Google has just launched a new “search options” feature on its main search page. When you click on “Search options” you can filter your search by different types of results (videos, forums, and reviews), by time (recent, past 24 hours, past week, past year), as well as seeing related searches, a “wonder wheel” view, or a timeline view.
At Google’s Searchology event, which is going on right now, Marissa Mayer listed the following as the hardest unsolved problems in search:
- Finding the most recent information
- Expressing that you want just one type of result
- Assessing which results are best
- Knowing what you’re looking for
- Expressing your searches in keywords
Notice that real-time search is the No. 1 problem. (Twitter and a bunch of startups from OneRiot to Tweetmeme are also working on it, with the latter two launching their own real-time search efforts today). And it certainly is a problem for Google, even with the new recent results option. Try searching for any of teh top trending results on Twitter right now like Miss California (vs. Twitter search results) or Star Trek (vs. Twitter results), and you don’t even get any Twitter results on Google.
While real-time search is still a big problem, it is not the only problem. Some of the new options address the difficulty of searching back through time. The recent results get as real-time as Google can get, but you can also expand the timeframe. And you can look at an actual timeline of results, which looks for dates within results and then places them chronologically (this is sort of hit or miss—just because a date is mentioned in a text does not mean the entire result is about or from that period of time). Google now also lets you see related searches as an option. And the Wonder Wheel is more of a visual aid to see how different related topics are clustered together. When you click on any spoke of the wheel, it then causes that search term to be at the center. We’ve seen many of these techniques in the past, but Google is giving them a higher profile by putting them in its main search page..
Searchology event, this could be a sociology of searching for instant knowledge
As for instant knowledge and real time search, Yauba is a pretty good choice. It seems to be a combination of Google search + Twitter search in one shot. http://www.yauba.com
Stop posting about yauba!
Never heard of it but I like the front design. Results are kinda bad though.
I completely agree, they just changed the game and may have put a bit of a damper on Twitter Search’s importance in real-time search (Twitter still has the real-time sentiment aspect going for it).
So I am shocked at the relatively low number of comments here, and the lack of response when I posted this to Twitter about 2 hours ago.
Think about it: With one click (24 hour results), you can now bypass nearly the entire search ranking structure, and get recency. Until today, Google’s results were in danger of getting more and more stale due to the authority algorithms:
E.g. if you did a search for say SXSWi panel notes in March, Google would come back with nearly all stuff from 2008, 2007, asf. because those pages had months or years collecting inbound links. On Twitter, a quick search for “#sxsw notes” gave you the 2009 results as they were being posted.
It is telling that Google has come around to taking this seriously enough to declare it “the greatest challenge”. Now will they buy Twitter to sew things up and preempt Microsoft?
(Could current re-increased anti-trust scrutiny prevent this?)
Or will MSFT wake up (now that Google has made this rather unmistakable gesture towards real-time search) and buy Twitter first, denying Google access to the firehose?
This is getting more interesting by the minute…
Marissa Mayer quoted:
Finding the most recent information
Alex said…
It is telling that Google has come around to taking this seriously enough to declare it “the greatest challenge”.
I think that another way to possibly solve this problem is to use Tensor calculus algorithms, which are mainly used by Physicists & Engineers, but its data analysis capability has been noted by data analysts, machine learning community, statisticians, information retrieval community. Current search don’t treat time-stamp as a variable (ie, another object or dimension). Both researchers at Google & Microsoft are already aware of tensors, and I wouldn’t be surprised if they’re working on PageRank tensors that includes time-stamp.
The reason I said that R&D people at both Microsoft & Google are already aware of tensors, because both companies sent representatives to participate the 2006 MMDS (Workshop on Algorithms for Modern Massive Data Sets), where several papers on tensors were presented.
So, it isn’t that Twitter has invented something that Microsoft & Google army of PhDs aren’t already aware of, but I think that hardware issues may be part of the problem that they’re trying to solve, since tensors is quite intensive & massive memorywise to crunch.
I think Google has a good approach on realtime. It is obvious that you need great resources to index and offer the most recent information. I think Google can do it.
Anyway, some post on important sites arrive very fast in Google index. It is just a matter of time until the realtime search is the new king.
Great stuff,
this is not Globalization it´s GoogleIzation
When can Google’s real time search comes to #1 on its SERP, before “search.twitter.com”? currently searching “real time search” yields twitter search
Google is too much smart.Still working with research & development approach.
Question: Is Twitter working on “real-time” search or twitter search?
Is twitter going to crawl my blog or my facebook page and return my up-to-the-second post/update as a result? Or is twitter going to return recent tweets?
If a tree falls outside of twitter – will twitter “real-time” search know about it?
Google is talking about a much bigger problem.
You nailed it.
This is great! I’m really interested to see how things play out over the next few months.
There’s A LOT of value is time sensitive search. The challenge is gonna be how to balance the results based on quality (authority if you will) and freshness.
It should be easy to get the dates of news/blog content (half the time it’s in the url) but it will be a lot harder for other types of content.
Is this live? I don’t see these options.
I don’t see it either. On the east coast of the USA.
So how long do you think it’s going to take for spammers to game Twitter real-time search results if it is give priority on SERPs?
I would like to see Google add real time search to their News section as well and implement some of these same functions.
Twitter is not ‘real time search’ unless you believe that the army of twitter users crawls the entire web for what’s been updated in a real time.
Twitter search is simply a real time search of people’s status/conversations, and what people are collectively talking about is most of the time, an echo chamber of news stories, and not necessarily a real time index of anything unique or important, unless you think cool links to weird photos or youtube videos are important.
Google helps me get work done. When I am stuck, 99% of the time, a Google search will unblock me. Asking people on Twitter rarely returns as timely a solution. Search.twitter.com is mostly useless for debugging anything.
I’m amazed that people think the Mechanical Turk/Crowdsource approach is going to solve the problem of scaling up real time search for the entire web in the face of exponential increases in information publishing.
When Google says “Real Time Search” they’re talking about a different animal entirely.
Ray said…
Twitter is not ‘real time search’ unless you believe that the army of twitter users crawls the entire web for what’s been updated in a real time.
Amen to that Ray. I have been saying that a few times here at TC, but the Twitter worshoppers are not convinced. They don’t seem to understand search indexing. Once you index new incoming information as raw data, you either re-compute the whole index of extracted fetatures again from the beginning (batch-type algorithm) or re-compute only features of the new arrival (online-type algorithm). These feature extractions processing are quite time-consuming & memory intensive and it cannot be true real-time if that is the one that Twitter is using. There will always be a delay. In Google’s case, I am not sure how often they update their PageRank, perhaps every hour or so. It may be longer than updating it hourly.
I seriously doubt that Twitter is using any online real-time algorithms at all, however there are a few of those algorithms already being published & made available in the literatures. Google & Microsoft R&D groups are already aware of these algorithms since this is what they do all day and nothing else (scouring the research literatures), but implementation-wise? I don’t know why they haven’t done so.
I said…
Google & Microsoft R&D groups are already aware of these algorithms…
Here is a paper on online-algorithm that was co-published by Microsoft Asia R&D and some other University Researchers in recent years, which appeared in the proceedings from 2007 IJCAI (International Joint Conference on Artificial Intelligence) conference.
Abstract:
——-
Detecting and tracking latent factors from temporal data is an important task. Most existing algorithms for latent topic detection such as Nonnegative Matrix Factorization (NMF) have been designed for static data. These algorithms are unable to capture the dynamic nature of temporally changing data streams. In this paper, we put forward an online NMF (ONMF) algorithm to detect latent factors and track their evolution while the data evolve. By leveraging the already detected latent factors and the newly arriving data, the latent factors are automatically and incrementally updated to reflect the change of factors. Furthermore, by imposing orthogonality on the detected latent factors, we can not only guarantee the unique solution of NMF but also alleviate the partial-data problem, which may cause NMF to fail when the data are scarce or the distribution is incomplete. Experiments on both synthesized data and real data validate the efficiency and effectiveness of our ONMF
algorithm.
PDF Download:
————
Detect and Track Latent Factors with Online Non-negative Matrix Factorization
The fact that the paper had been publicly made available, there is no doubt that Google researchers as well as others have read it. Either they (Google) develop something to perform better than the Microsoft’s ONMF (Online Non-negative Matrix Factorization) algorithm or tune the ONMF themselves to perform better than the published version.
I suspect that if Microsoft is using ONMF as a basis for product development, then they must have developed a superior version of ONMF for internal use, ie , they’re not publishing it, which is understandable. You can’t be Santa Claus and publish every bit of research that you do, or otherwise, there is no commercial advantage, since once you invent something, you go on to gift it to everyone else (including your competitors) to use.
Good contribution.. hope more people like you come here.
I do not know how forum/review spams will react to this.
Next to the issue of realtime crawling the web, you also have the ranking issue. How would you know – in realtime – which results are the most relevant to a given query?
i agree – twitter search turns up the more irrelevant results. It almost seems like twitter and google had/have opposite challenges. 140 chars limits the relevance of results and tiny urls don’t help the case for reults that may contain links with the up to date information i’m looking for.
This is awesome!
As a developer I’m always running into problems with old versions appearing at the top of Google’s search results.
Before this, if you search for anything about the Facebook, iPhone, Twitter API’s, etc., you’d always get old outdated results because their SEO rank is higher.
…and yeah, I like the “wonder wheel”. Weird name, but useful.
Google is definitely bent at doing something about real time search and whenever that is coming I just have a feeling it’s going to be better
Erick – Back in March we integrated real-time search results from OneRiot and Twitter onto the Google SERP: http://blog.sur...l-time-results/.
I’m really intrigued by the real-time vision and whatever Google eventually unveil it’s going to be innovative and as close to real-time as possible!
kudos to Google.. they are just taking online search to new paradigms.. but I wonder how would online advertisers would react to this. While using ‘Wonder Wheel’ I noticed that if I choose one of the suggested terms on the spoke no ads appear which otherwise appear while using normal search.
This is awesome. We launched a real time search on our site a few weeks ago…good to see so many other people catching up
seriously, though…”real time” as we’ve already seen from our own implementation, is problematic by default…better is the relevance sorting (on Google, on FunAdvice, or on Twitter) however, for twitter at least…they don’t have the relevance option, it’s real time or nothing.
Google & FunAdvice, though, both have these options…and I think Google’s doing it right, we should probably move back to relevance by default, and then use real time as a toggle.
When you publish an article on your blog you can sit around and wait for Google to index it and hope Google gives you a good placement or you can instantly submit your article on Twitter in Real Time.
The Real Time difference is who makes the choices on when your article can be found. This is why Google is promoting Real Time search because they are afraid of Twitter’s real time availability. Users will learn to build their own audience themselves and stop depending on Google for traffic. One Twitter account I have been building is now drawing 300 to 400 hits in minutes after I submit an article. Now that is Real Time!
Personally I am tired of pandering to Google’s SEO requirements that are always changing.
Tony said…
Google is promoting Real Time search because they are afraid of Twitter’s real time availability.
Umm, bullsh*t! Twitter is not real-time if it is , then it is not indexing the word-message features, just straight search on the raw data. The precision on searching using the raw data is far far inferior to one where searches is done on reduced sets of features extracted from the raw data.
Falafulu Fisi said…..
Twitter is not real-time…search on the raw data.
Raw data or not I my goal is instant results. Twitter drives instant traffic and my affiliates and Google provides contextual ads.
I will say the features of Related searches, timeline and Wonder wheel are cool and very useful for the end user. But the end result I want is INSTANT TRAFFIC.
Nice try on the “filtering” options, but the best benefit may arise when the searcher can choose whether they want stores in the search results or not.
(When searching on a movie, book, or band, you can learn a little about the work, but see a lot of offers to sell it to you. Adding “kurosawa -order -buy” and such doesn’t really work. Just filter out the stores when requested, and Google Websearch would become much more valuable.)
Another piece of low-hanging fruit is to weed out duplicate content. There are controls for it now, but searches still contain lots of search-rewarded spam.
i can’t see the "show options" link? not on the google.de page and not at google.com.
I see it at Google.com. Are you logged in? Are you coming from the US? Who knows how or why they’d segment the rollout…
You’ll find the options mantioned on the article clicking on "News". Immediatly above the SERP
yeah, logged in. i’m coming from germany… hmm. normally they mention it in the blog post, if the rollout is not for all countries.
@fluvio: i think, you should find the option directly at the results page, as shown in the video in second 18 or so.
I can’t find the video, where is it?
http://www.yout...h?v=MtirDMfcOKE
Thank you for the link. I red on Mashable that it is going to be on for all users by the end of the day. No idea why it’s not on for everybosy.
real time search – but how many people are actually wanting it?
I use generally use google for research and looking up code! However, I am fully waiting for wolfram to make it’s debut!!!
hmmmm the SQL query must be insane! Anyone have any idea what the query would look like? How would you link hundreds of db tables to accomplish real-time search? Anyone care to post a sample?
Kinda cool but I wonder if this is what will open up the market for another search provider to come in and provide a lean, clean and highly accurate search, similar to how Google crashed the search engine party a few years ago. Google’s results have degraded over the years but I don’t know that this is the solution.
I like the new wonder wheel but is there any limit to how much the wheel can grow or how many queries relate to the orginal queries?
Google is full of surprises….they are always coming out with great new features for googlers to use!
That’s a nifty addition! Gonna be easier to get the latest movie news.
Now, if only google was extending those new things to google custom search, would be so happy!
Wow, I’m tellin ya peep’s, one day Google is going to rule the world. Perhaps the entire universe!
RT
http://www.priv...resources.us.tc
OMGosh dude, thats insane. You do know that one day Google will rule the world right?
RT
http://www.priv...resources.us.tc
Hopefully this function will be available in other languages soon. It’s a great feature.
I think these are some legitimate issues that are long overdo. I love the idea of being able to search specifically for ‘current’ data. Nothing irritates me more than searching for health and fitness information and finding 2-year old posts at the top of the results! Very informational post!
I don’t like the new look at all, it looks kind of childish and thrown together very fast, not like google at all.
I like this new feature. Hopefully it will stop Google from showing pages from 1999 on my searches.
There’s a jelly talk in SF this Friday on open real-time search (from Google, Twitter, Yahoo)
http://wiki.wor...lySF-2009-05-15
You first have to do a search before the options menu becomes available on the left.
Its changing the game for SEO )) Now the date and fresh content become more crucial
It looks real messy…
The information Google is willing to expose is amazing.. although, i’m not sure whether the common “i-need-my-results-now!” searcher would are much for this.
google is now my official torrent search engine. lets see the MPAA and RIAA with google
Yeh…more search options, complete with more censorship.
This is in direct response to Wolfram Alpha. I hope W. A. kicks Google to the curb.
I noticed it yesterday. I just love the forum search feature
It’s interesting…thinking in the mindset of the searcher…seems the idea of being “fresh & new” wins. Rather than reading a trusted and aged article, they’d rather read a new and possibly untrustworthy article simply because it is “fresh & new.” Well, at least that was the response I got from 10 co-workers.
well i have to say its not going to be easy getting round this problem, what would be nice though is to have the first 20 sites in random positions, so first 20 group a and the next 20 group b ect. what do you guys think
It looks like you have to stay current with google to have any success at all to help your company with web exposure… I’m hopping on that train…