Google’s Norvig Is Down On Natural Language Search

December 18th, 2007 at 12:49 pm

That and the fact that people seem to have learned to think in google. I learned a long time ago that I needed to search the way the answer would be returned, not search with my question.

Microsofts help has tried to encompass this natural language stuff for a long time, I generally end up going to google and searching for msdn and my search terms.

December 18th, 2007 at 12:55 pm

“I think there is a world market for maybe five computers.” - IBM Chairman, 1943

I neither agree nor disagree about natural language search, but just because someone important from a big, important company says it, that does not make it true.

December 18th, 2007 at 1:10 pm

Natural Language Search is like speech recognition: perpetually lacking and perpetually funded.

December 18th, 2007 at 1:31 pm

>>>Most people are too lazy to type in more than one or two words into a search box anyway.

Definitely not true. Even in 2005, Hitwise data showed that 52% of search engine user queries were 3 words or more.

And, according to David Krane (also from Google) 25% of searches, daily, they’ve never seen before…hard to type in new one or two word phrases all the time.

December 18th, 2007 at 1:44 pm

If you’re reading, I want to thank who ever at Google sent my resume to Amazon in Seattle. I did a phone interview today but I bombed it pretty badly. I haven’t really used advanced SQL commands in a long, long time. I’m programming pretty exclusively in C++ and none of the questions grazed that. Thanks though
I also got a written exam from another SF company well known to TC though, and I am really good at written C++ exams so hopefully that will work out It was hard but I’m pretty good at C++.

As for natural language search, I think it’s ok, but like the researcher at Google said it’s pretty useless.

December 18th, 2007 at 1:49 pm

Natural language search is easier compared to speech recognition - because the problem of understanding whether someone said a phrase like “Austin” or “Boston” is not there - there is an unambiguous input mechanism.

The example quoted in the article is a simple one where you are going to get results as accurate as the keyword search.

But with a query complex such as “What is Paris famous for?” - one is better off asking the question and having NL technology do the interpretation of what all “famous” could mean and which Paris you’re talking about and organizing the results appropriately instead of learning to type a bunch of - and + after a few searches.

December 18th, 2007 at 1:51 pm

There’s the aspect of us being conditioned to use words and short phrases to describe what we are searching for. While I’m not certain, I suspect this has to do with the fact that it’s much easier/faster to type in a few words, scan the results, and iterate if the result set is not satisfactory.

If a search engine can return the right answer every single time, I can imagine typing in the full question. But, would you even know what the right question is?

December 18th, 2007 at 2:01 pm

Glad to see someone finally point out the “would people actually use it” question. I agree with Norvig’s points about the backend being important.

December 18th, 2007 at 2:23 pm

I agree with “Hustle Strategy”; people have learned to talk in “keywords”. I think that’s from a familiarization with the computer world. I mean, it was only recently that my mom stopped putting “www.hotmail.com” into the Google search box.

Of course, that kind of transitional generation will no longer be an issue as everyone being raised with computers learns to think in terms of tagging and categorization.

I think better than natural language searching, Associative searching would be cooler (and I am sure they do this to some degree). For example, if I put in the phrase:

how to html to Excel

… Google would use the “how to” to know to only bring up results that are tutorials. Of course, the problem is that the more natural language that gets, the more ambiguous it becomes.

I guess, that’s really what the problem is - natural language is ambiguous; but, tagging, keys, categorization is not (or less so).

December 18th, 2007 at 3:27 pm

This is akin to most in the IR (information Retrieval) community playing down the importance of natural language search. They always stress the “people being lazy and type only a few keywords” argument, BUT what the fail to highlight are the following points:

1. People are lazy and type in a few keywords because we were “conditioned” to do so by their algorithms since typing longer phrases or sentecnes failed to yield good results. If we were able to exlpoit NLP effectively from the get go, I am sure things would have been very different.

2. Disambiguation is part of NLP, so google is doing Natural Language Processing as well but at a smaller scale for obvious reasons outlined by others already.

3. To say that Natural Language Processing won’t help is an irrational argument to make. If that were the case, I am not sure how Norvig and many others process information (perhaps they simply utter 2-3 words at googleplex). I think it is more reasonable to say that current state-of-the art in NLP may not be good enough to handle web search per se. And there are many problems in Natural Language Processing that still need to be solved before we can effectively deliver a search engine that will handle full fledged sentences. Having said that regardless of how well powerset is able to pull it off, NL is and should be the future direction for “automated search”.

Also,

“We don’t think it’s a big advance to be able to type something as a question as opposed to keywords. Typing “What is the capital of France?” won’t get you better results than typing “capital of France.””

is just an example of an example hand picked to make his point. What one should ask is as humans when we engage in discourse, is it possible that we may also abridge the question “what is the capital of france” with “capital of france” and still maintain the “semantics”. In this case yes, since its not at all ambiguous.

Also as someone pointed out as well, the power of NL should come from the ability to index documents that contain full-fledged sentences that describe the content of the pages more so that processing the query.

December 18th, 2007 at 4:47 pm

People seem to be stuck on the way search engines are used today, keywords are claimed to be the somehow most natural way to search for things while actually the most natural way to do so is using full-blown sentences.
I mean, if you are lost somewhere, do you walk around yelling keywords at people or do you ask questions that model your request as precisely as possible? Certainly, a lot of human communication is made up of noise (e.g politeness patterns, repetition, re-phrasing) that doesn’t matter in a human-machine interaction setting, which is why abstracting over sentences by using keywords is a very efficient and useful method in most of the search use cases thought of today.
However, everyone can contrive requests (like the “What is Paris famous for?” example mentioned above) that neither can be easily expressed in keywords nor can be answered by conventional keyword-based search engines. You might say that these are just that: Contrived examples, but I think that we got used to keyword search so much that we cannot really think out-of-the-box anymore. There is plenty of space for further development of search engine technology, both UI- and backend-wise and this most certainly involves improved natural language understanding.

Nevertheless, a serious issue that still persists for natural language search in the backend is scalability when compared to simple keyword-indexing algorithms. While keyword indexing can be done in linear or loglinear time, for example naive parsing algorithms in the worst case can take exponential time, while even rather efficient parsing algorithms like the Earley parser often still need polynomial time (O(n^3) in this case). This is the actual reason why PowerSet still has to prove to me that their particular approach of setting up a large-scale all-the-Web natural language search engine is computationally feasible at all and not just some expensive toy for investors.

Maybe skipping the sentence level and concentrating on the text level is the way to go. Many of the elements that create textual coherence (and hence natural language understanding) operate beyond the scope of sentences. An engine that can create these kind of links could be very useful, though it would be beyond the notion that we have of search engines nowadays.

December 18th, 2007 at 5:27 pm

Note to self: Never call my users lazy

example of Google’s theory:

query: pallet jack

query: I want to buy a pallet jack

The results are 30% overlapping.

The keywords ‘pallet jack’ over lap - 34% / Google is good.

December 18th, 2007 at 6:21 pm

If you read between the lines correctly, the guy probably just wanted to say “Sergey will buy Powerset if the price is right.”

December 18th, 2007 at 7:57 pm

Natural language search will work as soon as the biologic intelligence can be reproduced in some way. The models must change enough to leave a path to natural evolution. May be the next step in life evolution leaves DNA for Silicon or any newer data-technology. The fact is that the actual research of AI is far from reaching a higher point, but the goal has been pointed out and many of us. (including me) are creating new paths everyday to lead to this target. Sooner or later, some of us (hopefully many) will succeed and this will be the moment in time where a next step has been reached in evolution: collective intelligence will be the answer. Today’s internet throws a glimpse on it, bettering every day, information is being debugged and improved by millions of people every hour. Wikipedia is a good example, good things prevail, bad things will disappear in noise.

December 18th, 2007 at 10:11 pm

… Vint Cerf - about 8 months ago spoke at my university…
He said Semantic web was very important for the future.

And I agree.
I know how these things work, I don’t type sentences in search boxes…

Many people I know do.
And every so often, I’m not quite sure how to phrase something so that I get the results I want.

but i’d still consider myself relatively proficient at ”Google Grammar” as such, though I daresay, the extension to the language allowed, and considerably more directed results can’t be all that bad for both advertisers and customers…

Surely an advertiser will pay more to be the ad for a query “Where can I buy Pallet Jacks in Seattle?” than “Pallet Jack” - the directedness is probably quite useful.

But chances are, if you’re viewing this website, you’re one of the more computer proficient individuals on the planet - and I guess such a change doesn’t mean as much to you or me as it might to my 40 year old mother, who gets cranky when results are irrelevent.

December 19th, 2007 at 12:04 am

@Mr. Crash

The concept Semantic Web actually hasn’t very much to do with search techology. While the latter is about machines somehow automatically ‘understanding’ the content generated by human beings as it is, Semantic Web means more or less manually enriching the content in order to make it easier to understand for machines.

This is why Semantic Web from my point of view won’t work at the large scale, I simply cannot imagine every content creator to manually create Semantic Web annotations just because Tim Berners-Lee has a vision. It might be justified in certain settings, where you have knowledge engineers dedicated to such kind of work, but don’t expect your average content provider to supply Semantic Web annotation any time soon.

December 19th, 2007 at 2:28 am

NLP is unachievable and unnecessary… but it’s a marketing tool for some. I’ve always thought it would be interesting to test Autonomy (a company which makes big claims about its software’s ability to understand text) against Google. Load Medline into both and see what happens.

December 19th, 2007 at 8:53 am

Interesting the timing on this. I just wrote a piece for Alt Search Engines titled Semantic and Personalized Search Are Fool’s Gold that makes a similar case.

I use the example of people searching for “calendar of events” and how it’s impossible to know exactly what that searcher wants. If a human can look at a search query and still not understand exactly what the searcher wants, how is a computer algorithm going to do it?

December 19th, 2007 at 10:11 pm

In my opinion people want to do things on the net . Google is the best tool available on the net if no specific site is known. Question is not wether few key words or a nlp query or phrase will solve the need. Question really is how do we empower the user with a tool as will let him do whatever he wants -shop,talk to friend,gather info… in the shortest time.

December 27th, 2007 at 8:23 am

Hello people - there are two pieces to the puzzle here:

1) Speech to Text

THEN

2) NLP

I for one, wouldn’t mind speaking to my computer or laptop and having it do things for me automatically. e.g. “Find me a recipe on making creme brulee” as opposed to typing that out. Why the hell would I type the entire thing out if I can just type, “recipe creme brulee”? It’s not about CONDITIONING - it’s about the INTERFACE (search box). I want to TYPE less because it’s a pain to type more than I need to.

We DON’T TYPE in natural language - WE SPEAK IT. That’s why it’s called LANGUAGE. If we can get the speech to text right - then we need the machines to understand the text part — this is where NLP comes into the picture.

Look at the BIG picture folks.