Here’s a head scratcher. BusinessWeek named search engine Cuil, which launched prematurely, lost their VP Product and now has near zero traffic, as one of the most successful U.S. startups of 2008.

Why? They say it attracted lots of attention when it launched (true, but it wasn’t positive attention), and they say that Cuil has a larger search index than Google (which doesn’t appear to be true). They miss Cuil’s big possible tech advantage, which is IP on how they handle search queries that may be much cheaper than the way Google and others do it.

BusinessWeek generally has intelligent coverage of startups. This time, they blew it. Cuil may yet live to see success, but 2008 was a bad, bad year for them and they had no place on this list.

Update: In the comments and here BusinessWeek says the list is based on who raised the most venture capital over the last four quarters. But Ning and others aren’t included who raised more.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

Rival, it never did. The launch of the search engine was nothing but a classic PR trainwreck, with much hype and little to show for. Cuil failed to deliver good enough results to drive anyone to change their search behavior, and quickly became the subject of backlash and criticism because of their poor performance and indexing methods that actually took websites down in the process. Last time we reported about Cuil, was when their VP of Products (and AltaVista founder) Louis Monier quietly resigned from the startup.

With the end of the year approaching, I took a peek at how they’re doing traffic-wise out of sheer curiosity. After all, with no less than $33 million in funding and a founding management team consisting of ex-Google search experts, something had to give, right?

Well, no. Cuil isn’t performing well any way you look at it, and I can only imagine how nervous the startup’s management team and investors must be by now. Based on the numbers and graphs we gather from Google Trends, Alexa, Compete and Quantcast, you could even say search engine traffic is nearing rock bottom. Apart from that, a Cuil search for ‘TechCrunch’ still displays a Gmail logo rather than our own.

Crunch Network: CrunchBase the free database of technology companies, people, and investors

This is a big blow to the troubled search engine – Monier was recruited away from Google a year ago, where he was working on advanced search products. Prior to Google he was the head of search at eBay and was the cofounder and CTO of AltaVista. He is widely considered to be one of the founding fathers of Internet search according to search experts Danny Sullivan, John Battelle and others.

Why did Monier leave? We’ve heard but haven’t confirmed that he and CEO Tom Costello just couldn’t agree on the Cuil product road map, and that the botched launch didn’t help things much either. Monier won’t return our calls, and we have an email in to Cuil for comment.

So far Cuil doesn’t seem to have a lot of stickiness with users. Tons of traffic at launch, but it spiked right back down immediately afterwards (Google Trends, Compete, Quantcast).

Update: Vince Sollitto, Cuil VP Communications, confirms Monier’s departure and says he will remain on board as a consultant, continuing to contribute and advise the company. “I’m told by Louis that he believes strongly in the goals and potential of the company, especially its technology,” Sollitto said, adding “We think this will be the best role for the company and for him going forward.” Sollitto says there were philosophical differences in approach and style between Monier and the other management, and that the new consulting role provides the best way for Louis to contribute to the company going forward.

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

Yahoo has highlighted a few more implementations of BOSS, the search API it launched in early July that allows third party websites to incorporate Yahoo search functionality seamlessly into their sites.

This is the second time Yahoo has showcased the fruits of BOSS developers. In early August, Yahoo drew attention to 4HourSearch, the Cuil knock-off formerly known as Yuil; PlayerSearch, a sports-focused search engine; Newsline, a tool for plotting news items on a timeline; and Tianamo, a 3D search visualization tool for Windows machines with Java installed.

Now we’re presented with three more implementations: 123People, askBOSS, and BuildaSearch.

123People

123People is a search engine designed to help you find information about ordinary people. It supposedly returns the best results for people living in Europe, although the index includes those living in the United States as well.

123People has used BOSS to show web results and images about people alongside the email addresses, instant messaging accounts, documents, phone numbers, and other information it collects from elsewhere. Unfortunately (or fortunately?), I couldn’t find more than a few photos of myself when searching with my name, even when I told it which area code to look within.

askBOSS

Built by a Yahoo employee, askBOSS is like Powerset for images. It processes natural language (i.e. “who was the first president of the United States?”) and returns images that are intended to “answer” your query.

In my tests, it worked very well with some queries (“who is the lead singer of the Rolling Stones?”) and returned amusing but inaccurate results for others (“what was the first machine gun?”). But then again, this kind of search isn’t supposed to be easy.

BuildaSearch

BuildaSearch intends to remove the whole programming aspect of implementing BOSS for your website. It simplifies the setup process by letting you pick just the colors, images, and scope of search results you desire.

It took me a a grand total of 30 seconds to set up site-specific search for TechCrunch, found here. Unfortunately, our logo only shows up on the first page so the engine isn’t truly white-labeled.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

An anonymous tipster wrote to us this morning to tell us that Cuil, the ill-fated “Google Killer,” has unleashed its Twiceler indexing bot on websites across the globe and in the process, has brought many sites down.

“I don’t know what spawned it, but when Cuil attempts to index a site, it does so by completely hammering it with traffic,” the tipster wrote. “So much, that it completely brings the site down. We’re 24 hours into this “index” of the site, and I’ve had to restrict traffic to the site down to 2 packets per second, while discarding the rest, or otherwise it makes the site unusable.”

The Admin Zone forums are abuzz over Cuil’s overzealous method for indexing. Countless posters on the site have said that their websites have been brought down because of the Twiceler robot and one user said it “leeched enormous amounts of bandwidth — nearly 2GB this month until it was blocked. It visited nearly 70,000 times!”

Website owners are also saying that the way Cuil indexes sites isn’t scientific in any way and is actually quite “amateurish.” According to those who experienced the Twiceler onslaught, the bot seems to “randomly hit a site and continue to guess and generate pseudo-random URLs in an attempt to find pages that aren’t accessible by links. And by doing this, they completely bring a site down to where it’s not functional.”

Upset site owners contacted Cuil to see why Twiceler was hitting sites so often. James Akers, Cuil’s Operational Engineer responded to the issue by saying that “Twiceler is an experimental crawler that we are developing for our new search engine. It is important to us that it obey robots.txt, and that it not crawl sites that do not wish to be crawled. If you wish I will glad to add your site to our list of sites to exclude, but I need you to tell the site name to block as email return addresses frequently from the domains that wish to be blocked.”

Akers also claims that Cuil has seen a “number of crawlers” that pretend to be Twiceler, and site owners should consult the company’s IP addresses page to determine if it’s really Cuil causing all the trouble.

Cuil has yet to respond to a request for comment, but it doesn’t look like the pelting of sites by the company’s Twiceler bot is an isolated incident. And if it’s true that Twiceler is trying to find pages on sites that don’t even exist to simply increase the index size, Cuil should work quickly to modify the bot before it receives even more negative publicity.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

The company’s founders have a good pedigree, and have developed a unique way to index the Web cheaply and at massive scale. But creating a big index is only half the battle. A good search engine has to bring back the best results from that haystack as well. Here Cuil falls short, as we pointed out an hour after the site launched and we could actually check it out.

The story quickly turned from Google-killer to Google’s lunch (make that an amuse bouche). The results Cuil returns aren’t particularly great, and sometimes completely off the mark. For instance, a search for “Cuil” doesn’t even bring up a link to itself on the first page of results. (See screen shot at end of post).

And when Cuil tries to pair images with sets of search results, it often chooses seemingly random images to accompany a set of results. For instance, “Wordpress” is associated with what looks like a TV newscast team and “TechCrunch” is paired with a Gmail logo. And I have no idea who that person is next to the results for our “About” section.

The backlash goes beyond the blogosphere. Friendfeed founder and former Googler Paul Bucheit started this thread of deservedly harsh comments on FriendFeed:

Paul Buchheit posted a message
“Maybe Cuil isn’t supposed to be good. They must know that the results are bad, but they launched anyway. Maybe they aren’t trying to build a full search engine, but just want to demo their crawling+indexing technology with the expectation that someone will buy the company and plug in better ranking.”
. . .
—Where “someone” = MSFT, who has already shown that they are willing to pay a lot for non-functional search engines. - Paul Buchheit
—That is what I was thinking, too. – Robert Scoble
—You’d think they’d work toward getting something relevant if you search on “cuil launch” – Michael Markman
—It’s the only thing that makes sense when you look at how bad the results are. You don’t roll out something like unless you are just showing off the interface. – Kevin Bondelli
—Would also explain the weak branding. Who cares, if it’s just going to get plugged into something else? – Chris Baskind
—Not very good results at the moment I will say – shinchi via twhirl
—It must be so hard to launch something in a realm where there isn’t much tolerance/patience for incremental improvement. The bar in this space is high and consumers are very picky. Look at Yahoo. Their search is actually pretty damn good. However, they keep losing share. – Sacca
—agree with Sacca – also, I thought the people behind Cuil had already sold some search technology to Google and hence they wanted to try it out on their own this time – so if anything they’d have more intent than others not to flip. – Adam Kazwell
. . .
—Google Killer? I do not even need to look to know the answer – Mike Reynolds
. . .
—It’s embarrassing IMO. The results are thin and the images completely wrong. Not good is one thing … Cuil seems a cut below not good right now. – AJ Kohn
. . .
—IMHO there have to be better ways to show off your very own crawling+indexing technologies than opening up to the public and getting thrashed for not delivering what people expect when you dub your service search engine. I have a hard time believing that the cuil/cuill guys did not know what they were doing though … Were they pushed to release by financiers? Were they in dire need of usage data? – Mustafa K. Isik

If you are going to feed the hype cycle, you’d better be able to deliver. Because it only lasts about 20 seconds if you don’t.

(Photo by Steve Jurvetson).

Crunch Network: CrunchBase the free database of technology companies, people, and investors

Launching a startup ain’t easy. And flatlining right after your launch is more of a rite of passage than an embarrassment. Here’s hoping Cuil is cool again by morning.

Update: Yay! It’s back.

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

Search engine Cuil launched earlier this evening, claiming a bigger index size (120 billion web pages) than Google or any other search engine. The pedigree of the founders and execs, which includes three ex senior Googlers, means the service will be compared to Google from day one. And the way they will be compared is index size and, more importantly, relevance/ranking of results.

We’ve been testing the engine for the last hour. Based on our test queries Cuil is an excellent search engine, particularly since it is all of an hour old. But it doesn’t appear to have the depth of results that Google has, despite their claims. And the results are not nearly as relevant.

A search for Dog returns 280 million results on Cuil and 498 million on Google. Judging relevance of results is subjective, but Google returns Wikipedia as the first result, then dog.com. Cuil returns Dog.com, wikipedia isn’t listed on the first page of results. Both are meaningful results, but Google is better.

More searches, Cuil v. Google: Apple (83 m v. 571 million) – neither mention the fruit. France (102 m v. 1.5 billion) – Cuil’s category refinement makes their results better for this query. Stonehenge (800k v. 8.5 million). Silicon Valley (3.2 m v. 24 m). Techcrunch (600k v. 6.5 m).

It seems pretty clear that Google’s index of web pages is significantly larger than Cuil’s unless we’re randomly choosing the wrong queries. Based on the queries above, Google is averaging nearly 10x the number of results of Cuil.

And Cuil’s ranking isn’t as good as Google’s based on the pure results returned from both queries. Where Cuil excels is with the related categories, which return results that are extremely relevant. With Google, we’ve all gotten used to trying a slightly different search to get the refined results we need. Cuil does a good job of guessing what we’ll want next and presents that in the top right widget. That means Cuil saves time for more research based queries.

And I want to reemphasize that Cuil is only an hour old at this point, Google has had a decade to perfect their search engine.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

The super-stealth search project was founded by highly respected search experts. Husband and wife team Tom Costello (CEO) and Anna Patterson (VP Engineering) were joined by Russell Power. Patterson and Power are also ex-Google employees, and the company has been the subject of intense speculation over the last couple of years.

Much of the secret sauce of Cuil is in the way they index the web and handle actual queries by users. Both are costly to scale, and Cuil claims to have found a way to massively reduce those costs. That allows them to run the search engine a lot cheaper, even at Google-scale should it ever reach that point. By some estimates, Google spends a billion dollars a year to run the back end infrastructure of it’s search business.

Cuil also claims to have better search results than Google and others based on how they index websites. They do not simply catalog keywords on a site and then rank the site based on its importance. They also work to understand how words are related (France – cheese – wine, for example), to return more relevant results to users. This is a semantic approach to search, but very different from Powerset’s natural language approach. Powerset uses artificial intelligence to try to understand what sentences on a website actually mean. Cuil, by comparison, simply tries to properly categorize and file a web page, even if the category name doesn’t appear on the site.

That means users search the same way they always have, but Cuil will try to return better results via refinements in a “explore by category” module to the right of results. A search for dogs, for example, will return category results for “water dogs,” “crossbreed,” “cocker spaniel,” etc. Some of these related terms do not include the term “dog.”

Cuil is experimenting with a new type of search interface as well. Results are shown in three columns and contain an image and more summary text than existing search engines. In addition to refinement by category, Cuil will recommend related searches via tabs across the top of search results. A search for New York, for example, also has tabbed results for recommended refinements like New York Times, New York City, New York Yankees, etc.:

Cuil also says that they will put user privacy at the top of their business objectives. User IP addresses are not recorded to their servers, they say, and cookies are not used to associate a computer with queries. The data is simply dumped as it is created. That means user data cannot be turned over to others, whether its via blind stupidity or lawsuits.

Cuil has raised $33 million over two rounds of financing from Greylock, Madrone Capital Partners and Tugboat Ventures.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.