Comments on: Is Cuil Killing Websites? http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/ Startup and Technology News Thu, 26 Nov 2009 10:52:58 -0800 hourly 1 By: ColtraneX http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-3081440 ColtraneX Sat, 07 Nov 2009 03:56:56 +0000 http://www.techcrunch.com/?p=21626#comment-3081440 Some user told me that his data was lost, so I checked the account and DB and data was actually lost. At first I thougth his account was owned, so we reset passwords and I start studing the Logs to know what this guy actually did, and try to get his IP. What did I found? I found almost 1 GEt Request per minute from Cuil executing code to delete the user account. Here is how those lines look on my access logs: crawl-3c.cuil.com - - [06/Nov/2009:20:55:40 -0600] "GET /administration/panel.php?mod=xxxx&op=delete&_pagi_pg=18 HTTP/1.0" 403 - "-" "Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)" WTF! This crawl was somehow accesing the user Panel (wich obviusly requires auth) and deleting 1 per 1 his information! I have banned every single IP and Domain from Cuil and now I am studing how the hell the crawler got the access to the user Panel. I should consider reviewing this situation with a lawer. I have the logs that prove that this Crawler did what it did. Some user told me that his data was lost, so I checked the account and DB and data was actually lost. At first I thougth his account was owned, so we reset passwords and I start studing the Logs to know what this guy actually did, and try to get his IP.

What did I found?

I found almost 1 GEt Request per minute from Cuil executing code to delete the user account.

Here is how those lines look on my access logs:

crawl-3c.cuil.com – - [06/Nov/2009:20:55:40 -0600] “GET /administration/panel.php?mod=xxxx&op=delete&_pagi_pg=18 HTTP/1.0″ 403 – “-” “Mozilla/5.0 (Twiceler-0.9 http://www.cuil...eler/robot.html)”

WTF!

This crawl was somehow accesing the user Panel (wich obviusly requires auth) and deleting 1 per 1 his information!

I have banned every single IP and Domain from Cuil and now I am studing how the hell the crawler got the access to the user Panel.

I should consider reviewing this situation with a lawer. I have the logs that prove that this Crawler did what it did.

]]> By: » Cuil odrovnáva weby alian.info http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2739518 » Cuil odrovnáva weby alian.info Sat, 09 May 2009 16:17:21 +0000 http://www.techcrunch.com/?p=21626#comment-2739518 [...] priebehu jedného dňa. To pochopiteľne vedie k zahlteniu indexovaného webu, ako o tom informuje techcrunch.com. Nieje teda divu, že robot Twiceler bol označený za amatérizmus a Cuil tak dostáva len [...] [...] priebehu jedného dňa. To pochopiteľne vedie k zahlteniu indexovaného webu, ako o tom informuje techcrunch.com. Nieje teda divu, že robot Twiceler bol označený za amatérizmus a Cuil tak dostáva len [...]

]]> By: Adam Haas » Fail: Internet Search Nearly Hits Rock Bottom http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2594927 Adam Haas » Fail: Internet Search Nearly Hits Rock Bottom Thu, 15 Jan 2009 19:07:41 +0000 http://www.techcrunch.com/?p=21626#comment-2594927 [...] of backlash and criticism because of their poor performance and indexing methods that actually took websites down in the process. Last time we reported about Cuil, was when their VP of Products (and AltaVista [...] [...] of backlash and criticism because of their poor performance and indexing methods that actually took websites down in the process. Last time we reported about Cuil, was when their VP of Products (and AltaVista [...]

]]> By: Why you need a Robots.txt | Mark8t: E-Marketing And Beyond http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2591887 Why you need a Robots.txt | Mark8t: E-Marketing And Beyond Tue, 13 Jan 2009 00:37:59 +0000 http://www.techcrunch.com/?p=21626#comment-2591887 [...] issue has arisen, over a new web search engine called cuil, and their robot twiceler. It has been reported that cuil’s robot twiceler may cause some harm to some webmaster’s bandwidth. We [...] [...] issue has arisen, over a new web search engine called cuil, and their robot twiceler. It has been reported that cuil’s robot twiceler may cause some harm to some webmaster’s bandwidth. We [...]

]]> By: rollin’ sysadm » Blog Archive » Beware of misbehaving crawlers http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2585130 rollin’ sysadm » Blog Archive » Beware of misbehaving crawlers Mon, 05 Jan 2009 13:25:10 +0000 http://www.techcrunch.com/?p=21626#comment-2585130 [...] also found an article describing the same issue were were [...] [...] also found an article describing the same issue were were [...]

]]> By: Cuil Fail: Traffic Nearly Hits Rock Bottom | Fidicaro.net http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2578512 Cuil Fail: Traffic Nearly Hits Rock Bottom | Fidicaro.net Sat, 27 Dec 2008 22:40:01 +0000 http://www.techcrunch.com/?p=21626#comment-2578512 [...] of backlash and criticism because of their poor performance and indexing methods that actually took websites down in the process. Last time we reported about Cuil, was when their VP of Products (and AltaVista [...] [...] of backlash and criticism because of their poor performance and indexing methods that actually took websites down in the process. Last time we reported about Cuil, was when their VP of Products (and AltaVista [...]

]]> By: Cuil Fail: Traffic Nearly Hits Rock Bottom http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2578208 Cuil Fail: Traffic Nearly Hits Rock Bottom Sat, 27 Dec 2008 15:40:04 +0000 http://www.techcrunch.com/?p=21626#comment-2578208 [...] of backlash and criticism because of their poor performance and indexing methods that actually took websites down in the process. Last time we reported about Cuil, was when their VP of Products (and AltaVista [...] [...] of backlash and criticism because of their poor performance and indexing methods that actually took websites down in the process. Last time we reported about Cuil, was when their VP of Products (and AltaVista [...]

]]> By: Horus Kol » Cuil is getting into hot water http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2573858 Horus Kol » Cuil is getting into hot water Sun, 21 Dec 2008 08:43:20 +0000 http://www.techcrunch.com/?p=21626#comment-2573858 [...] This is the main gist of the headlines covering Cuil today (for example, TechCrunch. [...] [...] This is the main gist of the headlines covering Cuil today (for example, TechCrunch. [...]

]]> By: Top 10 Web Dev Blunders of 2008 | Pedram Development http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2557933 Top 10 Web Dev Blunders of 2008 | Pedram Development Fri, 05 Dec 2008 10:10:05 +0000 http://www.techcrunch.com/?p=21626#comment-2557933 [...] However, the Cuil launch proved to be a huge disappointment. For starters, the site was down for hours, which is forgivable given the massive amount of traffic the site received. More importantly though, Cuil didn’t live up to its over-hyped billing. The search results weren’t even close to Google’s relevancy, and their indexing bot was crashing websites. [...] [...] However, the Cuil launch proved to be a huge disappointment. For starters, the site was down for hours, which is forgivable given the massive amount of traffic the site received. More importantly though, Cuil didn’t live up to its over-hyped billing. The search results weren’t even close to Google’s relevancy, and their indexing bot was crashing websites. [...]

]]> By: tinyComb » Blog Archive » The Ethics Of Cuil’s Indexing Approach http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2551408 tinyComb » Blog Archive » The Ethics Of Cuil’s Indexing Approach Mon, 01 Dec 2008 02:34:59 +0000 http://www.techcrunch.com/?p=21626#comment-2551408 [...] tiny from: TechCrunch.com original post View Jason Wilk's Profile      Subscribe via RSS RelatedBookmarksTags [...] [...] tiny from: TechCrunch.com original post View Jason Wilk’s Profile      Subscribe via RSS RelatedBookmarksTags [...]

]]> By: FLOSS In Church » Blog Archive » And, I’m back http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2539011 FLOSS In Church » Blog Archive » And, I’m back Wed, 19 Nov 2008 03:28:03 +0000 http://www.techcrunch.com/?p=21626#comment-2539011 [...] am thinking about blaming Cuil. Fortunately, I won’t be missing much traffic if I block them. These icons link to social [...] [...] am thinking about blaming Cuil. Fortunately, I won’t be missing much traffic if I block them. These icons link to social [...]

]]> By: Cuil als Googlekiller vermoord… Websites | Computertaal http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2491932 Cuil als Googlekiller vermoord… Websites | Computertaal Sun, 05 Oct 2008 08:01:07 +0000 http://www.techcrunch.com/?p=21626#comment-2491932 [...] Cuil is een zoekmachine die als Google killer voorgesteld werd. De basis is er in ieder geval: het personeelsbestand bestaat volledig uit ex-Google medewerkers. Alleen lijkt Cuil niet zozeer Google uit te schakelen, als wel de sites die het indexeerd. Cuil doet dat namelijk zo grondig dat hele websites down gaan doordat ze overladen worden met ‘verkeer‘. [...] [...] Cuil is een zoekmachine die als Google killer voorgesteld werd. De basis is er in ieder geval: het personeelsbestand bestaat volledig uit ex-Google medewerkers. Alleen lijkt Cuil niet zozeer Google uit te schakelen, als wel de sites die het indexeerd. Cuil doet dat namelijk zo grondig dat hele websites down gaan doordat ze overladen worden met ‘verkeer‘. [...]

]]> By: webman100 http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2480705 webman100 Thu, 25 Sep 2008 02:53:03 +0000 http://www.techcrunch.com/?p=21626#comment-2480705 Reading between the lines its quite obvious twicelers screwy design wasnt by accident. You have an Irish Techie who convinced VC's to dump 33 million in small change into his concept, which was based mainly on their massive index size. But guess what, when the VC auditors came to see the index size for real Mr. Tom had to fill it with something. So he came up with a spur of the moment idea to run the system dictionary agaisnt every sites web directory, and presto, Cuil now has generated millions if not billions of indexes to unique ip addresses, ofcourse they are all 40(1,2,3) errors, but who cares, since he was selling index size to his VC's not content. He passes the auditors test, they open up the bank account, and its muffins and chocolates forever (or at least until the money runs out and the VC's try to sell a goldmine full of fools gold. Reading between the lines its quite obvious twicelers screwy design wasnt by accident. You have an Irish Techie who convinced VC’s to dump 33 million in small change into his concept, which was based mainly on their massive index size. But guess what, when the VC auditors came to see the index size for real Mr. Tom had to fill it with something. So he came up with a spur of the moment idea to run the system dictionary agaisnt every sites web directory, and presto, Cuil now has generated millions if not billions of indexes to unique ip addresses, ofcourse they are all 40(1,2,3) errors, but who cares, since he was selling index size to his VC’s not content. He passes the auditors test, they open up the bank account, and its muffins and chocolates forever (or at least until the money runs out and the VC’s try to sell a goldmine full of fools gold.

]]> By: RoJo http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2480317 RoJo Wed, 24 Sep 2008 19:44:14 +0000 http://www.techcrunch.com/?p=21626#comment-2480317 Fair is fair and Cuil is fairly cool and the media hoopla on the Cuil launch is well deserved and totally understandable (even if a bit harsh). After all, Cuil was built by a team of top-notch ex-Google engineers. But did you know that another new search engine -- built by a team of top-notch ex-Google users -- has surpassed Cuil in traffic this month? And with nary a lick of media love. Check out NeXplore Search (www.NeXplore.com) vs. Cuil (www.cuil.com) for the month of September using whatever website traffic comparison tool you prefer -- Google Trends, Alexa, Compete, etc. Cuil’s focus -- more algorithmic complexity. NeXplore’s focus -- a more visually engaging and productive search results page. Seems pretty clear which approach real folk prefer... Fair is fair and Cuil is fairly cool and the media hoopla on the Cuil launch is well deserved and totally understandable (even if a bit harsh). After all, Cuil was built by a team of top-notch ex-Google engineers. But did you know that another new search engine — built by a team of top-notch ex-Google users — has surpassed Cuil in traffic this month? And with nary a lick of media love. Check out NeXplore Search (www.NeXplore.com) vs. Cuil (www.cuil.com) for the month of September using whatever website traffic comparison tool you prefer — Google Trends, Alexa, Compete, etc. Cuil’s focus — more algorithmic complexity. NeXplore’s focus — a more visually engaging and productive search results page. Seems pretty clear which approach real folk prefer…

]]> By: The Toybox mobile edition http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-3/#comment-2464556 The Toybox mobile edition Wed, 10 Sep 2008 14:55:15 +0000 http://www.techcrunch.com/?p=21626#comment-2464556 [...] might just be taking out the entire Internet, one site at a time. As TechCrunch's Don Reisinger reports, Cuil's Twiceler indexing bot is so bandwith-hungry that it's shutting down sites all over the Web: [...] [...] might just be taking out the entire Internet, one site at a time. As TechCrunch’s Don Reisinger reports, Cuil’s Twiceler indexing bot is so bandwith-hungry that it’s shutting down sites all over the Web: [...]

]]> By: » 100 | seoFM - der erste deutsche PodCast für SEOs und Online-Marketer http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2463604 » 100 | seoFM - der erste deutsche PodCast für SEOs und Online-Marketer Tue, 09 Sep 2008 18:27:56 +0000 http://www.techcrunch.com/?p=21626#comment-2463604 [...] Killt Cuil Websites? - Mehr Hunger als Slurp [...] [...] Killt Cuil Websites? – Mehr Hunger als Slurp [...]

]]> By: David http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2462644 David Mon, 08 Sep 2008 21:44:51 +0000 http://www.techcrunch.com/?p=21626#comment-2462644 This isn't a new problem with Cuil, in fact I've got an email from Cuil going back to 8/7/07 when I brought this problem to their attention. They responded saying that they were working on the bot and would block my site from the spider. I guess they still haven't fixed that problem over 1 year later. This isn’t a new problem with Cuil, in fact I’ve got an email from Cuil going back to 8/7/07 when I brought this problem to their attention. They responded saying that they were working on the bot and would block my site from the spider.

I guess they still haven’t fixed that problem over 1 year later.

]]> By: Favorite Posts of the Week: Aug. 31 - Sept. 6 « Techin’ it up http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2461548 Favorite Posts of the Week: Aug. 31 - Sept. 6 « Techin’ it up Sun, 07 Sep 2008 20:15:43 +0000 http://www.techcrunch.com/?p=21626#comment-2461548 [...] TechCrunch - Is Cuil Killing Websites? [...] [...] TechCrunch – Is Cuil Killing Websites? [...]

]]> By: BlogBuzz September 6, 2008 | Webmaster-Source http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2460535 BlogBuzz September 6, 2008 | Webmaster-Source Sat, 06 Sep 2008 10:01:50 +0000 http://www.techcrunch.com/?p=21626#comment-2460535 [...] Is Cuil Killing Websites? [...] [...] Is Cuil Killing Websites? [...]

]]> By: riccruz.com» Blog Archive » Google Killer Strikes Again But Kills Everybody But Google http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2460130 riccruz.com» Blog Archive » Google Killer Strikes Again But Kills Everybody But Google Fri, 05 Sep 2008 21:07:10 +0000 http://www.techcrunch.com/?p=21626#comment-2460130 [...] seems to be killing websites while indexing it. Way to go Cuil ” The Google Killer ” read more | digg [...] [...] seems to be killing websites while indexing it. Way to go Cuil ” The Google Killer ” read more | digg [...]

]]> By: Geek in Heels » Blog Archive Thursday Geek-isms http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2458678 Geek in Heels » Blog Archive Thursday Geek-isms Fri, 05 Sep 2008 00:53:04 +0000 http://www.techcrunch.com/?p=21626#comment-2458678 [...] Is Cuil Killing Websites? TechCrunch. Another step back for the company that marketed itself as the “Google Killer.” I’m starting to feel somewhat sorry for the guys. [...] [...] Is Cuil Killing Websites? TechCrunch. Another step back for the company that marketed itself as the “Google Killer.” I’m starting to feel somewhat sorry for the guys. [...]

]]> By: iwebie http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2457238 iwebie Thu, 04 Sep 2008 16:49:18 +0000 http://www.techcrunch.com/?p=21626#comment-2457238 Cuil will stick around im sure.. they will not compete with Google. Google has become a part of our vocabulary. Google it! http://www.iwebie.com/cuil-vs-google Cuil will stick around im sure.. they will not compete with Google. Google has become a part of our vocabulary. Google it!

http://www.iweb.../cuil-vs-google

]]> By: Royal Pingdom » The major Internet outages so far in 2008 http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2457158 Royal Pingdom » The major Internet outages so far in 2008 Thu, 04 Sep 2008 15:35:38 +0000 http://www.techcrunch.com/?p=21626#comment-2457158 [...] able to handle the number of visitors they got. Now there are also rumors that Cuil itself is causing downtime for some sites due to excessive crawling by its indexing [...] [...] able to handle the number of visitors they got. Now there are also rumors that Cuil itself is causing downtime for some sites due to excessive crawling by its indexing [...]

]]> By: Cuil e lo spider assassino | FDS http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2456800 Cuil e lo spider assassino | FDS Thu, 04 Sep 2008 08:21:54 +0000 http://www.techcrunch.com/?p=21626#comment-2456800 [...] lamentare l’invasività di Twiceler, il bot di Cuil, è un lettore di Techcrunch: nel tentativo di [...] [...] lamentare l’invasività di Twiceler, il bot di Cuil, è un lettore di Techcrunch: nel tentativo di [...]

]]> By: NEWS » Blog Archive » Cuil e lo spider assassino http://www.techcrunch.com/2008/09/01/is-cuil-killing-websites/comment-page-2/#comment-2456741 NEWS » Blog Archive » Cuil e lo spider assassino Thu, 04 Sep 2008 06:29:19 +0000 http://www.techcrunch.com/?p=21626#comment-2456741 [...] lamentare l’invasività di Twiceler, il bot di Cuil, è un lettore di Techcrunch: nel tentativo di [...] [...] lamentare l’invasività di Twiceler, il bot di Cuil, è un lettore di Techcrunch: nel tentativo di [...]

]]>