« Previous post
Next post »

September 1, 2006

Google Image Labeler Uses Human Labor

Michael Arrington

82 comments »

Google Image Labeler launched today. It’s a game, based on Luis von Ahn’s ESP Game, that puts two random users together and asks them to label/tag an image. The idea is that if two people come up with the same label, it is probably a good one and will make Google’s image search better.

See Google Blogoscoped for the initial story and Danny Sullivan who found a ton of additional facts, including a video by Luis von Ahn stating that this game could effectively label all Google indexed images in two months.

I tried the game. It’s fun, in a why-am-I-doing-this kind of way. I focused on labeling everything I saw as “purple” and “Donald Trump”.

  • Sphere It

This entry was posted on Friday, September 1st, 2006 at 11:15 am and is filed under Company & Product Profiles. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

Trackbacks/Pings (Trackback URL)

  1. TechCrunch Japanese アーカイブ » Google Image Labeler、人力でチューンアップ！

    September 1st, 2006 at 1:21 pm

  2. Diogo Azevedo

    September 1st, 2006 at 1:42 pm

  3. Google Image Labeler « Life of Robert

    September 1st, 2006 at 2:52 pm

  4. Google Image Labeler relies on crowdshop labor at FactoryCity

    September 1st, 2006 at 3:27 pm

  5. PJ Kix > Hi-tek / Lo-life » Blog Archive » Techcrunch » Blog Archive » Google Image Labeler Uses Human Labor

    September 1st, 2006 at 6:03 pm

  6. Technoogle » Blog Archive » Google Image Labeler, Play for Google’s Benefit

    September 1st, 2006 at 7:49 pm

  7. duncanriley.com » Google Image Labeler launched

    September 1st, 2006 at 9:44 pm

  8. Ajaxian » Google Image Labeler: Collaborative Tagging Game

    September 2nd, 2006 at 3:43 am

  9. 國生三年才開始

    September 2nd, 2006 at 6:00 am

  10. [[Woid's Blog|Woidův blog]]

    September 2nd, 2006 at 9:37 am

  11. www.techtagg.com - See Tech Taggers view on this story!

    September 3rd, 2006 at 7:15 am

  12. Google Image Labeler Uses Human Labor » Dee’s-Planet! Blog

    September 3rd, 2006 at 12:35 pm

  13. dmense - beyond the web 2.0 hype

    September 3rd, 2006 at 9:42 pm

  14. Google Image Labeler Uses Human Labor « Ultimate Bokness

    September 4th, 2006 at 12:04 pm

  15. CollaBlog » Blog Archive » Un “gioco” basato sul collaborative tagging

    September 5th, 2006 at 6:49 am

  16. Joheve » Google Image Labeler

    September 6th, 2006 at 7:13 am

  17. Techcrunch » Blog Archive » Blufr tests your b.s. radar

    September 14th, 2006 at 5:22 pm

  18. accelzone - techie weblog » Blufr tests your b.s. radar

    September 14th, 2006 at 5:29 pm

  19. Blufr tests your b.s. radar » JenIT

    September 18th, 2006 at 1:32 pm

  20. le blog des mash-up

    September 19th, 2006 at 6:16 am

  21. Google Image Labeler at Shavkatov’s Blog

    September 23rd, 2006 at 12:47 am

  22. Techcrunch » Blog Archive » All The Cool Kids Are Deep Tagging

    October 1st, 2006 at 11:18 pm

  23. Techcrunch » Blog Archive » Helium: click to compare other peoples’ topical writing

    October 11th, 2006 at 12:15 pm

  24. Techcrunch » Blog Archive » Riya’s Like.com Is First True Visual Image Search

    November 8th, 2006 at 12:57 am

  25. TechCrunch en français » Riya lance Like.com, le premier vrai moteur de recherche visuel

    November 8th, 2006 at 9:03 am

  26. Riya’s Like.com Is First True Visual Image Search » JenIT

    November 9th, 2006 at 11:59 am

  27. Google Image Labeler Fun - CyberNet News: Keeping you plugged in!

    December 3rd, 2006 at 11:20 pm

  28. Multimedias.mobi » Ookles To Launch in Early 2007

    December 18th, 2006 at 9:26 pm

  29. Techcrunch » Blog Archive » Polar Rose: Europe’s Entrant Into Facial Recognition

    December 19th, 2006 at 1:51 am

  30. Polar Rose: Europe’s Entrant Into Facial Recognition at Swiss Podcast Directory and Blog

    December 19th, 2006 at 3:09 am

  31. TechCrunch en français » Polar Rose: un nouvel entrant en Europe dans la reconnaissance faciale

    December 19th, 2006 at 6:41 am

  32. Start-up.co.nz » Due Picture Site set to launch in 2007

    December 20th, 2006 at 1:19 pm

  33. Polar Rose: Europe’s Entrant Into Facial Recognition

    January 8th, 2007 at 8:49 pm

  34. Google Image Labeler « iLEMONed

    January 28th, 2007 at 8:51 pm

  35. Facebook Taps Users To Create Translated Versions Of Site. Spanish, French and German Available Now : New Web 2.0 Magazine

    January 22nd, 2008 at 2:23 pm

  36. TechCrunch en français » TagCow tagge toutes vos photos pour vous...mais comment font-ils?

    March 29th, 2008 at 11:34 pm

  37. Google Experiments With Next Generation Image Search

    April 27th, 2008 at 11:33 pm

  38. Google Experiments With Next Generation Image Search

    April 27th, 2008 at 11:33 pm

  39. TechCrunch en français » Google expérimente la nouvelle génération de recherche d'images

    April 28th, 2008 at 12:37 am

  40. Google-Powered Work Avoidance: Image Labeler « Burned At Both Ends

    April 28th, 2008 at 1:30 am

  41. socialinnovation3.com » Blog Archive » Google Experiments With Next Generation Image Search

    April 28th, 2008 at 8:28 am

Comments

RSS feed for comments on this post.

  1. Charlie

    September 1st, 2006 at 11:21 am

    Google only save the results that match, so you’d have no luck whatsover… unless it was indeed a purple Donald Trump.

  2. Ryan W.

    September 1st, 2006 at 11:47 am

    After about 2 days, when the novelty wears off, why would users do this? Especially enough users to label google’s entire image index of billions of images? No incentive.

  3. stephen

    September 1st, 2006 at 11:54 am

    Maybe google will allow people that have X points to buy something in its store… or a site that use google checkout….

  4. Alex Iskold

    September 1st, 2006 at 11:54 am

    Totally agree with Ryan. There is no feedback loop. Some images are just plain weird and give you a headache. You need to do it the Flickr way, where people who post images have a vested interest in labeling them.

    Alex

  5. Kingsley Joseph

    September 1st, 2006 at 12:11 pm

    I think Google are smart enough to know that they’ll need an incentive system to make it work.

  6. Andrew

    September 1st, 2006 at 12:13 pm

    I noticed if you get a couple matchs with someone, that you can start to guess what words the other will using, but words that skew the meaning, or a re just simple artifacts found in the image such as colour.

    I would do this for hours if i could earn points towards Google Gear; or a T-shirt that says something funny; but I can think of anything funny…

  7. Andrew

    September 1st, 2006 at 12:17 pm

    I also wounder what physiological data Google can learn about you–adding to the value of your Gaccount–by the terms you use to rank images. Or the effect the terms you use to rate things to modify the personalised search behaviour Google uses to find stuff for you as an individual.

  8. Joseph Price

    September 1st, 2006 at 12:18 pm

    Ok, here’s an idea. How about contributors to the system earns credits that be used to buying content that’s on Google Video?

    I think that would be a sustainable incentive system.

  9. Orli Yakuel

    September 1st, 2006 at 12:22 pm

    Ryan, It’s like asking why people writing in wikipedia, for example.
    This is what we do when we spending so much hours on the Net,
    we’re either consume or provide information and knowledge FOR FREE.

  10. Jason M. Lemkin

    September 1st, 2006 at 12:31 pm

    As if we didn’t all already work for Google already in one form or another . . . this is my new diversion while on a boring call . . .

  11. Alan Patrick

    September 1st, 2006 at 12:37 pm

    Google blinks!

    It was Yahoo that went down the McTagging route first, Google initially tried to crack it all via Big Algorithms.

    Interesting…..

  12. ToddW

    September 1st, 2006 at 12:48 pm

    Interesting idea! Get the users to do the work for google for free!

  13. Jacob

    September 1st, 2006 at 12:55 pm

    Oh i like Andrew’s thought on google possibly using the data collected for personal search. I’m sure a lot could be deducted from the level of description that you use to help rate your search queries based around your potential IQ or knowledge in a particular area.

  14. Drew Olanoff

    September 1st, 2006 at 1:14 pm

    Too. Addictive. Must. Stop.

    Brilliant.

  15. Yuva Mani

    September 1st, 2006 at 1:15 pm

    Use Mechanical Turk … Use Mechanical Turk …

  16. piranha

    September 1st, 2006 at 1:17 pm

    oh so bad for me. :)

    why do we do anything for free? *duh*. because we like to share our knowledge. i use google image search a huge amount; i sure don’t mind helping improve it.

  17. Tom

    September 1st, 2006 at 1:28 pm

    This is pretty good fun, the key to getting a high score is choosing simple words, for example in one round I matched 3 pictures with ‘people’

  18. Nathan

    September 1st, 2006 at 1:43 pm

    This will work fine and dandy for simple/generic stuff. What happens, though, when one is faced with tagging a picture of a Colaptes cafer

  19. John

    September 1st, 2006 at 2:05 pm

    Interesting. I remember reading somewhere that Larry or Sergey thought that the best way to index was through computational means, not human… (including images).

  20. NJG from NYC

    September 1st, 2006 at 2:20 pm

    it was fun for about 5 minutes, then I got a bit bored. I’d keep doing it if I could earn a free t-shirt or some other benefit.

  21. Kalpesh Patel

    September 1st, 2006 at 2:42 pm

    It was no fun as the partner was dumb

  22. Susan Peterson

    September 1st, 2006 at 2:54 pm

    I think the idea of contributing to the Google Machine
    is kinda neat. My goodness, they have saved my
    butt many times with their browser!!

    Kudos to All Mighty Google!! (Hail, Hail).

    Excelsior!

  23. Arjun

    September 1st, 2006 at 3:13 pm

    whis is, I think, where Riya will clearly out-shine google.

  24. ToddW

    September 1st, 2006 at 3:17 pm

    It sure sucks to be paired up with someone who passes on 90%!! GRrrr!

  25. Ubertor

    September 1st, 2006 at 3:42 pm

    It is kinda fun… for about 2 minutes…. interesting way for Google to get some work done.

  26. Ryan W.

    September 1st, 2006 at 4:19 pm

    to those comparing this with wikipedia or other sites where users provide content for free, I’d say 2 things:

    1. wikipedia is non-profit (although many other web 2.0 sites are not). as such, I think people feel a little better about contributing knowledge knowing that some giant corporation isn’t going to be profiting off their free labor.

    2. writing a wikipedia article is, in my view, totally different than sitting and labeling an endless string of random images in the hopes that some other user will label them the same way and you’ll have contributed something useful. also, in many types of user generated content around the web, users are contributing primarily to benefit themselves (del.icio.us) and the secondary effect is the benefit to other users. it might be interesting for a few minutes, but without an incentive program, it’s not going to be incredibly useful (especially in terms of indexing billions of images in two months!). this is exactly like mechanical turk but without an incentive program…maybe amazon should start up a free mechanical turk program for those of you who (apparently) enjoy mindless repetitive tasks that ultimately pad someone else’s pockets.

    However, if they added some great incentives, I think this is very viable. It’s definitely not revolutionary, as you can accomplish the same thing with mechanical turk, but it’s viable.

  27. Alan

    September 1st, 2006 at 4:42 pm

    Is the Mechanical Turk heavily used…..payments don’t seem hugely enticing?

  28. Awesom-o

    September 1st, 2006 at 6:33 pm

    This is probably about collecting data for their machine learning algorithm. If they get good labels for a a few million images then they can use them to train their algorithm to correctly label the rest of the images in the index. They don’t have to get labels for all the images from the users/players. I believe that was the reason the original ESP Game was created. BTW, the other new game from the ESP guys, Phetch, is also partly about collecting data for machine vision type work.

  29. Jimbo

    September 1st, 2006 at 9:56 pm

    This is the dumbest “game” ever. I have a similar game idea. How about two people compete against each other to assemble products at home. The winner gets to assemble an extra product.

  30. Mike

    September 2nd, 2006 at 12:02 am

    I agree, useless - I tried that and I went through 5 pictures maybe - then I am gone. Another great example how not to launch a webservice.

  31. Grant

    September 2nd, 2006 at 12:05 pm

    You have to somewhat give props to Google for having a good idea to make people work for him. Although, I doubt I’d be a proud “top contributor.”

    It’s like free slavery.

  32. Vijay

    September 2nd, 2006 at 5:28 pm

    Idea is good, but we are helping to Google.

  33. Cam

    September 2nd, 2006 at 8:09 pm

    “I focused on labeling everything I saw as “purple” and “Donald Trump”.”
    Umm. Not Funny Michael.

  34. LeBain

    September 2nd, 2006 at 9:45 pm

    Here’s how to win: Everybody label everything asdf

  35. Mark Finnern

    September 2nd, 2006 at 10:55 pm

    Looks like Google bought the ESP Game from Carnegie Mellon. I played it a couple of years back already: http://www.futuresalon.org/200.....s_on_.html

    All they did is a bit of Google look and feel around it. I think you could do so much more to make it fun and exciting.

    Google where is your creativity?

    Bummer, Mark.

  36. Tom Parker

    September 3rd, 2006 at 8:10 am

    This thing is adictive, I was swearing at the guy who couldnt see the obvious in the pictures…

  37. Drew

    September 3rd, 2006 at 6:17 pm

    there is a major flaw with this! How we see an image when trying to think of words is different to the content and context of the image. Just because two people see a tiny image snippet as a map, does not mean the image holds any value as a map; and certainly should not be the first value returned when you search google for a map.
    C’mon there has to be a better way of Google getting some categorisation on images than this paired matching? Say there is 1 mil images in the database, each needs to be paired a few times to get a valid response, then that valid response needs to be weighted against other valid responses. Thats hundreds of millions of actions.

  38. Jon

    September 3rd, 2006 at 9:37 pm

    When today’s top contributor has the name ‘just_say_tits’ perhaps the results won’t be as good as everyone predicts.

  39. jazzle

    September 3rd, 2006 at 11:20 pm

    The images are far too small! Surely anyone designing & testing this could realise that it makes it impossible to be able to see enough detail to provide decent tags.

  40. Arthur

    September 4th, 2006 at 6:42 am

    Yes, the images need to be at least 300 pixels in size. Plus, why let people find matches simultaneously in pairs? Much easier would be assigning terms individually and automatically matching them up later.

  41. Phil

    September 5th, 2006 at 10:22 am

    Agree with Ryan, what’s the point. I also find the images too small to see easily, even with my reading glasses! :(

« Previous post
Next post »