« Previous post
Next post »

March 29, 2008

Image Recognition Problem Finally Solved: Let’s Pay People To Tag Photos

Michael Arrington

79 comments »

Most people have thousands of digital photos sitting on their hard drive. And the vast majority of those photos aren’t tagged or searchable. Want to find the 300 pictures of your youngest son amongst 10,000 others? It’s not going to happen. Unless you’ve been diligently tagging and categorizing those photos over the years, and who does that?

The problem is obvious. The solution, not so much. A trail of failed startups have tried to tackle the problem with a fairly serious application of technology, including: Riya (now focused on ecommerce via Like.com), Ookles (never launched), and Polar Rose (in private beta for nearly a year), among others.

And now suddenly TagCow appears, which allows users to upload photos and have them tagged within a few minutes. The technology appears to be “magic,” meaning there’s no explanation of it.

If there’s a mountain in the photo, it’s tagged. A dog? yep. A yellow cup? Absolutely. It does people, too. Upload an image of a person and say who it is, and all other images you upload will be tagged with that person, too. The service also integrates with Flickr and will auto tag the photos you have on the service.

Thomas Hawk, the CEO of photo site Zooomr, tried the service and declared it “really, really cool,” although he wonders how it works.

The answer is, humans do it. I note that the TagCow site is careful not to say anything about the tagging process, and never use the word “automated” or anything else that would suggests computers are doing the work. Munjal Shah, the founder of Riya/Like, agreed, noting that it recognized a witch in Thomas’ photo - he says this just isn’t something a computer can do today.

I haven’t confirmed this yet. I’ve emailed the company for a description of how the service works but have yet to hear back. Until we do, I’m betting that humans are the taggers. Note that Google has effectively thrown in the towel and uses humans for this kind of work, too.

TagCow appears to be offering the service for free, so the cost side of the business may be a problem for them down the road. And the business is definitely a little sketchy. Worried about the privacy of your data? Just don’t click on their Privacy Policy or Terms of Use: “Privacy policy is TBD.” and “Legal stuff TBD.” Not exactly a way to build confidence.

CrunchBase Information
TagCow
Information provided by CrunchBase
  • Sphere It

This entry was posted on Saturday, March 29th, 2008 at 6:46 pm and is filed under Company & Product Profiles. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

Trackbacks/Pings (Trackback URL)

  1. My daily readings 03/30/2008 « Strange Kite

    March 30th, 2008 at 4:46 am

  2. test 03/30/2008 « Strange Kite

    March 30th, 2008 at 10:31 am

  3. pligg.com

    March 30th, 2008 at 12:53 pm

  4. TechCrunch Japanese アーカイブ » 画像認識技術の課題が解決された―金を払って人間にタグづけさせればよい！

    March 30th, 2008 at 2:29 pm

  5. Wwwhat’s new? - Aplicaciones web gratuitas - » tagcow - Clasifica tus miles de fotos con etiquetas

    March 30th, 2008 at 8:31 pm

  6. if(JTeam && Toolman) {blog.read();} » Blog Archive » Who really uses Mechanical Turks?

    March 31st, 2008 at 12:48 am

  7. TagCow：自动为图片加标签 | SilenceWolf

    April 1st, 2008 at 2:54 am

  8. Automatisk tagging av bildene dine

    April 1st, 2008 at 3:33 am

  9. Computer Vision for Dummies » TechCrunch finally realizes that image recognition has not been solved

    April 1st, 2008 at 5:40 am

  10. » Resim algılama problemi çözüldü mü? Duymadim.com: Bunudamı duymadın? Araç Bilgisayar Bilim Donanım Fragmanlar Genel Komedi Magazin Müzik Mobil Oyunlar Son Dakika Spor Teknoloji Uzay Video Yazılım İnternet Yeni teknolojiler Yeni telef

    April 1st, 2008 at 7:52 am

  11. in Tech Today » Blog Archive » Need A Job? Make $1.20/hour Tagging Photos

    April 10th, 2008 at 9:05 pm

  12. Chinese Photo Site Tuyuan May Do Facial Recognition. Or It May Just Put Boxes Around People’s Heads — The People’s Republic Blog

    April 16th, 2008 at 10:17 am

  13. TagCow Sucks « La de da

    April 19th, 2008 at 1:08 pm

  14. Jhan-so-lee » My super power…

    May 12th, 2008 at 4:15 pm

  15. Techlife » TagCow - Photo tagging made easy

    May 13th, 2008 at 12:34 pm

Comments

RSS feed for comments on this post.

  1. bs

    March 29th, 2008 at 7:02 pm

    stupid

  2. Judson

    March 29th, 2008 at 7:05 pm

    Probably not automated. They probably have a couple people doing it, but they will probably eventually crack under load.

    Interested to hear what they say, if anything.

  3. Chris Thomson

    March 29th, 2008 at 7:06 pm

    I can imagine this happening… but because of their “Privacy Policy” and “Terms of Service” … nope. Lack of business model, too. Hmm…

  4. Steven Hodson

    March 29th, 2008 at 7:08 pm

    I saw Hawk’s mention of it on FriendFeed and went and had a look. Much like you I to found the whole thing rather suspicious. And the TBO for privacy .. should be a warning flag .. run .. run very fast in the opposite direction.

  5. Morgan

    March 29th, 2008 at 7:09 pm

    Sweet, Jeffery Bennett’s “Image Search for the Blind” from SxSW finally got funding

  6. amazed

    March 29th, 2008 at 7:11 pm

    LMAO @ TBD

  7. Alx Klive

    March 29th, 2008 at 7:13 pm

    Sounds like they are using Amazon’s Mechanical Turk

  8. Blowski

    March 29th, 2008 at 7:17 pm

    Besides the obvious interest, who cares whether it’s automated or done by humans? If it is done by people, then it’s basically the mechanical turk, a business model that has had a modicum of success elsewhere.

    For the average snapper, tagging is something you have to do yourself. If you get somebody/something else to tag your photos how will it know when/where/why a photo was taken, or who is in the photo? It might recognise famous faces and places, but what about a generic flower, or my baby nephew?

    Professional photographers might pay since more tags leads to more leads. I have no idea whether photographers currently do this themselves for sites like iStockPhoto, but if not, there is a potential business model in there.

  9. Leila Boujnane

    March 29th, 2008 at 7:19 pm

    Michael: I work in the image recognition and processing field (have been for a decade) and your answer is correct: the tagging is definitely not an automated process; it is manual. Total BS given the tag line “automatically tag thousands of photos”; perhaps “automatically” to them means “tagged by somebody else while you are not looking”.

  10. Michael Arrington

    March 29th, 2008 at 7:20 pm

    Blowski - it’s only an important distinction if you see a difference between, say, a secretary taking dictation v. a word processor on a computer.

  11. Michael Kimsal

    March 29th, 2008 at 7:23 pm

    what I don’t get is how they can tell who ‘dad’ or ‘emma’ are in the pictures I’d upload for tagging. For generic stuff (car, mountain, people, sunset, etc) this seems pretty straightforward, and yeah, they’re likely using mechanical turk or something like it.

  12. Susan Beebe

    March 29th, 2008 at 7:24 pm

    The *automagical* tagging technology ‘behind’ the http://www.Tagcow.com site is definitely not clear. Ok, so you think “humans do it.” Now, I am picturing this… there’s like this room of terribly sweaty, overworked ‘interns’ cranking out tags to match photos?! are you serious?

    If that’s the case, then wow, that’s definitely a ridiculous business plan; not viable.

    Don’t even get me started on the privacy policy “TBD” bit.

  13. Chris Thomson

    March 29th, 2008 at 7:24 pm

    “tagged by somebody else while you are not looking” -#9

    That’s probably exactly why they don’t have a privacy policy up yet.

  14. Blowski

    March 29th, 2008 at 7:38 pm

    @Michael: I would have compared it to a secretary taking dictation and voice-recognition software. You go with whichever gives you the best letter.

  15. Earle Ady

    March 29th, 2008 at 7:55 pm

    This is interesting. The complexities of solving this problem from a technical standpoint are computationally difficult to say the least.

    I think the better question here is, “Who Cares”? If its done by a human, its going to be more accurate….

    So if it works, and they’ll tag your photos for free, and it’s accurate — who cares?

  16. A

    March 29th, 2008 at 8:16 pm

    This business model was crushed by Luis von Ahn years ago.

  17. matt

    March 29th, 2008 at 8:16 pm

    “I’ve emailed the company for a description of how the service…”

    Looks like they want to keep it a secret. If the service is a success you will see numerous oDesk job posting for “image taggers”

  18. Jono

    March 29th, 2008 at 8:28 pm

    Personally I’m a little uncomfortable with humans looking at all my photos.

    Imagine if Gmail ads were added by humans reading your emails.

  19. greg

    March 29th, 2008 at 8:36 pm

    Why can’t it be a combination?
    80% automatic 20% human validation. The better it gets the more automatic it gets. Unlike riya it’s a progresive approach that learns from training.

  20. Kamal

    March 29th, 2008 at 9:09 pm

    Its not the business model that is the problem. Its viable as they can get it done at low cost destinations and the work can be done with little or no educational qualifications. Its the privacy issues that come to the fore.

  21. CARversation.com

    March 29th, 2008 at 9:22 pm

    Just Absurd

  22. amay

    March 29th, 2008 at 9:24 pm

    watch like they want to keep it a secret. If the service is a success you will see numerous

  23. Adam

    March 29th, 2008 at 9:43 pm

    It sounds like they’re ripping off Luis Von Ahn’s research:

    http://www.cs.cmu.edu/~biglou/

  24. ventureblogalist

    March 29th, 2008 at 9:57 pm

    cocomment test

  25. Rob

    March 29th, 2008 at 9:58 pm

    If there is a human factor to the matching process, you should have picked a better topic :) “Hey we got this figured out! Not!” could have worked.

    Also what are the privacy implications of this type of service?

  26. Mia

    March 29th, 2008 at 9:59 pm

    Sounds like a good idea in theory, but other than the obvious question marks around revenue model, operation, and everything else, there’s one more teensy weensy issue. How many people tag their photos by ‘mountain’, ‘yellow’ cup, ie. random description? I rarely tag my photos due to sheer laziness, but am I the only crazy person who does it by name of the people in the picture, event, date, location, ie. relevant details? How can any system, automated or monkey-operated do that?

  27. Markus

    March 29th, 2008 at 10:11 pm

    Umm, this wouldn’t have anything to do with a certain date fast approaching would it?

    You know, the date that makes people look like fools ;-)

  28. TriviaMania.com

    March 29th, 2008 at 10:19 pm

    Funny post, especially when you finish watch “Deja Vu” with Denzel Washington 10 minutes prior to reading the post!

  29. Han Solo

    March 29th, 2008 at 10:20 pm

    Thanks Mike, but an endorsement from a person who copies wholeseale another site’s code (*hello Thomas Hawk* *hello Zoooommerrr*) is hardly to be trusted.

    (unless of course they’re sending coke&ho’s your way, in which case I understand)

  30. FlashChick

    March 29th, 2008 at 10:24 pm

    Looks like they upgraded the tagline with the word “Automatically”

    ‘Automatically tag thousands of photos and make your photo library yours again’

  31. Sunil

    March 29th, 2008 at 10:25 pm

    Make no sense.

  32. Eric Xu

    March 29th, 2008 at 10:38 pm

    Manually? No way. This technology doesn’t seem to be new. I have been using oculr.com for a while.

  33. Vantrix

    March 29th, 2008 at 10:41 pm

    Too true to be believed at the moment if they are doing it automated way…

  34. ben

    March 29th, 2008 at 10:54 pm

    @ 23: Adam:

    Thanks for the link, amazing video on Human Computation here : http://video.google.com/videop.....0976635143

    Its the google techtalk from Luis Von Ahn … I started to watch it for a few minutes but could not stop until the end ^^

  35. Binaryday

    March 29th, 2008 at 11:04 pm

    It is practically impossible to get the tagging done by humans. I mean how many people can they employ? Hence my sense is it must be a combination of technology and human tagging. I mean the technology can tag 80% of the images and the humans can correct the rest 20%.

    I believe that is a good way strategy to enter this market. Enter the market, create a presence, then keep investing in the technology to a point where it becomes 95% or so correct. I think the human intervention shall be required for quite sometime to come.

    End of the day, as an end user I do not care how they do it. I just want to get my images tagged properly.

  36. tim

    March 29th, 2008 at 11:08 pm

    In corner A: letting TagCow hire random people to tag my *public* Flickr photos.

    In corner B: AOL, MSN, and Yahoo hand over their logs to the U.S. federal government when requested, and not even as part of a criminal investigation, and without telling their users.

    And I’m supposed to be concerned about TagCow? WTF?

  37. annonymous

    March 29th, 2008 at 11:22 pm

    #36, u have a point. i guess most ppl willnot be worried abt privacy. just look at how they put everything public in flickr,social n/w etc atleast they have some market.

  38. Bashar

    March 30th, 2008 at 12:32 am

    We’ve seen loads of ridiculous ideas turn into serious business on the WWW, but I would not bet on this one. Google Image Labeler is using human labeling based on Luis Von Ahn research and project, which is based on making use of people fun time. Paying for labeling is a nasty costly business you can’t afford it. All the time and money it takes to keep up with people uploading their some junk photos.

  39. GeMe

    March 30th, 2008 at 12:33 am

    This human-tagging process reminds me of Yahoo whose directory of URL is maintained manually by human. User suggests interesting links to Yahoo which they then add it to portfolio.

    However, human-tagging may not be efficient if the system gain critical mass. It will have to hire more human to tag the content, and that some content may just be missed due to human’s oversight. Moreover, it might be scalable.

  40. Thomas Hawk

    March 30th, 2008 at 6:51 am

    Also what are the privacy implications of this type of service?

    At least in my case I wasn’t really worried about their privacy policy. 100% of my Flickr photos are already public photos. Anyone and everyone in the whole world already has access to these photos by my choice to make them public images. Certainly someone with *private* photos might not want to turn them over to tagcow unless they felt comfortable with their privacy policy.

    Personally I was pretty impressed with the job that they did tagging my photos. They added thousands of highly descriptive tags. This means that more of my photos will likely show up in image searches both on Flickr as well as on Yahoo by extension as the photos are indexed for theses new tags.

    Personally I think that tagcow uses a combination of AI and human resources. I noticed what felt like text recognition on a lot of my photos that had text in them. But they also tagged many of my photos using highly complex conceptual tags that I don’t believe a machine could come up with.

    I believe that there possibly could be clustering of sorts going on as well but am not sure on this. For instance. If a photo is tagged witch, might you not automatically tag a photo with this tag also with “wicked.” Similarly, if a photo was tagged sunrise, might you also autotag the photo morning, etc.

    Certainly human tagging might not be economically feasible. Certainly it might not also scale. But, at least for the time being, it was pretty cool to have someone/somehow spend hours and hours making my photos on Flickr more descriptive at no cost to me. And whatever the case, one of the cooler things I’ve ever seen done on a personal level with the Flickr API.

  41. big

    March 30th, 2008 at 8:11 am

    interest

  42. john

    March 30th, 2008 at 12:58 pm

    this is like mechanical turk over at amazon isnt it?

  43. Niraj

    March 30th, 2008 at 1:44 pm

    “Note that Google has effectively thrown in the towel and uses humans for this kind of work, too.”

    I very much doubt that. If anything, they could easily use the information they get to train an algorithm. I’m sure the image labeler is a small piece that will help to feed some kind of automated recognition that could actually scale.

  44. Google FanBoy

    March 30th, 2008 at 2:51 pm

    Google hires 1,000’s of (contract) workers across the globe to classify and QA their web results.

    QED.

  45. Rajaram

    March 30th, 2008 at 3:16 pm

    “…Due to unexpected load (during our beta), tagging of photos will be delayed…”
    Now that clears things up :-)

  46. Sudoku Maniac

    March 30th, 2008 at 3:48 pm

    will they integrate with picasa ?

  47. Tony Chen

    March 30th, 2008 at 4:15 pm

    Its probably being done through Amazon’s Mechanical Turk.

    http://www.mturk.com

  48. Rupert Goldie

    March 30th, 2008 at 6:18 pm

    It’s “powered by” Munio Memory services which sells image insurance

    “Memory Insurance protects all your photos, children’s art, documents and journals.”

    So the free tagging service is most likely an attempt to attract customers for the paid service.

  49. jc

    March 30th, 2008 at 8:26 pm

    We will have to wait for the official word. Answers should have been before the launch of the site, without mentioning that privacy conditions are not clear either.

  50. Hendra

    March 31st, 2008 at 12:17 am

    I’m getting:

    503 Service Temporarily Unavailable

    The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

    Are they experiencing the techcrunch-effect already? ;)

  51. wikky

    March 31st, 2008 at 1:13 am

    http://www.china-kitchen-cabinets.com

  52. dailytwitter

    March 31st, 2008 at 2:04 am

    Has anyone of you suckers ever thought that this could be an April Fool joke given its April 1st tomorrow - maybe a get in early get you all excited and then pull the rug from under your feet routine….

    I shall be looking at this, but geepers yes its complex, but not impossible.

    another note from the dailytwitter

  53. kendy

    March 31st, 2008 at 2:47 am

    Top quality watches and handbags
    http://www.watchhandbag.com

  54. Susi Doring

    March 31st, 2008 at 3:11 am

    A good way to keep your photos organized (aside from human tagging) is with 2pad, an email based photo sharing application. Users send their photos and/or videos to 2pad@pad.com and store their media in their own private online galleries.

    Since 2pad stores the content of their emails with their photos. Users are able to search with keywords to find their media.

    A different solution to this human tagging approach.

  55. Harry Wang

    March 31st, 2008 at 8:29 am

    This task will be human powered for some time to come. I speak from experience. Poor, poor Wang!

    Harry “I bet they hired a slew of third worldies to do this” Wang

  56. Jay

    March 31st, 2008 at 12:44 pm

    Strange, searched on the main guys listed for the company and found another company they all founded as well: http://www.muniomemoryservices.com

    two companies doing the samething under different names by the same people????

  57. NB

    April 1st, 2008 at 9:56 pm

    hmmm….looking at the number of HITS that has been waiting for a worker at Mechanical Turk, they will be having indefinite delays in tagging it seems :) Another I have more money than sense, money-down-the-drain, web 2.0 madness.

  58. B

    April 1st, 2008 at 10:42 pm

    It is done through amazon mechanical turk.

    Workers get 5 photos and are paid $.04 to label all the text in the photo and to “Describe, using simple, generic terms, what you see in the image. (Examples: family, beach, drawing, child, child eating spaghetti, letter, disneyland, envelope, greeting card, etc.)”

    Tagcow’s response to the increased load has been an offer of a $10 bonus to every 200 hits completed (that’s 1000 photos as each hit has 5 photos).

    Be aware since anyone can sign up to mechanical turk, all your photos you want tagged are viewable by the public.

  59. Jerry

    April 1st, 2008 at 11:03 pm

    I tried a few. I tagged all images group sex, orgy, fellatio, swingers, voyeur etc. They were some guy’s Christmas photos

  60. Harry Wang

    April 2nd, 2008 at 11:01 am

    @59: Double-blind my friend, double-blind.

    Harry “covering all the angles” Wang

  61. Jerry

    April 6th, 2008 at 4:10 am

    @60 I have multiple accounts my friend, I have multiple accounts

    Jerry “hates fucking chinks” Wank

  62. dunke yao ming

    April 8th, 2008 at 3:46 am

    new knowlegre

  63. ClintJCL

    April 19th, 2008 at 1:12 pm

    Yea, this service just plain doesn’t work. Check it out; my wife guinea pig’ed it for me and it SUCKS SUCKS SUCKS SUCKS as a service:
    http://carolyncasl.wordpress.c.....cow-sucks/

    “I was a guinea pig for TagCow We figured if this was going to do what it says it’d do, that save us a lot of time tagging our pictures.
    In order to use TagCow, you have to register with their site, and upload the pictures to them. You can sign up for their flickr service, which I did, and they are supposed to tag your photo and then upload it to flickr. In order to tag people, you have to upload a picture of that person and assign their name to that picture. I uploaded a picture of me and Clint. Then I uploaded one picture. Several days later, the picture still wasn’t tagged.

    One day after I signed up and uploaded my one test picture, it was confirmed that it is actual humans tagging the pictures. I reported to Clint that my picture still wasn’t tagged. A week later, it still wasn’t tagged.

    Guess what.. I forgot about it.

    Until today. After realizing what the mystery was (”Hey, TagCow finally tagged my picture!”), we realized that it is still not perfect. First of all, why did they change the spaces to underscores in the title? That makes the title stretch all the way across the screen, and I have to actually use the horizontal scroll-bar! Secondly, it tagged Clint in the picture, but Clint isn’t in that picture. Maybe the human thinks Justin looks like the picture of Clint I uploaded? Who knows.

    Anyway, I will not be using TagCow in the future. It’s a neat idea, but in practice, they have a long way to go.”

  64. ClintJCL

    April 19th, 2008 at 1:12 pm

    ^ 19 days to get 5 tags on 1 picture. 1 of the tags was wrong, AND they messed up the flickr page by converting spaces to underscores.

« Previous post
Next post »