When famous computer scientist Jim Gray went missing a few days ago, the coast guard launched a large scale search that found absolutely nothing. On Thursday, they gave up.
Then Amazon stepped in. They arranged for a satellite sweep of the area and stored the images on their S3 storage service. They then created a task on their Mechanical Turk service to allow volunteers to scan the images to look for the boat. It’s a tough task - the boat would only be about six pixels in size in an image, and there was a lot of cloud cover obscuring large parts of the area scanned. But volunteers are pouring in to help out.
If you’d like to help, go to this task on Mechanical Turk. You’ll be asked to view five satellite images and note any that should be looked at more closely.
I hope that new web technologies will make helping people more common in the future. When tragedy strikes, people may be able to help in an effective and organized way instead of simply watching events unfold on their television.
I’m off to look at satellite images of the water outside of the San Francisco Bay. Please come help me.
This has been covered on local news, KRON4, NBC11, KGO7, and et al. but rarely do they talk about Jim Gray and his work, which is a really really big deal. I hope someone finds him still alive.
dang…thats terrible!
This kind of effort will no doubt save a life one day (and I hope it saves this one). Steve Jobs will go missing on the moon and everyone will be (effortlessly) panning/zooming lunar landscape photos on their iPhone, trying to find him.
It’s about time! Hurrah TechCrunch! Thank you!
If you look into Jim’s work a bit more, you’ll realize that most of us use something he helped create every day. For example, transaction integrity in DBMS, and concurrency control… allows us to maintain bank balances and charge credit cards on computers, in provably correct transactions.
Too bad this news is like 4 days old.
Im in!
This is like a modernized version of “Where’s Waldo”.
Here’s a post with a chronology of the search efforts and links to earlier video clips and articles: http://oakleafblog.blogspot.co.....-gray.html
Mike, thanks for your support.
We have now added the data from yesterdays NASA ER-2 flight to system. They are in a Mechanical Turk Group that is separate from the satellite images search. Please follow the link below to work on these.
http://www.mturk.com/mturk/pre.....MZYKZ34XSZ
We will load more satellite images soon also.
Actually delete that previous comment. Having two groups is too confusing. We have reloaded the data into the old group:
http://www.mturk.com/mturk/pre.....JZ5QY4F9M0
Sorry for the confusion
While I agree this type of search-and-rescue mission has the potential for a lot of good, this begs the question if we can expect the same type of Big Brother, a la 1984, oversight of our where-abouts.
Don’t get me wrong, I agree that this is a cheap, effective, benevolent effort on Amazon/the Internet’s part. Clearly they are only into this for good reasons. However, it does open the door for the acceptability of using people to find people doing the wrong thing, e.g. http://www.theregister.co.uk/2.....sunbather/
Good you told us about that. I completed 10 HITs so far and will continue to work on that. Could be a successful thing for Amazon, too.. Especially if it worked. It could give the Mechanical Turk a well needed promotion push.
Why does this need to be a manual effort? Given today’s technology, surely someone can develop a program to look for small patterns in the images and at least narrow down the results?
Awesome approach by Amazon. During the Kim search there were questions about how to process Satellite data quickly and accurately. This appears to be a great approach and I hope Amazon will consider making this available for future searches as well.
A great use for the technology, and hopefully a good way for a lot more people to learn about the possibilities with these kinds of technologies.
Get on over there and check it out.
Hope this works & Jim will be found soon!
Wow, those went fast. All the images have been checked. I wonder what the results will be?
Thanks for bringing this to the community’s attention. I’ll check the images and do my best to help.
Thanks,
Jaafer
I was going to try to help…but every image of the ocean I looked at had 200+ 4×10 white pixel blocks. I did not want to dismiss any of the images (fear of being incorrect) or flag any of the images for further review.
I am not sure if this technology will be any help - but I hope he is found.
#12, I was pondering the same. Where is Charlie Eppes (Numb3rs) when you need him.
Actually, if I were going to attempt a computer program to filter out noise, I’d like to start with 3 to 5 images of the same area taken 1 to 2 seconds apart. Then analyse the movement of all whiter pixel groups in relationship to their neighbors. Hopefully, if he has his sail up, he’d be travelling in a different direction or speed from all the waves.
Hope and pray for the best.
i hope they find him. (crying)
I really, really hope they find him.
Very cool that everybody’s helping to get the word out.
I’ll help spread the word. I hope he is found. I looked over a few of the images and to be honest, it is going to take a lot of us looking and re-looking!
Rex
Imagine this technology a few decades from Now
Did Jim not have any working communications devices?
Mammography, chest, and even colon x-ray Computer Aided (cancer) Detection (CAD) software does this kind of thing every day at hospitals now. Kodak, R2, and ICad are just three out of a group of companies providing such FDA aproved CAD systems. Much of that technology started as spin-offs from military image detection technology.
Thus technologically, the task is undoubtly quite doable. Of course, saddly, to update the software to assist in this task would take too long…
I have developed automation workflow to detect, analyze and measure image size and shape as Researcher more than 10 year ago. It is technological possible with image analysis software to calibrate background, filter noises and unwanted image size and shape, detect and highlight interested object using business rules and scripts.
If I successfully automate such image analysis process in early 90, certainly it can be done today in year 2007. This type of image analysis is usually conducted in basic research.
It is possible to highlight images with pre-selected size, shape and color in seconds. Given Jim Gray is missing on Jan 28, 2007 Sun, we have roughly a week to find him conscious before he gets over-dehydrated, assuming he is still on the boat.
Manually reviewing sallite images are too slow. To gather more clues on his whereabout or the boat location, automation is necessary to process thousands of images to cover more images taken at various day/time and larger areas.
I leave my comments here as to give hopes to those who care for Jim Gray and his contribution to Computer Science. If Microsoft runs out of options and time in figuring out image analysis and automation, at least there is one soul in Silicon Valley may be able to share my past technical experience from applied research. And perhaps I can lead the effort in such automation to save future lives.
IE Team Pete LePage and Chris Wilson have my personal email contact. Meanwhile there is a lot of work to do to figure out how to rescue SV and save http://WWW.
Founder of SVWB
http://www.svwebbuilder.com
this is the best example that web2.0 can help evan offline! great ideea.
“I hope that new web technologies will make helping people more common in the future.”
Me too, And i hope they find Jim. A story about me going missing isn’t likely to recieve enough media attention to make it worth while for amazon to care though.
I might be cynical, But face it, Amazon are doing this for selfish reasons, Free advertising and public relations.
Thanks for putting this up, Mike!
~1400 HITS left, btw…
Perhaps we could have done the same for James Kim. Here’s an interesting post: http://www.layoutscene.com/jam.....index.html
This is a really worthy cause, but the user interface of the actual HIT tasks is killing me, and not making good use of the volunteer’s image processing cycles.
The “training” section shows you pictures that are smaller than the one you’re actually going to look at. They give you a red rectangle to give you a rough size of what the boat should look like as “training”. But the red rectangle/boat-proxy is much larger in size relative to the small sample square than what a Jim’s-boat-sized area in the big, to-be searched images would look like.
This reality is noted in the small text instructions next to the “training” area, but text instructions like that have a hard time comparing to a nice visual comparison. Sadly, the only example we have of what to look for, is the wrong size.
CALL TO ACTION–Search Team Responsible for this Mechanical Turk Task:
1. How about an example of an actual sail boat in water, with the example image the SAME SIZE as the images to be searched? This information must exist, correct? Even a boat in a boat in the middle of the San Francisco Bay, in the right scale, would be helpful.
Right now, we have zero idea of what success or “something of interest” would look like when going through these images. I want to donate some human brain cycles, but this is really frustrating.
2. Failing that, how about a training sample image the SAME size of the images actually being searched by the volunteers–preferably NEXT to the space on the page where the image being searched shows up– and featuring a rectangle THE SAME COLOR as Jim’s boat. Not a red one. Give us something that you can point to and say “look for something that looks like this.”
If this can’t be done, searchers, be advised that a better size reference is the arrow point of your mouse cursor, which is about 10 px tall if your screen resolution is ~1000 pixels tall (my laptop is 1400×1050).
RE: Post 31 above:
Can we get a more realistic, and applicable to this particular scenario, sample image like this boat in the San Francisco bay: http://maps.google.com/?ie=UTF.....23&t=h
I don’t know what the right scale is, and obviously the calm water is different than the choppy open ocean waves.
However, the altitude from which I captured that link makes the boat in the dead center around 10 pixels tall. That boat appears to be a roughly 30 foot-ish boat, a little smaller than Jim’s boat. The link was captured from a sailboat marina near Pier 40 in San Francisco.
Something like this, but in the open ocean, would be really helpful as a reference.
Michael:
Thanks for bringing this seminal ‘real-world search by tech proxy’ to our attention. We join the many who hope for Jim’s quick and safe return to his friends and family, and remain thankful for them allowing us to be part of the search.
Clearly this incident is a turning point in showcasing the teamwork both beneficial and necessary to saving lives in times of duress.
Our staff at the Los Angeles Fire Department looks forward to participating in expanded trials of such open, seamless and participatory endeavors.
Respectfully Yours in Safety and Service,
Brian Humphrey
Firefighter/Specialist
Public Service Officer
Los Angeles Fire Department
LAFD Home Page: http://www.lafd.org
LAFD News Blog: http://www.lafd.org/blog
This is seems to be one of the highest level of technologies human being ever used. well, if we can use these for good human development values why not?
I hope Gray would be first among all of us.
IE Team Pete LePage just forward my personal email.
Scenario analysis:
It is very important to gather all the reliable facts, location/date/time of his sailing, boat specification, weather condition, ocean current, his medical condition, his sailing habits, his boat history and condition, food/water storage inside the boat, etc.
Useful:
Coastal guard and private plane’s coverage, date/time/altitude. We need to understand why they can’t find any sign or debris.
Boat photos:
As many as possible to draw any clues
Satellite Photos:
It is important to have access to the highest resolution images to identify pixel patterns, shape and color.
Founder of SVWB
http://www.svwebbuilder.com
Note: Would any web professionals help us to post our next Feb 28 meeting Browser Wars Episode II Attack of the DOMs to your professional network? Or tell your professional friends? http://browserwarii.eventbrite.com. I have to deliver no. of attendees for our speakers and sponsors and I am handling the Marketing campaign without any marketing budget.
Microsoft Question:
Does Microsoft R&D team or headquarter has any Image Analysis software license?
Budget Question:
Any funding to put Image Analysis Automation operation together in very short time? multiple machines/servers
Jim Gray’s Clothing:
What is he wearing? color of his clothing? What did he bring to his boat as far as clothing?
Talent Question:
Am I the ONLY one in web space have done this kind of image analysis and automation? Hellooooo! Anyone raise your mouse?
The search for Mr. Jim Gray assumes sea only?
I mean: Is there confirmation that his boat is not anchored in the bay?
You guys should check out this Greasemonkey script I’ve written to speed up the search. It allows you to process those satellite images on Mechanical Turk much faster by pressing ‘+’ or ‘-’ on your keyboard.
http://blog.nometers.com/?p=25
Its Feb 25th…what the result and or status of the mechnical turk endevour?