This morning brings another cautionary tale for anyone trying to build a Website or a business using data from another site. Visual search engine ManagedQ is broken right now because it took images of Websites from another visual search engine, Snap, without permission. (See screenshot above). Sound familiar? Alexaholic (now Statsaholic) ran into similar trouble with Amazon a year ago for taking graphs from Alexa before they were officially available through its API (read more about that dispute here).

It is unfortunate that Snap effectively disabled ManagedQ, which is run by a few programmers out of a basement in Palo Alto. But it goes to show that just because data is becoming more freely available on the Web, you still have to be careful about building a business on another company’s data. It appears that ManagedQ based its visual previews entirely on Snap’s images. As I wrote in a review last month:

Every time you do a search on ManagedQ, a grid appears on the right of the first six results so you can visually see what is on the other side of what is normally a blue link. If you click on one of the images, it opens up a larger, browsable window still within ManagedQ. The idea is that you can surf the Web without leaving the search application.

The way ManagedQ was using the images violated Snap’s terms of service (TOS), according to Snap CEO Tom McGovern. Snap does distribute these images through its Snap Shots widgets. (We use them on TechCrunch. If you mouse over any external link in this post, an image of the Web page on the other end will pop up). After coming across the site, his engineers figured out that ManagedQ was taking the images from Snap without any attribution or link, and cloaking the fact that it had done so. After contacting ManagedQ and not getting a response, McGovern ordered his engineers to block the site’s access to Snap’s images. Warns McGovern:

Folks really need to use services per the TOS. Otherwise they will go the way of ManagedQ or Alexaholic.

Ouch. At least his engineers didn’t replace the Website snap shots with goatse images. But the reaction does seem a bit harsh, especially for a tiny site like ManagedQ. Was McGovern justified in his response? Here’s what ManagedQ looked like before:

Update: ManagedQ founder David Stat has provided the following comment on their shutdown at the hands of Snap:

As we’ve been developing ManagedQ, we looked at several different
thumbnail services and decided on Snap due to their speed and high image
quality. ManagedQ is an experiment with visual Search, not a high volume
Search site. As such, we believed that Snap would not mind our use of
their service and may even encourage its novel and interesting
application. Before using Snap for our site, however, we performed a
traffic analysis and found that ManagedQ would consist of only about
0.01% of Snap’s traffic at most - hardly a share that would affect them
in any meaningful way.

It is most unfortunate that Snap has decided to block us, but I
understand that they are perfectly within their rights to do so. We did
not, however, receive a notice beforehand. We would certainly be
interested in pursuing an agreement with Snap that is outside the bounds
of their normal TOS, but we haven’t yet done so because we thought
ourselves too small for them to consider such a partnership.

Our focus is on continuing to create a new Search Experience with broad
appeal. We believe data should be open by default. We are at a loss
as to why a relatively big startup like Snap would feel threatened by a
small Search experiment like ManagedQ.

Update 2: Snap CEO Tom McGovern has also added these remarks to the situation:

We want sites to use the service in an unadulterated manner where the
actual Snap Shot is shown. There are lots reasons (server load,
business model, end user confusion) that this is important to us. For
developers that are working on a project or offering a commercial
service there are many other companies that offer a developer API
(Girafa, thumbshots, Alexa).

• Sphere It
• 

Creating a new search engine seems like a futile exercise. If Yahoo and Microsoft cannot compete with Google in search, what chance does a startup have? So instead of creating new search engines, we are starting to see the rise of search applications that sit on top of existing search engines.

Two recent examples are Surf Canyon, which publicly launched its browser add-on today, and ManagedQ, which launched its own site quietly a few weeks ago. I’ve been playing around with both for about a week. They both offer improvements to the pared-down search interface that we are all used to and point to areas where search can be made better. Not bad for two startups without any venture capital (Surf Canyon has raised $250,000 in angel money, and ManagedQ is run out of the founder’s basement in Palo Alto). Still, while both point in the right direction, neither one comes close to offering a better overall search experience than Google does on its own.

Surf Canyon is an application that sits on top of regular search results. The startup has its own Website where you can conduct searches, but the browser add-on makes it much more practical to use. The add-on is for either Firefox or Internet Explorer, and essentially allows you to re-order search results on Google, Yahoo, or Windows Live Search. (Google doesn’t like it when other Websites re-order its search rankings, but Surf Canyon doesn’t rely on Google’s APIs to do what it does and thus feels that it is not bound by Google’s restrictions).

Whenever you do a search, a little bullseye icon appears at the right of each result. If you click on the bullseye, Surf Canyon inserts three recommended search results that are similar to the one you clicked on. They appear indented under the result you are trying to drill down into. For instance, if you search for “techcrunch,” the three recommended results might be a link to TechCrunch UK, Crunchgear, and the TechCrunch Tech President Primaries (the recommended results change over time, even for the same search). You can drill down two more times within the recommended results to keep on refining your search. So if you click on the bullseye again next to one of the recommended links, you might get a link to TechCrunch on Amazon’s Kindle store from page 8 of the regular Google results, a mention in the NYTimes Bits blog from page 12, or a link to the TechCrunch Facebook group from page 5.

The results are hit or miss. Surf Canyon basically gets three chances per click to come up with a relevant recommendation. In general, it comes closer than if you hit the “Similar pages” link that Google provides with every search result, but it still feels pretty random. Showing more than three recommended results would help. But what I like best about Surf Canyon is the interface. It doesn’t take you to another Web page. The recommended results just appear underneath the appropriate link. It feels more like an application than a cumbersome Website where you have to click through multiple pages to find what you want. Google could take a lesson in interface design from Surf Canyon here with all of its Ajax goodness.

ManagedQ takes the more radical approach. It rethought the entire user interface to make it much more visual. Explains founder David Stat:

Search hasn’t changed in a decade. Result quality has improved, but what you see has not changed. The search interface has remained stagnant at the command level, So why not a search application, rather than create a search engine, we can sit on top of the results of any search engine. Currently we use Google.

Every time you do a search on ManagedQ, a grid appears on the right of the first six results so you can visually see what is on the other side of what is normally a blue link. If you click on one of the images, it opens up a larger, browsable window still within ManagedQ. The idea is that you can surf the Web without leaving the search application.

Presenting search results visually is nothing new. Sites like ViewFour have been doing it for years. But ManagedQ combines the visual search with a guided search experience.

On the left is a list of persons, places, and things to help you refine your search. ManagedQ uses natural language processing (NLP) to extract the main concepts from the entire search bin. And it does this very fast, in a distributed way using peer-to-peer technology. One of the drawbacks of NLP system is that they take a lot of time to parse and chunk large data sets. ManagedQ solves that problem.

When you click on a name or concept on the left, it is highlighted wherever it appears in the miniature Web pages on the right. So ManagedQ gives you a guided search experience with suggested terms that help you narrow your search. If you search for “Barack Obama,” it will suggest related people like “John Edwards,” “Hillary Clinton,” and “John McCain,” as well as other related search terms: “Harvard University,” “Keynote address,” Voting Record,” “Early life,” and “Senate career.”

The major drawback to ManagedQ is that if you want to see beyond the first result grid, you have to hit a “Next” button at the bottom. When you try to refine a search using one of the guided terms on the left, instead of bringing up the search results that contain that term, you are stuck with the existing grid half-filled with grayed-out boxes that say “No matches” on them. You have to click through the result set to find to find Web pages that match, in which case the terms are highlighted. (For more on ManagedQ, watch the tutorial).

That flaw alone makes ManagedQ not much more than an interesting experiment at this point. Searching Google is still much faster and gets you the results you want more directly. But again, Google can learn something here. Why not offer a decent-sized image of each Website next to search results to give searchers a visual cue as to what resides on the other side of that link? It is that extra little piece of information that, in some cases but not all, could help people sort through search results easier.

• Sphere It
•