Yahoo is letting a limited number of beta testers into its SearchMonkey developer platform, a service announced in February that allows site owners to customize their Yahoo search result listings.
With SearchMonkey, site owners create “applications” for Yahoo search that can be installed by users in the same sense that Facebook applications can be installed. Each application modifies results for a certain URL specification (for example, all reference pages on Wikipedia or product pages on Amazon). Modifications include both changes to the basic elements of a search result (the title and description) and additions such as an image, deep links, and key/value pairs.
Here’s the Yelp search result example we used in our first post about SearchMonkey:
Notice that the modified result has links to reviews and photos, plus quick data such as the restaurant’s rating, address, and phone number. This extra info is supposed to make the result more useful to searchers and hopefully entice them to click through at higher rates.
Creating an application
I had a chance to play around with the SearchMonkey application creation tool, which is actually quite straightforward (if you’re comfortable with PHP, XSLT, and DataRSS).
While you can import other people’s applications into SearchMonkey and modify them, you’ll probably want to start off from scratch when customizing search results for your website. The process starts by picking a name, description, category, icon, and type for your app.
There are two possible types: enhanced results and infobars. Most site owners will opt for the “enhanced result” option, which can display a bunch of information in full view as the Yelp example above does. The “infobar” option condenses supplementary information into an interactive bar below each search result. Users need to engage with it to reveal more information, but apparently it provides more flexibility in terms of HTML and styling (I couldn’t get it to work in my tests).
Once you set those basic settings, you need to indicate the URL structure that your application will target. The structure looks something like *.crunchbase.com/company/* where the asterisks allow wildcard matching at the beginning and/or end of your URL pattern. That example pattern would be used to customize search results for all company pages on CrunchBase.
After submitting up to ten URLs for which to test your new application, things start getting a little more complicated. You must select the sources for the data you want to show in search results, and then you need to tell SearchMonkey how to display that data. By default, you have access to 16 basic data fields from Yahoo. These include the result’s title and description, plus other less useful things such as its language, file size, and position among all the results.
But you want to pull information from the page being listed as a search result so users can get a taste of what they’d learn by clicking through. So you’re going to have to somehow inform Yahoo of this information. That requires creating a “custom data service”, which is a multi-step process in and of itself that I describe below.
For a second, though, assume that you’ve already created a custom data service. All that’s left is to tell SearchMonkey how to display the custom data, which consists of setting values in a simple PHP class. You can set up to 4 deep links (restricted to your own domain) and 4 key/value pairs. As you play around with the appearance of your result, you can preview it at the bottom of the page by clicking “Save & Refresh”. This gives you constant assurance that you’re actually doing things right (always a good thing).
Creating a custom data service
So those are the basic steps behind creating an application. Before you do that, however, you’ll probably have to set up a custom data service. These services use either semantic page scraping or web services to obtain the data you need for your customized result.
The page scraping method follows Yahoo’s previous announcement that it would embrace semantic web standards. It allows you to pull information from your webpages without creating any APIs, since it can be taught to crawl your HTML structure for just the data it needs.
Setting up page extraction requires the creation of an XSLT file, a process that’s made easier by the use of the XPather Firefox extension, which helps you extract the requisite code for identifying data on your site. I won’t go into any details about XSLT, but needless to say you’ll need to brush up on your knowledge of it. You’ll also most likely need to tweak your website code to ensure consistency across pages that are meant to display the same type of data.
As for the web services method of creating a custom data service, you’ll need to know XML in OpenSearch format or XSLT.
Our (half-baked) search result customization
So what did our test application produce? The modification you see below:
Obviously it’s not complete by any means, but you can see how much of the result is customizable – and that we were able to pull out Facebook’s address from its CrunchBase page without much configuration.
A step forward in the evolution of search
While the eyes of non-technical readers have probably glazed over at this point, I can assure even them that it’s not rocket science to customize search results with SearchMonkey. As such, I can see this becoming very popular with website owners big and small.
Perhaps the only question left, then, is how Yahoo will promote these applications to its search users. Facebook benefited from powerful viral channels when it kicked off its platform, but we have yet to see how Yahoo will attract users to its applications. Some of the applications (for larger, more prominent sites) may be enabled by default, but the vast majority will presumably require people to manually opt in.
Assuming that a vibrant app ecosystem does arise, SearchMonkey should be a significant step toward the more personalized, and potentially social, search that’s been anticipated by many. Once you allow 3rd-party developers to innovate – and users to adopt the best innovations – the market will have a direct ability to push search forward in its evolution.
 |
 |
 |
 |
 |
 |
I have a feeling that May is going to be a very big month for search news…
Good stuff…
Too good for developers. The results are very effective and user oriented. You can really tweak them as you want. Yahoo! has does it again. I have tried there API before and Pipes now this. Good going..
Looks interesting but I think the only way how SearchMonkey can succeed is if they enable the enhancements as a default across all Yahoo search. Without it there is a little incentive for webmasters to do that. If they make it default, everyone jumps onboard and only Yahoo will need to be policing it well so webmasters don’t abuse it.
Y! is fantastic… you just have to love them..
@Janusz
Not entirely true. Webmasters who did not even touch Search Monkey could potentially “spice up” their listings using existing sites such as Yelp and others that are to be integrated by default.
I think giving a level of control over the way results display makes sense in the long run…at least until there are programs that can automate these type of listings.
For online marketers, it’s a way to further enhance our offerings and possibly capture a more broad audience. I guarantee services will be created around this product if it is successful.
I’m anxiously excited to hear more about this.
(Half- Baked) On this Post is all i read {seesmic_video:{”url_thumbnail”:{”value”:”http://t.seesmic.com/thumbnail/IAL6f29c0V_th1.jpg”}”title”:{”value”:”(Half- Baked) On this Post is all i read ”}”videoUri”:{”value”:”http://www.seesmic.com/video/HWeGTfxv8f”}}}
This is an API developers dream! Possibly appears slightly underdeveloped but it’s a good start!
Mike: Any whispers on when we (as in the general unwashed masses) will be able to get our hands on it?
{seesmic_video:{”url_thumbnail”:{”value”:”http://t.seesmic.com/thumbnail/w5KcM0Cau5_th1.jpg”}”title”:{”value”:” ”}”videoUri”:{”value”:”http://www.seesmic.com/video/Faqj03npzV”}}}
yahoo recently banned me from their site because i put a link to my site on their message boards. hence, the email i sent them:
“yahoo sucks
your company, website, stock and mgmt are horrible.
no new features in 10 years. missed the boat on search advertising…remember overture?
google is eating your lunch…i really hope you are personally looking for other employment – basically any software/web company that has been around for over a week has better technology than yahoo.
i have no way to get in touch with my grandparents now because you wont give me my contact list back
IF I PUT TOO MANY LINKS ON A MESSAGE BOARD, THAN BAN ME FROM THE MESSAGE BOARDS, NOT MY EMAIL – your software probably isnt powerful enough to do so. Yahoo hasn’t even moved from static to dynamic yet.
Yahoo will be stuck in web 1.0 FOREVER
I hope my grandparents are still alive by the time I find them.
Absolute joke you guys run over there. #1 website on Alexa yet you don’t know how to monetize it. I think my dog can run a tighter ship.
Scott W”
I’m sure they were scathed by your withering criticism.
yahoo!’s clever move {seesmic_video:{”url_thumbnail”:{”value”:”http://t.seesmic.com/thumbnail/CXsoxKf2Gq_th1.jpg”}”title”:{”value”:”yahoo!’s clever move ”}”videoUri”:{”value”:”http://www.seesmic.com/video/lsz40a4MdD”}}}
Stop selling monkeys and sell out to MSFT.
http://www.meetingflex.com
Y! is the best…
I love that all these big companies are going on an “open-for-development” trend. People will definitely find use for this API.
{seesmic_video:{”url_thumbnail”:{”value”:”http://t.seesmic.com/thumbnail/BiezJjhyXr_th1.jpg”}”title”:{”value”:” ”}”videoUri”:{”value”:”http://www.seesmic.com/video/XPQXyWe4Mm”}}}
{seesmic_video:{”url_thumbnail”:{”value”:”http://t.seesmic.com/thumbnail/HdU99a2jZa_th1.jpg”}”title”:{”value”:” ”}”videoUri”:{”value”:”http://www.seesmic.com/video/psnoLhi1gQ”}}}
This has a lot of potential to make search more relevant to the searches. At least in your examples, it doesn’t come across as too much like an ad, but I can see ways to use this feature badly. too.
This is very interesting, it raises the bar on how search results can look, and I think it will push Google to improve their local search results too. I’d like to see these types of search results include information from a businesses Yahoo Local and Google Local profiles.
Wow. Looks like they “almost” stole Mailchimp’s Logo:
http://www.mail...mos/index.phtml
Hi Mark,
Good post.
A friend of mine told me that this wasn’t the kind of thing that TC was into, but it seems like you’ve taken a pretty good look at it, so he may be wrong.
Anyway, I’m doing tech talks about RDFa at Google and Yahoo! in SJ, week beginning May 19th, so if this whole thing isn’t old news for you guys on the cutting-edge, by then…well…I’ll be in town.
All the best,
Mark
@pfft…Um, no, fyi.
Mike -
any info on the question if publishers will have to pay for it or not (like in the search ads)?
Regards,
Tom
From what I know, its a free service.
This is good for the world. Google will strive harder to improve its search results. Competition rocks!
This seems like a great new feature, for the searcher, but especially for the small and mid size businesses that are looking to attract more clicks to their sites via seach…..I think that this should definitely become standard and just integrated into Yahoo’s algorythm….what I wonder is how thy will get word out and get website owners to initiate this process for them and their benefit…does anyone think that this may be a great new service idea (perhaps via telemarketers) to all the sites listed under thousands of keywords to market this change to them convincing them of the possible traffic increase to them by having a better search display? I think site owners may even pay for this to be setup for them….. Curious to hear replys…From Mark as well.
@Justin… without question that will work and this hits close to the SEM field in many ways. What is missing in this post (and I respect its tech testing focused and well done) are real world business case examples. You hit on one, I can see many areas where smart developers will create code to help others with traffic, branding, consumption of entertainment etc. With a smart engine and code on top of the search algorithms the ability to “discover” or find interesting and useful content is increased. Ultimately that is Yahoos goal and I would imagine they will acquire the best 3rd party code and applications over time. The BIG question remains why would I code something, whats in it for me?? Will yahoo share search revenues, will sites pay for my traffic or exposure increasing service? I think if you are thinking here, go for things which marketers will pay for as they already pay for SEM tools, shops and services.
This will be a VERY helpful tool for those of us who search (and scrape) for a living. I tested this out at the booth as well, having them conduct a site: search on LinkedIn for profiles w/ certain keywords. The ability to not have to go into each link to gather name/title/location etc. is such an incredible timesaver, and I think once the kinks are worked out of the beta, this will be a powerful (and FREE!!!) tool for us researchers.
This serves the ever present client desire to control the results on the search page and enhance results with images. I just wonder how long Yahoo’s going to be around to see this through.
Good luck Yahoo! Yahoo India and the Glue is also looking very good, but that may have been Halle’s impact more than the search result layout. And Mike (or should I call you Mr Arrington?), the Seesmic comments are ‘quality’, it really helps when I am constantly reading all day, and quite cool just to see you say your thoughts aloud… Although the video shy like me will continue to hide behind text for some time longer. Finally, congrats on getting yourself onto Time mag, a well deserved credit!
We at ShoppingNotes.com want to be the monkey for you!
We have developed the technology that SearchMonkey wants to do for the shopping vertical. Give us a product page URL and we’ll scrape the product image and price for you real-time. It’s all fully automated and you don’t have to deal with XML, XSLT, XPath, all that stuff. Try it live at http://www.ShoppingNotes.com or Read more at http://shopping...o-searchmonkey/
good
Good post !! Tanks