Google, Yahoo and Microsoft Agree to Standard Sitemaps Protocol
Marshall Kirkpatrick
82 comments »
In an encouraging act of collaboration, Google, Yahoo and Microsoft announced tonight that they will all begin using the same Sitemaps protocol to index sites around the web. Now based at Sitemaps.org, the system instructs web masters on how to install an XML file on their servers that all three engines can use to track updates to pages. This should make it easier to get your pages indexed in a simple and standardized way. People who use Google Sitemaps don’t need to change anything, those maps will now be indexed by Yahoo and Microsoft.
The protocol is offered under an Attribution-ShareAlike Creative Commons License, so it can be used by any search engine, derivative variations using the same license can be created and it can be used for commercial purposes.
Any time competitors agree on open standards, that’s an enabler of further innovation and something to celebrate. It’s also great to see Creative Commons receiving all the more validation.
Search engine guru Danny Sullivan wrote the following tonight about the move.
Overall, I’m thrilled. It took nearly a decade for the search engines to go from unifying around standards for blocking spidering and making page description to agreeing on the nofollow attribute for links in January 2005. A wait of nearly two years for the next unified move is a long time, but far less than 10 and progress that’s very welcomed. I applaud the three search engines for all coming together and look forward to more to come.
Several people have made early public statements indicating that the next move will be to develop meaningful standards support for robots.txt files. Imagine a future when these players agree on standards for user control of data, microformats or truly neutral party click-fraud tracking and prevention. Maybe that’s crazy.
GREAT
IDEA - there are many ways the ‘big three’ can cooperate and work together towards common goals for their collective common good….
Hope this is the beginning of a trend…
Wonder why Ask.com was not included ;-?
Awesome. I’ve already started working on a sitemaps wiki built on MediaWiki. Looking for up to 16 investors to put in $1 million. Accredited investors: don’t hesitate, contact me now.
great. hope it will be finalized before my site is online:)
Let the powers unite and rule all
This move is a necessity, and I guess it’s rooted on a previous news regarding Google getting sued by a news website for crawling its web pages and displaying them on search results (Google News).
This standardization effort will definitely prevent future lawsuits involving page crawling.
xml, bleh!
Great news for all of us!
I applaud the folks at the 3 companies for their collaboration. The folks working deep in the bowels of the ship at Yahoo! are doing the right think in this and a number of other specific areas.
The leadership of Yahoo, however, needs to start more clearly articulating their path forward for the company. Without strong leadership, the company is vulnerable for acquisition or influence from hedge funds who will push for change. Either way, change from the top will have to come sooner than later.
Eric
http://breakoutperformance.blogspot.com
This is a Great Idea from Google, Microsoft, and Yahoo.
Make all Simpler and better.
Baki
http://www.softwareas.com/agre.....ho-uses-it
Who’s actually using it? Supposedly Sitemaps improves ranking, but a quick sample - techcrunch, digg, odeo - found 0 out of 3 using it (404s for site.com/sitemap.xml and site.com/sitemap_index.xml).
…
I use them. Wordpress has a plugin that writes one for you. Re-writing it everytime you update or post. Oh yeah and it pings Google sitemaps too. Sweet.
This is a stellar idea that will definitely make the Internet a better place. However, the fact that Google doesn’t need to change anything makes me wonder - will Yahoo and Microsoft be effectively adopting a standard already endorsed by Google with its Sitemaps programme? In a twisted sort of way, is this effectively a victory for Google, with the two other search engine providers adopting a model that it has been using and promoting?
Mind you, I’ll admit that I don’t have extensive knowledge of how exactly Sitemaps as a concept was pioneered and what role Google had in it, so feel free to correct and criticise me as you see fit!
a victory of google?! hello, google already won the search engine war with its 50% market share on search
RE: Michael Mahemoff:
Sitemaps are useful when you have a site that has thousands pages, like e-commerce sites. The reason to use them isn’t necessarily to boost ranking but to get the search engines to crawl and index each page - especially for sites that have dynamic content.
In the case of a good sized e-commerce site, with products that may change daily due to product selection or inventory, this is a godsend.
For static sites, I’ve used sitemaps, but they’re only marginally useful after all fo the pages have been initially indexed.
I don’t bother to use sitemaps on my personal blog, because frankly, SE traffic tends not to convert all that well. Plus, Google tends to pick up new posts within 24 hours after going live.
Great to see some collaboration between the big boys. Standardization is a positive for everyone on the internet, both the startups and the crawlers. Although it does not particularly help me as a startup (web app, not many pages, mostly ajax), I think its a great step forward. Great news and in the words of Mr. Baron-Cohen “Great Success!”
Joy, I agree the concept is useful, it would be enticing if there was some good examples out there. WRT E-Commerce, once again no cigar for the world’s two biggest E-Commerce sites (Amazon + EBay). I find it kind of funny that everyone’s talking about how important this is, with so little discussion about why it hasn’t really taken off or examples of websites that use it.
To me, the interest is about the fact that these companies agreed on a standard (like the nofollow agreement), rather than the actual protocol they agreed on.
Re: 13. Anonymous
I disagree - Google may have won the battle, but not necessarily the war, and, despite its existing dominance, I think it would be very short-sighted - not to mention arrogant - on Google’s part to dismiss Yahoo, for instance. In fact, the moment such a perceptions sets in is the moment Yahoo - or whatever new rival emerges - start gaining market share at Google’s expense.
This is fantastic news. Not because XML sitemap standards are a very big whoop, but rather because this is another small win for collaboration and open source development.
I’m curious, where does W3C fit into all of this? I’m also curious to see how Jeffrey Zeldman weighs in.
-
Walter Stevenson
http://www.walterstevenson.com
Michael Mahemoff,
Amazon.com and Ebay are bad examples simply because they are overly well known. It would be foolish for any major search engines to neglect Amazon.com or Ebay, since alot of people are looking for it. The same reasoning falls for sites like CNN or BBC. Search engines are constantly crawling those sites because 1) the site is well-established in the public eye’s and relevant; 2) their type of relevancy is time-sensitive.
For medium-sized and lesser known e-commerce sites they are helpful in terms of indexing and in terms of diagnostic data. Google Sitemap program gives information on any 404s found and other data.
The sitemap program gives an easy way for webmasters to ensure all of their content are being found and crawled on a regular basis. Sure a simple “site:www.amazon.com” works too, but the sitemap tools are far easier to use and for tools like Yahoo Site Explorer, it actually gives different answers than a normal Yahoo query.
I’m extremely happy about this. Now I’m just waiting for the next round of wordpress/Joomla plugins to notify sitemaps.org on updates!
WOW, did Hell freeze over?
This is a long overdue consolidation. Great to hear that these three are capable agreeing on new standardizations!
Great news and it is about time. Sitemaps are much needed as ajax-driven single-page web apps are invisible to most html-based web crawlers as in many cases links are dynamically created on the DOM level in the client-side post the html page load. As these apps become more popular, the big 3 search engines had to agree on a standard to make it easy for webmasters to specify their pages or risk becoming less relevent and missing out lots of great web 2.0 driven content out there.
Yet, it is not yet clear to me how to submit sitemaps to y! and MSN but I’m sure this info is going to be available soon. If you have this info, it will be nice if you can share it here.
http://grouper.com/siteindex.xml
Now I have to create a sitemap, with all three companies now working togethor.
This is fantastic!
Good news. Increased efficiency and less time wasted on getting things to work across the board is always music to my ears.
This is an excellent step in the right direction. Seems like we are entering a new phase in internet etiquette.
The actual video of Vanessa Fox from Google and Tim Mayer from Yahoo announcing this at the Las Vegas Pubcon conference is here: http://videos.webpronews.com/2.....on-search/
“Imagine a future when these players agree on standards for user control of data…”
Thank you very much for your continued mention on this important issue. I wish others would constinually point out the important of user controlled data. Data portability will soon be the number one thing we consider when suggesting software or we web services for our clients.
It is both an important issue for internet end users and web service developers. With some projects data portability can mean the difference between hours and weeks of work.
This is great news! I just finished getting my sitemap working properly, excellent timing!
Thats great!
I haven’t seen yahoo or msn crawling my sitemap file yet. Any idea on implementation guidelines?