“Everything you blog goes on your permanent record!” How many times have we heard that lately? From employment to family situations, many people have been frustrated to find out that things they intended to write for a personal audience is now discoverable by anyone in the world via search engines. Bloglines proposed a new standard tonight to change that.
You can have private pages in places like Flickr and MySpace, but your page’s RSS feed can still be discovered by search engines. That’s what this new standard aims to change.
The proposed standard will allow XML/RSS/Atom feed publishers to keep their feeds out of search engines and unavailable for discovery by adding an access:restriction tag to the top of their feeds. Bloglines and Ask now support this tag and will keep feeds tagged as restricted out of their search and subscription results.
You’ll be able to pass a private feed URL to a friend you want to subscribe, but your prospective employer will not find it in participating search engines if you have a private account.
The Robots.txt protocol that tells search engines not to index web pages was agreed upon in 1994, but that’s just for HTML web pages. A growing number of search engines are now indexing the more dynamic XML/RSS/Atom feeds first. This new standard is an important part of the whole story around limiting distribution of our private accounts online.
I just talked to Robyn DeuPree, Senior Product Manager at Bloglines, and Paul Querna, Senior Software Engineer, and they told me that the company hopes that both content publishers (Flickr, Myspace) and search engines (Google, Google Blogsearch, Technorati, Icerocket) will get on board and make this feature available to users creating content who want their content undiscoverable by search engines.
No formal agreements have been made yet with any other company, but it’s hard to know why they wouldn’t accept the idea with enthusiasm. Many feed readers don’t support formally authenticated feeds (where passwords are required) but this should be easy to implement.
Will other feed readers respect this proposed standard? I sure hope they do – this is a great idea for which the time has come. Goodness knows I’ll start a whole new MySpace account if I know that it’s feed will be kept outside of search!