Bit.ly's Grand Plans, And Their Inevitable Clash With Digg: Bitly Now


URL shortener and analytics service Bit.ly has been working on a new set of products, being referred to as “Bit.ly Now” internally, which will define the next stage of the company’s growth. The company confirmed these plans to us today. The services will include both a destination website as well as a distributed service via expansions to the Bit.ly API.

The core Bit.ly service, which lets users shorten web URLs into something suitable for Twitter and other services with limits on characters per post, has continued to grow quickly. 7 million URLs are shortened via the service each day, the company says, and 2-3 million of those are unique URLs Bit.ly has not seen before. Those Bit.ly URLs are clicked on 150 million times per week across a wide range of services – Twitter, Facebook, instant messaging, email, etc. Twitter itself now uses Bit.ly for URl shortening, and the service has quickly taken the lead in their market.

The magic behind Bit.ly are the stats that the service makes available on the underlying domains being clicked. Investor John Borthwick explained it all to investors in an email we obtained earlier this month:

bit.ly has been on a tear since we launched it last summer — let me sketch out what it is, why its useful and offer some data points on progress. bit.ly is on its surface a link or URL shortener, helping people take long and unwieldy links and make them short and easy to share via email, Twitter, Facebook etc. But once you shorten a link with bit.ly the fun begins. You can put a simple “+” on the end of any bit.ly link and see, real time, the pace at which that link is getting shared and clicked on as it moves around these social distribution networks.

Bit.ly Now will take all of this deep (and wide) data on popular real time URLs and turn it into a service. That’s where the inevitable clash with Digg comes in.

Digg shows popular links based on what people vote on, filtered massively for fraud. The Digg home page is populated with the top stories voted on by Digg users.

But only 20,000 or so new links a day are submitted to Digg (compare that to 2-3 million for Bit.ly). And Digg has to constantly fight users who try to game the system and get access to home page traffic. They also rely on users to categorize links and provide other metadata about the stories.

That’s why Digg launched their own URL shortener service, and are planning a new real time product of their own. The goal is to reduce the dependence on this flawed human voting system.

Bit.ly’s new Bit.ly Now service will show popular links at any given time, just like Digg (for now, Bit.ly sends the most popular link every hour to a twitter account). When Bit.ly Now launches, that link data will be combined with additional metadata about the URLs. In particular, they plan to extract important entities, people and topics from the stories in real time, allowing for a categorized approach to popular links. Bit.ly says they are talking to a number of third party services, including Reuter’s Open Calais, to help them do this.

Those are two big advantages Bit.ly has over Digg – distributed link clicking data that is far harder to game than Digg, and automated real time categorization of links. But there’s a third advantage as well.

Bit.ly says that the data flow they are seeing is so massive that they are getting very good at predicting the number of clicks a link will get in the future. They look at acceleration of clicks as well as the source (Facebook, Twitter, IM, whatever) and whether people are clicking that are outside of the social graphs of other people clicking.

In other words, you could say that Bit.ly knows what will be on the Digg home page tomorrow.

They knew, for example, that the Neda Youtube video would be popular far before it was featured on CNN and other major media sites and then made its way to Digg.

The Bit.ly Now service will be both a destination site as well as a distributed service via the Bit.ly API. Third parties will be able to access the data based on topics or keywords. News sites may find this particularly valuable to monitor trends and supply additional relevant content to readers.

Perhaps even Digg may find this interesting. The real time stuff Digg is working on will overlap significantly with Bit.ly, we’ve heard. Digg will be looking for link information beyond what the Digg community adds directly.

The last thing Digg wants is to become reliant on Bit.ly data, though, with a directly competing Bit.ly destination site out there. If I were Digg, I’d start talking to Bit.ly now to see if I could find a way to avoid that situation.

It’s also clear that the new service will become a huge competitive advantage to Bit.ly’s core shortener service. Sites like ours, which use our own shortener service, will be left out of the Bit.ly service. Publishers who otherwise wouldn’t care will start to use Bit.ly to increase exposure in the ecosystem. Then the network effect kicks in – as more people use Bit.ly they get more data, making the service stronger, and forcing more people to use the service. It’s a great place to be.