URL shortener and analytics service Bit.ly has been working on a new set of products, being referred to as “Bit.ly Now” internally, which will define the next stage of the company’s growth. The company confirmed these plans to us today. The services will include both a destination website as well as a distributed service via expansions to the Bit.ly API.
The core Bit.ly service, which lets users shorten web URLs into something suitable for Twitter and other services with limits on characters per post, has continued to grow quickly. 7 million URLs are shortened via the service each day, the company says, and 2-3 million of those are unique URLs Bit.ly has not seen before. Those Bit.ly URLs are clicked on 150 million times per week across a wide range of services – Twitter, Facebook, instant messaging, email, etc. Twitter itself now uses Bit.ly for URl shortening, and the service has quickly taken the lead in their market.
The magic behind Bit.ly are the stats that the service makes available on the underlying domains being clicked. Investor John Borthwick explained it all to investors in an email we obtained earlier this month:
bit.ly has been on a tear since we launched it last summer — let me sketch out what it is, why its useful and offer some data points on progress. bit.ly is on its surface a link or URL shortener, helping people take long and unwieldy links and make them short and easy to share via email, Twitter, Facebook etc. But once you shorten a link with bit.ly the fun begins. You can put a simple “+” on the end of any bit.ly link and see, real time, the pace at which that link is getting shared and clicked on as it moves around these social distribution networks.
Bit.ly Now will take all of this deep (and wide) data on popular real time URLs and turn it into a service. That’s where the inevitable clash with Digg comes in.
Digg shows popular links based on what people vote on, filtered massively for fraud. The Digg home page is populated with the top stories voted on by Digg users.
But only 20,000 or so new links a day are submitted to Digg (compare that to 2-3 million for Bit.ly). And Digg has to constantly fight users who try to game the system and get access to home page traffic. They also rely on users to categorize links and provide other metadata about the stories.
That’s why Digg launched their own URL shortener service, and are planning a new real time product of their own. The goal is to reduce the dependence on this flawed human voting system.
Bit.ly’s new Bit.ly Now service will show popular links at any given time, just like Digg (for now, Bit.ly sends the most popular link every hour to a twitter account). When Bit.ly Now launches, that link data will be combined with additional metadata about the URLs. In particular, they plan to extract important entities, people and topics from the stories in real time, allowing for a categorized approach to popular links. Bit.ly says they are talking to a number of third party services, including Reuter’s Open Calais, to help them do this.
Those are two big advantages Bit.ly has over Digg – distributed link clicking data that is far harder to game than Digg, and automated real time categorization of links. But there’s a third advantage as well.
Bit.ly says that the data flow they are seeing is so massive that they are getting very good at predicting the number of clicks a link will get in the future. They look at acceleration of clicks as well as the source (Facebook, Twitter, IM, whatever) and whether people are clicking that are outside of the social graphs of other people clicking.
In other words, you could say that Bit.ly knows what will be on the Digg home page tomorrow.
They knew, for example, that the Neda Youtube video would be popular far before it was featured on CNN and other major media sites and then made its way to Digg.
The Bit.ly Now service will be both a destination site as well as a distributed service via the Bit.ly API. Third parties will be able to access the data based on topics or keywords. News sites may find this particularly valuable to monitor trends and supply additional relevant content to readers.
Perhaps even Digg may find this interesting. The real time stuff Digg is working on will overlap significantly with Bit.ly, we’ve heard. Digg will be looking for link information beyond what the Digg community adds directly.
The last thing Digg wants is to become reliant on Bit.ly data, though, with a directly competing Bit.ly destination site out there. If I were Digg, I’d start talking to Bit.ly now to see if I could find a way to avoid that situation.
It’s also clear that the new service will become a huge competitive advantage to Bit.ly’s core shortener service. Sites like ours, which use our own shortener service, will be left out of the Bit.ly service. Publishers who otherwise wouldn’t care will start to use Bit.ly to increase exposure in the ecosystem. Then the network effect kicks in – as more people use Bit.ly they get more data, making the service stronger, and forcing more people to use the service. It’s a great place to be.
I am squarely in the camp that believes that DIGG is old news and will essentially die a slow, painful death.
Already is, due to its complete lack of profits.
Are there any screenshots of bitly’s new aggregation page?
This concept sounds very similar to http://tweetmeme.com and http://tweetnews.me
Am I missing something? Seems like if one had Twitter data access you get a lot of what bitly is saving (and hence the ability to make such a pulse page).
I disagree, I think there are a lot of services that have access to very important, informative information that fail because that information isn’t presented in such a way that makes it easy for users to interact with it. I think digg does the best job of allowing users to interact with the data and I don’t think digg will lose that edge any time soon.
the filtering is the bottleneck and bitly seems to be merely gauging the votes by submission volume. smart. we’ll see what happens once someone starts pumping urls through bitly to game up the rankings.
As far as I’m aware, the Bit.ly service seems fairly game-proof. One of its nice features is that it recognizes previously shortened URLs and uses the same shortened URL for each subsequent shortening attempt. That means you can’t just shorten a link 1000 times and get yourself at the top.
Their stats track not how many times a link has been shortened, but how many times that shortened link is clicked, where it was clicked on from, and who did the clicking (IP address used for geographic data). Those metrics alone provide plenty of angles to restrict spamming, sufficiently.
I grew tired of Digg a year or so ago. I think a lot of early adopters may have left due to all the trolls and immaturity of the comments. One story in particular that got dugg to the top was about this poor nerdy kid who put up a singles ad & the vicious mob just ripped him apart for putting himself out there; poor kid!
It no doubt is still huge, but for me I find Twitter to be a better place to find, share & converse about interesting links with trusted sources.
Do you think twitter has lowered the value of digg? Would they still get 200 million?
Yes, indeed. Twitter has grossly reduced the value of Digg. Once upon a time, Digg was everything for information sharing. Now it is Twitter. The very fact that Twitter’s trending topics have become the talk of great blogs like TC is evident of its growth as an information sharing medium.
Now, with Bit.ly planning to launch Bit.ly Now, Digg may suffer a huge setback. Digg has to find out new ways to redeem itself, for sure.
I, too, was in that camp, but then I thought digg might become relevant, again, by it’s integrating Facebook Connect….but maybe not, who knows?
The idea that there can only be 1 winner is false, and dumb. Digg is still growing and has an extremely loyal user base. Bit.ly and Digg may be competitors, but they are hardly the same service. Not even close in fact.
On a related note, the more I read from Arrington, the easier it is to see why so many people think he is an asshole.
It’ll be great to see who wins in this clash. I think Bit.ly will win.
bye bye digg
Wow, you just blew my mind with the idea of bit.ly being a Digg killer. Real-time aggregation of link popularity, hard to game.
Kevin Rose better be sweating.
tr.im > *
good one beavis
I was initially using TR.IM for quite some time. It was not moving fast enough and could not hold a candle to what Bit.ly has going on. Sorry.
Great writeup.
I have been discussing the opportunities that URL shorteners have for months now.
I believe the long term value of Twitter, and specifically the updates from Facebook, are going to be in the information that is shared. Specifically, the URL’s that are shared.
Imagine the things you could discern if you had 2-3 million shortened URL’s a day, and you knew who was shortening them…
Bit.ly doesn’t seem to have a solid Business Model at this time and Digg on the other hand has progressed with their contextual ad-delivery in the past year. This seems to me to be a partnership opportunity for both companies which would go in Bit.ly’s favor.
Why is “real-time link popularity” so hard to game? Honestly, it would be even easier than Digg, which at least has a social graph. I think this is naive.
Still, should make for a cool service.
“Sites like ours, which use our own shortener service, will be left out of the Bit.ly service”
Well I think Bit.ly should simply allow services such as tcrn.ch to piggy back onto their infrastructure – this will allow such services to have their own brand, yet contribute to the massive data that Bit.ly will continue to process.
But regardless, Just being able to say this is huge: “you could say that Bit.ly knows what will be on the Digg home page tomorrow.”
Awesome.
That could be extended to other services as well…yes, it’s awesome, thanks to Twitter’s boost.
Hard to say what will kill digg until we see what they unveil in the next few months.
Iam hoping/expecting to see big things from them.
Bit.Ly sucks..Short URL Services all together suck. Twitter should integrate a Short-URL service.
Poor old tinyurl, who was it that was writing a year ago that this is what they should have been doing?
That’s an interesting analysis on bit.ly. Though it appears to be a trivial idea, it has rich data in it that can be easily monetized. Digg is a fading cloud in my opinion, as you’d clearly mentioned, Digg cannot stand against bit.ly or other similar services purely because of the number of posts made in Digg as opposed to the posts made in other websites. bit.ly might also be competing with other real-time search engine players like http://www.boilingpage.com, http://twettmeme.com, http://www.twiturly.com etc. While these guys track the URLs too, bit.ly definitely has an edge.
Thank God, I don’t even waste my time on Digg anymore. What a shithole it has become.
Digg is definitely old news. I’m surprised to hear people still use it.
digg is dead. they should have sold a long time ago.
As I said before, Digg is chopped liver and should have sold out when they had the chance. You always sell well things are going well and the future looks bright because one day it won’t be and you will be toast.
The article hits the nail on the head – bit.ly has a great rolling snapshot of rolling trends – that’s incredible and almost impossible to recreate for Digg.
However I think there’s more too it than click through’s – given the social graphs of twitter, facebook at the retweet could really be the currency that equates a +1. If that turns out to be true the the game shifts again to anybody who can track retweets.
What bit.ly gets that no-one else does is knowledge of the click throughs and how that correlates with links being passed around. To stay ahead bit.ly will eventually also need to include non bit.ly URLs in bit.ly now.
Interested to see how su.pr will compete with bit.ly, once it comes out of beta.
and none of you guys even think about how you “know” if a shortened link is valid, ie not coming from a trusted source/domain…
the whole reason behind the url structure was to provide a kind of visual indication that the link you’re going to might be valid…
good luck with the spamming/viruses as this grows…
unless you have a few major trusted providers of shortened urls… but then this becomes incredibly easy to game in an automated fashion, as i can have multiple levels of a shortened url referring to other shortened urls.. which ultimately tie back to my trojaned website..
the user doesn’t know what’s happening unti they link on the site…
good luck with this!
tinyurl will let you “pause” a shortened url at their site so you may preview the real link behind it. Very nice of them. It is a opt-in feature I believe.
And if you have OpenDNS running it may automatically save you anyway.
bit.ly doesn’t care what their own shortened URL is – they only care about the endpoint. Personally, I don’t care what the bit.ly URL of a link I generate is either, as long as it’s short.
As long as people maintain the attitude I have – I want to share this story and I only have 10 characters left – URL shorteners will continue to gain ground.
It’s amazing bit.ly is the first to take URL shortening to this level though – when I first saw TinyURL I had the same thoughts (analytics, real-time “breaking news” tracking, digg-functionality, etc).
That’s an incredibly myopic view on gTLD use in terms of shifting interfaces over time. Also, without PURL, any archive for all of this is moot in 20 years if the shorten service goes belly up. Compounding this, your concern is addressed already by companies like OpenDNS and by projects such as SURBL/URIBL/etc…
Someone in all of this hash happy mess is going to arrive back at an AOL Keyword clone. People want simple(r).
Great write-up. I do wonder what happened to Tiny Url? As for Digg, it still has great links but I go there mostly to read the insane comments.
One thing may stop Bit.ly and Twitter: SPAM. Its starting to become a real problem on Twitter and may do so on Bit.ly
“Those are two big advantages Bit.ly has over Digg – distributed link clicking data that is far harder to game than Digg, and automated real time categorization of links. But there’s a third advantage as well.”
Dude. Are you serious? Bot nets make this amazingly easy to game.
I don’t see how botnets obtain such a large advantage over bit.ly than they already would on digg.
digg is a URL submission service and they have algorithms in place to protect against collusion.
bit.ly is a URL submission service, post-digg, why would they not include algorithms in place to protect against collusion?
Digg relies on an internal social graph to assist with gaming identification. Definitely a useful asset.
That said, click fraud has been a long-standing problem for ad networks and I’m sure that the bit.ly guys will draft off that knowledge.
@timy11 actually with TweetDeck, clicking on a shortened URL gives a pop-up that shows the original URL and the destination’s title. If TweetDeck can do it, any other service can as well.
As for gaming the new bit.ly system, the first thing they should do is filter by IP address, to separate out unique visits from repeat. That would help. But yes, they’re going to need some sophisticated fraud detection at some point.
I am excited to see the new service. I love the bit.ly system that shows me my own shortened link click volume, so this will be fun to watch unfold.
If they can bring legitimacy to the data, I can see how this could eventually become one more data point that publishers can show to investors, advertisers…
URL shorteners are a disease on the web. Unfortunately I don’t think most people will realize this until the first big one goes under, and millions of links on the web instantly become broken.
Well, there’s a business idea in itself – start caching all of the shortened URLs you find. One day, the service will go down and you can start spreading your cached URL (with Google Ad position on the cache replaced with your own Publisher ID).
Solid write up. My immediate reaction is that Twitter needs to auction off its url shortening business – there’s a ton of value there.
“Those are two big advantages Bit.ly has over Digg – distributed link clicking data that is far harder to game than Digg, and automated real time categorization of links. But there’s a third advantage as well.”
Nope. Trust me. If Bit.ly or anyone rolls out a link click-based news site that draws significant traffic, spammers will be along soon after too, well, spam the hell out of it.
What save Bit.ly right now is that there’s no reason for spammers to bother with it. If you’re not reporting the top clicked URLs, there’s no easy traffic to tap into.
I mean, Direct Hit pioneered link tracking to produce “Top 10″ search results back in the late 90s. And that died in part because of the spamming issues.
Still, Bit.ly and other trackers have great data, and I’m interested to see how they’ll make use of it as guides.
“If Bit.ly or anyone rolls out a link click-based news site that draws significant traffic, spammers will be along soon after too, well, spam the hell out of it.”
Yep, look how often Google trends gets gamed. Bit.ly will inevitably get gamed at some point if they roll out a click-based news site, but that’s not important. How they stop people from gaming it is what is what matters.
They should probably employ the same system as digg, with editors who decide what appears on the front page.
Am I the only one who thinks url shorterners are a commodity? Once bit.ly’s urls grow in length, people will migrate to something else. Perhaps bit.ly has hundreds of 5 character domains and maybe that’s why they got investment. I fail to see the long term viability. Twitter will do to bit.ly what it did to tinyurl.
Good post Michael. They certainly are in a great spot. Brilliant execution really as tiny.cc, tinyurl and others were around.
There was always an opportunity for an intermediary like this to step in and control the data flow. Links are currency online and Bit.ly has them. Addthis.com presumably have similar data but have never done anything with it. That 2-3 million links vs 20,000 on Digg stat is killer. Digg are screwed. I rarely, if ever, see a digg link on twitter.
Twitter needed to buy this one along with Summize. Is Twitter asleep at the wheel or what?
If i were twitter i would play hard ball and kill off Bit.ly by pushing links out of twitter posts and denying API access to links. That way they control all links on Twitter (and we all get a full 140 characters).
“If i were twitter i would play hard ball and kill off Bit.ly by pushing links out of twitter posts and denying API access to links.”
Twitter, Summize, and Bit.ly all have the same investors. There’s a reason Twitter switched to bit.ly for auto-shortening.
Twitter actually bought bit.ly.
Cool. This is great. It is going to be big.
I wrote something similar using bit.ly API. It shows the top links I shared on twitter using bit.ly. I have the script here if you want to download and run your own. It was inspired by @davewiners top 40 links.
I would be excited if U.nu were doing this. Bit.ly’s URLs are still too big for me to make serious use of them.
This isn’t Bit.ly’s fault, though, as it can’t shorten them at its level of popularity without running out of permutations. It’s entirely Twitter’s fault for making such agonizing over two extra characters necessary.
Why don’t they use something like BackTweets.com to track what people are saying about a URL. It doesn’t matter what URL shortening service you use.
I didn’t even know that bit.ly provided stats on the links you shorten. hmm, I may need to change URL shortening services…
I love this, bit I have to disagree with the concept that the Digg system is flawed. Subjective? Sure. But not flawed, at least not because people are serving as gate keepers.
I personally believe there is no competition as the growth rate of digg is slowing down and the whole support Bit.ly allows user will bolster it ahead of digg. I think with the new plans in place it will steal the traffic from digg completely
.
I was recently privy to a demo of bimbs.com a URL migrating service, I can quite honestly say that for millions of businesses large and small across the gobe this thing, anticipated to launch next month is in a class of it’s own from the designers of sites that ooze inovation bimbs.com is one that needs to be looked at very carefully, I’m already getting my inventory videoed, photod and described….Bloody Marvellous !! A recent meeting with a few select product suppliers revealled that using a seb^rvice like bimbs.com could top almost destrpy the meaning all of their ROI….. Follow The Angels….
This is amazing!! I’m glad to hear that Bit.ly is starting to capitalize on all this data they are getting. I’m so excited to see how this plays out.
How does raw data collection from Twitter links replace the social graph editing of relevant stories? Google has page rank and that hasn’t killed Digg. There is no guarantee that Bit.ly can translate its popularity as a shortening and stats service into a destination site.
Don’t know how relevant would bit.ly stats be as 95% possibly comes from twitter where people are just selling stuff
how could anyone not like bit.ly? as someone who uses it happily and with frequency i’m a fan.
but a minor point gnaws at me: the fact that digg was apparently not asked to comment for this story.
i guess bit.ly wasn’t either, technically, but that’s not exactly the point. a battle has been set up without the generals weighing in. no?
Digg has an easy way to defend the invasion of Bit.ly: partner with Twitter to add a “URL Resolver” to each shortened URL. When the user mouses over the URL, it will display the real URL. When the user clicks on the real URL, it will bring the user to the site as well as recording the access. Thus Digg will end up having all the stats, not Bit.ly. This can be achieved with just a few lines of Javascript code.
The users will love it too: now they know what they are clicking (spammers will hate this).
Of course, Twitter can benefit from this by auctioning off this “URL Resolver” service.
I think iI thin
“You can put a simple “+” on the end of any bit.ly link and see, real time, the pace at which that link is getting shared and clicked on as it moves around these social distribution networks.”
Cool.. didn’t know this and I’m using bit.ly for a while now..
I’ve seen a demo of http://www.bimbs.com and I can see where you are coming from Bainzy, I never realised how something so simple has not been implemented before. I met the CEO, What a mad man
He along with that personality is such a breathe of fresh air in what is becoming a smelly sector. I tried to invest his answer and the smile on his face when he said the words ” Fu*k Off ” was a classic. We can expect great things coming out of the Alps shortly.