The background debate about whether or not Twitter can actually scale has intensified. More than a year ago I asked “Twitter At Scale: Will It Work?” Today Twitter is far, far bigger. And the uptime woes continue.
The big problem with Twitter is assymetric following without limitations on the number of connections, which means that a single account can theoretically have a number of followers limited only by the total number of Twitter users. This adds massive complexity to the system. Other services solve the problem by forcing both sides to agree to friendship, a one-to-one relationship. Others, like Facebook, limit the connections to 5,000 as well. But Twitter has no limits on complexity. And since they are a centralized, bottlenecked system, it is both hard to scale and easy to attack.
The short messaging format is popular, and it is now part of the web. It should thus be designed and implemented as a decentralized service like most other core web services (email, DNS, blogging etc.). The Internet was built to withstand a nuclear attack, and it is a platform that can’t be owned, attempting to completely centralize a new core service has never worked.
As Twitter grows, it needs to be architected more like the Internet.
New Twitter COO Dick Costolo says that he believes Twitter can scale in a centralized way, meaning the status quo will continue. But he acknowledges that it is a theoretical debate at this point, and he says that he hasn’t ruled out decentralizing Twitter.
We believe decentralizing Twitter solves two problems – it will help the service scale infinitely. And it is potentially a very lucrative source of revenue.
Email Is A Business – The Microsoft Exchange Model (Get Your Customers To Pay You And Do The Heavy Lifting, Too):
Twitter should look at how email, and commercial email servers such as Microsoft Exchange Server, developed. The business generates $2 billion or more in revenue for Microsoft, and powers the majority of corporate office functions (email, calendar, etc.). Businesses pay a few hundred dollars for Exchange, plust $50 or so per year per user. Plus, the businesses handle all the infrastructure costs (servers, bandwidth, etc.).
Twitter should sell Twitter Server just like Microsoft sells Exchange Server. They’d then run their own Twitter node on their own hardware.
Twitter likely couldn’t get $50/user/year out of Twitter Server, but they could certainly get more than the zero they are charging now. And they’d move the burden of scaling Twitter to businesses that want a highly stable solution. And users could still go to Twitter.com to create accounts for free, too. They just wouldn’t have the benefit of controlling the data on their own servers, and having the peace of mind knowing that their uptime was conditioned only on their own infrastructure, something under their control.
There would be some issues to work out, like the namespace and messaging between parties (If we had our own Twitter server, my user name would have to be something like @nik.techcrunch, or we could just use the existing global namespace – email). Twitter could build and sell a kick-ass Twitter server for corporations and those who wish to control their own messaging and their own brand.
But the benefits would be huge. Possibly hundreds of millions of dollars in revenue. And a partially decentralized service that would stay live even if Twitter.com went down.
So there are the benefits – revenue, lower operational costs, higher uptime. And there’s one more benefit, too. A decentralized Twitter would suck the air out of the idea that Twitter needs a decentralized competitor. Twitter could own the micro-messaging protocols and core service for the long term. Twitter owns the protocol, the users, the format, the trademarks, the brand and the name – why does it also need to host the whole damn thing?
Couldn’t you call DNS root zones the bottleneck of the internet?
name lookups are cheap and can be crazy cached – unlike this ‘real time’ stuff
I’m almost sure you are the same person that suggested that PHP randomly exposes its source code to visitors. Why are you still writing crap? =)
if your counter-argument relies on the security of PHP then you are not going to get very far
Are you suggesting that you cannot make a secure PHP application?
I don’t need a counter argument. But my analogy is that you still don’t research what you write about.
I believe he is talking about the day Facebook forgot to restore their Apache handler for .php files and exposed their home page source code.
http://www.devs...et/articles/php
This will happen for any language that has a missing server handler, not only on Apache but IIS also.
This has nothing to do with PHP or the webserver. This is the action of a bad SysAdmin. I bet the one at FB still has his job.
thanks for the reminder Chris – I wasn’t sure what he was referring to.
Nik,
I have to applaud your suggestion. Of all the Twitter coverage, yours is the first to connect an existing business model with a new platform. Further, I think it is a good idea. However, I do understand the desire to centralize — complete control can mean more $$$, if it works. But with the web, we’ve seen the power of decentralization. If Twitter doesn’t decentralize, someone else should do it. Are we seeing greed at Twitter that could be its downfall?
Someone else IS already doing it. check http://status.net these are the guys that built laconi.ca / identi.ca . Seems like they plan to allow you to build your own microblogging service and even if it seems free now I bet it’s going to charge some users.
And the code for it is free too… this looks more like the wordpress.org/wordpress.com module then the exchange model but it might be a good model.
could not agree more…
I think they are just looking to increase their pocket books. And as a business, rightfully so.
However, I think this is a bad move. It’s going to ruin twitter. It’s working perfectly right now! It’s going to go the way of myspace and loose a lot of people I think!
You see, with Posts like this, what you do is put a bunch of Consultants out of business; or, even close down whole divisions of a company.
What are they going to do with their new found $100MM if they get advice for free? Spend it on lavish Holiday parties?
How about a introducing a little capitalism to this space?
This is a terrible idea. The API would be totally destroyed. There already is a distributed micro-blogging platform identica. It doesn’t work so well. There are lots of issues with decentralization. Poorly researched piece. Also Twitter isn’t even remotely related to Exchange.
I would have to agree with Travis (@heinstrom) and Gavin (@gsmaverick) on this one. Twitter and Exchange are apples and oranges respectively.
Poorly researched, indeed.
you are the same guys who said ‘email is not a business’ back in 95
Rather than respond with an Ad Hominem attack right off the bat, perhaps you ought to think more critically about the difference between selling servers needed to maintain a core facet of company operations (i.e. internal communication) versus selling servers for maintaining a public-facing service that is, at best, frivolous with respect to day-to-day operations for 99.9% of businesses.
It’s absolutely an apples and oranges comparison you’re making. Plus if you wanted to sell a server for using Twitter for intraoffice communication, there’s already Yammer for that.
So explain to me what you didn’t explain in the article: what possible benefit would there be for my organization to buy our own publicly-facing Twitter server? We can already claim any of our trademarks on Twitter as it is currently and coming up with a user name like EmployeeAtBrandName is doable today.
ditto. you should have just gone to the beach today, nik.
I just learnt a new word
Yeah I agree.
Right now twitter names are global. How is a local “Twitter Server” supposed to send a message to 100,000 followers that are located on de-centralized twitter servers?
There is no name-to-server mapping like email@myhostname.com.
To establish such a mapping you would need something like a Twitter DNS — a centralized twitter-name-to-server mapping, so say if I wanted to send a message to @joetheplumber, my local server would know which twitter-server to send the message to.
It would work if twitter names were locally scoped such as LarryKing@cnn.com
kashif
more like apples and rotten oranges
How so? Both are communications platforms. Both are based on protocols. Please be specific.
You’re making my head hurt!
Decentralization seems a good idea, but where did the recent $100 million go? They are not going to improve their uptime with that money?
Does any of this matter?
Google Wave is going to wipe facebook and twitter off the map.
Id like to see that happen.
Have you used Google Wave? I have & I love it. However, they are two very different products.
They are two different products now but there is a lot of overlap.
And it is only a matter of time before more GW apps are created that make the overlap more of an eclipse.
I think facebook will be the new myspace (a kiddie site) and the grownups will shift to GW
Absolutely .. they are … as I have not used Google wave and Facebook and Twitter both are different things .. we cant even compare twitter and facebook .. So comparing Google wave with these social network sites would be a wrong thing to do … and I agree to Tracy as well .. that facebook will be more for teens … and kiddies .. as myspace has become….
Best ,
Daina
no, facebook is going to wipe twitter off the map.
+1
Twitter to announce their first fully funded tv show to begin this summer.The show working title is The Insider hosted by (awe shucks,me) Dr Rand Pink, the celebrity gynecologists known by many in the community as simply, The Orgasm Whisperer. Production costs are low because I will simply have a desk and a couch on the beach behind my home interviewing people in the entertainment industry.All inquires regarding our program please contact my bosses who are a bunch of characters: @biz @ev @jack.#JustAGynoLookingForAnOpening
This already exists and it’s called Laconi.ca. It’s the open source, federated, microblogging service behind Identi.ca and the TWiT Army.
Yahoo! Meme, Plurk, Jaiku, and Identi.ca have all proven to be better twitters than Twitter itself. Facebook – too.
Now, Microsoft, Yahoo!, Google, and AOL have their own versions of Twitter in profiles and instant messengers and I’m sure that the future is in distributed systems and aggregation (in Facebook/FriendFeed, Plaxo, etc), so, Twitter will be becoming less and less relevant.
They are lucky bastards though and are probably the only service with such record high downtime, slow (if any) product development, lack of revenue and any serious business plans that was able to collect so much cash.
Anyway, for a distributed Twitter, a project such as WebFinger should help solving the cross-service addressing issues. If Facebook, Twitter, and others adopt OpenID for their profiles, OpenID can be very helpful, too.
+1 for OpenID support!
Couldn’t agree more Nik, nicely put.
Interesting point. However, they would have to come up with a solution for synchronizing the data between Twitter nodes.
Compared to DNS, where your usual time to live (i.e. the rate with which information about domain names is update) is 24h, Twitter would demand a significantly shorter synchronization interval. Moreover, Twitter data is more complex than domain data.
I’m not saying it’s not possible but it would surely mean some hard work.
Maybe, they could take a look at distributed file sharing services like Kazaa and how they handled synchronization problems.
You can use DNS with a very short TTL this is how major websites like facebook move traffic between datacentres.
You can also have TXT records in DNS, this is longer than a Tweet…
Therefore with these two combined you could end up with:
With a TTL of 60 seconds:
DanBUK.twitter.com IN TXT “My Current Status…”
Then anyone can using a VERY simple API (a DNS request) to see my status. Update could also occur in this fashion, well with key signed updates over DNS.
You could also implement something like:
max.danbuk.twitter.com IN TXT “17″
then to iterate back in the updates I have made pull up
17.danbuk.twitter.com IN TXT “My status update then”
16.danbuk.twitter.com IN TXT “My older status update.”
15.danbuk
13..
..
Now I’ve typed all that out it seems like it could be a fun little project to write.
But to gain the stream aspect would require more coding on the client side than the current twitter API where you just ask for X stream.
Hope all of that made sense!
Cheers,
DanB.
A fun side-effect of what you suggested would enable you to check your friends’ updates from behind the paywall of a WiFi hotspot.
DNS (even TXT records) still works, even if HTTP gets redirected to a captive portal page.
It’s even possible to tunnel IP traffic using the same method.
I think you mean http://any.io/
Huh?
First, the follow method isn’t “asyncronous [sic]“, it’s asymmetrical.
Second, your post doesn’t even talk about how decentralized packaged “Twitter servers” would help with scaling of the service itself. If organizations started running their own Twitter servers, how does that help take the load off the main Twitter server? You’re implying that these organizations would willingly take on load from all users, not necessarily those associated with the organization (i.e. employees of a company).
This post doesn’t make any sense.
I agree, that’s going to be the issue. Business isn’t going to take on the load for everyone else. Business isn’t typically that altruistic and then again, this wouldn’t necessarily be altruism at play but them forking out money so people can use Twitter? Think about the shareholders? I mean I might personally like the idea, but I doubt you’ll find many shareholders willing to waste money like that.
There’s also a whole range of technical problems which have already been mentioned with regards to implementing something like this.
In saying all that, don’t stop thinking of ways to improve services online.
“The best way to have a good idea is to have a lot of ideas.” Dr Linus Pauling
(Who says “asynchronous”? This guy is commenting some other post.)
Now, do companies use twitter? Maybe some of them use yammer. Maybe you were discussing yammer. Yammer should probably do like you say. Twitter, I really don’t see.
Twitter isn’t a service though, it’s a website. You can’t compare Twitter with email and DNS, sure it’s mighty popular but their problems aren’t because it’s impossible to do, it’s because they’ve not yet found out how to do it right.
Should Facebook be treated like Email and DNS? Should all the big sites be treated as something that needs to be decentralized? DNS needs to be because it’s a core part of the internet, Twitter isn’t.
“Twitter isn’t a service though, it’s a website.”
you’re right. Sadly, the third party developers think otherwise based on what twitter has promised.
“Twitter isn’t a service though, it’s a website.”
disagree. it’s becoming less of a website and more of a service each passing day.
more than 3/4 of twitter usage is through 3rd-party clients.
in a few months time they could get rid of the web interface altogether and not lose much traffic.
stats from: http://twitstat...lientusers.html
and the third party apps are the first to get shut down when something goes wrong, which is my point. You are right, there are lots of third party developers for twitter.
Dangerous game, for sure.
Nicely put, Samuel. Add Michael’s support to it and the article is now just consultant-talk.
What is twitter’s biggest asset? We must not forget that the users are the biggest asset for twitter. The users make twitter what it is. Why would twitter give up its user base to third parties for a bit of upfront cash?
I am not saying it is not a good idea. I fully support federation. However, my support does not mean anything as far as the motivation for the twitter management is concerned.
Twitter engineers should define and claim TIP (Twitter Internet Porotocol).
Doesn’t really make much sense to me. Twitter is, basically, not that hard to reverse engineer (in fact, it’s already been done) – so selling Twitter servers will fail, people will use an open source version instead. In other words, Apache is a better model than Exchange.
Exchange sells due to its close ties to other Microsoft products, as a consequence of them, replacing Exchange is very hard. But replacing Twitter isn’t.
Given that, if Twitter does decide to let people run their own Twitter servers, they might as well open source their own implementation, and sell optional support (like Java, MySQL). Much, much less money than Exchange, but probably the most they can get for their server.
All that said, Twitter should be able to make more money in other ways.
“replacing Exchange is very hard. But replacing Twitter isn’t.”
then why aren’t the alternatives working? micro-blogging, micro-messaging, or whatever you want to call it, is here to stay – and it is synonymous with Twitter, so we may as well think about how to make it work with everybody using it
See http://status.net , they’re implementing a Twitter-compatible (and more) API on Open Source software.
To clarify, status.net is the new branding and hosted service of the former Laconi.ca, which Identi.ca is based on.
Formerly known as identica.
One word: userbase.
“Apache is a better model than Exchange”.
Apples and oranges.
Very well put.
Google and several other much larger than Twitter centralized operations seem to be counterexamples. Replace “Twitter” with “microblogging” or “live status updates” or similar, and you have a good sentence.
Good luck with that. What you’re really suggesting is that it be impossible to interoperate with the hypothetical Twitter Server, at least not with full features.
I don’t think you or TC understand what Twitter is. It seems like you’re trying to position Twitter in the same column as Wave Server.
Centralization is deep in Twitter’s blood, culture, and it’s original scaling effort back in 2007.
I here’s a presentation I did earlier this year that has a large section covering the history of Twitter scaling. http://www.slid...-we-are-1677682
You need this background info to understand that Twitter is centralized IO infrastructure for real time information dissemination.
It’s not a damn email server.
If you decentralize Twitter then the planet looses it’s only unified, free platform for real time dissemination of information.
The discussions focused on how to monetize, dissect or “fix” Twitter should include respect for its independence of commercial intent and tha value that brings to our lives.
This much might be beyond the scope of your publisher though.
SidGabriel
Exactly.
+1000
so true!
Like many other users on this site, I would have to agree that decentralisation doesn’t make any sense at all. In fact, it shows a lack of understanding of what Twitter actually is because Twitter is inherently centralised. The power of Twitter–and what it eventually involved into–is a real time global information stream. That’s why trend-tracking and hash tags are such core aspects of Twitter. Even if we had isolated islands of Twitter servers with their own set of users and tweets, someone would still have to pool all this data together to meaningfully track trends. This requires centralisation. And then we’re back to where we started…
It might not be, but the point of the post is that it should be. I think it’s a good suggestion, given that Twitter has no better idea.
wonder how the VCs that hopped in a $100M valuation would feel about selling server licenses!
nah, they are holding out for the pre-revenue Microsoft/Facebook/Google/Yahoo aquisition.
facebook? really?
well, the difference is that Exchange is a productivity tool which serves needs within a company where twitter is not. What do you think a big company will use twitter for? c’mon 140 characters of plain text and some links to replace mail, calendar, etc?
LOL, no kidding…What would a corporation need a Twitter server for?
Advertising/Marketing
Sales/Coupons
Press Releases
Customer service
Twice now I’ve bitched about the usability of a Microsoft product on Twitter, and both times someone explained what I was overlooking. Only afterward, looking at their profiles, did I notice they work at MS.
Does Microsoft count as a big enough company?
Yes, Microsoft as a company can use Twitter as a central service, but they have zero use for Twitter server…
Again, people miss the point about Twitter and its centralization being at its core.
If Microsoft ran their own Twitter server, and someone ‘bitched about the usability of a Microsoft product’ they would get fired rather than sales-talked.
Twitter is basically IRC reinvented with a centralized web frontend. aka “chat for the masses”.
now, decentralize it and we’re what, back to 1993 ?
no way Twitter would want to contribute to a public protocol similar to RFC1459. why would they, nobody was ever able to solve the IRC netsplit-factor.
so much for “The Internet was built to withstand a nuclear attack”. scriptkiddies are so underrated.
Makes no sense at all. If set on up for my site, I’m no longer asking people to follow my site on Twitter – but to follow my site on my site. What would be the point of CNN putting headlines on Twitter, if they’re in effect now just putting their headlines where they already are?
Distributed means loads of accounts, reading tweets in different places. You’d ask a friend if they’re on Twitter and they’d have to say something like “Yes, on Node #256 by DumbIdea, Inc.”, while their friend says “Oh, well, I’m on Node #137 by WhyBother, Ltd”.
Also, Twitter’s value isn’t in its software – it’s pretty crap, unreliable and lacks basic features. Twitter’s value is its users and their thoughts. There is, remarkably, no big idea or project they can share around the web. It simply is what it is.
“Twitter’s value is its users and their thoughts”
I am glad someone gets it.
Like Skype, which is decentralized… Yeah, we get it.
Great article in Fast Company last month about this idea of the value of a social network participant – it is the presumed reason why anyone would want to invest in these companies – but I agree with the earlier poster’s comment about the VCs holding out for a buyout by Google/AOL/Yahoo/MS/pick your favorite fat, dumb and filthy rich .com property.
I’m pretty sure that if you decentralize Twitter you essentially have email, though with an idiotic 14o character limit and much much less functionality. What you and many other people fail to realize is that Twitter’s success (?) directly correlates to the fact that it is free and easy. Take away those two core attributes and no one will use it.
I’m sick and tired of people writing about this company with wild speculation. It’s stupid and futile. The bottom line is that Twitter CAN NOT make money and IS NOT essential. Let’s just leave it at that and regard it for what it really is: a toy.
of course it can make money, of course it isn’t essential. It is a valuable tool though and definitely has created a purpose for itself. I would personally see absolutely no problem in paying $5/$10 a month to use Twitter and I’m a casual user at best, imagine how much the “big” users would be willing to pay.
They would be willing to pay $0. If they were really willing to pay, if others were really willing to pay, then they would be paying already. Again, people like you fail to understand that Twitter is only experiencing high usage because it is free. Add a fee and the user base and market collapses.
I’ve just said I’d pay. Are you assuming I don’t exist?
What I’m assuming is that you represent a small minority. In addition, what you fail to realize is that if Twitter became a paid service, a large majority of the user base would cease to exist, thereby impacting the usefulness of the service to people like you, and perhaps impacting your decision to pay. Ask yourself, would you still pay for Twitter if 95% of your follows/followers didn’t use the site?
Also, saying you’ll pay is different than actually paying.
I wouldn’t pay to use Twitter. I did just pay for an app for my iPhone to use Twitter, though. See how there’s money involved in Twitter? I say, get a percentage from that action and Twitter might actually be worth something to their investors.
-1
Mind you twitter has well over 1 Billion “CTR†Monthly. Now, you’d thought if ¼ of their CTR is converted into CPC Ads? It could turn out to be a gargantuan cash-cow!
I for once don’t think real time information/news or “RTI†is going to vanish any time soon.
I think monetization through search is the most effective and profitable form of internet advertisement to date.
If you want a good way to realise how stupid this is, replace every instance of “twitter” with “facebook” and read it.
This is a completely nitpicky comment, but…
>blockquote>As Twitter grows, it needs to be architected more like the Internet.
As an architect, the kind that designs buildings, I don’t understand why the term architected has come into favor. We don’t architect buildings, so why can’t they just design it more like the internet?
Apparently, I shouldn’t be allowed to use markup…
To steal your Google search results!
And to make things really confusing at monster dot com!
Wow, look at all these jobs….oh, wait a minute.
How do people like you get space on Techcrunch without the obvious ability to do any research?
What you’re talking about is the open source microblogging platform called Laconi.ca, no?
Ironically, this view is the exact opposite of the direction that Microsoft Exchange is taking.
An increasing number of businesses are using “cloud” based Exchange hosting to get out of the hassle of running their own system. Microsoft itself is offering hosted Exchange, and Exchange 2010 is the first version explicitly designed to be cloud-based.
The real problem is having the right algorithms and the software to implement them (which in turn means that the real problem is for twitter to hire the people who can develop them.)
I think this may happen, and I hope it’ll happen in the future. But I don’t agree when you say that the async “friendship” mode is the reason for which twitter doesn’t scale.
I mean, “followers” and “following” are something as simple as a database table with two column.
I’d like to see something about twitter technical structure, but it’s impossible. A simple explanation for twitter downtime/slowtime may be this: twitter was structured with an architecture that was not so scalable. I think they’re adding hardware, but it’s something temporary (and they get users faster than they get new hardware) and they’re planning on a new architecture (more scalable)
Nicola
I don’t care what the proponents of the status quo say the decentralization of Twitter is inevitable.
Its just a matter of what form it will take. Will Twitter spearhead it themselves? Will Google or Facebook make it happen? Or will Identica become powerful enough a force?
Personally I think Google and Facebook will have a hand in the destruction of Twitter, but only if they play together nicely…
Also, I just had a baaad feeling that Twitter’s business model will be an in house “App Store” per rumor of “Twitter Labs”…
Meaning Twitter may be ramping up to squash all of its API sites by creating Twitter-branded apps and selling them to users. It wouldn’t make economic sense for them to be involved in the decentralization, but anybody who currently has an app could feel pretty slighted and want to port them to an open platform if it happens..
The current value of Twitter is not in it’s technology, it’s in it’s userbase. You could write a better engine to do what they do, but you can’t convince enough people to use it.
Your suggestion to decentralize essentially has them giving credibility to external sites (who licence their engine) at a cost of what they value most (their users). Once I’m buying t-server from them, a new startup can compete on features..
I’m sure there are lots of good ways for Twitter to make money, but I don’t think that diluting their user base is one of them..
0.02c
see XMPP.
Dude, why in the hell does the fact that the “relationships” on Twitter are asymmetrical mean that they can’t scale? There are tens of millions more connections on Facebook than Twitter, and they don’t have 1/10 of 1% of the downtime Twitter does. It’s one more field in their database that says, hey, send the updates here, too. The reason there is massive downtime is that they spend too much time making pretty graphics and not enough time under the hood.
the demand for a DIY microblogging platform is sure to grow in time, but its hard to see why people would choose twitter over other commercial or open source alternatives. as someone rightly pointed out, exchange didnt succeed just because MS put it out there, it had huge interoperability advantages and MS’s deep relationships in corporate IT deptts that allowed them to dominate the space. Making an email server isnt hard, selling it is. The same is true for microblogging, probably even more so. Its a very weak analogy.
Also, its not clear how allowing businesses to set up their own twitter services will reduce load on twitter.com. It might well make twitter into another netscape, besides commoditizing the product completely. At the end of the day, if you want to make money, you’ve got to own the customer.. and i think twitter gets that. Increasing the distance between yourself and your users is rarely a good strategy. I’m sure there are less risky approaches to scaling the product.. rooted in technology rather than business strategy.
I think what makes Twitter so special is free unlimited access to real time people producing real time information, if we segregate the environment to be privatized i.e exchange then we limit the or close doors on that free information, granted that some companies do want a secure and private environment to micro blog with no outside access.
things like Bantam are cool http://www.bantamlive.com/
But I like the idea of being able to have your own twitter server in your company and allow it to have a secure environment of twitter along with a public one but i think if that was the case offering it for free would be ideal and then offer support packages i.e MySQL etc..
But whats the old saying every thing has a life expectancy, not to say that Twitter will die anytime soon, but what are all those dust balls blowing by at Myspace!! I imagine with time something else will come down the pipe thats better…
but until then i think people will keep on twittering while it’s hot! i know i do it’s like an information overload highway!!
+1 for OpenID support.
“a single account can theoretically have a number of followers limited only by the total number of Twitter users. This adds massive complexity to the system”
No it doesn’t. It just makes very big (but simple) tables.
who would buy and deploy to their network a server created by twitter? It’s not like they are known for their great security or anything…
Good post Nik – totally agree with you.
I’m surprised nobody’s mentioned usenet (i.e. “Internet Newsgroups”).
I think it would be possible to design a Twitter protocol that would mimic Usenet. However, unlike Usenet which has become over-run by spam (and has always been a bit more underground), Twitter data is extremely valuable.
There would be a huge incentive to hosting a Twitter node: You would get to keep huge reams of data, without running into the typical rate-limiting issues. Heck – sign me up! Data==$$$
There are all sorts of security-related issues which would have to be dealt with, but I think it’s worth the effort to attempt this model. It would truly lead to a win-win situation for Twitter and its users.
“Twitter data is extremely valuable”
I admit to being a Twitter skeptic. Could you elaborate on “valuable”?
Basic marketing info, consumer reactions and the like. There are tons of tweets on products and services ongoing and you can develop a “conversation” (as the marketing kids say these days) with a following there. Okay, maybe more of a series of telegraphs than conversation per se, but anyway it does make sense.
Of course it’s valuable only so long as there’s a large mass of users recommending, disparaging, and so on re products and services.
There are other valuable uses of Twitter, from a broadband announcement server (use cases include groups that use Twitter to receive instant notifications while mobile or grounded) to research on less commercial topics (though this varies a lot by subject) to the ability to do social mapping of how ideas spread.
I’m not suggesting any of these are unique solely to Twitter or even best performed there, but the current large audience and the way the messaging forces one into sound bites does provide/promote the above capabilities.
But Twitter does remain unchallenged in scale to date with its open-to-all communication mixed with targeted communication, and that alone creates a powerful bank of data. Undoing that immediately unravels the value. That said, if one believes Twitter cannot be sustained, then that unraveling is inevitable and such as the suggestion of this blog makes some sense.
Anyone that knows me knows I love the idea, but it’s too risky for twitter since almost immediately they’d have serious competitors in the space. G, M and F would have no barrier to entry. But as you imply a smalller slice of a “profitable” pie would be better. But would G, M, or F charge? Only for the enterprise, so it’s more of a gamble than you make it. More likely, is we do the whole IM silo thing again in the near future and the open networks nibble away at marketshare like XMPP does.
An entire post which basically says that Twitter should try to white label an email server. Brilliant.
Not only that.. But you’ve traded the problem of scale on one server farm, for an arguably even harder problem: multicasting to thousands of remote clients on every message. In other words, good luck.
Where’s the value here for anyone to buy their own internal server? Replicating this service is trivial for a closed internal environment leaving zero barrier to entry for any competitor other than not having the “Twitter” brand.
If it’s so difficult for twitter to figure out scaling issues, thats only going to open the door to Google or Microsoft to step in.
Twitter should just put the damn banner in footer of homepage and other pages ….and may be couple in the sidebar where it puts trending topics and get over with it!!!
It wont bother the regular users that much as its not right in middle and annoying (just like facebook ads)
Now that they have made their “deal with the devil” Twitter must monetize their services ASAP. Either as a platform or by providing services within the pre existing model. Perhaps they should have a contest to determine the best manner by which to customize their product??? (a la Netflix)
Yes to decentralizing Twitter’s architecture. I’m surprised they haven’t done so already. As far as Twitter servers, that makes sense for businesses who want an inside-the-firewall extension of Twitter.
But Twitter’s value is in its critical mass. So, a Twitter-like functionality on its own within companies is not as valuable as if it were an extension of the public Twitter. So, we’d have a Public/Private Twitter in the long term with on-ramps and off-ramps just like VPN’s & Intranets are today. That makes sense.
I’ve already argued that Twitter is becoming more like a utility- but an important one. All innovation is coming from outside of Twitter. So, Twitter could charge utility rates to the heavy users, like an ISP model of sorts.
A very bad idea indeed.
I don’t know if this has been said (too tired to read all the comments), but the problem here is that no protocol that can be resolved via DNS on the internet today is proprietary. (Their are localized proprietary records like ActiveDirectory records, but none that actually live on the root DNS servers to my knowledge).
As soon as they decentralize and we have TWX records in DNS, everybody and their brother will come out with a competing implementation of a Twitter server. There is a high chance that someone will do it better and cheaper than Twitter and now they’ve kissed their baby good-bye.
This may make Twitter more reliable, but it would be a horrible business move for Twitter.
I completely agree. Decentralicising Twitter would lead to competing implementations of the Twitter protocol. Why would companies want to pay Twitter when there is a Twitter-compliant open source alternative available?
You may be right. But what the Twitter will lose the most is the “control” on their contents! And, they don’t like to lose that. In fact, left to themselves, most people won’t like to lose control over their contents. The emotional appeal of this issue overshadows the visible advantage of “let us benefit all together by combining our resources”. Normally people behave in a manner conforming to “let me progress less; let me face some difficulties too; but I must not lose control over my activities”.
With the recently raised amount of $100 million, Twitter may be trying to raise more resources for its infrastructure. It may even try to “decentralize within its own organization” without ceding control to outsiders, until one day they find that their whole traffic is completely choked and people have started shifting to other networks!
Comparing twitter to email doesn’t make much sense. If twitter goes the decentralized way eventually, it won’t be because of scalability issues. Ask Google: you can scale a centralized service.
Email is not owned by a company, it’s just a standardized way of sending messages over the Internet (via specific ports, following certain technical rules, etc.).
Twitter is a platform, owned by a private company, that is mainly a website.
If they want to go the decentralized way, “tweeting” will have to become a standardized way of sending micro-messages over the internet and I’m sorry, but we’re far from that.
Twitter can scale, it’s not an issue.
Exactly.
They should really find a way to make a buck, especially while they’re hot. The longer they wait, the more money they forgo. Strike when the iron is hot!
I’ve thought about how twitter can be distributed. And it isn’t that easy. For example, how would you sarch the real-time stream then?
The answer, in my opinion, is just sharding the heck out of it. Well, I just see horizontal partitioning (sharding) as the answer to almost all scaling problems. And if you want to set up servers around the world to listen in on specific twitter traffic, then SOMETHING has to be centralized. Just shard that centralized thing and you’ll be fine, more or less.
Whether sharding will fix Twitter’s scalability problems or not, we can all agree that their current architecture does not scale. They have to redesign their architecture from the ground up, which would be a monumental task considering how many TB of data they’d have to convert over to the new database design.
Mike, here’s an interesting viewpoint I have been considering. The internet progression seems to be the following:
1) centralized mainframes with thin clients
2) decentralized services running over protocols
3) centralized social networks
So the web is back to being centralized. What advantages does this give over decentralized? Well, I can now see news feeds and other aggregated things. I can create collaborative environments like wikis or Google Waves.
I have thought about the possibility of
4) decentralized social networks
The problem is that a lot of the aggregation that happens in social networks will have to happen across the decentralized systems, and it can be very hard to accomplish this. Authentication is one thing, but then there are also updates. This will have to happen with some sort of push-based syndication.
I am writing up something more concrete about it on luckyapps.com/blog, after I revamp that blog
Oranges, like apples, grow on trees… Many people believe that Google Wave will eventually produce the next standard that will replace Facebook, Twitter etc. Is the technology that good or is the view just wishful thinking?!
Wishful thinking.