May 5, 2008

Twitter Can Be Liberated - Here’s How

Michael Arrington

126 comments »

Over the last few days a number of popular bloggers have complained, loudly, that it’s time to ditch Twitter and move to a decentralized version of the service that won’t go down every time usage spikes. Generally services like Twitter, once they reach a critical mass of users, can’t really be stopped because the network effect kicks in as a massive barrier to entry. But one aspect of Twitter - it’s openness - could also be its Achilles heel.

Scott Hanselman mused on how such a system might work yesterday. Dave Winer has also written extensively about this, although he’s more focused on simply backing up Twitter messages so that they are still available when the site goes down. He isn’t paying attention to the real benefit of Twitter - the spontaneous, asynchronous conversations that pop up between every changing groups of people. Instead, he just wants to make sure the data is secure.

Marc Canter also weighs in in a surprisingly lucid post. Infrastructure needs to be decentralized to be reliable, he says, pointing to DNS as an example.

Decentralizing Twitter isn’t about having backups of content if the service goes down. It’s about making sure that the service as a whole can’t go down, and allowing all those conversations to continue unabated no matter how popular the service gets.

Here’s How To Decentralize Twitter

Chris Saad, co-founder of DataPortability and founder of startup Faraday Media, thinks he knows one way Twitter can be decentralized effectively.

The key weakness in Twitter (and therefore opportunity for a new decentralized approach) is the fact that so much Twitter activity occurs off Twitter.com. Users are getting very used to using desktop clients (Twitterific, Alert Thingy, Twhirl, etc.), IM, SMS, and other interfaces to talk to Twitter. Those third party applications can be tuned to lock in to the new decentralized Twitter-like service instead or in addition to Twitter itself.

Users on the new system will post to a microblog. Third parties can create platforms for these blogs, and have them be certified as compliant with the microblogging standard - posts of 140 characters, no titles, etc. Users could also install compliant software on their own servers - much as they do with Wordpress.org today. There would certainly be an open source project around this shortly.

The hard part is putting these microblog posts together into a Twitter-like conversation where people subscribe to those writers they like, can respond via an “@[username]” mechanism, etc.

This can’t be done efficiently just via RSS because rapid and excessive polling would bring servers to a halt. Instead, Saad thinks wrapping RSS in XMPP, an open standards based instant messaging protocol that was originally created for Jabber and is now used in various applications including Google Talk, is the answer. XMPP allows for pushing of messages to subscribers, which removes the need for constant polling. For more of Saad’s thinking, see his site on their product SyncStream, and they’ve already written code that will do this based on their proposed standard called “GetPingd.” Twitter uses XMPP in their API already; third party applications like Google Talk integrate with Twitter via XMPP already.

If users begin to “twitter” using this new system, applications like Alert Thingy can simply add it to their functionality. Instead of using Alert Thingy to sign into just Twitter, you would also create an account at Alerty Thingy, too, which stores your subscription lists. This is analogous to what feed readers like Google Reader do today with RSS subscriptions.

XMPP already has a mechanism for tracking subscribers which performs the same funtion as the “followers” list in Twitter. Users can therefore have a list and count of followers back on their microblog.

Handling replies is a little more complicated (the @[username] feature on Twitter). One way to do this is to use the existing RSS infrastructure specifically with Google or Technorati Blog search to monitor for @replies and feed this to the user applicaitons. Results wouldn’t be limited to just microblogs, but I’m not sure that matters. It would also be simple to block spammy stuff by simply clicking on a button, the same way Twitter works today.

The net effect of this theoretical platform is to move the publishing to a completely decentralized network, and move the hard part of Twitter into the third party aggregators (which is already a competitive space). There would be no one central bottleneck that could fail.

Twitter (the company, the service, the site or the software) isn’t part of this new decentralized platform, of course, so they’ll oppose it. But can it happen? Absolutely, because it doesn’t require a hard reset to a new Twitter-free world. Existing Twitter clients could add support for GetPingd and the rest of the infrastructure and it would work seamlessly with the existing Twitter world. Anyone could create a website that duplicates what Twitter does today, but supporting the new decentralized framework. And a limitless number of microblogging applications could emerge to join in the fun.

And we’d never have to deal with outages again.

  • Sphere It

Trackbacks/Pings (Trackback URL)

  1. Decentralized Microblogging - Twitter 2.0? « Paying Attention
  2. Spontu
  3. Hueniverse
  4. Marc’s Voice » Blog Archive » RSS + XMPP = Decentralized Twitter
  5. Scripting News for 5/5/2008 « Scripting News Annex
  6. TechCrunch Japanese アーカイブ » Twitterは解放できる―これがシナリオだ
  7. Open, Distributed and Educational Twitter « mindstorms
  8. WinExtra » Twitter: IRC with a new shade of lipstick
  9. Maoxian » links for 2008-05-06
  10. borderlinetheory.com » Social Portability and the Free Internet
  11. Left Of Center » Blog Archive » How to Keep Twitter from FAIL.
  12. notes, thoughts, ideas and responses » With a little exericise, Twitter will be in great health
  13. Twitter as Expressive Identity Affordance « Jacob Kramer-Duffield thinks
  14.   De la aplicación al servicio web by Trisoft Peru
  15. P2P Blog
  16. Could Twitter go P2P? | Fresh Web 2.0 News
  17. » Decentralizing microblogging is about freedom, not because of Twitter outages
  18. Cloudlands
  19. Prototype for distributed / decentralised microblogging using semantics at Cloudlands
  20. Prototype for distributed / decentralised microblogging using semantics at Cloudlands
  21. links for 2008-05-09 « Unseen’d
  22. The Blood Brain Barrier
  23. Weblog on steroids: Twitter als Vorbild : Georg Holzer
  24. Technology and Gadgets Review » Blog Archive » The Blood Brain Barrier
  25. SIOC-based microblogging : Alexandre Passant
  26. 101 Twitter Resources | Traffikd
  27. Biznes na poważnie…. » Boostrapping a decentralized Twitter

Comments

RSS feed for comments on this post.

  1. Dan Delphin

    twitter is NOT going to like this post. :-)

  2. Nick Malaguti

    I think it’d be great to offer decentralized twitter, but who are the major players going to be? Who wants to foot the bill? Does everyone run their own server and hook up to the decentralized system or do we end up with 3 twitters instead of one?

    Who holds all the tweets ever made so we have a history? Does every server get every tweet and store it away or does twitter still end up as the canonical storage platform?

    How does track work? Do I have to jump from site to site to see everyone’s profile or can I still just go to twitter and check them out?

    I feel that by moving away from a centralized twitter we gain reliability but lose ease of use. Will we go in this direction? Yes. Who will be the big winner? Whoever can answer all these questions and has the easiest to use product.

  3. Shawn Farner

    What about SMS updates? That’s one reason I love Twitter.

  4. Internet Man

    For most it would be way too much work to actually use this. Only takes a couple seconds to sign up for twitter and they will have their infrastructure fixed eventually. It’s not like Twitter is that big of a site currently. They just chose a shitty language to build it in. Also, who is going to pay for SMS/IM support? SMS is costly and IM isn’t a walk in the park when dealing with high traffic.

  5. Michael Arrington

    Nick - the idea is to turn the notion of “twittering” into the same system that handles “blogging” - no one owns it, no one is a bottleneck. A limitless number of companies compete using a set of agreed upon standards.

  6. Lutfar Rahman Nirjhar

    Nice article! :)

  7. Rodney Rumford

    Metcalfe’s Law has kicked in big time. People are jumping on the twitter bandwagon at a rapid rate.

    I just don’t see people moving off twitter despite the reliability issues. Once people use it and “get it” they are hooked on this newly evolving way of communication and listening.

    The third party services built off of the twitter API is impressive and growing rapidly. Some are crap, some are really cool. Good ones like brightkite and twitpic are quite interesting.

    Decentralizing is an interesting concept in theory and not without significant technological challenges of it’s own. The jury is out.

    Cheers!

  8. jason goldberg

    This is exactly why google or microsoft or yahoo or aol or facebook should buy Twitter now. It is going to take a massive effort to scale twitter yet there is an opportunity now for someone to cement it as a de facto standard. Would anyone mind if Google for instance created the decentralized twitter envisioned in this post? But do it starting with twitter vs copying it.

    Forget about raising vc money for twitter. Go partner up now.

    Sure, open twitter makes a lot of theoretical sense but still needs an owner to make it happen. Who could build it from scratch and commit to it? Mozilla?

  9. FT

    The solution exists and has existed since the early 1990’s. It’s called IRC.

  10. Chris Saad

    I’ve added a few more comments on my own blog - but most of my input made it into the post :)

    http://chrissaad.wordpress.com.....witter-20/

  11. Alex Popescu

    Excellent read Mike! Your wrap up is making the entry I’ve worked on during the weekend a bit obsolete so I’ll have to re-write it, but this one is great.

    Alex

  12. Dimitris

    I agree with Shawn - what about SMS and offering a route directly to and from mobiles? That can’t be readily decentralised with an obvious way to cover up for the costs of SMSing. And bear in mind that SMS (not using mobiles to access the web) is probably the way to reach (even more of) the masses.

  13. Drew Freyman

    The decentralized approach you advocate comes at a cost: increased complexity, increased development effort, and increased running costs. Accordingly, in response to Twitter’s unreliablity it is more likely centralized competitors emerge and one or two of them succeed. Given Twitter’s problems and potential, Twitter 2.0 will probably choose technologies that will allow it to be more fault tolerant in high traffic conditions.

  14. Daniel Richard

    I like Twitter! Most likely won’t be moving off it anytime soon especially with the desktop app Twhirl that makes Tweeting so much easier.

    http://twitter.com/Daniel_Richard

    I haven’t faced any downtime by Twitter yet, most likely cos’ I’m one of those who signed up when Twhirl was released, and when I got an app from Facebook that allows status to be updated and sync between the two.

  15. kosmar

    can we haz decentralized flickr, fireeagle and facebook just at the same time, please ? O_o

  16. cocoy

    i don’t think recreating another twitter is the answer. problem could be in code or infrastructure. just throw money into it to scale up.

    there are lots of twitter-clones: pownce, jaiku. people use them as well, though differently but twitter’s got a bit of community already.

    twitter actually reminds me about irc but web2.0-ish and geared towards our 21st century mobile lifestyle.

    guess it’s time for twitter to find a business model or be bought by a google or google-like entity that can feed it money.

  17. Alaska Miller

    Oh wow, we’re progressing by essentially remaking AOL Instant Messenger.

  18. martin english

    “One way to do this is to use the existing RSS infrastructure specifically with Google or Technorati Blog search to monitor for @replies and feed this to the user applicaitons.”

    in a word, centralise :)

    Centralisation DOES provide some benefits. For example how reliable are these server ? not from uptime, but from ownership. if I see a message today from x saying “@y suks …” I can reasonably reliably say that IS @x saying it.’ Depending on the network links between various servers under you scenario, we may not hear from someoen in a timely fashion ofn a particular conversation. is this because they don’t care, because they’re thinking, or because their twitter-server is currently stuffed (or owned)?

    Perhaps implement Open-ID ? Even then we will have shelisrael.com BS all over the place again, if people don’t claim their names fast enough.

    There still needs to be a central point that ‘authorises’ and verifies the twitter-servers. Like DNS, it can be buffered at various hierachial levels, but again this means someone / somebody has to be responsible for the secuirty of this data, AND the propagation of this data. Who pays ? users via subscription ? Server owners via subscription ?

  19. Rick Curran

    Somebody mentioned this already, but what about the SMS part? That’s a pretty big part of the usefulness of Twitter for many is using SMS. At the moment Twitter covers the substantial cost of all the SMS notifications that people get, I don’t think that’s going to be an aspect that can be recreated so easily, basically as it doesn’t come free.

    It’s definitely interesting times for Twitter though, I can’t really see how they can monetise the service. Blog posts like this just show how much twitter has become part of the infrastructure, it’s more of a protocol really, and as the majority of it;’s usage is via the API I don’t think ads are going to be the solution.

    Very interested to hear Twitter’s response.

  20. Eran Galperin

    It’s much simpler than that - Simply go peer-to-peer (P2P). Liberating centralized networks since 1999 ;)

  21. Chris Saad

    @Dimitris An SMS service would be easy to run on top of the architecture described in the post.

    @Daniel Richard you would still use this imaginary new system via Twhirl or something very much like it.

    @cocoy The Twitter ‘clones’ you mentioned are not clones - they are very different to twitter in many fundamental and important ways.

  22. Ray Nimmo

    This sounds a great idea, by decentralizing the service it will allow it to operate if there is an outage in any part of the world, something that affects us in Asia nearly every day.

    I really cannot see this happening with twitter though, I have been waiting for a while now to see what twitter rolls out to actually make them money, yeah they have received a lot of funding recently, but how are the money men going to rake back their cash on a decentralized system?

    The most obvious way to bring in the dollars is through advertising and merchandise tie-ins; the merchandise I am sure they will make money on, but how are they going to rake in the advertising cash?

    Or are they running a charity :)

  23. Ben Myles

    In case anyone’s interested, last month I wrote a little about the inevitability of Twitter becoming decentralized, and what that means for Twitter’s business model (or lack thereof): http://benmyles.com/post/33195820.

  24. Alex Kritikos

    I read blog posts and comments about twitter’s scalability problems for a while now. What i keep wondering is how come no one is looking at the traditional ways the software industry is scaling large scale data distribution issues. Look at stock exchanges, brokers and financial institutions: Data is very volatile and needs to be distributed to massive numbers of subscribers, while at the same time it needs to be captured no matter what the load or connectivity state of the application is. The answer is Message Oriented Middleware (MOM). Of course traditional MOM technologies are not designed for the internet era as they provide HTTP/client APIs by using servlet like technologies which break end-2-end security as well as guaranteed delivery at the web server layer. There are solutions though that are designed for the internet such as our own Nirvana product, providing a built in HTTP/S stack and comet/ajax client APIs. For more details take a look at http://www.my-channels.com

  25. Nicole Simon

    Many do use Twitter in an IRC manner (or Jabber for advancements), and except for the SMS part it already is there.

    But what is sms really about? When I am on the road in Germany, I do use the mobile interface and not sms to communicate from and to Twitter, only when I have no data access it goes sms - and even then only occassionally.

    The one I really do use is dm for direct messages and that is basically because I may not have all your phone numbers. It would be something I might be willing to pay for, while at the same time I’d want better control of who can send me those sms I pay for.

    That set aside - why not go decentralized? It would finally allow for decent clients while at the same time allow for the idea to grow.

  26. Mickaël Rémond

    I explained two months ago how to build a decentralized Twitter with ejabberd Instant Messaging server.
    The article is here:
    http://www.process-one.net/en/.....on_server/

  27. Darren

    sorry I strongly disagree 50% of users are still using twitter.com according to rww research.

    http://www.techcrunch.com/2008.....heres-how/

    I commented this on the last post do you read your comments :p

    I find it an interesting problem to solve as most hackers do, The @ just needs a system to point a url of a user. The bigger problem is conflicting usernames on different system.

    1 final note I have is what are these problems people are having with twitter being up? I have not had any down time in a while and I use the .com for my client.

  28. cocoy

    @chrissaad, yep. i know… they’re all used and act differently from twitter. people use twitter differently as well. as a point of clarification i use the term “clone” very loosely.

    …twitteriffic has ads on its twitter feed. i don’t mind having those, especially if twitter’s getting its fair share.

    …DNS… isn’t exactly decentralized. there are “root” servers see wiki for details: http://en.wikipedia.org/wiki/D....._in_theory

    what i think people want is to get twitter as a service to be as reliable as dns, not exactly whether or not twitter is decentralized. it would be interesting to know what twitter thinks.

  29. Darren

    opps wrong URL for 27

    http://www.readwriteweb.com/ar.....e_list.php

  30. bagamare

    yes. i agree with the first commenter. ciao from romania.

  31. scott

    I don’t know why you would want to wrap RSS which is already a wrapper. How about XMPP wrapping a set of microformats? The formatting bits can be filtered out when the messages get translated to connectors like SMS.

  32. ozz314

    Open standards and Decentralization FTW. The “twittering” and “blogging” analogy mike gave makes it easy to understand. Same can be said for emailing. No one owns the ability to email.

  33. MyMesh

    One success of Twitter is SMS deliveries.. which could be expensive and not readily decentralizable (ie., not because of tech reasons)… might be when SMS becomes free… that said, a decentralized twitter without sms would still be very useful.

  34. vepa

    Great article and completely possible. All hard work (listing replies and updating followers) will be handle by google and/or technorati. And everyone offering services like sms, send as voicemail/email , will earn money form it, it will never be down as long as internet is not down :D , every blog can post to twitter similar to generating rss or have it as widget, sideblog.

    Only problem it needs to be done in 1 step hassle free process. Setting up an account and starting using it needs to be 1 click solution regardless of having host. Also it is better if all accounts are opened from one location to gain popularity to the service. otherwise it may die if everyone will refer to different service to signup. Unless term twitter is coined and added to international dictionary similar to term blog.

  35. James

    Does twitter have any patents relating to what they have developed to date to “protect” themselves from a decentralisation approach?

    eg if they had a patent for the use of @xyz or the 140 character payload then at least they could keep themselves in the picture during the development of this infrastructure.

  36. Andrew

    >> you would also create an account

    Oh great - yet another account somewhere. Here was the perfect opportunity to talk OpenID - yet.

    That’s a non-starter for me. The idea of services having to store their own tweets and then move them over later makes me envision those OWN services having their own scaling problems. It’s like solving a problem by simply spreading it out.

  37. Azhar

    We can do all these or twitter can just upgrade their servers. Keeping it simple is very important in everything we do.

  38. TedC

    Chris touches on the future of user owned content. Liberated service agents servicing personal requests. In this case, I’d configure my agent to serve up my tweets and alert for others. Unfortunately, the decentralized model changes the profit paradigm as well. Twitter.com’s profit potential lies in creating this messaging backbone

  39. laloj

    http://www.tech-exposed.com

    So what are the odds that this will happen? When could we expect change??

  40. Michael Kimsal

    So ‘twittering/tweeting’ becomes like ‘blogging’, and anyone can run their own service. Services will compete for tweeters, and we’ll see a handful of platforms that handle a huge majority of traffic. This will become another form of centralization, and will have the same uptime problems we have now, just slightly less impactful. Yes, anyone can set up a blog, but a handful of service providers - blogger, wordpress, livejournal, etc - provide blog service for a large percentage of blogs out there.

  41. Jakob

    twitter likes to be centralized - why else would the vc give them money?

  42. Dave Winer

    Chris has just restated what I’ve been sayimg, calling for a bootstrap, the first step of which is to get the data stored off twitter.com.

  43. Jakob

    nice linking to everything chris saad! i guess you get that as a house guest of casa arrington?

  44. Matt

    Michael, If you’re suggesting we move to a de-centralised twitter, then we need a de-centralised name for the initiative. Twitter is a brand mark of a commercial company.

  45. Jim McNelis

    Lots of wasted bandwidth if this thing has to communicate with millions of separate microblogs at the same time, rather then stemming from one place, no?

  46. Redge

    I think a decentralized Twitter (or Micro-blogging platform) could work very well if there’s a standard and if it’s easy enough to operate. Remember, not everyone is tech savvy and many people probably don’t want the trouble to operate their own “twitter” server.

    By the way, I’m curious to see what Google will do with its Jaiku acquisition…

  47. Louis-Philippe Chiasson

    To clarify a few differences:

    IRC, XMPP-based chat, SMS, microblogging, these are all, in essence, different ways to get one thing going: post a message so one or more users can read and may reply to it. But how they differ is in the intention you have when using the service.

    In IRC and similar chat services, your intention is to have an immediate response, a ‘conversation’ with one or more other users.

    In SMS, you can always expect a delay in response to your post, or have no response at all.

    In microblogging (in essence, just like blogging), your intention is to post a message to the masses without any direct response (in other words, no conversation), but users can comment on your post (comment, @reply, etc).

    Having that in mind:

    Twitter was developed more as a microblogging service than anything else, even though it now has more of a conversation feel to it.

    Chat services don’t usually store you conversations, generally speaking. They connect people and pass along messages. Whereas blogging and microblogging, all messages (posts) are stored on some server. For a decentralized microblogging service to work, all posts and replies would need to be semantically formated so they can be fetched with ease from any server and any similar product. Only then can the service be used by different products, but can still ‘plug into’ other similar products.

  48. Jason

    @coycoy - Please stop talking about something you know nothing about by backing it up with a wikipedia article.

  49. Scabr

    I need Decentralized Twitter ( like WordPress)!

  50. Robert

    Interesting proposal, but some things don’t really work.

    Replacing RSS with RSS wrapped in XMPP for example. Sounds like a great way to cut down when you have only a handful to share with, but this can get complicated. The advantage of RSS is that anyone can poll at any frequency provided they know the location and have permission (otherwise 403). With a push, if there are a ton of “subscribers”, you have issues:

    - Delay - last on the list gets it after the first.
    - Touch to cache - You can’t easily cache a push. You need to push for each user. This is really inefficient. RSS is cacheable because it’s a get. Both because you can leave a static file on a server, and because agents (think Google Reader) can make 1 GET for 1000’s of subscribers. If you push, that’s thousands of extra push’s. Just a giant waste.

    I’d suggest WordPress is really on track to conquer this. There’s even feeds for comments already. The question is how to aggregate this into something useful. If you monitor 10 people, it’s no big deal, but if you follow hundreds, your going to need a few servers to manage all those feeds.

  51. Loren Feldman
  52. Eran Hammer-Lahav

    Driving back from Social Graph FooCamp a couple of months ago with Blaine Cook, Eran Sandler, Matt Biddulph, Chris Saad, and myself, Blaine gave a detailed explanation of how a distributed microblogging service might work using Atom and XMPP. In fact, he’s been talking and writing about this for over 7 months now. I seem to recall Saad having a hard time buying into all this in the car… looks like someone has been taking notes…

  53. Hernan Garcia

    Come on! The one and only winner here is Saad.
    How convenient is to already have and be the owner of what he thinks may be the solution.
    And your support to all this theorical matter is just shocking Michael.

    Twitter the company should become a service provider and create the necessary infrastructure behind twitter the product, to scale well.

    Nonetheless, nice and very smart move Saad, DIVIDE AND CONQUER! It’s easy to re-use the community Twitter already have than start from scratch with a clone, right?

  54. Andrew

    Mike, I’m puzzled by: “one aspect of Twitter - it’s openness - could also be its Achilles heel.”
    What does openness mean in this context? My best guess is that it means that open API -> clients -> users -> overload on the centralized server.
    But the problem there is scalability, or lack thereof, rather than openness.
    Sorry if this seems nitpicky about a post obviously written in a great hurry.

  55. Mostafa Hussein

    How about making the process of sending SMS similar to distributed bittorrent or Tor servers.

    By allowing users to run local SMS servers in their countries to receive and deliver those messages to local users.

    It can be a subscription based or each server can limit the amount sent daily.

    This will circumvent delays in international SMS delivery and the fact that some operators block twitter SMS to start with.

  56. Marc Canter

    :-) Time for the shared social graph!

    And I found a typo. Is there a bounty paid?

    http://blog.broadbandmechanics.....ed-twitter

    Though the notion of complaint software is probably a hot Web 2.0 play Fred Wilson will fund!

  57. Joseph

    I love the decentralized concept in theory; not clear on business model (and thus, sustainability) for third-party service providers just yet.

  58. Marc Canter

    Business model? Twitter has no business model except to be bought.

  59. Bob Wyman

    Decentralized Twitter = XMPP + PubSub (XEP-0060). There is no mystery here on how to do this. The standard exists and was designed with this sort of application in mind. (Note: PubSub.com used XMPP and the XEP-0060 extensions years ago and we participated in the development of XEP-0060) The only “odd thing” about Twitter is the limit of 140 characters — but that is easily handed as a message type. Also, Mike’s base note says Saad suggests wrapping “RSS in XMPP”. That would be a mistake. What should be used is what Twitter is currently supporting in their existing PubSub feeds — that is “Atom over XMPP” (For a draft ID see: http://www.xmpp.org/internet-d.....fy-06.html)

    In my opinion, Twitter should be taken simply as a proof that inter-personal push communications makes sense. However, the network should not tolerate centralized designs that require significant communications to route tthrough a central hub. Twitter is architecturally wrong — dangerously so. Now that people seem to be understanding the value of publish/subscribe in this space, we should replace the existing system with something that facilitates the free flow of information on the net. Twitter will still, of course, have business opportunities and should be able to find a way to monetize participation in a significantly expanded universe of Open Twitter use.

    bob wyman

  60. Harold Gilchrist

    The future of Twitter and Microblogging in education is an open protocol

    http://hgilchrist.wordpress.co.....-protocol/

    There are many high scale school social mapping based applications that potentially will only evolve if an enterprise class microblogging standardized architecture exists. This would create high and reliable scalibility - the potential to scale beyond just one school … a whole district at the very least and the ability to scale higher - all in realtime - applications such as: all call announcement system, all call emergency system and many others based around staff to staff, administration to staff, admin to parent, parent to staff, student to student, student to staff etc. communications systems.

  61. Chris

    @Michael, have you not heard of NoseRub? It’s not even 1.0 yet but it seems to be screaming for some attention as the decentralized, micro-blogging platform of the future.

  62. Prick

    Why go through all this trouble so that guys with no lives who have no girlfriends can text message each other?

  63. dave

    uh, yonkly.com - open source twitter clone with twitter calls? anybody? or wordpress?

  64. Marcel

    interesting. Maybe Twitter should try to create the standard for this… or we might

  65. Bob Wyman

    Many of the blogs commenting on this thread seem to be debating the correctness of “microblogging.” The scaling issues with pull-based crawling of feeds are well known and clearly, a system based solely on file-based feeds would be a mess. Of course, others argue that microblogging should be supported since not everyone has access to a usable XMPP system or the knowledge needed to build one. However, it is also clearly the case that that file-based feeds and XMPP real-time feeds can usefully coexist. We proved this at the old PubSub.com years ago.

    We used Jabber/XMPP + XEP-0060 PubSub and “Atom over XMPP” for real-time updates but maintained “logs” of all traffic on each topic as an Atom feed. When clients attached to the system after having been off-line for some time (and thus missing some messages), they would pull the missing messages from their Atom feeds before beginning to accept new real-time Jabber messages. Thus, we used Atom both as a real-time feed (Atom over XMPP) and as a stored feed (normal Atom in a file). One of the great advantages of doing this, of course, was that we were able to provide support to either push or pull clients while also providing a natural feed history mechanism for the XMPP push clients.

    The Atom feed syntax has been designed to be useful in either push or pull modes. Atom + Jabber really provides all that is needed here.

    bob wyman

  66. Adam Simon

    Interesting idea, but despite the fact that a large portion of the tech community interfaces with Twitter via other means, the majority of Twitter users still tweet using the website. If my experience is any indication, there was a massive upswell of second-tier users in the past few months, ie, not early adopters, but average users.

    If this had happened last year, I would give it fairly good odds of succeeding, but with so many average users who don’t read tech blogs, have never even heard of Twhirl (or had to deal with an AIR installer…), and don’t freak out when the servers go down, it would just wind up creating class distinction and fragmenting the user base. The reason there’s so much complaining and not a lot of movement is because the social ties into Twitter’s network have grown pretty deep, and you can’t just transplant that to another service, no matter how much you whine. Most people would be stuck using both, which gets annoying fast.

    And I really question the effectiveness of tracking @ replies as mentioned in the article. It’s not nearly real time enough or as reliable. (Yes, reliable. Twitter works perfectly when it’s up, which is still most of the time.)

  67. Ernest Nova

    @28 DNS Root server are for practical purposes “decentralized” in that that they are highly replicated. Check Anycasting, and the large number of root servers.

  68. Aaronontheweb

    …. Comparing opening-out Twitter to the collapse of the Berlin wall? Are you fucking serious?

  69. Harish

    Twitter seriously dropped the ball on this one… if it hadn’t been for their frequent outages, I doubt there’d be such a strong movement for decentralization of the whole platform (because users wouldn’t be as worried about one entity having control of their data/conversations if it was an open and reliable one). They need to recover, and quickly, in order to take advantage of the microblogging phenomenon they helped create.

  70. Moritz Schroeder

    Great discussion, it is time that it starts. My list of what makes twitter great:

    - simple, scalable and compatibel with the TXT world
    - a live status/update/mood text ticker for the whole world
    - crrently potentially accessible by nearly half the whole world !
    - a great place to feed data
    - an incredible launchpad for apps
    - great fun and fascinating

    I think “distributed” would be better then “decentralised”. The immediacy is the kicker. This one global Internet ticker. The potential reach is phenomenal (3.3 billion phone contracts worldwide at the end of 2007, I think).
    Make it public, pay the twitter guys big money and turn it into something like a public service / foundation. Place it along the other communication / publication standards. Let it run in a distributed way across the backbones and data centers. Google & friends might finance all that, because they will earn a lot of money on the back of this service. And yes, if for some reason it shouldn’t happen, make a simplified jaikiu the new twitter.

  71. Bryan

    I’m not saying that Twitter ever had a business model, but it’s crazy how quick people can call for the open sourcification of something popular. If they fixed their downtime problems, would it still be an issue?

    So much for Twitter being a case study on the scalability of Ruby…

    How about we cry for openness, decentralization, API’s, and standards for salesforce.com? ;-) That would save me a fortune!

  72. kael

    The GetPingd idea looks very interesting.

    There exists a WordPress ATOM-over-XMPP Pubsub plugin (via), but we still need Pubsub ATOM aggregators, although some Jabber clients will probably integrate social streams soon.

    There’s also a XMPP PEP aggregator which looks similar to Twitter (via).

  73. Les Madras

    Why is twitter not a feature on Gchat?

  74. Bob Ngu

    As stated in the start of the post, I agree with this 100%
    “Generally services like Twitter, once they reach a critical mass of users, can’t really be stopped because the network effect kicks in as a massive barrier to entry.”

    And there are several Twitter clones to choose from now, the problem of moving off Twitter is not technology or availability of alternatives but rather Twitter critical mass of users.

  75. Q dub

    It makes me think, who profited from the many web standards that we benefit from today? Did anyone make a killing on DNS? HTML?

  76. Zach Weisman

    It seems that microblogging is inherently too simplistic to invoke the type of competition that blogging has. What value could a competitor add to a one liner broadcast? Speed? Reliability? (Both of which should be flawless anyway) Anything else?

  77. evgen

    To paraphrase an oft-used chestnut: “…and then someone suggests decentralization, now you have two problems.”

    There are a few reasons to decentralized a service or system, but the cost for this is increased complexity, fragility in the short-term as best practices for vague standards are developed, loss of security, and a sharp drop in system latency. Centralized systems can be made reliable and fault-tolerant with good planning and execution (e.g. Google searches) and they can almost always be done in a far more cost-effective and feature-rich manner than decentralized systems.

    I am sure that the spammers would _love_ a decentralized twitter. Naive designs (like all those proposed so far) provides all of the security nightmares of email with very little upside. This does not even start to address cross-network issues like the popular SMS features of twitter, or the additional functionality we are all enjoying through the availability of APIs and other services offered by a centralized twitter.

  78. Bob Ngu

    Another thing, I am sure everyone has experienced DNS outages as well (infrequent as they might be), granted that it isn’t an all-out DNS failure but still as a user, it sure seems that way to me, but then again how long did it take for DNS infrastructure to come together? A long time I am sure.

    The fact is that people have come to expect Twitter to function as a utility like electricity and water and that’s just ridiculous to expect from such a young startup. Due to the unrealistic expectations, every Twitter outage is overblown and over analyzed. And Ruby on Rails have become collateral damage along the way.

  79. Marc Canter

    @bryan - that’s called SugarCRM - no?

  80. Shannon Whitley

    I created OpenMB several months ago. It’s an open source microblogging platform. A lot of it is based on WordPress standards. You can host your own microblog, allow others to join, and communicate with other hosts. There’s still some work to do, but the hard parts are already complete. Anyone want to join me?

  81. Tony

    @James
    “patent for the use of @xyz or the 140 character payload”

    I’m pretty sure @xyz predates twitter - I’ve been seeing it around longer that twitter has existed. Fail on “Prior art”

    140 character payload? Again, probably fail on prior art, but even if not, well, I could do 139 characters.

  82. Chris Saad

    @Shannon Whitley sounds cool - does it support http://www.getpingd.com.

    Drop me a line

  83. mathew

    RSS is the wrong answer.

    Atompub over XMPP.

    http://tools.ietf.org/html/dra.....-notify-06

    We already have a draft standard based on actual RFCs, let’s implement it rather than inventing another one based on the obsolete RSS mess-o-standards.

  84. Ziad

    Congratulations everyone.

    Once you’ve decentralized twitter you’ll have invented email.

  85. AW

    @84: email with a really pretty front-end, though. :)

  86. Alex Schleber

    Twitter’s deeper uses come from mining it’s data (now so-so possible with Tweetscan but should be native, including tag/filter, search, and sort), from direct alerts on your highest significance keywords via the “track” feature (SMS), and the general SMS integration itself.

    There is no reason why Twitter couldn’t scale, all they need is a serious cloud to run it on. Enter Google…

    I expect them to buy it within 6 months, after determining that their own Jaiku service is too late to capture the mindshare lead (it’s not even out of invite-only beta). Same as with YouTube/Google Video.

    Interestingly, Twitter should be making money all along from a share of the SMS fees, but for some reason they don’t. Not sure why (seems to be philosophical or something)…

    If Google doesn’t buy them soon, someone else will. MSFT just kept $45B more in its pocket and is likely ready to spend like a drunken sailor…

  87. AW

    @86:

    Why would Google buy Twitter? What is the value? They already have Jaiku, so clearly it’s not technology they’d be fishing for.

    Users? Maybe. But Google has plenty of those already, lots of overlap with Twitter.

  88. Khris

    Wow, Chris!

    Great thought storm you set off. Always a pleasure to hear your
    thinking on the state and direction of the Internet :)

    Be Well,

    Khris

  89. Panamajack

    Interesting hearing early adopters complain about this just as Twitter is truly taking off.

    Question #1: what percentage of blogs are de-centrally served ? My gut feeling is the vast majority of blogs are hosted on blog-service websites (although many/most popular blogs like TC aren’t). That’s simply not an issue for the vast majority of bloggers who don’t use RSS.

    Centralized services like Twitter are fine for the masses, especially once a bigboy buys them out and and powers it from their server farms that are liberally sprinkled around the world.

    Question #2: What’s google doing with Jaiku ? Why didn’t they buy out Twitter?

  90. AW

    @89:

    I’m no expert, but for QUESTION #2, I can tell you, they are apparently ruining it.

    A million and one blog posts complain about the service being down every other request, no official posts on the blog for months at a time… (sound like someone you know?!)

    Seems like Google bought Jaiku for the technology and pretty much gutted the company for that reason.

    I don’t see “Join Google Jaiku today!” ads anywhere, that’s for damn sure.

    Man, I love being one of those guys who lurks 3 or 4 days after a post has fallen off the front page.

    Nobody to argue with me.

  91. Peter