Thank you!

The short answer may simply be, because scaling a web site after the launch is not an easy problem to solve. Especially if the architecture of the original design is antithetical to making the required changes as the site is running. The ideal scalability methods are ones which require deep integration with the underlying architecture of the web application. Ideally, the core API should be multi-threaded to have even a hope of performing an efficient dynamic distribution of requests over multiple servers under increasing load. If this was a web 1.0 site serving pages it would be easier, but twitter does lots of aggregation of twits and routing to subscribers…tons of db work through the app server layer…if that is not done efficiently, scale will not happen. If you don’t have smart multi-threaded code, it is going to be a B-och and a half to add in without major hiccups or out right downtime. The architect would have to build a parallel scalable system while the old system runs but do it fast enough to replace the old system before the increasing load kills it completely. That is the recipe for stress on a level we can only imagine. I’ve done my share of over nighters at a dot com and I’d imagine the stress of this type of re-architecting to be almost suicide inducing…so what do you do if you aren’t going to off yourself because of the stress of the job??? You jump ship.

That is my theory given the limited knowledge we have of the company internals and some personal experience in web application architecture.

my two bits

It seems pretty plain at this point that Arrington’s Blaine story has been illustrated to be pure personal vendetta. In other words, does this story make Mighdoll an amateur?

I couldn’t imagine Seesmic (good service btw) generating as much video messages as Twitter, it could bring down the internet most likely when you add that to P2P.

I’ve heard that too, that when you send small numbers of messages the providers charge you, but once you reach a certain tipping point you can negotiate with the providers to have them start paying you – since twitter generates so much revenue for the cell companies via text messaging charges (or unlimited plan charges) from all the twitter addicts.

anybody know any more details on this?

We did a combination of database optimizations, replication (both near real-time DB and scheduled SAN-based snapshots), clustering, caching, and a series of hardware and software upgrades. My message to the Twitter guys — “every” successful startup goes through this phase

Jason

On another note: going through a huge growth spurt, having downtime and changing management is just business as usual in the valley. Why is this a story? Anyone remember ebay’s growth issues? Didn’t kill them.

The second point is that when you do a Web 2.0 project like this, always, always, always, have a game plan. It seems so much these days that the game plan is “Let it get popular then monetize” and Twitter seems like a great example of that, except they forgot about the second part.

P.S.: I’ve never heard many positive things about Joyent as a service (doubly true when they unveiled it as a FaceBook partner), so good for Twitter for getting away from them.

Not sure who you are or what your affiliation is, but making a random statement about not supporting Twitter through their growth is insane. There are two sides to every story, but it was to everyone’s benefit to see Twitter succeed.

Out of respect for both companies, I will not disclose any more details here.