We’re hearing this from multiple sources: After nearly two years of high profile scaling problems, Twitter is planning to abandon Ruby on Rails as their web framework and start from scratch with PHP or Java (another solution is to stick with the Ruby language and move away from the Rails framework).
Former Chief Architect Blaine Cook famously said scaling Rails was “easy” in April 2007 (see image to right), but problems persisted after Cook claimed to have conquered the problem. The service most recently had a three day outage affecting their largest users.
Other massive Rails sites include Scribd, Hulu, and the popular Facebook app Friends for Sale. CrunchBase, our tech company database, is also built on Rails.
Switching off Rails may not solve all of Twitter’s problems. They have nearly two years of infrastructure built up and would face many more growing pains if they switched frameworks or rolled their own. As Twitter considers moving away from Rails, some companies are doing the opposite: last year, Yellowpages.com scrapped Java for Rails, and is now second on the unofficial Rails 100 wiki.
Rails has always bred controversy. Developers have argued that it is fundamentally flawed and unscalable; others have argued back saying the opposite (see here, here, and here). Earlier this year, one of the core community members and creator of the popular Rails web server Mongrel abandoned rails and trashed the community.
Update: Regarding Evan Williams’ statement here, all I can say is that multiple sources claim that Twitter is telling people they are planning on moving away from Ruby on Rails. This is not the first time a company has denied something that has turned out to be 100% true.
let’s call that a derailment
PHP or bust
They should go to PHP. Lots of experience to tap into to get a scalable solution.
Rails sucks. I never liked it, and never recommended it.
rails is beautiful. don’t hate.
Windows can be beautiful too, but that doesn’t mean it actually works
I disagree with that statement on so many levels. No Windows isn’t ideal, and while I’m a linux user I still recognize Windows as being a rock in terms of being a non-mac gateway into proprietary software.
That said, Ruby and Rails are not only beautiful syntactically, but are enormously functional, and as a long-time PHP programmer I recognize that I can develop web apps 2-5 times more quickly on rails, but don’t sacrifice any of the flexibility that comes with building an app without a framework. Not only that, but because Ruby organizes everything in classes, and Rails organizes everything into the MVC design pattern, my applications are boundlessly more accessible and easy to maintain, especially late in the development cycle.
I presume this is all back-end work? No impact on UI or functionality, right?
I will go one further..
Because of the similarity of Ruby on Rails and Cake PHP in the ORM layer and etc its more likely to be a move to PHP using the Cake MVC web framework.
The other choice is not Enterprise Java but say Plain Old Java Objects using the Groovy Graiils framework that works with Java.
Sometimes huge growing pains force change sooner than planned. Hope the transition goes smoothly. I’m just now getting into Twitter and although not an addict, yet, really enjoy the connecting that goes with the site.
Cindy
I thought database abstraction and transparency, almost as if you could avoid thinking about databases in traditional terms, was a big new bonus in Rails, why would you need to optimize the database?
Interesting post from Gigaom on rails getting some enterprise-level support: http://gigaom.c...-level-support/
@5 – Front-end’s all HTML, CSS and JavaScript. You could duplicate the UI and functionality in FORTRAN, if you really wanted to.
Oh no, this is going to cause absolute mayhem. Michael, do you realise what you’ve just unleashed? This is more potent than a DHH post on GENERATING PROFITS or four day works weeks.
I’m betting they won’t side with a PHP or Java framework, but that’s not going to stop the upcoming rage virus epidemic.
PHP?
Django a much better choice here.
they should try python & Django… their competitor Pownce uses it!
lol.. plus they’d be able to run it on Google App Engine…
Perl all the way!!
I’m not very surprised, things happen and they don’t always have an explication. I just like Twitter and nothing more.
They should just go with IIS7 and sql server 2008. I know it is the “in thing” to not pay for your webserver os and db engine, but why not build on something that is used in the enterprise? You can cache to your heart’s desire in asp.net, and there are hundreds of thousands of people learning C# in college right now. Seems like a no brainer to me.
They should go with C. This is a messaging protocol, not a Web app people. Would AOL Instant messenger implement their app in Rails? That would be preposterous. This really says very little about the Rails framework, if you ask me.
They could rewrite twitter in like a day.
PHP FTW. All I have to say.
You know. I know this probably won’t set well with everyone, but besides PHP and Java, I would recommend Twitter really look at using Asp.Net MVC framework.
It isn’t a secret that http://www.plentyoffish.com runs his whole entire operation on 2 front-end servers and one DB server, and he’s doing what like 30,000,000 page views an hour or a day or something ridiculous like that.
Compiled code beats scripting in speeds any day of the week.
You know what, have ‘em use IronRuby with the Asp.Net MVC framework, that way they can write ruby and still get performance.
My $.02.
-TF
Goodbye Rails {seesmic_video:{”url_thumbnail”:{”value”:”http://t.seesmic.com/thumbnail/HL9GlL3nKN_th1.jpg”}”title”:{”value”:”Goodbye Rails ”}”videoUri”:{”value”:”http://www.seesmic.com/video/UyItgSFEjU”}}}
Maybe it’s time for Twitter to leverage Amazon’s E2C which has no uptime agreement. At least Twitter would have someone new to blame for endless, unscheduled outages.
That’s May Fool
@16 – PHP, ROR, Python, etc. are used in the enterprise, unless you don’t consider Amazon.com, Google, and Yahoo! to be “enterprisey” enough.
There was a no-brainer in your post, just not the one you thought.
How do we know the recent three day service outage only affected large users? What about the little people?
Twitter is easy to implement using an ActiveMQ (Java) cluster for the subscriptions, PHP for the front end/API, and another Java farm for message delivery. You can put all your eggs in one basket. They are probably going down because they are pushing around too much data on the backend. Subscriptions should just be pushing around the index for messages. They should have a memcached farm as well for messages. Oh Well.
What about jruby on glassfish?
@20: “Compiled code beats scripting in speeds any day of the week. ”
That’s why you use an opcode cache.
Depends on their strategy here, but there are quite a few other ways they can go. The RR monolithic strategy can be tough to cope with when you need to do things like asynchronous operations. Other platforms are just better equipped to do those sorts of things, without your environment ending up as a hodge-podge of technologies.
I’m *sure* they’re having conversations about the front-end vs. back-end vs. middle-tier vs. db. Or, I hope they are anyway.
I see lots of people suggesting a PHP solution, but I think that’s a limiting path for 2 reasons.
First, Twitter is increasingly global, and (I believe) PHP still lacks Unicode support. Unicode support would make life much easier for them.
Second, for a site that gets as much traffic as Twitter, they’d benefit immensely from a language that handles multi-threading well. Sorry, that’s just not what PHP does.
Go Java, but be smart about it
earle: PHP is a template language meaning that it’s a language that works within a template system (in this case, HTML). That makes building smaller templates very easy and fast. That’s one of the reasons that you really want to divvy up business logic from display logic. Django is a template system within a language, which may seem like a minor quibble, but it requires data preconditioning in order to use it, meaning that templating is something that has to be kept different and requires a different syntax.
It really depends on how they’ve structured things for their site. Both choices are valid, but it’s a bad idea to dismiss either of them out of hand.
@Matt Baron: agreed this is a messaging app. Sticking a db into the flow and distribution of messages is a non-starter for anything with real aspirations to scale.
Just about any language can work fine … it’s the flow of msgs and what you do with them that matters.
Anyhow [osted some more on this yesterday.
@26 Your technology doesn’t support compilation so your defense in using it to scale is use an adon? I don’t get the argument. I / me / developers everywhere shouldn’t have to fight with a technology that doesn’t scale and bolt something else on to solve a problem. I should be able to focus on my code, write it, and move on, not worrying about having to one day at some point install some other third party tool to make it go faster.
Ridiculous argument. Wake up.
PHP and Cake FTW!
@26 – that thought scared me – are they really *moving* around multiple copies of tweets? I’ve seen people seemingly in awe of being able to “deliver” that many tweets to people (the twitter spewage charts a few days ago). There should only be one copy of a tweet in a database table, not dupes. I honestly don’t think they’d have been that dumb.
Also, not really sure why they’re having the scaling problems. I’m not a huge RoR fan, but I don’t really care to bash on it that much. I’ve been around to see crap code on every major platform out there – I know people who can make *any* system *not* scale. However, if they’re trying to keep everything within the confines of the RoR framework, that may be the problem. Frameworks are great, but there are times when you need to bypass the confines of the system, usually for performance reasons. The trick is knowing when and how to do it, documenting it, training people on why you deviated, etc.
I suspect if they just rebuild with the same architecture and mindset, just a new language, they’ll have the same problems. Likewise, if they rebuilt from the ground up in RoR (or just Ruby) they would likely be able to make it scale much better with all the lessons learned. Of course, given that many of the original brains are now gone, perhaps that wouldn’t be as useful, because those experiences are now gone too… ???
ha! gosling is probably wetting his pants laughing right now (remember last year when he said ruby couldn’t scale for the enterprise? well, here you go…)
Just because a site has scalability issues, it does mean Rails is the culprit. Most of the time when Sites are not scaling it is database which is the culprit, mainly for not being totally optimized.
Twitter is growing faster than they expected and can’t really blame someone here because the service is free and beggar’s(users) can’t choose.
PHP only limits people of lack of creativity. They will probably find the correct language for the application, but I would not say that PHP is not the right language for this.
They are obviously going migrate to Termite/Gambit and replace Starling with ejabberd, I mean, what else could possible handle the load?
Come on people, as if the web front end has anything to do with twitter’s particular technology challenges.
A default install of PHP isn’t the best thing in the world, but once you tweak some settings and add a few extensions like eAccelerator, it’s real fast. It could power Twitter easy.
Clicky’s tracking server handles almost 20 million hits per day to PHP, and it’s far from top of the line. I realize Twitter would have quite a bit more traffic than that, but the point is that 20M/day is still a LOT of traffic, and PHP does it without a sweat.
@30 – Eric:
Multithreading and application scaling – at the server farm level – have pretty much nothing to do with each other. One might almost argue *against* that approach, as the ’shared nothing’ approach of PHP has been proved as scalable as any other approach out there. IIRC, friendster was originally in Java and *just* *couldn’t* *scale*. No matter how much people tried to second guess them, they eventually moved over to PHP and they now scale. Wordpress seems to scale alright as well.
I’m not saying Java *can’t* do it, but there seem to be enough examples of high profile PHP scaling sites to say that it can’t do it, or that the problem at hand inherently requires a threading model.
Loren Feldman is now an expert on web technologies and a professional douche bag what a skilled guy.
Can we get some BRAINF_CK up in here?
http://esolangs.../wiki/Brainfuck
All the Rails people are going to cry and say it’s not Rails fault, it’s the developers fault.
I wonder if internal politics is partially driving the move, made easy by their past technical issues. I’ve seen it before, and sometimes the new solution doesn’t actually address the root cause. I hope this decision is based on rational thought instead of finding a expedient scapegoat.
Also, given the background of Steve the new guy (ex-google) i suspect Python or Java would be on the cards before PHP. Likely though, they’d investigate JRuby on Rails before anything else (if it hasn’t been done already). Last I read it was able to run 100% of the Rails stack without issue, and was faster than the standard C Ruby in many instances. http://antonioc...-ruby-shootout/ shows it faster than the Ruby 1.8.6, though slower than Ruby 1.9.
There are times when I don’t think wasting over 8 years of my life building enterprise financial systems where messaging was a routine part of the conversation was a waste of time. This is one of them, amateurs! Use a real messaging platform (Microsoft or Java (ActiveMQ). No wonder the tech team was fired! PHP is good for the web facing stuff and API calls. Twitter the web app or API calls should be reading data from a DB, PHP is more than fine and horizontally scales, so does the DB. All the fancy messaging magic is what is killing them and that should be able to queue load and scale horizontally as well to meet QoS targets.
We had the same problem that Twitter is having. RoR just doesn’t scale and it’s not ready yet.
Large-scale social networking sites sites, such as Facebook, have chosen PHP because of its execution model and because it’s a specialised scripting language designed for the Web.
Nothing beats Linux, Apache and mod_php. That’s why more than 20 million sites use it. That’s why System Administrators install it and Software Engineers use it.
Twitter’s lead developer and architect left the company. It’s time to scale, it’s time to use PHP.
PHP and CodeIgniter FTW.
http://codeigniter.com
I’m seriously interested in seeing if you will have enough butts and balls to post a retraction like a man