Amateur Hour Over At Twitter?

Trackbacks/Pings (Trackback URL)

1. PressReleasePoint
   

   April 23rd, 2008 at 8:39 am

   hoping to see better uptime for twitter with new hires.

2. Peter Urban
   

   April 23rd, 2008 at 8:40 am

   It is a bit baffling that it takes them so long to establish an infrastructure that scales properly given that they know of their success for a while now and that sufficient funding is available. However, you never know how flaky the idea was conceived in the first place and how much the system suffers from it’s experimental stage heritage.

3. Robert Scoble
   

   April 23rd, 2008 at 8:40 am

   I hope Twitter gets on top of its reliability problems. Too many things are counting on it now. Wait until the next earthquake when traffic will spike 1000x above where it is right now.

4. Will Johnston
   

   April 23rd, 2008 at 8:41 am

   Really hoping this fixes twitter’s issues, as you said, Mike, we’ve become dependent.

5. dumbquestion
   

   April 23rd, 2008 at 8:41 am

   Got a dumb question.
   How does Twitter make money?

6. Robert Scoble
   

   April 23rd, 2008 at 8:41 am

   It should be said, thanks to Blaine and team for building something we’re all addicted to. I do tap my hat to you and hope you do something interesting in your career path again. We’ll be watching.

7. johng
   

   April 23rd, 2008 at 8:46 am

   DumbQ - everybody asks that, and nobody knows. No doubt their biz model will become clear before much longer. (though I remember saying that about Napster a few years back)

8. Ken Camp
   

   April 23rd, 2008 at 8:46 am

   And given their time in operation, with no money making in sight, why exactly do we expect Twitter to be able to scale to support everything we want?

   Is Twitter anything more than a grand experiment in chitchat? I think not.

9. Patrick
   

   April 23rd, 2008 at 8:46 am

   Wow, someone certainly has an axe to grind.

10. Jay Virdy
    

    April 23rd, 2008 at 8:46 am

    I echo Scoble’s sentiment re: Blaine and team. Many of us have become addicted to Twitter thanks to their hard work.

11. Steven Van Tilburg
    

    April 23rd, 2008 at 8:46 am

    I hope that with the upscaling, they retain the functionality that Twitter already has, much less adding new function. It would be a shame to see things taken away because they can’t handle the load.

12. Jason
    

    April 23rd, 2008 at 8:46 am

    @dumbquestion - Twitter doesnt make money. Except for Twitter Japan which has ads. Which probably wont earn them much seeing as the majority of twitter activity happens over their API.

    I think Twitter should scheduled downtimes so you addicted folks will stop spamming stuff no one cares about.

13. Prokofy Neva
    

    April 23rd, 2008 at 8:48 am

    Is Ruby on Rails the problem and not the guy?

    http://antoniocangiano.com/200.....criticism/

14. Baker
    

    April 23rd, 2008 at 8:49 am

    Have to agree with Patrick - you guys sound venomous.

15. Tara Hunt
    

    April 23rd, 2008 at 8:52 am

    Personally, I think Blaine did a terrific job scaling something they never expected would grow like it did. He also built an amazing platform that millions love, hundreds have built apps off of and that works 95% of the time. It was Blaine’s idea to make the messages available via IM and by other means through the API. I don’t know about you, but I certainly couldn’t handle following 800+ people through my SMS.

    I think smudging Blaine’s skills is an unfortunate misrepresentation of the hard work, long hours and multiple weekends he spent trying to keep us all happy.

    FWIW…he left Twitter last week. This last weekend? Let me tell you, it would have been fixed in hours if he and @meangrape had been around (Jay was out of town).

16. Steve
    

    April 23rd, 2008 at 8:53 am

    So what’s up? Burying the twitter disaster thread?
    Anyway it could be the rails curse…

17. Kevin Clark
    

    April 23rd, 2008 at 8:54 am

    So, what you’re saying is:

    “I have no new information. I do have a scapegoat, though I have no concept of the technology I’m railing against. So let’s jump on the bandwagon and complain about someone, because it’s convenient.”

    Blaine is an excellent engineer who’s put out some really innovative stuff, and _you_ are the reason why many people still don’t consider bloggers journalists. You’re a gossip rag, nothing more.

18. Marco
    

    April 23rd, 2008 at 8:55 am

    You need different people at different stages of the product maturity. The article is a bit harsh about Blaine. Don’t forget he helped to get twitter all the way to this level. And I’m sure that they had to be rather innovative to keep up with the quick growth.

    It might take some time to see the results of the new people. I’m sure it’s not just “Buy X more machines…”

19. Pierre Carion
    

    April 23rd, 2008 at 8:56 am

    “Apparently Mike Arrington’s deleting comments from the post about my departure. Classy.” really ? http://twitter.com/blaine/statuses/795211971

20. Michael Arrington
    

    April 23rd, 2008 at 8:57 am

    Blaine in April 2007, on the ease of scaling twitter:

    

21. Eran Hammer-Lahav
    

    April 23rd, 2008 at 8:58 am

    Amateur Hour Over At TechCrunch? Yes!

    Blaine is the main reasons Twitter’s scaling problems were actually contained and under control for so long. To try and link his comments on Rails scaling with what is happening at Twitter now is a true demonstration of ignorance and bad reporting. The Twitter engineering team, which up until very recently was just 3 (!) people, deserves a lot of credit for being able to accomplish so much with so little. Pointing fingers at the developers demonstrates your lack of understanding and the low quality of your reporting. And if only you got some of your other facts right…

22. Yup
    

    April 23rd, 2008 at 8:58 am

    I can vouch for that deleting comment.
    Comments against Mike et al. are mysteriously disappearing into the ether.

23. Jeremy McAnally
    

    April 23rd, 2008 at 8:58 am

    Wow, this is the most unnecessarily vitriolic articles I’ve ever seen. Twitter and Blaine took Rails to places it hadn’t been before in terms of scale and traffic (just as other sites did with PHP, Java, ColdFusion, and so on), and so acting as if he’s some unskilled hippie that didn’t know what he was doing just makes you look like an idiot. Sure, they had some bumps along the way, but no one else had even gotten on the same road as them to encounter the bumps. Being forced to figure out things along the way because no one has done them before doesn’t make you an “amateur.”

    Even further, what the heck qualifies you to criticize someone’s ability to scale a website or doing anything remotely technical for that matter? You’re out of your league. Please stick to talking about business or whining about someone not giving you ad money.

24. westernworld
    

    April 23rd, 2008 at 8:59 am

    i met all kinds of new people due the absolute randomness of direct message distribution over the past few days, at least here in europe.
    so it did help the social aspect, but networking really needs work, lots of it.
    look what happened to pownce …people just left.
    everyone and their brother went on twitter, it’s the default now.

25. Michael Arrington
    

    April 23rd, 2008 at 8:59 am

    Eran - right. he’s awesome. that’s why he left and twitter hired all these other guys to scale the application.

26. Bob Lozano
    

    April 23rd, 2008 at 9:00 am

    @scoble sure I hope he lands on his feet, lessons learned and all that.

    But I think the bigger lesson is for the industry as a whole - the days of the throwing out a POC and figuring out scale issues later are probably long gone.

    I posted on this yesterday.

27. Kevin Clark
    

    April 23rd, 2008 at 9:00 am

    @Arrington: I don’t know. Aren’t you supposed to be reporting here? Shouldn’t you know if you’re going to post about it?

28. jimmy dean
    

    April 23rd, 2008 at 9:02 am

    Love this:

    This last weekend? Let me tell you, it would have been fixed in
    hours if he and @meangrape had been around (Jay was out of town).

    HOURS?!

    I’m not sure if that’s a compliment or criticism.

29. Mel Kirk
    

    April 23rd, 2008 at 9:02 am

    I have to second Tara’s comments. We were lucky enough to have Blaine speak at FOWA Miami back in February and he’s by far one of the nicest and most intelligent guys that I’ve had the pleasure of meeting.

    Blaine and the team have done an amazing job scaling Twitter to it’s present form and let’s face it, whilst we might be quick to criticise any down time, can you imagine life without Twitter? No, thought not…

    Good luck Blaine - hope rumours of you moving to the UK are true

30. Adam J. Schmidt
    

    April 23rd, 2008 at 9:03 am

    The most important question is how they’re planning on making money. Doesn’t matter how much we enjoy the product/service if it’s unsustainable. Maybe they just get bought out buy someone else who wants access to the users?

31. Fred Grott
    

    April 23rd, 2008 at 9:05 am

    I know form scaling several MVC frameworks including Rails that its NOT DAMN EASY IN RAILS!

32. John C.
    

    April 23rd, 2008 at 9:05 am

    Michael, from the SAI:

    “I am moving to the UK so that my partner can pursue her career. We’re Canadian and her visa makes it impossible for her to work in the US,”

    How hard is that to understand?

33. Tim G
    

    April 23rd, 2008 at 9:06 am

    Geez Arrington this post seems kind of personal. What’d Blaine hit on your boyfriend?

34. David Pollak
    

    April 23rd, 2008 at 9:07 am

    Blaine is a very smart guy, a fine engineer, and a professional. Twitter would not be many of the good things it is today without Blaine’s excellent work and passion. I’m toasting Blaine’s awesomeness and I know he’ll land someplace where he’ll be able to make another significant impact.

35. Mike Stenhouse
    

    April 23rd, 2008 at 9:07 am

    That’s a bit harsh. You’ve taken co-incidence and got causality without any evidence that I can see. Blaine’s leaving? Best of luck to him. He’s a super-sharp guy; I’m sure he’ll have companies beating a path to his door. As a service, Twitter is growing at a phenomenal rate with finite human resources. Something has to give - them’s the breaks. Twitter will be fine. Blaine will be fine. Techcrunch will be fine. Let’s all hug.

36. Charles
    

    April 23rd, 2008 at 9:08 am

    Twitter down time is good for the world economy; it’s the only time any work gets done. Don’t knock Twitter down time. It’s not like it’s an essential service anyway. I feel Arrington is being a little harsh and graceless here; Scoble’s comments are more appropriate.

37. Dave Litsky
    

    April 23rd, 2008 at 9:09 am

    Time for an acquisition. Your product is hot right now and who knows if the scaling issues will ever be fixed.

38. Charlie Wood
    

    April 23rd, 2008 at 9:09 am

    Wow, what’s the deal with the venomous personal attack? Leave the character assassination to Valleywag, Arrington.

    -Charlie

39. jimmy dean
    

    April 23rd, 2008 at 9:09 am

    C’mon folks, this is all in “fun.”

    Vitriolic? Sure. We’re human. Who ever said we’re supposed to be nice.

    If you want to put yourself out in the public, and to put it mildly, be overconfident (”It’s easy, really.”), then you better have a thick skin.

    Welcome to Hollywood.

40. Sabil Ardie
    

    April 23rd, 2008 at 9:09 am

    Hmmm… I wonder if the “Cache the hell of everything” makes it harder sometimes to delete posts.

41. Michael Arrington
    

    April 23rd, 2008 at 9:09 am

    Mike - if you agree with me that twitter has had serious uptime issues, it’s kind of hard not to follow the dotted lines to the chief architect.

42. Michael Arrington
    

    April 23rd, 2008 at 9:10 am

    Fred - no, see comment 20. scaling rails is quite easy.

43. Doug D
    

    April 23rd, 2008 at 9:11 am

    How is their infrastructure hosted these days? Are they still gambling on bleeding edge technology or have they moved to a more tried and true dedicated server solution?

    I’m not doubting that they have general architecture issues that would persist regardless of virtual or physical infrastructure, but I think they had originally placed too much faith in virtualization hype and attention to architecture suffered as a result (this is pure speculation, take with a grain of salt).

44. Eran Hammer-Lahav
    

    April 23rd, 2008 at 9:12 am

    @Arrington - the problem is, you have no clue about this story. You read it elsewhere, copied it, and made it into a personal attack. Your post is the same as blaming the TechCrunch graphic designer for your poor reporting.

45. Andrew
    

    April 23rd, 2008 at 9:12 am

    The tone of this post is why I’ve grown to dislike Techcrunch over the past months, and why it will never grow beyond a certain point.

    This post is bitchier than most found on Vallywag, and contains more opinion that news. Techcrunch seems to have replaced crunchnotes lately for Michael’s bitching and sideswiping.

    For a site that’s so hard on traditional media, there’s certainly a lot of lessons that could be learned about what constitutes journalism.

46. Pierre Carion
    

    April 23rd, 2008 at 9:12 am

    TC has also technical problems : I read in the comments, 10 mn ago, a reference to Blaine’s explanations about his leaving “I am moving to the UK so that my partner can pursue her career…” as in 32. , but this comment disappeared.

    Will the person in charge of the service at TC be fired too then?

47. Mark Madner
    

    April 23rd, 2008 at 9:13 am

    Software architects are second guessed as much as NFL coaches.

    I want to add my name to the list of people thanking Blaine. Getting a site like twitter to scale to the volume it now has on a shoestring budget is non trivial. I would feel lucky to work with Blaine in the future.

    Michael Arrington: You’re successful enough of a blogger to not have to resort to this kind of public humiliation of folks to gain traffic. If you have a beef with Twitter’s architecture, I would prefer you leave names out of it. You don’t know what went on behind the scenes, so you can’t judge who the problems should be attributed to. Let’s all make the internet a more hospitable place.

48. Raskin
    

    April 23rd, 2008 at 9:13 am

    Am i the only one who COULD live without twitter?

    Like, happily.

    And…easily.

49. Jeremy McAnally
    

    April 23rd, 2008 at 9:13 am

    @jimmy, Michael: I’d be fine with the criticism if it wasn’t coming form an uninformed dunce who knows about as much about scaling a high-traffic website as writing informative journalism. Which, by the way, isn’t much.

50. Coda Hale
    

    April 23rd, 2008 at 9:14 am

    Michael, I don’t think you know what you’re talking about, and I certainly I don’t think you’re familiar with the ins and outs of Twitter’s technical details. Personally, I’ve had all three of Twitter’s engineers (at the time) explain their architecture to me over dinner at Naan N’ Chutney, and I can tell you that Blaine Cook has *not* been their problem (nor has Ruby or Rails or any of the other technical issues favored by armchair analysts). This baseless scapegoating is unprofessional and your readers deserve better, even if you are unable to give them that.

51. Arun Vijayan
    

    April 23rd, 2008 at 9:15 am

    Crazy! Dont kill him for his slides. Those are like bible for some of us. Any engineer will think multiple times for taking a responsibility like this. And the kind of stuff he had done with limited infra, time is amazing. I think mike, you gotta learn to respect talent.

52. Scabr
    

    April 23rd, 2008 at 9:15 am

    Maybe new twitter team will remove twitter bugs

53. Michael Arrington
    

    April 23rd, 2008 at 9:16 am

    Jeremy, I’ve got two rails developers sitting ten feet away from me who seem to have their finger on the pulse of the rails community. And trust me, I toned the post down significantly. but you’re right, i don’t know much about journalism and even less about rails. but i do know that a for profit company that can’t get their shit together two years after launch deserves some criticism.

54. yah
    

    April 23rd, 2008 at 9:17 am

    Wow, TC has got over its love of Facebook. Can anyone say TC loves Twitter ?

    How many stories in a day do we need about Twitter ?

    Lets all sing , Techcrunch and Twitter sitting in a tree , K—i—S–S I–N–G.

    Dude enough of the Twitter love ,

55. Scott
    

    April 23rd, 2008 at 9:18 am

    only us geeks have ever heard of twitter anyway

56. Michael Arrington
    

    April 23rd, 2008 at 9:18 am

    coda - ok, what exactly in the post do you disagree with? the fact is cook said all their problems were solved a year ago. they’ weren’t. he looks a little dumb now. doesn’t he?

57. Mike Stenhouse
    

    April 23rd, 2008 at 9:19 am

    @Michael

    You _could_ follow that dotted line… but he might just be tired of working nights and weekends. Or maybe he’s got a startup idea of his own to pursue. Or perhaps he’s eloping with the missus and then joining the circus. Who knows? It’s speculation, not news.

58. Eran Hammer-Lahav
    

    April 23rd, 2008 at 9:19 am

    Re: 41 - So stability has nothing to do with management and operations? I don’t have the full story of why Twitter has some stability issue and can’t even tell if they are issues of scale or something else. The way you connect the dots is nothing more than wishful thinking. It might be true, but you *are* just guessing. Again, quality reporting!

59. jro
    

    April 23rd, 2008 at 9:20 am

    Someone can’t scale Rails, and that makes them an amateur? Wow, talk about uninformed writing. Some might even say “amateurish”.

60. Christopher Herot
    

    April 23rd, 2008 at 9:20 am

    Let’s face it - most sites never become popular enough to have this kind of scaling problems. And few have pushed RoR this far. I think we should thank Blaine for giving us Twitter and be grateful for the chances he took opening up the API, and wish the next guy well.

61. Michael Arrington
    

    April 23rd, 2008 at 9:21 am

    what else you guys got?

62. Andrew
    

    April 23rd, 2008 at 9:22 am

    “Mike - if you agree with me that twitter has had serious uptime issues, it’s kind of hard not to follow the dotted lines to the chief architect.”

    That’s a surprisingly ignorant statement.

    You can be the best architect, or anything, in the world but factors outside your control can change the end result in many ways.

    Maybe Blaine just became fed up at to the level of staff/software/hardware support he received while trying to scale and support such a massive platform as twitter?

    I don’t know if this is true, but I do know your statements are needlessly bitchy and unprofessional.

    As for the slides - pfft. Talk about taking things out of context.

63. Nick
    

    April 23rd, 2008 at 9:23 am

    @Michael

    If Twitter is such a hack-y piece of software, perhaps you should stop using it. No sense in supporting the work of an ‘amateur’ developer like Blaine.

    I know it might be hard… you know… not hearing you complain about stuff every 10 minutes on Twitter, but the rest of us (non-250 folks) will survive. Don’t you worry.

64. Mike M
    

    April 23rd, 2008 at 9:25 am

    I think Blaine’s done an excellent job with twitter. Getting RoR scaling this much is a mean feat. I don’t envy his successor.

    I believe Blaine could be right on horizontal scaling. Adding more servers can solve the problem. Perhaps the folks at procurement may have had cash issues? Perhaps they couldn’t find the cash to improve their infrastructure?

    One can only guess. But I don’t fault his choice of platform. (even though I’m a PHP guy).

65. Nick Taylor
    

    April 23rd, 2008 at 9:27 am

    Yea - Twitter in it’s entirety is a masterful bit of communicaion though - it’s like a tamagotchi crossed with a web-app. It’s small and cute and anthropomorhisable - and the fact that it wobbles sometimes doesn’t stop us loving it. Tamagotchi do wobble. They’re kindof supposed to.

    And the Blaine Cook? Never met him - but he seems like a tinkerer, and a project-doer and an all-round geek’s geek… whereas the writer of this blog most certainly does not.

    Which is my 2C why this article is such an own-foot-shoot.

66. jimmy dean
    

    April 23rd, 2008 at 9:27 am

    From No. 61:
    > what else you guys got?

    Now THAT’S the kind of reply Blaine should have posted here.

    Instead, we get the righteous defenders, “We love you Blaine,” which reminds me of the moms in the parks with their kids doing the “Good job!” Sickening.

67. nick halstead
    

    April 23rd, 2008 at 9:28 am

    Wow, this what happens when you annoy the Ruby/Rail’s community in any shape or form (but that is a different rant) - this is not about Ruby Scaling in anyway, although
    they have certainly suffered from not rebuilding from scratch when they should have at the outset. This is about the many-to-many problem in databases, and frankly given the timescales they have had the scaling/reliability issues should not have been an issue even if they had programmed the bloody thing in BASIC. So get over the ruby thing - this is just about not understanding scaling in general - as rule 1) performance DOES NOT equal scalability - so should he have gone? yes - about 6 months ago.

    sent from: fav.or.it [FID327972]

68. maximo
    

    April 23rd, 2008 at 9:29 am

    @Eran: You consider TechCruch real reporting? Please!! This is juicy tech gossip at best. Just read and enjoy it!

69. Ben
    

    April 23rd, 2008 at 9:30 am

    Mike, this isn’t valleywag. And you’re not very good at trolling.

70. Brick and Click
    

    April 23rd, 2008 at 9:30 am

    Every fifteen years or so, there’s a three foot ‘Nor’easter’ blizzard in the Washington DC region. Happened in ‘79 and ‘96. Shuts everything down for a week: Gov’t, schools, many businesses. You can’t drive until the plow truck comes, which may be five days for secondary streets. A lot of folks freak out at first, then acceptance sets in. Cross country skis and Flexible Flyers and Snow Shoes break out, and parents play with their kids again (rare opportunity to build world class snow forts and igloos). You re-meet the neighbors, shovel drives and sidewalks together, borrow supplies if needed, and pop Advil like candy because you worked long-forgotten muscle groups moving several thousand pounds of snow. And you take a zillion pictures of the most amazing winter landscape you’ve ever seen.

    Big snow in Twitterville. Enjoy life for a while.

71. Scott M.
    

    April 23rd, 2008 at 9:31 am

    @Prokofy Neva:
    Aww, c’mon, Prok. Can’t you keep your uninformed and deliberately inflammatory grandstanding/griefing to the Second Life world?

72. John Lynn
    

    April 23rd, 2008 at 9:31 am

    @Coda Hale You said “I can tell you that Blaine Cook has *not* been their problem (nor has Ruby or Rails or any of the other technical issues favored by armchair analysts)”

    If you know this is the case, then let Mike and the rest of us know what or who is responsible so he can update the post. Otherwise, your comments seem more baseless than Mike’s.

73. Dan
    

    April 23rd, 2008 at 9:32 am

    Speaking of amateur hour…

74. michael air
    

    April 23rd, 2008 at 9:32 am

    Put it on Google Appengine. Problem Solved

75. Loren Feldman
    

    April 23rd, 2008 at 9:33 am

    ROR Sucks {seesmic_video:{”url_thumbnail”:{”value”:”http://t.seesmic.com/thumbnail/La1XPF4uMI_th1.jpg”}”title”:{”value”:”R