The Amazon outage, first noticed by Twitter users at as early as 10:24 am PST this morning (I’d link to Twitter except that it’s down), has gone on for an hour now. It’s not necessarily news when a startup has minor downtime, but Amazon loses big revenue for every minute they aren’t out there selling.
My PR contacts there aren’t answering the phone, and a message to the main PR number hasn’t yet been returned. At least their web services appear to be humming. An AWS outage in February took an untold number of startups with it.
Update: Looks like www.amazon.com works now, but only sometimes. According to Twitter the site first came back up (sporadically) around 12:10 PM PST. Based on an average loss of $31,000 per minute, Amazon just theoretically lost out on around $3,286,000.
Update 2: Email from Amazon PR:
The Amazon retail site was down for approximately 2 hours earlier today (beginning around 10:25) – and we’re bringing the site back up.
Amazon’s systems are very complex and on rare occasions, despite our best efforts, they may experience problems. We work to minimize any disruption and to get the site back as quickly as possible.
Amazon’s web services were not affected nor were our international sites.
Ah yes, the power of Twitter. A quick visit to the Amazon site, however, reveals a couple performance issues as early as yesterday. Would that have any impact on the site overall?
Twitter is loading fine over here.
And we’re back…
Nevermind. Down again.
I have it on good authority that it’s part of their plan to compete with Twitter: http://www.chri...e-with-twitter/
MediaDefender is at it again.
The twitter api seems ok, using twhirl.
amazon.ca and .co.uk are fine. also, https://www.amazon.com/ works for me, interestingly.
The SSL version https://www.amazon.com is still working. I read the 80gig PS3 bundle with Metal Gear Solid 4 was going on pre-order at 10am PST at Amazon.com today. I wouldn’t be surprised if the load was the cause of the outage.
My guess is it has to do with the release of the new PS3 / metal gear solid release that is VERY coveted (was supposed to go on sale at 10am at amazon.com). I know it sounds silly, but remember a similar thing happened back when they were doing some other mega sale on a current-gen console – can’t remember which one though. Amazon didn’t go down, but the service was SUPER slow.
Still down.
It’s up now.
11.42
It looks like the terrorists have won.
Wow, you would at least expect them to have a better service down message this must of came out of nowhere.
Definitely due to metal gear solid…they just took that page down (goes to a custom error page)…so dumb. They sent an e-mail out a couple days ago telling everyong to look forward to that 10am pre-order launch…guess they should’ve sent it to fewer people.
I heard that there’s an all-hands going on right now. Someone must’ve tripped over the switch on the way there. No worries, I think they’ll be back soon to turn the site back on.
#14. That is a good point. But if the server is down there is no way for them to display any error message. As the requests cannot be served.
However, if it was a HTTP 500 or HTTP XXX they can display any desired message.
Cheers, Nag
Still down for me. Our Amazon Associates log in is working fine, but our widgets, etc. are not working. (See here for an example: http://farm4.st...141a1abab_o.png)
do you think they use a single master mysql setup?
http://www.tech...-questions-too/
https does not work, click on a link.
Looks like my first contribution to TC was noted by other TC users as well.
It doesn’t make any sense to me, but if you try to load Amazon on Internet Explorer it seems to load once in a while but it doesn’t seem to load at all on Firefox.
It is down again.
#14. I take it back. SORRY! You are right.
Looks like the server is up and it can respond to the requests. I AGREE THEY SHOULD HAVE A BETTER ERROR PAGE. So users know there is some issue with amazon.
I LIKE TWITTER THAT WAY, they mention that there is some issue ( DATABASE or something )
Cheers, Nag
It just came back up for me.
Back-of-the-napkin math: 2007 revenues around $14.5 billion. Assuming a little growth, suppose they have $15 billion revenues this year.
Flat math = $30,000 revenue/minute
First outage timemark I’ve seen = 10:30 am (pacific). First uptime timemark I’ve seen = 11:42 am (pacific). (Still not back up for me.)
72 minutes @ $30k =
$2,160,000 in lost/deferred revenue.
Yowza.
The site works perfectly in SSL https://www.amazon.com/ !! And the API calls seems to be answering well enough.. its just the site doesn’t reply in http mode? A hack attempt or technical error in configuration? And whats taking this folks so long to fix it? :O
Cheers!
the inner pages (deep links) still work. It’s a home page issue.
Finally able to see homepage @ 12:10pm Pacific.
I just reloaded the page and saw this new message:
We’re sorry!
An error occurred when we tried to process your request. Rest assured, we’re already working on the problem and expect to resolve it shortly.
If you were trying to make a purchase, please check Your Account to confirm that the order was placed.
We apologize for the inconvenience.
Continue shopping on the Amazon.com home page
It’s back online. Well here at least.
the interesting thing here is how much amazon would lose here ?
They would mainly lose the impulsive purchases, not the planned ones….
People who get the error message will re-visit, but those who are bored and would have bought impulsively, would spend their money at some other place.
Financially, its a small change for Amazon. Credibility however is being damaged.
Who’s next ?
I wonder what would happen if Weather.com would be unavailable….
I was able to get into some of my back-end systems at Amazon (i.e. seller account) but no luck with any other page on the amazon.com site.
Back just now!!
down again.. whats the heck..
This is crazy. I’ve never seen Amazon down for so long. I seriously doubt it was because of the ps3 bundle. I don’t think 1 single product would bring the main amazon website down. Deep links still seem to work fine.
Something incredibly funny to me about trying to get the word out about Amazon (which is almost never down) using Twitter (which is almost always down).
@Nag: Most of the time the AFE will redirect to a down page (or serve one from cache.) If they use Global Server Load Balancing (GSLB) then you’d expect it to go to an alternate site.
This suggests that something pretty serious is going on, either because none of their sites are working (which could be a vulnerability or a bad code change) or because something in the service lookup (global DNS) is horribly broken.
Generally:
- First get a working bunch of IPs from the DNS (in this case, ultraDNS is listed here: http://network-...host=amazon.com)
- The GSLB DNS server is supposed to know which destinations are available, and which are sending bad stuff or off the air
- Get to the destination. If it’s broken, the AFE should do an HTTP redirect to a working site (in the event the GSLB missed something)
- If there are no other sites, at least display a static/cached page from a separate server.
This suggests the problem is somewhere in the DNS/GSLB tree or at the local AFE. But I could be completely wrong and subjected to ridicule any time now.
Amazon Daily Blog is down. Main secure page (https) is rendering, but I haven’t tested many links yet.
I think Billary Clinton has something to do with it, since she’s furious about defeat. “If I go down, I’m taking Web 2.0 with me!” LOL
According to CNET, Amazon is losing $31,000 per minute.
Ditto what Shane said – Amazon handled all the Harry Potter releases just fine and other high profile releases – intuitively it seems crazy that they wouldn’t plan for increased traffic.
Must be some new front page features they added..
Interesting, amazon.com is down, but my widget is up and running, and the link from my widget is working [http://www.amazon.com/gp/aws/cart/add.html?SubscriptionId=D68HUNXKLHS4J&AssociateTag=thbrbiani-20&ASIN.1=0393057941&Quantity.1=1&adid=09CDCE786R47RVNMDGF0&linkCode=as1&OfferListingId.1=RfeOgW2fS4xsKtMO0i352dBUnrPVRpAUOxtVJC2OOD%252FuOpwsMFvqQfNX3KWin8NmqpIejnavmulFvyUmJm%252BC%252FA%253D%253D&submit.add.x=45&submit.add.y=24]
I guess I cannot complain then!
I wouldn’t be surprised if they see a pretty big spike in sales once the site is back up. If I went to Amazon to buy something (and it wasn’t terribly urgent) and the site was down, I’d probably just come back and buy it later. That’s not really a lost sale. Obviously not everyone’s gonna do that, but I doubt they end up losing as much as everyone’s saying.
@Alistair Croll
Wow! Good information.. Thanks Alistair..
Cheers, Nag
According to AdAge, Amazon just lost $1.8 million.
http://adage.co...ticle_id=127593
I’ve got a picture of a dog and the following message when I visit amazon.com:
“You have been denied access to this feature because we believe you violated the terms, conditions, rules, guidelines or policies of our site in the past. If you believe we have taken this action in error, you may contact us at ad-help-us@amazon.com. ”
What?!
I don’t believe any of these figures of loses of $2-4 million, most people would just come back later to complete their transaction. Amazon has a lot of loyal customers who don’t like to order from other sites.
seriously..boohoo they lost 3+ million bucks…. i can not even manage to get a loan for 25grand so I can go to school next year.. they make enough money that a losing 3+ million over a few hour period because of being down should not be too big of a deal.
I am getting redirected to their robots page, which gives me a pretty crappy user experience. The majority of people who see this page are going to be pretty annoyed (I would think).
“We’re sorry!
You have been denied access to this feature because we believe you violated the terms, conditions, rules, guidelines or policies of our site in the past. If you believe we have taken this action in error, you may contact us at ad-help-us@amazon.com.
We apologize for the inconvenience.”
You guys denying this is related to the PS3 bundle don’t understand how people who want these things operate. I guarantee you this was the cause of the downtime.
This bundle is in VERY high demand. It was posted about on EVERY major video game blog. MOST people who want a PS3 and don’t have one already have been waiting for this bundle to come out, myself included.
There were probably literally hundreds of thousands of people on amazon.com around 9:45am or so, starting to hit refresh like mad, waiting for the item to be buyable. This type of refreshing is unlike any other type of normal traffic. It does not apply to things like Harry Potter where there are millions of copies available right away – no one feels like they need to be there the second it comes out or they won’t get it. No, this ps3 bundle is in VERY short supply, so everyone wanted to be there the second it was able to be bought.
When the PS3 was first launched on costco.com, it brought down their entire web site too because of this same type of behavior. Millions of people sitting on one page hitting F5, F5, F5 over and over again. This is just like a DDoS attack basically. Seriously, read this article:
http://digg.com..._Before_8AM_PST
I guarantee the PS3 bundle was the cause of the downtime, without a doubt.
Amazon didn’t lose $3,286,000… they just didn’t make $3,286,000.