Amazon Takes on Oracle and IBM With SimpleDB

Trackbacks/Pings (Trackback URL)

1. MySQL
   

   December 14th, 2007 at 10:01 am

   What a great idea!
   -MySQL

2. Jonathan
   

   December 14th, 2007 at 10:01 am

   “Companies can now go ahead and fire their expensive database administrators—those engineers who keep the Oracle or IBM databases humming.”

   That’s a nice thing to say, sleep well at night do you?

3. Sean
   

   December 14th, 2007 at 10:04 am

   Erick, I think you’re overgeneralizing a bit here. I’m sure this will meet the needs of some, but hey, maybe I want to design my own database and indexes. I don’t trust an algorythm to figure out what would work best, because it doesn’t know how I’m going to be using that data, or what type of requests I’ll be making against it.

   Plus, you forget that most startups use MySQL or PostrgreSQL. Both 100% free. How does Amazon save me money? Oh, that’s right, they don’t - I would actually have to pay to use it. Let me guess - this service doesnt come with an SLA either? No thanks.

4. DBA
   

   December 14th, 2007 at 10:17 am

   Erick,

   Sorry man, but you have no idea hat the hell you are talking about here. Seriously have ever developed a web application yourself? For small startups, MySQL or PostegreSQL will do just fine, for larger companies the data models will be a lot more complex, how about stored procedures, security, providence of data, backups and fail over? Seriously do some homework or ask people who know what they are talking about.

   Yes this looks interesting, but your comments reflect your ignorance when it comes to databases and application development in general.

5. MajorNetworkNews
   

   December 14th, 2007 at 10:18 am

   Since most startups are using mysql, this will not affect pricing that much if at all.

6. Mark
   

   December 14th, 2007 at 10:19 am

   I’m not going to jump head first into this, but I’m definitely keeping my eye on SimpleDB. I run a startup that gets (areyouwatchingthis.com) that gets 75% of our traffic from our API. The ability to move that processing and storage into a cloud _might_ save me a lot on hosting.

7. AnonTroll
   

   December 14th, 2007 at 10:19 am

   Sounds great, but out goes all the backend support for SQL databases from PHP, Ruby on Rails, etc.

8. @All techcrunchers
   

   December 14th, 2007 at 10:20 am

   Why do you people want to fire Erick?
   I see nothing wrong…

9. Kevin Merritt
   

   December 14th, 2007 at 10:27 am

   I authored a post yesterday on the democratization of data (linked above). Amazon’s SimpleDB offering will boost awareness that for some applications you can consume database as a service. At blist we believe that building a robust database as a service is an important half the challenge. The other equally important half of the challenge is creating a new metaphor for organizing data and a dramatically easier user to use interface that allows mainstream users to organize data themselves. As some have already commented, MySQL and Postgres are viable behind the firewall options if you are a programmer or DBA, but when will organizing data be as accessible to mainstream users as creating spreadsheets or Visio diagrams?

   http://www.blist.com/blog/inde.....n-of-data/

10. Wouter
    

    December 14th, 2007 at 10:29 am

    Sean, Amazon saves you money by providing the servers so you’ll don’t have to buy them. If you need more capacity, Amazon can provide it without extra cost, besides the standard usage fees.

    SimpleDB is what the name says, simple, and not even close to being used for serious applications. But I like the way Amazon is going right now, S3 and EC2 are getting more mature and have proven usefulness for startups that can’t afford big hosting investments.

    I’ll definitely check it out!

11. MGZ
    

    December 14th, 2007 at 10:30 am

    I assume Erick was being facetious with the ‘fire DBA’ comments.

    Obviously this isn’t a replacement for advanced RDBMS requirements — its called SimpleDB. Amazon even points this out — many, many times — in their document that describes how it works.

    That being said, for the environment they designed it for (AWS, EC2, S3, etc), it is fucking BRILLIANT. I have many ideas for AWS, but most of them are stalled in the planning stage because up until now, it was a total PITA to manage structured, query-able data on EC2. You had to basically roll your own setup, where your database boots up each time and restores from backups made to S3, since EC2 has no permanent storage local to the machine. (Each new instance starts fresh from your OS image.)

    This solves most of that.

    The only thing I didn’t see in the ‘whitepaper’ is whether or not you can perform a query and get multiple items back. There was no reference to doing that, and that would be a big hole in this solution if that is the case. Fingers crossed.

    MGZ

12. Snyggast
    

    December 14th, 2007 at 10:30 am

    i backed out using s3 because of latency issues, but that was way back… they’ve solved that issue now. but I think amazon isn’t running aws off their unused capacity as I’ve previously thought. or maybe i’m confused.

    “..the massive computing infrastructure the company has built to run its online store has a great deal of spare capacity.”
    http://www.roughtype.com/archi.....tility.php

    AWS is completely separate since they needed more hardwares resulting in pricing change for S3. I thought they were running AWS on the same cloud and pipelines as their website. I guess Bezos was talking about the DC?

13. Greg
    

    December 14th, 2007 at 10:41 am

    I was pretty excited about this.. but at a glance there isn’t support for performing JOINs across domains. This one of the fundamental RDBMS operations, and without it, I’m afraid the SimpleDB model is extremely limited since you will need to do N queries to the API for a single join.

14. pwb
    

    December 14th, 2007 at 10:41 am

    It’s not obvious that Erick’s being facetious with the IBM/Oracle headline and the “firing”. If he is, that’s poor journalism. If he’s not, he’s an idiot.

15. Greg
    

    December 14th, 2007 at 10:42 am

    Actually strike that, not N queries for each row but N queries for each join, but you get the idea Having to do the join on the caller side is going to be terribly inefficient.

16. YDRIVE
    

    December 14th, 2007 at 10:47 am

    We are very convinced that this SimpleDB service — particularly being an offering from Amazon — is going to be very interesting and disruptive.

    It will not kill Oracle or MySQL — and the interesting point is not about killing a company or two. SimpleDb will make Web2.0 (and “Web3.0″, if any) services more reliable, more scalable, and thus more serviceable.

    Our confidence level is 100% about it.

    Right, only time will tell… but for now (when it becomes available), it will be very, very interesting to even simply explore — at the very least… /ac.

17. Chris R
    

    December 14th, 2007 at 10:49 am

    neato

18. John Willis
    

    December 14th, 2007 at 10:52 am

    Dude page down another page. You posted the S3 pricing. Also, I agree with all the DB posts this is not going to kill Oracle, DB2, or MySQL anytime soon. However, savvy developers might choose to put things that never needed to be in complex RDBMs in the first place into SimpleDB.

19. Roebot
    

    December 14th, 2007 at 10:52 am

    LMFAO, I’m sorry but you are a complete idiot. Why are you writing for a tech blog? Is this a tech blog?

20. NH
    

    December 14th, 2007 at 10:54 am

    I just crawled out of dot com crash trash bin form 2001. Recently I signed up for Techcrunch newsletter & its a sign I am in the peak of another doc com bust. In any case aren’t those Amazon guys selling ONLY books one time? Now they want to be your database vendor. Whats next? They will shine your shoes for a nickel.

    - NH

21. Open Source Everything
    

    December 14th, 2007 at 11:06 am

    Why won’t Erick respond to the MySQL and PostrgreSQL comments? All these people surely understand the business of database administration much more than Erick.

    As a long subscriber to the myth that Amazon Web Services will prevail this release seems a little out of step with reality. How the hell are they going to omit data modeling and “automatically indexes”? Sure there could be a better way to build a relational database; but what are they doing to replace this process?

    It seems as though the author of this article simply took this press release for it’s face value and published when he clearly did not no what he was talking about.

22. Erick Schonfeld
    

    December 14th, 2007 at 11:06 am

    Hmm, do we have some DBAs in the audience? Everyone is so touchy.

    Like any disruptive technology, this is not going to have all the bells and whistles that a full-fledged relational database has (as I mention in the post). The capabilities of SimpleDB and other Web-based databases, which are in their infancy, will improve over time.

    But that is not the point. It lowers the barrier to creating database applications. A startup can now have its storage, compute cycles, and database all up in the cloud, available on a pay as you go basis. That is going to make possible a lot more Web applications and companies that grow around them.

23. Rovion
    

    December 14th, 2007 at 11:08 am

    MGZ is correct, running any database (MySql, Postgres, etc) with EC2 is a complete pain the butt… these are virtal servers, with virtual harddrives that dissapear as soon as kill the instance. There has been some work done by Mark Attwood (http://fallenpegasus.com/code/mysql-awss3/) on porting MySQL to use S3 for storage, but so far the results have been less than spectacular (one key and one blog row is all you get).

    This may be a “simple” db that they are offering, but I’ve found that most database tasks, if fully normalized, can be done with simple structures. I’m hoping that version two will have SQL syntax.

24. YDRIVE
    

    December 14th, 2007 at 11:11 am

    > Whats next? They will shine your shoes for a nickel.

    reminds us of what John Battelle wrote in his The Search book, Chapter 5:

    “A Billion Dollars, One Nickel at a Time: The Internet Gets a New Business Model”

    (of ‘cos.. talking about adwords/adsense, not shoes shining, unfortunately)

25. Cormac
    

    December 14th, 2007 at 11:12 am

    But startups aren’t using IBM or Oracle, which is what the headline said. So basically you went for a cheap shot and now are surprised getting flack for it. How about you rewrite the post and make it actually newsworthy, because this is cool news, but the way you presented it was just lame. Sorry to say.

26. Marcelo Calbucci
    

    December 14th, 2007 at 11:13 am

    Here is a different perspective on SimpleDB:
    http://marcelo.sampasite.com/b.....y-is-t.htm

27. YDRIVE
    

    December 14th, 2007 at 11:21 am

    > but what are they doing to replace this process?

    Ever programmed in Lisp?

    Or more pertinently, Erlang?
    http://en.wikipedia.org/wiki/E.....anguage%29

    and, remember it’s SimpleDB — ie., don’t expect it’s a MySQL replacement…

28. YDRIVE
    

    December 14th, 2007 at 11:33 am

    @26, yeah looks like that’s another (seemingly correct) perspective also.. and of course, the more familiar way (to the non-Msft camp) to put “Directory Service” is, the all the more familiar LDAP! Hmm, in a sense, that’s true.. but obviously the SimpleDB attributes are going to have much bigger size data values..

29. =)
    

    December 14th, 2007 at 11:37 am

    I used to be a certified DBA for Oracle and DB2, and I work a lot with MYSql too. I am not too sure how the service will work, but for sure there are lots of things a DBA can do but a program can’t.

    You have to understand the use, and how to optimize a database, instead of throwing random index and see how it works.

    Scaling, capacity planning, partitioning and as well as bkup and recovery are all big topics.

    All these need to be think ahead instead of asking a generic program to do it. Oracle has some optimizing algothrim but it is no where close to having a professional DB analyst/DBA to work with business to resolve those.

30. Paul Freet
    

    December 14th, 2007 at 11:40 am

    This is truly the third leg that has been missing form Amazon’s offering. Big Contacts has been using S3 since day one, but we have not able to truly consider using EC2. The problem with managing the database was too great. Now we can seriously look at using Amazon as our infrastructure provider.

    Though I have not dug into it fully yet, there are a couple of things we would have to design around. One is the lack of a IN operator on queries. The other is a lack of TEXT data - the 1,024 chcracter limit is too limiting.

31. =)
    

    December 14th, 2007 at 11:42 am

    and more, as I am an entrepreneur now i am doing lots of work with MYSQL. I know there are lots of DB hosting out there which are cheap and always sufficient for a startup.

    If the startup grows enough to benefit from the traffic cost of amazon would be minimized. And probbaly the startup has to migrate again to a better RDBMS, just my 2 cents.

32. TechCrunch
    

    December 14th, 2007 at 11:44 am

    What a dumb title! The thing does not even have JOINs, let alone transaction support, but according to TC stuff it’s enough to challenge IBM and Oracle.

    Guys, you are turning into Web 2.0 tabloid. What’s next? Photos of naked FaceBook execs on a beach?

33. cbmeeks
    

    December 14th, 2007 at 11:45 am

    For my purposes, I can’t wait.
    http://codershangout.com

34. Paul
    

    December 14th, 2007 at 11:45 am

    This post delves more into the data model , API and the relaxed consistency model
    http://www.sriramkrishnan.com/.....rview.html

35. Zaid
    

    December 14th, 2007 at 11:48 am

    OK my first reaction was like “WTF? we already have mysql”.

    Then I realized we are paying about $1000/month just for our two DB servers that are constantly sucking up proc and RAM. If amazon can help us cut that cost who knows.

36. If No 1 Clicks Thru
    

    December 14th, 2007 at 11:52 am

    pffft. what makes you guys think he wrote the post. Amazon calls ” Hey you dumb ass over there”…E looks around,,, who me? Amazon ” Yeah you.” ” Here is a PR puff piece put your time stamp on it and feel free to add one or two comments…b ut not more. You know nothing about technology”
    “Ahhh, ummm sure I’ll get right on it. We will send you the invoice for the PR”

37. Another Paul
    

    December 14th, 2007 at 12:18 pm

    @22 Erick, re usefulness for start-ups….during the VC grilling in, say London:
    VC: “and what happens when something goes wrong with your servers…”
    SU: “we sit there, don’t answer the phones, and hope Amazon in Seattle fixes it fast…”
    VC “surely you mean ‘worst case we head down to the datacenter half-an-hour away, open up the rack, roll up our sleeves, and when the phone rings, tell the clients we’re on it’….
    Closes wallet and get’s up to go (VC, not SU)

38. Ivo
    

    December 14th, 2007 at 12:27 pm

    Ever thought of the “Cost of Ownership” of all those free MySQL, PostrgreSQL, etc databases? We spent months in achieving a reliable, always-on and performant MySQL environment and are more than happy to move away from this “free” nightmare to a cheap hosted service like SimpleDB!

    -Ivo

    http://www.web20friends.net

39. Shelley
    

    December 14th, 2007 at 12:28 pm

    “Companies can now go ahead and fire their expensive database administrators—those engineers who keep the Oracle or IBM databases humming.”

    What a ditz.

40. NH
    

    December 14th, 2007 at 12:41 pm

    Good observations form “Another Paul”.

    > “VC: “and what happens when something goes wrong with your servers…”

    Its no joke. I has happened with SUs who relied on SalesForce.Com and NetSuite. SUs were hosting their websites and back office apps with the hosted app providers. When the servers went down, both internal operations plus websites were down. Servers down meant no productivity and revenue. GREAT hosting model for SUs.

    > VC “surely you mean ‘worst case we head down to the datacenter half-an-hour away, open up the rack, roll up our sleeves, and when the phone rings, tell the clients we’re on it’….

    Forget about getting in to Amazon’s data centers let alone fixing the servers. Even Jeff B would have problem getting in there. You think they will let you touch their server when they are selling millions of copies of Harry Potties.

    - NH

41. DB guy
    

    December 14th, 2007 at 12:46 pm

    Erick,

    You are talking about something you have no knowledge at all.
    Therefore, your post is too simple, too naive.

42. Amit Raman
    

    December 14th, 2007 at 12:58 pm

    Erick,

    A database is already simple and easy to run. They are up 99.99% of the time and have self-healing, self-administering features. Most companies have a lean DBA staff anyways. Not sure your SimpleDB benefits really strike a chord. Also, good luck to enterprises and SMBs in their porting efforts.

43. wanker90210
    

    December 14th, 2007 at 1:03 pm

    It is a bit sad that they seem to have a stripped down db. For two reasons:

    a) Sql is a brilliant way of describing datasets. The stronger Sql the db has, the less procedural code I have to write to massage the data.

    b) Should the aws DB have been non-simple, one could have a “sloppy replication” (compare LDAP) from servers main db and then let a aws slave do costly queries (FTS, statistical etc).

    Right now, i have solved my db needs for my EC2 slaves by json/xml request to my main web server which was good enough where latency isn’t an issue.

    And please, mysql != postgres. I come from Oracle and have yet to find anything that I miss in my present Postgres databases. (Well, apart from “column varchar2(10 char)” on utf8 db:s.

44. Amit Raman
    

    December 14th, 2007 at 1:04 pm

    “But that is not the point. It lowers the barrier to creating database applications. A startup can now have its storage, compute cycles, and database all up in the cloud, available on a pay as you go basis. That is going to make possible a lot more Web applications and companies that grow around them.”

    Jeez, when was the last time you wrote a web application? The DBs are so cheap and easy to use. Also, each time you run a query on your OWN DB, your usage cost is $.00 + electricity. The Amazon model is expensive. Do your HW!

45. Saagar
    

    December 14th, 2007 at 1:14 pm

    Haha, nice comments from people about Eric’s post. And I was thinking Eric was one of the guys who writes posts sensibly enough to unwarrant strong criticism. Hmmm. Anyways, your intentions do not reflect in your post Eric. Maybe you should join a course on saying things right :). Amazon SimpleDB is what it says, a Simple DB. Unless there is tremendous progress in it, it cannot even dream of being a competitor to Oracle, IBM etc, let alone be a threat. And I am sure a huge Enterprise won’t put it s data dependency on Amazon ever. And why would they even do that.. I guess you have just started off with wild posts like Duncan Eric, the path from now on, is what you choose to tread

46. smartjuice
    

    December 14th, 2007 at 1:29 pm

    Get Ready for next bubble burst… its coming

    Happy New Year

47. Nag
    

    December 14th, 2007 at 1:53 pm

    Erik,
    Hmmmm….
    “Companies can now go ahead and fire their expensive database administrators—those engineers who keep the Oracle or IBM databases humming”

    Database and SimpleDb are completely different. If i am not wrong I think Amzon itself is using Oracle as the backend. Could you please confirm what database is being used for Amazon.

    More importantly makes sense for big database, why would i pay extra $’s for a small database hosted in the intranet. .. What about all the non web 2.0 companies and intranets.

    Cheers, Nag

48. DJ
    

    December 14th, 2007 at 2:13 pm

    Cool concept. No serious startup or web company is going to use this though. You can’t outsource something as important as your database. With most web companies if you lose your data your done. I’m not going to trust any company no matter how big to backup and run the core of my company.

    Niche market for boot strapped start ups maybe. But I can run postgres or mysql on a $1000 server with no extra costs and no company who relies on oracle is suddenly going to jump to outsourcing their DB needs.

    Postgres rocks. I’ll stick with that. Thanks amazon.

49. Ryan
    

    December 14th, 2007 at 2:33 pm

    You do understand that you just attacked one of the pillars of the LAMP technology stack. You were bound to be attacked.

50. Diogo
    

    December 14th, 2007 at 2:58 pm

    Outsource your most important asset, your data. No, thanks. Next, please.

51. Cartografia Extraordinoria
    

    December 14th, 2007 at 3:08 pm

    Wow, people here have *no* idea about how EC2 and S3 work.

52. Anatoly
    

    December 14th, 2007 at 3:10 pm

    @50 Diogo — do you carry your data with you? Like tuck it under the pillow at night?

    Its not like you wouldnt be able to take your data with you should you decide to leave the service.

    Regarding all the dba assaults on this post — a lot of web apps have simple CRUD (create update retrieve delete) requirement, with minimal need for joins (since joins can be moved to a separate relationship table anyways). Furthermore, the transactional semantics on a lot of these apps is very relaxed — no one cares about a distributed two phase commit on a twitter comment, except maybe Scoble.

53. OMG
    

    December 14th, 2007 at 3:38 pm

    Wow, not a single person knows enough about scalability to point out that this is a similar technology to Google’s BigTable and Lucene’s HBase (heavily supported by PowerSet)? It doesn’t do JOINs? That’s because it’s not supposed to be a Relational Database, it’s a very fast distributed lookup table. It doesn’t use algorithms to design your schema - it doesn’t have one. Neither does BerkeleyDB. This simply means that you can’t query on all the different fields, only particular lookup keys. Go read the BigTable paper (it’s beautiful) before dismissing this. And if you’re a backend programmer working at a web startup and made misinformed criticisms like that you should probably switch jobs.

54. TechCrunch
    

    December 14th, 2007 at 3:47 pm

    OMG: before starting to bitch about BigTable you should have scrolled up and re-read the original dumbass post by TechCrunch staffer.

55. Foo
    

    December 14th, 2007 at 5:07 pm

    Erick - has Arrington spanked you yet for posting such tripe?

    Amazon missed a major opportunity here, to build a “DB Cloud” system based around a real RDBMS.

    SimpleDB — No f*cking JOINs?!? You have a DBA and yet you’re going to fire him and use a system taht doesn’t have f*cking JOINs?

    This sytem will be used by a few very, very simple toy apps to do very simple things.

    - Wating for ComplexDB

56. DB guy
    

    December 14th, 2007 at 5:19 pm

    OMG: are you suggesting your super fast super scale BigTable is going to replace relational DB?

57. Wade
    

    December 14th, 2007 at 6:40 pm

    Yeah, I think this could be something. People are missing the point when they say MySql is free. But I’m not going to explain it to them.

58. David Mackey
    

    December 14th, 2007 at 8:37 pm

    Its a good idea…Though all databases are not expensive - namely MySQL and several other open source sql databases. Not that they don’t still require significant administrator overhead - but the software itself is free.

59. Steve Ballmer
    

    December 14th, 2007 at 8:56 pm

    … but they can’t come close to Access!

60. ididak
    

    December 14th, 2007 at 9:03 pm

    Only 10GB per domain, which means the biggest table you can have is 10GB, which is small even for MySQL bundled in cheap hosting plans these days. Dynamo is the Achilles’ heel of AWS.

    Considered the listed price: $1.5/GB/month, SimpleDB is weak even compared with their S3: $0.15/GB/month.

61. Alan Wilensky
    

    December 15th, 2007 at 8:30 am

    I would be very interested on what Kingsley Idehen has to say about this. He is the defacto heavy DB iron expert for all the types of things that simple DB can and cant and what MySql or Postgres can or can’t do.

    Stored procedures, ODBC, log shipping. etc. Of course, with the grid this new service runs on, it might make those types of services superfluous.

    see http://openlinksw.com

62. OMG
    

    December 15th, 2007 at 10:06 am

    @56: No, not at all - I am saying that there are specific applications and parts of applications that will really benefit from this. If you are doing web search or a lot of distributed data analysis this is is probably going to be very useful to you.

    Data structures people! Relational databases work quite well so that most of us have just been stuffing all our data into it by default. It’s great for many things but it’s not always perfect for every application: http://www.mooseyard.com/Jens/.....er-second/

63. Paul Duggan
    

    December 15th, 2007 at 4:00 pm

    Could they (Amazon) not simply have put the effort into providing persistent storage for Postgres/MySQL? I’ll bet that would not have been a huge technical challenge for them.

64. Shap
    

    December 15th, 2007 at 9:09 pm

    “Also, each time you run a query on your OWN DB, your usage cost is $.00 + electricity.”

    Perhaps if your site is so small and sees use so light that you don’t need at least one dedicated database server, your argument has merit. As soon as you get to the point where you need a dedicated server or cluster, your argument falls apart.

65. Mark
    

    December 16th, 2007 at 11:16 pm

    Whoa, all you database administrators, why so defensive? Maybe because deep down you know that SimpleDB is just the beginning, and it very well could become a threat to your jobs? Sure, it will take years and years, but if you can’t see that this is the future, I’m sure you are close to retirement anyways.

66. kayoone
    

    December 16th, 2007 at 11:37 pm

    i wonder which company would host their internal data on such a thing. Here in germany nobody would, i bet.

67. Mukul Gupta
    

    December 18th, 2007 at 4:33 am

    Sure it’s an innovation but not sure if its useful.

    Companies spend for DBA’s because A) They don’t know anything about database design and implementation and B) The complexity of data and chaning business requirements needs someone to ensure that the database is up-and-running. So relax DBAs! it won’t take your jobs away anytime soon. You can infact a lot of excel sheets which will require to be amazonized (Pun intended!)

    This service definetely does not solves any database design problems and as lot of people have already said, it does not know my business and hence it cannot guess what is important for me. I see it useful for very simple applications as the relational aspect of data will require to be manitained manually.

    I certainly wont be recommending it our clients anytime soon!

68. Andy
    

    December 19th, 2007 at 10:27 am

    Oh, and did not know about it. Thanks for the information …

69. n
    

    December 19th, 2007 at 2:51 pm

    Dudes, Amazon just launched DevPay. Sweet!
    http://www.amazon.com/gp/brows.....=342429011

    They seem to be really serious about SaaS.