http://blogs.smugmug.com/don/2.....the-money/

Your 1TB server is not a fair comparison. Amazon mirrors your data in multiple data centers so you will need to consider having at least 2 data centers and consider the b/w between them. The problem is that you are comparing cheap backup (as in, i don’t really care if I lose my backup because its just backup) to Amazon’s triple redundant, physically distributed storage.

our calculations are that you can build the storage yourself for less from a hw cost standpoint, even keeping copies in multiple data centers, but if you put even one head on the project fulltime to keep adding storage, fixing broken stuff, and scaling the system, the breakeven point is years out.

Reliability is not just one number. I did not mean availability (where amazon had some issues). I meant that Amazon won’t lose your data.

As for the SLA, we absolutely agree. It is needed. It is not really needed to use Amazon as backup, but it is certainly needed to make them primary storage. Feel free to email me at andrew@phanfare.com if you want more detailed cost numbers.

> You did not consider the cost of co-location and bandwidth for the server.

Actually, I did. If the server is used for backup, the bandwidth and colocation pricing seems to be about the same.

> Writing the virtualization layer so that it appears as limitless disk is non trival.

Sure, nothing is trivial when you’re developing software.

> Buying a solution (probably a SAN solution, will increase your storage cost a lot).

Hmm, not sure about that. 1TB storage for $1000, whether that’s 1TB for $1000 or 10TB for $10,000 (or less).

> S3 is considerably more reliable than a single 1TB server.

In theory perhaps, but that’s not what I’ve read. Check with Smugmug. Also, Amazon doesn’t even think S3 is reliable; they have no SLA.

> Your single terabyte server is not backed up in multiple locations.

So buy 3 or 4 more identical servers at $1000 each. It’s still cheaper to buy your own servers and colocate them.

> If you compare apples to apple and built a S3, you would be hard pressed to do it for less.

That’s what I keep hearing, but no one’s making the point with real numbers. Investing in technology based on speculation, without real numbers, doesn’t seem like the right way to do it. Unless your investors like the idea of working with Amazon and you like the exposure from TechCrunch.

Hey man … how are you? Scott mentioned something about you guys a few weeks ago. You guys still using the same technology when we last talk?

The secret sauce is AFS. That is all the magic you need. In my opinion it is not only the time that takes you to create a solution, but the cost of maintenance and mantainability, availability, risk through time, lover external dependencies, ownership and overall performance. I did not say that S3 is not a good solution; I said that there are better off-market solutions that you can build yourself and leverage to increase the valuation of your company in an M&A.

It is the same old argument of build vs. buy. I am more of the build mentality - but only for core technologies that do make a difference at valuation time. For example, I would not build a general ledger, but in the case of Pictage, I decided to build the image serving technology for two reasons:

1) Totally core to the company.
2) Even though there were some good products at the time, they were not cost effective.

At the end of the day, to this day, nobody has an Image Server as good, flexible, versatile as Pictage (and I built that) and if you do not believe me, look at Pictage’s Album Designer. The imaging back end is all done real time by the Image Server.

I am no longer at Pictage … good riddance … people there get treated like dogs. But the technology I left behind remains really powerful, including the storage solution.

InfintieFTP provides FTP access to Amazon S3.

Andrew

People, you can build very inexpensive distributed storage that works better than S3 or EMC or any other possible vendor. Contact me if you want to know hoe.

1. you will almost certainly wind up virtualizing that 1TB server if you buy 100 of them. Writing the virtualization layer so that it appears as limitless disk is non trival. Buying a solution (probably a SAN solution, will increase your storage cost a lot).

2. S3 is considerably more reliable than a single 1TB server. It is more like a single 1TB server and another server or two to back it up. Your single terabyte server is not backed up in multiple locations. S3 automatically does that for you.

If you compare apples to apple and built a S3, you would be hard pressed to do it for less. Certainly it is possible but your scale would need to be quite large to do it.

One of the reasons S3 seems expensive for backup is that it is not really backup. It is a reliable store that is also backed up, essentially. Hence, using S3 for backup is a bit of overkill. That is why to really clean up cost wise, you must make S3 be your primary data store. Whether you feel comfortable doing that without an SLA from Amazon is up to you.

A 1TB server costs about $1000 (or less) to buy. Expect at least a year of service from that one server, probably closer to 5 years, or longer when you replace the drives.

S3 costs $0.15 per GB per month, which is $0.15 x 1000GB x 12 months, which is $1800. For only one year. Two years is $3600. 3 years is $5400. In the meantime, your $1000 server is still humming along at 1/5th the cost. And will still be working for a full five years while S3 will have cost you $9000 in 5 years for the same amount of storage that you paid $1000 for.

The bandwidth costs seem to be about the same, especially if you’re just using this server for a backup.

Operations costs? Hands-on service at a co-lo can be as high as $200 per hour. So, let’s say a drive goes out once a year and needs to be replaced. You’re still much cheaper than S3.

Thanks for the reply. I hear you on RAID6 as your primary.

I am curious if Amazon facilitates periodic integrity checks and tests of back-ups (that do not themselves incur significant transfer costs, e.g. MD5 checks on both sides).

S3 is appealing. In outsourcing critical or back-up systems to S3 I would be looking for comfort that the triple nightmare scenario of “RAID6 failure, back-up corrupt” is demonstrably unlikely.

BackupReview.info

As this site ranks the top 25 online backup companies on a monthly basis, I have not seen even once Amazon’s S3 ranked in the top 25 in the past 2 years. If S3 is that good and relaible, then why is that it did not make the cut at least once?

It sounds fishy here, and it looks like that the more a company has money to promote its products and services, the more we hear about them, despite the fact that there are tons others with far better products.

Amazon S3 is a very interesting and cost effective service for web entrepreneurs that have significant data storage needs. Techcrunch just wrote an interesting post about it here. Below is an interesting beginners video on how to use the S3…

although they started advertising groceries on their homepage again

- not - so - brilliant -

Darin

Also, if we used Amazon for primary, and they went down, we would have almost no recourse. You can read the terms of service on Amazon web services. It basically says, if it goes down, tough cookies.

I actually believe that Amazon was extraordinarily conservative in crafting their terms of service and now that they have some operating experience, they will strengthen the terms of service, or offer traditional service level agreements on a tiered basis.

In any event, the Amazon web services concept is a great one, especially for the startup community. If was just one guy hacking my prototype, I would definitely see if I can get the whole thing on top of Amazon web services. After all, at that point, you have little to lose.

This past week in NYC, Amazon’s AWS evangelist presented at the NY Tech Meetup and I was surprised how many people had not heard of their services. He did a demo of an 3rd party software that monitors your EC2 servers and automatically increases / decreases capacity based on demand. Very cool.

PS #4 - PostgreSQL works with AWS.

Please feel free to contact me privately if you feel comfortable with that.
My email address is len [{ at }] thinkbbk dot com.

Thanks

ice