You’ll never win by metering, when others provide similar services that are not metered.

That’s aside from the whole security standpoint and the fact that banks, govt agencies and pretty much any other company that deals in CC numbers or personal info would not use this service because it breaks protocol.

NOOOOOOOOOOOOOOOOOOOOOOOoooooooooooooooooooooooo….

You buy several of them. Together you could make your own S3 and E2 distributed computing platforms by networking them with a LOCAL gigabit switch. That way it wouldn’t be metered. Some people, yes most people despise metered billing. Some people want to own the software or appliance outright. Such as….

Schools,
govt agencies,
public sector,
large scale project startups like ours,
SMBs that are bigger than a handful of people.

Metered billing is impractical for these entities. Govt regulations, and yes, I know what they are, we were in the http://www.merx.ca system on govt tenders for over a year, DO NOT ALLOW govt data to be sent to a remote. It violates specifications. Banks, online shops, anything that requires personal data will never send it to a remote.

It’s just not practical. So what do they do when they need distributed computing? They buy a preloaded Sun box at 80-120k a piece, or they build it from bulk purchases on cheap gear like us, if they don’t have to follow warranty mandates and guidelines.

Supercomputing and massive online storage isn’t for small time, it’s for large scale applications. Those who typically code such applications are the same ones who will build their own system.

Had Amazon put the systems in a box appliance like Google did, then let people buy as many as they wanted to make a distributed computing system, govt agencies and other businesses that can not, by mandate use Amazon services like that, could suddenly be customers. That’s called smart thinking. Aiming huge distributed computing services at geeks in a basement with the per minute style 1-900 billing they know all too well is stupid. Sure other small startups could use it, but that market is pathetically small.

The algorithm *IS* the service Chris. Secondly I have repeatedly asserted that your approach _is_valid_ under Google’s scale but not for 97% plus of todays startups. You keep ignoring that.

Besides I’m not even going to argue anymore since you’ve demonstrated your total lack of understanding of AWS by throwing out this gem (#34):

“Amazon should make the S3 and E2 services into appliances like Google did, then sell those, so we can use them non-stop as much as we want at our own office on our own duplex line. Like preload the appliances with the clustered computing platform.”

Make (virtually) unlimited storage and on-demand, (virtually) infinite CPU cycles available in a single box we can buy? Let me know when you figure out how to make these magic boxes and I too will sign up. C’mon man are you serious?

While you’re figuring these magic boxes, let me know how far along you are with that perpetual motion machine.

I’m done. Enjoy your hardware love affair, while the rest of us enjoy building the next generation web.

“Here is a hint, buying more CPU’s from Tigerdirect != unlimited”

This is pure ignorance. You are limited by the physical hardware. Even in a distributed computing cluster. Even on a supercomputer. Dividing physical resources by virtualizations != unlimited.
Even the @Home network peaked at a over a teraflop. Amazon’s doesn’t even come close to that. I’m beyond positive the backend is NOT a super computer like blue gene.

“Here’s a hint. In this age, no one cares about hardware as much as you obviously do. That’s fine. The rest of us will remain obsessed with our *services*”

Google didn’t win on services. They won on pure physical computing power, algorithms and raw bandwidth power. Before search engines were mere directories, after they were maps of the internet. No one has evolved the net that much since.

People are far to concerned with what’s “hot”, “popular” and in “vogue”. Flush these services and put the “science” back in computer science!
I don’t like remote wizard of oz services, the man behind the curtain controlling all of my code. I like to see and touch the hardware. I like to see the lights on the Cisco gear blinking away. I like to be there. I am not the only one.

“Don’t fault Amazon too much for that… requiring that the contest be US-only *dramatically* simplifies contest rules and legal expenses, and ensures there will be no legal problems/complications when it comes time to offer the winner an investment. I wouldn’t be surprised if there were multiple orders of magnitude in difference in cost/risk/etc. between a US company doing a US-only competition vs. a US company doing a whole-world competition.”

Yes I agree whole heartedly. But…is this content supposed to be about the startups and promoting their web services or is it about Amazon getting first crack at a sweet investment deal? It seems to me, that the later is more important to Amazon than the former.

Besides, what’s to prevent them from stating that the $50k investment is open to US Companies only but the $50k services in-kind prize is open to the world?

The answer?

How much better does a Wall Street announcement that they have invested in the _grand prize winner_ of its` contest look compared to an announcement that they have invested in the third or forth place finisher? It’s obvious they’re intentionally limiting the field for their own benefit.

Although I will continue to defend Amazon’s services because I believe they are game changing, this contest is all about Amazon corporate interests first and foremost. The people who run AWS obviously hatched this idea with the best of intentions, but once Amazon corporate got involved they screwed it up royally.

But then again they have an actual product…. investors… staff…. and that sort of thing.

Here’s a hint. In this age, no one cares about hardware as much as you obviously do. That’s fine. The rest of us will remain obsessed with our *services* while you continue to rant about how LEET your infrastructure is.

You still never managed to explain how transferring data across the globe is any better with our own North American data center vs Amazons. Nor did you explain how I can buy or build my own virtually unlimited compute on demand cluster. Here is a hint, buying more CPU’s from Tigerdirect != unlimited, infinitely scalable, on demand computing that is pay as you go. The fact you tried to suggest that it is, is nothing short of hilarious.

And lastly, try and be more concise. Six posts in a row (a TC record?) and I still haven’t heard a coherent argument why my startup should build my own infrastructure vs. using Amazons. To reiterate a key point (that you are ignoring), is that yes at some point it makes sense to build your own if you are using enough resources. But as I said before, 95-99% or more of startups don’t fit this category.

It’s simple math Chris.

Amazon should make the S3 and E2 services into appliances like Google did, then sell those, so we can use them non-stop as much as we want at our own office on our own duplex line. Like preload the appliances with the clustered computing platform.

If they did that, I would consider buying. But only if they make it as cheap as if we bought the parts and made it ourselves. I don’t want to rent equipment and pay by the minute like a sleezy 1-900 sex line. There’s my user feedback as a distributed computing user.

Don’t fault Amazon too much for that… requiring that the contest be US-only *dramatically* simplifies contest rules and legal expenses, and ensures there will be no legal problems/complications when it comes time to offer the winner an investment. I wouldn’t be surprised if there were multiple orders of magnitude in difference in cost/risk/etc. between a US company doing a US-only competition vs. a US company doing a whole-world competition.

15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform”

LOLZ

“Again, you’re also focusing on your hate of S3, but what about EC2? Does Tigerdirect sell cheap Compute-on-demand clusters that I don’t know about? (If so, please post the URL for that as well).”

tigerdirect.ca/applications/SearchTools/item-details.asp?EdpNo=2905631
tigerdirect.ca/applications/SearchTools/item-details.asp?EdpNo=486747&Sku=O261-7002

plus
$60 wholesale foxconn quad supported boards plus slim 1U socket 775 fans for $10 a piece, plus
cgi.ebay.ca/ws/eBayISAPI.dll?ViewItem&rd=1&item=300150507042
You can pick these up at Ciara Technologies in Montreal for $70 a piece

tigerdirect.ca/applications/SearchTools/item-details.asp?EdpNo=3206259&Sku=TSD-500AS4

You can squeeze 3 of these for 1.5 TB at $300 into a single 1U, pics/visual evidence :
http://www.flic...N03/1381909132/

Smush that sh*t in there eyeah!!!

So instead of virtual instances with virtual cores, which virtually sucks, you can be pushing physical 8MB cache quad CPUs with 8 GB of ram a piece. The foxconn board in that pic only doesn’t support 8GB, but you can buy those for about $10 more a piece.

The point is the hardware is too cheap, and the FOSS and clustered computing software is, well, free. There is no reason to do cloud computing with amazon if you have a few thousand to invest and you can build hardware from components. It would be cheaper to hire an employee to build it for you than to pay the amazon overhead. Especially if you do a lot of clustered computing.

The @home clustered computing where they have people volunteer CPU cycles on the other hand is cool. That’s a GOOD use of cloud computing. So is the SETI app. Paying for that is lame though. Doing distributed computing over Gigabit LAN is WAY the hell faster and cheaper.

http://en.wikip...i/MySQL_Cluster

“Currently a maximum of 64 nodes can belong to a single MySQL Cluster with up to 48 of those being data nodes. It is possible to change this at compile time, but that has not been thoroughly tested at this point.”

So you could easily cluster 48 x 1Us for 50+ Terabytes of LVM as a InnoDB cluster.
This would eliminate the cost of buying a Sun Micro SAN or other expensive SAN solution.

As for HTTP, you can easily load balance Apache 2 with something as retarded as Webmin GUI on your p-port 10000

If you want clustering in your apps, I believe there are some FOSS java libraries for that.

A half duplex line like that only runs about $1000 USD per month around these “NON-USA” parts. The actual upstream to your own SAN in a center cage goes through the LAN switch anyway so it’s never counted or billed.

The super service S3 offers is only half duplex to the end users. It’s like a glorified SOAP version of rapidshare.de from the docs I read a year or so ago.

By this I mean if there is a TCP error, mid stream or any type of connection drop or reset. Over a worldwide web, this is a likely scenario. Shuttling data in a center on a 10/100 or gigabit switch is the way to go, not shuttling it around the world.

Think about it for a second.

For 10k you could either get minimal bandwidth or a 100MBPS unmetered cage and 48, yes 48 servers full of 1+ TB hard drive LVM’s. Yes you have to manage them, but when you get into those amounts of money, let’s face it, the people have enough to hire employees.

Long range TCP soap on huge data objects is unsafe, unreliable and stupid, I do not retract. Plus if you mess it up you have to start ALL OVER AGAIN. Wasting even more bandwidth.

Executive Summary: People that can afford Amazon S3 can afford to shuttle the data on a local router in the datacenter, and not use the internet for data storage solutions. GPL backup solutions exist and so does RAID for Linux.

Why the hate?
This whole contest is a ruse to create affiliate tiers for service providers using the API. It is a pyramid. Why not create a compelling service instead and not have a contest?

I have been a member of the Amazon Developer program for almost 2 years now. I went through the services thoroughly over a year ago and came to the conclusion that it wasn’t a good idea.

With all the resources Amazon has, I wish they would have thought of something more practical than this. I ask myself why educated people would create SOAP services for remote SAN aimed at high revenue customers that can afford in house solutions?
Usually SOAP services not used in Enterprise are aimed at low income, high volume clients.
People that could effectively use this service are also those that know better than to use it in my opinion.

Unmetered, In Montreal, OUTSIDE the my precious US of A. $2399 USD. Which with the current exchange rate comes out to like 50 cents Canadian money. LOLZ

1. “Amazon built a pyramid storage scheme a couple years ago. It’s like if XDrive didn’t have enough customers so they put out an API to let other people (affiliates) find customers for them.”

Pyramid scheme? Explain that to me. Do you know what a pyramid scheme is? Besides, there is no affiliate program for S3 data storage anyways (that I’m aware of) so I’m not even sure what the heck you’re even talking about.

2. “This contest is to scheme in more affiliates for use of their storage, which let’s face it, IS DIRT F’ING CHEAP anyway”

Again, what are you talking about affiliates for? (See #1). But, besides that.. yes drives are cheap. But sysadmins, co-location fees, fail safe backups, redundancy, OS updates and so on are not. Co-Lo is especially expensive outside major US centers.

3. “Amazon doesn’t get it. The reason facebook and other API is so popular is BECAUSE the data is light, because they are not storing huge blobs cross country wasting bandwidth. It’s impractical, slow, and stupid. Maybe somebody should have actually thought it through before doing that as a SOAP interface.”

I’m none too fond of SOAP either. But that’s a minor detail. Also, third party services on Facebook are served on Facebook servers themselves, so your argument doesn’t make any sense whatsoever.

As for storing huge blobs cross-country being slow, yes I agree but how is that different from sending huge blobs across country to your own local data center? Oh I get it, my startup should have datacenters in _every_ region. I’ll just call up my investors for another few hundred grand while giving away another huge chunk of my startup. Good thinking.

4. “Amazon should have scrapped this service long ago, like when XDrive went downhill. This contest will not be successful. They should put this SOAP API on the back end, and come up with a service most people will want to use instead. Then market that. If the service is viable, they won’t have to offer cash & prizes. People will want to use it because it’s USEFUL.”

Ah yes, nobody is using Amazon Web Services. Right. What do you base this observation on? Ever visited the developer forums Chris? Have you read any of the articles about the showcased startups? There is some seriously cool stuff being built on top EC2/S3 right now. Sure it could be done with your own hardware but why the hell would I want to do that if I don’t need to?

Again, you’re also focusing on your hate of S3, but what about EC2? Does Tigerdirect sell cheap Compute-on-demand clusters that I don’t know about? (If so, please post the URL for that as well).

5. “Carting 8 gig data objects from Sweden to Amazon’s SAN so that people can download it from hi capacity servers at YOUR EXPENSE is RETARDED in any language. That’s what local datacenters are for. If you can afford to give Amazon 5k per year or more in transfer overhead, you can afford to build your own large cap servers and truck them down to a local data center.”

- So, what if your data objects are small, or at least reasonable (8 gig data objects… wtf!?) ? Does it still make economic and logistic sense then? And tell me how trucking servers down to my local data center alleviates the problem of long-distance, large bandwidth transfers from the other side of the globe? The data still has to travel from Sweden to my local datacenter doesn’t it? Try making sense.

- Secondly, if you spent any time in the developer forums you would realize that geographic localization is part of Amazon’s long term plan for S3 and EC2. It’s a no brainer, in fact I’d be surprised if it weren’t already being done to some extent. Amazon has data centers all over the world, what makes you think that the S3 domain resolves to their US data center from everywhere on the globe?

Lastly I’ll agree that the contest is kinda silly and that S3 bandwidth costs are still in the high end of what is reasonable but prices have already dropped significantly since the service launched (did you know that?), and will no doubt continue to do so.

Obviously, if you’re starting the next Youtube or Pr0n DVD download service (I’m trying to think of a site with 8gig data objects, but can’t think of much) then S3 makes no sense. For the other 97% of startups out there it makes perfect sense.

Regardless, why the strong hate? I’ve never seen someone get so wound up over a damn web service. Relax man, seriously. Get some air.

The more you know * * *

“Why don’t they just put short title win $50,000 startup challenge?”

Because if you reveal bologna as bologna, people get disinterested. Deceptive numbers are the name of the game in IT. “this site got 10000000000000000 hits last month” Don’t mention that they were all from vietnam and that they were robot hits. That’s how it largely works.

50k is nothing to Amazon. If people were to participate, and they won’t it would have made them much more money than it cost. Unfortunately, people already know about their API, and the drawbacks make it unacceptable. Large capacity hard drive storage space is just too cheap now, and putting drives together with LVM is just too easy. Also datatransfer is at an all time low.

Amazon should have scrapped this service long ago, like when XDrive went downhill.
This contest will not be successful. They should put this SOAP API on the back end, and come up with a service most people will want to use instead. Then market that. If the service is viable, they won’t have to offer cash & prizes. People will want to use it because it’s USEFUL.

Carting 8 gig data objects from Sweden to Amazon’s SAN so that people can download it from hi capacity servers at YOUR EXPENSE is RETARDED in any language. That’s what local datacenters are for. If you can afford to give Amazon 5k per year or more in transfer overhead, you can afford to build your own large cap servers and truck them down to a local data center.

This was super badly thought out by Amazon. Bad show Amazon, bad show.

Amazon built a pyramid storage scheme a couple years ago. It’s like if XDrive didn’t have enough customers so they put out an API to let other people (affiliates) find customers for them.

This contest is to scheme in more affiliates for use of their storage, which let’s face it, IS DIRT F’ING CHEAP anyway:
http://www.tige...;Sku=TSD-500AS4

Oooouuu, look, we’ll sell you 100 GB’s on a $100 hard drive for $5000 a year with SOAP. We are sooooo smart, and you are soooo stupid. Plus you will have to pay the bandwidth costs to store the data blobs.

And oh look, we built a whole pyramid scheme out of it! Let’s have teh contest to prove that we are da pimps of Web 2.0 RPC API!

Amazon doesn’t get it. The reason facebook and other API is so popular is BECAUSE the data is light, because they are not storing huge blobs cross country wasting bandwidth. It’s impractical, slow, and stupid. Maybe somebody should have actually thought it through before doing that as a SOAP interface.