Source: Google To Launch BigTable As Web Service

Google may be releasing BigTable, its internal database system, as a web service to compete with Amazon SimpleDB, according to a source with knowledge of the launch. There are also rumors that press is being pre-briefed on the product, although we haven’t been contacted by Google.

BigTable is a highly scalable database system used internally by Google to support over 60 of its products and projects. A source says Google has plans to announce next week that it will make BigTable available to outside developers as a service. Amazon provides a similar service through SimpleDB, a cloud database solution announced in December.

Google started development on BigTable in early 2004 and began using it actively in February 2005. The non-relational, proprietary system was designed internally to fulfill Google’s peculiar need for access to massive amounts of data at very high speeds (millions of read/writes per second). BigTable is based on the Google File System (GFS) and designed for distribution across thousands of commodity servers that collectively store petabytes of data. Services that rely on it include Google Search, Google Earth and Maps, Google Finance, Google Print, Orkut, YouTube, and Blogger.

The decision to open up BigTable would seem to mark Google’s challenge to Amazon Web Services (AWS) suite, which also includes the Elastic Compute Cloud (EC2) for cloud processing power and Simple Storage Service (S3) for cloud storage. The Amazon triumvirate of SimpleDB, S3, and EC2 is meant solve the scalability needs of web developers with a utility-like model. Customers pay for just the storage, computations, and bandwidth they need, and none they don’t. While Google has yet to announce the pricing for BigTable, we presume it will share the same model as AWS.

If Google does indeed announce public access to BigTable next week, expect the company to follow up with cloud storage and processing solutions as well, since there are substantial synergies between the three.

For more information about BigTable, see a paper (PDF) that was written about it in 2006. You can also watch a talk about it given at the University of Washington in October 2005.