« Previous post
Next post »

April 8, 2008

TechCrunch Labs: Our Experience Building And Launching An App On Google App Engine

Henry Work

100 comments »

Last night, Google announced App Engine, a hosted web application platform. We’ve now tested the service directly by writing and deploying a test application called appengine.crunchbase.com—a HotorNot popularity contest for startups. Our experience with building and launching an app is below.

Google promises developers two things with App Engine: to reduce the time from writing code to deploying it on a web server, and to leverage Google’s massive infrastructure. We decided write a simple app on the platform, deploy it, and get some traffic to really see how easy App Engine is.

Getting Started

Looking at the developer documentation, App Engine boasts a powerful API. The platform comes with a Python scripting runtime, static file serving capabilities, easy and tight integration with Google user accounts and email services (obviously a big play), simple access to a powerful persistence engine with queries and transaction support (aka a really good database), near real-time site monitoring and statistics, and the promise of consistent high performance and essentially linear scalability.

Despite its potential power and underlying sophistication, App Engine was surprisingly easy to get started with. Now, being Ruby on Rails guys, the fact that the only currently supported language is Python was a bit humbling. Still, the SDK provided by Google proved dead simple. Our first application was contained within a single Python script, making the path the code was taking to produce responses very clear. We incrementally added features: first accepting input from the users through forms, then storing and retrieving that data using the provided persistence API, and finally breaking out the user-facing web page code into a proper Django template.

The App

The web app we came up with is super-simple, shamelessly self-promoting, and easily game-able. It’s a one-page voting site using company logos slurped in from CrunchBase, our main project. The app has only two requests: one for rendering the page, the other for recording a vote (one GET, one POST request). The first real issue was getting our initial data, some URLs and names, into the database. While we later found that Google provides a slick bulk-update tool, we got going with a simple action that manually parsed our comma-separated values into the database. Another thing we wanted from the service was it to be served from our own domain. By default, Google hosts all App Engine projects on your-project.appspot.com (think blogspot), but they do offer domain services through Google Apps, which was pretty painless to setup if you just want App Engine functionality.

Deployment

The SDK provides a server that emulates the App Engine platform, making it possible to easily develop applications locally that will later deploy to Google’s cloud. Once we had a presentable first app coded up and tested locally, we deployed the app to Google’s server easily from the command line. This was particularly compelling; we’ve spent hours or even days deploying web apps to comparatively trivial servers.

Overall, the process from sign-up to deployment took about 4 hours, with the vast majority of that figuring out what we wanted to do and remembering how to do things like sort arrays in Python (we also spent an embarrassing 15 minutes on some poorly formatted hidden HTML fields). The rapid prototyping of the app and the ease of deployment is clearly the real power of App Engine right now. The scalability of App Engine is exciting but elusive; most apps won’t need it for a while. It’ll also be interesting to see how truly scalable it is, considering the linear scaling efforts on Amazon EC2. The redundancy and ease of deployment should be the immediate attraction for developers.

Try our app out: appengine.crunchbase.com.

Mark McGranaghan took the lead on coding for the site and contributed heavily to this article.

Update: Looks like our app was a little bit too popular:

Update #2: Back up now. Great success!

  • Sphere It

This entry was posted on Tuesday, April 8th, 2008 at 11:36 am and is filed under Company & Product Profiles. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

Trackbacks/Pings (Trackback URL)

  1. vrypan|net|weblog » Google App Engine

    April 8th, 2008 at 11:48 am

  2. Google Jumps Head First Into Web Services With Google App Engine

    April 8th, 2008 at 11:52 am

  3. Google Jumps Head First Into Web Services With Google App Engine

    April 8th, 2008 at 11:52 am

  4. appengine.crunchbase.com at AntyWeb

    April 8th, 2008 at 12:09 pm

  5. Google App Engine - GamersHell.com Community Forums

    April 8th, 2008 at 2:03 pm

  6. TechCrunch Japanese アーカイブ » TC体験ルポ「Google App Engineでアプリを作って公開してみた」

    April 8th, 2008 at 2:31 pm

  7. Why the web is doomed: The Riffraff are Coming! | Catalyze This

    April 8th, 2008 at 2:36 pm

  8. IdleTogether - Technology, design and impressive web applications » Blog Archive » Python to be the future of web languages?

    April 8th, 2008 at 5:51 pm

  9. Google gets all Amazony | My Slice of the Globe

    April 8th, 2008 at 6:57 pm

  10. Lushable - Technology news, web 2.0 start up news » Blog Archive » Google App Engine under minor turbulences

    April 8th, 2008 at 7:00 pm

  11. Quote by TechCrunch Labs: Our Experience Building And Launching An App On Google App Engine | Oliver's Stuff

    April 9th, 2008 at 2:25 am

  12. Первое приложение на Google App Engine | Vladimir Prudnikov

    April 9th, 2008 at 2:30 am

  13. links for 2008-04-09 « Breyten’s Dev Blog

    April 9th, 2008 at 4:31 am

  14. Isn’t Family Important? » position:relative - a Web and Now blog

    April 9th, 2008 at 10:29 am

  15. Juixe TechKnow » Google App Engine Analysis

    April 11th, 2008 at 4:07 pm

  16. nortypig » Blog Archive » Building on Google App Engine

    April 11th, 2008 at 10:11 pm

  17. Aspettatemi! « MasciBlog

    April 12th, 2008 at 3:55 am

  18. » Google App Engine et Ruby - RubyZness - Ruby on Rails, business et entreprises - Blog Archive

    April 14th, 2008 at 9:49 pm

  19. Techcrunch - Internet Marketing Sweetie - Google - Free Line Report 4.15.08

    April 15th, 2008 at 12:07 pm

  20. Google App Engine « The Multiplatform Game

    April 15th, 2008 at 2:48 pm

  21. Nome do Jogo » Artigo » Rails Podcast Brasil - Episdio 13

    April 16th, 2008 at 6:37 am

  22. Anonymous

    April 17th, 2008 at 3:52 am

  23. (中文版 Chinese version) Techcrunch - Internet Marketing Sweetie - Google - Free Line Report 4.15.08

    April 17th, 2008 at 9:18 pm

  24. New Google Web Services: Good News for Developers | TrendStarks.com

    May 8th, 2008 at 5:19 am

Comments

RSS feed for comments on this post.

  1. Hassan Hodges

    April 8th, 2008 at 11:40 am

    I’d love to see a skip button on there too.

  2. michael @ qik

    April 8th, 2008 at 11:44 am

    Fantastic stuff, Henry! It’s really useful seeing this from you guys, and I’m amazed it happened so quickly. Bravo.

  3. Michael Arrington

    April 8th, 2008 at 11:44 am

    Hassan - I doubt we’ll be doing much in the way of feature enhancements on the app. The point was just to test the platform. :-)

  4. Nik Kalyani

    April 8th, 2008 at 11:48 am

    It would be interesting to see an app that simulates load and how the performance compares on Google vs. Amazon.

  5. karthik

    April 8th, 2008 at 11:51 am

    Henry/Mike

    One thing that none of the blogs , TC included , addressed is the fact that GAE doesn’t permit the installation of any standalone software. Lets say you need to install a software that is not run on the app server but as a standalone component. EC2 would allow for this but not GAE. This might be a serious limitation.

  6. Todd

    April 8th, 2008 at 11:52 am

    That app is actually kind of fun. Hot or Not for startups.

  7. Please

    April 8th, 2008 at 11:55 am

    This is probably one of Google’s infamous 20% spare-time projects led by some technical guru with absolutely no business sense.

    No wonder people are leaving Google.

  8. Jorge

    April 8th, 2008 at 11:56 am

    App is cool. Agree that a skip option would be nice (i guess refresh works too). Awesome that you got it up so fast and wrote about it.

  9. FigureB

    April 8th, 2008 at 12:01 pm

    Are the 404 errors a result of Google apps or your programming? I am just curious.

  10. Sam Purtill

    April 8th, 2008 at 12:01 pm

    Mark/Henry, good work and thanks for the write up. Will be messing with Google App Engine later this week.

  11. Henry Work

    April 8th, 2008 at 12:10 pm

    Qik Micheal, Todd, Sam, Jorge: cheers.

    karthik: definitely a limitation. it exposes the differences of the approach.

    FigureB: not getting any 404 errors. I of course blame Google for all issues.

  12. Not Elder

    April 8th, 2008 at 12:11 pm

    “The scalability of App Engine is exciting but elusive; most apps won’t need it and those that do won’t use Google.”

    Can you elaborate why apps needing scalability won’t use Google? Thanks!

  13. Alan Wilensky

    April 8th, 2008 at 12:12 pm

    This is the single biggest service, bar none, that Techcrunch has ever contributed to the readership and community at large. It makes the preceding body of journalistic work pale.

    By just going and using the service, and clearly writing about the experience, you, sir, have distinguished yourself as unique from the awful pundits that are writing great fictions and strategy pieces about Amazon vs. google platforms.

    I am getting back to active programming after years of working the Product management side, and now it seems that I must add Python to the Ruby books.

    I would highly recommend that shops and developers seeking to deliver w20 apps to the small and med business community also look at http://Keroku.com.

    Thanks ever so much, again.

  14. Boring Market

    April 8th, 2008 at 12:12 pm

    I have been playing around with it for a minute and the number of votes when from 8,000 to 10,000. Is this a test to see if the new app engine can handle it? (its obvious it can handle it, if you have read Matt Cutts blog)

  15. Erick Schonfeld

    April 8th, 2008 at 12:13 pm

    This app is addictive, Henry. Great job. Maybe we should make it a real app after all. But we’ll need a name. Suggestions?

  16. Boring Market

    April 8th, 2008 at 12:18 pm

    I think you should make it a real app. Put limits on if you have voted for it, you can’t vote for it again. Maybe make it so you compare competitors. Call it ‘Start Down”.

  17. Henry Work

    April 8th, 2008 at 12:20 pm

    Thanks Erick: we probably should have a name.

    Also, we are also at 20% of our CPU quota for the day… uh oh. Might need to call Google.

  18. Henry Work

    April 8th, 2008 at 12:24 pm

    Anyone have any good name suggestions?

  19. Boring Market

    April 8th, 2008 at 12:25 pm

    I was wondering about this…5,000 votes in 10 minutes? I think you caught on to something. You should post stats from today in the next few days.

  20. Cameron Watters

    April 8th, 2008 at 12:25 pm

    Looks like somebody figured out how to game the thing and is pumping Mahalo.

  21. ppp

    April 8th, 2008 at 12:27 pm

    why do you guys hate pay per post so much. its hard for me to believe that TC users actually voted it down to be the worst. not that i care much, but just a thought.

  22. Boring Market

    April 8th, 2008 at 12:27 pm

    Cameron, You’re right. Someone twitter Jason Calacanis to stop.

  23. rh

    April 8th, 2008 at 12:28 pm

    How about startupstartdown? or startupdown? or sud?
    or the buttons could be ” ‘Sup ” (for up / hot) or ” ‘Sud ” (for down / not)
    I agree with 13, that you guys just distinguished yourselves, and if Henry Work did this on his own initiative, well, he deserves a raise.

  24. ZiZi

    April 8th, 2008 at 12:31 pm

    Michael, I would actually keep this app around if I was you, i just spent 30 minutes looking at startups.. a skip feature is a must though.

  25. Jesse Farmer

    April 8th, 2008 at 12:34 pm

    My suggestion: Fail OR Not

  26. michael @ qik

    April 8th, 2008 at 12:35 pm

    Think it just got hacked — blinkx is on a tear.

  27. michael @ qik

    April 8th, 2008 at 12:37 pm

    Looks like someone wrote a script to call the down-post for a given hidden key value over and over.

  28. Henry Work

    April 8th, 2008 at 12:38 pm

    I like startupdown… Fail or Not is solid too. It’s totally getting gamed now… which is understandable, because it’s really easy to game (the code we use is above). Someone must really dislike blinkx.

  29. Khang from Simplebucket

    April 8th, 2008 at 12:43 pm

    CrunchVote

  30. heri

    April 8th, 2008 at 12:46 pm

    how come total votes was reset to 0? did you restart the app?

    and don’t you save already voted startups? i keep getting the same startups

  31. David

    April 8th, 2008 at 12:47 pm

    I too would love to see updated screenshots of the dashboard. Thanks, this is a great post/app.

  32. Henry Work

    April 8th, 2008 at 12:49 pm

    Hup! Looks like the voting data got reset. Probably our fault.

  33. Michael B

    April 8th, 2008 at 12:51 pm

    That is a good idea, I was hoping they would make one of these.

  34. CG

    April 8th, 2008 at 12:54 pm

    Fun stuff. I applaud Goog’s efforts to further trivialize application development and scaling. This is a natural fit for a host-less facebook, opensocial and other widgety apps. Someone please tell rackspace their days are numbered.

    Fwiw, I clicked a few times & encountered this (no, I won’t say who I voted down):

    Traceback (most recent call last):
    File “/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py”, line 486, in __call__
    handler.post(*groups)
    File “/base/data/home/apps/techcrunch-demo/1.12/vote.py”, line 43, in post
    logo.down_votes = logo.down_votes + 1
    AttributeError: ‘NoneType’ object has no attribute ‘down_votes’

    Ok on second thought, don’t short rackspace yet.

  35. tony jarrenberg

    April 8th, 2008 at 12:55 pm

    2.44 AM - http://appengine.crunchbase.com/ is down

    is this the high reliability of google’s platform ?

  36. diystartupnews.com

    April 8th, 2008 at 12:56 pm

    I’d like to see the stats after 24 hours, traffic, data usage etc.

  37. Fixed

    April 8th, 2008 at 12:58 pm

    Your app is fixed to select a preferred selection…
    Its not random

  38. Michael Arrogantington

    April 8th, 2008 at 1:05 pm

    So this means I can vote for chunchbase 100 times?

  39. Henry Work

    April 8th, 2008 at 1:08 pm

    Fixed: no. Although the pool of companies is only 100 large (and that pool is fixed), we use the randrange() function (which presumably is pretty random).

  40. Fixed

    April 8th, 2008 at 1:10 pm

    yes, what a good idea, block me on Akismet while your at it

  41. Fixed

    April 8th, 2008 at 1:10 pm

    As I said the pool is fixed, why is that?????

  42. Joel

    April 8th, 2008 at 1:10 pm

    I agree, the app is kind of addictive. You should flesh it out and make it an actual app, it could be interesting

  43. Anon

    April 8th, 2008 at 1:14 pm

    Nice sample, I too got a 404 after voted down a company for the second time… will be interesting so see the stats when you max out the resources for the day.

  44. RBA

    April 8th, 2008 at 1:15 pm

    “Server Error”. And when it works, it’s terribly slow right now.

    Google Apps got Techcrunched? :-) How you like that? :-)

  45. Henry Work

    April 8th, 2008 at 1:16 pm

    Fixed: easier to implement. It’s just the first 100 companies in CrunchBase. I think it’s more interesting to do it this way than to have thousands of companies to choose from.

  46. Joe Arson

    April 8th, 2008 at 1:21 pm

    Looks like this isn’t as scalable as Google hoped it would be.

    Back to Amazon!

  47. John

    April 8th, 2008 at 1:23 pm

    Somebody needs to run some benchmarks. The site feel very very slow. Slower than 90% of websites out there. I have a lag of at least a full second when I submit a vote. Anybody else having this slow experience? If true, this would absolutely kill the idea of using AppEngine.

  48. Pankaj

    April 8th, 2008 at 1:23 pm

    i was hoping for an easy coghead like tool :(

  49. grah

    April 8th, 2008 at 1:30 pm

    This is easily one of the best posts by you guys this year.

  50. ;)

    April 8th, 2008 at 1:34 pm

    It’s just showing us simple logos and I just got a timeout server error :(

  51. Matt

    April 8th, 2008 at 1:59 pm

    Looks like lots of us are experiencing timeout and hipcuts, would be curious to know if it’s techcrunch or google’s fault :)

  52. Greg Duffy

    April 8th, 2008 at 2:13 pm

    I think you need to cache your handlers.

    http://code.google.com/appengi.....ching.html

  53. Robert

    April 8th, 2008 at 2:23 pm

    justin.tv and scribd are pounding the site

  54. Matthew

    April 8th, 2008 at 2:45 pm

    Others have already said this, but I’m going to say it as well:

    Great post Henry. TechCrunch needs more hands-on user experience articles like this one. I enjoyed the read.

  55. Sean

    April 8th, 2008 at 2:49 pm

    Agree, great post, thanks. It would be interesting to read the development of a more complex app too, to see what kind of annoying limitations people will be running into when trying to use this for something that’s actually useful. (Not meant as anything bad towards the current app, but it was obviously just for testing!)

  56. Henry Work

    April 8th, 2008 at 2:58 pm

    Just letting everyone know: we did put in some gaming protections and lower the vote counts a little bit.

  57. Michael Arrington

    April 8th, 2008 at 3:43 pm

    Henry - forget the gaming. get back to coding crunchbase. :-)

  58. Frank Church

    April 8th, 2008 at 4:04 pm

    I like “Fail or Sail”, but I guess this test-app doesn’t get the bosses approval for time allocation. Funny to see who bothered gaming, nice work.

  59. friarminor

    April 8th, 2008 at 4:05 pm

    Until then, we’d be keeping with our Rails apps and hosting them on theMorph Appspace.

    AWS and GAE sure make it more interesting.

    Best.
    alain

  60. ET

    April 8th, 2008 at 4:35 pm

    Your tiny app with its limited functionality brings up an amusing problem… I hope you don’t need all that much uptime.

    “Over Quota
    This Google App Engine application is temporarily over its serving quota. Please try again later.”

  61. John

    April 8th, 2008 at 4:47 pm

    I’m interested to see how fast it took you to meet the ~200k gigacycles

  62. Adam

    April 8th, 2008 at 4:54 pm

    yeah. “over quota”

    looks like it ain’t so scalable just yet.

  63. Ruslan

    April 8th, 2008 at 5:41 pm

    I agree with #7. I can’t express level of my disappointment seeing the way it was implemented. Why Python instead of opening up API via REST as all other Google services ? Why this ridiculous concept of uploading of my application into Google’s virtual application server ? I don’t know what exactly Google App Engine team is smoking but sure enough I wanna buy it too. So far it is far away from convenience of Amazon Web Services.

    Shame.

  64. John

    April 8th, 2008 at 6:52 pm

    I’m disappointed with app. It is simply a hack to have your CGIs to run on a remote machine. It lacks any innovation that we see in other Google tools.

  65. Ric Johnson

    April 8th, 2008 at 7:02 pm

    Great post.
    I asked to be a user last night, but got put on a wait list. WTF? How do I get to test this out? Anyone got an invite?

  66. Michael Arrington

    April 8th, 2008 at 7:55 pm

    John - you’re banned from techcrunch for stupidity.

  67. NHKC

    April 8th, 2008 at 8:36 pm

    app will not work, FAIL.

  68. Sachin Tendulkar

    April 8th, 2008 at 8:41 pm

    I want an account too! When are they going open fully?

  69. Hendra

    April 8th, 2008 at 8:42 pm

    Let’s just say your app is up and running and you decided to leave Google and host it yourself, will that be possible? Or is it a case of Hotel California’s you can check in but you can’t check out?

  70. Alan Wilensky

    April 8th, 2008 at 8:46 pm

    People, people:

    It’s not how the app performs at this early stage, it was henry’s experience in getting it running and deployed, and how well he told the tale in such an economical way. The story of the google cloud has not even begun.

    A year from now, this is going to be the shiz. As far as Python as the initial language, it could have been worse.

  71. Henry Work

    April 8th, 2008 at 9:53 pm

    Hendra: interesting question. In their talk they certainly broach the ‘how does an existing app port over to App Engine’ question, but the reverse is less clear. I don’t immediately see any db exporting stuff.

  72. anon

    April 8th, 2008 at 11:36 pm

    got this error while uploading

    Traceback (most recent call last):
    File “C:\Program Files\Google\google_appengine\appcfg.py”, line 48, in
    execfile(script_path, globals())
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 1530, in
    main(sys.argv)
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 1526, in main
    AppCfgApp(argv).Run()
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 1201, in Run
    self.action.function(self)
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 1405, in Update
    lambda path: open(os.path.join(basepath, path), “rb”))
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 1049, in DoUpload
    missing_files = self.Begin()
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 910, in Begin
    version=self.version, payload=self.config.ToYAML())
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 264, in Send
    self._Authenticate()
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 301, in _Authenticate
    super(HttpRpcServer, self)._Authenticate()
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 214, in _Authenticate
    auth_token = self._GetAuthToken(credentials[0], credentials[1])
    File “C:\Program Files\Google\google_appengine\google/appengine/tools/appcfg.py”, line 159, in _GetAuthToken
    response = self.opener.open(req)
    File “C:\Python25\lib\urllib2.py”, line 381, in open
    response = self._open(req, data)
    File “C:\Python25\lib\urllib2.py”, line 399, in _open
    ‘_open’, req)
    File “C:\Python25\lib\urllib2.py”, line 360, in _call_chain
    result = func(*args)
    File “C:\Python25\lib\urllib2.py”, line 1115, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
    File “C:\Python25\lib\urllib2.py”, line 1082, in do_open
    raise URLError(err)
    urllib2.URLError:

    C:\Program Files\Google\google_appengine\demos\helloworldsubin>

  73. Christian Knott

    April 9th, 2008 at 7:41 am

    A couple of points that I hope covers some of the misunderstanding out there:

    The Google platform is very much Beta. Of course they want people beating it to death.

    I believe that this is strategic for Google. Sure the tools aren’t there, you can’t install “x” where “x” is your current tool du jour, but just as with Amazon’s EC2, the middleware and tool-smiths will be there very rapidly.

    It comes down to this: do you want to maintain racks of servers, or pay someone else to host that, or do you want to use companies that have proven they manage 100% uptime on, for all practical purposes, an infinite server base?

    As chief “yelling at people about the cloud” geek at Alpha Software, I’ve been playing with EC2 for awhile, and I’ll be playing with Google’s AppEngine asap.

    The cloud, IMHO, is the future, whether you grok it now or not.

  74. frank

    April 9th, 2008 at 9:26 am

    Any chance in releasing the code to your demo app for those that want to get a peek at how it all fits together?

  75. francis dupuis

    April 9th, 2008 at 9:27 am

    I’m with Christian, running server farms isn’t what our company wants to be in business doing. Obviously this particular service needs to mature but there is a ton of potential here.

  76. Milan Cole

    April 16th, 2008 at 4:36 pm

    It’s just a matter of time before this matures. Releasing alpha/beta products is part of their MO and I’m sure they will be adding commercial pricing before long.

« Previous post
Next post »