I’ve been playing around with Google App Engine and wanted to share my first impressions. First and foremost I’d like to thank a Googler out there who got me into the App Engine beta. I’m not sure he wants it public that he got me in because I didn’t sign up for App Engine quick enough; I was put on a waiting list, posted on Twitter about it, and he got me an invite. Thank you very much!
For those of you who do not know about App Engine it is a way to run web applications on Google’s infrastructure. It throws a lot of ready made stuff at you including a web framework and a method of using Google accounts within your application. But the real draw is you get to use Google’s Bigtable through the Datastore API. Anyone building a web application knows that more often than not the database (and memory) is the limiting factor. With App Engine (assuming your queries are relatively simple) that is no longer an issue.
So I ignored everything other than the Datastore API and set to work porting FriendFeed Stats over to App Engine. FriendFeed Stats was already written in Python (specifically Django) so it was the simple matter of converting the models (StringProperty instead of CharField, IntegerProperty instead of IntegerField, DateTimeProperty instead of DateTimeField…you get the idea) and the queries (Service.all().order('name') instead of Service.objects.order_by('name') and queryset.filter('ffuser =', ffuser) instead of queryset.filter(ffuser = ffuser)…very simple) to use the Datastore API instead of the Django ORM. This only took a few minutes really. If you understand any ORM you should have no trouble with this.
Next I had to play with the FriendFeed API because it uses urllib2 and sockets. Sockets are a no-no on App Engine. Instead you use Google’s URL Fetch API. This took a little longer to debug and still isn’t perfect yet. I haven’t gotten POSTs working but I haven’t really tried because FriendFeed Stats doesn’t POST anything to the FriendFeed API.
And that is all it took to get FriendFeed Stats ported over to App Engine. To get the proper indexes I just ran the dev_appserver and clicked around which will auto generate an index.yaml file for you. To get around the fact that I don’t have cron anymore I switched to updating by accessing a “hidden” url. My server uses curl and cron to hit that url which causes App Engine to use the FriendFeed API and update the site; convoluted but effective.
I think Google has a real winner here. My only complaint so far is that count queries are limited to returning 1000. I understand that other queries need to be limited to returning 1000 rows (this is a free service after all) but limiting count seems kind of silly and makes FriendFeed Stats global pages just about wortheless. Any query that should have resulted in well over 1000 (like the Twitter service) is returning 1001 right now; I’ve submitted a report and hope that this is fixed.
