techno.blog("Dion")

I recently built an example of the Form History Pattern using an Addressbook case study.

I found myself talking about App Engine on the On Air tour, so I decided to change the example to not store the data locally with Gears, but instead to save it away into the cloud via App Engine.

Why did people want to hear about App Engine at an Adobe conference? I think that Jonathan Schwartz got it when he mentioned rich internet back-ends. As you build rich clients, you suddenly realise that the promised benefits of web services can kick into gear nicely.

Back to the sample. The architecture change was quite simple. Where I was doing a local DB save, I would call a back-end service such as /loadcontacts or /savecontact depending on the task.

You can view and download the full App Engine project code for this sample, and you can see it running live on App Engine

Check out the video below, or view the high quality version (recommended) to see the code walk through, and the various tools that App Engine gives you to develop, debug, and monitor your application running at a few thousand feet.

Nick Johnson, of Google, has created a nice bridge service smtp2web.com: Allow App Engine apps to receive email.

This is perfect timing, as after my App Engine talk in Prague, a nice gent came up to me and asked for just this. He wanted to process email in his application and didn’t think he could.

One shortcoming of purely HTTP-based webapps such as App Engine is
that they can’t receive email. I know that some people are wanting to
create App Engine apps that do this, so I put together a service to
facilitate this. It runs as a standard SMTP server, and sends all the
email it receives via HTTP POST to a user-designated URL. It’s free to
use (but not to abuse)

To use the service, you setup an email account (say [email protected]) and a URL to point too. Of course, if you want an email on your own domain you could alias [email protected] to [email protected]).

Then, you would configure an App Engine controller to receive the email contents:

from google.appengine.ext import webapp
import email
 
class EmailHandler(webapp.RequestHandler):
  def post(self):
    sender = self.request.GET.get("from", "")
    recipient = self.request.GET.get("to", "")
    message = email.message_from_string(self.request.body)
    # Do stuff with the email message

The code for the service has also been released as open source on Google Code so you can check it out. You will find that it runs as a Twisted application:

from twisted.application import service
from twisted.application import internet
from twisted.enterprise import adbapi
 
import sys
import os
sys.path.append(os.path.dirname(__file__))
 
import smtp2web
 
application = service.Application("smtp2web Service")
 
settings = smtp2web.Settings(secret_key="<enter secret key here>",
                             state_file="state", master_host="localhost:8081")
 
smtpServerFactory = smtp2web.ESMTPFactory(settings)
smtpServerService = internet.TCPServer(2025, smtpServerFactory)
smtpServerService.setServiceParent(application)

Thanks for doing this Nick!

NOTE: Mailhook is another service (pay) that does something similar

It was quite fun to see, right after Tim O’Reilly pondered the lock in strategy for App Engine, that Chris Anderson posted that he had ported the SDK to App Drop.

Now, there are some concerns here as the SDK itself isn’t built for performance, security, etc…. but this is a great start in a very short period of time, and it shows where people can take it.

Waxy has a good write up:

This proof-of-concept was built in only four days and can be deployed in virtually any Linux/Unix hosting environment, showing that moving applications off Google’s servers isn’t as hard as everyone thought.

How does it work? Behind the scenes, AppDrop is simply a remote installation of the App Engine SDK, with the user authentication and identification modified to use a local silo instead of Google Accounts. As a result, any application that works with the App Engine SDK should work flawlessly on AppDrop. For example, here’s Anderson’s Fug This application running on Google App Engine and the identical code running on EC2 at AppDrop.

Of course, this simple portability comes at the cost of scalability. The App Engine SDK doesn’t use BigTable for its datastore, instead relying on a simple flat file on a single server. This means issues with performance and no scalabity to speak of, but for apps with limited resource needs, something as simple as AppDrop would work fine.

Chris said: “It wouldn’t be hard for a competent hacker to add real database support. It wouldn’t be that hard to write a Python adapter to MySQL that would preserve the BigTable API. And while that wouldn’t be quite as scalable as BigTable, we’ve all seen that MySQL can take you pretty far. On top of that, you could add multiple application machines connecting to the central database, and load-balancing, and all that rigamarole.”

And for data, you can write built-in services that export your data from the store.

I have to admit, that as someone on the inside it is nice to see the Google App Engine out there as a way to show some of the way in which we do things, especially scale.

One big difference you will see is the lack of a RDBMS, and instead with Bigtable, you build models that you can do cool things with such as Expando. Being able to add data elements as you iterate is very nice indeed, and beats SQL land, even with migrations and such.

Now when a new engineer comes to Google, they won’t have to entirely swallow a big red pill, as they may have gobbled a little of it already.

Since I am still in London, my sarcasm quotient extends just a little more than usual (it’s pretty high in the US to start with, even if it is the “lowest form of humour”).

I had a fair amount of email from people, and saw some blogs, complaining about the fact that Google App Engine currently supports Python only. On the video, and in the docs, it is frequently mentioned that other languages are to come, and that the infrastructure itself is language neutral. We have to start somewhere!

I personally have a slight Ruby preference, but there is a funny thing about Python. Hardly anyone really hates it. It may not be the language of choice for many people, but a lot of people are telling me:

• Ah, was hoping to try more Python
• I was looking for an excuse to code a Django app
• I can deal with that. Thank god you didn’t force Java

This is a little like the iPhone SDK being Objective-C. People are picking it up. I think this happens then your desire to play with the new toy is greater than your religion!

I am very excited to see more languages and features on Google App Engine, but I know that I just have to be a bit patient, and in the meantime, I had been looking for an excuse to code a Django app.