techno.blog("Dion")

I was really excited when the Google I/O folks told me that Flight of the Conchords were going to be playing at the event.

I saw one of the chaps do standup in New York city, and he was hilarious.

I better get a back stage pass guys!

Oh, and I am speaking at the event with Ben, so give us a shout.

I really liked getting the Ajax Language API out into developers hands as god knows we shouldn’t have to worry about translations. Now we can use the API and have the Google back-end do all of the work.

I have recently had a couple of scenarios where I really wanted a quick translation. I had a few twitter messages pass through my stream in French and Spanish. I had the answer to some technical issues show up on foreign forums.

So, I decided to create a Translate bookmarklet that allows me to select any foreign text, click on the bookmark, and a little window pops up with the English translation if it can work it out. Automatic translation is far from perfect yet, but for many scenarios you can easily get the gist (e.g. you wouldn’t want to automatically convert a book).

This is how I created the bookmarklet:

The source

First, I have the raw JavaScript source that will become the bookmarklet. There are a few sections of the code. First, we setup a method that will go off and call the Ajax Language API, passing in the translation and language that we want. This is where you would change the language code for non-English.

if (!window['apiLoaded']) {
  window.apiLoaded = function() {
    var language = "en";
    var text = window.getSelection().toString();
    if (text) {
      google.load("language", "1", { "callback" : function() {
        google.language.detect(text, function(dresult) {
          if (!dresult.error && dresult.language) {
            google.language.translate(text, dresult.language, language, function(tresult) {
              if (tresult.translation) {
                translationWindow(tresult, dresult);
              } else {
                alert('No translation found for "' + text + '" guessing the language: ' + dresult.language);
              }
            });
          }
        });
      }});
    }
  };
}

Then we setup a method that is able to display a window showing the result. I used the Prototype UI Window object if available, and good old alert() if not:

if (!window['translationWindow']) {
  window.translationWindow = function(tresult, dresult) {
    if (window['UI']) {
      new UI.Window({theme:  "black_hud",
                   shadow: true, 
                   width:  350,
                   height: 100}).setContent("<div style='padding:6px'>" + tresult.translation + "</div>")
                   .setHeader("English Translation")
                   .setFooter("Language detected: " + dresult.language)
                   .center({top: 20}).show();
    } else {
      alert(tresult.translation + " [lang = " + dresult.language + "]");
    }
  }
}

Next, we load the Prototype UI window code, and accompanying CSS resources by dynamically adding the resources to the DOM:

if (!window['UI']) {
  var pw = document.createElement('script');
  pw.src = 'http://almaer.com/downloads/protowindow/protowin.js';
  pw.type = "text/javascript";
  document.getElementsByTagName('body')[0].appendChild(pw);
 
  var pwdefault = document.createElement('link');
  pwdefault.setAttribute('rel', 'stylesheet');
  pwdefault.setAttribute('type', 'text/css');
  pwdefault.setAttribute('href', 'http://almaer.com/downloads/protowindow/themes/window.css');
  document.getElementsByTagName('body')[0].appendChild(pwdefault);
 
  var pwblack = document.createElement('link');
  pwblack.setAttribute('rel', 'stylesheet');
  pwblack.setAttribute('type', 'text/css');
  pwblack.setAttribute('href', 'http://almaer.com/downloads/protowindow/themes/black_hud.css');
  document.getElementsByTagName('body')[0].appendChild(pwblack);
}

Finally, we load the Google API loader, and use the dynamic loading option with the ?callback=apiLoaded. This kicks off the main driver that we saw first, and if it is already loaded we call it directly (for multiple translations on the same page).

if (!window['google']) {
  var s = document.createElement('script');
  s.src = 'http://www.google.com/jsapi?callback=apiLoaded';
  s.type = "text/javascript";
  document.getElementsByTagName('body')[0].appendChild(s);
} else {
  apiLoaded();
};

“Compilation”

This is the raw form, and we need to get the bookmarklet form, which you can just use right away if you are wanting English. For this, I use John Grubber’s makebookmarklet Perl script to do the conversion.

The Server

The Prototype UI code lives on the server, so I put a striped down version over there which just contains a combined Prototype + Window JavaScript file, and just the one theme CSS set.

In Action

Unsure what I am talking about? Just watch it in action:

UPDATE: I also implemented Twitter Translate to automatically convert tweets to your language.

It was quite fun to see, right after Tim O’Reilly pondered the lock in strategy for App Engine, that Chris Anderson posted that he had ported the SDK to App Drop.

Now, there are some concerns here as the SDK itself isn’t built for performance, security, etc…. but this is a great start in a very short period of time, and it shows where people can take it.

Waxy has a good write up:

This proof-of-concept was built in only four days and can be deployed in virtually any Linux/Unix hosting environment, showing that moving applications off Google’s servers isn’t as hard as everyone thought.

How does it work? Behind the scenes, AppDrop is simply a remote installation of the App Engine SDK, with the user authentication and identification modified to use a local silo instead of Google Accounts. As a result, any application that works with the App Engine SDK should work flawlessly on AppDrop. For example, here’s Anderson’s Fug This application running on Google App Engine and the identical code running on EC2 at AppDrop.

Of course, this simple portability comes at the cost of scalability. The App Engine SDK doesn’t use BigTable for its datastore, instead relying on a simple flat file on a single server. This means issues with performance and no scalabity to speak of, but for apps with limited resource needs, something as simple as AppDrop would work fine.

Chris said: “It wouldn’t be hard for a competent hacker to add real database support. It wouldn’t be that hard to write a Python adapter to MySQL that would preserve the BigTable API. And while that wouldn’t be quite as scalable as BigTable, we’ve all seen that MySQL can take you pretty far. On top of that, you could add multiple application machines connecting to the central database, and load-balancing, and all that rigamarole.”

And for data, you can write built-in services that export your data from the store.

I have to admit, that as someone on the inside it is nice to see the Google App Engine out there as a way to show some of the way in which we do things, especially scale.

One big difference you will see is the lack of a RDBMS, and instead with Bigtable, you build models that you can do cool things with such as Expando. Being able to add data elements as you iterate is very nice indeed, and beats SQL land, even with migrations and such.

Now when a new engineer comes to Google, they won’t have to entirely swallow a big red pill, as they may have gobbled a little of it already.

Jason Robbins and Jenan Wise have released a new Ajax source code browser for Google Code project hosting:

We recently launched a new source code browsing tool as part of Google Code’s project hosting feature. This new tool makes it easy to navigate through a project’s Subversion repository. Key features include: fast directory browsing tree, syntax highlighting, history of changes, and easy-to-read diffs. See it yourself under the “Source” tab of any project that we host.

The new system certainly feels fast, and uses jQuery to flip around in short order, jumping through the revisions of your system, expanding into new directories, etc.

This change required a change to my Google Code greasemonkey script that adds a direct link to the trunk. It now takes you right to the trunk of the code in the new browser!

How many applications ask for your Google username and password to get access to your contacts? A lot of new services offer the “feature” to map contacts on their service to your buddies. For example, you signup to Dopplr and want to map your contacts over.

Giving your username and password to your email is a Bad Thing ™, so we have wanted to put out an API that does what you really want (access to contacts) without opening up the entire farm (e.g. look at your email, or worse).

Sebastian Kanthak and his team have released the Google Contacts API:

It gives programmatic access to your contact list. The contact list is shared among Google applications like Gmail, Reader, Calendar, and more.

The Google Contacts Data API allows you to own your own contact data. We expect the API to be useful for a big range of applications. For example, developers can use it to:

• Import a user’s Google contacts into their web or desktop application
• Export their application’s contact list to Google
• Write sync applications for mobile devices or popular, desktop-based contact management applications

The Contacts API allows developers to create, read, update, and delete contacts using the Google Data protocol, based on AtomPub. It also allows for incremental sync by supporting the “updated-min” and “showdeleted” parameters. Please take a look at our documentation to see all the options supported.

We know that this Google Data API is the most requested feed by our developer community, so we’re very excited about this release. We are committed to actively work with you to improve the Google Contacts Data API and we’d like to hear back from you in our Google Contacts API group.

I know many people have been waiting for this, and I am excited. What a day for tech!

EveryBlock is a fantastic new service that is a “news feed for your neighbourhood”, created by Adrian Holovaty of Django and other fame (and others of course).

The mapping side of their site is a strong focus, and they talk about why they went down a somewhat custom route:

For the technology-inclined reader, here’s how we put our map platform together. All web technologies are a “stack,” like layers in a map, and our map stack starts at the top with OpenLayers, which provides the click’n’drag’n’zoom interface. The next step down in the stack is Mapnik, which renders geospatial data into graphical images. At the bottom of the stack is our collection of geospatial data, which includes TIGER/Line and shapefiles from various city governments, as well as databases and tools that operate on that data, such as PostGIS, Shapely, and OGR. Finally, the “secret sauce” layer in the stack is TileCache, which actually sits between OpenLayers and Mapnik. It provides caching of the map tiles that are generated by Mapnik and served to OpenLayers. I say “secret” because it’s transparent to both the layers it sits between, and also because it greatly speeds up the entire stack, making it possible to deploy and scale a large mapping application. It’s worth mentioning that all these layers of the stack are open source or licensed freely. If you can build and deploy a web application, you can build and deploy a complete standalone geospatial and mapping application. The tools and data are there for the taking.

Comments aren’t enabled on that entry, so I thought I would just post here. I am a little surprised at the characterization of Google Maps (and other services):

With Google Maps or any other web-based mapping service, we’d be limited to the color palette, typeface, and other design elements that service’s designers chose. While those maps can be handsome products, their choices aren’t our choices, and don’t mesh well with our site’s aesthetics.

I find this a touch confusing, as you can create custom map types (with layers that you want), and you can extend the UI with custom overlays, custom controls, and you basically end up with something that doesn’t look at all like maps.google.com, other than the Google logo.

You can see this in action with some examples. Below you see Runway Finder, and you can also check out other custom maps such as Map WOW, NY Subway.

For UI changes you can see Rotten Neighbor, Ace Hardware and Lonely Planet.

There are plenty of reasons to have to jump from Google Maps to your own solution, and this may be the right choice for Everyblock (they know a touch more about their application than me!). You can go further and further with the online mapping solutions right now and I would find it hard to not use them.

Jack Black’s movies can be friggin funny, or too weird for me.

I hope that Be Kind, Rewind is a good one, and at least I now know what Jack’s Google interface would be like…. I wonder what it would be like working for him ;)

Man I love it when I can delete code. Seeing the line count go away and leaving a small amount of text is a sight for sore eyes. I got to delete a lot of code today thanks to the kind folks at Aptana.

I recently wrote a shim that allows the Google Gears API to run as is on the server side. It wraps the Gears Database API with the Jaxer one.

What is particularly cool about this is that you can then write some code and:

• If the user has Gears installed, it runs client-side
• If the user doesn’t have Gears installed just run it on the server

I want to setup a nice way to make this trivial to setup.

The majority of the code was a simple wrapper around the result set, so Aptana decided to directly support the Gears API itself which allowed me to get rid of it all!

This makes the shim as simple as this:

// -- Wrap this code so it is available if using a proxy call
function oncallback() {
 
  // Make up the namespaces to mimic Gears and a place for Jaxer holders
  google = {}; google.gears = {}; google.gears.factory = {}; google.gears.jaxer = {};
 
  // Create sets up a database instance to be used
  google.gears.factory.create = function(className, version) {
    if (className.indexOf('database') < 0) {
      throw new Error('I can only do Database work right now');
    }
    return new google.gears.jaxer.Db();
  }
 
// -- The Database Wrapper
  google.gears.jaxer.Db = function() {
    this.db = null;
  }
 
  google.gears.jaxer.Db.prototype.open = function(name) {
    this.db = new Jaxer.DB.SQLite.Connection({
      PATH: 'resource:///../data/' + name + '.sqlite',
      CLOSE_AFTER_EXECUTE: 'open'
    });
  }
 
  google.gears.jaxer.Db.prototype.execute = function(sqlStatement, argArray) {
    var rs = (argArray) ? this.db.execute(sqlStatement, argArray) : this.db.execute(sqlStatement);
    return rs;
  }
 
}

Philip Maker has also taken GearsORM and made it work with Jaxer. Very cool indeed.

As soon as I started to play with Aptana Jaxer, I saw an interesting opportunity to port the Google Gears Database API (note the Gears in the logo!)

If I could use the same API for both client and server side database access, then I can be enabled to do things like:

• Use one API, and have the system do a sync from local to remote databases
• If the user has JavaScript, use a local database, else do the work remotely
• Share higher level database libraries and ORMs such as Gears DBLib for use on server side data too

I quickly built a prototype to see if this would all work.

The Jaxer shim of the Gears API was born, and to test it out I took the database example from Gears itself and made it work.

To do so, I only had to make a few changes:

Run code on the server

I changed the main library to run on the server via:

<script type="text/javascript" src="gears_init.js" runat="server"></script>

I wrapped database access in proxy objects, such as:

function addPhrase(phrase, currTime) {
  getDB().execute('insert into Demo values (?, ?)', [phrase, currTime]);
}
addPhrase.proxy = true;

This now allows me to run the addPhrase code from the browser, and it will be proxied up to the server to actually execute that INSERT statement.

This forced me to separate the server side code from the client side code, which is a better practice anyway, but it does make you think about what goes where. In a pure Gears solution I can put everything in one place since it all runs on the client.

Create the new gears_init.js

A new gears_init.js acts as the shim itself. Instead of doing the typical Gears logic, it implements the Gears database contract. This wasn’t that tough, although there are differences between the Gears way, and the Jaxer.DB way. The main difference is to do with the ResultSet implementation, where Gears goes for a rs.next()/rs.field(1) type model versus the Jaxer.DB rs.rows[x] model.

I actually much prefer Gears DBLib as it hides all of that, and just gives the programmer what he wants… the rows to work on.

oncallback magic

In the current Jaxer beta, I ran into an issue where I wanted the Gears library to just “be there” for any proxy requests.

You have to think about the lifecycle of a Jaxer application, and the documentation tells you what you need to know to work around the issue.

In this case, I wrapped the code in:

function oncallback() {
  // create the wrapper here
}

This is less than idea, and Aptana is playing with nice scoping which would enable you to just say “hey, load this library once and keep it around for the lifetime of the server | application | session | page”. That will be very nice indeed.

You can do a little bit of this by opening up your jaxer_prefs file and adding the resource for your file:

// This option sets up an html document that will be loaded
// everytime a callback is processed.  This has to be a local file.
// If not specified, an empty document will be loaded.
// pref("Jaxer.dev.LoadDocForCallback", "resource:///framework/callback.html");

Future…

This is just the beginning. As I mentioned at the beginning, I am interested to see where you can take this to handle clients who do not support JavaScript, and also to deal with synchronization with minimal code (sync from local to remote with exactly the same SQL API).