techno.blog("Dion")

I am having a lot of fun with the AJAX Language API. Last week I talked about the translation bookmarklet that lets you translate anything that you select in the browser.

This time, I whipped up Twitter Translate, which watches the tweets in the page, and if the content isn’t your native tongue automatically converts it and replaces it inline. It then adds a mini logo with “translated from …” which you can click on to see the original text:

It is probably easiest to quickly see it in action:

I think that I am so excited about this API as it is a vertical service that you can just use for free. Think back on how you would be able to integrate language into your applications in the past. You would either:

• Have to work on the discipline yourself, which is crazy if it isn’t your core business
• Find a vendor that has some product that you can use, all of which are very expensive.

Now, we can use this service for free, and the best part? It will keep getting better behind the scenes without us having to do a thing!

Every now and then, normally when talking to a libertarian, I think about how we are actually all connected to each other. It is impossible to sandbox yourself from society which leads me to conclude that I need to embrace it and do what I can to work out what kind of society we want to be.

With the current Ext JS debacle, you get reminded of how connected your project and business are to other people. Just because you own a company, doesn’t mean that you control it. When I think about my own company, Google, I realize that the most important currency is user trust. It doesn’t matter how many PhDs and great technology releases we have, if we ever lost some of the trust. I think that Google has earned its reputation, but all it would take is something that goes against what we have stood for so far, and we could lose it just as fast. I actually like this fact, as it keeps us honest.

It is a little like your tennis ranking. A rolling year of past performance is what really matters here. It doesn’t matter if you won that grand slam one year and one month ago. This is why every tournament matters. A bad showing loses points.

Of course, with user trust it is a lot more nuanced, and the graph is more exponential (the longer you go back in time, the less it matters).

Anyway, enough side tracking. When you have a software project that is a library for developers, your end users are those developers. If the project is open source, then there is a clear communication of the rules through your license. This is why open source licensing is so important. It allows you to have a simple contract saying “this is what you can and can’t do”. As a developer I can see GPL, BSD, Apache, and I know right away what kind of community this is, and how I can play a role. It isn’t about one license being better or worse than another. It is about communicating rights.

If you are fortunate enough to gain a real community, where other developers are participating, then the game starts to change. Now you have people who are invested in your project, maybe building on it for you, or evangelizing it, writing documentation, or creating their own business. At this point you really start to see what kind of project it is going to be, above and beyond the source code licensing. This is the Open Community side of a project. It can range from: only people who work for company X contribute in anyway, to: active commiters from all over the Web. This paints a picture of the project as a whole, and will have large effects on project, including who uses it. This is all about governance.

This also comes into play in other ways. When you think of Apache, or the Dojo foundation, you know about the legal protection that comes through the process. You know that everyone has signed a CLA, and that the history of the code is clean and well known. This has a huge effect on getting large companies into the game (This is why companies like IBM and Sun are so involved in Dojo IMO).

Now that you have users of various stripes, and a community with varied roles, you also have connections through out. If you then change the open source license for your project, the contract in the community has changed. When you make a change you not only need a good reason, but it has to be transparent, and you obviously have to get all of your ducks in a row to even be able to pull it off (e.g. depending on the change you may need every author of a line of code to get involved).

With Ext JS, there was a strange situation. The original license of LGPL-ish was very confusing, which lead to a confused community. Some kind o change was required, and clarity needed to be brought in. Unfortunately, it seems that the move to GPL has caused more chaos and confusion. Developers who poured a lot of time into the community (e.g. by creating GWT-Ext) are upset. The chaos can rip the community apart and you end up with a true lose-lose. Jack has spent far too much time and grey hairs on this one, instead of writing great code and growing his business.

So, it acts as a reminder, that the community is all connected. Everyone may not be equal, but make sure that communication is incredibly clear at all times to make sure that something like this doesn’t happen.

I really liked getting the Ajax Language API out into developers hands as god knows we shouldn’t have to worry about translations. Now we can use the API and have the Google back-end do all of the work.

I have recently had a couple of scenarios where I really wanted a quick translation. I had a few twitter messages pass through my stream in French and Spanish. I had the answer to some technical issues show up on foreign forums.

So, I decided to create a Translate bookmarklet that allows me to select any foreign text, click on the bookmark, and a little window pops up with the English translation if it can work it out. Automatic translation is far from perfect yet, but for many scenarios you can easily get the gist (e.g. you wouldn’t want to automatically convert a book).

This is how I created the bookmarklet:

The source

First, I have the raw JavaScript source that will become the bookmarklet. There are a few sections of the code. First, we setup a method that will go off and call the Ajax Language API, passing in the translation and language that we want. This is where you would change the language code for non-English.

if (!window['apiLoaded']) {
  window.apiLoaded = function() {
    var language = "en";
    var text = window.getSelection().toString();
    if (text) {
      google.load("language", "1", { "callback" : function() {
        google.language.detect(text, function(dresult) {
          if (!dresult.error && dresult.language) {
            google.language.translate(text, dresult.language, language, function(tresult) {
              if (tresult.translation) {
                translationWindow(tresult, dresult);
              } else {
                alert('No translation found for "' + text + '" guessing the language: ' + dresult.language);
              }
            });
          }
        });
      }});
    }
  };
}

Then we setup a method that is able to display a window showing the result. I used the Prototype UI Window object if available, and good old alert() if not:

if (!window['translationWindow']) {
  window.translationWindow = function(tresult, dresult) {
    if (window['UI']) {
      new UI.Window({theme:  "black_hud",
                   shadow: true, 
                   width:  350,
                   height: 100}).setContent("<div style='padding:6px'>" + tresult.translation + "</div>")
                   .setHeader("English Translation")
                   .setFooter("Language detected: " + dresult.language)
                   .center({top: 20}).show();
    } else {
      alert(tresult.translation + " [lang = " + dresult.language + "]");
    }
  }
}

Next, we load the Prototype UI window code, and accompanying CSS resources by dynamically adding the resources to the DOM:

if (!window['UI']) {
  var pw = document.createElement('script');
  pw.src = 'http://almaer.com/downloads/protowindow/protowin.js';
  pw.type = "text/javascript";
  document.getElementsByTagName('body')[0].appendChild(pw);
 
  var pwdefault = document.createElement('link');
  pwdefault.setAttribute('rel', 'stylesheet');
  pwdefault.setAttribute('type', 'text/css');
  pwdefault.setAttribute('href', 'http://almaer.com/downloads/protowindow/themes/window.css');
  document.getElementsByTagName('body')[0].appendChild(pwdefault);
 
  var pwblack = document.createElement('link');
  pwblack.setAttribute('rel', 'stylesheet');
  pwblack.setAttribute('type', 'text/css');
  pwblack.setAttribute('href', 'http://almaer.com/downloads/protowindow/themes/black_hud.css');
  document.getElementsByTagName('body')[0].appendChild(pwblack);
}

Finally, we load the Google API loader, and use the dynamic loading option with the ?callback=apiLoaded. This kicks off the main driver that we saw first, and if it is already loaded we call it directly (for multiple translations on the same page).

if (!window['google']) {
  var s = document.createElement('script');
  s.src = 'http://www.google.com/jsapi?callback=apiLoaded';
  s.type = "text/javascript";
  document.getElementsByTagName('body')[0].appendChild(s);
} else {
  apiLoaded();
};

“Compilation”

This is the raw form, and we need to get the bookmarklet form, which you can just use right away if you are wanting English. For this, I use John Grubber’s makebookmarklet Perl script to do the conversion.

The Server

The Prototype UI code lives on the server, so I put a striped down version over there which just contains a combined Prototype + Window JavaScript file, and just the one theme CSS set.

In Action

Unsure what I am talking about? Just watch it in action:

UPDATE: I also implemented Twitter Translate to automatically convert tweets to your language.

Aaron Boodman was probably right in thinking that me wanting OpenID in the browser makes more sense as a browser feature, or separate plugin. This is a consumer level feature more than a developer focused one.

As I ruminate in the world of wishful thinking, I started to wonder about Jabber, and how it would be interesting to have XMPP as a native browser protocol.

We do have Jabber plugins, and there are JavaScript libraries that implement Jabber, but shouldn’t this be in the browser?

Jabber seems to be popping up all over the place. It started off as the IM protocol, and now has become a generic, scalable, messaging system. If XMPP was native and reliably in browsers, imagine how it could help the chat in Gmail? It could also transform the chat relationship with the Web. Social browsing as Me.dium does it could become more integrated. For example, I could have a trivial way to enable group chat on Ajaxian. I would love to be on the site and have others who are on there tell me how things are going, give me tips, and have comments feed into the same system. I shouldn’t have said “enable” as I wouldn’t have anything to do with it.

At an API level, we could also access XMPP from JavaScript.

Hmm, actually, is there a way in which this could tie into single sign-on and OpenID? Could XMPP be the way to login?

I mentioned the cross domain library, xssinterface, created by Malte Ubl on Ajaxian the other day.

His library abstracts on top of postMessage() and browser hacks to give you cross domain work. No sooner than I say “It would be nice if the library used cross domain workers if Gears is installed.” than Melde comments “Thanks for the hint to google gears. It is now implemented in trunk :)”. How about that for service!

Below is his first version that uses cross domain workerpools and the database to keep a message queue going. Very nice!

var wp       = google.gears.workerPool;
wp.allowCrossOrigin();
wp.onmessage = function(a, b, message) {  
  var origin = new String(message.origin);
  var parts  = origin.split("/");
  var domain = parts[2];
  parts      = domain.split(":"); // remove port
  domain     = parts[0];
 
  var recipient = domain;
  var channelId = message.text;
 
 
  var db = google.gears.factory.create('beta.database');
  db.open('database-xssinterface');
  db.execute('create table if not exists XSSMessageQueue' +
     ' (id INTEGER PRIMARY KEY AUTOINCREMENT, recipient_domain TEXT, channel_id TEXT, message TEXT, insert_time INTEGER)');
 
  // delete (and thus ignore) old messages
  var maxAge = new Date().getTime() - 2000;
  db.execute('delete from XSSMessageQueue where insert_time < ?',[maxAge]);
 
  // find new messages for me
  var rs = db.execute('select id, message from XSSMessageQueue where recipient_domain = ? and channel_id = ?', [recipient, channelId]);
 
  // there is a new message for the recipient
  if (rs.isValidRow()) {
          var id   = rs.field(0);
          var text = rs.field(1);
          db.execute("DELETE from XSSMessageQueue where id=?", [id]); // unqueue message
          wp.sendMessage(text, message.sender)
  }
 
  rs.close();
}

Man I love it when I can delete code. Seeing the line count go away and leaving a small amount of text is a sight for sore eyes. I got to delete a lot of code today thanks to the kind folks at Aptana.

I recently wrote a shim that allows the Google Gears API to run as is on the server side. It wraps the Gears Database API with the Jaxer one.

What is particularly cool about this is that you can then write some code and:

• If the user has Gears installed, it runs client-side
• If the user doesn’t have Gears installed just run it on the server

I want to setup a nice way to make this trivial to setup.

The majority of the code was a simple wrapper around the result set, so Aptana decided to directly support the Gears API itself which allowed me to get rid of it all!

This makes the shim as simple as this:

// -- Wrap this code so it is available if using a proxy call
function oncallback() {
 
  // Make up the namespaces to mimic Gears and a place for Jaxer holders
  google = {}; google.gears = {}; google.gears.factory = {}; google.gears.jaxer = {};
 
  // Create sets up a database instance to be used
  google.gears.factory.create = function(className, version) {
    if (className.indexOf('database') < 0) {
      throw new Error('I can only do Database work right now');
    }
    return new google.gears.jaxer.Db();
  }
 
// -- The Database Wrapper
  google.gears.jaxer.Db = function() {
    this.db = null;
  }
 
  google.gears.jaxer.Db.prototype.open = function(name) {
    this.db = new Jaxer.DB.SQLite.Connection({
      PATH: 'resource:///../data/' + name + '.sqlite',
      CLOSE_AFTER_EXECUTE: 'open'
    });
  }
 
  google.gears.jaxer.Db.prototype.execute = function(sqlStatement, argArray) {
    var rs = (argArray) ? this.db.execute(sqlStatement, argArray) : this.db.execute(sqlStatement);
    return rs;
  }
 
}

Philip Maker has also taken GearsORM and made it work with Jaxer. Very cool indeed.

What about the other languages?

• HTML / CSS
• COBOL
• Haskell
• JavaScript
• Arc
• Lisp
• Ruby
• Perl
• Java
• C++
• C

Got some good ones for other languages like Fortran, BASIC or assembly?

At first we had Nelly and XSL/T, and then we had Lisp. Now it is time to to evolve to C^HJavaScript.

ps. There is nothing geekier than a ^H joke.

As soon as I started to play with Aptana Jaxer, I saw an interesting opportunity to port the Google Gears Database API (note the Gears in the logo!)

If I could use the same API for both client and server side database access, then I can be enabled to do things like:

• Use one API, and have the system do a sync from local to remote databases
• If the user has JavaScript, use a local database, else do the work remotely
• Share higher level database libraries and ORMs such as Gears DBLib for use on server side data too

I quickly built a prototype to see if this would all work.

The Jaxer shim of the Gears API was born, and to test it out I took the database example from Gears itself and made it work.

To do so, I only had to make a few changes:

Run code on the server

I changed the main library to run on the server via:

<script type="text/javascript" src="gears_init.js" runat="server"></script>

I wrapped database access in proxy objects, such as:

function addPhrase(phrase, currTime) {
  getDB().execute('insert into Demo values (?, ?)', [phrase, currTime]);
}
addPhrase.proxy = true;

This now allows me to run the addPhrase code from the browser, and it will be proxied up to the server to actually execute that INSERT statement.

This forced me to separate the server side code from the client side code, which is a better practice anyway, but it does make you think about what goes where. In a pure Gears solution I can put everything in one place since it all runs on the client.

Create the new gears_init.js

A new gears_init.js acts as the shim itself. Instead of doing the typical Gears logic, it implements the Gears database contract. This wasn’t that tough, although there are differences between the Gears way, and the Jaxer.DB way. The main difference is to do with the ResultSet implementation, where Gears goes for a rs.next()/rs.field(1) type model versus the Jaxer.DB rs.rows[x] model.

I actually much prefer Gears DBLib as it hides all of that, and just gives the programmer what he wants… the rows to work on.

oncallback magic

In the current Jaxer beta, I ran into an issue where I wanted the Gears library to just “be there” for any proxy requests.

You have to think about the lifecycle of a Jaxer application, and the documentation tells you what you need to know to work around the issue.

In this case, I wrapped the code in:

function oncallback() {
  // create the wrapper here
}

This is less than idea, and Aptana is playing with nice scoping which would enable you to just say “hey, load this library once and keep it around for the lifetime of the server | application | session | page”. That will be very nice indeed.

You can do a little bit of this by opening up your jaxer_prefs file and adding the resource for your file:

// This option sets up an html document that will be loaded
// everytime a callback is processed.  This has to be a local file.
// If not specified, an empty document will be loaded.
// pref("Jaxer.dev.LoadDocForCallback", "resource:///framework/callback.html");

Future…

This is just the beginning. As I mentioned at the beginning, I am interested to see where you can take this to handle clients who do not support JavaScript, and also to deal with synchronization with minimal code (sync from local to remote with exactly the same SQL API).

I was chatting with someone about how, in 2008, you could build an application with GWT on the client side, and Rhino on Rails on the server side, and how that would mean flipping the roles of Java and JavaScript. Of course, this would be a flip BACK to the past:

Netscape LiveWire enables developers to create, modify, and maintain online sites and applications through a simple drag-and-drop, point-and-click environment. The environment uses the Java programming language and a Java-based scripting language to enable developers to create and execute Live Objects, or interactive multimedia content, within their applications.