“How exactly do you make your paints?”; What if Google hired artists? Feeling Touchy; Learning how to build great touch UI
Feb 15

Building an Web application from the inside out; Using node.js to bootstrap a server from client JS

JavaScript, Open Source, Tech, webOS Add comments

Over the winter holiday, Ben and I whipped together Project Appetite, an open source example that consumes the feeds from the Palm application ecosystem (both Palm Catalog and Web distribution). We didn’t have long to come up with something, and one of the interesting stories was how we took an API and mocked up the middle, allowing server and client to get going quickly.

The feeds that the Palm ecosystem puts out are RSS 2.0 XML feeds, with extra catalog-y info in an “ac” namespace. We converted the format to JSON to make life easier for the JavaScript client consumer. Although you can DOMParser() away (and use the Microsoft component) JSON is just so much easier. What lives at api.projectappetite.com is the result of the munging, so others could consume JSON directly if they so wish.

The Data

We quickly put together some dummy data, and for ease, put it in a file app.js that the client could consume. Once we iterated on the format, one of us could go off and write the XML to JSON code. We actually implemented this in a variety of ways as we experimented. Ben created a simple JDOM translation, and I fought with databinding, which still seems to be a royal pain with Java. The reason I checked that out was to create an open backend on app engine, and I wanted to go from XML to JSON to Java for persistence via JDO. From one set of Java objects I could @PersistenceCapable and @XStreamAlias("asset_url"). It wasn’t worth the effort.

The Client

We created an Appetite object that would be able to query the model and get the data that was needed. The constructor took the apps data, and then it did its magic. In this case is created some caches to make querying fast.

The public API contained:

  • app(id, locale): returns a single app based on id
  • find(opts): the core engine for querying the app data
  • types: contained the logic for filtering on top rated, top paid, newest, etc
  • sorts: contained the sorting functions to return the data in the right order (e.g. by download count versus date)

Very quickly the client was mocked up and the frontend was build out using this API. Again, at this point the entire front end could be built without waiting for server infrastructure.

The Server

Since the client API mocked out the functionality needed for the frontend, the server could actually reuse this logic. This is when the node.js server was born.

To play nice with node, we went back to the other files and added logic to export the data. E.g. in the client.js:

// check to see if you are running inside of node.js and export if you are
if (typeof GLOBAL == "object" && typeof GLOBAL['node'] == "object") {
    exports.Appetite = Appetite;

Once that was in place, we could load up all of the information that we need:

// -- Load up the libraries
var sys   = require("sys"),
   http   = require("http"),
   posix  = require("posix"),
   apps   = require("./apps").apps,
   client = require("./client");

And then we can get access to the client API via:

var a = new client.Appetite(apps);

We created a simple mapping that enabled a URL such as /app?id=[id]&locale=* to be converted to a method call of app({id:id, locale:locale}). We did this in a generic way that enabled us to add URL endpoints by simply added to the responder hash. The functions would return an object that could contain error codes and the like.

For example, the app end point:

// /app?id=[id]&locale=*
app: function(params) {
    if (!params.id) {
        return {
            error: 501,
            body: 'I need an id!' 
    return {
        body: a.app(params.id)

We also added a magical DEFAULT handler to take care of bad URLs.

Finally, we would boot up the listening server and handle responses:

// -- Create the HTTP server binding
var host = process.ENV['APPETITE_HOST'] || 'localhost';
var port = process.ENV['APPETITE_PORT'] || 8000;
if (process.ARGV.length > 3) { // overwrite with command line args
    port = process.ARGV[3];
if (process.ARGV.length > 2) {
    host = process.ARGV[2];
http.createServer(function(request, response) {
    var path = request.uri.path.substring(1);
    var output;
    var responder = (typeof responders[path] == "function") ? responders[path] : responders['DEFAULT'];
    output = responder(request.uri.params);
    var contentType = output.contentType || "text/javascript";
    if (output.error) {
        response.sendHeader(output.error, { "Content-Type": contentType });
    } else {
        var body = (contentType == "text/javascript") ? JSON.stringify(output.body) : output.body;
        response.sendHeader(200, { "Content-Type": contentType });
}).listen(port, host);

After 86 verbose, commented code, we had a server that would respond to URLs and return data. And in this time the frontend continued to be built out.

Being able to reuse the client JS and wrap it to become a server was a lot of fun. I am definitely a big node.js fan! Now, I am looking forward to doing a lot more with Project Appetite…. but for now we are working on a pretty cool webOS application, and getting the developer platform better and better.

Some coffee in your Node

In an aside, the node hello world has been ported to CoffeeScript:

# To run, first install node and coffee-script
# Node: http://nodejs.org/#download
# CoffeeScript: http://github.com/jashkenas/coffee-script
# In a terminal window:
# $ cd coffee-script
# $ ./bin/node_coffee -r hello_web.coffee
# Tested with Mac OS X 10.5.8, Node 0.1.26, CoffeeScript 0.5.0
sys: require "sys"
http: require "http"
http.createServer( (req, res) ->
  res.sendHeader 200, {"Content-Type": "text/plain"}
  res.sendBody "Hello, World!"
).listen 8000
sys.puts "Server running at"

Leave a Reply

Spam is a pain, I am sorry to have to do this to you, but can you answer the question below?

Q: What are the first four letters in the word British?