Jul 29

Capability based JavaScript loading; JS libraries catch up to GWT

Ajax, JavaScript, Tech with tags: 6 Comments »


Web developers have to walk around dragging a ball and chain. It is fantastic that we have a ridiculous install base (browsers, the Web runtimes) and a dynamic language in JavaScript.

The ball and chain though is the fact that we have to care so much about the payload of the application that we write. Less code means less bytes to download, and less for the JS runtime to load up.

This has bad consequences:

  • You are tempted to write hard to understand code
  • You have to balance functionality and code size much more than other environments
  • You end up getting tricky and come up with ways to dynamically load modules on demand
  • All of this time is time not spent on the app logic.

When you run into a problem like this in computer science what do you do? Build an abstraction!

GWT has done a great job here. It’s very nature requires a compilation step, and once you have to deal with that step…. you can do a lot at build time to change the above dynamics.

For example:

  • You can write nice explicit source code, and trust that the compiler will output optimized JS (which can look as ugly as sin for all anyone cares)
  • Your application can get faster with a new release of GWT, as the compiler gets better
  • A lot of advanced techniques such as code splitting, and deferred binding can be applied to allow logic to apply at build time as well as runtime.

Ignoring GWT, when you write a pure JS application for the Web, you are writing a cross platform application. Unlike clean portable C, you don’t have that compilation step to do clean up work for you on each platform, so you end up with a lot of runtime conditional logic.

You try to do object detection rather than user agent conditional, but still, you end up both downloading the code necessary to run in all environments, and you have the overhead of loading and executing that code.

Chances are you are smart and don’t do if() checks all the time, but maybe do so once when loading up. For large chunks of big differences, maybe you do something like:

var Foo = (sometest) ? function() {
    // do it like this
} : function() {
    // do it like that

If you take a look at some parts of popular JavaScript frameworks, you see that they are abstractions or fixes for various browsers. As browsers have stepped up to the plate recently, they have fixed a lot, and suddenly you take a step back. A lot of frameworks have a chunk of code to give us the ability to do smart things with CSS querying. Modern browsers support querySelectorAll (even with some bugs) and getElementsByClassName so a lot of the code is un-needed…. unless you care about the old browsers.

We shouldn’t have to download all of that cruft on the other side of the if() statement when we don’t use it!

The GWT compiler can output versions of your application targeted to a given browser. Only the fastest, minimal code, will get sent down. Deferred binding goes far beyond just browser type (can deal w/ locales and much more) too.

GWT isn’t the only horse in this race though.

Alex Russell started to sprinkle in some directives to Dojo that would enable you to do a build that is WebKit targeted. A major use case for all of this is making sure that you are sending down lean code in the mobile space. It can matter everywhere, but when you are dealing with grown up walkie talkies…. you want to really optimize.

And this brings us to YUI. I was really excited to see some of the features in the YUI 3.2.0 preview release. Great stuff for touch/gesture support, but what stood out for me was “YUI’s intrinsic Loader now supports capability-based loading”. A-ha!

I poked around the source to see how it was used. Here is one fake example that shows how it works. When you boot up YUI you can add in “conditions” which are in charge of working out whether to load something. You can currently tie on to user agent, or write a test function:

    modules: {
        lib2: {
            requires: ['yui'],
            fullpath: 'js/lib2.js',
            condition: {
                trigger: 'node-base',
                ua: 'gecko'
        lib3: {
            requires: ['yui'],
            fullpath: 'js/lib3.js',
            condition: {
                trigger: 'event-base',
                test: function(Y, req) {
                    return Y.UA.gecko;
}).use('node', function (Y) { .... });

The YUI team uses this themselves to only load certain ugly DOM stuff for IE6 when needed:

"dom-style-ie": {
    "condition": {
        "trigger": "dom-style", 
        "ua": "ie"
    "requires": [
YUI.add('dom-style-ie', function(Y) {
    // ....

This makes a ton of sense, especially for the libraries themselves to do this work. They are the ones that do the heavy lifting of cross browser ugliness, all to allow us to write to an API that works.

Beyond that, we can then ourselves split out code if it becomes a performance issue.

Of course, the value of the test() function is that you can do capability based testing (not just user agent testing…. hence the name!). This means that you can do a test say for Canvas support (as Zach mentions in the comments) document.createElement(’canvas’).getContext, and load up excanvas or another shim library if it doesn’t exist.

Using modernizr and ilk, we could build out “plugins” for YUI that auto load based on common capabilities. You can imagine running a YUI build, having it detect that you are using a capability, and then automatically load up a plugin that has the condition all loaded up. What a great user experience as a developer!

Maybe it makes sense to come up with a common pattern and conventions for dealing with this issue. How do you name your CSS/JS? Do we setup server side hooks so we don’t even need loaders necessarily?

I can certainly see a day where you may be asking the Google Ajax Library service for jquery.js, but it is returning jquery.ie.js to you.

Jul 14

I = V / R; Ohm’s law explains Developer Advocacy

Tech with tags: , , 3 Comments »


I consider myself a developer who happens to love community, evangelism, and advocacy.

When I put on the developer relations hat, I am constantly asking myself what value these roles have, and how to add more value to the process.

When I was working at Google, a certain engineering manager made it very clear that he felt advocacy and evangelism added close to zero value. His position was that you if you build a great product, developers will come. Rather than lash out, I keep that in the back of my mind, and think about how to make that view incorrect.

I keep coming back to Ohm’s law which explains the relationship between current, potential difference, and resistance and the laws of electricity.

I still remember the crazy physics teacher that I had that would scream:

“It’s the volts that jolt but the amps will kill ya!”

He would then kindly put kids through an experiment to get close to showing the truth behind that statement.

If you think of this law as a pipe and water flowing through it, there is the radius of said pipe, how fast the water is being pushed through it, and the resistance of the pipe.

How does this relate to developers and developer advocacy?

To reference the afore mentioned engineering manager, he has some truth. I feel like developer evangelism can help with the “push” (the volts). However, if you take the best developer relations crew… a crew that can push hard… they won’t be able to reach any potential if the pipe is tiny small, and the friction of the tools, APIs, platform, or economics provide.

Thus, the most important thing that you can do for a developer oriented initiative is first create a fantastic pipe. The pipe should have as little friction as possible, and should be as large as possible. Probably the most important way to achieve those goals is to provide developers with:

  • A fantastic economic opportunity. Let’s face it, if Apple restricted developers so they had to use punch cards to build iOS apps, we would still have a lot of apps :) Same goes for Facebook etc. Even though when you look at the numbers on even those platforms, success is a “hit based” economy, the hits do so well that many developers get in the water for their change
  • A chance to do something different. Create a platform with unique capabilities and the opportunity for developers to create something very different, and that itself will be a draw. New companies and products can be created where they didn’t exist. Nokia had (and has) a boatload of phones. At JavaOne for years a Sun exec would get up and say “developer a J2ME app and you can reach BILLLLLLLIONS of devices!” Few did in the scale that we see with so called “smartphones”. You just couldn’t create anything interesting or compelling (oh, and there are the carrier gateway issues of that time too)
  • Make your platform fun and generative. This is a sub-bullet of the last one in many ways. But, if you a developer can have a great time creating on top of your building blocks, great things will happen, and the network effect can flow.

Now, evangelism comes in as a way to:

  • Let developers KNOW about your fantastic economic opportunity, their chance to do something different, and how productive and fun it is to develop on.
  • With tools, samples, articles, conferences, and community in general…. help lower the friction for these things. Give developers a leg up.

This push can have a huge effect. Good evangelism will also create external evangelists who share your excitement and vision ending up with a nice network effect.

Only half the story

But evangelism is only a piece of the story. The reason that I like the term “developer advocate” so much (and why I posted on this topic) is because it shows the other side of the coin. You are the third party developers representative back at your platform company. You are letting the internal teams understand their needs. And, you get to be part of that community:

“I can advocate something, and part of that will hopefully be heavily listening, and participating in the open community. Of course, these are just words, and you have to make this happen. We could call ourselves evangelists and do a lot of listening, or become advocates and do none. The word choice though does make you think about what you should be doing.
More importantly, in my opinion, is the word developer. Rather than talking just about technology, we are talking about humans who use it. This again makes you feel like you need to be more part of the community, working with developers on their level.”

Every now and then I reflect on Ohm’s law (it helps to watch the current of a stream in the mountains as I have been able to do in Colorado this week!), and think about how me and my team can both relieve friction and push harder for developers on the platform.

I have been at HP for a week or so now, and thus it was a great opportunity to think about how we can come together as a team to do just this. I think that I will be personally changing what I do, and I can’t wait to work with the community, the internal engineers, and the product chaps as we take webOS to the next level.

Jul 12

Diffable; What if GitHub supported it natively?

JavaScript, Open Source, Tech with tags: 3 Comments »

Steve Souders told me about Diffable, when I saw him after his awesome Velocity conference.

Diffable is an open source project that allows you to only send down the deltas in your application versions, versus full new downloads (which may have a large amount of duplicate data).

In their presentation, Josh Harrison and James deBoer, talk about the details after the start with the core issues:


Frequently modified web resources must be downloaded in their entirety with every modification.
Even a small change invalidates the cache.
Rich internet applications often have large amounts of static content.


Initial application resources kept in cache.
Changes to cached versions transmitted as deltas.
Deltas merged client-side to generate latest JS version.


Faster page load times for users with cached resources.
Small changes to large resources incur only small costs.

Steve summarizes things well in his post:

Diffable uses differential compression to reduce the size of JavaScript downloads. It makes a lot of sense. Suppose your web site has a large external script. When a new release comes out, it’s often the case that a bulk of that large script is unchanged. And yet, users have to download the entire new script even if the old script is still cached.

Josh and James work on Google Maps which has a main script that is ~300K. A typical revision for this 300K script produces patches that are less than 20K. It’s wasteful to download that other 280K if the user has the old revision in their cache. That’s the inspiration for Diffable.

Diffable is implemented on the server and the client. The server component records revision deltas so it can return a patch to bring older versions up to date. The client component (written in JavaScript) detects if an older version is cached and if necessary requests the patch to the current version. The client component knows how to merge the patch with the cached version and evals the result.

With this technique in action, you end up sending down JS arrays as deltas that looks like:

[0,10,"red",40,3," leaps",25,15,16,3,"."]

The data that these guys share is impressive. The results seem to add up for applications as large as Google Maps. Do they measure up for smaller apps? If large apps have a lot of static content, couldn’t that content be put into another download and even app cached away?

Also, it is a lot of work to implement this for a developer. Work on both client side and server side. It would be great if we can experiment with Diffable and then move things lower into the stack. Why can’t HTTP itself be smart enough to deal with diffs?

It did make me think of my favourite chaps @github. I know that GitHub is about development rather than deployment…. but what if they supported this natively (since they kinda grok diffin’ etc already) and offered a client side loader so you could github.load("project", ...).

It all just makes me realise that GitHub is poised to pounce in many directions. Good on ‘em.