techno.blog("Dion")

Dealing with JavaScript scope issues; The tale of Alex kindly indulging me

JavaScript is about run-time. Run-time is great and all, but especially when dealing with the browser, and how your Web page has to bootstrap the entire world on every load, Ajax developers have to think about issues that other people don’t. Problems that others can compile away, or know that “that happens once when I start up the puppy on the server” are here for us to stay.

This often seems to mean that we have to deal with writing our applications in ways that aren’t as clean as we may like. We run across problems where for the Nth time someone was bitten as somewhere code did a “for in” that wasn’t guarded with hasOwnProperty and then someone throws up there arms. Never Again. Dojo does a lot of things out of this experience. It is out of real-world pain that choices were made in the toolkit. One of these is how they are very careful not to pollute the global namespace. This is great in that you don’t run into collisions, especially in a world where code is being sucked in from who knows where (e.g. some Ad code is sucking in things). As the author of some JavaScript code, you don’t actually know what else may get into your global area when running, so you need to guard against it.

The problem is that this means that you can lose out. Prototype feels so right to me in many ways as it is less of a “JavaScript library” than a “way in which JavaScript should have evolved”. We have seen some of its goodness get into ES 3.1 (e.g. bind()) but at that rate of progress we will get four more methods in 20 years ;)

"some content   ".trim(); // feels right
dojo.trim("some content   "); // doesn't feel right

I have to take a second to tease here. Did you notice that there are two trim methods in Dojo? dojo.trim and dojo.string.trim. This is a good example of the crazy things that we have to think about. It appears that dojo.trim is lean code, so it gets into base, but dojo.string.trim is more code, but it runs faster. Wow :)

Once you get a lot of code going, you suddenly realise that the word “dojo” appears 50 times per screen of code. It makes me want to create a Bespin plugin that change the shade of that one word. It feels like Java. Unlike Java, you can’t import away some of the pain. You can try to var foo = some.really.big.package; but that gets annoying quickly.

So, to the point of the blog post. When I saw Alex Russell at a nice wine bar in San Francisco last week, I told him how I would die for the ability to have the best of both worlds of Dojo and Prototype:

I want to be able to write code the way that feels right, without the verboseness, but ALSO not run into the scoping issues. I would even take a performance hit for this. What if I could have:

runSomeCodeWithMagic(function() {
    // in here I am in the lovely land where "some content   ".trim() works
    // and I can just forEach instead of dojo.forEach
    // but the outside world isn't affected
});

Well, Alex listened to me blather on, probably thinking I was a total idiot, and then went on to quickly indulge me by giving me a little of my wish by giving me dojo.runWith:

dojo._runWithObjs = [];
dojo.runWith = function(objs, func){
	if(!dojo.isFunction(func)){
		console.error("runWith must be passed a function to invoke!");
		return;
	}
	var rwo = dojo._runWithObjs;
	var iLength = rwo.length;
	// console.debug(func.toString());
	if(!dojo.isArray(objs)){
		objs = [ objs ];
	}
	var catchall = dojo.delegate(dojo.global);
	var fstr = [ "(", func.toString(), ")()" ];
	objs.unshift(catchall);
	var locals = {};
	objs.push(locals);
	objs = objs.reverse();
	dojo.forEach(objs, function(i){
		var idx = rwo.length;
		rwo.push(i);
		fstr.unshift("with(dojo._runWithObjs["+idx+"]){");
		fstr.push("}");
	});
	(new Function(fstr.join("")))();
	// allow us to GC objs passed as contexts, but don't rewind
	// further than we started (allowing nested calls)
	rwo.length = iLength;
	// TODO:
	//		iterate on locals and look for new properties that
	//		might have been assigned. Maybe give the with-caller a
	//		way to handle them or specify a policy like "make
	//		global"?
};

To use it I can do something like this:

<html>
    <head>
        <script src="http://o.aolcdn.com/dojo/1.3.0/dojo/dojo.xd.js"></script>
        <script src="runwith.js"></script>
        <script>
        dojo.addOnLoad(function() {
            dojo.runWith([ dojo ], function() {
                var sum = 0;
                forEach([1, 2, 3, 4], function(i) {
                    sum += i;
                });
                byId("result").value = "The sum is: " + sum;
            });
        });
        </script>
    </head>
    <body>
        <h1>Run with Wolves</h1>
 
        <input type="text" id="result">
    </body>
</html>

Very nice! Only a couple of dojo’s in sight!

Here are some of the thoughts from Alex himself:

Note/warning about the runtime cost: If your browser parses things fully in it’s JS engine up front, this function may hurt. A lot. It de-compiles a function using the toString() method, meaning that it does an uneval + eval + string concat + with() call. Each of these operations alone might be painful in a slow engine. Together they could be fatal. On the other hand, if you’re using these functions in, e.g., Chrome/V8, this could turn out to be relatively cheap, particularly as this is run-once kinda thing. The runtime cost involves namespace misses on locals, and that can be significant. I dunno. You’ll have to test to find out.

Note that you won’t easily be able to define globals from here by dropping a “var”. This might be a feature or a bug, depending on how you think about it.

Anyway, hope it’s useful. I’d imagine that you’d structure your files like this:

// something.js
dojo.provide("thinger.something");
dojo.require("thinger.blah");
 
dojo.runWith([ dojo, thinger ], function(){
       ...
});

What about getting the magic on the core objects? Again, only within the magically land of that scope do we want String to have the trim method…. and code outside of it shouldn’t see it. Can we swap onto the objects and their prototypes? Alex has some thoughts here too:

I think I can proxy intrinsics at some additional cost. I’d like to make it a protocol, though, so that you might be able to have a list or function that handles your extensions. E.g.:

var contextObj = {
    ":intrinsics": {
        "String": { ... },
        "Number": { ... },
            // ...
    }
};
 
dojo.runWith([ contextObj, ... ], ... );

This would give us a way to un-install them later, but I don’t have a solution for aync code as we do with with(){ … } which actually binds definitions at declaration time.

I might try to proxy intrinsic prototypes somehow, but I need to spend more time thinking about how to get that to work well. I’d like these things not to place big constraints on how you think about or use this system.

Wicked. This also ties into Pete Higgins and some of the very interesting work he is doing in similar but different veins with his dojotype and plugd which munge Dojo into interesting forms.

Since Dojo has pretty much everything that any other library has, you can start thinking about how you can bend it to look and feel like others, take on their APIs when it makes sense, and bend it to your whim.

With research like Alex’s we could see an interesting view when you can create worlds which don’t affect each other, let you have a view that makes sense for your code, but doesn’t affect others.

Even if you poo-poo some of it for the performance aspects, this is why it is incredibly exciting to see the latest JavaScript Vm work. With these guys running, they can optimize a lot of this a way, and things that used to be bottlenecks in the code will cease to be.

Thanks to Alex and Pete for indulging me, and taking the time to listen and produce really interesting solutions!

Updated: Using the Dojo Loader

James Burke has a very cool follow up on how to give a solution to this kind of problem using the Dojo Loader itself:

Setup your locals

dojo.provide("coolio.locals");
 
dojo.setLocalVars("coolio", {
 trim: "dojo.hitch(dojo, 'trim')",
 $: "dojo.hitch(dojo, 'query')",
 id: "dojo.hitch(dojo, 'byId')"
});

Now setup your actions that will use the locals:

dojo.provide("coolio.actions");
 
coolio.actions = {
 init: function(){
  $("#trimButton").onclick(coolio.actions, function(evt){
   id("trimOutput").value = trim(id("trimOutput").value);
  });
 }
}

Finally, use HTML to auto load:

<script type="text/javascript" src="dojo/dojo.js" djConfig="require: ['coolio.locals']"><script>