Vote for me! The case for: set strictlines on Bespin Concept Defrag Command; Exploring some fun visualizations
Mar 30

Dealing with JavaScript scope issues; The tale of Alex kindly indulging me

Ajax, JavaScript, Tech with tags: Add comments

Dealing with JavaScript scope issues; The tale of Alex kindly indulging me

JavaScript is about run-time. Run-time is great and all, but especially when dealing with the browser, and how your Web page has to bootstrap the entire world on every load, Ajax developers have to think about issues that other people don’t. Problems that others can compile away, or know that “that happens once when I start up the puppy on the server” are here for us to stay.

This often seems to mean that we have to deal with writing our applications in ways that aren’t as clean as we may like. We run across problems where for the Nth time someone was bitten as somewhere code did a “for in” that wasn’t guarded with hasOwnProperty and then someone throws up there arms. Never Again. Dojo does a lot of things out of this experience. It is out of real-world pain that choices were made in the toolkit. One of these is how they are very careful not to pollute the global namespace. This is great in that you don’t run into collisions, especially in a world where code is being sucked in from who knows where (e.g. some Ad code is sucking in things). As the author of some JavaScript code, you don’t actually know what else may get into your global area when running, so you need to guard against it.

The problem is that this means that you can lose out. Prototype feels so right to me in many ways as it is less of a “JavaScript library” than a “way in which JavaScript should have evolved”. We have seen some of its goodness get into ES 3.1 (e.g. bind()) but at that rate of progress we will get four more methods in 20 years ;)

"some content   ".trim(); // feels right
dojo.trim("some content   "); // doesn't feel right

I have to take a second to tease here. Did you notice that there are two trim methods in Dojo? dojo.trim and dojo.string.trim. This is a good example of the crazy things that we have to think about. It appears that dojo.trim is lean code, so it gets into base, but dojo.string.trim is more code, but it runs faster. Wow :)

Once you get a lot of code going, you suddenly realise that the word “dojo” appears 50 times per screen of code. It makes me want to create a Bespin plugin that change the shade of that one word. It feels like Java. Unlike Java, you can’t import away some of the pain. You can try to var foo = some.really.big.package; but that gets annoying quickly.

So, to the point of the blog post. When I saw Alex Russell at a nice wine bar in San Francisco last week, I told him how I would die for the ability to have the best of both worlds of Dojo and Prototype:

I want to be able to write code the way that feels right, without the verboseness, but ALSO not run into the scoping issues. I would even take a performance hit for this. What if I could have:

runSomeCodeWithMagic(function() {
    // in here I am in the lovely land where "some content   ".trim() works
    // and I can just forEach instead of dojo.forEach
    // but the outside world isn't affected
});

Well, Alex listened to me blather on, probably thinking I was a total idiot, and then went on to quickly indulge me by giving me a little of my wish by giving me dojo.runWith:

dojo._runWithObjs = [];
dojo.runWith = function(objs, func){
	if(!dojo.isFunction(func)){
		console.error("runWith must be passed a function to invoke!");
		return;
	}
	var rwo = dojo._runWithObjs;
	var iLength = rwo.length;
	// console.debug(func.toString());
	if(!dojo.isArray(objs)){
		objs = [ objs ];
	}
	var catchall = dojo.delegate(dojo.global);
	var fstr = [ "(", func.toString(), ")()" ];
	objs.unshift(catchall);
	var locals = {};
	objs.push(locals);
	objs = objs.reverse();
	dojo.forEach(objs, function(i){
		var idx = rwo.length;
		rwo.push(i);
		fstr.unshift("with(dojo._runWithObjs["+idx+"]){");
		fstr.push("}");
	});
	(new Function(fstr.join("")))();
	// allow us to GC objs passed as contexts, but don't rewind
	// further than we started (allowing nested calls)
	rwo.length = iLength;
	// TODO:
	//		iterate on locals and look for new properties that
	//		might have been assigned. Maybe give the with-caller a
	//		way to handle them or specify a policy like "make
	//		global"?
};

To use it I can do something like this:

<html>
    <head>
        <script src="http://o.aolcdn.com/dojo/1.3.0/dojo/dojo.xd.js"></script>
        <script src="runwith.js"></script>
        <script>
        dojo.addOnLoad(function() {
            dojo.runWith([ dojo ], function() {
                var sum = 0;
                forEach([1, 2, 3, 4], function(i) {
                    sum += i;
                });
                byId("result").value = "The sum is: " + sum;
            });
        });
        </script>
    </head>
    <body>
        <h1>Run with Wolves</h1>
 
        <input type="text" id="result">
    </body>
</html>

Very nice! Only a couple of dojo’s in sight!

Here are some of the thoughts from Alex himself:

Note/warning about the runtime cost: If your browser parses things fully in it’s JS engine up front, this function may hurt. A lot. It de-compiles a function using the toString() method, meaning that it does an uneval + eval + string concat + with() call. Each of these operations alone might be painful in a slow engine. Together they could be fatal. On the other hand, if you’re using these functions in, e.g., Chrome/V8, this could turn out to be relatively cheap, particularly as this is run-once kinda thing. The runtime cost involves namespace misses on locals, and that can be significant. I dunno. You’ll have to test to find out.

Note that you won’t easily be able to define globals from here by dropping a “var”. This might be a feature or a bug, depending on how you think about it.

Anyway, hope it’s useful. I’d imagine that you’d structure your files like this:

// something.js
dojo.provide("thinger.something");
dojo.require("thinger.blah");
 
dojo.runWith([ dojo, thinger ], function(){
       ...
});

What about getting the magic on the core objects? Again, only within the magically land of that scope do we want String to have the trim method…. and code outside of it shouldn’t see it. Can we swap onto the objects and their prototypes? Alex has some thoughts here too:

I think I can proxy intrinsics at some additional cost. I’d like to make it a protocol, though, so that you might be able to have a list or function that handles your extensions. E.g.:

var contextObj = {
    ":intrinsics": {
        "String": { ... },
        "Number": { ... },
            // ...
    }
};
 
dojo.runWith([ contextObj, ... ], ... );

This would give us a way to un-install them later, but I don’t have a solution for aync code as we do with with(){ … } which actually binds definitions at declaration time.

I might try to proxy intrinsic prototypes somehow, but I need to spend more time thinking about how to get that to work well. I’d like these things not to place big constraints on how you think about or use this system.

Wicked. This also ties into Pete Higgins and some of the very interesting work he is doing in similar but different veins with his dojotype and plugd which munge Dojo into interesting forms.

Since Dojo has pretty much everything that any other library has, you can start thinking about how you can bend it to look and feel like others, take on their APIs when it makes sense, and bend it to your whim.

With research like Alex’s we could see an interesting view when you can create worlds which don’t affect each other, let you have a view that makes sense for your code, but doesn’t affect others.

Even if you poo-poo some of it for the performance aspects, this is why it is incredibly exciting to see the latest JavaScript Vm work. With these guys running, they can optimize a lot of this a way, and things that used to be bottlenecks in the code will cease to be.

Thanks to Alex and Pete for indulging me, and taking the time to listen and produce really interesting solutions!

Updated: Using the Dojo Loader

James Burke has a very cool follow up on how to give a solution to this kind of problem using the Dojo Loader itself:

Setup your locals

dojo.provide("coolio.locals");
 
dojo.setLocalVars("coolio", {
 trim: "dojo.hitch(dojo, 'trim')",
 $: "dojo.hitch(dojo, 'query')",
 id: "dojo.hitch(dojo, 'byId')"
});

Now setup your actions that will use the locals:

dojo.provide("coolio.actions");
 
coolio.actions = {
 init: function(){
  $("#trimButton").onclick(coolio.actions, function(evt){
   id("trimOutput").value = trim(id("trimOutput").value);
  });
 }
}

Finally, use HTML to auto load:

<script type="text/javascript" src="dojo/dojo.js" djConfig="require: ['coolio.locals']"><script>

8 Responses to “Dealing with JavaScript scope issues; The tale of Alex kindly indulging me”

  1. Erik Arvidsson Says:

    Don’t do this on any production code. You have been warned :-)

    I’m pretty sure that the with statement will turn off jitting for all major engines. The reason why all browser vendors wants to remove with is not only that it is error prone and confuses the hell out of most people. It also prevents optimizations.

    The only solution to this problem that I’ve seen involves a compiler/preprocessor. While developing you can use eval (like base2) and then before you deploy you run a preprocessor that replaces those evals with inline code.

  2. Ray Cromwell Says:

    I dunno, given the extra verbiage of dojo.runWith, the object array, and the function() {} closure, you’d have to type the dojo ‘prefix’ more than 6 times before you’d gain any less verbiage, and you pay a high cost.

  3. henrah Says:

    If you’re going to use the `with` operator anyway, why bother using this runWith method? You’re incurring the considerable cost of decompiling and recompiling a function, plus the method setup and teardown process, on top of the existing penalty imposed by using `with` in the first place — and the API is not ultimately any simpler than a set of nested `with(object){}` blocks.

    The API may look cleaner and more idiomatic because it looks like you’re passing a closure/callback function in the style of modern libraries, but really you’re not passing a closure at all. You’re just passing the textual representation of a block, since all its context is lost when it gets converted into text and recompiled by the Function() constructor. What benefit does this confer? It seems to me like a very flimsy decoration over a widely despised language feature.

  4. Eugene Lazutkin Says:

    While JavaScript is not strictly a homoiconic language, it is fun to see what is possible. The biggest problem of this approach not with “with” but “func.toString()” — it doesn’t work on some mobile browsers as expected (does not return a function’s text sufficient to reconstruct the function).

    If only the committee found a way to standardize AST making JavaScript homoiconic (http://en.wikipedia.org/wiki/Homoiconic), we would have an elegant way to do this stuff and a lot more including custom pre-processing, DSLs, code and data specializations, and so on.

  5. Dion Almaer Says:

    Yes, yes, this isn’t meant to be a “do this in production” type thing, but rather an “experiment to see how JavaScript can be contorted in a way to make it both nice to work with, and sandboxed”

  6. Alex Russell Says:

    something I failed to mention in my original mail:

    I have some hope that we’d be able to actually introspect (vai a Rhino AST or something) on the APIs that you’d be using from such a mapping and roll them back at build time.

    As to Erik’s with(){} comment, it’s not really possible to turn off JIT in v8, but with(){} will cause polymorphic inline caching to become much less effective, hurting most kinds of lookups. It’ll still be happening in machine code (vs, say, IE which will need to look it up in the interpreter), but it will still be JIT’d.

    Regards

  7. toshiba Says:

    JavaScript is about run-time. Run-time is great and all, but especially when dealing with the browser, and how your Web page has to bootstrap the entire world on every load, Ajax developers have to think about issues that other people don’t.

  8. haberler Says:

    useful article thank you. And i loved you secret question :)

Leave a Reply

Spam is a pain, I am sorry to have to do this to you, but can you answer the question below?

Q: What is the number before 3? (just put in the digit)