Jun 29

Having fun with Canvas, but the aim is to have to use it less and less; Performance wars move from raw JS to DOM

JavaScript, Tech with tags: , 3 Comments »


We are having a great time using Canvas for Bespin and a few other projects. Having text, images, and boxes as the primitives on the Web just isn’t good enough, so leap frogging to HTML5-land where we have the ability to arbitrarily paint pixels is fantastic. Mix this with the ability to directly manipulate video, and you have new opportunities.

However, Canvas also has obvious issues. How do you make it accessible? Or searchable? We have some ideas there, but they are a long way off. The goal has never been to use Canvas as the shiny toy. With Bespin, we actually tried to do it with DOM, and it wasn’t so much chutzpah as “can’t get it to perform”-pah that lead us down the path of Canvas. The usual suspects in the stack weren’t working for us, whereas others were blazing (e.g. JavaScript performance has blown us away, as has Canvas itself).

With major browsers like Firefox 3.5 (Safari 4, Chrome 2, etc) I am very much excited to work with browsers (say, the great Firefox team :) to kick into gear on the next level of improvements. The current batch have been competing on the performance of JavaScript. The new engines are fantastic, and really change the game on what can be done. Add in Web Workers to the magnitude speed bump of JS and Web devs can truly build responsive low-latency applications. We are now seeing improvements to the DOM, the next frontier. It is all well and good to have your JS run nice and speedy, but if the DOM can’t keep up then we are still in trouble. This is why we had to go to Canvas for Bespin in the first place! I am hoping that we will be able to push on DOM and other APIs to get to a point where our DOM based Bespin can come back to town.

There is a time and a place for Canvas though. Rather than running the show, I much prefer the unobtrusive extensions that you can provide. I am also looking forward to seeing more tools that use a server side component to do some image manipulation… now work on the client through Canvas and ImageData. Vlad pondered why CSS spriting tools are all server-side for example.

Christian Effenberger has been doing this kind of work for a long time, and there is room for a lot more. I really like having Canvas available in more places, such as anywhere an image can be used (e.g. WebKit CSS extensions where you can use a Canvas as a background-image). That gives you full control to style something in zany ways, while still staying in DOM land. It also means that things can play together really nice. We continue to push more into CSS (transitions, gradients, etc.) and animation libraries (Scripty2 looks nice!).

There is still much work to be done on putting the pieces together. I don’t know about you, but doing some common things with layout and UI still drive me nuts. Having the low level new tools like border-image are cool, but I really long for a tool that graphically lets me take a mock, cut it up, and build it out as a generalized component (stretches in the right place etc). The tool could then spit out more than just a border-image version, but old style divs + CSS to make it work everywhere. One tool, run output anywhere.


So, time to take a breath, take a look at what bleeding edge folk are doing to get rich experiences, and push it into the core platform itself. This will mean less Canvas, but used in the right places, and we will move down the stack again. If we don’t push use cases down into the core, then we are doomed to stagnate or fork the Web.

May 15

Bespin gives you some Pie and will do more Web Work for you

Bespin, Tech, UI / UX with tags: , , 2 Comments »

There have been a couple of things that have been really fun to see and work on in Bespin recently.

First up, is our new Pie Menu that Ben just posted on. The short reasons why I like the new look are:

  • Giving as much area to your code as possible. Why show a command line all the time when you only need it when you run commands? Why have headers and subheaders?
  • Pushing the boundaries on subtle animated effects that full screen change the bits “on the glass”

The implementation is interesting too. We took the nice animation and easing functions that Dojo already provides, but instead of it powering the DOM, we hook in and have it power our Canvas animations instead.

For example, to show an incoming animation we use a fadeIn effect with the backOut easing. In onAnimate we use the opacity setting that the animation provides to render our own animation:

var anim = dojo.fadeIn({
    node: { // fake it out for Dojo to think it is changing the style :)
        style: {}
    duration: 500,
    easing: dojo.fx.easing.backOut,
    onAnimate: function(values) {
        var progress = values.opacity;

Here you can see it in action in video form:

Or, go to the live sample.

We are excited to pump up the “fun” in the UI (with options to tone it down of course). Once nice side effect is that the editor component will have this all baked in, so where ever you see a Bespin, you will be able to access the features you want.

Web Workers; Not having them do too much

Malte Ubl continues to do much appreciated interesting work for the Bespin project. His facade around Web Workers is good stuff.

Since we are trying to push the boundaries of the Web hard, and giving some sexy to the UI, we need to make sure that we keep responsiveness as the highest priority, else users will go nuts.

Web Workers give us the ability to do a lot more in Bespin, but outside of the main browser “thread”. For example, doing real-time analysis on the code itself (e.g. setsyntaxcheck all). The problem that we ran into as we started to use Workers for this type of thing, was making sure that we weren’t killing the poor CPU. It is all well and good to fire off work to a Worker, but how many are running crunching? Malte has done work to make sure that the CPU fan doesn’t go nuts if you are doing a lot of work, and this is something that people will have to generally work around as they start to do more and more in their Web applications. It may be nice in the future to have a richer API that you could use to time slice your background tasks in a better way, as there is a lot of work that fits into the “kick this out and run at your convenience” mould.

Malte recently posted on how added support for the Bespin event loop into the Workers themselves. This is really nice as a Bespin developer as you can use the same publish/subscribe event metaphor, and the Worker facade handles it all for you, so you can talk across the boundaries. This hugely changes the amount of code that you have to write.

He also added support to help the asynchronous issues around “Can you fire this, but make sure that it only fires after foo, bar, and baz are all fired”. We will probably need to cache certain published events so we don’t get into a situation where you run this code after a main initialization event has fired, and thus you will never see it again.

Building a Web application using the latest and greatest HTML5 features is a lot of fun, and we hope that developers will like it too.

Mar 25

Canvas 3D, standards, and where

Mozila, Open Source, Tech, Web Browsing with tags: , 6 Comments »

I was excited to hear about the Canvas 3D effort that Mozilla, Google, and Khronos are engaged in (and others can too of course).

Khronos is the group being OpenGL, and thus a good set of folks to be involved in the Canvas 3D approach that is in the mould of “OpenGL ES like for the Web” in that it is a low level API that others can build on top of. Others have played with higher level “Games” APIs, or virtual worlds, and this is not the same. It is a primitive that will enable people to do interesting things that sit on top.

I noted Ryan Stewart (friend and great chap) weighing in:

So it’s unfortunate to see that even the browser vendors have given up on moving the open web forward through standards. Whether it’s the WHATWG versus the W3C or the trials and tribulations of actually implementing HTML5, things are very broken and everyone is moving on regardless. I don’t blame any of them, but it doesn’t seem like it’s good for web developers.

Then, I saw John Dowdell, also of Adobe, talking about standards.

I already talked about how many of the leaps on the Web haven’t started in the W3C (and rarely start inside standards orgs first) and rather come out in browser implementations that are then shared. Think XMLHttpRequest. Think Canvas itself from Apple! Do something well, see people use it and get excited about it, and then get multiple implementations and standards. Everyone wins.

John’s wording is interesting:

“But Mozilla’s proposal relies upon further proprietary extensions to the experimental CANVAS tag”

“And you’d lose the moral fulsomeness of the ‘Web Standards for The Open Web!’ pitch when focusing on your own proprietary alternatives to existing standards.”

Look at how browsers have done some things recently. Take some of the new CSS work that Apple started out. When the Mozilla community liked what they saw, and had developers demanding, they went and implemented it too. When you see WebKit and Gecko doing this kind of work it is particularly Open because the projects are open source and you can check them out (well, if you are allowed ;) How great is that, to iterate nicely in the open…. and then when ready we can drive into the standards bodies.

Back to the Canvas 3D work. Having Mozilla, Google, and Khronos work on this in the open seems pretty darn good to me. This won’t be hidden behind a proprietary binary that no-one can see. There will be some work in marrying the world of OpenGL ES and JavaScript as nicely as possible, and there will be plenty of room for the jQuery/Dojo/Prototype/YUI/…. of the world to do nice abstractions on top, but this is good stuff. This is more than just throwing out an API on top of a proprietary system, and I can’t wait to see what comes of it all. Want to get involved? You can in this world.