techno.blog("Dion")

We are having a great time using Canvas for Bespin and a few other projects. Having text, images, and boxes as the primitives on the Web just isn’t good enough, so leap frogging to HTML5-land where we have the ability to arbitrarily paint pixels is fantastic. Mix this with the ability to directly manipulate video, and you have new opportunities.

However, Canvas also has obvious issues. How do you make it accessible? Or searchable? We have some ideas there, but they are a long way off. The goal has never been to use Canvas as the shiny toy. With Bespin, we actually tried to do it with DOM, and it wasn’t so much chutzpah as “can’t get it to perform”-pah that lead us down the path of Canvas. The usual suspects in the stack weren’t working for us, whereas others were blazing (e.g. JavaScript performance has blown us away, as has Canvas itself).

With major browsers like Firefox 3.5 (Safari 4, Chrome 2, etc) I am very much excited to work with browsers (say, the great Firefox team :) to kick into gear on the next level of improvements. The current batch have been competing on the performance of JavaScript. The new engines are fantastic, and really change the game on what can be done. Add in Web Workers to the magnitude speed bump of JS and Web devs can truly build responsive low-latency applications. We are now seeing improvements to the DOM, the next frontier. It is all well and good to have your JS run nice and speedy, but if the DOM can’t keep up then we are still in trouble. This is why we had to go to Canvas for Bespin in the first place! I am hoping that we will be able to push on DOM and other APIs to get to a point where our DOM based Bespin can come back to town.

There is a time and a place for Canvas though. Rather than running the show, I much prefer the unobtrusive extensions that you can provide. I am also looking forward to seeing more tools that use a server side component to do some image manipulation… now work on the client through Canvas and ImageData. Vlad pondered why CSS spriting tools are all server-side for example.

Christian Effenberger has been doing this kind of work for a long time, and there is room for a lot more. I really like having Canvas available in more places, such as anywhere an image can be used (e.g. WebKit CSS extensions where you can use a Canvas as a background-image). That gives you full control to style something in zany ways, while still staying in DOM land. It also means that things can play together really nice. We continue to push more into CSS (transitions, gradients, etc.) and animation libraries (Scripty2 looks nice!).

There is still much work to be done on putting the pieces together. I don’t know about you, but doing some common things with layout and UI still drive me nuts. Having the low level new tools like border-image are cool, but I really long for a tool that graphically lets me take a mock, cut it up, and build it out as a generalized component (stretches in the right place etc). The tool could then spit out more than just a border-image version, but old style divs + CSS to make it work everywhere. One tool, run output anywhere.

So, time to take a breath, take a look at what bleeding edge folk are doing to get rich experiences, and push it into the core platform itself. This will mean less Canvas, but used in the right places, and we will move down the stack again. If we don’t push use cases down into the core, then we are doomed to stagnate or fork the Web.

It was cool to see the second release of Jetpack with its storage, slidebars, and time travel from the .future().

I quickly hacked up a trivial slidebar that lets me mouse to see the source of the current tab, and click on it to have it stick around. All in a few lines of code that use the new future API, slideBar, and tabs:

jetpack.future.import("slideBar");
 
jetpack.slideBar.append({
  url: "view-source:" + jetpack.tabs.focused.url,
  width: 500,
  onSelect: function(slide) slide({ size: 500 }),
  onReady: function(slide) $(slide.doc).click(function() {
    slide({ size: 500, persist: true });
  })
});

Check out the screencast too:

With a wife with a masters in education, you tend to think and chat about education from time to time. I enjoy thinking about how social software could help education in the future. One of the fields that I enjoy thinking about is Maths. It was my favorite subject as a kid and I had the benefit of a great teacher. In general though, it feels like Math is often taught in such a painful way.

I remember being sent an article that discussed how Math is taught by comparing it to music. It postulates a world in which music is taught without instruments. You have to learn music theory for 10 years before you get to pick up that flute. What kid would get excited about a world of that music? That is what we do to them with Math.

Time for Pie

I mentioned the Bespin pie menu before. In our first version you jump to the different pieces with the keyboard. Of course, one of the benefits of a pie menu is that Fittss’s Law kicks in and you can easily jump to an item. (Aside: the dirty secret of our pie menu is that it isn’t really a pie menu, it currently sits at the bottom and not where you right click etc.)

As we start to put in mouse support, the first thing we need to do is be able to determine which slice of the pie is being clicked on so we can activate it. This gave me the opportunity to go down trigonometry lane.

If the pie was sliced with vertical and horizontal lines like the image above, then it would be very simple indeed. To start with, what information do you get when the user clicks? We get an x and y coordinate with an origin starting point in the top left. There are various x and y coordinates that you will get from the MouseEvent (x, y, offsetX/Y, pageX/Y, rangeX/Y, clientX/Y, layerX/Y). We care about the offsetX/Y or layerX/Y value (depending on the browser), which will give us the point with the origin starting at the top left of the canvas itself and not the window.

With the example above you could simply take the computed x and y coordinates and simply check them to determine the segment. For example, if x is less than the radius then it is a segment on the left. If y is less than the radius then it is a segment on top.

But we don’t have that luxury. Instead we have to do some simple Math. I drew out the problem with various bits of information here:

You will see the slices match the pie menu, and I drew in an X and Y axis that cuts through the center of the pie. In the diagram you will see an “x” marking the click in the right slice. How do we work it out? Well, what do we know? First we can calculate the location of the point based on the center of the circle. Remember that the browser will tell you the x and y based on the top left and not the center, so we quickly repurpose the coordinate based on the top left of 390, 180 and convert it to 140, 70. This is simply done via:

function centerPoint(x, y) {
    return { x: x - RADIUS, y: (y - RADIUS) * -1 };
}

Now we have the relative point from the center of the circle we can calculate the angle that will then in turn let us know which quadrant we are in.

Arctan and atan2()

The easiest way to calculate which quadrant we are in is to simply calculate the angle of the dangle. To do that we calculate the arc tangent between two lines in a triangle, the y=0 line across the x axis, and a line between y=0 and the point in question.

It turns out that we get this calculation for free, via Math.atan2:

function angle(x, y) {
    return Math.atan2(y, x) * 180 / Math.PI;
}

To explain how this works, see the scribble below that shows 4 different points and the resulting angles based on angle(). One gotcha that you will notice is that atan2() takes the coordinates in the order: y, x but elsewhere you are used to using the opposite x and then y. Be careful!

Once we have the angle based on the x axis, we can have a simple case statement to check the bounds.

If the angle is below -45 and 45 degrees, it is on the right. Greater than 45 and less than 135? Then we have the top. And so on.

With a little fun Math, we get to have Pi solve our Pie problem. To see the code at work, check out the test bed.

Ben and I both got new work 15″ Macbook Pro laptops, and Ben isn’t happy.

I have to admit that it doesn’t feel like the Apple designers may have gone one step too far with the trackpad. In one fell-swoop they took away the button and added new and exciting gestures.

The big problem I have with this is that I have learned to use the trackpad with one thumb at the bottom (where the button was) and I use my index finger to move around. There is a lot of muscle memory there. What this means in practice is that I am in Gmail and the pinch gesture kicks in which results in my font changing size, or the rotate gesture causes an image to skew in Keynote.

In theory, it is beautiful to get rid of the button. In practice I miss the tactile feedback, and the separation of concerns.

Maybe I should quickly grab a Macbook Air which still has the small little trim mouse button?

To those at you who have been using the latest generation of Macbooks, do you get over this? Will this be something that I forget in a month and life will actually be better?

Rich Hickey (Clojure creator) gave one of the best talks that I have been in for awhile. The title of the talk was scarily low level and “wow this must be boring”-y, but the title should have been something like “Learn about how the real world works”.

The first part of the talk discussed philosophy and how our software is totally messed up due to us not taking the factor of time into account properly. So many functional programming talks go into the bizarreness of Monad details and the like, in a way that makes a non-Functional chap’s head blow up.

Rich on the other hand didn’t go there. No need to say “Monads are in Haskell as an escape hatch past the whole state issue”. Instead he discussed the general notion of time and how it relates to variables. Short version: variables are variable in that they change over time, so really we want immutable values with different ones at different points. He talked about the different notions of time (before, at the same time, after, now, etc) and how the concurrency / thread problems become a touch more obvious when you think about things in that way.

And it went on from here. He talked about the great design of the data structures in Clojure and how they almost always uses a nice multiple version control system that enables a lot of things to “just work” in a concurrent and safe way. And of course, you have to have a little bit of fun with the Java Data API and how you can do things like someDate.setMonth().

This is the kind of talk that will make you think differently about your code, even if you are not in functional world. You will think about taking value snapshots to work with instead of passing around quite so many references. It was highly entertaining and will have me thinking for quite some time.

Related Interview: Rich Hickey on Clojure’s Features and Implementation

In this interview taped at QCon London 2009, Rich Hickey talks about all things Clojure: Software Transactional Memory, concurrency, persistent data structures, ports, AOT compilation, and more.

A contingent in the JavaScript community is pretty much solely focused on securing the language. Security is always important, but even more-so when you are dealing with a language that runs in a user agent to run the Web.

Whenever you try to add a feature to a language you are instantly dealing with legacy and backwards compatibility.

Perl has a strict mode that is very popular in the community. I remember projects where if I checked in a module without use strict; and -w I would get beaten up.

Back to JavaScript, how can we apply stricter rules on what we do in the JavaScript runtime after the face? One solution that the ECMAScript Edition 5 folks came up with is borrowing use strict but since they couldn’t add it to the language itself, they have you put it in a string "use strict";. You can place it at the top of the file, or even as the first statement in a function.

I find it incredibly ugly, yet pragmatic. It works.

However, then I saw some new items:

"use strict,cajita";

And currently you can’t switch the order around? It looks for absolute keys? That seems a little crazy. Surely all of these should be possible:

"use cajita,strict";
"use strict,cajita";
"use strict";
"use cajita";

Then this little pragmatic hack starts to get real messy. I start to joking imagine a ludicrous world like this:

(function() {
  "use strict";
  "use let"; // let's get let in there!
  "use cramda"; // I love Alex's cramda, let me use it!
  "use e4x"; // XML inline. Screw templates
 
  // If I see a var with the same name, REALLY use let instead
  "let x;"
  var x;
 
  // Now I can use the cramda
  "x = #(a, b) { "\
  " ... logic here ... "\
  "};"
 
  // And the let... ironic that it is in a string now
  "let y = <xml></xml>";
 
  // Annotations come to JavaScript!
  "@Serializable"
  function someThing() {}
})();

I like to think of myself as pragmatic, but why do I feel so cranky about "use strict";? :)

By now you have probably read about the critical Java vulnerability and how easy it is to take over a machine from a web page via a Java applet.

Apparently, Sun fixed it fairly quickly, but even then some people say their fix was too specific and thus there are still problems. Apple on the other hand, were not diligent and are still yet to provide a fix. Ouch.

Painful news as JavaOne approaches and Sun tries to push JavaFX. I was trying to think of a Java applet that I have used (knowingly) in the recent past, and I think the only candidate is the Facebook photo uploader.

I have been watching some research for a talk on JavaFX and I am really torn. In theory, the Java platform is phenomenal and should be a great choice for doing this kind of development. The scene graph work in JavaFX is very nicely done, but the implementation seems to be a touch off in much of what I have seen. Scrolling causing the applet to go blank? Browser crashes? Ouch. But, JavaFX is new, and has a chance to get better. Their problem is that they are squeezed from both sides. The browser platform itself is accelerating quickly, and it has the advantage of being native. Once you go to plugin land you are competing with Flash with its large share and proven ability.

Getting webby

I find myself wanting to get more and more webby. This is why I was excited to work with the Jetpack project that we just launched today at Mozilla Labs. Being able to extend the browser using Web technology itself is going to open the door for more playing. People have created 7000 addons for Firefox alone, but as someone who has done a touch of XUL, I am happy to stay in my familiar territory of HTML, CSS, JS. Jetpack has just been born, and is incredibly early stage, but I can’t wait to see it grow and get the APIs that people want in a secure extensible model.

You will see a lot of Bespin in there too. Below is a screencast where I create a Jetpack feature on the fly (no reloads!) using Bespin, and once you install Jetpack it embeds the puppy into the tutorial and the developer area too. Much more than can be done though!

There have been a couple of things that have been really fun to see and work on in Bespin recently.

First up, is our new Pie Menu that Ben just posted on. The short reasons why I like the new look are:

• Giving as much area to your code as possible. Why show a command line all the time when you only need it when you run commands? Why have headers and subheaders?
• Pushing the boundaries on subtle animated effects that full screen change the bits “on the glass”

The implementation is interesting too. We took the nice animation and easing functions that Dojo already provides, but instead of it powering the DOM, we hook in and have it power our Canvas animations instead.

For example, to show an incoming animation we use a fadeIn effect with the backOut easing. In onAnimate we use the opacity setting that the animation provides to render our own animation:

var anim = dojo.fadeIn({
    node: { // fake it out for Dojo to think it is changing the style :)
        style: {}
    },
 
    duration: 500,
    easing: dojo.fx.easing.backOut,
 
    onAnimate: function(values) {
        var progress = values.opacity;
        renderPie(progress);
    }
}).play();

Here you can see it in action in video form:

Or, go to the live sample.

We are excited to pump up the “fun” in the UI (with options to tone it down of course). Once nice side effect is that the editor component will have this all baked in, so where ever you see a Bespin, you will be able to access the features you want.

Web Workers; Not having them do too much

Malte Ubl continues to do much appreciated interesting work for the Bespin project. His facade around Web Workers is good stuff.

Since we are trying to push the boundaries of the Web hard, and giving some sexy to the UI, we need to make sure that we keep responsiveness as the highest priority, else users will go nuts.

Web Workers give us the ability to do a lot more in Bespin, but outside of the main browser “thread”. For example, doing real-time analysis on the code itself (e.g. setsyntaxcheck all). The problem that we ran into as we started to use Workers for this type of thing, was making sure that we weren’t killing the poor CPU. It is all well and good to fire off work to a Worker, but how many are running crunching? Malte has done work to make sure that the CPU fan doesn’t go nuts if you are doing a lot of work, and this is something that people will have to generally work around as they start to do more and more in their Web applications. It may be nice in the future to have a richer API that you could use to time slice your background tasks in a better way, as there is a lot of work that fits into the “kick this out and run at your convenience” mould.

Malte recently posted on how added support for the Bespin event loop into the Workers themselves. This is really nice as a Bespin developer as you can use the same publish/subscribe event metaphor, and the Worker facade handles it all for you, so you can talk across the boundaries. This hugely changes the amount of code that you have to write.

He also added support to help the asynchronous issues around “Can you fire this, but make sure that it only fires after foo, bar, and baz are all fired”. We will probably need to cache certain published events so we don’t get into a situation where you run this code after a main initialization event has fired, and thus you will never see it again.

Building a Web application using the latest and greatest HTML5 features is a lot of fun, and we hope that developers will like it too.

I was incredibly excited to see Dan Mills of the Weave team show off OpenID in the browser thanks to Weave. I have wanted this from long before joining Mozilla and Dan has done a great job and doing something smart to make it happen.

The key to this all is the notion of “logging into the browser.” With the Weave project this is exactly what you do. Although the first service that Weave offered was the ability to sync bookmarks, all along it has been setup for much much more, and this is a perfect example of what the integration of a browser with a service backend can mean. One thing to note is that the Weave team really cares about privacy. The way the Weave system works right now is that the server has encrypted bits that it can never look at and understand. Instead, the client itself has the private keys that it needs to do the real work of holding the data, thus the browser, which is a natural extension to yourself the user, has full control. Think about how that differs from many services up in the cloud that make a lot of money off of your data (not that making money is bad!)

Back to the point at hand. Dan shows us how, once you are logged into Weave (and thus the browser) it will watch out for input’s that match the openid spec (openid_identifier, openid_url) and replaces it with a simple button that tells you that you can just log in! With a click on the button you get logged in automatically via Weave and you are on your way. Now Ben will be able to login to the Web 2.0 Expo site with one click instead of a series of nightmare steps. I think this is a pretty big deal.

Here it is in action:

In one fell swoop we get:

• A usable login procedure that is trivial!
• We don’t have to give trust to some third party site that we are giving the keys to the treasure too. Imagine if you used OpenID to login to your bank… would you trust your OpenID provider enough for that?
• It is important to note that they also integrated the Firefox password manager into the mix. This means that you can set auto login right from the URL bar widget (with your password, as well as with OpenID!)

Of course, this is early days, as Dan says in his post

Keep in mind that this is just a prototype that we hacked together in a few of days, so there are some very rough edges. But we’re super excited about the possibilities already. When demoing it to people, they said things like “whoa! how did you do that?” and “I want this! How do I get it?”

The answer is, “soon”, or if you’re brave enough you can try it out right now by installing the latest Weave development snapshot. Please let us know what you think by posting on the Weave forum!

People and Places

Now we are getting the notion of “people” and Identity in the browser, and we also have “location” too, we are really getting somewhere. A great week!

I am really excited to see what comes out of all of the iPhone Tablet rumors.

My son is three years old now, and knows how to use my iPhone. When using the computer, he instinctively started to think of it like the iPhone and would start to touch (and smudge!) the screen.

In the real world we touch things. We grab them. We manipulate them. We don’t often have an abstract notion of moving X to get Y.

Well, there is one that Sam knows well:

These machines are pure evil when you have a young’un as they say “Daddy! Daddy! I want the pink one there!” and you have no chance in hell of ever getting it :)

Anyway, so Sam knows how to use multitouch, and is at an age where we are thinking about setting him up with his own computer. I don’t want to go through the pain to teach him a mouse and keyboard quite yet though, so what if there was a computer he could hold and could use his normal skills to access? The iPhone tablet could be that device!

As a side effect, it would also be the PERFECT home automation device, as well as an eBook reader, and a casual browser, and email checker, and the list goes on and on and on. The computer is changing in front of our eyes isn’t it.