techno.blog("Dion")

We are having a great time using Canvas for Bespin and a few other projects. Having text, images, and boxes as the primitives on the Web just isn’t good enough, so leap frogging to HTML5-land where we have the ability to arbitrarily paint pixels is fantastic. Mix this with the ability to directly manipulate video, and you have new opportunities.

However, Canvas also has obvious issues. How do you make it accessible? Or searchable? We have some ideas there, but they are a long way off. The goal has never been to use Canvas as the shiny toy. With Bespin, we actually tried to do it with DOM, and it wasn’t so much chutzpah as “can’t get it to perform”-pah that lead us down the path of Canvas. The usual suspects in the stack weren’t working for us, whereas others were blazing (e.g. JavaScript performance has blown us away, as has Canvas itself).

With major browsers like Firefox 3.5 (Safari 4, Chrome 2, etc) I am very much excited to work with browsers (say, the great Firefox team :) to kick into gear on the next level of improvements. The current batch have been competing on the performance of JavaScript. The new engines are fantastic, and really change the game on what can be done. Add in Web Workers to the magnitude speed bump of JS and Web devs can truly build responsive low-latency applications. We are now seeing improvements to the DOM, the next frontier. It is all well and good to have your JS run nice and speedy, but if the DOM can’t keep up then we are still in trouble. This is why we had to go to Canvas for Bespin in the first place! I am hoping that we will be able to push on DOM and other APIs to get to a point where our DOM based Bespin can come back to town.

There is a time and a place for Canvas though. Rather than running the show, I much prefer the unobtrusive extensions that you can provide. I am also looking forward to seeing more tools that use a server side component to do some image manipulation… now work on the client through Canvas and ImageData. Vlad pondered why CSS spriting tools are all server-side for example.

Christian Effenberger has been doing this kind of work for a long time, and there is room for a lot more. I really like having Canvas available in more places, such as anywhere an image can be used (e.g. WebKit CSS extensions where you can use a Canvas as a background-image). That gives you full control to style something in zany ways, while still staying in DOM land. It also means that things can play together really nice. We continue to push more into CSS (transitions, gradients, etc.) and animation libraries (Scripty2 looks nice!).

There is still much work to be done on putting the pieces together. I don’t know about you, but doing some common things with layout and UI still drive me nuts. Having the low level new tools like border-image are cool, but I really long for a tool that graphically lets me take a mock, cut it up, and build it out as a generalized component (stretches in the right place etc). The tool could then spit out more than just a border-image version, but old style divs + CSS to make it work everywhere. One tool, run output anywhere.

So, time to take a breath, take a look at what bleeding edge folk are doing to get rich experiences, and push it into the core platform itself. This will mean less Canvas, but used in the right places, and we will move down the stack again. If we don’t push use cases down into the core, then we are doomed to stagnate or fork the Web.

Imagine this scenario. Say you worked for a company that had a top website and a new browser came out. There were some known bugs when said browser used your website, so you would obviously change the website ASAP. Chance are that you are ahead of the game and already had the fixes ready as you tested with release candidates. Your users grab the latest and greatest browser and use your website with glee.

Now imagine if the following happened: the browser vendor was in charge of your website and you had to ask nee plea with it to update your site. The vendor refuses at first and you have no control at all! All of the users that have upgraded their browser are now getting a sub-par experience. It’s your site, but you have no control. You curse as you know that the fix is RIGHT FRIGGIN THERE but hasn’t been applied.

This is crazy talk. You would never cede control like that. But wait, isn’t that what can happen with the iPhone and Apple? Facebook is one of the most popular applications on the iPhone, yet I read the following from poor Joe Hewitt, the guru that single-handedly created Facebook for iPhone, as well as the original Firebug, worked on Firefox, and much much more:

Submitted an update to the Facebook app a few weeks ago to fix OS 3.0 issues, still awaiting approval.

As of right now, if I am running Facebook on the iPhone 3Gs there are known bugs that Joe has fixed. I can’t run it. This is how they treat THEIR TOP APPLICATIONS which has a brand name company behind it? Imagine how they treat the poor lowly buggers on the totem pole.

Well, actually, we don’t really have to wonder too much. How gutted are you if you bet a year of your company on something with some amount of apparent blessing from Apple, and then see it thrown back in your face with some rule cited … when you know that the guy next door got his work in even though he is doing the same or worse!

Back to Joe. He got the latest 3Gs and talked about how smooth it is. This is great news. I feel like I often have a frozen UI on my 3G (typing, scrolling, any interaction) and it bugs me. If Apple has gotten their device “responsive enough” where the average user doesn’t notice these glitches, they have gone a long way, and as Joe mentions…. will give him time to work on features and not tweaking his code to try as hard as possible to keep his UI responsive (which in some circumstances was proving impossible). Ben has told me “It’s like the PowerBook G4 to MacBook Pro upgrade dude” which is great to hear since recently all upgrades have felt like a wash (e.g. recent Macbook Pro upgrades).

Also, I took another look at the applications that I have installed on my phone, and quickly realized that I use ~5% of them. I have a huge number that I have probably run once. When I look at the ones that I do use, I also quickly saw that the vast majority could easily be web applications, especially with the HTML5 features of Mobile Safari. If Apple continue to open up the API to the WebView (or PhoneGap / Titanium Mobile continue to thrive) then it will be so easy to create a large swarth of applications using simple Web tech. Sure there will be reasons to go closer to the metal, but for a lot…. you won’t have to. That is exciting stuff. I would also love to chat more with Joe about his experience using Cocoa vs. Web tech for layout etc. We have chatted a little, but I would love to talk more.

I went back and forth on getting an upgrade this time around. I was a definite “No” at first due to the pricing, and then AT&T came out saying that they would be helping out, so Ben and I went to check it out. Note: we both were in the same line for the 3G last time around, and at the store we found out that HE got to get the phone for $200 (as a “valued customer”) whereas I would have the pleasure of paying $300 more. I couldn’t whip out my credit card and walk out with the same hardware as him for a shed load more. So, now he will get the chance to lord over me until I do make the upgrade “wow this compass is amazing. wow the responsiveness is fantastic. How did I live without video right in my pocket.” Or, maybe I will jump to a Pre and start hacking on the native SDK that is just Web itself?

It was cool to see the second release of Jetpack with its storage, slidebars, and time travel from the .future().

I quickly hacked up a trivial slidebar that lets me mouse to see the source of the current tab, and click on it to have it stick around. All in a few lines of code that use the new future API, slideBar, and tabs:

jetpack.future.import("slideBar");
 
jetpack.slideBar.append({
  url: "view-source:" + jetpack.tabs.focused.url,
  width: 500,
  onSelect: function(slide) slide({ size: 500 }),
  onReady: function(slide) $(slide.doc).click(function() {
    slide({ size: 500, persist: true });
  })
});

Check out the screencast too:

With a wife with a masters in education, you tend to think and chat about education from time to time. I enjoy thinking about how social software could help education in the future. One of the fields that I enjoy thinking about is Maths. It was my favorite subject as a kid and I had the benefit of a great teacher. In general though, it feels like Math is often taught in such a painful way.

I remember being sent an article that discussed how Math is taught by comparing it to music. It postulates a world in which music is taught without instruments. You have to learn music theory for 10 years before you get to pick up that flute. What kid would get excited about a world of that music? That is what we do to them with Math.

Time for Pie

I mentioned the Bespin pie menu before. In our first version you jump to the different pieces with the keyboard. Of course, one of the benefits of a pie menu is that Fittss’s Law kicks in and you can easily jump to an item. (Aside: the dirty secret of our pie menu is that it isn’t really a pie menu, it currently sits at the bottom and not where you right click etc.)

As we start to put in mouse support, the first thing we need to do is be able to determine which slice of the pie is being clicked on so we can activate it. This gave me the opportunity to go down trigonometry lane.

If the pie was sliced with vertical and horizontal lines like the image above, then it would be very simple indeed. To start with, what information do you get when the user clicks? We get an x and y coordinate with an origin starting point in the top left. There are various x and y coordinates that you will get from the MouseEvent (x, y, offsetX/Y, pageX/Y, rangeX/Y, clientX/Y, layerX/Y). We care about the offsetX/Y or layerX/Y value (depending on the browser), which will give us the point with the origin starting at the top left of the canvas itself and not the window.

With the example above you could simply take the computed x and y coordinates and simply check them to determine the segment. For example, if x is less than the radius then it is a segment on the left. If y is less than the radius then it is a segment on top.

But we don’t have that luxury. Instead we have to do some simple Math. I drew out the problem with various bits of information here:

You will see the slices match the pie menu, and I drew in an X and Y axis that cuts through the center of the pie. In the diagram you will see an “x” marking the click in the right slice. How do we work it out? Well, what do we know? First we can calculate the location of the point based on the center of the circle. Remember that the browser will tell you the x and y based on the top left and not the center, so we quickly repurpose the coordinate based on the top left of 390, 180 and convert it to 140, 70. This is simply done via:

function centerPoint(x, y) {
    return { x: x - RADIUS, y: (y - RADIUS) * -1 };
}

Now we have the relative point from the center of the circle we can calculate the angle that will then in turn let us know which quadrant we are in.

Arctan and atan2()

The easiest way to calculate which quadrant we are in is to simply calculate the angle of the dangle. To do that we calculate the arc tangent between two lines in a triangle, the y=0 line across the x axis, and a line between y=0 and the point in question.

It turns out that we get this calculation for free, via Math.atan2:

function angle(x, y) {
    return Math.atan2(y, x) * 180 / Math.PI;
}

To explain how this works, see the scribble below that shows 4 different points and the resulting angles based on angle(). One gotcha that you will notice is that atan2() takes the coordinates in the order: y, x but elsewhere you are used to using the opposite x and then y. Be careful!

Once we have the angle based on the x axis, we can have a simple case statement to check the bounds.

If the angle is below -45 and 45 degrees, it is on the right. Greater than 45 and less than 135? Then we have the top. And so on.

With a little fun Math, we get to have Pi solve our Pie problem. To see the code at work, check out the test bed.

Ben and I both got new work 15″ Macbook Pro laptops, and Ben isn’t happy.

I have to admit that it doesn’t feel like the Apple designers may have gone one step too far with the trackpad. In one fell-swoop they took away the button and added new and exciting gestures.

The big problem I have with this is that I have learned to use the trackpad with one thumb at the bottom (where the button was) and I use my index finger to move around. There is a lot of muscle memory there. What this means in practice is that I am in Gmail and the pinch gesture kicks in which results in my font changing size, or the rotate gesture causes an image to skew in Keynote.

In theory, it is beautiful to get rid of the button. In practice I miss the tactile feedback, and the separation of concerns.

Maybe I should quickly grab a Macbook Air which still has the small little trim mouse button?

To those at you who have been using the latest generation of Macbooks, do you get over this? Will this be something that I forget in a month and life will actually be better?

Rich Hickey (Clojure creator) gave one of the best talks that I have been in for awhile. The title of the talk was scarily low level and “wow this must be boring”-y, but the title should have been something like “Learn about how the real world works”.

The first part of the talk discussed philosophy and how our software is totally messed up due to us not taking the factor of time into account properly. So many functional programming talks go into the bizarreness of Monad details and the like, in a way that makes a non-Functional chap’s head blow up.

Rich on the other hand didn’t go there. No need to say “Monads are in Haskell as an escape hatch past the whole state issue”. Instead he discussed the general notion of time and how it relates to variables. Short version: variables are variable in that they change over time, so really we want immutable values with different ones at different points. He talked about the different notions of time (before, at the same time, after, now, etc) and how the concurrency / thread problems become a touch more obvious when you think about things in that way.

And it went on from here. He talked about the great design of the data structures in Clojure and how they almost always uses a nice multiple version control system that enables a lot of things to “just work” in a concurrent and safe way. And of course, you have to have a little bit of fun with the Java Data API and how you can do things like someDate.setMonth().

This is the kind of talk that will make you think differently about your code, even if you are not in functional world. You will think about taking value snapshots to work with instead of passing around quite so many references. It was highly entertaining and will have me thinking for quite some time.

Related Interview: Rich Hickey on Clojure’s Features and Implementation

In this interview taped at QCon London 2009, Rich Hickey talks about all things Clojure: Software Transactional Memory, concurrency, persistent data structures, ports, AOT compilation, and more.

A contingent in the JavaScript community is pretty much solely focused on securing the language. Security is always important, but even more-so when you are dealing with a language that runs in a user agent to run the Web.

Whenever you try to add a feature to a language you are instantly dealing with legacy and backwards compatibility.

Perl has a strict mode that is very popular in the community. I remember projects where if I checked in a module without use strict; and -w I would get beaten up.

Back to JavaScript, how can we apply stricter rules on what we do in the JavaScript runtime after the face? One solution that the ECMAScript Edition 5 folks came up with is borrowing use strict but since they couldn’t add it to the language itself, they have you put it in a string "use strict";. You can place it at the top of the file, or even as the first statement in a function.

I find it incredibly ugly, yet pragmatic. It works.

However, then I saw some new items:

"use strict,cajita";

And currently you can’t switch the order around? It looks for absolute keys? That seems a little crazy. Surely all of these should be possible:

"use cajita,strict";
"use strict,cajita";
"use strict";
"use cajita";

Then this little pragmatic hack starts to get real messy. I start to joking imagine a ludicrous world like this:

(function() {
  "use strict";
  "use let"; // let's get let in there!
  "use cramda"; // I love Alex's cramda, let me use it!
  "use e4x"; // XML inline. Screw templates
 
  // If I see a var with the same name, REALLY use let instead
  "let x;"
  var x;
 
  // Now I can use the cramda
  "x = #(a, b) { "\
  " ... logic here ... "\
  "};"
 
  // And the let... ironic that it is in a string now
  "let y = <xml></xml>";
 
  // Annotations come to JavaScript!
  "@Serializable"
  function someThing() {}
})();

I like to think of myself as pragmatic, but why do I feel so cranky about "use strict";? :)

By now you have probably read about the critical Java vulnerability and how easy it is to take over a machine from a web page via a Java applet.

Apparently, Sun fixed it fairly quickly, but even then some people say their fix was too specific and thus there are still problems. Apple on the other hand, were not diligent and are still yet to provide a fix. Ouch.

Painful news as JavaOne approaches and Sun tries to push JavaFX. I was trying to think of a Java applet that I have used (knowingly) in the recent past, and I think the only candidate is the Facebook photo uploader.

I have been watching some research for a talk on JavaFX and I am really torn. In theory, the Java platform is phenomenal and should be a great choice for doing this kind of development. The scene graph work in JavaFX is very nicely done, but the implementation seems to be a touch off in much of what I have seen. Scrolling causing the applet to go blank? Browser crashes? Ouch. But, JavaFX is new, and has a chance to get better. Their problem is that they are squeezed from both sides. The browser platform itself is accelerating quickly, and it has the advantage of being native. Once you go to plugin land you are competing with Flash with its large share and proven ability.

Getting webby

I find myself wanting to get more and more webby. This is why I was excited to work with the Jetpack project that we just launched today at Mozilla Labs. Being able to extend the browser using Web technology itself is going to open the door for more playing. People have created 7000 addons for Firefox alone, but as someone who has done a touch of XUL, I am happy to stay in my familiar territory of HTML, CSS, JS. Jetpack has just been born, and is incredibly early stage, but I can’t wait to see it grow and get the APIs that people want in a secure extensible model.

You will see a lot of Bespin in there too. Below is a screencast where I create a Jetpack feature on the fly (no reloads!) using Bespin, and once you install Jetpack it embeds the puppy into the tutorial and the developer area too. Much more than can be done though!

There have been a couple of things that have been really fun to see and work on in Bespin recently.

First up, is our new Pie Menu that Ben just posted on. The short reasons why I like the new look are:

• Giving as much area to your code as possible. Why show a command line all the time when you only need it when you run commands? Why have headers and subheaders?
• Pushing the boundaries on subtle animated effects that full screen change the bits “on the glass”

The implementation is interesting too. We took the nice animation and easing functions that Dojo already provides, but instead of it powering the DOM, we hook in and have it power our Canvas animations instead.

For example, to show an incoming animation we use a fadeIn effect with the backOut easing. In onAnimate we use the opacity setting that the animation provides to render our own animation:

var anim = dojo.fadeIn({
    node: { // fake it out for Dojo to think it is changing the style :)
        style: {}
    },
 
    duration: 500,
    easing: dojo.fx.easing.backOut,
 
    onAnimate: function(values) {
        var progress = values.opacity;
        renderPie(progress);
    }
}).play();

Here you can see it in action in video form:

Or, go to the live sample.

We are excited to pump up the “fun” in the UI (with options to tone it down of course). Once nice side effect is that the editor component will have this all baked in, so where ever you see a Bespin, you will be able to access the features you want.

Web Workers; Not having them do too much

Malte Ubl continues to do much appreciated interesting work for the Bespin project. His facade around Web Workers is good stuff.

Since we are trying to push the boundaries of the Web hard, and giving some sexy to the UI, we need to make sure that we keep responsiveness as the highest priority, else users will go nuts.

Web Workers give us the ability to do a lot more in Bespin, but outside of the main browser “thread”. For example, doing real-time analysis on the code itself (e.g. setsyntaxcheck all). The problem that we ran into as we started to use Workers for this type of thing, was making sure that we weren’t killing the poor CPU. It is all well and good to fire off work to a Worker, but how many are running crunching? Malte has done work to make sure that the CPU fan doesn’t go nuts if you are doing a lot of work, and this is something that people will have to generally work around as they start to do more and more in their Web applications. It may be nice in the future to have a richer API that you could use to time slice your background tasks in a better way, as there is a lot of work that fits into the “kick this out and run at your convenience” mould.

Malte recently posted on how added support for the Bespin event loop into the Workers themselves. This is really nice as a Bespin developer as you can use the same publish/subscribe event metaphor, and the Worker facade handles it all for you, so you can talk across the boundaries. This hugely changes the amount of code that you have to write.

He also added support to help the asynchronous issues around “Can you fire this, but make sure that it only fires after foo, bar, and baz are all fired”. We will probably need to cache certain published events so we don’t get into a situation where you run this code after a main initialization event has fired, and thus you will never see it again.

Building a Web application using the latest and greatest HTML5 features is a lot of fun, and we hope that developers will like it too.

I was incredibly excited to see Dan Mills of the Weave team show off OpenID in the browser thanks to Weave. I have wanted this from long before joining Mozilla and Dan has done a great job and doing something smart to make it happen.

The key to this all is the notion of “logging into the browser.” With the Weave project this is exactly what you do. Although the first service that Weave offered was the ability to sync bookmarks, all along it has been setup for much much more, and this is a perfect example of what the integration of a browser with a service backend can mean. One thing to note is that the Weave team really cares about privacy. The way the Weave system works right now is that the server has encrypted bits that it can never look at and understand. Instead, the client itself has the private keys that it needs to do the real work of holding the data, thus the browser, which is a natural extension to yourself the user, has full control. Think about how that differs from many services up in the cloud that make a lot of money off of your data (not that making money is bad!)

Back to the point at hand. Dan shows us how, once you are logged into Weave (and thus the browser) it will watch out for input’s that match the openid spec (openid_identifier, openid_url) and replaces it with a simple button that tells you that you can just log in! With a click on the button you get logged in automatically via Weave and you are on your way. Now Ben will be able to login to the Web 2.0 Expo site with one click instead of a series of nightmare steps. I think this is a pretty big deal.

Here it is in action:

In one fell swoop we get:

• A usable login procedure that is trivial!
• We don’t have to give trust to some third party site that we are giving the keys to the treasure too. Imagine if you used OpenID to login to your bank… would you trust your OpenID provider enough for that?
• It is important to note that they also integrated the Firefox password manager into the mix. This means that you can set auto login right from the URL bar widget (with your password, as well as with OpenID!)

Of course, this is early days, as Dan says in his post

Keep in mind that this is just a prototype that we hacked together in a few of days, so there are some very rough edges. But we’re super excited about the possibilities already. When demoing it to people, they said things like “whoa! how did you do that?” and “I want this! How do I get it?”

The answer is, “soon”, or if you’re brave enough you can try it out right now by installing the latest Weave development snapshot. Please let us know what you think by posting on the Weave forum!

People and Places

Now we are getting the notion of “people” and Identity in the browser, and we also have “location” too, we are really getting somewhere. A great week!