Jun 29

Having fun with Canvas, but the aim is to have to use it less and less; Performance wars move from raw JS to DOM

JavaScript, Tech with tags: , 3 Comments »


We are having a great time using Canvas for Bespin and a few other projects. Having text, images, and boxes as the primitives on the Web just isn’t good enough, so leap frogging to HTML5-land where we have the ability to arbitrarily paint pixels is fantastic. Mix this with the ability to directly manipulate video, and you have new opportunities.

However, Canvas also has obvious issues. How do you make it accessible? Or searchable? We have some ideas there, but they are a long way off. The goal has never been to use Canvas as the shiny toy. With Bespin, we actually tried to do it with DOM, and it wasn’t so much chutzpah as “can’t get it to perform”-pah that lead us down the path of Canvas. The usual suspects in the stack weren’t working for us, whereas others were blazing (e.g. JavaScript performance has blown us away, as has Canvas itself).

With major browsers like Firefox 3.5 (Safari 4, Chrome 2, etc) I am very much excited to work with browsers (say, the great Firefox team :) to kick into gear on the next level of improvements. The current batch have been competing on the performance of JavaScript. The new engines are fantastic, and really change the game on what can be done. Add in Web Workers to the magnitude speed bump of JS and Web devs can truly build responsive low-latency applications. We are now seeing improvements to the DOM, the next frontier. It is all well and good to have your JS run nice and speedy, but if the DOM can’t keep up then we are still in trouble. This is why we had to go to Canvas for Bespin in the first place! I am hoping that we will be able to push on DOM and other APIs to get to a point where our DOM based Bespin can come back to town.

There is a time and a place for Canvas though. Rather than running the show, I much prefer the unobtrusive extensions that you can provide. I am also looking forward to seeing more tools that use a server side component to do some image manipulation… now work on the client through Canvas and ImageData. Vlad pondered why CSS spriting tools are all server-side for example.

Christian Effenberger has been doing this kind of work for a long time, and there is room for a lot more. I really like having Canvas available in more places, such as anywhere an image can be used (e.g. WebKit CSS extensions where you can use a Canvas as a background-image). That gives you full control to style something in zany ways, while still staying in DOM land. It also means that things can play together really nice. We continue to push more into CSS (transitions, gradients, etc.) and animation libraries (Scripty2 looks nice!).

There is still much work to be done on putting the pieces together. I don’t know about you, but doing some common things with layout and UI still drive me nuts. Having the low level new tools like border-image are cool, but I really long for a tool that graphically lets me take a mock, cut it up, and build it out as a generalized component (stretches in the right place etc). The tool could then spit out more than just a border-image version, but old style divs + CSS to make it work everywhere. One tool, run output anywhere.


So, time to take a breath, take a look at what bleeding edge folk are doing to get rich experiences, and push it into the core platform itself. This will mean less Canvas, but used in the right places, and we will move down the stack again. If we don’t push use cases down into the core, then we are doomed to stagnate or fork the Web.

Jun 20

iPhone 3Gs: “S” for responsiveness, but also subordination?

Apple, iPhone with tags: 7 Comments »


Imagine this scenario. Say you worked for a company that had a top website and a new browser came out. There were some known bugs when said browser used your website, so you would obviously change the website ASAP. Chance are that you are ahead of the game and already had the fixes ready as you tested with release candidates. Your users grab the latest and greatest browser and use your website with glee.

Now imagine if the following happened: the browser vendor was in charge of your website and you had to ask nee plea with it to update your site. The vendor refuses at first and you have no control at all! All of the users that have upgraded their browser are now getting a sub-par experience. It’s your site, but you have no control. You curse as you know that the fix is RIGHT FRIGGIN THERE but hasn’t been applied.

This is crazy talk. You would never cede control like that. But wait, isn’t that what can happen with the iPhone and Apple? Facebook is one of the most popular applications on the iPhone, yet I read the following from poor Joe Hewitt, the guru that single-handedly created Facebook for iPhone, as well as the original Firebug, worked on Firefox, and much much more:

Submitted an update to the Facebook app a few weeks ago to fix OS 3.0 issues, still awaiting approval.

As of right now, if I am running Facebook on the iPhone 3Gs there are known bugs that Joe has fixed. I can’t run it. This is how they treat THEIR TOP APPLICATIONS which has a brand name company behind it? Imagine how they treat the poor lowly buggers on the totem pole.

Well, actually, we don’t really have to wonder too much. How gutted are you if you bet a year of your company on something with some amount of apparent blessing from Apple, and then see it thrown back in your face with some rule cited … when you know that the guy next door got his work in even though he is doing the same or worse!

Back to Joe. He got the latest 3Gs and talked about how smooth it is. This is great news. I feel like I often have a frozen UI on my 3G (typing, scrolling, any interaction) and it bugs me. If Apple has gotten their device “responsive enough” where the average user doesn’t notice these glitches, they have gone a long way, and as Joe mentions…. will give him time to work on features and not tweaking his code to try as hard as possible to keep his UI responsive (which in some circumstances was proving impossible). Ben has told me “It’s like the PowerBook G4 to MacBook Pro upgrade dude” which is great to hear since recently all upgrades have felt like a wash (e.g. recent Macbook Pro upgrades).

Also, I took another look at the applications that I have installed on my phone, and quickly realized that I use ~5% of them. I have a huge number that I have probably run once. When I look at the ones that I do use, I also quickly saw that the vast majority could easily be web applications, especially with the HTML5 features of Mobile Safari. If Apple continue to open up the API to the WebView (or PhoneGap / Titanium Mobile continue to thrive) then it will be so easy to create a large swarth of applications using simple Web tech. Sure there will be reasons to go closer to the metal, but for a lot…. you won’t have to. That is exciting stuff. I would also love to chat more with Joe about his experience using Cocoa vs. Web tech for layout etc. We have chatted a little, but I would love to talk more.

I went back and forth on getting an upgrade this time around. I was a definite “No” at first due to the pricing, and then AT&T came out saying that they would be helping out, so Ben and I went to check it out. Note: we both were in the same line for the 3G last time around, and at the store we found out that HE got to get the phone for $200 (as a “valued customer”) whereas I would have the pleasure of paying $300 more. I couldn’t whip out my credit card and walk out with the same hardware as him for a shed load more. So, now he will get the chance to lord over me until I do make the upgrade “wow this compass is amazing. wow the responsiveness is fantastic. How did I live without video right in my pocket.” Or, maybe I will jump to a Pre and start hacking on the native SDK that is just Web itself?

Jun 12

Jetpack: View-source Slidebar from the future

Mozila, Tech with tags: 3 Comments »


It was cool to see the second release of Jetpack with its storage, slidebars, and time travel from the .future().

I quickly hacked up a trivial slidebar that lets me mouse to see the source of the current tab, and click on it to have it stick around. All in a few lines of code that use the new future API, slideBar, and tabs:

  url: "view-source:" + jetpack.tabs.focused.url,
  width: 500,
  onSelect: function(slide) slide({ size: 500 }),
  onReady: function(slide) $(slide.doc).click(function() {
    slide({ size: 500, persist: true });

Check out the screencast too:

Jun 04

Pi and Pie: Enjoying some time with trigonometry

Bespin, Tech with tags: , , 7 Comments »

With a wife with a masters in education, you tend to think and chat about education from time to time. I enjoy thinking about how social software could help education in the future. One of the fields that I enjoy thinking about is Maths. It was my favorite subject as a kid and I had the benefit of a great teacher. In general though, it feels like Math is often taught in such a painful way.

I remember being sent an article that discussed how Math is taught by comparing it to music. It postulates a world in which music is taught without instruments. You have to learn music theory for 10 years before you get to pick up that flute. What kid would get excited about a world of that music? That is what we do to them with Math.

Time for Pie

I mentioned the Bespin pie menu before. In our first version you jump to the different pieces with the keyboard. Of course, one of the benefits of a pie menu is that Fittss’s Law kicks in and you can easily jump to an item. (Aside: the dirty secret of our pie menu is that it isn’t really a pie menu, it currently sits at the bottom and not where you right click etc.)

As we start to put in mouse support, the first thing we need to do is be able to determine which slice of the pie is being clicked on so we can activate it. This gave me the opportunity to go down trigonometry lane.


If the pie was sliced with vertical and horizontal lines like the image above, then it would be very simple indeed. To start with, what information do you get when the user clicks? We get an x and y coordinate with an origin starting point in the top left. There are various x and y coordinates that you will get from the MouseEvent (x, y, offsetX/Y, pageX/Y, rangeX/Y, clientX/Y, layerX/Y). We care about the offsetX/Y or layerX/Y value (depending on the browser), which will give us the point with the origin starting at the top left of the canvas itself and not the window.

With the example above you could simply take the computed x and y coordinates and simply check them to determine the segment. For example, if x is less than the radius then it is a segment on the left. If y is less than the radius then it is a segment on top.

But we don’t have that luxury. Instead we have to do some simple Math. I drew out the problem with various bits of information here:


You will see the slices match the pie menu, and I drew in an X and Y axis that cuts through the center of the pie. In the diagram you will see an “x” marking the click in the right slice. How do we work it out? Well, what do we know? First we can calculate the location of the point based on the center of the circle. Remember that the browser will tell you the x and y based on the top left and not the center, so we quickly repurpose the coordinate based on the top left of 390, 180 and convert it to 140, 70. This is simply done via:

function centerPoint(x, y) {
    return { x: x - RADIUS, y: (y - RADIUS) * -1 };

Now we have the relative point from the center of the circle we can calculate the angle that will then in turn let us know which quadrant we are in.

Arctan and atan2()

The easiest way to calculate which quadrant we are in is to simply calculate the angle of the dangle. To do that we calculate the arc tangent between two lines in a triangle, the y=0 line across the x axis, and a line between y=0 and the point in question.

It turns out that we get this calculation for free, via Math.atan2:

function angle(x, y) {
    return Math.atan2(y, x) * 180 / Math.PI;

To explain how this works, see the scribble below that shows 4 different points and the resulting angles based on angle(). One gotcha that you will notice is that atan2() takes the coordinates in the order: y, x but elsewhere you are used to using the opposite x and then y. Be careful!


Once we have the angle based on the x axis, we can have a simple case statement to check the bounds.

If the angle is below -45 and 45 degrees, it is on the right. Greater than 45 and less than 135? Then we have the top. And so on.

With a little fun Math, we get to have Pi solve our Pie problem. To see the code at work, check out the test bed.

Jun 03

When beauty can become a beast; Don’t take away all of my buttons

Apple, Tech with tags: 23 Comments »

Ben and I both got new work 15″ Macbook Pro laptops, and Ben isn’t happy.


I have to admit that it doesn’t feel like the Apple designers may have gone one step too far with the trackpad. In one fell-swoop they took away the button and added new and exciting gestures.


The big problem I have with this is that I have learned to use the trackpad with one thumb at the bottom (where the button was) and I use my index finger to move around. There is a lot of muscle memory there. What this means in practice is that I am in Gmail and the pinch gesture kicks in which results in my font changing size, or the rotate gesture causes an image to skew in Keynote.

In theory, it is beautiful to get rid of the button. In practice I miss the tactile feedback, and the separation of concerns.


Maybe I should quickly grab a Macbook Air which still has the small little trim mouse button?

To those at you who have been using the latest generation of Macbooks, do you get over this? Will this be something that I forget in a month and life will actually be better?