techno.blog("Dion")

As we watch and wait for the update to the Google Mobile iPhone application we are once again aware of the gatekeeper situation on that platform.

Google doesn’t know when it will launch, Apple does. Google had great PR into the launch (which they were told would be Friday… at least at some point). I had the pleasure to see the voice feature and was actually kinda gobsmacked with it when I saw it at work. It isn’t like we haven’t seen voice recognition apps for years. However, this one seemed to actually work, and not just for simple words but for complex queries. Random place names were picked up. Wow. Maybe the work behind GOOG-411 is paying off :)

But, the iPhone app isn’t out there, yet. If this was a Web application, the Google engineers could cut a release and be on their way. But, how would you read in the audio? You could go for Flash or a custom plugin, but it reminded me of the audio API support and Gears. When you think Audio API you think of the HTML 5 audio tag and the API that goes with it… specifically the “play” support. What interested me from the first design doc in Gears was the other side of things and the support for “record”:

// an object of this class can be got from 
// google.gears.factory.create('beta.audiorecorder')
AudioRecorder class
{
  // ---- error state ----
  readonly attribute AudioRecorderError error;
 
  // ---- recording state ----
  // says whether recorder is currently recording or not
  readonly attribute boolean recording;
  // says whether recorder is paused or not
  readonly attribute boolean paused;
  // the amount of sound detected by the microphone
  // 0 - no sound detected to 100 - maximum sound detected
  readonly attribute int activityLevel;
  // specifies the length (in milli seconds) of the audio recorded
  readonly attribute float duration;
 
  // number of channels, currently can be 1 (mono) or 2 (stereo)
           attribute int numberOfChannels;
  // sample rate for the recording
           attribute float sampleRate;
  // sample type for the recording, possible values need to be defined
  // signed 16 bit little endian linear PCM
  const unsigned short S16_LE = 0;
           attribute short sampleFormat;
  // audio file type (container and codec), possible values need to be defined
           attribute string type; 
 
  void record();
  void pause();
  void unpause();
  void stop();
 
  // ---- controls ----
  // 0.0 - silent to 1.0 - loudest
           attribute float volume;
           attribute boolean muted;
  // the amount of sound required to activate the microphone
  // 0 - capture even minutest sound to 100 - capture only loudest sound
           attribute int silenceLevel;
 
  // ---- cue ranges ----
  // provides ability to set callbacks at specific points in playback time.
  // similar to API in Audio class. Look at HTML5 spec for explanation.
  void addCueRange(in DOMString className, in float start, in float end, 
                  in boolean pauseOnExit,
                  in VoidCallback enterCallback, in VoidCallback exitCallback);
  void removeCueRanges(in DOMString className);
 
  // ---- access blob ----
  // returns handle to the blob object containing the audio data
  Blob getBlob();
};

I can’t wait to get full audio support available in the Open Web itself. Yet another barrier knocked down. Of course, the publicity around the Google Mobile app is nothing but good in many ways :)

I talked about having a best viewed in [insert new browser here] day where prominent web sites (and anyone who cares too) display some kind of sign, maybe on the first Monday of the month, that says:

If you like this application now, you will love how it runs if you were using a modern browser

This happens if a non-Grade A browser is detected. We can even go further and push a new version of the browser the person is using. For example, if someone comes in with IE 6, we prominently show IE 7, and then also offer Firefox 3, Opera 9.5, Safari 3.1 for Windows.

As Web developers we have put in so much time just to make sure that IE 6 runs our application well. This is like writing desktop software that runs well on an 8 year old machine!

It hasn’t mattered all that much until the last year or so, as now we are developing richer and richer Web applications.

As I look at apps like Gmail, Mobile Me, 280 Slides, Yahoo! Mail, etc etc…. I say screw it. People upgrade their computers when they want to run things faster, why can’t we ask them to upgrade their browsers.

If you have a simple content site, then it is fine to support everyone, but as you build rich apps, maybe it is time NOT to be a crutch and support these browsers.

It is interesting to note that Mobile Me doesn’t support IE 6. People have shouted at the SproutCore chaps, but SproutCore actually DOES support IE 6. Interesting huh?

If Apple can do it… who next?

I have been hearing the same tune recently, and it escalates: “Man, if we could just not worry about IE 6.” It sounds familar. Every few years, we get another popular but lagging usage that holds us back.

Imagine if the majority of people were still using phones from 2001 too:

Oh wait, sometimes it does feel that way doesn’t it?

We shouldn’t be blaming IE 6 itself of course. When it was released it was a great browser. We can’t forget how Microsoft had a large part to play in pushing the Web back in the day, bringing us DHTML itself, and offering up features that never caught on as people saw them as proprietary (behaviours are awesome for example).

So, we can’t blame the IE 6 engineers. We can get a little frustrated with the fact that new machines still come with IE 6 installed on them, even though IE 7 is out there, and IE 8 is making us all feel like Microsoft could care again.

Frank, Microsoft needs to push the button. The last four times I have had to do tech support for family or friends (I need the “just because I work for Google doesn’t mean that I can fix your computer” t-shirt) they were running IE 6. I couldn’t believe it. Some of these were fairly savvy folks too.

Say we get Microsoft, and IT folk, to push the button and push out IE 7 to the masses, we will still be short on a large part of the users. Namely, the folks across the world that are running Windows illegally. Will this ever end?

Maybe it is time for large sites to start a trend. On the first Monday of the month, when user comes to your site for the first time with a browser that isn’t Grade A you show them some kind of notice that “If you like this site running Browser X, you would like it so much more if you download any of these browsers”.

This isn’t about IE 6. This is about all browsers that are low on the pack. IE 6 happens to be the one that is talked about the most as it has the huge market share as well as being painful. This also isn’t about picking one browser. We aren’t saying “Download Firefox!” We are talking about downloading ANY grade A browser. Firefox, WebKit, or even IE 7!!!!

I never liked seeing the “Better viewed in IE/FF” images, but I think this is different. At some point we need to rev our platform. It is like the cars on the road. It isn’t enough to just improve fuel efficiencies in cars. We then need to get people to get rid of their gas guzzlers and buy the responsible one!

I love how Firefox lets me hit Apple (or Splat as James Strachan calls it!) plus a number to go to the given tab in the browser window.

I often follow the pattern of “open new link in a tab and go to it at the end” and also have windows open with known apps up in the same tab location so I can get to them quickly, without even thinking. Being able to Splat-9 to go to the end is key to this of course.

Although Firefox does this by default, WebKit/Safari does not, so I changed it via a simple hack that others have mentioned. Some people saw me doing this at JavaOne and asked me how I did it, so I decided to quickly show how it is done in the window below.

The steps to follow are:

• Generate AppleScripts to tell Safari and WebKit to go to the right tab. This is talked about here, and I have a slightly modified version that I will paste at the end of this post
• Tie together the AppleScript to the browsers. I tried to use simple keyboard bindings in the keyboard pref panel, but that didn’t work, so I ended up using Quicksilver triggers

See how it all fits together below!

Russ Beattie has closed up shop for Mowser and people are rushing to declare the death of the mobile Web.

I like Russ, and was glad to see him back on the scene and blogging a storm, even if he can be a touch offensive from time to time ;)

But, just because he couldn’t find the right niche for Mowser, doesn’t mean the “mobile Web” is dead before born.

Take a look at what he really said:

In other words, I think anyone currently developing sites using XHTML-MP markup, no Javascript, geared towards cellular connections and two inch screens are simply wasting their time, and I’m tired of wasting my time.

I agree. Where are the mobile apps today? They are the iPhone specific ones, and a few stripped-down versions. The mobile Web is growing strong from where I sit. I just have to look around at how my own wife uses her laptop less and less, and her mobile browser more and more.

I am so bullish about the Web on the phone that I believe it will be THE platform for building mobile applications in the future.

If you are a hardcore mobile app builder you may snortle a little. Really? Cheesy Web technology can compete with rich application frameworks? Never.

They can, and they will. I was listening to someone talking about the battle of IPX versus NetBEUI. The viscous battle between Microsoft and Novell. This person said: “If you had told me that this TCP/IP thing would beat both of us I would have laughed in your face”. Some crappy thing uses in academia that doesn’t have all of the features that we do? In the battle of IPX and NetBEUI, TCP/IP won.

It will keep winning, and it will come to win in the mobile world. This is why I am excited about Gears for Mobile, and any other work that will come through in HTML 5 and browsers such as Mobile Safari.

It may take awhile, but would you really bet against it? The mobile Web will just be the Web. We will have limitations of course. 3G will take awhile, and the size of screens isn’t going anywhere until we have the dream of projection into your eyes and such.

Would you bet against it?

First came IE behaviours (HTC), which allowed you to bind script and binary behaviour to tags via CSS, for IE only of course. Mozilla decided that it needed a language to tie behaviour to XUL widgets and XML elements in general, and they created the first version of XBL.

I met Jonas Sicking of Mozilla for the first time recently, and found out that he is working on XBL 2.0, a spec that was edited by none other than Ian Hickson. That man is a machine!

XBL 2.0 cleans up the original work from Mozilla, and gives you a really nice way to add scripting behaviour to tags.

The example application below gives you a good view of how you could implement ‘an inaccessible implementation of the proposed HTML5 details disclosure element: it opens and closes when clicked, and reflects its current state in the element’s “open” attribute.’

<xbl xmlns="http://www.w3.org/ns/xbl">
 <binding element="details">
  <template>
   <div>
    <div><content includes="legend:first-child">Details...</content></div>
    <div state="hidden" id="container"><content/></div>
   </div>
  </template>
  <handlers>
   <handler event="click" phase="default-action">
    this.open = !this.open;
   </handler>
   <handler event="DOMAttrModified" attr-name="open" attr-change="addition" phase="target">
    this.shadowTree.getElementById('container').setAttribute('state', 'visible');
   </handler>
   <handler event="DOMAttrModified" attr-name="open" attr-change="removal" phase="target">
    this.shadowTree.getElementById('container').setAttribute('state', 'hidden');
   </handler>
  </handlers>
  <implementation>
   ({
     get open() { return this.boundElement.hasAttribute('open'); },
     set open(val) {
       if (val)
         this.boundElement.setAttribute('open', 'open');
       else
         this.boundElement.removeAttribute('open');
       return this.open;
     },
   })
  </implementation>
  <resources>
   <style>
    #container[state=hidden] { display: none; }
   </style>
  </resources>
 </binding>
</xbl>

What I find interesting is how you bind this behaviour via CSS a la IE behaviours:

details { binding: url(details.xml#details); }

You can also do so programatically:

$$('details').each(function(el) {
  el.addBinding("details.xml#details");
});

Once you have this behaviour setup, you are fully embedded in the DOM. You see ‘details’ as a true node, rather than a processing hack that would put in divs and spans in their place. You will notice the shadowTree that let’s you have a full DOM behind the scenes.

Imagine if this spec was implemented by a series of browsers. Is there a way to have a shim that could take XBL 2 and map it to IE behaviours? That is a question for Dean Edwards ;)

In general, this seems like a great extension point to let us take generic behaviour and reuse it. It could be a great way to add functionality to older browsers too, just as we were able to add alpha transparency to IE via binary behaviours.

Do you see value?

Other posts on Web archelogy:

• Java Pluglet API

Even since Ben and I looked at the notes for the first version of Mozilla that supported XMLHttpRequest, which suddenly took the technology from “Some ActiveX for IE” to “Ajax”, I have been interested in hidden technologies that maybe never made it. In those notes for that release we saw no mention of XMLHttpRelease, but technology such as FIXptr was prominently mentioned.

Also, something interesting about Ajax is exactly the fact that the technology was available since 1997, but didn’t make it big until many more Dilbert calendars later.

This points to the fact that there may be some hidden gems in the past that could also be resurrected in the now! As I look back in time, I thought I would talk about any that interest me in some way. Hence, the Web Archeology set of postings. If there is a technology that I am missing, please let me know!

Today I am going to talk about the Java Pluglet API. This technology is part of the Blackwood project at Mozilla, where it was created in 1999 by Igor Kushnirskiy & Akhil Arora.

Let’s walk back to 1999 for a second. Imagine working on Mozilla in a world where you had to futz with a lot of XPCOM and C++ to build things. XUL came about a way to reach out via more Web-y technology to get work done (XML, JavaScript, CSS, etc). In 1999, Java was a sexy language, and everyone was getting ready for fantastic server side Java with great technology like EJB ;)

What if the Java developers could get in on the browser action and develop rich plugins for Mozilla? This is where the Java Pluglet API comes in. It allows you to do just that, mimicking the C++ side of the house:

It was a conscious design decision to have the Pluglet API resemble its C++ counterpart as much as possible, while being able to reflect all of its functionality to Java, so that Plug-in writers will not have to learn yet another API. This concern, in our opinion, outweighed other alternatives which offered a cleaner, more Java-like look. Support for other Plug-in APIs can be easily added by contributing adaptors.

How you register the pluglet is via mime types. You could create application/wicked-cool and when that comes back from the server, Mozilla will say “hmm, I don’t understand this mime type, Pluglet Engine do you?”

At a high level the idea of writing extensions in Java makes total sense to me. It is obviously cross platform, but still low level enough, with a huge library set to get a lot done.

Isn’t this dead?

Probably? Ed Burns did step up to the plate though and reenergized the project recently. As part of the resurrection you can now interface directly between the Java side and JavaScript itself.

If you squint a little, you see an interesting plugin path for the browser. I often say that I would love for Gears to be made using Java instead of C++!

I heard the term “Social Web” too many times today. It is the type of term that can bug you a la “Web 2.0″. What the hell do you actually mean? Just like everything else, the Web itself will consume any of the good thoughts on the meme, and we will make fun of the term more and more.

Although a lot of thought is placed around social “sites” and putting little applications on those sites, we will quickly move away from this model.

We only have to look at the past to see this.

Do you remember life with AOL and Compuserve back in the day? We had the walled gardens where they controlled the content and the entire experience. When you have that kind of control you can do interesting integrating, but we moved away from this as soon as we have the building blocks that were required to move us. First we had Gopher, which failed partly due to greed (charging for server licenses) and partly due to functionality (ridged folder based structure), and then we got the world of the Web including HTTP and HTML.

Tim Berners-Lee managed to unify fresh islands using hypertext (as many other systems had before) in a very simple way. The decoupling of client and servers started the viral growth, starting with academia but quickly getting into the commercial world.

With HTTP and HTML acting as the bridges, what about the cars? We started with very simple browsers, with Mosaic being the first big one that added features to what it would understand in HTML.

When we skip forward in time, what do we get to drive now?

• Internet Explorer: This is the Ford of browsers. Good ‘ole American made, highly marketed.
• Safari: Coming from Apple this is maybe a clean BMW?
• Firefox: a GTI that I can pimp?
• Opera: A euro-SAAB?

We do not have a huge choice on the type of car that we drive, but at least they have opened up on how much we can tinker with them. Firefox lead the charge with its plugins/add-ons and now the others have some form of this.

Being able to tweak my experience via JavaScript technology is empowering. You don’t need to be a hard core C++ hacker to do some tweaking, and again Firefox breaking out to give us the XUL approach was fantastic.

As an extension point, plugins are OK, but are also too much for many people, and can be a pain to manage. What if I want to tweak something of my experience on a Web site? This is of course where Greasemonkey came in and truly empowered people to take over their experience. They could grab a Greasemonkey userscript that would let you use Amazon to find things, but also show you the price from other stores. Amazon.com became the meta search engine.

I find it painful when I jump on Gmail on someone elses computer as it doesn’t have my Greasemonkey extras, let alone if you use something like GTDInbox that drastically changes the experience.

This is about the user taking control of their world. Gears on the other hand is about giving the Web an update mechanism, allowing developers to extend the Web platform to do new tricks.

All great, but we need more. Even though the Web is open through HTML and HTTP, it isn’t reving fast enough, and it doesn’t feel like we are moving forward as fast as we could.

What would it mean if we take some of the characteristics from the social networks and put them into the cars themselves instead of just in some islands?

Me.dium is trying to build a social Web experience. So far we have seen the plugins and widgets that allow you to chat and explore the Web together. That is somewhat interesting, but I would love to take it further and have the service give me a personal TechMeme based on my friends.

For me, it keeps coming back to social extension points. There are hundreds of Greasemonkey user scripts out there, but you never know about it. Only the die hards are going through userscripts.org. Could we do better and have:

• Social notifications that tell you when extension points are available. “There is a script that would allow you to do X on this site”
• Have the scripts weighted by my social graph, or the wisdom of crowds (narrowed by my graph a little), or other algorithms

The end result would be a push of interesting behaviour that is created and added to sites other than those who run the actual site. The comprimise here is that we have to deal with security, and gaming the system.

I would love to see the citizens having the power to change the Web in this way, so maybe it could happen. No more social networks. The Web would be implicitly social. I need to think more about explicit extension points that sites could put out (so it isn’t about monkey patching).

Above is a slide from one of my Gears decks. I get a lot of questions around Gears and HTML 5, and I wanted to be explicit about the role that each play, and how excited we are about HTML 5 (and other standards in fact).

I do not see HTML 5 as competition for Gears at all. I am sitting a few feet away from Hixie’s desk as I write this, and he and the Gears team have a good communication loop.

There is a lot in common between Gears and HTML 5. Both are moving the Web forward, something that we really need to accelerate. Both have APIs to make the Web do new tricks. However HTML 5 is a specification, and Gears is an implementation. Right now you can see differences between the two. I overheard a good analogy for this in a meeting this morning. You can see how similar parts and pieces are, and it will be natural to bring these together just like the image above shows the sides of a zipper.

It is my belief that HTML 5 and Gears will converge in the not too distant path. This has already happened a little, as the WorkerPool API has been proposed to be added as a standard. We are also working on converging the database APIs, with great work from Dimitri Glazkov. To be clear, we have actually been working with the HTML 5 group (starting with WHATWG) from the beginning, but life isn’t perfect and it can take some time to converge.

Standards bodies are not a place to innovate, else you end up with EJB and the like. Instead, they are a place to take work that is already being done and spreading it out. Hixie is doing just that. He studies the Web, how it works, and then works with everyone to come up with nice general standards. HTML 5 is also a large spec that has some additions that do not make sense for a Gears plugin, items that are low level browser features that we couldn’t touch. All implementations of specs are not perfect, and thus we know that Gears will not have exactly the same implementation of HTML 5 as others. On the other hand, Gears is always going to be a superset of HTML 5 as we are pushing the Web hard and innovation is key.

Gears came out of the real-world need within Google to create more and more compelling Web applications. We took Google Reader offline and learnt along the way. We saw the pitfalls, wrestled with synchronization issues, and along with other applications came out with the first release of Gears last May. Now we work with ever expanding groups at Google and learn from their needs, and since we went public in May, we have gotten to learn from you folks.

Gears is a battle hardened Web update mechanism, that is open source and ready for anyone to join and take in interesting directions. Web developers get to go guerilla and fit issues that they see. The good ones? We can bring them back to the HTML 5 standards party.

I guess the zip will always be opening and closing. For some APIs (Database and such) we will join at the hip, and then as we come up with new ones we will break away again and the innovation will start.

My hope is that we will constantly be coming back together.

When you view Gears through the lense of “A bleeding edge implementation of HTML 5″ that is also battle hardened since it has to be deployed for Google (so it isn’t an experimental toy!) you can also see how different it is to AIR and Silverlight. They can play together just fine, as the future of Gears gets into HTML 5. This means that WebKit will support it, and Gears and AIR will be together in that world. Hopefully the IE team will continue on their standards march and you will see the same there. It is nice to see a world where you can work together instead of make stark choices.

The first post I read this morning was Alex Russell giving some more thought to the future of the Open Web where he concludes:

Gears has an open product development process, an auto-upgrade plan, and a plan for N+1.

At this point in the webs evolution, I’m glad to see browser vendors competing and I still feel like that’s our best long-term hope. But we’ve been left at the altar before, and the IE team isn’t giving us lots of reasons to trust them as platform vendors (not as product vendors). For once, we have an open, viable Plan B.

Gears is real, bankable progress.

Oh, and @simonw, I am with you. I want to see that too.

These are my personal opinion and may or may not be that of my employer

We tunnel everything through HTTP these days. Poor old port 80 is the backdoor through the firewall, but the hole is small and only truly suited to certain protocols. One of the crazy wrappers is taking streaming video and putting it in the HTTP envelope. This is nuts as HTTP is of course running on TCP/IP which gives you reliable ordered packets. The overhead of making sure this is the case is large, and is not needed for fire and forget type applications. Video, VoIP, gaming, they all favour UDP as you just want to shove bytes down as fast as you can and if you miss a packet or two it doesn’t matter, you are onto the next.

Wikipedia tells the tale:

Difference between TCP and UDP

TCP (”Transmission Control Protocol”) is a connection-oriented protocol, which means that upon communication it requires handshaking to set up end-to-end connection. A connection can be made from client to server, and from then on any data can be sent along that connection.

• Reliable – TCP manages message acknowledgment, retransmission and timeout. Many attempts to reliably deliver the message are made. If it gets lost along the way, the server will re-request the lost part. In TCP, there’s either no missing data, or, in case of multiple timeouts, the connection is dropped.
• Ordered – if two messages are sent along a connection, one after the other, the first message will reach the receiving application first. When data packets arrive in the wrong order, the TCP layer holds the later data until the earlier data can be rearranged and delivered to the application.
• Heavyweight – TCP requires three packets just to set up a socket, before any actual data can be sent. It handles connections, reliability and congestion control. It is a large transport protocol designed on top of IP.
• Streaming – Data is read as a “stream,” with nothing distinguishing where one packet ends and another begins. Packets may be split or merged into bigger or smaller data streams arbitrarily.

UDP is a simpler message-based connectionless protocol. In connectionless protocols, there is no effort made to setup a dedicated end-to-end connection. Communication is achieved by transmitting information in one direction, from source to destination without checking to see if the destination is still there, or if it is prepared to receive the information. With UDP messages (packets) cross the network in independent units.

• Unreliable – When a message is sent, it cannot be known if it will reach its destination; it could get lost along the way. There is no concept of acknowledgment, retransmission and timeout.
• Not ordered – If two messages are sent to the same recipient, the order in which they arrive cannot be predicted.
• Lightweight – There is no ordering of messages, no tracking connections, etc. It is a small transport layer designed on top of IP.
• Datagrams – Packets are sent individually and are guaranteed to be whole if they arrive. Packets have definite bounds and no split or merge into data streams may exist.

What if we had nice UDP support in the browser itself, something that could fall back to HTTP if necessary. You would need that to make sure that the firewall gods wouldn’t stop things working, but think of the YouTube traffic and how much better streaming video would be! And games! And, Skype!

Can we do it, or is there too much inertia around “HTTP won. It’s good enough”