techno.blog("Dion")

By blurring the boundary between Ajax and RIA, Google has found a way to grow into the Mesh that Microsoft is close to delivering from IT outward. In many ways, this strategy is supportive of the new Microsoft as much as it is disruptive of the old. Just as Microsoft can’t be stopped from executing on its cloud strategy in the enterprise, neither can Google from its base in the user cloud. Where the two platforms meet in the middle looks a lot like a hybrid of iTunes and Office.

That is from Steve Gillmor as he compares video chat with Silverlite :)

You can look at this as some amazing plan, or maybe a Gmail chat team that thought video would be a natural progression?

One key aspect of the new Gmail chat is mentioned as part of the launch blog post:

We designed this feature using Internet standards such as XMPP, RTP, and H.264, which means that third-party applications and networks can choose to interoperate with Gmail voice and video chat.

Once again, standards lead the way for a Google team. This shows how this can be so much more than just an end user feature.

Let’s do a thought experiment: What if?

• This was not yet another plugin (a la Lively 3D), but rather just a Gear? Something that could be reused by developers right away so they could add video and audio in a way that reaches many end users, using standards
• And what if it used the audio and video HTML 5 tags? Chrome could implement them, and Gears could give us a shim to at least give us the APIs, if not more. Of course, other browsers have implementations too!

Google’s “Silverlite” is already here: Gears. If we all kept building on that we could do so much. Add the ability to load and update seperate Gears (modules) so in this case people would have gotten a video/audio module update to their existing plugin.

This is important

Video is huge, and is exploding. It is something that the Open Web doesn’t have a good answer for yet, and we need one. Right now you have to use Flash or Silverlight, and I would prefer more choice ;)

First we need to get players and codecs out there. The video/audio tags are fine, but what can they play? Apps such as Gmail video could deploy that technology. Then the next step is in tooling. How do we plug in to the current video development process? How do we reach the creative types? Without the toolchain, the technology won’t matter.

Can we get from here to there?

Last week I shared the WhereAreYou? application that used the Ajax APIs ClientLocation API to access your location via your IP address.

At the same time, we announced support for the Gears Geolocation API that can calculate your address using a GPS device, WiFi info, cell tower ids, and IP address lookups.

Add to all of that, the W3C Geolocation API that Andrei Popescu of the Gears team is editing. You will notice that it looks similar to the Gears API, with subtle differences. The ClientLocation API is quite different.

To make life easier, I decided to put together a shim called GeoMeta that give you the W3C Geolocation API, and happens to use the other APIs under the hood.

If you have the Geolocation API native in your browser (no one does yet, future proof!) that will be used. If you have Gears, that API will be used, and finally, with nothing the ClientLocation API will be used behind the scenes.

To you the API will look similar:

// navigator.geolocation.getCurrentPosition(successCallback, errorCallback, options)
navigator.geolocation.getCurrentPosition(function(position) {
      var location = [position.address.city, position.address.region, position.address.country].join(', ');
      createMap(position.latitude, position.longitude, location);
}, function() {
      document.getElementById('cantfindyou').innerHTML = "Crap, I don't know. Good hiding!";
});

At least, that is what I would like. Unfortunately, there are a few little differences that leak through.

• The W3C API only seems to give you a lat / long, so you have to do the geocoding to get address info
• The Gears API gives you an additional gearsAddress object attached to the resulting position object. This can contain a lot of information on the resulting area (street address to city to …) however for certain providers the API returns that as null, the same as the W3C standard
• That gearsAddress object has slightly different information from the address data that the ClientLocation API returns.

To give you control when you need it, you can ask the navigator.geolocation object what type it is. navigator.geolocation.type will be null if it is native, but ‘Gears’ or ‘ClientLocation’ if a shim kicks in. You can also check navigator.geolocation.shim to see if it is augmented code.

Implementation

There is some fun implementation code in there if you poke around. For example, for the ClientLocation API, when you make a call, it will be added to a queue if the Google Loader hasn’t fully loaded yet, and it will kick off that call when finished. Dealing with dynamically creating <script src> as a loading mechanism sure is fun!

I like the idea of jumping straight to the W3C standard and updating the shim as the APIs change. That way, when browsers catch up, the code will still work using the native APIs and you don’t have to change a thing.

We just announced two new ways to get location info from a browser client.

The Gears GeoLocation API is very detailed. It is able to use GPS, cell towers, WiFi, and ip addresses to work out the location, and you get an “accuracy” parameter to see what was available. As well as getting a position, you can watch a position so you are updated when a change happens. This is perfect for mobile devices that have Gears installed, and since the community is working on the W3C Geolocation spec it should be in many more places soon.

To go with the Gears API, we also have an API that goes along with the AJAX APIs, called ClientLocation.

This is an ip based geocoder that we have made available, and is very simple.

I put together a trivial example called Where Are You? that ties together this API with the Maps API:

You get access to the data from google.loader.ClientLocation, which is null if it can’t be calculated.

Here is a bit of JavaScript that ties it together:

google.load("maps", "2.x");
 
google.setOnLoadCallback(function() {
    if (google.loader.ClientLocation) {
        var cl = google.loader.ClientLocation;
        var location = [cl.address.city, cl.address.region, cl.address.country].join(', ');
 
        createMap(cl.latitude, cl.longitude, location);
    } else {
        document.getElementById('cantfindyou').innerHTML = "Crap, I don't know. Good hiding!";
    }
});
 
function createMap(lat, lng, location) {
    var mapElement = document.getElementById("map");
    mapElement.style.display = 'block';
    var map = new google.maps.Map2(mapElement);
    map.addControl(new GLargeMapControl());
    map.addControl(new GMapTypeControl());
    map.setCenter(new google.maps.LatLng(lat, lng), 13);
    map.openInfoWindow(map.getCenter(), document.createTextNode(location));
}

Passpack is an online password organizer that provided offline access early in its life via Gears.

Last week they announced an Adobe AIR version, and Ryan quickly commented saying:

For a while they were using Gears as a way to store those passwords on the client but they’ve just recently switched over to using Adobe AIR.

One of the reasons given was Safari support for Gears. You have to do a build for Safari which is not what we want, and we are working with the Apple team to clean things up and have a really nice Safari plugin. Unfortunately, Safari doesn’t have a plugin model, so you always feel like you are doing some hack using unsupported mechanisms like Input Managers. Grr, come on Apple, play nice with others! :)

At the same time, Air support for the Encrypted File store (a very nice feature!) doesn’t work on Linux yet (I am sure that will be fixed soon too).

Of course, the solution isn’t Gears vs. AIR. They aren’t really competing here! The Gears users of the product started to get vocal and we quickly saw Gears support added again.

There are two very different use cases:

• Users who want to be using Passpack in the browser, who happen to want to use functionality such as offline, or any other Gears component
• Users who want a desktop-like application, outside of the browser. This is where the AIR application fits in.

So, both can make sense for Passpack. In other cases, one of the choices could make sense.

I also expect to see more joint applications. Gears functionality is working into HTML5 the standard, which will end up in WebKit (as Apple is great in that regard), and AIR uses…. WebKit as its renderer!

I really hope that AIR will be able to bridge to those APIs, and you get the best of all worlds. I would love to use the Workerpool API from within an AIR application that is doing a lot of JavaScript work for example.

A core vision of Gears is that the same APIs are available on the desktop, and mobile versions.

That being said, there are APIs that will potentially make more sense on one platform or another. One of those that has garnered excitement on the mobile platform is the Location API.

Another is the newly proposed Camera API that “enables a script to trigger the camera viewfinder and allow a picture or video to be captured using the device’s onboard camera.”

That being said, this could still tie into a computers webcam.

Summary

The goal of this API is to allow a Web application to launch the native camera application and obtain the resulting media file.

On devices with built-in camera, all of the UI interaction is left to the corresponding native camera application, not Gears.

On devices that do not have a built-in camera (e.g. laptop or desktop computers), Gears will scan for attached Web cameras and will provide a basic viewport that allows the users to grab a picture or record a movie. If several cameras are attached, Gears will provide a way for the users to select the device they want to use.

This API is based on the “action-driven permission model” - the user is involved in the action and hence a separate permission prompt is not required.

You would use the API like this:

// Create a camera object
var camera = google.gears.factory.create("beta.camera");
 
// Capture an image.
var image = camera.captureImage(function(media) {
    uploadImage(media.content);
});
 
// Capture of a video, specifying a desired resolution and duration.
var video = camera.captureVideo(function(media) {
    uploadVideo(media.content);
}, 5000);

The full API proposal is:

interface Camera {
  // Acquisition methods
 
  // Triggers the camera viewfinder and invokes the callback after the user
  // has either captured the image or canceled the operation.
  void captureImage(CaptureCallback callback, optional CaptureOptions options);
  // Triggers the camera viewfinder and invokes the callback after the user
  // has either captured the video, canceled the operation or maxDuration milliseconds
  // have elapsed.
  void captureVideo(CaptureCallback callback, double maxDuration, optional CaptureOptions options);
 
  // Query methods
 
  Array availableImageFormats();   
  Array availableVideoFormats();   
};
 
void CaptureCallback(Media media);
 
interface CaptureOptions {
  string format;  // MIME type
  double width;   // desired image or video frame width
  double height;  // desired image or video frame height
};
 
interface Media {
  readonly string format;  // MIME type
  readonly double width;   
  readonly double height;
  readonly Blob content;   // reference to captured media
};

What do you think? What haven’t we thought about yet?

Other Future APIs

• Resumable Uploads via PUT/POST
• Open ID and OAuth
• Services / Daemon API
• Notification API
• Crypto API
• Developer Tools
• Factory API updates
• Blob API
• Logging API
• Messaging API
• Location API
• Desktop Shortcut API
• Image Manipulation API
• Introduction to the series

Disclaimer: These APIs are being discussed by the Gears community and do not represent live products. Do you have ideas for cool Gears that make the Web better? Let us know!.

I was sitting on the tube a few months ago in London when I looked up to see Matt Mullenweg, Om Malik, and another nice chap whose name escapes me. A little random to bump into them in the middle of London, but we were all in town for the “Future of Web Apps” conference. I had the fortune to chat with Matt a little about Gears and Wordpress. The marriage of which would make me a happy man as I use both technologies on a daily basis (this blog and Ajaxian both run Wordpress).

Brad Neuberg got to working with the Wordpress team, and after a short IRC session they had great progress. At first it can seem daunting “Oh man, won’t it be a ton of work to rewrite Wordpress to work offline?”

Speed Up!

This leads us to this post by James who did an “svn up” recently, and saw new support for Gears which lead to some pleasant surprises:

As a side note and introduction to what has been sped up, here’s a little rant.

I personally LOVE the changes that were implemented with WordPress 2.5.

But, some of the new features (and features I’ve just started using now that I use the Visual Editor) just aren’t as cool thanks to the not-so-great internet speeds in South Africa.

For example, if you want to create a link. Every time you click the link icon in the editor’s toolbar, it has to download the same stuff over and over…

Well, it looks to me like the WordPress Google Gears implementation has solved that. The link and the “insert embedded media” popups are now instantaneous!

Thank you to whoever decided to do this.

It also seems that switching between each “pane” in the admin section is a LOT faster… Believe me, working on the South African tubes (via iBurst), this makes a HUGE difference!

I am really proud of the Gears team whenever I open up Google Docs, Reader, Zoho, or any other application that uses Gears to let me access the application while offline. There have been situations where it really saved me, and offline is an important boundary.

However, Gears is so much more than offline, and it is really exciting to see “Speed Up!” as a link instead of “Go Offline?”

This is just the beginning. As the Gears community fills in the gaps in the Web development model and begins to bring you HTML5 functionality I expect to see less “Go Offline” and more “Speed Up!” and other such phrases. In fact, I will be most excited when I don’t see any such linkage, and the applications are just better.

With an embedded database, local server storage, worker pool execution, desktop APIs, and other exciting modules such as notifications, resumable HTTP being talked about in the community…. I think we can all get excited.

Kudos to the Wordpress team for finding a great way to increase the performance of their great application, and I can’t wait to see how you use Gears in the future.

Aaron Boodman wrote a fantastic post on Gears and Standards which I am very passionate about myself.

In it he talks about the HTML5 Storage API and how we are all working together to unify the database access semantics.

You can see the Database 2 API which aims too:

• Enable Javascript developers to easily write code that works with both Gears and browser database APIs
• Reduce developer “mind-print” by implementing the same API that is available in browsers
• Support the proposed HTML5 database standard with an implementation available for all browsers that Gears supports
• Implement an asynchronous API that can be called from the UI thread without freezing the UI
• Implement a synchronous API to simplify usage inside workers
• Implement a thread pool abstraction that can be used in other modules for asynchronous operations (bonus)
• build a new module from scratch using the new Dispatcher model (bonus).

It would allow you to write code such as:

var dbman = goolge.gears.factory.create('beta.databasemanager');
var db = dbman.open('pages', '0.0.1.0',
  'Collection of crawled pages', 3000000);
 
function renderPageRow(row) {
  // insert page row into a table
}
 
function reportError(source, message) {
  // report an error
}
 
function renderPages() {
  db.transaction(function(tx) {
    tx.executeSql('CREATE TABLE IF NOT EXISTS Pages(title TEXT, lastUpdated INTEGER)', 
      []);
    tx.executeSql('SELECT * FROM Pages', [], function(rs) {
      for(var i = 0; i < rs.rows.length; i++) {
        renderPageRow(rs.rows.item(i));
      }
    })
  })  
}
 
function insertPage(text, lastUpdated) {
  db.transaction(function(tx) {
    tx.executeSql('INSERT INTO Pages VALUES(?, ?)', [ text, lastUpdated ],
      function() {
        // no result returned, stub success callback
      },
      function(tx, error) {
        reportError('sql', error.message);
      });
  });
}

There is a full technical design, so you can get involved and take part in the open source / open community process that we have going on in Gears land.

I will again end with my visualization of the zipper :)

I have sat watching an upload happening, with a spinning “I am doing something” indicator, wondering how long to go, and if anything is actually really happening. Some sites do a better job and give you some feedback, but most still don’t.

If the Internet connection goes down and you are at 90% you cry out as you know that you will have to tell the browser to send it all up again.

This is why we want to implement Content-Range for HTTP POST and PUT:

The primary primitive necessary to enable large file uploads is the ability to specify byte ranges in POST and PUT requests. Byte ranges are already standardized for GET requests, and there are implementations of byte-range PUT used by WebDAV, but to our knowledge there has been little effort to use them for POST. Byte-range POST/PUT could be used to resume incomplete transfers, or to explicitly break transfers into smaller sized chunks. We propose standardizing byte range POST/PUT in a manner analogous to byte range GET.

The use of Content-Range headers in POST/PUT is allowed by the standard. Section 9.6 explicitly suggests the possibility of using Content-Range headers in PUTs: “The recipient of the entity MUST NOT ignore any Content-* (e.g. Content-Range) headers that it does not understand or implement and MUST return a 501 (Not Implemented) response in such cases.” However, such functionality has not been widely deployed, and therefore there exists no reference implementation or standardized semantics for how it should be used.

Use Cases

The use of Content-Range POST/PUT could be used to compose a chunked transfer protocol where a large POST/PUT was broken up into many small POST/PUTs. This would be similar to the protocols used by several current resumable uploaders. The basic algorithm for this model is to continue sending each chunk (probably in serial order) until it is successfully acknowledged. Note that if used for POSTs, some server-specific mechanism would be required to uniquely identify the the set of chunks constituting a single logical transfer. If it is necessary to accommodate server-side chunk loss (e.g. due to failures partially masked by replication) some additional custom protocol components would be necessary for the server to indicate to the client which chunks it had successfully received.

For some web applications, it may be desirable to upload a subset of a file to the server to reduce the latency between when a file is selected, and when the user can manipulate it in the application. A concrete example of this might be an image hosting website, which would like to upload the 64KB EXIF data segment from a JPEG file, so that the user can quickly view a thumbnail version of the file in the application while the rest of the image is transferred in the background.

It makes sense for this to work with the Blob API, adding:

[java]
interface Blob {
…
Blob slice(int64 offset);
Blob slice(int64 offset, int64 length);
…
}
[/java]

Other Future APIs

• Open ID and OAuth
• Services / Daemon API
• Notification API
• Crypto API
• Developer Tools
• Factory API updates
• Blob API
• Logging API
• Messaging API
• Location API
• Desktop Shortcut API
• Image Manipulation API
• Introduction to the series

Disclaimer: This is me rambling about APIs and tools that I would love to see in Gears, or the Open Web as a whole. Do you have ideas for cool Gears that make the Web better? Let us know!.

Above is a slide from one of my Gears decks. I get a lot of questions around Gears and HTML 5, and I wanted to be explicit about the role that each play, and how excited we are about HTML 5 (and other standards in fact).

I do not see HTML 5 as competition for Gears at all. I am sitting a few feet away from Hixie’s desk as I write this, and he and the Gears team have a good communication loop.

There is a lot in common between Gears and HTML 5. Both are moving the Web forward, something that we really need to accelerate. Both have APIs to make the Web do new tricks. However HTML 5 is a specification, and Gears is an implementation. Right now you can see differences between the two. I overheard a good analogy for this in a meeting this morning. You can see how similar parts and pieces are, and it will be natural to bring these together just like the image above shows the sides of a zipper.

It is my belief that HTML 5 and Gears will converge in the not too distant path. This has already happened a little, as the WorkerPool API has been proposed to be added as a standard. We are also working on converging the database APIs, with great work from Dimitri Glazkov. To be clear, we have actually been working with the HTML 5 group (starting with WHATWG) from the beginning, but life isn’t perfect and it can take some time to converge.

Standards bodies are not a place to innovate, else you end up with EJB and the like. Instead, they are a place to take work that is already being done and spreading it out. Hixie is doing just that. He studies the Web, how it works, and then works with everyone to come up with nice general standards. HTML 5 is also a large spec that has some additions that do not make sense for a Gears plugin, items that are low level browser features that we couldn’t touch. All implementations of specs are not perfect, and thus we know that Gears will not have exactly the same implementation of HTML 5 as others. On the other hand, Gears is always going to be a superset of HTML 5 as we are pushing the Web hard and innovation is key.

Gears came out of the real-world need within Google to create more and more compelling Web applications. We took Google Reader offline and learnt along the way. We saw the pitfalls, wrestled with synchronization issues, and along with other applications came out with the first release of Gears last May. Now we work with ever expanding groups at Google and learn from their needs, and since we went public in May, we have gotten to learn from you folks.

Gears is a battle hardened Web update mechanism, that is open source and ready for anyone to join and take in interesting directions. Web developers get to go guerilla and fit issues that they see. The good ones? We can bring them back to the HTML 5 standards party.

I guess the zip will always be opening and closing. For some APIs (Database and such) we will join at the hip, and then as we come up with new ones we will break away again and the innovation will start.

My hope is that we will constantly be coming back together.

When you view Gears through the lense of “A bleeding edge implementation of HTML 5″ that is also battle hardened since it has to be deployed for Google (so it isn’t an experimental toy!) you can also see how different it is to AIR and Silverlight. They can play together just fine, as the future of Gears gets into HTML 5. This means that WebKit will support it, and Gears and AIR will be together in that world. Hopefully the IE team will continue on their standards march and you will see the same there. It is nice to see a world where you can work together instead of make stark choices.

The first post I read this morning was Alex Russell giving some more thought to the future of the Open Web where he concludes:

Gears has an open product development process, an auto-upgrade plan, and a plan for N+1.

At this point in the webs evolution, I’m glad to see browser vendors competing and I still feel like that’s our best long-term hope. But we’ve been left at the altar before, and the IE team isn’t giving us lots of reasons to trust them as platform vendors (not as product vendors). For once, we have an open, viable Plan B.

Gears is real, bankable progress.

Oh, and @simonw, I am with you. I want to see that too.

These are my personal opinion and may or may not be that of my employer

When I was looking over Brad Neuberg’s Paper Airplane thought experiment I noticed the single sign on feature, where you login to the browser, and then you are done.

I realized that this is what I actually want. Having one signon via OpenID is really nice. It allows me to plug in “http://almaer.com” as my identifier. However, I always have to go around finding the OpenID option (if it exists) and put that in.

What I really want is for the browser to do that work for me. If a site groks OpenID the browser should be able to pass that over without having me intervene at all. It could hide the entire login process if we came up with a microformat to let all sides know what is going on.

It would be a breath of fresh air to be able to jump through sites leaving comments on blogs, and checking out my todo list, all without me once having to actually login.

I wonder if a Gear could be made with a complementary microformat / server side handshake that could then give us single sign-on in all of the browsers.

As Brian McCallister suggests:

<link rel="openid-auth" href="..." />

Other Future APIs

• Services / Daemon API
• Notification API
• Crypto API
• Developer Tools
• Factory API updates
• Blob API
• Logging API
• Messaging API
• Location API
• Desktop Shortcut API
• Image Manipulation API
• Introduction to the series

Disclaimer: This is me rambling about APIs and tools that I would love to see in Gears, or the Open Web as a whole. Do you have ideas for cool Gears that make the Web better? Let us know!.