Jan 11

The table stakes of 2010 and beyond

Comic, Mobile, Tech 1 Comment »

From websites to mobile apps

Remember the first dot-com boom when people started to think that creating a company was just about creating a website? They would register a domain before they had a business plan? Crazy days of sock puppets and and people renaming themselves as dot com versions. Behind the craziness was the reality that to a large extent you did have to have a website. A website was table stakes, and this continues to be the case.

With the iPhone another gold rush occurred. The initial rush is over and you can’t just throw an application into the appstore and hope to make a fortune. You actually have to create something compelling and differentiated to make hour mark. Imagine that!

The “quick buck” allure isn’t there and we are onto the next level of maturity. Having an experience for iPhone users has become table stakes for some too.

So, now you have created your website, and iPhone app, do you go to an Android app? Facebook? One for a new 3D TV? :) It is starting to get daunting.

devices

Walking around at CES the consumer in me gets very excited to see the proliferation of devices. Everything is connected, everything has an API.

Then the developer in me wonders how this is going to scale and how we are going to deliver great experiences to all.

My hope is that the Web can become a great unifying platform. It isn’t easy to build Web apps that will just work anywhere. I just talked about the issues and fun of developing for devices and we are going to be getting a lot more form factors out there.

Jan 04

Gearing up your applications to be touched and the horizontal scroll

Mobile, Tech, webOS 2 Comments »

I have been having a great time hacking on code that let’s me do interesting things on my device. With webOS you get to use your old favorites (HTML/JS/CSS and Web APIs that you know) and now you are ready to take Web applications that you may have even already written, and make them work for the mobile form factor.

The two biggest challenges I have found so far on the design side have been dealing with the real estate available (screen size) and the touchy feely-ness of the device.

We have been trained on the desktop, thanks to the mouse as our pointer interface, to click lots of buttons. The tactile feedback that we get is the button looking depressed, but that is about it. The mouse has a lot going for it. The fact that I have a set of states due to the fact that there is a difference between having the cursor located somewhere, and having a click on that location is useful. It can be especially useful for discoverability. As a user mouses around you can unveil information “hey, if you click this button X will happen mate!” It also has the nice side effect that your hand can rest on it.

The infamous Minority Report interface looked great, and spawned many “look how we can do this now” prototypes, but try holding your hands in front of you for 10 minutes, let alone 8 hours :)

The mouse has a lot of drawbacks though. It is a level of abstraction. You aren’t touching the objects, you are touching a device that takes your input and does the real touching. This of course leads to touch screens and bypassing that abstraction. Watching my son tap and drag and manipulate a touch screen has been great to see. It matches the physical world a lot more. He tried to touch my laptop screen at first too.

Since you are mimicking the real world more though, you get into the uncanny valley problem. If you drag something…. it SHOULD drag! One of the interesting design choices that iPhone made was to allow you to drag things even though some folks would say “don’t let them do that, there is nothing back there”. You then saw studies where people were taking their browser screens and dragging them around just for fun. Crazy :)

Applications started to take advantage of this. iPhone started with many buttons. The back button is the classic one that is all over the shop. On webOS there is a gesture area that trains you quickly to swipe back whenever you want, like going back a page. It is fun to watch a webOS user try to swipe when using an iPhone. You get used to it, you like it, it feels more natural.

tweetie_refresh

Applications like Tweetie 2 groked this too. They took away the refresh button and added a draggable notion. You drag the list down and let go to spin it into action. In many ways this is more work than a simple tap and is a touch less discoverable, but it has the benefit of “feeling nice” and once you know about it, there isn’t a darn button in front of you all the time. The UI can get out of the way.

newsroom-drag

The Newsroom feedreader app on webOS does a really nice job here too. You flip and drag and move throughout most of the interface. After awhile it feels like you have a large virtual space and you are just moving the view port around to see what you want to see. It feels nice. I strangely find that it makes me want to use the application much more than an app with buttons and simple taps. Maybe I am just strange.

I find that I really enjoy the horizontal scroll. I wanted to build a simple HorizontalScroller component for webOS, but first I will detail the low level side of using a Scroller webOS component to get this effect.

The code for all of this is in my HorizontalScroller GitHub repo.

A simple way to build a horizontal scroller is to have a div that contains all of the pieces of content (pre-loaded, or you can lazily load by adding nodes to this div). It extends wider than the 320px viewport size of the Pre/Pixi and contains other elements that are each 320px. We have a view that contains this, and a controller that sets it all up:

The View

Here is how we setup the component with 3 items:

<div x-mojo-element="Scroller" id="scrollerId">
    <div class="scrollerContainer">
        <div class="scrollerItem" id="scrollerItem:1">first</div>
        <div class="scrollerItem" id="scrollerItem:2">second</div>
        <div class="scrollerItem" id="scrollerItem:3">third</div>
    </div>
</div>
</div>

and we style them:

/* A scroller needs a container element */
.scrollerContainer {
    width:  960px;
    height: 400px;
    border: yellow solid 1px;
}
 
/* An item that is viewable in the scroll pane window */
.scrollerItem {
    float:     left;
    width:     320px;
    min-width: 320px;
}

The Controller

The code that turns the divs into life lives in the controller.

this.controller.setupWidget("scrollerId", {
                   mode: 'horizontal-snap'
               }, this.model = {
                   snapElements: { x: $$('.scrollerItem') },
                   snapIndex: 0
               }
        );

The key is the “horizontal-snap” mode that make the drag physics feel right and ties to the snapElements which are the DOM elements for the inner divs. The widget takes care of snapping to those elements for you!

Of course, you can use this base to do a lot. The afore mentioned Newsroom styles the divs so it looks like switching cards.

Next up: I will get the HorizontalScroller component done and dusted to make life even easier.

Conclusion

It is enjoyable to use the constraints of screen size and the natural feeling of touch and put them together to make great apps that live with you in your pocket. I thought screen size would be a huge issue, but in some ways it has been nice to have that constraint. It means you focus on the core functionality. Dealing with the touch vs. mouse issue is an interesting one too and will have you ending up with a very different interface if you embrace the feel.

What have you found enjoyable about using mobile apps vs. desktop ones?

Nov 16

iPhone Web applications and the Record API

Mobile, Tech, Web Browsing, iPhone with tags: 3 Comments »

As we watch and wait for the update to the Google Mobile iPhone application we are once again aware of the gatekeeper situation on that platform.

Google doesn’t know when it will launch, Apple does. Google had great PR into the launch (which they were told would be Friday… at least at some point). I had the pleasure to see the voice feature and was actually kinda gobsmacked with it when I saw it at work. It isn’t like we haven’t seen voice recognition apps for years. However, this one seemed to actually work, and not just for simple words but for complex queries. Random place names were picked up. Wow. Maybe the work behind GOOG-411 is paying off :)

But, the iPhone app isn’t out there, yet. If this was a Web application, the Google engineers could cut a release and be on their way. But, how would you read in the audio? You could go for Flash or a custom plugin, but it reminded me of the audio API support and Gears. When you think Audio API you think of the HTML 5 audio tag and the API that goes with it… specifically the “play” support. What interested me from the first design doc in Gears was the other side of things and the support for “record”:

// an object of this class can be got from 
// google.gears.factory.create('beta.audiorecorder')
AudioRecorder class
{
  // ---- error state ----
  readonly attribute AudioRecorderError error;
 
  // ---- recording state ----
  // says whether recorder is currently recording or not
  readonly attribute boolean recording;
  // says whether recorder is paused or not
  readonly attribute boolean paused;
  // the amount of sound detected by the microphone
  // 0 - no sound detected to 100 - maximum sound detected
  readonly attribute int activityLevel;
  // specifies the length (in milli seconds) of the audio recorded
  readonly attribute float duration;
 
  // number of channels, currently can be 1 (mono) or 2 (stereo)
           attribute int numberOfChannels;
  // sample rate for the recording
           attribute float sampleRate;
  // sample type for the recording, possible values need to be defined
  // signed 16 bit little endian linear PCM
  const unsigned short S16_LE = 0;
           attribute short sampleFormat;
  // audio file type (container and codec), possible values need to be defined
           attribute string type; 
 
  void record();
  void pause();
  void unpause();
  void stop();
 
  // ---- controls ----
  // 0.0 - silent to 1.0 - loudest
           attribute float volume;
           attribute boolean muted;
  // the amount of sound required to activate the microphone
  // 0 - capture even minutest sound to 100 - capture only loudest sound
           attribute int silenceLevel;
 
  // ---- cue ranges ----
  // provides ability to set callbacks at specific points in playback time.
  // similar to API in Audio class. Look at HTML5 spec for explanation.
  void addCueRange(in DOMString className, in float start, in float end, 
                  in boolean pauseOnExit,
                  in VoidCallback enterCallback, in VoidCallback exitCallback);
  void removeCueRanges(in DOMString className);
 
  // ---- access blob ----
  // returns handle to the blob object containing the audio data
  Blob getBlob();
};

I can’t wait to get full audio support available in the Open Web itself. Yet another barrier knocked down. Of course, the publicity around the Google Mobile app is nothing but good in many ways :)

Aug 22

Where are you? Using the new Ajax ClientLocation API

Ajax, Gears, Google, Mobile, Tech with tags: , 22 Comments »

We just announced two new ways to get location info from a browser client.

The Gears GeoLocation API is very detailed. It is able to use GPS, cell towers, WiFi, and ip addresses to work out the location, and you get an “accuracy” parameter to see what was available. As well as getting a position, you can watch a position so you are updated when a change happens. This is perfect for mobile devices that have Gears installed, and since the community is working on the W3C Geolocation spec it should be in many more places soon.

To go with the Gears API, we also have an API that goes along with the AJAX APIs, called ClientLocation.

This is an ip based geocoder that we have made available, and is very simple.

I put together a trivial example called Where Are You? that ties together this API with the Maps API:

You get access to the data from google.loader.ClientLocation, which is null if it can’t be calculated.

Here is a bit of JavaScript that ties it together:

google.load("maps", "2.x");
 
google.setOnLoadCallback(function() {
    if (google.loader.ClientLocation) {
        var cl = google.loader.ClientLocation;
        var location = [cl.address.city, cl.address.region, cl.address.country].join(', ');
 
        createMap(cl.latitude, cl.longitude, location);
    } else {
        document.getElementById('cantfindyou').innerHTML = "Crap, I don't know. Good hiding!";
    }
});
 
function createMap(lat, lng, location) {
    var mapElement = document.getElementById("map");
    mapElement.style.display = 'block';
    var map = new google.maps.Map2(mapElement);
    map.addControl(new GLargeMapControl());
    map.addControl(new GMapTypeControl());
    map.setCenter(new google.maps.LatLng(lat, lng), 13);
    map.openInfoWindow(map.getCenter(), document.createTextNode(location));
}
Apr 14

The future of the Mobile Web is strong

Mobile, Tech, Web Browsing with tags: 5 Comments »

Mobile Web

Russ Beattie has closed up shop for Mowser and people are rushing to declare the death of the mobile Web.

I like Russ, and was glad to see him back on the scene and blogging a storm, even if he can be a touch offensive from time to time ;)

But, just because he couldn’t find the right niche for Mowser, doesn’t mean the “mobile Web” is dead before born.

Take a look at what he really said:

In other words, I think anyone currently developing sites using XHTML-MP markup, no Javascript, geared towards cellular connections and two inch screens are simply wasting their time, and I’m tired of wasting my time.

I agree. Where are the mobile apps today? They are the iPhone specific ones, and a few stripped-down versions. The mobile Web is growing strong from where I sit. I just have to look around at how my own wife uses her laptop less and less, and her mobile browser more and more.

I am so bullish about the Web on the phone that I believe it will be THE platform for building mobile applications in the future.

If you are a hardcore mobile app builder you may snortle a little. Really? Cheesy Web technology can compete with rich application frameworks? Never.

They can, and they will. I was listening to someone talking about the battle of IPX versus NetBEUI. The viscous battle between Microsoft and Novell. This person said: “If you had told me that this TCP/IP thing would beat both of us I would have laughed in your face”. Some crappy thing uses in academia that doesn’t have all of the features that we do? In the battle of IPX and NetBEUI, TCP/IP won.

It will keep winning, and it will come to win in the mobile world. This is why I am excited about Gears for Mobile, and any other work that will come through in HTML 5 and browsers such as Mobile Safari.

It may take awhile, but would you really bet against it? The mobile Web will just be the Web. We will have limitations of course. 3G will take awhile, and the size of screens isn’t going anywhere until we have the dream of projection into your eyes and such.

Would you bet against it?

Mar 08

jPhone?

Comic, Java, Mobile, Tech, iPhone 6 Comments »

jPhone

Paul Krill reported that Sun has looked at the iPhone SDK and thinks that it can port Java to the iPhone. It will be placed up on AppStore as an application, so I wonder what the user experience will be for apps that actually run Java, especially for the first time on a phone that doesn’t have it installed.

Don’t get me wrong, I want Java on the phone, just like I want full RubyCocoa, PyCocoa, CocoaJS, and any other language that you fancy.

Background Processes

So, the iPhone doesn’t allow background processes:

Only one iPhone application can run at a time, and third-party applications never run in the background. This means that when users switch to another application, answer the phone, or check their email, the application they were using quits.

This makes me worry about Java too. Startup has never been a good point of the JVM. If I flip to a Java app am I going to have to wait for the bugger to startup? Is there going to be a way to load one VM and keep it loaded (doesn’t seem like it).

It seems like this is just a guideline and not a firm issue:

I’m a programmer and I just tried it [using the iPhone SDK] and you can keep your app running in the background in the normal way ApolloIM and iFob do it. I.e. overriding applicationSuspend.

Even more worrying though is this part of the developer agreement:

3.3.2 An Application may not itself install or launch other executable code by any means, including without limitation through the use of a plug-in architecture, calling other frameworks, other APIs or otherwise. No interpreted code may be downloaded and used in an Application except for code that is interpreted and run by Apple’s Published APIs and built-in interpreter(s).

Although it has been suggested that this is to stop non AppStore code, it seems to go a lot further than that.

Open source and the iPhone

Also, Mark Pilgrim has written up thoughts on whether iPhone apps can be GPL:

Since all iPhone apps must be distributed through a third-party (Apple’s “App Store”), that would make Apple the “distributor.” Which would mean that Apple — acting as the distributor of GPL-licensed object code — must provide source code or a written offer to provide source code. It’s analogous to a Linux distribution — they distribute binaries of upstream GPL programs, so they need to host the source code as well.

Feb 29

BBC takes a look at Android

Android, Mobile, Tech, iPhone 8 Comments »

Darren Waters sat down with Andy Rubin to take a look at an early version of Android:

The software stack, I was told, was Alpha, so not even Beta; but what I was shown gave a good indication that Android should be taken seriously by competitors like Windows Mobile and Symbian.

Google says they are driving the Android initiative because they want to see internet-style development on mobile platforms in the way that the openness of the web has given rise to Facebook and the Web 2.0 movement which should be able to migrate to the mobile phone.

Of course, coming in at the ground level of Android will give Google plenty of opportunity to tailor its own applications.

I got to spend some time with a few Windows Mobile devices this week. I found them incredibly hard to use. I felt like an old person using a device. I think that we forget what the iPhone has done for the mobile industry. it is just so easy to use. Going back to all of these magical keys and no touch screen is soooo painful and backwards. I can’t wait for my touchscreen-screen.

Feb 25

The future is mobile…. soon.

Adobe, Comic, Mobile, Tech with tags: , No Comments »

The future is mobile…

I wonder if 2008 will be the tipping point over in the US where we see more developers targeting mobile versus desktop. I am sure it is going to happen some day…. but when is that?

Kevin Lynch at Engage told us that he believes we will be developing for the mobile form factor and extracting desktop interfaces from there, rather than the other way around.

I am twittering the Engage event using hashtags.

Dec 21

Gears Future APIs: Location API

Gears, Mobile with tags: , 7 Comments »

I have spoken at a bunch of conferences in Europe this quarter. From the Future of Web Apps, and @mediaAjax in London, to JavaZone and JavaPolis in Oslo and Belgium. When I speak about Gears there, I get a lot of questions about Mobile Gears.

A lot of the features of Gears arguably make even MORE sense on a mobile device. Allowing Web developers to build applications for phones has taken off well thanks to the iPhone. Gears can help out in these high latency devices.

One very handy API to have would be a Location API (although it would be useful in other contexts too):

The purpose of this API is to provide means to fetch the location of a device running a Web browser with Gears.

The Location API is an abstraction for the various LBS APIs that currently exist on mobile platforms (GPS-based, network/cellid-based). The API consists of the Location class, which encapsulates various location attributes (latitude, longitude, etc), and also provides the means to query the platform for a location fix. This API also adds a new event type that is fired every time the location changes. Location implementations can be straightforward mappings to native LBS APIs (e.g the S60 Location Acquisition API) or have a more complex design that combines several location providers (e.g a GPS-based provider and a cell id-based provider) and returns the location from the most accurate provider at any given time.

Here is the API as a code example using it:

// Getting the object
var location = google.gears.factory.create( "beta.location", "1.0" );
 
// Setting up a callback to handle "location changed" events
location.onlocationstatechanged = function() {
   switch (this.state) {
     case 1:
         SetStatusText("Connecting");
         break;
     case 2:
         SetStatusText("Acquiring");
         break;
      case 3:
          SetStatusText("Location accuracy:", this.accuracy);
          MoveMap(this.latitude, this.longitude);
          break;
      case 5:
          HandleError(this.error);
          break;
      default:
         alert("Unknown state!");
   }
}
 
// Initiate a fix. This leads to the onlocationstatechanged event handler being called exactly once for each
// of the "connecting" and "acquiring" states and one or more times for the "fixed" state (for the initial
// fix and every time the location changes, after that).
location.startLocationUpdates(); // async call, initiates fix (powers up GPS if needed, etc)
 
...
 
// Getting the last known location
if (location.latitude != -1 &&
    location.timeUTC > threshold) {  // the location info is valid and not very old
  Foo(location.latitude, location.longitude);
}
 
// Cancel the request. This leads to the onlocationstatechanged event handler being called for
// the "canceled" state. This call will power down the GPS HW / close HTTP connection
// (depending on the location providers that were in use).
location.stopLocationUpdates();

I can imagine the fun games that I could write here, let alone the interesting business apps that could take the location context into consideration.

Other Future APIs

Disclaimer: This is early days, and who knows what the final API will look like, or if it will even make it. Do you have ideas for cool Gears that make the Web better? Let us know!.

Nov 30

So happy to see a Erik Thauvin post

Mobile, Tech with tags: , , , , 1 Comment »

Erik Thauvin’s link blog was huge back when I was doing TheServerSide. If you read both the TSS feed and Erik’s you were pretty sure that you were getting a look at most of the interesting content in the Java, and Enterprise Java communities.

I had huge respect for him, and it was sad to see his link blog die out as he got interested in other aspects of life. I can’t blame him. I also moved on from the world of Enterprise Java and got into other technologies and interests. Ajaxian was born to fill my “I love working with communities” itch.

Erik was developing top notch mobile applications and as soon as I launched devphone I knew I had to reach out to him.

So, I can’t tell you how excited I am to see this at the bottom of a post on how Google Maps Mobile works:

devphone 11/30/07 11:00 PM erik Java Networks Windows Mobile

Welcome, Erik. Who know where the ride will lead, but whatever happens, I am honoured to have some fun with you along the way :)