Feb 05

Google Gears Database API on the Server

Gears, Google, JavaScript, Tech with tags: , , 2 Comments »

As soon as I started to play with Aptana Jaxer, I saw an interesting opportunity to port the Google Gears Database API (note the Gears in the logo!)

If I could use the same API for both client and server side database access, then I can be enabled to do things like:

  • Use one API, and have the system do a sync from local to remote databases
  • If the user has JavaScript, use a local database, else do the work remotely
  • Share higher level database libraries and ORMs such as Gears DBLib for use on server side data too

I quickly built a prototype to see if this would all work.

The Jaxer shim of the Gears API was born, and to test it out I took the database example from Gears itself and made it work.

To do so, I only had to make a few changes:

Run code on the server

I changed the main library to run on the server via:

<script type="text/javascript" src="gears_init.js" runat="server"></script>

I wrapped database access in proxy objects, such as:

function addPhrase(phrase, currTime) {
  getDB().execute('insert into Demo values (?, ?)', [phrase, currTime]);
}
addPhrase.proxy = true;

This now allows me to run the addPhrase code from the browser, and it will be proxied up to the server to actually execute that INSERT statement.

This forced me to separate the server side code from the client side code, which is a better practice anyway, but it does make you think about what goes where. In a pure Gears solution I can put everything in one place since it all runs on the client.

Create the new gears_init.js

A new gears_init.js acts as the shim itself. Instead of doing the typical Gears logic, it implements the Gears database contract. This wasn’t that tough, although there are differences between the Gears way, and the Jaxer.DB way. The main difference is to do with the ResultSet implementation, where Gears goes for a rs.next()/rs.field(1) type model versus the Jaxer.DB rs.rows[x] model.

I actually much prefer Gears DBLib as it hides all of that, and just gives the programmer what he wants… the rows to work on.

oncallback magic

In the current Jaxer beta, I ran into an issue where I wanted the Gears library to just “be there” for any proxy requests.

You have to think about the lifecycle of a Jaxer application, and the documentation tells you what you need to know to work around the issue.

In this case, I wrapped the code in:

function oncallback() {
  // create the wrapper here
}

This is less than idea, and Aptana is playing with nice scoping which would enable you to just say “hey, load this library once and keep it around for the lifetime of the server | application | session | page”. That will be very nice indeed.

You can do a little bit of this by opening up your jaxer_prefs file and adding the resource for your file:

// This option sets up an html document that will be loaded
// everytime a callback is processed.  This has to be a local file.
// If not specified, an empty document will be loaded.
// pref("Jaxer.dev.LoadDocForCallback", "resource:///framework/callback.html");

Future…

This is just the beginning. As I mentioned at the beginning, I am interested to see where you can take this to handle clients who do not support JavaScript, and also to deal with synchronization with minimal code (sync from local to remote with exactly the same SQL API).

Feb 01

Google Social Graph API Released

Google, Tech with tags: , 2 Comments »

Would you like to be able to make a quick call to get a JSON response that ties together a social graph made up of resources available on the Web?

Social Graph API

Brad Fitzpatrick, Kevin Marks, and others at Google have released a new Social Graph API that does just that:

The new Social Graph API makes information about the public connections between people on the Web easily available and useful. You can make it easy for users to bring their existing social connections into a new website and as a result, users will spend less time rebuilding their social networks and more time giving your app the love it deserves.

Here’s how it works: we crawl the Web to find publicly declared relationships between people’s accounts, just like Google crawls the Web for links between pages. But instead of returning links to HTML documents, the API returns JSON data structures representing the social relationships we discovered from all the XFN and FOAF. When a user signs up for your app, you can use the API to remind them who they’ve said they’re friends with on other sites and ask them if they want to be friends on your new site.

This is exciting to me as:

I gave it a quick test drive, and when I say quick, I mean 5 minutes :)

I built a tiny JavaScript library that takes a base URL, and it graphs out the relationships using Canvas.

You get to call loadGraph(URL, { width: w, height: h }) and the graph will be injected away.

It needs to be nicely abstracted and isolated so you can call it willy-nilly, but it works.

Watch the introduction video:

Jan 29

My interview with Steve Yegge on Rhino on Rails

Google, JavaScript, Tech, Web Frameworks with tags: , , 17 Comments »

I have been a long time follower of Steve Yegge and his long blog entries that manage to keep your attention. He has the opposite style to me (I know how curt and bad my writing style is!), so I envy it a little.

Ever since he presented on the ‘Google Rails Clone’ at FooCamp and posted about the internal Google Rhino on Rails project, people have been curious to learn more.

  • What does it mean to port Rails to JavaScript?
  • What can’t you do since JavaScript doesn’t have the same meta programming facilities?
  • Rails = a group of Active*, so did you re-implement everything?
  • What do you gain out of having JavaScript all the way down?
  • Does it actually make sense to have jjs? Server side JavaScript generating client side JavaScript? Argh!
  • What is the state of Rhino?
  • Will Rhino support JavaScript 2?
  • How does the JVM help you out?
  • What are the ramifications of implementing ActiveRecord with Hibernate
  • Fun other languages to play with

And of course, the big questions:

When do I get to see it!

I happen to be in Seattle at the Google offices, so I was able to ask all of these questions and more. Steve was a fantastic host, and I really enjoyed chatting with him.

This is the kind of video I want to explore at Google. We have many great developers working on cool technology. I want to get them on camera, participating with the community when I can. Sometimes we can talk about products and APIs, but sometimes we will talk about fun ideas and projects that we are working on such as Rhino on Rails.

Anyway, give it a watch and let me know what you think:

Jan 27

Valkommen and Good-ay to Jon Tirsen

Google, Tech with tags: 1 Comment »

But what if you’re tired of software development as a concept? What if you’ve had it with yet another programming language? What if you want to build apps that maybe even your grandmother would use (I have two very internet-savvy grandmothers)? What if you want to focus on what to build not on how to build it?

This is the reason that Jon has given for why he is leaving ThoughtWorks and joining Google.

I am really excited to see Jon join. I met him at an AOSD conference and we shared a train trip back from the middle of nowhere to London. I immediately liked him for so many reasons. Anyone who I can have a good laugh with is right by me, and he is smart to boot.

I hope that we get to work together in some way, allowing me to sneak to Down Under to meet up with him. See you in Mountain View shortly though mate!

Jan 23

Dynamically Loading Google Maps for Performance

Google, Tech with tags: 8 Comments »

A fellow JavaScripter was running into an issue where the standard way to load the Google Maps API was causing a perceived slow down of a single app page. The API has added the ability to get away from document.write() and dynamically load everything up.

Example code

function loadScript() {
  var script = document.createElement("script");
  script.type = "text/javascript";
  script.src = "http://maps.google.com/maps?file=api&v=2.x&key=PUT-YOUR-KEY-HERE&async=2&callback=loadMap";
  document.body.appendChild(script);
}
 
function loadMap() {
  var map = new GMap2(document.getElementById("map"));
  map.setCenter(new GLatLng(37.4419, -122.1419), 6);
  map.addOverlay(new GMarker(map.getCenter()));
}

Here it is at work in an iframe below:

Jan 15

Gears Future APIs: Services / Daemon API

Gears, Google, JavaScript, Tech, Web Frameworks with tags: 4 Comments »

Daemon

One of the comments on the Notification API was by lunatix:

It would be really useful if we could set a background javascript process (via WorkerPool) that continue to run after browser is closed and which is able to send System Tray notification.

I would definitely like to see an API (or as part of the Notification API, or WorkerPool in general) that allows you to attach some work to a process that can always be running, or be scheduled to run (a la cron).

This can be particularly important with Offline and syncing that comes along with it. Imagine a world where Gmail worked offline. The need for a service that downloads email in the background may not be that important, as people tend to leave a tab with Gmail running in it.

But, if you take Zoho Writer as another example. You may not have Writer open all the time. You open it when you get a new document that you want to work with. Let’s take a look at a scenario:

  • Bob edits a shared document that discusses the travel for the next quarter
  • Bob closes the document and keeps working for the rest of the day
  • Meanwhile, Harry, Linda, and Chou all edit the document
  • Bob goes offline and heads to the airport
  • Bob opens up the document on the plane

Does Bob just have the document with content from his step? If there was a service that was running in the background, it could detect the changes and bring them down. Then Bob would have the latest and greatest up to when he turned off his computer.

The obvious issues

Of course, a key issue here will be making sure that this isn’t abused, and what UI do we give users. I personally never like it when I find services running on my machine when I closed an app. You know the culprits…. Quicktime and the like. I want to be able to know about every service and prune the list, but also not be burdensome for the average Joe. Does this mean that the system tray notification system has a way to see all of the services?

Also, I don’t want some rogue daemon taking up my resources, and of course, it can’t have special access to things such as key logging ;)

You can write Gears too!

A few people have emailed me about potential Gears saying that they wish they could build a new Gear. Remember, Gears is a true open source project, so you CAN write your own Gears. If you want to implement a new API, start out by emailing the group with your proposal and then get to it!

Other Future APIs

Disclaimer: This is me rambling about APIs and tools that I would love to see in Gears, or the Open Web as a whole. Do you have ideas for cool Gears that make the Web better? Let us know!.

Jan 14

Prediction: Google will produce more open source in 2008

Google, Open Source, Tech with tags: 2 Comments »

I personally believe that Googlers will produce more open source in 2008 than prior years, and this post will discuss why. The same reasons apply to many other companies, which leads me to believe that we will see an increase in open source across the board.

Google does a lot of open source

When I am on my travels I get enough questions about Google and open source that has me realize that Google does a bad job in showing what a good open source citizen it really is.

Many people do know about the fantastic programs that Leslie Hawthorn and her team puts on which include: the Summer of Code and the Google Highly Open Participation Contest. This is the tip of the iceberg with respect to what the open source group does though.

As I talk to more and more people at Google, I find out about more and more people like Jeremy Allison, or the Samba team, who gets to spend much of his time hacking on the Samba project. There are Linux kernel hackers, Firefox contributors, and then you get to smaller pieces such as Hibernate Shards and MySQL tools.

If you have a butchers on Google Code itself you will find 135 current projects from Google, and this doesn’t include the magnitudes more that Googlers contribute too. 20% time really helps here of course, and it is a popular choice for engineers to dabble in the open source world one day a week. I can imagine the situation of working on some internal project, potentially something that is hush hush and secret, and having the option of spending some time on something very public indeed would be appealing.

So, we contribute a lot of open source code to various communities, we put out a bunch of our own code, and of course there are a number of high profile projects that have open source components to them: Gears, OpenSocial, and Android.

Erm, so if that is a lot… how come more?

Now I have hopefully shown a glimpse of the top of the iceberg for the amount of open source development that Google does, you may be wondering why I think 2008 will be ever better. Is it a numbers game based on the fact that we are constantly hiring more top notch developers? Well, that will hopefully help.

The core issue is that “doing open source” can mean many things. The code side of it is often the easiest part. Then you have the legal crud. What can be really challenging though is the process, especially when you have legacy code and procedures which everyone but the newest startups have (and the startups will have them soon too, as things change again).

Google is known for its engineering practices, and tools that help make them happen. I think that we do a really good job at providing a process that makes sense when you have as many programmers as we have, but not being heavyweight. A constant tight rope walk for sure.

One of the tools that has been talked about before is Mondrian, a Web based tool written by Guido himself, to help the review process of code. It is the kind of tool and process than if I left Google I would want to replicate, and would miss.

It is well known that Google uses perforce, yet most open source projects are using Subversion, or Git, or [insert your favourite new shiny toy]. This leads you to a problem when you decide to open source some code. How do you manage the fact that your code AND TOOLS are for X, when the open source world wants Y? If you are lucky, you can do a dump and have everyone using the other land. That sounds like the simple solution, but what if that code is being used in other projects within your company? This is where the real world gets complicated.

In many enterprise companies that I have had the pleasure to work with:

It is common for these process issues to be so painful that doing the open source thing isn’t worth it!

A lot of companies are still working out how best to handle these situations. It is easy for a small new company to come along and base its work on open source. You use what the open source world uses, and your life is easier. This shows how it can make sense to go down this path and favour open source software where appropriate. If you think that you may open source some of your code later, or want to integrate with open source, why make it hard for yourself?

One way to help with the friction is to open source more and more of your own code. You quickly find that although you thought you were just open sourcing Foo, you actually need to open source a slew of dependencies, and it would help to open source some of the tools too!

I only just ran across Review Board, which looks a little like Mondrian with support for various VCS. Promising indeed (Django/Python app).

Google is getting better and better and handling these issues, more and more tools and libraries are open, which means the path to open sourcing code in new project gets easier.

This is why I think that 2008 will be a fantastic one for open source at Google, and throughout the development community, as other large companies get better and better at it too.

What do you think? Have you run into these issues?

Poisinous Open Source

Jan 13

It’s a lot of money

Google, Microsoft, Tech No Comments »

Microsoft as a company still posts considerably higher revenue than Google does. Microsoft’s executives last week trumpeted the fact that the company — consisting of everything from the Office productivity suite to the Xbox 360 game console — surpassed $50 billion in annual revenue for the first time in the fiscal year ended June 30.

Two companies. Lots of money.

Jan 10

Gears Future APIs: Notification API

Gears, Google, JavaScript, Tech with tags: , 14 Comments »

Growl

The browser is a fantastic platform to develop applications on. The reach of the Web is unprecedented and the psychology of “installing an app” versus “going to a web page” is huge.

There are still desktop integration features such as desktop shortcuts that link the browser to the underlying host in a nicer way.

One of the missing links is between the browser and the system tray and notification system.

I want to see a Notification API Gear that allows you to tie this together.

Use Cases

Instead of having to install notification agents for applications that you use such as Gmail and Google Calendar, the notification can be built into the applications themselves. You upload a video to YouTube, and it will get back to you when it is done. You choose to “watch” a particular post on the Web? It tells you when there are updates. Google alerts on certain tags do the same.

I find that in my river there are a few occasions where I want to be interrupted (which notifications can do). Depending on your preferences these occasions may be “when my wife emails” or “when someone talks about some cool Ajax thing” or “when someone emails me directly” or “when someone @twitters me”. If I really want to know, I want a notification. If it is slightly less important, I want an email.

In fact, if it is TRULY important, I may even want a text message. This is where configurability comes in. Growl is an event notification for Mac OS X, and it nails all of this. It abstracts the applications from the user. I can go in and change the dials so I get notifications when I want them, how I want them (e.g. in my face, SMS, email), and what they look like (styling). This also includes being able to turn them off.

I want Growl for the Web. Applications can get permission to add events to a notification queue, and I the user can choose how to deal with those notifications.

The API

The API itself should be as simple as possible. This is pretty much a messaging pub/sub system, where the application can publish messages on a queue for its app, with different message types for the type of events.

var notifier = google.gears.factory.create('beta.notifier', '1.0');
 
notifier.notify({
       application: "My App",
       title: 'warning',
       description: 'some text',
       priority: 2,
       sticky: 'True',
       password: 'Really Secure',
});

Growl allows you to have actions in the message itself. That would mean allowing anchor links in the description in the Web world.

There is also the thought that you could also subscribe to various events from other applications, but that can get messy.

System Tray

This notification system could tie into the system tray, or that could be its own API. I worry about the proliferation of tray icons though :)

Beware

The big worry is that people will start spamming the notification system. This is a real concern, and needs to be seriously considered from the beginning. The balance will be in how much we ask users to do, without asking too much so no notifications ever get turned on. With a growl like system it will be easy to turn on and off notifications per application.

What do you think? Is this a useful API?

Other Future APIs

Disclaimer: This is me rambling about APIs and tools that I would love to see in Gears, or the Open Web as a whole. Do you have ideas for cool Gears that make the Web better? Let us know!.

Jan 07

The connection between Hope and Web innovation

Gears, Google, Tech No Comments »

There are a couple of pet ideas that I have had with respect to JavaScript and Gears. When you think of Gears as a base platform that can upgrade the Web, you think about the ECMAScript 4 fun in a new way.

A common thought pattern starts with optimism:

*ding* Wow, wouldn’t it be cool if my browser could do X?

or

As a developer, I would love to be able to do Y.

And then you often get to disillusionment:

Bugger, but if IE doesn’t support it, I am screwed.

I think that there is probably a good connection between Hope and Web innovation. I have felt this many times. If I feel like I can do something, I go after it with vigour. On the other hand, if I know that after a ton of work it won’t work for a large number of people, then I am far less likely to take her home.

How does this relate to JavaScript again?

What if JavaScript 2 (ECMAScript 4) comes out at a subset of Web developers are really excited about the new features. They want to get into packaging/namespacing/programming units. They grok optional types and like them. The pythonista in them enjoys the generator-like support. And, I could keep going with the large number of features. After you get over your initial excitement, you think about how a new set of libraries could make life a lot better. Prototype 2 is born that supports JS 2.

And then you start thinking about deployment. Darn it, maybe Microsoft doesn’t support it. Hmm. Of course, this is where the monkey comes in from Mozilla, but I immediately thought that Gears could help here. We could see if a script is asking for JavaScript two, and on demand install Action Monkey or anything else. Maybe together, Gears and the monkey could push out JavaScript 2 in a way that would push Microsoft to implement it themselves at some point (IF they don’t right away!).

What about the script of JavaScript?

I have talked about this a bit before, but I would also love to do smart work with respect to registering libraries enabling sites to not have to re-download and compile them all the darn time. We register Dojo, Prototype, jQuery, YUI, GWT, and [insert your favourite library] in a versioned way, and these versions can even be optimized for a particular browser. Instead of the if (thisbrowser) crud that we still see now and then (instead of the if (FEATURE)) we could have browser specific code, and all of the code can be compiled to be super fast.

When an application loads Dojo v1.0.2, it is swapped in with crazy speed.

If we ever get to this reality, then think about how things change. All of the optimizations and “keep my library to 3kb” arguments change. We always want nice small libraries, but we won’t have to make the compromises that we have had to in the past. We can do more in these libraries.

Gears isn’t the only solution here of course. Browsers can do this too. Ideally, browsers would do it, and Gears would be there to once again mop up for browsers that don’t implement it.

Wot no server?

There is also the server side of the equation. I have long thoughts that:

JavaScript needs a CPAN

It needs to be:

  • Crazy fast
  • CDN (see: crazy fast)
  • Access is given to the projects themselves to manage (not run by one company person)
  • Community driven

We have had JSAN, and other sites out there, but we need a big guy to come along. Yahoo! does it for YUI. AOL does it for Dojo. I want someone to do it for everyone. Of course, I would love Google to be that place, but I don’t care who actually does it. Google already has a bunch of people using Google Code to hotlink to their libraries, as it actually fulfills most of the requirements. There are some issues such as the fact that it only works for open source projects, and how it isn’t explicitly made for this purpose.

With script registration, and a fast distributed server for the script, we have a much better place. Add to this new JIT JavaScript VM’s, and 2008+ looks like a different place.

And there is more…

You can also take the case where Gears swaps in a JS 2 engine and generalize it. What if someone came up with a new CSS engine? Or any other library that makes sense to be cross-browser-platform. Hmm.

Other Future APIs

Disclaimer: This is me rambling about APIs and tools that I would love to see in Gears, or the Open Web as a whole. Do you have ideas for cool Gears that make the Web better? Let us know!.

Loading...