techno.blog("Dion")

The last week or so has been a stark reminder of how hard it is to do “Open”, and how the term itself doesn’t mean anything. There are many shades of grey when it comes to open. Let’s take a peak at what happened, and then try to come up with some tools to help us communicate what we truly mean.

Ext JS

This one has been talked to death. At its heart, the project originally had a license that was hard for people to understand. The “LGPL unless” clauses were unclear, and many thought untenable. At best, it was very cloudy.

After some people discussed the issues, mainly in private, with Jack we saw a new Ext 2.1 release until the GPL license (no more special clause). Some were unhappy about this, mainly because it was also unclear that the team actually said “we were wrong” and that these folks had a right to fork the project due to LGPL. Jack quickly came out with an exceptions clause and people are trying to iron this out.

SpringSource Application Server

SpringSource announced a new product for JavaOne, an application server. The Spring Framework itself is licensed under the Apache license, yet the new application server is GPLv3.

This caused a bit of a stur, as people were unclear of the difference, and I read a few posts saying that everything had been changed (which is totally untrue).

Marc Fleury came out of the biotech wilderness to comment on this all, claiming that this is just a packaging of the old stuff with a new license to kick into gear a new business model:

So voila, we now have a box drawn around an OSGi kernel, the Spring framework and Hibernate/Tomcat, and it has a name: it’s an application server. It is the same thing you had yesterday for free, except it is now under the GPL and a proprietary subscription license.

Rod and the team consistently argued that there is actually a lot of engineering here. One little embarrassing moment was in one of the TSS replies. Both Rod and Adrian replied to a message with the same boilerplate response:

Creating an application platform that makes the benefits of OSGi available to end users was a huge investment for us. There’s a *lot* of technical innovation under the hood which won’t be immediately apparent but which enables us to make a generational leap. If we’re giving that technology away in open source, we wanted others who build on it to also give away the results in open source.

When the community saw that, they thought that this was very much canned.

Adobe SWF / FLV Binary Specs

Adobe made a big PR announcement around the opening of Flash and more.

When you come out and say “now we are Open” it gets everyone excited. Their stock bumped up, which may or may not have been related to this (a lot of stocks went up at the same time).

I was ecstatic to see the news on the wire, because I think this will be great for the Web. If we could get Flash to be part of the Open Web, I would love to see it as a win-win.

Unfortunately, when I looked into the details, there wasn’t much to see. The claim was that the FLV/SWF/F4V binary formats will be Open, and there will no longer be the restriction that said you can’t RUN the code.

The problem was that there was no license to go along with this claim, which means that we can’t actually do much with it yet. Adobe isn’t more “Open” today than it was the day before the announcement. This will hopefully change very soon when we actually see the license, and hopefully see even more.

Time to learn

What did I learn through all of this? All of this licensing lark is about clarity and communication. I actually like all of the three parties here. I consider Jack, Rod and many Spring-ers, and lots of the Adobe folk as friends.

When it comes to licensing:

1. You need one, please
2. Please make it a standard one
3. Be careful which one you choose first, as changing it later will cause a lot of issues.

All of the concerns have been due to communication through licenses and on top of them. The license is a great starting point, but it isn’t enough.

So, what does it actually take to be Open?

• 0 points: Say you are open
• 10 points: Choose an OSI license
• 20 points: Define the governance of the code, or the protocols / specs. If the spec gets a license that is great, but how does it get changed? Does Adobe hold all of the cards still? Can others participate? For code, who participates? Can anyone patch? Can you, and if so how do you become a committer? At the core: HOW ARE DECISIONS MADE
• 30 points: An reference implementation under an open source license
• 40 points: Where does the IP stand? Did you donate it to Apache or some other foundation? For an example, you can see Exhibit B: Patent Non-Assertion Covenant for the OpenSocial Foundation Proposal

Now, this can be a gradual thing. It is common to start at one end and then slowly move down the stack. It took some time to get the OpenSocial Foundation in place for example, and everyone involved is still working out the governance model.

Also, no answer is “right”. You can put code out there as open source and hold all of the cards. That is your prerogative. It is so easy for us to sit back and say “Oh come on Jack, just put it all out there!” or “Spring? GPLv3? Come on!” or “Adobe, just open source the entire Flash VM!”. These decisions have huge business ramifications, which huge potential consequences. You can understand how it is hard.

All we can really ask is to have the clear communication. Just be honest with us. Be clear with your intentions. The ramifications really do effect us too. I may get more involved in a project that isn’t just run by one company, where they can change things on a whim. If the purpose for using open source is more than the insurance of “if they do something I can fork it” then this stuff matters hugely. Some are in the game for insurance, but in general I think that people like to also get behind causes. They want to put energy into something they believe in. As soon as this happens your project has a part of us in it, and you need to respect that.

I am really excited to see a day though where SWF/FLV does have a clear license, thoughts on governance and how the community can get involved there, and frankly, guidance from Adobe on why they are doing this. Based on that information, people will get more or less excited. Others have already reverse engineered the Flash formats, and a Flash player that lives in the wild under full control of just Adobe means a certain kind of “Open”… one that isn’t very. I have belief that over time, the need and desire will push Flash over the edge. You only have to look at where things have gone with Flex, Tamarin, and other open source projects at Adobe. Macromedia is winning, and over time Flash will surely be open source, especially as Silverlight gets better.

The Peeps Don’t Care

Finally, I know that 99% of the developers out there may not even care, let alone users. There are open source wonks who like to argue about licensing and methodologies. As I watched the John Adams HBO series, I felt a little like those fine chaps arguing over the finer details of things. Many of the people didn’t know what was going on there, or why a particular Article was written the way it was. But, they had drastic implications for the people. I think that the same can hold here for some of the projects.

We need to pat backs when they deserve it, and hold the feet to the fire a little when the details don’t match the rhetoric. I can’t wait to see a better software world continue to grow over time.

Every now and then, normally when talking to a libertarian, I think about how we are actually all connected to each other. It is impossible to sandbox yourself from society which leads me to conclude that I need to embrace it and do what I can to work out what kind of society we want to be.

With the current Ext JS debacle, you get reminded of how connected your project and business are to other people. Just because you own a company, doesn’t mean that you control it. When I think about my own company, Google, I realize that the most important currency is user trust. It doesn’t matter how many PhDs and great technology releases we have, if we ever lost some of the trust. I think that Google has earned its reputation, but all it would take is something that goes against what we have stood for so far, and we could lose it just as fast. I actually like this fact, as it keeps us honest.

It is a little like your tennis ranking. A rolling year of past performance is what really matters here. It doesn’t matter if you won that grand slam one year and one month ago. This is why every tournament matters. A bad showing loses points.

Of course, with user trust it is a lot more nuanced, and the graph is more exponential (the longer you go back in time, the less it matters).

Anyway, enough side tracking. When you have a software project that is a library for developers, your end users are those developers. If the project is open source, then there is a clear communication of the rules through your license. This is why open source licensing is so important. It allows you to have a simple contract saying “this is what you can and can’t do”. As a developer I can see GPL, BSD, Apache, and I know right away what kind of community this is, and how I can play a role. It isn’t about one license being better or worse than another. It is about communicating rights.

If you are fortunate enough to gain a real community, where other developers are participating, then the game starts to change. Now you have people who are invested in your project, maybe building on it for you, or evangelizing it, writing documentation, or creating their own business. At this point you really start to see what kind of project it is going to be, above and beyond the source code licensing. This is the Open Community side of a project. It can range from: only people who work for company X contribute in anyway, to: active commiters from all over the Web. This paints a picture of the project as a whole, and will have large effects on project, including who uses it. This is all about governance.

This also comes into play in other ways. When you think of Apache, or the Dojo foundation, you know about the legal protection that comes through the process. You know that everyone has signed a CLA, and that the history of the code is clean and well known. This has a huge effect on getting large companies into the game (This is why companies like IBM and Sun are so involved in Dojo IMO).

Now that you have users of various stripes, and a community with varied roles, you also have connections through out. If you then change the open source license for your project, the contract in the community has changed. When you make a change you not only need a good reason, but it has to be transparent, and you obviously have to get all of your ducks in a row to even be able to pull it off (e.g. depending on the change you may need every author of a line of code to get involved).

With Ext JS, there was a strange situation. The original license of LGPL-ish was very confusing, which lead to a confused community. Some kind o change was required, and clarity needed to be brought in. Unfortunately, it seems that the move to GPL has caused more chaos and confusion. Developers who poured a lot of time into the community (e.g. by creating GWT-Ext) are upset. The chaos can rip the community apart and you end up with a true lose-lose. Jack has spent far too much time and grey hairs on this one, instead of writing great code and growing his business.

So, it acts as a reminder, that the community is all connected. Everyone may not be equal, but make sure that communication is incredibly clear at all times to make sure that something like this doesn’t happen.

I was talking to a friend that does a lot of work in the realm of Open Source Java. He is someone who talks to people high up in the chain, and discussed how a lot of the CIO folks are getting a bit confused with the offerings. They had gotten used to JBoss. And, now they get Spring. But, they keep getting bombarded with more things. Next they had Mule, and Groovy.

Get some of these guys in a discussion about Mule vs. ServiceMix and they froth. Spring did something smart via Spring Integration, but maybe it is time for some consolidation? SpringSource + MuleSource + G2One? SpringyMule?

Jason Robbins and Jenan Wise have released a new Ajax source code browser for Google Code project hosting:

We recently launched a new source code browsing tool as part of Google Code’s project hosting feature. This new tool makes it easy to navigate through a project’s Subversion repository. Key features include: fast directory browsing tree, syntax highlighting, history of changes, and easy-to-read diffs. See it yourself under the “Source” tab of any project that we host.

The new system certainly feels fast, and uses jQuery to flip around in short order, jumping through the revisions of your system, expanding into new directories, etc.

This change required a change to my Google Code greasemonkey script that adds a direct link to the trunk. It now takes you right to the trunk of the code in the new browser!

One of the interesting differences between a liberal opensource project, and a commercial endeavor is that you don’t know who is using your stuff.

If someone wants to extend or integrate beyond your work, they don’t need to deal with you to make it happen.

The fun side-effect is when you get the code drop. Out of the blue, someone sends you a message and says “oh, by the way, I just ported X to work with Y”.

This really keeps you on your toes, and it gives you a gift. The “oh man, that is so amazing that they did that work, now I don’t need too!” gift.

I personally believe that Googlers will produce more open source in 2008 than prior years, and this post will discuss why. The same reasons apply to many other companies, which leads me to believe that we will see an increase in open source across the board.

Google does a lot of open source

When I am on my travels I get enough questions about Google and open source that has me realize that Google does a bad job in showing what a good open source citizen it really is.

Many people do know about the fantastic programs that Leslie Hawthorn and her team puts on which include: the Summer of Code and the Google Highly Open Participation Contest. This is the tip of the iceberg with respect to what the open source group does though.

As I talk to more and more people at Google, I find out about more and more people like Jeremy Allison, or the Samba team, who gets to spend much of his time hacking on the Samba project. There are Linux kernel hackers, Firefox contributors, and then you get to smaller pieces such as Hibernate Shards and MySQL tools.

If you have a butchers on Google Code itself you will find 135 current projects from Google, and this doesn’t include the magnitudes more that Googlers contribute too. 20% time really helps here of course, and it is a popular choice for engineers to dabble in the open source world one day a week. I can imagine the situation of working on some internal project, potentially something that is hush hush and secret, and having the option of spending some time on something very public indeed would be appealing.

So, we contribute a lot of open source code to various communities, we put out a bunch of our own code, and of course there are a number of high profile projects that have open source components to them: Gears, OpenSocial, and Android.

Erm, so if that is a lot… how come more?

Now I have hopefully shown a glimpse of the top of the iceberg for the amount of open source development that Google does, you may be wondering why I think 2008 will be ever better. Is it a numbers game based on the fact that we are constantly hiring more top notch developers? Well, that will hopefully help.

The core issue is that “doing open source” can mean many things. The code side of it is often the easiest part. Then you have the legal crud. What can be really challenging though is the process, especially when you have legacy code and procedures which everyone but the newest startups have (and the startups will have them soon too, as things change again).

Google is known for its engineering practices, and tools that help make them happen. I think that we do a really good job at providing a process that makes sense when you have as many programmers as we have, but not being heavyweight. A constant tight rope walk for sure.

One of the tools that has been talked about before is Mondrian, a Web based tool written by Guido himself, to help the review process of code. It is the kind of tool and process than if I left Google I would want to replicate, and would miss.

It is well known that Google uses perforce, yet most open source projects are using Subversion, or Git, or [insert your favourite new shiny toy]. This leads you to a problem when you decide to open source some code. How do you manage the fact that your code AND TOOLS are for X, when the open source world wants Y? If you are lucky, you can do a dump and have everyone using the other land. That sounds like the simple solution, but what if that code is being used in other projects within your company? This is where the real world gets complicated.

In many enterprise companies that I have had the pleasure to work with:

It is common for these process issues to be so painful that doing the open source thing isn’t worth it!

A lot of companies are still working out how best to handle these situations. It is easy for a small new company to come along and base its work on open source. You use what the open source world uses, and your life is easier. This shows how it can make sense to go down this path and favour open source software where appropriate. If you think that you may open source some of your code later, or want to integrate with open source, why make it hard for yourself?

One way to help with the friction is to open source more and more of your own code. You quickly find that although you thought you were just open sourcing Foo, you actually need to open source a slew of dependencies, and it would help to open source some of the tools too!

I only just ran across Review Board, which looks a little like Mondrian with support for various VCS. Promising indeed (Django/Python app).

Google is getting better and better and handling these issues, more and more tools and libraries are open, which means the path to open sourcing code in new project gets easier.

This is why I think that 2008 will be a fantastic one for open source at Google, and throughout the development community, as other large companies get better and better at it too.

What do you think? Have you run into these issues?

Today, I was able to launch a new section of Google Code that is focused on open source code related to the Mac. Since I am passionate about both open source AND the Mac, it was a really fun little side project to work on.

The core Google Mac team, and other developers at Google, actually have a fair amount of Mac related open source code. From small pieces of helpful code such as the Quartz Composer patches for Leopard, to large projects like MacFUSE to fun tools like iPhoneDisk to a new meta status updater called Statz. This is just the beginning of course, and now there is a home for this code.

One of the pleasures of working at Google is the slew of great people that you get to meet. On this project it was a real pleasure to work with Amit Singh (MacFUSE and much more), Nicholas Jitkoff (Quicksilver), and Dave MacLachlan (Adobe and much more).

Congratulations to Bob, Cedric, Romain, and the many many engineers that worked on Android.

Today the Android SDK was released, and along with it a raft of video content and documentation.

For all intents and purposes for developers Android == the SDK right now (until killer phones ship in short order). The development experience for Java programmers will be a dream, and I really like the architecture. They have really learned that declarative markup for UI is a Good Thing &tm;.

If you want to get a high level look at what this is about, I would start off by seeing what the phone can do:

Then, if you want to write some code, start by watching Dan build a simple application on Android and then delve deep into the Androidology that shows you the full architecture. Learn what .dex files are. See how cool Intents are. Check out the markup.

devphone.com: phones are decent now

I am really excited to see the bar being pushed by Apple, Google, and other players out there. You know that when the iPhone came out, Nokia had a lot of meetings and engineers got a better budget for doing innovative processes on their phones. With Android pushing the bar too, in a different way, I think that we can safely say that the phone that we hold in a year or two is going to be amazing.

I am quite astounded at how little Emily uses her laptop since she got an iPhone (apart from Scrabulous. She uses her laptop for her scrabble addiction.). You can see the future today by visiting Europe and Asia. Ben and I were so excited about this, that we went looking for a community to rally this excitement, kinda like Ajaxian for mobile. We were surprised that we couldn’t find it. It seems like most developers hang out in the forums and such of the various platforms. Since we are interested in development that transcends one implementation, we decided to start devphone as a place to throw all of the ideas into. It just launched and is very raw, so who knows what will come of it. Check out our welcome, an interview with Joe Hewitt, and subscribe to the feed

One of the fascinating effects of working for a company that so many love to keep a look on is that you get to be on the inside watching the thoughts of “analysts”, press, and random folk.

Watching the speculation around the “GPhone” has been fun. I particularly loved it when people would come up with suggestions such as:

• “The GPhone will read ads into your ear before each call”
• “The GPhone will have scrolling ads through the screen”

Riiiiight. That would go down really well wouldn’t it! I love how some think that Google has to literally put ads everywhere to make it worthwhile. Google needs the web to keep expanding and to have more people on more devices on it. If that happens, Google will do well.

Android

So there isn’t a GPhone, but instead there is the Open Handset Alliance, or Android (for a more fun name), which is an Apache licensed open source stack for mobile. No more walled garden. This is pretty huge. I can’t wait for the SDK to get out in the open on November 12th. I wish we could have gotten more information out there today to be honest. There are a bunch of usual suspects that people who read my blog also read that are a major part of this, so I am really happy for them that this is getting out in the open!

What I am looking forward to

The applications of course. Smart location-aware services will be fun, but what I really wish I could get is for a mobile digital wallet so I don’t have to use cash/credit cards. I want to use my phone for this just as they are able to do in parts of Europe and Asia. With an open platform that anyone can build applications for, I know it is going to happen.

Here is the fluffy look at Android, but for developers, please think about what apps you would like to see on a phone, and come back on November 12th for the real announcement that we care about… the SDK itself so you can see what you can do!

I played around with getting Ruby to work in the browser while watching some cheesy TV this weekend.

I was able to get things slightly working, and put this work in progress up on Google Code here.

That link sends you to a page that will evaluate Ruby for you, both from a script tag, and from your own input.

If you take a look at the top of the file, you will see:

<script type="text/ruby">
#class Foo; def initialize; @foo = 'whee'; end; end; f = Foo.new; f
Time.now
</script>

This script is found by the Ruby in the browser system, and is run through JRuby via a little applet. The page takes the output at the end and throws it into the container HTML on the right.

You can also edit the text area at the bottom left and run your own Ruby code, and the output will be placed on the right too.

Now, this is just the first baby step to get to something feature full like:

<script type="text/ruby">
document.ready do |dom|
dom["table tr"] <<"<td>test</td>"
end
</script>

Just being able to run some Ruby doesn’t mean much. We need to give it rich support for the browser to be able to interact with the DOM and Ajax libraries to do cool things, using the power of Ruby.

So, there are a lot of TODOs here:

• The applet is a download of JRuby complete, which is huge, and isn’t all needed. I need to work with the JRuby folk to produce a slimmed down applet version.
• Come up with a solution that allows you to puts data out (maybe put that data in a div somewhere), and more importantly, talk to the DOM directly from Ruby. Since the applet can contain Ruby files themselves, we can create a Ruby module that lets you do DOM stuff, that becomes JavaScript which will be eval’d to run in the browser. Or, we just give direct access. I am still playing with what makes sense there
• Only tested in Firefox. There are some known issues in Safari.
• I am lazily using an applet tag instead of the magic object/embed stuff that would make this work in other places

Any direction make sense to you?