techno.blog("Dion")

You know that the Mac ad campaign is doing well when you get parodies left right and center.

The latest is Rails vs. Java.

It is of course gratuitous and silly…. but it’s fun :)

I often get side tracked seeing JRuby as just a cure for Rails deployment issues. Although it is huge to think of a fast Ruby container, and having access to JVM internals for management and such, JRuby also has a fantastic set of Java integration features.

This means that while you are in Java-land, you can actually hide Ruby implementations of features behind nice Java interfaces. Rob Harrop showed off some of this at his JRuby DSL talk at JavaOne last Friday.

Here is a simple example of having Java code doing some kind of calculation, and behind the scenes the implementation is in Ruby.

The pieces are:

• Java client code that does the calculation
• Factory to return the object that can do the calculation
• Calculation interface
• The Ruby object that does the work
• The Ruby file that makes the object implement the interface

Java Client Code

Calculator c = CalculatorFactory.getCalculator();
String calculation = "1 + 3";
System.out.println("Calculate(" + calculation + ") = " +  c.calculate(calculation));

Calculation interface

public interface Calculator {
public String calculate(String calculation);
}

Calculator Factory

The factory loads up the Ruby code, and then munges the Ruby object to return it back to Java-land as the Calculator interface:

public static Calculator getCalculator() {
Ruby runtime = Ruby.getDefaultInstance();

try {
runtime.evalScript(getContents("rcalculator.rb"));
runtime.evalScript(getContents("makejcalculator.rb"));
} catch (Exception e) {
System.err.println(e.toString());
e.printStackTrace();
}

Object c = runtime.evalScript("RCalculator.new");
c = JavaEmbedUtils.rubyToJava(runtime, (IRubyObject) c, Calculator.class);
return (Calculator) c;
}

The magic is done in JavaEmbedUtils.rubyToJava(runtime, (IRubyObject) c, Calculator.class);.

Ruby implementation of Calculator

The Ruby implementation is simple:

class RCalculator
def calculate(s)
return eval(s).to_s
end
end

To make it implement Calculator we monkey patch it by also loading the makejcalculator.rb file:

require 'java'

class RCalculator
include Java::Calculator
end

You could obviously do this all at once, but this pattern can allow you to share the Ruby code, and you may have other Ruby that you can’t just hack on.

This pattern will be common, so having a java_implement RCalculator, Java::Calculator simple piece of meta-code is useful.

All in all, JRuby gives you everything you need to do as much of your business logic in Ruby, and never expose it. This gives me thoughts of working on a Java team and telling my team mates about the interfaces they can use, and writing the entire app in Ruby behind the scenes ;)

I played around with getting Ruby to work in the browser while watching some cheesy TV this weekend.

I was able to get things slightly working, and put this work in progress up on Google Code here.

That link sends you to a page that will evaluate Ruby for you, both from a script tag, and from your own input.

If you take a look at the top of the file, you will see:

<script type="text/ruby">
#class Foo; def initialize; @foo = 'whee'; end; end; f = Foo.new; f
Time.now
</script>

This script is found by the Ruby in the browser system, and is run through JRuby via a little applet. The page takes the output at the end and throws it into the container HTML on the right.

You can also edit the text area at the bottom left and run your own Ruby code, and the output will be placed on the right too.

Now, this is just the first baby step to get to something feature full like:

<script type="text/ruby">
document.ready do |dom|
dom["table tr"] <<"<td>test</td>"
end
</script>

Just being able to run some Ruby doesn’t mean much. We need to give it rich support for the browser to be able to interact with the DOM and Ajax libraries to do cool things, using the power of Ruby.

So, there are a lot of TODOs here:

• The applet is a download of JRuby complete, which is huge, and isn’t all needed. I need to work with the JRuby folk to produce a slimmed down applet version.
• Come up with a solution that allows you to puts data out (maybe put that data in a div somewhere), and more importantly, talk to the DOM directly from Ruby. Since the applet can contain Ruby files themselves, we can create a Ruby module that lets you do DOM stuff, that becomes JavaScript which will be eval’d to run in the browser. Or, we just give direct access. I am still playing with what makes sense there
• Only tested in Firefox. There are some known issues in Safari.
• I am lazily using an applet tag instead of the magic object/embed stuff that would make this work in other places

Any direction make sense to you?

John Resig gave an enjoyable presentation on Javascript libraries.

This wasn’t a talk about choosing library A over library B. It was meta. It was about what libraries give you, and why they are here.

John also showed his text/jquery DSL that is up at jquery2 (a bad name that will not probably be jquery2).

We are seeing more and more of this. People are following the pattern of sneaking in their own script by putting in their own mime type and then using DOM scripting to go back and do the right thing.

I am hopeful that we will see a <script type=”text/ruby”> soon:

<script type="text/ruby">
document.ready do |dom|
dom["table tr"] <<"<td>test"
end
</script>

Yummy. Then Try Ruby would literally be… try ruby in the browser.

I am producing a new podcast and since I was using Blogger for the podcast blog, I really wanted to be able to throw enclosures right into the feed.

I had thought that Blogger didn’t support enclosures, but I found that this wasn’t the case. Although there isn’t an area in the UI to upload files to a blog post, the GData Blogger Data API does allow you to attach an enclosure to an entry.

Since the GData API works with Atom feeds, you will need to attach a correct link tag (as opposed to an <enclosure> tag from RSS) to the blog entry that you want to attach the podcast.

E.g. a simple enclosure can be added as

<link rel=”enclosure” type=”audio/mpeg” title=”MP3″ href=” http://foo.com/episode2.mp3 ” length=”19283″ />

You can also add other types such as BitTorrent.

To implement adding an enclosure I created a ruby script to do the work called addenclosure.

This script takes a bunch of arguments:

• Blog ID: If you don’t know your blog id, simply go to your feed at http://yourblog.blogspot.com/ /feeds/posts/default and look for the id field. It should look something like tag:blogger.com,1999:blog-4808741160899251111
. The numbers are the id that you need to plug in (4808741160899251111)
• Entry ID: To find this id, you can either look in the same feed and hunt for the correct entry: <entry><id>tag:blogger.com ,1999:blog-4808741160899251111.post-5004114743910071111</id>... and the numbers after the ‘post-’ are for the entry (5004114743910071111). You could also view the entry itself and look for the comment feed ( http://yourblog.blogspot.com/feeds/5004114743910071111/comments/default ) or various post links, that have the post id.
• Enclosure URL: You need to tell the script the location (URL) of the podcast itself ( e.g. http://foo.com/episode2.mp3)
• Enclosure Length: How big is that file? Let me know.

I fully admit that it is a little ugly to have to scrounge for the various IDs, but once you run this script you can visit the feed to see a brand new enclosure link. Throw this into a podcatcher such as iTunes and Bob’s your Uncle.

How the script works

The script itself uses my, alpha quality, GData Ruby library. The library was first factored out of gspreadsheet and consists of:

• GData::Base: The base library knows how to talk GData, including being able to authenticate to Google and use X-HTTP-Method-Override for HTTP methods such as PUT. It has the low level GET, PUT, POST API that other service specific libraries will build on
• GData::Blogger: The Blogger code knows how to speak to blogger and get and modify feeds and entries. It includes high level methods such as add_enclosure(..)
• GData::Spreadsheet: Likewise, the spreadsheet module has high level methods such as add_to_cell and evaluate_cell

Along with the library, scripts are also packaged to show the library usage:

• addenclosure: what we have been talking about
• bloggerfeed: view the entire feed
• gspreadsheet: run formula
• removeenclosure: nuke an enclosure from an entry

In the future I want to work with more GData folk to have a fully fleshed out API for Ruby-folk to talk to any GData service. A couple of people have already contacted me asking to add features ( e.g. high level support for the new Picassa APIs).

For more coverage, quotes, and photos, check out Ben’s thoughts.

The session that I was looking forward too was chatting with John Lam (RubyCLR) and Jim Hugunin (IronPython) to discuss dynamic languages on the CLR. This was the first session that reached out the crowd, so we got to shout out questions.

There was an immediate warning that they won’t be able to answer all questions. If it was the end of the month we would be in better shape. Come on Queensland. You know that Ruby – IL is coming soon.

Jim discusses his story on how he ended up at Microsoft, and how he wasn’t hired to write IronPython, but to be in the CLR team and make dynamic languages run better. Currently, he has a team that does most of the work.

Questions:

These were in the order that people shouted them out, and very much not in the order that they were answered.

1. Ruby or Python, which sucks less?: Jim is not a language biggot. Python happens to fit his mind best. There are lots of people who seem the same as me but get this mind meld with Ruby. “It annoys me that I can’t play around with a Rails apps without learning a new language. I want the languages to get along better together”. Hopefully that means that we will see some compelling examples of this in the future (of course both on the CLR mean that you can do some things). John finds the DSL meta model that Ruby gives you superior.
2. How are you going to lock us into your platform?: Two sides to it. IronPython’s goals have been about becoming a compliant implementation of Python, and being integrated with .NET. It is a challenge to do both at once. A few examples:

   s = "abc"
   s.strip
   vs.
   s.Trim()
   

   What should they do? Allow non-python Trim? throw an exception? Use import clr to allow Trim and friends.

3. What is the plan for PHP?: No one in the audience thought that it was really needed.
4. What is it going to take for Rails to run on top of .NET?: The audience wants this to happen. Michael Koziarski of the Rails Core team would like to see this so support questions can be answered. Others want to be able to deploy Rails into enterprise customers who deploy on Windows. Also, the CLR could take up the weakness of Rails/Ruby deployment and make it the best platform.
5. When is the veil of silence going to be lifted?: Jim explained how he was stuck under the veil for 8 months, and it drove him nuts. He is happy that now he is out there. IronPython 1.1 is now out, 2.0 is being talked about. We shouldn’t be harsh on John as he has only been part of the company for a matter of months. “If 8 months go by without an announcement, something has gone wrong.”
6. Why dynamic languages?: I want more freedom than I can get from these languages. Unit tests are the solution. We discussed how unit tests are great, but how many developers REALLY practice this discipline. Writing unit tests is like winning in Vegas. Everyone says that they are at least even with the house. Judging by the new $7bn complex being built on the strip, someone is lying. Jim hasn’t been able to persuade static folk over the years, but who cares? Lots of people love it, so let them program in it!
7. Do we want to run the CLR in IE?: The audience wants it. Jim/John like the idea. The use would be: fast JIT’d JavaScript, Ruby and such in the browser, and the ability to tie into the blackbox that is currently the JavaScript VM on IE.
8. Mono?: It’s getting better?
9. Should dynamic languages be an Intro CS100 language?: They have a lower barrier to entry. The concept count is lower than in a static (classes, methods, static method calls, …. to write hello world in Java, C#). This low bar means that you may see crappier code, as “non-programmers” can write it. This is a good thing. Jim spends half of his time in Python, half in C# and he likes it.
10. Phoenix?: wrong group.
11. Erlang? Haskell? OCAML? F#?: …..
12. What about IDE support?: They aren’t seeing the desire for IDE support in the community. John showed off his WPF Black on Black irb world. He is a super star :)

Two great guys. I wish we could have gotten in depth with what is coming up soon. Without that, it was hard to get value out of the conversation in the same way.

I used to cringe at having to work with XML. These days there are nice ways to work with it… from E4X to Groovy builders, and of course with Hpricot.

I wanted to take my OPML file and grep out the URLs so I could create a custom search engine that would search over my buddies (from the OPML file).

It is basically a one-liner with Hpricot:

require 'rubygems'
require 'hpricot'

filename = ARGV.first || 'mysubscriptions.opml'

doc = open(filename) { |f| Hpricot(f) }

(doc/"outline[@htmlurl]").each do |url|
puts url.attributes['htmlurl']
end

In my case the OPML file is just sitting on disk there, but I could easily have it grab the file from a URL:

require 'open-uri'
doc = Hpricot(open("http://almaer.com/mysubs.opml"))

Not bad until we implement JsDOM ;)

Oh, and here is my nice custom search engine:

Kris Rasmussen and Andy Holt from Competitious gave a SDForum tech talk on their experience with Rails.

They start out talking about their site Competitious and then jump into things. A few pseudo-quotes:

• “Rails actually scales quite elegantly”
• “Maintainability is an issue…. harder to debug”
• “We chose Rails as I got tired writing configuration code”. The component dream didn’t work out for me in Java
• “We built the first version of Competitious in 3 weeks”
• Problems: refactoring is hard, heavy processes, takes work to make production ready, does not support some complex queries, fetches all the data, all the time, so it is easy to over-query the DB.
• Production check list: use automatic deployment, filter_parameter_logging :password, store session in memory, cache data, make web server store the static files, security reviews (don’t rely on with_scope).
• Acts as everything: acts_as_notable, acts_as_loggable, acts_as_securable, acts_as_taggable, acts_as_mediable
• Biggest performance issue: Waiting on other services: ObjectCache.cache(key, CACHE_TIME) { third_party_stuff } (Alexa goes down a lot)
• Timeouts: status = Timeout::timeout(3) { Net::HTTP.get(url) }
• Asynchronous Polling: startup a new process/thread to communicate with the third party service. Use backgroundDRb (lightweight)
• JavaScript Templates: define_js_template(”our.tmpl”) / render_js_template. Share the view.
• Activity Logging: model: acts_as_loggable :name. controller: Media.with_logging(:user => @current_user) { .. }

I am spending time hacking away on Google APIs to really see what it is like.

I remember seeing the Google Spreadsheet Data API that allows you access to spreadsheets in Google Docs & Spreadsheets.

There is a full API that gives you access to create and modify spreadsheets even to the level of REST requests for each cell.

I was also surprised at the number of formulas available.

Suddenly I realised that I could create a spreadsheet and use a cell to do various calculations for me, so I hacked up a ruby script to do this:

% gspreadsheet [insert formula]
e.g.
% gspreadsheet ‘GoogleFinance(”GOOG”)’
467.3
% gspreadsheet ’sin(0.2)’
0.19866933079506

Behind the scenes the script used the Google APIs to put the formula in a cell, and then read from that field to get the calculated output.

Writing the script

To get this all working I just had to:

• Find a Ruby library to the Google APIs
• Work out how to authenticate using the Google ClientLogin API
• Work out the location for the REST requests
• Work out why I was getting a 404 error

Find a Ruby library to the Google APIs

I was surprised to not be able to find a nice API to Google Spreadsheets. I did find this script that helped a lot though.

I am working on packaging a gdata-ruby module that I will place in rubyforge soon. It will start out with just APIs such as ClientLogin and Spreadsheets, but hopefully we can grow it to cover more of them.

Work out how to authenticate using the Google ClientLogin API

The key to authentication is using the ClientLogin API to get the auth token, and hiding it away in a member variable so other requests will add it to the headers:

response = Net::HTTPS.post_form(https://www.google.com/accounts/ClientLogin,
{'Email'   => email,
'Passwd'  => password,
'source'  => "formula",
'service' => 'wise' })
@headers = {
'Authorization' => "GoogleLogin auth=#{response.body.split(/=/).last}",
'Content-Type'  => 'application/atom+xml'
}

Work out the location for the REST requests

To find out the location for feeds it helps to GET the feeds themselves and look for the post URLs in link tags.

To get the A1 cell (a.k.a. R1C1) you would use something like:

/feeds/cells/#{@spreadsheet_key}/1/#{@headers ? “private” : “public”}/basic/A1″

• The spreadsheet key is the magic key for each of your spreadsheets that looks something like: pSYwzniwpzSFfn0KFRg9oWB.
• The 1 right after that is the worksheet id.
• The private/public check get changed in this based on if the code is authenticating you or not (and if you have allowed public access to the spreadsheet in question).
• The ‘basic’ is a projection value. basic means just basic atom. ‘values’ means a full feed minus formula data, and ‘full’ means a full read/write feed with everything
• Finally we give the A1 cell info

For reading this cell we just used basic projection, but for writing data into the cell we need to use the URL:

“/feeds/cells/#{@spreadsheet_key}/1/#{@headers ? ‘private’ : ‘public’}/full”

Notice that we use full here (as we want full access) and yet we do not put in the cell in question. This is because we will POST or PUT an entry piece of XML:

<entry xmlns='http://www.w3.org/2005/Atom' xmlns:gs='http://schemas.google.com/spreadsheets/2006'>
<gs:cell row='1' col='1' inputValue='=sin("0.2")' />
</entry>

This has the row and column and input value (in this case a formula).

Work out why I was getting a 404 error

At first I was getting 404 errors when I posted data up. The reason was that I wasn’t putting the full namespaces in the entry doc:

<entry xmlns='http://www.w3.org/2005/Atom' xmlns:gs='http://schemas.google.com/spreadsheets/2006'>

vs.

<entry>

XML always seems so frustrating and when these things happen. Really? You couldn’t work out what to do without that namespace? Really? Nice and forgiving. Give me JSON or YAML ;)

The full code

It was fun to be able to be productive with these APIs immediately because they are just basic REST actions that just require Net::HTTP to access. I will work on getting some nice helper libraries so the low level stuff doesn’t even need to be done.

For those that are interested, here is the quick hack as one script file. The code is ugly… I am sorry. The library version is a lot nicer (and is just a few lines of code after the require’s). You can also get weird behaviour if you use single ticks. For now use ” and all is well.

gspreadsheet

#!/usr/bin/env ruby

require 'net/http'
require 'net/https'
require 'uri'
require 'rubygems'
require 'hpricot'

#
# Make it east to use some of the convenience methods using https
#
module Net  class HTTPS < HTTP
def initialize(address, port = nil)
super(address, port)
self.use_ssl = true
end
end
end

class GoogleSpreadSheet
GOOGLE_LOGIN_URL = URI.parse('https://www.google.com/accounts/ClientLogin')

def initialize(spreadsheet_key)
@spreadsheet_key = spreadsheet_key
@headers = nil
end

def authenticate(email, password)
$VERBOSE = nil
response = Net::HTTPS.post_form(GOOGLE_LOGIN_URL,
{'Email'   => email,
'Passwd'  => password,
'source'  => "formula",
'service' => 'wise' })
@headers = {     'Authorization' => "GoogleLogin auth=#{response.body.split(/=/).last}",
'Content-Type'  => 'application/atom+xml'
}
end

def evaluate_cell(cell)
path = "/feeds/cells/#{@spreadsheet_key}/1/#{@headers ? "private" : "public"}/basic/#{cell}"

doc = Hpricot(request(path))
result = (doc/"content[@type='text']").inner_html
end

def set_entry(entry)
path = "/feeds/cells/#{@spreadsheet_key}/1/#{@headers ? 'private' : 'public'}/full"

post(path, entry)
end

def entry(formula, row=1, col=1)
<<XML
<?xml version='1.0' ?>
<entry xmlns='http://www.w3.org/2005/Atom' xmlns:gs='http://schemas.google.com/spreadsheets/2006'>
<gs:cell row='#{row}' col='#{col}' inputValue='=#{formula}' />
</entry>
XML
end

def add_to_cell(formula)    #puts entry(formula)
set_entry(entry(formula))
end

private
def request(path)
response, data = get_http.get(path, @headers)
data
end

def post(path, entry)
get_http.post(path, entry, @headers)
end

def get_http
http = Net::HTTP.new('spreadsheets.google.com', 80)
#http.set_debug_output $stderr
http
end
end

if __FILE__ == $0
formula = ARGV.first || 'sin(0.2)'

gs = GoogleSpreadSheet.new([INSERT YOUR SPREADSHEET KEY])
gs.authenticate('[email protected]', 'your password')
gs.add_to_cell formula
puts gs.evaluate_cell('A1')
end

I’ve got to admit it’s getting better
a little better all the time

IntelliJ Ruby support keeps getting better.

I wish there was a lightweight mode where I could just open one .rb file and have it edit without needing the project setup.