techno.blog("Dion")

The releases kicked in over in Rails land:

• Rails 1.2 (the big one)
• Prototype 1.5
• Haml 1.0

Haml you ask?

!!!
%html
%head
%title= controller.controller_name
= stylesheet_link_tag 'main'
%body
#header
%h1 BoBlog
#content= yield
#footer
%p All content copyright © Bob

Watching all of the alternative view technologies gives me a bit of a Deja Vu to other technology. JHTML, JSP, Velocity, XMLC, Freemarker, ….

Before long we will have to fight against Enterprise Ruby Beans? ;)

I am not sure how I feel about a->foo = b->foo, having a magic -> operator for properties.

In theory I actually like syntactic sugar. It can be a pain to learn, but once you do you have more power in your own hands. On the other hand I really hate having > as part of an operator (which bugged me about generics). The peeve? Working with HTML (blog posts, html docs, etc) make it a royal pain. How many entries have you seen that make no sense and then you realise the the generic stuff wasn’t escaped. Grr.

I also prefer to think of sending messages to objects. If the message happens to correspond to a property, that is great. a.foo in Ruby could mean many things hidden behind various implementations, and there is no need to call out THIS IS A PROPERTY.

And then of course we like being able to method_missing and more, so a.foo may not even exist yet.

Could someone call James Duncan Davidson and get him to drive up to Seattle from Portland?

All joking aside, with the great progress on the deployment side, we still have room to improve.

It is still fantastic to see Amazon in the game and I am sure this is temporary.

I am sure there is a better way to do this, so please let me know.

It somehow came up that someone wanted to feel JavaScripty when creating objects with methods.

People often follow the pattern below in JavaScript:


{ foo: function() { ... }, bar: function() { ... } }


How about in Ruby? Lucas Carlson talked about dynamically adding methods to classes through their objects in Ruby, and of course it is simple to do:

o = Object.new
def o.foo
puts 'whee'
end

But what if you want to make this more JavaScripty:


x = {:a => lambda { puts 'whee' }, :b => lambda { puts 'foo' }}.to_o
x.a


A simple bit of code did this (without good checking of things):

class Hash
def to_o
o = Object.new
self.each do |k,v|
if v.is_a?(Proc)
o.class.instance_eval do
define_method(k.to_s, v)
end
end
end
o
end
end

I had a weird dream last night (literally a dream, not in the MLK sense). In the dream I was building a Rails app (I know, couldn’t I think about more interesting / racy things?) and the designer for the project got hit by a bus.

Instead of finding another designer, I delivered the UI that was already built. No, not even Streamlined but instead:


% ruby script/console


I didn’t JUST give them the command line, I also had a piece of paper that listed the domain models and documentation for them to grok, but even that told them how to look at methods on the objects to delve deeper.

This is Naked Objects taken to the extreme, but a lot of us probably use this “UI” don’t we?

This is because the developers of the application put the power in power users. We have to know everything about the system, and often have to dig deep into the system to fix bugs and such.

script/console is my best friend. It means that I do not have to drop to SQL to poke around my DB. Instead I have a more powerful tool to munge the data, in a language that I prefer for day to day tasks. Also, it means that I do not bypass any validation / business logic when playing with the data. I have known many a project that has SQL loading scripts that end up being invalid when business logic changes (and this logic wasn’t constrained in the DB).

Writing scripts that use the bootstrapping is great too. Now instead of doing a bunch of work in Ruby and then dropping to SQL, you can stay in Ruby land and it will do the hard work for you.

So, here is to script/console. The power users tool.

A lot of developers know the dangers of concatenation with Strings and objects. In Java we had the StringBuffer.append vs. + (and now StringBuilder) knowledge transfer.

Ruby has the same issue, and people have talked about it before.

We ran into this issue in one of our projects, and I remember Dave Thomas talk about a problem that was fixed by moving from string concatenation to putting the contents on an array.

I think this benchmark says it all:

require 'benchmark'
Benchmark.bm do |x|
x.report do
a = 'foo'
100000.times { a += ' foo' }
end
x.report do
a = 'foo'
100000.times { a << ' foo' }
end
end

Output

dion@stewart [~]$ ruby t.rb
user     system      total        real
13.790000  25.180000  38.970000 ( 40.102451)
0.060000   0.000000   0.060000 (  0.064342)

So, favour << unless you really want to copy strings around.

I am a long time Lucene fan, and was excited about being able to use Ferret in Ruby land to work on the same files.

That dream just died as David Balmain (Mr. Ferret) has jumped away from the Lucene file format:

This is the first Ferret announcement I’ve put up for a while, the reason being, the most recent releases of Ferret have been alpha releases. I completely rewrote Ferret from the ground up so that it no-longer uses Lucene’s file format and I was able to gain so great performance improvements in the process.

Do I need Ferret to use the same file format? Often-time no, as the app is in pure Rubyland, however, I know of a few projects in which being able to access the index from both worlds is a definite plus….. and not through a web service ;)

I guess we will have to use one of the other lucene ports for that.

“Hmm, this action is taking 45 seconds”

Subscriber.count: entire action == 1 second

“I am not getting the output that I expected”


<%=

render :partial => "most_recent_registrations", :locals => load_most_recent_registrations
render :partial => "summary_totals", :locals => load_summary_totals
%>


I often talk about how much I like clustered caching for a certain set of large scale systems (Give your DB a break).

I recently had the pleasure to add some simple caching to a Rails app. This time it didn’t make sense to add memcached and friends (some interesting work there with ActiveRecord support) because the problem was simple:

One page in a stats package that would be hit by ~4 users, a couple of times a day was very slow (30 secs) to dynamically generate. The data is NOT time sensitive.

Since this piece didn’t need to scale to thousands of concurrent users, and stale data was fine, we thought it would be enough to add some HTML caching.

One of our requirements was that we didn’t want one poor sucker to have to wait 30 secs to fill the cache. We wanted the cache to fill itself. Again, we ended up with a simple solution to this rather than some killer cache work.

Here are some steps though:

1. Rails Page Caching

At first it seemed simple to just add page caching.

All you need to do here is:

• Add “caches_page :action1, :action2″ to your controller
• Have some way to expire the page. You can create a Cache Sweeper, or have an action that expire_page’s things, or simply have something that nukes the html file.

Sounds great! However, all that is happening here is the caches_page will make sure that an after_filter is applied that saved the response content out to the file system. By default, a .html file is created in the public directory so apache can just slurp it right up and serve it. Very fast.

The sweeping is where you have to write some code, but it is simple to do, so you can’t totally complain (although it could actually be even simpler: I would love to see a system where you have a cache config so you put in expiration info in there and rails auto handles expiry).

The problem is that this full page caching only works if you CAN actually cache the entire page. Erm, surely that is most of the time right? Well, not really. If you have any user-specific work on the page it will not work. This could be as simple as showing a login/register area for non-logged in people, to a ‘Welcome Dion’ link to the users account if they are logged in. If you have special UI changes for admin users it will not work (e.g. add ‘Edit’ links next to content to give your admin users a simple interface).

This all means that you often can NOT use full page caching (and this was the case for us).

2. Action Caching

Another option here is to cache an action rather than the full page. The only difference is that: “unlike page caching, every request still goes through the Action Pack. The key benefit of this is that filters are run before the cache is served, which allows for authentication and other restrictions on whether someone is allowed to see the cache.”

So, it lets you still do filters (which we would need to do as we wouldn’t want the stats to be viewed by anyone), but still has the full page limitations.

3. Fragment Caching

The first two items sit on top of Fragment Caching.

Fragment caching is used for caching various blocks within templates without caching the entire action as a whole. This is useful when certain elements of an action change frequently or depend on complicated state while other parts rarely change or can be shared amongst multiple parties.

So, we could simply wrap the content that took a long time to load with:

<% cache do %>
... all of the long running pieces ...
<% end %>

And we can tweak keys via:

<% cache(:action => "list", :action_suffix => "all_topics") do %>

Now we finally have just a piece of content cached, so the full UI is still dynamic (get the logged in features) and all of the filters will of course run.

By why out of the box does this version still take a long time to load?

• If you are testing in development, caching will not happen for you because config/environments/development.rb will probably have:

  config.action_controller.perform_caching = false
  

  You are used to thinking of this in the form of “in dev mode Rails reloads files that I have changed”. In this case it is the HTML caching too. To test, change this setting to true in dev mode, and remember to change it back later ;)

• Where are you DOING the long running task? Just because you have a cache() wrapping the VIEW doesn’t mean that the controller isn’t running the action. If your action has @data = long_running_method that the view then uses, the caching will do nothing for you. Make sure that the action doesn’t do the long running thing, but that is kicked off from within the cache block itself. E.g. literally by calling the long_running_method, or by doing a render passing in the long running piece (which is inside the cache)

Ok, done right? Not quite. This has just created the cache file. Unless you want this to happen once and have that same version used for the rest of time you need to clean up.

We simply put something in cron to expire that fragment every X minutes (as it was simple). When working on an app that does more of this caching, then we create a cache config object that knows more about expiration and does the deed for you.

Also, remember the requirement not to screw the first user after expiry? The cron script does this in a hacky way. After it nukes the fragment it accesses the url of the page immediately to kick off a new cache page. This isn’t totally trivial as in our case the page is behind authentication and some filters that can trip the script.

To solve this we have a cookie that the process uses to get through the auth piece (security worry) and we turn off one of the filters (cookies_required) for this beast.

Conclusion

Rails has a bunch of built-in page caching mechanisms, but they aren’t THAT useful out of the box. You need to tweak and play around to get what you need, and most of the time you will NOT use the simple solutions.

For our large scale sites we still love the event-driven memcached approaches.

What have you done?