Charles Miller has written about Ruby Performance.
The XML file was large. 25MB. And Ruby was spending far too much time in GC.
It would be interesting to look at the code and see what is going on. I have never had to really push Ruby hard, but have never had any problems like this.
Of course, do you want to use a DOM in this situation anyway? ;)