Metaprogramming Roundup: Speed, Ruby Macros, Screencasts

Matt Aimonetti recently took a look at the speed of metaprogramming, in particular the speed of different ways of defining methods. He found that code using define_method with a Proc to define utility methods was running considerably slower than code using statically defined methods (ie. defined with def method_name).
However, in a follow-up article, Matt finds the reason and a solution for the speed difference. The reason for the speed difference:

Today Wycats pinged me about this post and told me that the issue was define_method and that class_eval is effectively the same as regular code, it gets evaluated in eval.c, just like regular Ruby code. On the other hand, defined_method has to marshall the proc.

Matt provides a modified version of the code he previously benchmarked using class_eval, which now runs as fast code using statically defined methods. A useful bit of information to keep in mind when working with metaprogramming.

On the other end of the metaprogramming spectrum is Reginald Braithwaite's Rewrite gem. The Rewrite gem uses an approach found in languages like LISP or Scheme called Macros or Macro expansion. In Ruby, this approach requires to step (slightly) outside the language - normally Ruby code doesn't have access to a representation of loaded Ruby code - Reflection stops as the method body level. To get access to code - to get the AST from the Ruby interpreter - the ParseTree extension is used for this in MRI 1.8. Rubinius can support ParseTree s-exprs (eg. the Debugger allows to access the s-exprs of methods) and there is an incomplete version of ParseTree for JRuby (it lacks support for accessing the AST from certain types of methods). ParseTree doesn't support Ruby 1.9.

The current incarnations of the Rewrite gem are used to solve one particular issue, but the principles in it will be generally usable. What is solves now is the problem of adding methods to classes (Open Classes). The problem is the global nature of modifying a class.  If one piece of code adds, say, a foo method to Object, all code in the runtime will see this method as well - which can cause problems like name clashes if other bits of code add a method with the same name to the same class.

The solution implemented with the Rewrite gem: limit the visibility of an added method to the scope of a block, which can look like this:

with(andand) do
 foo().andand.bar(blitz())
end 

The code in the Block handed to the with method is turned into a ParseTree s-expr, analyzed and rewritten. In this case, there is no andand method in any of the classes involved - instead the with method finds the code that invokes this method and rewrites it so it returns  the expected behavior.

It's important to keep in mind that the current ideas implemented in the Rewrite gem are ways to experiment with Macros in Ruby, and many more ideas are conceivable. While Ruby's convenient syntax for Blocks allows to achive a concise notation for many things, passing multiple pieces of lazily evaluatable code gets harder, requiring to more verbose methods of creating Procs.

For an introduction to the concepts like macros, a good place to start is "Practical Common Lisp". There are other projects making use of ParseTree to analyze code - InfoQ showed how Ambition, Sequel and merb use ParseTree. Also see Joel Klein's discussion of - what he refers to as lazy-lambdas - which are somewhere between Macros and Lambda's.

Finally, for everyone who's not familiar with the possibilities of metaprogramming in Ruby, a new series of screencasts by Dave Thomas (PragDave) gives an introduction to the concepts. The screencast series has already received a few very positive reviews: review by Antonio Cangiano and review by Mike Riley.