BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Interviews Dr Nic Williams on Sustainable Ruby Open Source Development

Dr Nic Williams on Sustainable Ruby Open Source Development

Bookmarks
   

1. We are here at Future Ruby 2009 in Toronto. We are here with Dr Nic. Why don't you tell us a little bit about yourself?

I am actually a doctor. I have brought in my thesis and I think now, here, on http://www.infoq.com/, we can formally say that the thesis is being presented.

   

2. It looks very official, so we guarantee it.

I am primarily a Ruby developer, Javascript, did some Cocoa, MacRuby - I played with , anything, as all programmers do. I think historically I've become a Ruby developer because it's awesome, so what's not to love? Rails; I run a consultancy called Mocra doing Rails and iPhone development and are based at Brisbane, Australia, which is a lovely place. You should come and visit. It's really hot around Christmas, but what's not to love?

   

3. In the Ruby community you are known for the myriad of tools. What are some of your favorite tools that you created?

My pet project is NewGem, which is the one I created after I made a couple of gems - the composite keys and the magic models and then I realized that they are bit of a pain in the bum to write. Just getting started is..., Jay Fields has written this simple 'How to start a Ruby Project' script. I turned that into another gem that pretty is his script except it's what I thought I wanted and then that became what I've evolved and because I use it all the time. Then, I extracted out - a couple of years later - the Rails generator code from Rails that Jerry Kemper wrote into its own gem so that Merb and Adhearsion and many others can use.

   

4. That's the rubigen Project.

I know, it's tragically named. It looks great when you write it out, but you can't say it without saying Ruby Gem. If I'd maintained it, I might have renamed it, but who cares? It uses that name so now it has a really nice sweet definition of what gets generated and has internal generators, so you can generate a Cucumber, C extension starting points, support Test::Unit, RSpec and everything. It makes looking after gems a lot easier. Internally, it uses Hoe for all the rake tasks, etc. That's my pet project and everything else flows from that, and then the TextMate bundles.

   

5. Talking about TextMate, you maintain a bunch of tools for TextMate, or bundles or whatever they're called?

TextMate is an extension of tools. It's an editor that enables integrated tooling so you can do your own thing and I love that.

   

6. Why do people use TextMate?

We use TextMate because David told us to. When he launched the 15 minute 'How to use Rails' video, he said you use TextMate and then Rails works. After a while, you figure out that there is a level of independence between using TextMate and running Rails, but initially you don't know that. Even when I first started Rails - obviously that's a joke - I did it on Windows machines, so, I used RadRails, back in the da. But like for every Windows person and aspiring TextMate owner, TextMate is really expensive, because it's like 3,000 $ or something. You get a free Mac with your TextMate editor, but it's relatively expensive to get started with TextMate, if you are not owning a Mac.

That was the first piece of software that I purchased when I bought a new Mac. I know more that you can do with TextMate now that I actually own it. But even before that I wanted to use it, so it's something special about it. It lets you configure nearly any aspect, invoke commands, invoke snippets. Because in the background it's either just static text snippet or a command and the command can be written in any script. So we just write the commands in Ruby and therefore we can write tests for them and it's production code that we can share and have a good time. A new editor is coming out - theoretically, one of the new editors might be TextMate 2. In the meantime, there are other editors, but they have so much to catch up to in order to get to what we expect from TextMate, I feel bad for them. I don't know how you communicate to you if you are not using TextMate. It's more like Twitter. Who knows what Twitter is for?

   

7. It's more like VI users - they also can't tell the users why they use VI.

VI and emacs - they are the same. You can extend them. I did try to use emacs on Windows - this was in 2006-2007 - but the 21.4 build, I think, no one has done a compilation of that for Windows and that had all the cool icon changes. I remember this now, for 2 days I tried to move away from RadRails onto something that was more powerful and at that time that would have been emacs. I couldn't get it to work the way I wanted to and then I was a sad boy. Then, as a tribute to that, I ported as many TextMate bundles, the Rails and HTML bundles and CSS and those 2 bundles I ported to RadRails, as much I could. That was my little tribute to Rails, I got those.

   

8. What do those bundles actually do? Can they run 'generate', or do they insert snippets?

There are a couple of aspects - generators are one, it's like in-line generators. It might be a method snippet. Instead of having to type - it's kind of a simple one - you can type def, press Tab and it will give you the starting point for the method. def will give you the def self. and type a method name in, and that's just the starting point! There are snippets for starting points, but then you can activate entire chunks of code that it could figure out the filename. We now have to create a class. You type cla, you press Tab, it figures out the name of the file and gives the default class name based on the name of the file.

So, you have the ability to really do anything. In application structures like Rails, where it's fixed, you got apps, folders, models, views, controllers, there is going to be test or spec, the upside of that static structure. It means that the tool can then know about that. So we have ways to go from a model to its unit test, from a controller to its unit test and from anywhere to anywhere, really, because we have that knowledge. I don't know what it does anymore, I just use it. My fingers and the F key are binding so there is some snippets are activated, but later I am going to be press Tab and others are going to be key bindings and I lose track. My fingers know stuff. They know more than I do. Top tip! A lot of people still don't know. If you want to learn about TextMate, press command, control T, because it might be that way on the video. Or going to the bundles menu as a top item and then you can start to figure out all the different things you've got based on wherever your cursor is. TextMate top tips for you, InfoQ!

   

9. The TextMate bundles are just one of your many projects. How does the average Australian handle tons of projects? Or how do you handle them?

You don't. You can't. It's just mathematically not possible. The talk we gave here was that I did some back, like of an envelope - I don't know if we actually have any envelops nor pens with which to write on the back of an envelope -, but metaphorically I wrote on the back of an envelope that I think I figured out I don't know how many real projects I have. It might be somewhere between 50 and 100 and that's three years. Let's just say 75, because it divides nicely by 3. That's 25 a year, and I plan, I try to live a life style whereby I can keep doing this for another 37 years. I don't know where it came from! One day I just said someone "I'm going to do this for another 40 years".

I kept saying "40 years" for several years, then I realized that otherwise I was never going to get to stop, so now I do subtract some time - so 37 to go. That's 40 years x 25, that's 1,000 projects that you potentially have and obviously that might just be Ruby, Javascript and Objective-C, that would be God knows what languages by the end. You got to acknowledge that you are not going to support all of them, so I just progressively and proactively don't support them very quickly. Just because I wrote them, it doesn't mean that I have to look after them. I thought I'd use them, then I'm not touching them. GitHub is a tremendous community tools for people being able to discover and take over maintainership, if they want. I make too many projects and I have to not maintain them.

   

10. You try to stay sane by simply dropping projects or do you tell people to go away if they ask you for enhancements? What do you do? Do you separate projects into important projects or ones that could die?

I'm not a machine. So... ChocTop - I don't know if you've talked about ChocTop - but it's a tool for packaging and bundling Cocoa apps into those pretty DMGs, so all you need to provide the background picture, some icons and the positions and it does everything. Wonderful tool! I originally wrote it because of a project we were going to relate to called CommitChap We never released CommitChap Technically, I don't need a packaging and deployment tool, if I've never packaged and deployed anything. Therefore it does become hard to maintain because I don't use it. Nonetheless, there have been occasions when people have written to me and said "It doesn't do this. Can it?" and I said "Here is the code, go for gold!" And there have been other times, when for no reason - I don't know, perhaps the name look good - I just felt like adding their feature.

I'm still human and I still maintained stuff mostly because at that point in time it seemed fun, but I don't have a "to do" list. The worst place anything can get is on my "to do" list. I don't know where the "to do" list is, I never look at it. So, it either gets done straight away or it never gets done. That's why I end up with so many projects - because I have that perspective. If I have an idea of what needs to be written - "Wouldn't it be new and shiny if we had this?" - I typically do it right away, because it's no point in having that thought. I would not come back to that thought, so it's a linked process.

   

11. Which projects do you actively maintain? Which are the top 5 projects that you still keep alive?

There is no list. I said that NewGem is my pet project - that's probably it. In terms of what I would say if I ever got to hang my hand on something to say I'm proud of this. There are probably others as well that I'm proud of. ChocTop I'm proud of, nobody uses it, I'm very proud of it though. If anyone ever finds it and uses it, the things that it saves you if you ever tried to figure out that custom DMG stuff, I'm very happy that it exists now. Everything else is not pet project, so therefore the only stuff I maintain is the stuff I come across on a daily basis. I won't maintain any TextMate bundles until I decide "I need this" in which case I'll add it and while I'm adding it, I'll then double check. Now I'm touching a project, so this is my implicit process. When I'm touching a project that's using it inside the code base, I'll go after GitHub and because there is a GitHub gem that can go gh home or gh network web.

I think the last one show up the commits that everyone's had. While I'm actually in that code, that's a good time. There is no cost, there is no transition cost to go and doing maintenance. I'm there! Now I'll go and have a look to see what else needs to be done, see if there is anything interesting and then you can come back to the command line and go 'gh pull name --merge' and I'll get their stuff. That's pretty much my low cost maintenance process. That's TextMate bundles, that's libraries, that's anything. Every now and again, for laughs and giggles, I will maintain something. I'll say "Well, today I'll go back to NewGem or rubigen or whatever" and I'll give it a bit of love, but very rare because I have kids and they are actually shiny interesting now and it's fun to play with them. They can talk, they acknowledge that I'm interesting, it's nice to have a fan base at home. My wife still doesn't understand a thing I do, but that's fine. No, I don't maintain anything officially, just as I touch it, it gets some love. That's my process - I make sure that if I touch, I do some love.

   

12. With that experience, what tip would you give to new aspiring open source commiters? If someone wants to create an open source project, what first steps should they do?

There is a lot to learn. Obviously, every different thing is different. Ruby Gem VS Javascript library VS Cocoa app, everything is very different. I know which one you want to talk about. Use NewGem or one of the other types of gem starting points and do what it tells you. Like Rails, it's a convention over configuration, so you don't care. You honestly don't care whether it's called lib, source, public, or whatever. That's where you put your code and it's probably a reason for it and until you have a good reason to change it, what do you care? Just do what they do and then you get the benefit of all the rake tasks to do packaging, bundling. But look inside other libraries, other gems and feel free to ask questions. It's part of my zero maintenance. That doesn't mean I'm zero help. I answer questions all the time for anyone that cares.

I don't call that maintenance, I call that community building and the more people are at it, knowing how to write gems, more people running actually writing unit tests for Java script, which I find a challenging concept because no one writes Java script unit tests. I say no one, but there is perhaps a point percentage of people who have ever written Javascript and write unit tests for it, and even smaller than a part of point do TDD on Javascript. That is production Javascript has to be written with tests, which I find challenging to comprehend. I help anyone, because the more people I help, the more people can then help and it goes on that way. Since no one reads blog posts any more, you got to go back to teaching people person by person.

   

13. NewGem is based on Hoe and add some other features. Could you go into that?

NewGem pre-dated my knowledge of Hoe's existence and I assume Hoe did exist at the time, but that's ok. The primary purpose of Hoe is to create a scaffold, like Rails. It's the Rails command equivalent to creating a scaffold for gems. Rails itself comes with all its rake tasks. Most people wouldn't even know where to look for in the source of Rails. It used to do that too when it came with some tasks, then I discovered Hoe, shortly after it. "Somebody else has done work and it's pretty good. Fine, I'm using this stuff". So, now what it does, it generates the Hoe DSL for specifying the spec, which is 3 lines long, versus - if you want to do the raw gemspec it's like - 5,000 lines long, so it's ridiculous. It automatically, intuitively figures stuff out so you look inside your Read Me file for the description, it will look inside a file called History.txt for the most recent history.

Then, there is a task, when it packages up the gem, there's a task called 'rake release version'. It's 'rake release version=1.2.3' It pulls all this information implicitly from places, packages it up your gemspec and puts it on RubyForge, which is the only place to put your Gems. Then you can do 'rake post news' and it will do a similar thing pulling in the description from the Read me, the most recent history information from the history file and then push that to the RubyForge news our RSS feed, which is a really good feed to subscribe to. A whole bunch of other tasks like 'rake multi' for MultiRuby tests that runs all the tests against all the versions of Ruby that you have installed and it's being maintained by very intelligent people over in Seattle - Ryan David and Eric Hodel and all the guys that work with them. In my perspective that's a great cheat, using Hoe. If you don't use NewGem, do use Hoe! Once you are inside the generated Gem, once you are working, then it's like Rails has a script/generate command and now you can generate things that are helpful like installing Cucumber.

If you want to do a C extension there is extconf and it generates the starting point for the C library and the test harness, so you can do test driven C extensions and lots of little things like that, lots of little generators. It has generators for creating generators. If you are creating a framework like Merb or Adhearsion or Rails, you create a new project - whatever your project name is - let's say Adhearsion. You go 'newgem adhearsion' and then you go into that adhesion directory and then you run the application generator. You go to script generator/application generator/adhearsion. That generates a command line app called Adhearsion or whatever you want to call it that will run a generator, like the Rails command. Then you can specify all the things, all the files and all the folders that you want everyone to get by default.

So, it's a simple way for your framework to come to life. Then you can write generators for within that project. All the things that will turn up under script/generate. You create those with a component generator generator. If you wanted to have a model generator, you create an Adhearsion project and inside there, and you want a model generator, then, back over in your code you'll run "script/generate component generator model" and then it will give you the stub for that and it's all test driven. I don't know if it makes any sense. If you want it, just ask me. Actually I did talk about this - RubyConf 2007 has a big long video where there was more tutorial and less entertainment than I would normally like and that has a very cool A-Team theme. If you just watch the RubyConf 2007 talk just to watch the A-Team intro, it's pretty cool. That's on www.confreaks.com, so just go to confreaks website and say I can delegate the rest of this conversation on how it works to that video.

   

14. We'll outsource that to confreaks.com. You say everybody should put their Gems on the RubyForge, why can't I put my Gem on GitHub?

Understand the irony of what I'm going to say. I going to tell you in a second, I'll repeat it that there is no need for your name to be prefixed to your Gem. Why? You are just not that special. The irony is, one of my first projects was called The Magic Models, it was actually called Dr Nic's Magic Models. I don't know what sort of person feels the need to put their name at the front, I did it because it was funny, back in 2006. That was one of my first projects, so that introduced me and that sort of thing. That was the irony - having previously actually explicitly put my name in the front, I'm now here to say that there is no need for that. Other than the simplicity aspect that we can talk about later, the only reason that you want to use the GitHub for releasing gems might be for patch level gems. Cucumber uses it for 0.3.11.6.

On RubyForge, the most recent one is 0.3.11, on GitHub it's 0.3.11.6 that gives someone access to that level of download. The other reason might be if you were a forker and you want people to be able to access yours. The rest should go on RubyForge. Aslak, I love you, dude, I cannot spell your name! He pushes his gems to RubyForge. There is an example of the problem. I know the thing's called Cucumber, what if gem install cucumber doesn't work? What have I got to do? I got to go to GitHub search and look for all these 7 different people sharing the gem. With things like Hoe it's just not that hard to push stuff to RubyForge. There is no need. But I'm wrong! I can say this, I can have my conservative stance on this issue. GetHub announced they are serving more gems than RubyForge, so I guess RubyForge is losing rapidly, but I think that's just because GitHub makes it so easy. RubyForge is probably never going to compete with that, because it's maintained by one guy and that's hard.

   

15. Finally, a question that every interviewee gets. What about 1.9?

I love 1.9. My production level contact with 1.9 is actually through MacRuby. I've never written code that is 1.9 only, I still live and breathe 1.8.6 because that's what I get in OSX, though in Snow Leopard 1.8.7 is bundled, so we have already started the coming in contact with some stuff. One of my developers has got Snow Leopard installed, he added some code that didn't work on my machine because it was a variation that didn't work backward compatibly. No one at Mocra has 1.9 installed as their primary Ruby. Hampton Catlin just pinged me and said "Can you help me fix this one bug" I'll have a better look at that. I still follow the zero maintenance thing; I can't fix things until I care about them. I probably at some stage I need to move out of the 1.9 just so I start to care.

I probably need things start breaking on me and that's probably anyway what's going to happen. A lot of the new stuff looks really good. I've seen it through MacRuby the new syntax for methods, wit the Javascript like Hash syntax and I love it. Who doesn't want the speed? I do need to take some responsibility and actually move to ahead of the curve and I might get round to that one day. It's looking good. The community is moving forward and there are more and more articles lately about how to move to 1.9. 1.9 is the precursor to 2. We can't get a 2 until everyone's on 1.9. Look he's hoping that we will move in by the end of the year and we can get a Christmas present at Christmas time and that's when the Ruby release is coming out. Who doesn't want Ruby 2.0?

   

16. If you are not using Ruby 1.9, how do you ensure that it will work on other versions?

It would be great if it just got bugs and people would tell you and then you couldn't reproduce the bugs and would look silly. There is a gem called dentist that you install and that's the gem that includes all autotest, which continuously runs your tests and only reruns failing ones and also includes an app called MultiRuby. MultiRuby allows you, in a nice happy place inside your personal folder to install the edge or the most recent releases of every major version. The most recent releases of every major version, so you get the latest 1.8.6, the latest 1.9 and puts those there and then it allows you to run some Ruby commands against all of them. It's a little bit like using capistrano to run something on multiple machines, but now you are running something against multiple versions of Ruby to see what happens. You could just do print, you could run it.

All the options you would otherwise pass to Ruby you could pass to MultiRuby, because it just passes them through. You could just get it to print something out or, more interestingly, inside a project, you get it to run all the test cases, so you say "For every version of Ruby run all my test cases". What happens? That enables you to safely test your code. You need obviously tests in your app, but hopefully everyone's running tests. That's how I live relatively comfortably figuring out that my code would work or not work.

Oct 21, 2009

BT