[Editor's note: some of the websites mentioned in the answer below may offend some readers] I'm a Ruby developer based out of Toronto and I work for Avid Life Media. We own a bunch of dating sites, the most notable being www.establishmen.com . We handle massive load, lots of users from all over the world. We've overcome a lot of technical challenges and we've arrived at a best of breed platform that we built in Merb with an Erlang backend that we use for driving our most recently developed sites like www.establishedmen.com and a whole bunch of sites that we are working on. It's been a really interesting experience, taking the learnings from all of our older sites, like www.hotornot.com and www.ashleymadison.com and just rolling that into a site built using the absolute newest technology. Ruby, Merb, Erlang have really enabled us to throw something together that's really great in very little time that has amazing performance from a development standpoint and also from the user's perspective. It's an exciting time.
Yes, that's correct. At this point, we are more concerned with getting the platform totally feature complete and stable. We are not about to go and take all of our sites and collapse all the PHP, but ultimately, the hope would be that we can have one platform that can handle everything. Given our rate of development, the speed of development that we have with all of our modern tools, I expect us to get there relatively quickly, so that's exciting. But, at this point, we're developing a series of sites, each more difficult, more complex forcing us to add more and more functionality to the platform in order to prove the platform under very different conditions and also to allow us to incrementally build features into it. It's definitely a lot of fun, there is a lot of challenge obviously. We are trying to keep everything abstract and generic, but also be able to be very specific for each site.
We use what's become like this standard kind of stack with the JQuery in the front and Haml and SASS for handling all of our display and then we use Merb and DataMapper for the site itself, but, in order to keep things snappy, we have Erlang backend, using RabbitMQ. Basically, anything that can be taken asynchronously we fire off to Erlang and then we process it like that. From the user's perspective, they very quickly see their page, but behind the scenes, things are still processing. When somebody logs in, immediately we start doing things in the background, in order to proactively get things ready for them or we are off finding out who they should be looking at next or trying to prepopulate a list of potential search targets and stuff like that. There are certainly a lot of needs coming on behind the scenes, but Merb is the vast majority of the logic.
4. What's your experience with Erlang?
I came across Erlang a few years ago, when I was working for the University Help Network. I started looking into it as a possible ETL (extract, transform and load) language to take data from all different hospital systems and then transform it all together and then load it into a data warehouse or something like that. Everything right now is either being done in custom proprietary languages that are functional languages or people are just munging it together using whatever language they happen to know. I thought Erlang would be a really good language to do that and then out of that, I started to find that it was missing a lot of the things a lot of things that we, as Rubyists are enjoying.
There is no inflection libraries you can pluralize or singularize, strings or anything like that, so I got into taking some of the stuff in Ruby, importing it to Erlang, which is - I think - really cool, because the vast majority of Erlang developers are doing telecom. So, they are never going to write a pluralization library or an inflection library, but, if Erlang is going to get general acceptance, it needs to have this stuff and this stuff that made Ruby great needs to be in Erlang. Having said that, professionally, we only really use Erlang to do the messaging and background processing and that kind of thing. The hope is that they keep adding to the libraries and the functionality of the language that eventually will be in a place where it's comfortable to write an entire web app using it.
It's neat to see in the Ruby community that people are getting really excited about Erlang. At RubyFringe last year, a whole bunch of us started talking about how we're all playing with Erlang individually and a lot of people feel it may be the next big thing for Rubyists. So, we started the Toronto Erlang Group T.Erl and it's mostly Rubyists that get together and talk about Erlang and we have very few people who are actually being paid to do Erlang day in day out, but I think over time we are seeing slowly more people getting paid to do Erlang and more and more Rubyists that are able to actually find uses for Erlang at work.
We use RabbitMQ - it's a message queue. More or less we fire off messages onto the queue from Ruby and then, behind the scenes they get processed - that's in some cases. The other way in which we interact is using XMPP, which is a Jabber protocol - depending on the site, or more so, depending on the usage. If we are looking at doing something like a notification or something that we are trying to be close to realtime, then we'll basically treat it like a chat protocol. We have custom chat packets that will fire off to ejabberd, which is the Erlang Jabber daemon and then that will basically distribute it to all the people who need to know that a gift has been sent, or that a new private message has arrived for them.
That's absolutely a good question. In the past I used Rails exclusively and until you use Merb, you never realize what you are missing. In hindsight Rails is pretty inconsistent. The API doesn't feel like this well thought concise API in the way that Merb does and I understand that's because Merb had the benefit of learning from Rails and not needing to be backwards compatible or anything like that. At this point, it's hard to imagine how those are going to be reconciled because the APIs are different enough that as a Merb developer and a Rails developer I find that can't just switch back and forth.
If I write Merb code all day and then I go home, I can't easily step back into it, because it's just close enough and my guess is that, since it is a larger user base of Rails people, that Rails 3.0 is going to be more railsy than merby. From our perspective, we'll either have to bite the bullet at some point and convert everything over to Rails 3.0 or if we find that there is enough support to keep Merb alive, then, we'll stick with it, but the sense I'm getting is that most of the people using Merb are going to have to move to Rails 3.0.
7. There seem to be plans to provide tools to move over from Merb to Rails. Have you seen any of these?
No, I haven't tried any of them. I'm a little skeptical that there will be anything like fire and forget. I don't think it will be like you run something in it. There were tools like even moving from Rails 1.0 to Rails 2.0 to identify anything in their code, like parameters and add parameters and things like that. The changes in the API would just to a text search and identify where you are using them and then you would have to go off and fix it yourself. I'm sure that it would be stuff to help, but also it doesn't change the fact that it's a big effort. Even upgrading a minor revision of Rails is a big deal.
In the past, at MDL (Medical Decision Logic) where I worked, our policy was that if it was a feature that we wanted to use in a newer version of Rails, that would be our justification for moving forward, but we wouldn't upgrade just for the sake of upgrading, because in many cases, the sites are working perfectly and everyone is really happy with it. We are not being slowed down by any inconsistencies with that version, it's really hard to justify spending the time. Active Resource was something that forced us. We had an application at MDL for clinical research, for cancer research and it was something like 55-60,000 lines of Ruby code - that's large for a Rails application. We dragged our feet on upgrading to a new version of Rails and also it was Active Resource that made us move, because we needed to connect disparate systems and it seemed the best way to do it.
8. You created that new version of ActiveResource?
Yes, we went through all the trouble of upgrading this massive application to a new version of Rails, and then we go to use ActiveResource and we find that it really hold up to all the claims, like literally you can fire it up and look at all the documentation and the claims are incorrect. It's supposed to have the same API as active record, but it has a very small fraction of it and even in some cases, where the same method is implemented, it's implemented with different semantics and behaves differently. It's just close enough so that not only will you find things you expect to be able to do you can't and you often times find that you can do it and it's suddenly wrong behind the scenes.
What we did was we made HyperactiveResource that extends Active Resource and ten basically modifies it's behavior and adds in new behavior so that it supports things like associations so that an active resource can belong to another active resource or it can belong to a model so that you can have these situations where you have active record and active resource coexisting and referencing each other. I'd say that given you are going to write mapping you are using ActiveResource raw, you really need to look at Hyperactive Resource, because there is a world of pain waiting for you otherwise.
9. You mentioned you were working in the medical field. What were you doing there?
I started working for a series of hospitals in Toronto and a hospital network that more or less most of our applications were internal and they were to support various different functions within the hospital. When I first stared to learn Ruby and Rails first came on the scene, was long before version 1.0, we saw an opportunity to use it in a very experimental fashion, where we had 5 or 6 users, they were all internal, they were all very open to downtime, any of the weird issues that we expected to run into.
So, we introduced Rails for chart tracking and various other, like health records. Over time, it grew, because it was so easy for us to add new features. More and more of the functions that happened behind the scenes were tracking where patient chart was gone or a correspondence between doctors and so on became done in our Rails application. Ultimately, I moved from that to doing a medical research software, where we assist doctors in running large clinical trials, where you have thousands of patients say taking a new cancer drug and you need to track their progress and their lab results and all that kind of stuff.
10. Is it basically database applications?
Not strictly speaking. I guess one of the big things that is part of clinical research management system is timetabling and reconciling the fact that you have a drug that can't be taken within 2 days of a particular lab result because it will skew the results or because this drug conflicts with that. When you look at a patient that may be on 20 or 30 drugs in all these different labs, it's actually a real problem finding what days they should take which drugs and when the lab should be done.
We allow them to enter metadata about this study and it actually uses with them invoke Prolog from within Ruby and Prolog is good at doing constraint solving, so effectively the Ruby would define all these constraints and then the Prolog would reconcile that into the ideal time for everything to occur and that would translate into a calendar for a patient. They can say "OK, I know I have to come in on Monday or Tuesday for this drug and if I go any later, for some reason I'll be conflicting with something else."
11. Does that mean that you create a Prolog code from Ruby or Prolog input data?
Yes, we create a Prolog input data. Ultimately, there is not that much code in Prolog. It was because the language is so well suited to the problem, it's neat. At RubyFringe, there was a lot of discussion about polyglotting and personally I found that that's the way of life. We use Erlang where we need to, we use Ruby, we have PHP in the past Prolog, but in that case, the Rails application generated the Prolog inputs and we basically fired off to a running Prolog process and then it would get the results back and parse it out.