NServiceBus 3.0 was recently released and InfoQ talked to Udi Dahan, the creator of NServiceBus, about the new features of the latest release. One of which is the ability to integrate with RavenDb, a fairly new document data store written in .NET and created by Oren Eini (Ayende Rahien). InfoQ also took the opportunity to ask Ayende some questions about the NServiceBus integration and what that means for his own Rhino Service Bus.
InfoQ to Udi: How would you describe NServiceBus?
Udi: NServiceBus is the most reliable platform for building distributed systems in .NET. For data that you can’t afford to lose when servers crash, when databases deadlock, or even when a system upgrade causes a deserialization failure – NServiceBus goes above and beyond the standard WCF, outperforms BizTalk, and can be run both on-premise and, now with version 3.0, in the cloud as well.
For developers familiar with other messaging technologies like ActiveMQ, RabbitMQ, and ZeroMQ, NServiceBus handles the million little things that happen at higher levels like multi-threading, transaction management, message definition, serialization, and versioning, dependency injection, impersonation, encryption, as well as capabilities usually associated with business process engines – all in one developer-friendly lightweight package.
InfoQ to Udi: If you get to choose 3 features to highlight for NServiceBus 3.0, which features would those be and why?
Udi: I’d have to say that my favorite feature in version 3.0 is deferred messaging – the ability to send messages to yourself in the future. Simple in its implementation but profound in its impact, this feature can help developers handle state in a more scalable and maintainable fashion in long-running processes. For example, let’s say you want to implement a business process to model “buyer’s remorse”, the fact that a large percentage of cancelled orders occur within a short time of the purchase. When processing a SubmitOrder message, instead of just doing the regular logic and saving that state in a database, you could have that same message sent back to you in 2 hours, within which time you may have received a CancelOrder message for that same order ID. If the cancellation comes within the 2 hour window, you could close the process down without once talking to a database. If the cancellation comes too late, you could come back to the customer telling them that they now have to pay some kind of fee for cancellation – they don’t get a full refund, potentially keeping the state flowing through messages for even longer. It is the modeling of these kinds of complex and ever-changing processes on top of a fully distributed and scalable messaging topology that really shows the power of NServiceBus.
The second area I’d mention is monitoring. We’ve always taken a very lean approach to monitoring as we’ve seen too many customers drown in floods of data without being able to find meaning in it all. In this version, we’ve been able to distill the most important metrics of message-based systems down to a single counter – SLA violation countdown. Developers set the Service Level Agreement (SLA) for an endpoint (say 10 seconds), and NServiceBus watches the trending of message wait times in the queue as well as processing time and calculates how soon the endpoint will end up violating the SLA (assuming load continues along the same trend). Administrators and business stakeholders can now know what is the status of each part of the system in business terms – not just percent CPU utilization and other low-level metrics. When taken together with the load balancing capabilities in NServiceBus, this ultimately tells administrators which part of the system needs to be scaled out and how quickly that needs to be done. I’m not sure everybody in the IT organization is ready for this level of transparency :-)
Another very important (although less visible) stride we made with version 3.0 was around extensibility. While we’ve always had a pluggable architecture, in the past year we’ve seen customers want to take much deeper control over what happens “in the pipes”. Enabling features like property-level and full message encryption and compression, override-able routing, and integrating 3rd party unit-of-work and transaction management, many of our most technically advanced customers chose to use NServiceBus 3.0 in production even before it was in beta to get these features. It’s also been really exciting watching our community take NServiceBus and integrate it with new frameworks like SignalR in just a few lines of code (as shown here) and then show others how to build things like full-push auction sites (as described here). Our objective has always been: do anything you can do to help others build cool stuff faster.
InfoQ to Udi: Have there been any simplifications to how NServiceBus is deployed and/or will there be in coming versions?
Udi: There have been massive improvements in this area. For developers using all of NServiceBus’ advanced features like durable timeout management, multi-site + cross-protocol communication, and clustered message processing, we’ve introduced a powerful convention-over-configuration model that reduces the already small configuration by almost an order of magnitude.
We’ve also focused more strongly on the security side of things so that elevated privileges would be required only at install time leaving a locked-down production environment. This was quite an interesting balancing act – keeping developer productivity at a maximum when you’re building a system so that any queues you need are created on the fly, while doing the exact opposite in the production environment.
Also, our choice to use RavenDB (an open-source 2nd generation document database) in version 3.0 gave much greater production-time visibility into things like the dynamic routing of NServiceBus, providing administrators much more control of their deployment environment.
InfoQ to Udi: This release comes with integrated support for RavenDb at no additional licensing cost. How and why did this deal come about?
Udi: NServiceBus requires persistent storage for things like remembering which endpoints are subscribed to each other (in case there are power outages) as well as retaining the state of long-running processes. In previous versions, developers had to provide a connection to a database for that. In many organizations, that required getting the approval of a DBA (which didn’t always come easily). The introduction of RavenDB into NServiceBus allowed developers to set up a complete solution entirely on their own.
One of the primary reasons we chose RavenDB was its support for distributed transactions – a feature that means that developers don’t have to implement de-duplication logic to handle “at least once” messaging – instead they get the full “once and only once” model NServiceBus has always provided them. The responsiveness of the RavenDB development team was also a big factor.
With regards to licensing, we really didn’t want our customers to have to sort out licensing issues with multiple vendors. So what we did was sort out the financials between our organizations to make things as simple as possible for our customers. The short story is that a portion of each sale of NServiceBus now goes to RavenDB – everybody wins that way.
InfoQ to Ayende: What was your motivation for integrating RavenDb with NServiceBus and why the free licensing model?
Ayende: I can't speak from NServiceBus point of view about the RavenDB integration, but from our point of view, this have several major advantages. To start with, this gives us entry into many enterprise, with NServiceBus as the release vehicle. It gives IT organizations and potential customers the reassurance that RavenDB has been in production use by many customers and have withstood the test of fire under heavy load and demanding environments.
In addition to that, of course, we are exposing RavenDB to NServiceBus users, and thereby promote the core product.
The step from having NServiceBus internally use RavenDB and having your application use RavenDB is a small and a natural one, and we are confident that users will discover and be very happy with the abilities, features and API that RavenDB gives them and their applications.
With RavenDB, things that are extremely hard to do in relational databases, become oh so natural and easy.
InfoQ to Ayende: You have previously written a service bus for .NET yourself, namely the Rhino Service Bus. Should users of Rhino Service Bus now reconsider and move to NServiceBus?
Ayende: I built Rhino Service Bus around 2008. I built it mostly because I wasn't happy with the state of the other service buses at the time. I have had different concerns and direction when building my service bus, but that was 4 years ago. In that time, I think that NServiceBus made great strides in becoming an easier to use product and having a much better out of the box development story. If I was starting out with service buses today, I strongly doubt that I would be building my own.
InfoQ to Udi: How’s the migration story from earlier (more recent) versions of NServiceBus to 3.0? Especially considering the story where Saga persistence is to be migrated to RavenDb for a system already in production.
Udi: We’ve always gone to great lengths to provide backwards compatibility with NServiceBus. We allow you to run subscribers built on a newer version of NServiceBus with publishers running a previous version, and vice-versa. We’ve provided migration tools for taking timeouts being managed in the previous version and bringing them to the current version – all with zero downtime.
When talking about long-running process state (called “sagas” in NServiceBus), we recommend a deployment of the new version (vCurrent) alongside the previous version (vLast) where all new long-running processes will be started in the vCurrent endpoint and old processes will “drain” out of vLast. This is done by utilizing the interception capabilities built in to the NServiceBus pipeline allowing us to prevent new processes from being started in the vLast endpoint.
Our customers hold us to a very high standard when it comes to zero downtime upgrades.
InfoQ to Udi: What can we expect to see in coming versions of NServiceBus?
Udi: In addition to improvements to our hosting environment enabling running multiple endpoints in a single process giving a much more manageable topology in large scale integration scenarios, we’ll be looking to scale NServiceBus down into a single process for developers looking for an in-memory message-passing programming model (like F# and Erlang) but with the familiar imperative style of regular C#.
Also, we released a preliminary drop of some modeling tools for NServiceBus last year and are investing very heavily in that area this year. We’re looking at pulling in some runtime profiling to create, what we’re calling, NSB Studio – a fully integrated modeling, development, and profiling environment for message-based and service-oriented systems.
Very exciting times ahead!
