ServiceStack is an Opensource .NET and Mono REST Web Services framework. InfoQ had the opportunity to get some insights from Demis Bellot about the project. In Part 2 of this two-part interview, we learn more about ServiceStack features and the role of Microsoft and Mono in open source .NET. You can find Part 1 of this interview here.
InfoQ: What exactly is a message-based Web service?
Demis: In essence, a message-based service is one that passes messages to facilitate its communication. A good metaphor for illustrating the differences between a RPC method and a message-based API can be seen in Smalltalk or Objective-C's message dispatch mechanism vs a normal static C method call. Method invocations are coupled to instance they are invoked on, in a message-based system the request is captured in the message and sent to a receiver. The receiver doesn't have to handle the message as it can optionally delegate the request to an alternative receiver to process instead.
Message-based design is enabled in ServiceStack by capturing the Services Request Query into a Request DTO that's completely de-coupled from any one implementation. You can think of making a ServiceStack request as a Smalltalk runtime method dispatch at a Macro scale, where the ServiceStack host is the Receiver, the HTTP Verb is the selector and Request DTO is the message.
It doesn't matter on which of the endpoints the Request is sent to as the request can be populated with any combination of PathInfo, QueryString and Request Body. After the Request binding, the request travels through all user-defined filters and pre-processors for inspection where it can optionally be handled before it is able to reach the service implementation. Once the request reaches the service it invokes the best matching selector, by default it will look for a method with the same name as a HTTP Verb, if it doesn't exist it falls back to using a catch-all 'Any' method that can be used to handle the request on any endpoint or route in any format. Even when inside the Service implementation it is able to further delegate the request to an alternative service or easily proxy the request to a remote sharded instance if it needs to.
Conceptually in ServiceStack you're just sending a message to a ServiceStack instance, the client is not concerned with what ultimately handles it, only that a response is returned for reply requests or that the request is successfully accepted for oneway messages. In a RPC API you're conceptually invoking a remote method which has the request tightly coupled to its remote implementation method signature.
There are many natural benefits gained when adopting a message-based design they offer better resilience, flexibility, versionability then their RPC cousins. An example of one of the benefits possible is when you send a request to its one-way endpoint, if the ServiceStack instance has a MQ Host configured, the request is automatically deferred to the configured MQ Broker and processed in the background. So even if the ServiceStack Host goes down none of the pending messages are lost and are automatically processed the next time the Host starts up. This is the type of behaviour that is enabled for free in ServiceStack. When no MQ host is enabled the request is just processed normally, i.e. synchronously by a HTTP web worker.
Most of the benefits of message-based designs are gained over time as you're developing and evolving your existing services and adding support for more clients. One immediate benefit is being able to provide an end-to-end typed API without the use of code-gen. This is impossible to achieve without a message-based design which ensures the essence of your Service Contract is captured in re-usable DTOs. Being able to share your server DTOs you defined your web services with on the client completely by-pass the normal development workflow required in re-generating your clients proxies from your services interim WSDL/XSD schemas.
Typed clients are the underpinnings for Native SDK's which provide the most value to end-users of your service as they reduce the most of the burden required in order to consume your API. This approach is popular for companies that really, really want you to use their APIs, i.e. where their businesses success depends on its popular use. This is the preferred approach taken by Amazon EC2, Google App Engine, Azure, Facebook, Ebay, Stripe, Braintree, etc.
More importantly, message-based designs encourage the design of coarse-grained and more re-usable services. By contrast RPC method signatures are generally designed to serve a single-purpose, i.e. Rather than adding more RPC methods for every client requirement (which introduces a new external endpoint each time), message-based designs instead encourages enhancing existing services with extra functionality since they can be added without friction. This additionally has the benefit of providing instant utility to existing clients already consuming existing services, since they can easily access the extra features without introducing a new code-path to call a new external endpoint.
This is an especially important approach to take whenever implementing service-intensive systems like SOA platforms, as services routinely end up out-living and serving more clients that the original client that consume them, so it's important not to have your service APIs driven by ad hoc client-specific requirements. It's more useful to think about designing APIs from the system's perspective with the goal of exposing the underlying systems capabilities in a generically re-usable API. This is the main reason why I now only ever adopt message-based designs for all my services endpoints as Coarse-grained APIs naturally encourage the design of more re-usable and feature-rich APIs.
Benefits of message-based designs are already well-known to developers of leading distributed frameworks who have adopted message-based designs in leading platforms, e.g: Google's Protocol Buffers, Amazon's Web Services platform, Erlang processes, F# mailboxes, Scala's Actors, Go's Channels, Dart's Isolates and Clojure's agents, etc.
InfoQ: You recently introduced a razor engine making ServiceStack a more complete web framework than just a web services framework - what was the motivation behind that?
Demis: We've always wanted to have a good HTML story with ServiceStack. From the point of view of a Services Framework, HTML is just another Content-Type although it does have the special property of being supported in all browsers making it the only format capable of rendering a universal UI that's supported on most computing devices. Since ServiceStack could be easily hosted along-side any ASP.NET or MVC Web Framework there wasn't an immediate need to support HTML as we were able to satisfy Single Page Apps by serving static assets and whenever we needed to dynamically generate content we could just leverage the hosting Web Framework. Although this worked well enough, we weren't completely satisfied. Our choices were to use WebForms which like WCF, we considered to be a leaky over-architected server-side abstraction or MVC which remains a comprehensive framework but due to its increasing complexity and additional moving parts added on each release, was frustrating trying to get working in Mono.
The eventual motivation for providing our own HTML Story was largely due to our ambitions of providing full-support for Mono so we could run on each of the exciting platforms Mono supports. That and our Self-Hosted HttpListener host was becoming more popular but being held back because it wasn't able generate dynamic HTML views as both WebForms and MVC required an ASP.NET host. So we decided we needed to provide support for an integrated HTML View Engine. Unfortunately the only View Engines available at the time was WebForms which we didn't like very much or Razor which was beautiful, but closed-source at the time and poorly documented. So we decided instead to create our own View Engine based on our 2 favourite markups: Markdown and Razor. ServiceStack is nicely archictected so we're able easily plug in new Content-Types without disrupting the other formats and endpoints. Two weeks of development later, Markdown Razor was born - a nice blend of Markdown (the ideal markup for content) and Razor - providing its dynamic functionality.
We were happy with the result as we we're now able to build rich ajax-powered, documentation-heavy websites like ServiceStack Docs using just ServiceStack and Markdown Razor. Markdown has the distinct advantage of being natively supported in GitHub, which let us import our existing GitHub pages as-is as well as live-edit and preview content directly from our public GitHub repository. This still wasn't a complete solution as although Markdown is perfect for content it's not ideal for precise HTML layout, the most ideal View Engine for this task that's familiar to most .NET developers was Razor, so once it became Open Source we jumped at the chance to integrate it. With the help of our good friends from NancyFx(another good alternative Web Framework) who told us how to get Razor support so that it works with VS.NET intell-sense, we were able to add Razor View Engine support to ServiceStack that now works equally well on all supported hosts and platforms.
But we didn't stop there, now that we had a complete stack and total control over the HTML rendering, we're able to introduce some of our own unique functionality that can be seen on our showcase site: razor.servicestack.net. Thanks to our virtual file system we're able to serve Razor views from sources other than the file-system, like embedded inside a .NET dll. Using the Self-Host, this lets you package your website, complete with Razor views inside a managed .NET .exe. Another unique feature we've introduced was being able to embed Partial views of other View Engines, this enables the ideal scenario of creating the structure of your page with Razor and HTML and maintain the Content of your page in Markdown which can easily be embedded as a Partial view.
We also think we've provided more appealing alternatives to some of MVC's features like our Cascading Layout templates as a simpler more intuitive way of maintaining multiple website layouts than MVC Areas and ServiceStack Bundler which is a faster, simpler and more flexible cross-platform node.js-based alternative to MVC Web Optimization.
InfoQ: Are there any scenarios where you think WCF/Web API/MVC might be better suited than ServiceStack?
Demis: MVC is a comprehensive Web Framework which will continue to be more suitable for large write-heavy server-generated websites. ServiceStack is more focused towards providing a great experience for Web Applications with a heavy services component such as Single Page Apps like the kind built on the leading JavaScript Frameworks: Backbone.jsand AngularJS and exciting up-coming initiatives like Dart's WebComponents. We also expect our integrated Markdown and Razor View Engines to be attractive for hosting content and documentation heavy sites.
If you believe in the value of following REST and HATEOAS Contraints in developing server-driven systems, you're going to want to use WebApi and join the developing culture in that community. If you would like maximum utility for your Services and be hosted in alternative endpoints like SOAP, MQ's (and upcoming TCP support), ServiceStack is the better choice.
If you're an MVP or Microsoft Gold Partner you're going to naturally want to stick with the MVC and Web Api technology stacks as Microsoft will have you covered with Sql Server, AppFabric all the way to Azure. We see more value and opportunity in supporting the better scaling and performing alternative platforms which is primarily where we'll be focusing our efforts on, providing a great story for Amazon's EC2 and Linux-only clouds like Google Compute Engine as well as the alternative RDBMS solutions OrmLite supports and high-performance NoSQL solutions with a continued heavy investment around Redis and integrated adapters for the Cloud-hosted data stores.
InfoQ: Microsoft has collaborated with open source projects in the past (JQuery, NuGet for e.g.) and MS folks like Scott Hanselman seem quite open about adopting good open source solutions into the MS default stack wherever possible - do you foresee any such collaborations that could benefit the .NET community in general?
Demis: As project lead who's led ServiceStack as an Open Source run project for 4 years, building a thriving Open Source .NET community is something I care deeply about, though I'm not sure I'd be willing to give Microsoft gold stars for their collaboration with existing Open Source projects just yet - especially .NET ones. Until now it seems they've only adopted Open Source libraries after their competing attempts have failed. E.g. Most JavaScript developers were already ignoring ASP.NET AJAX javascript libraries in favor of jQuery long before it was officially adopted.
When it was launched, the NuGet project received criticism for their lack of collaboration with other pre-existing Open Source solutions. But overall I think NuGet has turned out to be Microsoft's most helpful contribution in reducing the effort to use Open Source libraries to date as it provides a window in VS.NET for developers to be able to easily reference external packages. We've definitely seen a greater adoption of ServiceStack's libraries since publishing to NuGet with more than 200k downloads in the last 18 months.
As for Open Source .NET libraries, Microsoft have only just adopted their first Open Source .NET library with JSON.NET earlier this year when it was bundled with Web Api. Like any company it makes sense for them to adopt superior Open Source alternatives when they exist, as all their previous JSON Serializer attempts to date haven't matched the features of JSON.NET or the performance of ServiceStack's Serializers. Although this provides a great boost for a single library like JSON.NET which has seen an order of magnitude more downloads than all other JSON Serializers combined, it doesn't have a halo-effect in helping the adoption of any other library - it actually has a negative impact. The power of defaults means .NET developers need a good reason to deviate and adopt an alternative. We're lucky to have a reason as .NET's fastest JSON Serializer making it popular with performance conscience companies like StackOverflow who've adopted it for their JSON needs. But we're a distant 2nd place with only 1/14 of JSON.NET's market share, the next Open Source JSON Serializer in line only has 1/110 of its market share. As serialization performance is critical in developing high-performance services we view our serializers as a core component and we're committed to maintaining the best and fastest serializers we can.
Other than JSON.NET I believe DotNetOpenAuth is the only other Open Source .NET library they've adopted since which again makes sense for any company to avoid un-necessarily re-inventing the wheel. So whilst they now seem to be open to adopting superior external Open Source libraries when it's in their interest to, although this change in behavior itself hasn't provided any noticable benefits to the community.
But thanks to their change in business models, Microsoft has begun to open up a lot more with most of their libraries and frameworks around Windows Azure being Open Sourced. This is great as it lowers the barrier for everyone in adopting new software and provides the direct benefit in reducing the burden off the Mono Community who've had to previously expend efforts in duplicating functionality and are now able to re-use Microsoft's Open Source released software as-is and contribute back patches to improve its support of Mono. F# is a great example of this which as it's completely open sourced has surprisingly great support for Mono, in-fact all my forays into fsharp were done in Mono/OSX with Sublime.Text. One of Microsoft's Open Source releases has even created an active community in SignalR which has shot to the top of GitHub C#/.NET Charts and who's SignalR-poweredJabbR.net chat has become a popular hangout for .NET developers.
Areas that haven't been as fortunate were alternative Open Source frameworks that were new to adopt approaches popularized on other platforms, filling the void before Microsoft introduces a competing solution. E.g. ASP.NET MVC was created years after the MVC MonoRail Framework which has since deflated that community. Their latest attempt at an ORM Data Access Layer in Entity Framework has negatively impacted the earlier prominent ORM NHibernate's once thriving community as well. Despite being several times slower than every other Open Source .NET ORM, EF has succeeded in attracting more downloads than all other ORMs combined. Likewise they seem to be in a perpetual state of creating and releasing new service frameworks, of which many have come and gone including .asmx, CSF, WCF, WCF/REST, WSE, WCF DataServices, WCF RIA Services and now Web Api. During the years between releases many alternative Open Source service frameworks have provided more than capable alternatives, that would've been more suitable options for various use-cases if it weren't for their existence remaining largely unknown to the wider .NET community.
The .NET platform is like no other; Microsoft has PR channels, Evangelists, MVP Award programs and control over VS.NET that commands a strong influence over .NET mindshare where they're seen as the authoritative voice for the .NET ecosystem to most developers. Historically they've only used their influence to validate their own libraries and frameworks which has contributed to many .NET companies being reluctant to deviate from Microsoft's prescribed technology stacks and explore alternative solutions. We've experienced this ourselves on a number of occasions where developers have wanted to use ServiceStack but were unable to convince their company to adopt an alternative framework when a Microsoft solution exists, despite presenting favourable examples and benchmarks. I expect many other alternative Open Source libraries and frameworks to have sufferred similar fates.
This environment makes it hard for Open Source communities to thrive and C#/.NET has seen its relative popularity decline,recently slipping off GitHub's top 10 languages list (the home of Open Source) where there currently exists sparingly few Open Source .NET projects that have rallied this trend and have been able to form communities around their independent technologies. The Mono project is by far the brightest spark with the most healthy and talented developer community that is providing a lot of value in enabling .NET applications to run on popular alternative platforms like iOS, Android, Linux and OSX. For many projects, supporting Mono is providing the best opportunity to expand their reach which has been one of our primary motivations for always ensuring first-class support of ServiceStack on Mono. Behind Mono, NancyFx and ServiceStack are doing their best to grow the Open Source .NET community attracting more than 200 unique contributors between them - many of whom were first-time contributors to Open Source. MonoGame and RavenDB are two notable others also gaining popularity. We're definitely proud to be amongst the top GitHub projects encouraging Open Source in .NET, but we have a desire to see more engaging .NET communities grow and more developers encouraged to pursue the Open Source development model.
Raising awareness for alternative libraries and frameworks is where I believe some collaboration from Microsoft would provide the most benefits. This is a situation where trying to grow a bigger Open Source .NET community would provide more benefits then trying to take a bigger slice. Up to this point Scott Hanselman has probably single-handedly provided the most awareness by covering various Open Source libraries and frameworks on his popular personal blog. After Scott, Glenn Block has helped in promoting various frameworks from his very active @gblock twitter account. More efforts from Microsoft in raising awareness and validating the various projects will amongst other things encourage more .NET developers to get into Open Source and give existing developers enough momentum to continue to enhance their own projects - both are vital ingredients in sustaining a thriving Open Source community.
Being a profit-motivated company Microsoft would need some financial incentives to encourage promoting alternative libraries and frameworks. I would hope that someone could build a business case around how building a thriving Open Source .NET community would encourage more developers to choose .NET which in-turns creates a larger potential customer base for their Windows server tools and Azure services. Even if they only promote alternative .NET frameworks in the context of running on Windows Azure like they do with node.js,python and java) that would still be an improvement. To fully thrive Open Source .NET would need buy-in from Microsoft where increasing Open Source .NET communities is seen as a desirable metric to strive for, at which point they would recognize Open Source contributions in their MVP award programs and promote them in their PR channels. Failing any future Microsoft involvement, I believe the Mono project holds the best hope for encouraging more .NET developers to join Open Source.
InfoQ: You made a comment recently on one of the forums - "I'm hoping next year to be even better, as I have a few things planned that should pull us away from the other frameworks" - would you elaborate what features you have in mind for the future?
Demis: Well, I was being intentionally vague in the forums as we want to save some features for surprise announcements as we believe we have an opportunity to deliver some unique and news worthy products and features. But the general theme is to remain a value-focused services framework and implement the reamining useful bits in WCF to provde a more appealing upgrade path to ServiceStack. Some publically known features we've mentioned we have planned for next year, include:
-
Merging the Async branch and its async pipeline
-
Create new fast Async TCP Endpoints
-
Enable fast, native adapters for node.js and Dart server processes
-
-
Enable integration with more MQ Endpoints (i.e. RabbitMQ and ZeroMQ)
-
VS.NET Integration and our improved solution for WCF's 'Add Service Reference'
-
Integrated Development workflow and story for Mono/Linux
-
Enable automated deployment story to Amazon EC2 and Google Compute Engine clouds
-
-
Signed and Long-term stable commercial packages
-
Starter templates around popular Single Page App Stacks: Backbone.js, AngularJS, Yeoman and Dart
-
Starter templates for creating CRM and SharePoint-enabled support systems
-
Website re-design and improved documentation
InfoQ: Thanks a lot, Demis, for taking the time out for this interview.
About the Interviewee
Demis Bellot is a developer at Stack Exchange where he maintains StackOverflow Careers 2.0 Back Office Web & MQ services built on ServiceStack. He is the creator and project lead of ServiceStack.
