Introducing: Restful Objects

Restful Objects is a public specification of a hypermedia API for domain object models. Version 1.0.0 of the specification has just been released and may be downloaded from here, and there are already two open source frameworks that implement the specification - one for the Java platform and one for .NET.

How does Restful Objects differ from other RESTful standards or frameworks - such as the JAX-RS (JSR311) for the Java platform, or Microsoft’s Web API for .NET? While such frameworks typically help in abstracting away pure network issues, they don’t ensure any uniformity in the structure of the resources defined to interact with different types of domain object. And they provide little or no support for what is arguably the most important principle of RESTful systems: "hypertext as the engine of application state", or HATEOAS. In plain language this means that it should be possible to access the entire functionality of the system just by following hypermedia controls (links) from a home resource. Perhaps the closest match to what Restful Objects is attempting is OData. However, the big difference is that while Odata is really targeted to CRUD (Create, Read, Update, Delete) functionality, Restful Objects provides access to the full behaviour (the methods) of the objects.

The new frameworks that implement the Restful Objects spec don’t just make the task of writing a HATEOAS-conformant API to a domain object model easier - they actually eliminate the work altogether! You can literally take a domain object model - written as plain-old Java objects or plain-old C# objects respectively - and create the complete RESTful API to them in minutes.

Moreover because the resources and representations provided by these frameworks conform to the public specification, it means that a client written to work with a server running one framework will work with the other also. Our hope is that over the next few months we will see more server-side implementations of the Restful Objects specification (we know of at least one other party planning to do so). We're also starting to see the emergence of standard client-side frameworks to consume a Restful Objects API; more on that later.

Applicability and benefits

The Restful Objects specification is most likely to appeal to developers who use domain-driven development (DDD) and who therefore already have or are developing a domain object model, and who also want to provide some sort of RESTful API to this model. Our own experience of DDD involves very large scale domain object modeling [1, 4, 5], with the majority of the behaviour encapsulated as methods on the domain entities. We are looking forward to using Restful Objects to further leverage such domain model assets.

Apart from the enormous savings in effort, there are a couple of other benefits to be gained from using a framework that implements the Restful Objects spec:

• Testing. Since all business logic is expressed only in domain classes, such business logic may be tested using fast and cheap in-memory unit tests. In contrast, if a RESTful API is custom-written and tuned to specific use cases, then it can only be effectively tested through (slower) integration tests running across a webserver.
• Documentation. Because Restful Objects is driven from a domain object model, it can be documented using established techniques (eg UML, or simply Javadoc/Sandcastle). This is far preferable to inventing some new notation to explicitly document a hand-crafted RESTful API.

Before we go on, we should also say that Restful Objects is agnostic with respect to the semantics of the domain object model. So, whether you believe that domain objects should represent the entities of the business, or whether you believe that there should be a separate ‘resource model’ representing instances of use-cases, say - the Restful Objects specification is equally applicable. For simplicity, in this article we'll predominantly use examples that assume the former (domain objects as entities), but we'll revisit this topic later on in the article.

Resources

In the Restful Objects specification each domain object is a resource; for example the following URI addresses a customer with the Id of 31:

~/objects/customers/31 

where ~ is the server address (such as http://myserver.com), customers defines the type of object and 31 is the ‘instance identifier’ of that type. The spec doesn't define the format of this identifier; any unique string will do. Why the need for /objects? This is to distinguish the resource from other top-level resources, including services (singleton objects such as repositories and factories that have methods but no properties) and domain-types (type metadata); each of these top-level resources has their own sub-resource structure.

Each object resource will have a number of sub-resources representing members of that object. So:

~/objects/customers/31/properties/dateOfBirth

addresses a particular Customer's date of birth, while:

~/objects/customers/31/collections/orders

addresses the collection of Orders associated with the Customer. The table below shows the object, property, and collection resources mapped against the HTTP methods used to access them. Where no useful meaning could be assigned to a particular HTTP method then a 405 error is specified.

The above analysis is hardly innovative - many bespoke RESTful APIs feature something similar. The Restful Objects specification breaks new ground, though, in applying the same approach to object ‘actions’ - meaning those of an object’s methods that are to be accessible through the RESTful interface. To the extent that this is tackled at all in other designs (for example in [2]), it is usually to map an action's invocation to HTTP POST. Our objectives led us to distinguish between providing information about an object action - such the set of parameters required - and actually invoking the action. Thus, the URL:

~/objects/customers/31/actions/lastOrder

describes an action that will retrieve the last order for customer 31, while:

~/objects/customers/31/actions/lastOrder/invoke

is the resource to invoke this action.

We also realised that requiring all actions to be invoked with a POST was not making correct use of HTTP: what about idempotent actions, or query actions that just retrieve other objects (sometimes described as ‘side-effect free’)? The table below shows how we resolved this:

In some domain models, most of the behaviour (business logic) is implemented on services, with domain entities acting as little more than data structures passed to and from the service actions. The Restful Objects specification copes well with this pattern; the only difference is that the URL would identify a service instance rather than a domain object instance. However, the spec copes equally well with a pattern where most of the behaviour is implemented as methods on ‘behaviourally-rich’ domain objects; services then play the much more minor role of providing access to objects (finding existing objects, or creating new ones).

Representations

Each resource returns a representation; the Restful Objects spec defines that these representations are in JSON (JavaScript Object Notation). For resources called with a PUT or POST method, a representation must also be provided as a body to the request, describing how the resource is to be modified.

The specification defines a few primary representations:

• object (which represents any domain object or service)
• list (of links to other objects)
• property
• collection
• action
• action result (typically containing either an object or a list, or just feedback messages)

and it also describes a small number of secondary representations such as home, and user. Each of these representations has a formal media type, exposed in the HTTP Content-Type header. For domain objects, this takes the form:

   application/json;profile="urn:org.restfulobjects:repr-types/object";
   x-ro-domain-type="http://~/domain-types/customer"

Formally, this is of type application/json, with two optional parameters: profile, and x-ro-domain-type. These two parameters provide additional semantics to the consuming client, and can be thought of as layered one on top of another. At the lowest level, application/json tells the client merely that the payload is JSON. For some clients, such as developer plug-ins to browsers, this is all the information that is required. On top of this, the profile parameter indicates that the representation is of a domain object. Finally, the x-ro-domain-type parameter indicates the type of the object: in this case a Customer. A client might use this to customize the user interface, or just to verify that the representation returned is the one expected. Incidentally, the specification uses the .x-ro- prefix to minimize namespace conflicts whenever there is no existing or draft standard that can be leveraged. This necessarily gives rise to a number of custom parameters and query strings; Restful Objects defines no custom HTTP headers, however.

Links

In line with the HATEOAS principle, each representation contains links to other resources, and each of these links has a rel parameter defining the nature of the relationship. The specification re-uses several IANA-defined rel values (for example, self, up, describedby), plus some custom ones. For example:

   urn:org.RESTfulobjects:rels/details;action="placeOrder"

defines a link from an object representation to a resource for an action on that object. The prefix:

   urn:org.restfulobjects:rels/details

is sufficient for a generic client, while the additional parameter action - indicating in this case that this link is for the placeOrder action - may be of use to a bespoke client.

Putting all the above together, the table below shows a typical flow through a set of resources to accomplish some user goal.

Server implementations

Earlier we mentioned two separate open source frameworks that implement the Restful Objects specification. Restful Objects for .NET is a complete implementation of the specification, in beta only because it makes use of the Microsoft Web API framework (part of ASP.NET MVC4 - in ‘RC’ stage at the time of writing). The second framework, Restful Objects for Isis runs on the Java platform; this framework is working, but currently implements an earlier draft of the specification and needs further development and testing before release. We are actively involved in the development of these frameworks.

With each of these frameworks, it is possible to take a domain object model, written as POCOs and POJOs respectively, and create a complete RESTful API conforming to the Restful Objects specification, without writing any further code. An online video recording (using the .NET framework) shows this being done in just a few minutes.

This is possible because both of the frameworks build upon existing frameworks that implement the naked objects pattern - whereby an object-oriented user interface is created automatically from a domain object model using reflection, with public methods made available (by default) as user actions. The new Restful Objects frameworks reflect over a domain object model in a similar fashion, but render the objects’ capabilities as a RESTful API rather than a user interface. In both cases, they delegate to the existing frameworks (Naked Objects for .NET, and Apache Isis respectively) the responsibility for reflection, for object persistence, and other cross-cutting concerns.

The new frameworks both recognise some simple domain object coding conventions, and annotations (‘attributes’ in .NET). Any public method on an object, for example, will by default be made available as an action in the Restful Objects API, but this may be overridden by annotating the method as Hidden. If an object defines a public method foo([params]) and also a public method validateFoo([params]), the latter will be recognised as providing logic for validating parameters supplied to the former before it is executed.

Both frameworks also provide fine-grained authorization based on the user’s identity and/or roles. If, for a given domain type, the user is not authorized to view a given property, collection or action, then in the corresponding representations the links to that object member will never be rendered for that user. Were the user to attempt to gain access by constructing the URL for the resource directly, they would be returned a 404 error; if the user has permissions to view the object member, but not to modify it, then attempting the latter would return a 403.

The source code for Restful Objects for .NET can be downloaded from Codeplex or the framework may be installed as a NuGet package; Restful Objects for Isis may be downloaded as source code or installed through a Maven archetype.

For any developer already using Naked Objects for .NET or Apache Isis, the new frameworks mean that they can create a RESTful API to their existing domain models with literally minutes of effort. But our intent was always to appeal to developers with no knowledge of, nor interest in, the naked objects pattern, and we are indeed seeing interest from such developers.

Clients

Most of our development effort to date has been focused on the server side - the code that generates a Restful Objects API from a domain object model. We have also done some limited work on client applications that consume such APIs.

In one example, the client is a conventional web application (written using ASP.NET MVC), where the controller methods make calls to a Restful Objects API implemented on another server. We hope in future to develop a small client-side application library for making the calls to the Restful Objects server and for transforming the returned JSON representations into objects that can be manipulated by C# or Java, say.

Another example client we have written is a ‘single-page application’, consisting of a single web page, containing just a few lines of static HTML, and supported by a rich set of JavaScript functions using JQuery. The JavaScript makes the calls to the Restful Objects API on a server and renders the results as HTML in the browser. Again, we hope in future to be able to write a small JavaScript application library, specific to consuming a Restful Objects API, but drawing heavily upon JQuery - or perhaps someone else will pick up that baton.

The generic nature of the Restful Objects specification suggests another possibility: generic clients that can work, unmodified, with any domain model via the RESTful API. We are aware of three different open source generic clients already under development, and these are listed on the Restful Objects website. All of them are single-page apps, written in JavaScript and using existing libraries. The JavaScript code within of these generic clients could also be used as the basis for creating one or more custom clients also. The three generic clients have very different styles of user interaction. A screenshot from one of them - A Restful Objects Workspace (AROW), being developed by Adam Howard, is shown below:

(Click on the image to enlarge it)

Counter arguments

The idea of exposing domain entities via a Restful API is controversial. Some commentators have suggested that it is a bad idea; or that it creates security holes; others have even suggested that it is not possible to create a true Restful API that way. Let’s look at those arguments in detail.

Rickard Öberg has asserted that exposing domain entities as resources "cannot be HATEOAS because there is no sensible way to create links between resources that expose the application state" [3]. But this is plainly not true: Restful Objects exposes entities as resources; is fully HATEOAS; and has very clear rationales for the links.

Jim Webber has asserted that even if is possible, it is a bad idea because this results in a tight coupling of client and server, whereas in a Restful system client and server should be able to evolve independently [6]. He and others argue that you should expose only ViewModels and/or objects representing use-cases, both of which should be versioned, and which effectively insulate the client from underlying changes in the domain model.

We suggest that this argument, while not totally wrong, is being applied indiscriminately. The truth, we believe, is much more nuanced: there are situations where this line of argument is valid, but many where it is not. Two factors need to be considered.

The first is whether the client and server are both under the control of the same party. For a publicly-accessible Restful API - such as one operated by Twitter or Amazon - then the server and clients are owned by (many) separate parties and here, we agree, that exposing domain entities is not a good approach. Restful Objects, however, can work perfectly well with view models and/or use-case objects in this case, and these patterns are discussed in more detail in the specification.

However, there are a great many potential uses of Restful APIs where client and server evolution are controlled by the same party - in the form of enterprise applications for primarily internal use. Here, exposing domain entities is not only safe, it is actually a good idea. Such applications, which fall into the category of ‘sovereign’ systems (as defined by Alan Cooper), typically need to give the user access to a much broader range of data and functionality than in a public-facing (or ‘transient’) application.

The second factor is whether the client is ‘bespoke’ (written specifically to work with a particular domain model) or ‘generic’ (capable of working, unmodified, with any domain model). To date, almost all clients consuming Restful APIs, whether for intranet or public internet, are bespoke - so there is a need to insulate them from changes to the domain model. But that is not true of a generic client, which will respond automatically to changes in the domain model. To date, few people have considered the possibility of generic clients, though, because there was no comprehensive standard against which they could be written. We suggest that the concept of a generic client is actually more in-line with the spirit of REST (and the letter of it too) than a bespoke client. After all, at one level, a web browser is just a generic client to a restful interface, operating at a document level. Restful Objects enables the idea of a generic client operating at a higher level of abstraction: that of a domain object model.

Portraying these two factors as a 2x2, as shown below, we can see that if you are dealing with an intranet application and/or a generic client, then exposing domain entities through a RESTful API is both safe and effective. Only where you are working on a public internet application and with bespoke clients is it necessary to heed the advice that domain entities should be entirely masked by ViewModels and/or use-case objects, to insulate clients from server changes.

To reiterate: Restful Objects is equally applicable to all positions on this grid. The fact that the bottom-right corner has received most attention to date is, we suggest, more a reflection of the difficulties of building bespoke Restful APIs than an inherent limitation. Another common nostrum - that Restful APIs can only be built for applications that have a small, tightly-defined, state-transition diagram - is further evidence of people thinking only in that box.

Öberg has further suggested that exposing the domain entities as a RESTful API cannot work for a public application because of authorization issues, citing the example where access to a ResetPassword action may be regulated by role, but access to a ChangePassword action must be restricted to a specific user. This particular example is actually trivial to code in both of the Restful Objects framework implementations described above. We suggest that a better example for the point that he is trying to make would be access to a specific object instance, rather than a method. This is a common need in public-facing systems: for example where a user must be able to access their own ShoppingBasket, but not be able to access anyone else’s ShoppingBasket just by guessing at the URL.

While neither of the two Restful Objects implementations currently support instance-based authorization per se, there is another perfectly viable solution to this problem. As we mentioned earlier, the form of instance identifiers in URLs is not prescribed in the Restful Objects spec. It follows that this part of the URL may be encrypted by the server, for example using a session-generated private key. Invoking a service action resource to return MyShoppingBasket might then return an object representation whose "self" link was:

~/objects/baskets/xJDgTljGjyAAOmvBzIci9g

The URL for the Total property on that shopping basket would therefore be:

~/objects/baskets/xJDgTljGjyAAOmvBzIci9g/properties/Total

So parsing of resource identifiers remains straightforward, but it would be effectively impossible to retrieve another Basket directly. Certainly this is less pretty, but it is worth pointing out that REST is not about creating ‘pretty’ URLs. As Tim Berners-Lee has stressed, apart from the server address, URLs should be treated as opaque; it's the "rel" value that matters. That’s part of the point of HATEOAS.

In both of the two Restful Objects frameworks described in this paper we plan to make session-key encrypted instance identifiers as a configurable option..

Conclusion

Creating a RESTful API to a large enterprise system has, to date, been a very expensive exercise in development terms - requiring huge resources in design, development, testing and documentation. Using a framework that implements the Restful Objects specification means that the whole exercise becomes almost trivial - leaving the developers to concentrate their efforts where it is most needed - on the domain object model.

Moreover, the standardization of resources and representations encouraged by the Restful Objects spec opens up a huge opportunity for client-side libraries, that will work with any Restful Objects server, and even for completely generic clients, that will work for any application coded to the standard.

We hope you'll explore the Restful Objects specification and the framework implementations, and we'd be delighted to hear from anyone interested in writing a new open source framework or library designed to conform to the Restful Objects spec.

References

[1] Haywood, D, "Domain-driven design using Naked Objects", 2009, Pragmatic Bookshelf
[2] Masse, M. "REST API Design Rulebook", 2011, O’Reilly
[3] Öberg, R, "The Domain Model as REST Anti-pattern"
[4] Pawson, R. "Case Study: Large-scale pure OO at the Irish Government", QCon London 2011 presentation
[5] Pawson, R, "Naked Objects", 2004, PhD thesis, Trinity College, Dublin.
[6] Webber, J, "Rest and DDD".

About the Authors

Richard Pawson has worked in IT for 35 years  -  claiming to have been the very first person in Europe to write and run a program in a Microsoft language. As well as periods spent in advanced robotics, toy design, and technology journalism, he was 14 years with Computer Sciences Corporation as Director of Research. His 2004 PhD thesis is the definitive reference on the 'Naked Objects' architectural pattern, and since then he has been Managing Director of Naked Objects Group. He lives in Henley-on-Thames, UK, in a house that he designed using Christopher Alexander's A Pattern Language.

Dan Haywood is a freelance consultant, developer, writer and trainer, specializing in domain-driven design, agile development, and enterprise architecture on the Java and .NET platforms. He's a well-known advocate of Naked Objects, and a leading committer on the Apache Isis project.  Dan is the author of the Restful Objects spec, and has also written a number of books on DDD and OO.  He lives near Oxford, UK.