Designing, implementing, and maintaining APIs for the Web is more than a challenge; for many companies, it is an imperative. This series takes the reader on a journey from determining the business case for APIs to a design methodology, meeting implementation challenges, and taking the long view on maintaining public APIs on the Web over time. Along the way there are interviews with influential individuals and even a suggested reading list on APIs and related topics.
This InfoQ article is part of the series “Web APIs From Start to Finish”. You can subscribe to receive notifications via RSS.
Mike Kelly is a software entrepreneur and founder of Stateless, an API consultancy firm assisting companies with their strategy, design and implementation challenges. Kelly currently resides in the small city of Winchester outside London with his wife and their three children. Active in the Web space since he was a teenager, Kelly is a well-known voice in the API and REST communities.
In 2011 he released the Hypertext Application Language (HAL) media type for APIs. Since that time, HAL has grown to be the dominant hypermedia format for APIs and Mike Kelly is considered one of the leading experts in Web APIs. He does not often attend public events but recently made an exception and participated in the 2014 API-Craft event in Detroit, MI where he talked about HAL and about the future of hypermedia and APIs in general.
Not long after that event, we were able to have a conversation with Mike to talk about his reasons for creating HAL and his experiences over the last three years with web developers and the API community.
InfoQ: It was in July 2011 that the Hypertext Application Language (HAL) media type was registered with the IANA. What was your motivation to create HAL?
Mike: I was working with a client on the new API of their SaaS product. The roadmap for the product included significant backend changes which were going to affect URL structures in the API. I wanted to design the API so that we could roll out those changes with as little friction as possible, and hypermedia seemed like the ideal style for that.
InfoQ: So, one the key goals was to lessen the impact of "change over time"?
Mike: Right. Another objective was to optimise the developer onboarding experience since API integration was a key value proposition of the product and a key driver of sales. For me that meant we needed to have clear, accessible documentation and the API itself had to be easy to play around with.
For that, I wanted to build an API browser that would allow developers to jump around and explore the API and its documentation in a similar way you do with a web app. So I needed a general purpose format we could use across the whole API that we could then build a browser against.
InfoQ: So you initially created HAL to solve a problem for one of your projects. To use Eric Raymond's words, you "scratched an itch" you had. So, if HAL was built to solve your SaaS problem, how did it grow to be such a common format used by so many others?
Mike: Well the approach seemed to work really well, so I shared the idea with some other people in the API space by sketching out some examples of a new general purpose format based on JSON. Feedback was really positive and a couple of people even started to adopt this sketched-out format in their APIs. Momentum built and, to cut a long story short, I ended up having to build a web page describing the sketch in more detail (I called it the Hypermedia Application Language, or HAL) and this eventually evolved into a formal internet draft.
InfoQ: HAL was the first in a wave of new API-centric media types registered in the last few years. What do you think is going on here? Why are these new designs popping up now?
Mike: I think there's a growing realisation that, although our APIs may cover very different business domains, there's a lot of common territory shared in terms of architectural style. Media types are a way we can start to collectively map some of this territory on the web of APIs and allow us to develop and share tools and techniques that are applicable to a broad range of problems.
InfoQ: Yes, on the idea of "common territory", one of the important design choices of HAL is that it focuses on 'just links and resources' in the message and relies on human readable documentation to explain how to manipulate those resources using HTTP methods. Why did you design HAL this way?
Mike: I find that most APIs I work on need to optimise the developer onboarding experience because it is what drives up key success metrics like API-user acquisition and activation. In my opinion, to achieve this you need concise API messages and rich, human readable documentation that clearly demonstrates to developers what those messages will look like.
InfoQ: So your design relies on both a machine-readable format and human-readable documentation. What about the idea that the message should contain all the details for executing a transition such as the arguments, method names, etc.?
Mike: I think there is a law of diminishing returns when adding features to a media type - more is not always better, since more features mean more burden for consumers of the format. HAL is my interpretation of where the sweet spot lies in hypermedia type design.
InfoQ: In early 2014, it was announced that Amazon's AppStream API was using HAL. What role did you play in helping Amazon learn how to use HAL for their API?
Mike: Encouragingly, I didn't play any direct role at all. I think this is a good indication that HAL's design and specification are heading in the right direction. APIs using HAL are popping up all over the place, it's impossible to keep track of them!
InfoQ: So the API community is just picking this up on their own and solving problems with HAL. I know you have a mailing list for HAL (HAL Discuss) and it seems like a pretty active list. Lots of people asking questions and sharing experiences with HAL I noticed, though that you don't seem to post a lot there. Why is that?
Mike: Another promising thing that's been happening is that I am not having to answer many questions or give advice on the mailing list any more, since the community of HAL adopters are actively sharing their experience and interpretations of the spec. I think that's a really healthy thing because, at this point, HAL has evolved beyond my intended design and is actually about what I managed to communicate and how that has been interpreted and deployed by the community. Fortunately, those two things seem to be largely in unison!
InfoQ: I don't have any figures, it seems that, of the new API designs that have come out in the last few years, HAL is the most-used hypermedia format. Is that true? and why do you think that's happening? Why are so many developers attracted to HAL?
Mike: I don't have any figures on that either, but I would agree it does seem that way. I'd like to think this was because HAL's design is well balanced and introduces hypermedia without compromising too much on the simplicity of JSON. In reality, it's probably partly that and partly that it just appeared in the right place at the right time.
InfoQ: Right, this gets back to something we talked about earlier about your design decisions. For example, one topic that comes up with some regularity on the HAL Discuss mailing list is the fact that HAL lacks inline "forms" like those in HTML, Siren, Collection+JSON, and a few others. What do you say to people who want to add these features to the HAL spec?
Mike: Whilst it is possible to try and add more dynamic affordances to HAL, I've actively resisted doing that because I feel that the added complexity of doing so damages the developer experience by making the messages less concise and at the same doesn't solve enough real world problems which cannot be solved using a simpler design.
One of the reasons often cited for needing forms in an API message is that they're required for a GUI application. My issue with this is that HTML is a perfectly adequate hypermedia format for representing GUI forms - it's ubiquitous and developers are very familiar with it. Most of the media types that are introducing forms in JSON seem to be reinventing that relatively complicated wheel, which seems like a lot of effort for not very much gain.
InfoQ: So you really see HAL as a format for machine-to-machine (M2M) use. And you don't see the concept of this added information inline very useful for machines, right?
Mike: My opinions on forms intended for M2M are probably a whole other interview on its own! In summary, I'm very skeptical and I haven't seen many compelling examples (theoretical, or real world) that convince me they are worth the added complexity.
Having said all of that, a while ago I did sketch out an extension to HAL called HALO which would introduce this type of dynamic affordance. People are interested in it but, to be honest, I don't really feel convinced enough about it's usefulness to push it forward right now.
InfoQ: Over a year ago, you started a HAL RFC. The current document has expired. Do you have plans to pick this up again?
Mike: Yes, I recently sent out a call to action on hal-discuss asking for some final feedback and adjustments to the specification. If any readers have input to share here please raise a pull request!
InfoQ: It's been three years since you first published HAL. Looking back, is there anything you might have done differently? Anything you know now that you wish you'd known then?
Mike: I'm fairly happy with the way things have gone and how things have worked out.
I probably should have written and released the HAL browser sooner, since seeing that has really made things "click" for a lot of people. Also, I should have proactively engaged high profile open source web frameworks like rails-api and ember.js in their early stages; since they're good candidates to adopt a general purpose media type like HAL as their default message format, which would make adopting HAL even easier. I should mention there are actually third party HAL libraries for both rails and ember, and for many other languages and frameworks too.
InfoQ: OK, one more question: It seems every few months a new format appears. As one of the first in this new wave of media-type designers, do you have any advice to those out there thinking about doing what you did -- creating a new media type?
Mike: My advice would be not to worry too much about having the longest feature list. The shorter your specification, the better. Scratch an itch, don't look for a problem to solve. Don't try to sound clever, use plain language and simple examples. Make sure you share it with others a early as possible so that you can iterate on the wording and make it as clear as possible. And finally, try to avoid reinventing HTML in JSON!
About the Interviewee
Mike Kelly is a software entrepreneur and founder of the API consultancy Stateless - assisting companies with their strate gy, design and implementation challenges. He lives just outside London in a small city called Winchester with my wife and three children. Mike work primarily with clients in London and Europe and recently traveled to Detroit, Michigan as a guest speaker and panelist for Apigee's API Craft event where he talked about Hypermedia APIs and his very popular hypermedia design named Hypertext Application Language (HAL).
Designing, implementing, and maintaining APIs for the Web is more than a challenge; for many companies, it is an imperative. This series takes the reader on a journey from determining the business case for APIs to a design methodology, meeting implementation challenges, and taking the long view on maintaining public APIs on the Web over time. Along the way there are interviews with influential individuals and even a suggested reading list on APIs and related topics.
This InfoQ article is part of the series “Web APIs From Start to Finish”. You can subscribe to receive notifications via RSS.