As a distributed systems developer and architect, I find the diversity of voices and ideas in the expanding world of APIs enriching, daunting, and, at times, disconcerting. Something I have noticed is that the definition of the elusive “good” web API is highly influenced by one’s exposure to a plethora of successful and failed concepts evolving over the past 40 years. With the relatively recent rise of the “API Economy” fueling unprecedented exploration of opportunities, I am always looking for better ways to articulate the principles of API design.
Filling a Gap
Absent some good mentors, opportunities, and guidance--as I have been privileged to in my career--it is no surprise when I hear the stories of startups and companies unnecessarily “re-discovering” many of the same API problems and solutions as others before them.
In the circles I run in, a common theme I hear is the tension of product managers, architects and developers having to just get something done without being able to really spend the time and effort to learn better ways to think about the problems they are trying to solve. I have often lamented not having a concise, practical mechanism to bridge the API knowledge gap in teams I work with and facilitate a common language and understanding between business and technical stakeholders. It’s either been pay your dues, suck it up and figure it out yourself, or basically just continue to repeat the past.
As such, I was pleasantly surprised when I recently watched Mike Amundsen’s video series “Designing APIs for the Web,” published by O’Reilly. Here was a straightforward, holistic dive into the A-Zs of the API lifecycle. With a format and a pace suitable to a broad set of stakeholders, it seamlessly bridges the divide between business analysts, product owners, architects, entry level and senior engineers.
In a nutshell
If you have ever had a chance to hear Amundesen expounding on API design, you know you are in for a rich blend of practical advice, new ideas and a bit of controversy thrown in to keep you on your toes. His years of experience in the field as an API consultant and architect shine through in the material to keep it relevant and engaging.
The thirteen-part series roughly breaks down into four general areas, though some of the boundaries blur a bit at times. I summarize as follows:
Distributed Computing Concepts and Architecture
- Practical synthesis of some of the heady, yet foundational, aspects of distributed messaging
- Compulsory survey of the fallacies of distributed computing
- Novel insights into understanding the boundaries of systems
- Overview of the evolution of the Web, its protocols and the advent of web APIs
Design
- Practical strategies to target your API
- Tried-and-tested methods for mining requirements from business stakeholders
- How to make your API a pleasure to use with longevity
- Survey of the pros and cons of API styles
Implementation
- Design artifacts that consolidate business requirements for implementation
- System Architectures and Components to support your API
- Service Implementation Examples and analysis
- Client Implementation Examples and analysis
Support
- Documenting APIs consistently
- Providing great Developer Experience (DX)
- Evolving APIs so they don’t break
Digging in
One of the questions on my mind as a senior level API architect when I started to watch the series was “Would this be worth my time to go through thirteen videos?” Part of me was curious who was the target audience and was I in it? Part of me was looking forward to learning new things and part of me was interested in comparing notes.
Turns out Amundsen deftly covered a spectrum of topics and material relevant to those new to APIs and those who have been working in the field for years. He shines throughout the series at communicating foundational ideas in a unique, multi-disciplinary fashion.
It never feels like the series is grinding an axe or trying to convince you of a better way. Rather, Amundsen comes across as an objective presenter of the strengths and weaknesses of a variety of approaches leaving it to the listener to draw their own conclusions.
Re-schooled on the Fundamentals
Launching into some of the fundamental limitations of distributed systems, Amundsen ties in Einstein, Newton and a bit of Pat Helland as he covers the nature and evolution of API messages in a system. Though much of this was not new material for me, what was new was the simplicity with which Amundsen mined the essence out of some pretty heady material and tied it to practical aspects of implementing APIs.
His explanation of relativistic effects on messaging and accounting for the fact the related underlying data in the messages may changed by the time you receive them, referencing burned out stars, was an insightful takeaway.
Expounding on Helland’s work on the boundaries of systems, his ideas about trust from messages on the inside versus the outside of the system were a new perspective on some old ideas for me. Tying that to how one should think about writing code was reflective of how Amundsen consistently grounds theory in application.
Absent in all this, however, were credits to Leslie Lamport, who had done some of the foundational work on distributed systems back in the ’70s that Amundsen was referencing.
What struck me overall was the spectrum of background topics covered, including the basics of the Web, were what I considered some of the most helpful concepts I had amassed through the years. If I were to generalize the background I consider basic to everyone crafting APIs, Amundsen had not only touched on it, but had consistently pointed out how to practically apply it.
Tweaked again
As the series transitioned from laying foundations to framing the design of APIs, I felt like I was getting the goods from someone who had spent a career refining what actually works for customers.
If I was to say Amundsen has an agenda for the video series, it starts to show up as he starts systematically defining a framework for all aspects of understanding the consumers, domain and business objectives of an API. It is clear that Amundsen is passionate about helping people really understand the API space and how to succeed in it.
As he discusses leveraging focus groups, hackathons, and other creative strategies to understand the developers who will consume your API, including a gamut of other relevant mechanisms to refine business objectives and discover usable APIs, one gets a sense of clarity about how to lead design initiatives.
Having spent a lot of time personally in the discipline of modeling and object-oriented analysis and design, I found some of Amundsen’s ideas challenging, if not a bit unsettling at first. He patently advocates avoiding any attempt to define a model, be it a data model or a resource, per se. Rather, he focuses on surfacing the data elements and actions associated with that data and then categorizing/grouping that information by the different consumers of subsets of the data.
I had to ponder that concept for a while and it bothered me at first. Ultimately, I realized the downstream flexibility this affords architects and developers. Doing so allows them to focus on how best to implement APIs satisfying diverse customers without the constraints of forcing the API into preconceived data structures.
It is this kind of thought provoking, preconception-breaking exposure to new ideas that I appreciate most about Mike Amundsen.
A bit of a mixed bag
One of the sections I most enjoyed in the series was Amundsen’s example of pulling all the design materials together into a concise, multi-purpose design document for engineers to implement an API and as a precursor to external facing documentation for client developers. His overview covers the vocabulary and actions of the API, media-type definitions and diverse examples.
In my experience, the presence or absence of such a design artifact dictates whether API engineers have clear direction or whether they wallow in ambiguity, pulling ideas out of wireframes or the head of product owners haphazardly. Amundsen nails this as he presents a complete sample as a precursor to the API he implements.
Interestingly, on the heels of what I considered one of the strongest parts of the series, I found the actual implementation-focused videos the weakest part overall. In an effort to give examples of how to create and consume APIs, Amundsen spends a significant amount of time scrolling through snippets of code in an attempt to illuminate the ideas he had been building.
His service architecture, while reasonable, really did not add much to the discussion as it felt more targeted to someone with little experience in implementing APIs to get them started. I believe it would have been a better use of the time to give links to code and spend more time highlighting the resulting functionality and maybe some direction on frameworks to play with.
A missed opportunity showed up in URI-style (CRUD) client vs. REST-style (Hypermedia) client discussion. Amundsen meticulously presented the fundamentally different way to code the two clients consuming his sample APIs. He then went on to tell--not show--the audience what the difference would be when a Hypermedia API under its associated client evolved.
To the uninitiated, the difference would be completely lost. URIs looked the same, code looked like code. A few extra minutes to demonstrate adding a change to the Hypermedia API would have made for a great “Aha” moment as the Hypermedia client automatically picked up the changes while the CRUD client would have completely missed them, requiring more code to be written to accommodate the changes.
Amundsen rounds out everything with an overview of documentation and DX best practices and then puts his flag in the sand on the API versioning debate for good measure.
Getting Ahead of the Game
It was clear in the overall presentation that Amundsen had labored to find the balance between too little and too much depth. Overall, I think the series succeeded. There was enough there that if a team or organization merely understood and leveraged the presented content, they would be ahead of the game in developing APIs that had longevity and the potential to delight users. And if you wanted to go deeper, he regularly pointed the way.
I guess in the end of the analysis, as much as I see a lot of benefit to others ramping up in API development watching this video series, I have to admit that I often find my mind revisiting ideas that stuck with me. They have leaked into discussions with my colleagues and are changing a number of fundamental practices of mine--which I expect they will continue to do. I am thinking better about the problems I am trying to solve from the experience. I am glad I took the time to dive in and I will be going back for more.
About the Author
Mark W. Foster started out as a research scientist in optoelectronic bio-sensing, meandered through the world of Systems and Database Admin, IT infrastructure, Enterprise computing and came out an API developer and architect. He currently works at Apiary.io and participates in a variety of projects pioneering Hypermedia API tooling. You can follow Mark on Twitter at @fosrias.
