API design affects all developers. Some APIs are a pleasure to work with, others are annoying at times and yet others can be downright frustrating. But what is the difference between a good one and a bad one? What are the qualities that makes one API easy to use and other hard? The ACM Queue recently published an article by Michi Henning about API design; an article that analyzes the aspects of good and bad and the effects APIs can have on the productivity.
We all recognize a good API when we get to use one. Good APIs are a joy to use. They work without friction and almost disappear from sight: the right call for a particular job is available at just the right time, can be found and memorized easily, is well documented, has an interface that is intuitive to use, and deals correctly with boundary conditions.
So, why are there so many bad APIs around? The prime reason is that, for every way to design an API correctly, there are usually dozens of ways to design it incorrectly. Simply put, it is very easy to create a bad API and rather difficult to create a good one. Even minor and quite innocent design flaws have a tendency to get magnified out of all proportion because APIs are provided once, but are called many times. If a design flaw results in awkward or inefficient code, the resulting problems show up at every point the API is called. In addition, separate design flaws that in isolation are minor can interact with each other in surprisingly damaging ways and quickly lead to a huge amount of collateral damage.
Before giving his advice on API design, Michi starts out by looking at an anti-example. He analyzes the Select() call in the .NET-framework to show common problems that can be found in many APIs.
Select() takes lists of sockets to be monitored. In most applications, the sockets used won’t change often, so these lists are commonly the same ones during a long period of time. But:
Because Select() overwrites its arguments, the caller must make a copy of each list before passing it to Select(). This is inconvenient and does not scale well: servers frequently need to monitor hundreds of sockets so, on each iteration, the code has to copy the lists before calling Select().
When it comes to waiting and blocking, Select() accepts a time-out, but it doesn’t give an easy access to weather the call returned due to the fact that a socket became ready or not.
To determine whether any sockets are ready, the caller must test the length of all three lists; no socket is ready only if all three lists have zero length. If the caller happens to be interested in this case, it has to write a rather awkward test. Worse, Select() clobbers the caller’s arguments if it times out and no socket is ready: the caller needs to make a copy of the three lists on each iteration even if nothing happens!
Furthermore, the documentation gives no indication of how to wait indefinitely. By experiment, Michi realizes that giving a timeout value of zero or less returns immediately and that there seems to be no way to wait for more that 35 minutes; which forces him to implement a wrapper of his own to be able to continue to wait.
Writing this wrapper gives new insights of problematic API design:
Another problem with Select() is that it accepts lists of sockets. Lists allow the same socket to appear more than once in each list, but doing so doesn’t make sense: conceptually, what is passed are sets of sockets. So, why does Select() use lists? The answer is simple: the .NET collection classes do not include a set abstraction. Using IList to model a set is unfortunate: it creates a semantic problem because lists allow duplicates. (The behavior of Select() in the presence of duplicates is anybody’s guess because it is not documented […])
One of the problems with poor API design is that more or less every developer who uses the API have to compensate for the poor design. The larger the API user base is, the more waste of developer time.
Poor APIs often require not only extra code, but also more complex code that provides more places where bugs can hide.
However, there are differences in effect, depending on where a design mistake is made.
The lower in the abstraction hierarchy an API defect occurs, the more serious are the consequences. If I mis-design a function in my own code, the only person affected is me, because I am the only caller of the function. If I mis-design a function in one of our project libraries, potentially all of my colleagues suffer. If I mis-design a function in a widely published library, potentially tens of thousands of programmers suffer.
Any widely public library can be considered more or less unchangeable. Any change to an API of that kind will break backwards compatibility and cause a lot of trouble. Signature changes will cause the client code to either not compile or to crash and behavioral changes will create more subtle errors. Even a bugfix will cause problems, since there will be code that depends on the original buggy behavior.
In the world of dynamically linked code, the client code is particularly vulnerable to API changes. Even in the case where a versioning mechanism involved, how can the client code determine if a very small step in a library version number affects any current assumptions?
Even though the API change itself may make the framework better, every user of the API will have to pay a price for the upgrade. It is also possible to make a change that doesn’t affect one single line of client source code, but which still forces every client to be recompiled to avoid a crash, since the binary representation can change even though the source representation remains.
An API designer has got a lot to think about, but despite the fact that most decisions will be based on experience, there are good recommendations out there. Joshua Bloch shares his thoughts in his Javapolis/InfoQ presentation about API design, and one of his most fundamental recommendations is to to keep an API as small as possible (but not any smaller), since:
You can always add, but you can never remove
Joshua talks about everything from the importance of concepts like immutability (to keep classes immutable unless there’s a good reason to do otherwise), to smaller, but important, details like the practice of returning empty lists instead of null values if possible (since the risk of a null value requires extra checking in the client code).
Michi Henning also gives his rules and guidelines of how to do better, in his article:
- An API must provide sufficient functionality for the caller to achieve its task.
- An API should be minimal, without imposing undue inconvenience on the caller.
- APIs cannot be designed without an understanding of their context.
- General-purpose APIs should be “policy-free;” special-purpose APIs should be “policy-rich.”
- APIs should be designed from the perspective of the caller.
- Good APIs don’t pass the buck.
- APIs should be documented before they are implemented.
- Good APIs are ergonomic.
Since most people would acknowledge that these things are important, one would expect it is taught at the universities, but that doesn’t seem to be the current standard procedure. Ed is a blogger who commented Michi’s article by describing the lack of such knowledge in class:
I realize now that I had an inadequate education. My classes were all about finding the best algorithm or discovering new data structures that behave like old ones (anyone remember implementing a deque?). And assignments were NEVER graded on design; it was purely on speed of execution.
But if the education systems doesn’t teach API design, surely older colleagues will? Michi is not too optimistic on this point.
It takes time and a healthy dose of “once burned, twice shy” to gather the expertise that is necessary to do better. Unfortunately, the industry trend is to promote precisely its most experienced people away from programming, just when they could put their accumulated expertise to good use.
Which makes the whole learning process tricky. So, how should we as an industry improve?
