At the inaugural O’Reilly Software Architecture conference, Bryan Cantrill shared lessons learnt from building a business around the early adoption of OS-virtualised containers. Core advice included; don’t innovate in too many directions at once; identify the technical differentiators, and focus on ease of use/adoptions for everything else; and don’t give up - technological revolutions happen much slower than people think they should.
Cantrill, CTO at Joyent, began the talk by stating that for nearly a decade Joyent have believed OS-virtualised containers are the future of computing. While the efficiency gains may be tremendous, they have not alone been enough to propel containers into the mainstream. The emergence of Docker, and its promise of developer agility, has finally pushed container technology along the adoption curve towards the early majority.
Cantrill began the story of his journey with the early adoption of technologies by starting that virtualisation has provided the catalyst for cloud technologies. In the 1960s pundits foresaw a multi-tenant compute utility, but it took the emergence of the Internet, commodity computing and virtualisation to yield cloud computing. Cantrill discussed how virtualisation is the essential ingredient for multi-tenant operation, but the primary question is where in the stack to virtualise?
Choices around virtualisation capture tensions between elasticity, tenancy and performance.
Virtualisation choices drive economic tradeoffs.
Hardware virtualisation presents a tenant with an operating system of their choosing, and there are as many operating systems (OS) as tenants. However, hardware virtualisation has a large cost, as operating systems are not designed to share compute resources. Virtualisation at the application platform layer addresses the tenancy challenges of hardware virtualisation, but abstractions are ‘dictated’ to the developer. Cantrill suggested that this problem can be seen within the APIs and environment constraints provided by modern Platform as a Service (PaaS) offerings.
Cantrill suggested that virtualising at the OS level provides an ideal solution to the above problems. Providing a single kernel allows for efficient use of hardware resources, which maximises tenancy and performance, and disjoint virtual instances can also be securely compartmentalised by the OS. Users have access to what appears to be a virtual machine on which to run higher-level software, and this is essentially the functionality of Infrastructure as a Service (IaaS), without the associated cost of hardware virtualisation.
OS-level virtualisation provides the ease of a PaaS with the generality of IaaS.
Joyent runs OS containers in the cloud via their SmartOS product. Cantrill shared several stories exploring the challenges Joyent have faced over the past 10 years, as OS-level virtualisation has moved from the fringes into mainstream adoption. Case studies included the creation of a node.js and OS-based virtualised PaaS, the combination of KVM with SmartOS, and the development of the Manta big data platform.
Cantrill suggested that being a successful early adopter of a revolutionary technology can make it easy to dismiss the inconveniences as ‘casualties of the future’. However, some conveniences are actually constraints - although it can be very difficult to discern which. When adopters must endure painful scenarios in order to enjoy the technological differentiators, the economic advantages of a revolution are undermined.
Even as the understanding of an emerging technology develops, and the associated thinking does shift, the process of mainstream adoption can take a long time. This can be a problem if a company's business model is based on the adoption of this technology. Quoting the economist John Maynard Keynes, Cantrill reminded the audience that “the market can stay irrational longer than you can stay solvent”
Cantrill stated that counter to the above perils, the promise provided by early adoption of a successful technology can provide payoffs that are largely outsized with respect to the risk.
Placing gutsy technological bets attracts like-minded technologists - which can create uniquely fertile environments for innovation.
If early adoption is based within an open source community of like-minded technologists then innovation is not confined to within a company’s walls. Cantrill suggested that in this respect, open source is the farm system.
Cantrill concluded the talk by stating that while building a business around early adoption is not for everyone, every organisation should be engaging in early adoption of technology somewhere, and ideally in the open. Cantrill provided a series of guidelines for this:
- Don’t innovate in too many directions at once.
- Identify the technical differentiators, and focus on ease of use/adoptions for everything else.
- Stay flexible and adaptable - it is possible to be right on trajectory, but wrong on specifics.
- Don’t give up. Technological revolutions happen much slower than people think they should - and then much more quickly than anyone would think possible.
The slides for Bryan Cantrill’s “The Peril and Promise of Early Adoption: Arriving 10 Years Early to Containers” talk can be found on slideshare, and the recording will be available on the O’Reilly Software Architecture 2015 conference website.