The first "o11ycon" conference provided a comprehensive look at the emerging concept of observability in software and systems. Observability allows people to understand if things are working as expected, and to diagnose problems and identify solutions both during development-driven testing and production situations.
Charity Majors, CEO at Honeycomb, an organisation that provides a platform for real-time system debugging and observability, remarked during her opening statements that there are many definitions of observability, and she hoped that this event would help refine the meaning of the term as well as best practices around observability.
Majors noted some challenges in working with modern systems, including testing in production environments and knowing the state of a complex production system. Majors contends that having better observability helps with these challenges and more.
Christine Spang, keynote speaker and CTO of Nylas, a provider of a synchronization API for email, calendar, and contacts integrations, introduced the growing need for better observability. Spang explained that in a world where we use software for everything, software architecture has changed dramatically from the days of the LAMP stack and a single server and database to SOA, microservices, and distributed systems.
Spang noted that a key trend in monitoring is that it has become increasingly difficult to determine what is happening. In articulating the challenges and benefits of observability, Spang explained that:
Software is opaque by default, it must generate data in order to clue humans in on what is going on. Observable systems allow humans to answer the question "is it working properly?", and diagnose the scope and impact and identify what is going wrong if the answer is no. Observable systems not only have the data available to understand them but the data is accessible explorable and understandable in a fast user-friendly manner.
O11ycon featured several breakout sessions where attendees met to discuss topics and then reported a brief summary of their findings at the end of the day. In the "observability and tracing" track, the overlap and differences between observability, tracing, and monitoring was considered. One challenge that the group discussed frequently was the need for having an open and consistent way to observe and trace across disparate systems and languages. The Open Tracing and Open Census projects are two efforts to try to help in this area.
Nitzan Shapira, CEO at Epsagon, explains that:
We learned that in modern distributed distributed systems, it's not enough to monitor standard metrics, but rather [we need] to focus on the entire system. Morever, in serverless environments, things get much more complicated because many elements are managed, e.g. message queues. When there are many of these managed elements, they can greatly impact the performance of the application, and you don't have any access to their underlying infrastructure, which makes it very challenging to monitor. Hopefully advances in observability can help solve this challenge moving forward.
Peter Alvaro, assistant professor of computer science at UC Santa Cruz, spoke about the "twilight of the experts" and the future of observability. His focus was on how to take ideas and processes that are often bottlenecked in the minds of one or a few people, and make them approachable for everyone on a team, with the goal that everything can be observed rather than just the insights gathered by an expert.
Joe Beda, CTO at Heptio, and co-creator of Kubernetes, explained the rate of updates and releases, and that while it used to be acceptable to have quarterly or annual release cycles, companies now realize that they don't have a choice of speed of rolling out updates and releases, and that observability is a crucial component in making faster releases possible.
For further information on observability, videos from the o11ycon sessions will be available in the near future. InfoQ also published an emag focused on the topic of Observability earlier in the year.