Day Three of the 17th annual QCon San Francisco conference was held on October 4th, 2023, at the Hyatt Regency San Francisco in San Francisco, California. This five-day event, consisting of three days of presentations and two days of workshops, is organized by C4Media, a software media company focused on unbiased content and information in the enterprise development community and creators of InfoQ and QCon. It included a keynote address by Will Larson and presentations from these four tracks:
- Architecting for the Cloud
- Hosted by Khawaja Shams, co-founder and CEO at Momento
- Offers attendees a chance to share practitioner-driven insights on what works (and what doesn't) as an inspiration to make the most out of a developer's cloud computing journey
- Deep Tech: Pushing the Boundaries of Hardware+Software
- Hosted by Allison Randal, board member at sfconservancy.org, openinfra.dev, and openusage.org
- Offers attendees the opportunity to explore the latest trends in deep tech, including artificial intelligence, machine learning, big data, IoT/Edge, security, quantum computing, and more
- Emerging Trends in the Frontend
- Hosted by Jeff Wagner, director of engineering at Snowflake
- Offers attendees the chance to leverage new frameworks and capabilities to create more flexible, faster and engaging applications for users.
- Lessons in Building Organization Resilience
- Hosted by Courtney Hemphill, partner and head of product, Engineering and Design (PXEL) at WestMonroe
- Offers attendees the chance to dive into approaches teams and engineering leaders have taken to improve the resiliency of their organization over time
There was also one sponsored solutions track.
Wes Reisz, technical principal at Thoughtworks, creator/co-host of The InfoQ Podcast and QCon San Francisco 2023 program committee chair, and Danny Latimer, content product manager at C4Media, kicked off the day three activities by welcoming the attendees and providing an overview from day two. There were 20 editorial presentations, four unconference sessions and six presentations from sponsors.
Reisz highlighted a list of recommended day two sessions based on attendee feedback, namely: How Netflix Ensures Highly-Reliable Online Stateful Systems presented by Joseph Lynch, distributed systems engineer at Netflix; How to Get Tech-Debt on the Roadmap presented by Ben Hartshorne, principal engineer at Honeycomb; Building Guardrails for Enterprise AI Applications with LLMs presented by Shreya Rajpal, founder at Guardrails AI; and Defensible Moats: Unlocking Enterprise Value with Large Language Models presented by Nischal HP, vice president of data science at Scoutbee.
Latimer provided an overview of how QCon conferences are organized. The process starts with a program committee six to seven months in advance, the selection of tracks and track hosts, and how the track hosts select the speakers for their respective tracks. There is no call for papers as speakers are selected by invitation, and are typically senior software practitioners and real-world technical experts.
Including himself, Reisz introduced the program committee, namely: Monica Beckwith, Java champion, First Lego League Coach, passionate about JVM Performance at Microsoft; Haley Tucker, principal software engineer for platform engineering at Netflix; Sid Anand, chief architect at Datazoom, Committer/PMC Apache Airflow; Courtney Hemphill; and Justin Cormack, CTO at Docker.
Latimer acknowledged the many QCon San Francisco staff and volunteers, the exhibitors and sponsors.
The aforementioned track leads for Day Three introduced themselves and described the presentations in their respective tracks.
Courtney Hemphill introduced the keynote speaker, Will Larson.
Keynote Address: Use Engineering Strategy to Reduce Friction and Improve Developer Experience
Will Larson, CTO at Carta and author of "An Elegant Puzzle" and "Staff Engineer", presented his keynote address entitled Use Engineering Strategy to Reduce Friction and Improve Developer Experience. Larson kicked off his keynote with the alternate title, "Solving the Engineering Strategy Crisis!" Referencing the book "Good Strategy/Bad Strategy" by Richard Rumelt, Larson enumerated the attributes for a good engineering strategy, namely: diagnosis, guiding policies and coherent actions. This is the basis for his equation:
Engineering strategy = honest diagnosis + practical approach
As an example, Larson provided a step-by-step engineering crisis scenario using a fictitious company called "Widget & Hammer Company."
- A software engineer joins Widget & Hammer Company
- The engineer's team works in a Python monolith to build the Widget product
- The CTO hates monoliths and mandates a service migration
- The engineer switches teams to build a brand new Hammer product in a new service
- Two years later, the engineer's old team and the Widget product are still in the monolith
- The engineer has no idea how to share code between the Widget and Hammer products
Focusing on the honest diagnosis operand of the above equation, Larson asks, "How can engineering strategy help the Widget & Hammer company?" To start, he presented a number of dishonest diagnoses, namely: "We can migrate to services in three months;" "We have de-risked our approach by moving meaningful complex component out of our monolith;" We are willing to invest heavily in migrating our services, even if it means slowing down product velocity in the short term;" and "We are willing to expand out Developer Tools team to build new tools for services in addition to supporting our existing monolith."
Interestingly, the list of honest diagnoses was exactly the same because it is quite possible that any one of them can be accomplished. Larson then defined honest diagnosis as "A reality-based assessment of your circumstances. Nothing is universally honest."
Focusing on the practical approach operand of the above equation, Larson asks, "How can engineering strategy help the Widget & Hammer company?" He maintained that practical approaches acknowledge tradeoffs when documenting strategies. A small example showed the process of how an engineering strategy evolved from something that would most likely fail, to something that would most likely succeed. Context and knowing the tradeoffs are essential.
Larson then asked, "Can something so simple be so useful?" To answer that question, he elaborated on some concrete strategy examples from Stripe ("We run a Ruby monolith"), Calm ("We are a product engineering company") and Uber ("We run our own hardware"). For each company, he provided the diagnosis, the approach and the impact.
The engineering strategies for all three companies were successful because: many interesting properties are only available through universal adoption ("We run our own hardware"); concentrate on tooling investment ("We run a Ruby monolith"); reduce energy lost on conflict ("We are a product engineering company"); control your innovation budget (all three); and new hires, especially senior new hires, are forced to explicitly engage with strategy rather than having the option of ignoring it. (all three).
There is also an impact on missing engineering strategies with examples of: a good diagnosis, but highly impractical approach; a diagnosis reasoned back from an approach as that approach was determined to be built on a shaky foundation; and two reasonable, but conflicting diagnoses that culminated in a flawed approach.
Larson stated that every company has an engineering strategy, but it is rarely documented. With a written strategy, it's easier: for new hires to find; to get feedback; to explain updates; to clarify confusion; and to hold employees accountable. If a strategy is struggling, it either due to a dishonest diagnosis or an impractical approach.
Highlighted Presentations: Streamlining Cloud Development with Deno, Unconference Sessions
Streamlining Cloud Development with Deno was presented by Ryan Dahl, co-founder & CEO at Deno and software engineer best known for creating Node.js. Dahl kicked off his presentation by stating that the web has become the medium of human information and that JavaScript is inherently tied to the web.
As the creator of Node.js, his goal was to force developers to easily build fast servers by only exposing asynchronous I/O in JavaScript. However, building those fast servers requires more than just asynchronous I/O. Issues that need to be addressed include: managing complex cloud configurations; some sort of geographically replicated state; navigating a plethora of software, workflows and toolchains; and supply chain security. This was the inspiration behind Deno, an open source next-generation JavaScript runtime.
Dahl introduced Deno as: a browser for command-line scripts; a single executable file at 100MB; support for 14 different web standards; and secure by default. Deno offers native support for JavaScript and TypeScript and the language includes a standard library.
Dahl then provided a demo on how to quickly build an asynchronous compression stream application that would open a file, then compress it to a gzip file. Upon executing the application, Deno requested read access and write access to the file to be opened and the gzip file, respectively. Libraries from Node.js can be imported in Deno.
Deno Node Transform (DNT), a Deno-to-npm package build tool that can transpile JavaScript for distribution on npm. Deno tests can also be transpile and executed on Node.js. Dahl provided a demo on how to build an Express server.
Deno Deploy, the "easiest serverless platform," as Dahl claimed, features: scaling to zero cost; support for npm packages; built-in storage and compute; low global latency in 35 regions; fast cold starts; and powers Netlify Edge functions. Dahl then provided a demo on how to take the aforementioned freshly-built Express server and deploy it.
Deno KV, a datastore anchored by ACID transactions and powered by FoundationDB. Features include: zero configuration; ACID transactions; scaling to zero cost; and built-in to Deno Deploy. Dahl stated that Deno KV doesn't replace a real database, but it is useful for sharing state.
Unconference Sessions, facilitated by Danny Latimer, content product manager at C4Media, are facilitated, bottom up and self-directing discussions among experts. They are designed as "by the people, for the people." There were a total of seven unconference sessions spread over the first two days of presentations related to the conference tracks:
- Languages of Infra
- Beyond YAML
- Architectures You've Always Wondered About
- Staff+ Engineering
- Platform Engineering Done Well
- JVM Trends; Modern ML
- Designing for Resilience
A typical unconference session featured a round of introductions followed by attendees writing down topics as candidates for discussion. Afterwards, attendees would then peruse through the submitted list of topics and select the ones that they felt were worthy of a more detailed discussion. Approximately eight topics were selected and areas for each topic were established in the conference room for attendees to participate in the conversation.
Summary
Day Three consisted of an opening keynote address, 20 editorial presentations, five presentations from sponsors and a closing keynote address, Mission, Culture, and Values: Using Them to Guide Your Company Through Good and Challenging Times, delivered by Heather McKelvey, vice president of engineering at LinkedIn.