Of most of the applications we have in the world, maybe 90% of them, are perfectly served by a monolithic approach; to avoid overengineering, we should therefore start with a simple architecture and evolve it as needs arise, Randy Shoup declared at the Reactive Summit 2018 conference. In his recently published presentation he describes his experience with companies that started small and then grew into large global internet companies, how their architecture evolved during that time, and his recommendations from an IT perspective when starting a new company or a new product.
Shoup, with experience working for eBay, Google and Stitch Fix, is currently VP of engineering at WeWork. His first example was eBay, which started in 1995 as a three-day weekend project, not as a proof of concept (PoC) for a business, but to see if it was possible to do something interesting on the web. Today, it’s on its 5th complete rewrite of the infrastructure and Shoup characterizes the company as a polyglot set of microservices, and adds that both Twitter and Amazon have gone through a similar evolution.
Starting with some form of a monolith and ending up with a set of microservices is for Shoup a common pattern for large companies, and notes that there are two parts in this pattern — no one starts with microservices, and above a certain scale everyone ends up with microservices.
The companies mentioned are all very large and Shoup points out that the architecture that works at their scale is completely inappropriate for most companies. Most applications are perfectly served by a monolithic approach, and Shoup’s recommendation when building a new application or system is to start simple and evolve the architecture as needs arise. He likes to rephrase this as:
If you don’t end up regretting your early technology decisions, you probably overengineered.
For Shoup, a common progression curve for most companies and products includes an idea and start phase, a scaling phase where distributed systems may be relevant, and commonly an optimizing phase:
In the idea phase Shoup claims that we shouldn’t have any fancy architecture at all – we are just prototyping. To avoid overengineering we should constantly ask ourselves, "What problem are we trying to solve?" The goal in this phase is to explore the solution space as rapidly as possible and with minimum cost to:
- Find a business model
- Find a product market fit
- Acquire our first customers
If possible, he recommends that we try to stay away from all technology, for example by using Google ads to see if any clicks on it, or by using paper prototypes or Excel spreadsheets.
When moving into the starting phase, the goal is to meet near-term customer needs, and do this as cheap as possible. Typically at this stage there is only one team consisting of something like 4 – 6 people. They work with a fairly short time horizon, maybe 3 – 4 months. One argument for this is that it’s often hard to see far ahead in terms of what features to build. We only need a minimal amount of architecture, enough to get us moving forward. Shoup emphasizes that it is still not about scaling; we should use simple and familiar technology, and definitely a monolithic architecture — a single application and a single database. The infrastructure should be minimal and not built by yourself. Instead, he suggests using Platform as a Service (PaaS) or similar technology.
The advantages using a monolithic architecture at this point include:
- It’s simple
- It’s fast because it only has in-process latencies
- You get a single build deployment unit
- It’s very resource-efficient at small scale
Besides lack of scaling and a single point of failure, a major disadvantage of a monolith is the poor enforcement of modularity. It’s doable but it requires programming and team discipline. But Shoup points out that none of this is of any concern in this phase. Still, as we see a need to prepare for scaling, he thinks it’s worthwhile to abide to a modularity discipline by using components or modules within the monolith. This will simplify future modification or replacement with external services.
Looking at when we may need to rearchitect this monolith, Shoup has experienced some warning indicators:
- Velocity: your ability to deliver slows down due to coupling and lack of isolation
- Scaling: vertical scaling no longer works, or different parts of the system need to scale independently
- Deployment: different parts evolve at different speeds and therefore need independent deployment
When we enter the scaling phase the goal is to stay ahead of a rapidly growing business and to keep our application up and running. In the organisational context we now typically increase the number of teams and work with a longer time horizon. Commonly there is also a need to introduce repeatable processes.
In the technical context, the scaling phase often means a move to a technology that scales. Typically, we now start to split out services from the monolith, and it’s very common to try to decrease the load on the single database, for instance by creating read-only replicas of some data. It’s also common to start with separating out special services like payment and billing and introducing an event queue or message bus.
For Shoup, this is typically also the time to consider whether we should migrate our monolith into small independent components, what we today call microservices. We must also consider if the single primary storage used is still the right way to store data. At QCon New York 2017 Shoup showed how to do an incremental migration of a monolithic application into microservices and several independent storage mechanisms for different domains.
In the book The Art of Scalability, a three dimensional scalability model (the AKF Scale Cube) is described where the three axis represents different types of scaling:
- X: Horizontal duplication and cloning
- Y: Functional decomposition and segmentation (microservices)
- Z: Horizontal data partitioning (shards)
The last step is the optimizing phase, and Shoup emphasizes that if reached it’s a sign of success. The goal is to keep the set of functionalities, but use less resources, maybe with fewer teams. We also have a longer time horizon, maybe 2-5 years.
Shoup concludes by stating that the reason for rearchitecting in this context is a positive thing:
Getting to rearchitect a system is a sign of success, not failure.
The business is doing well and you have the resources to do it. It’s not that you have to rearchitect, but rather that you get to do it.
