At Craft Conference 2015 in Budapest, Dan North and Jessica Kerr presented a keynote session which cautioned developers that complexity is often found outside of the code. The key messages included: identify and manage areas of complexity, such as UX, security, deployment and concurrency; treat learning as a first-class citizen; focus on working to sustainably minimise lead time to business impact; and nurture a supportive team, and contribute to the wider software development community.
Kerr opened the keynote by stating that during the emergence of the computer programming industry in the 1940s, complexity was inherent in the methodology used to write software. Grace Hopper, computer scientist and United States Navy rear admiral, created the first compiler which overcame some of this initial complexity. However, with modern software development Kerr and North both warned that we are now fighting multiple areas of complexity, such as UX, security, deployment and concurrency.
Kerr cautioned that many software systems are still designed using the layered ‘box-arrow-box-arrow-cylinder’ architecture, and the anti-pattern of aligning functionality within layers in akin to organising Lego blocks by their colour:
Assigning software functionality into a layered architecture is like organising Lego blocks by colour - it may look nice, but isn’t very useful when building things
A common anti-pattern accompanying the layered architecture is the shared single database. Kerr stated that any service sharing a database with another service within a service-oriented architecture (SOA) is not truly a service. North warned that in the same respect, the use of an enterprise service bus (ESB) tends to push complexity into the messaging middleware, and in effect moves complexity up from a shared database to a shared communication medium.
Microservices may be a useful pattern to address some of the architectural complexity, but care should be taken to avoid pushing complexity into the supporting ‘connective' functionality sitting between associated services, such as communication, fault-tolerance and monitoring. Lessons should be learnt from languages such as Erlang, which is used to build massively scalable real-time systems in the telecoms and banking domains:
Erlang has been providing [the connection between services] for literally 25 years.
As we get more and more sophisticated microservice implementations, each one grows their own crappy version of Erlang
Kerr and North stated that learning within software development should be a first-class citizen. The process of 'explore, evaluate, familiarise, understand and validate’ should be repeated as required. North briefly discussed the benefits of the Lean Startup methodology and the 'build-measure-learn' cycle, and stated that ‘nothing should be built without a reason’. The impact on the customer is vital, and the uncertainty of the potential cost of and revenue generated by a business idea must be explored early.
Discovery work could be undertaken to understand how to maximise revenue, or experiments conducted to determine the complexity of a software implementation. North suggested that rather than give a simple number for revenue or cost, a range (preferably using a probability distribution or confidence interval), should be given, as this more accurately reflects real-world scenarios.
Kerr and North both stated that the goal of a software development is to minimise lead time to business impact, and to do this in a sustainable fashion. ‘Lead time’ is a technical term, taken from the lean methodology, and is essentially the wall-clock time from a business requirement being formulated, to a solution being implemented which impacts the business in a positive fashion (which may, or may not, require the development of software).
The goal of software development is to sustainably minimise lead time to business impact
Kerr and North concluded the keynote by stating that nurturing a supporting team is vital within software development. It should not be a problem to point out bad things and not have a solution, as some people are naturally good ‘canaries’, and others are good at creating solutions. A diverse team is beneficial, and it helps to have a range of expertise across the areas of complexity identified at the opening of talks, such as UX, security, deployment and concurrency.
In addition to nurturing a strong team, the development of a supporting community is also beneficial, and much can be achieved by contributing at the this level. For example, submitting bug fixes and patches to upstream open source projects that are used by the team. Kerr and North closed the keynote by stating that developers should embrace uncertainty, and work together for the best results.