At the microXchg 2016 conference, Fred George presented “It’s Not Just Microservices”, and argued that microservices can enable an organisation to ‘go faster’ and rapidly deliver business value. However, the implementation of microservices alone will not lead to success, and inhibitors to increasing business agility within the context of technology, process and the organisation must be identified and addressed.
George, principal consultant at Fred George Consulting, began the talk by stating that many of the current industry ‘hot topics’, such as microservices, Docker, and Lean Startup, all revolve around the desire to ‘go faster’ in regards to delivering business value. With reference to the Cynefin framework, it was suggested that many organisations are working on complex problems, where the relationship of cause and effect cannot be established a priori, and what “works one day may not necessarily work the next”. The desire to ‘go faster’ when solving complex problems requires that technology, process and the organisational ‘inhibitors’ be identified and removed.
The concept of technical inhibitors was introduced, and the discussion began with the exploration of ‘Valley Tech’. The exploitation of cloud technology, specialised databases, open source frameworks, and continuous releases by successful businesses based within the Silicon Valley area has frequently driven other organisations to question if they are being constrained by their own technology choices. In particular, the move from deploying applications within a bare metal data center to a virtualised cloud and container-based environment has reduced hardware lead times, and hence application deployment time, by several orders of magnitude.
George argued that the reduced lead time and improved flexibility of modern hardware platforms has contributed to the evolution of service design, and over time the industry innovators have moved from creating a single monolithic application to developing several large-sized service-oriented architecture (SOA) services. Ultimately this has led to the creation of many small single-responsibility (micro)services, which enables easier code changes, small iteration size and rapid experimentation.
Emerging technology, such as the use of a persistent event bus (utilising technology like Kafka), coupled with another tier of lightweight messaging brokers to reduce read load on the core message bus (powered, for example, by ZeroMQ), allows the creation of a loosely-coupled and performant application architecture. This concept has been explored further in an additional presentation by George entitled “Challenges in Implementing Microservices”
An example application within the domain of a car rental business was presented, where one service may send a message to the event bus requesting a rental quote based on customer data, and other services could respond with a bid. The loose-coupling and asynchronous message-driven architecture allowed new bidding services to be added easily (for example, if a developer believes they have created an improved pricing algorithm), and also promotes fault-tolerance, as the failure of a single bidding service does not necessarily cause a cascade failure within the system.
The discussion of mitigating technical inhibitors was concluded with the exploration of the benefits of open source, and it was suggested that by not utilising technologies such as the Netflix OSS stack or Docker, organisations may waste resources by building their own platforms. George also argued that for some use cases, functional languages may actually be a ‘silver bullet’, and cited anecdotal evidence from a rewrite of a large Java-based legacy system consisting of 130 thousand lines of code (KLOC) into a 4 KLOC Clojure-based application.
The talk continued with a discussion on how process inhibitors can be mitigated, and a key component of this is first understanding the problem the organisation is working on. Obvious and complicated projects may be solved with the traditional structured approach to software development. However, complex problems require a different strategy. The traditional ‘waterfall’ approach of creating a list of requirements that are divided into tasks for the development team at the beginning of the project is not a valid approach when working within a complex problem domain. Instead “experimentation drives innovation”, and focus for the developers should be moved from the task level up to the ‘feature’ level.
Stakeholders should discuss the business goal that is to be achieved directly with developers, and the measures of success and the ‘levers’ available to the development team should be clearly identified. It is essential to “measure what matters”, and bad metrics like ‘lines of code’ and ‘milestones met’ should be replaced with business-focused metrics such as sales volume, number of clicks and customer retention rates.
The discussion of mitigating organisational inhibitors began with a recommendation to avoid over-specialisation within the development teams. George has found that the theory suggesting specialists are more productive is frequently challenged by the overhead of communication and the unbalanced workloads that lead to queueing and delays. A case study of over-specialisation within an organisation was presented, and it was argued that the creation of niche job titles did not contribute to delivering business value to a project.
We had 50 IT professionals, 25+ titles and zero people understanding the project they were working on...
The suggested fix for over-specialisation within this use case was to define a short list of key technologies that were strategically important for the organisation, and create new titles based on the apprentice, journeyman and master model of skill recognition. The new job titles defined within the case study were presented as ‘graduate developer’, an individual that is not yet competent in a technology; ‘developer’, an individual who is competent in a key technology; ‘senior developer’, who is an expert in a key technology; ‘system developer’, someone who is competent is five or more key technologies; and ‘master developer’, an individual who is an expert in three or more key technologies.
In addition to job title simplification, it is essential to “fix the furniture” within an organisation’s workplace, with the goal being to allow teams to closely co-locate and communicate with high bandwidth. The implementation of “non-dedicated leadership” avoids creating unnecessary hierarchy, and enables flexibility for dealing with the rapidly changing challenges that emerge when solving complex problems. The final piece of advice offered was to “bring work to the team”, as the constant formation and disbanding of short-lived project teams is not effective because of the time it takes for a team to form into a productive group.
George concluded the talk by suggesting that although microservices are a very good approach to solving problems within a complex business domain, they cannot be leveraged to effectively deliver business value without dealing with other technical, process and organisational inhibitors. The final message for the developer-focused microXchg audience was that sysadmins/operations must be included within development teams (‘DevOps’), and the notion of fullstack developers embraced.
The video for Fred George’s microXchg talk “It’s Not Just Microservices”, can be found on the conference YouTube channel.
