Neal Ford, Software Architect at ThoughtWorks, and Rachel Roumeliotis, Software Architecture Conference Program Chair at O'Reilly Media, opened the conference by stating that the role of a software architect frequently involves making complex decisions that will have long term consequences for the technical and business domains of an organisation. When this fact is combined with the wide-ranging topics involved in modern software engineering, the role requires constant learning and practice.
Jonas Bonér, founder and CTO of Lightbend, presented the opening keynote, ‘blah, blah... microservices...blah, blah’. Bonér stated that he believes that traditional architecture and platforms are now obsolete, and proposed that a monolith often attempts to do many things, and does not model real business domain very effectively. Although ‘micro’ may be a bad choice of word, the concept of single responsibility (from the Unix-inspired SRP philosophy) is valuable, and developers should design systems that ‘embrace the network’ by staying ‘mobile and addressable’. As all microservice-based applications are effectively deployed as distributed systems, the iterative approach of ‘guess, apologise and compensate’ for communication and action is a valuable pattern, as is the Saga pattern, which can correctly handle the eventual consistency that is often found in the real world.
‘micro’services may be a bad choice of naming, but the concept of single responsibility (from the Unix-inspired SRP philosophy) is very valuable
Next, the importance of ‘the evolution of evolutionary architecture’ was presented by Rebecca Parsons, CTO of ThoughtWorks. Parsons proposed that the software development industry initially began with low expectations that were met with monolith architectures and SQL-based data stores, but the agile manifesto has driven organisations to seek more. The introduction of design patterns and emergent design, in combination with the safety net of continuous integration, has allowed architects to continually “evolve design” to meet the constantly increasing demands on software. Parsons cautioned that any data migration performed as part of an evolution is always difficult, as data complexity grows in relation to time spent running in production. The original service-oriented architecture (classical SOA) was also a good first attempt at building composable systems, and now microservices may take this concept to the next level. However, the early and late majority of organisations require “existence proof” before technologies and techniques are considered ‘enterprise ready’.
The introduction of practices like continuous integration provided a 'safety net' when evolving design
For the first breakout session of the day, David Strauss, CTO at Pantheon Systems, presented ‘Death Star Security’ - the concept of reinforcing the edge of systems, but suffering catastrophe when this is breached. The talked was aimed at advanced practitioners, and as such, the security basics of L2/L3 firewalls, web application firewalls, OWASP Top 10, DDoS controls, and applying software patches and updates quickly, were only briefly covered. Strauss discussed the prevention of password leaks in depth, and suggested that techniques such as salting or peppering passwords, password stretching and multi-factor authentication (MFA) are vital for good security in relation to authentication. ‘Capability-based security’, such as signing requests or activity tokens, is also very useful for authorisation. Mandatory Access Control (MAC), such as that provided by SELinux improves security, and is easier to implement up-front in a project, rather than as an afterthought.
Strauss suggested that the handling of sensitive or personally identifiable information be delegated to trusted third parties, such as federated identity management services or payment gateways, which can reduce the internal management of security risks. Keeping ‘hands off data’ is also a valuable approach, and patterns like the ‘Black Hole API’ can be used for a service to irretrievably rid itself of data, meaning that even if the service is compromised in the future it cannot be coerced into providing the information once it has been sent. Data should be protected on the move and at endpoints, and tools such as HashiCorp Vault or Square’s Keywhiz can enable good practices such as secure key management and key rotation. Strauss cautioned that ‘security by obscurity’ is not a valid approach, and suggested that organisations practice ‘white hat’ style hack events to determine how secure their systems are.
Mike Amundsen, Director of API Architecture, API Academy, presented ‘Twelve Patterns for Hypermedia Architecture’. Amundsen began the talk by stating that hypermedia is a concept, not a product, and is essentially the language of the World-Wide Web (WWW). Humans typically communicate via messages, and hypermedia, with semantic elements and links to other entities, affords effective communication. Referencing Christopher Alexander’s classic ‘The Timeless Way of Building’ and the GoF ‘Design Patterns’ books, Amundsen suggested that applying patterns to messages would be beneficial for hypermedia and API system developers.
General design patterns included: ‘pass messages, not objects’, as the sharing of a model introduces coupling between services; ‘share vocabularies, not models’, as representation and relations are easier to standardise than objects; ‘use a representer’, to enable translation of messages (much like using the strategy pattern to determine which content-type to communicate with across two services); and ‘publish profiles’, as having a ubiquitous language for talking about APIs allows both humans and machines to know what is being talked about.
‘Basic principle patterns’ presented consisted of: ‘must ignore’, creating tolerant reader services that can be forwards and backwards compatible, and follow Postel’s Law; ‘must forward’ - don’t edit the content for other downstream services; ‘provide MRU’, including most recently used actions and document links; and ‘use idempotence’, where actions should designed to be idempotent. ‘Shared agreement patterns’ presented included: ‘use related’ - return related links to assist with navigation; ‘use navigation’, and provide a way for a human or machine to navigate through a procedure/process; ‘partial submit’ - allow incremental submission of data; and ‘state watch’, providing state to enable machines to behave autonomously, set their own goals and act accordingly.
Next on the agenda, Michael Nygard, VP at Cognitect, discussed the concept of ‘Architecture Without an End State’. The talk began with a suggestion that organisations often seek a perfect ‘steady state’ architecture, and attempt to implement this with a series of (often ill-conceived) cycles of change. The process is often simplified to determining the current state, specifying the end state and pitching a plan (“it always appears to be a three year plan”). Nygard suggested that the majority of organisations are complex systems, and in reality “the steady state is a superposition of ongoing waves of change”. Eight rules were presented to assist in developing an effective evolutionary architecture (and the process behind this).
- Embrace plurality - there will often be multiple systems of record, which represent the multiple domains within the business.
- Contextualise downstream - minimise the number of entities that all systems need to know about
- Beware granularity - do not create canonical data models
- Decentralise - centralised groups should define constraints, and not simply impose designs on other teams
- Isolate failure domains - modularity has a high economic value, and limits the ‘blast radius’ when failure does occur
- Data outlives applications - and...
- Applications outlive vendor integrations - create appropriate ‘anti-corruption layers’ between integration points, and explore the use of the hexagonal architecture (‘ports and adapters’) to minimise coupling
- Increase discoverability - make work visible (e.g. internal blogs or wikis), and facilitate an internal open source culture
Nygard concluded his talk by suggesting that the creation of an ‘engineering culture’ at the grassroots is beneficial. Architects cannot be prescient. The associated organisation, business and market are in constant flux, and therefore we should stop designing for the ‘end state’.
The final talk of the day, 'Leading Simplicity' was presented by Sam Lambert, Senior Director of Infrastructure at GitHub. This was a story of how the GitHub technical platform has evolved over the past several years, and the how stable and proven technologies have been chosen over other newer, competing technologies. GitHub is running primarily as a Ruby-on-Rails monolith, and is deployed multiple times a day. The GitHub team embrace the ‘remote first’ working pattern, and their engineers are distributed across the globe. The team communicates via IM, emails and video calls. The technical leadership values of the organisation are defined to favour: the Unix-inspired single responsibility principle for code and tooling; experiment regularly with new technologies, but spend ‘innovation tokens’ wisely; and simplicity:
It is easy to add complexity, but difficult to take it away...
Other values discussed included: favour pragmatism and predictability - this is the reason why the organisation stores the majority of their data within using the production-proven MySQL technology; avoiding the creation of complex systems by peer review and discussion - “elegance is a simple solution to a complex problem”; and that stability is vital - customers rely on GitHub to be constantly available.
More details on the O’Reilly Software Architecture conference that ran in New York City can be found on the conference website. Additional events in the O’Reilly Software Architecture Conference will take place this year in London 18th-21st October (CFP deadline 2nd May) and San Francisco 13th-16th November (CFP deadline 1st June), and the corresponding call-for-proposals web pages are now available.