Architecture and Design InfoQ Trends Report - January 2019

Both InfoQ and QCon focus on topics that we believe fall into the “innovator, early adopter, and early majority stages”. What we try to do is identify ideas that fit into what Geoffrey Moore referred to as the early market, where “the customer base is made up of technology enthusiasts and visionaries who are looking to get ahead of either an opportunity or a looming problem”. We are also on the lookout for ideas that are likely to “cross the chasm” to broader adoption.  It is perhaps worth saying, in this context, that a technology’s exact position on the adoption curve can vary. As an example, microservices is widely adopted at this point amongst Bay Area companies, but may be less widely adopted, and perhaps less appropriate, elsewhere.

This article provides a summary of how we currently see the “architecture and design” (A&D) space, which focuses on fundamental architectural patterns, realisation of patterns in technology frameworks, and the design processes and skills that a software architect must cultivate.

Notable changes since we last reviewed this topic include "microservices" moving into the late majority, but our internal discussions also highlighted that the closely related themes of "correctly designing distributed systems", and designing for reactivity and fault tolerance are not so far along the adoption curve. We may well be approaching the bottom of the microservice “trough of disillusionment” in respect to the Gartner Hype Cycle.

We have also speculated that there are areas of architecture that will never move along the adoption curve to early majority or late majority, and unfortunately, they include several highly-effective patterns -- such as event-sourcing/CQRS-based or Actor model-based systems -- which can provide highly effective solutions for certain organisations and business problems.

Although we feel the term "serverless" is slightly vague, we appreciate the potential to drive focus on designing modular, event-driven systems, and the possibility of automating several underlying operational platform concerns. On a related theme, we're also seeing more discussion around the need for evolutionary architectures that support future changes in requirements and constraints. 

We are increasingly seeing that the role of "architect" is becoming focused on soft skills such as technical leadership, in additional to modern technical skills like architectural pattern recognition and framework awareness, and dealing with the design of cross-cutting concerns.

For context, here is what the topic graph looked like for the second half of 2018. The 2019 version is at the top of the article.

The following is a lightly-edited copy of the corresponding internal chat-log between three of the InfoQ architecture and design topic editors, which provides for more context for our recommended positioning on the adoption graph.

Daniel Bryant, Independent Tech Consultant, Product Architect at Datawire, and InfoQ News Manager:

As a starter-for-ten, I'm thinking that HTTP2 moves to Early Adopter (EA), and HTTP3 comes in at innovator. GraphQL (and potentially gRPC) could be EA (or innovator?). I also think Chaos Engineer belongs in the DevOps queue. Microservices could be Late Majority (LM), as could BDD, DDD and TDD.

I would be keen to see "evolutionary architecture" appear somewhere -- potentially EA? And what about "Architect as technical leader" (emphasising the non-technical evolution of the role).

I'm interested in hearing what your thoughts are here, and also to ask if we need to move things around, or add or remove topics? 

Jan Stenberg, IT consultant with more than 25 years in building systems on both .Net/C# and JVM/Java platforms:

I think that Architecture & Design (AD) to some extent differs from the other topics we cover at InfoQ. 

In AD we don’t have a regular base of new or updated versions of architectures coming up. Instead existing ideas get popular again, and perhaps packaged and branded, due to new tooling, frameworks or smart architects that make them feasible.

We also have areas that may fit in two queues. At a higher level they may be suitable for AD and the more technical parts to another queue. I think that Serverless is an example of that, at a higher level it’s an important area for AD, the more technical parts may belong to the Cloud queue. Micro frontends and similar techniques are another example, is it AD or HTML5 and JavaScript?

There are also areas or architecture that I think will never end up in EM or LM, and unfortunately, they include several of my favourite architectures like event-sourced/CQRS-based or Actor model-based systems. I think they for the foreseeable future will be niche architectures used by a few. I’m not sure how we should deal with those, maybe they just fade away when architects and developers stop talking about them?

So, my take on the AD future (or my hopes maybe):

• Serverless. From the presentations I’ve listened to the last year, my impression is that this will be more and more automatic, with less work on the underlying infrastructure.
• Workflow platforms like Camunda. I think they are very important in microservices or distributed systems with more complex business logic.
• Event-sourcing/CQRS. I hope it will become more mainstream. Probably EA or EM.
• Event-driven architectures. EA or maybe EM.
• Actor Model / Reactive. Last year I talked to Vaughn Vernon about this and he believed it will become mainstream someday but I’m sceptical.
• Evolutionary architecture is interesting, and I think EA is correct.
• Chaos Engineering. Yes, normally it’s DevOps, a presentation discussing the subject from an AD perspective might be an exception.
• GraphQL and similar tooling is I or EA I think, replacing REST (hopefully also properly implemented).
• Architect as technical leader. I’ve been in meetings with various architects at home and for many of them their main work is getting the business/government domain experts to understand their own domain. But maybe it’s then more of an Agile queue story?
• Microservices is LM. (I think that Microservices soon will be “SOA of today”. Many are using them in a good way. Too many have put the label on a distributed monolith).
• DDD is late majority, but I hope it still is an interesting subject for InfoQ.
• BDD is late majority or rather “late minority”
• TDD still has some more or less interesting discussions. Too little or too much, unit tests or black box tests, ice-cream cone or what, but LM, at least.

When I meet architects, developers, domain experts and others in daily life, not at conferences and similar events, I too often realize that many of the concepts we are talking about here are unknown or very diffuse for them, which also make it hard for them to see the benefits of InfoQ. I remember a presentation I listened to about two years ago from a developer conference (I think it was in Canada) where Vaughn Vernon asked how many that knew about DDD and about half of the audience raised their hand.

When I started to write for InfoQ I wrote a fair bit about updates of frameworks and libraries that I thought added functionality that could influence architecture, but with time my writing has more and more focused on interesting blog posts and presentations, with just a few items about frameworks like Axon, Akka, and others that I think are closely related to a specific architecture. 

This kind of discussion would be great to have during a QCon conference.

Charles Humble, Editor in Chief at InfoQ:

I’m with Vaughn Vernon with regards the actor model -  and think it is likely to become mainstream - either directly or through something like messaging. In the JVM space Akka is doing a good job of popularising the approach, and messaging-based systems have been a popular way of doing something like actor-like in financial systems for a long time. 

Actors seem to be easy for people to grasp and reason about, and a good way of dealing with large-scale parallel work.  I'd love to see momentum build up around Ponyas an example of a modern, actor-based system, but I have to say I think that is unlikely.

With regards evolutionary architecture I was interested to hear Martin Fowler talk about this on the podcast last year and the paralel he drew to extreme programming.  I’m looking forward to reading the Thoughtworks’ Book on this.

Thomas Betts, Principal Software Engineer at IHS Markit and InfoQ Architecture Queue Lead:

At a very high level, I agree with most of what Daniel proposed. And Jan is correct that some of the architectural patterns fit well into the natural progression of the topic graph, while others will probably never move beyond the early adopter phase because they aren't widely applicable.

I do sometimes struggle with the overlap between A&D and the other topics on InfoQ, especially Culture & Methods. Blame Conway's Law, I suppose. So much about an architecture comes down to communication -- What are the external communication points entering and leaving my system? How will my internal services communicate with each other? How will my data be persisted and accessed? 

In many ways, a company's way of answering these questions, and the options they are able to choose from, will be based on their place on the adoption life cycle curve for both A&D and C&M. I'd like to say A&D is the technical side of the equation, and C&M is the non-technical, but that seems almost too simplistic. Also, the technical implementationwould probably fall into the development and/or language queues. A&D is in the squishy place in between, dealing with cross-cutting concerns, hopefully providing direction on how to implement the system.

I'll stop the philosophical rant, and just add a few specific discussion points. 

• Serverless - While I personally dislike the term, because it doesn't seem to mean anything specific, serverless needs to be represented, probably in EA.
• Reactive - Probably EA. I think reactive architectureswill become more common because developers are getting familiar with reactive programming, especially in JavaScript. That might be the tail wagging the dog.
• DDD - While DDD itself can probably move to LM, there are a lot of spin-off ideas that get closely associated with DDD, and are in I or EA. For example, Event Sourcing could merit mentioning as EA/LM. However, I don't think many those sub-topics warrant inclusion on the AD topic graph.
• Microservices - Right up there with "serverless" as a commonly misused or misunderstood term. I see this as moving into late majority in terms of widespread adoption, but might only be EA for a solid distributed architecture.
• Distributed Systems - I don't think it would work to have this as an item on the topic graph, because it's too broad. But I'd like to see us talking about designing with distribution in mind. Ideas like reactive and failure tolerance are critical to a robust distributed system in ways that weren't important in a monolith. That would be the argument for leaving chaos on the A&D topic graph.

I fully support having this discussion at a QCon!

The InfoQ editorial team is built by recruiting and training expert practitioners to write news items and articles and to analyse current and future trends. Apply to become an editor via the editor page, and get involved with the conversation.

About the Authors

Thomas Betts is a Principal Software Engineer at IHS Markit, with two decades of professional software development experience. His focus has always been on providing software solutions that delight his customers. He has worked in a variety of industries, including retail, finance, health care, defense and travel. Thomas lives in Denver with his wife and son, and they love hiking and otherwise exploring beautiful Colorado.

Daniel Bryant is leading change within organisations and technology. His current work includes enabling agility within organisations by introducing better requirement gathering and planning techniques, focusing on the relevance of architecture within agile development, and facilitating continuous integration/delivery. Daniel’s current technical expertise focuses on ‘DevOps’ tooling, cloud/container platforms and microservice implementations. He is also a leader within the London Java Community (LJC), contributes to several open source projects, writes for well-known technical websites such as InfoQ, DZone and Voxxed, and regularly presents at international conferences such as QCon, JavaOne and Devoxx.

Charles Humble took over as editor-in-chief at InfoQ.com in March 2014, guiding our content creation including news, articles, books, video presentations and interviews. Prior to taking on the full-time role at InfoQ, Charles led our Java coverage, and was CTO for PRPi Consulting, a renumeration research firm that was acquired by PwC in July 2012. For PRPi he had overall responsibility for the development of all the custom software used within the company. He has worked in enterprise software for around 20 years as a developer, architect and development manager. In his spare time he writes music as 1/3 of London-based ambient techno group Twofish, whose debut album came out in February 2014 after 14 years of messing about with expensive toys, and spends as much time as he can with his wife and young family.

Jan Stenberg is working as an IT consultant since more than 25 years in northern Sweden, experienced in building systems on both .Net/C# and JVM/Java platforms. His experiences range from large distributed and service based systems through web based and rich client applications down to hardware related software.