Twenty Years of Patterns’ Impact

This article first appeared in IEEE Software magazine and is brought to you by InfoQ & IEEE Computer Society.

Good advice in software design is difficult to come by. General design principles can guide us, but reality tends to force tradeoffs between seemingly conflicting goals, such as flexibility and maintainability against size and complexity. Likewise, code libraries can go a long way in helping us avoid reinventing the wheel, but the vision of lesserskilled developers effortlessly wiring together readymade components remains fiction.

Design patterns have helped narrow this gap by documenting a well­working solution to a problem that occurs repeatedly in a given context. Instead of presenting a copyandpasteready code snippet, patterns discuss forces impacting the solution design. Examples of such forces are performance and security in Web applications: encryption and decryption algorithms improve security but introduce processing overhead. Ward Cunningham once described the best patterns as your older brother teaching you how to do something right.¹

Although patterns have become popular, their impact as a design technique is more difficult to quantify than the impact of a specific software product (which is what previous installments of this column have examined). This installment highlights both the breadth of patterns available after 20 years of patternwriting conferences and the depth of impact some patterns have had on open source software.

How It All Began

Building architect and philosopher Christopher Alexander inspired Kent Beck and Ward Cunningham to write their first small pattern language in 1987 for designing Smalltalk windows. In 1993, Beck and Grady Booch sponsored a mountain retreat in Colorado that triggered the formation of the nonprofit Hillside Group to foster pattern writing through the Pattern Languages of Programming (PLoP) conference series, which is celebrating its 20th successful year. PLoP conferences follow a highly collaborative style based on “shepherding” before submission and peerbased feedback workshops during the conference. Many successful pattern papers and books have emerged from this process.

In 1994, Erich Gamma and his colleagues’ Design Patterns catapulted the concept of patterns to a broad audience; as of this writing, it has sold more than 500,000 copies in 13 languages.² Two years later, Frank Buschmann and his colleagues produced the first volume of the PatternOriented Software Architecture series,³ closely followed by Martin Fowler’s Analysis Patterns.⁴ (Resources for further reading are available elsewhere.^5–9) The pattern format’s apparent success even tempted some authors and publish ers to gratuitously add the word “patterns” to their titles—the price of success, we feel. As of 2013, an Amazon search on “patterns” among computer and technology books yields more than 5,500 unique hits (including a minor number of false positives on visual pattern detection).

The early hype around patterns has settled, and people realize that patterns neither replace design skills nor solve all problems. Still, wellcrafted patterns provide valuable nuggets of relevant advice based on actual experience. Because learning by doing (learning from making mistakes) often isn’t an option for realworld projects, patterns can provide a way to learn from others’ experience (and mistakes, which can make good antipatterns).

No Sign of Pattern Fatigue

The widespread diversity of pattern domains makes determining the exact number of documented patterns difficult. Linda Rising’s The Pattern Almanac 2000 listed more than 1,000 patterns.¹⁰ The PLoP conferences, sponsored by the Hillside Group (www.hillside.net), have accepted more than 1,500 papers. The submission rate to those conferences has been constant, at approximately 100 papers per year. A conservative estimate of four patterns per paper, plus all the books and face design, mobile app development, adaptive systems, sustainable architectures, domainspecific patterns, metaarchitectures, workflow, faulttolerant systems, and security.

Many people accept the definition of a pattern as a proven solution to a problem in a context. In The Timeless Way of Building, Christopher Alexander clarifies that “the pattern is, in short, at the same time a thing, which happens in the world, and the rule which tells us how to create that thing, and when we must create it.”¹¹ Patterns present a reusable solution, provide information about its usefulness and tradeoffs, and encapsulate knowledge about proven best practices.

For example, many integration architectures include the Broker pattern, which acts as an intermediary between clients and servers and handles message routing, including serialization and deserialization of message content.² The Web’s communication infrastructure implements this pattern; workflow engines such as YAWL (Yet Another Workflow Language) also include rich implementations of this pattern.¹²

Many patterns are part of a pattern­ library; examples include ­http://developer­. yahoo.com/patterns and www. securitypatterns.org­. Many companies, including Amazon, Google, IBM, Lucent, Microsoft, Oracle, and Siemens, have written similar pattern collections, some of which are available in books and on websites. One example of such pattern collection is the IBM patterns for ebusiness catalog. Among many other recurring designs, it featured implementations of the Enterprise Service Bus in the context of IBM WebSphere products.¹³ Connected sets of interrelated patterns building on each other can form pattern languages, which support a generative, domainspecific development process.¹⁴ There’s even a pattern language for writing patterns.¹⁵

Enterprise Integration Patterns

Patterns’ success in software architecture and design has motivated attempts to integrate them more closely into programming tools to boost productivity and more closely align design and implementation mindsets. Alas, most attempts have stumbled because patterns are inherently a medium for documenting and passing knowledge between humans, not a programming construct. Still, some pattern languages have directly affected how software solutions are built.

Around 2003, the term Enterprise Service Bus (ESB) gained traction for describing the integration platform for serviceoriented architectures. ESBs route, filter, and transform XML messages between services; they represent the evolution of traditional enterprise application integration products that implement the Broker pattern. Ironically, although these products aimed to unify the Tower of Babel of disparate enterprise applications, no shared vocabulary to describe such solutions’ design was available.

(Click on the image to enlarge it)

Figure 1. Apache Camel core code growth over time. Linear growth of the Java code base suggests a stable committer community and sustained engagement. The amount of JavaScript jumped in 2009, but was later reduced, likely to the availability of libraries and frameworks.

Developers of open source ESB implementations that aimed to overcome this apparent gap soon realized that Enterprise Integration Patterns (EIPs) provide a coherent vocabulary of 65 patterns,⁷ ranging from integration styles to message routing and transformation. This can describe a large portion of meaningful ESB solutions. In the absence of an ESB industry standard, the open source projects adopted the EIP vocabulary as a de facto standard.

Open Source ESBs

Since the emergence of open source ESBs in 2005, almost a dozen open source ESB products have embedded the EIP language in their products’ domainspecific languages or programming models. The most widespread examples are Mule, Apache Camel, WSO2 ESB, Spring Integration, and OpenESB.

The nature of open source projects makes tracking code size relatively easy. However, tracking volume is relatively difficult because sales figures don’t exist and download numbers are often tainted by mirroring, caching, or automated downloads.¹³ Apache Camel comprises some 890 KLOC, created by 62 committers over the course of more than 18,000 individual commits over six years. The (Java) code base’s growth has been amazingly linear (see Figure 1), which suggests consistent engagement by a stable set of committers. Commercial adaptations of the open source core—for example, by Red Hat or Talend—augment the code base significantly with design or runtime management tools.

Download figures derived from Maven Central have averaged about 25,000 a month with a peak of more than 30,000 in July 2013—higher than YAWL, which reported about 1,000 downloads per month in 2010.¹³ Mule reports 3.6 million downloads on its homepage but doesn’t indicate whether all of them are individual userinitiated downloads.

Community engagement provides another insightful metric of open source success. Apache Camel community traffic quickly ramped up after its initial release in 2007 and holds steady at about 2,500 messages a month. This indicates a healthy community that collaborates to resolve issues and drives the product’s evolution. For comparison, Mule’s community page counts more than 150,000 members, and its forum counts 26,600 total posts.

Patterns as a Design Tool

After the EIP vocabulary’s integration into those products proved popular, some ESB projects went one step further and adopted the EIP pattern sketches as the visual language for their design studios. For example, developers can access the EIP icon language within the Red Hat Fuse IDE (integrated development environment) or Mule Studio. Unlike prior, somewhatcontrived “visual programming” attempts, the simple pipesandfilters architectural style of asynchronous messaging solutions makes this visual composition of patterns natural. Figure 2 shows a visual Camel route that directs an incoming message to one of two possible message endpoints via a message router. ESB developers can now think, design, communicate, and implement their solutions using the EIP vocabulary, even if they’re using different runtime platforms.

(Click on the image to enlarge it)

Figure 2. Creating messaging solutions using the visual pattern language from Enterprise Integration Patterns (EIPs)7 inside the Redhat Fuse IDE (integrated development environment). Messages arriving from a file-based message endpoint are routed by a content-based router to one of two potential message endpoints based on the city specified inside the message content. The content-based router pattern describes a reusable design for routing messages to a correct recipient based on message content.

Figure 3. Playing cards based on Enterprise Integration Patterns. The visual pattern language allows for an interactive, almost playful usage of the patterns. Each card displays the pattern icon together with the name and solution statement.

The EIP playingcard deck handed out at the inaugural CamelOne conference is likely the most creative pattern adaptation to date (see Figure 3). Each card shows a pattern from the pattern language together with the solution statement. It’s satisfying to see design patterns, which were created to improve human communication and collaboration, finding their way (literally) into the hands of architects and engineers in such an approachable, useful way. The statistics we presented here indicate that pattern languages have had a broad impact on the software design community over the past 20 years. Many research questions around patterns remain open, however. For example, good patterns aren’t always easy to find, which invites more work to organize and catalog the large body of existing patterns. We also envision pattern language authoring tools, perhaps using semantic wiki technologies. Finally, patterncentric design tools promise to be more appealing to the software engineers than mere componentandconnector drawing tools. Will the patterns community ever lose momentum? We don’t think so: existing pattern languages will continue to be implemented as domainspecific languages, just like EIPs. And domains still exist for which patterns have yet to be captured. For instance, typical conversations both between applications (via technical protocols) and between humans (such as via social networks) could be preserved in pattern form. The future of patterns is bright. We invite you to help shape it by furthering the development of a pattern tool or writing and sharing your design wisdom in pattern form.

References

¹W. Cunningham, “Tips for Editing Patterns,” Dec. 2002.
²E. Gamma et al., Design Patterns, AddisonWesley Professional, 1994.
³F. Buschmann et al., PatternOriented Software Architecture, Volume 1: A System of Patterns, John Wiley & Sons, 1996.
⁴M. Fowler, Analysis Patterns: Reusable Object Models, AddisonWesley Professional, 1996.
⁵J. Kerievsky, Refactoring to Patterns, AddisonWesley Professional, 2004.
⁶M. Fowler, Patterns of Enterprise Application Architecture, AddisonWesley Professional, 2002.
⁷G. Hohpe and B. Woolf, Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions, AddisonWesley Professional, 2004.
⁸E. Evans, Domain Driven Design: Tackling Complexity in the Heart of Software, AddisonWesley Professional, 2003.
⁹V. Vernon, Implementing DomainDriven Design, AddisonWesley Professional, 2013.
¹⁰L. Rising, The Pattern Almanac 2000, AddisonWesley, 2000.
¹¹C. Alexander, The Timeless Way of Building, Oxford Univ. Press, 1979.
¹²M. Adams, A.H.M. ter Hofstede, and M. La Rosa, “Open Source Software for Workflow Management: The Case of YAWL,” IEEE Software, vol. 28, no. 3, 2011, pp. 16–19.
¹³M. Keen et al, Patterns: Implementing an SOA Using an Enterprise Service Bus, IBM, 2004.
14. F. Buschmann, K. Henney, and D. Schmidt, “Past, Present, and Future Trends in Software Patterns,” IEEE Software, vol. 24, no. 4, 2007, pp. 31–37.
15. G. Meszaros and J. Doble, A Pattern Language for Pattern Writing, Hillside Group.

About the Authors

Gregor Hohpe is chief enterprise architect at Allianz SE and a member of the Hillside Group. Contact him at info@enterpriseintegrationpatterns.com.
Rebecca WirfsBrock is president of WirfsBrock Associates and treasurer of the Hillside Group. Contact her at rebecca@wirfsbrock.com.
Joseph W. Yoder is president of The Refactory, Inc., and of the Hillside Group. Contact him at joe@refactory.com.
Olaf Zimmermann is a professor and institute partner at the Institute for Software at the University of Applied Sciences of Eastern Switzerland, Rapperswil (HSR FHO). Contact him at  ozimmerm@hsr.ch. 

This article first appeared in IEEE Software magazine.IEEE Software's mission is to build the community of leading and future software practitioners. The magazine delivers reliable, useful, leading-edge software development information to keep engineers and managers abreast of rapid technology change.