Author Jason McC. Smith speaks in-depth with InfoQ regarding his book, "Elemental Design Patterns", and the methodology behind it.
InfoQ: Let’s begin with an introduction. Would you please tell us a bit about yourself professionally and how you became interested in this topic of design patterns?
Jason McC. Smith: I started out programming when I was 12, on an old Apple ][+, like a lot of my colleagues. I found myself drawn to elegant tricks and techniques that could transcend the more mechanical programming styles of the time, and was always looking for that 'clever hack' that would make the system sing and dance beyond expectations.
Over time, as I moved into a more professional programming environment in college, I came to realize that these tricks and techniques, while elegant and novel, made collaborative software production miserable. I began to look for more expressive ways of injecting elegance and simplicity into a program. While completing my MS in Computer Science, I took my first object-oriented course, and was immediately hooked. It made sense to my mathematics and physics trained brain in ways that procedural programming never quite did.
From there, I took a position at Evans & Sutherland in Salt Lake City working on commercial and military flight simulators, bringing legacy C forward into a C++ environment. It was during this work that I was introduced to design patterns through the seminal Gang of Four book, which had just been published. They were the elegant solutions that enhanced collaborative software production that I had been waiting for. Immediately, they proved useful and immensely clear. From that point, I began researching ways to leverage patterns in novel ways.
InfoQ: The book's preface states that "This book assumes you have a passing familiarity with design patterns as a field but have not used or studied them in detail". How do you feel this book compares/aligns with Design Patterns? Do you feel your book should be read before or after reading Design Patterns? I see several references (UML, Gang of Four, etc) to modern software development, so it seems your book would benefit from a grounding in those areas first.
Smith: A passing familiarity with UML isn't a bad thing before diving into this book, and I suggest a couple of quick references in the Preface if the reader has not encountered it before. For other connections to modern techniques, such as refactoring, etc, I provide enough material to whet the reader's appetite if they have not yet run across these concepts, and pointers to further reading. For readers already working with those techniques, I show how the EDPs can be used to enhance what they already know.
For those who are not familiar with the Gang of Four publication, I see the EDP book as rather like an extended introduction to the Design Patterns text. As an educator, I have seen students struggle with understanding the finer points of design patterns when thrown into the GoF book. Too often, students take the position that since they are told to use these solutions, they will use them... but rarely do they understand them in a deep way that is necessary for using patterns effectively. Too often, students will blindly copy and paste code implementations from a patterns text without putting much consideration into whether the pattern can or should be modified for their particular problem.
The GoF design patterns are simply too large and complex for beginning programmers to digest and truly understand. Since then, the design patterns literature has reached for larger and more abstract patterns, which is a fabulous endeavor, but it's left the GoF patterns as the 'ground floor'. Unfortunately, this 'ground floor' is about three stories too high for anyone to walk into from the street. The EDPs are the ground floor, the elevator, and the stairs to get you to the GoF design patterns in an easy to understand, methodical manner. From there, the sky is the limit.
This isn't to say, however, that those already familiar with the GoF patterns can't find plenty in this book to spark their interest. The decompositions of the GoF patterns using the EDPs can be highly illuminating, as they bring to light many similarities between the larger abstractions that have been well hidden until now. It can be quite an eye opener to learn the fine points of the composition process and how it can be applied in our everyday tasks. New ways of looking at how we structure our systems and the thoughts behind them will almost always provide new insights. And of course, the EDPs alone are just plain rather fun to read. It's rather like revisiting old friends, and finding out new things about them that you never realized.
InfoQ: You describe one of your projects, System for Pattern Query and Recognition (SPQR) and state it was designed to identify design patterns. Can you elaborate a bit more? Was the identifying design patterns seen as a way to produce better documentation, or did the process of producing documentation cause the use of design patterns to be identified?
Smith: SPQR started as an effort to document instances of known design patterns used in source code. Given a large body of code, SPQR could give you a list of the GoF patterns used in the system, and how they were implemented. The project got its start from an experience at Evans & Sutherland in 1997. I worked with a group responsible for one library in a state of the art commercial flight simulator system. One developer meeting, the application group thanked a number of us for a piece of functionality we had added to the system. Turned out, however, that not a single library group had added the behavior... it was an emergent behavior from three libraries working in concert to produce an example of the Decorator pattern, accidentally. The sad thing was, it took us over 200 hours of dedicated work by the domain and implementation experts to find this out. I thought "There has to be a better way..." and SPQR was born.
While SPQR started from this desire to document existing software, we found many more applications for it as time went on. In the process of trying to train SPQR to find design patterns, the EDPs and underlying rho-calculus formalism were created, enabling new approaches to education and automated analysis. For instance, a side project involves mining large software repositories for previously unknown design patterns, and using the EDPs to build an understanding of what these abstractions represent.
InfoQ: In Chapter 4 ("Working with EDPs") there is a discussion about how identifying design patterns in code benefits the documentation process given that design patterns are widely understood. Thus developers can focus on the darker, less-common areas of the code base. Has this shown an easily measurable (either quantitatively or qualitatively) benefit in your experience?
Smith: We were just getting into this kind of quantitative research at IBM Watson Research when I was there, but the work was not finished prior to my leaving for my current position at TSRI. I'd love to resurrect it however!
In the anecdotal realm, though, yes, this definitely lets developers concentrate on the aspects of their systems that are unique to their implementation. By being able to identify where design patterns are used in your software, you can mentally set those aside as well-documented, well-understood portions of your system, and concentrate on the sections where you bring the most value. I'm not sure I'd call them darker areas, but perhaps not yet quite as illuminated.
InfoQ: You introduce Pattern Instance Notation (PIN) in chapter 3 of your book. Can you provide a brief description for our readers?
Smith: Design patterns are rather esoteric beasts, but they have specific traits. They all have a name, for instance, and a list of roles that must be fulfilled by participants for them to be implemented. An instance of a design pattern is a concrete representation of a design pattern, and it describes how a single combination of software elements such as classes, methods, and fields come together in a unique way to form a larger abstraction.
The PIN allows a designer to depict individual instances of design patterns in a graphical notation. The basic entity in the PIN is the PINbox, a rectangle containing the name of the pattern, and the roles arrayed around the outside. PINboxes can be dropped into a variety of other notations, such as UML, to indicate the presence of design patterns independent of the underlying basic notation, and arrows used to connect the roles of the PINbox to the entities in the other notation that fulfill them.
PINboxes can be used on their own, however, to illustrate only instances of design patterns, and how they relate to and interact with one another. This is one way that the hierarchical composition of design patterns from the EDPs is demonstrated in the book. There's quite a bit more to PIN, including expanding and collapsing PINboxes to show varying levels of detail, and I recommend your readers consult the book for the remainder.
InfoQ: Do you feel that UML's complexity hinders understanding of design patterns such that it creates the need for PIN?
Smith: Definitely. The original UML was a simple, concise system for describing software implementations in object-oriented systems. As time has gone on, it has undergone an incremental expansion and aggregation into other realms, and only briefly touches on the abstract nature of design patterns. While a couple of notations, such as pattern: role tags and collaboration entities have been produced, they are additive to UML. In other words, you need a comprehensive UML diagram of an implementation (or very concrete design) before you can start to discuss the abstractions behind it. This is usually backwards. A designer starts with ideas, abstractions, and then uses those to provide the seeds of a design. PIN provides designers with a unique notation for noodling over those abstractions directly, wiring them up, coming up with a preliminary design from that basis, and only then needing to start thinking about the implementation.
InfoQ: The comparison of alchemy vs chemistry is made early with Elemental Design Patterns representing a move towards chemistry. Do you see EDPs as equivalent of the periodic table?
Smith: Yes, very much so. The periodic table brought a sense of order - and more importantly, prediction -- to alchemy and transmuted it to the science of chemistry, if you'll pardon the pun. Prior to the periodic table, there was a gross understanding of the effects of chemical reactions, and an only vague comprehension of the underlying processes and relationships between the elements. Mendeleev’s main contribution was that, by organizing the known elements into a relational representation, the gaps in the knowledge became as important as the known information. By looking at where an element should be in the table, chemists were able to predict the properties of the unknown element, and refine their search for it.
The EDPs underwent a similar process. Initially, a handful of elemental patterns were identified by investigating existing higher-level design patterns and looking for similarities. This is normally how design patterns are found, by identifying recurring solutions, so it makes sense in this context. Over time, relationships between the identified smaller patterns started to form. These were formalized in the rho-calculus, which provided a larger framework in which to place the already identified elemental patterns.
At that point it was a matter of looking at the gaps, predicting the behaviors, expected problems in which they would be useful solutions, and such, and then verifying their existence in already implemented software. The EDP formalisms provide more than a descriptive system, they provide a predictive system. In that way, they are very much analogous to the periodic table.
While I can't adequately portray the process through which the EDPs grew, I can draw the analogies for further researchers. Remember, the EDPs provided in the current book are only one quarter of the conceptual space. There is a much richer, much larger EDP playground to investigate, all from the same formal basis. While I wrote the book to reduce or eliminate the need for understanding the formalisms, they are the key to extending the research. I invite anyone interested in doing so to contact me, it's a wild world of widgets.
InfoQ: Anything else you would care to add regarding your book and your goals for it?
Smith: I have to say that I am thrilled to see the interest growing in this material. The EDPs are, I believe, a foundation for a new view on what we all do every day in our practices as software engineers. Designed originally to enable automated software analysis systems, the EDPs have grown into a tool for teaching students as well, to give them an easier approach to the richness of the larger design patterns literature.
I would encourage anyone, however, who wants to gain a greater appreciation for the gears and cogs of our discipline to take a look at this book. Even the simplest concept of programming has a tremendous wealth of distilled information behind it, and the more we familiarize ourselves with the tribal wisdom of our field, the faster our progress can be building on that foundation in novel ways. Innovation is driven by knowing what has come before, so we can concentrate on what to do next.
About the Book Author
Jason McC. Smith received his Ph.D. in Computer Science in 2005 from the University of North Carolina at Chapel Hill, where the Elemental Design Patterns were born as part of the System for Pattern Query and Recognition (SPQR) project. He then spent four years at IBM Watson Research with the opportunity to apply the lessons of SPQR and the EDP catalog and compositional approach to immense bodies of software, both legacy and modern. Dr. Smith is currently Senior Research Scientist at The Software Revolution, Inc. in Kirkland, WA, where he continues to refine the EDP catalog and look for ways to enhance the company's goal of automated modernization and transformation of legacy systems.