The article Agile and SEMAT-Perfect Partners describes what the Software Engineering Method and Theory (SEMAT) initiative adds to agile and how SEMAT can be used when organizations adopt agile.
InfoQ interviewed Ivar Jacobson and Ian Spence about what SEMAT is and how agile teams can use it, and on combining Essence from SEMAT with other software engineering frameworks like SAFe, CMMI, etc.
InfoQ: Can you describe what SEMAT is?
Ivar: SEMAT is a community of people interested in software engineering, which have agreed that we have several significant problems in the way we practice software development and that we see no simple way to fix these problems but to refound software engineering as a discipline.
But, let me say this already now, make no mistake SEMAT doesn’t compete with any other community of people - pre-agile or agile. SEMAT just support them and make them more successful.
InfoQ: Agile software development has its roots in the values and principles of the manifesto for agile software development. What are the roots of SEMAT?
Ivar: The agile values and principles are fully supported in everything we do within SEMAT; we are agile when working with methods and practices. However, SEMAT is not dogmatic on which practices to use in a particular context. The reality is that the software community has a huge portion of legacy software that needs to be maintained as well as it builds new software using agile and lean methods. SEMAT is agnostic to which methods people want to use, but it helps teams to move from old ways of working to new ways.
The paramount concerns and issues that challenge the field of software engineering are for example the reliance on fads and fashions, the lack of theoretical basis, the abundance of unique methods that are hard to compare, the dearth of experimental evaluation and validation, and the gap between academic research and its practical application in industry.
Against these challenges, a solution based on a solid theory, proven principles and practices is developed. At its heart the solution is a kernel including a collection of elements present in any software solutioning context. For instance, we always have a software system, we always have requirements (whether we document them or not), we always have a team. The kernel would provide the common ground reference to among other things help practitioners (e.g., developers, testers, project leads, etc.) to compare methods and make better decisions about their practices. The kernel would not just be used as a description, but it will help the teams in their daily work.
This far SEMAT has been very successful in getting a standard kernel, called Essence, and it is now developing practices using this kernel as a base. For instance practices like Scrum, TDD, ATDD, Continuous Integration, User Stories, Use cases are practices in the pipeline.
InfoQ: Can you give some examples of how the SEMAT kernel Essence can support agile teams when they want to improve the way that they develop software?
Ian: There are lots of ways Essence can support agile teams. These include helping them:
- Detect systemic problems early and take appropriate action.
- Measure their progress and health regardless of the method or practices selected.
- Be agile with methods, easily and safely evolving their set of practices as they inspect and adapt their way of working.
- Start conversations and brighten up their retrospectives by playing games with the cards
- Establish lightweight practice independent checklists to enable agile governance
- Evaluate the completeness of the set of practices selected, and understand the strengths and weaknesses of their way-of-working.
InfoQ: In both agile and SEMAT teams play an important role. Can you elaborate on that?
Ian: Both Agile and Essence recognize that it is teams of people that develop software and the importance of giving them the insight and tools they need to be successful.
Ivar: In Essence the team is described as an alpha, which means that it will have a state. Having a state allows us to measure the progress and health of the team. The team progresses through the following states: seeded, formed, collaborating, performing, adjourned. To reach a particular state a number of checkpoints should have been achieved. The value of these states and checkpoints are that the team clearly understands what needs to be achieved for the team to produce good software. When the team alpha is just ‘seeded’, nothing can be produced. The team needs to get to state ‘performing’ to be efficient. To progress the state some practices needs to be applied.
InfoQ: SEMAT supports teams to self-assess their way of working. Can agile coaches also play a role in this? How can they help teams when they want to use Essence?
Ian: Essence provides a number of tools and techniques that coaches can use to start conversations and help teams to inspect and adapt their way of working. By adding the Essence cards and games to their toolkits coaches can provide even more help to teams as they self-organize and take ownership of their way of working.
Ivar: One of the key differentiators of Essence is its very simple, intuitive graphical representations of practices. It uses cards of the size of normal playing cards, and they are used to play serious games. At the moment there are seven games to be found at Alpha State Card Games, one of them – Chase the State - has the objective to find out where the team is now and agree on where it should go next (in for instance the next sprint).
InfoQ: Could SEMAT be used in agile team retrospectives? Do you have examples of this?
Ivar: Work has been done at Carnegie Mellon West by Cecile Peraire and Todd Sedano comparing retrospectives in Scrum with retrospectives (they call it reflections) in Essence. Their finding is that they are complementary and not competing. In fact Essence provides as expected more fundamental retrospectives, on top of which practice-specific retrospectives are complemented with each new practice composed on top of Essence.
Ian: It certainly can for example I have used the Alpha state cards to start conversations and help teams to reflect on the strengths and weaknesses of their way-of-working and to assess the overall health of their projects. Using the state cards and the game boards can provide some much needed variety to the retrospectives and make a change from just collecting post-it notes.
InfoQ: There are many different agile methods and frameworks. Which ones can be combined with Essence? Are there any which you would not recommend to combine with Essence? Why?
Ian: The Essence Kernel is a completely different kind of framework from the various agile frameworks such as Scrum, SAFe etc. It attempts to capture the common ground shared by all software development endeavours rather than provide a description of any particular approach. The agile frameworks tend to be compositions of smaller practices all of which could be expressed in the Essence language. Basing the agile frameworks on Essence would allow teams to pick and choose the practices they need to solve their problems without signing up to any particular brand of agile.
Ivar: As an example several people have showed how Scrum is improved by complementing it with Essence. For instance Brian Elvesaeter is investigating how Scrum practices may be mapped to the SEMAT Kernel and Language. June Park, Paul McMahon and Barry Myburgh are writing a paper showing how Scrum practice can be described using Essence kernel and what values are added to Scrum by doing that (paper is not yet generally available).
InfoQ: Combining 2 frameworks like agile and Essence can be more complex and difficult for people to use them, because of the differences. Do you have any suggestions how people can deal with that?
Ian: The goal of Essence is to directly address this complexity and enable people to be able to share, compose, compare and use their practices in an easy and intuitive fashion. Using Essence will make the life of the practice author and methodologist a little more difficult and require more discipline on their part, which is very reasonable, but the end result should be easier for the team to use. For example represented in the Essence format will be no harder to understand than in any other format but with the added value of easier composition with other practices and useful cards to use during workshops and training events.
InfoQ: There are organizations which deploy the CMMI combined with agile to improve their software development and management capabilities. Are there things in SEMAT that such organizations could use to improve?
Ivar: CMMI is a set of practices focusing at the organizational level. Agile can also be described as a set of practices focusing at the development level. Agile and in particular Scrum is essentially a practice for backlog driven development complemented with a number of useful patterns (mini-practices) such as daily stand ups, retrospectives, etc. Understanding these practices with Essence as a foundation with its alphas, states and checklists help teams discuss important issues that might not otherwise surface. Practitioners can start applying Essence immediately on their project with very little start up training. Thus Essence doesn’t compete with CMMI or Agile. Instead it is complementary, you may say that they can both be powered by Essence.
Let me also add that CMMI as a whole is more than most organizations can swallow, but the essentials of CMMI - continuous improvement and measurements - are useful. Essence allows teams to have practices at different levels of detail. They can be tacit (just in the head of the developers), they can be lightly described focusing on the essentials, or they can be fully described – the 600 pages as in the CMMI guidelines.
InfoQ: What if an organization is currently using waterfall processes. Can SEMAT help them to become agile? How?
Ian: It certainly can as it enables them to adopt a practice-based approach preserving their good practices and combining these with the appropriate new agile practices making sure that they don’t throw the baby out with the bath water by, for example, losing important compliance and development practices that ensure the quality and integrity of their existing systems.
They can also make the transformation practice-by-practice enable a more iterative and incremental transition.
InfoQ: Many organizations are looking for ways to scale agile. Are there specific things that SEMAT has which can help them with this?
Ian: There are many ways that Essence can help companies scale agile. These range from 1) the use of the Essence concepts to create lightweight, governance practice that empower teams to innovate within the constraints of the larger system or enterprise can to 2) creating vibrant practice exchanges that support encourage a true learning organization.
Ivar: The learning organization flourishes since all practices are understood on top of a common ground – the kernel. Before we had a kernel every methodology had to describe all its practices, even practices not very different from what people already were using. Now, you don’t have to relearn what you already know and only learn what is really new. We can reuse our existing practices when there is no good reason to change.
InfoQ: You just published this article about SEMAT and agile. What's next?
Ivar: SEMAT addresses the interest of many different groups of people. The “Agile and SEMAT - Perfect Partners” was directed to the developer community. We have also published another paper “ Major League SEMAT - why should an executive care?” Many other papers have been published on the topic ‘A General Theory in Software Engineering’. Several papers are in the pipeline for instance experience papers from large organizations, a paper comparing Scrum with or without being powered by Essence. We are also now expecting to get more experience papers.
About the Interviewees
Ivar Jacobson, chairman of Ivar Jacobson International, is a father of components and component architecture, use cases, the Unified Modeling Language, and the Rational Unified Process. A contributor to modern business modeling and aspect-oriented software development, Jacobson is one of the leaders of SEMAT, working to renew software engineering as a rigorous discipline.
Ian Spence is chief scientist at Ivar Jacobson International, and has been involved in many large-scale agile adoptions over the years reaching thousands of people. He also led the work of the Essence kernel and has co-authored three software development books.