This article first appeared in IEEE Software magazine and is brought to you by InfoQ & IEEE Computer Society.
In software engineering, a tight relationship exists between nonfunctional requirements (NFRs) and software architectures (SAs). As early as 1994, Rick Kazman and Len Bass asserted that SA is intimately connected to NFR achievement.1 This idea has pervaded software development over the years and explains why development projects invest a lot into fulfilling NFRs.2
This general statement becomes more concrete when we consider how the concept of SA has evolved from a simple structural representation to a decision-centric viewpoint.3 In this perspective, NFRs serve as selection criteria for choosing among myriad designs and implementations.4 For instance, we can justify choosing a layered architectural style in terms of maintainability or portability, or choosing a particular database technology in terms of efficiency.
Software architects must continually deal with NFRs as part of their SA design responsibility. They must know a systemfs NFRs and how architectural decisions affect the NFRsf fulfillment. Here, we present an empirical study that uncovers relevant practices for software architects dealing with NFRs during decision making. This study is based on a two-part survey. First, we analyzed NFRs from an engineering perspective, specifically in relation to three requirements-engineering activities: elicitation, documentation, and validation. Then, we investigated how NFRs influence architectural decision making.
The Survey
We conducted semistructured interviews regarding a single software development project in which the respondents had participated as software architects. Compared to other qualitative research strategies such as structured questionnaires, semistructured interviews provide more flexibility and let us better investigate interesting issues that appear during conversations. Furthermore, considering a single project instead of a general perception of the architectsf rationale lets us better interpret and assess contextual information.5
The Respondents and Their Organizations
The survey involved 13 respondents at 12 software-intensive organizations covering a variety of business areas and application domains (see Table 1). The organizations comprised three types of business:
- Software consultancy companies that primarily perform software development tasks for different clients.
- IT departments that usually perform or outsource software development tasks to meet the organizationfs internal demands.
- Software houses that develop and commercialize specific proprietary solutions.
The projects were diverse in terms of functionality and team size.
Although all the respondents performed as architects in their respective projects, their organizations had no specified software architect role. Instead, the organizations chose architects for each project on the basis of technical knowledge or experience. In all but one case, the respondents also played other roles (project manager, consultant, or developer).
(Click on the image to enlarge it)
/filters:no_upscale()/articles/non-functional-requirements-in-architectural-decision-making/en/resources/table-large.jpg){kind=link}
Execution
We created an interview guide and tested it with two researchers and two software architects to ensure its effectiveness. We then sent the guide to the respondents in advance, which let them become familiar with the topic and choose which project to discuss. We conducted the interviews face to face; each interview took about one hour. We recorded the interviews and prepared them for analysis by manually transcribing the audio records into text documents. Then, the respondents validated them. In a few cases, we explicitly requested clarification of certain aspects. We used NVivo software to assess the collected data. The team met several times to generate categories by grouping sentences or phrases describing the same idea, action, or property. Finally, we analyzed the data according to the organizationsf and projectsf characteristics.
Limitations
We knew our sample wasnft random and therefore didnft necessarily represent the broader population of software development projects. So, we tried to mitigate possible bias by letting the organizations choose respondents and letting the respondents choose projects. We recognized that respondents might be inclined to select the more successful projects. To mitigate this, we explained that our study didnft aim to analyze best practices but to learn how things are done. We also stated that responses would be kept confidential. Most of the organizations were small or medium sized, and all were in Spain. These factors certainly might have affected the surveyfs results. Furthermore, most of the projects dealt with noncritical domains. However, because we sought to reveal industrial practices used in the analyzed projects rather than to formulate general theories, this isnft a fundamental weakness.
How Architects Deal with NFRs
We asked software architects for details about how they elicited and documented NFRs and how they later validated systems against them. This is important for understanding the context of architectsf decision making in the projects. Details on this part of the study appear elsewhere,6 and the list of questions is here. Because we implemented semistructured interviews, the questions served as a guideline, but the interviews evolved depending on the conversations. Topics such as requirements elicitation, documentation, and validation naturally emerged during the interviews.
Who Elicits NFRs?
Requirements elicitation obtains systems requirements from stakeholders and other sources and refines them. Both researchers and practitioners consider it one of the most challenging parts of requirements engineering. Numerous techniques (surveys, creativity workshops, and so on) help express requirements precisely and unambiguously.
These techniques assume that stakeholders on the customer side (end users, managers, and so on) contribute the most to elicitation. Some respondents acknowledged this assumption when referring to functional requirements. For example, respondent A said, “[The business analyst] writes a detailed document reflecting all the [functional] requirements specified by the customer.”
However, this assumption didnft hold for NFRs: in 10 projects, the software architect was the main source of the NFRs. Some clients never mentioned NFRs. Respondent E said, g[The client] never mentioned [a requirement] that webpages could not take more than two seconds to load, but he complained about it afterward.h The other clients provided only broad indications, usually in the form of cost or efficiency constraints. Respondent L2 said, “The client mentioned a basic [NFR], and we added others based on our experience. ”
This number exceeds the already high rate (60 percent) Uwe van Heesch and Paris Avgeriou reported of architects significantly involved in requirements elicitation.
The organizations of respondents D, H, and I were the only ones with client-led NFR elicitation. Interestingly, they were also the only ones in which the respondents were working on outsourced projects (managed by an aerospace company, a software company, and a bank). This situation is due to the subordinated role of the respondentsf organizations in these projects. Even in these cases, however, the architects played an active role in defining the NFRs. Respondent D said, “Our client was an aerospace system department. Therefore, all the NFRs were very well defined. We also added other NFRs based on our experience.”
How Are NFRs Elicited?
NFRs’ elusive nature can make them difficult to elicit up front. In accordance with this general belief, all respondents agreed that NFR elicitation is an iterative process that expands along the system life cycle. This could be because part of the systemfs expected behavior might not be uncovered until a certain milestone (typically, a prototype) has been achieved. Respondent J said, ”We determined first some relevant NFRs - for example, compatibility with other systems - and then developed a prototype and analyzed alternatives.”
Furthermore, we can't consider NFRs complete after the first development. They’ll require corrective maintenance to fix some misbehavior that wasnft originally expected. Respondent K said, ”In relation to efficiency, we had to make changes because the necessary level of service was not specified at the beginning of the project.” Such activity is justified. Some NFRs, such as those related to security, canft be completely checked until the system is deployed in its expected environment and unexpected conditions arise.
How Well Are NFRs Documented?
Academics and standards organizations have proposed many notations and templates to write system requirement specifications to make documentation more efficient. However, nine of the 13 respondents acknowledged that they hadnft documented the NFRs. Respondent H said, ”[Functional requirements] came in UML, using conceptual models and use cases, but there was no mention of NFRs.” Some respondents said that documentation is only necessary if the client or the critical nature of the domain require it. The four respondents who explicitly documented their NFRs used different methods. Respondent Dfs organization uses a domain-specific language. ”Since we work in the field of aerospace, our NFRs had to be clearly stated and verifiable. We have special templates, and we use different techniques from other engineering disciplines, such as risk models, failure trees, etc.” Two respondents used natural language with a certain structure. Respondent B used Volere templates that provide a high degree of structure to the requirements; Respondent K used plaintext classified under the ISO/IEC 9126 quality model.
Respondent J simply wrote a plaintext document. Respondents B and D were the only ones who maintained up-to-date requirement documentation; J and K documented only the initial NFRs. Respondent K said, "At first, we wrote down some initial ideas for NFRs in natural language, c but afterward, we did not keep track of any of them or of any other NFRs arising during the design process." It seems natural to think a relationship exists between measurability and continual (or at least regular) documentation updates, but confirming this link will require further studies.
So, we can see that NFRs are more tacit or even hidden than documented. When they are documented, their accuracy and timeliness are seriously compromised. This situation can be explained in terms of cost and benefit. Respondent C stated it plainly: “I rarely appropriately document my projects, basically because it costs money.” If practitioners donft perceive a clear benefit from shaping NFRs into fully engineered artifacts, NFRs will remain elusive.
How Are Systems Validated against NFRs?
Validating system behavior against elicited NFRs is difficult. Because every NFR differs, the appropriate methods differ. Nevertheless, 11 respondents claimed that all NFRs had been satisfied by the projectfs end, although there was always room for improvement. However, when we asked how they validated the NFRs, their answers were vague. Respondent D said, ”Compliance with some [not all] NFRs is only informally discussed with the client, since it is not easy to test.” So, the need exists to distinguish the perception of NFR compliance, which 11 respondents (85 percent) achieved, from real validation, which eight respondents (61 percent) achieved for only some types of NFRs. (This is in line with a previous study that reported 60 percent.8)
Eight interviewees performed some validation, but against only one to three NFRs. They considered only a few types of NFRs:
- Performance efficiency. Respondent H said, ”We ran load and stress tests to evaluate performance.”
- Correctness. Respondent A said, “For each hour of coding, we spend one hour testing for bugs.”
- Usability. Respondent K said, ”We made a prototype just to ensure client satisfaction with the interface.”
- Reliability. Respondent J said, ”We have enforced some errors to see what happens and [to] control loss of data.”
No one mentioned one highly relevant type of NFR: security. Respondent F represented an extreme case of nonvalidation: ”We wait for the client to complain. He will notice when something goes wrong.” Although this response is unsatisfactory, it shows again (as in the case of documentation) how budget and time considerations can interfere with ideal engineering practices.
In contrast, respondent D used formal techniques based on statistical analysis and simulation to check the systemfs reliability. Of course, this validation approach was likely because the project involved an information system for aerospace, a critical domain. This observation aligns with a previous survey that suggested that the evaluation approach depends on the evaluation goals.9
One finding of our study that might align with previous results is the link between documentation and validation. As Andreas Borg and his colleagues said, ”When expressed in non-measurable terms testing is timeconsuming or even impossible.10”Only two respondents expressed the NFRs in a measurable form, which could be one reason for the overall low level of validation.
How NFR s Influence Architectural Decisions
Not surprisingly, the respondents all agreed that NFRs influence their decisions. The details of their answers revealed some nuances, though.
Types of Decisions
NFRs drove four types of decisions.
Architectural patterns.
Given the types of projects, a layered architecture was the natural option for most of the respondents, although some of them explicitly justified the decision. Respondent J said, ”We used a layered architecture to support later changes.”
Implementation strategies.
Several types of requirements might require strategies at a detailed architectural level. The situation could be a general design decision. Respondent L1 said, ”We opted for single sign-on to improve the integrability of different subsystems.” Or, it could be a detailed decision about some components. Respondent A said, ”The tables of a database were duplicated because the access time was too high.”
Transversal decisions.
Some NFRs carry the weight of affecting the whole architecture. Respondent L1 said, ”We prefer to use technologies we have already mastered.” One recurrent matter is the use of third-party components and especially open source software (OSS). Respondent D said, ”We wanted to have access to code for maintainability reasons; thus, we opted for OSS solutions.”
Technological platforms.
NFRs might be satisfied by technological choices in the database, the middleware, and so on. In this case, they can be system-wide. Respondent K said, ”We need high availability, and this requirement was ensured only by Oracle.” Or, NFRs can be more localized. Respondent H said, “One of the queries was implemented directly in JDBC [Joint Database Connectivity] instead of Hibernate due to efficiency reasons.”
Diverse Decision-Making Processes
One particular aspect of decision making that emerged from the survey was the intertwining of technological and other decisions. We found three types of responses. Four respondents said that nontechnological decisions come first. Respondent C said, ”The architect should manage which technologies are appropriate to cover the previously designed logical structure.” On the other hand, four said that fundamental technological decisions come first and subsequent decisions should adapt to it. Respondent H said, ”We had some limitations from the client-for example, architecture based on OSS and Java.” The remaining five argued that both types of decisions overlap and feed each other, which could be considered a local twin peaks model11 at the SA design level.
Amount of Influence
We asked the respondents which NFRs they took into account when making architectural decisions. We consolidated their answers using the ISO/IEC 25000 quality standard as a unifying framework.
Explicitness.
Clearly, architects expect certain NFRs, even if they havenft explicitly identified them as a requirement. Respondent I said, ”I would never think of a system that it is not secure.” Often, these tacit NFRs percolate into architectsf minds owing to the current features of the technologies and platforms that they usually adopt. Respondent E said, ”We didn’t think about the security of the documents because it is done by the management system of SharePoint.”
Source.
Some requirements come directly from the development team or architect. Respondent B said, ”We were the ones that would maintain the system, so it was important for us to ensure its maintainability.” These NFRs are closer to the decision-making process than those from the client because the technical staff already thinks in terms of the solution, compared to the clientsf problem-oriented thinking.
Nontechnicality.
Nontechnical NFRs donft refer directly to the softwarefs intrinsic quality but to the context of the system under analysis, such as license issues or cost.12 Architects consider these essential; approximately 40 percent of the NFRs that respondents considered in their projects were nontechnical. In some cases, they considered these their highest priority. Respondent J said, ”Money rules, and everything has to be adapted to it.”
Importance.
We asked the respondents which types of NFRs they considered most influential. Licensing issues, usability, reliability, performance efficiency, and maintainability were mentioned most, whereas only two respondents mentioned portability. We cross-checked this information with the decision examples the respondents gave during the interview; performance efficiency and maintainability NFRs drove most of the examples.
Terminology issues.
We experienced terminological difficulties when talking about NFRs with the respondents. Some of them required additional clarification (for example, about the meaning of gavailabilityh or gaccuracyh). Others used terms inappropriate in the given context (such as gergonomich to mean gusableh). Some even used terms with incorrect definitions (for example, “Maintainability is very important, because when something is working, we canft make changes”). Another recurrent problem was the indiscriminate use of “performance” and “efficiency,” which required further questions from our side. ISO/IEC 25000 defines gperformance efficiencyh as “performance relative to the amount of resources used under stated conditions.”13 This definition could help resolve this confusion.
To check whether the observations from this study hold, we’re conducting a new study with several big IT corporations. We expect to see improvements in the level of documentation and validation practices, and maybe differences in the importance of the different types of NFRs. We’d also like to explore how the explicit representation of the relationship between NFRs and architectural decision making (which trade-offs are considered, which options are discarded, and so on) influences the final choice of system architecture. For a look at other research related to ours, see the sidebar.
Acknowledgments
We thank the study participants for their time and valuable contributions. The Spanish project TIN2010-19130-C02-01 partly supported this research.
References
1R. Kazman and L. Bass, Toward Deriving Software Architectures from Quality Attributes, tech. report CMU/SEI-94-TR-10, Software Eng. Inst., Carnegie Mellon Univ., 1994.
2F. Buschmann, “Value-Focused System Quality,” IEEE Software, vol. 27, no. 6, 2010, pp.84.86.
3P. Kruchten, R. Capilla, and J.C. Duenas, “The Decision Viewfs Role in Software Architecture Practice,” IEEE Software, vol. 26, no. 2, 2009, pp. 36.42.
4L. Chung and J.C.S. do Prado Leite, ”On Non-functional Requirements in Software Engineering,” Conceptual Modeling: Foundations and Applications, A.T. Borgida et al., eds., Springer, 2009, pp. 363.379.
5R. Conradi et al., ”Reflections on Conducting an International Survey on Software Engineering,” Proc. Intfl Symp. Empirical Software Eng., IEEE, 2005, pp. 214.223.
6D. Ameller et al., gHow Do Software Architects Consider Non-functional Requirements: An Exploratory Study,h Proc. 20th IEEE Int‘l Requirements Eng. Conf., IEEE, 2012, pp.41.50.
7U. van Heesch and P. Avgeriou, “Mature Architecting-a Survey about the Reasoning Process of Professional Architects,” Proc.9th Working IEEE/IFIP Conf. Software Architecture (WICSA 11), IEEE CS, 2011, pp.260.269.
8R.B. Svensson, T. Gorschek, and B. Regnell, “Quality Requirements in Practice: An Interview Study in Requirements Engineering for Embedded Systems,” Requirements Engineering: Foundation for Software Quality, LNCS 5512, Springer, 2009, pp. 218.232.
9M. Ali Babar, L. Bass, and I. Gorton, “Factors Influencing Industrial Practices of Software Architecture Evaluation: An Empirical Investigation” Software Architectures, Components, and Applications, LNCS 4880, Springer, 2007, pp. 90.107.
10A. Borg et al., “The Bad Conscience of Requirements Engineering: An Investigation in Real-World Treatment of Non-functional Requirements,” Proc. 3rd Conf. Software Eng.Research and Practice in Sweden (SERP 03), CSREA Press, 2003, pp. 1.8.
11B. Nuseibeh, “Weaving Together Requirements and Architectures”, vol. 34, no. 3, 2001, pp. 115.119.
12J.P. Carvallo and X. Franch, “” Extending the ISO/IEC 9126-1 Quality Model with Non-technical Factors for COTS Components Selection,h Proc. Intfl Workshop Software Quality (WoSQ 06), ACM, 2006, pp. 9.14.
13Software Engineering.Software Product Quality Requirements and Evaluation (SQuaRE).Guide to SQuaRE, ISO/IEC 25000, International Org. for Standardization, 2005.
Related Work in Non-functional Requirements
We tried to consolidate our findings with previous empirical studies of non-functional requirements (NFRs), but not many exist. Richard Svensson and his colleagues reported on 18 studies somewhat related to our proposal.1 However, none of these studies addressed the relationship among NFRs and architectural decisions in research questions; this clearly differentiates our study. Although most of these studies focused on NFR aspects, others have explored software architecture (SA) issues in more depth; Uwe van Heesch and Paris Avgeriou specifically addressed relationships among NFRs and SAs.2
Some of our observations match previous findings - for example, software architects performed other duties. Other observations hadnft been previously reported. For example, the participants in our study perceived NFRs complied even though they performed only light validation. A few of our observations contradict previous studies.for example, the NFRsf measurability was poor, which contradicts some of the reported cases.3 We also analyzed the influence of the different types of NFRs. We found some similarities to previous research.for example, Jose de la Vara and his colleagues also named efficiency and usability as the most important factors.4 However, we had difficulty comparing the studiesf results with the necessary rigor. For one thing, the studies used different NFR classification schemas. In addition, the roles involved were often different, so the NFR types differed, too.1
References
1R.B. Svensson, M. Host, and B. Regnell, gManaging Quality Requirements: A Systematic Review,h Proc. 36th Euromicro Conf. Software Eng. and Advanced Applications (SEAA 10), IEEE CS, 2010, pp. 261.268.
2U. van Heesch and P. Avgeriou, "Mature Architecting.a Survey about the Reasoning Process of Professional Architects", Proc. 2011 9th Working IEEE/IFIP Conf. Software Architecture (WICSA 11), IEEE CS, 2011, pp.260.269.
3A. Borg et al., “The Bad Conscience of Requirements Engineering: An Investigation in Real-World Treatment of Non-functional Requirements”, Proc. 3rd Conf. Software Eng. Research and Practice in Sweden (SERP 03), CSREA Press, 2003, pp. 1.8.
4 J.L. de la Vara et al., “An Empirical Study on the Importance of Quality Requirements in Industry,” Proc. 23rd Intfl Conf. Software Eng. and Knowledge Eng. (SEKE 11), Knowledge Systems Inst. Graduate School, 2011, pp. 438.443.
About the Authors
David Ameller is a PhD student at BarcelonaTech-Universitat Politecnica de Catalunya. Hefs also member of the Software Engineering for Information Systems research group. His research interests include software architecture and requirements engineering. Ameller received an MSc in computing from BarcelonaTech.Universitat Politecnica de Catalunya. Contact him at dameller@essi.upc.edu.
Claudia Ayala is a researcher and lecturer at BarcelonaTech-Universitat Politecnica de Catalunya. She’s also a senior member of the Software Engineering for Information Systems research group. Her research interests include requirements engineering, software architecture, and open source software. Ayala received a PhD in informatics from BarcelonaTech.Universitat Politecnica de Catalunya. Contact her at cayala@essi.upc.edu.
Jordi Cabot leads the AtlanMod INRIA research team at Ecole des Mines de Nantes. His research interests include model-driven engineering, software modernization, and formal verification. Cabot received a PhD in informatics from BarcelonaTech.Universitat Politecnica de Catalunya. He’s a member of IEEE and ACM. Contact him here.
Xavier Franch is an associate professor of informatics at BarcelonaTech. Universitat Politecnica de Catalunya. He’s also the scientific leader of the Software Engineering for Information Systems research group. His research interests include requirements engineering, software architecture, and open source software. Franch received a PhD in informatics from BarcelonaTech.Universitat Politecnica de Catalunya. Contact him at franch@essi.upc.edu.
This article first appeared in IEEE Software magazine. IEEE Software's mission is to build the community of leading and future software practitioners. The magazine delivers reliable, useful, leading-edge software development information to keep engineers and managers abreast of rapid technology change.