This article first appeared in IEEE Software magazine. IEEE Software offers solid, peer-reviewed information about today's strategic technology issues. To meet the challenges of running reliable, flexible enterprises, IT managers and technical leads rely on IT Pro for state-of-the-art solutions.
Talking with users might change how you see the context of your software project, often in unexpected ways. Drawing from his experience on spacecraft operations software projects, Francisco Torres shares stories on how listening to users taught him to stop making assumptions and helped him de ne his software’s operating range: the set of quality properties in which a software system can successfully run. - Cesare Pautasso and Olaf Zimmerman
They say that experience is what you get when you were expecting something else. This happened to me some years ago while I was studying human-machine interface concepts for a software system for spacecraft operations. One initial project activity involved analyzing the existing user interface to identify improvement areas and new concepts to prototype and explore. To that end, my colleagues and I designed a questionnaire on usability aspects, which a sample population of the users answered. Some of the answers were counterintuitive, to say the least. In time, and after giving it some thought, it suddenly dawned on me that I had lost sight of one key element: context. With context in mind, everything made more sense.
This lesson about context applies to not only GUI-related topics but also many other software-engineering areas, such as processes and tools. They all have a nominal operating range, so to speak. More on this later, but first I’ll walk you through some of the GUI attributes our study explored. In each case, I'll brief you on the feedback we received through the questionnaire and explain why, despite not matching my expectations, the users' point of view ultimately made sense.
When the Stakes Are Too High
The questionnaire asked the respondents to assess how intuitive and user friendly the various displays were. We got some answers along the lines that some displays were "not user friendly", "far from intuitive", and "clumsy in the extreme". However, one theme also recurred in many of the answers. Learnability and intuitiveness were less of an issue because the users had been formally trained to operate the system, including rehearsals and simulations.
One respondent stated that it wasn’t possible to use the system simply intuitively; some a priori knowledge was required. Another one even proposed to redesign some of the displays, trading intuitiveness for efficient screen real estate use. The reasons were that novices weren’t allowed to command the spacecraft and that users could be assumed to have undergone weeks of training.
The same results applied to online help. In most desktop applications, the last item on the menu bar is Help, which accesses some form of user assistance. Although none of the applications in our study featured online help, the respondents didn’t perceive this as a serious weakness because, as one of them put it, "In real operations there is in general no time to consult help". Also, spacecraft operations are performed according to a set of flight control procedures (FCPs), which cover both nominal and contingency scenarios. These FCPs provide detailed step-by-step descriptions of what the user is expected to perform and check at any given time. So, little if any room exists for improvisation.
In summary, sometimes the stakes are too high. Maybe your application deals with valuable assets (in this case a satellite, which costs millions to build and launch), or human lives can be at risk (for example, with medical devices or nuclear plant control systems). In these cases, some level of training will always be required, and reliance on purely intuitive use of the system simply isn’t an option.
Configurability: No, Thanks
Because computers are pervasive, we’ve grown accustomed to the multiple configuration options that most OSs and applications offer users. (Examples include fonts, colors, customizable toolbars, and languages). So, you can imagine our surprise when the survey respondents told us that configuration options were generally frowned on and should be used carefully.
The opinions ranged from saying that the application shouldn’t be configurable at all to saying that it should be configurable only by the administrator (which, mind you, effectively makes the same configuration available to all users). The main reason for this configurability aversion was that this isn’t a personal application. Many people use it, such as spacecraft operations engineers and spacecraft controllers working shifts. Sometimes the application is used serially, such as when one user sits in front of the system after another one has just left. Sometimes it’s used simultaneously, such as when the engineers and controllers gather around a given workstation to perform an investigation.
An overall consensus existed on language configuration. Although this was an international working environment with personnel from various European countries, the respondents expressed clearly that users shouldn’t be allowed to change the user interface language. They showed a strong opinion about sticking to English, which was the language used operationally and in which all the operational procedures were written.
In summary, in this environment, configurability wasn’t a particularly good idea. Some configuration options could turn out to be just a minor annoyance, like when someone decided that Comic Sans was cool and made it the default system font. Others could be harmful: imagine a controller starting her workday only to realize in horror that her German colleague on the previous shift changed the default language to German and had already left the building. Even if, in your application or environment, it makes sense to let users change the GUI language, you might probably be better off by sticking to a single standard language (which in most cases will be English) in your log files and configuration artifacts. This will certainly make life easier for your support team when it deals with queries and tickets from users in another country.
Context Is King
Don’t worry. I’m not advocating that you throw away everything you learned about usability and good user interface design1.That’s all good; as much as possible, strive to design and deliver user-friendly applications. All I’m saying is that the user survey proved that some of the enhancements I had in mind were either not appropriate or not the topmost priority for that organization at that point in time. Put another way, they weren’t a good fit for that context.
This lesson is valid for you, too The system or the application you’ve been tasked to develop and deliver will operate in a given context, rather than in isolation. Keep this context in mind because it can have a huge impact on some of the decisions you’ll need to make along the way if you really want to come up with a sensible, successful solution.
The operating-range metaphor might help you understand my point. For example, the operating temperature is the temperature at which an electrical or mechanical device operates. The device will operate effectively within that range; outside that range, the device might fail. The user manual for your laptop, printer, and refrigerator likely included a short chapter detailing the device’s operating environment. If you’re like me, you probably paid little attention to that section or skipped it. In most cases this is fine because you’re using these devices in the environment for which they were designed. But if you were to use them in an extreme environment, that information would suddenly become quite relevant.
I have the impression that sometimes we become so enamored with our favorite tools and techniques that we lose sight that they also have a nominal operating range. And, by doing so, sometimes we apply them in situations for which they aren’t a good fit. At best, this will result in a suboptimum (yet usable) solution. If we’re not so lucky, this can be a big mistake that seriously hinders progress or jeopardizes the project’s success.
Take the following two examples. First, there’s much talk these days about the convergence of mobile and desktop OSs. But no one can serve two masters. A mobile OS is conceived first and foremost with touch and mobile devices in mind, and it’s optimized for tablets and smartphones. In the same way, a desktop OS is designed mainly for interaction with a keyboard, mouse, and normal-size, nontouch display. In trying to achieve convergence, the designers will probably have to make some trade-offs and settle for some compromise solutions.
Imagine that overall these design decisions lean toward optimized use in mobile devices. It should then be no surprise that the user experience in a desktop setting will suffer because the OS will be running outside the operational environment it was optimized for. For a satisfactory experience on both mobile and desktop devices, the converged OS would need to adapt to the device it’s running on. For two-in-one devices, this adaptation will probably need to take place on the fly depending on the mode (for example, tablet or laptop) in which the device is being used.
The second example involves agile methods2. I won’t go into their pros and cons, but I hope you agree with me that they’re not suitable for every situation or project. However, I’ve seen invitations to tender in which the contracting agency put out a statement of work in which bidders were expected to use Scrum yet comply up front with a fixed set of (already defined) requirements and bid for a firm fixed price. I think that those conditions (fixed scope and fixed price) aren’t compatible with the iterative, incremental nature of agile software development.
Certainly I didn’t cover any new techniques in this article. Some of you might argue that the problem was that my initial expectations were unfounded or plain wrong. In hindsight, that’s probably true; I should have known better. Yet it was a humbling experience that helped me learn quite a few things. Here are some takeaways that I hope help you too.
First, keep in mind the adage that one size doesn’t fit all. No matter how well a solution has worked for you in the past, ask yourself whether it will be working within its nominal operating range in your new project. Your new system’s expected quality attributes3 can have a big impact on a given solution’s suitability.
Similarly, if you’re moving to a new application domain, don’t assume that everything you learned in your previous projects still applies to the problem at hand. Some bits will; others won’t. Even if you stay in the same application domain, the same holds true if any of the other context attributes4 in your new project differ significantly from those in your earlier "standard" projects.
Also, make sure the processes you use are suitable for your project. Although agile development is probably mainstream for in-house development, the waterfall model is still widespread in many outsourced IT projects. In these firm fixed-price contracts, the requirements and deliverables are typically set in stone at the project’s start, and there’s little room to revisit them or deviate from the initial project scope.
Likewise, keep in mind that software is so pervasive that our expectations about how to interact with an application are heavily influenced and shaped by our everyday use of our office application, browser, or mobile OS of choice (just to name a few things). But before you implement a particular feature that you think is cool or that you think everyone will take for granted, double-check whether it’s relevant to the problem you’re addressing.
Finally, resist the temptation to jump on the next technology trend wagon (NoSQL databases, cloud computing, microservices-you name it) just for the sake of it. Ask yourself whether it’s a good fit for the problem at hand, and make sure you’re not using it just because it will look good on your resume.
References
1. R. Hartson and P. Pyla, The UX Book: Process and Guidelines for Ensuring a Quality User Experience, Morgan Kaufmann, 2012.
2. The Agile Manifesto, Agile Alliance, 2001.
3. M. Barbacci et al., Quality Attributes, tech. report CMU/SEI-95-TR-021 ESCTR-95-021CMU/SEI, Software Eng. Inst., Carnegie Mellon Univ., 1995.
4. P. Kruchten, "Contextualizing Agile Software Development", J. Software Maintenance and Evolution: Research and Practice, vol. 25, no. 4, 2011, pp. 351–361.
About the Author
Francisco Torres is a monitoring and control system engineer at GMV Aerospace and Defence. Contact him at ftorres@gmv.com.
This article first appeared in IEEE Software magazine. IEEE Software offers solid, peer-reviewed information about today's strategic technology issues. To meet the challenges of running reliable, flexible enterprises, IT managers and technical leads rely on IT Pro for state-of-the-art solutions.