The definition of usability in the ISO 9241 standard is: "The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use". This definition, however does not mention concrete ways of evaluating the ease of use or better still, the usability of the system. In a recent discussion on the Agile Usability group, members discuss various ways for evaluating system usability.
Andrea started the discussion about the evaluation criteria when she mentioned that she had to compare two products for their ease of use in an objective manner. This would help her organization sunset one of the products in the favor of other.
Daniel Naumann suggested that since the products were live products, the best way to compare them was to compare their respective trends. He suggested,
Since these are existing products you should have a wealth of info, such as, log files, completion rates and times, drop out rates and drop out points, etc. And best of all you'll have existing users you can hopefully go talk to to get feedback.
On similar lines Marjorie Pries suggested,
I would recommend you survey all the hard-evidence you already have, first. Then do some open-ended interviews and unstructured live observations with a representative sampling of your customers. What do they think is important? What products do they use as a basis for comparison? What features do they avoid using and where are they introducing workarounds or using external help?
Whitney Quesenbery suggested that measuring usability goes beyond the ease of use. She suggested that measuring usability is a function of five E’s (effective, efficient, engaging, error tolerant, easy to learn). She summarized the planning of usability evaluations using the following table
|
Characteristic |
Type of Usability Evaluation |
|
Efficient |
Time (or count clicks or page views) realistic tasks. Must use working versions of the software and plausible sample data. |
|
Effective |
Evaluate tasks for how accurately they were completed, and how often they produce errors. |
|
Engaging |
User satisfaction surveys or qualitative interviews can gauge user acceptance and attitudes towards the software. |
|
Error Tolerant |
Include task scenarios with potential problems in test use scenarios |
|
Easy to Learn |
Control how much instruction is given to test participants, or carefully recruit users with different levels of domain knowledge and experience. |
According to her, measuring the usability against the five E’s would give an objective and fair analysis of the usability of the system.
Similarly Jakob Nielsen suggested ten general principles for user interface design. They are called ten usability heuristics and the usability can be measured against those.
Andy Edmonds mentioned the Common Industry Format standard, which is the standard for reporting usability test findings. According to him,
This provides a highly rigorous way of designing, conducting, and even presenting the results of a test.
Thus, there are various ways of gathering usability statistics of a system. Some of them are more involved than others in terms of effort and time. An Agile team can chose a method based on the detail of their need and ascertain usability in an objective manner.