Hallway testing can be used to enhance the usability of products and make your UX better. You can also use it to test APIs as Ewa Marchewka, head of software integration and test department at Nokia, presented at TestCon Europe 2019. It’s cheap, straightforward, there’s no need for complicated tools, and it’s fast, getting feedback from the end-user almost instantly.
Hallway testing can be described as using "random" persons or group of people to test software products and interfaces. "Randomness" of a person depends on what we are trying to test. Marchewka suggested trying to engage people who will be using the product (i.e. members of the target group) to get the best understanding of how they will do that. For their hallway testing session they invite a truly random group of people if they are checking mobile app, and a random group of API users if they are verifying UX of an API.
Using the specific background and experience of all the people taking part in a particular session of hallway testing (i.e. cognitive diversity), we can uncover all inefficiencies of the user interface in a tested product, said Marchewka.
The app or software does not need to have a GUI to benefit from hallway testing; it can be used as part of API prototyping activity, as Marchewka explained. Consumers of API can be asked to use an early version during a hallway testing session; for example, creators can find out if methods are named correctly. Creating an API you invent the scenarios in which it will be used, she said. You invent a target audience (i.e. developers who will be using what you created) and ask them to try to use your API. Then, watch and learn what they will see as inefficient or hard to use.
Marchewka mentioned two major "don’ts" that we should avoid when doing hallway testing:
For one, you should try to avoid giving hints or guide tester through your app/SW program. Do not help him/her unless they are stuck and cannot move forward. By telling the tester what to do you will lose all the value of hallway testing - getting unbiased feedback from a different person.
The second thing you should try to avoid is asking the tester to perform overly complicated tasks. If you skip the basic use case scenarios, you will miss all the usability bugs that are hidden there. Also, a very complicated task will probably frustrate the tester and take him/her a long time to complete,
InfoQ interviewed Ewa Marchewka after her talk at TestCon Europe 2019 about hallway testing at Nokia.
InfoQ: How can we apply hallway testing to enhance the usability of products?
Ewa Marchewka: Let’s start with the simplest case possible. You want to create a mobile app that will be used for online shopping. You have a prototype ready and you want to find out if your app is free of any major usability flaws.
First, you identify your target audience (for an online shopping app, most probably a truly random person would be enough). In the simplest case, you can ask your family member or neighbor for help. Offering coffee or lunch helps!
The next step is defining user scenarios. Select very simple, basic use cases that will be performed by your end-user quite often (in our example it could be "adding an item to a shopping cart" or "searching for an item").
You need to reserve time for your testing session, depending on the number of features/functionalities that you want to check. Sometimes it is hard to define how much time you would need. You can estimate it by performing the tasks yourself, measuring the time you took and then adjusting the duration by adding 5-10 minutes per task, as the tester will not be familiar with the application.
During the session, make sure that you note down the results carefully. If the tester is stuck or struggles with executing a task - he just uncovered a usability bug in your software! This is really valuable information and you do not want to lose it. Note down all the comments done by tester, even if you are not happy with them (e.g. if the tester is saying the coloring looks awful, this is still useful feedback). This is the whole point of hallway testing - getting fresh, unbiased feedback.
The way you capture the results may vary - in basic cases, pen and paper are enough. If you are going for an expensive version of hallway testing, you can use different tools, e.g. eye-tracking devices to see which elements of your app are most attractive for the user.
After the session is finished, analyze the results, and fix the bugs :)
The best results can be achieved if you repeat the testing sessions a few times, preferably with a new tester who has not used the application before.
InfoQ: Which benefits did hallway testing bring you?
Marchewka: Hallway testing is perfect for uncovering major usability flaws in your software, like a counterintuitive GUI, missing functionality or bad color design.
It is cheap, as usually you are getting help from your friends, family or colleagues at the office, so there’s no need for hiring additional professional testers. You do not need expensive equipment; to perform a session of Hallway Testing you just need a mobile/PC with your app/webpage and a pen and paper to note down the results.
This usability testing method is also pretty straightforward - you are asking the tester to try out basic use cases that will be performed by the end-user. Again - no need for complicated tools here.
Hallway testing is also fast - you will get feedback from the end-user almost instantly.
InfoQ: If readers want to learn more about hallway testing, where can they go?
Marchewka: For an overall usability testing introduction, I recommend these books:
- A Practical Guide to Usability Testing: Joseph S. Dumas, Janice C. Reddish
- Handbook of Usability Testing: Jeffrey Rubin
Blogs and articles about Hallway Testing:
- Take Your UX on a Trial Run with Hallway Testing by Jamie Ayers
- 10 Tips for Better Hallway Usability Testing on Digital.gov
- How to Conduct Hallway Usability Tests without any Hallways by Jon Ericson
Why we need a small number of users for Hallway Testing:
- How to Test Your Application with 5 Users before You Launch by Thomas Pehan
- Why You Only Need to Test with 5 Users by Jakob Nielsen
