Usability Testing Guide & Benefits
By Laveena Ramchandani
How important is it that we can navigate smoothly through applications and complete tasks?
It’s crucial that users are able to navigate through applications with ease. It’s even more vital that we focus on the user’s experience when we are creating applications. To do this we can perform usability testing to reveal pain points, confusions and unveil opportunities to improve a user’s overall experience.
Usability testing is a form of non-functional testing with the aim of understanding how usable an application is. We can achieve this through observing end-users, as they navigate through the app or perform certain tasks. If we focus on understanding and hopefully improving a user's experience we are not just helping them but also building a better relationship with them.
Make your products sticky, so your users can stay stuck to them!
A brief history of ‘usability’
Usability, as a professional career, was advanced in the 1980s through the books. Its origins can be traced back to WWI and WWII with reports of soldiers being taught how to assemble and disassemble weapons in the dark. One of the first books focused on usability was published in 1983 at Carnegie Mellon and Xerox Parc, “The Psychology of Human-Computer Interaction”. From then onwards more guidelines and usability material was published including Donald. A. Norman’s “The Psychology of Everyday Things''. The speciality about this book is around how a product design serves as a medium of communication between the user and the product leading to a pleasant user experience.
This type of user research is very important with any application that is user-facing and new designs. Without usability testing, you may be stuck with a design process that your team members understand, but your target audience would not.
Usability testing is focused around assessing how usable your application is - can the end-user complete their tasks without getting obstructed? This testing can also provide answers to a very important question: have we built the right product? In this blog, I will take you from the very basics of usability to my personal experience researching consumers testing an application.
The benefits of usability testing
The key rationale for performing usability testing for an application is:
To identify any problems that may exist
Uncovering any opportunities to improve your product
Learning the end user’s preferences and behaviours
A software development team can sometimes miss out on the basics around usability, they are busy working towards requirements and new features that come in. Therefore usability of the product can often be ignored. Having your end user’s test an application brings a fresh pair of eyes and more rounded feedback. This is an effective and efficient way to determine whether the end-users can:
Work their way through the application without getting confused
Manage to complete their actions to complete a task
Challenges faced during this experience will enable them to see the various usability bugs/defects that engineers will need to address.
Your application will always benefit from user testing as user experience is important. Whilst not a typical user-facing type application, we can still perform usability on pretty much everything, including API’s. The key is to understand and ideally categorise how the user of the application performs their job, i.e. what tasks do they need to complete to achieve their overall goals - service a customer/ arrange a trade etc. Once you have identified these tasks, organise and prioritise them and then try to get the users to complete those tasks with a prototype of the application.
With an API we can test if it is plugged into an actual customer system or create our own interface to surface the data output from the API and see if it can be consumed easily. For API's that have no interface, you can test how easy it is for a developer/ tester to review it and how easy it is to potentially get data from it.
You can keep getting constructive feedback from your users to help improve the product over time. Having users test the product can be the first crucial step in an effective feedback loop. Usability testing can bring the following benefits:
Early validation of a prototype solution at the beginning of the development process.
Enables the team to understand the end-users reactions before creating the final product. Testing a prototype helps you assess the value of any future functionality before you put all your efforts into building the entire application.
Confirmation that your product is meeting the user’s expectations.
Your product may have some complex user flows, therefore usability testing can help identify if these steps need to be broken down into several stages making it simpler and more intuitive to use.
It will also help you understand if the user is noticing and paying appropriate attention to what you deem as important areas of the application and if not, it gives you the opportunity to find out why?
A fresh pair of eyes can be more likely to find errors that the engineering team might have overlooked such as grammatical mistakes, broken links, site errors or any cosmetic issues. These all contribute to the applications reliability, readability, maintainability making it trustworthy and professional.
Empathy, a software development team who is working on the product day in day out should know the product inside out, but they also need to bear in mind that this may not be the case for its users. Empathy for the users is vital therefore, having them use the product through usability testing can help you understand their perspective better.
Types of usability tests
This part is the most important part when adding usability testing within your project. If this step is done thoroughly and with good planning, the entire process can be smooth not just for the team, but also for the users who will be completing the testing. The test planning phase, test execution phase, data analysis and reporting phase are the key stages of performing this type of testing. These are some methods that you could adopt for usability testing:
Lab usability testing
The moderator and the tester are present in the same office or physical space. It is a specially built usability lab where the users will conduct specific tasks on the product and the moderator is observing and asking questions. Stakeholders can also be involved when observing the testing. This can be a high budget task.
Guerilla usability testing
Users are chosen randomly from a public space, such a coffee shop. The tests are very short and easy to understand and quite low budget. Their task is to perform a quick test in exchange for a small incentive sometimes.
Moderated Remote usability testing
Performed via the internet with the moderators. It is all remote and there are multiple testers involved to gain useful insights, similar to mob testing. Usually, the testers would need a computer or a phone device and a good internet connection depending on what the application is. This is a medium budget task.
Screen share sessions
A moderator provides instructions to the testers to complete a specific task and also collect useful feedback from the testers whilst testing. This method is more budget-friendly as it can include users from a wide area and helps collect more metrics in a small time frame.
Usability testing stages
In the test planning phase, you need to identify the scope of what needs testing. In other words what functionalities and features are important to be tested by the user? You need to also think of your testers or your target customers. It would be great if you can invite some of your project end-users with varied product knowledge to test the prototype/end product. Also create a set of objectives you would like to achieve via the session to gain more clarity such as:
What are we going to show to the stakeholders?
What do we expect from the users?
What do we want to achieve from the session?
Lastly, you need some types of metrics to follow and measure the user's actions such as:
How much time did it take the user to complete the test you have created?
How easy is it for the user to perform a task?
Did the user use the application as expected and did it work as per the user's expectations?
The above should be identified upfront so that you can proactively track them during the session.
Next, the test execution phase:
Usability tests are executed at this point where the moderator is responsible for moving the session forward by introducing the user to the tasks to be completed. These tasks should incorporate the main areas or functionalities of the application.
The moderator should also observe and record the spoken feedback and the actions of the user. Here you need to think about two ways of assessing: quantitative information and qualitative information. Quantitative information is a number based in nature. For example in this context - the time taken to complete a task in minutes and seconds. Of the tasks set for the user, you can record how many were completed successfully - 3 of 5 = 60%.
Whilst numbers are most comforting for managers and team members to hear, given the numbers of users and tasks likely to be tested - statistics like this can be a little misleading. In situations where the users and the moderator get into a healthy discussion, this could mean the stats would change as the timer in that situation won’t be useful. Also if you are talking to 5 users who are testing 3 tasks each, it is really a small number to talk through their experiences and the severity of problems each faced. You can identify where the severe issues are easily without taking percentages. Percentages make the audience sometimes feel it is more scientific. What is more robust is getting to the heart of each problem and addressing it rather than finding ways to say 4 tasks out of 5 had some kind of issue leading to an 80% failure rate.
Qualitative information is centred around the human reactions to the experience of attempting to complete tasks using the proposed solutions. These can centre around the perceived ease of completing the tasks or the frustrations both in terms of level and frequency. The real measures of success or failure are largely to be found here. Also, the particular pain points observed where users become stuck or delayed in completing their tasks provide a list of things to address. In reality, usability testing is not simply there to assess whether the users can complete the tasks, it should be there to measure the ease (effectiveness and efficiency) with which they can achieve them.
The penultimate step is around the analysis of the results from the test conducted. Once you have concluded the test it is important to analyse the results and to extract some insights. It can help your team better understand particular user behaviours and patterns. From the results, you can bring in recommendations to the team to improve the product to be more user friendly.
The last step is the reporting phase where a report will contain the metrics used, the feedback received and any recommendations. Then a discussion can take place around the findings and understand what can be improved and plan on how to implement the changes.
Understanding the findings
After conducting the usability testing you will have quite a lot of findings to go through and analyse. Interesting insights can surface. These can help you improve your product, creating greater user satisfaction. You can categorise and prioritise issues, what is crucial vs what is good to have. Remember it is vital to understand all the quantitative and qualitative findings. If there is a particular piece of functionality that needs extra work, then you can even write down some recommendations and keep things more transparent between the engineering team and the users. Once the results are analysed and decisions are made then the team can plan to add the new implementations in future design iterations.
Conduct usability testing by selecting a method of the team’s choice
Include information about the participants (users can be those who actually use similar products to what you want to test, as different patterns and journeys can be perceived that are important to them)
Collate quantitative metrics and qualitative findings
Create a test report which is easy to understand and drive a team discussion for action plans
State what worked well and what were the pain points and prioritise issues to be fixed
Write actionable recommendations with clear steps to bring into future iterations
The cost of usability testing
The cost of running usability testing varies depending on what method you decide to use. Compared with methods that are performed in a lab or when end users are invited to the firm, using a budget-friendly method ensures the costs won’t mount up. An approach to reduce costs may be to test with people within your firm, who do not directly work with you and who come from different business areas.
It is critical for a firm to be able to build a user-friendly application and one which is tested and accepted by its end users. Sometimes it's important to look beyond the obvious costs. It may end up costing you more if you choose not to do usability testing. What I mean by this is, if this testing is not conducted the product may go live, however not meet user standards. This can lead to negative feedback and a negative reputation for the product. Usability testing brings incredible value when testing with real people as you gain a lot of information about the product and the users. Think about it as an investment that will help your product and the client.
I had participated in the usability testing as a software tester and was one of the observers taking all the notes from the end-users who were testing the product. It was a great experience as I find it quite intriguing to see how every user thinks. What I may think is straightforward may not be as easy for the end-user or vice versa. I took notes for about 4 different end-users.
I learnt quite a few things like certain aesthetics were not useful, certain wordings were confusing, suggestions were also made which were quite interesting as the engineering team might have not thought about them. I noticed some features were tested quite successfully but certain areas of the application were not clear to the user. The moderator did an excellent job of running the session and helping the users where they got stuck and strategically got feedback whilst testing. Some of the tasks the moderator had to do for a smooth usability testing session were:
Have clear objectives which were agreed with the team members.
Have a discussion guide with timings planned (10 minutes for an introduction/ 5 minutes on page 1/ 10 minutes on page 2, etc).
Do a test run to check the timings with a team member before the session kicks off (just to make sure everything can be covered).
When running the session take enough pauses to check on the discussion guide and make sure the main areas/points are covered before moving on.
We then did a three amigos session to discuss our findings and add them onto a joint board of ideas. This helped us view all the other observers’ points made during the session. The next step was to bring the new recommendations as features into sprints and share this with the end-users to gain more feedback.
Thanks to the usability testing session I started to think of other edge cases whilst exploring the application to make sure whether I am comfortable with it or not. Being close to it, if I am not comfortable then the end-user may also find it quite tricky. At the end of the day, it's worse if we cannot live up to the user's expectations, therefore bringing in usability testing may increase your budget but stop your team/project from having unplanned issues which becomes more expensive. It is super important to see the thought process of the end-user and their ideas too.
Usability testing is vital as it gives you the opportunity to learn about your product through the end user’s eyes. Technology is advancing so quickly these days, that if your product is not easy to use then you have already lost potential users as they will race towards other, perhaps easier to use, options available. Therefore, plan the usability testing thoroughly and recruit the users that are required for it. Think about the budget you have but don't dwell too much into it, as saving a tiny bit could lead to missing key insights that could cost hundreds of thousands. A happy customer means a happy product.
“If you think good design is expensive, you should look at the cost of bad design.”— Ralf Speth
The Tech world is ever-growing, and Laveena has been working in Tech for over 8 years now. Laveena is a software tester and a quality advocate. She quite likes the tester's role as it is a good mix of technical and business awareness roles. She has learnt a lot through her career and looks forward to gaining more knowledge and at the same time inspire more testers around the world and UK.