This is the fifth post in a series that takes a look at information architecture (IA) best practices in higher education. Each post will focus on a specific best practice. In this post we’ll discuss usability testing.
For those new to information architecture, it is the practice of organizing information—in this case websites and the content found within them—to make it understandable and findable. Information can be messy and IA tries to make sense of it.
What is usability testing?
It’s the act of testing a website to see how well people can use it. And it’s important to note that it’s usability testing, not user testing. The word choice may seem subtle, but it makes a huge difference—we’re not testing the user, we are testing the website. Usability testing can take many forms. This post will focus on the most traditional format—in-person, one-on-one sessions where a facilitator observes participants as they try to complete tasks and find information on your website.
Why should you care about usability testing?
For starters, it’s a great way to see how your audiences are using your website. It’s easy to make decisions based on assumptions about user behavior. But if you take some time to test with and talk to your users you’ll be surprised at how they challenge those assumptions.
Secondly, it’s a great way to check your work and spot problems before you’re too far along in a project. Test early in a project to make sure your structure is sound. But also test later, when a site has real content, to make sure it’s working for your audience.
Thirdly, it’s a great way to settle differences of opinion or see if a new idea will work. Are you working with someone who insists on a structure or content that you don’t think will work? Test it. It’s much easier to fight for your ideas if you can prove they’re effective.
How to get started with in-person usability testing
Step 1: Find participants
Participants should be potential site visitors from key audience groups. Ideally they should have no knowledge of the project that is being tested.
We recommend testing with at least five participants. We usually don’t test with more than 10 participants as we often see clear results/patterns within that amount. If possible, conduct a series of smaller tests throughout the entire project, so you can see how your work is improving over time.
Incentives are helpful in recruiting people. Gift cards and swag are usually good. There are services that will help you find people within your target audience for a fee. If you’re in a real pinch you can open it up to people outside of your audience, although you may have to tailor your test questions to better fit a wider audience. Usability.gov has a helpful recruitment email template you can download.
Step 2: Create your tasks
You’ll need to develop a list of tasks for participants to complete. A good number of tasks is 15 to 20. Any more than that and the participant will start to get fatigued.
Questions should find a balance between specific and general. You don’t want to spell out the answer in the question, but you have to have enough specificity that the question makes sense without the site having content.
Step 3: Getting ready for your tests
For in-person tests, you’ll need a quiet space where participants will feel comfortable. You’ll also need a computer or mobile device with an Internet connection. If you’re using a laptop, you should also have a mouse.
You’ll want to print out the list of tasks for each audience group. Participants will use these lists as a guide.
Make sure you have a way to take notes. Whatever works best for you. It can be helpful to make an audio or video recording of the sessions to back up your notes and to share with other team members.
If you’re giving away incentive,s make sure you bring them to the sessions and/or tell participants how they can access them.
Step 4: Conduct the test
You’ll want to welcome each participant as they arrive and thank them for participating.
Before starting the test you should tell the participant:
- Why you’re conducting the test.
- That you are testing the website, not the participant.
- That there are no wrong answers—you’re actually trying to identify potential issues, so if they are having trouble it is ok.
- That someone else created the site so they can’t hurt your feelings if they have problems.
- That they should try to navigate the site as they would at home. It is fine if they can’t find the information. If they would give up looking in real life they should give up here.
- Ask them if it is ok to record the session. You can also have them sign a consent form.
- To speak aloud as they try to complete each task. This helps you understand what they are looking for as they navigate the site. You will have to continually remind them to do this.
When ready, have each participant read the task aloud and then try to use the website to complete the task.
Try not to answer any questions they ask you. For example if they ask “Is this the right place?” try to respond with “do you think this is the right place?” You don’t want to lead them on or accidentally answer the questions for them.
Go through all the tasks. When those are done, ask any follow up questions you might have. Feel free to ask the participant questions as they move through the tasks, as long as you don’t help or influence the participants.
The session is over once the tasks are completed and the follow up questions/discussion are done.
Step 5: Reviewing the results
- For each task identify if the participant:
- Completed it easily (found it right away or with little trouble)
- Completed it with some difficulty (it took a few tries to find it)
- Did not complete it (either didn’t find it or went to the incorrect page)
- Try to notice any patterns. Were there things that no one could find? Did several participants behave in the same manner? Did they have similar questions or concerns?
- You can calculate your quantitative results and note the qualitative results into a report if you want. Usability.gov provides sample reports to get you started.
- The most important thing is to identify any major problems with the website that need to be addressed. You’ll probably also learn some things about the types of content your audience wants and the vocabulary they use.
Beyond in-person testing
If your circumstances don’t allow for in-person testing consider other options such as:
- Guerilla testing—Nothing is scheduled, you simply go to a public place and try to find willing participants. It’s your best option when you don’t have time to recruit and/or can’t get anybody to commit. It is better than no testing, but you usually can’t ask as many questions and you’re not guaranteed to get participants in your target audience.
- Remote testing—Similar to in-person testing, but done remotely. There can be technical issues with remote testing so it is good to have a few extra participants lined up. It also requires a bit more pre-work (sending people the files, preparing instructions to send, etc.).
- Online testing—Using a third party service to handle the testing for you. You have to pay for the service and don’t get to watch in person, but you can get more people and it takes less time.
When it comes to usability testing it doesn’t matter how you do it, but simply that you do it. It will always provide information that helps you understand your audience and create a better website.