17 Oct 2024
|11 min
Usability testing questions
We share our top tips on writing effective usability testing questions, including specific and open-ended questions, avoiding leading or biased questions, and focusing on the user's goals and tasks.
How to ask the best usability testing questions
As someone with experience in marketing academia, and as a market researcher and participant, I’ve found usability testing to share some similar concepts.
Marketing practitioners like to assume their target audience thinks the same way they do about a product or concept — but the reality is that people respond differently, even within the same target group.
And it’s the same with user experience. So to create a great UX experience, researchers and designers need to understand who their users are, as well as what they want or need from a product, and why. But how?
The short answer: by asking potential users lots of questions!
To get more specific, before you start doing comparative usability testing, and other testing you need to ask “usability testing questions”, which is a period of research UX designers, UX researchers, product managers, and marketers conduct throughout product development and before any official launches (and again when you propose any changes to your design later, or to check everything is still working as intended).
In this guide, we’re going to cover the following areas:
Why you need to ask usability test questions
Usability testing question categories (with example questions)
Best practices for asking test questions
Let’s dive right in!
Why do you need to ask usability testing questions?
Asking potential users questions about your product or UX design is important because it helps you validate your hypotheses or any ideas you have, find any issues with your design before launch, and understand how users interact with similar products or services offered by the competition.
While we’re going to cover best practices further down, one rule of thumb is that you should ask questions at each stage of the user research process — this way you get more in-depth results and gain the maximum benefits from it.
Your go-to user research platform
The best teams use Lyssna so they can deeply understand their audience and move in the right direction — faster.
Usability testing question categories
As mentioned above, you should organize your usability testing questions based on your current research stage. But to make sure you’re asking users the right questions, you need to define the purpose of your research — what do you want to find out?
By defining the overall research goal, you’ll have a far easier time narrowing down the list of questions you’ll need to ask at each stage of the process.
With that in mind, there are three core stages of UX design research where you can ask users questions: before the test (to screen participants), during the test (to help answer your research questions), and after the test (to gain feedback).
Another aspect to consider is whether you’ll be using remote usability testing tools as opposed to an in-person test, and whether the test will be moderated or unmoderated. You can find more details about the nuances between these testing methods in our usability testing guide.
For now, let’s take a look at these categories in more detail.
Screening questions
As much as we like to think our designs will be useful for everyone, they won’t be. Instead, we need to focus our attention on users that will be the most likely to use the product — our target audience.
To do that, you need to find the right participants for your test by asking screening questions.
So first, define the type of people you want to involve in your usability tests. Then, to find those people, you can ask “about you” questions to filter your pool of users. These questions are usually about demographics, background, and experience.
Demographic questions
Asking demographic questions can help you find trends or narrow down your participant pool by age, gender, nationality, relationship status, and religion (if relevant). However, in many countries, these points of data are protected characteristics and can be sensitive information for some people.
Therefore, it’s best to ask these questions in ranges or let the users define themselves (and/or give “prefer not to answer” or “other” options). Here are some examples:
What age group are you in? (Give ranges as options, e.g. 18–25, 26–34, etc.)
How would you describe your gender?
How would you describe your ethnicity?
What is your household income? (Again, giving ranges as an option)
How would you describe your relationship status?
To help you further filter your participants, you can ask additional, more specific background questions.
Background questions
Asking background questions can help you get a better context of the person and how your product or service fits into their life.
You should also find out if they’re an existing user, a user of competing products, or if they’re completely new to it — each of these offers different insights and you should take them into account during your analysis of the results.
Some background questions to ask include:
What does a day in your life look like?
Do you already use X product, and if so, how often?
What type of product do you use to do X?
Which device would you normally use to do X?
How much time do you typically spend online? (If building a digital product)
How experienced are you in using X type of product?
Using these screening questions can help you find the right participants, as well as help explain any abnormalities in your results.
In-test questions
Now to the questions you probably had in mind when you were thinking about this topic — what to ask during your usability test. Contrary to what you might think, this is the period when you should avoid asking too many questions, as this can distract your users and stop them from taking actions natural to them.
However, the most important rule to remember when asking usability test questions is to not ask leading questions, meaning a question that influences the user to think a certain way, e.g. “how good was the experience of using X feature?”.
Leading questions like this will skew the results and won't provide an accurate picture of the experience. Instead, ask neutral, broad questions that encourage users to offer honest answers.
Examples of non-leading questions can include:
What do you think about X design?
Can you name any competitors that you currently or would use X for?
What features do you find most valuable, and why?
How would you describe the language used on this page?
Can you give an example of where you would use X?
I noticed you [describe something they did]. Why?
How was your experience completing X task?
Open-ended questions like these help you get an in-depth understanding of the user’s experience while using the product in the test, which is far more useful than simple ‘yes’ or ‘no’ based questions.
Post-test questions
After they finish a test, users might still have some opinions about the product or testing experience. So the end of the test is a great opportunity to ask some final questions while you still have their attention (at most, ask these up to a few days later so the memory is still relatively fresh).
These questions can help you improve the testing experience or gather opinions on aspects of the product you hadn’t considered before. Some usability testing examples of these questions can be:
How would you describe the overall experience of the product?
How would you describe the overall experience of this test?
Were any of the tasks difficult to complete? (Or) How would you rate the difficulty of the tasks on a scale of 1–10 (10 being extremely difficult)?
What do you expect X product/feature to be like in the future?
If you could make any changes to X, what would they be and why?
Do you have any additional comments or questions?
Another option can be to use a standardized quantitative research feedback survey (such as the System Usability Scale) to help gather any long-term trends or to compare with other similar products using the same survey format.
Best practices for asking usability testing questions
As you’ve probably picked up, we’ve already introduced two best practices for usability testing:
Ask questions at each stage of the research process.
Don’t ask leading questions; ask open-ended questions instead.
However, there are a few other best practices to keep in mind to help you get the most out of your usability test questions.
Watch the participant's body language and facial expressions
If a participant has allowed you to record them, pay attention to how they physically respond during the test. This can give you a lot of insight into parts of the test when the user subconsciously expresses like/dislike or confusion.
Ask participants to think aloud
Similar to the above, some people find it difficult to express their thoughts out loud. Asking them to do so gives them “permission” or reminds them you want to understand their state of mind.
Make use of technology
While in-person tests are still great, using remote usability testing tools can help automatically process data, such as click maps, path analysis, heatmaps, and time to complete.
Heatmap results from a first click test in Lyssna
Don’t provide too much guidance or ask too many questions during a test
I completed a usability test (as a participant) recently for a banking app. During the test, I often felt the need to ask “how do I find that?”, but the researcher encouraged me to figure it out for myself — for good reason. It helps researchers and designers to see where users might struggle with navigation, and pinpoint potential problems.
Improve your designs by asking the right questions
Great usability tests help to create better user experiences and can take a lot of time and effort to get right. Being prepared with good questions can help you maximize your return on investment (in time, insights gathered, and costs to conduct tests).
By planning your questions out in stages, you can identify problems that come up before launch, and find out if your product addresses what users want and need. Sometimes, they can also help inspire new ideas based on user feedback from the test.
Elevate your research practice
Join over 320,000+ marketers, designers, researchers, and product leaders who use Lyssna to make data-driven decisions.
Frequently asked questions about usability testing questions
--
Alexander Boswell is the Founder/Director of SaaSOCIATE, a B2B SaaS, MarTech and eCommerce Content Marketing Service and a Business PhD candidate. When he’s not writing, he’s playing baseball and D&D.
You may also like these articles
Sign up to our newsletter
We'll keep you updated with the latest UX insights and more.
