01 Oct 2024
|15 min
How to do usability testing on a budget
Discover how UX consultant Mark Jones conducts usability testing on a budget and gets stakeholder buy-in for UX.
If you're reading this article, you probably already understand the importance of good user experience. You also know how user research and usability testing can help us design products and services that enable users to achieve their goals.
Some of you may work for an organization that embraces UX and provides you with all the necessary resources. But for each of these organizations, there are many more that don’t recognize the advantages of investing time and money in research and testing.
One problem is that teams are constantly under pressure to deliver. They have targets to hit and an ever-growing backlog of tasks. They therefore need to find ways to get things done as quickly and effectively as possible.
Frequently in these situations, UX becomes compromised. I often hear statements such as "We know exactly what the problems are" and "We know what the solution is" based on a stakeholder's bias.
Mark Jones
UX consultant
Consequently, the argument for conducting research to solve a problem, and performing usability testing to ensure a solution meets user needs before development, often falls on deaf ears.
Throughout my career, I've encountered various situations in which I've sought to gain buy-in for user research and usability testing. In this article, I'll share the approaches I’ve taken to conduct usability testing with little or no budget to work with.
Your go-to user research platform
The best teams use Lyssna so they can deeply understand their audience and move in the right direction — faster.
Do-it-yourself usability testing
Imagine you've been asked to design a customer support section for a website. The objective is to reduce the number of calls by offering information on the website that customers typically inquire about over the phone.
You've generated ideas for how to present this information and have created a prototype. Now, you want to seek feedback on whether users can find the information easily and without resorting to calling. However, you don’t have a budget for usability testing. How can you test your prototype?
A do-it-yourself approach to usability testing is perfect if you’re working with a tight budget and few resources.
I’ll first discuss how to recruit participants using this method, followed by ways to conduct testing.
Recruiting participants
It goes without saying that you’ll need people to test your prototype, but how can you recruit when you have no budget?
1. Existing customers
One strategy is to engage with existing customers. First, confirm that they’ve agreed to be contacted. Then, send an email inviting them to participate in your test.
While offering an incentive for participation is ideal, it doesn't necessarily have to be monetary. A discount code for their next purchase could be an attractive incentive.
This method is effective for obtaining feedback from people familiar with your brand or product, but can be challenging to execute. You need to identify who can be contacted and draft an enticing invitation, and the rate of responses can be low or slow, which may not be suitable when you’re working under a tight deadline.
2. Friends and family
Another approach is to solicit help from friends and family. Due to your personal relationship, they’re likely to help you for minimal or no reward.
However, this method has a downside. Your relationship might deter them from giving honest feedback as they might not want to upset you. Even if you emphasize the importance of their honesty, it could still be challenging for them to separate their personal feelings from their critique.
3. Colleagues
You might also consider requesting feedback from your colleagues. The primary benefit of this approach is that it can be a quick and cost-free way to gather feedback.
However, there’s a disadvantage. Because they work for your company, they may provide feedback influenced by their personal biases and opinions.
When analyzing your findings, watch for recurring themes. Be cautious of feedback that seems subjective and may not accurately represent your users' perspectives.
4. General public
Another method is guerrilla usability testing – asking people on the street or at your local coffee shop. The advantage of this approach is that you're testing with people who have no relationship with you or your company, minimizing bias.
In exchange for a coffee, you might find people willing to provide their feedback. However, this approach has its downsides too. It may be uncomfortable approaching strangers, and recording the session can be challenging in an offsite environment.
How to conduct do-it-yourself usability testing
After recruiting your participants, the next step is to test your prototype. For a do-it-yourself approach, there are a few main methods you can use.
Remote testing
Advancements in video meeting software have simplified remote usability testing. Your participants can be anywhere in the world, and you can conduct these tests for free by simply sending a meeting invite and providing the user with a prototype link.
During the test, the user shares their screen, and you ask them questions and assign tasks while observing their behavior. It's important to record the session for later analysis.
While this method is quick and easy to set up, the main drawback is that it's time-consuming to conduct and analyze sessions.
Given it will be moderated, you need to feel comfortable conducting sessions that avoid leading a user to take certain actions. It can be incredibly tempting to assist the user when you see them struggling, but doing so will skew your results.
Most video meeting software now provides transcripts for session analysis, helping to identify key moments. However, you'll still need to manually review each session, extract important findings, and create your own highlight reels.
In-person testing
If a meeting room or similar space is available, participants could be invited for an in-person usability test session.
It's necessary to plan the room setup and make sure that all required equipment is available. Additionally, consider how you'll record the session.
One approach I've used involves attaching a phone to a tripod and positioning it behind the computer the participant is using to conduct the test. You then record the session on your phone.
An advantage of in-person testing is that it enables better empathy and possibly a more accurate sense of user behavior. However, the major drawback is the limited pool of participants, as they need to be able to attend in-person.
Elevate your research practice
Join over 320,000+ marketers, designers, researchers, and product leaders who use Lyssna to make data-driven decisions.
Usability testing software benefits
While the do-it-yourself approach can be beneficial when working with a limited budget and resources, it has several drawbacks. These include the time and effort required to set up, conduct, and analyze your tests. The need to compromise on the type of participants, such as family or colleagues, can also affect the quality of the feedback.
An alternative approach involves using online software that specializes in usability testing. Even on a limited budget, these tools allow you to conduct tests that yield quality feedback quickly.
Here's how online usability testing software can be beneficial in our given scenario.
1. Range of methodologies
The success of our customer support solution relies on us being able to assess if a user can successfully complete a key task using our prototype and measuring how easy it was.
We also need to understand how users navigate to find this information on the website, and what their first impressions are when they land on the page.
Relying on a single methodology to gauge the success of a prototype may not always offer a comprehensive understanding of a user's experience.
Usability testing tools usually offer a variety of methodologies, so you can run different tests depending on what you want to find out. In our scenario, we could conduct a prototype test to see if users can reach a goal screen, a tree test to find out if users can locate information correctly, and a five second test to see whether they can recall the purpose of the customer support landing page.
Not sure where to start? Explore our template library to discover a wide range of pre-designed templates tailored to specific jobs to be done.
2. Speed
Our goal when creating a prototype is to quickly determine if an idea solves a particular problem. If it doesn't, we take the insights gathered from testing to create a revised version. This process is repeated until we're confident that our solution fulfills the intended objective.
The speed at which we can iterate based on user feedback is where online usability testing tools come into their own.
I know that I can quickly create a test on Lyssna, recruit participants based on demographics and screener questions, launch my study, and often receive results within minutes. This allows me to quickly iterate and improve my prototype.
Why is this important? It negates the argument that usability testing hinders the speed of design work. Even better, it ensures that the delivered solutions are informed by user feedback and that any significant usability issues have been addressed.
3. Access to a pool of participants
Many online usability testing software solutions offer the ability to recruit participants from a research panel.
Typically, you can filter participants based on demographics. It's also beneficial to use screener questions where applicable to further refine your participant selection. This ensures you select individuals who closely resemble the actual users of your product or service.
Recruiting participants through a panel is often fast and cost-effective. It also eliminates many of the biases and challenges that may arise from testing via the do-it-yourself approach.
With tools like Lyssna, you’re also able to recruit from your own network. In that case, all you're paying for is the cost of your subscription or the cost to set up the unmoderated test.
4. Unmoderated testing
With the do-it-yourself approach, you'll typically conduct tests on a moderated basis. While this is a great way to uncover deep insights, it can be time-consuming when you're trying to quickly iterate on your prototype ideas.
Online software offers the advantage of setting up self-guided, or unmoderated, usability tests. You simply need to formulate the questions and tasks, and then wait for the results.
These unmoderated sessions are particularly useful during prototyping as they provide you with feedback more quickly. They’re typically less costly than moderated studies, allowing you to conduct multiple rounds of unmoderated testing at a similar cost to a single moderated study.
5. Automated results
Many online software solutions help you analyze your results. They provide quantitative data, such as time on task and rating the task's difficulty.
Additionally, it's usually possible to create instant highlight reels. These allow you to quickly share key findings with stakeholders. This method effectively shows any difficulties a user may have with the prototype. It tends to have a more significant impact on stakeholders compared to simply presenting numerical data.
How to choose the right usability testing approach
The do-it-yourself approach can be advantageous when working with a limited or non-existent budget. However, it tends to be more time-consuming, permits only sporadic testing due to the time and effort required, and involves recruiting participants who may have inherent biases.
On the other hand, specialized usability testing software requires a budget. With many options available, choosing the best solution for your needs can be difficult. However, once you've made a decision, the main benefits of this method are the speed and frequency with which testing can be conducted, and access to a larger pool of participants for testing.
How to get buy-in for your UX budget
While the do-it-yourself approach demonstrates that it's possible to conduct usability testing without much of a budget, you'll need access to a tool in order to run more frequent and faster testing.
I understand that securing a budget for usability testing can be challenging. I've personally experienced this in an organization where usability testing wasn't integrated into our workflow, and the work we designed and delivered went untested.
To get a budget for testing, consider taking the following steps.
Show outcomes from the do-it-yourself approach
The do-it-yourself approach is an excellent way to demonstrate the importance of usability testing to stakeholders without requiring their financial investment.
Although setting up, analyzing, and reporting findings from these sessions is time-consuming, presenting your results to stakeholders can be a powerful initial step. It highlights the value of testing in your workflow and can spark their interest.
Make your case, and start small
I was optimistic that by showing my senior stakeholders how testing could fit into our workflow, and the benefits it would bring to our output, I’d be able to secure a modest, continuous budget.
Initially, I used pay-as-you-go usability testing software. To strengthen my case, I requested a small investment to conduct a usability test that focused on a significant user task on the live website.
This test uncovered several recurring issues. When I showed the highlights to two of my senior stakeholders, they were taken aback. This revelation was the turning point that reshaped our team's workflow and led to the formation of a UX team. Our team then developed over the years to assist other teams in the organization to solve problems through a human-centered approach.
The takeaway is that you don't need to aim for a high budget right off the bat. You can begin with a free plan on a tool like Lyssna, paying only for user recruitment in order to start building your case.
Conduct effective usability testing on a budget
Navigating the hurdles of budget constraints in usability testing can feel like a steep climb. But through DIY initiatives and the support of tools like Lyssna, you can still prioritize and improve the user experience of your products.
By sharing the real impact of testing and gradually rallying support for a UX budget, you can pave the way for more empathetic and effective product design, enriching the lives of your users one step at a time.
--
Mark Jones is a freelance UX designer, researcher, and strategist with over 15 years of industry experience. Throughout his career, he has effectively demonstrated the benefits of and implemented people-centered approaches to problem-solving in the organizations he's worked with. Currently, he works with SMEs within the ecommerce, travel, and tourism sectors, helping them identify, understand, and solve the right problems to increase conversions on their digital platforms.
You may also like these articles
Sign up to our newsletter
We'll keep you updated with the latest UX insights and more.