Launching a new product can be a hit-or-miss affair. Often, what separates a market leader from a market flop isn't just innovation, but understanding and meeting real user needs. And this is where many products stumble – failing to connect with their intended audience right from the start. 

What if you could not only catch these missteps before they happen, but ensure your product launches with a bang?

Enter evaluative research: your key to unlocking the insights that transform average products into user favorites. 

In this article, we dive into what evaluative research really involves, why it's crucial, and how to apply it effectively. We'll provide you with actionable strategies that ensure your product doesn't just meet, but beats, your users’ expectations.

What is evaluative research?

Unlike generative or exploratory research, which is all about brainstorming new ideas at the start of the design process, evaluative research focuses on refining and validating product designs or prototypes. By using different methods like usability testing, surveys, and A/B testing, we can pinpoint exactly where users might struggle and what they prefer. 

With targeted evaluation, you can make informed decisions that significantly enhance UX, design, usability, and – ultimately – the product’s ability to compete in the market.

Why is evaluative research important? 

Evaluative research plays an essential role in shaping products that are functional and highly attuned to user needs. Here’s why it’s an indispensable part of the development process.

Enhances product usability and experience 

Through rigorous testing with real users, evaluative research identifies usability challenges, big and small. Addressing these issues allows you to refine the product to align with user behavior and expectations. 

Drives cost efficiency in development 

Identifying potential design flaws and user experience issues early through evaluative research can prevent costly changes after the launch and damage to the brand’s reputation (the Apple Maps disaster is a prime example – and shows that it can happen to the best of us!). 

Boosts user satisfaction and retention 

Products refined through user feedback are more likely to resonate with their intended audience, resulting in higher satisfaction rates. This satisfaction fosters loyalty and can turn users into brand advocates, enhancing word-of-mouth marketing and significantly improving retention rates.

Supports data-driven decision making 

The insights gathered from evaluative research provide a solid empirical foundation for product development decisions. This data-driven approach makes sure every modification or addition is backed up with evidence, increasing the likelihood of the product's success in meeting user needs and achieving business goals.

Incorporating evaluative research throughout the product development life cycle is key to delivering solutions that are both effective and competitive.

How different industries use evaluative research

Evaluative research is a powerful tool, adapting to the unique challenges of various sectors. Here’s a closer look at how it can make improvements across key industries.

Banking, financial services, and insurance

Evaluative research is crucial in improving the security, usability, and personalization of fintech and insurtech apps and platforms. It ensures that complex transactions and data privacy features meet user expectations while complying with stringent regulatory standards.

Energy

In the energy sector, evaluative research fine-tunes user interfaces for control systems and customer portals, making them more accessible and effective. This helps users better manage their energy consumption and interact with services more efficiently.

Travel

The travel industry benefits greatly from evaluative research by refining booking systems and mobile apps. This makes navigation simpler and enriches the user experience with relevant, timely information to meet the dynamic needs of travelers.

Software

For software developers, evaluative research is indispensable in iterating products that are user-friendly and meet the diverse functional requirements of all users.

Marketplaces 

Online marketplaces use evaluative research to enhance the user journey, focusing on ease of use, trust, and transaction security – vital for customer retention and satisfaction.

Ecommerce and retail

Evaluative research streamlines online shopping experiences, testing and perfecting new features – like augmented reality in shopping apps – to ensure they meet consumer expectations.

Fast-moving consumer goods (FMCG) 

Companies in this sector apply evaluative research to test and improve packaging, branding, and usability of products, making sure they’re both appealing and practical for consumers.

Media and publishers 

Media outlets and publishers use evaluative research to tailor content delivery platforms to the preferences of their audience, optimizing the consumption of content across apps, websites, and other interactive platforms.

Healthcare 

In healthcare, evaluative research develops user-friendly interfaces for electronic health records, appointment systems, and telehealth services, prioritizing accessibility and ease of use for better patient experiences.

Harnessing evaluative research allows industries to enhance product design and user experience while significantly boosting customer satisfaction and loyalty, which, in turn, drives business success. It's quite the virtuous circle …

Elevate your research practice

Join over 320,000+ marketers, designers, researchers, and product leaders who use Lyssna to make data-driven decisions.

Sign up for free

Sign up for free

Sign up for free

Evaluative vs generative research 

Evaluative research is primarily concerned with refining existing products or prototypes. Typically quantitative, it validates hypotheses about user behaviors and preferences, making existing products more user-friendly and effective.

On the other hand, generative research is more exploratory and predominantly qualitative. Used at the beginning of projects, it dives deep into user needs, motivations, and potential issues. This type of research is key to sparking new ideas, concepts, and features based on real user insights. Methods used in generative research include user interviews, ethnographic studies, and observational research.

By integrating both evaluative and generative research, you can gain a comprehensive understanding of what users truly need and want. This dual approach informs the innovation process, ensuring solutions are practical and align well with user expectations.

Types of evaluative research 

Evaluative research is divided into two main types: formative and summative. Each type plays a distinct (and essential) role in the product development lifecycle.

Formative evaluative research

Formative research is integral during the product development process, focussing on the continuous monitoring and improvement of the product's performance. It's an ongoing process that allows UX designers to keep refining their solutions until they achieve the best user experience.

Summative evaluative research 

Conducted at the end of the development cycle, summative research assesses a product's overall effectiveness and user satisfaction. Its primary aim is to determine if the product meets the predefined objectives and fulfills user needs. Results here often inform critical decisions regarding product launches or significant updates.

By applying the correct method at each stage of the development process, you can secure both immediate improvements and long-term success, enhancing user satisfaction and product viability.

Evaluative research methods

So, that’s the what; now let's look at how all this fine-tuning is done in practice.

Here's an overview of the most common techniques.

Usability testing

Usability testing observes how users interact with a product, allowing us to pinpoint where they encounter difficulties and what aspects are most user-friendly. This hands-on method offers direct insights into the product’s functionality and user interface effectiveness.

A/B testing

This technique pits two versions of a product against each other to see which one users prefer. It's an excellent way to measure user engagement and satisfaction with specific features or designs.

Surveys 

Surveys gather both quantitative and qualitative data by asking users directly about their experiences and perceptions. This provides valuable feedback on the product’s impact and areas for improvement.

Heatmaps and click tracking 

By visually mapping where users click and how they navigate through a product, these tools reveal which areas attract the most attention and interaction, offering clues on how to structure the user journey better.

Tree testing

Tree testing assesses how well users can find information based solely on the navigation structure. It's invaluable for evaluating the clarity and effectiveness of your site or app’s organizational schema.

Closed card sorting

In closed card sorting, participants categorize information into predefined groups, helping to refine the information architecture to align more closely with user expectations and mental models.

Session recordings

Watching real user sessions helps identify where users experience friction or confusion, making it easier to pinpoint exactly where improvements are needed.

Eye tracking 

By understanding where users look first and what holds their attention, eye tracking provides insights into how to design more engaging and effective user interfaces.

Choosing the right evaluative research method will depend on your project's very specific goals and needs. But, with careful choice and application, you can unlock the feedback you need to improve your product's design and user experience. 

Evaluative research questions

Effective evaluative research hinges on asking the right questions – both directly to users and internally among development teams. This section outlines questions for various methods, from direct user feedback during A/B testing and surveys to internal reviews following session recordings and eye-tracking. 

Usability testing

• How easily can you add a product to your shopping cart?

• Were there any parts of the process that were confusing or frustrating?

• What was your overall satisfaction with using our website?

A/B testing

• Which version of the home page led to more user engagement: Version A or Version B?

• Did users find the checkout process easier in Version A or Version B?

• Which layout version resulted in higher conversion rates?

Surveys and questionnaires

• On a scale from 1 to 10, how satisfied are you with the features of our app?

• What improvements would make you use our app more often?

• How likely are you to recommend our service to a friend?

Heatmaps and click tracking

• Where do users click most frequently on the product page?

• Are there areas of the page that users consistently ignore?

• What path do users typically take when navigating through the site?

Tree testing

• Were users able to find the information they were looking for using our website’s navigation?

• How many steps did it take for users to locate a specific item in the menu?

• What sections of the site do users most frequently access from the home page?

Closed card sorting

• How do users categorize information related to our product features?

• What labels do users find most intuitive for grouping our services?

• Are there any categories that users consistently misunderstand or misplace?

Session recordings

• What are the common actions users take before abandoning their shopping cart?

• How do users interact with the search functionality on our site?

• At which points do users show signs of frustration or confusion?

Eye tracking

• What elements on the landing page attract the most immediate attention from users?

• How do users' eye movements differ when presented with different content layouts?

• Which advertising placements do users notice first, and for how long do they focus on them?

How to conduct evaluative research

Conducting evaluative research effectively involves a structured process to ensure the data collected is actionable and insightful. Here’s a step-by-step guide to help you navigate the process.

1. Define your goals

Clearly define what you want to learn from the evaluative research. Set specific, measurable goals that align with your product’s development objectives.

2. Choose the right methods 

Based on your goals, select the appropriate evaluative methods. Consider factors such as the stage of the product, the information you need, and the resources available.

3. Recruit your participants

Gather a representative sample of users who reflect your target audience. Ensure diversity in the participant pool to capture a wide range of experiences and opinions. (Using the Lyssna research panel gives you access to over 530,000 active panelists across 100+ countries and 37+ demographic attributes.) 

4. Prepare your study 

Develop the tools and materials you need for the research, such as survey questions, testing scripts, or prototype designs. Make they're clear and directly relate to your research goals.

5. Conduct your study 

Execute the chosen methods. This might involve observing users during usability tests, distributing surveys, or tracking user interactions through software.

6. Analyze the data

Systematically analyze and synthesize the data you gather to uncover trends, patterns, and insights. Use statistical and qualitative analysis techniques to interpret the results.

7. Report your findings 

Collect the findings into a report summarizing the insights, implications, and recommended actions. Make sure the report is clear and actionable.

8. Implement changes 

Use the insights gained to make informed decisions about product enhancements. Prioritize changes based on the impact and feasibility.

9. Reevaluate

After implementing changes, consider conducting additional evaluative research to assess the effectiveness of your changes and continue refining your product.

This structured approach ensures that as well as providing deep insights into user behavior and preferences, your research directly informs product improvements.

Your go-to user research platform

The best teams use Lyssna so they can deeply understand their audience and move in the right direction — faster.

Sign up for free

Sign up for free

Sign up for free

Evaluative research best practices

Here's a roundup of essential practices to make sure your research is as accurate and effective as possible.

By embedding these best practices into your research process, you'll enhance the reliability and the impact of your findings, paving the way for more informed decision-making.

Minimize bias 

Keep your data clean and unbiased by using random sampling, neutral questions, and well-trained data collectors.

Ensure adequate sample size 

Make sure your conclusions are solid by analyzing a sample size that truly represents your target audience.

Define clear objectives 

Know exactly what you're looking for before you start to keep your research focused and relevant.

Implement robust data management 

Protect the integrity of your data with strict handling, storage, and documentation protocols.

Account for contextual factors  

Consider the setting and circumstances of data collection to avoid misinterpretations.

Conduct pilot testing 

Identify and fix issues early by testing your research design and instruments beforehand.

Standardize data analysis 

Ensure consistent and valid results by sticking to standardized analysis methods and consulting experts when necessary.

Evaluative research in action

We don’t like to ‘toot our own horn’ too much, but we’re privileged to have played a part in helping companies around the world make smart, impactful design decisions. Here's one example from Stayery.

Stayery’s story

In Berlin's bustling startup scene, Stayery, a leading serviced apartment company, use the power of evaluative research to fine-tune its offerings and ensure each property is ready for guest satisfaction. Eveline Moczko, Head of Product at Stayer, shares how their commitment to rigorous testing has been instrumental in their growth trajectory.

The challenge and strategic approach

Stayery faced the challenge of making sure that its customer journey and property design were flawlessly aligned with guest expectations before committing to expansion across Europe. Eveline knew preemptive testing was essential: "Before you replicate a customer journey, you want to get it right. Fixing it afterward gets expensive."

To tackle this, Eveline and her team used various testing methods, focusing on gathering rapid, actionable feedback. From preference tests evaluating copy and image choices for listings on Booking.com and Airbnb, to more detailed tests exploring guest preferences, the team left no stone unturned.

Efficient testing with Lyssna

Lyssna proved invaluable for Stayery, offering quick tests that provided not just quantitative results, but rich qualitative feedback that guided more informed decisions. This depth of information was crucial for Eveline and her team as they refined every aspect of the guest experience.

The ability to conduct these tests rapidly (and at low cost) allowed Stayery to avoid potentially massive financial mistakes – like a costly investment in inappropriate electronic locks for their properties, which could have led to significant expenses across multiple locations.

Proven results

Using Lyssna not only empowered Stayery to make smart design choices; it significantly impacted their business outcomes. By testing key elements of their customer journey before scaling, Stayery made sure that each new property was optimized for success and met their high standards.

As Stayery prepares for expansion, its story serves as a compelling example of how effective evaluative research – when combined with tools like Lyssna – can lead to substantial savings and refined customer experiences. This approach sets Stayery up for a successful rollout across Europe, demonstrating the profound impact of smart, data-driven decision-making in the hospitality industry. 

For more detailed insights into how companies have used Lyssna, visit our customer stories.

Elevate your product with evaluative research

Whether you’re a startup veteran or a first-time founder, the power of evaluative research is your key to making data-driven decisions that propel your product to new heights. When you harness these strategies, your team isn't just making products that work – they're creating experiences that resonate

We hope you’ve enjoyed reading and can put this knowledge to good use! 

And if you're ready to take your product development to the next level and truly connect with your users, we're here to help. Explore our resources or sign up for a free plan to try our platform for yourself. 

--

Pete Martin is a content writer for a host of B2B SaaS companies, as well as being a contributing writer for Scalerrs, a SaaS SEO agency. Away from the keyboard, he’s an avid reader (history, psychology, biography, and fiction), and a long-suffering Newcastle United fan.

FAQ

What is the main goal of evaluative research?

What are the characteristics of evaluative research?

How does evaluative research differ from other forms of UX research?

Can evaluative research influence business decisions?