We’re excited to announce the release of a brand new feature to Lyssna – card sorting! 

Card sorting is easily one of our most requested testing features, and our team has been busy working away behind the scenes to bring you this update. We can’t wait for you to try it!

Simply create a card sort test in your Lyssna account and invite your users to participate, or order quick responses from your target audience using our integrated 690,000+ participant panel.

If you’re interested in finding out more about what card sorting is and what it’s used for, check out our comprehensive introduction to card sorting guide. Otherwise, let’s dive in and explore how it all works.

Create cards and categories in a few simple steps

It might sound pretty obvious, but the first thing you’ll need to do when setting up a new card sorting exercise is create your cards. 

We’ve made it really easy to create cards within the Lyssna test builder. You can create cards by:

• Manually entering the names for each card.

• Copying and pasting a list of names from another document into the test builder (one card name per line) to create cards in bulk.

• Dragging and dropping a group of image files to create cards in bulk. You can then add names to each card, but this is optional. 

Speaking of images, they can be a great way to help your participants sort physical products into categories. You can add images to cards in bulk using the above method, or upload images to individual cards.

Cards can also be text-only and support optional descriptions, if you’d like to give your participants some more context.

For closed card sorts, you’ll also need to define categories so your participants know where to group cards. Category names can be entered manually or in bulk by pasting a list of category names into the test builder. 

As with cards, categories can support optional descriptions. But unlike cards, you aren’t able to add images to categories.

Additional card sort task options

We wanted to highlight a few other features you can use when creating card sort tasks.

Allow participants to sort all, or only some, cards

By default, participants will be asked to move all cards into one of the available categories. But if you want participants to have the option to leave some cards uncategorized, you can do that too. 

Limit the number of cards shown to your participants

By default, participants will see all of the cards you create. But you can limit participants to work with a smaller selection from your list of cards. For example, participants can be asked to categorize only 20 out of a total of 50 cards.

When this option is selected, our system tracks the number of times each card is shown to a participant and automatically optimizes this to keep views as balanced as possible. As more participants take your test, we'll present the cards with the lowest frequency first to ensure no cards are missed.

Why limit the number of cards shown? If you have a lot of cards to sort, the task can become difficult and time consuming for your participants. We recommend limiting a card sort to around 30–40 cards. This helps keep the task manageable while still allowing a large number of cards in your study to be sorted. Our system automatically balances the number of times each card is shown to participants.

Randomize how cards and categories are displayed

By default, the order of your cards is automatically randomized for each participant. This can be disabled in the test builder. When unchecked, cards will be ordered as they are arranged in the test builder. 

When running a closed card sort, categories aren’t randomized by default. But again, you can change this in the test builder.

Guide participants, ask additional questions, and use test logic

When participants open your card sort, they’ll be shown some detailed instructions on how to complete the activity. You can also enter further instructions if you’d like to add some more context to guide your participants.

Within the text builder, you can also add follow-up questions for participants to answer after they complete the card sorting task. 

For example, you might ask participants how easy they found the test on a scale from 1 to 5, with 1 being easy and 5 being difficult. If you want to ask follow-up questions to those who found the task difficult (e.g. those who chose a 4 or 5), you can then apply conditions using test logic. The language you use in your follow up-questions can be tailored to those participants. When analyzing your test results, you can then filter responses to those individuals (more on that below).

Create a seamless test experience for your participants

Our testing interface is responsive and supports card sorting tasks on both desktop and mobile devices. This means your participants can complete a card sorting activity no matter which device they’re using.

There’s a smooth drag and drop UI, and accessible buttons for sorting without drag and drop. 

If you’re interested in seeing what the card test looks like, why not take our open card sort test, where you can help us organize the site architecture of our (fictional) electronics e-commerce site.

As with all of our test sections, card sorting tasks support 30 languages. Selecting your language will ensure the test-taking interface (e.g. the default instructions and button text) is automatically translated. You’ll just need to enter custom field text in your chosen language.

Not sure where to source participants from? Our integrated panel allows you to quickly and affordably recruit research participants that precisely match your target audience.

Analyze your card sorting data in a variety of ways 

You’ve created and sent out your test, and the results are rolling in. Now comes the fun part – sorting and analyzing the data!

We know UX researchers and UX designers don’t have time to sift through lots of responses. Our aim is to make this part of the process as intuitive as possible, with different ways to view, sort, and analyze your data.

If you're curious to see what the card sort test results look like in practice, here are the results from our electronics e-commerce store example above.

View card sorting data

Here’s a breakdown of the different ways you can view data within your Lyssna account.

Card view

The default for viewing test results is the card view. This is where you’ll see the complete list of your cards and the categories each card is sorted into. This applies to both open and closed card sorts. 

You’ll also see:

• The percentage of participants that sorted each card into a category.

• The number of participants that sorted each card into a category.

You can also filter within a category to only show participants who selected that option, along with any follow-up responses they gave to other tasks within the test (e.g. a five second test or preference test). 

Category view

For deeper analysis, you have the option to review results by category. In this view, a complete list of all categories is displayed, including:

• Each card that’s sorted into the category.

• The percentage of participants that placed each card into the category.

• The number of participants that placed each card into that category.

Similar to the card view, you can filter to only show participants that sorted a particular card into that category, and any other feedback they added throughout your test.

Agreement matrix

The agreement matrix is a great way to review your results at a high level. 

On the left side of the matrix you’ll see cards listed in separate rows. The categories are displayed in columns. On the cells you’ll see the percentage of participants that placed each card into the category. You can hover over each cell for more detailed info. We automatically highlight the card’s highest agreement score in deep blue, and highlight in light blue if there’s a tie.

Similarity matrix view

For open card sorts, you can use the similarity matrix to see how often pairs of cards are sorted into the same category, regardless of which specific category they were sorted into. The matrix is ordered to show clusters of cards that are sorted together. We automatically highlight pairs of cards with more than 30% similarity in dark blue and pairs of cards with more than 0% similarity in light blue. Those with 0% are left empty and displayed in gray.

In addition to the above views, you can also filter test results based on demographics, view how each individual participant completed the card sort, and export to CSV.

Standardize open card sort data

After collecting results in an open card sort, you’ll likely need to standardize the categories created by your participants in order to get some useful data.

For example, say you’re researching categories for a travel booking site. Some of your participants might opt to create a category called ‘Day trips’. Others might enter the same term but use lower-case capitalization, or those with really strong feelings may even go for max caps. Some of your participants might type in ‘daytrips’ or ‘daytrps’ by accident. Others may have used different but related terms, like ‘Theme parks’ or ‘Road trips’. 

There are other considerations too, like international spellings (we’re looking at you color/colour) and usage (hello zucchini/courgette).

Your job as the researcher is to decide which categories mean the same thing.

Thankfully, our system automatically groups categories with exactly the same spelling, even if different title cases are used. So in the example above, ‘Day trips’ and ‘day trips’ would automatically be grouped together under whichever capitalization the word received first. 

Typos and different spellings won't automatically get grouped together, but you can easily do this manually, and undo automatic groups if you’d like to.

Categories can be grouped and standardized under the categories tab on the results page. You can also merge grouped categories together here. For example, you might have a group of eight categories all named ‘Day trips’ and a group of two named ‘Theme parks’. If you decide they’re all referring to the same thing, you can merge them together. You can also customize the name of a group of categories.

Ready to get started with card sorting?

Open card sorting and closed card sorting is ready to use in the Lyssna test builder. And if you’re looking for specific card sorting how-to’s, check out the video below, our help center articles, or reach out in-app to our awesome support team. 

Our new open and closed card sorting feature is available to customers on all Lyssna plans. Sign up to today and unlock a new way to test and gain research insights into your information architecture.