Card sorting for UX

Card sorting is a well established user experience (UX) research method where users group topics into categories according to criteria that makes the most sense to them. Join us in this video where Paul, one of Lyssna's Software Developers, walks through how to use the card sorting feature in Lyssna.

Transcript

What is card sorting? (0:00)

Kia ora koutou everybody. My name is Paul. I'm a software engineer here at Lyssna and I'm based in Aotearoa New Zealand. I'm here to talk to you today about card sorting which is a research methodology that we've recently added to the Lyssna platform. 

Today we're going to cover what card sorting is, what kind of research problems that it can help you with, and how you could use the Lyssna platform to create a card sort to recruit participants and to analyze your results. 

Card sorting is a tool that lets you give your participant a list of cards and asks them to sort those cards into different categories. Those categories can either be predefined, which we call a closed card sort, or you can get the participants to create their own groupings in whatever way makes sense to them. 

Card sorting is often used when evaluating potential information architecture. An open sort could be a great way to generate different ways of grouping your content. A closed sort could be useful to validate whether a specific IA makes sense to your participants. 

There are lots of different potential use cases for card sorting though it's a great tool to have any time you want to understand your users mental models of a system or to understand how people group things. 

How to create a card sort test in Lyssna (1:01)

Let's jump in now and have a look at the process of how you could create a card sort on the Lyssna platform. So here I am on the Lyssna dashboard. I'm going to create a new test. Lyssna tests can have all sorts of different kinds of sections on it and you'll see at the bottom here there's a new card sort section which you're able to add so you could add a card sort in combination with other types of of sections or you could add multiple card sorts to one test if you'd like. 

Today we'll just do a simple single card sort and let's try to do an open card sort today so I can show you some other really cool grouping tools in the results section. So when we create our sort we have several options. We can either require participants to sort all of the cards or perform a partial sort. The participants are going to be creating names for each of their groupings that they put together but we could allow that to be optional as well and we could just show them a subset of the cards if we have a large number of cards and we don't want to show all of them. 

Then I'll have an instruction section here and my list of cards. You can type in the cards one by one but if you have a predefined list you can also just paste it in, so here's a list of sports that I prepared earlier. I'm going to copy that, paste it into the card section, so now we have our cards. 

Today the sort I'm going to work on is a list of different sports and I'm going to ask people to group them in a way that makes sense to them, so perhaps we're organizing a sports website and we want to know how the information architecture might make sense when we have these different sports to select from.

You can optionally add descriptions. If you have more information you need to pass about your cards or even images if you have a visual sort that you'd like people to do, or you could add images to each card and you can have a combination of cards with and without images, if that makes sense for your sort. 

Today let's just do the simple test sort. The last option we have here is you have the ability to randomize the order of cards. If you think the order they're presented will affect how people do the sort perhaps it would be useful for you to randomize that, and that is on by default. 

And then as with any Lyssna section you can add follow-up questions afterwards if you want to get more information about how the participant completed the sort or how they found the experience. Today let's just go with the card sort. 

How to preview your card sort test (3:12)

Once I'm happy with my card sort, I will need to give it a name, Sports, this is just for internal use. We can go to the save and preview option and this will let us see what the participants are going to experience when they fill out our card sort. So you can see first of all, you'll get an instructions page that explains how a card sort works. 

The user-defined instructions that I put in are going to show here at the top and there's an image that demonstrates how it's going to work then we begin the card sort so I've got an empty section here where I can create categories and I've got my list of cards over here on the left and I can take these either using drag and drop or with keyboard navigation and I can drag them across to an empty space over here and then that will create a new group the first time and I can give that group a name or if I want to come back later once I've figured out a bit more about the sort and name them at the end. Then I have the option to do that too so let's drag these cards into place.

How to recruit participants from Lyssna’s integrated panel (4:16)

Once you're happy with the way your test is set up you can come over here to the save and continue button and that will take us through to the recruit page, so this page is going to let us find participants to take our tests. 

There's two main ways that you can do that in Lyssna. The first way is you can recruit from our panel, so we have a global pool of participants and this will let you test with your target users. You can choose the demographics that you like and you can specify exactly what you need to get the people that you want. One click, very easy. 

The other option is if you have your own participants, your own pool of people to do research with, you could set up a recruitment link over here and you could send that out yourself to different people you'd like to talk to. Today let's do a panel order.

So I'm going to ask for 30 participants. You can choose the number that you like, the costs will be based on the length of your test and the number of responses that you've asked for. Over here you can see demographic filters to specify who you'd like to fill out your test, if that's important for the results that you're asking for. Today, I'm not too worried about who's going to do it, so let's submit the order like this.

Okay the order has been placed. It will only take a few minutes for the responses to start coming in, so let's come back here when they do. 

How to review your card sort results (5.30)

And once the results have started coming in. We can click over to the results tab in Lyssna to start seeing information about how people have filled out our card sort. So you can see here I've got nine total participants, so we haven't yet got the full 30 that I'm waiting for but more will keep coming in and we can have a look at the interim results in the meantime. 

You'll see down the left you can do some filtering of different demographics that people have entered. If you need to filter the results for some reason you can also have a look at everyone who's filled out your tests so far. You can see we've got people from across the globe already which is cool. 

If you like, you can click into the individual participants that have filled out your card sort. You can see more information about them and you can also see exactly how this person filled out the card sort. So this person's decided to group our sports into three categories: high intensity sports, low medium intensity sports, and ball sports, and that's what they've understood from our exercise. Because we didn't give them much direction on how to sort it, you're going to see a lot of different approaches across the different people who have filled out your test. 

So let's have a look at the aggregated results over here. So in our card sort section we've got four main tabs that we can work through to see what's going on in our card sort. The one we're on by default here is the cards tab. That's going to show us a list of all of the cards that we defined. So you can see our sports down here: football, basketball, rugby, and it's going to show which categories those were most commonly sorted into. And you can see where people have used the same names for open categories. We've grouped them into the same category together, so all of the ball sports entries are here. So that was the most common one so far out of the nine people that have filled it out so far. We can scroll down to see more if we want to and we can click the show all button to see the entire list of all of the categories that this was sorted into. 

The next tab is the categories tab, so this will show us the reverse view, so it'll show us down the left hand side all of the different categories that people created and we can see which cards were most commonly sorted into them. So you can see ball sports have all of the sports with balls in them. Makes sense. Let's see one down here, you can see this person has created a category called horse racing with just horse racing in it, because I guess they couldn't figure out where else to put that. 

With open categories like this, sometimes people will create similar categories that aren't identical but really they should be grouped together, so for example we have sports with ball and probably this is the same thing as ball sports just expressed a slightly different way, so we could click that here and we could go and find the other entry for ball sports right at the top, and now we can group these two together because probably these people are expressing the same concept. And when we do that we'll see this modal pop up that will tell us the different people's categories that we're sorting in and how many cards are involved in them.

How to standardize open card sort categories (8:07)

We can give our new grouping a name, choosing one of the names that participants used or a whole new name, if that's something that makes more sense to us and helps with our analysis. 

Then once we've done that we can save, and now that it's grouped you'll see it appears slightly different here with the stacked cards and the numbers over here have changed to incorporate all four of those different participants' results into their grouping. 

There's also some really great sorting and filtering tools that you can use on both the cards and categories tab to help you find what you need if you're just looking for ball sports. For example, you can do that quite easily here. 

How to use the agreement matrix (8:39)

Okay, the next tab I'd like to show you is the agreement matrix tab and you'll see I've changed to a slightly bigger sort with a bit more data so you could see what this might look like after you'd gone through that grouping exercise. 

So in this case we're dealing with an airline website and the agreement matrix is going to show you a table of all of the groupings that you've created and all of the cards that your sort has, and then what percentage of people put that card into their grouping. 

So you can see here for example we've got a booking grouping. Book flights. 46% of the participants in your test put book flights under booking, and you can see that the cards are ordered in such a way that it kind of naturally makes these potential groupings, these blue background cells are the highest percentage for that card so this card was most commonly put into booking out of all of these different possibilities. And you can see that naturally forms clusters along here of these, maybe would be the groups that you might want to put those cards into or it gives you a starting point to figure that out. And you can scroll along and see all of the data in this test. It's quite a big test so lots of information to look at. 

How to use the similarity matrix (9:40) 

And the last tab, I'm going to show today is the similarity matrix. So this is purely a card based look at your data, so it's thinking about pairs of cards and how often those cards were sorted into the same place. Doesn't matter what that category was named or what other things were in there, it's just thinking about pairings and how often the two cards will sit together. 

So you get this kind of triangular matrix and you'll see down the diagonal, there's natural clusters forming. Maybe these loyalty points and membership benefit cards were often sorted together, so perhaps there's some kind of natural grouping forming there. These reports, careers, story, and vision, these things tend to be sorted together, so perhaps there's something there for us to think about when we're arranging our content. 

How to export your card sort data to CSV (10:23)

And the last part of the results I'd like you to show you today is the CSV export. So it's really important to us that you're able to get your data out and process it further if you'd like to use it with another tool or do your own analysis.

It's really powerful to be able to have access to that raw data, so on any Lyssna test, at the top here you can hit the export results as CSV button, and when you do that you're going to get a CSV that looks a little bit like this, so each row in your CSV is going to be one participant who filled out your study and you can see all the demographic information about them. If you had other sections on your test you'd be able to see how they filled out those sections. 

You'll be able to see how long they took to do certain things and then in the card sorting section it's going to show each card is going to have two columns. The first one will be the group that the participant put in for that card and then the second one will be how you've grouped that in the results. In this case we haven't changed the grouping so it shows the same in both columns but if you had regrouped the data you'd be able to see the participants' raw information as well as your own grouped information, and it's going to show that for every card in the sort. So quite a lot of information, but if you need access to that to do some further analytics it's all there for you. 

Conclusion (11:38)

And that's all there is to it. Thanks for joining me on this card sorting journey today. We've just really scratched the surface of what our new tool can do but hopefully it's inspired you to jump in and have a play and try to do some card sorting studies of your own. There's a lot of interesting research opportunities that this tool unlocks and we're really looking forward to seeing what people do with it. 

Thanks so much for your time today, really appreciate it. If you have any questions or feedback for us please don't hesitate to reach out on social media and make sure you like and subscribe to see more information about what we're working on next.