Workshop overview

No matter what type of site or product you work on, there’s one common challenge: ensuring users can quickly and easily find the content that’s valuable to them. Whether it’s products in an e-commerce store, events on a meetup platform, or tours on a travel app, discoverability is key to user satisfaction and business success.

If you’ve ever struggled with questions like “Where should we put this?” or “How do we make this easier to find?” this workshop is for you. You’ll leave with practical tools and strategies to improve navigation, enhance discoverability, and create a better user experience.

About the host

Joe Formica leads workshops and creates educational content at Lyssna, bringing over 10 years of experience in UX and education. His practical, hands-on teaching approach has helped thousands of designers develop their skills and create better products.

Transcript

[00:00:00] What's up, everybody welcome. My name is Joe. I'm going to be your instructor guide today for all things. Navigation card, sorting tree, testing information architecture. We're going to cover a lot of good stuff in the next 90 minutes or so. Into this session, I'm excited for it. We got a lot of examples, activities, studies, and results. 

[00:00:26] My name is Joe Formica. I'm the design advocate here at Lyssna. I've been a senior product designer and researcher for a while now. And I've also been teaching design and research in all different areas now for a while as well.

[00:00:40] So I know we had a couple of people here whose names I recognize who probably joined a lot of my classes either recently or with bite size courses. So it's good to see you all here. And the main thing I'm going to be bringing a lot of. My work experience and some examples into really every workshop, but definitely a few big ones tonight.

[00:01:00] And in my background, I've worked on lots of different projects for all different types of companies, all different size companies, all different. Subject matter areas. So in that time, I've seen a lot of how things can go, let's say, either in design and research. And I'm looking forward to bringing that here tonight and answering your questions as they come up and hopefully having a really productive session.

[00:01:25] No matter what topic we're talking about. My goal for teaching any session is for all of you to leave with some new tangible skills and some confidence to improve your day to day work. So I want you to learn stuff and have new information in your brain. But more importantly, I want you to leave here, especially those of you who have projects that you're working on or redesigns that you're doing.

[00:01:50] I want you to leave here coming to that project tomorrow. feeling like you have some new tools and some new confidence to do it well. So I'll ask you these questions at the very end of the workshop, right before we break. Number one, did you enjoy this workshop? Hopefully, you're spending your Wednesday night or Thursday morning with me. It's enjoyable and we have some fun here tonight.

[00:02:13] More importantly, do you feel like a better, more confident designer or researcher than you were 90 minutes ago? So that's my goal. I'll loop back to those right before we break for the evening and hopefully get at least a couple of thumbs up from everyone here. All right, some quick tips for getting the most out of our workshop.

[00:02:32] Everything is recorded and I'll be sending out the resources. I have them all in this fig gem here, which I'm going to send out with the slides, links for everything. I'll share it all in the chat, but just know that if you miss anything, if you have to leave early, if you want to come back and review some of this stuff.

[00:02:51] You will have it all long after the live workshops done. Keep the chat open and active. We're going to use the chat a lot tonight. I'm going to be sharing some links in there. I'm also going to be throwing some questions out to the group and having everyone respond in the chat with some thoughts or some answers.

[00:03:08] If you have questions, you can drop them in the chat as well. Everyone's doing a good job of that as usual, but keep the chat open and I got my eye on it over here. And we'll be using it quite a bit. Finally, stay comfortable, happy, and hydrated. I'm sure everyone's had, either had a long day or a long day ahead, and you're taking an hour plus out of your day to be here with me.

[00:03:31] While we're going to cover a lot, Make sure to take care of yourself. If you need a quick break, take it. And if there's anything that I can do to make things more comfortable, just let me know. All right, cool. Let's get started. We're going to start with a study. You guys are all going to be participants in a tree test, and we're going to come back to this later.

[00:03:53] I'm going to talk about what tree tests are, why they're important, how they loop into the project that we're mini project we're going to be working on tonight. But we're going to start by you're going to start as a participant. So I'm going to copy this link and drop it in the chat

[00:04:10] and go ahead and click on that link. And you should see this tree test here. Let me make sure that's the right one before we go far. Yes. Okay. So all the instructions are on there, and this might seem a little out of context, but read through them. Imagine that you want to book a kayak rental on the Hudson River, maybe for an upcoming trip to New York.

[00:04:36] Use the navigation below and indicate where you'd expect to find information about this activity. Just give me a thumbs up. Everyone can open this link. Everyone can see what I'm seeing here. Awesome. So just take a minute or two to complete this and We'll leave it at that. We'll, we're gonna come back later, check out some of the results, and again, see how they loop into the project that I'll introduce in a second.

[00:05:02] I'm just gonna pull up the results on my screen just so I can make sure these are coming in. All right, cool.

[00:05:09] Give everyone another minute or so on That. Good? Good, good. Nice.

[00:05:16] Awesome. That tree test is gonna make a lot more sense in a couple minutes. Cool. So you can finish that up and we'll come back to it in a few minutes to check it out. All right, let's get started with kind of an overview here. The title of this workshop, design better navigation with using card sorting entry testing.

[00:05:39] You already have a tree test under your belt, and we're going to get into card sorting and some other methods. But first, I want to just take a step back and give an overview of really the bigger topic that we're talking about here, which is information architecture. Information architecture is a simplified definition, but I think a good one and one that follows a lot of the themes we'll be talking about here tonight.

[00:06:02] Basically, the behind the scenes organization of content, navigation, and structure that help users find and discover and browse things on your site app, digital product. And I wanted to include that behind the scenes aspect because we're going to connect some of these research methods and deliverables that you create for information architecture to organize the content on your site.

[00:06:30] They seem behind the scenes, but can really have a major impact on the user experience and the things that the user's directly interacting with. So quick little overview on some information architecture methods and deliverables information architecture, all about organizing, grouping. The content on your, let's say website, and we have a bunch of research methods and deliverables that kind of fall under that umbrella card sorting, which we're going to get into more detail on is used to understand how users naturally group prioritize and categorize content.

[00:07:07] So a very valuable and useful research method. That is going to contribute to good information architecture site that's well organized and easy for users to find the things that they're looking for tree testing, which we're going to cover as well is used to test and validate navigation structures basically by seeing how easy it is for users.

[00:07:30] To find content where they click first, the paths that they take to get there. Think of it as a mini, a very low fidelity stripped down test. That's meant to see how efficiently and effectively. people can, users can find things on your site. Site maps and taxonomies, more of deliverables and something we're not going to cover as much tonight, but still an important part of information architecture, diagrams and site maps like this used to define the overall structure and hierarchy of a website or an app.

[00:08:06] Think of basic parent pages and sub pages as an example. That also falls under information architecture and even user flows, which are used to map out the step by step start to finish paths that a user can take through an experience. All of these methods and deliverables all come back.

[00:08:27] Under the umbrella of information architecture and all contribute in different ways to creating a site and an experience that's clear, organized and easy for users to navigate. What I mentioned when I brought up, let me pull the chat up here just to make sure I'm not missing anything, just about to push go on card sort test.

[00:08:47] Nice. Information architecture feels invisible. until it's bad. And I think this goes right along with the, 99 percent of design is invisible and all those good design is invisible, bad designs, pretty apparent. Same thing with information architecture. Even if you just look at, some of these graphics that I showed here, these are all diagrams.

[00:09:10] These are all behind the scenes maps that are creating some kind of structure for your site, but it's not what the end user. I see it so often when I teach information architecture for the first time or when students are getting used to it. It's hard to make that connection between what's going on there and really the impact that it has on the end user experience.

[00:09:35] So good information architecture. is based on research just like anything else in a good product. It's based on understanding your users, knowing their needs and reflecting that in the experience and what it looks like or what it feels like when there is good, researched, structured information architecture where users are able to navigate confidently.

[00:09:58] They know where to find things they can quickly move around on your site, they can jump back and forth between the different things accomplished tasks efficiently. So there's this level. I think confidence is the word I keep coming back to where users are able to navigate and browse confidently.

[00:10:16] And that is a very direct. Results of good, researched information architecture. Bad I wrote AI here. I knew I was going to mess that up somewhere. Bad information architecture or a site that's disorganized or maybe not based on users mental models of how things are organized really leads to confusion, hesitation, and frustration.

[00:10:42] And I got some examples that I think from my own experience that hopefully Might be relatable to some of you. These are the things I will say to myself when I'm on a site that has a maybe confusing navigation. How do I get back to that thing that I was just looking at? I use, this happens to me with Pinterest all the time.

[00:11:03] I use Pinterest for like design inspiration stuff and it's good for that because there's just so much stuff there. But I always find myself like finding something cool and then say, two minutes later saying, wait. How do I get back to that? I'm in 50 different pop ups of that screen. Where was that thing that I just found?

[00:11:20] I saved it. Where did it go? I don't know if anyone else feels this with interest or any other apps, but the feeling of how do I get back to that thing? I was just looking at very let's say, obvious sign or symptom of a. Maybe information architecture that needs some work. If I click this, will I lose my spot?

[00:11:41] I think everyone's probably felt that hesitation, your fingers like hovering over the mouse or your trackpad. You're like, I think this is what I'm supposed to do, but I don't know where it's going to take me or if it's right. I felt this when I was renewing my driver's license recently on a DMV website.

[00:12:01] filling out a long form and then that very scary click at the end where you're not really sure. What's going to happen or where it's going to go and where am I right? I did a video a while ago on Zara because so many people told me that Zara's website is bad I decided to do a breakdown of it and this was how I felt using it.

[00:12:22] Where am I? How do I get back to that thing? What clothes am I looking at right now? How do I get to see all of the I don't know shirts in this category and just feeling confused and lost? Honestly, not navigating it confidently and really feeling situated with the site. So these are a couple examples just more bringing it to the user side of how it feels and the impact that good or bad information architecture can have on the experience.

[00:12:54] Give me a thumbs up. Does anyone, you guys, can you think of an, even if you can't think of a specific example, you've felt this before, right? The risky click, the where am I? Okay, good. All right. I thought so. Information architecture is more than just menus. Information architecture also affects Lots of other areas of a site or an experience search filters, recommendations, personalizations.

[00:13:24] We see these little tags up here, on the GAP website, a navigation up top here. Just in this one little screenshot, there are a lot of categories, tags. Ways of grouping things that are all based on information architecture. So a lot of times it's easy to, Oh, Sally, Canva, me too. I love Canva actually, but I do feel that way, especially when I create a project and it's just Dawn or in a different.

[00:13:53] I don't know. I agree with that one. Anyway, a lot of a lot of comparisons with information architecture. Just talk about navigation like this top bar up here. And that's a really straightforward and good connection between I. A and the user interface. But I also wanted to expand that a little bit to so many different aspects of a site and how people navigate it, not just.

[00:14:18] The top navigation or what's in the hamburger menu. All right. I've vented enough about sites that I feel maybe could have an improved information architecture. This is a bad screenshot, the only one I could find of Spotify. I love Spotify. I use it every day, but I have recently found like I have a really hard time.

[00:14:40] Finding something that I was just listening to, and I feel like it should be easier. I always have to go to that recent section and just see what were the last, whatever podcasts or songs that I listened to. I get very mixed up in the library and all the other stuff there. That might just be me though.

[00:14:56] So maybe they don't have to improve. So that is the big umbrella of information architecture, seeing some methods, some examples, some deliverables. But what we're going to focus on today is card sorting and tree testing using these two research methods and using them in Lyssna to understand users mental models, to improve navigation and discoverability, and to reduce friction in finding content.

[00:15:25] We're going to see how these are used together in the example project that I'm going to introduce. Let me check the chat here. If you're not premium Spotify, I am premium Spotify and it's still bad. They did it on purpose. How to find Spotify recently. We listened to, yeah, TRI had to do, I found it and I was like, oh, this, why is it hidden all the way in this menu?

[00:15:46] This is what the main thing I need to go to see Esther always have a hard time finding listening history. IA is not important. People search now. But we can, Jamie, we're going to come back to that one. Why doesn't every app just have a history? Yeah. And it's all tucked away in Spotify in that menu.

[00:16:04] And that makes me think either I'm doing something wrong or that should be like one of the main things on the bar. All right. You're all making me feel seen and heard right now with my examples. Thank you. All right. So let's talk about the example that we're going to be working on tonight. Explore City, a fictional app but with some very real.

[00:16:25] information architecture challenges that we are going to solve or at least make some progress on through card sorting and tree testing. So a quick brief on the project that we're working on. Explore City was originally an app for finding and booking guided tours. So if you were traveling somewhere you could browse, learn about some of the different tours that were offered, and book them directly through the app.

[00:16:55] This little screenshot just an example I put together, but we're not going to look at too many visual examples, because just about everything that we're doing is going to be, again, not behind the scenes. I work, but explore city originally not for finding and booking guided tours, which leads us to explore city expanding due to demand explore city now includes.

[00:17:18] Non tour experiences. So there's still guided and self guided activities. There are events, outdoor adventures, wellness retreats, expanding beyond just maybe those tours of a museum or a city or a downtown restaurant tour, something like that. A larger amount and a more diverse group or pool of events and activities that people can book.

[00:17:51] So the problem is that the current navigation is failing and you just took a tree test that we're going to check out the results of in a second. But to summarize them, and I'm pretty confident these will be consistent with. How you guys called out the study, but we'll see there are a couple of things going wrong.

[00:18:12] People often clicked on the wrong categories versus someone looking to book either a tour or some experience and trying to find it often clicked on the wrong category or had to backtrack to find what they were looking for. And the other high level takeaway is that users eventually found the right event or what they were looking for.

[00:18:35] But it wasn't really a direct path. It was probably a little frustrating. And like I said, took some backtracking. So let's actually look at the results for this one. And I'm going to share this link in the chat so you can check them out as well. And this, these are the results for the study that you guys just did a couple minutes ago.

[00:19:00] So we have our tree test here and it should look familiar. This is what it looks like. This is what you saw. And this is the results that we're seeing here on the list now. See our instructions and The familiar prompt that you just got of wanting to book a kayak rental on the Hudson River for your, let's say, upcoming trip to New York.

[00:19:24] The instruction was to use the navigation below and indicate where you'd expect to find information. About this activity, so you click through this as you just did see towards an experience and I don't know how many of you found it, but it was down here in new experiences and kayak rental on the Hudson River.

[00:19:49] I'd find it here. Let's see what came out of our true test. We're going to look at these results and a separate version of this test a little bit later, but just to give the overall summary of the challenges and the things that we're going to be trying to improve in this workshop. When we're doing a tree test, it's evaluating the success rate, how many users or what percentage of users were able to find that piece of content that you asked for in the prompt, in this case, kayak rental on the Hudson River, and also the directness, percentage of respondents who selected the correct answer without backtracking.

[00:20:33] So a good, a successful tree test that is going to validate your navigation structure. You're going to see a high success rate. Everyone's able to find the thing that they're looking for and a high directness rate, a hundred percent is perfect. Meaning that everyone found the most efficient path to that content.

[00:20:54] right on their first try. Now, without going into all of every breakdown of this tree test, I want you to see a few things that stand out. The first is that we had 43 percent of users, and this was not everyone here. This includes your responses, but I also did some before this. 43 percent found the content.

[00:21:19] Kayak rental on the Hudson River, so people were able to find the content, but we see the directness here was 3%. That needs both need improvement, but that being the most right. Everyone was able to find it, but there was a lot of backtracking and I bet I can predict what some of these common paths were.

[00:21:40] Let's see this 1, 5 participants, so 5 participants took the exact same path. You clicked on Tours and Experiences. You clicked on Outdoor Adventure Tours, and then you realized it's not in there. Backed up, went to New Experiences, looked around a little bit, and found Kayak Rental on the Hudson River.

[00:22:04] You got it. It was a success, but it was indirect. You had to loop back around to find it. We also see the time here, just to mention this in a tree test, which can be really valuable if you're trying to improve on how efficiently people can find things. But if I were looking at this on a real project, like these exact results here, My main concern would be the directness that's going to improve the time and that's more of a major red flag, right?

[00:22:36] Barely anybody is find it is immediately finding a quick, the quickest path or most direct path to that content here. So we're going to come so Jenny said, what is a good directness percentage? There is not, there's not a. 1 number that I can throw out there. That is going to be, universal, right?

[00:22:59] It's going to depend on the site, the amount of content, how early in the process you are of figuring out your information architecture. I can say 3 percent is bad. I can't think of any site that if you had this rate, that's going to, you're going to be happy about it. And I would also say, finding 90 to 100 percent is does not, you could be under that and it could still be a good experience, but it totally depends on benchmarks that you have. And it's really about improving it and testing until you see it max out or see diminishing returns on how well you can improve it. But I think it's fair to say 3 percent is.

[00:23:42] pretty bad. You can also look at other metrics that have, that it has an effect on of like conversion rates and stuff. So there are a lot of ways to figure out like a goal of what you're trying to get to. But we got a pretty low bar to start out with here with our 3 percent directness. So we are going to be looking at we're going to be trying to improve this, right?

[00:24:05] We're going to actually be doing a cards or looking at a card sort. Seeing how we might redesign that navigation and running the exact same tree test. Now with a different navigation at the end of the workshop, and we're going to see how it compares. And if we can improve on either time success rate, and definitely that directness rate,

[00:24:31] what I wanted to highlight here is something that I wanted to include in this example, because it's a really common mistake, or I guess scenario in so many projects that involve information architecture. I call it The dumping ground problem, basically what we saw in the tree test that you all just participated in was.

[00:24:53] All the tours organized and then a bunch of other secondary content, but then we saw this bucket called new experiences where the kayak rentals located. That was just a big random group of stuff. Basically, anything that was a tour was nicely organized under tours and everything else. That was not a tour.

[00:25:15] These new experiences just got. dumped into the same bucket with this vague title of new experiences. This is a really common occurrence especially with sites that have a lot of content sites that have been around for a long time and sites that are adding content or maybe changing the type of content on their site.

[00:25:38] But not really adjusting the navigation or how it's organized. So just like how explore city started out as tours. And now it's adding other experiences, the navigation and the organization behind it didn't keep up with that. Instead, the problems are stemming from them. Just dumping all of these other experiences into that new experiences.

[00:26:04] Bucket. This project on the right, this is a website I worked on a long time ago. It's for lawyers and legal professionals. And it was definitely to this day, the most challenging information architecture project that I've ever worked on. And the main reason was that even at the time, which was a while ago, it was a 20, 25 year old website with an enormous amount of content for lawyers and legal professionals.

[00:26:30] And. Basically, it was a mess. There were so many things added to that site over the years that fell victim to the dumping ground solution that it was almost impossible to find anything. And I'm happy to share some stuff after if you want to check it out, but we did a lot of card sorts. A lot of tree tests and spent a lot of time trying to pull up 1st of all, just take stock of all of this content and information and then figure out how we even begin to organize this in a way that's going to make sense for lawyers, legal professionals.

[00:27:08] judges, people in the legal industry who are trying to find it.

[00:27:13] So our goal, like I said, we're going to improve information architecture so users can find the right activity faster with fewer misclicks. And to do that, we need to understand how users naturally group these activities. We're going to abandon the dumping ground model of just throwing all these new experiences under a single group.

[00:27:34] And we're going to learn One by running a card sort to how different users or some of the trends and naming conventions and groupings that people use to organize this bucket of tours and experiences. We're going to use those results to design a better navigation. And as I said, at the end of the workshop, we're going to run the exact same tree test with new participants, obviously and see if we can improve on that success rate or that directness.

[00:28:07] Let's see for a redesign of a current nav, would you recommend with starting with a tree test or a card sort? It depends, but I would say the order that would probably. Work for just about any example would be doing a tree test first, just to get almost exactly what we're doing in this workshop, which is do a tree test first with the current navigation that you have and see what comes back, right?

[00:28:37] See it set a baseline of directness time. It takes to find something, identify some potential issues and just know where you stand with the quality or discover ability. Of your navigation, and then that might help you focus in on some areas with your card sort. For example, if you gave two different tasks, right?

[00:29:01] And one was a really high directness and success rate, then you know that's pretty clear the way it is, and you might not need to do card sorts or future studies on that. But you might have another task book around trip flight to something that really has a low success rate or low directness or takes a lot of time to do.

[00:29:24] Then you can shift your focus to that and improving the areas that need it the most. So it could work anyway, but I'd say a tree test to start is a good and very efficient and effective way to just understand how people are navigating what some of the areas of red flags or areas of focus you want to hone in on.

[00:29:45] Awesome. All right. Tree test, success rate and directness. Needed to make changes. Yeah, there you go, Johnny. You got it answered my question. Cool. So we're going to try to improve that. We did our tree test and see that there are clearly some issues and now we're going to get into our next step of improving it, and we're going to start that with some card sorting.

[00:30:09] All right, this, let me set the timer on my phone since my. My Chrome extension isn't working here. I'm going to set a one minute timer and I want you to have your chat open and just fire off as many possible ways that you can think of in one minute of how you could organize these six items on that you're seeing on the screen here.

[00:30:35] So some easy ones to start. You could say you could group them by color. You could group them by fruit or vegetable. You could group them by where it grows, on a tree versus the ground. I want you to get, try to get some of the obvious ones, but also go for some of the weird ones, right? This is not, we're not thinking of what's the most logical or most common way to group these.

[00:31:00] I want you to try to get as many possible variations or criteria as you can. All right, they're already coming in, so let's do it. I'm going to put a minute on the clock. You guys get started throw those in the chat and we'll take a look in just a 2nd.

[00:31:14] Nice. You got 30 more seconds

[00:31:17] and we got some original ones here. I like it used for skin care. That's good. Could any of these be used for skin care? Other than cucumbers still works though. James. I like it. Jenny slice chop or grill edible seeds. First letter, Jamie, that's a good one color fruits, veggies. All right. 3 more seconds to 1 and stop.

[00:31:46] Nice. Hey, that was pretty good for a minute. I'm not going to count how many responses we have in the chat, but you can just do a little scrolling and see we have a lot. Just with this very small set. Let me see if I can find my favorite 1. I liked I like use for skin care.

[00:32:01] James, that's a good one. Likes heat or dies with too much heat? Sally, that, that was I wouldn't have come up with that one, even though I've done this lesson a few times. Nice. Fruit, tree, bush fruit, vegetable, soup or no soup. Awesome. We have a lot of possible ways that we could organize this very small set of Items or content here, in just a minute.

[00:32:23] We came up with a ton of ways and while some might be more common or more familiar than others. They're all technically correct ways that you could group these items. Some are going to instinctively be make it easier to find 1 if we were putting them in categories. But they're all factually correct, just as equally factually correct ways that you could group these six items that we're seeing here.

[00:32:52] So if you think about that small set of content, our six fruits and veggies there, and then expand that out to a site like Netflix or a streaming site with tons of videos or YouTube, even bigger. You can imagine that just grows exponentially, right? There's all this content, each piece of content, each movie has so many characteristics that you could categorize it by, whether that's the genre, the director, the leading actor, the length, whether it's a show or a series or a movie, so much, so many different options.

[00:33:32] Millions, probably that you could organize these and when you think about it with our quick example, and then something bigger like this question really comes down to what is the right way to do it. And while there's probably a shorter list that you can pick from that, maybe you and your team are going to agree on.

[00:33:52] The answer is not to try to figure it out amongst yourselves, right? Or to have one person in the meeting, whoever is most stubborn with their opinion, get their way of how things should be organized. That's how it happens a lot. But the right way to do it is based on your users and their mental models.

[00:34:13] It doesn't matter really how you, as a, design or research or product team organized things. What matters is how your users think about it. And if they're ultimately going to find in this case, the movie or the show that they're looking for. So card sorting is a great method to do that. Card sorting is used to understand how users naturally.

[00:34:37] group, prioritize and categorize content. And it helps create an intuitive navigation, which is what we're focusing on here today. It can help with labeling systems like tags, and it can even help with things like priorities, maybe certain features or jobs to be done. That you want to evaluate with people so most commonly used for navigation and categories, but a very effective tool for a lot of different research.

[00:35:10] You might do to learn about how your users group and name different things. We're going to cover a good amount of card sorts and look at some results in a 2nd, but I'm going to share some resources after the workshop on some further reading. On card sorting some other things you would want to consider in terms of sample size, some other things in terms of breaking down the results, different varieties of card sorts moderated, unmoderated, qualitative, quantitative follow up questions.

[00:35:44] So there are a lot of. Different tweaks and scenarios you can make on a card sort, depending on what you're trying to learn from your research we're going to cover the big ones, but I'll also share some other methods and again, tweaks on the card sort method that can be really useful for some, maybe less common cases of learning about your user's mental models.

[00:36:11] Okay, so the two main types of card sorts or the two big ones to know are an open card sort and a closed card sort. We're going to start with an open card sort. Essentially an open card sort You are giving users a bunch presenting them with a bunch of cards. Those cards can have some kind of content on them.

[00:36:37] In this case, it's activities or tours, the things that we saw in the Explore City app. And users or part research participants are dragging those cards into groups. And what makes it an open card sort is they're also adding a title to that group. So if you see the little recording I did here, this is me as a participant actually creating a title of what a name that I think best summarizes or describes the different cards that I put in that group.

[00:37:12] An open card sort is Often used early in the process when you want to explore how users group different content and also how they name it. So that's not, across an across the board rule, but in my work experience and basically the vast majority of examples I've seen, if you're start, if you're thinking about.

[00:37:36] doing a navigation redesign and you're doing card sorts. Usually it's best to start with an open card sort because you're learning more, you're keeping it more open ended. How do users group these items and also what are the terms that they're using to describe those groups.

[00:37:55] A closed card sort is. Users or participants are placing content into categories that you've already provided. In this example here, I know it's a little hard to see on the screenshot, but it says articles, recipes, exercise and courses. Those are the 4 categories that you've already provided. Main titles of these groups and in a closed card sort participant would be doing the same thing dragging the cards into the category that they think best describes it and grouping them, but they're dragging them into categories that are already that we, as a research or product team have already created for them.

[00:38:34] They're not actually naming. The categories, there's also hybrid card sort where you can have some predefined categories and also allow users to create their own as well. Like I said, lots of different variations of the card sort, but the two most critical to know open and closed and in a closed card sort, this is a really good method after you've done maybe some open card sorts and you've identified some initial patterns, you feel like maybe we don't want to spend the rest of our lives So if you're a blog post trying to get 100 percent agreement on whether this category should be called articles or blog posts, let's call it articles, let's do a closed card sort, and let's see if people can still put the articles under that, and if they're in agreement on the types of content that should go in there.

[00:39:28] So closed cards are usually comes a little further down the line after you've got some more solidified evidence and you want to more so validate that. The groupings and test out the titles that you've already come up with some interesting and I think kind of fun things about cards or when you're creating cards, they can include just about any content in the card sort that we looked at that I just showed the example of for explore city.

[00:39:58] It's the titles of events. Each card is an activity or a tour words, right? A quick title that we're asking people to group. But cards can be images, videos, photos I guess full articles if you wanted to do that. It can be just about anything as long as it's the content type. Is consistent.

[00:40:22] So if you wanted people to group different articles of clothing, you could use each of these illustrations here and put it on a card and see how people group them into different categories. What you don't want to do is have an illustration of, I don't know, these socks down here and then also have another card with a photo of a model wearing this sweater up here.

[00:40:49] That's going to lead to. Some maybe bias or inconsistencies. So you want to make sure all the cards are consistent in the, in what you're presenting the user with, but it doesn't just have to be text. It can really be anything that you want people to group organize. And in some cases, Name groups for I thought this was a cool example of how that can branch out, which is this is sound snap, which is a website for video editors to find sound effects and there are hundreds of thousands on this site and they're all in categories like.

[00:41:30] Whoosh wind anticipation. I wonder if the researchers and designers at sound snap did a card sort with audio. That'd be pretty cool to hear play a 2 2nd whooshing sound and see if people participants drop that into the wind. Or the whoosh or the ambiance bucket here. It seems pretty well organized.

[00:41:53] So they might have done something like that, but furthering the example, anything can be on a card. It could be a two second sound effect. And you are as a researcher are curious if it should go in the wind bucket or the whoosh bucket to help people find it easier. Let's see a question from Jenny. If we want to see how people group categorize our products.

[00:42:14] We could put images of those products and let them name those. Yeah, absolutely. Yeah, you can have all the images. You can do that and listen as well. I'm going to pull it up, but you could have all the images. And even with no text or description, see how people group and categorize and name, name those categories.

[00:42:32] Definitely.

[00:42:33] All right. So let's check out the card sort results because this is what's going to be leading us into our hopeful improvement of the navigation and the tree test for Explore City. So let me pull this results link up. I'm going to pull the edit link I have in the back here, and then I'll share the public results with with all of you guys in the chat.

[00:42:59] And I'm going to pull up the actual study as well, just so you can see what we're talking about here. Okay, so this is not the study that you participated in. This is a card sort that I recruited for and created before the class. So we're not actually going to be participating or organizing cards as users, but we're going to look at the results.

[00:43:22] of this card sort and using them to see if we can find some ways to improve the navigation for Explore City. So here's what it looks like for the research participant. Hello there, you've been invited to participate in a short study for Explore City, an app that helps people find and book amazing tourism experiences.

[00:43:42] Your feedback will help us improve how people discover activities that match their interests. I'm going to skip past this because the instructions are here as well. But just to show you what this card sort looks like for the user. How would you organize these experiences? Sort them into groups that make sense to you.

[00:44:01] You can name the groups however you'd like. There are no right or wrong answers. So if I'm a participant in this study, I'm pulling some of these activities or experiences or tours, a list of them that I created for Explore City, dragging them over here. And I'm going to take these 2 and maybe I wanted to call this nature and tours here.

[00:44:34] I'm going to do a real quick version of this. So you can see the so you can see the question that I have after this. I'm going to skew our results a little bit here, but I will show you what I'm. what it looks like completed. Let's call this one history. Let's see some other nature and outdoors ones.

[00:44:56] That's good. Yeah, hot air balloon. This is a quick, I'm rushing this one, but it'll work. We got a bunch of responses in there already. Let's do some other history. This one I think could go in there too. Jazz, Shakespeare, stew. This one has, I'm going to call this food and wine or food and drinks. Let's say And I'm going to put all these theater, jazz, Shakespeare down here, this in there, and I'll call this arts and music.

[00:45:39] Okay, so that was a not too much thought went into that one. So I don't want you to focus on how quickly I did it or what I named it. But that's what the participant would do. Say I'm done. And I'm going to skip this question because we're going to come back to it in a sec. But just to show you another additional aspect you can include in card sorts, that's very common and very useful thing to include.

[00:46:01] In addition to having the user or participant sort and name the categories, you can also include some other questions. And this can be really valuable where you can collect a lot of great data on the stuff over here, how they organized it, but also learn about some other things. So this is the question I chose to include.

[00:46:22] which is asking the participant what other information would help you choose an activity or experience for your next trip. Beyond the category or the type of experience, what details would be useful in deciding if an activity is right for you. So we're going to look at some of the responses from these in a sec, but some things that you can pair together to still create a pretty quick study, but really get a lot Of valuable information on how people organize it and some other stuff that will become useful for us as we think about other aspects of organizing these activities.

[00:47:00] All right, so that's what it looks like from the front and let's look at the results here and I will share these with everyone in a 2nd, but just to, to go through this here. Okay. So what happened in this card sort that I sent out before our workshop, we had 18 participants, including the myself, the submission I just did here.

[00:47:30] So we don't have a huge data set for this 1. But we do still have some good stuff that we could pull out of this and use to redefine or redesign or rethink our navigation. We see all the cards and everything here, and we see a couple of different ways, basically. To look or visualize the results of this card sort, even with a pretty limited sample size we're able to see some pretty interesting stuff and some patterns in how people both organized and named some of the different the cards that we just saw here.

[00:48:12] So first thing that we're seeing, I'll go to this one categories right here, what we're seeing here are basically groups of similar where people put similar cards and agreed that a certain amount of card, a certain set of cards or activities should go together. And we're also seeing in this categories section.

[00:48:38] We're getting some insight into what people named. These groups, so I'll show you this 1. this group is called outdoor activities and we see this is a group that has. A bunch of different titles, all pretty similar to outdoor activities that people gave as a name of the group. We have outdoor adventures, outdoor experiences, adventure, outdoor nature activities, outdoor activities, a lot of exact copies right here.

[00:49:10] We're seeing that pretty much there were a lot of people who named a group something like, or very close to outdoor. Activities and what we can see over here are the cards that people put in that group seeing 72 percent of people had autumn foliage hike rock climbing kayak rental went into outdoor activities.

[00:49:39] We're seeing a pretty strong agreement here that not only did people put these activities in the same group, a lot of people named it. Something similar, like outdoor activities. We also see there are some outliers, right? Someone put food and wine tasting in outdoor activities, but we're seeing the same thing here with food and drink, right?

[00:50:03] A really high agreement, 89 percent of people put these activities in outdoor activities. in a category called or similar to food and drink here. So with this category section, basically we're getting some insight into what people named the trends or patterns in how people named these groups, and also seeing how consistently they, or how the rate of agreement of cards that went into that group.

[00:50:33] And for a lot of these, We're seeing pretty high agreement and pretty similar names for them. I'm going to go over here to the similarity matrix to I won't spend as much time on this 1, but this is where you can compare. how often different cards or groups of cards were put together in the same group.

[00:50:56] It doesn't give you much, or I think any info on what those groups were named, but more so just the agreement rate, how commonly were these grouped together? So if you look at this one here if I click on or hover over this 100, We see rock climbing and autumn foliage hike. 100 percent of all participants put these in the same group.

[00:51:23] They might have given the group different names. They might probably had other cards in that group. It wasn't just these two together. But if you're looking for patterns on which content should be put together or in the same group, this similarity matrix is going to be a useful tool for you. You're going to be able to see some of those patterns emerge maybe smaller or even larger groups that have a high agreement method.

[00:51:50] So we can even see, let me find another good example. Here's a good one right here. Let's see. How do we get these? These two are together. Let's see. Whoops. Here. Let me share these results with everyone to before I forget. I'll put these in the chat. Basically, you're able to see how the rate of agreement and I'm going to share some key takeaways from that coming back to the slides.

[00:52:18] Let's see 56, so you can compare any 2 of these to see what their agreement was and you can see how they cluster down here where we're seeing all of these darker green ones showing that we can compare 2 at a time, but we can also see a high rate of agreement between the larger group here of these outdoor activities.

[00:52:39] You can check out the results. I put the link in the chat and I'm also going to put some further resources for you to check out after. On the specifics of each of these visualizations, but I wanted to give you the overview and then come back to what the big takeaways that I would get from this.

[00:52:58] When I look through the results of that card sort, even though it was 18 people and maybe nothing that we saw in there is absolutely rock solid, 100 percent the way, guaranteed way to organize things. There were a couple. Trends that stuck out to me. First one is the big one for the most part users didn't really separate or participants didn't really separate tours from experiences.

[00:53:29] We saw in the original tree test, the 1 that each of you did at the beginning of class that there was the initial navigation was the dumping ground problem. We had guided tours with all those nice neat categories under them. And then we had that category more or new experiences. With everything else, essentially, the way the navigation was broken up in the original one was tours and everything else.

[00:54:01] Everything else in this 1 big bucket. And as we saw, that wasn't aligned with. Your mental models or how you would find something and we're seeing that again here where it's not lining up. What's being presented in the app is tours and everything else. But what we're seeing in the card sort is that people didn't really organize them by whether they were a tour or not.

[00:54:27] They organized them by the type. Of activity. It was whether it was a tour or not. And I think there's a good example here. 100 percent of all participants sorted food and wine tasting tour of the finger like wineries and street food festival in Long Island City. Together 1 is a tour. 1 is an event or an activity, but they're both related to food and drink.

[00:54:53] And they are 100 percent of people. All 18 people sorted those together. So we're seeing a big difference here of tours and everything else. And realizing that the way that users are actually browsing for this is more based on the type of activity. I'm summarizing this a little bit because there are some more, there are some things I could nitpick in the results of the card sort.

[00:55:17] But for the sake of moving forward and pulling out some of the big themes to go with, these were the most common groups. And I tried to name these with names that were. The most frequent or popular names for these groups in the card sort. Basically, we have outdoor and adventure and that had all of those outdoor activities tours or otherwise.

[00:55:40] We have food and drink, including tours and non tours, tastings, classes, food festivals cultural and historical experiences, and art, music, and theater. These were the main groups that came out of that card sort. Again, we could refine this further. We could test it. Again, if we really wanted to be sure, do it with a bigger sample size and probably tweak some of the names of those, but this is how, if this was all the information that I had on it, I'd be pretty confident saying that grouping them by this instead of what you saw in the first version is going to lead to some improvement.

[00:56:20] Okay, so the new navigation that we're going to test out here is. Going to look like this we have tours and experiences, let me just pull this up actually. And here you guys can check this out too. I'll put it in the chat.

[00:56:37] All right. So don't actually complete this study yet. I want to keep it keep it unbiased. We're going to send it out to the panel and get some fresh answers in it, but I'm going to put it in the chat on honor system that you can just. Just check it out. And if we need some more responses I'll give you the green light to fill it out.

[00:56:57] But just so you can see what I'm looking at here and see the updated navigation based on some of the stuff I just talked about from the card sort. We have our same intro. Actually, everything about this study is the same except for the actual content of the tree test. So we have our prompt here. That should look familiar.

[00:57:21] Click start. We get our instructions and we even get the exact same prompt here. You want to book a kayak rental on the Hudson River. So we are running the exact same test, same task. And we're doing that as a method of keeping, keeping it. The variable isolated to this navigation so we can easily and fairly compare if we made an improvement on it.

[00:57:48] Same prompt, everything identical right here. But then if you come down into the actual study, you'll see that it's different. We have tours and experiences. We don't have new experiences down here. We totally got rid of that other bucket and we're grouping these and naming them in a way that reflects what we just saw in the card sort.

[00:58:14] Under historical experiences, that was the most common. Name for that group. We have all of our historical experiences tours or otherwise. Let's see. We have outdoor and adventure here where we now have our kayak rental along with the outdoor tours and guided activities, food and drink with all of our food, wine, drink stuff and then art, music and theater.

[00:58:45] There is nothing separating tours from non tours. We are following what our 18 users told us about how they organize information, which is by the topic or the yeah, subject of the activity, not whether it's categorized by a guided or non guided. All right, so I'm not going to submit this. Don't don't submit it either because we're going to get some responses on this 1.

[00:59:11] But I'm going to send it out there in just a second. I'll walk through how to do that. Let me just pause for a second while I'm setting that up and preparing to get some responses for this one. If anyone has any questions. Let me know in the chat. I'll keep an eye on it. Otherwise, I'm going to get this launched.

[00:59:32] We'll wait for some results to roll in and then I do a few other things before we come back and check it out

[00:59:41] and get that set up. And then I'll come back to this 1.

[00:59:44] All right. So let me check the chat. You have any questions here. I think so. Okay, cool. Feel free to drop a question. And if you have one, I'm going to keep going on this so we can get some results in before we break here tonight. But this is the back end of the lesson where I set up the that new tree test that you just saw.

[01:00:03] So we got 2 people. Who responded to it, so that might skew our results a little bit, but that's okay. We're going to we'll look at them all together. And hopefully everyone found it easier than last time in the build section. Just setting up the, this is how I set up the study that you just saw with the welcome screen tree test and the examples that we all plugged in here.

[01:00:27] Now with the new categories and hierarchy. And we're going to recruit some participants for this. So let me find some participants and I'm going to do just a few here. Let's see, and we could do a better sample size of this 1, but let's do 20 participants. For this one, I think we had 30 for the original, so it won't be as much, but we'll still be able to get some participants in.

[01:01:00] So what I'm doing right now, instead of recruiting with the link that I shared with all of you at the beginning of class. I'm recruiting from Lyssnas panel, a bunch of people out there that we could target a little bit more and use some types and demographics in, but I'm going to keep it totally open ended here.

[01:01:19] And submit the order. So the idea here is we don't want you guys retaking the street test. We want to get some fresh eyes on it and again, see how we did. And if we were able to improve both success and the directness, so test looks good, new navigation looks pretty good recruitment looks good.

[01:01:42] Let's put it out there. And see what we get back. Awesome. Okay. So the order's in progress. I'm going to come back to this and we'll start to see, hopefully. some results soon. I'll keep this link up here to come back and check out. All right. I will pull that up in a second to to see how we did. But let me come back to one thing I wanted to mention here as well.

[01:02:10] I talked about this when I was walking through the card sort and that last question we had the actual card sort activity, and then we had a question after it. An open ended question, asking people what other information would help you choose an activity or experience for your next trip. And I talked about that briefly, but that can be a really effective addition to a card sort when you were doing a card sort.

[01:02:40] Users are working with the cards that you give them and the prompt that you give them. So you're asking them a question. How would you organize and group these activities and they have a set group of data that they can use within that, but you're not asking them. all of the other things that still might be important in learning about how they determine what activity or tour that they want to do.

[01:03:07] So I like to include, maybe not always, but early on in card sorts, I like to include a question at the end, an open ended question. That will allow me to get some more insights from them, not just about how they organize that set of cards that I gave them more broadly, like an interview question, how would they anything else?

[01:03:31] They want to tell me about the question that we're trying to ask, which is. How can we effectively organize and help people find the best activity or event for them? So in this one, I asked what other information outside of just, what group you would put it in, what other things would you want to know about a certain event or activity?

[01:03:52] And I'll just show you some of the answers for this one down here. No answer given for some of them because I made it optional, but and you can check this out on your own using the link. Reviews, length of time, testimonials of people who were with the tour guide, if there was one, what's the average time people actually spent there versus what the listing says.

[01:04:15] Ratings matter, but also see why people rated it low. Was it bad or just not what they expected? Let's see if there are any more on that. Picks from real people, not just promo shoots at 100 percent go to reviews first. So what we're seeing here is not, maybe not directly related to the categories that people would put these different activities in or what they would name them, but we're able to get a little insight into other factors that influence their process of finding an event.

[01:04:48] So just looking at this you can see a couple of themes reviews. Testimonials, understanding how much time it takes or how much effort it is if it's something for a hike. These might not be immediately important to create our navigation or just simply organize these events into categories, but they might be useful down the road.

[01:05:11] But for example, if we're designing a new landing page for the walking tour or some kind of outdoor activity, we're seeing a lot of feedback here that people want to know. The level of effort or how strenuous it is that might become a filter that we use, right? Challenging leisurely in between good for kids and family is highest reviewed.

[01:05:37] We get a lot of insight in here into more supplemental things that we could include that are still going to have a big impact. Over how users make decisions on what event they want to do. So I think of these post card sort questions as like a freebie, right? Someone's already doing your study.

[01:05:56] They're already motivated to provide feedback on the topic. Yes, it's great to get their card sorting groups and the titles, but for an extra maybe, minute of their time and a quick question, you can often pull a lot of other useful information that is going to be useful. Maybe in different organizational methods or different informing different content that you want to include for them.

[01:06:26] Let me check back on our results here. Let's see how we're doing. We had four when I refreshed it last time. All right. We have eight right here. Okay. So let's I'm going to go down to our wrap up and some key takeaways and next steps. And then we'll take one peek at that. Hopefully we'll get a couple more.

[01:06:45] Results and see how we did with see if we improved on that 3%. Directness rate that we started out with that low bar that we set for ourselves. Let's actually check it now. Why not? Then we'll do our wrap up. I don't know if I shared these with everyone in the chat already, but I'll put the link in there 1 more time.

[01:07:04] Even if we don't get all 20 of ours you can come back to that and see the results using that link and just refresh it. So even if it's after we wrap up here, you'll be able to check it out and see how we did. All right. So we don't have all the data in yet. We got eight responses, but we are seeing a pretty big improvement here.

[01:07:26] Success rate 63 percent. I think it was 43 on the last one and a huge improvement here. Again, I think a little bit skewed because I Responded to it with a direct path. And we only have 8 responses here. So I'm not going to put too much stock in the actual number, but I think this is giving me positive signals.

[01:07:48] Good vibes that we just made some improvements on this. We see the common path, the most common path of 5 participants. Was a direct success. They were able to find that kayaking trip and they were able to find it in the most direct path possible. Again, the time not really as concerning here because we can see that we're making an improvement in the biggest area that we needed to, which is the directness not having people.

[01:08:19] Backtrack and need to find that activity or browse around for it, knowing and feeling confident in the category that's in and seeing it there. And that is not because we just. Guessed or used our assumptions because we did, we asked users a different set of users, but we learned that through a card sort and really did it as simply as reflecting it back to them rather than trying to organize things on our own or use our best assumption as to what would, improve it for them. All right. So this is looking good. Let's see. We got any more. We got 12 now. All right. This is feeling positive. You all have that link. You can come back and check it out whenever, but I'd say I'm feeling pretty good that we're going to beat our 3 percent directness on this one.

[01:09:08] Someone's got to be on watch, though, as I do our key takeaways and wrap up here. If somehow we have just a devastating, devastatingly bad 10 more responses to it and no one can figure it out and things go the other way, give me a a red alert emoji in the chat and I'll go check it out.

[01:09:26] But I think we're looking pretty good. I think we can safely say that this would have made some improvements on the navigation. So thank you everybody. We covered. A lot here tonight thanks for sticking with me and walking through these examples. It was a little ambitious. I wanted to put a lot of material in here.

[01:09:47] Really? So you guys could see the process. And what it looks like from start to finish of improving it, even if we. Fast forwarded a little bit through some of the examples and analysis here. My hope is that you're able to see how these different pieces fit together and how these things that seem a little bit.

[01:10:09] Out there, like a tree test or a card sort, really have a direct impact and how they really aren't just these things that happen in the background, but research studies that the results flow through directly to the end user experience and can really make a big difference. Even in a simple example.

[01:10:28] like we covered here. So I want to wrap up with some key takeaways. I got three of them here and a quick wrap up. I hope you learned more than three things here tonight, but these are the big ones that I hope, if nothing else, you come away with. First one is seeing just the importance and impact of information architecture and the research that goes into creating good information architecture.

[01:10:58] I introduced it as a background process. It's not UI design. We're not testing out prototypes or animations. We're not really seeing directly. How this affects the end user experience and what they're clicking, tapping, looking at and interacting with, but it is the backbone of a great experience, even if you have a great looking UI, cool features, transitions, animations, really interesting layout for something, and you're doing a lot of other things well.

[01:11:30] If the content on your site or your app is not well organized and not scalable, then that's going to hurt. And it's not going to be able to good UI is not going to be a bandaid that's able to fix some of those challenges. Users are still going to be frustrated that they can't find what they're looking for.

[01:11:50] Second one, good IA scales as content grows. And what I wanted to focus on here is It's just like all things in research and product design, the iterative nature and need for kind of constant improvement. We saw the dumping ground example with both my legal site and Explore City. And that happens all the time.

[01:12:15] A company grows, their content offering expands, they add new products to their site, they change their positioning or, drop certain products or content from things. it never is reflected in the organization of the site. So good information architecture scales. It's continuous where you're constantly learning and updating how you're organizing content on your site.

[01:12:42] And if you do that well, You never have to really do that huge overhaul, right? You'll be able to easily bring in new products, new content, new movies if you're Netflix, and do maybe a little bit of work or that kind of little upkeep to make sure that they're finding a good home. And that is a much more effective and a much more feasible way to make sure that your information architecture is good because you're constantly checking in on it.

[01:13:14] And improving it as new things get added my last big takeaway here as an insider. What I mean by that is a designer, a researcher, a founder, a business person, someone who's working on the product. It's easy to assume that users group things the same way that you and your organization does. And I can tell you they probably this is another cause of that disconnect between how things are organized and presented on a site versus how users think about them.

[01:13:48] A lot of times there are internal ways of thinking about given products. Maybe it's something that you, a name that you and your team use for a certain type of product or content. Maybe it's something that's just been ingrained in the product. with explore city tours and everyone who works at explore city just thinks tours and everything else and that mindset kind of leaks into what the user is seeing.

[01:14:16] This is, I would say the most common cause of a confusing navigation and bad information architecture is assuming that the way that you and your teammates really talk about the products or content on your site or how you think of them is going to be consistent with how users do. And, just like all things in UX design and research, those assumptions are probably not right.

[01:14:41] They might not be wildly wrong, but I would almost guarantee that if you tested it out, you would find some interesting differences between the way that you organize those items internally and that users. Browse for them on your site. And those can often be pretty high impact things. And if you fix them, it can really make a big difference on the user experience.

[01:15:06] Alright since we're just about at time here we're gonna wrap up I hope it was worth your time and not just, good information, but I hope an enjoyable experience and some clear examples that maybe clear things up for you.

[01:15:21] I know there's more to learn and I'm going to follow up with some other resources and I might not you might not be feeling like. an expert on all of this stuff just leaving this course, but I do hope that you're feeling more clear about the process and more confident to put this into action on your own work.

[01:15:39] So I want to thank you all for joining, for taking time out of your day or night to sit with me for an hour and a half, listen through all my examples,

[01:15:47] and we got a couple other workshops coming up next week and then launching a bunch after that for April and May. So I'll be teaching keep an eye out on those emails and hopefully you'll be back for some more.

Workshop transcript