Card sorting: a definitive guide - Boxes and Arrows: The design behind the design

Card sorting: a definitive guide

byDonna Spencer and Todd Warfelon 2004/04/07 | [43 Comments]

Introduction

Card sorting is a technique that manyinformation architects (and related professionals.) use as an input tothe structure of a site or product. With so many of us using thetechnique, why would we need to write an article on it?

While cardsorting is described in a few texts and a number of sites, mostdescriptions are brief. There is not a definitive article that describesthe technique and its variants and explains the issues to watch outfor. Given the number of questions posted to discussion groups, anddiscussions we have had at conferences, we thought it was time to getall of the issues in one place.

This article provides a detaileddescription of the basic technique, with some focus on using thetechnique for more complex sites. This article does not cover someissues such as the use of online tools, which will be covered in afuture article.

Why

Card sorting is a quick, inexpensive, and reliablemethod, which serves as input into your information design process. Cardsorting generates an overall structure for your information, as well assuggestions for navigation, menus, and possible taxonomies.

Whilecard sorting might not provide you with final structure, it can helpyou answer many questions you will need to tackle throughout theinformation design phase. For example, more than likely there will besome areas that users disagree on regarding groupings or labels. Inthese cases, card sorting can help identify trends, such as:

• Do the users want to see the information grouped by subject, process, business group, or information type?
• How similar are the needs of the different user groups? >
• How different are their needs?
• How many potential main categories are there? (typically relates to navigation)
• What should those groups be called?

Card sorting can help answer these types of questions, making you better equipped to tackle the information design phase.

Definition

Card sorting is a user-centered design methodfor increasing a system’s findability. The process involves sorting aseries of cards, each labeled with a piece of content or functionality,into groups that make sense to users or participants.

According to Information Architecture for the World Wide Web,card sorting “can provide insight into users’ mental models,illuminating the way that they often tacitly group, sort and label tasksand content within their own heads.”

Card sorting is a great,reliable, inexpensive method for finding patterns in how users wouldexpect to find content or functionality. Those patterns are oftenreferred to as the users’ mental model. By understanding the users’mental model, we can increase findability, which in turn makes theproduct easier to use.

Method

There are two primary methods for performing card sorts.

• Open Card Sorting: Participants are given cards showing site content with no pre-established groupings. They are asked to sort cards into groups that they feel are appropriate and then describe each group. Open card sorting is useful as input to information structures in new or existing sites and products.
• Closed Card Sorting: Participants are given cards showing site content with an established initial set of primary groups. Participants are asked to place cards into these pre-established primary groups. Closed card sorting is useful when adding new content to an existing structure, or for gaining additional feedback after an open card sort.

Closed card sorting will be detailed in a future article.

Advantages and disadvantages

As with any other method,card sorting has both advantages and disadvantages. Keeping these inmind will help you determine whether the technique is appropriate foryour situation and make decisions about how you run the activity.

Advantages

• Simple – Card sorts are easy for the organizer and the participants.
• Cheap – Typically the cost is a stack of 3×5 index cards, sticky notes, a pen or printing labels, and your time.
• Quick to execute – You can perform several sorts in a short period of time, which provides you with a significant amount of data.
• Established – The technique has been used for over 10 years, by many designers.
• Involves users – Because the information structure suggested by a card sort is based on real user input, not the gut feeling or strong opinions of a designer, information architect, or key stakeholder, it should be easier to use.
• Provides a good foundation – It’s not a silver bullet, but it does provide a good foundation for the structure of a site or product.

Disadvantages

• Does not consider users’ tasks – Card sorting is an inherently content-centric technique. If used without considering users’ tasks, it may lead to an information structure that is not usable when users are attempting real tasks. An information needs analysis or task analysis is necessary to ensure that the content being sorted meets user needs and that the resulting information structure allows users to achieve tasks.
• Results may vary –The card sort may provide fairly consistent results between participants, or may vary widely.
• Analysis can be time consuming – The sorting is quick, but the analysis of the data can be difficult and time consuming, particularly if there is little consistency between participants.
• May capture “surface” characteristics only – Participants may not consider what the content is about or how they would use it to complete a task and may just sort it by surface characteristics such as document types.

When should card sorting be used?

Card sorting is a user-centered, formative technique. It should be used as an input to:

• designing a new site
  
• designing a new area of a site
  
• redesigning a site

Card sorting is not a silver bullet to create an informationstructure. It is one input in a user-centered design process and shouldcomplement other activities such as information needs analysis, taskanalysis, and continual usability evaluation. It is most effective onceyou have completed:

• research into what users need out of the site
  
• a content (functionality) audit/inventory (for an existing site) or detailed content list (for a new site). For an existing site, it is crucial that the content inventory is examined carefully to include only content that is needed by users.
  

Card sorting will provide benefit to most sites, but can bechallenging to use against some sets of information. The table belowsummarizes when card sorting works well and provides good results, andwhen it is challenging both to run and to analyze.

For sites with characteristics listed in the last column, cardsorting will provide less direct input into the information structure;you may need to undertake a range of card sorts and more user-centereddesign activities.

Card sorting can be useful to demonstrate topeople that others think differently. We have successfully included itas an exercise in workshops for web site and intranet authors.

Preparation

Preparing for a typical card sorting exercise requires the following:

1. Selecting content
2. Selecting participants
3. Preparing the cards

Selecting content

The first step in conducting acard sort is to determine the list of topics. This list should be drawnfrom a wide variety of sources:

• existing online content
• descriptions of business groups and processes
  
• planned applications and processes
  
• potential future content

By including potential future content it becomes possible to create astructure that not only works now, but also will work for futurecontent and functionality. Adding new items in the future should requireminimal rework if the structure is designed correctly.

Granularity and sampling content.

Contentselected for the cards can be individual pages, functionality, smallgroups of pages, or whole sections of the site. Be consistent with yourchosen granularity—participants will find it difficult to group contentat different levels of granularity.

If you choose to use smallgroups of pages or sections of the site, ensure that the groups are ofitems that belong together. For example, don’t include a grouping of“media releases,” as this may not suit users and their tasks (they mayprefer individual media releases to be grouped with other pages ofsimilar topic.). Instead, include some individual media releases and seewhat participants do with them.

The content for the card sortshould be representative of the site (or the part of site that you areinvestigating). It is important to ensure that the content has enoughsimilarity to allow groupings to be formed. If the content chosen is toovaried, participants will not be able to create natural groupings.

Selecting participants

Cardsorting may be performed individually or in groups. Keep in mind thatthe exercise will be performed multiple times. So, if you’re usingindividuals, try and get seven to ten for a good sampling. If you’reusing groups, our preferred method, five groups of three participantsper group (a total of 15 participants) works best. Whether you choose touse individuals or groups, the most important aspect of selectingparticipants is that they come from and are representative of your usergroup. (If you have multiple user groups, it is important to include arepresentative sample from each, as they may view the informationdifferently).

Scheduling individuals can be easier thanscheduling groups of participants, especially if you have individualslocated remotely. However, individuals can find it difficult to sortlarger numbers of cards, providing less valuable input.

A benefitof group sorts is that they typically provide richer. data thanindividual sorts. Whereas individuals need to be prompted to “thinkaloud,” groups tend to discuss their decisions aloud openly. Combinethis with the group’s ability to handle larger numbers of cardseffectively and their tendency to walk each other through questionsabout content or functionality, and you have a rich data set withgreater insight into users’ mental model.

The number of groupsneeded may depend upon the size and complexity of the site or product.However, we’ve found that patterns tend to emerge within five groups.These patterns become the basis for the site or product’s informationarchitecture.

When inviting participants, it’s not necessary totell them they’ll be performing a card sort. Instead, simply tell themthey’ll be asked to perform a simple task, or exercise that will helpyou (re)design the site or product. Additionally, let them know theydon’t need to prepare ahead of time; they should simply come as theyare.

Preparing the cards

Each item on your list shouldbe placed on a card. The labels you use on the cards are extremelyimportant. They should be short enough that participants can quicklyread the card, yet detailed enough that participants can understand whatthe content is. When necessary, the label can be supplemented with ashort description or image on the back of the card.

Labels may beprinted on standard (Avery) mailing labels, or printed by hand. Werecommend using mailing labels as this saves time and the labels will bemore legible.

Mark each card with a letter or number to make analysis easier once the sorting is done.

Youcan use whatever cards you have on hand, but we recommend 3” x 5” (10cmx 15cm). Index cards are durable, easy to see from a distance, andreadily available at office supply stores. You may also use Post-it®notes, but it is our experience that cards are more durable and easierto handle.

Number of cards.

While there is nomagic number, we have found that between 30 and 100 cards works well.Fewer than 30 cards typically does not allow for enough grouping toemerge and more than 100 cards can be time consuming and tiring forparticipants. However, we have performed successful card sorts with over200 cards where participants understood the content well.

Inaddition to the labeled cards, be sure to include some blank cards incase participants need to add something. And don’t forget a pen.

Execution

For the purpose of this article, we willdescribe an ideal execution for a card sorting exercise. Keep in mindthat there are several variations, as described above.

The cardshave been labeled using Avery labels on 3” x 5” index cards. On the backof each card is a letter/number combination, as well as a shortdescription or image as necessary. The letter/number combination will beused during analysis; the short description or image is provided toclarify titles that might prove confusing. The cards are shuffled priorto participants entering the room. The shuffled cards, a stack of 20blank cards, and an ink pen are placed on the table. Three participantsare brought into the room and given an introduction with some basicinstructions, like these:

First of all, we’d like to thank you for coming. As you may be aware, we’re in the initial stages of (re)designing a (web site, product, intranet). In order to make it as easy to use as possible, we’d like to get some input from the people who will be using it. And that’s where you come in. We’re going to ask you to perform a very simple exercise that will give us some great insight into how we can make this (web site, product, intranet) easier to use.

Here’s how it works. In front of you is a stack of cards. Those cards represent the content and functionality for this (web site, product, intranet). Working together, you should try and sort the cards into groups that make sense to you. Don’t worry about trying to design the navigation; we’ll take care of that. Also, don’t be concerned with trying to organize the information as it is currently organized on your (web site, product, intranet). We’re more interested in seeing how you would organize it into groups you would expect to find things in.

Once your groups are established, we’d like to have you give each group a name that makes sense to you. You are allowed to make sub-groups if you feel that’s appropriate. If you feel something is missing, you can use a blank index card to add it. Additionally, if a label is unclear, feel free to write a better label on the card. Finally, if you think something doesn’t belong, you can make an “outlier” pile.

Oh, and one last thing. Feel free to ask questions during the exercise if you feel the need. I can’t guarantee that I can answer them during the exercise, but I’ll do my best to answer them when you’re finished.

Facilitating card sorts can be tricky. During the exercise, your mainjob is to observe and listen. Your secondary job is to keep themomentum going without leading the participants. Take notes on a smallnotepad to keep track of insightful comments made by participants, orquestions that come up during the session.

Try to make sure eachparticipant has the opportunity to provide input. If one of theparticipants tries to “take over” the sort, gently prompt the otherparticipants. If one participant sits back, gently prompt thatparticipant. If the group creates a “miscellaneous” group, ask them ifthey are satisfied with that group, or if they would like to takeanother look at it to see if it needs to be sorted further. Make surenot to lead them too much.

Once the sort is complete, you may see something that looks like this:

Once the participants are finished, walk them through a particulartask. This helps validate the results. For example, if the site has sometype of account management, or profile feature, ask them to walk youthrough updating their address information.

Analyzing the results/next steps

Analyzing card sortdata is part science, part magic. Analysis can be done in two ways: bylooking for broad patterns in the data or by using cluster analysissoftware.

When performing analysis on smaller numbers of cards,you may be able to see patterns by simply laying the groups out on atable, or taping them on a whiteboard. You will be able to see patternsthrough similar groupings and labeling.

When performing analysis on larger numbers of cards, we suggest using a spreadsheet. Enter the results into a spreadsheet, making sure to capture the title and number on each card. If theparticipants changed the label on a card, record the new label and placethe old label in parentheses. Once you’ve entered the data, beginlooking for patterns across the groups. Keep in mind the discussionsheld between the group participants during the sort, as they provideadditional insight that might not appear in the spreadsheet. At thispoint, you are not looking for a definitive answer, but for insights andideas.

Another technique for analyzing data can be found in “Analyzing Card Sort Results with a Spreadsheet Template”;by Joe Lamantia.. Follow the instructions in Lamantia’s article toprepare the spreadsheet. As he mentions, look at the results forhigh-agreement cards and low agreement cards.

In both types ofanalysis, patterns will emerge. These patterns will likely be sensiblefor the actual users. It is important to note that areas of differencealso provide useful insights. Areas of difference tell us about:

• content that participants haven’t understood well
• content that could belong to more than one area
• alternative paths to content (for example, a list of all “how-to” articles could be created)
• how different types of participants see information

There definitely is some magic in the analysis step, and it isdifficult to provide exact instructions on what to look for. Allowyourself some time to explore more than one organizational model basedon the information provided from your analysis. Remember that it is notnecessary to jump straight to a taxonomy at this point. Your card sortresults can be supplemented with additional user research and taskanalysis.

Issues/Variations.

There are a range of additional tasks that you can ask participants to do during the exercise, including these:

• Home page content: ask participants to put to one side content that they would use so often that they would want a link on the home page to it.
• Information- seeking task: after the exercise, bundle up the piles of cards on the table so only the top level is showing. Ask participants where they put particular content. (It is worth doing this if you suspect that the participants were not thinking about how they would use the content as they sorted)

The resulting draft information architecture can be evaluated using Donna’s card-based classification evaluation.This technique provides additional information about the grouping ofthe content, as it focuses on tasks that users would do rather than justfocusing on content. Frequently, participants will create groupings ofcontent in a card sort that they then cannot use when asked to perform ascenario.

Summary

In summary, card sorting is a simple, reliable,and inexpensive method for gathering user input for an overallstructure. It is most effective in the early stages of a (re)design. Andwhile it’s not intended to be a silver bullet, when done correctly, itis instrumental in capturing helpful information to answer questionsduring the information design phase – ultimately making the producteasier to use.

Invitation

One reason we wanted to write this articlewas to get a detailed explanation of card sorting in one place. Pleaseexpand this article into a definitive card sorting resource by addingcomments with your own variations or observations.

• The IAwiki has a page on card sorting that is updated frequently.
  http://www.iawiki.net/CardSorting
• Gaffney, Gerry. “What is Card Sorting?” Information & Design (2000).
• Maurer, Donna. “Card-Based Classification Evaluation,” Boxes and Arrows, April 2003.
  http://www.boxesandarrows.com/view/card_based_classification_evaluation.
• Rosenfeld, L. and Morville, P. Information Architecture for the World Wide Web: Designing Large Scale Web Sites. O’Reilly & Associates, 2002.
• Warfel, Todd. “Modeling Organization – Methods for Increasing a System’s Findability,” Message First Corp., 2001.
  http://messagefirst.com/downloads/ModelingOrganization.pdf