upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Card Sort Analysis Best Practices

Carol Righi, Janice James, Michael Beasley, Donald L. Day, Jean E. Fox, Jennifer Gieber, Chris Howe, and Laconya Ruby

Journal of Usability Studies, Volume 8, Issue 3, May 2013, pp. 69 - 89

Article Contents

Review the Items Placed in the Categories

Now that you have a first pass at your categories, and you have a first pass at standardized category labels, you will want to take a closer look at your data to see whether there is enough agreement between items within the category before moving on to the next step. This can be done with the item-by-category matrix (Figure 4). Results are presented in a table with categories on one axis and the individual content items on the other. As mentioned before, cells for each pair contain the percentage of times participants placed the item into that category. Cells are also color-coded to indicate how strong the association is. In some online tools, (Figure 4) the colored background cells indicate the highest percentage of times, whereas in other online tools, the background color graduates in darkness. The darker the background color, the higher the percentage is of participants who used this mapping.

Next, look at the percentages for an item across the different categories participants provided. Do you have a lot of items with high percentages across few categories (high agreement)? Or, do you have a lot of items with low percentages across many categories (low agreement)? If your answer is the latter, then you’ll need to understand why. Again, this may be due to having two or more distinct participant groups who have very different mental models of the content. If you haven’t already, try separating your participants into two different groups based on relevant demographics and recheck the matrix for each group separately. If splitting the data doesn’t result in stronger agreements, or if you didn’t even have enough agreement to get that far, then your dataset needs to be more closely examined. Again, recheck for outliers and try removing them from the data to see if results become clearer.


Previous | Next