A Modified Delphi Approach to a New Card Sorting Methodology
Journal of Usability Studies, Volume 4, Issue 1, November 2008, pp. 7-30
Article Contents
Next Steps
The Modified-Delphi cart sort provides a promising alternative to Open card sorting for use as a pre-design method. However, it is a new method and must be further researched in order to refine the methodology and maximize the return of investment it requires. Questions about the method surfaced during the design of method and analysis and preliminary reporting of results. Some of these questions included the following:
- Does the number of 8 to 10 participants hold up for all datasets and participant diversity cases?
- How do characteristics of the dataset affect the validity of the method?
- How much does participant diversity matter (for example, using a single user group versus a sampling of all user groups)?
- How much influence does the selection of the seed participant have on the study?
- How does the method hold up with groups of participants per session, rather than single participant sessions?
- Does the Modified-Delphi card sorting method provide as good of results executed as a web-based method as it does in the laboratory?
The answers to these questions can only be obtained by conducting additional Modified-Delphi card sorting studies. Parallel Open and Modified-Delphi studies are not cost effective and only make sense for research purposes. Modified-Delphi card sorting has already been proven to be as at least as good and very likely better than Open card sorting. It would be interesting to conduct this method as a web-based method to see if it offers benefits to web-based methods as it has to laboratory-based methods. I encourage practitioners to utilize this method early in the design process and share their results with the rest of the community.
Acknowledgments
This research was partially funded by the Information Architecture Institute 2006 Process Grant (Paul, 2007) and introduced at the 2007 Information Architecture Summit in Las Vegas, Nevada. I would also like to thank Kathryn Summers of the University of Baltimore and my colleagues Bill Killam and Marguerite Autry of User-Centered Design, Inc. for their guidance and support during the project.