Comparing Computer Versus Human Data Collection Methods for Public Usability Evaluations of a Tactile-Audio Display
Maria Karam, Carmen Branje, John-Patrick Udo, Frank Russo, and Deborah I. Fels
Journal of Usability Studies, Volume 5, Issue 4, August 2010, pp. 132 - 146
Abstract
We present a public usability study that provides preliminary results on the effectiveness of a universally designed system that conveys music and other sounds into tactile sensations. The system was displayed at a public science museum as part of a larger multimedia exhibit aimed at presenting a youths’ perspective on global warming and the environment. We compare two approaches to gathering user feedback about the system in a study that we conducted to assess user responses to the inclusion of a tactile display within the larger audio-visual exhibit; in one version, a human researcher administered the study and in the other version a touch screen computer was used to obtain responses. Both approaches were used to explore the public’s basic understanding of the tactile display within the context of the larger exhibit.
The two methods yielded very similar responses from participants; however, our comparison of the two techniques revealed that there were subtle differences overall. In this paper, we compare the two study techniques for their value in providing access to public usability data for assessing universally designed interactive systems. We present both sets of results, with a cost benefit analysis of using each in the context of public usability tests for universal design.
Practitioner’s Take Away
We have found that it is important to consider the following concepts when creating systems using universal design principles:
- Use automated questionnaires to increase participant numbers in evaluations.
- Use public usability studies to supplement lab experiments with real-world data.
- Modify existing evaluation tools to extend into the public domain.
- Include as wide an audience in evaluations to ensure universality of the design.
- Expect to alter major features of the design to ensure target users are addressed.
- Select only technology that is robust enough to withstand constant public use.
- Reinforce and secure systems to ensure safety of users and the equipment.
- Restrict public studies to short, succinct questions and questionnaires to maintain ease of ethics approval, and focus the study on the broader aspects of system interactions.
- Ensure proper assistance is in place to accommodate users with special needs during the study. Sign language or other interpreters, or special access needs are essential to address when conducting studies on universal designs.
Article Contents
Download Article