Reliability of Self-Reported Awareness Measures Based on Eye Tracking
William Albert and Donna Tedesco
Journal of Usability Studies, Volume 5, Issue 2, Feb 2010, pp. 50 - 64
Abstract
Participants in a usability evaluation are often asked whether they noticed certain elements after some level of interaction with a design. The purpose of this study was to evaluate the reliability of self-reported awareness measures using eye tracking data. Participants were shown 20 popular homepages for 7 seconds each and then asked afterwards if they saw 2 particular elements on each page. The results showed that self-reported awareness measures are reliable, but can vary depending on question structure and object type. These findings have implications for how usability practitioners ask questions about object awareness, and how that information is used in the design process.
Practitioner’s Take Away
- Usability practitioners should feel confident in collecting self-reported awareness measures from participants.
- If a practitioner wants to minimize the chance of making an incorrect conclusion, they should use a continuous (5- or 7-point) scale for self-reported awareness (similar to Experiment 2).
- If a practitioner wants to maximize the likelihood of confirming that a participant did or did not see an element, they should use a discrete set of questions for self reported awareness (similar to Experiment 1).
- Participants are more reliable in their self-reported awareness for navigation and image elements, than functional elements, regardless of question structure.
Article Contents
Download Article
Reliability of Self-Reported Awareness Measures Based on Eye Tracking