Heuristic Evaluation Quality Score (HEQS): Defining Heuristic Expertise
Journal of Usability Studies, Volume 4, Issue 1, November 2008, pp. 49-59
Abstract
This paper identifies the factors affecting heuristic expertise and defines levels of expertise permissible to conduct an evaluation. It aims to standardize skills or define heuristic expertise worldwide and also suggests ways to improve issue categorization.
An online heuristic evaluation competition was hosted on the World Usability Day website in November 2007 by Usability Professionals’ Association (UPA), Bangalore. Twenty contestants from the U.S. and India with heuristic evaluation experience ranging from 0 to 10 years participated. Contestants were judged on a quantitative framework that assigns weights to the severity of the identified issues (Kirmani & Rajasekaran, 2007). Results indicated that the group with average heuristic experience of 2 years found a mean HEQS% of 8% in 1 hour. Previous studies identified that evaluators found a mean of 34% of issues but did not address issue quality (Nielsen & Landauer, 1993). Further studies on heuristic expertise quality would make the standards more reliable.
Practitioner's Take Away
- A Heuristic Evaluation Quality Score (HEQS) can be used to quantify heuristic evaluation expertise to ensure evaluations of a certain standard. It is critical for evaluations to be of a certain standard to build reliability and trust among evaluators and ultimately provide the end users with high quality applications.
- The average HEQS% is 8% for evaluators taking 1 hour with heuristic experience of 2 years.
- Evaluations are recommended to be done by above average and exceptional evaluators. Above average evaluators have an HEQS% of 8% or more and exceptional evaluators have an HEQS% of 15% or more.
Article Contents
Download Article
Heuristic Evaluation Quality Score (HEQS): Defining Heuristic Expertise