Industry Usability Reporting and the Common Industry Format (ANSI-NCITS 354-2001)
By Avi Parush and Emile Morse
What is the Common Industry Format?
The Common Industry Format (CIF) is a standard method for reporting usability test findings. The format is primarily for reporting results of formal usability tests in which quantitative measurements were collected and is particularly appropriate for summative/comparative testing. The CIF targets two audiences: usability professionals and stakeholders in an organization. Stakeholders can use the usability data to help make informed decisions concerning the release of software products or the procurement of such products. While the CIF is formally aimed for software products, it can be used for hardware usability as well.
The format includes the following main sections: Executive Summary, Introduction, Method, and Results. The Method and Results sections are particularly important and provide usability professionals with a good framework for the report. The Method section prescribes the inclusion of details concerning the participants and their profile, the details of the context of use employed in the test, technical aspects of the testing facility and apparatus, and finally all the study design aspects (such as variables and measurements). The Results section includes sub-sections for the presentation of performance data (e.g., times or error rates, etc.) and a sub-section for the presentation of satisfaction results. It is important to note that the CIF assumes that best practice is used in designing and conducting a usability test. The CIF does not tell you what to do; it tells you how to report on what you did. The CIF is based on the usability definitions of ISO 9241-11, and as such is appropriate for use when the test procedures follow both ISO 13407 for Human Centered Design and ISO 9241-11.
The CIF has its origins in the Industry USability Reporting (IUSR) effort, started in 1997 by the National Institute of Standards and Technology (NIST) with the objective of increasing the visibility of software usability. The participants worked through a series of workshops to develop the format; then they ran several pilot studies to show the utility of using the format. In December 2001, the American National Standards Institute (ANSI) approved the CIF as a recognized standard, ANSI-NCITS 354-2001.
Current status and future agenda
The most recent workshop dealing with the CIF took place at NIST in Gaithersburg in the beginning of October 2002. This workshop had four primary objectives:
- To identify appropriate revisions to the current CIF: Some of the required revisions that were identified were: elaboration of usability measures (e.g., efficiency), inclusion of a usability problems list, and addition of an optional section for conclusions and implications. Another future direction is to explore how to develop a CIF appropriate for formative testing; a white paper will be written on this issue.
- To initiate an effort to develop a standard format for usability requirements: The main issues in developing a CIF for usability requirements were identified and a working draft of such a format was developed. The basic approach is that the requirements CIF will also follow the ISO 13407 and ISO 9241-11, and will be complementary with the ANSI-NCITS 354-2001.
- To assess the need for a separate CIF for hardware usability: After some discussion, the group agreed that while formally the CIF is aimed for software products, it could be applied in its current form to reporting usability tests of hardware as well.
- To discuss international standardization: Various routes to turn the CIF into an international (ISO) standard were examined. The keys to success are identifying an existing ISO subcommittee that has both an interest in usability and a streamlined process. The workshop participants agreed that the fast-track should be taken with the CIF as it is now.
Additional information about how the CIF was developed, how you can join the project team and how to obtain the CIF can be found in: http://www.nist.gov/iusr/index.html
ISO 13407 Human-centered design processes for interactive systems, ISO 1997.
ISO 9241 Ergonomic requirements for office work with visual display terminals - Part 11: Guidance on usability.
ANSI-NCITS 354-2001 Common industry format for usability test reports.
Scholtz, J., A. Wichansky, K. Butler, E. Morse, & S. Laskowski. 2002. Quantifying Usability: The Industry USability Reporting Project. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (Baltimore, MD), 1930-1934.
About the authors:
Industrial Management & Engineering
Israel Institute of Technology,
Haifa, Israel, 32000
100 Bureau Drive, Stop 8940
Bldg. 225, A223
Gaithersburg, MD 20899