The LEO Kit: Usability Testing for Local Election Officials

This project started with a problem and an idea for a solution.

The problem is how to get ballots across the country usability tested for each election. We're not talking about scientific, rigorous testing, but a way to find any problems with the ballot design or other election materials before the election.

The crazy idea is that local election officials can do the testing themselves.

To make it happen, we came up with a testing kit for people who have no specific training in usability or human factors engineering. We needed a test that doesn't require any special equipment or a laboratory.

 

We created this kit with these goals in mind:

• Make it more likely the voter’s intention is carried out
• Make it easier for voters to use the ballot
• Identify problems in the design of the ballot that could lead to residual votes or drop-off
• Make it more likely that media coverage of an election is positive

Usability studies like this one provide opportunities to observe how easy or difficult it is for voters to use a ballot (either paper or electronic), to ask questions with measurable answers, and to confirm (or challenge) assumptions. In this test, you will observe individual people using a ballot to learn about how users go about using the ballot, and to learn why things work or don’t work for them.

We identified three times in the election cycle when it makes sense to run this "ballot check" usability test:

• When something about the voting situation has changed since the last election, such as new machines, a new ballot layout, new regulations or ordinances have been enacted.
• When you have a good idea of what is going to be on the ballot for the next election.
• When some significant event happens that may cause the overall layout of the ballot of page to change.

The test can be done in a central, public place, such as the town hall or city hall, or in a place similar to a regular polling place, following the same guidelines for layout, traffic management, amount of space, light, and so on. If your elections are all mail-in, simulate a kitchen or dining room or some other home-like setup, if possible.

 

 

Download the LEO usability test kit materials

These files were updated May 14, 2009.

 

Usability Testing Ballots: What you need to know
An introduction that acts like a planning guide for to how to use the LEO kit, including guidelines for testing at different stages in an election cycle, or when changes have been made to the voting system or ballot design.

Testing Ballots (DOC-226kb) or PDF (159kb)

Session Script: Ballot usability testing
The session script is a set of complete instructions and forms to use in conducting the ballot usability test. It provides direction for moderating individual usability testing sessions, complete with what to say, when

Session Script (DOC-196kb) or PDF (95kb)

Session Materials: Forms for Participants
Consent and information forms for participants, and the demographic and post-session questionnaires, ready to print.

Participants Forms (DOC-198kb) or PDF (103kb)

 

Materials from specific tests:

Washington State Voter Registration Form (July 2009)

Sample Test Report: Ballot Usability Feedback
A template for writing a report on the usability test, to use to report results internally to elections departments or to the public

Sample Report (DOC-755kb) or PDF (707kb)

Training workshop handouts
Dana Chisnell developed this training course to introduce election officials to usability testing and to the LEO usability testing kit.

Ballot Design Task Force training slides (PDF - 881kb)

First version, used in Washington State (PDF - 397kb)

Poster, with portraits of people in the elections process and an overview of the LEO Kit - 33" x 47" (PDF - 88Kb)

 

MSU-UPA working group

These are the people who originated the LEO usability testing kit, in a workshop at the Michigan State University Usability and Accessibility Center

• Dana Chisnell has a user research and usability consulting practice in San Francisco.
• Whitney Quesenbery is a user researcher and usability consultant based in New Jersey.
• Sarah Swierenga and Mike Elledge are the director and assistant director (respectively) at the MSU Usability and Accessibility Center
• Josephine Scott is a usability specialist at TechSmith.
• Fred Conrad is a social sciences professor at University of Michigan
• Laurie Kantner does user research and consulting through Tec-Ed, Inc.

You can read more about the LEO usability testing kit on our Ballot Usability blog.

For more information, contact the UPA Voting and Usability Project.