Teaching Election Officials Usability Testing
Whitney is a past-president of UPA, and leads the Usability in Civic Life project. In civilian life, she works on user research, usability and IA projects for organizations like the UK's Open University and the US National Cancer Institute.
The election calendar is very tight, with legally mandated deadlines and other constraints, all conducted in the public view. The UPA Voting and Usability Project wanted a way to fit usability testing into that schedule, and give election officials a way to do what they all want: run excellent elections.
A group of us** got together to develop a testing kit for people who have no specific training in usability or human factors engineering. We needed a test that doesn't require any special equipment or a laboratory.
Our goals are to:
The result is "LEO Usability Testing Kit". We hope it will be part of a process of changing how the elections community thinks about the design challenge of election materials. Our goal is for the usability of the ballot, instructions and other election information to be just as (maybe even more than) important as other considerations.
Dana Chisnell ran the first training session over a year ago for officials in Washington State. Afterwards, we heard that several counties used the kit, and the results went beyond the ballot as they found other problems in voter education material -- and were able to fix them before the election!
With the release of Better Ballots, the report from the Brennan Center, and its endorsement of the importance of usability and the LEO Usability Test Kit, there has been a lot of interest in both the ballot design issues discussed in the report, and usability testing as a final check of ballot layout and instructions. We have already done presentations and training in Ohio and Iowa, and plan to present in several other states this summer.
Will it work? After the 2000 election, there were many articles saying that thousands of participants would be needed to find ballot usability problems. We disagree. We are not looking for performance numbers or "valid results" in that sense, but for a diagnostic view, in which we can consider hesitation, requests for assistance and other signs of "success after a problem" as an indicator of a larger problem that is fixable.
At the UPA conference in Baltimore this summer, Steve Krug reported that in his workshops, he runs short, informal tests on a web site volunteered by a member of the audience. and has never failed to find at least one problem. We've had some of the same experiences in training and in usability testing we have observed. Here are two examples:
The LEO Kit talks about when it's time to call for assistance, and when this quick test will not be sufficient. We hope to provide an understanding of best practices in the same way any of us might train a newly minted user experience group in a company.
This is not the only usability in the election lifecycle. There are usability and accessibility standards for voting systems, tested design templates and opportunities for public review. The LEO Testing Kit is designed to fill the last step: a final check on the finished ballot.
** The LEO Usability Testing Kit was developed in a workshop at the Michigan State University Usability and Accessibility Center with Dana Chisnell, Whitney Quesenbery, Sarah Swierenga, Mike Elledge, Josephine Scott, Fred Conrad and Laurie Kantner.
|Contact the Voice|