Resources: UPA 2004 Idea Markets
How do you
turn usability test findings into recommendations?
Activator: Kris Betcher
No surprise here! The answer to this question is it depends. It depends on the usability specialist’s role within the company, it depends on their relationship with the developer/designer, and it depends on the product. During Idea Market 2004, we learned a variety of strategies and techniques from the participants to help answer our question.
A key element in turning usability test findings into recommendations is the usability specialist’s role within their company. At some companies, the usability specialist works with a designer, sometimes they work with one person who is both designer and developer, and sometimes they are an outside consultant working with a developer or manager.
The strategies and techniques to create and implement test recommendations vary with these roles. Outside consultants have an easier time getting their recommendations accepted. Often, the outside consultant’s recommendations are accepted as more credible, whereas a specialist working within a company can find it more difficult to get the usability results implemented (seen more as just coworkers). These roles also depend on how integrated usability is into the development lifecycle. If usability is an accepted part of the process, the designers and developers are more open to recommendations.
The usability specialist’s relationship with the designer or developer is another key element in turning usability test findings into recommendations. A developer is much more likely to implement recommendations from someone they accept as credible and accurate.
Bottom-line advice? Determine your role within your company, forge close
relationships with designers/developers, and provide consistent recommendations
with data or clear rationale to back it up.
Our discussions were guided by the following questions, and below we list all of the answers from the Idea Market participants.
- What is the role of the usability specialist in formulating
specific, implementable recommendations?
- This depends on the usability specialist’s role within the company. In some companies, the usability specialist works with a designer to resolve usability issues, and in other companies they work directly with the developer.
- Some companies only want results, not recommendations; others want recommendations, which is a big part of the usability specialist’s role.
- This is a collaborative effort, but the ultimate responsibility lies with the usability specialist.
- Think of yourself as a usability “developer” instead of a usability “specialist.”
- What is your role in seeing that recommendations are actually
- Some specialists work with the designer to develop recommendations, and then bring recommendations to developers for implementation.
- After the final report is distributed, how involved are you
in the project?
- I give the developers a list of recommendations with priorities assigned, and it’s up to them to implement those recommendations.
- What specific techniques do you use when formulating recommendations?
- Work hand in hand with the developer.
- Enter issues into a database (bug tracking software) and assign to the designer and track which recommendations are implemented. The bug is assigned back to the usability specialist after it is fixed in order to confirm resolution.
- Create a matrix of issues and invite the designer/developer to a meeting to go through them.
- I create a spreadsheet of change requests and submit it to the project manager to determine what will be changed and when. >
- I recommend small incremental changes over time, not drastic ones.
- Carefully consider the language you use in reports, and back up the results with data and rationale.
- Sit as close to the developers as possible. This allows you to easily, frequently reinforce your role and follow up on your recommendations.
- How receptive are developers to your recommendations?
- This depends on the relationship with the developer.
- Finding results the developers accept is difficulty. They sometimes criticize the users or techniques used for testing.
- It is important to use direct feedback (quotes) from users to add credibility to the test results.
- Bringing the designer/developer to the testing is crucial.
- Get okay on the users and the test plan ahead of time from stakeholders and developers to avoid rejection of results later.
- Keep it objective. Always go back to the user data – it is about the users’ needs, not the designer.
- How do you track the findings, the recommendations, and the
actual changes to the product or Web site?
- We learned that some people enter issues in their project tracking software so that the developer needs to address the issues as it they were any other bug.