Remote Usability: Insight into New Tools

By Mike Madaio

Mike Madaio is currently the Chief IA for QVC.com and the Secretary of PHICHI, the Philadelphia Chapter of the ACM SIGCHI. He also keeps a user experience-related blog at http://mikemadaio.com.

Ethnio and MindCanvas
On a hot Friday evening in August, Philadelphia’s CHI chapter turned up to learn about two new remote research tools available to UX professionals: Ethnio and MindCanvas. Nate Bolt, president and CEO of Bolt|Peters (Ethnio), and Rashmi Sinha, founding principal of Uzanto (MindCanvas), were both on hand after presenting at AdaptivePath’s UX Week in D.C. a few days before.

One might think it strange that Bolt and Sinha chose to present together – after all these companies do play in the same space. Interestingly, however, both presenters feel that these products are not competitors, but wonderful compliments to one another, mainly because Ethnio is a facilitated testing solution while MindCanvas provides automated testing.

Ethnio
The goal of Ethnio is to offer moderated usability testing in a native environment via a telephone/web interface. Instead of recruiting weeks ahead of time and hosting users in an unfamiliar usability lab, Ethnio recruits in real-time and allows users to participate on their own computer. The main advantage to this method is that it catches people in the act of completing a task, and as such ensures a realistic, focused test experience.

To engage possible testers, Ethnio displays invitations via DHTML; interested parties are asked to fill out a short qualifying survey. At the same time, a facilitator (or group of facilitators) can be monitoring the test queue to act quickly once a response is submitted. The test itself is conducted through both a phone and web connection. Ethnio’s server calls both parties and records the ensuing conversation; the user must also install a small program (either Active X, Firefox plug-in or EXE), which Bolt described as "by far the quickest desktop setup of all the remote products." After the test is complete, a flash video of the user's desktop is synched up with the telephone audio to produce a complete presentation of the test.

Ethino displays invitations to test with DHTML

Ethnio desktop

Enhancements to Ethnio are in the works; meeting attendees were happy to hear that event tagging and collaborative note-taking features should be implemented shortly, and that video editing is also on the way.

The speed with which users can be recruited for Ethnio allows for more testing to be done in a day that with normal recruiting methods, which can save both time and money. Some projects, however, require an even larger testing group, which is when Uzanto’s MindCanvas becomes a viable option.

MindCanvas
MindCanvas is an unmoderated research tool that takes online surveys to a higher level. Research methods such as card sorting, click maps, free listing, build-your-own interfaces and more are transformed into fun exercises that engage and entertain the user while gathering vital data.

Rashmi first showed off a brilliant flash-based card sorting game that supports complex features such as duplicate cards, groups within groups, and make-your-own card. By pulling the basics of traditional card sorting into this easily distributed format, MindCanvas can allow researchers to gather valid card-sorting data from a much larger audience. The flexible, detailed reporting tool ensures that this data can be used quickly and effectively.

MindCanvas Card Sort

Some other MindCanvas tools include:

• Click Map: The user is shown a website and asked to find a particular element. This program will track each click and then provide a map with the entire results set.

MindCanvas Click Map

• Sticky Notes: Ask users to associate terms with images. For example, a design is shown alongside sticky notes with text descriptions (i.e. “innovative”). The user is asked to label page elements with the sticky notes.

MindCanvas Stickly Notes

• Build-Your-Own Interface: Allow users to construct an interface via drag-and-drop tool, based on page elements they are given.
• Free Listing Study: Find out what is on users’ minds in regards to a specific topic. For example, a user might be asked: “What is important to you when looking for a cell phone?” and given a certain amount of time to list out everything that comes to mind.

More Information
All in all this was a very entertaining and informative presentation. Nate and Rashmi are open to feedback and questions and welcome anyone interested in either product to contact them to discuss your project or company.

Nate Bolt
President & CEO, Bolt | Peters
nate@boltpeters.com
http://www.ethnio.com/

Rashmi Sinha
Founding Principal, Uzanto
rashmi@uzanto.com
http://www.themindcanvas.com/

PHICHI
Philadelphia Region's Chapter of the ACM SIGCHI
http://www.cis.drexel.edu/PhiCHI/