Your Website's Search Interface: WebRef Update Feature | 2
Collecting Feedback About Your Website's Search Interface
Conduct the Sessions
Get the Person to Talk: Make the person feel at ease before the session. Ask them to think out loud as they work. Tell them to tell you what they like or don't like, and what they understand and don't understand.
Feedback During the Sessions: There is a tendency to want to empathize with the testers. You want to make them feel better when they are confused, answer their questions, laugh at jokes. Empathizing is fine. Distracting is not. Do not give any verbal or visual cues that you are agreeing, disagreeing, surprised, or confused. If you are quiet, the tester will get used to doing the evaluation and ignoring you. As you become more adept at running sessions, you can establish a kind of nod that acknowledges the user's comment, but doesn't agree or disagree with the statements.
If the tester is absolutely waiting for a response, consider using one of these phrases:
"I have noted your comments. Please continue when you're ready."
"We can go to task four when you're ready."
"That's a great idea."
"That is an excellent point."
These might make testers think: 1) they should elaborate about the one topic they think you are interested in, 2) other points they make are not great, 3) they should try to redesign the interface for you.
Evaluate the Data
When analyzing search data, it's very important to realize how the task you constructed might have affected the user's behavior. For example, the task is: Read about Ford and Toyota, the user types Ford and Toyota cars. Your conclusion should not be that users do not know what "and" means in a Boolean context. Instead, consider whether the wording in the task lead them to choosing their search string. (Really, a better task would have been: Your brother is happy with his Ford. Your sister is happy with her Toyota. Get information about each company to help you decide which car to buy.)
Test Search using a Paper Prototype
Search interfaces can be tested with paper prototypes. The earlier you collect usability feedback, the more changes you typically can make. And, if you collect feedback using a prototype before the site is online, you will also spend less money redesigning than you would if you waited to test the online version. Preparing a search paper prototype is time-consuming, but worth it. Some simple things to remember:
* Prepare search results for various search scenarios, including: totally right results, totally wrong results, results that have some right and wrong links, many results that spread over several pages, results that fit on one page, no results at all.
* Anticipate the words and phrases users might type, so that you have realistic-looking search hits.
* During the sessions, have the users actually do the steps. This will teach you the exact words they would type with any separators they might use. And, it will tell you if they know how to start the search. Ask them to click the mouse (their finger). When the cursor is in the search field (a plain white strip) hand them a pencil. Make sure they type (write). If they would have clicked a search button in the actual interface, they should have to do this here too before you make any results appear. This might seem like overkill, but it's not. A person might say: Well I'd type phones or phone systems with two lines and caller id. But, if they were to type it, they might actually write any of the following, which could return different results:phones or phone systems with two lines and caller id phones with two lines and caller id phone systems with two lines and caller id phones, two lines, caller id etc.
WebReference Interview with Jakob Nielsen: http://www.webreference.com/new/nielsen.html
Summary and review of "Designing Web Usability": http://www.webreference.com/new/nielsenbook.html
About the Authors:
Jakob Nielsen, Ph.D., is a principal of Nielsen Norman Group. Before founding this company he was a Sun Microsystems Distinguished Engineer. His recent book, "Designing Web Usability: The Practice of Experience" has a quarter million copies in print in eleven languages: http://www.useit.com/jakob/webusability/
Kara Pernice Coyne is a Senior User Experience Specialist with Nielsen Norman Group. Before joining the Nielsen Norman Group she managed the usability group and was a principal specialist for Lotus Notes and Domino products.
Nielsen and Coyne are just about to embark on the User Experience World Tour: a conference about Web usability that is coming to twelve cities on four continents: http://www.nngroup.com/worldtour/
Written by Jakob Nielsen and Kara Pernice Coyne and
Revised: Oct 27, 2000