David Alfaro: Scrum Costa Rica

Yesterday I heard someone saying proudly that he was able to conduct successfully usability test for non-internal-use product using his employees because they fit the target user profile, so recruiting users for the “usability exercises” was easy. That comment made me feel impelled to state clearly why you should look for REAL users to conduct usability test, and BTW what does REAL mean?

In Aggiorno we are totally concern about letting our users to shape the product, to give them value in the way the want. In Aggiorno we have a rigorous process to find the appropriate users to participate in our usability tests, so I can tell you we are able to observe high quality behavior during the usability tests, why?

Carolyn Snyder states it clearly in her book Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces (Interactive Technologies) when talking about recruiting in general Usability Tests (that is, not only applies for paper-prototyping):

“Co-workers aren’t real users. Co-workers are usually not representative of real users, even if they talk to users every day or used to be users themselves. Your co-workers know stuff that real users don’t, such as your company’s business strategy, acronyms, brand, other products, and so on. Very likely, your co-workers know more about your interface—even if they haven’t seen it—than real users do. Therefore, the behavior of a co-worker usually isn’t a good proxy for what real users will do. (Possible exception: You’re designing an intranet or other interface specifically intended for employees of your company.) ”

Anyone who has assisted to any Usability Week will recall (as I do) the guidelines about recruiting once you have done a user profile that fit your persona description:

1. Look for people who never had seen the product you are going to test. Your target market has never seen your product, you need to watch the first impression the final users is going to get, the pains that someone you want to convince will go through.
2. Look for people who had never been in a usability test. Once someone participates in one Usability Test, that person, consciously or not, will learn what makes a facilitator taking notes, will try please or annoy the facilitator. Neutrality and the effort to emulate a real setting are lost. At least, allow six months between tests of different products for the same user.
3. Look for people unfamiliar with you. Any feeling linked with the familiarity with you easily mares the objectivity of the test. Consider possible user’s thoughts: “My friend/girlfriend/father/husband/boss has worked so hard in this product! I would be uplifted during the test, I’m sure this is a great product”, “I don’t want to be fired” “I don’t want to hurt my friend’s feelings” “I hate this guy, I will be sure to let him know that his product sucks”.

All the companies that charges you for recruiting users for usability tests set good part of their quality by finding users that comply with those guidelines.

Any “tropicalization” of such guidelines of methodologies doesn’t make them better. The realities behind them are still valid everywhere. Even when the product is created in a “tropical” country, there is no reason to loose those guidelines, specially when your target market is not “tropical”.

Still, I am not against reviewing the UX with co-workers, even so remember this piece of advice of Carolyn Snyder:

“You don’t want more opinions. I’ve yet to work at a company where there’s a shortage of opinions. The whole purpose of usability testing is to gather data from real users, not yet another internal opinion. It may not be wise to redesign an interface solely on the basis of internal feedback. If a co-worker gives you feedback that you don’t want to act upon (at least not yet), then you’re put in the awkward position of explaining why you’re apparently ignoring the valuable advice you asked him or her to give.”

Remember, a usability test is getting a REALITY CHECK, not yet another co-workers discussion. Is that easy Not at all! In the case of Aggiorno, we had had to go the places where our real users are, we had had to move our usability lab in order to get them. We want to listen, to watch as real as possible the Aggiorno use.

Feel free to let know any other useful guideline in recruiting users for usability tests.