I just finished the round #2 of Usability Tests of Aggiorno. It was amazing! I found out many important issues regarding how the user expects Aggiorno to behave given the Interaction Design it presents. And because I assume you are here to receive value I will tell you the generalities of doing a usability test and the general lessons.
I can’t show you specific details about the detected issues neither clips because of privacy matters outside of my control. However, in a near future post I can talk a bit about the design principles you always need to remember when designing User Interface.
The Process
The Usablity Test was conducted in a web development company called Cis-Solutions. From that company, five users were picked for this test. The selection process was based on a questionnaire where we were sure each user fits our target persona. The main aspects in this sample are:
1. College’s Bachelor Degree.
2. 20 to 30 years old.
3. 1 to 3 years working in ASP and/or ASP.NET
4. Uses Visual Studio Environment usually.
5. Develops sites of 16 to 50 pages
6. Maintains sites of +50 pages
Due technical problems during test executions, I wasn’t able to gather data to do a System Usability Scale (SUS). Instead of SUS, I resorted to ask the users to score (from a scale from 1 to 7) the satisfaction level using Aggiorno. Additionally, I asked them to score how useful Aggiorno is.
After a short but complete explanation of what Aggiorno is, does and how it is invoked, each user was asked to do some tasks with Aggiorno in a Web site already opened in Visual Studio 2005. Those tasks were carefully selected in order for us to see user’s behavior in our intended User Interface.
Conducting a Usability Test
Every time I conduct a usability test, I learn a lot and become more aware of more details. “User A is tapping his feet” “User B is humming” “User C is yawning” That’s behavior!! That together with “Think aloud” exercise allow me to collect the user is feeling (i.e. usability). I always ask the users: “Please pretend I am hot here (neither is any observer) so can swear freely in the case you want to.” I collected a big amount of notes in my chronological log. This is an art!
Finding patterns in raw data and reporting
Reviewing collected data is like drinking water from a hydrant. My preferred method for organizing that data into themes is Affinity Diagrams where each sticky note has a color correspondent to a user: User A has sticky notes in green and User B in yellow and so on. So I and observers (it would be great if any) organize in themes all the sticky notes. If a theme has all the colors available it means that that is a common issue, worthy to be noted.
Because the sticky notes are OBSERVATIONS, that is, OBJECTIVE NOTES, you have to describe consistently the behavior (crystallize the pattern) and make it an issue for the report.
What to do with that?
Soon I’ll let you know how to manage the Debriefing meeting.
There are so many details I am skipping in doing this post, so feel free of contacting me.
Why is User Studies so important for me? Because it means getting closer to user, being iterative, being open to the issues you are going to find and the potential changes they imply, learning to watch as an alternative to listen, inspecting over the collected data, adjust your course because of the lessons your learn from the tests. Do you follow me? For me THAT’S BEING AGILE!
People who looked at this item also looked at…
