What is Qualitative Testing?
Qualitative testing—which can take the form of interviews or other directly-observed usability tests—tends to be exploratory and with the goal of gaining a more in-depth understanding of the user’s experience. It can be in-person or via video-conferencing tools like Skype or Google Hangouts.
Qualitative testing aims to gather information on a user or group’s everyday life experiences and motivations to see how these details might affect their use of the product or tool.
Results
When using qualitative research methods, it is important for the researcher to recognize that they may have an effect on the research results. Since it is the human experience that is being researched instead of hard data (number of clicks, login location trends, etc.), it is difficult to remain completely objective during testing or when assessing results.
The results also won’t necessarily be reproducible. This is because the factors that affect a user’s experience are varied and can change from day to day.
These factors can take the form of a user being more stressed one day than another due to new events in their life or having more responsibilities that day, users having completely different life experiences than one another, bad weather affecting the user’s mood, and anything else that affects a person’s personal or emotional life.
Results of qualitative testing are typically shown as themes or categories instead of numbers.
Example
Here’s a great example from The Interaction Design Foundation on the difference between quantitative research and qualitative research:
To illustrate the difference, let’s say you want to study a user group’s exercise habits. You can choose to study these using either quantitative or qualitative research methods.
If you use a quantitative method, you could create an online survey and distribute it to a large number of participants. Participants must answer predefined questions about their exercise habits such as “How many hours per week do you exercise?”
If you do your job dutifully, the survey answers can be numerically summarized without bias from your own opinion or personal experience with exercise.
If you use a qualitative method, you could choose to do interviews with a limited number of participants, where you talk to the participants about when, where, and how they exercise.
Because the interview is similar to a conversation, your results will depend on how you ask follow-up questions to the participants’ answers – and how you do that will to some extent depend on your personality and your own experience with exercise. Similarly, the results of the interview also, to some extent, depend on a subjective interpretation of what the participant has told you.
Testing Methods
From ”12 UX Research Techniques (Quantitative and Qualitative)” by Kevin Dalvi:
Interviews
Typically covers three types of interviews:
- directed interviews where the researcher asks specific questions to the users and attempts to compare responses with other users
- non-directed interviews where the researcher attempts to have more of a general discussion with the user(s)
- ethnographic interviews where the researcher observes the user(s) in their own environment to understand how they approach certain aspects, accomplish certain tasks.
Surveys and Questionnaires
A quick way to collect information from a large number of users but their obvious limitation is lack of any interaction between the researcher and the users.
Usability Tests
Involves asking user(s) to use the app/product to accomplish certain goals. There are three variations of such tests:
- moderated testing—where the users are brought into the lab and given specific tasks or tests to perform.
- unmoderated testing—where the users complete the test on their own time typically remotely.
- guerrilla testing—a more casual form of testing where random users at a social or community location are asked to use the app/product and provide informal feedback.
Card Sorts
The primary goal of a card sort test is to understand how users perceive relationships and hierarchy between various content, categories and other information. This is typically used to generate appropriate information architecture or site maps.
Tree Tests
Similar to card sort, the primary goal is to test whether the product/app has the appropriate level of information hierarchy is being designed within the product.
A/B Tests
Focuses on providing the user with two or more options and documenting the user’s preferences amongst the options. There are also advanced or focused A/B tests tests on specific aspect of the product such as the Design Elements, Information Hierarchy, Navigation and more.
Persona Development
This is essentially a representation of a group of a user who exhibit a very similar pattern in terms of behavior of using the application regardless of age, gender, location, education and profession.