Tips and Techniques
Focus Groups vs. Usability Testing
When To Use Them and Why
There is a common misconception that focus groups are an appropriate method for assessing the quality of the user experience of a product or a website.
While focus groups are a useful research tool, like all research methods they have their limitations in terms of the kind of information provided for decision making purposes. The choice of research method depends on the kind of decisions which need to be made.
Focus Groups - Strengths
Focus groups are best used in the early stages of during Discovery and Assessment and can provide rich information on opinions and attitudes of the target audience. They are ideally suited in an exploratory situation - to get a feel for the range of opinions, to understand the reasons underlying preferences, to get a basic list of user requirements - information which can provide direction to a development team in the early stages of development. They are not a good source of behavioural data.
What people say they will do, compared to what they actually do, is notoriously unreliable.
Intent and action are two different language concepts for good reason. What we say we will do (intent) and what we actually do (action or behaviour) often vary because we can't predict contextual factors which may alter our behaviour in a given situation. Nor are we fully aware of all of our behaviours - some everyday actions are so commonplace that we sometimes forget about them when we are asked to recall a set of behaviours. For instance, if you ask people something as simple as "what features on a telephone do you use and approximately how often?" and compare the results to direct observation data on the same question; the results are usually staggeringly different.
The more complex the behaviour pattern you ask people to predict or recall, the worse people are at predicting or recalling. So how do you get around this problem? That's where observational techniques like usability testing come in.
Usability Testing - Strengths
Usability testing provides behavioural information by assessing users' performance on pre-determined tasks critical to the successful use of a product or website. Performance measures - like time to complete task, number of errors, success rate - are collected along with ease of use ratings for a series of tasks within a typical usage scenario. This information is used to identify roadblocks and lesser obstacles to successful use of the product or website.
Usability testing is best used throughout development to provide performance-based information on how effectively the target audience can use the functionality of the website or product. Comparative testing is used at the user requirements stage of development to compare different potential design options. Assessment testing is used once a design has been partially developed to identify potential roadblocks; while validation testing is completed at the end of the development cycle to determine the effectiveness of the design implementation in meeting the user requirements. User interaction testing can also be used to benchmark the user interaction of a site before and after re-design. By using performance-based metrics in user interaction testing you can quantify the improvement in meeting your business objectives.
| Focus Groups | Usability Testing | |
|---|---|---|
| Method | Group discussion with 6-8 participants | Individual testing of 6-8 participants |
| When to Use | Pre-development, early in program | Throughout development (concept, detailed design, verification) |
| Key motivation | What are customers' wants and desires? | How well can end users use the functionality a product provides? |
| Key focus | What will motivate potential customers to buy the product? | What will make the product easy for end users to learn and use? |
| Output | Opinions, attitudes, preferences | Behavioural measures and preferences based on usage, ease of use ratings |
Although focus groups typically include 6-8 participants, to the extent that participants interact with each other and can influence opinions of other participants the outcome of the session should be treated as one data point. The same number of participants in a user interaction test may be treated as independent data points.
Focus groups are best used early in a development program to aid in defining user requirements. Once development has begun, user interaction testing should be used at set milestones - as early as a wire frame design - to test basic information organization and navigation structures.
Focus groups are often used to ask participants for their preferences between two or more concepts. This technique has value if it is used to explore the factors which cause participants to choose one concept over another, but it is a poor predictor of future behaviour. We humans are much better at making a satisfactory choice after we have tried out the product; that is why we all like test drives, or 30 day return policies.
In summary, both research methods can play a useful role in supporting design decisions in any product or web development project. To get the most value for your research budget it's important to know which method to use and for what purpose.