The ultimate guide to consumer product testing

How is your product received by consumers or business decision-makers? What are the pros and cons of a change in an existing product feature or new varieties of your current big-sellers? Why is a product failing to perform? You want to tweak a formulation, messaging or packaging to cut costs or reach new audiences – but will that scupper or supercharge your sales? The answers to all these questions can be found in product testing research.

What is product testing?

With product testing, we’re not looking to establish general consumer attitudes or behaviours. Nor is this about standing up a new concept or looking for gaps in the market. The primary job of product testing is to tell us how people respond to an actual product – including how they use it and what they think its qualities are – allowing brands to decide whether and how to market it.

When should I do product testing research?

Let’s look at the natural marketing life-cycle to explain how product testing research can support the emergence, and successful exploitation, of a product – and place it in the context of a wider field of market research:

• Ideation – dreaming up an idea worth pursuing. Research helps identify unmet consumer needs, value-chain opportunities, potential applications of innovation and new markets.
• Screening – a rigorous approach to deciding which ideas are worth pursuing, again drawing on research and feasibility work, and assessing potential audiences.
• Concept testing – seeing how the manifestation of ideas might work in the market, leading to additional screening out of less viable concepts.
• Prototyping – designing products to prove mass production feasibility, form factors and feature sets.
• Early product testing – evaluating consumer attitudes to the product itself, either in controlled settings, in the field or in everyday contexts; typically unbranded or with highly simplified packaging.
• Late product testing – which might include feedback from earlier tests to refine messaging, packaging and final form factor.
• Testing iterations of a product to forecast the impact (on sales and usage) of changes to features, formulations, targeting and marketing – often in response to changes in sales patterns or negative customer feedback.

In other words, product testing is distinct from concept testing. It’s all about refining the delivery of something that is (or soon will be) a finished product. This might include changes to feature sets, the marketing pitch, pricing, ideal target audience and other details. It’s not so much whether the product works – it’s how the product will work best.

Use cases for product testing research

In summary, then, you can apply product testing to:

• Find out how a close-to-final version of a new product might perform.
• Tweak that product to optimise its performance on launch.
• See how a new product is performing post-launch.
• Test the effect of changes to product design or presentation.
• Evaluate or explore how a product is marketed.
• See how well consumers in a new market will accept an existing product.
• Undertake ‘penalty analysis’ to see which qualities, when changed, alter consumer options about a product.

Product testing in action – a case study

One way to think about the value of product testing is as a way to optimise the introduction or evolution of a particular product. A good example is work we’ve done with a beverage brand to launch a new range of iced teas. The client wasn’t launching a brand-new product – they had worked up some new flavours and wanted to know which they could launch successfully, how these might affect brand perception and what consumers made of them.

There were eight formulations to be tested. We added in an established variant to act as a benchmark, giving us a way to test the relative strength of the products. We measured various metrics with consumers to provide comparable scores as a key insights.

The product isn’t necessarily going to change in this case. It’s a chance for the client to check which variants might work best, to optimise the roll-out and then make minor refinements if the research delivers consistent feedback on particular elements. How research subjects describe the teas might also help shape marketing and packaging for instance.

In other cases, clients might test out product names and straplines on consumers while they’re testing the product to create a range of possibilities, not just on the branding and marketing, but also on likely target audiences and even pricing. Does the product live up to a premium positioning? Or will it chime with more down-to-earth messaging?

The role of product testing guidelines

We work with many clients that have well-established product testing guidelines – a set of standards that enable them to better evaluate products over time and give them clearer benchmarks for making decisions. For large corporates in particular, the product testing research project is an exercise in generating fairly standardised numbers – data that fits a well-established, almost algorithmic approach to evaluating potential product performance.

When this isn’t the case – or if the guidelines are relatively basic – it’s a good idea to establish some clear ground rules at the start of a project to ensure it delivers insights that will shape client decisions. You might need to agree:

• How the product should be stored, prepared, presented and used.
• The audience it should be tested with, and how to recruit them.
• Where to conduct tests with participants (see below).
• What metrics to record.
• How to record their experiences – and other feedback.
• The research methodologies that will work best.
• Whether to use a control product for comparative purposes.

Introducing a framework helps everyone understand what success looks like for a product: for it to go forward, what will the research need to show? Is it being on par with an existing or rival product on overall performance? Does it need to be statistically stronger on key metrics?

In some ways, it’s akin to a science experiment: you outline the aim (proving it’s better than the existing product) based on your prediction (the product design); we provide a sound, rigorous methodology to test that assumption (the research); and the result gives conclusive result to tell you how to proceed.

How does product testing work? Where and how to test

Different objectives of product testing will suit different methodologies. A lot depends on what brands already know about the product and the way it’s perceived; on what they want to learn from the tests (see below); and the type of product under review.

There are broadly three environments to conduct product tests. Let’s look at the use-cases and the pros and cons.

1. Central Location Testing (CLT)

This is where participants are invited to a facility to undertake the test. This is ideal for evaluating products in controlled conditions, especially when testing a variety of use cases. It’s also suited to products that won’t be used as much in the home – especially in food outlets, for instance.

A good example would be a new foamless cappuccino we tested. To get comparable results, the same machine was used in different central testing centres, with the client providing an expert barista to produce the same product every time.

CLT is ideal for evaluating products under controlled conditions – testing different fragrances, say, is hard if the conditions allow for cross-contamination of scents. But it’s also very useful where confidentiality is important. We set up a CLT in a hotel, for example, so that a new tech product could be tested by invited consumers without the design leaking. Non-disclosure agreements might be a feature of any product test, but for this kind of commercially or technologically sensitive research, the controlled setting can be helpful too.

The other advantage of CLT is liability management. Some products – foodstuff and cosmetics, in particular – might cause adverse reactions with test subjects, and it’s easier to screen and monitor them on-site.

You can find out more about central location testing in our guide.

2. Street Intercept Testing (SIT)

This is literally grabbing participants in an ambient setting for a few minutes to get them to try something and test their reactions. This works well for relatively simple research – the questionnaire will need to be relatively quick in a supermarket or street setting – and for targeting particular participants. Testing a new cheese at the deli counter in a supermarket would be one application.

It’s also ideal for capturing insights within specific use locations – when a central facility would be a little abstract. We worked with a sports beverage brand to test a special protein-rich drink. The use-case is post-exercise, so intercepts with the target market in a gym setting yielded much more insights than a central location could have.

3. In-Home Testing (IHT)

For many products, the consumer’s home (or, in some cases, their workplace) will be the usual usage location. Getting the products into the home for a period of use, then running online, telephone or face-to-face follow-up questionnaires is a great way to see how they work ‘as intended’.

In-home testing tends to be ideal for more sustained testing. The taste of a new iced tea or reformulated cheese can be tested fairly immediately. But a toothpaste, cleaning product, in-home device or even lightbulb, will only reveal itself properly over a few days’ use. Out of the control conditions, we can learn more about how good instructions for use are; we can see how consumers might use the product in their daily lives or in combination with other products; and we can monitor evolving opinions about the product as they get used to it.

Obviously in-home testing has been popular during Covid-19 lockdowns – not least because many products are now being consumed or used in the home that might otherwise have been ambient products; but also because centralised or street intercept tests have been harder to run for biosecurity reasons.

Note also that IHT allows for different research methodologies. As well as post-use surveys, we can get consumers to keep diaries of use, highlighting a wider variety of situations and providing more qualitative inputs.

How to do product testing

Where to start

For many companies, product testing isn’t the start of their journey with us. This kind of research is often part of a much bigger engagement process around a brand or product line; or it might be commissioned by a brand we already do different kinds of work with. So the starting point is rarely a cold introduction to a product.

But even with some engagement beforehand, the first step in product testing is to look at the product and the client’s requirements, and then design an approach that will answer their key questions.

For some, those questions will be extremely precise. For example, one detergent brand asked us to test out a new toilet cleaning product. They knew exactly what segment they wanted to target – ABC1 consumers in their 30s and 40s who were already familiar with the brand – and even the methodology they wanted (in-home testing).

That’s largely a logistical challenge – getting the product and a control cleaner into their homes, in plain packaging, so they can be tested side-by-side; then running an online survey to generate some quantitative data and some qualitative feedback comparing the product to a known comparator.

Another example might be a commercial-kitchen mayonnaise we tested. The client was keen to assess not just how the product performed against other formulations of mayo, but also what professional chefs thought of it in different applications. Will it be at least on par with the existing product? What recipes or dishes did it suit? And how did it compare commercially?

One thing to bear in mind is that you should be testing for things you might change as a result of the insights we generate. Knowing what can change (from packaging and marketing, to cosmetic attributes or even key design features) as a result of the research findings – and what you definitely can’t alter – will ensure the tests are focused and useful.

Methodology reflections

We find that CLT is generally better for ‘sequential monadic’ testing. ‘Monadic’ means the consumer is evaluating a single product, and this is obviously possible in any environment. Even ‘paired comparison’ testing – head-to-head – can be done in-home. But sequencing the comparisons scientifically (standalone, then in head-to-head, for example) often generates more reliable data.

In terms of participant recruitment, clearly targeting the audience accurately – whether in the field, via lists of consumers, or panels – is key. They are often motivated because they receive free products. But in some cases, especially with the more in-depth or time-consuming studies, the chance to earn money is also a motivator.

Hard and soft questions

Product testing can answer a lot of questions. For seasoned clients, they’re often very precise ones – they’re seeking standardised data on usage and performance that will help them contextualise the product within a portfolio.

A good example of a ‘hard’ question might be pricing. Using techniques such as the Gabor–Granger method (to understand price elasticity) or the Van Westendorp Price Sensitivity Meter (which creates an optimal price point for a given audience), research can reveal a lot about the economics of a product.

Hard questions like that are often central, even when we’re working with smaller companies that are looking to take a product from prototype to production and need to calculate the risk/rewards involved.

Smaller companies, however, are more likely to be asking ‘soft’ questions, too, where quantitative surveys are augmented with qualitative insights. They might be trying to learn more about consumer attitudes to the category as well as the product; or develop a deeper understanding either to tweak the product being researched – or inform future innovations.

A good market research agency can really help with this part of the process. For example, in some companies there might not be rigid product testing guidelines in place. But by explaining what they need to know to market the product, what they might be able to change about it and what they’re not sure about, we can help companies come up with fieldwork that will deliver clear metrics and provide answers to their key questions.

What’s the outcome?

A well-planned, well-run product testing project is rarely just looking for a blunt ‘go/no-go’ answer to a product roll-out or adjustment. Although many big brands have a well-established formula for conducting product tests – designed to plug data into their tried-and-tested algorithms – even these clients will often use the test as an opportunity to learn more about the product in different dimensions.

Sometimes that’s just a by-product of a sufficiently expert and thoughtful product test. As market research professionals, we learn a lot more about products during tests than the raw data suggests. Often it’s the degree of flexibility the market research team brings to the product test that makes it most valuable.

That’s true whether the primary objective is standardised data on product attributes – or semi-quantitative work with a healthy dose of qualitative inputs to shape decisions. By making sure the parameters for the product’s adaptation are clear and the questions about it well framed, we can ensure the right blend of methodology and insights meet the client needs.

A good example of that would be taste tests for a new formulation at a chocolate brand. ‘Super tasters’ working at the client will arrive at some finely calibrated formulation, created to be aligned to brand values and differentiate the product. But it’s ordinary consumers whose verdict will shape its ultimate success.

Looking to embark on product testing research?

With experience in product testing research, we can combine the inputs and recommend robust methodologies to make sure the product hits the sweet spot in the market. Find out more about our product testing capabilities or get in touch to discuss with our team.