Glossary

What is concept testing, and why is it important?

Image of the post author Kadence International

What is concept testing? Essentially, it’s the process of getting your audience to evaluate or give feedback on an idea before releasing it to the public.

It can be used for adverts, physical products, websites, landing pages, and more. Here are some of the main reasons to use concept testing:

  • It’s a great way to evaluate ideas early on and make tweaks before sending something out to the wider public
  • It helps your team get company buy-in on ideas, justifying decisions by proving they work with empirical data

For these reasons and more, companies draw on the power of concept testing to make sure the ideas they come up with are actually viable in the real world before putting them out.

In this article, we’ll take a deeper dive into concept testing and what it’s all about. We’ll cover how to do it and the steps you’ll need to take.

How Does Concept Testing Work

Concept testing takes different forms depending on what you want to test, but generally, the process involves asking current or target customers e to engage with your concept and then evaluate it – either in person (for instance in a focus group) or online (either in an online community or by completing a survey or questionnaire to evaluate it.)

One of the most popular and effective ways of testing concepts is through an online survey so in this article, we’re going to focus on our top tips for getting this right.

When testing concepts through an online survey we ask questions to determine the appeal of the product, how innovative consumers thought it was, and so on. The goal is to gather feedback and to find out how the typical customer will respond to the concept, along with identifying any areas that need improvement.

So how do you get started?

How To Approach Concept Testing Through An Online Survey

Set a Clear Goal at the Outset

Before beginning, it’s extremely important to set a clear goal for your test. This is crucial because it’s what gives the survey its overall structure and direction.

Ask yourself and your team questions like:

  • Why are you doing this?
  • What kind of action will you take with the results?
  • How does this test fit into your overall strategy?

The answers to these questions will help you select your testing population better, choose the right metrics to track, and conduct a more effective and coherent survey.

Pick the Right Metrics to Track

The metrics you choose to track in your concept test are extremely important. They can make the difference between a useful survey that drives genuine positive change, and one that has very little effect other than to waste resources. Some examples of metrics to track include:

  • Purchase intent (how likely would you be to hit the buy button?)
  • Innovativeness (how innovative is the concept?)

These metrics will form the basis of your questions and shape your survey. Questions should focus on areas such as likes and dislikes, overall appeal, uniqueness, and innovativeness.

Give Your Survey A Clear Structure

Your survey should have a clear and coherent structure. Each section should follow intuitively from the next, with similar questions grouped together. You don’t want participants feeling  confused or frustrated at any point — it should all make sense.

A good way to help this along is by including an introduction section that clearly outlines what the respondent should expect from the survey, , along with instructions on how to complete it.

It’s also important to consider which method you want to use. Here are the main options:

Concept Testing Survey Methods

Get regular insights

Keep up to date with the latest insights from our research as well as all our company news in our free monthly newsletter.

The two main types are monadic design, which uses a single concept, and sequential design, which asks the participants to compare multiple concepts.

Let’s break that down a little further:

  • Single-concept (monadic) design. Here participants analyse one concept, learn the features, and then give their feedback and opinions in a survey. It’s a great way to get in-depth responses around one single concept, but it can be time-consuming and expensive.
  • Multi-concept (sequential) design. In this kind of test, participants do all the above, but then move onto another concept and repeat, and so on. It saves time and money by combining multiple concepts into one session, but there is a risk of participants getting bored and giving lower-quality responses as time goes on. What’s more, responses can be influenced by the order in which concepts are presented, so we make sure to always randomise the order to avoid any bias.

How Long Should a Concept Testing Survey Be?

There are many different opinions on how long your survey should be but our view is that it shouldn’t be any longer than 20 -25 minutes, At this point, respondents can suffer from what’s known as “survey fatigue” and the quality of responses can deteriorate as attention drops For example, SurveyMonkey recommend keeping it under 30 questions. Fewer focused questions can be better than dozens of less relevant ones.

Ultimately, it will depend on your concept, your audience, and the resources you have available.

Ask Screener Questions and Demographic Questions

Screener questions are a way to see if your respondents are relevant to your customer base. They’re usually placed at the beginning of the survey, and the goal is to filter out unqualified participants.

You shouldn’t be too specific here — you don’t want to filter everyone out. For example, let’s say you’re testing an online payment service. The screener question might be, “How often do you shop online?” 

Demographic questions usually come at the end of the survey. They’re a way to find out a little more about the people you’re speaking to and . usually focus on things like age or profession. Collecting information on demographics allows you to ensure that your survey is representative, as well as to analyse responses based on age, gender or socio-economic grade to identify any significant differences. Demographic questions can be considered sensitive, so by putting these at the end of a survey, you avoid people being put off from taking part in the survey from the outset.

Send the Survey Out

Once the survey is ready, it’s time to select a sample of people to test it on. This should be  representative of your overall population you are interested in targeting.

Analysis and Action

The final (and most important) part of the process is to analyze the results of your concept test and take action.

The way you analyze the results depends on how the survey was structured. For example, if your survey used Likert scores (asking participants to answer on a scale of ‘very unlikely’ to ‘very likely’, you can use something called top 2 box scores in your analysis. This allows you to use one metric to compare across concepts or attributes. Remember –  never pick a ‘winner’ unless there’s a statistically significant difference.

When you have your results, it’s time to put them to use. You can use your results to justify a product launch, get buy-in for a concept, or fuel further R&D.

When sharing your results, for example with company management, it’s important to be as clear and detailed as possible. Do all the work for them, and leave them in no doubt about your findings.

Concept testing can be an extremely effective way to get hard data in support of a concept. It helps you and your team make clear-headed, evidence-based decisions and create products that delight your customers and drive the company forward.

To find out how Kadence can help you harness the power of concept testing — and take a more data-driven approach to your organisation in general — reach out to request a proposal.