Product development and market research

In discussing market research and new product development, a famous story often comes up about Steve Jobs dismissing consumer market research as a tool for shaping new Apple products. The driving force behind the Mac, the iPod, and the iPhone famously said in a 1985 Playboy interview, “We were the group of people who were going to judge whether it was great or not. We weren’t going to go out and do market research.”

It is, of course, one of the most widely debunked stories in business. Apple does conduct consumer market research – and is, arguably, in its pre-eminent position precisely because it innovates using insights generated by analyzing in incredible detail its consumers’ behaviors and the market appetite for its products. (There’s video of a young Jobs extolling the virtues of market research for these purposes – it’s 90 seconds well spent.)

In the world of new product development, market research often reveals that most new offerings are iterations of existing products. For every genre-busting innovation, there are tens of thousands of new iterations of existing ideas, tweaks to brands, and updates to proven sellers. In most cases, some kind of market research will have shaped the new iteration and how it was conceptualized; helped stand up the business case for it; framed the marketing; and guided its introduction to consumers. So how does market research help businesses design and launch successful new products?

Integrating Market Research in New Product Development: Each Stage of the Innovation Funnel

There are lots of different ways to describe the innovation process, broadly broken down into three phases: ideas, concepts and creation. It’s not a science with a standard formula, however, but there are some common steps. For example, some experts recommend breaking the process into 5Cs:

Capture intelligence about market gaps and organizational potential.
Connect opportunities to capabilities.
Convert ideas and available resources into concepts for products.
Confirm these products are viable in the market.
Conclude by executing a market entry plan for them.

Another way of thinking about it is a series of questions that need to be asked at each stage of the product development process. Market research can help answer them all.

1: What’s the opportunity?

Desk research, analysis of existing customer data and some qualitative investigation can help frame likely areas for innovation. In many cases, an organization will face an internal problem – overcapacity, falling margins, consumer appetites shifting away from existing products – that also frame the need for new products. The output here is an extrapolation of big trends to identify emerging needs, changing behaviors and whitespace for innovation.

2: What ideas might thrive there?

In some organizations, internal R&D will have a ready supply of potential innovations that might be applied to the opportunity. More likely, R&D and marketing teams will benefit from a brief developed from the ‘opportunity’ phase to direct R&D in more concrete areas. This process might include brainstorming inside the organization or more formal ideation sessions with an external research agency. At this point surveys can be harnessed to give more shape to the ideation process. In the search for an iterative new product (rather than a genuine technological innovation) there might be 30 broad ideas that can be tested in quantitative surveys to thin down the field.

3: What concepts deliver on those ideas?

In the next stage, focus groups and market analysis can clarify which concepts ought to progress further by exploring the strengths and weaknesses of each idea. This is also where the innovation and R&D efforts of the business are properly moulded around consumer and market insights – and some iteration takes place to align the two. Note that research here isn’t just among consumers in the core market. Channel partners, consumers and suppliers in adjacent industries are all valuable sources of insight and inspiration. For example, when Kadence worked with an airline to develop new first and business class seats, we looked to bedding experts, audiophiles and high-end restaurant maîtres d’hôtel to shape the concepts.

4: How might those concepts perform in the market?

By this point, an organization should have narrowed its ideas down to a small number of solid concepts. At this stage, a large-scale quantitative survey can be used to identify the concept with most potential to take forward, as well as the size of the potential customer base.

5: What’s the investment case for launch?

The insights gained from market and concept testing will allow numbers to be attached to the product at this point. What might revenues be? What’s the cost to produce the product or service? With research around pricing, what’s the margin likely to be? Does this justify retooling a factory or investment in marketing? This is the “go/no go” point for a new product.

6: What should the final product look like?

Using the research on market potential and consumer attitudes to the new idea, a business can shape decisions on final feature set, ancillary products or services (again, both quant and qual market research will illustrate the need or potential for these), packaging, marketing and pricing.

7: How do we get it out to market?

Research can also highlight optimum product launch strategies, including distribution, adverting and partnerships to make the most of both existing markets and potential follow-ons – whether that’s mass-market adoption for a product designed initially for a niche or early adopters; new demographic segments; or launch into different international markets.

Get regular insights

Keep up to date with the latest insights from our research as well as all our company news in our free monthly newsletter.

The known unknowns for new products

There are broadly two types of business keen to answer these questions. First there’s the radical innovators, the people who come up with brand new ideas and product concepts and want to understand whether they stand a chance in the market. This group are interested in ‘unknown unknowns’, the broader trends in consumer behavior that might hint at acceptance of a brand new idea. We’ll come on to these Steve Jobs types later.

The second, much larger, group understands the innovation funnel in more detail and seeks data to optimize a pipeline of new products. They are interested in ‘known unknowns’ and using the answers to justify, shape and execute a launch.

For this group, the challenge is modelling the potential performance of a new product against a number of variables already visible in the market. These organizations often have a sophisticated process in place to test new ideas and are keen to benchmark any new product in order to validate investment. They will have an algorithm for product development. The more variables they can pin down using market research, the higher the confidence in making those investments.

‘Benchmarking’ in this case might be looking at the performance of products within the target market; or evaluating consumer attitudes to particular features or benefits. This makes it a largely quantitative methodology.

This kind of quantitative approach is often applied with good reason. Standardized questionnaires and clear, consistent methodologies can help ensure that the market research process is more reliable and easier to interpret. And for many larger organizations with a wide portfolio of potential innovations, a fixed investment budget and the need for reliable returns, this rigour can be highly valuable.

But beyond simply looking at the “go / no go” result, it’s important to dig into the reasons why products didn’t pass this hurdle. This can provide valuable insights to inform future development.

Competitor analysis can also reveal opportunities for developing successful product iterations. Research might include:

Rivals’ marketing strategies – what’s their targeting and messaging; what are they missing?
Customer satisfaction with competitor products– where are there discontents that might be satisfied by your product?
Other gaps in the market – such as different price points or localized versions for international consumers.
Other competitor strengths and weaknesses – consider brand halo effects or financial status.
Early-adopter behaviors – in similar markets or using new technologies that might be adapted to your own target markets.

Learning from History: New Coke as a Case Study in New Product Development Market Research

But it’s not always done right. There’s no shortage of case studies of new product launches that didn’t go well. And often that’s not because an organization didn’t do any market research. It’s because they didn’t use it deftly enough.

New Coke is a great example. Coca Cola is an innovative business and wields one of the greatest brands in history. In the 1980s, management decided to rebuild its dominant position with a new formula. Clearly this was a huge decision, and as a market research powerhouse, it took no risks. It spent $4m on development and conducted over 200,000 taste tests across the US to research how consumers would score the new flavor against rival Pepsi. And based on those tests, New Coke was going to be a hit.

But management made a series of errors. In a classic case of confirmation bias, they tended to put more weight behind positive views expressed in focus groups, ignoring those who warned a change would turn them off the brand. They discounted emotional feedback on their brand. And they over-focused on differentiation with Pepsi, which had long marketed itself as the sweeter product.

One big mistake was conducting sip tests instead of researching how consumers would feel drinking a whole can of the sweeter formulation. But narrowing down their research focus – ignoring the context for consumption – they ended up launching a product that turned consumers off the brand altogether.

The error, then, was not failure to conduct market research. It was failure to treat research objectively and apply appropriate methodologies. Management sought justification for their decision – not confidence that it was the right one.

The impact of market research on new product development – giving you the confidence to guide a product launch

The key word here is ‘confidence’. Even iterating an existing product entails risks. Using market research for product development helps reveal and manage that risk – and allows decision-makers to test rigorously against hypotheses for new products, rather than head off down potentially blind alleys.

Note that qualitative research plays a crucial role in helping product developers fine-tune their approach and create innovations more suited to particular audiences. And as the New Coke example shows, qual research can capture the emotional components of product change much better than quantitative analysis might. Every new product launch is a balance between gains and losses for the consumer and understanding that balance is vital.

When it comes to qualitative research, organizations shouldn’t just ask themselves whether to conduct it, but how to conduct it. Whilst central location testing for instance, allows you to ensure the product is experienced in a consistent way during the testing process pandemic lockdowns have obviously accelerated this shift towards at-home testing. New technologies are helping. Augmented reality (AR), for example, is an ideal way to help consumers visualize new products even at the concept stage. Using their mobile phones, they can ‘see’ products in their own home or a work setting, providing valuable depth to qualitative studies at even earlier stages. This is something we’ve piloted with Asahi to test their London Pride packaging and are seeing a number of benefits, such as respondents using AR organically noticing and commenting on small visual details that aren’t picked up by other respondents assessing a 2D concept.

Using market research to guide blue-sky thinking

Our market research for new product development includes testing against quantitative benchmarks to validate each phase of product innovation. Additionally, our qualitative studies in new product development market research focus on testing the emotional reaction to new products, shaping their evolution for market success. A third aspect of market research in new product development is ideation – generating new ideas at the outset.

This is often called ‘ideation’ and it’s an area where market research has played a key role since the birth of the industry – regardless of what Steve Jobs said. He was right that consumers are typically quite poor at predicting what might define or satisfy meet their own future needs. But understanding how R&D and human appetites come together is core to the market research offering.

Take a dairy business, as an example, that’s facing a slow decline in consumption. One solution would be to increase the appeal of organic products. How might they craft a brief to their own product development team?

Working with Kadence, the company use a structured approach to frame where this innovation might gain some traction in the market. Using proven research techniques, they also explored possible options for further innovation. These can be tweaked and repositioned using further research.

This approach can be further optimized if like us, the research agency has an in-house creative team that can quickly visualize concepts based on consumer feedback. We worked with a global beverage brands wanting to relaunch its range to make this happen. Based on focus groups, we were able to redesign the packaging in a matter of hours in a way that capitalized on insights from the research.

One other process to consider: the ideation sprint. Rather than gradually piecing together some R&D, market analysis and internal feedback before gradually building out a new product for consumer testing, this involves getting all the stakeholders into a project group together to develop new ideas within a short timeframe.

Kadence has conducted these sprints with food manufacturers – where that combination of chefs, technicians, marketing experts, salespeople and researchers working in concentrated bursts over a couple of days can see a menu of ideas created, tested with consumers and refined incredibly quickly. And because these sprints are cross-departmental, buy-in for the new product internally is much greater.

Concluding thoughts

Product development is risky even when you’re not launching a category-busting innovation or changing the world. New flavors, revived branding, tweaked feature-sets or version updates can upset existing product performance or result in costly investment in ideas that might not fly.

Far from stifling product development, market research can deliver reassurance and confidence at every stage, helping inform the choice of new products to pursue, their key attributes, how they might be marketed and what contribution they make to a business operationally and financially.

This is an area where Kadence has extensive expertise. Find out more about our product development research services or contact us to discuss how our market research can optimize your new product development brief.