Why Understanding Your Market Requires More Than Knowing Your Business.

Boston Consulting Group recently published a finding that should make business leaders uncomfortable. In a survey of 645 CEOs, 87% were confident their employees trusted their leadership. Yet when BCG asked 500 employees about their confidence in their CEO's leadership, judgment, and vision, only 46% expressed the same level of confidence.

The size of that perception gap is striking, especially given that the research concerns people on opposite sides of the same employer-employee relationship. If executives can misjudge the confidence of people working within their own organizations, how accurately can they assess the expectations and behavior of customers they rarely meet?

The disconnect is hardly confined to the workplace. PwC's 2025 Customer Experience Survey found that 89% of executives believed customer loyalty had increased, compared with just 39% of consumers who reported feeling more loyal to brands. Different research, different audiences, but a similarly striking disparity between what businesses believe and what the people they depend on actually report.

Executives know their businesses intimately. They understand the products, the financial pressures, the competitive landscape, and the decisions behind their strategies. But that depth of internal knowledge doesn't necessarily translate into an equally accurate understanding of the people buying their products or choosing their competitors.

And that's where I think businesses need to question their own confidence. Familiarity has a way of turning assumptions into accepted truths, sometimes long after the market has moved on.

The same disconnect exists in the marketplace

It's tempting to assume that sales figures, customer retention rates, and years of category experience provide a reliable picture of how consumers feel about a brand. They certainly tell us something, but they don't always tell us why people behave as they do, what alternatives they're considering, or how easily their loyalty might shift.

Take customer retention. A business might interpret repeat purchases as evidence of satisfaction and loyalty, when customers may simply be staying because switching is inconvenient, alternatives are limited or the price remains acceptable. The commercial outcome looks the same, but the underlying relationship is very different. One suggests a brand with customers who actively want to stay. The other suggests a business that may be more vulnerable to competition than its performance figures imply.

The implications extend well beyond customer retention. Businesses make decisions worth millions based on what they think consumers want, often drawing on years of sales history, customer feedback and market performance. The figures may support a convincing business case without revealing how consumer expectations are changing. A profitable product line, for example, might conceal growing demand for something the company doesn't yet offer. By the time those changing expectations show up in declining sales or lost market share, competitors may already have responded.

Why experience can become a blind spot

There is a well-documented psychological phenomenon known as the false consensus effect, where people tend to overestimate how widely others share their own attitudes, preferences, and beliefs.

Inside a brand, this creates an interesting problem. The people responsible for understanding customers are also consumers themselves, with their own purchasing habits, expectations and opinions about what makes a product or service appealing.

Research published in the Journal of Marketing Research by Walter Herzog, Johannes Hattula and Darren Dahl examined this issue across six studies involving 714 marketing executives. The researchers found that marketing professionals were susceptible to projecting their personal preferences onto consumers. More concerning was their tendency to favor research findings that supported those preferences while discounting evidence that challenged them.

Consider what this means for commercial decision-making. A marketing director who personally values sustainability might overestimate its importance in customers' purchase decisions. A product team enthusiastic about a new feature might assume consumers will share that enthusiasm. Neither needs to consciously manipulate the evidence. The bias can influence which questions are asked, which findings receive attention and ultimately which recommendations reach the boardroom.

What's particularly interesting about the research is that access to consumer data doesn't automatically correct the problem. If decision-makers selectively trust the findings that agree with their existing views, collecting more information may do little to improve the quality of their decisions.

The concern isn't a lack of consumer knowledge among experienced executives. It's how easily personal perspectives can shape the interpretation of that knowledge, even when credible research is available.

When research reinforces what businesses already believe

One of the more uncomfortable realities of market research is that commissioning a study doesn't automatically mean a business is prepared to act on its findings. By the time a research brief is written, there may already be considerable investment in a particular strategy, product or market opportunity. Budgets have been allocated, senior executives have expressed support, and product teams have spent months developing something they want to succeed.

That can influence the research long before anyone sees a set of results. A company preparing to launch a product might ask which features consumers find most appealing, without first establishing whether there is sufficient demand for the product itself. A brand considering international expansion might concentrate on purchase intent while giving less attention to the cultural, practical or competitive barriers that could prevent consumers from acting on that interest.

These aren't necessarily examples of poor research. The questions may be entirely legitimate, and the findings statistically sound. What concerns me is how easily a study can deliver a convincing answer while leaving the more consequential question unanswered.

And even when research uncovers something unexpected, getting an organization to act on it can be another challenge entirely. An inconvenient finding can be explained away as an anomaly, attributed to the wrong audience, or set aside because it conflicts with what previous studies suggested. Sometimes those objections are justified. Sometimes they reflect how difficult it is to abandon an idea in which an organization has invested considerable time, money, and reputation.

For anyone commissioning or interpreting research, I think that's where the real responsibility lies. We should be as interested in the evidence that threatens a business case as we are in the findings that support it. Otherwise, research risks becoming an exercise in building confidence around a decision that has effectively already been made.

The value of a perspective from outside

This is where independent market research can make a meaningful difference. An external research partner brings something that can be difficult to achieve from within an organization: the freedom to examine a commercial problem without being invested in how the business has traditionally understood it.

An experienced research partner should be willing to challenge the brief itself. Is the company investigating the right problem? Are the proposed audiences representative of the market opportunity, or simply the customers the business already knows? Does the research need to explore demand for a new product, or should it first establish whether consumers have a problem worth solving? These conversations can fundamentally change the direction of a project before a single interview takes place.

I also think businesses underestimate the value of experience gained outside their own organization. Researchers working across industries, categories, and international markets encounter different consumer behaviors, emerging expectations, and competitive dynamics. That exposure can help identify opportunities or risks that aren't immediately apparent to teams focused on the day-to-day realities of their own brand.

Of course, independence comes with responsibilities. A good external research partner should be prepared to defend its methodology, explain the limitations of its findings and challenge interpretations that the evidence cannot support. That includes having uncomfortable conversations when the results call into question a strategy supported by senior leadership.

For me, the real value of an external research partner is the quality of thinking they bring to the relationship. A well-executed study matters, but so does the willingness to ask difficult questions, draw on experience beyond the client's business and offer an informed perspective that improves commercial decisions.

The commercial cost of misplaced confidence

The consequences of misreading a market aren't always immediately obvious. A product may launch to respectable initial sales before demand begins to flatten. A pricing strategy might improve margins in the short term while gradually encouraging customers to consider alternatives. An international expansion can appear promising on paper, only to encounter consumer attitudes or purchasing habits that the original business case failed to account for.

One of the most instructive examples remains Coca-Cola's introduction of New Coke in 1985. The company had conducted taste tests involving nearly 200,000 consumers, with 55% preferring the new formulation over the original in blind comparisons. Yet the decision to replace the original Coke triggered an extraordinary consumer backlash, forcing its return just 79 days later. The research had identified a genuine taste preference, but that was only part of the purchasing relationship. Consumers' emotional attachment to the original product proved far more consequential than the company had anticipated.

Coca-Cola's mistake wasn't a lack of research. It was treating the answer to one question as evidence for a much bigger decision. A preference for a sweeter drink in a taste test did not necessarily translate into a desire to see a familiar and much-loved product disappear. The company had measured which formulation consumers preferred without fully accounting for what replacing the original would mean to them.

Forty years later, the lesson remains relevant, particularly when businesses interpret early commercial performance as proof that a strategy is working. Strong initial sales may reflect promotional investment or curiosity rather than sustained demand. The figures can be accurate, the analysis technically sound and the conclusions still wrong.

The less visible cost, however, is often the opportunity a business never pursues. Companies can measure the financial consequences of unsuccessful launches or underperforming campaigns. It's far harder to quantify the growth lost because an emerging consumer need went unnoticed or a competitor recognized a shift in purchasing behavior first.

This is where the quality of the original research question becomes commercially significant. Understanding demand, the barriers to adoption and the circumstances in which consumers might change their behavior can influence whether a business commits further investment, changes direction or identifies an opportunity it hadn't previously considered.

Its commercial value lies as much in knowing when to reconsider an investment as it does in identifying the next opportunity for growth.

Knowing when confidence is justified

Leaders will always need to make decisions with imperfect information. The question is whether they know which uncertainties matter most, and how much confidence the available evidence really warrants.

For me, the real test isn't whether a business invests in research. It's what happens when the findings challenge a strategy that already has the backing of the boardroom. It's relatively easy to embrace evidence that supports a decision. Far harder is acknowledging that an investment may need rethinking, or that an opportunity everyone was excited about isn't quite what it appeared to be.

I believe we sometimes underestimate the value of research that tells us we're right. That reassurance can give businesses the confidence to invest, expand or move ahead of competitors. But I've always thought the findings that make people uncomfortable deserve particular attention. They can expose risks that might otherwise go unnoticed, reveal opportunities outside the original brief or prompt a business to reconsider where its resources will deliver the greatest return.

The willingness to change your mind in response to credible evidence isn't a weakness in leadership. It's a sign of good judgment. Ultimately, one of the greatest contributions market research can make is helping a business change its mind before the market forces it to.

How confident are you that your business sees the market as it really is?

At Kadence International, we help businesses look beyond internal assumptions to understand what customers think, what drives their decisions and where opportunities may be hiding in plain sight. Through independent, evidence-led research across global markets, we bring the perspective and commercial insight needed to challenge established thinking and make better-informed decisions.

Talk to Kadence about what your business might be missing.