If you’re interested in business, you’ve probably heard the “know your customer” mantra more times than you’d care to remember. Major enterprises spend billions of dollars a year on business intelligence software used to mine customer information from corporate databases, and on painstaking market research and consumer surveys.
Every year, thousands of shoppers are buttonholed as they enter stores or call customer service centers and then are quizzed about their shopping experience.
There is no doubt that understanding the needs and wants of customers in an ever-more competitive economic environment is critical. But in a series of studies, Itamar Simonson of Stanford GSB has demonstrated that information gleaned from some widely used types of customer surveys can be misleading and even counterproductive.
“People in business greatly exaggerated the value of matching the consumer‘s presumed preferences and giving them what they want,” he says. Indeed, “measuring consumer expectations can backfire by producing misleading information.” Even worse, some popular survey methods actually put consumers in a negative frame of mind, hardly the results the companies paying for the survey had envisioned.
Simonson’s work, done in collaboration with Professor Chezy Ofir, the academic director of the Davidson Research Center at Hebrew University, strongly indicates that priming consumers to anticipate an upcoming interview leads to unexpected, and generally negative, results.
“The mere fact that consumers are told in advance to form evaluations leads them to believe that they are expected to focus on negative aspects, and they act on that assumption,” Simonson writes in “Customer Compliance with Presumed Market Research Goals,” a working paper currently under peer review. It was co-written by Simonson, the Sebastian S. Kresge Professor of Marketing; Ofir of Hebrew University; and Song-Oh Yoon of Singapore Management University. It marks a distinct departure from prior academic work that concluded polling consumers just prior to shopping or encouraging buyers to compare products on the shelves was likely to be useful.
Simonson and Ofir’s latest paper builds on seven years of studies that included interviews with hundreds of consumers in Israel and the United States. The results, he says, “were robust. It was not a close call.”
Businesses have learned to listen to consumers, but taking their input without a hefty dose of skepticism can be a mistake, says Simonson. Most consumers want to be honest, and even helpful, when they participate in a survey. But to many consumers helpful means being (constructively) critical or at least offering suggestions for improvement, as opposed to simply stating their opinions—good or bad. Moreover, taking a survey and giving a thoughtful answer to complex questions about why they behaved in a certain way isn’t easy. “Most consumers don’t have real insight into their own motivations,” he says.
An earlier paper by Ofir and Simonson exemplifies the researchers’ methodology. To prevent the survey takers from inadvertently tipping off the shoppers, even the surveyors were not told the real object of the study as they questioned customers of a major computer hardware and software company. The participants, representatives of companies who had called for service, were divided into four groups:
Those who were told that a technician would service their problems and that they would subsequently be asked questions about the tech’s performance and attitude. Those who were asked what they expected; for example, how long they would have to wait for the technician. This group was not told there would be an exit interview. Those who were told they would be surveyed and were also asked to state their expectations going in. The control group that was told nothing, but was interviewed after their business was finished.
The results were telling: The more shoppers were asked in advance, the more negative they became. Most negative of all were the consumers who knew they would be quizzed and who were asked in advance how well they thought their problems would be handled.
Simonson and Utpal Dholakia of Rice University have also discovered pitfalls in asking customers to compare two or more brands. “Telling consumers to make comparisons, which is a practice that marketers use a lot, can be very uncertain because it can change the behavior of consumers in very fundamental ways,” said Simonson.
Consumers who decided on their own to compare products behaved as you might expect, looking for the best buy. But consumers who were told to compare became unusually cautious. “The mere fact that we had asked them to make a comparison caused them to fear that they were being tricked in some way,” said Simonson.
A seemingly unbiased method of studying consumer preference can nevertheless change behavior. And that calls into question the accuracy—and usefulness—of the opinion survey.
Ofir and Simonson delved deeper in a paper published in early 2007 in the prestigious Journal of Marketing Research. They deliberately chose a supermarket with a reputation for poor service, asking exiting customers what they remembered about the visit. Once again, those who had been asked their expectations in advance left with a more negative impression than those who had not been tipped off.
Another study conducted in Israel, where pharmacies have a poor reputation for service and variety, asked consumers to evaluate a new pharmacy based on a newspaper review of the chain’s first store. The fictitious review was constructed by the researchers.
After reading the review, participants were given four minutes to write down as many characteristics of the store as they could remember and rate them as positive or negative.
Once again, those who voiced expectations in advance were significantly more negative and even rated the same store features such as cleanliness or the professionalism of employees more negatively than the control group.
Although Simonson called his overall research results “robust” he cautions that consumer surveys may sometimes prompt the participants to do things they would never do in real life. After all, how many of us go into a store and then rate it on 10 characteristics?
Still, it’s clear that the costs of surveys taken in advance of shopping exceed the benefits of these surveys by introducing an unacceptable level of bias and producing negative evaluations. Future research, says Simonson, is needed to find ways to remove that bias.
