One of the toughest challenges faced by a product design team is concept selection—comparing any number of potential product ideas in order to choose a winner. Traditionally, companies have relied on costly physical prototypes or complicated statistical analysis to test the whims of consumers. Now the Internet introduces a tantalizing option: Virtual prototypes. “It’s a market research tool that has a promising future,” says Stanford Business School marketing professor V. “Seenu” Srinivasan.
In a research study, selected as the best paper published in 2000 by the Journal of Product Innovation Management, Srinivasan tested the ability of Internet-based virtual prototypes to accurately predict consumer choice. He concluded that virtual prototypes provide nearly the same results as physical prototypes. Furthermore, virtual prototypes cost considerably less to build and test than their physical counterparts, so design teams using Internet-based product research can afford to explore a much larger number of concepts. “In short, the Web can help to reduce the uncertainty in a new product introduction by allowing more ideas to be concept tested in parallel,” says Srinivasan, who is the Ernest C. Arbuckle Professor of Marketing and Management Science at the Business School.
Working with research coauthor Ely Dahan, a Stanford Business School PhD who is now an assistant professor of management science at MIT’s Sloan School of Management, Srinivasan pitted the virtual prototype against conventional market research methods including both physical prototypes and non-visual, attribute-only conjoint analysis — a complex statistical comparison of consumer trade-offs between price and other characteristics. Nine concepts for a new portable bicycle pump competed against two existing bike pumps in the tests, which surveyed respondents who were university students screened for bicycle use. Characteristics such as price, time for inflation, size, ease of inflation, and durability were included as product attributes. Both still-picture and animated virtual prototype tests produced market shares that closely mirrored those obtained with the physical products. And, the visual prototypes outperformed the set of predictions produced in the attribute-only conjoint analysis, which failed to capture the aesthetic and usability aspects of the product.
Interestingly, says Srinivasan, the attribute-only conjoint analysis identified the top three products in correct order. However, it predicted market shares for the top three products to be well below those achieved using physical prototypes. This sort of forecasting gap may be filled at least in part by the realistic animations of virtual prototyping. Indeed, a key breakthrough in virtual prototyping that allowed the researchers to conduct their study has been the development of Virtual Reality Markup Language (VRML), which produces high quality color animation within very compact data files that can be easily downloaded by consumer research participants in far flung locations.
Yet virtual prototypes are not perfect. Sometimes there is a disconnect between the results of a physical prototype and a virtual one. In the Dahan-Srinivasan study, the Web-based survey predicted that a bike pump nicknamed Gecko could compete against the two commercially available products and ranked fourth in the market. But when customers handled the real thing, Gecko couldn’t score. It ranked last. One reason may be that the computer renderings of Gecko made it look and feel better in virtual reality. The fit and finish of the rubber material that gave Gecko its green, lizard-like texture and appearance was not of high quality, but that only became obvious when consumers actually touched the prototype.
The crucial question is which product characteristics are most accurately communicated only through a physical prototype? Sensory experiences such as smell, touch and taste have yet to be mastered in a virtual environment. “It remains to be seen which goods are best suited to virtual, visual testing, but we expect that many durable goods categories can be represented accurately using animation and compared using the simulated shopping experience,” says Srinivasan. For unfamiliar products, an educational step could precede the concept tests, say the coauthors. One thing is already clear: The Internet provides a cheap way to test more ideas than ever before.
