|
|
 |
 |
Lessons for Business from Olympics Judging
February, 2003 by Eric Zitzewitz Forcing decision makers to boil their opinions to a single vote can encourage bias and vote trading, says a researcher who studied Olympic winter sport judging. When designing a group decision-making process for potentially biased managers, intuition tells us that throwing out extreme opinions neutralizes the most severe biases among the group’s members and leads to better decisions. Surprisingly, new research suggests our intuition may be wrong. In a corporate setting, empirical data on the impact of biases in group decision making is hard to come by. Luckily, a parallel setting exists that offers a rich dataset of biased decision makers coming together to attempt to reach an impartial decision: winter sports judging in the 2002 Olympics. Consider figure skating. As we’ve known since long before the 2002 Olympic vote-trading scandal, figure skating judges are nationalistically biased. As a response to these biases, figure skating under-weights extreme opinions more than any other judged Olympic sport. Judges’ scores are compressed into an ordinal ranking of skaters, and then the winner is determined by who is ranked higher on a majority of scorecards. If four judges think the Canadian couple skated a lot better than the Russian couple but five judges think they skated a little worse, the Russians win. It sounds like a good idea to throw out extreme scores when judges may be biased, but this actually makes things worse. Throwing out extreme scores throws away a lot of information. Using techniques I developed to study stock analyst forecasts, I find that extreme opinions have information, noise, and bias in the same proportions as less extreme opinions. In short, those extreme opinions are not as biased as we think. The same is true for analysts. When an analyst says something very different from the pack, listen to her! Also, turning opinions into votes actually can make vote trading easier. It’s easier to agree on "You vote for my skater" than "You add 0.1 point to whatever you would normally give my skater." I find evidence of vote trading throughout the figure skating judging data, and the judging blocs are exactly what you’d expect from the 2002 scandal: The Cold War countries of Russia, Ukraine, and Poland aligned with France on one side; the United States, Canada, Italy, and Germany on the other. The other judged winter sports–ski jumping, moguls, aerials, and snowboarding–have much less-biased results and no evidence of vote trading. And this gives us our second lesson: Career concerns matter. In figure skating, national federations choose the judges, and, not surprisingly, they select for the Olympics the judges with the most nationalistically biased track records. Think of Yuri Balkov, who was caught on tape (successfully!) fixing the 1998 Olympic ice dancing competition and was nonetheless chosen by Ukraine for Salt Lake City in 2002. In the other sports studied, the International Skiing Federation selects the least biased judges. Ski jumping judges still have some nationalistic biases–they hide them in minor events but express them in the Olympics. But rather than trading votes, other judges compensate for each other’s biases, so an unrepresented athlete is not at a disadvantage. So what does this teach us in the corporate world? Suppose the promotion committee of a professional services firm is choosing a new partner. Or a company’s division heads must decide among several projects to greenlight and devote resources. In both cases, the chair or group leader must contend with the inherent favoritism of the partner who is most familiar with the candidate or the division head who would reap the most benefit from the approval of a particular project. Olympic judging is most similar to a committee that allows biased decision makers to participate rather than recuse themselves. Organizers do not adjust for known biases, but as observed in ski jumping, sometimes other judges do. Also, organizers can use career concerns to create incentives for limiting biases. An important departure from sports judging is that most organizations do not insist on simultaneous voting, preferring instead to reach consensus and resort to formal voting only when opinions become deadlocked. Even so, I offer three observations from my research: • Value fairness. To the extent that organizations can cultivate a taste for fairness among decision makers, nearly unbiased outcomes can be achieved. • Be aware of career concerns. If managers are accountable to someone who cares primarily about the quality of the decision, rather than which decision is taken, you will get better decisions. Part of the role of a strong committee chair is to make sure that members are rewarded for providing un-biased information and act as a counterweight to interested parties within the organization. • Recognize the cost of opinion truncation. Contrary to intuition, truncating opinions into votes uses information inefficiently, with the additional likelihood of encouraging bias and vote trading. This is why voting should be a last resort in group decision making. In a well-functioning committee, extreme opinions are respected and given extra weight, and members are selective about when they give them. If members are not selective enough, though, committees can fall into a situation where every opinion is extreme, and the only way to make decisions is to count votes. Keeping groups small can help, but the committee chair is important as well. Heed these lessons from Olympic winter sports judging, and no one will get clubbed in the kneecap. Eric Zitzewitz is assistant professor of strategic management at the Business School. |
|
|
|
|
 |
|
|
|
Lessons
for Business from Olympics Judging
