Does the Hot Hand Really Exist? “Jeopardy!” Has Some Answers.

Back in 2006, Sridhar Narayanan got interested in casinos. Specifically, he wanted to know how gamblers were influenced by marketing and whether there was any truth to some of their common beliefs, such as the “hot hand” effect.

A hot hand is when someone experiences a state of elevated performance in a competitive situation where there are repeated actions, like playing cards or shooting craps. Narayanan discovered that while many gamblers fervently believe in the hot hand, in situations where the outcome is determined entirely by chance — like a slot machine — it doesn’t exist.

“It’s not rational,” says Narayanan, a professor of marketing at Stanford Graduate School of Business. “But gambling itself is not rational.”

But he couldn’t dismiss the phenomenon altogether. When skill comes into play in activities like throwing darts or making layups, the existence of the hot hand is more ambiguous. Narayanan came back to the topic a few years ago when he was paired with PhD student Tony Kukavica for an independent study.

Along with a shared interest in studying behavioral phenomena, Narayanan and Kukavica discovered they also had a mutual love of Jeopardy! Narayanan, in fact, watches it religiously with his children every day. Kukavica had also been an avid fan of the show since childhood.

The researchers assembled data from an online archive with play-by-play details of 230,000 questions from 3,900 episodes dating back to the early 2000s. (The archive was originally created by superfans armed with VCRs, though Narayanan suspects the task has since been delegated to AI. It is mostly used by contestants prepping for the show, though researchers have found it useful as well.) Narayanan and Kukavica then began brainstorming ways they could use it in an experiment.

“I’ve always felt that game shows are underrated in terms of their ability to help us study all kinds of questions about how people behave under risk and uncertainty,” Kukavica explains. He had read Narayanan’s casino paper, and over the course of their conversations, the idea of studying the hot hand in Jeopardy! emerged. They recently wrote up their findings in a paper, “Rational and Irrational Belief in the Hot Hand: Evidence from Jeopardy!”

As they read through previous literature on the hot hand, Kukavica and Narayanan noted the many ways other researchers had tested for both the existence of the phenomenon as well as people’s belief in its existence. But there had been minimal formal investigation of these issues simultaneously — specifically the question of whether people’s beliefs were aligned with reality or whether they were biased.

There were a couple of additional difficulties. In a lab experiment, there’s not enough at stake to recreate the excitement (or desperation) of a real-life hot hand situation. And the real world presents its own complications: Athletes, for example, can adjust their game to thwart an opponent who’s on a hot streak. So far, few researchers had found an effective way to study the hot hand effect in a real-world setting with meaningful consequences. And the researchers who had studied the hot hand only looked at it in terms of physical rather than cognitive tasks. Consequently, the initial consensus dismissed the hot hand phenomenon as a fallacy.

In a lot of ways, Jeopardy! was an ideal setting to test for the effect. “It’s a game that involves some skill, concentration, confidence — a lot of mechanisms through which it’s possible that there’s a hot hand, just like in sports,” Narayanan explains. In Jeopardy!, it’s possible for a player to put the kibosh on another’s streak by being faster on the buzzer. But the game also has a unique feature that made it ideal for Narayanan and Kukavica’s purposes: the Daily Double.

Betting on Yourself

Daily Doubles are hidden on the game board more or less at random. When players who have control of the board stumble across one, they get to choose how much of their current winnings to wager on getting the next answer correct. “That wager gives us a way to measure a player’s belief in the hot hand along with the existence of the effect,” Narayanan says.

It would be impossible to track down contestants and ask them what they were thinking when they made their Daily Double wagers. But in the dataset of questions and responses, Narayanan and Kukavica had a record of the contestants’ actions and the results. They could determine if a contestant was in a “hot” state by seeing how many questions they had answered correctly before landing on the Daily Double, among a number of other possible measures. They could then test for how much impact being hot had on a wager, and also whether any change in the wager was consistent with the player’s actual performance.

Jeopardy! contestants bet $100 to $500 more on Daily Doubles when they were on a winning streak, suggesting that they do believe — consciously or subconsciously — in a hot hand effect. The data backed them up, to a point: Contestants were 1% to 8% more likely to answer questions correctly when they were on a hot streak.

There was a real hot hand effect, though its magnitude was small. And the effect was not nearly as profound as contestants seemed to believe. The researchers devised a clever econometric way to measure the degree to which increased bets after a winning streak were consistent with the actual extent of the hot hand effect. Only about 9% to 18% of the increased bet could be explained by the true effect; the remainder was the extent to which contestants were overreacting to the effect.

Mind Games

In addition to contestants’ winnings, the Jeopardy! dataset also contained information about their professions. Kukavica and Narayanan noticed that players whose jobs required less quantitative or analytical skill tended to have a stronger belief in the hot hand. Yet even software engineers, economists, and others who were highly trained in analytical reasoning overestimated the hot hand effect. “That was surprising,” Narayanan says.

The results provide new insights into the psychology behind the hot hand. “A hot hand can be caused by things like focus and concentration and also confidence,” Narayanan says. “As you get more confident, you might actually do better, and you might trust yourself better. So we looked at whether things like focus and concentration and lack of fatigue might cause you to have an elevated performance versus confidence.”

Jeopardy! has several short ad breaks between rounds. These breaks, Narayanan explains, allow contestants to rest and restore their levels of focus and concentration. Their confidence, however, should remain unaffected. He and Kukavica found that the hot hand effect fades after commercial breaks. “This gives us an indirect indication that it’s perhaps not confidence but focus and concentration that cause the effect to even exist,” he says.

These findings have implications beyond America’s most popular game show. Hot streaks don’t just happen in games, after all. As Narayanan puts it, “There are many professions where you make judgment calls based on how well you think you’ve been making decisions.” Think stock traders, doctors, or air-traffic controllers.

Some athletes claim to know when they’re in a hot state. But there hasn’t been much study of what actually generates that feeling, Kukavica says. “That’s starting to get into research questions that are more biological, neurological, or physiological in nature.”

Both researchers are interested in studying these ideas more closely. “We indirectly examined whether focus and concentration versus confidence led to these effects, but would like to more directly examine the underlying mechanisms for hot hand effects, and beliefs in them,” Narayanan says. “We would also like to determine what kind of training can help overcome these biases.”

Let’s take further study for $800, Ken.