Executives' Words Contain Clues to Deception
Language may be a better predictor of a company's health than accounting reports, says research by David Larcker.
by Robert D. Hof
Just days after Bear Stearns collapsed and was sold for a pittance to J.P. Morgan Chase, rival Lehman Brothers was weathering the economic downturn pretty well, to hear chief financial officer Erin Callan tell it. In a quarterly conference call on March 18, 2008, she assured financial analysts that the company's businesses remained "strong." In fact, Callan used that word 24 times during the call, added "great" 14 times, and piled on "incredibly" 8 times. She used "challenging" 6 times and "tough" just once. "Well, that wasn't too bad," a Wall Street Journal writer blogged at the close of the call. Yet by September, after it became obvious that risky investments masked by accounting gimmicks might sink the company, Lehman had plunged into the largest bankruptcy in U.S. history.
Lehman shareholders, who lost most of their investment, might have had a better chance to divine the company's true plight if they had had the benefit of recent research by David F. Larcker, the James Irvin Miller Professor of Accounting, and Anastasia A. Zakolyukina, a PhD candidate in accounting at the Graduate School of Business. Their research paper, "Detecting Deceptive Discussions in Corporate Conference Calls," essentially showed that it's possible to figure out when executives are likely to be lying about how their company is doing. In particular, they found that companies whose chief executives and chief financial officers used words and other linguistic patterns associated with deception later restated their earnings by a material amount.
More than three years in the making, the study was published last summer for Stanford's Rock Center for Corporate Governance. Larcker, right, a Rock Center senior faculty member who also directs Stanford's Corporate Governance Research Program, and Zakolyukina analyzed the transcripts of more than 16,000 U.S. quarterly earnings conference calls between 2003 and 2007 — specifically, the less rehearsed question-and-answer portion between analysts and executives. The research got a lot of media attention partly because it centered on material that almost any investor can obtain easily.
But what's most interesting to investors is that language patterns in conference calls appear to be a better predictor of negative financial surprises down the road than the more traditional accounting-based method. The latter analyzes differences between a company's reported earnings, which can involve executive discretion on various measures, and its actual cash flows. That model predicts financial restatements about 3% better than a random guess, while Larcker and Zakolyukina's model does 4% to 6% better than random.
That may not sound like much, but it's potentially a significant improvement for analysts and portfolio managers responsible for managing hundreds of billions of dollars in investments. "Ultimately, you could imagine using this to weed out big losers in your portfolio" before they go south, Larcker says.
Larcker and Zakolyukina, left, began their research by identifying words and categories of words that psychologists and linguists have shown to be related to deception. For example, CEOs trying to deceive are more likely to display extreme positive emotions, using words such as "fantastic" instead of merely "good" or "solid," and less likely to show extreme negative emotions.
They also were less likely to say "I" or "we," which imply personal ownership of a situation, and more prone to employ impersonal pronouns or phrases such as "the company" or "the team." They often made references to general knowledge with phrases such as "you know" and "everybody agrees." And they tended to make relatively short statements with few hesitation words, presumably because they had planned some answers in advance and preferred to move on quickly from their deceit.
To those well-established word categories, the researchers added several of their own. They looked at terms such as "shareholder value" and "value creation," phrases commonly used by corporate executives. And they found that deceptive executives were less likely to use them. The likely reason: They may fear that statements appearing to promise shareholder value creation might later be used against them in lawsuits.
The researchers created electronic documents from transcriptions of each conference call's Q&A session, obtained from FactSet Research Systems Inc., with answers by both the CEO and the CFO. Using serious restatements of income in the same quarters identified by the corporate governance analysis firm Glass, Lewis & Co., they classified each of those documents as truthful or deceptive. Then, in each document, they counted the words and phrases from their list of deceptive language to determine any correlation.
Using Stanford's computer clusters to test the model, they ran the resulting mathematical model for determining deceptive conference calls on a different set of conference calls. For a direct comparison, they also crunched the numbers on each company's quarterly financial data to test the traditional accounting model of determining potential restatements. The result: Larcker and Zakolyukina's method was about twice as good as the accounting model at predicting problems down the road.
The research has plenty of limitations, Larcker says, which is why the linguistic method correctly classified a random sample of conference call narratives as truthful or deceptive between 50% and 65% of the time — not, say, a more definitive 80% to 90%. For one thing, the study assumes that CEOs and CFOs know something is amiss in their companies at the time of the conference call, though it's quite possible they did not know and thus weren't being deceptive. Still, Zakolyukina notes, she and Larcker deliberately chose to look at large restatements that also were associated with serious accounting problems such as the disclosure of a "material weakness," a change of an auditor, or a late financial filing. In those cases, it's more likely executives knew they at least weren't telling the whole truth.
What's more, while the research assumes that the Q&A portion of conference calls is largely unrehearsed, that's not always the case. Individual executives also may have their own unique speaking patterns that match deceptive language even when they're not actually trying to lie. On the flip side, other executives who are lying may have speech patterns that don't fit into the usual pattern of deception.
For all those reasons, not everyone is convinced the research will be useful. One skeptic is Joe Navarro, a former FBI expert in behavioral analysis and author of What Every BODY Is Saying: An Ex-FBI Agent's Guide to Speed-Reading People. On Psychology Today's blog, he wrote that CEOs are inherently cheerleaders, so their extremely positive language has dubious value for predicting actual lies. Even if they do appear to be lying, he says it's not necessarily clear that they're lying specifically about the company's financial condition.
Larcker concedes that the research is just a start and that others will need to take it further to turn it into a usable tool, though Boston-based Business Intelligence Advisors Inc. appears to use similar methods already. "It's a little early," Larcker says. "But we think it's worth pursuing and refining." Nonetheless, he says he has received many calls and emails from investors, analysts, and portfolio managers, some of whom sarcastically thanked him for exposing their own methods.
"It's some of the best research to detect lying CEOs ever done," says former hedge fund manager and private investor Cory Johnson, now a journalist with Bloomberg TV. "He's found an empirical way to find the cockroaches."
It's not the first time Larcker's work has challenged conventional wisdom. His past research has flouted the assumptions of corporate governance ratings firms, company board members, and lawmakers. For a study released in June 2010, for instance, he joined with Stephen A. Miles, vice chairman of the executive search firm Heidrick & Struggles, to survey 140 CEOs and corporate board members on how well they planned for the time their companies would need a new CEO. Most directors thought they did a good job at succession planning, even asking Miles and Larcker whether the survey was a joke. But it turned out that more than half of companies surveyed could not immediately name a successor to their current CEO. (See related story.)
Larcker raised perhaps the biggest ruckus with a study released in September 2009 that examined the correlation between a company's rating from governance rating firms such as RiskMetrics Group Inc. and its financial performance. Larcker, with coauthors Robert Daines, the Pritzker Professor of Law and Business at Stanford Law School, and Ian Gow, PhD '09, flatly found little correlation at all, especially for RiskMetrics' ratings. Earlier this year, that firm dropped its previous ratings system and instituted a new one. Larcker and governance experts such as James J. Hanks Jr. of the corporate law firm Venable LLP say it's not much different, so it's tough to tell whether Larcker's research forced the change. RiskMetrics, whose acquisition by investment analysis firm MSCI Inc. was completed in June, did not return several emails and phone calls.
Former and current students and colleagues invariably cite Larcker's independent streak. "He digs into the data," says David Chun, CEO of executive compensation data provider Equilar of Redwood City and a student of Larcker's during the professor's 20 years at the Wharton School. "He definitely takes a more challenging view of the consensus."
Joseph A. Grundfest, the W.A. Franke Professor of Law and Business at Stanford Law School, agrees: "Dave has a terrific sense of what's important in the real world and then brings a very rigorous empirical approach to testing ideas that often pass for ‘common wisdom' to demonstrate that they might not actually be that wise."
Larcker isn't the usual picture of an academic. He repairs and rides Harley-Davidson motorcycles, a passion apparent in the black Harley T-shirt, cycle boots, and jeans he often wears. On his office desk is a lava lamp with black goo issuing from a small canister labeled Genuine Harley-Davidson Oil, and prints of classic Harley models are tacked on one wall. The other wall is crammed with books such as Logistic Regression and Corporate Financial Disclosure 1900–1933 and comics legend Gary Larson's The Far Side Gallery.
Larcker's research is one big reason Stanford's corporate governance work has a sterling brand, Heidrick's Miles says. Together, the Rock Center and the Corporate Governance Research Program are "seen as one of the tier-one think tanks in corporate governance," Miles says.
That reputation may be further burnished later this year with the publication of a book, tentatively titled Governance Matters. Larcker didn't intend to become an author. Written with governance program case writer Brian Tayan, MBA '03, the book will be based on a collection of case studies he and Tayan wrote from Larcker's research. Until Larcker came to Stanford in 2005, he hadn't written a single case study. But he realized that if he didn't, he would have no material to teach his classes early next year — a sign of his pioneering work in corporate governance.