Susan Athey: Applying Machine Learning to the Economy

Written

Susan Athey: Applying Machine Learning to the Economy

The Stanford GSB professor talks about her data-driven research and experiences as one of the first “tech economists.”
January 29, 2018
Susan Athey and her chicken, Violet. | Tricia Seibold
When not at Stanford GSB, economist Susan Athey serves on several company boards and raises chickens. | Tricia Seibold

Susan Athey’s pioneering work as a “tech economist” has helped industry and academia alike better understand the constantly shifting digital era.

About Susan Athey

Hometown

Rockville, MD

Education

BA in economics, computer science and math, Duke University PhD in economics, Stanford GSB

Joined Stanford GSB

2013

Research Interests

  • Marketplaces and intermediaries
  • Machine learning
  • Causal inference

Personal Interests

Keeping up with my 3 kids

Athey, a professor of economics at Stanford Graduate School of Business, seeks to understand the impact of marketplaces and digital platforms on the economy, touching disparate fields such as timber auctions, virtual currencies, the news media, and online advertising. By marrying machine-learning techniques with statistical tools to analyze large and novel data sets, she helps answer thorny questions about cause and effect.

How does Google News affect market shares of small and large news outlets? What would happen if a marketplace changed its rules or auction design?

“There are so many opportunities around digitization,” Athey says. “The digital transformation of the economy is going to lead to an explosion of new data and the ability to intervene in the environment, run experiments, and learn iteratively.”

Athey recently spoke with Stanford Business about her experiences at the intersection of academia and industry, what motivates her as a researcher, and what it’s like to help chart a new career path for a fresh generation of economists.

What drew her to the field:

Athey was initially attracted to economics by the ability to use rigorous theoretical and empirical analysis to answer policy questions. “Economics — and in particular, designing marketplaces like search, ride-sharing, and vacation rentals — is an area that allows you to approach important issues that have a lot of impact on people, in settings where answers and mechanisms are often quite subtle,” she says. “So there’s tremendous value in having a good conceptual framework.”

An essential theme in her research:

Researching and designing auction-based marketplaces has been a fixture throughout Athey’s career. She, for example, developed the auction-based pricing system that has been used to price most of the timber in British Columbia for more than a decade and that helped resolve a major trade dispute with the U.S. Athey developed a much broader insight from studying timber auctions: “The rules of the game have an obvious short-term effect on how prices get set and how business gets allocated,” Athey says. “But the impact of the market design on who participates is more important.” In other words, how does the design influence the overall mix of small and large bidders in an auction? Understanding that is more valuable than understanding how they behave once they get there, Athey says.

Athey’s wide-ranging work extends that theme into other contexts. Her work regarding the news industry studied how online aggregators and intermediaries — like Google News and social media — redistribute reader attention away from large news purveyors and toward smaller outlets. “That can have a pretty big effect in the long run in terms of what kinds of news gets created, and how investments are made,” Athey says. “If large news outlets do a lot of the in-depth reporting, then we expect to see less in-depth reporting when the outlets lose market share. Aggregators and intermediaries force news outlets to compete on the basis of catchy headlines and snippets rather than on the basis of a reputation for quality investigation. These same themes come up again and again in all kinds of marketplaces. As an intermediary, you need to think about the effect of the decisions you make on the structure of the market participants, because that ultimately affects the quality of what’s provided and the competitiveness of the market.”

The most significant turning point in her career:

In 2007, Athey took time off from her academic career to serve as consulting chief economist to Microsoft, where she helped design and optimize the auction-based search advertising marketplace. There, she was exposed to machine learning and big-data analytics, an experience that profoundly changed the course of her academic work. “It made me realize that a lot of the statistical techniques we’d used in economics over the past few decades were really not up to the task of working with very large data sets with lots of covariates,” she says. “I thought that the techniques coming out of machine learning could be a very powerful set of tools to bring into economics.”

She also observed that the way that machine learning was being taught at major universities, as well as the way it was being applied at large technology firms, focused mainly on prediction and classification; for example, classifying images as cats or dogs, or making simple predictions like whether a user will click on an advertisement. Not, Athey says, on the greater opportunity to answer questions and pick apart cause and effect in order to drive decisions. “A lot of people who were trained in the machine learning paradigm weren’t taking advantage of what we knew and understood from economics to answer questions like what was the effect of a marketing campaign? What would happen if the firm reduces prices? What was the impact of introducing a new product?”

Athey is now one of a small group of researchers pioneering this new area, importing the latest advances from machine learning into traditional economic approaches but with much larger data sets, dramatically improving the performance of the traditional methods while maintaining the ability to answer questions about cause and effect.

“It’s a small but exploding literature, and I’m really excited about it. I had to completely retool, but I saw it as a unique opportunity to be present at the beginning of an exciting and important literature and help shape the way it is going to evolve.”

Her proudest professional highlight outside of academia:

Athey’s work as Microsoft’s consulting chief economist was, at the time, something of a novelty for a microeconomist. The role contrasted with most other industry economists, Athey says, who traditionally worry about forecasting and other macroeconomic issues.

“What was so new for these tech companies was that they were inventing business models and market designs that had new strategic issues that hadn’t been played out before,” she says. “Whereas one person could set the price of Windows, say, it’s a whole other thing to think about pricing every click on a search engine or every ride on Uber in an efficient way. These multi-sided platforms are built on algorithms and auctions, so the way the business operates is really quite complex. New science and economics are being invented inside of the firms.”

Fast-forward to today, when Amazon has a team of over 100 “tech economists,” and countless other companies model C-level roles after the discipline that Athey helped pioneer. “This is now a career. It’s an opportunity for academics — either former academics who go into industry or people like me who have a foot in each — to have a huge impact on business. It’s a career path that has the potential to start at the lowest levels and become the right-hand person to the CEO,” Athey says. “It was an amazing opportunity to help shape and define that role. Now I spend a lot of time mentoring young people in those roles and helping other companies create similar positions.”

On the companies she admires:

Athey serves on the board of a number of companies that operate in a wide range of industries, including travel (Expedia), finance (Ripple), and dog-sitting (Rover). What is it about those companies that piques her interest as an economist?

For Ripple — whose mission is to move money instantly, 7 days a week, 24 hours a day, around the world — it’s the opportunity to address the frictions in global financial transactions. “Those frictions are basically a huge tax on the global economy, particularly for parts of the world that are not well-connected,” Athey says. “You have people working all week to remit money home, and then 20% of that money gets burned in the remittance process. Or people are trying to do business from Africa but can’t move money in less than 10 days or two weeks. It’s very regressive and very inefficient. So it’s exciting to be part of a company that has a real chance of making a dent in that problem.”

As for the online dog-sitting marketplace Rover, Athey was drawn by the company’s ability to create value for people on each side of the transaction. This service, Athey says, allows pet owners to do things that would have been problematic before, whether it’s simply staying late at work or traveling without having to put their pets in the “doggie jail” of a kennel. On the other side, she says, the pet sitters are passionate about animals. “It just creates enormous value. It’s exciting as an economist to see a service that gives people something that’s so much better than their next alternative, and have the service providers get paid for something that they love,” Athey says.

“They almost didn’t let me on the board because I don’t have a dog,” she laughs. “I have chickens, but they don’t love you the way a dog does.”

For media inquiries, visit the Newsroom.

Explore More

May 18, 2018
Written
2017 Impact Design Immersion Fellow rolls out a plan to make wheelchairs affordable and customizable.
Rachael Wallach's personal experience with wheelchairs caused her to ask why they couldn't be customizable to meet users' varying needs.
May 10, 2018
Written
After being faced with a difficult decision on whether to accept a job or go back to school, Mahere aims to use his Stanford GSB experience to transition into other business sectors.
Tawanda Michael Mahere, MBA ‘19 | Kiefer Hickman
May 7, 2018
Written
It’s all about connections for the winner of the Jerry I. Porras Latino Leadership Award.
José E. Feliciano: If you can make those around you better, you can succeed at anything. | Joe Bradwell