Browse or search publications from faculty affiliated with the lab.
The Surrogate Index: Combining Short-Term Proxies to Estimate Long-Term Treatment Effects More Rapidly and Precisely
A common challenge in estimating the long-term impacts of treatments (e.g., job training programs) is that the outcomes of interest (e.g., lifetime earnings) are observed with a long delay. We address this problem by combining several short-term…
Presidential Address: The Economist as Designer in the Innovation Process for Socially Impactful Digital Products
This paper provides an economic perspective on data-driven innovation in digital products, focusing on the role of complex experiments in measuring and improving social impact. The discussion highlights how tools and insights from economics…
Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewal
In many settings, interventions may be more effective for some individuals than others, so that targeting interventions may be beneficial. We analyze the value of targeting in the context of a large-scale field experiment with over 53,000 college…
Choosing the “Right” Default Donation Amounts for Each Donor to Balance Multiple Fundraising Objectives
This report describes insights gleaned from the Data Fellows collaboration between PayPal and the Golub Capital Social Impact Lab at Stanford University’s Graduate School of Business. By embedding researchers in PayPal’s charitable giving team,…
Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects
There are a number of available methods for selecting whom to prioritize for treatment, including ones based on treatment effect estimation, risk scoring, and hand-crafted rules. We propose rank-weighted average treatment effect (RATE) metrics as…
Federated Offline Policy Learning
We consider the problem of learning personalized decision policies from observational bandit feedback data across multiple heterogeneous data sources. In our approach, we introduce a novel regret analysis that establishes finite-sample upper…
Qini Curves for Multi-Armed Treatment Rules
Qini curves have emerged as an attractive and popular approach for evaluating the benefit of data-driven targeting rules for treatment allocation. We propose a generalization of the Qini curve to multiple costly treatment arms that quantifies the…
Qini Curves for Multi-Armed Treatment Rules
Qini curves have emerged as an attractive and popular approach for evaluating the benefit of data-driven targeting rules for treatment allocation. We propose a generalization of the Qini curve to multiple costly treatment arms that quantifies the…
Policy Learning with Adaptively Collected Data
In a wide variety of applications, including healthcare, bidding in first price auctions, digital recommendations, and online education, it can be beneficial to learn a policy that assigns treatments to individuals based on their characteristics…
Data-driven Error Estimation: Upper Bounding Multiple Errors with No Technical Debt
We formulate the problem of constructing multiple simultaneously valid confidence intervals (CIs) as estimating a high probability upper bound on the maximum error for a class/set of estimate-estimand-error tuples, and refer to this as the error…
Towards Costless Model Selection in Contextual Bandits: A Bias-Variance Perspective
Model selection in supervised learning provides costless guarantees as if the model that best balances bias and variance was known a priori. We study the feasibility of similar guarantees for cumulative regret minimization in the stochastic…
The Heterogeneous Impact of Changes in Default Gift Amounts on Fundraising
When choosing whether and how much to donate, potential donors often observe a set of default donation amounts known as an “ask string.” In an experiment with more than 400,000 PayPal users, we replace a relatively unused donation amount ($75) on…
Battling the Coronavirus ‘Infodemic’ among Social Media Users in Kenya and Nigeria
How can we induce social media users to be discerning when sharing information during a pandemic? An experiment on Facebook Messenger with users from Kenya (n = 7,498) and Nigeria (n = 7,794) tested interventions designed to…
Digital Interventions and Habit Formation in Educational Technology
We evaluate a contest-based intervention intended to increase the usage of an educational app that helps children in India learn to read English. The evaluation included approximately 10,000 children, of whom about half were randomly selected to…
Impact Matters for Giving at Checkout
We conducted two experiments on PayPal’s Give at Checkout feature to learn about the effect of 1) information about charity outcomes on donations, and 2) exposure to these point-of-sale microgiving requests on subsequent giving. In this “…
Optimal Experimental Design for Staggered Rollouts
In this paper, we study the design and analysis of experiments conducted on a set of units over multiple time periods where the starting time of the treatment may vary by unit. The design problem involves selecting an initial treatment time for…
Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects
There are a number of available methods for selecting whom to prioritize for treatment, including ones based on treatment effect estimation, risk scoring, and handcrafted rules. We propose rank-weighted average treatment effect (RATE) metrics as…
Proportional Response: Contextual Bandits for Simple and Cumulative Regret Minimization
In many applications, e.g. in healthcare and e-commerce, the goal of a contextual bandit may be to learn an optimal treatment assignment policy at the end of the experiment. That is, to minimize simple regret. However, this objective remains…
Proportional Response: Contextual Bandits for Simple and Cumulative Regret Minimization
In many applications, e.g. in healthcare and e-commerce, the goal of a contextual bandit may be to learn an optimal treatment assignment policy at the end of the experiment. That is, to minimize simple regret. However, this objective remains…
Can Personalized Digital Counseling Improve Consumer Search for Modern Contraceptive Methods?
This paper analyzes a randomized controlled trial of a personalized digital counseling intervention addressing informational constraints and choice architecture, cross-randomized with discounts for long-acting reversible contraceptives (LARCs),…
Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewal
In many settings, interventions may be more effective for some individuals than others, so that targeting interventions may be beneficial. We analyze the value of targeting in the context of a large-scale field experiment with over 53,000 college…
Targeting, Personalization, and Engagement in an Agricultural Advisory Service
ICT is increasingly used to deliver customized information in developing countries. We examine whether individually targeting the timing of automated voice calls meaningfully increases engagement in an agricultural advisory service. We define,…
Semiparametric Estimation of Treatment Effects in Randomized Experiments
We develop new semiparametric methods for estimating treatment effects. We focus on a setting where the outcome distributions may be thick tailed, where treatment effects are small, where sample sizes are large and where assignment is completely…
Estimating Heterogeneous Treatment Effects with Right-Censored Data via Causal Survival Forests
Forest-based methods have recently gained in popularity for non-parametric treatment effect estimation. Building on this line of work, we introduce causal survival forests, which can be used to estimate heterogeneous treatment effects in survival…