Adaptive & Iterative Experimentation

Rapid testing of new features and algorithms allows organizations to innovate more quickly as well as customize its services to the needs of individuals.

Our lab researchers help social impact organizations test hypotheses about their products or services. We design experiments and analyze the data to determine the impact of interventions on different individuals. Methods for adaptive experimentation, such as multi-armed bandit algorithms, contextual bandit algorithms (that incorporate individual characteristics), and reinforcement learning are important tools to improve an organization’s ability to experiment effectively. These methods need to be carefully tailored to the setting and respect organizational constraints.

Featured App

An application that assists researchers in planning adaptive experiments using a method that makes experiments more effective. By assigning better-performing treatments to more participants, the researcher learns which treatment is most effective faster and using fewer resources.

Project Abstracts

Read about a few of the research projects the lab is currently working on.

Enhancing Digital Education Technology

17EdTech is an online learning platform offering homework solutions for K-12 students, parents, and teachers in China. In partnership with 17EdTech, lab researchers are designing a personalized delivery system of homework questions for users of its mobile app and are reviewing effective learning products for parents to assist in children’s learning, particularly in low-income regions. Lastly, the team is estimating the impact of gamification on student engagement.

Education

Learning How to Incentivize Giving

With a financial technology firm and ImpactMatters, this project is identifying approaches to increase contributions to charitable organizations. Lab researchers are analyzing donation data to understand consumer behavior and motivations around charitable giving, and designing experiments to expose consumers to giving opportunities during electronic commerce. The experiments will guide strategies to motivate consumers to increase their giving and direct their giving to impactful organizations.

Charitable Giving

Testing Efficacy of Personalized Approaches to Encourage Charitable Donations

Lab researchers are using adaptive experimentation to test the efficacy of personalized approaches to encourage charitable donations. Algorithms are utilized to choose which questions to ask a user about their interests; different options are then offered based on the users’ answers. In addition, the research is also establishing best-practices of using experiments to consider how people use tools to make donations, focusing on how to optimize the experimental design to learn how to target interventions to users’ characteristics.

Charitable Giving

Adaptive Experiments to Help Patients Make Informed Choices About Contraception

Lab researchers are working with Yaounde Gynecology, Obstetrics and Pediatrics Hospital in Cameroon to help women make informed choices about contraceptives. Adaptive experiments identify effective and efficient strategies, including price subsidies, for providing information through a tablet application. The experiment accelerates learning in an environment of many potential alternatives and improves expected patient outcomes by allocating more and more patients to the tablet design that works best.

Health

Academic Publications

Publication Search

Interviews

Learn firsthand from researchers and practitioners associated with the lab.

Thought Leadership

University of Pennsylvania, Wharton

Susan Athey shares how to apply machine learning in behavioral science experiments and use those applications in social impact organizations, especially designing for heterogeneous treatment effects in experiments. She highlights a specific project with a fintech company identifying approaches to increase contributions to charitable organizations.

Indiana University

Preliminary Findings from a new Collaborative: Using financial technology and big data to increase the quantity and quality of charitable giving.

Advances with Field Experiments Seminar Series, University of Chicago, Department of Economics

Susan Athey presents "Confidence Intervals for Policy Evaluation in Adaptive Experiments".