Doctoral Thesis: Causal inference for Socio-Economic and Engineering Systems

Doctoral Candidate: Anish Agarwal

Abstract:
What will happen to Y if we do A?

A variety of meaningful socio-economic and engineering questions can be formulated this way. To name a few: What will happen to a country’s economy if policy-makers legislate a new tax? What will happen to a patient’s health if they are given a new therapy? What will happen to a company’s revenue if a new discount is introduced? What will happen to a data center’s latency if a new congestion control protocol is used? This thesis explores how to answer such counterfactual question using observational data—what is increasingly available due to digitization and pervasive sensors—and/or very limited experimental data.

Towards this goal, the key framework introduced in this thesis is connecting causal inference with tensor completion. In particular, a general way to represent the various potential outcomes of interest is through an order-3 tensor, where rows index units (e.g., individuals, sub-populations, geographic regions), columns index measurements (e.g., outcomes over time), and matrix slices index interventions (e.g., discounts, health therapies, socio-economic policies). If we could fully observe this tensor, we would be able to effectively do counterfactual decision-making. However, without the ability to do extensive experimentation and build a reliable model of the system of interest, we only observe a sparse, noisy subset of the entries of the tensor. An additional challenge with observational data is the issue of confounding, i.e., unobserved factors that determine both the entries of the underlying tensor and the missingness pattern in the observed tensor.

To tackle these challenges, the key theoretical results is shown on high-impact real-world applications. These include working with: (i) A pharmaceutical company to correct for bias from patient dropouts and proposing novel clinical trial designs to reduce the number of patients recruited for a trial. (ii) A large e-commerce platform on how to re-design their A/B testing platform to potentially run 10-20% of the millions of A/B tests they conduct to ensure a personalized customer experience. (iii) U.S. and Indian policy-makers to evaluate the impact of mobility restrictions on COVID-19 mortality outcomes. (iv) The Poverty Action Lab (J-Pal) at MIT to make personalized policy recommendations to improve childhood immunization rates across different villages in Haryana, India.

Details

• Date: Thursday, November 18
• Time: 10:00 am
• Category: Thesis Defense
• Location: E18-304

Additional Location Details:

Thesis Committee: Profs. Alberto Abadie, Munther Dahleh, Devavrat Shah

To attend via zoom, please contact the doctoral candidate at anish90@mit.edu