Finding Regions of Heterogeneity in Decision-Making via Expected Conditional Covariance

Abstract

Individuals often make different decisions when faced with the same context, due to personal preferences and background. For instance, judges may vary in their leniency towards certain drug-related offenses, and doctors may vary in their preference for how to start treatment for certain types of patients. With these examples in mind, we present an algorithm for identifying types of contexts (e.g., types of cases or patients) with high inter-decision-maker disagreement. We formalize this as a causal inference problem, seeking a region where the assignment of decision-maker has a large causal effect on the decision. Our algorithm finds such a region by maximizing an empirical objective, and we give a generalization bound for its performance. In a semi-synthetic experiment, we show that our algorithm recovers the correct region of heterogeneity accurately compared to baselines. Finally, we apply our algorithm to real-world healthcare datasets, recovering variation that aligns with existing clinical knowledge.

Publication
Proceedings of the 35th Conference on Neural Information Processing Systems
Justin Lim
Justin Lim
Master’s student
Christina X Ji
Christina X Ji
PhD Student

Christina is interested in characterizing variation in treatment policies, examining the theoretical assumptions behind off-policy evaluation of reinforcement learning for healthcare, and developing algorithms for disease progression modeling.

Michael Oberst
Michael Oberst
PhD Student

Michael’s research interests include developing learning algorithms for dealing with non-stationarity / dataset shift in predictive modelling, as well as robust learning of treatment policies from observational data.

David Sontag
David Sontag
Associate Professor of EECS

My research focuses on advancing machine learning and artificial intelligence, and using these to transform health care.

Related