My main line of research concerns external validity of experimental findings. Randomized controlled trials (RCTs) hold a special position atop the hierarchy of causal analysis methods. Randomization allows us to know that a treatment and control group are similar on all characteristics except who received treatment, and thus we can attribute differences in outcomes among these groups to the causal effect of treatment, giving them strong external validity. However, ethical, budgetary, and time constraints often mean researchers conduct RCTs outside of the target context of inference, on non-representative samples of the population, using varying degrees of realistic treatments, and/or measuring proxies for the primary outcome of interest. The differences on these dimensions from the target research question have lead to long-standing debates about the external validity of experimental findings.
I work on methods that allow researchers generalize experimental results beyond the units, treatments, outcomes, and contexts upon which they were conducted. In Elements of External Validity: Framework, Design, and Analysis (with Naoki Egami), we derive a framework for considering issues of external validity among four dimensions (units, treatments, outcomes, and contexts). We also discuss design and analysis techniques for applied researchers interested in population treatment effects.
Other work of mine has considered methods for estimating population treatment effects, such as Covariate Selection for Generalizing Experimental Results (with Naoki Egami) or From SATE to PATT (with Richard Grieve, Roland Ramsahai, and Jasjeet Sekhon), which also provides a falsification test for the policy relevant quanity of interest–the population average treatment effect on the treated. In Leveraging Population Outcomes to Improve the Generalization of Experimental Results (with Melody Huang, Naoki Egami, and Luke Miratrix) we propose a way to leverage outcome population data from a target population to improve precision when estimating population effects.
For a minimally technical introduction to generaliability of experimental findings, see Generalizing Experimental Results and Generalizability and Transportability.
Even with these advances, there is much work to be done in providing applied researchers with effective tools for addressing external validity.
See the full list of relevant manuscripts and publications below!
