Methods for combining experimental and population data to estimate population average treatment effects

With increasing attention being paid to the relevance of studies for real-world practice (such as in education, international development, and comparative effectiveness research), there is also growing interest in external validity and assessing whether the results seen in randomized trials would hold in target populations. While randomized trials yield unbiased estimates of the effects of interventions in the sample of individuals (or physician practices or hospitals) in the trial, they do not necessarily inform about what the effects would be in some other, potentially somewhat different, population. While there has been increasing discussion of this limitation of traditional trials, relatively little statistical work has been done developing methods to assess or enhance the external validity of randomized trial results. This talk will discuss design and analysis methods for combining experimental and population data to assess and increase external validity, as well as general issues that need to be considered when thinking about external validity. Implications for how future studies should be designed in order to enhance the ability to estimate population effects will also be discussed.