Design and Analysis of Clustered Observational Studies

Friday, October 4, 2023,  9:30am - 11:00am ET               

Speaker: Luke Keele, University of Pennsylvania

Registration is required to view this webinar and can be completed here. 

Many interventions in education occur in settings where treatments are applied to groups. For example, a reading intervention may be applied to all the students in some schools and withheld from the students in other schools.  When such treatments are non-randomly allocated, outcomes across the treated and control groups may differ due to the treatment or due to baseline differences in the groups. We classify such study designs as the observational analogue to clustered randomized trials or clustered observational studies (COS). Clustered observational studies (COS) are a critical analytic tool in educational effectiveness research. In this workshop, we will introduce participants to key aspects of the COS design including the possibility of bias from differential selection.  Differential selection can occur when the units in a cluster respond to the treatment status of the cluster. We will outline when analysts need to condition on school and student covariates, or just on school covariates. We will review new methods based on balancing weights to adjust for covariates in the COS design. We will compare and contrast the balancing weights approach to multilevel matching methods.

In this webinar, we will:

• Explore a number of conceptual issues that arise when the goal is to estimate treatment effects with the COS design. Emphasis will be placed on the need to understand how the treatment assignment mechanism operates.  
• Introduce new, specialized methods for balancing covariates in the COS design.
• Assess balance at multiple levels of the data
• Estimate treatment effects, and
• Examine the various practical choices needed in the analysis process.

 ___________________________________________________________________________________