Generalized, Multilevel, and Longitudinal Psychometric Models for Evaluating Educational Interventions

Sponsor
U.S. Department of Education Institute of Education Sciences
$899,995
Principal investigator
Matthew Madison
Assistant Professor, Department of Educational Psychology
Co-principal investigators
Minjeong Jeon, University of California, Los Angeles
Michael Cotterell, Senior Lecturer, Department of Computer Science
Active since
July 2022

Abstract

Diagnostic classification models (DCMs) can provide useful information about students’ strengths and weaknesses in an academic domain beyond the norm-referenced or percentile results typically achieved through item response theory. DCMs, however, need to be able to accommodate the multilevel framework in which most education takes place, or they run the risk of providing substantially inaccurate results. The purpose of this project is to develop and make accessible to applied researchers a multilevel extension to the longitudinal diagnostic classification model to help researchers take into account contextual effects that can impact the fidelity and effectiveness of an educational intervention.

Our research team will first develop the mathematical models and then program them for testing via Monte Carlo simulation studies to evaluate the validity and reliability of the proposed modeling framework in a variety of research contexts. They will also conduct secondary data analyses to demonstrate the utility and added value of the proposed methods relative to other common approaches. The results of the simulations and the secondary data comparisons will render a synthesis of practical recommendations for researchers interested in applying the proposed methods in intervention studies. Our research team will develop user-friendly software for conducting multilevel longitudinal DCM analyses and will provide workshops to applied audiences in addition to supporting the software with online training materials.