All impact evaluations depend on quantitative data that typically come from surveys, direct observations of children or service providers, and administrative data like scores from exams. To detect impacts of the interventions under evaluation, we need data that will appropriately measure changes in the outcomes we want the intervention to affect. How do we know we are using the right data and analyzing it appropriately? Enter psychometrics, which is the branch of psychology focused on the design, administration, and interpretation of quantitative assessments. Psychometric evaluations can tell you if you are using the right tools for your measurement goals.
Learning
Andrew Ho, professor of education at the Harvard Graduate School of Education, delivering a mini-course at the World Bank on psychometrics. He is a psychometrician whose research aims to improve the design, use, and interpretation of test scores in educational policy and practice. The following links offer a view into his course, along with corresponding materials.
Lectures
Mini-course presentation (Part 1), June 11, 2018, “Statistical and psychometric methods for measurement: Scale development and validation.”
Mini-course presentation (Part 2), June 27, 2018, “Statistical and psychometric methods for measurement: G Theory, DIF, & Linking.”
Studies referenced in the lectures
Jessen A, Ho AD, Corrales CE, Yueh B, Shin JJ. Improving Measurement Efficiency of the Inner EAR Scale with Item Response Theory. Otolaryngology–Head and Neck Surgery
2018, Vol. 158(6) 1093–1100.
Appendix, Stata code (Inner Ear All Analyses Codeshare.do)
Duckworth AL, Quinn PD. Development and Validation of the Short Grit Scale (Grit–S). Journal of Personality Assessment, 91(2), 166–174, 2009.
Other code and data referenced in lectures
- Sample data for generalizability
- STATA code for generalizability
- Sample Excel code for generalizability
- STATA code for differential item functioning
Research
Halpin, P. F., Wolf, S., Yoshikawa, H., Rojas, N., Kabay, S., Pisani, L., & Dowd, A. J. (2018, November 8). Measuring Early Learning and Development Across Cultures: Invariance of the IDELA Across Five Countries. Developmental Psychology.