IEA INSIDER 2025
Research & Development Matters: From Idea to Influence
BY RALPH CARSTENS AND LAUREN MUSU
IEA’s international large-scale assessments provide insights into how and why education matters. By extension, the IEA Research and Development (R&D) fund plays a consistently important role for advancing methods and ensuring that IEA studies keep pace with an evolving educational landscape.
With 16 projects completed for Calls 1–3, seven projects in progress for Call 4, and the current Call 5 for 2025 just closing, the field-initiated program has solicited an impressive range of research conducted by experts from within IEA and the wider IEA network. Across the projects to date, several overarching themes have emerged—each contributing critical insights to IEA’s practice and the field of survey and assessment methodology at large. • One set of projects has aimed to bring innovations to assessment design and instrument development , in particular: the use of large language models for automatic item generation; AI-driven item verification and alignment; validity and measurement properties of technology-enhanced items; effects of positively and negatively worded item on respondents’ behavior; dimensionality and mode effects; novel question types in electronic surveys; and how the instructional sensitivity of items can reflect differences in classroom teaching. • More recently, a set of projects have explored aspects related to global relevance, fairness, and diversity including contextual questionnaires in low- and middle- income countries, improvements to accessibility, accommodations, and inclusion; and cross-cultural comparisons in ILSAs. • In terms of sampling and weighting , a range of projects have investigated student exclusions; the relative performance of variance estimators; frameworks
for and desirable field trial sample sizes to drive decisions for instrument development. understanding school non-response; • Once data is collected, an equally strong set of projects have proposed methods for better understanding respondent behavior through cognitive modeling and similar: approximate areas of interest to study student motivation and engagement; students’ computational thinking strategies in digital tasks; and effects of survey mode and trends over time. • Another timely strand of research has explored the application of AI and neural networks on response data to enhance data quality and assist in human coding processes through the classification of image-based responses; the automated scoring of text responses; and possible improvements to the coding of parental occupation. • Two projects have strived to improve psychometrics and imputation , namely a variable selection approach in latent regression to increase model precision; and a two-step imputation approach combining IRT and deep learning to missing data. • Finally, one project is piloting retrieval-augmented generation (RAG) to make the vast and complex results and documentation generated by our large- scale assessments more accessible and user friendly.
70
Made with FlippingBook - PDF hosting