DATA IN ACTION
Measuring Educational Opportunities and Outcomes
achievement tests and background questionnaires to develop instruments and construct educational measures. During the development stage, we use field trial data to examine the dimensionality of tests or questionnaire scales, provide statistical criteria for items selection, and evaluate the overall performance of the instruments. After the main survey data collection, we provide scores derived from confirmatory factor analyses (CFA) or item response theory (IRT) models. We also compute plausible values by combining test and survey data to enable unbiased population estimates. Given that international studies include data from multiple countries, a critical aspect of our work is the analysis of measurement invariance. This ensures that comparisons across countries are based on equivalent measures, which is essential for making valid inferences about education systems globally. The same holds for comparisons between different grades, social groups such as gender, or over time. Our rigorous approach guarantees that the data we produce is both accurate and fair, enabling meaningful international comparisons. CAPACITY BUILDING AND RESEARCH IEA Hamburg is committed to enhancing capacity in educational measurement through specialized training programs. For nearly two decades, we have been offering regular introductory and advanced workshops in educational measurement and psychometrics, covering topics such as item response theory, measurement invariance testing, structural equation modeling, and multilevel structural equation modeling. These workshops take place in Hamburg, at international conferences, and we also offer tailored training on demand for national clients. A notable upcoming event is our workshop on IRT and population modeling at the IRC 2025 (see page 50). Further information about training opportunities can be found on the IEA website here. Our commitment to excellence is further demonstrated through our advanced research initiatives. We employ and continually develop state-of-the-art methodologies. We have secured research funding through the IEA Research and Development Fund, supporting projects like Improving Data in Electronic Surveys (IDES), Field Trial Sample Size for Scales (IDES), and Dimensionality and Mode Effects (DIMO) in PIRLS 2016 and PIRLS 2021. These projects explore innovations and operational issues in large-scale studies, such as digital surveys, sample size requirements, and the comparability of the PIRLS assessments. ■
BY PURYA BAGHAEI & ROLF STRIETHOLT
A central goal of international large-scale studies in education is to measure educational opportunities and outcomes. This is often done using tests and questionnaires to capture complex constructs such as students' mathematics performance, their reading engagement, or teachers' job satisfaction. The measurement of these constructs is a cornerstone of IEA's work. Worldwide, there are very few organizations with the expertise in test and questionnaire development and the corresponding psychometric and statistical analyses to conduct international educational studies. While IEA has often collaborated with external partners, the Research and Analysis (RandA) unit has increasingly built-up expertise in this area, successfully supporting IEA studies and those of other organizations. SUPPORTING IEA STUDIES AND EXTERNAL PARTNERS Over the past few years, IEA has continuously built its capacity in educational measurement, psychometrics, and the scaling of both test and survey data. Our RandA team is currently responsible for the psychometric analysis and scaling of context questionnaires and achievement data for several IEA studies. RandA takes pride in supporting a wide array of external organizations, including OECD, UNESCO, and UNICEF, as well as various national organizations. We also support various national assessments across the globe, providing tailored solutions that meet the unique needs of different education systems. Additionally, we provide consultation and support to national clients in various countries in developing their national assessments. Notably, RandA is currently responsible for three major international studies: ICILS, TALIS, and ICCS. EDUCATIONAL MEASUREMENT IS A COMPLEX TASK Educational measurement involves applying advanced statistical and psychometric models to data from
47
Made with FlippingBook Online newsletter maker