The Reading Measure Study: Phase I & II There were two primary phases of the Scholastic Reading Measure study: the first phase was an expert review of the CAT algorithm and the second phase was a field-based reliability and validity study of the Reading Measure with American Institutes for Research (AIR), an independent research firm.
Phase I: Review of CAT Algorithm
A senior psychometrician at the independent research firm (AIR), who is an expert on CATs, reviewed the Reading Measure CAT algorithm.
Phase II: Reliability and Validity Study of the Reading Measure
Prior to initiating the reliability and validity study, the Reading Measure was subjected to usability and other product testing to evaluate how real students would interact with the measure and to address any technical issues that might affect the performance of the Reading Measure .
A field-based reliability and validity study was conducted in order to ensure: • confidence in replication of consistent results (reliability) • confidence in the accuracy of the measure (validity)
Approximately 10,000 students in Grades 1–6 were targeted for participation in the study to ensure a robust sample for analyses, with approximately 500 students at each grade level providing complete data (i.e., two complete administrations of the Reading Measure and either progress monitoring/formative assessment data or state reading assessment scores). Additionally, a minimum of 400 students with complete data were targeted for subgroup analyses for English learners, students with disabilities or those eligible for special education services, students receiving free and reduced-price lunch, and other groups of interest.
As part of the reliability and validity study, the independent research firm also intended to gather contextual data through classroom observations.
In order to conduct the reliability study using the test-retest model, students would take the Reading Measure two times in a short time frame: Reading Measure during Week 1 (Administration 1), no research activities in Week 2, and Reading Measure during Week 3 (Administration 2). Additionally, in order to conduct the validity study, data from the reliability study would be analyzed in conjunction with achievement data from the district for students who participated in the reliability study.
SCHOLASTIC READING MEASURE RELIABILITY AND VALIDITY STUDY 3
Made with FlippingBook - Online catalogs