TECHNICAL MANUAL
REPORTING ON RELIABILITY & VALIDITY
Concurrent Validity Concurrent validity measures the degree to which assessment scores from a new test compare to a well-established test. This requires both variables to be measured at approximately the same time (i.e., concurrently), to prevent external factors from affecting the variables of interest. Because the L2M assessment was specifically designed to capture a student’s letter-naming, decoding, encoding, and comprehension ability, the components of literacy common to many reading-focused assessments can be used to determine concurrent validity. Specifically, the degree to which a particular metric (such as the GE score from an L2M assessment) actually captures the underlying variable of interest (in this case, literacy) is referred to as validity, and positive correlations between different assessments demonstrate that those assessments are measuring the same underlying characteristic and provide evidence of an assessment’s concurrent validity. Correlation coefficients at and above 0.70 are commonly taken to indicate strong relationships (Moore et al., 2013). Data collected from 3,954 third-grade students at the end of the 2020–21 school year was used to evaluate correlations between the A2i L2M assessment and eight third-party literacy assessments. This analysis uncovered robust correlations* between L2M and each of these external assessments. These correlations indicate that L2M assessments capture important information about student literacy and demonstrate a high concurrent validity for L2M across varied conditions, locations, and populations.
Figure 5. Concurrent validity of L2M assessment demonstrated by correlation with third-party assessments. L2M Grade Equivalent scores are significantly correlated with the scores produced by third party literacy assessments and do so more efficiently (i.e., in less time than any other assessment save for STAR Reading, which also takes approximately 15 minutes).
Correlations Between A2i and Third-Party Assessments
Assessment
*These correlations were conducted using student-specific growth models to predict student performance on L2M at the time of the third-party assessments. These growth models pool information from our large database of student assessments and effectively smooth out much of the unavoidable noise that occurs from assessment to assessment. However, these growth models have not yet been incorporated into the A2i system, so these correlations represent best case concurrent validity.
Back to Top
Scholastic.com/A2i
Page 18
Made with FlippingBook - Online catalogs