Florida RISE/Up Students Outperform Peers on Standardized Fluency and Reading Comprehension Assessments
Lead Researcher: Rachel L. Schechter, PhD.
Research Design During the 2022–23 school year, Scholastic contracted with LXD Research to conduct a quasi-experimental study that examined the implementation and influence of Scholastic RISE and RISE Up (RISE/Up) on students’ reading abilities in grades 1 to 5 in a school district in Florida. The evaluation tracked the literacy performance of students (as measured by the Renaissance Early Learning/Reading, FAST English Language Arts Reading, and Fastbridge CMBreading assessments), which used RISE and RISE Up as a Tier 2 pull-
out intervention, as well as an equivalent group of students who did not use the program 1 from January 2023 to May 2023 2 . The study aimed to determine whether “striving” students using RISE/Up outperform those in comparison group classrooms. To better understand how RISE/Up was implemented and how educators perceived it, the study included program logs, surveys, classroom observations, and interviews with teachers and district leaders. RISE/Up Implementation in Florida Public School Districts The Scholastic RISE/Up program is a Tier 2 intervention program designed for students in grades 1 to 5 that provides targeted, small-group instruction to address and prevent reading gaps using explicit, structured comprehension, word study, phonics instruction, and guided writing. It recommends usage of 45–60 minutes per day, with one to four teachers. Scholastic RISE/Up offers a kit with teacher resources, books, short reads, and digital access for teachers and students. Scholastic RISE is leveled C–N and targets comprehension, word-solving, fluency, and writing. In the Florida Public School district, groups of up to four students rotate through four instruction stations for 30 minutes daily (e.g., 12–15 minutes per station over two days). RISE Up is leveled O–Z and is focused on advancing student comprehension. For RISE Up, groups of four students rotate through three instructional stations focused on comprehension strategies on a short text for 30 minutes (12–15 minutes per station over two days). 1 This study used a matched-comparison group design to evaluate the effects of RISE/Up on student achievement. The analytical sample consisted of: (1) a group of students who used RISE/Up, and (2) a comparison group of equivalent students who did not. To ensure baseline equivalence, researchers used one-to-one, nearest-neighbor propensity matching to identify a comparison group whose baseline characteristics were similar to those of RISE/Up students at the beginning of the intervention. First, researchers paired students whose grade level, gender, racial ethnicity, special education status, and English language status were identical to the RISE/UP sample. If a match using these procedures was unavailable for the RISE/Up student, that student was removed from the analysis. Propensity score matching procedures were conducted separately for grades 1 and 2 because the tests for these grades have different subtests at the start of the year for each grade level. Propensity Score Matching was conducted for grades 3 to 5 because all students took FAST with grade-agnostic scale scores. Researchers used paired t-tests to determine whether performance on the Fall Renaissance Star Early Learning, Star Reading, and FAST ELA differed significantly between the treatment and comparison groups. No significant differences were found; therefore, the matching procedures successfully created equivalent groups. See Appendix A. 2 Interviews with district leadership, conversations with educators during site visits, and analysis of teacher surveys showed that student learning during the fall semester was disrupted multiple times. The school district approved the research study in mid-September, coinciding with massive teacher shortages. Shortly after the teachers were trained on RISE, the category 4 Hurricane Ian hit Palm Beach County. A few weeks later, in early November, category 1 Hurricane Nicole inflicted significant damage and closed the schools for multiple days. With Thanksgiving break and winter break just weeks later, measuring the impact of RISE between January and May would more accurately reflect its effectiveness. This period also coincided with the study’s weekly implementation log collection, providing the researchers with better insight into implementation and instructional patterns to triangulate with quantitative results.
Results Program Dosage
Interventionists and teachers input weekly RISE/Up activities between January and May 2023 in a log. Log data showed that students completed less than half of the expected weekly stations (Figure 1). Rather than completing between 8 and 10 stations a week, students only completed an average of 4.5. Considering that RISE was designed for four stations a day, every day, this implementation was approximately only one-quarter of the daily recommended dosage.
Figure 1. Program Usage from Teacher Logs, by Grade Level
Avg. Total Stations Completed
Avg. # of Station Cards Used
Avg. Stations per Week
Avg. Cycles per Week
Unique Skills Covered
Grade
# of Students
5
5.2
62 52 52 56 36
1.6
14 12 28 27 10
13 21 27 31 16
1 2 3 4 5
17 12 28
4.36 4.33 4.65 3.02
1.15 1.06 1.54
7
1.0
FastBridge CBM FastBridge CBMreading is an oral reading fluency assessment that measures students’ reading rate. During the assessment, students read aloud for one minute from grade- or instructional-level passages while the teacher marks any errors. The resulting score is the number of words read correctly per minute (WRCPM). A student’s reading rate details how automatically the student can recognize, decode, and understand each word. Beginning in January 2023, grades 1 to 5 interventionists in the study were asked to administer the FastBridge CBM assessment every two weeks for 18 weeks, except during the spring break week (i.e., Week 10) and during state testing (i.e., Weeks 15 to 18). LXD Research analyzed FastBridge CBMreading data from 258 students who took the assessment at least six times (45 RISE participants and 213 Comparison group participants) from Week 2 to Week 14 3 . The RISE/Up and comparison group WRCPM scores did not differ statistically from each other at the beginning of the intervention (i.e., at Week 2/3; t(256) = 0.97), indicating that they were similar at the start of the testing period. RISE/Up students showed significantly more growth in WRCPM than students in the comparison group by the end of the year (F(1, 5) = 3.3, p < .01, partial eta squared effect size = .06). While RISE/Up students gained an average of +21 words correct per minute, the comparison group only gained +7 WRCPM. RISE/Up students demonstrated fluency growth three times greater than those who did not use the program. These results are promising because the dosage was less than half of the intended usage (Figure 3). Figure 2. FastBridge Words Read Correctly per Minute (WRCPM) Scores, RISE/Up and Comparison Students Assessment Condition N Pre/Post-Mean (Change) SD
Occasion* Condition Interaction F Score
p-Value Partial-eta Squared Effect Size
FastBridge WRCPM
Comparison
213
83 - 90 (+7)
33.7 34.0*
F(1, 5) = 3.3
p = .002
.06
RISE/Up
45
88 - 109 (+21)
*Note: Pooled SD = 33.9
1 There was significant missingness in six of the 18 total weeks: Week 1 (73% missing), Week 10 (100% missing), and Weeks 15 to 18 (74% missing). Week 1 missingness was due to communication issues at launch, Week 10 was spring break, and Weeks 15 to 18 were interrupted by state testing. Therefore, Weeks 1, 10, and 15 to 18 were excluded from the analysis. In addition, the participants were asked to respond biweekly, so missingness at the individual level typically followed a biweekly pattern. Therefore, scores were averaged in two-week intervals (e.g., Weeks 2 and 3 were averaged, and Weeks 4 and 5 were averaged). This averaging resulted in a reduction to six occasions of measurement. After completing this method, missingness was reduced to an average of 16% across the six two-week intervals.
Figure 3. FastBridge Words Read Correctly per Minute (WRCPM Scores), RISE/Up and Comparison Students
*Note: N = 213 Comparison and 45 RISE students. Week 10 was Spring break and was therefore excluded from the analysis.
Star Early Literacy and Star Reading Assessments LXD Research also analyzed scaled scores from the Winter 2023 and Spring 2023 administrations of the Star Early Literacy, Star Reading, and FAST ELA assessments (See Figure 4). While students in grades 1, 2, and 3 outperformed the comparison group, the results were not statistically significant.
Figure 4. Winter to Spring 2023 Gains on the Star Early Literacy, Star Reading, and FAST ELA Tests RISE/Up and Comparison Students by Grade Level
Perceptions LXD Research conducted interviews to gather details on the implementation and experiences with RISE/Up. The third-party research firm also administered an online teacher survey to gauge perceived impacts and attitudes toward the program. Data indicate that RISE/Up interventionists and teachers' perceptions of the program were overwhelmingly favorable, especially regarding content, student engagement, and achievement. In an online survey, 93% of teachers indicated that the students particularly enjoyed the content of the books, and 86% of the teachers selected the group discussions.
When asked why the students preferred the content of books, teachers responded with: “the texts were interesting,” “students loved getting a new book,” and “the content is engaging and fun for them.” Teachers said the students enjoyed class discussions because “it allowed them to share their understanding and also learn what their classmates understood from the text.” In addition, more than 90 percent of teachers agreed that RISE/Up helps improve student engagement, vocabulary, fluency, comprehension, sense of belonging, and re-engagement (See Figure 5).
Figure 5. Teacher Survey ( N = 14) RISE/Up Interventionists/Teachers Perceptions of the Program Teachers Agreed that RISE Helped Their Students With:
Conclusion A third-party research firm, LXD Research, analyzed student growth over time using winter and spring scores from Renaissance Star Reading, Renaissance STAR Early Literacy, FAST ELA, FastBridge CBMreading, and all available student- level demographic variables. The key results and conclusions of this evaluation are as follows: • RISE/Up students outgained comparison students on the Fastbridge CBMReading assessment by 14 points in just three months. This advantage was statistically significant ( p = .002). • While grades 1, 2, and 3 RISE/UP students outgained the comparison group, the results were not statistically significant. • Teacher perceptions of RISE/Up were generally favorable, especially about student engagement and achievement. • This study provides ESSA “Promising” or Tier 3 evidence of the efficacy of RISE/Up in improving student achievement in reading for students in grades 1 to 5. Considering it was the first year using RISE/UP, the growth seen in this paper speaks to the product’s ease of use and potential to support students who had been showing resistance or lack of progress with the school’s other tools in the intervention toolkit.
Appendix A: School District of Palm Beach County, RISE/Up and Comparison Group Demographics by Grade Level after Propensity Matching
Number of Students
Gender (Male)
Ethnic Minority
Free/Reduced Lunch
Special Education
Condition
RISE/UP
34 32 33 81
56% 57% 58% 52%
71% 75% 67% 64%
44% 56% 48% 53%
32% 31% 30% 26%
1st Grade
Non/RISE/IP
RISE/UP
2nd Grade
Non/RISE/IP
RISE/UP
101
3rd–5th Grades
63%
78%
67%
33%
Non/RISE/IP
139
55%
81%
73%
27%
School District of Palm Beach County, RISE/Up and Comparison Group Beginning of Year (BOY) Reading Test Score by Grade Level after Propensity Matching
Number of Students
Reading Test Score BOY
Condition
Significance
RISE/UP
34
694.1
1st Grade STAR Early Literacy Scale Score
p = 0.96
Non/RISE/IP
32
693.6
RISE/UP
33
755.3
2nd Grade STAR Reading Scale Score
p = 0.97
Non/RISE/IP
81
756.0
RISE/UP
60
268.8
3rd Grade FAST ELA Scale Score 4th Grade FAST ELA Scale Score 5th Grade FAST ELA Scale Score 3rd–5th Grades FAST ELA Scale Score
p = 0.70
Non/RISE/IP
84
267.8
Non/RISE/IP
23
278.0
p = 0.28
RISE/UP
28
282.3
Non/RISE/IP
18
285.5
p = 0.63
RISE/UP
27
283.2
RISE/UP
101
273.9
p = 0.94
Non/RISE/IP
139
273.7
Appendix B: School District of Palm Beach County, RISE/Up and Comparison Group FastBridge Sample
Condition
Comparison
Rise
Total
Count
8
2
10
1st Grade
Percentage
4%
4%
4%
Count
34
5
39
2nd Grade
Percentage
16%
11%
15%
Count
86
9
95
3rd Grade
Percentage
40%
20%
37%
Count
46
19
65
4th Grade
Percentage
22%
42%
25%
Count
39
10
49
5th Grade
Percentage
18% 171
22%
19% 209
Count
38
3rd–5th Grades
Percentage
80%
84%
81%
213
45
258
Total
Page i Page ii Page 1 Page 2 Page 3 Page 4Made with FlippingBook - Online catalogs