Struggling students in the Hot Springs School District (HSSD) in Arkansas were not ready for grade-level mathematics once they reached Grade 9. The district was looking for a program to help these middle school students bridge their knowledge gap. After seeing promising results in its first year of implementation, the district chose the MATH 180 program for all their struggling students in Grades 7 and 8 during the 2016–2017 school year. The district was interested in understanding the impacts of the program on student growth. This study involved 187 students who participated in MATH 180 for the full 2016–2017 school year. The study sought to answer the following research questions:
1. What are the effects of MATH 180 on student mathematics achievement?
2. How does MATH 180 differentially affect subgroups of students? Can these impacts be seen in secondary outcome measurements, such as NWEA MAP?
3. What is the association between student mathematics achievement and program implementation—are changes in MATH 180 participants’ mathematics test scores associated with variations in program progress?
Hot Springs School District recognized their students struggling in mathematics needed a change in curriculum. The district used block periods to work towards getting students ready for grade-level mathematics.
MATH 180 is a mathematics intervention program designed for struggling students in Grades 5–12. MATH 180 rebuilds the foundational skills that struggling students need to prepare for algebra readiness. The program is divided into two courses each with nine blocks of instruction, with each block covering three topics. MATH 180 utilizes a blended learning model of instruction. Students rotate between teacher-led group instruction and personalized software that adapts to their individual needs. MATH 180 Course I focuses on rebuilding students’ understanding of multiplicative thinking, division, fractions, and decimals. MATH 180 Course II prepares students for algebra by focusing instruction on rates, ratios, proportional and linear relationships, and functions. Essential to the program is rebuilding the belief in every student that they can be a mathematics learner. This is accomplished through the program’s emphasis on developing a growth mindset.
Struggling mathematics students from Hot Springs School District began using MATH 180 Course I and MATH 180 Course II in October of the 2015–2016 school year. The program was implemented at Hot Springs Junior Academy, the district’s only Grade 7 and Grade 8 school. The district saw promising growth among their students and chose to continue using the program for the 2016–2017 school year. Students enrolled in MATH 180 used the program for 98-minute blocks each day, completing two rotations each consisting of teacher-led instruction and student-led software exploration.
MATH 180 was the sole mathematics course for all enrolled students. The district’s goal is for students to be ready for their grade- level mathematics course either midyear or at the end of the school year. Students are moved to their grade-level mathematics course based on Math Inventory score, proficiency level, and course progress. All students move to their grade-level mathematics course, regardless of MATH 180 course progress, when they begin Grade 9.
The district universally tested all students using NWEA MAP and used those scores, in addition to teacher feedback, to identify students in need of mathematics intervention. Identified students took Math Inventory to confirm their MATH 180 placement. Math Inventory scores were used to determine if students were correctly placed in MATH 180 and if they should start at the beginning of the program or in a designated course midpoint. Most students begin in Block 1 for both courses, but some students are provisionally promoted to begin at a designated course midpoint for Course I and Block 5 for Course II. In the 2016–2017 school year, 11 Course I students out of 121 students were provisionally promoted to a designated course midpoint of the program. No students in Course II of the program began in an advanced block.
In total, 187 students used MATH 180 for at least 40 sessions during the 2016–2017 school year. One hundred and twenty-one students used MATH 180 Course I and 94 students used MATH 180 Course II. Primarily, Grade 7 students used MATH 180 Course I and Grade 8 students used MATH 180 Course II during the 2016–2017 school year. There were a few exceptions; 25 Grade 8 students used MATH 180 Course I and 13 Grade 7 students used MATH 180 Course II. Twenty-two of the twenty-five Grade 8 MATH 180 Course I students had used the program in the previous school year. Twelve of the thirteen Grade 7 MATH 180 Course II students completed MATH 180 Course I before beginning MATH 180 Course II.
District leaders described the student population as increasingly transient, with many students changing schools throughout the school year, withdrawing from the district midyear, or transferring into the district late into the term. As a direct result, many students did not finish a full year of the program or started the program late. Only students who were enrolled for a full year of MATH 180, and completed at least 40 MATH 180 software sessions, were included in this analysis. MATH 180 Course I students used the software for an average of 31 minutes per session and MATH 180 Course II students used the program for an average of 30 minutes per session. Students needed to have both a fall and spring score to be included in the Math Inventory and NWEA-MAP analyses.
As reported by district leaders, roughly 80% of students are eligible for a free or reduced-price lunch. Between 15–20% of students are English learners, a percentage which has been increasing recently. The district identified 37.1% of their students as White, 36% as Black or African American, and 19% as Hispanic for the 2016–2017 school year. While demographic data was not available for all students, Table 1 shows that Black or African American students are overrepresented in the MATH 180 student population as compared to district enrollment. The program was used by Grade 7 and Grade 8 students and by male and female students at similar levels of enrollment.
The students represented in this sample were highly diverse and considered to be at significant risk for not attaining Algebra readiness by the 9th grade.
Growth in mathematics achievement was measured by the Math Inventory®, an adaptive assessment measure that can be taken 3–5 times a school year. Math Inventory is not a part of MATH 180 but rather an external measure that is used to track students’ changing mathematics abilities while using MATH 180. Math Inventory measures student growth in mathematical knowledge through changes in their Quantile® measure.
Mathematics growth was also measured by NWEA® MAP®, a K–12 assessment program that can be used as a universal screener. The adaptive program measures what students know and what they’re ready to learn next. Student performance is reported out using a RIT score. All students in HSSD take NWEA MAP in the fall and spring.
Overall, MATH 180 Course I and Course II students saw growth of 160.8 Quantile measures on the Math Inventory, from 420.9Q to 581.7Q (see Figure 1). A paired samples t-test indicated the difference was statistically significant with t = 9.2(160) and p < .01. A similar examination was conducted using the math section of the NWEA MAP test where MATH 180 students saw an average growth of 2.3 RIT points, from 203.7 to 206 (see Figure 2). A paired samples t-test indicated the difference was statistically significant with t = 3.2(155) and p < .01. In addition to growth for the entire MATH 180 sample, changes from fall to spring were also examined by ethnicity. MATH 180 students in all ethnic groups saw an increase in their Math Inventory scale scores, however those who were identified as Black or African American saw the largest average growth of 178.9 Quantile measures. Students representing each ethnic group also saw increases in their NWEA MAP and students who were identified as Hispanic saw the greatest average growth from 204.2 to 208.2.
MATH 180 Course I and MATH 180 Course II each consist of 27 topics over nine blocks of instruction. Students move through the software at their own pace. HSSD MATH 180 students completed between five and 52 topics of software instruction during the 2016–2017 school year. To determine the level of association between program use, as measured by topics completed, and growth in mathematics achievement, the mean growth for each number of topics completed was calculated and plotted yielding a Pearson correlation of .64 (see Figure 3). In an effort to verify the linearity of this relationship, a series of models was tested to ascertain the best fit for this data. Based on these comparisons, the two lines that yielded the best fit or accounted for the most variance were linear R2= .41 and logarithmic R2= .51.
During the 2016–2017 school year, Hot Springs School District implemented the MATH 180 Course I and Course II programs. Students were identified for the programs based on their NWEA MAP performance during the prior spring. To document progress in their mathematics achievement, two outcome measures (Math Inventory and NWEA MAP) were administered in the fall and again in the spring of the school year. Results indicated that MATH 180 students achieved significant gains on both sets of measures. When viewing the results by ethnicity, a similar pattern of gains was noted where all subgroups increased in their achievement scores. African American students demonstrated proportionally higher gains on the Math Inventory and Hispanic students demonstrated proportionally higher gains on NWEA MAP.
For a program to provide the intended effect on student achievement, it must be implemented with fidelity. This study also examined the relationship between amount of program use and growth in student achievement as measured by the Math Inventory. For the purposes of this analysis, level of program use or fidelity was defined as the number of topics the students completed in the software component of the program. Results indicated that the number of topics completed was significantly associated with growth in math achievement. Although a linear model was reasonable in explaining the relationship between program use and achievement growth, a series of alternative models depicting this relationship was also tested. The results indicated that a logarithmic curve provided a slightly better estimation of this data set than the linear association. The difference was modest but pointed to a more nuanced connection between program use and growth. This relationship (depicted in Figure 3) implies that student growth increases at a faster rate initially then begins to stabilize as the completion of topics meets a threshold. Growth continues as student completion of topics increases but at a slower and more sustainable rate. Although this model of student growth does begin to acknowledge the additional complexity of a students’ learning trajectory, the true nature of this relationship is likely even more complex. Future research should acknowledge the integrated nature of student educational inputs and continue to unpack these relationships.