Do The Math: Miami-Dade County Public Schools (2012)

Do The Math is an intervention program for students who are struggling with fundamental mathematics skills. The program consists of 13 scaffolded modules that focus on rebuilding fluency with whole numbers and fractions. Each module includes a series of 30 thirty-minute lessons. Every lesson provides step-by-step instruction and comprehensive teacher support. For this midyear implementation, Scholastic staff and district advisors chose three modules for each grade, covering seven different modules, as seen in Table 1.

Six hundred ninety-five students in Grades 3–5 from 11 elementary schools in the Miami-Dade County Public School District were enrolled in Do The Math during the spring semester of 2012. Nine schools taught Do The Math as a pull-out supplemental math program during the school day, and two schools taught Do The Math in after-school programs. Do The Math supplemented the district’s core math program. Math Solutions^® staff provided one-day professional development sessions for teachers, math coaches, and principals, with 43 school staff being trained. Math Solutions coaches visited Do The Math classrooms three times, providing ongoing professional development, in-classroom coaching, and support throughout implementation.

Six hundred ninety-five students in Grades 3–5 who were identified as being in need of math intervention participated in this study. Grade 3 students were identified by receiving a low score on the previous year’s Stanford Achievement Test™. Grade 4 and 5 students were identified if they performed in the lowest two levels on the previous year’s FCAT^® exam.

The demographic characteristics of the students enrolled in Do The Math varied slightly from the demographics of the district. Over half of the students (54%) were African American, 44% were Hispanic and 1% were White. Fourteen percent of Do The Math students were classified with disabilities and 28% were English learners. Nearly all of the Do The Math students (97%) were eligible for free or reduced-price lunch (see Table 2).

Observations, Staff Interviews, and Surveys

Scholastic contracted with an independent research firm to observe Do The Math classes and conduct interviews with teachers, math coaches, and principals. Researchers visited Do The Math classrooms twice during the spring semester, using an observation protocol rubric to capture the essential features of Do The Math and monitor effective teaching skills. Interviews with teachers, math coaches, and principals were conducted at the end of the school year. Teachers completed surveys in winter 2012 and spring 2012 to assess their beliefs about effective math instruction. The spring 2012 survey asked teachers to rate Do The Math in terms of perceived effectiveness in improving students’ math skills.

Math Inventory

Math Inventory^® is a computer-adaptive assessment that measures students’ readiness for math instruction for Grade 2 through Algebra 1. It can be used as a universal screener to monitor progress and inform instruction. Students receive a score on the Quantile^® Framework that shows their current level of math achievement and readiness. These scores also correspond to performance levels for each grade. The performance levels are Below Basic, Basic, Proficient, and Advanced.

Florida’s Comprehensive Assessment Test (FCAT)

Florida’s Comprehensive Assessment Test (FCAT) is a state test for students in Grades 3–10. Students complete the FCAT Mathematics assessment over two days in April. For Grade 3 students, the FCAT Mathematics assessment is a multiple-choice test. For students in Grades 4 and 5, FCAT Mathematics consists of multiple-choice and gridded response questions. Scores are placed in five achievement levels. Level 3 indicates satisfactory performance. Conversion charts allow for results from the 2011 FCAT to be compared to the 2012 FCAT 2.0 format. Table 3 displays the student sample by grade and by test completion.

Teachers completed a log to track the number of lessons completed. The number of lessons completed varied by class; teachers reported completing between 22 and 60 lessons. According to classroom observations, lessons were taught with an average of 76% fidelity based on inclusion program components. Division C was the module observed to be taught with the highest fidelity, 95%.

During interviews, teachers reported that Do The Math positively impacted students’ confidence and math understanding as well as improved students’ grasp of foundational math concepts. About half of the teachers interviewed felt that the games in Do The Math played a large part in student success because students had fun while learning. Half of the math coaches interviewed said that students gained confidence in their ability to succeed in math and other subjects after using Do The Math.

According to a survey, teachers viewed the program components very positively. Teachers most strongly felt that the lessons had clear objectives and were sequenced to help students build on previous skills. Additionally, teachers felt that the lessons were helpful in building computational fluency and focused on building conceptual understanding, while the instructions for teachers were clear and complete and explicit vocabulary instruction improved student understanding of math concepts.

Six hundred thirty-four Do The Math students completed Math Inventory administrations in both winter 2012 and spring 2012. Change in MI performance level can be seen in Graph 1. Three hundred fifty-four students performed at the Below Basic level in the winter, but after interacting with Do The Math, only 240 students performed at Below Basic in the spring. The percent of students performing at Proficient or Advanced levels nearly tripled, from 11% (n=70) in the winter to 31% (n=194) in the spring.

The mean Quantile measure at each grade level shows statistically significant growth from winter 2012 to spring 2012. As seen in Table 4 and Graph 2, students Quantile measure and grade percentile increased significantly after using Do The Math.

One measure for assessing student growth on FCAT Mathematics examines students’ demonstrated learning gains. Students demonstrate learning gains on FCAT by achieving a minimum scale score increase from spring 2011 to spring 2012. As seen in Graph 3, a majority of all the Do The Math students did show growth on the 2012 FCAT. Interestingly, when these results were disaggregated by FCAT Performance Levels 1 and 2, the data reveal that a disproportionately higher percentage of FCAT Level 2 students met FCAT growth expectations. Since students begin taking FCAT Mathematics in Grade 3, the only third graders represented here have repeated the grade.

As displayed in Graph 4, 38% of Do The Math fourth graders who performed at FCAT Level 1 met FCAT growth expectations. By comparison, only 18% of their non-Do The Math counterparts met FCAT growth expectations.

As displayed in Graph 5, 74% of Do The Math fourth graders and 84% of Do The Math fifth graders who performed at FCAT Level 2 met FCAT growth expectations. By comparison, only 35% of non-Do The Math fourth graders and 35% of non-Do The Math fifth graders (performing at FCAT Level 2) met growth expectations. Overall, more than 80% of Do The Math students previously at the Level 2 performance level demonstrated growth on the FCAT math test.

Additionally, 28% of all Do The Math students’ 2012 FCAT math results were one or more performance levels higher than their 2011 FCAT results.

Do The Math students showed significant improvements on Math Inventory testing, with increased percentile rankings and a greater proportion of students performing in the Proficient and Advanced ranges of the test. While Do The Math modules focus on core foundational skills, these improvements on Math Inventory suggest that students were making connections to grade-level content.

In addition, Do The Math students were more likely than their counterparts at the same schools to demonstrate growth on the 2012 math FCAT. While all of the students were enrolled in the same regular core mathematics program, the Do The Math students showed greater gains with the addition of the intervention math program midway through the school year. These results were even more pronounced for students who previously performed in the Level 2 range, with more than 80% of students meeting FCAT Mathematics growth expectations.

Overwhelmingly, teachers, principals, and math coaches reported that Do The Math had a positive impact on their students’ confidence and understanding of mathematics. However, teachers did report that the implementation of Do The Math midway through the school year made for a less perfect implementation. While the significant growth observed with the midyear implementation of Do The Math was impressive, it is likely that the results would have been even greater with a full year of implementation.