Do The Math: Math Intervention in NYC Schools
At a glance
  • Promising Evidence
  • Subjects: Intervention Curriculum, Math
  • Report Type: Efficacy Study
  • Grade Level: Elementary
  • Region: Northeast
  • District Urbanicity: Urban
  • District Size: Large
  • District: New York City Schools, NY
  • Participants: N=94
  • Outcome Measure: Do The Math Pre- and Post-Module Assessments, Student Surveys
  • Evaluation Period: 2008
  • Study Conducted by: Scholastic Research
Abstract
“One thing I learned is that I take too much for granted in the classroom. I’ve assumed that students’ know things they really don’t. The [Do The Math] program reveals their misconceptions.”—ESL and AIS Math Teacher

In spring 2008, the Scholastic Research and Validation Department collaborated with independent research consultants to conduct a study in six public schools in New York City to gather data and document the impact of implementing Do The Math . Developed by Marilyn Burns and a team of Math Solutions® master classroom teachers, Do The Math is a research-based math intervention program designed to support students who are struggling with elementary arithmetic. With an emphasis on Number and Operations—the cornerstone of elementary math education—the program helps students build a strong foundation in computation, number sense, and problem solving for immediate and long-term learning.

Researchers found that the program could be implemented within various intervention models, including before and after school, pull-out, and self-contained special education classrooms. With regard to impact, researchers found that diverse populations of students, including students with special needs, English learners, and general elementary school students who have been identified as low performing, made gains in their understanding of and skill at performing multiplication. Researchers also discovered that students acquired key academic math vocabulary, and that students’ confidence levels in themselves as math learners improved as a result of their participation in the program. In addition, all participating teachers, regardless of their experience and expertise, were able to implement the program successfully. Teachers also reported that through teaching the program, they gained a deeper understanding of multiplication and learned new strategies for teaching it.

Student achievement data and descriptive portraits distinct to each school provide detailed information about student performance and scenarios of implementation. While each school’s situation is unique and had its share of challenges, the results of the program revealed overall benefits for both students and teachers.

Introduction

At the end of eighth grade, roughly two-thirds of students score at or below basic level as measured by the most recent National Assessment of Educational Progress (NAEP) Mathematics test (Lee, Grigg, & Donahue, 2007) . This downward trend in math achievement can be attributed to curricula that quickly move students through extensive math content in the early grades. In March 2008, the National Mathematics Advisory Panel, charged with providing guidance on how to improve mathematics achievement for all students, released a report that calls for streamlining the curricula to focus on three key foundational topics that are critical for students’ success with algebra. Two of the three foundations, fluency with whole numbers and fluency with fractions, were identified as being the major emphasis for elementary curriculum. Considering that two-thirds of eighth grade students in the United States score at or below basic level as measured by the NAEP Mathematics test, students are not fully prepared to achieve success in algebra.

“Conceptual understanding, computational procedural fluency, and problem solving skills are equally important and mutually reinforce each other.”—National Math Panel, 2008

To compound this challenge, one percent of school-age children have a math disability not associated with any other learning disability, and two to seven percent experience serious math deficits. Students with math disabilities and deficits struggle to perform as well as their peers on basic operations, and the discrepancy between them and their peers increases with age (Cawley, Parmar, Yan, & Miller, 1996 ). Learning math may also present a challenge for many English learners, as this content area possesses its own unique academic vocabulary, which is often presented abstractly.

One of the goals articulated by the National Council of Teachers of Mathematics in their Principles and Standards For School Mathematics (2000) is that all students become mathematical problem solvers, learn to communicate and reason mathematically, use representations to model problem situations, and make connections among mathematical ideas. In addition, the National Mathematics Advisory Panel (2008 ) recommends that math curricula for elementary and middle school be a coherent progression of key foundational topics with an emphasis on proficiency within these key topics. For students who struggle, meeting these goals is challenging, especially if they only receive the 50 minutes that schools generally dedicate to math instruction each day. These students need instruction that helps them build the foundational mathematical concepts that they are missing.

Do The Math aims to address these learning challenges that students face. The program’s instructional design applies what is known about reaching a wide variety of students who struggle with math to achieve proficiency with arithmetic concepts and skills. This report shares the results of a research study that was conducted on the implementation and impact of Do The Math during the spring of 2008 in six New York City schools.

Do The Math

Do The Math is a research-based arithmetic intervention program that will help educators reverse the downward trend in mathematics achievement. Developed by Marilyn Burns and a team of Math Solutions master classroom teachers, the program gives students who are at risk of falling behind or who have already fallen behind the chance to catch up and keep up with grade-level content. The program carefully and intentionally scaffolds four key topics (Addition & Subtraction, Multiplication, Division, and Fractions) to build fluency with whole numbers and fluency with fractions, both of which are critical foundations that prepare students for success with algebra.

Do The Math gives students who have fallen behind the chance to catch up and keep up by offering:

  • carefully scaffolded and sequenced content, in small chunks and with appropriate pacing that allows students to build conceptual understanding and skills;
  • instruction that gradually moves students from explicit instruction, guided practice, and cooperative pair work to working independently;
  • visual directions that support students who may not read well or who are developing English skills;
  • consistent routines of having partners work together to solve problems, communicate their ideas, and explain their reasoning;
  • explicit instruction that presents carefully sequenced experiences through which students develop understanding of concepts, learn skills, see relationships, and make connections among mathematical ideas;
  • concrete manipulatives and models to help students build understanding and practice skills;
  • games and literature that engage and motivate students, providing them with opportunities to strengthen and reinforce their learning;
  • carefully selected vocabulary and simple sentence structures that support students’ understanding of the mathematics;
  • explicit instruction based on the  see it, hear it, say it, write it, read it routine for learning mathematical vocabulary; and
  • access to materials that teachers can use immediately to meet students’ diverse needs.
Math Solutions Professional Development

Founded by Marilyn Burns, Math Solutions offers professional development in several formats for teachers who are implementing Do The Math. For this research study, Math Solutions provided two days of professional development with the goal of preparing the teachers to implement the program, make them aware of the instructional strategies built into the Do The Math lessons, and to the uses and management of the program.

The Study

Research was conducted on the implementation and impact of Do The Math in six schools in New York City between January 30  and June 15 of 2008. Scholastic partnered with the New York City Department of Education (NYC DOE) to select schools that would represent the city’s diverse student population and where the program could be implemented with fidelity. Half were general education elementary schools, and half were within District 75, a district that serves students with special needs. Entire classes or small groups of students were instructed using one of two Do The Math multiplication modules. Multiplication A focuses on the basic concepts underlying multiplication, and Multiplication B focuses on multiplication facts through 12 x 12. Administrators, math coaches, and in some cases Academic Intervention Specialist (AIS) coordinators, often in collaboration with teachers, selected those students who were most in need of additional instruction and support in math to receive the treatment. The fourteen participating teachers received the two-day professional development introductory course provided by Math Solutions.

Research Questions

The purpose of the research was twofold: First, to determine the impact of Do The Math on students from diverse populations, including special education, English learners, and general education students in Grades 3 through 6; and second, to document student attitudes and teacher perceptions about math as a result of the treatment. The research sought to address these four questions:

  1. Does using Do The Math Multiplication A or B result in improved performance in math achievement and vocabulary?
  2. Do students’ dispositions toward learning math and their confidence in doing math improve as a result of their participation in Do The Math?
  3. Is using Do The Math effective in a variety of settings and with a variety of implementation models?
  4. What effect does using Do The Math have on teachers’ math knowledge and instructional practices?
Study Participants and Implementation
Schools

There were three criteria for selecting the six case-study sites. Researchers first looked for sites where there would be a significant number of students with special needs, students who are developing English skills, and/or students who have been identified as low performing. Secondly, researchers sought to select sites that would represent a geographic dispersion among the five boroughs of New York City (four of the five boroughs are represented in the study). Lastly, researchers wanted sites that would be representative of the different implementation models typically used for intervention. For specific details on each of the six schools, see the Appendix.

In collaboration with the NYC DOE, three general education elementary schools, two of which serve a large number of English learners, and three District 75 schools that serve special education students were selected. All three of the special education schools chosen serve students in Kindergarten through Grade 8 and have a population of students who are designated as “ungraded.” Four of the five boroughs of New York were represented by two schools in Manhattan, one in Brooklyn, one in Queens, and two in the Bronx.

Teachers

From the six schools, fourteen teachers were chosen to teach Do The Math to either small groups or entire classes of students. The teachers implementing Do The Math have a wide range of experience and expertise and hold various positions in the school system.Within the group, there is a school math coach, an Academic Intervention Specialist (AIS) coordinator, three AIS math teachers, an English as a Second Language (ESL) teacher, two general education classroom teachers, a special education math coach, a special education AIS teacher, and four self-contained special education teachers. The experience among the group of teachers ranges from a first-year teaching fellow to a 30-year veteran. Their expertise in math also ranges from a math coach who has attended numerous professional development workshops focused specifically on teaching math, to an ESL teacher who normally doesn’t teach math. All teachers participated in the two-day professional development course on Do The Math conducted by a Math Solutions master classroom teacher. The first day of training was held on January 30, 2008, and the second on March 18, 2008.

Students

In all six schools, students selected to participate in Do The Math were those who were struggling with elementary arithmetic. Administrators and teachers worked together to review the results of multiple assessments, including state test score data, periodic assessment data, and in most cases, teacher recommendation and judgment to select the students. These students then took the Beginning-of-Module Assessment included in Do The Math Multiplication A. If a student’s score was greater than 80% on the assessment, the student was placed in Multiplication B.

The study was conducted on 94 students across the six schools. The sample was divided into students in the general education schools who participated in Do The Math (54 students or 57% of the sample) and students in the special education schools (40 students or 43% of the sample). Seventy-eight students (82% of the sample) participated in Multiplication A and 16 students (18% of the sample) participated in Multiplication B. The sample of students was ethnically diverse (Graph 1). Also, 60% were boys, and 83% of the total students were eligible for free or reduced lunch (Table 1).

3.13 Table2 1
GRAPH 1. Student Race/Ethnicity
3.13 Table1
Implementation Models

While all participating students received the same dosage of Do The Math (one module consists of thirty 30-minute lessons), three different implementation models are represented within the six schools in the study. They are: 1) before and after school, which in New York City is referred to by the Department of Education as “Extended Day” and conducted for 37.5 minutes either before or after the school day begins or ends; 2) pull-out, which occurs at designated times during the school day; and 3) self-contained special education classrooms, which means that the program is implemented during regular school hours within the classroom. Typically with before or after school, students receive the intervention from an Academic Intervention Specialist (AIS), a math coach, or a regular classroom teacher. In a pull-out program, students are pulled out of their regular classrooms at specified times. They generally work with an AIS teacher or math coach in a space or classroom separate from their regular classroom. In some models, the AIS teacher(s) works with groups of up to 24 students; in others, he/she works with a small group of eight students or fewer. In a self-contained classroom, the regular classroom teacher teaches the program to either the whole group of students or a small group within the classroom (Table 2).

3.13 Table3 1
Data Collection and Measures

The independent research consultants designed and customized research instruments to learn as much as possible about each of the six schools. They collected results from the program’s Beginning-of-Module Assessment (pretest) and End-of-Module Assessment (posttest), administered student surveys, and conducted classroom observations. Formal interviews with the teachers and informal interviews with students provided information beyond the assessment results. Researchers analyzed all of the data collected to draw conclusions about the impact of the program’s implementation.

Student Achievement Data

Teachers administered the Beginning-of-Module Assessment for either Multiplication A or Multiplication B, and the End-of-Module Assessment after completing all thirty lessons. Even though the questions on the Beginning-of-Module Assessment and the End-of-Module Assessment differed slightly, they measured the same set of specified multiplication objectives.

Multiplication A Objectives:

  • Calculate products with factors 1 through 6;
  • Represent combining equal groups with related addition and multiplication equations;
  • Write a multiplication equation for a word problem;
  • Write a word problem for a given multiplication equation;
  • Calculate the product when one factor is 0; and
  • Apply the Commutative Property of Multiplication using factors 0 through 6.

Multiplication B Objectives:

  • Calculate products with factors 0 through 12;
  • Represent arrangements of equal rows and rectangles with multiplication equations;
  • Use the Commutative Property of Multiplication to solve problems; and
  • Recall products for facts through 12 x 12.
Student Surveys

The student survey consists of six multiple-choice questions that focus on students’ disposition toward math, confidence levels in themselves as math learners, and beliefs in the importance of math in terms of their future success. One question asks students about their preference regarding the way they like to work in math class (small group, with a partner, by themselves, using a computer, listening to teachers, or whole-class discussions). Two open-ended questions ask students to share what they like best and least about math. The same survey was administered before the implementation period began and after it ended.

Site Visits

During the months of February, March, and April 2008, researchers conducted multiple site visits to each one of the six schools. During visits, researchers observed teachers teaching and students participating in Do The Math. Observations took place wherever teachers at each site normally taught the program, be it before, during, or after school. After the observation, researchers conducted formal interviews with the instructor. In many instances, researchers also talked informally with students about their experiences.

Data Analysis

Researchers conducted paired t-tests and ANOVA (Analysis of Variance) to determine the statistical significance of the change in scores between the Beginning-of-Module Assessment (pretest) and the End-of-Module Assessment (posttest). Then they analyzed and compared the pretest and posttest scores with students’ responses to the two administrations of the student survey to determine how they statistically correlate.

Key Findings
Research Question 1: Does using Do The Math Multiplication A or B result in improved performance in math achievement and vocabulary?

Analysis of students’ results on the Beginning-of-Module Assessment (pretest) and the End-of-Module Assessment (posttest), along with data from the classroom observations and teacher interviews, reveals that students experienced the following with regard to impact:

  1. gains in multiplication; and
  2. acquisition of key academic vocabulary in math.
“I have noticed that it [Do The Math] alleviates anxiety for some of the students. They will say to one another, ‘Just use Circles and Stars.’ They now have tools and strategies that they can use to solve problems.”—Grades 3, 4, and 5 combined self-contained Special Education teacher, Queens

A. Students made gains in multiplication that were statistically significant.

In all six schools, treatment students, including English learners, students with special needs, and general education students, made gains in multiplication that were statistically significant (t=11.45, p<0.001). The results of the 20-item Beginning-of-Module Assessment (pretest) and End-of-Module Assessment (posttest) revealed that whether students received instruction in Multiplication A (n=78) or Multiplication B (n=16), whether they were in a general education school (n=54) or a special education school (n=40), or whether they received treatment in a before or after school program (n=31), pull-out situation (n=23), or a self-contained special education classroom (n=40), they made significant gains (Graphs 2-5).

As shown in Graph 2, students made gains in each school that were statistically significant: Washington Heights (t=8.72, p<0.001), Brooklyn (t=7.01, p<0.001), UWS (t=5.37, p<0.001), Bronx Special Education #1 (t =4.19, p<0.001), Queens Special Education (t=3.67, p<0.001), and Bronx Special Education #2 (t=3.33, p<0.001). Mean gains ranged from 2.7 to 6.8 points on a 20-point scale. A significant finding was that the greatest overall mean gains (6.8 points) made by students occurred where teachers pulled small groups of students out of regular classroom settings to receive Do The Math instruction.

3.13 Graph1
GRAPH 2. Mean Gains Between Pretest and Posttest Scores
3.13 Graph2

GRAPH 3. Comparison of Raw Scores for Pretest and Posttest Scores

Note. Graph 3 shows that the End-of-Module Assessment mean results ranged from 15.3 questions correct to 17.7 correct out of 20 questions. The differences in pretest scores by school were statistically significant (F(5, 94)=4.85, p<0.001).

Overall, the 78 students who participated in Multiplication A made slightly higher gains (mean gain of 4.3 points) than the 16 students who participated in Multiplication B (mean gain of 3.1 points). However, because of the large difference in sample sizes, the small sample in Multiplication B, and the slight difference in the Multiplication A and Multiplication B assessments, any comparisons drawn between the two groups should be viewed with caution. The sixteen students who received Multiplication B attended two out of the three general education elementary schools.

3.13 Graph3
GRAPH 4. Comparison of Mean Gains of Students’ Posttest Scores for Multiplication A and Multiplication B

The pretest and posttest data for students who participated in Multiplication A and Multiplication B reveal roughly a one-point difference in gains made (Graph 4). Differences in the Beginning-of-Module Assessment (pretest) results by module were not statistically significant, suggesting that both groups began with relatively equivalent scores on their respective pretests. Pretest and posttest averages by module are presented in Graph 5.

3.13 Graph4
GRAPH 5. Comparison of Average Gains in Raw Scores Between Multiplication A and Multiplication B Pretest and Posttest

Disaggregating the results by general education and special education schools reveal that significant effects persisted for both groups (Special Education: t=6.33, p<0.001; General Education: t=9.79, p<0.001). Students in the general education schools made slightly higher gains than students in special education schools–approximately 26 %growth as compared to 17 % growth (Graphs 6 and 7). Overall, students in the three general education schools achieved a mean gain of 5.2 on a 20-point scale, and students in the special education schools achieved a mean gain of 3.4 points on a 20-point scale. Only the results for Multiplication Module A are represented in Graphs 5 and 6, as no students in the special education schools received instruction in Multiplication B.

3.13 Graph5
GRAPH 6. Comparison of Mean Gains of Students’ Posttest Scores for Multiplication A between Special Education and General Education Schools

The pretest data reveal that students in the general education schools scored lower than the students in the special education schools. While the differences in pretest scores were not statistically significant, the trend revealing an average lower performance for the general education students may be due to the fact that those students were in earlier grades (third, fourth, and fifth) as opposed to the students in the special education schools, two-thirds of whom were in sixth, seventh, and eighth grade.

3.13 Graph6
GRAPH 7. Comparison of Average Gains in Raw Scores for Multiplication A between Special Education and General Education Schools

Teachers in special education schools confirmed the positive results with anecdotal evidence about individual students. Many of these students were initially unable to complete independent work. Teachers relayed that with Do The Math several of these students were able to work independently, some for the first time.

In both the general and special education schools, teachers attributed students’ overall success to having gained access to new, comprehensible, easy to implement strategies to solve multiplication problems, such as Circles and Stars. Teachers also credited students’ achievement to the ongoing success they experienced while participating in the program. When armed with the new strategies, students were able to solve problems correctly and efficiently, and thus, they grew increasingly more confident. As a result, according to teachers, many students who previously had not liked math not only began to succeed but also began to enjoy doing math.

“The specific detailed steps gave him the language and tools he needed to articulate what he was learning. When he couldn’t express himself he would open up his book, point to examples, and with support, tell us what he was thinking.”—Special Education district-level math coach

B. Students acquired and utilized the key academic math vocabulary presented in the program.

A qualitative analysis of the teacher interview data reveals that students who participated in Do The Math acquired the key math vocabulary presented in the program and began to communicate using those vocabulary words. These interview findings were consistent with data obtained from classroom observations in that the researchers also noted that students were using words such as factor, product, equation, and Commutative Property, correctly and with regularity, both when they participated in the lesson, while playing the games, and when they completed independent work.

Several teachers noted that they did not realize before teaching Do The Math how important it is to explicitly teach key math vocabulary. According to teachers, it was the consistency and structure of the program that supported students in achieving fluency with the vocabulary words and mastering the underlying concepts. Teachers at schools with large populations of English learners were especially appreciative of the program’s emphasis on vocabulary and language development.

“One of the things we learned is that the students need language development. They don’t know the words ‘equation’ and ‘sum,’ for example. The teachers didn’t know that the students didn’t know these terms until they started using the [Do The Math] program.”—ESL teacher, Brooklyn

Multiplication A Key Math Vocabulary:

  • Commutative Property of Multiplication
  • Equal
  • Factor
  • Multiplication
  • Multiplication equation
  • Multiply
  • Product
  • Times

Multiplication B Key Math Vocabulary:

  • Multiplication equation
  • Factor
  • Product
  • Commutative Property of Multiplication
  • Square number
  • Multiple
  • Zero Property of Multiplication
Research Question 2: Do students’ dispositions toward learning math and their confidence in doing math improve as a result of their participation in Do The Math?

An analysis of pre- and post-surveys was conducted to reveal results about students’ attitudes toward learning math, confidence in their own math abilities, and their perceptions of themselves as math learners. Surveys were distributed to all teachers; however, pre- and post-survey results were collected for only 25 student participants attending three of the six participating schools. The three schools are: Brooklyn Elementary School, Upper West Side Elementary School, and Queens Special Education School. Teachers from the remaining three schools administered the student surveys, but only at one time-point, which was either before the implementation period or afterward.

The analysis of the responses from the 25 surveys revealed that students’ confidence improved from the time they began to participate in the program until they completed it (Graph 8). In interviews, teachers also reported observing a rise in students’ confidence levels. In fact, several teachers in both the general and special education schools described watching Do The Math students actually teach fellow classmates, who were not in the program, strategies such as Circles and Stars and games like Capture and Pathways.

3.13 Graph7 1

GRAPH 8. Comparison of Responses to Survey Item 2* Before and After completing Do The Math (N=25)

Little change was found in regard to students’ attitudes toward learning math and their beliefs about whether good math skills will help them to get good jobs and go to college, but it should be noted that student responses in the pre-survey generally were positive—all participants reported that they like math at least a little. Almost 65% of students also reported that their favorite way to work in math class is working with others (22% prefer to work with a partner, and 23% prefer to work in a small group). Working in a small group and with a partner are at the core of Do The Math’s gradual-release process.

“The [Do The Math] program empowered kids who really struggle with mathematics and have for years. It gave them confidence.... The program enabled them to become more involved and understand the concepts behind what has previously been a blur of ‘facts’ and misunderstanding. It was eye opening and a welcome relief to them to actually finally understand and enjoy what they were doing.”—Special Education district-level math coach
Research Question 3: Is using Do The Math effective in a variety of settings and with a variety of implementation models?

Analyses of the pre-post data, classroom observations, and teacher interviews revealed that Do The Math could be implemented with success using various implementation models. Across the six schools, three models were represented including before or after school, extended day, pull-out, and special education self-contained classes. Students in all three of the intervention models made positive gains on the End-of-Module Assessment. Students who participated in a pull-out model at the Washington Heights elementary school made the greatest gains (6.8 points on a 20-point scale–Graph 2). At this site, AIS teachers pulled out small groups of four to six students during the school day. These groups represented the fewest number of students per group in the study. Students therefore may have received more individualized attention that addressed their specific needs. Teachers at all six schools implemented Do The Math as an intervention for both Tier 2 and Tier 3 students within a Response to Intervention (RTI) Framework .

According to teachers, Do The Math was easy to implement because New York City’s Extended Day program provides 37.5 minutes per day to be dedicated to instruction for those students who need it. Teachers found that because each Do The Math lesson is 30 minutes long, they could teach one lesson a day and use the remaining 7.5 minutes for setup and cleanup. Some teachers mentioned that in the case of some lessons, they would have liked to have had more than 30 minutes for children to practice the skill taught, or play a game that had been introduced.

In the special education self-contained classrooms teachers reported that certain students required more time to master key math concepts or skills. In general, special education teachers struggled more than those teachers in the general education schools to carve out the time necessary to implement the program because most special education schools did not have an Extended Day program, and the math block during the school day was designated for teaching the district–selected core math program. To solve this implementation issue, at some schools teachers required the support of administrators to adjust schedules so the program could be implemented.

“I enjoyed and appreciated the philosophy behind Do The Math. Struggling students need scaffolding, explicit instruction, and useful and fun strategies that enable them to learn. The strategies and games, such as ‘Circles and Stars,’ ‘Where the Lines Cross,’ and ‘Pathways,’ were engaging for both teachers and students.”—School-based math coach, Upper West Side
Research Question 4: What effect does using Do The Math have on teachers’ math knowledge and instructional practices?

A qualitative analysis of the teacher interview data showed that teachers improved their own knowledge of multiplication and learned new instructional strategies for teaching it to struggling students. After they received the Math Solutions training and taught Do The Math, several teachers said they deepened and expanded their own knowledge of the underlying concepts of multiplication. Many expressed that teaching the program also helped them make the connections between math concepts and procedures. Several also noted that they greatly valued learning new strategies to teach struggling students.

One teacher in particular said that teaching the program raised her self-esteem. She explained that in the past, she has tried to help students memorize their facts, and when they weren’t able to, she felt as though she was failing. With Do The Math, she learned many different strategies that worked for her students. Other teachers also reported that by teaching the program, they gained a deeper understanding about struggling students’ misconceptions.

In addition, teachers reported finding the lessons easy to teach and the program easy to use. They felt that the content was structured in a manageable and comprehensible way, ensuring that they could grasp the math concepts before teaching them. While Scholastic offered professional support throughout the implementation period, few questions arose and when they did, they were easily resolved.

“It [Do The Math] was instrumental support for my teaching. I also learned a lot about multiplication myself. I am not a math person. It was fun learning, alongside the students, the many relationships between numbers through the lens of multiplication. I also realized how [the lack of ] English language proficiency can limit an enhanced number sense.”—ESL teacher, Brooklyn
Implementation Challenges

Researchers noted two challenges associated with the implementation of Do The Math. As with many schools across the country, when it comes to implementing an intervention program, challenges revolve around issues common to supporting struggling students, such as lack of time for intervention and high rates of absenteeism.

Time for Intervention
“The challenges were the same ones we find with all math instruction–time, teacher preparation, teacher understanding, carefully listening to students–but having a program that is so specific really helps.”—District-level math coach

While overall all six schools were able to create effective models of use and implement the program with fidelity, several teachers expressed feeling challenged by having to teach the program on a consistent basis (30 minutes a day for at least four days a week). Even though extended day before and after school programs ensure that a 37.5-minute time period is designated for teaching Do The Math, teachers reported that there were a number of days when the extended day program was cancelled or replaced by another event.

Challenges also exist with the pull-out model. In one case, a teacher resisted having her students pulled out of the classroom. This challenge is not atypical when it comes to pulling students out of the classroom to receive intervention; classroom teachers worry that those students who are pulled out will miss valuable content and that they will be treated differently by their peers. These concerns are exacerbated by the fact that the students who are being pulled out for Do The Math are often the same ones who are being pulled out to receive intervention in other subject areas, and consequently already contending with these issues.

Absenteeism

Unfortunately, a large proportion of students who are identified as at-risk or who struggle academically are those who are also absent frequently. In particular, high rates of absenteeism present a chronic problem within the special education schools. Teachers therefore felt challenged when it came to implementing the program with fidelity. In the case of the school where they implemented Do The Math within the extended day before school program, teachers grappled with the erratic attendance of some students, making it challenging to keep all of the students on the same page. Teachers notified parents about the importance of their child attending everyday, and eventually dismissed those students who were not attending the program regularly.

Conclusion

This Do The Math research study reveals positive results for students who struggle with elementary math and for the teachers who work with them. The four-month study showed that diverse populations of students in Grades 3 through 6 made statistically significant gains on the program’s End-of-Module Assessment. They also acquired the key math vocabulary presented in the program. In addition, it showed that students’ confidence in themselves as math learners improved from the time when they began the program until they finished it.

The study also revealed that Do The Math could be implemented effectively within various intervention models, including before and after school, pull-out, and self-contained special education classrooms, and employed as a Tier 2 or 3 intervention within a Response to Intervention (RTI) framework. It found that teachers who held different positions and embodied a wide range of experience and expertise could implement the program successfully. Moreover, through teaching the program, teachers said that they gained a deeper and more thorough understanding of multiplication and learned new strategies to teach it.

The results are promising for schools, teachers, and students searching for a research-based intervention program that supports struggling students to become proficient in elementary mathematics.

References
  • Cawley, J., Parmar, R., Yan,W. F., & Miller, J. (1996). Arithmetic computation performance of students with learning disabilities: Implications for curriculum. Learning Disabilities: Research and Practice, 13(2), 68–74.

  • Lee, J., Grigg,W., & Donahue, P. (2007). The Nation’s Report Card: Reading 2007. Washington, DC: National Center for Education Statistics, Institute for Education Sciences, U.S. Department of Education.

  • National Council of Teachers of Mathematics. Principles and Standards For School Mathematics (2000). Reston, VA. http:/www.nctm.org

  • National Mathematics Advisory Panel (2008). Foundations for Success: The Final Report of the National Mathematics Advisory Panel. Washington, DC: U.S. Department of Education.

Appendix: School Implementation Portraits

The following school portraits offer more detailed information about each of the six school sites involved in the study:

  • The school context, including demographic data;
  • A description of the implementation scenario (e.g., before or after school, pull-out, or self-contained special education classroom); and
  • Teachers’ perceptions of the program’s impact.

Each school selected the implementation model that best aligned with the structure already in place. The requirement was that participating students receive the recommended schedule of thirty 30-minute lessons either four or five days a week.

General Education Schools

UPPER WEST SIDE ELEMENTARY SCHOOL

Located on the Upper West Side in Manhattan, this school enrolls students in Kindergarten through Grade 5, and serves students who represent a wide array of ethnicities, cultures, and home languages.

3.13 P24table Updated

Implementation:

The school’s math coach and two fifth grade teachers taught Do The Math to three separate groups of students after school. The math coach taught six third grade students using Multiplication A. The two fifth grade teachers taught the program to a total of 14 students (one teacher taught 10 students using Multiplication B and her partner taught four students using Multiplication A).

The three groups received Do The Math instruction for 30 minutes a day, four days a week. After teachers taught the first module, they moved on to teaching the next module (either Multiplication B or C). On several days, Extended Day was either cancelled or replaced by a special event or assembly.

Impact:

Teachers at the school were impressed with the structure of the program. They appreciated how the content was gradually released so that struggling students could comprehend, practice, and master math concepts. The math coach claimed that he gained a deeper understanding of what struggling students need through teaching the program.

WASHINGTON HEIGHTS ELEMENTARY SCHOOL

Located in Washington Heights, a neighborhood in Manhattan where the population is predominantly Hispanic, this school serves students in pre-K through Grade 5. Many students come from homes where Spanish is the first language. To serve students, the school offers dual-language classes, which are conducted in Spanish and English.

3.13 P25table Updated

Implementation:

The AIS coordinator worked with two AIS math teachers to select 23 students in fourth and fifth grade who were struggling with math. They reviewed students’ results on the state math test and on some of the periodic assessments administered by teachers. They then divided students into four small groups of approximately six students each. Students from the two different grade levels were combined within each small group. Three out of the four groups began in Multiplication A and the remaining group in Multiplication B. During first period, the AIS coordinator taught the group that started in Multiplication B, and the AIS math teachers each taught a group in Multiplication A. One of the AIS math teachers also taught Module A to a small group of students during the final period of the school day.

The AIS coordinator and teachers pulled students out of their classrooms to receive Do The Math for the prescribed 30 minutes a day, five days a week. All three teachers completed the thirty lessons in the first module they taught and went on to teach the following module (either Multiplication B or C). Before they started the program, scheduling proved a challenge as many of the students who were identified to receive the program were already being pulled out to receive additional support in either literacy or English. One classroom teacher was uncooperative because she did not want students pulled out of her class. The AIS coordinator remained sensitive to this issue.

Impact:

Overall, students who participated in Do The Math at this school made the greatest mean gains on the End-of-Module Assessment. Interestingly, it was also the only school where teachers pulled out small groups of six or fewer students in fourth and fifth grade to provide instruction. The teachers who taught the program felt that mixing students from different grade levels was beneficial. According to one, “Everyone is doing the same thing. It’s great to see kids from different grades working together.” A few, who usually struggle within their own classrooms because they are performing at a lower level than their peers, were able to feel successful and ultimately more confident.

“It gave them an in-depth understanding of multiplication. Additionally, it gave them a way of thinking about numbers in a more systematic fashion that they could utilize with other types of math challenges.”—Math Intervention Specialist

BROOKLYN ELEMENTARY SCHOOL

Located in Brooklyn, this neighborhood school enrolls students in pre-K through Grade 5. The surrounding neighborhood is ethnically diverse; the majority of students speak a language other than English at home. To support the large population of English learners, administrators hired several full-time English as a Second Language (ESL) teachers who support regular classroom teachers by working with students right in their classrooms.

3.13 P26table Updated

Implementation:

To select students to participate in the program, the school’s AIS math teacher and ESL teacher identified fourth and fifth grade students who were most in need of additional support in math. To do so, they reviewed students’ state test results. Once the group was established, they taught Do The Math within the Extended Day 37.5-minute period that is held everyday before school. The two teachers advised parents that their child had been selected to participate in the program, so parents and students would know to arrive 40 minutes before the school day began.

The AIS math teacher and the ESL teacher co-taught Do The Math every morning for 30 minutes a day, five days a week. Students completed the lessons in both Multiplication A and B (only the results for Multiplication A were analyzed). The first challenge the teachers experienced in terms of implementation was locating an available space where they could teach the class. Once they secured the library, the next challenge was how to deal with the erratic attendance of some students. After a few weeks, teachers decided that students who were not attending regularly would no longer be allowed to participate. A total of 11 students remained in the program through its completion.

Impact:

The teachers who co-taught Do The Math most appreciated how the program revealed students’ misconceptions about multiplication, lack of number sense, and struggles with underlying math concepts. Both teachers felt that they gained a deeper understanding of what struggling students need, as well as math strategies they can use. According to the teachers, the program had a positive impact on the majority of student participants. One teacher described the positive changes she witnessed in one student in particular: “He resisted in the beginning of Do The Math by showing up late and having sporadic attendance. Recently at the end of the program, he came up to me to show me the results of his latest math test. He did very well and said he realized he’s not afraid of numbers anymore.” This same student’s mother wrote a note to the school’s assistant principal asking that he continue to receive Do The Math instruction.

Special Education Schools

QUEENS SPECIAL EDUCATION SCHOOL

Even though the school is located in the southern part of Queens near the Brooklyn border, most students are bused in from other parts of New York City. The school enrolls students in pre-K through Grade 11 and some students who have been designated as “ungraded.” Students in the elementary grades participated in the program.

3.13 P27table Updated

Implementation:

Do The Math was implemented in two third, fourth, and fifth grade combined self-contained classrooms everyday for 30 minutes at the end of students’ lunch period and during the beginning of the following period. The district math coach helped teachers to carve out time within their daily schedules, as the math block is reserved for teaching the district-selected core math program. Teachers continued to teach the core math program mandated by the district and used Do The Math to provide additional instruction.

The two participating classrooms have a 6:1 student teacher ratio, but the needs of the students in each classroom span a broad range. Teachers therefore found it challenging to keep all of the students on the same page; some students wanted to move faster, and others needed more help. Furthermore, the school wrestles with high rates of absenteeism, making it additionally hard for teachers to keep all of the students together.

Impact:

Nonetheless, the two teachers who taught Do The Math valued the fact that some of the students in their classrooms, who typically struggle to work independently, could complete the tasks and assignments within the program by themselves. One of the teachers observed, “Students are able to participate in all aspects of the program. They are receptive on all levels.” Additionally, teachers found that the students made connections among math concepts that they had never made before.

“The kids learned that multiplication is repeated addition. They are making the connection between addition and multiplication. The students brought up the Commutative Property. They made the connection themselves.”—Special Education teacher, Grades 3, 4, and 5

BRONX SPECIAL EDUCATION SCHOOL #1

This school is located in the South Bronx. Many students who attend come from the surrounding neighborhood, but a large number are bused in as well. The school enrolls students in Kindergarten through Grade 8 and some who are designated as “ungraded.” Administrators and coaches decided to implement Do The Math in classrooms and with students in sixth, seventh, and eighth grade.

3.13 P28table Updated

Implementation:

The school’s math coach worked with the principal to select two classrooms where they believed that the teachers and students involved would benefit from using and participating in Do The Math. Both of the teachers who were chosen teach sixth, seventh, and eighth grade combined self-contained classes.

Teachers selected students who needed the most support in math and were not being pulled out of the classroom to receive other services at the time when they could teach the program. The math coach also pulled out students from other classrooms who were experiencing serious difficulties with multiplication in order to provide them with the opportunity to participate in the study.

Impact:

Teachers appreciated the multiple strategies that students learned in Do The Math. The teachers reported that students loved the games and were completely engaged when playing them. Furthermore, teachers claimed that students used the strategies they learned, such as Circles and Stars, to solve problems they encountered outside of the program. One teacher shared that she even witnessed one student using it when working to solve a multiplication problem on the state math test.

“Some of the kids said that the program was too easy, but I observed those same kids using some strategies they learned in the program when they took the test (the state math test).”—Special Education teacher, Grades 6, 7, and 8

BRONX SPECIAL EDUCATION SCHOOL #2

This school is also located in the South Bronx. It is a middle school that serves students who struggle to function within District 75  because they require specialized support. The school enrolls some of the most severely emotionally disabled students in the New York City public school system.

Implementation:

The AIS math teacher taught Do The Math in two sixth, seventh, and eighth grade combined self-contained classrooms for 30  minutes a day. While scheduling proved relatively simple due to the existence of a period designated specifically for AIS math, dealing with behavioral issues and student absenteeism proved to be a significant challenge in terms of implementing the program with fidelity. Nonetheless, 15 students were able to receive the program on a consistent basis and achieve success.

Impact:

Across the two classes that received Do The Math, students’ abilities and behavior ranged significantly. For several students, it is a challenge to simply remain in their seats for a concentrated period of time. Researchers observed that the majority of the students were not only engaged during Do The Math lessons, but were also able to make connections between the math concepts and the mathematical procedure, as well as work independently in their workbooks. According to the teachers, students also felt ownership over their workbooks.

“The greatest impact it had was it, for the most part, caught students up while keeping others engaged.”—District-level math coach

In 2015, HMH acquired Scholastic Education, including Do The Math.

Response to Intervention (RtI) is the practice of providing high-quality instruction and intervention matched to student needs, monitoring progress frequently to make decisions about changes in instruction or goals, and applying student response data to implement educational decisions.