Measuring Reasoning Skills Among English Learner Students

Ability tests, as I mentioned in my last post, differ from achievement tests. While achievement tests measure classroom learning, ability tests prompt students to demonstrate thinking they have learned from everyday life, beyond the formal classroom. When we measure verbal, quantitative, or figural reasoning, we ask students to recognize patterns, make inferences about relationships, and use their creative problem-solving skills in each of those content domains. In verbal domains, this means students must apply their existing knowledge of words, infer the meaning of new words, comprehend incomplete verbal information, use context clues, and recognize cognates (words that look similar in different languages).

With the growing number of English learner (EL) students in many schools, educators have been struggling with how to appropriately adapt assessment systems to their needs. When it comes to ability testing, many educators believe they need to switch to a “nonverbal,”—for example, figural or reasoning—task alone. However, evaluating EL students’ ability to reason with language provides useful insight for planning their instruction. The challenge is that we can’t interpret the scores in the same way we do for other students.

When we think about the experience of EL students, it becomes clear that verbal reasoning skills are absolutely critical for them. Short and Fitzsimmons suggest that EL students do “double the work,” because they acquire content knowledge like other students while also developing English language proficiency. The ability to learn new words and grammar involves the same verbal skills discussed before—inferring the meaning of new words, comprehending instruction from incomplete information, using context clues, and recognizing cognates.

So, contrary to popular belief, it is more important to measure verbal reasoning skills for EL students because this helps us recognize when students will struggle more with acquiring English proficiency. It also allows us to provide support and differentiated instruction for these students. That said, obviously a fully English-based test will not be appropriate for EL students in their first few years of English language instruction. However, there are ways to assess verbal reasoning that are language-reduced and ways to interpret the scores of EL students taking English-based verbal reasoning tests that give us this important insight into their verbal reasoning abilities.

Reduced Language Formats

A traditional language-based format may use word problems that are specific to one language. It is not reasonable to simply translate these items, as specific language features and nuances are easily missed in translation.

Language-Based Format:

Apple   Orange   Pear        

A) Fruit    B) Carrot    C) Pea    D)* Lemon    E) Onion

Below is a sample of a picture-based verbal test item like those used in the primary levels of CogAT.  Using picture based items allows students to work with the language in their heads—whether English, Spanish, or another language—to reason through these items. Once they are led through a couple of sample items, students can demonstrate their ability to reason with language without being constrained to a single language—in essence, a nonverbal verbal test.

Picture-Based/Reduced Language Format:

Subgroup Norms

Another way to make appropriate and helpful inferences about verbal reasoning ability for EL students is to compare them only to students with similar opportunity to learn (OTL). These are called subgroup norms. Test norms are the context that allows us to interpret a student’s test performance in comparison to students with similar OTL. Comparing EL students solely to the national norms, based on age or grade peers, misrepresents their ability because they have substantially different OTL from the national norms. For EL students, a more equitable comparison may be made by comparing these students only with other students who have been in the U.S. (or in English-speaking schools) for the same number of years. 

The figure below shows the results of a sample of EL students taking the Verbal Battery of CogAT. For example, the top student in the figure below has a verbal scale of 105. When we compare this student with those of a similar age, this score is only slightly above the national average of 100 (on a scale of M=100, SD=16). It may not seem remarkable in the context of the national norms, but when we compare this student with other EL students who have similar OTL, we realize this is an impressive performance. This student likely acquires language more easily than others and may struggle less with understanding classes taught in English than other students with similar exposure to English. It is only when we compare EL students to similar EL students that we are able to identify those with the strongest and weakest verbal ability. This is not as precise as the national norms, but vastly more useful for planning instruction and identifying students who require more support to acquire English. 

***

If you're looking to implement differentiation in the classroom, you can get a complete picture of students' abilities with the CogAT assessment, which measures their Verbal, Quantitative, and Nonverbal reasoning. Dr. Joni Lakin will also explore CogAT and related topics during free webinars in September and October.