This is an open access article distributed under the terms of the Creative Commons
Attribution License (Phonological Processing and Working Memory in Early Numeracy Development
The National Center for Education Statistics reported that there were approximately 5.1 million students classified as English Learners (ELs) in public schools in 2019, which accounts for 10.4% of the population of students attending public schools in the United States (NCES, 2022). Using evidenced-based practices to meet the needs of multilingual students is essential, as only 14% of students classified as English Learners in 4th grade and 4% of students classified as English Learners in 8th grade have scored at or above proficient levels in mathematics per the National Assessment of Educational Progress compared with 40% of 4th graders and 29% of 8th graders not classified as English Learners (NCES, 2022). Mathematics outcomes for English Learners are more complex than indicated by NAEP data. Given the rapid development of English skills for those classified as English Learners and the current understanding of mathematics development of multilingual students, further analysis of the interaction between math achievement and linguistic proficiency in early elementary grades could provide essential foundational knowledge that could be used to support the mathematics development of multilingual students. While the rate of improvement in math outcomes indicates a positive trend for students classified as English Learners in the United States, there is a subgroup of students, those at risk for math learning difficulties, that require additional instructional supports to reduce the probability of future difficulties in mathematics.
Students classified as English Learners will be referred to as emergent bilinguals throughout the present study because participating students are in kindergarten. They are learning multiple language systems simultaneously; thus, they are not yet fluent in either their home language or the language of instruction. When providing instructional support to emergent bilinguals in early grades, educators must consider factors related to both second language acquisition and cognitive development. Further, within the field, numerous terms have been used to label students struggling to gain proficiency in mathematics. Generally, the term math difficulty refers to students with either an identified mathematics learning disability (i.e., dyscalculia) or low performance in mathematics and no identified mathematics learning disability (Nelson & Powell, 2018). In the United States, students with specific learning disabilities are often identified in later elementary grades, thus, students demonstrating risk for mathematics difficulties in kindergarten are often considered to be at risk for math difficulty (O’Connor et al., 2013).
Kindergarten emergent bilinguals receiving instruction in English learn new early numeracy skills in their second language, while simultaneously learning their second language. This creates additional challenges for emergent bilingual students at risk for math difficulty because the development of mathematical skills is supported by each student’s current level of proficiency in the language of instruction (LeFevre et al., 2010). For this reason, it is imperative that further research is conducted to inform the design and delivery of mathematics interventions to specifically support the development of early math skills for students who both are receiving math instruction in their second language and demonstrating risk for future difficulties in mathematics.
Literature Review
Development of Language and Early Math Skills for Emergent Bilingual Kindergarteners
Language development and the acquisition of early math skills are inextricably linked for emergent bilinguals in the United States, especially in kindergarten, which is the first time that most emergent bilingual students receive both formal instruction in mathematics and instruction provided in English. During this critical period of mathematics development, kindergartners develop number sense, which includes the development of counting principles and cardinality, magnitude comparison, and the ability to compose and decompose numbers (Jordan et al., 2010). Number sense skills in kindergarten and at the beginning of first grade have been linked to math achievement in third grade, therefore, the development of these skills is critical for emergent bilingual students in kindergarten (Jordan et al., 2010; Jordan et al., 2009).
For monolingual students, their vocabulary and understanding of English grammar supports the acquisition of early math skills as they learn new concepts and terms through verbal instruction (Méndez et al., 2019). Within the context of mathematics instruction, monolingual students call on background knowledge related to the vocabulary used to describe arithmetic problems such as those that involve addition and subtraction (Méndez et al., 2019). Comparatively, emergent bilinguals call on background knowledge related to vocabulary and comprehension of math content in multiple languages (Swanson et al., 2018). This process is known as cross-linguistic transfer, which is the ability to call on resources from both language systems in service of completing a task (Genesee et al., 2006; Swanson et al., 2018). Cross-linguistic transfer may pose additional challenges for emergent bilingual students as they must simultaneously develop both their oral language skills in English and their early numeracy skills.
Students learning mathematics in their second language are exposed to additional cognitive demands aside from simply acquiring early numeracy skills (Genesee et al., 2006; Méndez et al., 2019). One instructional method that supports the development of these oral language and early numeracy skills is explicit instruction (Doabler et al., 2019). Explicit instruction may be beneficial for emergent bilingual students at risk for math difficulty because it employs scaffolded instructional tactics such as frequent opportunities for practice, response to the instructor, and corrective feedback that support the development of procedural fluency and verbalization within the area of instruction (Doabler et al., 2019; Gersten et al., 2009).
Tier II Math Intervention for Emergent Bilinguals
Students struggling to gain competence with whole number foundations generally struggle with concepts of numeral identification, counting and cardinality, number relations, and number operations (Devlin et al., 2022). In kindergarten specifically, early numeracy skills involving number relations are predictive of mathematics achievement outcomes for students at risk for MD (Devlin et al., 2022). In order to address mathematics achievement gaps, mathematics interventions have been developed to increase treatment intensity and dosage of mathematics instruction (Nelson & McMaster, 2019b). Academic interventions are delivered within the multitiered systems of support (MTSS) framework in the United States, which allows for students to progressively receive increased instructional support in the academic domains of reading and mathematics based on targeted screening and progress monitoring. The MTSS system is divided into three tiers, each of which deliver increased instructional support. Most students receive mathematics instruction at Tier I (core instruction) in the general education setting (Nelson & McMaster, 2019b). As students demonstrate greater need for instructional support in mathematics, they receive additional instructional support at Tier II in the form of small group instruction. If a student demonstrates lack of progress at Tier II, they receive instructional support with increased dosage and treatment intensity at Tier III, which typically involves individual instruction (Nelson & McMaster, 2019b). Researchers have developed and validated numerous evidence-based Tier II mathematics interventions which are designed to be delivered in small groups by a trained interventionist (Fuchs et al., 2017). Despite strong evidence supporting effectiveness of these interventions, not all students respond adequately (Fuchs et al., 2017). Previous research has indicated that approximately 5-10% of students receiving validated Tier II interventions demonstrate need for intensive intervention based on lack of adequate response (O’Connor et al., 2013).
According to a meta-analysis examining effects of early numeracy interventions, kindergarten early numeracy interventions yielded moderate to large effects (g = 0.75; SE = 0.15) (Nelson & McMaster, 2019b). While a significant number of mathematics interventions have been developed and studied for early elementary grades, few intervention studies have focused specifically on emergent bilingual students (Nelson & McMaster, 2019a; Orosco, 2014; Orosco et al., 2013). Results of previous intervention studies with emergent bilingual kindergartners demonstrate mixed findings. Jordan and colleagues (2012) evaluated the efficacy of a Tier II number sense intervention for kindergartners and determined that language skills did not have a statistically significant effect on math outcomes for emergent bilinguals. Additionally, Foster et al. (2018) examined the impact of the Spanish Building Blocks software program for emergent bilingual kindergarteners. Results favored students in the treatment group, as students in receiving the Building Blocks intervention demonstrated higher post-test mathematics scores in Spanish than their peers in the control group (Foster et al., 2018). During the efficacy trials for the ROOTS intervention, the Tier II kindergarten math intervention used in the present study, ROOTS worked equally well across a diverse sample of emergent bilingual students with differing levels of English proficiency and mathematics skills (Doabler et al., 2019). To date, there have been no other studies published in English examining treatment outcomes for emergent bilingual kindergartners receiving Tier II mathematics intervention.
In terms of intervention response, previous studies indicate that math outcomes demonstrated at pre-testing do not always predict responsiveness to intervention for students at risk for math difficulty. This supports the notion that all students cannot be expected to respond predictably to a given intervention based on initial math skill (Fuchs et al., 2019; Fuchs et al., 2016). Demonstrated variations in the predictors of response for at-risk students necessitate further analysis of such predictors for students at risk for difficulties in mathematics. This is particularly relevant for emergent bilingual students, as few studies have been conducted examining outcomes for this population. When considering math interventions for emergent bilingual students, multiple factors must be considered, such as: language skills and cognitive mechanisms that underlie mathematics development. One promising avenue for this investigation is to study the role of domain general cognitive skills in relation to the development of early numeracy skills.
Domain General Cognitive Processes and the Development of Early Math Skills of Emergent Bilinguals
Multiple domain general cognitive skills such as phonological processing, rapid automatized naming, visual spatial reasoning, and working memory support essential cognitive processes used during mathematical processing (Geary et al., 2017). Previous work has indicated that difficulties in mathematics may be linked to lower skill levels in these cognitive processes because deficits in numerical processing correlate with executive skills such as working memory, short term memory, and processing speed (Johnson et al., 2010). Students with mathematics difficulties have been found to exhibit deficits in verbal and numerical working memory in comparison to peers without mathematics difficulties and both working memory and phonological processing have been associated with math performance (Lin & Powell, 2022; Peng & Fuchs, 2016; Yang et al., 2022).
Previous research has indicated that bilingualism influences cognitive functions, including domain general cognitive processes (Tao et al., 2021). While this body of work has indicated that bilingual children and infant groups may demonstrate cognitive and academic advantages when compared with monolingual peers, these studies have not examined differential outcomes for students at risk for learning difficulties (Tao et al., 2021). For this reason, examining the associations between domain general cognitive skills related to the development of early numeracy skills, such as working memory and phonological short-term memory for emergent bilingual students at-risk for academic difficulties could provide critical insights related to their academic needs.
Working Memory
Working memory (WM) refers to the ability to actively maintain and manipulate information within in service of the completion of complex cognitive tasks (Baddeley, 1992). According to Baddeley’s model of working memory, the central executive processes information through working memory, retrieves information stored in long-term memory through various retrieval strategies, and engages in arithmetic processes while the phonological loop and visual spatial sketch pad process specific types of information such as speech or visual stimuli (Baddeley & Logie, 1999; Gathercole, 1998). Within the context of mathematics instruction, students rely on their working memory capacity to process word problems, store verbally presented information, and manipulate mental images. Students use decoding and comprehension skills that draw on working memory capacity to engage in mathematical problem solving (Swanson, 2004).
Growth in working memory skills has been found to be predictive of growth in mathematical word problem skills in both languages spoken by emergent bilingual students between first and third grade (Swanson et al., 2021). 5-year-old emergent bilingual students have demonstrated greater efficiency and accuracy than their monolingual peers on tasks related to working memory, suggesting that bilingual students in this age group demonstrate greater levels of executive control on working memory tasks when compared to their monolingual peers (Morales et al., 2013). Working memory abilities are predictive of mathematical problem-solving abilities in students’ home and target languages, and the relationship between working memory and mathematical problem-solving ability functions independently from other skills such as vocabulary, reading, and short-term memory (STM) (Swanson et al., 2015). Swanson et al. (2018) posited that variations in bilingual proficiency may be linked to executive functioning skills, while variations in math achievement may be linked more explicitly to STM skills. For emergent bilingual students at risk for math difficulty, it is plausible that the relationship between bilingual proficiency and early numeracy skills may influence growth in mathematics skills.
Phonological Memory
Phonological memory relies on the phonological loop, which holds verbal stimuli for a period of time in short-term memory while information is maintained through rehearsal (Swanson et al., 2015). It is used in mathematical problem solving as phonological and numerical information is maintained and manipulated (Matejko et al., 2023; Peng et al., 2016). Phonological memory is associated with mathematics tasks such as counting, double and single-digit computation, and retrieval of arithmetic tasks and procedures from long-term memory (De Smedt, 2022; Hecht et al., 2001; Peng et al., 2016). Differences in efficiency within the phonological memory system are associated with individual differences in math computation skills, and students with learning disabilities in mathematics tend to demonstrate lower phonological memory skills than their peers (Hecht et al., 2001).
The existing body of evidence related to the cognitive development of emergent bilinguals indicates that learning multiple languages influences domain general cognitive skills (Barac et al., 2014). Both cognitive skills and linguistic ability have been found to influence the development of early numeracy skills for emergent bilinguals in kindergarten (Foster et al., 2018; Swanson et al., 2011). Further, previous studies have indicated that both working memory and phonological short-term memory influence math achievement for emergent bilingual students (Swanson et al., 2018).
When examining the influence of phonological memory skills on the acquisition of skills in mathematics for emergent bilingual students, it is critical to consider the role of cross-linguistic transfer and language-specialization within the phonological loop. Cross-linguistic transfer occurs when bilingual students use skills from their home language to engage in tasks that require them to use their second language, and it is supported by cognitive processes such as working memory and phonological awareness (Chung et al., 2019). Previous research demonstrates mixed findings as to whether there is a moderating effect of cross-linguistic transfer on the relationship between working memory and ability to solve math problems (Swanson et al., 2015). However, students with greater phonological memory skills may experience better outcomes than their peers when engaging in instructional tasks in their second language because they may experience more ease processing verbal information than peers with lower phonological memory skills (Chung et al., 2019; Hecht et al., 2001). Phonological short-term memory has been linked to vocabulary acquisition in the first and second languages of emergent bilingual students and is known to support calculation tasks (Engel de Abreu & Gathercole, 2012; Fürst & Hitch, 2000).
Present Study
Working memory and phonological memory have a significant influence on math outcomes for students at risk for math difficulty (Barnes et al., 2020). Given the age and developmental level of the population in this study, the purpose was to examine the association between specific aspects of memory, particularly working memory and phonological short-term memory, both of which are critical within the process of developing of early numeracy skills and accessing content in their non-dominant language during instruction. The present study explores the degree to which emergent bilinguals at risk for math difficulty demonstrate differences in mathematics outcomes and response to intervention based on domain general skills (working memory and phonological memory) in kindergarten. The following research questions were examined:
Research Question 1: What are the associations between domain general skills and scores on pre-test mathematics measures for emergent bilingual kindergartners? It was hypothesized that moderate correlations would be demonstrated between domain general skills and pre-test mathematics scores for emergent bilingual kindergartners. Given previous examinations of the association between domain general skills and mathematics outcomes, it was predicted that emergent bilingual students will demonstrate similarly moderate to strong associations between domain general cognitive skills and mathematics outcomes (Geary et al., 2017; Shanley et al., 2021).
Research Question 2: What are the associations between domain general skills and gains in mathematics? Given the associations reported in previous studies regarding the associations between working memory (Peng et al., 2016) and mathematics outcomes and phonological memory (De Smedt, 2022; Hecht et al., 2001; Peng et al., 2016) and mathematics outcomes, it was hypothesized that domain general skills would be positively associated with gains in mathematics.
Research Question 3: To what degree is the response to the ROOTS intervention moderated by domain general skills? Given the associations between domain general skills and mathematics achievement present in previous research, it was predicted that students with higher domain general skills would demonstrate stronger outcomes in regard to mathematics achievement (Geary et al., 2017). However, given previous findings related to differential response to the ROOTS intervention for multilingual students (Doabler et al., 2019) and examination of outcomes related to domain general cognitive skills for a broader sample (Shanley et al., 2021) it was hypothesized that the ROOTS intervention would be similarly effective for all students, regardless of initial domain general skill. Thus, it was hypothesized that response to the ROOTS intervention would not be moderated by domain general skills to a statistically significant degree.
Method
This study uses data from an IES funded efficacy trial of the ROOTS intervention, a randomized control trial conducted over the course of four years. Two cohorts of students were included in this study, one of which was in Oregon, while the other was located in Massachusetts. Math achievement data were collected at pre-test and post-test during the students’ kindergarten year and follow-up testing was conducted six months into the students’ first grade academic year. The primary aim of the present study was to determine the degree to which domain general cognitive skills and initial mathematics skills influenced math achievement for emergent bilingual students who received the ROOTS intervention in comparison to their emergent bilingual peers in the control condition.
Design
The study was a randomized control trial which randomly assigned students, blocking on classroom, to receive ROOTS, a Tier II mathematics intervention, or the business-as-usual control condition. ROOTS was provided in addition to the core curriculum provided in general education classes. ROOTS is designed to support students struggling with Tier 1 mathematics content by focusing on the foundational concepts, skills, and vocabulary associated with whole numbers.
Screening
Consent was obtained and kindergarten students were screened for participation in the late fall of the academic year. Students were screened using two standardized mathematics measures selected based on their utility for assessing number sense. Students were considered eligible to receive the ROOTS intervention if they earned a score of 20 or less on the Number Sense Brief Screener (NSB; Jordan et al., 2008) and a composite score that placed them in the strategic or intensive range on the Assessing Students Proficiency in Early Number Sense (ASPENS; Clarke et al., 2011). Cut scores were determined based on prior research establishing predictors of future risk in mathematics. Students scoring below the cut score on the NSB would be likely to fail to meet state standards for math achievement at the end of third grade and students scoring in the strategic or intensive ranges on the ASPENS would likely struggle to meet expectations for math achievement at the end of kindergarten (Clarke et al., 2011; Jordan et al., 2008). School districts provided language proficiency data for each student and indicated that students were considered English learners based on their scores on the ACCESS for ELLs or English Language Proficiency Assessment (ELPA; American Institutes for Research, 2014).
Interventionists
The ROOTS intervention was delivered by interventionists hired by the research team and district employees. Bilingual status was not available for interventionists; however, the intervention was delivered in English.
Participants
The sample for this study is a subgroup from the ROOTS efficacy trial. It includes 131 emergent bilingual kindergarteners (47% female), located across six school districts in Oregon and Boston. The participants in this study were drawn from 40 kindergarten classrooms taught by 38 teachers. Classrooms include different sessions by the same teacher (e.g., half-day kindergarten classes). Students were classified as English Learners by their respective school districts based on district selected measures of English language proficiency and spoke 23 different languages. Descriptive statistics for the sample are reported by condition in Table 1.
Table 1
Descriptive Statistics for Student Characteristics by Condition
| Characteristics | ROOTS
|
Control
|
Total
|
|---|---|---|---|
| n (%) | n (%) | n (%) | |
| Total sample (n) | 88 (100%) | 43 (100%) | 131 (100%) |
| Sex | |||
| Male | 47 (53%) | 22 (51%) | 69 (53%) |
| Female | 41 (47%) | 21 (49%) | 62 (47%) |
| Race | |||
| Asian | 3 (3%) | 1 (2%) | 4 (3%) |
| Black | 2 (2%) | 2 (5%) | 4 (3%) |
| Hispanic | 59 (67%) | 33 (77%) | 92 (70%) |
| Hawaiian/Pacific Islander | 1 (1%) | 0 (0%) | 1 (1%) |
| White | 21 (24%) | 7 (16%) | 28 (21%) |
| More than one | 2 (2%) | 0 (0%) | 2 (2%) |
| Special education | |||
| Not eligible | 79 (90%) | 35 (81%) | 114 (87%) |
| Eligible | 9 (10%) | 8 (19%) | 17 (13%) |
| M (SD) n | M (SD) n | M (SD) n | |
| Age | 5.3 (0.4) 88 | 5.3 (0.5) 43 | 5.3 (0.4) 131 |
Note. The percentages represent the proportion of students for whom each variable was reported. The sample sizes (n) for each variable will not sum to the total sample (top row) due to missing responses. Race and ethnicity categories were exclusive, so we did not receive reports, for example, of students as White and Hispanic. M = mean, SD = standard deviation.
ROOTS Intervention
ROOTS is designed to align with the Common Core State Standards (CCSS) for mathematics and the Texas Essential Knowledge and Skills (TEKS) standards in mathematics. Students identified to be at risk for math difficulties were randomly assigned their respective conditions within their classrooms. The lowest performing students were assigned: (1) a ROOTS instructional group with a 2:1 student teacher ratio, (2) a ROOTS instructional group with a 5:1 student teacher ratio, or (3) business-as-usual, with no treatment condition. Analyses of impact by group size indicated no difference in outcomes by group size, thus for the purpose of this study assignments to groups were collapsed by treatment and control (Clarke et al., 2017). Students in the treatment condition received the 50 lesson ROOTS intervention 3 to 5 times a week from interventionists trained by the research team. Intervention sessions lasted 20 minutes. The curricula of the ROOTS intervention are designed to facilitate the development conceptual understanding of whole number foundations through the usage of mathematical models to represent key concepts. Throughout the intervention, students develop procedural fluency and automaticity through systematic practice and review that allows students to master, maintain, and generalize skills through frequent opportunities to respond and receive feedback and regular opportunities to engage in individual checks for student mastery through review worksheets. Throughout the intervention, students develop math vocabulary skills and engage in math discourse as key vocabulary terms are explicitly taught throughout the intervention.
Control Condition
The students in the business-as-usual condition received only the core math instruction. Core instruction was delivered in English to students in the treatment and control condition daily. Information about the control condition was collected using teacher surveys and direct observation. Observations indicated that teachers did not use ROOTS materials during core math instruction. Teachers utilized various curricula for daily core mathematics instruction including Scott Foresman, en Visionmath, Houghton Mifflin, and Everyday Mathematics.
Measures
Trained researchers administered five different measures to assess mathematics achievement and four subtests from standardized neuropsychology batteries to assess domain general cognitive skills. Cognitive measures were administered once at pre-test and all other measures were administered to participants at pre-test and post-test. Unless otherwise stated, raw scores were used in all analyses.
ROOTS Assessment of Early Numeracy Skills (RAENS)
RAENS (Doabler et al., 2012) is an individually administered instrument consisting of 32 items and was developed by the ROOTS research team that assesses counting and cardinality, number operations, and the base-10 system. The predictive validity of RAENS ranges from .68 to .83 with commonly used measures of mathematics achievement including the Test of Early Mathematics Ability-Third Edition (TEMA; Ginsburg & Baroody, 2003) and the Number Sense Brief (NSB; Jordan et al., 2012). Inter-rater scoring agreement was reported to be 100% (Clarke et al., 2016b) and internal consistency was also high, coefficient alpha = 0.91 (Clarke et al., 2016a). RAENS was administered at pre-test and post-test.
ASPENS
ASPENS (Clarke et al., 2011) is used to measure components of early number sense such as number identification, magnitude comparison, and determining the missing number in a sequence. The measure utilizes three curriculum-based measures and has moderate to high test-retest reliabilities for kindergartners and a predictive validity that ranges from 0.45 to 0.52 when used in the fall of the kindergarten year and compared to spring scores on the TerraNova 3 (Clarke et al., 2011). ASPENS was administered at pre-test and post-test.
Number Sense Brief (NSB)
The NSB (Jordan et al., 2012) is administered individually to students and assesses skills of early numeracy such as counting principles, number recognition and comparison, subitizing, story problems, and number combinations. For use with students at the beginning of the first-grade year, it has a coefficient alpha of 0.84 (Jordan & Glutting, 2012). The NSB was administered at pre-test and post-test.
Test of Early Mathematics Ability – Third Edition (TEMA)
The TEMA-III (Ginsburg & Baroody, 2003) is individually administered to students as a measurement of early numeracy skills. It is norm-referenced and assesses student understanding of whole number concepts. Authors of the TEMA-III report alternate form reliability of 0.97 and test re-test reliability that ranges from 0.82-0.91. The concurrent validity with similar criterion measures is reported to range from 0.54-0.91. The TEMA-III was administered at pre-test and post-test.
Comprehensive Test of Phonological Processing (CTOPP) Subtests: Memory for Digits and Nonword Repetition
The CTOPP (Wagner et al., 1999) is composed of 13 subtests that assess phonological skills including phonological awareness, phonological memory, and rapid naming. In this study, the Memory for Digits and Nonword Repetition subtests were used to provide a composite score for Phonological Memory, which is referred to as the CTOPP PMC.
CTOPP Memory for Digits
The CTOPP Memory for Digits subtest assesses the degree to which a student can repeat a series of numbers. It is a 21-item test, and the digit series range from two to eight digits. The subtest is used to measure the phonological loop of the student’s short-term memory and is presented to students by an administrator who plays an audio-recording of a series of numbers at a rate of 2 numbers per second. Students must listen to the recording and repeat the set in the same order in which it was presented. The score for CTOPP Memory for Digits subtest is calculated by counting the number of correct responses up to the ceiling, and the test is discontinued if the student misses three items in a row.
CTOPP Nonword Repetition
The CTOPP Nonword Repetition subtest is used to assess the extent to which the student can repeat nonwords of various length, that range from 3 to 15 sounds. The subtest is used to detect efficiency of the phonological loop. The nonwords are presented via audio-recording and the student must repeat the words back to the examiner verbally. The total score for the subtest is calculated by adding the correct number of test items achieved before the student reaches the ceiling or misses three items in a row. The Nonword Repetition subtest has internal consistency of Cronbach’s alpha = 0.80 and test-retest reliability is 0.68 for students between the ages of 5-7.
Digit Span Backwards (DSB)
Digit Span Backwards is a researcher created measure that is commonly utilized to measure working memory ability (Hilbert et al., 2015). The subtest is presented verbally and requires participants to repeat a series of numbers presented by the examiner in reverse order.
English Language Proficiency Assessment (ELPA)
In Oregon, students were classified as English learners using the English Language Proficiency Assessment (ELPA; American Institutes for Research, 2014), the assessment used by the Oregon Department of Education to provide placement information regarding students’ mastery of the English language (Gilkey et al., 2013). The assessment is a standardized computer-based measure administered individually to students that places a student’s level of English proficiency at one of five levels of English proficiency (Beginning, Early Intermediate, Intermediate, Early Advanced, and Advanced). Student proficiency scores were standardized by their respective district for placement regarding English language proficiency and analysis for the present study.
ACCESS for ELLs (ACCESS)
In Massachusetts, ACCESS for ELLS was used to assess the English language proficiency of English learners (WIDA Consortium, 2015). ACCESS measures proficiency in listening, speaking, reading, and writing which is used to calculate an Overall Composite and composite scores by category (oral language composite, literacy composite, and comprehension composite) which range from 1-6 (WIDA Consortium, 2015). Students who receive a category score that is less than 5 were considered to be eligible for services as an English learner. The ACCESS for ELLs technical reports that the kindergarten overall composite score has a reliability of 0.97 (WIDA Consortium, 2015). For the purpose of this study, ACCESS scores were standardized by district.
Statistical Analysis
Main Effects
Before examining the three research questions pertaining to relations between domain general cognitive skills and intervention response, we first confirmed that the intervention effects for the specific sample in the current study were consistent with prior studies. We assessed intervention effects on each of the primary outcomes with a mixed model (multilevel) Time × Condition analysis (Murray, 1998), which test for net gains from the fall (T1) to spring (T2) of kindergarten between conditions (Murray, 1998) and provide an unbiased and straightforward interpretation of the results (Allison, 1990; Jamieson, 1999).
The statistical model included Time, Condition, and the interaction between the two. The models estimated effects for Time, coded 0 at T1 and 1 at T2, Condition, coded 0 for control and 1 for ROOTS, and the Time × Condition interaction. The Time parameter estimated gains on mathematics outcomes in the control group, Condition compares ROOTS and control groups at baseline, and the Time × Condition interaction estimated the difference between intervention and control students on gains on mathematics outcomes. We used Satterthwaite approximation to determine the degrees of freedom.
The models accounted for students partially nested within small groups (Baldwin et al., 2011; Bauer et al., 2008). As the ROOTS groups but not the unclustered controls required a group-level variance estimate, the analytic model accounted for the potential heterogeneity of residual variances across conditions (Roberts & Roberts, 2005). We tested for the noninferiority of homoscedastic and heteroscedastic models with a likelihood ratio test and reported the simpler model if equivalent. We reversed the null and alternative hypotheses, a common procedure for equivalence or noninferiority trials (e.g. Dasgupta et al., 2010; Piaggio et al., 2006). For this reason, as well as the limited statistical power to detect differences in variance structures (Kromrey & Dickinson, 1996), we set α = .20. We refer readers to Clarke and colleagues (2016b) for a complete description of the analysis approach.
Correlations
After examining intervention effects, Research Questions 1 and 2 required tests of associations amongst cognitive measures and both pretest mathematics measures and gains on mathematics measures across time. Pearson product-moment correlations were calculated for these analyses.
Interactions—Moderation
Research Question 3 asks whether intervention effects differed by students' domain general skills or, equivalently, whether the association between cognitive variables and mathematics gains depended on participation in the ROOTS intervention. To examine the influence of phonological memory, for instance, we added the phonological memory composite score to the model as well as its interaction with each of the Time, Condition, and Time × Condition terms. Statistical support for interaction effect implies that the association between cognitive variables and gains in mathematics depends on participating in the ROOTS intervention.
Model Estimation
We fit the statistical models to our data using SAS PROC MIXED version 14.2 (SAS Institute, 2016) with full-information maximum likelihood estimation to minimize the potential for bias due to missing data (Allison, 2009; Graham, 2009). To account for missing data, the Time × Condition and growth models were fit with all available data on mathematics measures but not domain general measures (1.1% to 2.5% missing).
Reporting
The American Statistical Association (Wasserstein & Lazar, 2016; Wasserstein et al., 2019) has strongly urged researchers to abstain from using bright-line rules for claims of “statistical significance” such as p < .05 or other metrics (e.g., effect sizes > 0.25). We reported p. We then estimated model probabilities (w), described next, to characterize the strength of evidence for the alternative hypothesis over the null hypothesis. We used tables to summarize parameter estimates with standard errors.
p values, defined as a measure of incompatibility between the observed data and all assumptions of the statistical model including the null hypothesis, H0, are difficult to interpret (Wasserstein & Lazar, 2016). They provide information about neither which assumptions are incorrect nor the importance of the association (Greenland et al., 2016). We have therefore complemented p values with a model probability (also called Akaike weights; Akaike, 1973) that describes the strength of evidence for the hypothesis of an effect (Burnham et al., 2011).
The model probability is based on the AIC, a second-order, small-sample bias correction to the AIC (Akaike, 1973; Anderson, 2008). The probability or weight, w, can be interpreted as the probability that the same model would be selected with a “replicate data set from the same system” (Burnham et al., 2011, p. 30). We defined a model for each of two hypotheses, the hypothesis of an interaction effect between a domain general skill and condition (HA) and the hypothesis of no interaction effect (H0), and reported the model probability, w, for the model with the interaction (HA). With only two models, the model probability for H0 (no interaction) is 1 – w. For example, if w = .75, it suggests the probability of HA is .75 while the probability of H0 is .25. The model for HA is estimated to have an approximately 75% chance of being the better-fitting model given the data and the two models. Equivalently, the model for HA is three times as likely as the model for H0. We interpret model probabilities as a continuous level of evidence for an interaction and avoid using a particular probability level as a cutoff for “significant.”
Results
Descriptive Results
Table 2 presents descriptive statistics including sample size, mean, standard deviation, and range for the standard score and standard normative percentile for the measures of domain general cognitive skills. The mean standard score of the Phonological Memory Composite, which includes CTOPP Memory for Digits and CTOPP Nonword Repetition was 7.35 (SD = 1.95), with scores ranging from 2.50-14.50. The average standard score for CTOPP Memory for Digits was 6.18 (SD = 2.17), with scores ranging from 1.00-14.00 and the average standard score for CTOPP Nonword Repetition was 8.51 (SD = 2.54), with scores ranging from 4.00-16.00. Comparatively, the average normative percentile for CTOPP Memory for Digits scores was 14.91 (SD = 16.61), with student scores falling in percentiles ranging from < 1.00-91.00 and the average normative percentile for CTOPP Nonword Repetition scores was 33.89 (SD = 25.91), with student scores falling in percentiles ranging from 2.00-98.00. Students earned scores on the Phonological Memory Composite across a wide range of normative percentiles. On Digit Span Backwards, students scored an average of 0.18 (SD = 0.57), with scores ranging from 0.00-3.00.
Table 2
Descriptive Statistics for Cognitive Measures, Standard Scores and Normative Percentiles, and Math Measures at Baseline and Posttest
| Measure | n | M | SD | Min | Max |
|---|---|---|---|---|---|
| Cognitive Measures | |||||
| CTOPP PMC | 127 | 7.35 | 1.95 | 2.50 | 14.50 |
| CTOPP DMss | 127 | 6.18 | 2.17 | 1.00 | 14.00 |
| CTOPP NRss | 127 | 8.51 | 2.54 | 4.00 | 16.00 |
| CTOPP DMpc | 127 | 14.91 | 16.61 | -1.00 | 91.00 |
| CTOPP NRpc | 127 | 33.89 | 25.91 | 2.00 | 98.00 |
| Digit Span Backward | 129 | 0.18 | 0.57 | 0.00 | 3.00 |
| English Proficiency z-score | 126 | 0.00 | 1.00 | −2.04 | 4.81 |
| ELPA Score | 52 | 488.40 | 6.58 | 475.00 | 509.00 |
| ACCESS Score | 74 | 1.74 | 0.49 | 1.10 | 4.10 |
| Baseline Math Measures | |||||
| RAENS | |||||
| Full Sample | 129 | 8.57 | 5.41 | 1.00 | 29.00 |
| Intervention | 86 | 8.28 | 5.07 | 1.00 | 25.00 |
| Control | 43 | 9.16 | 6.05 | 1.00 | 29.00 |
| ASPENS | |||||
| Full Sample | 125 | 12.11 | 13.96 | 0.00 | 57.50 |
| Intervention | 84 | 12.39 | 14.37 | 0.00 | 57.50 |
| Control | 41 | 11.54 | 13.23 | 0.00 | 57.00 |
| NSB | |||||
| Full Sample | 131 | 9.99 | 3.07 | 0.00 | 20.00 |
| Intervention | 88 | 9.94 | 3.16 | 0.00 | 20.00 |
| Control | 43 | 10.09 | 2.91 | 4.00 | 16.00 |
| TEMA-III | |||||
| Full Sample | 128 | 12.70 | 5.81 | 1.00 | 35.00 |
| Intervention | 85 | 12.76 | 5.89 | 1.00 | 35.00 |
| Control | 43 | 12.56 | 5.72 | 2.00 | 26.00 |
| Posttest Math Measures | |||||
| RAENS | |||||
| Full Sample | 123 | 19.42 | 7.58 | 1.00 | 31.00 |
| Intervention | 81 | 21.93 | 7.10 | 1.00 | 31.00 |
| Control | 42 | 14.60 | 6.03 | 4.00 | 27.00 |
| ASPENS | |||||
| Full Sample | 121 | 68.74 | 37.69 | 0.00 | 158.10 |
| Intervention | 81 | 74.99 | 37.55 | 0.00 | 158.10 |
| Control | 40 | 56.10 | 35.13 | 0.00 | 153.70 |
| NSB | |||||
| Full Sample | 121 | 16.92 | 4.54 | 6.00 | 29.00 |
| Intervention | 81 | 17.48 | 4.60 | 6.00 | 29.00 |
| Control | 40 | 15.78 | 4.24 | 7.00 | 24.00 |
| TEMA-III | |||||
| Full Sample | 123 | 21.74 | 7.53 | 4.00 | 40.00 |
| Intervention | 81 | 22.68 | 7.71 | 4.00 | 40.00 |
| Control | 42 | 19.93 | 6.93 | 5.00 | 39.00 |
Note. CTOPP = Comprehensive Test of Phonological Processing; PMC = Phonological Memory composite score; DMss = Digit Memory standard score; NRss = Nonword Repetition standard score; DMpc = Digit Memory normative percentile; NRpc = Nonword Repetition normative percentile; ELPA = English Language Proficiency Assessment; RAENS = ROOTS Assessment of Early Numeracy Skills; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability.
Main Effects
Previous research indicated condition differences favoring the ROOTS intervention with subsets of the sample used in the present study (Clarke et al., 2017; Fien et al., 2016), but as noted above main effects for the specific sample include in the present study had not been tested previously. Therefore, condition differences in gains on the RAENS, ASPENS composite, NSB total raw score, and TEMA measures were examined, and statistically significant main effects (p < .05) were found for each measure (see Table 5). These results are consistent with analyses that include some of the sample utilized in the present study (Clarke et al., 2017; Fien et al., 2016) as well as replications with other samples (Clarke et al., 2016a). Given the consistency of results across analyses, condition effects are not extensively discussed herein.
Associations Amongst Domain General Skills and Pretest Mathematics
To test the extent to which performance on pretest mathematics measures was associated with phonological memory and working memory each baseline math measure was correlated with CTOPP DM, CTOPP NR, digit span backward, and English language scores. The cognitive variables were modestly associated with the mathematics outcomes and are reported in Table 3. Correlations ranged from .21 to .50 (4% to 25% overlapping variance) for the phonological memory composite, working memory, and English language proficiency. For all correlations, pBH (p after the Benjamini-Hochberg correction) was less than .05. Language proficiency was moderately correlated with each baseline math measure with correlations ranging from 0.42 to 0.50.
Table 3
Correlations Between Math Measures and Domain General Skills and Language Proficiency at Pretest
| Measure | CTOPP-PMC | Digit Backwards | English Proficiency |
|---|---|---|---|
| RAENS | 0.27 | 0.28 | 0.50 |
| ASPENS | 0.21 | 0.30 | 0.42 |
| NSB | 0.29 | 0.30 | 0.42 |
| TEMA-III | 0.29 | 0.32 | 0.46 |
Note. CTOPP-PMC = Comprehensive Test of Phonological Processing Phonological Memory composite score; RAENS = ROOTS Assessment of Early Numeracy Skills; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability.
Associations Between Domain General Skills and Gains in Mathematics
To test associations among gains in mathematics and domain general measures, each of the cognitive measures were correlated with gain scores (i.e., pretest to posttest) on each mathematics measure. See Table 4 for details. No statistically significant correlations were found (p < .05).
Table 4
Correlations Between Gains on Math Measures and Domain General Skills and Language Proficiency
| Measure | CTOPP-PMC | Digit Backwards | English Proficiency |
|---|---|---|---|
| RAENS | -0.01 | -0.04 | 0.04 |
| ASPENS | 0.02 | -0.07 | 0.17 |
| NSB | 0.02 | 0.14 | 0.17 |
| TEMA-III | -0.01 | -0.12 | 0.18 |
Note. CTOPP-PMC = Comprehensive Test of Phonological Processing Phonological Memory composite score; RAENS = ROOTS Assessment of Early Numeracy Skills; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability.
Moderation Effects
Next, the extent to which the association between domain general skills and mathematics achievement differed by receipt of the ROOTS intervention was examined. The interaction between each domain general skill and condition tests whether the association between domain general measures and mathematics gains differ by condition. This is equivalent to a test of whether domain general skills moderate intervention effects on gains in mathematics skill. Model probabilities, ws, ranged from .25 to .76 indicating that models without interaction terms produced a better fit to the data in most cases. Thus, participating in the ROOTS intervention did not change associations amongst domain general skills and gains in mathematics; equivalently, domain general skills did not change the relationship between the ROOTS intervention and mathematics outcomes. See Tables 5, 6, 7, 8, and 9 for details.
Table 5
Main Effects of Condition Differences for the ROOTS Intervention
| Main Effects | RAENS | ASPENS | NSB | TEMA |
|---|---|---|---|---|
| Heterogeneity of Variance Test | ||||
| Likelihood Ratio χ2 | 2.26 | 1.00 | 2.03 | 0.69 |
| p | .3231 | .6056 | .3622 | .7083 |
| Fixed Effects | ||||
| Intercept | 9.16 (0.92) | 11.09 (4.28) | 10.09 (0.58) | 12.56 (1.00) |
| Time | 5.48 (0.91) | 45.16 (5.19) | 5.71 (0.67) | 7.47 (0.74) |
| Condition | -0.81 (1.17) | 1.61 (5.28) | -0.13 (0.72) | 0.27 (1.28) |
| Time × Condition | 8.21 (1.10) | 17.68 (6.27) | 1.85 (0.84) | 2.40 (0.91) |
| Variances | ||||
| Group-Level Intercept | 8.11 (5.10) | 74.19 (75.53) | 0.13 (1.61) | 12.69 (5.32) |
| Group-Level Gains | -1.04 (2.27) | -27.54 (63.54) | 1.11 (1.32) | 0.00 (.) |
| Member-Level Intercept | 11.18 (5.01) | 142.96 (95.56) | 5.09 (1.94) | 18.84 (5.03) |
| Residual (Error) | 18.50 (3.32) | 563.74 (98.86) | 8.09 (1.57) | 11.42 (1.45) |
| ICC | -.06 | -.05 | .12 | .00 |
| Test of Time × Condition Estimate | ||||
| Hedges’s g | 1.22 | 0.48 | 0.41 | 0.32 |
| Model probability (w) | > .99 | .94 | .79 | .91 |
| p | < .0001 | .0057 | .0294 | .0092 |
| df | 106 | 108 | 101 | 124 |
Note. Table entries show χ2 values, p values, parameter estimates with standard errors in parentheses, intraclass correlation coefficients (ICCs), Hedges’s g values, model probabilities (w), and the degrees of freedom (df). The Likelihood ratio test in the first two rows compared homoscedastic residuals to heteroscedastic residuals with a criterion α of .20 and one degree of freedom; all models supported homoscedastic residuals. Tests of fixed effects in the next four rows accounted for small groups as the unit of analysis within the ROOTS intervention condition and unclustered individuals in the control condition. RAENS = ROOTS Assessment of Early Numeracy Skills; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability-Third Edition.
Table 6
Interactions Between Digit Span Backward and Condition Differences
| Moderation Effects | RAENS | ASPENS | NSB | TEMA |
|---|---|---|---|---|
| Heterogeneity of Variance Test | ||||
| Likelihood Ratio χ2 | 2.44 | 1.17 | 2.73 | 0.91 |
| p | .2950 | .5578 | .2548 | .6345 |
| Fixed Effects | ||||
| Intercept | 9.36 (0.90) | 11.49 (4.27) | 10.15 (0.55) | 12.73 (0.97) |
| Time | 5.41 (0.91) | 45.06 (5.20) | 5.83 (0.66) | 7.40 (0.73) |
| Condition | -1.06 (1.13) | 1.07 (5.26) | -0.10 (0.68) | 0.04 (1.24) |
| Time × Condition | 8.28 (1.10) | 17.86 (6.29) | 1.59 (0.83) | 2.49 (0.90) |
| DSB | 5.02 (1.94) | 11.90 (9.04) | 1.58 (1.19) | 4.44 (2.10) |
| DSB × Condition | -3.05 (2.19) | -6.21 (10.25) | -0.47 (1.35) | -1.98 (2.36) |
| DSB × Time | -1.62 (1.95) | -5.33 (11.13) | 2.15 (1.44) | -1.83 (1.56) |
| DSB × Time × Condition | 1.01 (2.21) | 1.36 (12.56) | -1.56 (1.62) | 0.95 (1.78) |
| Standardized Estimates for Fixed Effects with DSB | ||||
| DSB | .34 | .17 | .18 | .31 |
| DSB × Condition | -.18 | -.08 | -.05 | -.12 |
| DSB × Time | -.08 | -.05 | .17 | -.09 |
| DSB × Time × Condition | .04 | .01 | -.11 | .04 |
| Variances | ||||
| Group-Level Intercept | 6.67 (4.59) | 66.15 (74.35) | -0.38 (1.30) | 10.54 (4.97) |
| Group-Level Gains | -1.13 (2.25) | -23.95 (64.40) | 0.98 (1.23) | 0.00 (.) |
| Member-Level Intercept | 10.50 (4.72) | 145.55 (95.13) | 4.45 (1.69) | 18.79 (4.93) |
| Residual (Error) | 18.46 (3.30) | 557.32 (98.56) | 7.95 (1.51) | 11.19 (1.43) |
| Test of DSB × Time × Condition | ||||
| Model probability (w) | .27 | .25 | .34 | .28 |
| p | .6476 | .9141 | .3388 | .5927 |
| df | 124 | 120 | 123 | 123 |
Note. Table entries show χ2 and p values for the likelihood ratio test of residual variances, parameter estimates with standard errors in parentheses, standardized estimates for fixed effects, model probabilities (w) and p values for the highest-level interaction effect, and the degrees of freedom (df). DSB = Digit Span Backward; RAENS = ROOTS Assessment of Early Numeracy Skills; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability.
Table 7
Interactions Between the CTOPP Phonological Memory Composite and Condition Differences
| Interactions | RAENS | OC | ASPENS | NSB | TEMA |
|---|---|---|---|---|---|
| Heterogeneity of Variance Test | |||||
| Likelihood Ratio χ2 | 3.13 | 0.14 | 2.78 | 8.88 | 1.17 |
| p | .2095 | .9324 | .2487 | .0118 | .5563 |
| Fixed Effects | |||||
| Intercept | 9.46 (0.92) | 12.50 (2.40) | 11.70 (4.30) | 10.33 (0.52) | 12.88 (0.99) |
| Time | 5.54 (0.93) | 19.78 (3.13) | 46.06 (5.28) | 5.53 (0.57) | 7.46 (0.76) |
| Condition | -1.21 (1.15) | 1.08 (3.07) | 0.80 (5.31) | -0.35 (0.67) | -0.22 (1.26) |
| Time × Condition | 8.16 (1.13) | 5.35 (3.97) | 17.05 (6.35) | 1.96 (0.78) | 2.49 (0.93) |
| PMC | 0.98 (0.43) | 1.49 (1.13) | 2.29 (2.03) | 0.60 (0.24) | 0.95 (0.47) |
| PMC × Condition | -0.43 (0.56) | -0.55 (1.46) | -1.22 (2.62) | -0.23 (0.33) | -0.21 (0.60) |
| PMC × Time | -0.08 (0.45) | 1.25 (1.45) | 3.12 (2.45) | -0.02 (0.26) | 0.02 (0.36) |
| PMC × Time × Condition | -0.28 (0.57) | -1.82 (1.88) | -5.24 (3.14) | -0.09 (0.38) | -0.18 (0.47) |
| Standardized Estimates for Fixed Effects With PMC | |||||
| PMC | .22 | .15 | .11 | .23 | .23 |
| PMC × Condition | -.08 | -.04 | -.05 | -.07 | -.04 |
| PMC × Time | -.01 | .09 | .11 | -.01 | .00 |
| PMC × Time × Condition | -.03 | -.10 | -.14 | -.02 | -.02 |
| Variances | |||||
| Group-Level Intercept | 6.00 (4.86) | 16.29 (34.55) | 69.81 (72.63) | -0.41 (1.49) | 10.78 (4.91) |
| Group-Level Gains | -0.99 (2.39) | 51.81 (48.74) | -28.51 (65.20) | 1.44 (1.72) | 0.00 (.) |
| Member-Level Intercept | 11.66 (5.06) | 30.18 (31.11) | 136.34 (94.07) | 18.54 (4.87) | |
| Residual (Error) | 18.53 (3.43) | 141.33 (43.23) | 562.22 (100.51) | 11.49 (1.47) | |
| ROOTS Residual | 8.82 (1.99) | ||||
| ROOTS Pre-Post Covariance | 4.55 (2.04) | ||||
| Control Residual | 4.86 (2.23) | ||||
| Control Pre-Post Covariance | 5.13 (2.40) | ||||
| Test of PMC × Time × Condition | |||||
| Model probability (w) | .27 | .35 | .57 | .25 | .26 |
| p | .6269 | .3353 | .0981 | .8158 | .7081 |
| df | 123 | 115 | 116 | 109 | 124 |
Note. Table entries show χ2 and p values for the likelihood ratio test of residual variances, parameter estimates with standard errors in parentheses, standardized estimates for fixed effects, model probabilities (w) and p values for the highest-level interaction effect, and the degrees of freedom (df). PMC = Comprehensive Test of Phonological Processing Phonological Memory composite score; RAENS = ROOTS Assessment of Early Numeracy Skills; OC = oral counting; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability.
Table 8
Interactions Between CTOPP Digit Memory and Condition Differences
| Interactions | RAENS | OC | ASPENS | NSB | TEMA |
|---|---|---|---|---|---|
| Heterogeneity of Variance Test | |||||
| Likelihood Ratio χ2 | 2.14 | 1.22 | 1.26 | 3.57 | 1.11 |
| p | .3437 | .5447 | .5330 | .1677 | .5751 |
| Fixed Effects | |||||
| Intercept | 9.54 (0.93) | 12.93 (2.41) | 12.32 (4.38) | 10.48 (0.56) | 13.22 (0.99) |
| Time | 5.59 (0.95) | 19.91 (3.18) | 45.60 (5.41) | 5.57 (0.68) | 7.19 (0.76) |
| Condition | -1.43 (1.15) | 0.34 (3.06) | -0.12 (5.36) | -0.57 (0.69) | -0.71 (1.25) |
| Time × Condition | 8.11 (1.14) | 5.28 (4.01) | 17.48 (6.48) | 1.93 (0.84) | 2.78 (0.93) |
| DM | 0.66 (0.40) | 1.63 (1.04) | 2.55 (1.89) | 0.62 (0.24) | 1.20 (0.43) |
| DM × Condition | 0.01 (0.51) | -0.16 (1.32) | -1.06 (2.39) | -0.19 (0.31) | -0.33 (0.54) |
| DM × Time | 0.08 (0.42) | 1.01 (1.36) | 0.94 (2.32) | 0.07 (0.29) | -0.47 (0.34) |
| DM × Time × Condition | -0.17 (0.52) | -1.45 (1.74) | -1.42 (2.95) | -0.08 (0.37) | 0.32 (0.42) |
| Standardized Estimates for Fixed Effects With DM | |||||
| DM | .17 | .18 | .14 | .26 | .32 |
| DM × Condition | .00 | -.01 | -.04 | -.06 | -.07 |
| DM × Time | .01 | .08 | .03 | .02 | -.09 |
| DM × Time × Condition | -.02 | -.08 | -.04 | -.02 | .05 |
| Variances | |||||
| Group-Level Intercept | 3.88 (5.30) | 10.84 (33.63) | 56.17 (73.30) | -0.57 (1.42) | 9.33 (4.82) |
| Group-Level Gains | -1.31 (2.40) | 48.88 (49.30) | -30.13 (65.93) | 1.21 (1.67) | 0.00 (.) |
| Member-Level Intercept | 13.28 (5.74) | 30.58 (31.45) | 146.82 (97.27) | 18.88 (4.93) | |
| Residual (Error) | 18.99 (3.51) | 143.90 (44.47) | 574.36 (102.30) | 11.30 (1.45) | |
| ROOTS Residual | 8.97 (2.01) | ||||
| ROOTS Pre-Post Covariance | 4.41 (2.04) | ||||
| Control Residual | 5.01 (2.17) | ||||
| Control Pre-Post Covariance | 4.77 (2.25) | ||||
| Test of DM × Time × Condition | |||||
| Model probability (w) | .26 | .32 | .27 | .25 | .31 |
| p | .7498 | .4050 | .6310 | .7792 | .4525 |
| df | 125 | 121 | 121 | 111 | 124 |
Note. Table entries show χ2 and p values for the likelihood ratio test of residual. variances, parameter estimates with standard errors in parentheses, standardized estimates for fixed effects, model probabilities (w) and p values for the highest-level interaction effect, and the degrees of freedom (df). DM = Digit Memory standard score; RAENS = ROOTS Assessment of Early Numeracy Skills; OC = oral counting; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability.
Table 9
Interactions Between CTOPP Nonword Repetition and Condition Differences
| Interactions | RAENS | OC | ASPENS | NSB | TEMA |
|---|---|---|---|---|---|
| Heterogeneity of Variance Test | |||||
| Likelihood Ratio χ2 | 1.98 | 0.86 | 1.62 | 2.98 | 0.90 |
| p | .3720 | .6521 | .4450 | .2250 | .6384 |
| Fixed Effects | |||||
| Intercept | 9.23 (0.93) | 12.16 (2.42) | 11.23 (4.30) | 10.19 (0.57) | 12.66 (1.01) |
| Time | 5.53 (0.92) | 19.40 (3.11) | 45.20 (5.21) | 5.51 (0.67) | 7.46 (0.74) |
| Condition | -0.91 (1.17) | 1.56 (3.10) | 1.47 (5.29) | -0.15 (0.71) | 0.11 (1.29) |
| Time × Condition | 8.13 (1.11) | 5.68 (3.96) | 17.63 (6.24) | 1.97 (0.83) | 2.46 (0.92) |
| NR | 0.63 (0.32) | 0.57 (0.83) | 0.85 (1.46) | 0.26 (0.20) | 0.27 (0.35) |
| NR × Condition | -0.45 (0.43) | -0.55 (1.11) | -0.65 (1.96) | -0.14 (0.27) | -0.02 (0.46) |
| NR × Time | -0.14 (0.32) | 0.75 (1.06) | 2.77 (1.75) | -0.06 (0.23) | 0.30 (0.26) |
| NR × Time × Condition | -0.24 (0.42) | -1.06 (1.41) | -4.98 (2.32) | -0.04 (0.30) | -0.36 (0.35) |
| Standardized Estimates for Fixed Effects With NR | |||||
| NR | .19 | .08 | .05 | .13 | .09 |
| NR × Condition | -.10 | -.05 | -.03 | -.05 | -.01 |
| NR × Time | -.03 | .07 | .13 | -.02 | .06 |
| NR × Time × Condition | -.04 | -.07 | -.17 | -.01 | -.06 |
| Variances | |||||
| Group-Level Intercept | 7.94 (4.97) | 21.52 (36.47) | 77.12 (76.87) | 0.01 (1.45) | 12.45 (5.27) |
| Group-Level Gains | -1.11 (2.40) | 52.33 (48.71) | -40.91 (64.76) | 1.12 (1.27) | 0.00 (.) |
| Member-Level Intercept | 10.70 (4.91) | 30.49 (31.97) | 139.78 (97.51) | 4.92 (1.80) | 18.88 (5.01) |
| Residual (Error) | 18.55 (3.45) | 141.37 (43.04) | 566.04 (101.56) | 7.81 (1.51) | 11.39 (1.46) |
| Test of NR × Time × Condition | |||||
| Model probability (w) | .28 | .30 | .76 | .25 | .37 |
| p | .5794 | .4546 | .0338 | .8959 | .2944 |
| df | 119 | 118 | 111 | 117 | 123 |
Note. Table entries show χ2 and p values for the likelihood ratio test of residual variances, parameter estimates with standard errors in parentheses, standardized estimates for fixed effects, model probabilities (w) and p values for the highest-level interaction effect, and the degrees of freedom (df). NR = Nonword Repetition standard score; RAENS = ROOTS Assessment of Early Numeracy Skills; OC = oral counting; ASPENS = Assessing Student Proficiency in Early Number Sense; NSB = Number Sense Brief; TEMA = Test of Early Mathematics Ability.
Discussion
The development of early numeracy skills significantly influences the trajectory of math achievement throughout elementary school (Jordan et al., 2010). Understanding factors that impact this development is critical, especially for emergent bilingual kindergartners simultaneously learning a second language and developing early numeracy skills. Further, variations in cognitive skills and linguistic proficiency must be considered when engaging in instructional decision making to ensure that adequate instructional supports are provided to students at risk for difficulties in mathematics.
In this study we examined the role of domain general skills in early numeracy development within the context of a kindergarten math intervention for emergent bilingual students. The results indicate that there is a statistically significant relationship between the domain general cognitive skills of working memory and phonological memory and baseline mathematics scores. Within the sample, students demonstrated a vast range of cognitive abilities, which contributed to statistically significant differences in their mathematics scores at pre-testing. Emergent bilingual students with lower domain general skills demonstrated lower math scores at pre-test, indicating that domain general skills may have influenced the development of mathematics skills prior to intervention. Further, a statistically significant relationship between baseline mathematics scores and English proficiency was found. This finding indicates that for emergent bilingual kindergarteners receiving mathematics instruction in English, the degree to which they are proficient in English has a statistically significant influence on their math performance at pre-test. Thus, demonstrating proficiency in the language of instruction had an impact on initial math skill. Results indicate that baseline scores on mathematics measures were influenced by both domain general cognitive skills and language proficiency. These findings were consistent with previous studies indicating that there is a relationship between domain general cognitive skills and math skills (Geary et al., 2017; Peng et al., 2016). However, they offer an extension of previous studies by establishing the relationship between these constructs for emergent bilingual students at risk for math difficulty by examining language proficiency, gains in mathematics, and response to intervention. Although language proficiency was not associated with gains in mathematics or intervention response, future research should continue to investigate the role of language proficiency and mathematics development generally and in the context of receipt of intervention. In the instructional context, these results indicate the importance of examining proficiency in the language of instruction prior to engaging in intervention for emergent bilingual students.
In previous research, working memory and phonological processing skills have been found to be predictive of academic outcomes for monolingual and emergent bilingual students (Barnes et al., 2020; Swanson et al., 2021). However, previous studies have not analyzed the degree to which working memory and phonological memory may be predictive of growth in mathematics skills for emergent bilingual kindergartners (Nelson & McMaster, 2019a). The present study examined the degree to which domain general cognitive skills predict growth in early numeracy skills for emergent bilingual students. Results indicated that no significant associations were found between students’ working memory and phonological memory skills and gains in mathematics and are consistent with the results determined by Shanley et al. (2021), which indicated that domain general cognitive skills did not moderate response to the ROOTS intervention for a larger sample of primarily monolingual students. It is plausible that students demonstrated similar gains in mathematics, regardless of individual differences in domain general cognitive skills as a result of the design of the ROOTS intervention. Designed to support students with a range of difficulties in early numeracy skills, the ROOTS intervention incorporates instructional practices recommended for at-risk learners including frequent opportunities to respond, use of multiple forms of mathematical representation, and use of instructional examples designed to encompass the scope and sequence of Common Core State standards. Within the ROOTS intervention, prerequisite knowledge is identified and pre-taught to student and within each lesson students engage in frequent and systematic practice opportunities. For emergent bilingual students learning early numeracy skills in English, engaging in systematically designed instruction and frequent practice opportunities is likely highly supportive of the linguistic and cognitive demands they experience as multilingual learners, which in turn may make the mathematics content within the intervention easier to access.
Of greater interest, is the moderating effect that domain general skills may have on response to intervention for emergent bilingual students. In the present study, no moderation effect was found between domain general skills and response to the ROOTS intervention. Thus, domain general cognitive skills did not significantly influence response to the ROOTS intervention, as students with a range of cognitive skills demonstrated a similar response. These findings demonstrate that effectively designed intervention that utilizes frequent opportunities to respond and receive feedback, concrete representations of mathematics concepts, and systematic practice for building procedural fluency is beneficial for emergent bilingual students at risk for math difficulty regardless of their baseline domain general cognitive skills. As mentioned previously, it is likely that the incorporation of high-quality principals of instructional design and delivery support individual differences in language proficiency and domain general cognitive skills to the extent that students are able to gain early numeracy skills through participation in the ROOTS intervention.
Implications for Research and Practice
Further research is warranted to determine the nature of the intricate relationship between cognitive skills, multilingualism, and development of early numeracy skills. While previous work in the field has demonstrated the robust relationship between domain general skills and mathematics outcomes, the lack of moderation effect demonstrated in the present study has direct implications for future work in this area of research in terms of selection of measures and interventions.
As demonstrated in this study, use of explicit and systematic instruction consistently yields positive results for students at risk for difficulties in mathematics (Gersten et al., 2009). It could be hypothesized that due to the use of ROOTS and its design as an explicit and systematic instructional intervention, varied levels of cognitive and linguistic skills did not significantly impact performance on the intervention. While findings for this study were limited as to the role of domain general skills, further investigation of domain general skills is warranted to better understand the relationship between domain general skills and math outcomes. Moreover, future research would benefit from an expansion of the domain general skills examined. For example, including processing speed, visuospatial skills, and fluid reasoning may provide additional insights related to individual differences in response to mathematics intervention. Continued study with emergent bilingual students with varied levels of English proficiency would be beneficial for establishing which intervention techniques are most effective for emergent bilingual students at risk for math difficulties. Moreover, collecting language proficiency data at pre-test and post-test would also be beneficial as it could allow for further study of the joint development of language and early numeracy skills. When developing interventions for emergent bilingual students, researchers must consider the implications of the cognitive processes associated with language acquisition and the complicated relationship between emerging language skills and mathematics. According to the meta-analysis conducted by Kieffer and Thompson (2018), English Learners in the United States demonstrated significant improvement in NAEP proficiency measures in mathematics between 2003 and 2018. When compared to monolingual peers, students classified as English Learners demonstrated a rate of improvement that was two to three times greater in mathematics in both grades 4 and 8 (Kieffer & Thompson, 2018). As such research will need to be attuned to and investigate multiple factors that influence the mathematics development of emergent bilingual students. While the present study demonstrated that phonological short-term memory did not significantly moderate response to the ROOTS intervention, the relationship between phonological short-term memory and mathematics outcomes highlighted in previous studies and within the correlational results of the present study indicate the importance of the relationship between these two variables (Engel de Abreu & Gathercole, 2012; Fürst & Hitch, 2000). As researchers continue to develop intervention materials for students at risk for math difficulties, it is important that they continue to include principles of instructional design that support students with individual differences in cognitive and linguistic skills, such as emergent bilingual students.
As the field progresses to develop a deeper understanding of the relationship between domain general skills and response to intervention for multilingual students, educators may have the opportunity to make more informed and targeted intervention decisions involving training on domain general versus domain specific skills (Peng & Lee Swanson, 2022). For students who do not respond to Tier II interventions, such as ROOTS, continued study with other interventions is warranted to determine which forms of instructional architecture benefit students with different cognitive and linguistic profiles. Previous studies have established links between gains in training of executive functions and mathematics outcomes, however, lasting effects of such results remain uncertain (Barnes et al., 2016; Clements et al., 2016; Ramani et al., 2017). Replication of interventions targeting cognitive training with multilingual students is essential for determining which interventions would be most beneficial for students with low domain general cognitive skills who are also at risk for difficulties in mathematics. Future work in this area should target a broader range of cognitive variables and expand the measurement net to ensure a more sensitive categorization of students’ cognitive skills.
As the present study was conducted only with kindergarten students, it would be beneficial for this work to be replicated and extended students in different grade levels using different math interventions. While the present study provides a snapshot of the relationship between early numeracy skills and domain general skills for emergent bilinguals in kindergarten, results cannot be generalized in relation to other grade levels. For this reason, replication and extension of mathematics intervention studies with emergent bilingual students across grade levels and intervention programs will provide critical evidence for determining which instructional supports are most beneficial for student simultaneously learning English and mathematics.
Limitations
The present study was conducted as part of a larger efficacy trial for the ROOTS intervention; therefore, a preliminary limitation of the study is that findings can only be generalized in relation to the ROOTS intervention. For this study, participants were assessed using a limited measurement net for domain general cognitive measures. Working memory was measured using the Digit Span Backwards test, which requires students to remember digits. As the students in this study were in kindergarten, and potentially still learning the names of numbers, requiring them to memorize digit names could have caused challenges in terms of measuring working memory. This limitation could have contributed to the limited range in scores on the Digit Span Backwards. Further, as the students were considered emergent bilinguals, it is a limitation of the study that cognitive measures were not administered in both the students’ home and target languages, as students could have encountered difficulties due to language proficiency. This was not considered to be a significant issue for the purpose of the study because the students were all receiving general education instruction and the ROOTS intervention in English. However, future research should include assessments in both students’ native and target languages, particularly when working with students in early elementary grades. Additionally, future research examining intervention outcomes with this population would benefit from employing the use of an active control condition to account for the influence of additional instructional time provided to the intervention group in the treatment condition.
In this study, English language proficiency was measured and standardized by district, however, it could have been beneficial to include a measure of mathematical language, so as to have a consistent measurement of the participants’ understanding of mathematics vocabulary across school districts. Arizmendi et al. (2021) conducted a meta-analysis examining language-focused interventions on mathematics performance for students classified as English Learners in which results indicated small effect sizes overall and significant moderators of grade level, focus of the intervention, and dosage. Future studies working with multilingual students in early elementary grades and across school districts should consider incorporating a measure of mathematical language within the measurement net. For this reason, further analysis of the relationship between language proficiency and intervention outcomes is an essential future direction of research in this field.
Conclusion
As educators seek to provide evidence based academic interventions to culturally and linguistically diverse students, it is imperative that the interventions are designed to account for variations in their academic, cognitive, linguistic needs. Research exploring these complex relationships will shed light on the complex development of mathematics skills for emergent bilinguals. In doing so, we may gain greater understanding of how best to design and deliver interventions for all students to build strong positive early numeracy skills and establish positive trajectories for later math development.