The solution proposed here uses technology to automate some components of the assessment process. The previous section highlighted that the critical challenge is children’s lack of autonomy, making assessment expensive due to the required mediation or subjective due to the teacher-dependent observation process. Hence, an assessment solution should be designed that can overcome children's lack of autonomy and remain affordable and objective. The core idea presented here is to automate the individual mediation processes through mobile devices and gamification to allow group applications of the assessment (all children in a classroom) in a simple, automated, and affordable way. In this regard, tactile mobile devices such as tablets and smartphones can facilitate children's autonomy (Holloway et al., 2013). These devices can provide verbal instructions, represent abstract ideas in interactive visual models, or allow manipulation of virtual objects through touch screens, thus automating specific evaluation processes.

Gamification could also increase the attention span of early education students (as early ages are associated with short attention spans). In fact, educational interest in gamification has grown exponentially due to its ability to engage students innovatively and increase their motivation and creativity (Zainuddin et al., 2020). Although using electronic devices is not recommended at early ages due to the increase of emotional problems such as anxiety and social isolation, these adverse effects are associated with the intensive use of these devices during prolonged periods (Lin et al., 2020). The solution proposed here considers an adequate use of these devices by applying them only occasionally.

The present work aims to develop a proof of concept for an automated assessment tool that facilitates the quick and straightforward acquisition of high-quality information about children's early math abilities by allowing group assessments. This report presents a proof of concept of the "Test of Preschool Mathematics" (TPM), which enables group assessments by automating the mediation process through mobile devices and gamification. Its design aims to assess mathematical skills in prekindergarten and kindergarten levels in alignment with international curricular standards, and it uses digital games to present assessment tasks that evaluate numerical or visuospatial reasoning. We hypothesize that the data collected through this group assessment strategy would be highly informative about children's abilities related to early math skills. In this regard, we will present preliminary evidence about the validity of the TPM by collecting and analyzing data from more than 700 children in prekindergarten, kindergarten, and first grade levels.

The organization of this work is as follows. The next section provides an overview of international curricular standards and their relationship with Chile's curriculum—the background of the present work. Additionally, it briefly describes the expected mathematical learning for children in prekindergarten and kindergarten classrooms. The third section presents a brief description of the TPM development process. The methods section describes the data collection, the sample, and the participants, while the results section presents the data analysis supporting the TPM's criterion and construct validity. Finally, the last section discusses the results and argues that the TPM is an assessment tool that combines efficient resource use and good information quality regarding math performance in early education. The discussion highlights that this tool could present opportunities to assess early childhood math skills and develop timely and appropriate strategies to strengthen under-resourced educational communities.

Developing early numeracy is a powerful predictor of future math performance (Watts et al., 2014, 2017, 2018), mainly because it lies at the foundation of mathematical structure, allowing us to describe generalizations of predictable sequences. Consequently, many countries have emphasized opportunities to strengthen students' mathematical learning in general and also in early childhood (Clements & Sarama, 2011; Melhuish & Petrogiannis, 2006). Indeed, progress has been made in consolidating robust curriculum standards for mathematics teaching at the international level. One of the curricular frameworks of reference is the Curriculum Focal Points (NCTM, 2006), which drove the emergence of collaborative environments promoting the creation and development of high-quality tools such as assessment instruments, curriculum frameworks, and instructional materials (e.g., books and software). For example, by drawing on the Focal Points curriculum, the Common Core State Standards (CCSS, 2010) have been massively adopted in different U.S. territories, creating a common framework that facilitates and promotes collaboration and curriculum development for a coherent educational system.

A comparative analysis between the Curriculum Focal Points and the Common Core State Standards concludes that, although the two documents differ in terms of their levels of specificity, both standards are highly comparable in their coherence, focus, and content (Achieve, 2010). According to the NCTM website, comparative analyses between the Canadian mathematics curriculum and the Focal Points are highly similar. Regarding math education in prekindergarten and kindergarten classrooms, these documents state that the majority of instructional time should focus on (1) representing, relating, and operating with whole numbers and (2) describing shapes and spatial relationships (CCSS, 2010).

In coherence with these standards, Chile's curriculum (MINEDUC, 2018) strongly emphasizes the development of number knowledge and aims to develop reasoning skills about spatial relationships. A detailed description of these learning goals is provided as supplementary material (see Navarrete-Ulloa et al., 2025S-b, Table S2). Chile’s curriculum aims to teach students numerical skills, using numbers to represent quantities and solve quantitative problems. The learning objectives include comparison relationships; the notion of number, quantifier function, and numerical sequence; concrete, pictorial, and symbolic (COPISI) numerical representations of quantities; and solving problems through addition and subtraction operations using concrete and pictorial representations (up to 10). Regarding spatial relationships, the learning objectives include the creation of two- and three-element patterns, identification/classification of shapes and objects by their attributes, seriation of objects by various attributes (such as height, length, and capacity), and spatial/temporal orientation and representation of objects from different perspectives (top, bottom, sides).

We reviewed the literature for tasks to assess early mathematical skills. Experts analyzed each task to verify its alignment with the Chilean early education’s learning objectives (MINEDUC, 2018). Not all target learning objectives could be covered, so we designed specific tasks to cover them by drawing on theoretical frameworks of analogical reasoning and analogical representations (Kalra & Richland, 2022; Navarrete & Dartnell, 2017; Navarrete et al., 2018; Navarrete-Ulloa & Munoz-Rubke, 2022; Ramani et al., 2012, 2020; Richland et al., 2004). Analogical reasoning consists in comparing the structure of two entities and is particularly useful for processing abstract domains through concrete representations (e.g., love is a journey) by identifying a structural correspondence between a known concrete domain and the abstract concept (Andrews et al., 2006; Halford et al., 2010; Holyoak, 2012; Lakoff, 2006). For example, a balanced scale can be a friendly representation of some first-degree equations (Araya et al., 2010). This property of analogical reasoning helps to design concrete representations of abstract and complex concepts (Gentner, 2010; Gentner & Colhoun, 2010; Navarrete & Dartnell, 2017; Navarrete-Ulloa & Munoz-Rubke, 2022; Richland & McDonough, 2010; Richland et al., 2012).

The methods described above led to sixteen proposals for digital tasks to measure math abilities in children. These proposals consisted in a conceptual model of the task with focus on assessing the target learning goal along with a basic design of the user interface and its functionality. Each proposal was presented by a member of the team and evaluated by five experts who used a scoring rubric built on the following criteria: (a) factibility of development, (b) face validity based on literature support, and (c) intuitiveness of the user interface. From these proposals, twelve were selected for further development. The five experts conducted three cycles of review, evaluation, and feedback to develop these tasks further and improve them. Three of these experts had a doctoral degree and research experience, whereas the two other experts had a master's degree and preschool classroom experience. These experts then selected the best nine tasks based on the same criteria described above. Afterward, the development team implemented these nine tasks as digital games. To explore the usability and validity of the assessment, we carried out a piloting process where a group of children performed all nine digital games. One task was eliminated because its average duration was deemed too long for children to complete in actual testing. Another task was eliminated because its user interface was not intuitive enough and difficult to use, and thus, its validity was called into question.

The seven remaining tasks compose the TPM and cover nine out of the twelve learning goals of the Chilean curriculum in early education (see Navarrete-Ulloa et al., 2025S-b, Table S2). Furthermore, these seven tasks are strongly aligned with international curriculums such as the Focus Points (NCTM) and the Common Core State Standards (CCSS), as these tasks focus on (1) representing, relating, and operating with whole numbers and (2) describing shapes and spatial relationships (CCSS, 2010).

Table 2 briefly describes the seven TPM tasks, which, through narrative, illustrations, and animations, emulate digital games. The graphic and interaction design of each task had an edutainment focus implemented by an illustrator and a programmer, both experts in designing and implementing games for children. To lower children's anxiety, there was a progress indicator on the screen; the character's voice was calm and soft, and a ludic story surrounded each game, where the characters had to achieve a critical mission.

From now on, we will refer to each task of the TPM as a game. Each “game” (task) consists of "ten levels" (ten items) scored from 0 to 100. In other words, the TPM presents ten levels for each game, but each level differs due to slight modifications to modulate the difficulty level (see Navarrete-Ulloa et al., 2025S-b, Table S2). The TPM has seven games, each one with ten levels, meaning that the TPM presents 70 items to children. The TPM does not give feedback to children about the correctness of the answered item, but congratulates them each time they complete a game. The structure of every game has initial directions (given by a character), an example (accompanied by an animation showing the interaction with the screen), and the ten game levels (the task items). The Supplementary Materials contain the audio transcript (translated from Spanish) of the “Numerical Comparison” game to illustrate the types of questions posed to children (see Navarrete-Ulloa et al., 2025S-b, Table S3). Additionally, the ten game levels had a progression of difficulty: The first was the easiest, and the last was the most difficult. Nevertheless, every child had to answer all assessment items, since there was no algorithm for selecting the items presented to children according to their previous performance.

This paper argues for the TPM’s validity based on the analysis of the data collected from a sample of Chilean children aged 4 to 6 years old in April 2022. This data collection was conducted just after the COVID-19 quarantine. It raised concerns that the data could reflect atypical learning progress, as participants had been isolated in their homes for almost two full years. For this reason, we decided to conduct a second round of data collection after roughly three months of regular school attendance. The data collected in this second round was thus expected to better reflect typical learning progress for children of these ages.

Measurements of children's mathematical learning were conducted using the TPM in a sample of 824 participants in Prekindergarten (PK), Kindergarten (K), and Grade 1 (G1) classrooms, recruited from five educational centers located in O’Higgins Region, Chile. In the first and second rounds of data collection, 718 and 653 children participated, respectively. In what follows, we report the results of the second round of data collection and provide analog results for the first round in the Supplementary Materials (see Navarrete-Ulloa et al., 2025S-b). Participants’ mean ages and percentage of girls for each educational level are presented in Table S9 (first round) and Table 3 (second round). The protocols of this research were approved by the Scientific Ethics Committee of Universidad de O’Higgins, certificate 05/2019, and the guardian of each participant signed an informed consent. Each participant also gave informed assent before the start of the activities.

The design of the TPM’s implementation in classrooms considered the acquisition of students’ lists beforehand. Hence, typing a test identifier in the TPM showed the students’ list, which facilitated a fast start of the assessment process and diminished the application time in classrooms. When children finished a game, their performance data was uploaded to an internet cloud for subsequent analysis. For the interested reader, a prior work details the TPM architecture (Navarrete-Ulloa et al., 2023).

Data was collected in the classrooms by assessing complete groups during each visit. For data collection, two research assistants visited each classroom carrying the necessary technological equipment (tablets, mobile internet, etc.). When entering the classroom, the assistants connected a router to provide wireless internet to the classroom. After the assistants introduced themselves and gave group instructions, they provided each child a tablet with the app ready for use and headphones. Afterward, they approached each child to type the test identifier and selected the child’s name to start the first game. The assistants followed a protocol including the following guidelines: a) Provide instructions where the TPM is presented as a fun game that they brought for them to play; b) In case they notice a child does not understand the instructions, they should approach the child and explain the task at hand personally. However, they should try their best for these explanations not to influence the child’s performance; c) For each child, there should be a five-minute break in the middle of the assessment process (at the end of one of the games).

Children carried out the TPM games autonomously using a tablet, with few exceptions where an adult was required to mediate the test application. In PK classrooms, the assessment process lasted less than 80 minutes. In K and G1 classrooms, the evaluation process lasted less than 60 minutes. The application of the games in each classroom was divided into two sessions, separated by a five-minute break and monitored by the leading researcher. Ninety-four percent of the participants completed both sessions on the same day, while 6% completed the sessions on different days (M = 14 days; SD = 6.4; Range = 1-21 days). Regarding class size, classrooms of the PK, K, and G1 levels had an average of 17.6 (range 6-29), 24.4 (range 15-33), and 26.8 (range 22-33) children, respectively.

To verify the TPM’s criterion validity, the analyses presented below seek to confirm that the TPM can detect significant differences between the three educational levels considered in the present study (PK, K, G1). Concerning construct validity, confirmatory factor analyses were performed to show that the TPM captures two dimensions of children's mathematical thinking: numerical thinking and spatial reasoning (according to international curricular standards). The test has been designed to facilitate diagnostic and formative assessment procedures two years before school; therefore, it is valuable to interpret the data collected in terms of this objective. The three cohorts in our sample represent different assessment points in the learning trajectory of the educational period of interest. The first assessment point is seen as a diagnosis for entry to early education (Pre-Kinder cohort; PK), the second assessment point is a measurement of progress at the end of the first year of early education (Kinder cohort; K), and the third assessment point evaluates the competencies acquired at the end of early education (first grade cohort; G1). Thus, our selection of cohorts allows us to conceptualize three different levels of student progress regarding their early math skills.

Table 3 shows descriptive statistics of the sample of children in the second data collection round. Table S4 (Navarrete-Ulloa et al., 2025S-b), provides Pearson correlations between each game’s scores and age.

We first looked at item discrimination levels per task. We computed discrimination indices for each item and task by subtracting each item’s average scores for the 27% of participants with top and bottom overall task scores (Ebel & Frisbie, 1991). In what follows, we considered an item’s discrimination as good or not based on a threshold of 30% (Ebel & Frisbie, 1991). Number Line Estimation was the task with the least number of good items, while all the items in Numerical Comparison and Perspective Visualization were above the threshold. Table 4 presents the number of items with above-threshold discrimination and Cronbach’s alpha coefficients for each task. The Numerical Comparison and the Perspective Visualization tasks obtained the lowest alphas (about .60). In contrast, the Pattern Creation task obtained the highest value (about .80). From this point onwards, we discarded the nine items that did not reach a good level of discrimination.

As already mentioned, the first collection of data (T1) was performed in April 2022, and the second one after roughly three months (T2). Table 5 presents the correlations between scores of T1 and T2 for each of the tasks, considering only the subset of children who provided complete data in both times of testing. Table 5 shows strong correlations between the scores of each task at both times (ranging from .37 to .68). However, these correlations should not be taken as estimates of test-retest reliability, due to the long time elapsed between the two measurements: Nunnally and Bernstein (1994) have suggested a 2-week interval for estimating test-retest reliability regarding achievement-type tests. Hence, correlations in Table 5 more likely suggest a change in the sample. For example, a comparison between Figure 1 and Figure S1 (Navarrete-Ulloa et al., 2025S-b) shows that children improved their knowledge during the 3-month period. Recent research on analysis of pretest-posttest data indicates that the acquisition of knowledge between T1 and T2 is associated with lower correlation values between the scores obtained in T1 and T2 (Navarrete-Ulloa, 2024). Hence, these relatively low correlation values more likely reflect a change in the sample due to learning, a better adaptation to school environment, children’s developmental changes, among other factors that are confounded with the time elapsed. Nevertheless, Table 5 show significant correlations between all the TPM tasks, and suggest that all of them measure somewhat stable constructs.

Figure 1 presents the average scores obtained by children in each educational level and game. As expected, average scores on all the TPM games increased along with educational level. We used t-tests to separately compare children’s scores across consecutive educational levels (PK vs. K, K vs. G1) for each game. Since this amounts to 14 contrasts, we corrected for multiple comparisons using the Holm-Bonferroni procedure (Holm, 1979). This process showed statistically significant differences between all three educational levels in all but one case: only the Number Line Estimation game failed to show a significant difference (between PK and K, p = .08; all other ps < .0003).

We then looked at the seven-game set and asked about its factorial structure. According to our theoretical framework, we hypothesized that these tasks would be grouped into two subsets: numerical tasks (Numerical Comparison, Concrete Pictorial Symbolic Mapping, Additive Problem Solving, and Number Line Estimation) and visuospatial tasks (Pattern Creation, Attribute Seriation, and Perspective Visualization). A confirmatory factor analysis considering these two factors in predicting total scores per task showed a good degree of fit: RMSEA = .015, SRMR = .017, CFI = .999, TLI = .998, χ²(13) = 14.90, p = .31. Table 6 shows the factor loadings for this model, revealing that Number Line Estimation was the only task with a loading smaller than .60 in its respective factor.

To understand how the TPM behaves at the different individual tasks and grade levels, we computed Cronbach’s alpha values separately for each task, dimension, and level. Table 7 shows that tasks improve their internal consistency with grade level, that most tasks show acceptable values at the target level (K), and that two tasks show very low alpha values in the youngest age group (PK): Numerical Comparison and Perspective Visualization. It should be noted that both dimensions (Numerical Thinking and Visuospatial Reasoning) have good internal consistency for all levels.

Finally, we evaluated whether the two factors underlying the TPM show measurement invariance between boys and girls, that is to say, if the factor structure is comparable between these two groups (Kline, 2023). There are different measurement invariance levels: configural, metric, scalar, and strict (Bialosiewicz et al., 2013). Each of these introduces additional requirements to the previous ones. In configural invariance, both groups exhibit the same factor structure. In metric invariance, factor loadings are equal between groups. In scalar invariance, factor loadings and intercepts are equal between groups. Finally, in strict invariance, factor loadings, intercepts, and residuals are equal between groups. The data should display at least scalar invariance to meaningfully compare factor means for boys and girls.

We computed four multigroup (boys/girls) CFAs, one per each invariance level. A chi-square difference test revealed no significant difference between the configural invariance model and the metric invariance model (χ²_diff(5) = 6.3, p = .27), and a significant difference between the metric invariance model and the scalar invariance one (χ²_diff(5) = 20.6, p = .001). In addition, the metric invariance model presented a good degree of fit: RMSEA = .033, SRMR = .033, CFI = .993, TLI = .990, χ²(31) = 41.8, p = .09. Altogether, this suggested that the two-factor model for the TPM exhibits metric invariance.

We analyzed each factor's measurement invariance separately to shed further light on the TPM’s properties. For the Numerical Thinking factor, the chi-square difference test revealed no significant differences between models (χ²_diff(3) = 3.3, p = .34; χ²_diff(3) = 3.5, p = .32; and χ²_diff(4) = 7.8, p = .10, respectively). Since the strict invariance model displayed a good fit (RMSEA = .029, SRMR = .040, CFI = .993, TLI = .994, χ²(14) = 17.9, p = .21), we concluded that this factor exhibits strict invariance. Instead, for the Visuospatial Reasoning factor, the chi-square difference test indicated no significant difference between the configural invariance and the metric invariance models (χ²_diff(2) = 2.2, p = .34), and a significant difference between the metric invariance and the scalar invariance models (χ²_diff(2) = 17.8, p = .0001). Together with the good fit indices of the metric invariance model (RMSEA = .016, SRMR = .021, CFI = .999, TLI = .999, χ²(2) = 2.2, p = .34), we concluded that this factor shows metric invariance.

In summary, the measurement invariance analysis suggests that the TPM’s Numerical Thinking factor is measured with the same degree of precision for both boys and girls (Kline, 2023). In contrast, the Visuospatial Reasoning factor only exhibited metric invariance, indicating that means between boys and girls should not be directly compared.

In the Supplementary Materials, we provided percentile scales for the Numerical Thinking and Visuospatial Reasoning dimension scores and total TPM scores for the entire sample (Navarrete-Ulloa et al., 2025S-b, Table S5). We also provide percentile scales for the PK (Table S6), K (Table S7), and G1 (Table S8) subsamples.

The participants in this study have demographic characteristics associated with the geographic region where it was carried out, so the results should not be generalized to the population of Chile in general. This point is relevant because although the scores obtained through this assessment instrument can be used to make specific comparisons between students (or schools), the lack of further information at the national level that identifies the levels of competence obtained makes it challenging to interpret the test results objectively. More details about this would enable the interpretation of the results to facilitate educational decision-making relevant to the student's (school's) reality and appropriate to their regional or national environment.

Considering that the children took the first TPM during the first days of the 2022 school year, which coincided with the return to face-to-face classes interrupted by the COVID-19 quarantines, the scores of this first data collection could have reflected atypical mathematical learning trajectories in contrast to an everyday schooling context. We thus decided to present the analysis of the second data collection, expecting it to reflect a more typical mathematical learning trajectory. While overall scores were indeed higher in the second data collection (compare Figure 1 and Figure S1, Navarrete-Ulloa et al., 2025S-b), this difference might not be only due to math learning and development but also due to children being more familiar with the teaching methodology and school environment, reflected in an improved relation with the teacher and better behavior in the classroom. Children feeling more at ease in the classroom might have provided better conditions to conduct the second TPM assessment and might partly explain the improved performance. Future research should establish whether there is any significant difference in the score distributions relative to cohorts whose educational process is typical. Researchers aiming to generalize the results presented here would need to evaluate a population at this academic level with a stratified sample according to key educational variables such as geographic region, administrative unit, gender, and educational level of participants. Better yet, in the future, a more rigorous validation study could consider better standards of criterion validity and add discriminant validity criteria.

This work presented evidence of reliability and construct validity resulting from data analysis associated with two different data collections. This redundancy is because the first data collection was conducted at the end of a lockdown period of almost two years (due to the COVID-19 health contingency). Consequently, participants lacked school experience, their levels of autonomy were compromised, and their learning trajectories were likely atypical. These factors suggest that the evidence of construct validity and reliability from the first data collection might be underestimated (see Navarrete-Ulloa et al., 2025S-b). There is a similar effect for the second data intake, but in the opposite direction, as most students had prior experience with the instrument. This suggests that the evidence associated with the second data collection might be overestimated. By analyzing both data collections, we can ensure that the evidence of construct validity and reliability for the TPM is bounded by the levels of evidence associated with these two data intakes.

The TPM’s design strongly aligned with worldwide curricular guidelines, which point out that prekindergarten and kindergarten levels should spend the majority of time learning numbers, their relations and representations (numerical reasoning), along with spatial concepts, their relations and logic (spatial reasoning) (Achieve, 2010; MINEDUC, 2018; NCTM, 2006). Nevertheless, those researchers and practitioners focused on assessing only numerical skills might want to drop the dimension of spatial reasoning to obtain a shorter version of the TPM which might be more efficient to this aim. Using different frameworks would have suggested measuring other important precursors for arithmetic learning, such as verbal counting and ordinality (e.g., Merkley & Ansari, 2016). Still, our theoretical framework and the technology constraints prioritized measuring the seven tasks reported in this study. In this context, estimating the number line tasks has been widely used in the infant mathematical learning literature (Ramani & Siegler, 2008; Siegler & Ramani, 2009; Siegler et al., 2012). Our results indicate that the associated task (NumLine) has standardized factor loading λ = 0.35, which is much lower than the loadings associated with the other three number sets. Hence, there is room to improve the TPM by modifying this estimation task. Along the same lines, the invariance measurement analysis indicates that the Visuospatial Reasoning factor shows only metric invariance, implying that at least a subset of these items may be approached differently by boys and girls, so the comparisons of group means may be contaminated (Gregorich, 2006). Further research is required to understand the origin of such invariance results for this TPM dimension.

Although the TPM technology assesses large groups of children more efficiently, each child experiences an activity of around sixty minutes. This long period of activity might cause exhaustion, and such a factor might have impacted the results presented here. Still, in our experience, the tablet and game-based design succeeded in maintaining children's attention, and we did not see signs of exhaustion in kindergarten. However, some signs of fatigue were observed in prekindergarten. In this context, practitioners might want to apply the two assessment sessions on two separate days, especially for younger children. Additionally, since the TPM does not require a teacher to mediate between a child and their assessment, it cannot capture the learning dimensions that are more readily observable through social interaction with the adult or between children. Concerning criterion validity—that is, the degree of effectiveness with which the TPM detects levels of early mathematical ability—convergent, retrospective, and predictive validity criteria should be studied. For convergent validity, the correlation between the TPM scores and a test of mathematical skills should be estimated, for example, the "TEMA-III battery" (Hoffman & Grialou, 2005), a widely used international standard of early mathematical skills which has been adapted to Spanish. For retrospective and predictive validity, we suggest a comparison of participants’ TPM scores with their overall school performance in the academic semester immediately prior to (retrospective) and at the end of (predictive) the academic semester in which the TPM assessment is performed. Concerning discriminant validity, it is essential to distinguish whether the TPM measures early mathematical skills’ development or the general cognitive development of boys and girls. For this purpose, a picture vocabulary test such as the PPVT-III may be used, expecting higher correlations between the TPM and TEMA-III and lower correlations between the TPM and PPVT-III.

This paper highlights the difficulties in conducting mathematical learning assessments in early childhood, especially for under-resourced educational communities with large student groups. Considering international curricular standards for early childhood mathematics, this paper presented the Test of Preschool Mathematics (TPM), which uses automation and gamification technologies to deliver a pleasant, effective, and efficient assessment experience. Although there is room for improvement of the TPM, the current version meets the minimum desirable requirements that could provide valuable information as one of the inputs considered for the assessment of 4- and 5-year-old children’s development of mathematical abilities.