Fraction Errors in a Digital Mathematics Environment: Latent Class and Transition Analysis

Why Do Students Struggle With Fractions?

Prior research has offered a number of reasons why students struggle with fractions: fractions are multifaceted (e.g., Pantziara & Philippou, 2012), fraction algorithms are far more complex than natural numbers (e.g., Hiebert, 1992; Stafylidou & Vosniadou, 2004), and students often experience a gap between conceptual and procedural knowledge (e.g., Bailey, Siegler, & Geary, 2014; Jordan et al., 2013; Jordan, Resnick, Rodrigues, Hansen, & Dyson, 2017). The following sections explain how these factors contribute to student difficulty with fraction understanding.

What Types of Errors Do Students Make?

It is important to understand both why students struggle with fractions and how they struggle with fractions. To understand how students struggle with fractions, researchers have looked at the errors students make when solving fraction problems (e.g., Ashlock, 2001; Bottge et al., 2014; Brown & Quinn, 2006; Malone & Fuchs, 2017). To do this, researchers give participants a test/activity and observe errors made. Most tests given in this manner have been free response (e.g., 1/2 + 3/4 = ______), although some have included multiple choice (e.g., Brown & Quinn, 2006). Although this procedure is used among most fraction error researchers, there are differences in methods in terms of what skills are tested and how errors are addressed. For example, some researchers have focused on specific fraction knowledge, like ordering or adding/subtracting fractions (Bottge et al., 2014; Malone & Fuchs, 2017), whereas others have focused on a broad range of fraction knowledge (Ashlock, 2001; Brown & Quinn, 2006). Additionally, some researchers define categories of errors a priori and then examine tests to see how often students make these errors (Bottge et al., 2014; Malone & Fuchs, 2017), whereas other researchers first give tests and then code and define errors from student answers (Ashlock, 2001; Brown & Quinn, 2006). Regardless of the method used, two broad categories of fraction errors have emerged, general misunderstanding of the concepts of fractions and improper application of whole number rules to fraction problems (e.g., Ashlock, 2001; Bottge et al., 2014; Brown & Quinn, 2006; Malone & Fuchs, 2017). These a priori errors and their origins will be further discussed in the context of the results of this study (see Results sub-heading Fraction Errors).

Motivation for Current Study

There is an extant body of work identifying student errors in fraction problems and examining why students might make these errors (e.g., Ashlock, 2001; Brown & Quinn, 2006). Although these studies have contributed to our understanding of the structure of student struggles with fractions, they have largely viewed student errors in isolation, without consideration of the instructional context. Work examining student fraction errors using assessments embedded in authentic educational contexts might provide information more immediately applicable to the classroom. Additionally, previous research has used student errors to enhance instruction (e.g., Ashlock, 2001), but often focuses on individual errors and what they mean. However, it is likely that students who make one type of error also make another type of error. By classifying error profiles, teachers may be able to more efficiently address a combination of errors. Finally, examining profiles of errors both before and after an instructional event may reveal both typical patterns of change and those errors and patterns that may be resistant to instruction. This can lead to refinement in instruction aimed at common profile groups and the identification of smaller groups of students for more targeted intervention. Similar research has been conducted to see what algebraic errors are persistent throughout instruction, but this has not yet been extended to fractions (Booth, Barbieri, et al., 2014).

Current Study

Using data from within the digital mathematics software ST Math, we identify common fraction errors, examine co-occurrence of errors to define student error profiles, and investigate the changes in profiles from before to after instruction. Specifically, we ask:

We expect that certain errors will be more prevalent in the pre-quiz due to a naïve understanding of fractions, such as the ratio error or whole number ordering (Ashlock, 2001; Malone & Fuchs, 2017). We also expect errors related to fraction magnitude to be persistent into the post-quiz due to the construct’s complexity (Booth, Barbieri, et al., 2014; Rinne et al., 2017; Siegler et al., 2011).

Context: ST Math

ST Math, created by MIND Research Institute (MIND), is an interactive instructional software for computers and tablets that is based on theory that suggests that the ability to visualize mathematics concepts leads to better conceptual knowledge and performance (see Geary, 1995; National Research Council, 2005; Shaw & Peterson, 2000). ST Math has previously been shown to result in small improvements in math achievement, especially on topics involving number sense (Rutherford et al., 2014; Schenke, Rutherford, & Farkas, 2014). ST Math is currently used in 45 states with over one million students and is designed to align with both Common Core and relevant state standards, including those in Florida, the site of this study.

Within ST Math, students progress through a number of objectives focused on specific math concepts. The ST Math content follows a hierarchical pattern: objective, sub-objective, game, level, puzzle. Within the objective/sub-objective, there are a variety of games that use the same imagery and design throughout their levels. Each of these games contains between one and 10 levels that increase in difficulty. Within each level, students complete interactive puzzles, which are the delivery method for the mathematics content. For each level, the student has between one and three lives—if they answer more puzzles incorrectly than they have lives, they are removed from the level and can chose to replay it or replay a previously-passed level. Before students begin the objective, they must complete a five-question multiple choice pre-quiz on that objective’s content. After demonstrating mastery of the content within an objective by successfully completing all levels within, the student then completes a five-question multiple choice post-quiz that mirrors the pre-quiz, question-for-question in topic, but uses different specific examples and numbers. The pre- and post-quizzes have either three or four answer choices for each question (see Table 5 on page 15 and Figure 1 on page 25 for examples).

The third-grade curriculum for the 2015-2016 Florida version of ST Math had 23 required objectives and eight optional objectives. These objectives covered content such as place values, addition and subtraction, multiplication and division, and fractions. The three objectives covering fraction content were part of the required curriculum and began approximately 65% of the way through the curriculum.

Participants

Participants were third graders from a school district in central Florida participating in an NSF-funded project relating gameplay within ST Math to student achievement and motivation. Students within the district played ST Math as part of their normal instruction during the 2015-2016 school year. The sample (N = 4,290) was limited to third graders with pre- and post-quiz data from the three fraction objectives (n = 2,187), and who also had district-provided demographic and achievement data (n = 1,431). The 1,431 students attended 75 different schools, 54% were male, 60% were white, 16% were Hispanic, 13% were black, 5% were Asian, and 46% qualified for free or reduced lunch (see Table 3). The analysis sample differed at statistically significant levels (ps < .05) on most demographics from the larger sample of third graders. There were fewer students with disabilities, who qualified for free/reduced lunch, were English Language Learners, and who were black. Conversely, there were more gifted students, Asian students, and White students. Because the sample was limited to students that had completed all three fraction objectives, the reduced sample may have a lower percentage of students using ST Math for remedial reasons.

Measures of Fraction Errors

As implemented in Florida during the 2015-2016 school year, the ST Math curriculum aligned with the Florida Standards, which introduced fractions in the third grade (Mathematics Florida Standards, 2014; MIND Research Institute, 2017). The three fraction objectives within the ST Math third grade curriculum during this year were Fraction Concepts, Fractions on the Number Line, and Comparing Fractions. The pre- and post-quizzes for these objectives tested fraction concepts and algorithms. Table 4 describes the types of questions within each objective’s quizzes.

Procedure

Fraction Errors

Three of the 10 error codes were developed a priori—illogical sizing/spacing (Ashlock, 2001), ratio (Ashlock, 2001), and whole number ordering (Malone & Fuchs, 2017)—and the other six were a posteriori—complement, filler, incomplete information, reciprocal, reducing fractions to whole numbers, and same numerator/denominator ordering. Table 5 summarizes the types of questions and the errors possible within each question type. Overall, students made an average of seven errors on pre-quizzes and four errors on post-quizzes (out of 15 questions each for both pre- and post-quizzes, see Table 6). Table 7 details the proportions of times each error was made and how many students made it. See Figure 1 for example of errors.

Pre-Quiz Latent Class Analysis

As noted above, nine types of errors could be made in the third-grade quizzes—complement, filler, illogical size/spacing, incomplete information, ratio, reciprocal, reducing to whole numbers, same numerator/denominator ordering, and whole number ordering. Analyses were run in Mplus using the proportion of times each of the nine errors were made to identify categorical latent variables.

Latent Transition Analysis

For the latent transition analysis, a post-quiz model was constrained to three classes for two reasons. First, the three-class model had the lowest sample size adjusted BIC value (see Appendix B). Second, it allowed the number of classes to remain constant across the pre- and post-quiz models. The three-class post-quiz model was estimated by regressing the post-quiz model onto the pre-quiz model. This regression determined the thresholds for the post-quiz classes. The thresholds determine how often each error is made in each class, i.e., what the prominent pattern of errors is for each class. Additionally, the regression determines the transition probabilities based on the models, i.e., how likely a student is to remain in a given class for pre-quiz and post-quiz.

Research Question 1: Fraction Errors in St Math

Within this study, we expand upon previous fraction error analysis in both the context and content of our questions and in the identification of our errors. First, breaking from the tradition of using open-response researcher-administered questions (e.g., Ashlock, 2001; Bottge et al., 2014; Brown & Quinn, 2006; Malone & Fuchs, 2017), we used a multiple-choice test provided in the course of students’ actual instruction. Although the multiple-choice nature of our instrument limited the scope of errors students could make, it allowed us to examine errors in-situ within an authentic educational context. Additionally, if designed carefully and with common errors in mind, the use of such tests may make error identification more efficient—if researchers can make multiple choice answers with specific errors in mind, they can assume the error by the answer choice instead of coding each written answer. However, because students were required to pick an answer choice, selecting a random wrong (or right) answer was more likely to occur. It is therefore possible that students may have picked an answer choice that matched an error code without actually following the logic to get that error, resulting in more noise. This may be especially true of students new to fraction problems, who, on open-ended questions, might not provide any answer. By forcing an answer choice, students who are new to fractions may make multiple errors. In fact, it appeared that this may have borne out in the propensity of lower-performing students within our pre-quiz Distributed Errors Class.

As our second contribution to the nature of error identification, we described new categories of errors. Of the nine errors coded, six of them were a posteriori in that they were not previously described in research on fraction errors—complement, filler, incomplete information, reciprocal, reducing fractions to whole numbers, and same numerator/denominator ordering error. These a posteriori errors were made just as often, if not more, than those assigned a priori codes from prior research. It remains to be seen if these errors would be found outside of the context of ST Math or in non-multiple-choice examinations.

As the multiple-choice nature of the ST Math quiz questions may have influenced the errors that arose, so too may have the visual nature of the questions. The focus within ST Math on visual-spatial instruction is reflected in a number of quiz questions that used visual representations, such as shaded figures or number lines. These questions may elicit different errors than those seen in prior studies, which, other than writing factions based on visual models, relied primarily on questions that were symbolic (e.g., 1/2 + 3/4 = ____; Ashlock, 2001; Bottge et al., 2014; Brown & Quinn, 2006; Malone & Fuchs, 2017).

Research Question 2: Models of Struggling Students

Three classes of errors were identified using LCA at pre-quiz—Few Errors, Distributed Errors, and Whole Number Ordering Error. Students in the Few Errors Class tended to have the highest pre-quiz averages. The students in this class made few errors, and when they did make errors, tended to make the same error the majority of the time. Although the LCA identified this group as a class, many of the students had little in common regarding the type of errors they made—instead, they were joined together merely by their propensity to make few errors. It is intuitive that high performers would make few errors overall and that these errors may not be consistent among the high performers. There are two possibilities we offer for why these students did not make similar types of errors. First, it may be that even high-performing students experience moments of carelessness, and therefore the distribution of errors these students make is the result of a mostly random process. Alternately, it may be that there are sub-groups within this class that cluster on their tendency to make certain types of errors, but our instrument was not sensitive or extensive enough to capture these differences.

The Whole Number Ordering Error Class contained less than five percent of students who did not make the whole number ordering error and almost 60% who made it at least half of the time. The high rate of this error—ordering only numerators or denominators without considering the relationship between the two—is likely due to a naïve understanding of fractions and an improper reliance on knowledge of whole numbers and their ordering schema (Malone & Fuchs, 2017). Although other errors were made by students in this class, none were as concentrated as the whole number ordering error. Students in the WNO Class tended to score lower on the pre-quizzes and were less likely to be labeled as gifted.

Lastly, the largest group of students were placed in the pre-quiz Distributed Errors Class. This class made a wide range of errors but did not make most errors more than 25% of the time. The seemingly random nature of their errors may indicate that most of their errors are due to a general lack of fraction knowledge and unfamiliarity with fraction problems. This conclusion is further supported by students in this class having the lowest pre-quiz averages, indicating a lower level of fraction knowledge. This may indicate that students with little fraction knowledge do not make errors that indicate one type or a few types of conceptual or procedural misunderstandings, but instead, exhibit a pattern that may be more indicative of guessing. It may be that these students would be those that would leave open-ended questions blank, and therefore would not be attributed to a specific error or classified into an error profile on the types of exams frequently used in prior fraction error research.

Research Question 3: Class Membership Changes Pre to Post

Both class membership and class structure differed between pre- and post-quizzes. This was expected, because the number of total errors being made decreased as the students learned from ST Math. Two of the three pre-quiz classes remained in the post-quiz—the Distributed Errors Class and the Few Errors Class. Interestingly, the percent of students in the Few Errors Class increased by 10% from pre- to post-quiz. The class also changed slightly in its composition of errors, as depicted in Figure 4, wherein the pre-quiz error distribution is shown in the left columns and the post-quiz in the right.

The most notable change from pre to post is that the proportion of errors increased for the filler and illogical size/spacing categories in both classes and the complement error in the Distributed Errors Class. For the illogical size/spacing error, students who make this error have some aspect of the question correct—they understand that 1/5 is one of five parts (for example) but do not recognize that the parts must be equal, or they correctly order fractions but do not properly space them on the number line. Thus, the students who make this error may have a partially developed understanding of fractions. Alternately, students who selected answer options in line with this error may have been confused at encountering what may be viewed as a tricky option. Questions that elicit this error are necessarily complex–changing the spacing of fractions on the number line may have taxed student working memory. The filler error may have been more prevalent in the post-quiz classes because students were not experiencing a specific misconception, rather they likely made careless errors. Thus, although they have applied certain misconceptions in the past, those misconceptions are suppressed and thus the student relies on a different strategy, in this case, choosing an answer choice that does not fit under a specific misconception (see Booth, Barbieri, et al., 2014, describing the arithmetic error for an example of an error not representing a misconception). This theory is supported by the Overlapping Waves theory that states that children’s strategies fluctuate as they learn (Siegler, 1996).

Only movement into the Few Errors class could be predicted by student characteristics. Students who had higher pre-quiz averages and were labeled as gifted were more likely to move from the Distributed Errors Class in the pre-quiz to the Few Errors Class in the post-quiz. Gifted students, in addition to English Language Learners, were also more likely to move from the Whole Number Ordering Class to the Few Errors Class. Movement between the classes from pre- to post-quiz followed a pattern similar to the one found with Resnick et al.’s (2016) fraction magnitude growth trajectories. They found three growth classes—consistently accurate, inaccurate with growth, and inaccurate with minimal growth. Although our project did not specifically look at growth, we did find that students who made few errors in the pre-quiz continued to make few errors in the post-quiz (similar to the consistently accurate class), students moved from one of the lower performing classes—Distributed Error and WNO—to the post-quiz Few Errors class (similar to the inaccurate with growth class), and some students were never in either of the Few Errors class (similar to the inaccurate with minimal growth class).

Limitations

As noted, although the multiple-choice nature of the questions allowed us to collect authentic educational data from a large number of students, it also provided a limitation in that it constrained the possible number of errors to those already provided within the software. It also allowed for the number of times an error could be made to vary. For example, the same numerator/denominator ordering error could only be made on one specific type of question—comparing fractions. Similarly, the number of times an error could be made varied between the pre- and post-quiz, preventing a true measure of difference, although this difference was small (differences ranging from zero to two possible errors). Additionally, although we grounded our coding decisions in both prior research of fraction errors and understanding of mathematics and mathematics education, we lacked insight into each student’s thought process as they solved the problems. Future studies may rectify this shortcoming by providing for methods such as cognitive interviewing to complement error coding.

Implications and Future Directions

Our results are immediately applicable to the digital platform from within which the data came, ST Math. First, we can work with the platform developers to reduce the number of filler items in ST Math quizzes and provide for a variety of error options with which to identify student misconceptions. These same guiding principles can be used by other test and platform developers to allow for more fine-grained data collection and identification of opportunities to assist student learning. Even after instruction, students demonstrated difficulty with the proper placement of numerator and denominator (reciprocal error). It may be that these are difficult concepts for all students, which would explain the persistence of these errors at post-quiz. Alternately, it could mean that ST Math is not presenting content in a way that teaches this idea as well as it does other fraction concepts. Examination of the curriculum and the puzzles that cover material related to these concepts can help to answer this question. Experimental studies altering this material and examining resulting error patterns can inform future iterations of both ST Math and other elementary fraction curricula.

Our work can also be applied to the realm of personalized or individualized instruction (e.g., learner profiles; Pane, Steiner, Baird, & Hamilton, 2015). As typically conceptualized, personalized learning aims to tailor each student’s instructional needs to meet that student “where they are.” This may be optimally accomplished by a computer program, such as ST Math, that can require mastery of a skill before a student moves on to more advanced skills. However, technology cannot work in isolation—knowledgeable teachers are necessary to support student learning (Anderson, Corbett, Koedinger, & Pelletier, 1995; Hew & Brush, 2007). By identifying typical patterns of student errors and grouping students based on the classes that define these patterns, teachers may be able to efficiently offer a type of personalized learning outside of the digital environment. At pre-test, a number of students displayed naïve schemas about fractions as demonstrated by the clustering of their errors around whole number ordering. The needs of these students in the classroom may be different than those who have demonstrate what we view as a more general unfamiliarity with fraction problems as seen in the Distributed Errors Class at pre- and post-quiz.

Conclusion

We set out to better understand third grade students’ errors with fractions in a digital learning environment using multiple choice tests of fraction concepts and algorithms. We identified new types of errors, such as incomplete information and reciprocal errors. We then examined the co-occurrence of these errors using Latent Class Analysis, identifying three patterns of student errors at pre-quiz: one that demonstrated a naïve fraction schema, one that contained high performers without consistent errors, and one that included the majority of the students and likely reflected student unfamiliarity with fraction problems. We found that, although some patterns remained the same, such as distributed or few errors, some patterns changed after instruction. Our work contributes to the growing field of research on fractions, adding to the understanding of the complexity of fraction knowledge and offering insights into the particulars of student struggles, insights that can contribute to the design of assessment and instruction, and ultimately, to improved fraction and mathematics achievement.