The current study explores the effects of physical spacing within mathematical expressions on student performance. A total of 2,152 students in 5th-12th grade were randomly assigned to one of four conditions within an online problem set, with terms in algebraic expressions spaced 1) neutrally, with no spaces in the expression, 2) congruent with the order of precedence through grouping terms, 3) incongruent with the order of precedence, or 4) mixed, a combination of the previous conditions. Results show that students who viewed incongruent problems made more errors and had to solve more problems to complete the assignment than those who viewed congruent or neutrally spaced problems. Additionally, students who viewed problems with mixed spacing had to solve more problems to complete the assignment than students who viewed congruent problems. These findings suggest that viewing expressions with spacing that is incongruent with the order of precedence presents challenges for students. Overall, these results replicate prior research in perceptual learning in a natural homework environment and support the claim that physical spacing between terms does influence student performance on order of precedence problems.
Formal mathematics is a commonly used example of how humans make sense of abstract symbolic reasoning (
Beyond being abstract and requiring conceptual knowledge, reasoning about mathematics is also inherently perceptual (
Although subtle visual manipulations are irrelevant to the mathematical meaning of notation, visual manipulations of notation can lead to attentional biases and create perceptual groupings among terms and operands. For instance, terms and operands spaced in close proximity within a mathematical notation tend to be seen as a group, such as viewing “4 x 6+3” and wrongly grouping “6+3” together based on the spatial proximity of those terms (
A large body of research has demonstrated that the physical spacing between terms and operands within equations and expressions contributes to students’ perceptions of
Similarly,
Overall, this body of literature shows that the perceptual grouping of mathematical terms in an expression or equation influences both novices and experts during problem solving. Specifically, when terms are spatially organized in groups that mirror the order of precedence, students are more likely to have higher performance (
The present study asked 5-12th grade students to simplify order of precedence expressions in ASSISTments, an online tutoring system (
This study extends prior research on perceptual learning in four key ways. First, the majority of studies examining the effects of physical spacing between mathematical terms have been conducted with undergraduate students in controlled laboratory settings rather than with school-aged children in authentic classroom and learning contexts. Second, the study is conducted through an online homework assignment assigned to students by their teachers using the ASSISTments platform (
To extend prior research on perceptual learning as it pertains to mathematics performance, we present a randomized controlled trial with upper elementary, middle, and high school students in ASSISTments, an online tutoring system. This study is designed to explore the impact of physical spacing between terms on students’ mastery speeds when solving a series of order of operations problems. Specifically, we investigate our hypotheses by posing the following questions:
If there are differences in assignment mastery speed based on condition,
If there are differences in assignment mastery speed based on condition,
Data for this study was collected from 2015-2019 in ASSISTments, an online tutoring system that features free content for K-12 students with a primary focus on mathematics (
This randomized controlled trial was created by the authors and deployed as an available Skill Builder problem set covering order of operations content (targeting 7th grade) within ASSISTments. “Skill Builders” are optional problem sets that teachers can assign to provide students with fluency practice on topics commonly featured on standardized mathematics tests. Skill Builders map onto content areas from the Common Core State Standards and present problems from a given content area in a randomized order. These problem sets are designed to challenge a student in a mathematics topic until that student achieves content mastery.
Under default settings, students must consecutively answer three problems in a row correctly to achieve mastery status for the Skill Builder assignment. If a student answers a problem incorrectly, the problem count restarts and they continue to receive problems until they correctly answer three problems in a row. Therefore, in this context, a slower mastery speed (solving more problems in order to get three problems correct in a row) is an indicator of higher error and lower mathematics performance on a Skill Builder assignment. Mastery speed has been used as an outcome measure of student performance in previous ASSISTments studies (e.g.,
The final sample included in the analyses were 2,152 students (48.0% male, 35.2% female, 16.9% unknown) who completed more than three problems in the Skill Builder problem set and completed the assignment by achieving mastery. Participants were 5th-12th grade students assigned to complete the given problem set by their classroom teacher. The 2,152 students included in the final sample from this study came from 199 classes taught by 115 teachers from 83 schools in 64 districts from 16 states. The students were distributed across several grade levels, with a majority of students in middle school classrooms (0.6% fifth, 11.7% sixth, 30.4% seventh, 9.8% eighth, 18.5% ninth, 1.0% tenth, 0.1% eleventh, and 1.1% of reported cases in twelfth grade; with the remaining 26.9% of cases missing grade level information).
Many more students initially opened the problem set but were dropped from this study for the following reasons. A total of 6,238 students opened the problem set, however, 4,053 students were excluded due to assignment completion within three problems or stopping the assignment within the first three problems, thus never seeing an experimental condition. Additionally, a small subset of participants was also excluded due to having an unknown mastery status for the problem set (
When students opened the problem set, they were first exposed to three neutrally spaced expressions to solve (
Problem with neutral spacing as shown in first three assignment problems.
Example assignment screen for a participant in the incongruent spacing condition.
Tutorial: All Participants | ||||
---|---|---|---|---|
1. | 6*3+4*4 | |||
2. | 14-5*2 | |||
3. | 3*3+3+3*3 | |||
Neutral | Congruent | Incongruent | Mixed | |
4. | 5+2*4 | 5 + 2*4 | 5+2 * 4 | 5 + 2*4 |
5. | 7*2+8*5 | 7*2 + 8*5 | 7 * 2+8 * 5 | 7 * 2+8 * 5 |
6. | 4*3+2 | 4*3 + 2 | 4 * 3+2 | 4*3+2 |
7. | 4*(2+5)+12-2*3 | 4*(2 + 5) + 12 - 2*3 | 4 * (2+5)+12-2 * 3 | 4*(2 + 5) + 12 - 2*3 |
8. | 5+3*2 | 5 + 3*2 | 5+3 * 2 | 5+3 * 2 |
Most students continued to solve problems until they achieved mastery (answering three consecutive problems correctly on the first try). However, if a student answered a problem incorrectly, they could not move on to the next problem until typing in the correct answer. To support students as they moved through the assignment, one hint restating the order of operations was available to click on at the beginning of each problem (
Hint available to participants on each problem in the assignment.
The study remained open as an active Skill Builder for the order of operations standard without exponents (Common Core Standard 7.NS.A.3. EX) that teachers could easily assign to their students at any time for three years. At the end of the three years, the data from the study was aggregated using the ASSISTments Assessment of Learning Infrastructure (ALI) report that was automatically generated by the ASSISTments team for external researchers and provides aggregated data files at various levels of granularity such as student-level and problem-level (
Prior to data analysis, the following measures for analyses were defined and extracted from the ASSISTments report as necessary for analysis.
As an estimated measure of prior mathematics performance, ASSISTments calculates a prior proportion correct value (from 0-1). This value represents the proportion of all previous ASSISTments problems completed from other assignments that each student answered correctly prior to the current experiment. However, the type of content may have varied and some participants may have had extensive experience with ASSISTments over years whereas others might have been first- or second-time users. Although participants varied in previous exposure and practice with ASSISTments, this value serves as a proxy for prior mathematics performance and has been used in studies that were deployed using the ASSISTments platform (e.g.,
The ASSISTments ALI report also provided an ordinal value representing the reported grade level of each participant by the classroom teacher. With values ranging from 5-12, grade level was treated as a continuous variable in all analyses.
After preprocessing the data, descriptive statistics were calculated in SPSS to determine means and variability for each variable and relations between each construct. Next, we conducted a one-way analysis of variance (ANOVA) with condition (neutral, congruent, incongruent, and mixed) as the independent variable and mastery speed as the dependent measure. We also conducted post hoc tests with Bonferroni correction to examine where there were significant differences in average mastery speed between conditions.
In addition to the ANOVA, we examined the impact of condition, above and beyond prior performance and grade level. To determine whether or not multilevel analysis would be appropriate, we calculated the intraclass correlation coefficient (ICC) from an unconditional 2-level hierarchical linear model (HLM; Model 1). An unconditional HLM model predicting mastery speed suggested that approximately 10% of the variance in mastery speed was attributable to differences at the class level. As this value exceeds the 7% variance threshold to suggest that using HLM would be appropriate (
Next, four two-level HLMs were conducted to explore our research questions. Model 2 estimates how the covariates, grade level and prior performance, impact participants’ assignment mastery speed while accounting for any nested effects between the student and class levels. Model 3 includes the three condition variables (neutral, congruent, and mixed, with incongruent as the reference group) and estimates how the physical spacing between terms impacts participants’ assignment mastery speed (compared to the incongruent condition) while accounting for any nested effects between the student and class levels.
Model 3 in HLM has the following form:
where
Interaction terms were created and added to the hierarchical linear model to examine interactions between
Overall, all students completed the assignment by eventually achieving mastery status (answering three problems correctly in a row) at some point in the assignment (
Population | Average Prior Performance ( |
Average Mastery Speed ( |
|
---|---|---|---|
Overall | 2,152 | .70 (.14) | 6.38 (3.24) |
5th Grade | 13 | .86 (.15) | 7.23 (5.00) |
6th Grade | 251 | .71 (.13) | 6.34 (2.96) |
7th Grade | 654 | .67 (.14) | 6.39 (3.13) |
8th Grade | 210 | .76 (.15) | 6.25 (3.72) |
9th Grade | 399 | .71 (.13) | 5.90 (2.44) |
10th Grade | 21 | .77 (.11) | 6.57 (3.16) |
11th Grade | 2 | .72 (.09) | 5.50 (0.71) |
12th Grade | 24 | .73 (.08) | 6.79 (3.90) |
Next, we conducted a preliminary one-way ANOVA to examine differences in average mastery speeds by condition. Results indicate that there were statistically significant overall differences between groups in mastery speed,
Parameter | Model 1 |
Model 2 |
Model 3 |
Model 4 |
Model 5 |
|||||
---|---|---|---|---|---|---|---|---|---|---|
β | ||||||||||
6.42** | 0.11 | 6.23** | 0.11 | 6.79** | 0.22 | 7.67** | 1.04 | 8.04** | 1.09 | |
Grade Level | -0.01 | 0.09 | -0.02 | 0.09 | -0.02 | 0.09 | -0.22 | 0.20 | ||
Prior Performance | -2.31** | 0.69 | -2.32** | 0.70 | -3.70* | 1.61 | -2.33** | 0.70 | ||
Neutral | -0.78** | 0.24 | -2.21 | 1.77 | -3.62* | 1.72 | ||||
Congruent | -0.92** | 0.26 | -2.68* | 1.35 | -2.43 | 2.15 | ||||
Mixed | -0.46† | 0.24 | -0.82 | 1.70 | -2.33 | 1.54 | ||||
Neutral x Prior Performance | 2.02 | 2.37 | ||||||||
Congruent x Prior Performance | 2.51 | 1.77 | ||||||||
Mixed x Prior Performance | 0.52 | 2.27 | ||||||||
Neutral x Grade | 0.37† | 0.21 | ||||||||
Congruent x Grade | 0.20 | 0.25 | ||||||||
Mixed x Grade | 0.25 | 0.18 | ||||||||
Variance Components | ||||||||||
Student Level | 9.52 | 8.90 | 8.80 | 8.79 | 8.78 | |||||
Teacher Level | 1.02 | 0.43 | 0.42 | 0.43 | 0.43 | |||||
Total Variance | 10.54 | |||||||||
Level 1 | 0.90 | |||||||||
ICC Level 2 | 0.10 | |||||||||
% Explained at Student Level | 0.07 | 0.01 | 0.00 | 0.00 | ||||||
% Explained at Classroom Level | 0.58 | 0.02 | -0.02 | 0.00 |
†
Model 2 shows the influence of students’ grade level and prior performance in ASSISTments on assignment mastery speed. Prior performance was a significant predictor of mastery speed, where students with higher prior performance on ASSISTments problem sets had lower mastery speeds (β = -2.31,
A 2-level HLM model (Model 3) was conducted to examine the impact of condition on mastery speed, controlling for grade and prior performance. The incongruent spacing condition was treated as the reference group for the hierarchical linear models since the ANOVA indicated that there were significant differences between the incongruent spacing condition and two other groups. Results were consistent with the ANOVA; there was a significant effect of two conditions on assignment mastery speeds. The analysis revealed that students in the congruent condition (β = -0.92,
Next, we tested whether there was an interaction effect between students’ prior performance and condition to understand if the effect of spacing condition was moderated by prior performance. Model 4 presents all Level-1 variables in addition to the prior performance interaction terms within Level-1. The interactions between prior performance and each spacing condition (neutral, congruent, mixed) were not significant predictors of assignment mastery speed (β = 2.02,
Lastly, we tested whether there were interaction effects of grade level × condition on assignment mastery speed. Model 5 presents all Level-1 variables in addition to the grade level interaction terms within Level-1. The interaction between grade level and the neutral spacing condition was not significant but was trending towards significance (β = 0.37,
The goal of this study was to explore whether manipulating the physical spacing between mathematical symbols would impact students’ assignment mastery speed on order of operations problems. In addition to replicating the difficulty that algebra learners experience with incongruent spacing in order of operations problems, we were particularly interested in examining whether spacing effects exist in both younger and older students in authentic homework environments such as an online tutoring system. Two main findings emerged from this study: 1) students in the incongruent condition had slower mastery speeds (solving more problems to achieve mastery) than students in the congruent or neutral conditions, and 2) there were no significant interactions between grade level and condition, or prior performance and condition, on mastery speeds. Together, these results suggest that viewing incongruent spacing within mathematical expressions led to more errors and lower performance for most students, regardless of age or prior performance, compared to those who viewed problems with congruent or neutral spacing between terms.
We predicted that viewing congruent or neutral spacing within problems would lead to faster mastery speeds compared to viewing problems with incongruent or mixed spacing. The results mostly supported this hypothesis; students who viewed problems with congruent or neutral spacing tended to master the assignment in significantly fewer problems than students who viewed problems with spacing that was incongruent with the order of precedence. However, there were no significant differences in mastery speed between the neutral and mixed condition.
One explanation for why congruent spacing may lead to greater performance over incongruent spacing is that visually modifying the physical spacing of terms may bias people to naturally group proximal terms into grouped objects (
The finding that viewing congruent and neutral spacing led to higher performance than incongruent spacing is consistent with prior studies (e.g.,
Other work has suggested that spacing is used as an action-guiding cue; incongruent spacing elicits errors while congruent and neutral spacing in mathematical notation helps facilitate improved performance. Consequently, viewing congruent spacing in expressions may not be significantly more helpful than viewing neutral spacing because the perceptual cues from physical spacing would be redundant to cues from operands. This notion is supported by findings that individuals attend to multiplication operands quicker than addition operands and treat narrow spacing between terms similarly to multiplication operands (
There is a common view that students’ computational errors are an indication of their conceptual misunderstandings about mathematics. Consistent with this idea, students’ prior performance in ASSISTments significantly predicted their mastery speed, suggesting that, on average, students with higher prior performance made fewer errors when solving order of operations problems. However, even when controlling for prior performance, incongruent spacing still affected student performance on the problem set. Additionally, while previous findings have largely focused on college-aged participants in laboratory settings (e.g.,
Taken together, these findings reinforce previous evidence that subtle changes in physical spacing can impact students’ performance on computing order of operations problems regardless of the student’s age or knowledge level. It seems that perceptual grouping, through spacing, may be acting as an irrelevant but substantial lure that is hard for students to ignore. More broadly, the differences in performance across conditions supports the notion that people use space as a perceptual cue when interpreting and acting on mathematical symbols. As such, these results provide further evidence that visual and perceptual processes can drive reasoning about mathematics computations (
The current study suggests that minor and relatively meaningless changes to the visual presentation of mathematical notation have implications for how students interpret and use symbolic notation to perform computations. Although perceptual cues influence mathematical reasoning, few instructional approaches or interventions make use of the power of perception. Future learning interventions for algebra learners could include purposeful manipulations to the presentation of mathematical expressions and equations which could affect students’ abilities to learn and apply arbitrary mathematical rules such as the order of precedence. More broadly, the prevalence of a spacing effect on mathematics performance across upper elementary through high school age groups poses interesting questions and may have theoretical implications about when perceptual cues begin to drive cognitive processes in learning and development. Future work could investigate how early spacing effects emerge in young learners and how spatial manipulations may drive students’ cognitive processes and actions at younger ages.
There are several limitations to the dataset as well as the methods used in these analyses. For instance, the Skill Builder structure (where students must correctly answer three problems in a row) is not a common approach used in classroom instruction and does not easily lend itself to a pretest/posttest design. The Skill Builder structure also allows students to stop working after they achieve mastery status in an assignment by correctly answering three problems in a row. As a result, participants only answered an average of six problems in the assignment. Additionally, since the final dataset excluded students who answered the first three problems correctly, this sample does not take into account the behavior of the highest-performing mathematics students on this particular problem set.
Another limitation of the study is that there was limited demographic data available on the students. Since ASSISTments problem sets were assigned by teachers around the country who use the platform for homework, we are unable to collect specific data about children, teachers, or the classroom context. While this is certainly a limitation, the fact that ASSISTments Skill Builders are used and assigned widely by teachers allowed for more ecological validity and a much larger sample size than would have been collected if this study was conducted by researchers with recruited teachers in local classrooms. Additionally, effects of spacing were found even while controlling for prior performance and grade level.
While the current project focused solely on assignment mastery, subtle spacing manipulations have been found to influence student problem-solving behavior at the action level (
One difference in our study from prior work is that we included a mixed spacing condition. While findings suggest that the mixed spacing condition was more difficult for students compared to students in the congruent spacing condition, interestingly, there were no differences in mastery speed between the mixed condition and the neutral condition, or the mixed and incongruent spacing conditions. Given these mixed findings, future experimental work should further examine the effects of mixed spacing and test plausible mechanisms for the impact of perceptual spacing.
It is also important to note that, across conditions, students were provided with an optional hint on each problem to remind them of the rules of precedence. Roughly a third of participants viewed the hint at least once while working on the assignment. That said, little to no work has studied how conceptual knowledge reminders about the order of precedence may mitigate the effect of physical spacing on mathematics performance. To develop a richer understanding of how perceptual grouping may affect student behavior, our team is currently exploring patterns of student behavior within problems, such as response times, help-seeking behavior, and error types, to see how effects of spacing manipulations extend to a broader population of students. More thorough error analyses at the action level, within problems, could provide insight about whether students who are presented with incongruent spacing tend to make predictable errors based on how the symbols were visually grouped. We also plan to explore students’ actions after viewing the available hint to examine whether order of precedence errors continue to occur after a visual reminder of the order of precedence.
This work demonstrates that irrelevant, but salient visual information in notation, such as spacing, can influence students’ reasoning of mathematics. Specifically, perceptual cues, even those that are mathematically misleading such as incongruent spacing, are difficult to ignore. This study extends prior work in the following ways. First, to our knowledge, this is one of the first randomized controlled trials conducted on physical spacing in the context of an online learning platform with school aged children, showing that perceptual grouping occurs in authentic learning environments in addition to laboratory settings. Second, these results reveal that incongruent spacing between terms does impact 5th to 12th grade students’ performance on mathematics assignments, resulting in slower mastery speeds, even when a reminder about the order of precedence is available as a hint on each problem. Conversely, terms in mathematical expressions that are neutrally spaced or grouped together to be congruent with the order of precedence increase assignment mastery speeds. These findings further support the notion that subtle perceptual cues, such as physical spacing, do not bear any practical implications in mathematics yet have effects on mathematical cognition and performance for students in upper elementary through high school. Learners and experts alike utilize perceptual strategies when reasoning about mathematics.
For this study, a dataset is freely available (
The de-identified data used in this analysis is available on the Open Science Framework. Additionally, the original data report from ASSISTments is available for further reference (for access, see
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. 1645629. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Additionally, we are grateful for the support of the Graduate Assistance in Areas of National Need fellowships #P200A150306 and #P200A180088.
We have no competing interests to disclose.
We thank David Landy for his consultation on this project, as well as Janice Kooken and Anthony Botelho for their contributions to data analysis. We also thank the ASSISTments team for access to their research testbed.