This is an open access article distributed under the terms of the Creative Commons
Attribution License (Algebra is a central topic within school mathematics (Hodgen, Küchemann, & Brown, 2014). It facilitates a smooth transition for students from primary school to higher education (Coles & Brown, 2001) because it underlies many higher-level mathematics and science courses (Matthews & Farmer, 2008). Algebra also plays a major role in mathematics proficiency tests in university entrance exams in many countries. Thus, a good understanding of algebra is essential for studying Science, Technology, Engineering, or Mathematics (STEM) subjects (Crisp, Nora, & Taggart, 2009).
There are large differences between individuals in the way they solve and perform on algebra questions (e.g. Kirshner & Awtry, 2004). Given the prominent role that algebra plays in students’ academic achievement, it is important to understand the factors that influence and explain differences in students’ algebra performance. Mathematical equivalence is a foundational concept in mathematics (ACME, 2008), and previous research has identified students’ conceptions of mathematical equivalence to be one of those concepts that relate to their algebra learning (Booth, Barbieri, Eyer, & Paré-Blagoev, 2014; Booth & Davenport, 2013; Byrd, McNeil, Chesney, & Matthews, 2015; Carpenter, Franke, & Levi, 2003; Matthews, Rittle-Johnson, Taylor, & McEldoon, 2010; McNeil & Alibali, 2005).
In most research, students’ conceptions of the equals sign (=) have been investigated in two ways: as an operator indicating the result of an arithmetic operation, and as a relational symbol indicating that both sides of the equals sign are the same and interchangeable (e.g. Knuth, Stephens, McNeil, & Alibali, 2006). Jones et al. (2012) argued that while the sameness component is often investigated, the interchangeableness or substitution conception has largely been neglected. The present study investigated for the first time whether endorsement of substitution is a unique predictor of algebra performance independently of the other two conceptions.
Conceptions of the Equals Sign and Algebra Performance
Many students hold an operational conception of the equals sign, i.e., they view it as a ‘place-indicator’ for the result of an operation (Frieman & Lee, 2004; National Research Council, 2001), and/or as a ‘do something’ signal (Kieran, 1981). Such students typically anticipate that the equals sign is always preceded by an expression and followed by a result. Formal education, however, aims to enable students to consider the equals sign as a relation rather than just an operation. A relational view involves establishing equivalence between the two sides of a given arithmetic equation (Carpenter et al., 2003). Students who hold this view are said to embrace a relational conception of the equals sign (Matthews, Rittle-Johnson, McEldoon, & Taylor, 2012). Such students establish equivalence by either calculating the value on both sides of an equation, or by exploiting arithmetic shortcuts to avoid the need for calculation (Rittle-Johnson, Matthews, Taylor, & McEldoon, 2011). We refer to this as the ‘sameness conception’ of the equals sign.
The general consensus among researchers, who investigate the relation between students’ understanding of the equals sign and their success at further mathematics, is that developing a relational conception of the equals sign, and moving on from the basic operational conception, is important for learning algebra (Booth et al., 2014; Byrd et al., 2015; Knuth et al., 2006; Matthews et al., 2010; McNeil, Rittle-Johnson, Hattikudur, & Petersen, 2010). For instance, Knuth, Alibali, McNeil, Weinberg, and Stephens (2011) reported that students who provided a sameness interpretation of the equals sign were more likely to judge that the two equations 2 × __ + 15 = 31 and 2 × __ + 15 – 9 = 31 – 9 had the same solutions compared to students who offered an operational interpretation. Likewise, Matthews et al. (2012) found that children who could interpret equivalence in terms of sameness were more likely to solve different types of algebra items correctly (e.g., m + m + m = m + 12).
Although these previous studies have defined relational definitions as “equal and interchangeable” the researchers have tended to focus exclusively on the sameness component in their operationalization of the definition, likely because children rarely spontaneously provide definitions that invoke the interchangeability of the two sides of an equation. Thus, previous work has led the field to assume that developing the sameness conception of the equals sign, and moving on from the operational conception, are assumed to be central to algebra performance.
A New Conception for the Equals Sign: Substitution
Jones et al. (2012) emphasised that a relational understanding of the equals sign involves not just sameness but also substitution, which relates to viewing the equals sign as meaning one side can be exchanged for the other.
The substitution conception arises from the formal mathematical definition of equivalence. If a relation is binary, reflective, transitive and symmetrical, it is an equivalence relation. The substitution conception is based on transitivity and symmetry properties of equivalence. Transitivity means that if a = b and b = c then a = c, and symmetry means that a = b is identical to b = a. According to this definition equivalent expressions can be used to replace one another at any time in mathematical equations (e.g., given y = 4x + 1, then 4x + 1 can be used to replace y, and vice versa). This leads to the hypothesis that conceptions of the equals sign that conform to the mathematical definition of equivalence should include the idea of substitution.
The substitution component of equivalence tends not to have been operationalised by education and cognitive researchers (see Carpenter et al., 2003; Matthews et al., 2012). An exception was Jones et al. (2012), who administered an instrument (adapted from Rittle-Johnson & Alibali, 1999) to children in England and China. The instrument comprised operational, sameness, and substitution definitions of the equals sign and asked participants to rate how ‘clever’ they thought each definition was. Jones et al.’s results showed that the three types of conceptions loaded strongly and distinctly on three different factors. These results provided initial evidence for their hypothesis that substitution is theoretically distinct from sameness.
Jones et al. (2012) found that the English students, on average, rated operation more highly and sameness about the same as their Chinese peers. Importantly, they also found that the Chinese students, rated substitution more highly than the English students. Their findings suggested an additional dimension to the discussion as to why Chinese students generally outperformed English students in algebra on international tests. Existing theoretical accounts of how students understand equivalence would lead us to conclude that those students who are weaker in algebra might be resistant to changing their entrenched operational view of the equals sign (e.g. McNeil & Alibali, 2005). However, Jones et al.’s (2012) findings suggested that there may be more to individual differences in algebra performance than the operation-sameness dichotomy because students also varied in their endorsement of substitution. Thus, identifying the role of substitution could further explain individual differences in algebra performance. The main purpose of the research reported here was to investigate this hypothesis.
Substitution in Algebraic Equations
Solving simultaneous equations can necessitate an understanding of the mathematical notion of substitution. Secondary-school students are usually taught to solve pairs of equations involving two unknowns such as Items 8 and 9 depicted in the instrument (see the Supplementary Materials section). A typical method to solve such problems is to express one unknown in terms of another, and then substitute that expression into an equation. However, as evidenced in several studies, students often struggle to substitute one expression for another, even in the later years of secondary schooling (Filloy, Rojano, & Solares, 2003, 2010; Godfrey & Thomas, 2008; Kosyvas, 2016). This difficulty may be in part due to students not appreciating the transitive property of equivalence relations, namely the fact that a = b and b = c implies a = c (Godfrey & Thomas, 2008). On this basis, we hypothesised that understanding the substitution component of equivalence, which implies the transitive property of the equals sign, will play a unique and important role in secondary students’ algebra performance.
The Present Study
Despite the large body of research investigating the role of the sameness and operational conceptions of the equals sign in learning algebra, the role of substitution on students’ algebra performance is yet unclear. To fill this gap in the literature, the present study examined the role of substitution on algebra performance. On the basis of the existing literature, we hypothesised that:
-
On average secondary students would endorse the sameness definition more highly than the substitution definition. This hypothesis was based on previous work using similar instruments (Jones et al., 2012; Jones, Inglis, Gilmore, & Evans, 2013), and we expected to replicate this finding.
-
Endorsement of substitution would be a unique predictor of secondary students’ algebra performance independently of operation and sameness. This hypothesis was based on the finding that Chinese students, who outperform English students on algebra items, rated substitution more highly than their English peers.
Method
Participants
A total of 57 secondary school students (Mage = 15.34 years, age range = 14-16 years, SD = 0.54) from two urban schools in England participated in this study. The students were in Year 10 (N = 27) and Year 11 (N = 30) of secondary school. According to their mathematics teachers, all students had been taught linear and simultaneous equations within the previous eight months.
All students gave written consent to participate in the study. The mathematics teacher of each class was informed about the purpose and procedure of this study. Study procedures were approved by the Loughborough University Ethics Committee.
Materials and Procedure
The students were given an instrument, which included an equals sign test and an algebra test (see the Supplementary Materials section). The instrument was completed by the students within their usual lesson time under test conditions and administered by their regular mathematics teachers. The students worked individually through the instrument and were allowed 20 minutes to complete all 11 items.
The Equals Sign Test
The equals sign test (Items 1 and 2) consisted of a subset of items based on Jones et al.’s (2012) prior work, and assessed which definitions of the equals sign are endorsed by students. The test used by Jones et al. (2012), which was adapted from Rittle-Johnson and Alibali (1999), presented nine definitions (three definitions for each of three conceptions of the equals sign). Participants rated these definitions as not so clever, sort of clever or very clever. For Item 1, we kept one definition from each component (operation, sameness, and substitution) since Jones et al.’s analysis showed the components to be clearly distinct. There was also one control definition that had no meaning in terms of conceptions of the equals sign. Students’ ‘cleverness ratings’ of the operational, sameness, and substitution definitions of the equals sign were scored as 0 for not so clever, 1 for sort of clever and 2 for very clever. Similarly, Item 2 in which students were asked to select the best definition of the equals sign among the presented definitions was used by Rittle-Johnson et al. (2011) previously, inspired by methods used in the psychology literature to assess people’s knowledge of concepts (e.g. Gelman & Meck, 1983).
For each student, substitution, sameness, and operation scores were calculated through adding 1 point to the score for Item 1 if the student chose the related conception as the best definition of the equals sign for Item 2. For instance, if a student rated ‘= means the same as’ as ‘very clever’ (2 points) and chose this definition as the best, their sameness score was 3 (2 + 1) or if a student rated ‘= means the two sides can be swapped’ as ‘sort of clever’ (1 point) and chose another definition as the best, their substitution score was 1 (1 + 0). Each conception score for each student ranged from 0 to 3.
We used only closed items in which students rated definitions of the equals sign since students tend not to offer substitution definitions to free response items asking them to define the symbol ‘=’, although other authors have used free response items (e.g. Knuth et al., 2006; Matthews et al., 2012).
Algebra Test
The algebra test was designed to explore students’ performance on solving algebraic questions. We adapted nine items from existing instruments (Brown, Hart, & Küchemann, 1984; Küchemann, 1978; Rittle-Johnson et al., 2011).
Three items adapted from Rittle-Johnson et al. (2011) were designed to investigate students’ understanding of arithmetic equations. Students were asked to identify whether the given equations are true (Items 3 and 5), and in what circumstances a given equation becomes true (Item 4).
Students were asked to solve linear and simultaneous equations in Items 6 to 9. These were based on typical items from high-stakes national examinations (General Certificate of Secondary Education) sat by most students in England at age 16. Finally, Items 10 and 11 asked students to assess the mathematical equivalence of equations consisting of letters. These last two questions were taken from the Increasing Competence and Confidence in Algebra and Multiplicative Structures (ICCAMS) project survey (see Brown et al., 1984; Küchemann, 1978) and have been used extensively to assess conceptual understanding of algebra (see Hodgen, Küchemann, Brown, & Coe, 2009; Hodgen, Coe, Brown, & Küchemann, 2014).
We checked the performance of the items in the algebra test in terms of the internal consistency. Items 3, 5, and 6 were excluded from the analysis due to ceiling effects (see Table 1). We found an acceptable Cronbach’s alpha of .80 for the remaining algebra items.
Table 1
Descriptive Statistics of the Test Items
| Question | N | Min | Max | M | SD |
|---|---|---|---|---|---|
| 3A | 57 | 0 | 1 | 0.95 | 0.225 |
| 3B | 57 | 0 | 1 | 0.96 | 0.186 |
| 4 | 57 | 0 | 2 | 1.79 | 0.491 |
| 5 | 57 | 0 | 1 | 0.96 | 0.186 |
| 6 | 57 | 0 | 1 | 0.98 | 0.132 |
| 7 | 57 | 0 | 1 | 0.86 | 0.350 |
| 8 | 57 | 0 | 1 | 0.79 | 0.411 |
| 9 | 57 | 0 | 1 | 0.26 | 0.444 |
| 10 | 57 | 0 | 1 | 0.54 | 0.503 |
| 11A | 57 | 0 | 1 | 0.82 | 0.384 |
| 11B | 57 | 0 | 1 | 0.32 | 0.469 |
| 11C | 57 | 0 | 1 | 0.40 | 0.495 |
| 11D | 57 | 0 | 1 | 0.19 | 0.398 |
Note. Min/Max = Theoretical min/max score.
Students’ responses to the algebra test were scored for correctness irrespective of the solution method (i.e., 1 for a correct answer, and 0 for an incorrect answer). For each student, total scores were calculated, which we call here ‘algebra performance’.
Results
Descriptive statistics for all measures are displayed in Table 2. As expected, students rated sameness as a cleverer definition than substitution (Hypothesis 1), and this difference was significant (t(56) = 7.39, p < .001), replicating the finding of Jones et al. (2012).i As shown in Table 3, operation and sameness negatively correlated with each other but, importantly, neither correlated with substitution. This provides further support for the unique contribution of substitution to a relational meaning of the equals sign (Jones et al., 2012).
Table 2
Descriptive Statistics for the Three Conceptions and the Dependent Measure, i.e., Algebra Performance
| Variable | M | SD | Min | Max |
|---|---|---|---|---|
| Algebra performance | 5.98 | 2.45 | 0 | 10 |
| Substitution | 0.95 | 0.86 | 0 | 3 |
| Sameness | 2.18 | 0.98 | 0 | 3 |
| Operation | 1.77 | 0.85 | 0 | 3 |
Note. N = 57, Min/Max = Theoretical min/max score.
Table 3
Correlations Between Algebra Performance, the Different Conceptions of the Equals Sign, and School Year
| Variable | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| 1. Algebra performance | – | |||
| 2. School year | .76** | – | ||
| 3. Substitution | .46** | .35** | – | |
| 4. Operation | -.34** | -.55** | -.07 | – |
| 5. Sameness | .38** | .35** | .09 | -.55** |
Note. N = 57.
**p ˂ .01.
Since school year, operation, sameness, and substitution were significantly correlated with the dependent measure, all of these variables were used as predictors for algebra performance in a simultaneous multiple regression analysis. In the multiple regression model, we entered all variables in the same step to test our hypothesis.
Regression results showed that the model was significant, F(4, 52) = 24.98, p ˂ .001, with an Adjusted R2 = .63, which meant that our model explained 63% of the variance in algebra performance. Table 4 depicts the regression coefficients. For the dependent measure, i.e., algebra performance, three variables emerged as significant predictors: school year, sameness and substitution. Operation did not explain unique variance in algebra performance. Most importantly, as expected, substitution explained unique variance in algebra performance above and beyond school year and sameness.
Table 4
Predictors of Algebra Performance
| Predictor | Β | η2p | B (SE) | 95% CI |
|---|---|---|---|---|
| Model for All Items (Total R2 = .66) | ||||
| Constant | 1.48 (1.03) | [-0.59, 3.56] | ||
| School year | .72** | .48 | 3.50 (0.51) | [2.48, 4.53] |
| Operation | .19 | .05 | 0.55 (0.32) | [-0.10, 1.19] |
| Sameness | .21* | .08 | 0.53 (0.24) | [0.04, 1.02] |
| Substitution | .20* | .09 | 0.56 (0.27) | [0.07, 1.05] |
| Model for Typical Items (Total R2 = .47) | ||||
| Constant | 0.14 (0.45) | [-0.77, 1.06] | ||
| School year | .76** | .41 | 1.35 (0.22) | [0.90, 1.80] |
| Operation | .32* | .10 | 0.37 (0.14) | [0.05, 0.62] |
| Sameness | .26* | .08 | 0.23 (0.11) | [0.02, 0.44] |
| Substitution | .05 | .00 | 0.05 (0.11) | [-0.26, 0.17] |
| Model for Conceptual Items (Total R2 = .49) | ||||
| Constant | 0.24 (0.81) | [-1.40, 1.87] | ||
| School year | .60** | .31 | 1.96 (0.40) | [1.15, 2.76] |
| Operation | .07 | .00 | 0.13 (0.25) | [-0.38, 0.63] |
| Sameness | .10 | .02 | 0.17 (0.19) | [-0.21, 0.56] |
| Substitution | .23* | .09 | 0.43 (0.19) | [0.05, 0.82] |
Note. N = 57. CI = confidence interval.
*p ˂ .05. **p ˂ .01.
The algebra test contained items that are typical of those used in national examinations (Items 7 to 9) and items sourced from a conceptual test (Brown et al., 1984) designed for research purposes (Items 10 and 11). As can be seen from the instrument, these two types of items are qualitatively different and the participants could be expected to be familiar with the examination style items but not the conceptual style items. We were interested in whether our result for all algebra items holds across the two distinct types of item. To this end we calculated separate participant scores for Items 7 to 9 (‘typical items’) and Items 10 and 11 (‘conceptual items’) and then repeated the regression analysis for each set of scores.
For the typical items the regression model was significant, F(4, 52) = 13.29, p ˂ .001, with an Adjusted R2 = .47, explaining 47% of the variance in scores. The regression coefficients are shown in Table 4. Operation and sameness explained unique variance in typical item scores, but substitution did not. For the conceptual items the model was again significant, F(4, 52) = 14.48, p ˂ .001, Adjusted R2 = .49, explaining 49% of the variance in scores. As shown in Table 4, substitution explained unique variance in conceptual item scores, but operation and sameness did not.
In summary, operation and sameness were significant predictors of performance on those algebraic items that are typical of national examinations, whereas only substitution was a significant predictor of performance on items designed to test conceptual knowledge of algebra. We consider the implications of this in the discussion section.
Developmental Analyses
Jones et al. (2012) reported a post-hoc analysis to investigate whether there are individual differences in the order in which the sameness and substitution conceptions develop. We replicated their post-hoc analysis for the data reported here. We coded students as ‘rejecting’ a conception (having a score of 0 or 1 on the equals sign test) or ‘accepting’ a conception (having a score of 2 or 3). For the students who accepted sameness, 56.1% (N = 32) rejected substitution, whereas for the students who accepted substitution, only 3.5% (N = 2) rejected sameness. Therefore, the present data suggest that substitution may emerge after students have developed a conception of sameness, contrary to Jones et al. who suggested they emerge concurrently. We return to this discrepancy in the discussion.
To explore developmental issues, albeit tentatively, we considered the extent to which patterns of ‘accepting’ or ‘rejecting’ each definition relates to algebra performance in our data.ii The full results are shown in Table 5. Those students who accepted operation (N = 31, M = 5.29) scored lower on the algebra test than those who rejected operation (N = 26, M = 6.81), t(54.57) = 2.45, p = .017, as would be expected. Those who accepted sameness (N = 42, M = 6.50) scored higher than those who did not (N = 15, M = 4.53), t(32.61) = -3.24, p = .003, again as expected. Interestingly, those who accepted substitution (N = 12, M = 7.92) also scored higher than those who did not (N = 45, M = 5.47), t(31.19) = -4.56, p < .001. This post hoc analysis provides further support that substitution is associated with the development of algebra performance. Moreover, it is interesting to note that accepting substitution is associated with a higher score than accepting sameness. A comparison of students who only accepted sameness (N = 32, M = 6.06) with those who also accepted substitution (N = 10, M = 7.90), t(25.62) = -2.80, p = .010, supports the importance of substitution over and above sameness for algebraic development.
Table 5
Scores by ‘Acceptance’ or ‘Rejection’ of Equivalence Conceptions
| Operation | Sameness | Substitution | N | M |
|---|---|---|---|---|
| R | R | R | 1 | 6.00 |
| R | A | R | 19 | 6.32 |
| R | A | A | 6 | 8.50 |
| A | R | R | 12 | 3.83 |
| A | R | A | 2 | 8.00 |
| A | A | R | 13 | 5.69 |
| A | A | A | 4 | 7.00 |
Note. A = accept; R = reject.
Discussion
The findings of the present study shed light on the role that substitution plays in students’ algebra performance. Multiple regression analysis demonstrated that endorsement of substitution was a unique predictor of algebra performance independent of operation, sameness and school year. This novel finding has important implications for educational assessment and practice; it indicates that enhancement of substitution could potentially influence students’ success in algebra. Future research should further examine whether, through training, developing a substitution conception of the equals sign can improve students’ algebraic learning and problem solving.
We found a positive correlation between algebra performance and sameness, and a negative correlation between algebra performance and operation. These results were expected because it has been evidenced that students who mainly endorse a sameness conception perform better in algebra than those who view the equals sign operationally (Byrd et al., 2015; Knuth et al., 2006; Matthews et al., 2012; McNeil et al., 2010). Importantly, substitution positively correlated with algebra performance. Multiple regression analysis indicated that endorsement of substitution uniquely predicted secondary school students’ algebra performance independently of school year and the conceptions of operation and sameness. This suggests that the more students endorsed the concept of substitution the better they performed in algebra. Surprisingly, operation did not contribute significantly to the regression model unlike sameness. This finding suggests that sameness and substitution appear to sufficiently explain variance in algebra performance on their own.
However, for the three typical Items (7 to 9), which were based on national examination (GCSE) items, only operation and sameness were significant predictors, whereas for the three conceptual items (10 and 11), only substitution was a significant predictor. We are unsure as to the explanation for this but a possibility is that the typical items lend themselves to being solved using arithmetic methods. For example, Item 7 can be completed by guessing values for y and calculating the result for comparison. Evidence based on the scrutiny of GCSE scripts suggests that it is common for candidates to obtain high scores for algebra items using such arithmetic methods (Noyes, Wake, Drake, & Murphy, 2011). Conversely, Items 10 and 11 cannot be solved using such guess-and-calculate strategies. It is unclear why sameness was not also a significant predictor of conceptual item scores, but it may be that endorsement of the substitution definition of equivalence may be associated with a more conceptual understanding of algebra. Our exploratory post-hoc analysis, summarised in Table 5, supports this: endorsing substitution is associated with the development of algebra performance, and with a higher score on the algebra items.
Our finding that endorsement of the concept of substitution uniquely predicted algebra performance overall and on the conceptual items further highlights that substitution is an important conception of the equals sign which relates to students’ algebra learning. Given that Rittle-Johnson et al. (2011) proposed a continuum of equivalence knowledge from operation to sameness, this finding provides evidence for the existence and importance of the conception of substitution, which should also be considered in students’ development of the equals sign knowledge alongside operation and sameness.
We also replicated previous findings showing that English students rated the sameness definition of the equals sign more highly than the substitution definition (Jones et al., 2012, 2013). Our results further demonstrate that even older secondary school students’ views of the equals sign are dominated by operation and sameness. However, some of our analyses contradicted previous findings. We found that operation and sameness did not correlate with substitution, whereas in Jones et al.’s (2012) study they both correlated significantly with substitution. In addition, we found a negative correlation between operation and sameness whereas Jones et al. (2012) found that they did not correlate. Jones et al. proposed two possibilities regarding the development of sameness and substitution conceptions: 1) that substitution and sameness develop somewhat concurrently, or 2) that substitution emerges only after students have developed a sameness conception. A post-hoc analysis reported by Jones et al. suggested that the conceptions may develop concurrently. We replicated their post-hoc analyses using our data, and this suggested that substitution may emerge after students have developed a conception of sameness, contrary to Jones et al. (2012). The discrepancy across the two studies may be as a result of the different instruments used. Jones et al. (2012) administered a nine-item instrument to assess the understanding of the equals sign and conducted a confirmatory factor analysis. Here we used a two-item instrument based on the outcome of Jones et al.’s (2012) analysis. The discrepancy may also be as a result of the different characteristics of the participants across the two studies. First, Jones et al. sampled younger secondary school students (ages 11 and 12), whereas here the sample came from older secondary students (ages 14 to 16). Perhaps the mathematics that the students experienced in their later years of schooling explains why substitution lost its correlation with operation and sameness. Second, Jones et al.’s (2012) sample comprised students from schools in England and China. The students in China endorsed substitution more highly, and operation less highly, than the students in England. Therefore, cultural differences may also be a reason why the observed associations between the equals sign conceptions were different across the studies. Longitudinal and cross-cultural studies are required to address the issue of the developmental trajectories of the different types of conceptions.
In conclusion, the present study contributes new insights into how students’ algebra learning relates to their conceptions of the equals sign. Previous research operationalised a sophisticated understanding of the equals sign in terms of sameness and tended to neglect substitution. Our findings show that the operation, sameness, and substitution conceptions of the equals sign are different constructs and substitution is a unique predictor of algebra performance independent of operation and sameness. Designers of both classroom instruction and of instruments designed to investigate students’ conceptual development should consider incorporating the equals sign in terms of substitution.