Empirical Research

Same-Different Judgments With Alphabetic Characters: The Case of Literal Symbol Processing

Courtney Pollack*a

Journal of Numerical Cognition, 2019, Vol. 5(2), 241–259, https://doi.org/10.5964/jnc.v5i2.163

Received: 2018-01-11. Accepted: 2018-08-24. Published (VoR): 2019-08-22.

*Corresponding author at: McGovern Institute for Brain Research, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, Massachusetts, 02139, United States. E-mail: courtney.pollack@post.harvard.edu

This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Learning mathematics requires fluency with symbols that convey numerical magnitude. Algebra and higher-level mathematics involve literal symbols, such as "x", that often represent numerical magnitude. Compared to other symbols, such as Arabic numerals, literal symbols may require more complex processing because they have strong pre-existing associations in literacy. The present study tested this notion using same-different tasks that produce less efficient judgments for different magnitudes that are closer together compared to farther apart (i.e., same-different distance effects). Twenty-four adolescents completed three same-different tasks using Arabic numerals, literal symbols, and artificial symbols. All three symbolic formats produced same-different distance effects, showing literal and artificial symbol processing of numerical magnitude. Importantly, judgments took longer for literal symbols than artificial symbols on average, suggesting a cost specific to literal symbol processing. Taken together, results suggest that literal symbol processing differs from processing of other symbols that represent numerical magnitude.

Keywords: literal symbols, artificial symbols, number processing, same-different, distance effect

Success in mathematics requires fluency with numerical symbols and their relation to one another. Indeed, research shows a link between efficient comparison of Arabic numerals and mathematics achievement in children and adults (Castronovo & Göbel, 2012; De Smedt, Noël, Gilmore, & Ansari, 2013; Holloway & Ansari, 2009; Schneider et al., 2017). However, success in higher-level mathematics such as algebra requires fluency with other symbols that represent numerical magnitude, namely literal symbols (e.g., x) (Rosnick, 1980, 1982; Schoenfeld & Arcavi, 1999). The present study concerns processing literal symbols compared to other symbols that represent numerical magnitude.

Students experience substantial and persistent difficulty with literal symbols from first exposure through college algebra (Akgün & Özdemir, 2006; Booth, 1999; Christou & Vosniadou, 2005; MacGregor & Stacey, 1997; McNeil et al., 2010; Philipp, 1992; Rosnick, 1982; Trigueros & Ursini, 2003; Ursini & Trigueros, 2004). Student difficulty results from many factors, including mathematical syntax (Schoenfeld & Arcavi, 1999), novel algebraic notational conventions (Kieran, 2007), and insufficient explanations in mathematics curricula (Rosnick, 1982). Importantly, student difficulty also stems from literal symbol processing – connecting the literal symbol to its referent (MacGregor & Stacey, 1997; Philipp, 1992; Rosnick, 1982, 1999; Stacey & MacGregor, 1999). Even the basic act of connecting literal symbols to their referents requires more complex processing compared to Arabic numerals because of properties inherent to literal symbols and how they function in mathematics.

Arabic numerals and literal symbols co-occur in algebra problems (e.g., 3x = 12), but the two symbolic representations qualitatively differ in three main ways. First, literal symbols do not have consistent numerical magnitudes across mathematical contexts like Arabic numerals (e.g., x can be 3, -4, or ½). Second, literal symbols do not have a single magnitude, like Arabic numerals (e.g., ‘3’ never stands for both 3 and 4 objects; x can stand for two numbers or all real numbers [Usiskin, 1999]). Third, literal symbols are not specific to numeracy, like Arabic numerals. Children initially develop fluency with literal symbols in the context of literacy. When children see literal symbols in a mathematics context, they bring strong prior associations for those symbols related to reading and writing. For instance, young students may confuse a literal symbol’s numerical magnitude with its numerical position in the alphabet, or confuse the linear ordering of numbers with the linear ordering of the alphabet (MacGregor & Stacey, 1997; Wagner, 1983). Students may also incorrectly associate literal symbols with objects (e.g., a = apples) rather than a related numerical magnitude (e.g., a = the number of apples), or confuse the two associations (McNeil et al., 2010; Rosnick, 1982). Associating literal symbols with objects can interfere with the notion that literal symbols can have a numerical referent (Booth, 1999) and can lead to difficulties with word problems that persist into college (McNeil et al., 2010; Philipp, 1992; Rosnick, 1982). Nascent research into the cognitive mechanisms of literal symbol processing suggests that even for their most restricted use, when they represent one magnitude like Arabic numerals, literal symbols are processed differently (Pollack, Leon Guerrero, & Star, 2016).

In sum, literal symbols signify a substantial departure from the symbol-magnitude mapping that students learn with Arabic numerals or number words. Students’ familiarity with literal symbols in the context of literacy and their inherent properties in mathematics show that literal symbols require more complex cognitive processing compared to Arabic numerals. However, an understanding of this cognitive processing is lacking. A useful starting point, as undertaken in the present study, is to target the core, lower-level cognitive processes that support literal symbol use in numerical contexts. Such work can inform future research related to higher-level mathematical contexts that involve literal symbols.

Measuring Literal Symbol Processing

One way to investigate literal symbol processing is to adapt tasks, such as number comparison, which are commonly used to measure numerical magnitude processing (Dehaene, Dehaene-Lambertz, & Cohen, 1998; Schneider et al., 2017). In these tasks, participants may view two arrays of dots or two Arabic numerals and judge which one is larger in number. Error rates and response times decrease as the distance between the two numbers increase, a behavioral signature known as the Comparison Distance Effect (CDE) (Moyer & Landauer, 1967). The CDE may arise from the engagement of a mental number line, which indicates numerical magnitude processing (Restle, 1970). Numbers that are closer together are harder to compare since their representations overlap more on the mental number line than numbers that are farther apart. However, many argue that the CDE may result from alternative cognitive processes, such as decision-making associated with comparison itself (Cohen Kadosh, Brodsky, Levin, & Henik, 2008; Krajcsi & Kojouharova, 2017; Smets, Gebuis, & Reynvoet, 2013; Van Opstal, Gevers, De Moor, & Verguts, 2008; Verguts, Fias, & Stevens, 2005) or the strength of associations between discrete quantities in a semantic network (Krajcsi, Lengyel, & Kojouharova, 2016). Alternative numerical judgment tasks can elicit distance effects that result from processing numerical magnitude rather than decision-making processes, including a priming task (Reynvoet, De Smedt, & Van den Bussche, 2009) or a same-different task (e.g., Van Opstal & Verguts, 2011).

In a priming task, participants view two sequentially presented numbers (i.e., prime-target pairs) and compare the target to a fixed standard (i.e., 5). Response times and error rates are smaller when the distance between the prime and target is smaller. This is known as the priming distance effect (PDE), which occurs because the first number primes the comparison (e.g., Defever, Sasanguie, Gebuis, & Reynvoet, 2011; Van Opstal et al., 2008). For example, given the prime-target pairs 3 – 4 and 1 – 4, comparing ‘4’ to ‘5’ will be faster in the first pair because ‘3’ primes ‘4’ more than ‘1’ primes ‘4’.

Pollack et al. (2016) used a priming paradigm to investigate the differences between literal symbol and Arabic numeral processing in adults. The authors tested whether literal symbols produced a PDE when they were assigned a particular numerical magnitude (e.g., y = 9) to use during comparison. Literal symbols could be the prime (i.e., first number in the pair) or the target (i.e., second number in the pair). Consistent with prior literature (Defever et al., 2011), Pollack et al. (2016) found a PDE with Arabic number pairs. However, there was no PDE with literal symbols, even though participants learned the numerical magnitudes with a high level of accuracy. Because participants compared both the prime and target to ‘5,’ the authors used the prime comparisons to look for a CDE and found similar results; a CDE for Arabic numerals, but not literal symbols. These findings suggest that literal symbol processing may fundamentally differ from Arabic numeral processing, even when literal symbols represent one numerical magnitude. Alternatively, the task may not have required participants to process numerical magnitude. Because participants pressed left and right buttons for ‘less than 5’ and ‘more than 5,’ participants may have merely associated literal symbols with ‘left’ or ‘right.’ Further, this study did not include numerical judgments with a non-alphabetic symbol set. Therefore, differences between literal symbol and Arabic numeral processing, and whether pre-existing associations in literacy affect literal symbol processing, remain open questions.

The Present Study

The present study addresses both limitations discussed above. First, it examines differences between processing literal symbols and Arabic numerals using a same-different task, in which participants decide whether two simultaneously presented symbols represent the same or different numerical magnitudes. This task produces a Same-Difference Distance Effect (SDDE) that results from numerical magnitude processing, in which response times and error rates decrease as different numerical distances increase. A same-different task is ideal to probe whether literal symbols can also produce distance effects, as with other number symbols. Prior studies have consistently elicited SDDEs with dot arrays, number words, Arabic numerals, and their cross-format combinations, such as ‘TWO’ and ‘7’ (Defever, Sasanguie, Vandewaetere, & Reynvoet, 2012; Dehaene & Akhavein, 1995; Duncan & McFarland, 1980; Van Opstal & Verguts, 2011). Importantly, cross-format combinations produce SDDEs from access to numerical magnitude representations, not decision-related processes (Van Opstal & Verguts, 2011) or perceptual effects (Cohen, 2009). Further, while letters (without numerical magnitude) induce comparison distance effects (Lovelace & Snodgrass, 1971; Parkman, 1971), they do not produce SDDEs (Van Opstal & Verguts, 2011), which further supports using a same-different task to investigate numerical magnitude processing with literal symbols. Second, the present study incorporates same-different judgments with artificial symbols to test whether pre-existing associations with literacy affect literal symbol processing.

Participants completed three same-different judgments, with Arabic numerals and number words (i.e., numbers only condition), Arabic numerals and literal symbols (i.e., literal symbols condition), and Arabic numerals and artificial symbols (i.e., artificial symbols condition). For the latter two, participants learned symbol-magnitude associations for the literal or artificial symbols. An SDDE was expected with Arabic numerals and number words. If pre-existing linguistic associations hamper literal symbol processing, there would be an SDDE for artificial symbols but not literal symbols. Alternatively, both could produce SDDEs, but literal symbol processing may be more difficult, resulting in increased error rates and/or response times.

Because the literal and artificial symbol sets require learned associations, participants with higher working memory capacity may respond more accurately and quickly than participants with low working memory capacity. Indeed, taxing working memory can affect performance on some number comparison tasks (van Dijck & Fias, 2011; van Dijck, Gevers, & Fias, 2009). A working memory task was included to account for this possibility.

Method

Participants

Twenty-four typically developing adolescents aged 14-18 (M = 16.63, SD = 1.28, 67% female, 87% right-handed) from the Boston area participated. A sample of adolescents of high school age provides an opportunity to examine literal symbol processing in the earlier stages of developing proficiency with literal symbols. Students in this age range have been introduced to literal symbols in a mathematics context and thus have learned (even if implicitly) that literal symbols can represent numerical magnitudes. Based on prior studies, a sample of 24 is adequate to elicit group-level symbolic SDDEs (e.g., Dehaene & Akhavein, 1995; Van Opstal & Verguts, 2011).

Participants were recruited via flyers, online postings, and through schools. All participants were native English speakers, since the language in which numbers are learned contributes to differences in mathematics performance (Dehaene, 1997). To ensure adequate introductory exposure to literal symbols, participants had previously passed Algebra I and either had taken, or were concurrently enrolled in, a subsequent mathematics class (e.g., Geometry). However, participants’ highest level of mathematics was not recorded. Participants who were 18 years old provided consent; those under 18 years old provided assent along with parental permission. The study was approved by the Harvard Committee on the Use of Human Subjects and participants received small monetary compensation for participating.

Stimuli and Procedures

Participation involved a one-hour testing session with three same-different tasks, two training tasks, and a working memory task. Training and same-different judgments were designed using OpenSesame 2.8.3 (Mathôt, Schreij, & Theeuwes, 2012) and presented on a Google Nexus 7 tablet using the OpenSesame Experiment Runtime Application. To assess working memory, participants completed a backward digit span task (Mueller, 2011), which was administered using The Psychology Experiment Building Language (Mueller & Piper, 2014) on a Macbook Pro running OSX.

Participants completed the tasks sitting in a quiet space at their school, in a university research lab, or at a public library. Participants first completed same-different judgments with Arabic numerals and number words, and ended with the backward digit span task. In-between, participants completed two sets of tasks that involved associating literal or artificial symbols with unique numerical magnitudes and subsequently performing same-different judgments with those symbols. The symbol set (i.e., literal symbols, artificial symbols) that participants worked with first was counterbalanced.

Same-Different Judgment With Numbers Only

Participants viewed cross-notation pairs of Arabic numerals and number words using 1, 2, 7, and 8; and “ONE,” “TWO,” “SEVEN,” and “EIGHT,” in Arial font size 72. Cross-notation pairs eliminate a visual matching strategy that interferes with semantic processing (Defever et al., 2012; Van Opstal & Verguts, 2011). Participants judged whether the pairs represented same or different magnitudes. Same pairs (e.g., ONE-1) had a distance of zero and different pairs were either Near, with a distance of one (e.g., TWO-1), or Far, with a distance of five, six, or seven (e.g., 8-ONE). There were 8 trials of Same pairs, each shown four times, 8 trials of Near pairs, each shown twice, and 16 trials of Far pairs, each shown twice, for 80 trials per block. There were three blocks, resulting in 240 total trials, shown in a pseudorandom order. As in Van Opstal and Verguts (2011, see Experiment 2), participants saw Near and Far pairs equally often. Same pairs were shown twice as often to provide approximately the same number of left and right responses. Trials began with a fixation dot displayed for 500 ms, followed by the number pair, which displayed until response. Participants touched the left side of the screen if the number pair represented the same magnitude and touched the right side of the screen if the number pair represented different magnitudes. The trial ended with a 500 ms intertrial interval.

Literal Symbols

Participants first completed a paired-associate learning task with four symbol-magnitude associations, similar to a procedure used in Pollack et al. (2016). Table 1 provides the set of literal symbols (adopted from Van Opstal & Verguts, 2011) and their numerical equivalents. The goal was for participants to equate the literal symbols with numerical magnitudes for use in a subsequent same-different task. Initially, participants viewed the four associations (e.g., Q = 1) for as long as needed, but for a minimum of 20 seconds. Then participants completed a test of the associations. Participants saw a symbol for 750 ms and were asked to vocalize the associated numerical magnitude and select it from a choice screen containing 1, 2, 7, and 8, each in one quadrant of the screen, displayed until response. The number positions changed for each trial. Feedback of “Good job!” or “Oops!” displayed for 750 ms and the correct symbol-magnitude association was displayed for 750 ms. The trial ended with a 500 ms intertrial interval. The task began with eight practice trials, two of each association. To test all associations approximately equally, trials were presented in the same pseudorandom order for all participants. Trials continued until the participant reached an accuracy of at least 93% with at least 21 additional trials.i On average, participants completed 29 trials (SD = 23, range: 21-109). This task took approximately five minutes.

Table 1

Arabic Numerals Used in All Three Conditions With the Associated Literal and Artificial Symbols

Arabic numeral Literal symbol Artificial symbol
1 Q jnc.v5i2.163-fs1
2 G jnc.v5i2.163-fs2
7 R jnc.v5i2.163-fs3
8 H jnc.v5i2.163-fs4

After paired-associate learning, participants completed a same-different task. Pairs were from the numerical same-different task, but in the different number pairs, the number words “ONE,” “TWO,” “SEVEN,” and “EIGHT” were replaced with Q, G, R, and H. Half of the Same pairs had the literal symbol on the left (e.g., Q-1) and the remaining half had the literal symbol on the right (e.g., 1-Q). All procedures were the same as the numerical same-different task.

Artificial Symbols

As with literal symbols, participants completed a paired-associate learning task and then completed a same-different task. All participants successfully completed the learning task with 8 practice and 21 additional trials. In the same-different task, different number pairs consisted of the numbers 1, 2, 7, and 8, and four artificial symbols, which were corresponding Gibson figures used in prior research on artificial symbol training (Tzelgov, Yehene, Kotler, & Alon, 2000). Gibson, Gibson, Pick, and Osser (1962) created these figures in accordance with characteristics of letters (e.g., number of curves, lines, and angles; open and closed forms; symmetry).

Table 1 displays the four artificial symbols, and their literal symbol and numerical equivalents. As with literal symbols, half of the Same pairs had the artificial digit on the left and the other half had the artificial digit on the right. All other procedures were the same as in the literal symbols condition.

Backward Digit Span

Finally, each participant completed a backward digit span task (Mueller & Piper, 2014) to measure working memory capacity. Each participant saw a sequence of numbers 3 to 10 digits in length. After the sequence ended, the participant typed the sequence in reverse order. If the response was correct, the next sequence was one digit longer. If incorrect, the next sequence was the same length. Length and number of correct items were recorded; participants’ backward digit span was measured as the previous string length after two unsuccessful recalls at the same string length. Average backward digit span was 6.71 (SD = 1.55, range: 4-10).

Data Analysis

Following Sasanguie, Defever, Van den Bussche, and Reynvoet (2011) and Van Opstal and Verguts (2011), same pairs were excluded from analysis since the SDDE only manifests between Near and Far pairs, even for cross-notation pairs. Mean error rate was calculated for each participant, separately for Near and Far pairs for each symbol set (i.e., numbers only, literal symbols, artificial symbols). Median response time was calculated for each participant for correct responses, separately for Near and Far pairs for each symbol set.

Error rate and response time were modeled as separate outcomes. The relationship between distance and each outcome was estimated using random-effects multi-level models. This approach is preferred for data analyses because it accounts for repeated-measures of distance (i.e., near, far) for each participant, simultaneously provides estimates of differences between all three symbols sets, allows for the inclusion of covariates, and facilitates model comparison. Equation (1) describes the two-level multi-level model:

1
Y i j = β 0 + β 1 N E A R i j + β 2 N U M i j + β 3 A S i j + β 4 X j + ( e i j + u j )

In Equation 1, Y i j represents each outcome (i.e., error rate, response time) for each distance i and each participant j. NEARij is a dichotomous predictor in which Near trials are coded as ‘1’ and Far trials are the reference category. A set of dummy predictors, NUMij and ASij, represent the Arabic numeral (NUMij = 1) and artificial symbol (ASij = 1) sets for each distance i and participant j; literal symbols serve as the reference category to facilitate comparison between participants’ performance with literal symbols and the other symbol sets. A vector of participant level covariates, Xj, includes backward digit span, age, gender, and order of administration (for potential order effects beyond counterbalancing). Finally, the model includes random effects for the repeated-measures (eij) and participant levels (uj) to account for the multi-level nature of the data. There are three parameters of interest; β 1 captures the effect of distance on the outcome, β 2 represents the average difference in the outcome between same-different judgments with number symbols and literal symbols, and β 3 represents the average difference in the outcome between same-different judgments with artificial and literal symbols. Together, β 2   and β 3 represent the effect of symbol set.

Models were fit using maximum likelihood estimation and bootstrapped standard errors (200 replications) to address violations of L1 and L2 residual normality. For error rate, Shapiro-Wilk tests showed violations of Level 1 (W = 0.955, p < .001) and Level 2 (W = 0.905, p = .03) residual normality. Similarly, for response time, Shapiro-Wilk tests showed violations of Level 1 (W = 0.876, p < .0001) and Level 2 (W = 0.891, p = .01) residual normality. Lastly, due to a software malfunction, one participant’s data for the literal symbols same-different condition was not recorded.

Results

Descriptive Statistics

Table 2 displays the descriptive statistics for error rate and response time for each of the three symbol sets, both overall and by Near and Far distances. As Table 2 shows, error rates were larger for Near trials than for Far trials, in the sample. Sample response times were longer for Near trials compared to Far trials. Further, sample error rates were higher on average for literal and artificial symbols than for the numbers only condition. Average sample median response times were longer for literal and artificial symbols compared to the numbers only condition.

Table 2

Descriptive Statistics for Error Rate and Response Time Outcomes, Overall and by Distance, for Each of the Three Symbol Sets

Symbol Set Error rate (%)
Response time (ms)
Overall Near Far Overall Near Far
Numbers only
M 3.56 2.95 1.17 772 809 770
SD 3.29 3.63 1.61 130 148 145
Literal symbols
M 4.91 4.26 1.59 921 953 931
SD 5.44 4.05 3.09 220 189 257
Artificial symbols
M 4.58 4.60 1.91 843 872 846
SD 4.05 4.56 2.31 140 185 147

No Differences Across Symbolic Formats for Error Rate

Table 3 displays a taxonomy of fitted models estimating the effects of distance and symbol set on each outcome, and includes parameter estimates, standard errors, random effects, and goodness-of-fit statistics. The left portion of Table 3 shows three fitted models estimating the effects of distance and symbol set on error rate.

Table 3

Selection of Multilevel Models Specifying the Effects of Distance and Symbol Set on Error Rate (i.e., Models 1-3) and Response Time (Models A-C) (nL1 = 142, nL2 = 24)

Parameter Error rate (%)
Response time (ms)
Model 1 Model 2 Model 3 Model A Model B Model C
Intercept
β ^ 2.718*** 1.528*** 1.650** 860.870*** 846.683*** 924.343***
SE 0.470 0.397 0.582 29.875 30.392 42.817
Near
β ^ 2.377*** 2.377*** 28.366*** 28.366***
SE 0.346 0.346 6.772 6.772
Numbers only
β ^ -0.777 -149.068***
SE 0.545 37.193
Artificial symbols
β ^ 0.417 -79.516**
SE 0.582 28.951
Random effects
σ ^ u 2.239** 2.302** 2.311** 146.366*** 146.505*** 147.899***
σ ^ e 2.737*** 2.406*** 2.344*** 119.368*** 118.350*** 98.240***
ρ ^ 0.401* 0.478** 0.493** 0.601*** 0.605*** 0.694***
Goodness-of-fit
Log-likelihood -363.647 -348.463 -345.344 -908.041 -907.030 -884.888

*p < .05. **p < .01. ***p < .001.

Model 1 in Table 3 displays the fitted error rate across all distances and symbol sets. The intra-class correlation (i.e., ρ ^ ) for Model 1 shows that 40% of variance in error rate was attributable to differences across participants. As Model 2 shows, there was a distance effect for error rate. The error rate on Near trials was 2.377 percentage points higher than the error rate on Far trials, on average (z = 6.88, p < .0001). The third model estimates the effects of distance and symbol set on error rate. As Model 3 shows, there was not a statistically significant relationship between symbol set and error rate (ps > .15). Further, the relationship between distance and error rate was unchanged, showing the distance effect on error rate remained when controlling for symbol set. Taken together, these models show an SDDE for error rate across all symbol sets, but no effect of symbol set.

Additional models were fit to test for interactions between symbol set and distance and to test the effects of the backward digit span, gender, age, and order covariates. None produced statistically significant relationships with error rate (ps > .34 and ps > .13, respectively). Figure 1 illustrates the fitted relationship between error rate and distance by symbol set (i.e., Model 3).

Click to enlarge
jnc.v5i2.163-f1
Figure 1

Fitted relationship between distance and error rate for all three symbol sets.

Note. The difference in error rate between Near and Far conditions illustrates the main effect of distance. However, there was no statistically significant difference in error rate between the three symbol sets, controlling for distance. Error bar represents standard error associated with fitted relationship (Model 3).

Differences Across Symbolic Formats for Response Time

The right portion of Table 3 shows three models estimating the effects of distance and symbol set on response time. Model A shows the fitted average response time across all distances and symbol sets. The intra-class correlation for Model A shows that approximately 60% of the variance in response time was attributable to differences between participants. Model B shows a statistically significant distance effect for response times. On average, participants took about 28 ms longer to respond to Near trials than to Far trials (z = 4.19, p < .0001). Model C shows the distance effects remained when controlling for symbol set.

Model C also shows there was a statistically significant difference in response time between literal symbols and the numbers only condition; it took about 149 ms longer, on average, to make same-different judgments with literal symbols, controlling for distance (z = -4.01, p < .0001). Crucially, there was a statistically significant difference in response time between literal symbols and artificial symbols. Average response times were about 80 ms longer when making same-different judgments with literal symbols, controlling for distance (z = -2.75, p = .006). Further, a General Linear Hypothesis test showed that, on average, participants took 69.55 ms longer to make same-different judgments with artificial symbols compared to Arabic numerals (SE = 19.63, z = 3.54, p < .001). This shows that same-different judgments were fastest in the numbers only condition, which is expected because of participants’ fluency with Arabic numerals and number words, and a lack of automatic associations between numerical magnitudes and the other two symbol sets.

Subsequent models were also fit to test for interactions between distance and symbol set, and to test for effects of the backward digit span, gender, age, and order covariates. None showed a statistically significant relationship with response time (ps > .40 and ps > .18, respectively). Figure 2 illustrates Model C, the relationship between response time and distance for each symbol set.

Click to enlarge
jnc.v5i2.163-f2
Figure 2

Fitted relationship between distance and response time (ms) for the three symbol sets.

Note. The figure illustrates the statistically significant distance effect and the statistically significant differences in average response times among all three symbol sets, controlling for distance.

To further investigate the response time differences, an additional multilevel model was fit using trial level data, adjusting the standard errors to account for the clustering of trials within participants. Unlike the more common participant-level analyses, which weight each participant’s aggregated response times equally, the trial level analysis differentially weights participants according to the number of trials each participant contributes (i.e., how accurate the participant is). Table 4 displays three models (A1, B1, C1) that relate to the participant level Models A-C (see Table 3). As Table 4 shows, the results were consistent across both analyses. For distance, comparison of Near trials took longer on average than Far trials. For symbol set, comparison with literal symbols took longer than comparison with numbers only and with artificial symbols, and the difference was greater in magnitude for the former than the latter. The trial and participant level estimates differ slightly, which is expected based on the differential weighting of the data (since only accurately-answered trials were included in the response time analysis). All subsequent models testing covariates and interactions were consistent with the participant-level analysis.

Table 4

Taxonomy of Multilevel Models Specifying the Relationship of Response Time With Distance and Symbol Set, Using Trial Level Data and Clustering Standard Errors at the Participant Level (n1 = 9,903; n2 = 24)

Parameter Response time (ms)
Model A1 Model B1 Model C1
Intercept
β ^ 946.508*** 937.758*** 1045.849***
SE 39.080 39.777 62.150
Near
β ^ 26.699*** 26.917***
SE 6.923 6.877
Numbers only
β ^ -207.910***
SE 54.421
Artificial symbols
β ^ -108.312*
SE 43.908
Random effects
σ ^ u 199.857*** 199.873*** 198.492***
σ ^ e 406.508*** 406.314*** 397.485***
ρ ^ 0.195 0.195 0.200
Goodness-of-fit
Log-likelihood -73600.353 -73595.646 -73378.444

*p < .05. **p < .01. ***p < .001.

Taken together, these results show that in addition to digits and number words, literal and artificial symbols produced an SDDE. As one would expect, it took longer, on average, to make same-different numerical magnitude judgments with symbols that are not Arabic numerals or number words. Importantly, these results suggest that some symbol sets may be more difficult to work with than others. Specifically, while there was no difference in accuracy when working with different symbol sets, same-different judgments with literal symbols elicited a cognitive processing cost compared to artificial symbols, which manifested as a longer response time, on average, regardless of distance.

Discussion

The present study used a series of same-different judgments to examine the nature of literal symbol processing related to numerical magnitude. It replicated an SDDE with Arabic numeral - number word pairs and examined whether pre-existing associations for literal symbols hinder literal symbol processing. Each aim is discussed in turn.

SDDEs Across Symbolic Formats

Participants compared cross-notation pairs of number words and Arabic numerals that either represented the same magnitude or differed in magnitude by a small or large amount. Participants showed an SDDE; error rate was lower and response times were shorter on average for number pairs with greater distance. These results support the presence of a symbolic SDDE in adolescents, which aligns with prior research on the symbolic SDDE in children (e.g., Defever et al., 2012) and adults (e.g., Dehaene & Akhavein, 1995; Van Opstal & Verguts, 2011).

After paired-associate learning, participants completed same-different tasks with literal and artificial symbols. Both symbol sets produced SDDEs for error rate and response time. Building on prior research on the SDDE (e.g., Dehaene & Akhavein, 1995; Van Opstal & Verguts, 2011), these findings imply that same-different judgments with newly-learned symbol formats involve processing numerical magnitude. These SDDEs also align with related studies that show congruity effects for numerical Stroop tasks with artificial symbols (Cohen Kadosh, Soskic, Iuculano, Kanai, & Walsh, 2010; Tzelgov et al., 2000) and comparison distance effects with artificial symbols associated with non-symbolic numerical magnitudes (Lyons & Ansari, 2009). The present results extend this work, suggesting that non-numeric symbols associated with symbolic, cardinal representations of number show behavioral signatures of numerical processing.

Further, the magnitude of the SDDEs did not differ across symbolic formats. This may seem surprising, since the literal and artificial symbol trainings may be akin to the development of symbol-quantity associations. Research on changes in the CDE over developmental time suggests that distance effects may be smaller (Lyons, Nuerk, & Ansari, 2015) or larger (Halberda & Feigenson, 2008; Holloway & Ansari, 2009; Sekuler & Mierkiewicz, 1977) for symbols learned more recently. However, literature examining SDDEs over developmental time suggest a stable magnitude across people of different ages (Defever et al., 2012). Priming paradigms also appear to produce stable distance effect magnitudes across developmental time (Defever et al., 2011; Reynvoet et al., 2009). These differences, between the stability of the SDDE and the PDE on one hand, and the variability of the CDE on the other, may stem from the latter’s lack of dependence on access to numerical magnitude representations.

Literal Symbol Processing

The present study lends additional insight into literal symbol processing of numerical magnitude. An SDDE when making judgments with literal symbols stands in contrast to prior related work in which literal symbols did not produce a PDE, while Arabic numerals did (Pollack et al., 2016). Differences in findings across the two studies may result from methodological differences. It may be that the same-different task required numerical magnitude processing while the priming task in Pollack et al. (2016) did not, since participants could use a ‘left’-‘right’ strategy rather than processing numerical magnitudes. Alternatively, same-different and priming numerical judgments may require a different level or strength of association between symbols and their numerical referents. Arabic numerals have strong, permanent associations with numerical magnitudes that are overlearned through formal schooling, whereas associations between literal symbols and numerical magnitudes in authentic mathematical contexts are temporary. Temporary associations may not be encoded strongly enough to elicit a PDE as with Arabic numerals, but may be strong enough to elicit an SDDE. Finally, the SDDE and PDE may arise from different mechanisms, such as a differential reliance on automatic versus intentional processing (Sasanguie et al., 2011), which could explain the discrepant findings across the two studies.

Even though an SDDE was present for literal symbols for both error rate and response time, the present study suggests that literal symbol processing involving numerical magnitude may differ from processing with other symbolic formats. It is expected that same-different judgments would be fastest in the numbers only condition because of strong associations between numerical magnitudes, Arabic numerals, and number words. However, when comparing performance between the literal and artificial symbol conditions, participants took longer on average to make numerical judgments with literal symbols, controlling for distance. Put another way, even though one would expect a processing cost when recalling newer associations generally, that cost is greater in magnitude for literal symbols compared to artificial ones. This suggests that the cognitive processing cost that manifests between the literal and artificial symbols conditions results from literal symbol processing per se.

What might account for this difference? First, one might argue that the difference in average response time between literal and artificial symbols may depend on the specific symbols and associations. For example, all symbol-magnitude associations were randomly assigned, which resulted in a numerical magnitude ordering for literal symbols that does not preserve the linear ordering of the alphabet (i.e., Q, G, R, H), which could in turn produce longer response times. However, this seems unlikely here given that the linear ordering of the alphabet does not affect performance on the same-different task (Van Opstal & Verguts, 2011). If ordinal positions of letters in the alphabet were to affect numerical magnitude processing, this would suggest one mechanism by which literal symbols are more difficult to work with in numerical contexts due to their pre-existing associations from literacy (i.e., ordinal representations). Such hypotheses can be tested explicitly in future studies, for example by contrasting performance utilizing symbol-quantity mappings that preserve and do not preserve the ordinality of the alphabet.

Second, the cognitive processing cost associated with literal symbol same-different judgments may reflect executive function ability. While in the present study working memory was not a statistically significant predictor of performance, inhibition may play a role. Participants may inhibit a literal symbol’s competing representation in the context of literacy when processing associated numerical magnitude information. Indeed, recent research has shown that inhibitory control plays an important role in performance on tasks that measure numerical magnitude processing (e.g., Gilmore et al., 2013). Future research can examine whether inhibitory control, measured in numerical and non-numerical contexts, may mitigate the magnitude of the literal symbol processing cost.

Limitations

There are several limitations to the present study. First, participants likely had heterogeneous levels of knowledge and instruction related to literal symbols. Participants came from different schools and may have had different (or no) formal instruction related to the role of literal symbols in mathematics (see Nie, Cai, & Moyer, 2009). Second, the participant age range was relatively large, suggesting the sample spanned beginning high school to beginning college. Age was not a statistically significant predictor of either outcome, which aligns with prior research on the SDDE (Defever et al., 2012; Duncan & McFarland, 1980). However, there may be differences in literal symbol processing across age, grade, or level or type of exposure to literal symbols. Future studies can address this limitation with a cross-sectional study that tests groups of participants from different grades in the same school, for example, and by confirming with a larger sample size that the covariates tested in the present study are unrelated to response time. A third limitation pertains to the Arabic numeral and number word stimuli. As noted in prior studies on symbolic number processing (Peters, De Smedt, & Op de Beeck, 2015), numerosity is confounded with the length of number words, which may affect same-different judgments when comparing Arabic numerals and number words. For example, ‘1’ is a small numerosity and ‘ONE’ has relatively few letters, whereas ‘8’ is a larger numerosity and ‘EIGHT’ has relatively many letters. Finally, the same-different task contrasts performance between different numerosities with Near and Far distances. As such, this task precludes a fine-grained analysis of distance that is more common to comparison tasks. However, same-different tasks eliminate the confound of distance effects that arise from decision-related processes (Van Opstal et al., 2008). These considerations underscore the importance of task selection when measuring numerical magnitude processing.

Conclusion

The present study examined differences in numerical magnitude processing across three symbolic formats. It focused on differences between literal and artificial symbol processing using same-different numerical judgments. Participants completed three same-different tasks in which they compared Arabic numerals to number words, literal symbols, or artificial symbols. To work with literal and artificial symbols, participants first associated these symbols with numerical magnitudes. All three symbol sets elicited an SDDE, evidence of numerical processing across all symbolic formats. Importantly, same-different judgments with literal symbols took longer than their artificial symbol counterparts, on average, regardless of distance. These findings suggest that literal symbol processing of numerical magnitude may engage mechanisms that require additional cognitive processing steps, such as reducing interference from competing representations or processing verbal labels, that are not required for working with other number symbols.

To learn algebra, students must develop fluency and flexibility with literal symbols, including when they represent numerical magnitude. The core of this fluency and flexibility is symbol-referent connections, investigated here in a restricted case, in which literal symbols represented one numerical magnitude as Arabic numerals do. While persistent student difficulty with literal symbol processing results from myriad factors, the present study suggests that difficulty partially results from unique, lower-level cognitive processing demands required for basic processing of literal symbols as symbols that represent numerical magnitude in mathematics contexts. Additional research can illuminate the nature of such mechanisms and how they impact students’ understanding of literal symbols, which in turn can support learning in higher-level mathematics.

Notes

i) One participant did not achieve a high level of accuracy for the literal symbols condition (69%). Sensitivity analyses showed that when the models were fit without this participant, magnitudes and statistical significance of relationships for distance and symbol set were essentially unchanged. Accordingly, results presented here include all participants.

Funding

This work was supported in part by the Data Collection Travel Grant from the Harvard Graduate School of Education.

Competing Interests

The author has declared that no competing interests exist.

Acknowledgments

I thank Jon Star, Gigi Luk, and Bert De Smedt for their mentorship on this project and their feedback on prior versions of this manuscript. I also thank the schools and students who participated, and Steven Hoffman and Darrick Yee for guidance with model fitting.

Data Availability

For this study, a dataset is freely available (see the Supplementary Materials section).

Supplementary Materials

The data for this study can be accessed via the PsychArchives repository (see Index of Supplementary Materials below).

Index of Supplementary Materials

References

  • Akgün, L., & Özdemir, M. E. (2006). Students’ understanding of the variable as general number and unknown: A case study. The Teaching of Mathematics, 9(1), 45-51.

  • Booth, L. (1999). Children’s difficulties in beginning algebra. In B. Moses (Ed.), Algebraic thinking, grades K-12: Readings from the NCTM’s school-based journals and other publications (pp. 299–307). Reston, VA, USA: National Council of Teachers of Mathematics.

  • Castronovo, J., & Göbel, S. M. (2012). Impact of high mathematics education on the number sense. PLOS ONE, 7(4), Article e33832. https://doi.org/10.1371/journal.pone.0033832

  • Christou, K. P., & Vosniadou, S. (2005). How students interpret literal symbols in algebra: A conceptual change approach. In Proceedings of the XXVII Annual Conference of the Cognitive Science Society (pp. 453–458). Stresa, Italy.

  • Cohen, D. J. (2009). Integers do not automatically activate their quantity representation. Psychonomic Bulletin & Review, 16(2), 332-336. https://doi.org/10.3758/PBR.16.2.332

  • Cohen Kadosh, R., Brodsky, W., Levin, M., & Henik, A. (2008). Mental representation: What can pitch tell us about the distance effect? Cortex, 44(4), 470-477. https://doi.org/10.1016/j.cortex.2007.08.002

  • Cohen Kadosh, R., Soskic, S., Iuculano, T., Kanai, R., & Walsh, V. (2010). Modulating neuronal activity produces specific and long-lasting changes in numerical competence. Current Biology, 20(22), 2016-2020. https://doi.org/10.1016/j.cub.2010.10.007

  • Defever, E., Sasanguie, D., Gebuis, T., & Reynvoet, B. (2011). Children’s representation of symbolic and nonsymbolic magnitude examined with the priming paradigm. Journal of Experimental Child Psychology, 109(2), 174-186. https://doi.org/10.1016/j.jecp.2011.01.002

  • Defever, E., Sasanguie, D., Vandewaetere, M., & Reynvoet, B. (2012). What can the same–different task tell us about the development of magnitude representations? Acta Psychologica, 140(1), 35-42. https://doi.org/10.1016/j.actpsy.2012.02.005

  • Dehaene, S. (1997). The number sense: How the mind creates mathematics. New York, NY, USA: Oxford University Press.

  • Dehaene, S., & Akhavein, R. (1995). Attention, automaticity, and levels of representation in number processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(2), 314-326. https://doi.org/10.1037/0278-7393.21.2.314

  • Dehaene, S., Dehaene-Lambertz, G., & Cohen, L. (1998). Abstract representations of numbers in the animal and human brain. Trends in Neurosciences, 21(8), 355-361. https://doi.org/10.1016/S0166-2236(98)01263-6

  • De Smedt, B., Noël, M.-P., Gilmore, C., & Ansari, D. (2013). How do symbolic and non-symbolic numerical magnitude processing skills relate to individual differences in children’s mathematical skills? A review of evidence from brain and behavior. Trends in Neuroscience and Education, 2(2), 48-55. https://doi.org/10.1016/j.tine.2013.06.001

  • Duncan, E. M., & McFarland, C. E. (1980). Isolating the effects of symbolic distance and semantic congruity in comparative judgments: An additive-factors analysis. Memory & Cognition, 8(6), 612-622. https://doi.org/10.3758/BF03213781

  • Gibson, E. J., Gibson, J. J., Pick, A. D., & Osser, H. (1962). A developmental study of the discrimination of letter-like forms. Journal of Comparative and Physiological Psychology, 55, 897-906. https://doi.org/10.1037/h0043190

  • Gilmore, C., Attridge, N., Clayton, S., Cragg, L., Johnson, S., Marlow, N., . . . Inglis, M., (2013). Individual differences in inhibitory control, not non-verbal number acuity, correlate with mathematics achievement. PLOS ONE, 8(6), Article e67374. https://doi.org/10.1371/journal.pone.0067374

  • Halberda, J., & Feigenson, L. (2008). Developmental change in the acuity of the “number sense”: The approximate number system in 3-, 4-, 5-, and 6-year-olds and adults. Developmental Psychology, 44(5), 1457-1465. https://doi.org/10.1037/a0012682

  • Holloway, I. D., & Ansari, D. (2009). Mapping numerical magnitudes onto symbols: The numerical distance effect and individual differences in children’s mathematics achievement. Journal of Experimental Child Psychology, 103(1), 17-29. https://doi.org/10.1016/j.jecp.2008.04.001

  • Kieran, C. (2007). Learning and teaching of algebra at the middle school through college levels: Building meaning for symbols and their manipulation. In F. K. Lester Jr. (Ed.), Second handbook of research on mathematics teaching and learning (Vol. 2, pp. 707–762). Charlotte, NC, USA: Information Age Publishing.

  • Krajcsi, A., & Kojouharova, P. (2017). Symbolic numerical distance effect does not reflect the difference between numbers. Frontiers in Psychology, 8, Article 2013. https://doi.org/10.3389/fpsyg.2017.02013

  • Krajcsi, A., Lengyel, G., & Kojouharova, P. (2016). The source of the symbolic numerical distance and size effects. Frontiers in Psychology, 7, Article 1795. https://doi.org/10.3389/fpsyg.2016.01795

  • Lovelace, E. A., & Snodgrass, R. D. (1971). Decision times for alphabetic order of letter pairs. Journal of Experimental Psychology, 88(2), 258-264. https://doi.org/10.1037/h0030922

  • Lyons, I. M., & Ansari, D. (2009). The cerebral basis of mapping nonsymbolic numerical quantities onto abstract symbols: An FMRI training study. Journal of Cognitive Neuroscience, 21(9), 1720-1735. https://doi.org/10.1162/jocn.2009.21124

  • Lyons, I. M., Nuerk, H.-C., & Ansari, D. (2015). Rethinking the implications of numerical ratio effects for understanding the development of representational precision and numerical processing across formats. Journal of Experimental Psychology: General, 144(5), 1021-1035. https://doi.org/10.1037/xge0000094

  • MacGregor, M., & Stacey, K. (1997). Students’ understanding of algebraic notation. Educational Studies in Mathematics, 33(1), 1-19. https://doi.org/10.1023/A:1002970913563

  • Mathôt, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 44(2), 314-324. https://doi.org/10.3758/s13428-011-0168-7

  • McNeil, N. M., Weinberg, A., Hattikudur, S., Stephens, A. C., Asquith, P., Knuth, E. J., & Alibali, M. W. (2010). A is for apple: Mnemonic symbols hinder the interpretation of algebraic expressions. Journal of Educational Psychology, 102(3), 625-634. https://doi.org/10.1037/a0019105

  • Moyer, R. S., & Landauer, T. K. (1967). Time required for judgments of numerical inequality. Nature, 215(5109), 1519-1520. https://doi.org/10.1038/2151519a0

  • Mueller, S. T. (2011). The PEBL digit span test [Computer software]. Retrieved from http://pebl.sf.net/battery.html

  • Mueller, S. T., & Piper, B. J. (2014). The psychology experiment building language (PEBL) and PEBL test battery. Journal of Neuroscience Methods, 222, 250-259. https://doi.org/10.1016/j.jneumeth.2013.10.024

  • Nie, B., Cai, J., & Moyer, J. C. (2009). How a standards-based mathematics curriculum differs from a traditional curriculum: With a focus on intended treatments of the ideas of variable. ZDM, 41(6), 777-792. https://doi.org/10.1007/s11858-009-0197-1

  • Parkman, J. M. (1971). Temporal aspects of digit and letter inequality judgments. Journal of Experimental Psychology, 91(2), 191-205. https://doi.org/10.1037/h0031854

  • Peters, L., De Smedt, B., & Op de Beeck, H. P. (2015). The neural representation of Arabic digits in visual cortex. Frontiers in Human Neuroscience, 9, Article 517. https://doi.org/10.3389/fnhum.2015.00517

  • Philipp, R. A. (1992). A study of algebraic variables: Beyond the student-professor problem. The Journal of Mathematical Behavior, 11(2), 161-176.

  • Pollack, C., Leon Guerrero, S. L., & Star, J. R. (2016). Exploring mental representations for literal symbols using priming and comparison distance effects. ZDM, 48(3), 291-303. https://doi.org/10.1007/s11858-015-0745-9

  • Restle, F. (1970). Speed of adding and comparing numbers. Journal of Experimental Psychology, 83(2, Pt. 1), 274-278. https://doi.org/10.1037/h0028573

  • Reynvoet, B., De Smedt, B., & Van den Bussche, E. (2009). Children’s representation of symbolic magnitude: The development of the priming distance effect. Journal of Experimental Child Psychology, 103(4), 480-489. https://doi.org/10.1016/j.jecp.2009.01.007

  • Rosnick, P. (1980). The presentation of the concept of variable and the development of problem solving skills: A multi-text review. Retrieved from ERIC database (ED295804): https://eric.ed.gov/?id=ED295804

  • Rosnick, P. (1982). Students’ symbolization processes in algebra. Retrieved from ERIC database (ED300230): http://eric.ed.gov/?id=ED300230

  • Rosnick, P. (1999). Some misconceptions concerning the concept of variable. In B. Moses (Ed.), Algebraic thinking, grades K-12: Readings from the NCTM’s school-based journals and other publications (pp. 313–315). Reston, VA, USA: National Council of Teachers of Mathematics.

  • Sasanguie, D., Defever, E., Van den Bussche, E., & Reynvoet, B. (2011). The reliability of and the relation between non-symbolic numerical distance effects in comparison, same-different judgments and priming. Acta Psychologica, 136(1), 73-80. https://doi.org/10.1016/j.actpsy.2010.10.004

  • Schneider, M., Beeres, K., Coban, L., Merz, S., Susan Schmidt, S., Stricker, J., & De Smedt, B. (2017). Associations of non-symbolic and symbolic numerical magnitude processing with mathematical competence: A meta-analysis. Developmental Science, 20(3), Article e12372. https://doi.org/10.1111/desc.12372

  • Schoenfeld, A. H., & Arcavi, A. (1999). On the meaning of variable. In B. Moses (Ed.), Algebraic thinking, grades K-12: Readings from the NCTM’s school-based journals and other publications (pp. 150–156). Reston, VA, USA: National Council of Teachers of Mathematics.

  • Sekuler, R., & Mierkiewicz, D. (1977). Children’s judgments of numerical inequality. Child Development, 48(2), 630-633. https://doi.org/10.2307/1128664

  • Smets, K., Gebuis, T., & Reynvoet, B. (2013). Comparing the neural distance effect derived from the non-symbolic comparison and the same-different task. Frontiers in Human Neuroscience, 7, Article 28. https://doi.org/10.3389/fnhum.2013.00028

  • Stacey, K., & MacGregor, M. (1999). Ideas about symbolism that students bring to algebra. In B. Moses (Ed.), Algebraic thinking, grades K-12: Readings from the NCTM’s school-based journals and other publications (pp. 308–312). Reston, VA, USA: National Council of Teachers of Mathematics.

  • Trigueros, M., & Ursini, S. (2003). First-year undergraduates’ difficulties in working with different uses of variable. In Research in collegiate mathematics education (Vol. 5, pp. 1–29). Providence, RI, USA: American Mathematical Society.

  • Tzelgov, J., Yehene, V., Kotler, L., & Alon, A. (2000). Automatic comparisons of artificial digits never compared: Learning linear ordering relations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(1), 103-120. https://doi.org/10.1037/0278-7393.26.1.103

  • Ursini, S., & Trigueros, M. (2004). How do high school students interpret parameters in algebra? In The 28th International Conference of the International Group for the Psychology of Mathematics Education (Vol. 4, pp. 361-368) [Proceedings]. Retrieved from ERIC database (ED489663): https://eric.ed.gov/?id=ED489663

  • Usiskin, Z. (1999). Conceptions of school algebra and uses of variables. In B. Moses (Ed.), Algebraic thinking, grades K-12: Readings from the NCTM’s school-based journals and other publications (pp. 7–13). Reston, VA, USA: National Council of Teachers of Mathematics.

  • van Dijck, J.-P., & Fias, W. (2011). A working memory account for spatial–numerical associations. Cognition, 119(1), 114-119. https://doi.org/10.1016/j.cognition.2010.12.013

  • van Dijck, J.-P., Gevers, W., & Fias, W. (2009). Numbers are associated with different types of spatial information depending on the task. Cognition, 113(2), 248-253. https://doi.org/10.1016/j.cognition.2009.08.005

  • Van Opstal, F., Gevers, W., De Moor, W., & Verguts, T. (2008). Dissecting the symbolic distance effect: Comparison and priming effects in numerical and nonnumerical orders. Psychonomic Bulletin & Review, 15(2), 419-425. https://doi.org/10.3758/PBR.15.2.419

  • Van Opstal, F., & Verguts, T. (2011). The origins of the numerical distance effect: The same–different task. Journal of Cognitive Psychology, 23(1), 112-120. https://doi.org/10.1080/20445911.2011.466796

  • Verguts, T., Fias, W., & Stevens, M. (2005). A model of exact small-number representation. Psychonomic Bulletin & Review, 12(1), 66-80. https://doi.org/10.3758/BF03196349

  • Wagner, S. (1983). What are these things called variables? Mathematics Teacher, 76(7), 474-479.