Empirical Research

Not Toeing the Number Line for Simple Arithmetic: Two Large-N Conceptual Replications of Mathieu et al. (Cognition, 2016, Experiment 1)

Jamie I. D. Campbell*1, Yalin Chen1, Maham Azhar1
[1] Department of Psychology, University of Saskatchewan, Saskatoon, Canada.

Journal of Numerical Cognition, 2021, Vol. 7(3), 248–258, https://doi.org/10.5964/jnc.6051

Received: 2019-09-05. Accepted: 2020-02-13. Published (VoR): 2021-11-30.

Handling Editors: Mojtaba Soltanlou, University of Surrey, Guildford, UK; Krzysztof Cipora, Loughborough University, Loughborough, UK

*Corresponding author at: Department of Psychology, University of Saskatchewan, 9 Campus Drive, Saskatoon, SK, Canada, S7N 5A5. Phone 306-914-2691, Fax 306-966-1959. E-mail: jamie.campbell@usask.ca

This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

We conducted two conceptual replications of Experiment 1 in Mathieu, Gourjon, Couderc, Thevenot, and Prado (2016, https://doi.org/10.1016/j.cognition.2015.10.002). They tested a sample of 34 French adults on mixed-operation blocks of single-digit addition (4 + 3) and subtraction (4 – 3) with the three problem elements (O1, +/-, O2) presented sequentially. Addition was 34 ms faster if O2 appeared 300 ms after the operation sign and displaced 5° to the right of central fixation, whereas subtraction was 19 ms faster when O2 was displaced to the left. Replication Experiment 1 (n = 74 recruited at the University of Saskatchewan) used the same non-zero addition and subtraction problems and trial event sequence as Mathieu et al., but participants completed blocks of pure addition and pure subtraction followed by the mixed-operation condition used by Mathieu et al. Addition RT showed a 32 ms advantage with O2 shifted rightward relative to leftward but only in mixed-operation blocks. There was no effect of O2 position on subtraction RT. Experiment 2 (n = 74) was the same except mixed-operation blocks occurred before the pure-operation blocks. There was an overall 13 ms advantage with O2 shifted right relative to leftward but no interaction with operation or with mixture (i.e., pure vs mixed operations). Nonetheless, the rightward RT advantage was statistically significant for both addition and subtraction only in mixed-operation blocks. Taken together with the robust effects of mixture in Experiment 1, the results suggest that O2 position effects in this paradigm might reflect task specific demands associated with mixed operations.

Keywords: replication, simple addition and subtraction, spatial attention

Cognitive links between the representation and processing of number and space are indicated by numerous behavioral, neuropsychological and brain imaging studies (Fias & Bonato, 2018; Fischer & Shaki, 2018; Knops, 2018; Shaki, Pinhas, & Fischer, 2018; Walsh, 2003). Dehaene, Bossini, and Giraux (1993) first demonstrated a relationship between numerical and spatial cognition using a parity-judgment task in which participants decided whether a single-digit number was odd or even by making a left- or right-side keyboard response. A response time (RT) advantage occurred for left-side responses to small numbers (i.e, 1, 2, 3, 4) and right-side responses to large numbers (i.e., 6, 7, 8, 9) compared to reverse mapping of response side and number size. Dehaene et al. named this the spatial–numerical association of response codes (SNARC) effect and proposed that it reflects a mental number line (MNL) representation of numerical magnitude that is spatially organized from left (smaller numbers) to right (large numbers). Since then, many experiments using a wide range of paradigms have confirmed a link between number magnitude and space (Casarotti, Michielin, Zorzi, & Umiltà, 2007; Fischer, Castel, Dodd, & Pratt, 2003; Fischer & Shaki, 2018; Shaki, Fischer, & Petrusic, 2009; Viarouge, Hubbard, & Dehaene, 2014; see also Fias & Van Dijck, 2016).

Beyond judgements about individual numbers, directional effects have also been observed in connection with performance of addition and subtraction (e.g., Hartmann, Mast, & Fischer, 2016; Li, Liu, Li, Dong, Huang, & Chen, 2018; Liu, Cai, Verguts, & Chen, 2017; Masson & Pesenti, 2014; Masson, Pesenti, Coyette, Andres, & Dormal, 2017; Mathieu, Epinat-Duclos, Sigovan, et al., 2018; Mathieu, Gourjon, Couderc, Thevenot, & Prado, 2016; McCrink, Dehaene, & Dehaene-Lambertz, 2007; Pinhas & Fischer, 2008; Pinhas, Shaki, & Fischer, 2015; Pinheiro-Chagas, Dotan, Piazza, & Dehaene, 2017). Of particular interest here is the paradigm developed by Mathieu et al. (2016, Experiment 1). This is a well-known paper with more than 60 citations in just a few years but there are several reasons to pursue a replication. First, this experiment was among the first (see also Wiemers, Bekkering, & Lindemann, 2014) to demonstrate evidence of a visuospatial attention shift acting on arithmetic performance itself, as opposed to an effect of arithmetic operation on efficiency of visuospatial attention (e.g., Masson & Pesenti, 2014). Consequently, it is important to affirm its replicability. Second, most studies of an association between space and arithmetic have used multi-digit problems (78 + 6, 78 - 6), whereas Mathieu et al. tested single-digit items (e.g., 8 + 6, 8 - 6). Solving multi-digit problems, however, includes solving one or more single-digit component problems; therefore, the mechanism of spatial associations for single-digit problems would also contribute to corresponding effects in multi-digit problems. This accentuates the importance of pursuing spatial effects for single-digit problems. Third, as explained further on, some of the Mathieu et al. results were not statistically compelling,

Mathieu et al. (2016, Experiment 1) analyzed the results of 34 students from the University of Lyon, France, on elementary addition and subtraction problems in random mixed-operation blocks. In their Experiment 1 paradigm, single-digit addition and subtraction problems (e.g., 5 + 4, 5 – 4) were displayed with the first operand (O1), the operator (+ or -), and the second operand (O2) presented sequentially. The spatial position of O2 was displaced either 5o to the left or to the right of the central fixation point where the first two elements appeared. The inter-stimulus interval (ISI) between the operator and O2 was 150, 300 or 450 ms. Mathieu et al. (2016) refer to these intervals as stimulus onset asynchronies (SOAs), but their Figure 1 correctly depicted ISIs of 150, 300 or 450 ms following offset of the 150 ms operator.1 Therefore, the SOAs between operator onset and O2 in their Experiment 1 were 300, 450 or 600 ms, whereas the ISI between the operator and O2 offset was 150, 300 or 450 ms. Although this is a potentially important discrepancy for comparing the time course of spatial effects in the literature, it has little bearing on the present issue of replication.

With the 300 ms ISI there was strong statistical evidence reported (p. 233) that O2 position had different effects on RT for addition and subtraction, F(1, 33) = 11.16, p = .002, MSE = .006, η p 2 = .25, BF10 = 24.28.2 Specifically, addition was 34 ms faster on average with O2 displaced rightward relative to leftward, t(33) = 2.67, p = .012, SE = 12.7, η 2 = .18, BF10 = 4.77, whereas subtraction was 19 ms faster with O2 displaced leftward relative to rightward, t(33) = -2.10, p = .04, SE = 9.0, η 2 = .12, but BF10 = 1.45. So, there was little evidence of an O2-leftward advantage for subtraction at the 300 ms ISI. At the 150 ms and 450 ms ISIs there was also little statistical evidence based on BF values we calculated that the position of O2 affected RT for either addition or subtraction. The authors concluded (p. 234) that “these results demonstrate that solving single-digit addition and subtraction problems is associated with on-line horizontal shifts of attention. This is consistent with the idea that these problems activate procedures that may involve shifts to the right or left of a MNL”. Other features of the results were reported to substantiate this claim. A subset of 15 participants in Experiment 1, tested on the advice of a reviewer, were also presented with addition and subtraction problems involving zero (e.g., 3 + 0, 3 - 0), which did not present effects of O2 position at the 300 ms ISI, 25 ms, t(14) = 0.22, p = .83, SE = 113.6, η 2 = .003, BF01 = 3.77; but no test of effects of O2 position on zero vs. non-zero problems was reported. The reported result fits with the attentional shift theory because zero-problems “should not require any shift along the MNL” (p. 231). Furthermore, Experiment 2 (n = 22) tested blocks of mixed single-digit addition and multiplication problems based on the same number pairs as Experiment 1 and using only the 300 ms ISI between the operator (+ or x) and O2. As in Experiment 1, addition presented an RT advantage with O2 displaced rightward compared to leftward, 36 ms, t(21) = 3.01, p = .007, SE = 11.96, η 2 = .30, BF10 = 11.02, whereas multiplication did not, -12 ms, t(21) = 0.79, p = .44, SE = 15.19, η 2 = .03, BF01 = 3.40. The results for addition replicated Experiment 1 and the null effect of O2 position for multiplication was predicted based on previous evidence that multiplication facts are retrieved directly from long-term memory (e.g., Campbell & Xue, 2001) and would not involve the MNL. Mathieu et al. concluded that there is RT facilitation for non-zero single-digit subtraction and addition problems when the spatial position of the second operand in a sequential presentation of problem elements is congruent with the operation-dependent scan direction on a MNL, running rightward up the number line for single-digit addition and leftward down the number line for single-digit subtraction.

The Replication Experiments

We conducted conceptual replications of Mathieu et al. (2016, Experiment 1). While the evidence was strong that addition benefitted from a rightward O2 shift compared to a leftward shift, the evidence for subtraction benefitting from a leftward shift was weak (BF10 = 1.45) despite p < .05. It would be worthwhile to provide additional evidence about this finding. Furthermore, rather than the effect of O2 position reflecting an intrinsic spatial-attentional component of simple addition and subtraction, instead the Mathieu et al. results might reflect demand characteristics of their experimental paradigm. After presentation of O1 for 500 ms the operation to be performed was not determined until the operator appeared 500 ms after O1 offset. The operation sign (+ or -) was displayed very briefly (150 ms) followed by a delay of 150 ms up to 450 ms before the second operand (O2) appeared. Following encoding of O1, how is the operator information encoded to prepare for problem processing once O2 appears? The requirement to remember which operation to perform might induce a strategy to encode the operation sign as a spatial instruction about direction relative to O1 (e.g., if + then answer higher than O1; if - then answer lower than O1), akin to the more-or-less heuristic identified by Shaki et al. (2018). Use of a task-dependant spatial strategy to encode operation could be sensitive to the congruency of the direction planned (i.e., higher or lower) and shifts in the spatial location of O2. In this view, encoding O2 for addition is faster with shifts to the right and encoding of O2 for subtraction faster with shifts to the left. To eliminate this possible task-specific explanation our conceptual replications included pure addition and pure subtraction conditions along with the mixed-operation condition employed by Mathieu et al. We tested the pure-operation conditions first because practicing the mixed-operation condition first could induce a strategic procedure that is carried over to the pure-operation conditions.

Experiment 1

Method

Participants

The power is .9 for a sample of 72 or more to detect an operation-specific O2 position effect about half the size ( η p 2  = .13) observed by Mathieu et al. Experiment 1 for the 300 ms ISI condition, F(1, 33) = 11.16, η p 2 = .25 for the main effect of operation; p. 233. We tested 74 and replaced two participants with high error rates especially on large addition problems (sum > 10). The final sample included 45 women and 29 men, aged 17 to 30 years, (M = 20.8, SD = 3.3), with 71 right-handed. They were recruited at the University of Saskatchewan and received course credit or $7.50 CAD. Sixty reported English as their first language for arithmetic, three Mandarin, two reported Chinese, two reported Spanish and one each reported Afrikaans, Arabic, Bengali, Korean, Portuguese, Tagalog and Urdu.

Apparatus

A Microsoft Windows-based computer used E-Prime 2.0 software (Schneider, Eschman, & Zuccolotto, 2012) to display stimuli on a 15 inch CRT monitor. A chin rest was centered in front of the monitor at a distance of about 40 cm. A microphone contected via the E-prime 2.0 response box provided the stop signal to measure verbal RT.

Stimuli and Design

The addition and subtraction stimuli were the same as Mathieu et al. (2016, Experiment 1). Small problems utilized the number pairs 21 31 32 41 42 43 51 52 53 54 and the larger problems were based on pairs 65 75 76 85 86 87 95 96 97 98. These pairs were used for both the addition and subtraction problems (e.g., the digit pair 54 yields 5 + 4 and 5 - 4). Stimuli appeared in black, Courier New 36-point font with a white background. This was inadvertently different from Mathieu et al. (2016, Experiment 1), which displayed white characters on a black background.

Participants received three conditions: pure addition, pure subtraction, and mixed-operation addition and subtraction problems. Each condition comprised four trial blocks and within each block each problem appeared once with problem order randomized. For the first block of each condition, a random half of the problems were assigned to the O2-left condition where O2 appeared 5° to the left of the central fixation point while the other half were assigned to the O2-right condition with O2 appearing 5° to the right of fixation. The position of O2 for each problem then alternated across blocks resulting in each problem being tested twice with each O2 position. The order of the pure addition and pure subtraction conditions was counterbalanced across participants and the pure conditions always occurred prior to the mixed-operations condition. This avoided the possibilty that practice with mixed operations could have carry-over effects on potential spatial processes engaged in the pure-operation conditions.

Procedure

The 30-minute experimental session took place in a quiet room with an experimenter present. Instructions emphasized both speed and accuracy. There was no performance feedback given to the participant. For each trial participants held the microphone in their preferred hand with their chin on the chin rest. The sequence of events on each trial (see Figure 1) was the same as the 300 ms ISI condition in Mathieu et al. (2016, Experiment 1).

Click to enlarge
jnc.6051-f1
Figure 1

Trial Event Sequence in the Replication Studies of Mathieu et al. (2016) Experiment 1

The trial started with a central fixation dot for 500 ms. The first operand (O1) then appeared at the fixation point for 500 ms followed by the fixation dot for 500 ms. Then the operator (+ or -) was displayed for 150 ms and replaced by the fixation dot for 300 ms. The second operand (O2) appeared to the left or right of the fixation dot for 150 ms followed by a blank screen for up to 3000 ms, which was the maximum time allowed for a response. Response timing began with the presentation of O2 and stopped when the participant spoke their answer into the microphone. During the 3150 ms response window, when a response was detected the screen switched immediately to a blank response recording screen. This allowed the experimenter to flag RTs that were spoiled because the microphone failed to detect the onset of the spoken response. Once the participant’s response was recorded the fixation dot for the next trial appeared.

Results

Analysis

We analyzed the data using an Operation (addition, subtraction) × O2 Position (left vs. right) × Mixture (pure vs. mixed-operation blocks) repeated-measures design. Mathieu et al. (2016) included problem size (small vs. large) as a factor but found that problem size had no important effects with respect to the O2 position manipulation. Therefore, we omitted it in our reported analysis to simplify the design. Mathieu et al. also performed a log transform on RTs prior to analysis. Our results were essentially the same with or without a log transform on RT and we report the latter. For completeness, however, the Supplementary Materials include a JASP file (JASP Team, 2019) containing the data and analyses with RTs log transformed and coded by problem size.3 Mathieu et al. reported one-tailed t-tests for analyses of spatial effects, although a one-tailed assmption is questionable given that this experiment was the first trial of their new paradigm. Nonetheless, in their Experiment 1 these tests were nominally significant using a two-tailed criterion. We report p-values for two-tailed tests throughout.

Response Time

A total of 1124 RTs (4.7%) were excluded from the analysis because they exceeded the 3150 ms second response deadline, were marked as spoiled by the experimenter, or were more than 4 SD from a participant’s mean RT in each Operation × O2 Position × Mixture cell. Table 1 presents mean RT and SE for correct answers as a function of operation, mixture, and O2 position. On average, addition was slower (988 ms) than subtraction (838 ms), F(1, 73) = 88.41, p < .001, MSE = 37544, η p 2 = .55, BF10 > 10000. There was weak evidence of an interaction between operation and mixture, F(1, 73) = 4.52, p = .04, MSE = 7332, η p 2 = .06, but BF10 = 1.07, reflecting slower subtraction in mixed-operation blocks relative to pure-operation blocks (854 vs 822 ms), t(73) = 2.64, p = .01, SE = 11.87, η 2 = .09, BF10 = 3.40, whereas mean addition RT did not differ between pure-operation (987 ms) and mixed-operations blocks (989 ms), t(73) = .11, p = .92, SE = 13.19, η 2 = .0002, BF01 = 8.55. We might expect mixed-operation blocks to be slower than pure-operation blocks for both operations owing to task switching or mixing effects (e.g., Campbell & Arbuthnott, 2010). Mixed-operation blocks always followed pure-operation blocks, however, and the same problems were tested in the pure-operation and mixed-operation blocks. Consequently, mixed-operation RT costs would be masked by facilitative effects of practice following pure-operation blocks.

Table 1

Performance in Experiment 1

Mixture Add
Subtract
O2 Left O2 Right L - R O2 Left O2 Right L - R
RT
Pure 983 (29) 991 (31) -7.5 (8.6) 824 (22) 821 (22) 3.3 (6.8)
Mixed 1004 (34) 972 (33) 31.5 (9.3) 856 (26) 852 (26) 4.1 (7.1)
Errors
Pure 4.3 (0.8) 4.6 (0.7) -0.3 (0.5) 2.1 (0.4) 1.4 (0.3) 0.6 (0.4)
Mixed 4.7 (0.6) 5.0 (0.7) -0.3 (0.4) 2.8 (0.5) 2.1 (0.3) 0.7 (0.3)

Note. Mean RT and mean percentage of errors (SE in brackets) in Experiment 1 by operation, O2 position and mixture. L – R = O2 Left – O2 Right.

There was a Mixture × O2 Position interaction, F(1, 73) = 7.94, p = .006, MSE = 1841, η p 2 = .10, BF10 = 5.30, owing to an overall 18 ms RT advantage with O2 appearing to the right of fixation in mixed operation blocks, t(73) = 3.01, p = .004, SE = 5.92, η 2 = .11, BF10 = 8.69, but no effect of O2 position in pure-operation blocks [mean RT difference of -2 ms, t(73) = -0.36, p = .72, SE = 5.87, η 2 = .002, BF01 = 8.07]. There also was weak evidence for the three-way interaction, F(1, 73) = 5.97, p = .02, MSE = 2267, η p 2 = .08, BF10 = 2.13. As Table 1 shows, in mixed-operation blocks the mean RT for addition was 32 ms faster with O2 appearing to the right than to the left of fixation, t(73) = 3.39, p = .001, SE = 9.31, η 2 = .14, BF10 = 25.62, but O2 position had no effect on addition RT in pure-operation blocks, -7.5 ms, t(73) = -0.88, p = .38, SE = 8.58, η 2 = .01, BF01 = 5.85. A separate analysis that included only addition trials confirmed a difference in the effect of O2 position between the mixed- and pure-operation conditions, F(1, 73) = 11.26, p = .001, MSE = 2499, η p 2 = .13, BF10 = 23.46.4

In contrast to addition, the results showed no effect on subtraction RT of the left vs. right position of O2 in either pure-operation, 3 ms, t(73) = .49, p = .62, SE = 6.75, η 2 = .003, BF01 = 7.61, or mixed-operation blocks, 4 ms, t(73) = .57, p = .57, SE = 7.13, η 2 = .005, BF01 = 7.29. Thus, the mixed-operation condition replicated the effects of the spatial position of O2 on addition RT observed by Mathieu et al. (2016), but we did not replicate an effect of O2 position on subtraction RT.

We used the MSE values generated in a Mixture × O2 Position ANOVA of the subtraction RT data to estimate expected variability in potential effects of O2 position. Using MorePower 6.0.4, we determined that the experiment had power of .9 for a main effect of O2 position of 17 ms or greater (MSE = 1958), and power of .98 to detect a Mixture × O2 Position interaction effect as large as that observed in the addition data (39 ms, MSE = 1609). Thus, the experiment was virtually certain to have detected a comparable effect of O2 position in the subtraction data, had it existed.

Percentage of Errors

Mathieu et al. (2016, Experiment 1) did not present an analysis of errors. For completeness here, Table 1 includes mean percentage of errors (i.e., the rate of incorrect arithmetic answers) for each Operation × O2 Position × Mixture cell. A factorial analysis is not warranted because 250 of the 592 cells in the data set (i.e., 2 × 2 × 2 × 74 participants) had a value of 0 (i.e., were at the measurement floor). This potentially inflates the Type I error rate owing to artificially low variances.

Discussion

Experiment 1 replicated Mathieu et al. (2016) with respect to faster addition with O2 rightward than leftward of fixation, but subtraction did not demonstrate the reverse pattern reported by Mathieu et al., although the experiment had ample power to detect an effect for subtraction. Additionally, the effect of O2 position in addition occurred only in the mixed-operations condition and not in pure addition blocks. This raises the possibility that effects of O2 position in this paradigm arise from demand characteristics of the mixed operations combined with a very brief 150 ms display of the operator to cue operation. A potentially important factor, however, distinguishing Experiment 1 here and the Mathieu et al. Experiment 1 is that the mixed-operation condition used by Mathieu et al. was preceded here by pure blocks of addition and subtraction. In Experiment 2, we reversed the situation with the mixed-operation condition preceding the pure-operation conditions.

Experiment 2

Method

Seventy-four people who had not participated in Experiment 1 (45 women and 29 men, aged 18 to 54 years, M = 27.7, SD = 7.6, 67 right-handed) were recruited as in Experiment 1. Forty-seven reported English as their first language for arithmetic, nine Persian, six Mandarin, four reported Spanish, three Vietnamese, two Farsi, and one each reported Bangla, Bengali, Chinese, Portuguese, Telugu and Urdu.

The apparatus, stimuli, design and procedure were the same as Experiment 1 except that the mixed-operation condition was presented before the pure addition and pure subtraction tasks. The order of the pure-operation tasks was counterbalanced across participants.

Results

Response Time

A total of 1318 RTs (5.6%) were excluded from the analysis by the same criteria as Experiment 1. The RT data from Experiment 2 were analyzed by operation, O2 position and mixture as in Experiment 1. The Supplementary Materials include a JASP file containing the data and analyses with RTs log transformed and coded by problem size to match Mathieu et al. (2016, Experiment 1). Table 2 presents the mean correct RT and SE for each cell. On average, addition was slower (942 ms) than subtraction (810 ms), F(1, 73) = 96.60, p < .001, MSE = 26897, η p 2 = .57, BF10 > 10000, and mixed-operation blocks were slower on average (898 ms) than pure operation blocks (854 ms), F(1, 73) = 25.88, p < .001, MSE = 11540, η p 2 = .26, BF10 > 10000. As in Experiment 1, there was some evidence of an interaction between operation and mixture, F(1, 73) = 6.64, p = .01, MSE = 3146, η p 2 = .08, but BF10 = 2.91, although the form of the interaction was different, probably reflecting the reversed order of the mixed and pure operation conditions: Subtraction was 33 ms slower in mixed-operation blocks relative to pure blocks (826 vs. 793 ms), t(73) = 4.22, p < .001, SE = 7.83, η 2 = .20, BF10 = 372.91, whereas mean addition RT was 57 ms slower in mixed compared to pure blocks (971 vs. 914 ms), t(73) = 4.85, p < .00001, SE = 11.71, η 2 = .24, BF10 = 3587.37. This pattern likely reflects, at least in part, RT gains from practice of problems in the mixed-operation condition transferring to the same problems tested again subsequently in the pure-operation conditions, with greater RT gains for the relatively more-difficult addition problems.

Table 2

Performance in Experiment 2

Mixture Add
Subtract
O2 Left O2 Right L - R O2 Left O2 Right L - R
RT
Pure 918 (27) 909 (26) 9.1 (8.3) 799 (17) 788 (16) 10.6 (5.9)
Mixed 979 (29) 962 (28) 16.9 (8.3) 835 (18) 818 (18) 16.5 (8.0)
Errors
Pure 3.9 (0.8) 3.8 (0.7) 0.1 (0.5) 1.8 (0.3) 2.1 (0.4) -0.3 (0.4)
Mixed 5.8 (0.9) 6.5 (1.0) -0.7 (0.6) 3.6 (0.5) 2.6 (0.6) 1.0 (0.4)

Note. Mean RT and mean percentage of errors (SE in brackets) in Experiment 2 by operation, O2 position and mixture. L – R = O2 Left – O2 Right.

There was also a main effect of O2 position because mean RT with O2 displaced rightward (869 ms) was slightly faster overall than with O2 leftward (883 ms), F(1, 73) = 9.68, p < .003, MSE = 2698, η p 2 = .12, BF10 = 11.62. Inspection of Table 2 might suggest that this effect was larger in mixed-operation blocks, 17 ms, t(73) = 2.66, p = .01, SE = 6.27, η 2 = .09, BF10 = 3.57, than pure blocks, 10 ms, t(73) = 1.85, p = .07, SE = 5.33, η 2 = .04, BF10 = 1.57, but the test of the interaction disconfirmed this possibility, F(1, 73) = 0.743, p = .39, MSE = 2316, η p 2 = .01, BF01 = 5.91. More importantly, there was good evidence against an Operation × O2 Position interaction, F(1, 73) = 0.007, p = .94, MSE = 1503, η p 2 < .001, BF01 = 8.57, as well as against the presence of a three-way interaction, F(1, 73) = 0.016, p = .90, MSE = 2182, η p 2 < .001, BF01 = 8.53.5

Percentage of Errors

The overall mean rate of incorrect arithmetic answers was 3.8% of trials. Table 2 includes mean percentage of errors for each Operation × O2 Position × Mixture cell. Again, a factorial analysis is not warranted given that 260 of the 592 cells in the data set contained a value of 0, the measurement floor.

Discussion

With respect to O2 position effects, Experiment 2 replicated only the rightward position RT advantage for addition observed in Experiment 1, but in this experiment subtraction also showed some evidence of a similar rightward advantage, and O2 position did not interact with arithmetic operation as in Experiment 1. Both addition and subtraction showed a significant effect of O2 position only in the mixed-operation condition, but the test of the interaction with pure vs. mixed operations strongly supported a null effect rather than the interaction observed in Experiment 1.

General Discussion

We replicated Mathieu et al. (2016, Experiment 1) with respect to a RT advantage for addition when the position of O2 was shifted rightward relative to O2 in a position left of fixation. We found no evidence, however, that subtraction RTs were faster with O2 shifted leftward; in fact in Experiment 2 there was some evidence that subtraction was faster with O2 shifted to the right. Nonetheless, there is evidence from a variety of experimental paradigms that subtraction is associated with a leftward direction or location (e.g., Blini, Pitteri, & Zorzi, 2019; Li et al., 2018; Liu, Cai, Verguts, & Chen, 2017; Masson & Pesenti, 2014; Zhu, You, Gan, & Wang, 2019), but there are also other exceptions (e.g., Masson, Letesson, & Pesenti, 2018). It may be, however, that the Mathieu et al. paradigm does not reliably engage this spatial association for subtraction. As mentioned previously, their own evidence for an effect for subtraction was very weak with a BF10 value of only 1.45. We conclude that their paradigm using the 300 ms ISI before O2 is sensitive to a rightward spatial bias for addition but not sensitive to a generalized leftward spatial association for subtraction.

Furthermore, in Experiment 1 the RT gain for addition with O2 rightward occurred only in mixed-operation trial blocks and not in the pure-addition blocks; but we did not replicate an effect of mixed vs. pure-operation blocks in Experiment 2. Perhaps, with the mixed-operation condition first as in Experiment 2, participants acquired a problem-encoding procedure to accommodate the trial-by-trial unpredictability of the operation to be performed after O1 was presented. This task set could then persist into the subsequent pure-operation blocks, washing out mixture effects in Experiment 2 that were robust in Experiment 1.

The present results raise the possibility that spatial bias effects in the Mathieu et al. (2016) experiments were the product of the task-set procedures that participants develop to handle unpredictable intermixed arithmetic operations. Randomly mixing operations across trials is a common practice in research into spatial biases in arithmetic; but mixing may invite spatial strategies such as the more-or-less heuristic (Shaki et al., 2018) that produce spatial effects not necessarily intrinsic to performance of addition or subtraction generally but rather arise as artefacts of paradigm-specific task demands. This is consistent with evidence that spatial and numerical representations must be jointly involved to observe effects suggesting a spatially organised MNL (Pinto, Pellegrino, Marson, Lasaponara, & Doricchi, 2019). As mentioned by Mathieu et al. (p. 234), the plus sign (+) cues that the answer will be more than O1 whereas the minus sign (-) cues that it will be less than O1. We suggest that this spatial encoding of operation (i.e., go up or down relative to O1) may be particularly likely if the operator sign is presented very briefly in the context of mixed-operation trials. With this operation encoding strategy, the addition operator shifts attention rightward, facilitating encoding of O2 when it appears on the right relative to the left side. In this view, the effect is not intrinsic to the addition process per se, but instead resides in speeding up O2 encoding time. In theory, this spatial encoding effect would be reversed for subtraction, although our experiments did not find an advantage for subtraction with O2 displaced leftward. This result, combined with the O2-rightward RT advantage in Experiment 2 for both addition and subtraction, indicate that no single or simple mechanism can explain the present results. Nonetheless, the context sensitivity of spatial effects in the Mathieu et al. paradigm suggests that the task does not measure an automatic, functionally integrated spatial processing component for single-digit addition or subtraction.

Notes

1) Jérôme Prado confirmed (25/11/2019) that Figure 1 in Mathieu et al. (2016) correctly depicted the trial procedure.

2) The Bayes Factor (BF) values reported in this paper were calculated using MorePower 6.0.4 (Campbell & Thompson, 2012). The program implements the Bayesian Information Criterion (BIC) as proposed by Masson (2011; see also Jarosz & Wiley, 2014; Nathoo & Masson, 2016; Wagenmakers, 2007), which approximates the unit-information prior as a default objective Bayes prior probability (Wagenmakers, 2007). BF01 denotes the odds ratio of the null (H0) over the alternative hypothesis (H1) and BF10 is the odds ratio of H1 over H0. A conventional interpretation is that BF greater than 10 provides relatively strong evidence for the hypothesis, BF values of 3 to 10 provide moderate evidence, whereas BF less than 3 provides little evidence one way or the other for H0 or H1 (e.g., Wetzels, van Ravenzwaaij, & Wagenmakers, 2015).

3) Log transformation is a common remedy to normalize positively skewed RT distributions, but factors such as problem size in arithmetic studies can distribute RTs from some experimental cells disproportionately into the upper tail of the experiment-wide distribution of RTs (Campbell & Penner-Wilger, 2006). Upper tail RTs are affected most by log normalization; consequently, log transformations can mask genuine effects on RT or produce misleading artefacts. Both the benefits and potential costs of RT transformations need to be considered (Whelan, 2008).

4) As observed by Mathieu et al. (2016), the RT advantage for addition in mixed-operation blocks with O2 appearing rightward of fixation compared to O2 leftward of fixation was observed both for numerically smaller (sum ≤ 10) problems, 22 ms, t(73) = 2.68, p = .009, SE = 8.23, η 2 = .09, BF10 = 3.77, and larger (sum > 10) problems, 40 ms, t(73) = 2.59, p = .01, SE = 15.35, η 2 = .08, BF10 = 2.97.

5) Twelve of the 74 participants reported that their first language for arithmetic was Farsi, Persian or Urdu, which are languages with right-to-left reading direction. The SNARC effect appears to be sensitive to participants’ habitual reading direction such that right-to-left readers have demonstrated a reverse SNARC effect (Dehaene et al., 1993; Shaki et al., 2009) suggesting a reverse mapping of number magnitude to space between right-to-left and left-to-right readers (see also Azhar, Chen, & Campbell, 2020). To our knowledge effects of reading direction on number-space associations have not been investigated for simple arithmetic. We repeated the analyses for Experiment 2 excluding right-to-left readers and found the same pattern of results for RT with the remaining 62 participants. This analysis was included among the Supplementary Materials.

Funding

This work was supported by a Discovery Grant awarded to Jamie Campbell by the Natural Sciences and Engineering Research Council of Canada.

Acknowledgments

The authors have no additional (i.e., non-financial) support to report.

Competing Interests

The authors report no conflicts or competing interests.

Ethics Approval

This research was approved on ethical grounds by the Behavioural Research Ethics Board of the University of Saskatchewan, Canada.

Data Availability

For this article, a dataset is freely available (Campbell, Chen, & Azhar, 2020).

Supplementary Materials

The Supplementary Materials contain the data, materials, and code for this study (for access see Index of Supplementary Materials below).

Index of Supplementary Materials

  • Campbell, J. I. D., Chen, Y., & Azhar, M. (2020). Supplementary materials to "Not toeing the number line for simple arithmetic: Two large-n conceptual replications of Mathieu et al. (Cognition, 2016, Experiment 1)" [Research data, materials, and code]. OSF. https://osf.io/9b45h/

  • Journal of Numerical Cognition. (Ed.). (2021). Supplementary materials to "Not toeing the number line for simple arithmetic: Two large-n conceptual replications of Mathieu et al. (Cognition, 2016, Experiment 1)" [Open peer-review]. PsychOpen GOLD. https://doi.org/10.23668/psycharchives.5221

References

  • Azhar, M., Chen, Y., & Campbell, J. I. D. (2020). Reading direction and spatial effects in parity and arithmetic tasks. Psychological Research. Advance online publication. https://doi.org/10.1007/s00426-020-01397-y

  • Blini, E., Pitteri, M., & Zorzi, M. (2019). Spatial grounding of symbolic arithmetic: An investigation with optokinetic stimulation. Psychological Research, 83(1), 64-83. https://doi.org/10.1007/s00426-018-1053-0

  • Campbell, J. I. D., & Arbuthnott, K. A. (2010). Effects of mixing and cuing simple addition and multiplication. European Journal of Cognitive Psychology, 22, 422-442. https://doi.org/10.1080/09541440902903629

  • Campbell, J. I. D., & Penner-Wilger, M. (2006). Calculation latency: The μ of memory and the τ of transformation. Memory & Cognition, 34, 217-226. https://doi.org/10.3758/BF03193400

  • Campbell, J. I. D., & Thompson, V. A. (2012). MorePower 6.0 for ANOVA with relational confidence intervals and Bayesian analysis. Behavior Research Methods, 44, 1255-1265. https://doi.org/10.3758/s13428-012-0186-0

  • Campbell, J. I. D., & Xue, Q. (2001). Cognitive arithmetic across cultures. Journal of Experimental Psychology: General, 130, 299-315. https://doi.org/10.1037/0096-3445.130.2.299

  • Casarotti, M., Michielin, M., Zorzi, M., & Umiltà, C. (2007). Temporal order judgment reveals how number magnitude affects visuospatial attention. Cognition, 102(1), 101-117. https://doi.org/10.1016/j.cognition.2006.09.001

  • Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122(3), 371-396. https://doi.org/10.1037/0096-3445.122.3.371

  • Fias, W., & Bonato, M. (2018). Which space for numbers. In A. Henik & W. Fias (Eds.), Heterogeneity of function in numerical cognition (pp. 233-242). https://doi.org/10.1016/B978-0-12-811529-9.00002-9

  • Fias, W., & Van Dijck, J. (2016). The temporary nature of number—Space interactions. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 70(1), 33-40. https://doi.org/10.1037/cep0000071

  • Fischer, M. H., Castel, A. D., Dodd, M. D., & Pratt, J. (2003). Perceiving numbers causes spatial shifts of attention. Nature Neuroscience, 6(6), 555-556. https://doi.org/10.1038/nn1066

  • Fischer, M. H., & Shaki, S. (2018). Number concepts: Abstract and embodied. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1752), Article 20170125. https://doi.org/10.1098/rstb.2017.0125

  • Hartmann, M., Mast, F. W., & Fischer, M. H. (2016). Counting is a spatial process: Evidence from eye movements. Psychological Research, 80(3), 399-409. https://doi.org/10.1007/s00426-015-0722-5

  • Jarosz, A. F., & Wiley, J. (2014). What are the odds? A practical guide to computing and reporting Bayes factors. Journal of Problem Solving, 7, Article 2. https://doi.org/10.7771/1932-6246.1167

  • JASP Team. (2019). JASP (Version v0.10.2) [Computer software]. Retrieved from https://jasp-stats.org

  • Knops, A. (2018). Neurocognitive evidence for spatial contributions to numerical cognition. In A. Henik & W. Fias (Eds.), Heterogeneity of function in numerical cognition (pp. 211-232). https://doi.org/10.1016/B978-0-12-811529-9.00011-X

  • Li, M., Liu, D., Li, M., Dong, W., Huang, Y., & Chen, Q. (2018). Addition and subtraction but not multiplication and division cause shifts of spatial attention. Frontiers in Human Neuroscience, 12, Article 183. https://doi.org/10.3389/fnhum.2018.00183

  • Liu, D., Cai, D., Verguts, T., & Chen, Q. (2017). The time course of spatial attention shifts in elementary arithmetic. Scientific Reports, 7(1), Article 921. https://doi.org/10.1038/s41598-017-01037-3

  • Masson, M. E. J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679-690. https://doi.org/10.3758/s13428-010-0049-5

  • Masson, N., Letesson, C., & Pesenti, M. (2018). Time course of overt attentional shifts in mental arithmetic: Evidence from gaze metrics. Quarterly Journal of Experimental Psychology, 71(4), 1009-1019. https://doi.org/10.1080/17470218.2017.1318931

  • Masson, N., & Pesenti, M. (2014). Attentional bias induced by solving simple and complex addition and subtraction problems. Quarterly Journal of Experimental Psychology, 67(8), 1514-1526. https://doi.org/10.1080/17470218.2014.903985

  • Masson, N., Pesenti, M., Coyette, F., Andres, M., & Dormal, V. (2017). Shifts of spatial attention underlie numerical comparison and mental arithmetic: Evidence from a patient with right unilateral neglect. Neuropsychology, 31(7), 822-833. https://doi.org/10.1037/neu0000361

  • Masson, N., Pesenti, M., & Dormal, V. (2017). Impact of optokinetic stimulation on mental arithmetic. Psychological Research, 81(4), 840-849. https://doi.org/10.1007/s00426-016-0784-z

  • Mathieu, R., Epinat-Duclos, J., Sigovan, M., Breton, A., Cheylus, A., Fayol, M., . . . Prado, J. (2018). What’s behind a “+” sign? Perceiving an arithmetic operator recruits brain circuits for spatial orienting. Cerebral Cortex, 28(5), 1673-1684. https://doi.org/10.1093/cercor/bhx064

  • Mathieu, R., Gourjon, A., Couderc, A., Thevenot, C., & Prado, J. (2016). Running the number line: Rapid shifts of attention in single-digit arithmetic. Cognition, 146, 229-239. https://doi.org/10.1016/j.cognition.2015.10.002

  • McCrink, K., Dehaene, S., & Dehaene-Lambertz, G. (2007). Moving along the number line: Operational momentum in nonsymbolic arithmetic. Perception & Psychophysics, 69(8), 1324-1333. https://doi.org/10.3758/BF03192949

  • Nathoo, F. S., & Masson, M. E. J. (2016). Bayesian alternatives to null-hypothesis significance testing for repeated-measures designs. Journal of Mathematical Psychology, 72, 144-157. https://doi.org/10.1016/j.jmp.2015.03.003

  • Pinhas, M., & Fischer, M. H. (2008). Mental movements without magnitude? A study of spatial biases in symbolic arithmetic. Cognition, 109(3), 408-415. https://doi.org/10.1016/j.cognition.2008.09.003

  • Pinhas, M., Shaki, S., & Fischer, M. H. (2015). Addition goes where the big numbers are: Evidence for a reversed operational momentum effect. Psychonomic Bulletin & Review, 22(4), 993-1000. https://doi.org/10.3758/s13423-014-0786-z

  • Pinheiro-Chagas, P., Dotan, D., Piazza, M., & Dehaene, S. (2017). Finger tracking reveals the covert stages of mental arithmetic. Open Mind, 1(1), 30-41. https://doi.org/10.1162/OPMI_a_00003

  • Pinto, M., Pellegrino, M., Marson, F., Lasaponara, S., & Doricchi, F. (2019). Reconstructing the origins of the space-number association: Spatial and number-magnitude codes must be used jointly to elicit spatially organised mental number lines. Cognition, 190, 143-156. https://doi.org/10.1016/j.cognition.2019.04.032

  • Schneider, W., Eschman, A., & Zuccolotto, A. (2012). E-Prime user's guide. Pittsburgh, PA, USA: Psychology Software Tools, Inc.

  • Shaki, S., Fischer, M. H., & Petrusic, W. M. (2009). Reading habits for both words and numbers contribute to the SNARC effect. Psychonomic Bulletin & Review, 16(2), 328-331. https://doi.org/10.3758/PBR.16.2.328

  • Shaki, S., Pinhas, M., & Fischer, M. H. (2018). Heuristics and biases in mental arithmetic: Revisiting and reversing operational momentum. Thinking & Reasoning, 24(2), 138-156. https://doi.org/10.1080/13546783.2017.1348987

  • Viarouge, A., Hubbard, E. M., & Dehaene, S. (2014). The organization of spatial reference frames involved in the SNARC effect. Quarterly Journal of Experimental Psychology, 67, 1484-1499. https://doi.org/10.1080/17470218.2014.897358

  • Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779-804. https://doi.org/10.3758/BF03194105

  • Walsh, V. (2003). A theory of magnitude: Common cortical metrics of time, space and quantity. Trends in Cognitive Sciences, 7, 483-488. https://doi.org/10.1016/j.tics.2003.09.002

  • Wetzels, R., van Ravenzwaaij, D., & Wagenmakers, E.-J. (2015). Bayesian Analysis. In R. L. Cautin & S. O. Lilienfeld (Eds.), The Encyclopedia of Clinical Psychology (pp. 274-283). https://doi.org/10.1002/9781118625392.wbecp453

  • Wiemers, M., Bekkering, H., & Lindemann, O. (2014). Spatial interferences in mental arithmetic: Evidence from the motion-arithmetic compatibility effect. Quarterly Journal of Experimental Psychology, 67, 1557-1570. https://doi.org/10.1080/17470218.2014.889180

  • Whelan, R. (2008). Effective analysis of reaction time data. The Psychological Record, 58, 475-482. https://doi.org/10.1007/BF03395630

  • Zhu, R., You, X., Gan, S., & Wang, J. (2019). Spatial attentions shifts in addition and subtraction arithmetic: Evidence of eye movement. Perception, 48, 835-849. https://doi.org/10.1177/0301006619865156