Special Thematic Section on "Tracking the Continuous Dynamics of Numerical Processing"

Tracking the Continuous Dynamics of Numerical Processing: A Brief Review and Editorial

Thomas J. Faulkenberry*a, Matthias Witteb, Matthias Hartmanncd


Many recent studies in numerical cognition have moved beyond the use of purely chronometric techniques in favor of methods which track the continuous dynamics of numerical processing. Two examples of such techniques include eye tracking and hand tracking (or computer mouse tracking). To reflect this increased concentration on continuous methods, we have collected a group of 5 articles that utilize these techniques to answer some contemporary questions in numerical cognition. In this editorial, we discuss the two paradigms and provide a brief review of some of the work in numerical cognition that has profited from the use of these techniques. For both methods, we discuss the past research through the frameworks of single digit number processing, multidigit number processing, and mental arithmetic processing. We conclude with a discussion of the papers that have been contributed to this special section and point to some possible future directions for researchers interested in tracking the continuous dynamics of numerical processing.

Keywords: eye tracking, hand tracking, mouse tracking, numerical cognition

Journal of Numerical Cognition, 2018, Vol. 4(2), doi:10.5964/jnc.v4i2.179

Published (VoR): 2018-09-07.

*Corresponding author at: Department of Psychological Sciences, Box T-0820, Tarleton State University, Stephenville, TX 76401, USA. E-mail: faulkenberry@tarleton.edu

This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The last 20 years have seen a great increase in the interest surrounding fundamental questions in numerical and mathematical cognition. Along with this increased interest, methods for approaching these questions have increased in complexity and diversity (c.f., Cohen Kadosh & Dowker, 2015). Early on, chronometric techniques allowed researchers to develop the early theories of basic numerical representation (e.g., Moyer & Landauer, 1967) and mental arithmetic (e.g., Groen & Parkman, 1972). Since those seminal works, these theories have matured, ever spawning new predictions and further deep questions about the nature of numerical and mathematical thinking.

Though chronometric methods still comprise a large portion of the current work in numerical cognition, there is a burgeoning literature that uses continuous, dynamic measures of cognitive processing to work on open problems in our field. Such methods include eye tracking (Hartmann, 2015; Hartmann & Fischer, 2016; Mock, Huber, Klein, & Moeller, 2016), hand tracking (Santens, Goossens, & Verguts, 2011; Song & Nakayama, 2008), and computer mouse tracking (Faulkenberry & Rey, 2014; Fischer & Hartmann, 2014). Compared to chronometric methods, which provide a measure of time elapsed for an experimental task, these dynamic methods additionally provide a continuous stream of information recorded during an experimental task. As such, methods such as eye tracking and hand/mouse tracking have begun to give us an unprecedented window into the mathematical mind.

The added value of these methods is crucial in order to advance the understanding of the mechanisms underlying simple number processing and mental arithmetic. For simple number processing (e.g., number comparison), continuous dynamical methods have allowed us to tackle a number of difficult problems, such as probing the temporal dynamics of mental number line activation (Myachykov, Ellis, Cangelosi, & Fischer, 2016) and the sequence of decade and unit activations in two-digit numbers (Moeller, Fischer, Nuerk, & Willmes, 2009). Whereas some of these studies have allowed us to capture the fundamental characteristics of how numerical representations are formed, others have allowed us to further test competing theories of numerical phenomena, including explanations for the dynamics of the numerical distance effect (Faulkenberry, 2016) and the size congruity effect (Faulkenberry, Cruise, Lavro, & Shaki, 2016). For mental arithmetic, continuous dynamical methods can reveal the timing of activation of the different elements of the arithmetic problem (operands and operators) and their spatial association (Hartmann, Laubrock, & Fischer, 2018; Marghetis, Núñez, & Bergen, 2014; Pinheiro-Chagas, Dotan, Piazza, & Dehaene, 2017), and provide insights about how people select strategies in mental arithmetic (Huebner & LeFevre, 2017).

Against this background, we have edited a special collection of articles for the Journal of Numerical Cognition that focuses on the dynamics of numerical and mathematical processing. The purpose of this special collection is to bring together a collection of work in numerical and mathematical cognition that exploits techniques such as eye tracking and hand movement tracking in order to foster a deeper understanding of the processes involved in numerical and mathematical representation.

In this review and editorial, we have two immediate purposes. First, we wish to provide a brief review of the two dominant continuous dynamic methods in numerical cognition: eye tracking and hand movement tracking. Second, we will briefly describe the contributions of the authors in this special collection and conclude with an evaluation of the current state of affairs with respect to continuous methods in numerical cognition.

Eye Tracking in Numerical Cognition [TOP]

Eye tracking studies are based on the online recording of eye movements during an experimental task. These movements, consisting of rapid jumps called saccades and resting periods called fixations, provide a direct record of the cognitive processes involved in scene exploration, goal-directed behavior, problem solving, language comprehension, and mental imagery (e.g., Liversedge & Findlay, 2000; Richardson, Dale, & Spivey, 2007). Particularly, fixation positions indicate the locus of an observer’s interest and focus of attention, and the duration of a fixation indicates the cognitive processing time of the fixated item (Just & Carpenter, 1980; Rayner & Pollatsek, 1989). By recording eye movements at a high frequency (usually on the order of hundreds of times per second), researchers can obtain a detailed record of the processes that occur during an experimental task, yielding comparatively richer descriptive data than can be obtained by recording response times alone.

Though eye tracking has a long history (Wade & Tatler, 2005), especially in the study of reading comprehension and associated processes (Rayner, 1998), its use in the study of numerical and mathematical processing has primarily emerged in the last decade. Though a comprehensive review of this literature is beyond the scope of this editorial, we will highlight some of the recent results which use eye tracking to investigate single-digit numerical processing, multi-digit numerical processing, and mental arithmetic. The interested reader is invited to consult the reviews of Hartmann and Fischer (2016) or Mock et al. (2016) for further reading.

Single Digit Number Processes [TOP]

Early research in this area focused on basic aspects of symbolic number processing, such as processing speed and reading time. For example, Brysbaert (1995) found that reading time increases with increasing numerical magnitude (in analogy to the word frequency effect), suggesting that semantic attributes such as number meanings are activated in an early processing stage. Much of the more recent research that has used eye tracking to study single digit number representation has focused on the well-known observation of Dehaene, Bossini, and Giraux (1993) that people are faster when using leftward responses to categorize small numbers and rightward responses to categorize large numbers. This observation of a spatial-numerical association of response codes (i.e., the SNARC effect) has resulted in a large body of research on the “mental number line”, a hypothesis that numbers are cognitively represented as a spatial continuum with an implicit ordering, where small numbers are represented to the left of large numbers.

While most of the early research on the mental number line was based on chronometric analysis, several recent studies have shown that eye movements also show this spatial-numerical association. For example, Loetscher, Bockisch, Nicholls, and Brugger (2010) demonstrated that leftward and downward shifts of eye gaze reliably predicted that the next number generated by a participant in a random number generation task would be smaller than the previously generated number. Similarly, a rightward and upward movement reliably predicted that the next number would be larger than the previous. Remarkably, this ocular association between number and space seems to be spontaneous even in passive listening tasks. This was demonstrated by Myachykov et al. (2016), who found that even when participants were asked to maintain fixation on a central point while listening to auditory-presented number names, they showed reliable leftward drift in eye movements when listening to small numbers, and conversely, rightward drift when listening to large numbers.

Multidigit Number Processes [TOP]

Whereas single-digit number tasks seem to evoke a tight correspondence between directional eye movement and spatial number representation, studies involving numbers with multiple components (e.g., multidigit Arabic numerals, such as 24, and fractions, such as 3/7) have used eye tracking to uncover other interesting results. One of the early studies in this regard was Moeller et al. (2009), who used eye tracking to tackle the classic problem of sequential versus parallel architecture in two-digit number representation. Moeller et al. showed that eye fixation patterns implied that two-digit number comparison proceeds in a decomposed fashion (i.e., separate comparisons of decades and units), but these digit comparisons were performed in parallel rather than one digit at a time.

Along a similar line of research, Huber, Mann, Nuerk, and Moeller (2014) studied two-digit number processing through the lens of the unit decade compatibility effect, a classic signature of decomposed processing (Nuerk, Weger, & Willmes, 2001). To explain this effect, consider the two-digit numbers 28 and 43. If one were to compare these numbers in a decomposed fashion (that is, separately compare the unit/decade digit of one number to the unit/decade digit of the other), one might note that 2 is less than 4, but 8 is greater than 4. In other words, the decomposed comparisons are in opposite directions; Nuerk et al. (2001) called such number pairs incompatible. On the other hand, we define pairs for which the decomposed comparisons are in the same direction (e.g., 23 versus 48, where 2 is less than 4 and 3 is less than 8) as compatible pairs. The unit-decade compatibility effect occurs when responses to incompatible pairs are slower and more error-prone than compatible pairs. Huber, Mann, et al. (2014) used eye tracking to show that this effect depends on factors related to experimental design. In their study, Huber, Mann, et al. manipulated the proportion of same-decade filler trials. Remarkably, they found that participants made more fixations on units in target trials (which can be answered by focusing only on decades) as the proportion of same-decade filler trials increased. Thus, eye tracking can reveal a substantial role for top-down influences on two digit number representation.

Several researchers have also used eye tracking to study fractions, for which questions similar to those pursued for two-digit numbers can be asked (e.g., Faulkenberry & Pierce, 2011). One of the first studies of this type was conducted by Huber, Moeller, and Nuerk (2014), who showed that participants can adaptively adjust their eye fixation patterns depending on experimental context. Participants’ eye movements indicated decomposed processing (i.e., focusing on direct comparison of numerators or denominators) on trials for which those components held decision-relevant information. For example, in a same-numerator pair such as 2/3 versus 2/5, the fraction comparison can be made by focusing solely on the magnitude of the denominator components. However, when the experimental context did not allow participants to make strong predictions about the relevance of components to the overall fraction comparison, the eye movement patterns were adjusted accordingly. In all, Huber, Moeller, and Nuerk (2014) demonstrated a pattern of top-down influence on fraction representation similar to that revealed earlier in the context of two-digit numbers (Huber, Mann, et al., 2014).

Other recent studies have used eye movements to infer participants’ strategies used when comparing fractions. For example, Obersteiner and Tumpek (2015) showed that even with complex fraction stimuli (e.g., two-digit components), participants tended to prefer decomposed strategies when fractions had common numerators or denominators. On the other hand, participants preferred holistic strategies for fraction pairs that did not contain common components. Interestingly, Ischebeck, Weilharter, and Körner (2016) showed that eye movements may serve to define the processing strategies used in fraction comparison. Participants’ eye movements exhibited scanning patterns that served to identify the type of fraction being compared, followed by the deployment of a specific comparison strategy adapted to the identified fraction pair type. Such results indicate that eye movements not only reflect the most fundamental aspects of numerical cognition, but also serve to shape the processing strategies used when making numerical decisions.

In a similar vein, the analysis of eye movements also revealed different strategies used in a number line task (Schneider et al., 2008). In this task, participants are asked to place a given number onto its correct spatial position on a horizontal line (Siegler & Opfer, 2003). Young children tend to start from the endpoint of the line and count upward (or downward) in whole units until they reach the target position. This strategy is reflected by a linear distribution of fixations along the line (from one end to the final position). On the other hand, older children start to count from the midpoint when the target position is closer to the midpoint than to one of the endpoints of the line. An increased use of such a midpoint strategy is associated with greater arithmetic competence (Schneider et al., 2008). Finally, adults use a fully proportion-based strategy: fixations are distributed along proportional reference points (e.g., endpoint, midpoint, points between the endpoint and midpoint; Sullivan, Juhasz, Slattery, & Barth, 2011). Thus, eye tracking reveals task strategies that are related to specific developmental stages of numerical concepts. This is particularly helpful when studying populations with impairments in numerical cognition. For example, van’t Noordende, van Hoogmoed, Schot, and Kroesbergen (2016) found that dyscalculic children attend to different features of the number line: specifically, they make less efficient use of reference points and are less capable of adapting their strategy when compared to age-matched controls (see also van Viersen, Slot, Kroesbergen, van’t Noordende, & Leseman, 2013).

Mental Arithmetic Processes [TOP]

Compared to single digit and multidigit number representation, the use of eye tracking to investigate mental arithmetic processes has received less attention (see Mock et al., 2016, for a systematic review). Among others, eye tracking has been used to further study spatial-numerical associations during mental arithmetic. Particularly, it has been hypothesized that mental addition is conceptualized as rightward movement along a mental number line, whereas mental subtraction is conceptualized as leftward movement (McCrink, Dehaene, & Dehaene-Lambertz, 2007). In line with this hypothesis, Hartmann, Mast, and Fischer (2016) found that spontaneous eye fixatons on a blank screen were located more rightward (and upward) when mentally counting upwards than downwards. Similarly, when solving verbally presented arithmetic problems, Hartmann, Mast, and Fischer (2015) found that participants exhibited more covert upward eye movements when solving addition problems compared to subtraction problems, and interestingly, the degree of horizontal movement was partially influenced by the magnitude of the problem operands (for a similar approach see also Yu et al., 2016; Zhu, Luo, You, & Wang, 2018).

Other researchers have used eye tracking to provide a more sensitive measure of strategy use in mental arithmetic (e.g., Green, Lemaire, & Dufau, 2007; Hartmann et al., 2018; Susac, Bubic, Kaponja, Planinic, & Palmovic, 2014). For example, Susac et al. (2014) recorded participants’ eye movements as they rearranged algebraic equations and found a correlation between the number of fixations and the efficiency of solution. Susac et al. also found that eye movement data provided a more accurate measure of solution strategy than did explicit verbal self-reports. In a similar vein, Hartmann et al. (2018) found that superior arithmetic performance in simple addition and subtraction tasks was characterized by fewer revisits to the first operand while computing the solution, probably due to better task representation in working memory. Further, Curtis, Huebner, and LeFevre (2016) measured eye movements as a function of problem size and operation in simple arithmetic problems. Curtis et al. found that for addition and multiplication problems (as well as small subtraction problems), participants made more fixations on the operator. Conversely, for large subtraction problems and division problems, participants fixated more on the operands. Furthering this line of inquiry, Huebner and LeFevre (2017) focused on mental subtraction and combined analysis of eye fixation patterns with distributional analyses of response times to conclude that eye movements can be used to distinguish fluent solvers (who use primarily memory retrieval) from nonfluent solvers (who tend to use procedures).

Hand Movement Tracking in Numerical Cognition [TOP]

Compared to eye tracking methods, the use of hand movement measures in cognitive research has been a much more recent phenomenon. Consequently, the collection of work in this area is much less developed. Much of this research paradigm owes its origin to the groundbreaking work of Spivey, Grosjean, and Knoblich (2005), who used computer mouse tracking to index the dynamics of cognitive processing during language comprehension. In their study, participants were instructed to click on one of two response labels after hearing an auditory word presentation. In the case where the two response labels represented phonological competitors (e.g., candle versus candy), participants’ computer mouse movements were reliably deflected toward the location of the incorrect competitor on the computer screen. Spivey et al. interpreted this result as evidence for dynamic competition between parallel and partially active responses during the decision process. Methdologically, the work of Spivey et al. (2005) gave rise to the notion that online tracking of hand movements during experimental tasks can provide an observable behavior that directly indexes the underlying cognitive processes of interest in the task.

Though the work of Spivey et al. (2005) set the stage for using hand movements to index cognitive processes, it was arguably the work of Freeman and colleagues (e.g., Freeman & Ambady, 2009; Freeman, Dale, & Farmer, 2011) that popularized the method. Importantly, Freeman and Ambady (2010) introduced the free software package MouseTracker (freely downloadable from http://www.mousetracker.org), which provided an easy-to-use interface for programming and delivering mouse tracking experiments. As a result, the mouse tracking method has been made accessible to a greater number of researchers to investigate problems in a wide variety of contexts. Since the release of this software, additional solutions for hand/mouse tracking have been released, including software solutions such as Mousetrap (Kieslich & Henninger, 2017) and increasingly sophisticated analysis methods (Calcagnì, Lombardi, & Sulpizio, 2017; Hehman, Stolier, & Freeman, 2015; Zgonnikov, Aleni, Piiroinen, O’Hora, & di Bernardo, 2017)

In the field of numerical and mathematical cognition, the types of problems investigated with hand and/or mouse tracking has largely mirrored that of eye tracking. Indeed, the recent literature can be broadly classified as we did above with our review of the eye tracking literature. To this end, we will describe some of the recent papers that have used hand tracking to investigate numerical and mathematical processes.

Single Digit Number Processes [TOP]

Some of the first studies to use the hand tracking methodology in the context of numerical cognition were aimed at investigating phenomena with single digit number stimuli. The first of these was Song and Nakayama (2008), who recorded participants’ hand movements as they manually touched either the left or right side of a computer screen, depending on whether the presented single-digit stimulus was less than or greater than 5, respectively. Song and Nakayama found that the recorded hand trajectories became more curved toward the center of the screen as the target number approached the comparison standard 5. They explained this increasing curvature through the lens of a direct mapping account, positing a direct correspondence between the position of a manual response and the externally projected position of a number on a mental number line.

This led to a follow-up study by Santens et al. (2011), who argued that since Song and Nakayama (2008) used only one response rule throughout their experiment (less than 5, touch left; greater than 5, touch right), the results of Song and Nakayama could just as easily be accounted for by a competition-based model of small number representation (Gevers, Verguts, Reynvoet, Caessens, & Fias, 2006; Verguts, Fias, & Stevens, 2005). Such an account would require no assumption of a spatial relationship between manual responses and a mental number line. To test between these two accounts, Santens et al. (2011) used finger tracking on a touchpad with two response rules, and were thus able to manipulate whether the response rules were congruent or incongruent with the typical “smaller-to-larger” arrangement of a mental number line, thus removing a possible confound between number and direction. Since the two accounts made opposite predictions regarding the patterns of curvatures in the number-line incongruent response mapping, Santens et al. (2011) were able to construct a strong test of the direct mapping account proposed by Song and Nakayama (2008). Their results were opposite what would be predicted by direct mapping, and instead provided support for a competition account of small number decisions. Faulkenberry (2016) further extended the results of Santens et al. (2011) by using distributional analyses of computer mouse trajectories to show that the patterns of hand movements observed in both Song and Nakayama (2008) and Santens et al. (2011) were indeed the result of graded competition effects, and not the result of some averaging artifact (see also Freeman et al., 2011).

Another context in which computer mouse tracking has been used to arbitrate between competing theories of numerical representation is the size congruity effect (Henik & Tzelgov, 1982), which is the finding that people are slower and more error prone to choose the physically larger of a digit pair when the numerical magnitude is incongruent with the physical magnitude (e.g., a large 2 presented with a small 8). Faulkenberry et al. (2016) recorded computer mouse trajectories in a physical comparison task. Across three experiments, Faulkenberry et al. found that the interference between numerical and physical magnitude was present in the response trajectories, but not in measures of response initiation. Such results indicated that the competition between physical and numerical magnitude persisted into the motor response, and consequently, was not isolated to the encoding stage. Faulkenberry et al. (2016) argued that these results supported a late interaction account of the size congruity effect.

In all, these studies demonstrate one of the primary advantages of the hand tracking paradigm. Specifically, recorded hand trajectories can provide rich data to arbitrate between competing theories of numerical representation. In addition to these studies, several other recent experiments have used computer mouse tracking to investigate the nature of basic number processing, including investigations of the dynamics of the SNARC effect (Faulkenberry, 2014), number line bisection (Haslbeck, Wood, & Witte, 2016), and distance and end effects in number comparison (Ganor-Stern & Goldman, 2015).

Multidigit Number Processes [TOP]

Similar efforts with hand tracking have been employed to attack problems in multidigit number processing. Dotan and Dehaene (2013) presented some early data on this problem by recording participants’ finger movements while locating two-digit numbers on a presented number line. Dotan and Dehaene used regression analyses to trace the dynamics of predictors for these finger locations, and found that participants first form a quick representation of the unit digit magnitude, followed by both holistic and decomposed representations of the two-digit number. Their results indicated that both holistic and decomposed representations may play a role in two-digit number representation (supporting the hybrid model of Nuerk et al., 2001).

Similarly, Bloechle, Huber, and Moeller (2015) tracked finger movements in a two-digit number comparison task. Bloechle et al. (2015) had participants point to the larger of a pair of two-digit numbers on a touch screen. They found that the final landing point of the finger on the target number was significantly left of the midline between the decade and unit digit, a finding they termed “decade bias”. Further, this decade bias was reduced on unit-decade incompatible trials, possibly because on these trials the magnitude information from the unit digit became more salient. They argued that this behavior reflected “an embodied representation of the place-value structure of two-digit numbers” (Bloechle et al., 2015, p. 480). Notably, the experiment of Bloechle et al. (2015) did not use trajectories, but rather the ending points of the manual responses. Faulkenberry, Cruise, and Shaki (2017) responded to this claim by showing that trajectory patterns which might indicate decade bias were reversed when response labels were also reversed. Similar to Faulkenberry (2016), they claimed that this trajectory data showed support for a competition model of two-digit number comparison rather than a purely embodied representation of place value.

As with eye tracking, hand tracking has also been used to investigate the dynamics of fraction representations. Faulkenberry, Montgomery, and Tennes (2015) recorded participants’ computer mouse movements while they made decisions about whether presented fractions were greater or less than 1/2. Faulkenberry et al. (2015) found that trajectories showed competition effects from both component magnitude and holistic magnitude, but the competition from component magnitude happened much earlier in the decision process. Faulkenberry et al. (2015) interpreted these results in terms of a hybrid model of fraction representation, where both decomposed and holistic magnitudes are processed together in a continuous, competitive fashion.

Mental Arithmetic Processes [TOP]

The literature employing hand tracking to understand mental arithmetic processes is considerably less developed compared to numerical representations. However, there are a few important first strides in this area. The first such study to examine manual dynamics of mental arithmetic was Marghetis et al. (2014), who measured participants computer mouse movements as they clicked the answer to presented single-digit arithmetic problems. Marghetis and colleagues found that the recorded hand trajectories showed significant rightward deflection for addition problems and leftward deflection for subtraction problems, consistent with the predictions of theories which conceptualize arithmetic as being supported by a dynamic, spatial processing component (Mathieu, Gourjon, Couderc, Thevenot, & Prado, 2016).

This conclusion is also supported by the recent work of Pinheiro-Chagas et al. (2017). In their study, Pinheiro-Chagas and colleagues tracked participants’ finger movements as they pointed to the results of single-digit addition and subtraction problems on a number line. Similar to Marghetis et al. (2014), they found that participants’ finger movements showed rightward attraction on addition problems and leftward attraction on subtraction problems. Further, these spatial attractions were predicted by magnitude properties of the problem operands. Pinheiro-Chagas et al. (2017) interpreted these results as support for a fast procedural view of mental arithmetic, where addition and subtraction are solved via simulated motion on a mental number line. Studies such as these are important because they can potentially shed new light on some classic debates in mathematical cognition, such as the “procedures versus memory retrieval” debate in mental arithmetic (e.g., Ashcraft, 1985; Baroody, 1985).

The Present Collection [TOP]

The papers that we have collected for this special section of Journal of Numerical Cognition represent an important continuation the above-mentioned work on tracking the continuous dynamics of numerical representation. Across these five articles, the reader will see a variety of approaches, ranging from computer mouse tracking to eye tracking and visual search methods.

Two of the papers used some version of hand tracking to investigate number processing. The article by Erb, Moher, Song, and Sobel (2018, this section) addresses one of the current shortcomings of hand tracking research by extending this methodology to a developmental focus. In their experiment, Erb and colleagues adapt the original hand tracking study of Song and Nakayama (2008) to a sample of 5- and 6-year-old children and track hand trajectories as these children respond to centrally presented number stimuli by pressing a left-side rectangle for small numbers and a right-side rectangle for large numbers. In addition to replicating the original finding of Song and Nakayama (2008), where reach trajectories became more curved as the numerical distance to the comparison standard decreased, Erb et al. (2018) also showed that the observed numerical distance effects were more pronounced in measures that reflected response execution than in measures reflecting response initiation. These results open up a potentially exciting avenue of research on the processing locus of the numerical distance effect.

The second paper that used hand tracking to investigate numerical processing was offered by Ursula Fischer, Martin H. Fischer, Huber, Strauss, and Moeller (2018, this section). In their paper, Ursula Fischer and colleagues developed a novel method whereby they asked participants to respond to parity judgment and magnitude comparison tasks via a continuous swiping movement. That is, they asked participants to perform actions similar to those that have recently become commonplace with smartphones and tablets, where screen navigation is initiated by touching an object and swiping it rightward or leftward. In addition to reporting some interesting data with regard to the SNARC effect and the MARC effect (the finding that right hand responses are faster for even numbers, whereas left hand responses are faster for odd numbers; Nuerk, Iversen, & Willmes, 2004), Ursula Fischer et al. (2018) found that the magnitude of the finger swipe increased with numerical magnitude. As the first such report of this finding, this work is likely to be influential for a host of future efforts to better understand this spatial-numerical phenomenon.

Along with these papers on hand tracking, we also collected two papers that used eye tracking to study the dynamics of number processing. As described earlier, decisions with fractions pose special cognitive processing demands as a symbolic fraction represents a (rather abstract) relative magnitude. People often struggle with interference between holistic and decomposed strategies which treat the numerator and denominator either as a single whole number or as separate quantities. Obersteiner and Staudinger (2018, this section) assessed the eye movement patterns of adults during a mental arithmetic task with fractions. Extending the dominant literature which focuses solely on fraction comparison tasks, Obersteiner and Staudinger used fraction addition problems of varying complexity. Eye tracking data revealed that task difficulty had a direct impact on the strategies chosen. Participants mostly exhibited component-based saccades, either between numerators for same-denominator tasks or between denominators for all other fraction addition tasks. Holistic saccades were also common, and further research is needed to clarify whether these eye movements, which were not necessary to solve the task, may represent double-checking behavior or reading. Similar to some of the past work on eye tracking with fractions, the current study by Obersteiner and Staudinger (2018) shows that adults adjust their eye movements in a way focusing on the most relevant components of a fraction. Additionally, we think that the work of Obersteiner and Staudinger might be highly relevant to helping students who struggle with fraction arithmetic, providing a tool to assess and compare individual skills of mathematical learning in greater detail.

The second paper of our collection that uses eye tracking reports on research by van der Weijden, Kamphorst, Willemsen, Kroesbergen, and van Hoogmoed (2018, this section), who used a combination of eye tracking along with cued retrospective reports to provide a qualitative, exploratory description of unbounded number line performance in both typical adults as well as those with dyscalculia. Perhaps surprisingly, van der Weijden and colleagues found little difference between typical adults and those with dyscalculia, indicating that the previously observed performance deficits in individuals with dyscalculia may stem from an impaired system of fast number knowledge (e.g., Ashkenazi, Rubinsten, & Henik, 2009), rather than an impaired number sense. Further, van der Weijden et al. (2018) demonstrate a newly observed strategy in unbounded number line tasks (one they term “use of previous number”).

Finally, the paper by Lee, Sobel, York, and Puri (2018, this section) provides a different take on a fundamental problem concerning the dynamics of numerical representation. Specifically, Lee et al. (2018) employed a visual search task in order to further explore the temporal dynamics of the mechanisms underlying how adults extract numerical meaning from digits. More specifically, they examined response times required for estimating the average of digit arrays of varying sizes in order to determine whether numerical meaning is extracted in a serial or parallel fashion. Their results suggest that semantic information from multiple digits may be extracted by a parallel processing mechanism, thus contributing to the debate about the integration of perceptual and semantic information (see also Sobel & Puri, 2018; Sobel, Puri, & Faulkenberry, 2016; Sobel, Puri, Faulkenberry, & Dague, 2017).

Conclusions [TOP]

The papers collected in this special section add to an ever-growing literature focused on tracking the continuous dynamics of numerical representation. As this field of inquiry is still relatively young, there is plenty of room for researchers to apply these methods in a wide variety of empirical contexts, ranging from description to theory testing. The next decade will be an exciting time indeed, as the techniques and tools for measuring continuous dynamics in numerical representation will mature and become more widely available as time progresses. We think the present collection of papers will form a core set of work that will be widely cited in many of these future studies. We further hope that the ideas presented in these papers will generate a wide variety of new questions and techniques for future research.

Funding [TOP]

M. H. was funded by the Swiss National Science Foundation [grant number PZ00P1_167995].

Competing Interests [TOP]

The authors have declared that no competing interests exist.

Acknowledgments [TOP]

We thank the Editor in Chief and the Editorial Board for their support in the development of this Special Section of the Journal of Numerical Cognition. We appreciate the efforts of the authors and reviewers and thank them for their positive contributions to this work.

References [TOP]

  • Ashcraft, M. H. (1985). Is it farfetched that some of us remember our arithmetic facts? Journal for Research in Mathematics Education, 16(2), 99-105. doi:10.2307/748367

  • Ashkenazi, S., Rubinsten, O., & Henik, A. (2009). Attention, automaticity, and developmental dyscalculia. Neuropsychology, 23(4), 535-540. doi:10.1037/a0015347

  • Baroody, A. J. (1985). Mastery of basic number combinations: Internalization of relationships or facts? Journal for Research in Mathematics Education, 16(2), 83-98. doi:10.2307/748366

  • Bloechle, J., Huber, S., & Moeller, K. (2015). In touch with numbers: Embodied and situated effects in number magnitude comparison. Journal of Cognitive Psychology, 27(4), 478-489. doi:10.1080/20445911.2014.1001760

  • Brysbaert, M. (1995). Arabic number reading: On the nature of the numerical scale and the origin of phonological recoding. Journal of Experimental Psychology: General, 124(4), 434-452. doi:10.1037/0096-3445.124.4.434

  • Calcagnì, A., Lombardi, L., & Sulpizio, S. (2017). Analyzing spatial data from mouse tracker methodology: An entropic approach. Behavior Research Methods, 49(6), 2012-2030. doi:10.3758/s13428-016-0839-5

  • Cohen Kadosh, R., & Dowker, A. (2015). The Oxford handbook of numerical cognition. Oxford, United Kingdom: Oxford University Press.

  • Curtis, E. T., Huebner, M. G., & LeFevre, J.-A. (2016). The relationship between problem size and fixation patterns during addition, subtraction, multiplication, and division. Journal of Numerical Cognition, 2(2), 91-115. doi:10.5964/jnc.v2i2.17

  • Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122(3), 371-396. doi:10.1037/0096-3445.122.3.371

  • Dotan, D., & Dehaene, S. (2013). How do we convert a number into a finger trajectory? Cognition, 129(3), 512-529. doi:10.1016/j.cognition.2013.07.007

  • Erb, C. D., Moher, J., Song, J.-H., & Sobel, D. M. (2018). Numerical cognition in action: Reaching behavior reveals numerical distance effects in 5- to 6-year-olds. Journal of Numerical Cognition, 4, 286-296. doi:10.5964/jnc.v4i2.122

  • Faulkenberry, T. J. (2014). Hand movements reflect competitive processing in numerical cognition. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimentale, 68(3), 147-151. doi:10.1037/cep0000021

  • Faulkenberry, T. J. (2016). Testing a direct mapping versus competition account of response dynamics in number comparison. Journal of Cognitive Psychology, 28(7), 825-842. doi:10.1080/20445911.2016.1191504

  • Faulkenberry, T. J., Cruise, A., Lavro, D., & Shaki, S. (2016). Response trajectories capture the continuous dynamics of the size congruity effect. Acta Psychologica, 163, 114-123. doi:10.1016/j.actpsy.2015.11.010

  • Faulkenberry, T. J., Cruise, A., & Shaki, S. (2017). Reversing the manual digit bias in two-digit number comparison. Experimental Psychology, 64(3), 191-204. doi:10.1027/1618-3169/a000365

  • Faulkenberry, T. J., Montgomery, S. A., & Tennes, S.-A. N. (2015). Response trajectories reveal the temporal dynamics of fraction representations. Acta Psychologica, 159, 100-107. doi:10.1016/j.actpsy.2015.05.013

  • Faulkenberry, T. J., & Pierce, B. H. (2011). Mental representations in fraction comparison. Experimental Psychology, 58(6), 480-489. doi:10.1027/1618-3169/a000116

  • Faulkenberry, T. J., & Rey, A. E. (2014). Extending the reach of mousetracking in numerical cognition: A comment on Fischer and Hartmann (2014). Frontiers in Psychology, 5, Article 1436. doi:10.3389/fpsyg.2014.01436

  • Fischer, M. H. [Martin H.], & Hartmann, M. (2014). Pushing forward in embodied cognition: May we mouse the mathematical mind? Frontiers in Psychology, 5, Article 1315. doi:10.3389/fpsyg.2014.01315

  • Fischer, U. [Ursula], Fischer, M. H. [Martin H.], Huber, S., Strauß, S., & Moeller, K. (2018). The influence of number magnitude on continuous swiping movements. Journal of Numerical Cognition, 4, 297-316. doi:10.5964/jnc.v4i2.135

  • Freeman, J. B., & Ambady, N. (2009). Motions of the hand expose the partial and parallel activation of stereotypes. Psychological Science, 20(10), 1183-1188. doi:10.1111/j.1467-9280.2009.02422.x

  • Freeman, J. B., & Ambady, N. (2010). MouseTracker: Software for studying real-time mental processing using a computer mouse-tracking method. Behavior Research Methods, 42(1), 226-241. doi:10.3758/BRM.42.1.226

  • Freeman, J. B., Dale, R., & Farmer, T. A. (2011). Hand in motion reveals mind in motion. Frontiers in Psychology, 2, Article 59. doi:10.3389/fpsyg.2011.00059

  • Ganor-Stern, D., & Goldman, R. (2015). Reaching towards an end: Numerical end and distance effects in motor movements. Journal of Cognitive Psychology, 27(4), 490-498. doi:10.1080/20445911.2014.950662

  • Gevers, W., Verguts, T., Reynvoet, B., Caessens, B., & Fias, W. (2006). Numbers and space: A computational model of the SNARC effect. Journal of Experimental Psychology: Human Perception and Performance, 32(1), 32-44. doi:10.1037/0096-1523.32.1.32

  • Green, H. J., Lemaire, P., & Dufau, S. (2007). Eye movement correlates of younger and older adults’ strategies for complex addition. Acta Psychologica, 125(3), 257-278. doi:10.1016/j.actpsy.2006.08.001

  • Groen, G. J., & Parkman, J. M. (1972). A chronometric analysis of simple addition. Psychological Review, 79(4), 329-343. doi:10.1037/h0032950

  • Hartmann, M. (2015). Numbers in the eye of the beholder: What do eye movements reveal about numerical cognition? Cognitive Processing, 16, (S1), 245-248. doi:10.1007/s10339-015-0716-7

  • Hartmann, M., & Fischer, M. H. (2016). Exploring the numerical mind by eye-tracking: A special issue. Psychological Research, 80(3), 325-333. doi:10.1007/s00426-016-0759-0

  • Hartmann, M., Laubrock, J., & Fischer, M. H. (2018). The visual number world: A dynamic approach to study the mathematical mind. Quarterly Journal of Experimental Psychology, 71(1), 28-36. doi:10.1080/17470218.2016.1240812

  • Hartmann, M., Mast, F. W., & Fischer, M. H. (2015). Spatial biases during mental arithmetic: Evidence from eye movements on a blank screen. Frontiers in Psychology, 6, Article 12. doi:10.3389/fpsyg.2015.00012

  • Hartmann, M., Mast, F. W., & Fischer, M. H. (2016). Counting is a spatial process: Evidence from eye movements. Psychological Research, 80(3), 399-409. doi:10.1007/s00426-015-0722-5

  • Haslbeck, J. M. B., Wood, G., & Witte, M. (2016). Temporal dynamics of number-space interaction in line bisection: Comment on Cleland and Bull (2015). Quarterly Journal of Experimental Psychology, 69(6), 1239-1242. doi:10.1080/17470218.2015.1095773

  • Hehman, E., Stolier, R. M., & Freeman, J. B. (2015). Advanced mouse-tracking analytic techniques for enhancing psychological science. Group Processes & Intergroup Relations, 18(3), 384-401. doi:10.1177/1368430214538325

  • Henik, A., & Tzelgov, J. (1982). Is three greater than five: The relation between physical and semantic size in comparison tasks. Memory & Cognition, 10(4), 389-395. doi:10.3758/BF03202431

  • Huber, S., Mann, A., Nuerk, H.-C., & Moeller, K. (2014). Cognitive control in number magnitude processing: Evidence from eye-tracking. Psychological Research, 78(4), 539-548. doi:10.1007/s00426-013-0504-x

  • Huber, S., Moeller, K., & Nuerk, H.-C. (2014). Adaptive processing of fractions — Evidence from eye-tracking. Acta Psychologica, 148, 37-48. doi:10.1016/j.actpsy.2013.12.010

  • Huebner, M. G., & LeFevre, J.-A. (2017). Selection of procedures in mental subtraction: Use of eye movements as a window on arithmetic processing. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimentale. Advance online publication. doi:10.1037/cep0000127

  • Ischebeck, A., Weilharter, M., & Körner, C. (2016). Eye movements reflect and shape strategies in fraction comparison. Quarterly Journal of Experimental Psychology, 69(4), 713-727. doi:10.1080/17470218.2015.1046464

  • Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329-354. doi:10.1037/0033-295X.87.4.329

  • Kieslich, P. J., & Henninger, F. (2017). Mousetrap: An integrated, open-source mouse-tracking package. Behavior Research Methods, 49(5), 1652-1667. doi:10.3758/s13428-017-0900-z

  • Lee, K. R., Sobel, K. V., York, A. K., & Puri, A. M. (2018). Dissociating parallel and serial processing of numerical value. Journal of Numerical Cognition, 4, 360-379. doi:10.5964/jnc.v4i2.133

  • Liversedge, S. P., & Findlay, J. M. (2000). Saccadic eye movements and cognition. Trends in Cognitive Sciences, 4(1), 6-14. doi:10.1016/S1364-6613(99)01418-7

  • Loetscher, T., Bockisch, C. J., Nicholls, M. E., & Brugger, P. (2010). Eye position predicts what number you have in mind. Current Biology, 20(6), R264-R265. doi:10.1016/j.cub.2010.01.015

  • Marghetis, T., Núñez, R., & Bergen, B. K. (2014). Doing arithmetic by hand: Hand movements during exact arithmetic reveal systematic, dynamic spatial processing. Quarterly Journal of Experimental Psychology, 67(8), 1579-1596. doi:10.1080/17470218.2014.897359

  • Mathieu, R., Gourjon, A., Couderc, A., Thevenot, C., & Prado, J. (2016). Running the number line: Rapid shifts of attention in single-digit arithmetic. Cognition, 146, 229-239. doi:10.1016/j.cognition.2015.10.002

  • McCrink, K., Dehaene, S., & Dehaene-Lambertz, G. (2007). Moving along the number line: Operational momentum in nonsymbolic arithmetic. Perception & Psychophysics, 69(8), 1324-1333. doi:10.3758/BF03192949

  • Mock, J., Huber, S., Klein, E., & Moeller, K. (2016). Insights into numerical cognition: Considering eye-fixations in number processing and arithmetic. Psychological Research, 80(3), 334-359. doi:10.1007/s00426-015-0739-9

  • Moeller, K., Fischer, M. H., Nuerk, H.-C., & Willmes, K. (2009). Sequential or parallel decomposed processing of two-digit numbers? Evidence from eye-tracking. Quarterly Journal of Experimental Psychology, 62(2), 323-334. doi:10.1080/17470210801946740

  • Moyer, R. S., & Landauer, T. K. (1967). Time required for judgements of numerical inequality. Nature, 215(5109), 1519-1520. doi:10.1038/2151519a0

  • Myachykov, A., Ellis, R., Cangelosi, A., & Fischer, M. H. (2016). Ocular drift along the mental number line. Psychological Research, 80(3), 379-388. doi:10.1007/s00426-015-0731-4

  • Nuerk, H.-C., Iversen, W., & Willmes, K. (2004). Notational modulation of the SNARC and the MARC (linguistic markedness of response codes) effect. The Quarterly Journal of Experimental Psychology, 57A(5), 835-863. doi:10.1080/02724980343000512

  • Nuerk, H.-C., Weger, U., & Willmes, K. (2001). Decade breaks in the mental number line? Putting the tens and units back in different bins. Cognition, 82(1), B25-B33. doi:10.1016/S0010-0277(01)00142-1

  • Obersteiner, A., & Staudinger, I. (2018). How the eyes add fractions: Adult eye movement patterns during fraction addition problems. Journal of Numerical Cognition, 4, 317-336. doi:10.5964/jnc.v4i2.130

  • Obersteiner, A., & Tumpek, C. (2015). Measuring fraction comparison strategies with eye-tracking. ZDM, 48(3), 255-266. doi:10.1007/s11858-015-0742-z

  • Pinheiro-Chagas, P., Dotan, D., Piazza, M., & Dehaene, S. (2017). Finger tracking reveals the covert stages of mental arithmetic. Open Mind, 1(1), 30-41. doi:10.1162/OPMI_a_00003

  • Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372-422. doi:10.1037/0033-2909.124.3.372

  • Rayner, K., & Pollatsek, A. (1989). The psychology of reading. Englewood Cliffs, NJ, USA: Prentice Hall.

  • Richardson, D. C., Dale, R., & Spivey, M. J. (2007). Eye movements in language and cognition. Methods in Cognitive Linguistics, 18, 323-344. doi:10.1075/hcp.18.21ric

  • Santens, S., Goossens, S., & Verguts, T. (2011). Distance in motion: Response trajectories reveal the dynamics of number comparison. PLoS One, 6(9), Article e25429. doi:10.1371/journal.pone.0025429

  • Schneider, M., Heine, A., Thaler, V., Torbeyns, J., De Smedt, B., Verschaffel, L., . . . Stern, E., (2008). A validation of eye movements as a measure of elementary school children’s developing number sense. Cognitive Development, 23(3), 409-422. doi:10.1016/j.cogdev.2008.07.002

  • Siegler, R. S., & Opfer, J. E. (2003). The development of numerical estimation: Evidence for multiple representations of numerical quantity. Psychological Science, 14(3), 237-243. doi:10.1111/1467-9280.02438

  • Sobel, K. V., & Puri, A. M. (2018). Size congruity influences visual search via the target template. Acta Psychologica, 183, 66-74. doi:10.1016/j.actpsy.2018.01.003

  • Sobel, K. V., Puri, A. M., & Faulkenberry, T. J. (2016). Bottom-up and top-down attentional contributions to the size congruity effect. Attention, Perception & Psychophysics, 78(5), 1324-1336. doi:10.3758/s13414-016-1098-3

  • Sobel, K. V., Puri, A. M., Faulkenberry, T. J., & Dague, T. D. (2017). Visual search for conjunctions of physical and numerical size shows that they are processed independently. Journal of Experimental Psychology: Human Perception and Performance, 43(3), 444-453. doi:10.1037/xhp0000323

  • Song, J.-H., & Nakayama, K. (2008). Numeric comparison in a visually-guided manual reaching task. Cognition, 106(2), 994-1003. doi:10.1016/j.cognition.2007.03.014

  • Spivey, M. J., Grosjean, M., & Knoblich, G. (2005). Continuous attraction toward phonological competitors. Proceedings of the National Academy of Sciences of the United States of America, 102(29), 10393-10398. doi:10.1073/pnas.0503903102

  • Sullivan, J. L., Juhasz, B. J., Slattery, T. J., & Barth, H. C. (2011). Adults’ number-line estimation strategies: Evidence from eye movements. Psychonomic Bulletin & Review, 18(3), 557-563. doi:10.3758/s13423-011-0081-1

  • Susac, A., Bubic, A., Kaponja, J., Planinic, M., & Palmovic, M. (2014). Eye movements reveal students’ strategies in simple equation solving. International Journal of Science and Mathematics Education, 12(3), 555-577. doi:10.1007/s10763-014-9514-4

  • van der Weijden, F. A., Kamphorst, E., Willemsen, R. H., Kroesbergen, E. H., & van Hoogmoed, A. H. (2018). Strategy use on bounded and unbounded number lines in typically developing adults and adults with dyscalculia: An eye-tracking study. Journal of Numerical Cognition, 4, 337-359. doi:10.5964/jnc.v4i2.115

  • van’t Noordende, J. E., van Hoogmoed, A. H., Schot, W. D., & Kroesbergen, E. H. (2016). Number line estimation strategies in children with mathematical learning difficulties measured by eye tracking. Psychological Research, 80(3), 368-378. doi:10.1007/s00426-015-0736-z

  • van Viersen, S., Slot, E. M., Kroesbergen, E. H., van’t Noordende, J. E., & Leseman, P. P. M. (2013). The added value of eye-tracking in diagnosing dyscalculia: A case study. Frontiers in Psychology, 4, Article 679. doi:10.3389/fpsyg.2013.00679

  • Verguts, T., Fias, W., & Stevens, M. (2005). A model of exact small-number representation. Psychonomic Bulletin & Review, 12(1), 66-80. doi:10.3758/BF03196349

  • Wade, N., & Tatler, B. (2005). The moving tablet of the eye: The origins of modern eye movement research. doi:10.1093/acprof:oso/9780198566175.001.0001

  • Yu, X., Liu, J., Li, D., Liu, H., Cui, J., & Zhou, X. (2016). Dynamic mental number line in simple arithmetic. Psychological Research, 80(3), 410-421. doi:10.1007/s00426-015-0730-5

  • Zgonnikov, A., Aleni, A., Piiroinen, P. T., O’Hora, D., & di Bernardo, M. (2017). Decision landscapes: Visualizing mouse-tracking data. Royal Society Open Science, 4(11), Article 170482. doi:10.1098/rsos.170482

  • Zhu, R., Luo, Y., You, X., & Wang, Z. (2018). Spatial bias induced by simple addition and subtraction: From eye movement evidence. Perception, 47(2), 143-157. doi:10.1177/0301006617738718

Copyright (c) 2018 Faulkenberry; Witte; Hartmann