Decoding Fact Fluency and Strategy Flexibility in Solving One-Step Algebra Problems: An Individual Differences Analysis

Prior Research

Previous research has revealed reasoners’ cognitive processes in arithmetic (Ashcraft, 1992), which may undergird their understanding of algebra, especially missing-operand problems. One method used to solve arithmetic problems is retrieval, in which the relevant arithmetic fact is directly retrieved from memory to answer the question (Ashcraft & Guillaume, 2009; LeFevre et al., 1996). Research indicates that people are faster to evaluate arithmetic expressions or to verify arithmetic sentences involving ‘smaller’ operands (e.g., to evaluate 2 + 3 or verify 2 + 3 = 5) than ‘larger’ operands (e.g., to evaluate 8 + 9 or verify 8 + 9 = 17). This ‘problem size effect’ (Parkman & Groen, 1971) has been demonstrated robustly and throughout mathematical development (Ashcraft & Guillaume, 2009). The problem size effect is a key characteristic of the representations involved in arithmetic problem solving.

Reliance on direct retrieval to solve arithmetic problems is a common strategy even for young children (Ashcraft & Fierman, 1982; Siegler & Shrager, 1984). However, it is not the only strategy. People also use non-retrieval strategies, including more effortful transformational strategies. There are various transformational strategies used by young children learning arithmetic (Carpenter et al., 1981; Hiebert, 1982), but transformation remains common among adults, especially for larger problems. For example, as addition problem sums exceed 10, young adults become increasingly likely to self-report that they decomposed one operand to create a more familiar arithmetic fact (e.g., 7 + 4 = 7 + (3 + 1) = (7 + 3) + 1 = 10 + 1 = 11; LeFevre et al., 1996). Non-retrieval strategies may be especially prominent for subtraction, for which reasoners increasingly shift from retrieval to non-retrieval for minuends 11 and greater, sometimes using an inverse strategy for subtraction expressions (e.g., 13 – 7 = _) and recruiting the corresponding addition facts (e.g., 6 + 7 = 13) (Seyler et al., 2003). This subtraction-by-addition strategy is more common when the operands are large (Campbell, 2008) and when the subtrahend is considerably smaller than the difference (Peters et al., 2010; Torbeyns et al., 2018). It is even demonstrated among children, sometimes flexibly (Hickendorff, 2020; Torbeyns et al., 2018). However, it is less clear whether inverse transformations play a role in missing-operand problems, especially for single-digit operands (i.e., below 10), and how this may relate to individual differences in arithmetic fluency.

Earlier research has shown that young children struggle with open number sentences (e.g., [ ] – 3 = 5; Groen & Poll, 1973), especially when the unknown is the first operand or when represented as word problems (Briars & Larkin, 1984; Hiebert, 1982). Children’s performance on such word problems improves after instruction mapping their informal strategies onto the mathematical symbols of open number sentences (Carpenter et al., 1988). Across these and other studies, children vary in their strategy choice, which can relate to achievement (Siegler, 1988) and working memory capacity (Seyler et al., 2003).

Finally, research has demonstrated that when individuals switch between strategies across problems, they may incur a ‘switch cost’ (Lemaire & Lecacheur, 2010). Despite this cost, most participants in the above studies were willing to use different strategies depending on problem characteristics. This suggests that people may also perceive a strategy ‘switch benefit’ for some problems which outweighs the switch cost enough to compel them to switch strategies for those problems. If so, sensitivity to this strategy switch benefit may be a key characteristic of flexibility in algebraic problem solving and mathematical achievement more generally. We examine this ‘switch benefit’ in missing-operand algebra problems and whether individuals vary in sensitivity to it.

The Current Study

These findings provide an arithmetic basis on which to investigate strategy flexibility in simple algebraic problem solving. Given the importance of simple algebra problems and the foundational nature of arithmetic to algebra, we sought to assess whether individual participants’ arithmetic and algebraic problem-solving patterns are related for single-step missing-operand problems, and if so, whether an individual’s unique patterns could be used to ‘decode’ their strategies for missing-operand problems. The current study does this through addressing five research questions.

The first and second research questions concern whether arithmetic and algebraic problem solving utilize common mental representations and processes. After ensuring replication of the problem size effect on the arithmetic verification task, the first research question extends the problem size effect to one-step algebra problems: Is there a problem size effect on the missing-operand task? The second research question connects arithmetic and algebraic problem solving through an individual differences analysis: Is an individual’s problem size effect on the arithmetic verification task positively correlated with their problem size effect on the missing-operand task? Given the centrality of the problem size effect to the representations involved in arithmetic thinking, extending the problem size effect to one-step algebra problems and finding correlations between the effect in arithmetic and algebra tasks would suggest that participants may repurpose arithmetic facts for algebraic problem solving.

The third research question addresses the arithmetic-algebra relation at the problem level to measure individuals’ flexibility in algebraic problem solving. Recall that there are two primary ways to solve missing-operand problems: by retrieving the direct fact or by transforming and retrieving the inverse fact. We predict that people will differ in strategy use as a function of their fluency for the direct fact and their ability to flexibly switch to the inverse retrieval when it is beneficial. To assess individual variation in flexibility, we formalize a switch benefit (alternatively, a stay cost) for missing-operand problems as the potential gain in switching from the direct fact – given its disfluency – to the inverse fact (or alternatively, the potential penalty of staying with the direct fact given its disfluency). We operationalize this at the participant level: On missing-operand problems (e.g., x – 7 = 2), is a given individual more likely to switch to the inverse strategy if their time to retrieve the direct arithmetic fact (9 – 7 = 2) is longer than their time to retrieve the inverse arithmetic fact (2 + 7 = 9) — that is, if direct retrieval of the corresponding arithmetic fact is relatively disfluent for that individual on that problem? An individual’s switch benefit for a problem may predict which strategy the individual uses to solve the problem. We propose a participant-level regression model in the analysis to address this question.

The fourth research question concerns the validity of this switch benefit operationalization. We assessed this question by comparing individuals’ ‘switch benefit’ estimates from the regression model to their explicit strategy self-reports collected at the end of the study.

The fifth research question concerns the relation between strategy selection, flexibility, and mathematical achievement. If flexibility is important for mathematical thinking more generally, then is it the case that sensitivity to switch benefit when solving missing-operand problems is associated with higher mathematical achievement more generally (as measured by standardized ACT and SAT scores)? We predicted that sensitivity to switch benefit would be positively associated with mathematical achievement.

To address these research questions, we created an experimental paradigm to ‘decode’ participants’ strategy choice flexibility from their response times (RTs). In this paradigm, participants make speeded judgements in the missing-operand task of interest (e.g., x – 7 = 2) and also in an arithmetic verification task where they verify the corresponding direct and inverse arithmetic facts (e.g., 9 – 7 = 2 and 2 + 7 = 9). We addressed the first and second research questions by analyzing participant RTs on these tasks as a function of problem size and also operation (addition or subtraction). We addressed the third research question by using regression to estimate how well each participant’s missing-operand RTs are predicted by direct fact fluency and/or the benefit of switching to an inverse strategy. This switch benefit is operationalized for each participant using their own RTs on the corresponding arithmetic fact verifications. We addressed the fourth research question by correlating participants’ switch benefit terms from the regression model with their self-reported use of the inverse strategy. We addressed the fifth research question by using regression to evaluate whether an individual’s sensitivity to switch benefit was associated with their own standardized mathematical achievement score.

Participants and Design

We recruited an initial sample of 62 of an intended 80 undergraduate students at a university in the Midwest U.S., before the onset of COVID-19 brought an end to face-to-face data collection. Because the experimental paradigm is new, we could not rely on the literature to estimate the number of participants and trials. We were instead guided by a pilot study in our lab of 64 participants experiencing 224 trials across the two tasks, which detected notable individual variation between-participants. Although we were unable to increase the sample size of the current study for external reasons beyond our control, we almost doubled the number of trials across the two tasks, to 432. (This is the number that they could complete in approximately 40 minutes before beginning to experience fatigue.) Given that most of our analyses are within-participants, we believe our increased number of trials provided adequate power. Two participants were excluded for having also participated in the pilot, and one participant was excluded because their accuracy was considerably lower than 90% on the tasks, leaving 59 participants.

Participant ages ranged from 18 to 23 years old (M = 20.04, SD = 1.28); 42 identified as women and 17 as men; and 45 identified their race as white, 12 as Asian, 1 as Black, and 1 as multiracial. Recruitment and study procedures were approved by the local IRB. Each participant received $12 USD compensation for the study, which lasted approximately 40 minutes.

Materials

The experiment was implemented using PsychoPy (Peirce et al., 2019). Participants were instructed in both tasks to respond to each trial as quickly and accurately as possible. Response time and accuracy were recorded. The experiment script and all stimuli files are provided as Supplementary Materials.

Procedure

For each task, participants first completed eight practice trials with feedback. The experimental trials that followed were divided into two blocks with a break halfway. The order of the two tasks was counterbalanced across participants. After finishing both tasks, participants completed a strategy questionnaire about how they solved the missing-operand problems and what percentage of the time they utilized each strategy type (i.e., direct vs. inverse). Lastly, they completed a demographics form.

Problem Size Effects

Before answering our first research question, we assessed whether we replicated the problem size effect for the true trials in the arithmetic verification task. Each participant’s mean RT for each problem size and operation type (addition and subtraction) are visualized in Figure 1, along with the overall means. For each participant and each operation type (addition and subtraction), we computed the Kendall’s rank correlation ( $\tau$) between their RT and the problem size, operationalized as the sum of the operands.³ For the addition problems, all participants had descriptively positive correlation coefficients, with individual $\tau$s ranging from .05 to .55 (Median = .29). The individual $\tau$ coefficients were significantly different from zero for 48 of the 59 participants. A one-sample t-test found that overall, participants’ correlations for addition problems were different from zero, t(58) = 18.8, p < .001. For the subtraction problems, 57 of the 59 participants had descriptively positive correlation coefficients, with individual $\tau$s ranging from -.005 to .42 (Median = .19), of which 32 were significantly different from zero. Participants’ correlations for subtraction problems were different from zero, t(58) = 14.5, p < .001. Thus, the problem size effect was replicated across both operations at the group level and also at the individual level for the majority of participants.

The first research question was whether the problem size effect extended to the missing-operand problems (Figure 2). We operationalized the problem size as the sum of the problem’s operands, one unknown and the other known. We computed the Kendall’s rank correlation ( $\tau$) between RT and problem size for each participant and operation. For the addition problems, most participants had descriptively positive correlation coefficients (individual $\tau$s ranging from -.11 to .36, median of .20), and the individual $\tau$ coefficients were statistically different from zero for 36 of the 59 participants. A one-sample t-test found evidence that overall, participants’ correlations for addition missing-operand problems were different from zero, t(58) = 11.6, p < .001. A Bayesian one-sample t-test (Faulkenberry et al., 2020) indicated that these correlations were more likely under a problem size effect model than the null model (BF₁₀ = 5.5 × 10¹³). For the subtraction problems, participants were almost evenly split between positive and negative correlation coefficients (individual $\tau$s ranging from -.19 to .25, median of -.004), and only 2 correlations differed statistically from zero. Participants’ correlations for subtraction problems were not significantly different from zero, t(58) = 0.5, p = .61. The Bayes factor was BF₀₁ = 6.211, indicating the observed correlations are 6 times more likely under a null model. Thus, the problem size effect was extended from arithmetic problems to missing-operand problems, but only for the addition context. This provides initial evidence for some overlap in the mental representations and processes recruited for arithmetic, specifically addition, and simple algebra. Notably, we observe a dip in the mean RT for addition facts with a problem size of 10 (Figure 2, left). This may reflect practice effects given the role of 10 sums in ‘transformation’ strategies (LeFevre et al., 1996).

The second research question concerned whether overlapping mental representations and processes are recruited for arithmetic and algebra. We addressed this question using an individual differences approach. Specifically, individuals varied in the strength of their problem size effect in both arithmetic and simple algebra contexts (Figures 1 and 2). We assessed whether these differences covaried across the tasks by performing a simple linear regression predicting participants’ problem size effect for missing-operand problems from their problem size effect for arithmetic problems. For each problem class (i.e., missing-operand and arithmetic verification), the problem size effect was operationalized as the Kendall’s rank correlation between the problem size and the RTs (Figure 3). We conducted these analyses separately for the addition and subtraction contexts. For the addition context, there was an association between the missing-operand and arithmetic problem size effects, b = .45, 95% CI [.23, .67]), t(57) = 4.13, p < .001, R² = .23. A Bayesian linear regression indicated this association was more likely under the model where problem size effects are correlated across task than uncorrelated (BF₁₀ = 191.1). This is consistent with the recruitment of the same mental representations and processes for the two problem classes. For the subtraction context, there was no such association, b = .20, 95% CI [-.05, .46], t(57) = 1.58, p = .11, R² = .04. Here, the Bayes factor was BF₀₁ = 1.34, indicating the observed correlations are slightly more likely under a null model, i.e., weak support for the absence of this correlation.

Flexible Strategy Use

The third research question asks whether participants are fluent and/or flexible across problems in their strategy selection during the missing-operand task. Can we ‘decode’ whether participants are sensitive to the benefit of switching to the inverse strategy when solving missing-operand problems? For this analysis, multiple linear regressions were run for each participant individually, using all cases where the missing-operand problem and the direct and inverse arithmetic facts were all three answered correctly and within the time cutoffs.

We operationalized the ‘switch benefit’ for missing-operand problems as the ratio of an individual’s speed in retrieving the direct vs. inverse arithmetic facts during the arithmetic verification task:

This switch benefit ratio reflects the degree to which switching from direct retrieval to the inverse strategy would reduce an individual’s RT (or alternatively, the degree to which staying with direct would cost their RT). Consider the missing-operand problem x – 7 = 2. If participant A verified the direct match 9 – 7 = 2 in 1500 ms and the inverse match 2 + 7 = 9 in 1000 ms, then this ratio would be 1500 / 1000 = 1.5. That is, A would reduce their RT if they switched to the inverse route when solving x – 7 = 2. If participant B verified these same arithmetic facts in 1200 ms and 1800 ms, respectively, B’s switch benefit ratio would be 1200 / 1800 = .67, indicating that they would not reduce their RT if they switched; instead, B should stick with direct retrieval. To measure whether participants were sensitive to their individual switch benefit, we included it as a term in a regression predicting their missing-operand RT. Note that this variable is a reverse indicator: The larger the switch benefit ratio, the greater the benefit of switching from the direct strategy to the inverse strategy. Thus, if a participant is sensitive to this ratio and switches adaptively to the inverse strategy, then this term will be negatively predictive of their missing-operand RT (by reducing RT).

Thus, we regressed each participant’s RTs for missing-operand problems on their switch benefit term, and also on their RT for the directly matched arithmetic fact. The direct match RT operationalizes the participant’s retrieval fluency. We also included block number (1 or 2) as a predictor to control for practice effects. The full equation is:

Given the full regression model, the switch benefit ratio predictor thus measures strategy flexibility over and above fact fluency.

Figure 4 visualizes the results of the individual-level regressions. Each point shows the t-value for the direct RT predictor (x-axis) and the t-value for the switch benefit ratio predictor (y-axis) for a participant. The figure shows an overall trend ( $\tau$ = -.49, p < .001) as well as a range of individual differences. When participants’ missing-operand RTs are more positively predicted by their direct match RT (more positive on Figure 4’s x-axis), they tend to also be more negatively predicted by their switch benefit (more negative on y-axis), as indicated by the overall negative trend between the two t-values. Those participants with positive t-values for direct RT may be inferred to have high arithmetic fact fluency, and those with negative t-values for switch benefit may be inferred to have both (1) sensitivity to the ratio of each fact’s fluency (recall that this variable is reverse-coded) and (2) the strategy flexibility to switch to the inverse strategy when beneficial. This suggests an overall positive relationship between fact fluency and strategy flexibility. Additionally, there are clear individual differences, with a couple of participants positively predicted by switch benefit and not direct RT alone (top-middle of Figure 4), several by direct RT alone and not switch benefit (middle-right), some by both (lower-right), and some by neither (middle).

The regression analyses indirectly estimated the switch benefit for each participant from their RTs. The fourth research question sought converging evidence for these analyses. Do the resulting estimates correspond to participants’ self-reports about their strategy usage? We evaluated this by correlating the t-values for the switch benefit ratio to the percentage of the time they self-reported using the inverse strategy on the survey at the end of the experiment. As predicted, the two variables were negatively, though modestly, correlated, $\tau$ = -.25, p = .007. Thus, the more sensitive participants were to their switch benefit (the more negative the t-value associated with the switch benefit ratio variable), the higher their self-reported use of the inverse strategy. This correlation provides external validation for the strategy flexibility of participants that we indirectly measured (i.e., decoded) in our regression models.

The fifth research question asked whether strategy flexibility on missing-operand problems was associated with higher mathematical achievement more generally as measured by college entrance exam scores. We obtained standardized test scores for 58 of the 59 participants, 50 for the ACT college entrance exam and 8 for the SAT; the latter were converted to equivalent ACT scores using a concordance table.⁴ We fit a regression model predicting their ACT-Math (or equivalent) scores from the t-value for their switch benefit ratio predictor and the t-value for their direct RT predictor. In contrast to our predictions, we did not find a significant negative relationship between switch benefit and ACT-Math scores, b = -0.65, t(55) = -1.53, p = .13. However, direct match fluency was a significant and negative predictor of ACT-Math, b = -1.09, t(55) = -2.92, p = .005. We did not make any explicit hypotheses regarding this latter relation and do not have a clear interpretation of it. (These two results remained the same after removing an outlier with an ACT-Math score more extreme than 3 SDs below the mean, p = .26 and p = .027, respectively.) A Bayesian linear regression with both terms revealed the model with the highest odds increase from prior to posterior included only the direct match fluency predictor (BF_M = 2.5), followed by the model with both predictors (BF_M = 1.9). These results provide moderate evidence that direct match fluency is associated with ACT-Math scores (posterior inclusion BF = 5.2), but not switch benefit (inclusion BF = 0.73).