This is an open access article distributed under the terms of the Creative Commons
Attribution License (Individual Differences in Math Factors That May Impact Health Decision Making
How do people make health decisions involving trade-offs between costs and benefits (see Figure 1)? What math (e.g., comparison of the magnitudes of risks) and non-math information (e.g., prior beliefs and experiences, emotions, or severity of the disease) do people use to make decisions, specifically health-related decisions?
People might receive disease risk information from medical providers, risk tools, genetic tests, and the media, and the hypothetical problem in Figure 1 reflects one example of this type of information. For example, there are often trade-offs in risks of side effects that should be considered before one decides to take a new medication. Critically, the type of health information conveyed often involves ratios (i.e., whole number frequencies, fractions, and percentages). However, ratios, such as fractions, are often difficult to comprehend, and adults make predictable errors when reasoning about them (Braithwaite & Siegler, 2018; Denes-Raj & Epstein, 1994; Fitzsimmons et al., 2020; Ni & Zhou, 2005; Resnick et al., 1989; Reyna & Brainerd, 2007; Rittle-Johnson et al., 2001; Sidney et al., 2019; Siegler & Pyke, 2013; Siegler et al., 2011; Thompson et al., invited revision; Yamagishi, 1997). Further, people are quick to admit that they are “not math people” (Miller-Cotto & Lewis, 2020). In fact, even math teachers who teach about fractions report higher anxiety for fractions than whole numbers (Mielicki & Thompson, in preparation), and children and adults endorse holding more negative attitudes about fractions than whole numbers (Sidney et al., 2021). These individual differences in rational number understanding and affective factors may make it especially challenging for people to make accurate decisions when numerical information is involved, such as in health contexts.
Another individual difference, the strategies that people use as they compare and estimate ratios and complete fraction arithmetic problems, has illuminated common errors that children and adults make as they reason about rational numbers in neutral, non-health contexts. Such errors have been identified by examining trial-by-trial, self-reported strategies as people solve math problems (Fazio et al., 2016; Fitzsimmons et al., 2020; Sidney et al., 2019; Siegler & Thompson, 2014; Siegler et al., 2011). These strategies provide converging evidence when combined with people’s performance accuracy: Not only can researchers calculate how accurate participants are on any given math task, they can also assess why participants were correct or incorrect. For instance, a participant may incorrectly indicate that 3/15 is greater than 3/10, and their strategy report could highlight their explicit focus on the whole number denominator components if they say that 15 > 10. Both performance accuracy and strategy reports are related to overall math achievement (Fazio et al., 2014; Fazio et al., 2016; Sidney et al., 2019; Siegler & Thompson, 2014; Siegler et al., 2011). Furthermore, those participants who perform well on math tasks describe more savvy strategies used to solve the problems and have higher overall math skills than those who perform poorly. Those who perform poorly often describe strategies that involve math misconceptions, such as those that reflect whole number bias (Alibali & Sidney, 2015; Ni & Zhou, 2005; Thompson et al., invited revision) as illustrated in the above example.
Math Factors That Are Related to Performance in Pure Numerical Contexts Should Also Be Related to Health Decision-Making Accuracy
Our overarching goal in the current paper is to determine whether individual differences in math skills, math attitudes and anxiety, and math strategies that account for variance in performance accuracy in pure numerical contexts also account for variance in accuracy in hypothetical health decision-making tasks. We hypothesize that health decision-making problems that involve numerical information should be solved in very similar ways to math problems in pure numerical contexts. If this were to be the case, health decision-making researchers should consider assessing factors that are known to be associated with math performance outside of health contexts to understand performance accuracy within health contexts. To accurately compute the change in total risk in the problem in Figure 1, people likely use multiple math-based strategies (e.g., translate a fraction into a percentage to facilitate interpretation, compare risks, add and subtract risks). However, because people have negative attitudes about rational numbers (Mielicki & Thompson, in preparation; Sidney et al., 2021), and math anxiety is related to lower performance on rational number tasks (Choi et al., 2020; Sidney et al., 2019; Sidney et al., 2021), people may choose not to engage with the numeric information at all (Hattikudur et al., 2016; Nokes & Belenky, 2011).
Furthermore, to our knowledge, there is very little work in which open-ended strategy reports are collected as adults make hypothetical or real-world health decisions (cf. Thompson et al., invited revision), despite their demonstrated utility in studies of math problem solving. For example, Figure 1 presents an example of the type of questions that are often used in health decision-making research (adapted from Waters et al., 2007). In such problems, there is often a high likelihood of correct guessing; participants have a 33.33% chance of guessing correctly because the problem has three possible choices. Therefore, collecting strategy report data, in addition to measuring key individual differences, can help illuminate when correct performance reflects accurate reasoning. Further, we acknowledge that non-math factors, such as prior beliefs and experiences with the particular health contexts in question (e.g., health-related media coverage: Thompson et al., invited revision) could also be associated with performance on health decision-making problems.
To summarize our rationale, math problems concerning one’s own or others’ personal health, even if the problems are hypothetical, have the potential to engage the processes typically involved in math problem solving as well as additional processes that might influence the selection and use of math strategies compared to more neutral, “pure” numerical contexts. For example, people may possess pre-existing strategies when it comes to their health, such as an aversion to taking medication. Or, it is possible that the emotional salience of thinking about serious health concerns may use up cognitive resources that would be otherwise devoted to solving a problem involving complex health statistics. The primary goal of the present study was not to measure or assess the impact of these non-math factors on decision making, as many health studies have already examined these factors. Instead, the goal was to determine whether the same math skills, strategies, and processes that are engaged in pure numerical contexts also accounted for some proportion of the variance when predicting decision making in “hotter” health contexts that are likely to be more emotionally salient and perhaps personally relevant. We anticipated that math skills and strategies would play a role and conducted an empirical study to test this hypothesis, rather than assuming that findings from pure numerical contexts would generalize to health contexts. As such, we hope that this paper is a first step in highlighting why we believe major theories and commonly-used measures of math cognition should be considered in future health decision-making research.
Figure 1
Example of a Health Decision-Making and a Number-Line Estimation Problem
Note. Participants were randomly assigned to receive one of three number types (i.e., whole number frequency, fraction, or percentage) in each decision-making scenario. For example, those assigned to the percentage condition saw only percentages in their problems. The correct answer was that taking the drug would decrease total cancer risk (adapted from Waters et al., 2007).
The Current Study
We explored whether key individual differences in math cognition – math anxiety, measures of magnitude understanding (e.g., number-line estimation and magnitude comparison), arithmetic fluency, calculation accuracy on multi-step arithmetic tasks, math attitudes, math anxiety, and strategy use – were associated with performance accuracy on health decision-making problems given our own and others’ previous research in pure numerical contexts. Specifically, we anticipated that higher math anxiety and lower magnitude understanding would negatively predict decision-making accuracy.
Further, we included measures of objective and subjective numeracy (Fagerlin et al., 2007; Weller et al., 2013), which are often used in health decision-making research, to assess participants’ overall math skills. Our goal was to assess whether these measures accounted for more variance in decision-making accuracy than commonly-used measures from the math cognition literature used to measure math skills (cf. Thompson et al., invited revision). One advantage of using objective math cognition measures to assess math skills is that measures like number-line estimation and magnitude comparison are quick to administer and do not require participants to solve word problems, like those involved in objective numeracy measures, which are notoriously difficult (Koedinger et al., 2008; Nathan & Koedinger, 2000). If one goal for health decision-making researchers is to assess specific math skills in the areas of statistics and probabilities in low-pressure, non-anxiety provoking ways (Fagerlin et al., 2007), results from the current study may be of interest.
Method
Participants
The sample included 90 participants recruited through Amazon Mechanical Turk’s online panel (MTurk) (M = 36.72 years, SD = 10.42, range = 22-72; 51.11% male; 85.56% White; 19.32% completed some college and 52.23% were college graduates) who completed the health decision-making problems, and 93 MTurk participants (M = 36.39 years, SD = 10.57, range = 22 - 66; 41.94% male, 75.27% White; 23.33% completed some college and 45.56% were college graduates) who completed the non-health decision-making problem. MTurk is an online panel of “workers” who can complete online tasks for pay. In recent years, researchers have used this Amazon platform to collect data for their empirical studies in a relatively short amount of time. These MTurk workers completed the parent study (Sidney et al., 2021) in which one of the goals was to establish whether adults possessed differential attitudes about whole numbers, fractions, and percentages. At the end of the survey (M length = 45.72 minutes), participants completed the decision-making problems. The first health decision-making problem was adapted from Waters et al. (2007). We had originally planned to analyze the predictors of just the Waters et al. problem; other items were researcher-generated and served as pilot items for a secondary study. However, to provide additional evidence of our findings, we present analyses collapsed across all four health items, including the researcher-generated items, and a non-health decision-making problem. After participants solved each decision-making problem, they provided a strategy report to explain how they solved it.
In an attempt to obtain data from high-quality responders, we included only MTurk participants in the parent study who had previously completed at least 100 Human Intelligence Tasks (HITs) and had at least 95% approval from other requestors. Only three participants from the parent study were excluded for poor quality data (i.e., random responding). The study was approved by the Kent State University IRB, and consent was provided by each participant prior to beginning the survey. Prior to launching data collection for the parent study, we conducted an a priori power analysis for a 3 (between subjects: number type) x 2 (between subjects: health vs. non-health) design to detect a small effect size with 80% power and alpha level of .05.
Upon launching the survey on MTurk, participants informed us that the study took longer than advertised, resulting in underpayment based on length of time and difficulty of the math tasks. We increased the pay to 10 cents/minute for those who had already completed the survey and decided to stop data collection before obtaining the desired sample size.
As part of the study design, we included an experimental manipulation in which we presented the numerical information with different formats (e.g., fractions, whole number frequencies, or percentages) because number type is a well-established factor that influences risk comprehension and decision-making accuracy (Cameron et al., 2017; Cuite et al., 2008; Nelson et al., 2008; Schwartz et al., 2017; Schwartz et al., 1997; Waters et al., 2016; Waters et al., 2006, 2007; Witteman et al., 2011). Because we did not obtain the desired sample size and were underpowered to examine the effect of number type, we collapsed over the number-type conditions for our analyses.
See the Appendix for the survey flow of objective and subjective math tasks, health and non-health decision-making tasks, and demographic questions. Note that these measures were included to meet the main goal of the parent study (Sidney et al., 2021): To determine which measures were related to math attitudes.
Objective Math Measures
Magnitude Understanding
We operationalized magnitude understanding as the precision of adults’ number-line estimation skill as indicated by percent absolute error (PAE; Siegler & Booth, 2004). PAE is calculated as ((|actual - estimate|)/range of scale)*100. Higher PAE values indicate less precise estimates. We averaged PAE across fraction estimation tasks in the 0-1 and 0-5 ranges as well as the whole number estimation task in the 0-1 billion range because performance on these individual tasks was strongly correlated (see Table 4).
Fraction number-line estimation in the 0-1 and 0-5 range
Estimation tasks are strongly correlated with concurrent standardized math achievement scores (Siegler & Thompson, 2014; Siegler et al., 2011), predict algebra performance (Siegler et al., 2012), and provide a proxy for math skills more broadly. In the current study, participants placed 20 fractions, one at a time, on number lines ranging from 0 to 1 and in a separate block placed 20 additional fractions on number lines ranging from 0-5 (Siegler et al., 2011; Sidney et al., 2021). Participants completed estimates in the 0-1 range first and the 0-5 range second.
Number-line estimation in the 0-1,000,000,000 (1 billion) range
In this task (adapted from Landy et al., 2017), participants made estimates one at a time in the large numerical range: 0-1,000,000,000. PAE was averaged across all trials. Given that approximately half of adults mistakenly place one million at the midpoint between 0 and 1 billion (Landy et al., 2013), we included this measure to assess which adults may have a more precise understanding of numbers within this larger whole number range. Whole number estimation and fraction estimation are also strongly correlated (Fazio et al., 2014).
Magnitude Comparison Accuracy
In the 36 trials of the magnitude comparison task, participants saw two fractions and had to choose which fraction was the largest. The largest fraction appeared an equal number of times on the left and right sides of the screen. Fraction comparison is strongly correlated with other measures of magnitude knowledge, like number-line estimation (Fitzsimmons et al., 2020; Sidney et al., 2019; Siegler & Thompson, 2014; Siegler et al., 2011; Thompson & Siegler, 2010; Thompson et al., invited revision). The variable was calculated as the percentage correct across the 36 problems, and then scaled for interpretation such that M = 0, SD = 1.
Arithmetic Accuracy
Arithmetic accuracy was averaged across a fraction arithmetic task and a whole number fluency task involving addition, subtraction, and multiplication.
Fraction arithmetic
We measured participants’ fraction arithmetic performance across 24 problems: six problems blocked by addition, subtraction, multiplication, and division operation type. Participants were given three minutes to solve each set of six problems. Two problems in each section involved two fraction operands (Siegler et al., 2011), and four problems involved a whole number and a fraction (Sidney & Alibali, 2017). The variable was calculated as the number of correctly answered questions. Accurate performance on fraction arithmetic problems is related to precise estimates in the 0-1 and 0-5 numerical range (Siegler et al., 2011; Siegler & Pyke, 2013).
Whole number arithmetic fluency
We measured participants’ whole-number arithmetic fluency with the Calculation Fluency Test (Sowinski et al., 2014). Participants had three minutes to complete as many two-digit whole-number arithmetic problems out of 180 total as they could (i.e., one minute to complete up to 60 problems in each operation: addition, subtraction, and multiplication). The variable was calculated as the number of correctly answered questions. A measure of whole number fluency was included because we reasoned that those individuals who were more accurate on basic arithmetic should be able to accurately apply this knowledge to finding common denominators in our health decision-making problems.
Multi-Step Arithmetic Accuracy
Participants solved four multi-step fraction arithmetic problems in which the correct steps involved finding common denominators (for two problems), adding sets of fractions, and subtracting the resulting fractions. If participants solved these problems correctly, they should also correctly complete the procedural steps necessary to solve the health and non-health decision-making problems (Table 1) because the underlying mathematical formulas were identical. That is, these problems corresponded to the math that should have been used in the health decision-making problems. The variable was calculated as the number correct across the four problems.
Objective Numeracy
Participants completed the Rasch-based objective numeracy scale (Weller et al., 2013), a measure frequently used in health research, as a measure of their ability with numeric information, including probabilities. Five out of the eight problems required rational number understanding to solve. The variable was calculated as a summed score out of a total of eight problems. Our primary reason for choosing this objective measure of numeracy over others (e.g., Berlin Numeracy; Cokely et al., 2012) was to compare the explanatory power of an objective numeracy scale, commonly used in health decision-making research, with that of a number-line estimation task. Although other researchers have examined how number-line estimation precision in the 0-1,000 range related to measures of objective numeracy (Peters & Bjalkebring, 2015), adults’ performance in this range was likely at ceiling levels of precision (Siegler & Opfer, 2003; Thompson & Opfer, 2010). We reasoned that a fraction number-line estimation task or even an estimation task in a larger whole number range (Landy et al., 2017), which show more variability across adult participants than does a whole-number estimation task in the 0-1,000 range, would be a better indicator of adults’ basic math skills and perhaps be more closely related to objective numeracy performance given the prevalence of questions involving rational number understanding on this particular objective numeracy task. Furthermore, we wanted to assess whether objective numeracy or estimation precision on an age-appropriate number-line estimation task accounted for more variance in decision-making tasks. The Rasch-based objective numeracy scale was validated in a sample with a range of ages and educational attainment and has been shown to predict decision-making preferences and risk judgments (Weller et al., 2013).
Subjective Math Measures
Subjective Numeracy
The subjective numeracy scale (Fagerlin et al., 2007) is correlated with objective measures of numeracy. It was initially created to be a quicker assessment of math performance than objective numeracy scales. Further, because it does not involve math computations, it also minimizes the impact of negative attitudes and anxiety about math on participants’ responses. We included this measure because some of the questions were conceptually similar to those we included on the math attitudes questionnaire, see below, which allowed us to compare explanatory power across these conceptually similar measures. Six of the eight problems required participants to rate their rational number understanding. The variable was calculated as a summed score out of a total of eight problems.
Math Attitudes
Participants answered 20 questions (MAQ: Sidney et al., 2021) pertaining to their attitudes about math in general as well as their specific attitudes about whole numbers and rational numbers, such as fractions and percentages. We reasoned that those who had more negative math attitudes may be less likely to correctly answer the decision-making problems. This math attitudes questionnaire involved subscales for perceived ability, preferences, frequency of use, and importance. The variable was calculated as an average math attitudes score across items.
Math Emotions
Although we asked participants to rate their math emotions on seven-point Likert scales anchored by opposites (e.g., bad vs. good; sad vs. happy; Peters & Bjalkebring, 2015), these ratings were not the focus of the current study, and they are thus not analyzed.
Math Anxiety
Participants rated their overall math anxiety on a 10-point scale with higher scores indicating greater anxiety (Ashcraft, 2002). Higher math anxiety is related to lower accuracy on magnitude comparison tasks and lower precision on number-line estimation tasks (Sidney et al., 2019). It has been proposed that those who experience higher math anxiety may choose to avoid engaging with mathematical tasks (Ashcraft & Krause, 2007). According to Ashcraft’s (2002) seminal review paper on math anxiety, this single-item measure was correlated from .49 to .85 with scores on an abbreviated version of the original 98-item Mathematics Anxiety Rating Scale created by Richardson and Suinn (1972). Ashcraft states that the 1-item measure is sufficient “for a quick determination” of math anxiety (Ashcraft, 2002, p. 181).
Decision-Making Problems and Strategy Reports
Health and non-health decision-making problems. See Table 1 for the wording of all decision-making problems. Responses to each problem were scored in terms of accuracy (0=incorrect; 1=correct). The number of accurate responses was summed across the four health problems to create a continuous variable that could range from 0 to 4. The health decision-making problems were presented in the same order for all participants.
Table 1
Health and Non-Health Decision-Making Prompts
| Health Decision-Making Problem 1 (adapted from Waters et al., 2007) |
|---|
| Imagine that you are talking to your doctor about your risk of getting cancer. Your doctor says that your risk of getting stomach cancer is and your risk of getting colon cancer is . A new drug would cut your risk of getting stomach cancer to . But, the drug has a side effect that would increase your risk of getting colon cancer to . Think about your risk of getting either stomach cancer or colon cancer. Would taking the drug decrease your total cancer risk, increase your total cancer risk, or leave it unchanged? |
| Health Decision-Making Problem 2 |
| Imagine you are hospitalized for a severe allergy, and you are thinking about transferring to a new hospital with special allergy doctors. Your doctor tells you that your risk of your severe allergy symptoms getting worse in the current hospital is and your risk of catching pneumonia in the current hospital is . If you transfer to the new hospital, your risk of your severe allergy symptoms getting worse is cut to . But, transferring would increase your risk of catching pneumonia to . Think about your risk of either your symptoms getting worse or getting pneumonia. Would transferring decrease your total risk, increase your total risk, or leave it unchanged? |
| Health Decision-Making Problem 3 |
| Imagine that you're considering surgery, and the surgery will affect two systems in your body. The surgeon says that your current risk of heart failure in the future is to and your current risk of kidney failure in the future is . The surgery would cut your risk of heart failure to . But, the surgery has a side effect that would increase your risk of kidney failure to . Think about your risk of either having heart failure or kidney failure. Would having surgery decrease your total risk, increase your total risk, or leave it unchanged? |
| Health Decision-Making Problem 4 |
| Imagine that you have found radon (a natural radioactive gas) in your home, and you are thinking about having it removed. If you do nothing, then your risk of developing cancer because of radon is and your risk of developing cancer because of asbestos is . Hiring someone to remove the radon would cut your risk of radon-related cancer to . But, the procedure would increase your risk of asbestos-related cancer to . Think about your risk of getting either radon-related cancer or asbestos-related cancer. Would having the radon removed decrease your total risk, increase your total risk, or leave it unchanged? |
| Non-Health Decision-Making Problem |
| Imagine that your home floods every year and you are thinking about getting a flood-resistant treatment for your floor. If you do nothing, then your risk of needing to repair your floor is and your risk of needing to replace your floor is . Hiring someone to install the new treatment would cut your risk of repair to . But, the treatment would increase your risk of needing to replace your flood to . Think about your risk of either needing to repair your floor or needing to replace your floor. Would getting the treatment decrease your total risk, increase your total risk, or leave it unchanged? |
Note. The numbers-types--fractions, percentages, and whole number frequencies, which were equivalent in magnitude--varied by randomly assigned condition. See Figure 1 for an example.
Strategy reports
Participants typed their strategy report (Table 2) in a textbox after rating their willingness to take the action mentioned in the decision-making problem (e.g., Health decision-making problem 1: take a new drug with a side effect; see Figure 1 for prompt). The specific strategy report prompt was, Please describe how you made your decision about risk and willingness. Open-ended strategy reports are uncommon in the health decision-making literature; however, they are widely reported in the math cognition literature as they are strongly correlated with estimation precision (e.g., Fitzsimmons et al., 2020, 202118; Sidney et al., 2019; Siegler et al., 2011; Siegler & Thompson, 2014; Thompson et al., invited revision).
Table 2
Open-Ended Strategy Reports
| Strategy | Definition | Example | Non-Health Problem | Waters Problem | Four Health Problems |
|---|---|---|---|---|---|
| Attention to Math | Any attention to math including comparison (e.g., smaller, bigger), estimation (e.g., about 3%), calculation (e.g., adding or subtracting risks), specific references to numbers, or magnitudes (e.g., drastically, miniscule). | The much larger change in the risk of getting stomach cancer outweighs the small increase
in getting colon cancer. This is a significant drop in my risk for stomach cancer and the chances for each remain small. |
54.84% | 54.44% | 50.56% |
| Specific Numeral | Referenced a specific numeral. | I think that the decrease in one of the cancers is large enough to make up for the 3 percent increase. | 15.05% | 13.33% | 9.44% |
| Percentage | Referenced a specific percentage, or used the word “percentage.” | The overall risk of getting stomach cancer is almost halved, whereas the risk of getting colon cancer only goes up 2%. | 12.90% | 13.33% | 9.17% |
| Fraction | Referenced a specific fraction, or used the word “fraction.” | It would cut my chances of stomach cancer down by nearly half, but increase my colon cancer risk threefold. Tough choice. | 2.15% | 4.44% | 4.17% |
| Whole Number | Referenced a specific whole number. | I like the odds for stomach cancer better with the meds and the 2 point increase in other would still be very low. | 1.08% | 2.22% | 1.11% |
| Restated Prompt | Restated the strategy report prompt (e.g., risk would increase or decrease) without additional description of why, how, or in what manner this would occur. | It would decrease my total cancer risk. It upped my risk of getting colon cancer. |
11.83% | 25.56% | 13.61% |
| Non-Math | Exhibited at least one of the non-math subcategories below. | 23.66% | 25.56% | 24.17% | |
| Prior Experiences and Beliefs | Mentioned personal prior experiences or beliefs, such as opinions. | My family has a history of colon cancer. I would do nothing to increase that risk. Don't believe in prescription drugs. |
19.35% | 8.89% | 11.94% |
| Severity | Referenced the potential danger or harm of cancer. | Stomach cancer is also more dangerous. | 4.30% | 5.56% | 8.06% |
| Emotions | Used emotional language. | I would be apprehensive about this drug. | 0.00% | 5.56% | 2.50% |
| Doctor | Mentioned doctors. | If my doctor is telling me that I need to I would. | 0.00% | 3.33% | 1.39% |
| Difficulty | Problem was difficult or confusing. | I’m totally confused about the fractions, I’m lousy working with those. I’m trying to be as logical as possible. | 0.00% | 2.22% | 2.22% |
| Wants More Information | Expressed a desire for additional information. | I would need more information about the drug and side effects to make a more informed
decision. Are there other more serious side effects than cancer? |
0.00% | 2.22% | 2.50% |
| None | No explanation provided. | I went with what makes the most sense. | 8% | 2% | 5% |
Note. Proportion of use is calculated by adding the total instances of each strategy and dividing by the sample size. For the non-health problem, N = 93. For the Waters Health problem, N = 90. For the combined health problems, N = 360 (because each participant reported 4 strategies). Each strategy report could be assigned multiple codes. That is, the codes were not mutually exclusive.
We used a grounded theory approach (Strauss & Corbin, 1994) to code strategy reports (Fazio et al., 2016; Sidney et al., 2019; Siegler & Thompson, 2014; Siegler et al., 2011). Author 4 (PGS) created an initial coding scheme by examining participant responses to identify emerging themes. Author 3 (CJF) completed a preliminary coding of the entire data set. A research assistant also coded 25% of the dataset; inter-rater reliability was 97%. Finally, Authors 1 (CAT) and 2 (JMT) worked simultaneously to finalize the coding of the four health and one non-health decision-making problems together to reach 100% agreement. See Table 2 for a summary of the proportion of math and non-math strategies used.
Procedure
For each participant, the order of the objective math skills tasks (i.e., number-line estimation, magnitude comparison, whole number and fraction arithmetic, multi-step arithmetic, and objective numeracy) was randomized. The math attitudes, subjective numeracy, math anxiety, and math emotions tasks were completed in that order by all participants because the main goal of the parent study was to assess math attitudes with a new researcher-generated measure. Half of the participants completed the math skills tasks before they completed the math attitudes, subjective numeracy, math anxiety, and math emotions tasks.
Participants in each study were randomly assigned to one of three number-type conditions: whole number frequencies (41 out of 100), fractions (41/100), or percentages (41%). The numbers that participants saw in their decision-making problems corresponded to their randomly assigned experimental condition (see Figure 1 for an example). As noted above, we were underpowered to test the effect of number type on decision-making accuracy, so we did not include it in our analyses.
All participants completed the decision-making problems at the very end of the experimental session because, as mentioned above, analyses related to these decision-making problems were largely exploratory given that they were peripheral to the main goal of the parent study. Prior to completing the decision-making questions, participants rated their emotions and perceptions of severity with the scenarios/diseases mentioned in the questions. For example, on the health decision-making problem adapted from Waters et al. (2007), we asked participants, Imagine that you are talking to your doctor about cancer, and you are thinking about your risk of stomach cancer and colon cancer. How does thinking about your risk for stomach[colon] cancer make you feel (Very negative to Very positive)? How serious would it be if you had stomach[colon] cancer (Not at all serious to Extremely serious)? We used these questions to assess whether the scenarios/diseases were viewed as equally serious and elicited similar emotional reactions to ensure that the particular scenarios/diseases did not impact the participants’ responses. Table 3 indicates that the descriptive statistics for each example within a scenario (e.g., stomach vs. colon cancer) were rated similarly within each health and non-health decision-making problem.
Table 3
Descriptive Statistics for Emotion and Severity Ratings for Each Health and Non-Health Decision-Making Scenario
| Problem | M | SD | Mdn | min | max | skew | kurtosis | SE |
|---|---|---|---|---|---|---|---|---|
| Health Problems | ||||||||
| Emotion Ratings | ||||||||
| Health 1 Emotion 1 | 1.84 | 0.82 | 2.00 | 1.00 | 4.00 | 0.53 | -0.70 | 0.09 |
| Health 1 Emotion 2 | 1.77 | 0.78 | 2.00 | 1.00 | 4.00 | 0.56 | -0.73 | 0.08 |
| Health 2 Emotion 1 | 1.90 | 0.82 | 2.00 | 1.00 | 4.00 | 0.54 | -0.46 | 0.09 |
| Health 2 Emotion 2 | 1.87 | 0.86 | 2.00 | 1.00 | 5.00 | 0.88 | 0.67 | 0.09 |
| Health 3 Emotion 1 | 1.41 | 0.72 | 1.00 | 1.00 | 4.00 | 1.75 | 2.55 | 0.08 |
| Health 3 Emotion 2 | 1.50 | 0.75 | 1.00 | 1.00 | 4.00 | 1.40 | 1.27 | 0.08 |
| Health 4 Emotion 1 | 1.76 | 0.85 | 2.00 | 1.00 | 4.00 | 0.91 | 0.04 | 0.09 |
| Health 4 Emotion 2 | 1.69 | 0.89 | 1.00 | 1.00 | 5.00 | 1.29 | 1.32 | 0.09 |
| Severity Ratings | ||||||||
| Health 1 Severity 1 | 4.53 | 0.74 | 5.00 | 1.00 | 5.00 | -2.03 | 5.42 | 0.08 |
| Health 1 Severity 2 | 4.50 | 0.74 | 5.00 | 2.00 | 5.00 | -1.57 | 2.34 | 0.08 |
| Health 2 Severity 1 | 4.01 | 0.85 | 4.00 | 1.00 | 5.00 | -0.77 | 0.64 | 0.09 |
| Health 2 Severity 2 | 3.97 | 0.84 | 4.00 | 1.00 | 5.00 | -0.61 | 0.43 | 0.09 |
| Health 3 Severity 1 | 4.69 | 0.77 | 5.00 | 1.00 | 5.00 | -2.87 | 8.20 | 0.08 |
| Health 3 Severity 2 | 4.63 | 0.73 | 5.00 | 1.00 | 5.00 | -2.64 | 8.29 | 0.08 |
| Health 4 Severity 1 | 4.50 | 0.71 | 5.00 | 2.00 | 5.00 | -1.23 | 0.76 | 0.07 |
| Health 4 Severity 2 | 4.53 | 0.67 | 5.00 | 2.00 | 5.00 | -1.32 | 1.27 | 0.07 |
| Non-Health Problem | ||||||||
| Emotion Ratings | ||||||||
| Non-Health 1 Emotion 1 | 1.59 | 0.76 | 1.00 | 1.00 | 4.00 | 1.11 | 0.63 | 0.08 |
| Non-Health 1 Emotion 2 | 1.50 | 0.81 | 1.00 | 1.00 | 4.00 | 1.50 | 1.34 | 0.08 |
| Severity Ratings | ||||||||
| Non-Health 1 Severity 1 | 3.75 | 0.96 | 4.00 | 1.00 | 5.00 | -0.73 | 0.15 | 0.10 |
| Non-Health 1 Severity 2 | 4.12 | 0.95 | 4.00 | 1.00 | 5.00 | -1.35 | 1.96 | 0.10 |
Note. Emotion 1 and Emotion 2 refer to the two emotion ratings, and Severity 1 and Severity 2 refer to the two severity ratings for each example presented in the scenario.
Results
Confirmation of Random Assignment to Condition
We determined that random assignment to the health and non-health conditions was successful: Demographic factors, including participants’ age, t(180.94) = 0.22, p = .829, gender distribution, χ2(1) = 1.06, p = .303, and level of educational attainment (coded as proportion of college graduates: yes vs. no; χ2(1) = 0.04, p = .843, did not differ for those who were randomly assigned to complete the health vs. non-health decision-making problems. Additionally, there were no gender differences in accuracy for the health problems, t(88) = 1.13, p = .260, or the non-health problem, χ2(1) = 0.44, p = .509. Pearson correlations across all objective and subjective math variables and demographic variables for the full sample are included in Table 4. The percentage of participants who answered each health and non-health problem correctly is indicated in Table 5.
Table 4
Correlations Among Variables for Participants Randomly Assigned to the Health and Non-Health Conditions
| Measure | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| 1. PAE 0-1 | 0.70* | 0.50* | -0.22* | 0.23* | -0.41* | -0.32* | -0.30* | -0.44* | -0.36* | |
| 2. PAE 0-5 | 0.62* | 0.47* | -0.46* | 0.33* | -0.52* | -0.49* | -0.38* | -0.57* | -0.56* | |
| 3. Whole Number PAE | 0.33* | 0.11 | -0.33* | 0.38* | -0.38* | -0.19† | -0.35* | -0.42* | -0.44* | |
| 4. Math Attitudes | -0.18† | -0.34* | -0.16 | -0.54* | 0.44* | 0.21* | 0.40* | 0.48* | 0.76* | |
| 5. Math Anxiety | 0.25* | 0.42* | 0.18† | -0.64* | -0.46* | -0.23* | -0.27* | -0.45* | -0.58* | |
| 6. Fraction Arithmetic | -0.56* | -0.65* | -0.27* | 0.45* | -0.50* | 0.44* | 0.39* | 0.63* | 0.51* | |
| 7. Multi-step Arithmetic | -0.33* | -0.45* | -0.16 | 0.26* | -0.32* | 0.46* | 0.16 | 0.29* | 0.37* | |
| 8. WN Fluency | -0.38* | -0.40* | -0.22* | 0.36* | -0.21† | 0.53* | 0.38* | 0.37* | 0.43* | |
| 9. Objective Numeracy | -0.47* | -0.48* | -0.35* | 0.40* | -0.31* | 0.62* | 0.34* | 0.40* | 0.49* | |
| 10. Subjective Numeracy | -0.32* | -0.37* | -0.22* | 0.68* | -0.57* | 0.50* | 0.29* | 0.42* | 0.58* |
Note. Values above the diagonal are from the non-health condition. Values below the diagonal are from the health condition. PAE = Percent Absolute Error on the number-line estimation tasks. WN = Whole Number.
†p < .10. *p < .05.
Table 5
Accuracy for Each Decision-Making Problem, Health and Non-Health
| Problem Type | Health 1 | Health 2 | Health 3 | Health 4 | Health Average | Non-Health |
|---|---|---|---|---|---|---|
| Accuracy | 78% | 58% | 62% | 78% | 2.74 (1.26) | 82% |
Note. For individual problems, the percentage of participants who accurately responded is reported. Average score on health problems was calculated as a sum score out of four possible points. Standard deviation of the mean is presented in parentheses.
Variables Associated With Problem-Solving Accuracy
The overarching goal of our analytic plan was to determine which objective and subjective variables accounted for significant variance in health and non-health decision-making accuracy. We conducted a logistic regression for the non-health problem and a linear regression for the four health decision-making problems. To avoid overfitting our model, we aggregated some variables that were highly correlated so that our final model included nine predictors, in line with suggestions for sample size to predictor ratios (Babyak, 2004). Specifically, we aggregated the number-line estimation scales into a single PAE measure and aggregated the whole-number fluency and fraction arithmetic accuracy into a single arithmetic measure, both scaled such that M = 0, SD = 1. We entered all predictors into the model simultaneously.
Whether people attended to math, as measured by their open-ended strategy reports, was the only variable that accounted for significant variance in non-health decision making accuracy (Table 6). Note that about half of the participants in the non-health condition attended to math, and averaged across participants, about half of health decision-making problems involved participants mentioning that they attended to math (Table 2). Multi-step fraction arithmetic, magnitude comparison, and math anxiety accounted for significant variance in problem-solving accuracy across the four health-related problems (Table 7).
Table 6
Accuracy on the Non-Health Problem With Predictors Entered Simultaneously
| Predictors | Odds Ratio | LR Chi-square | p-value |
|---|---|---|---|
| Attention to Math | 10.89 | 8.29 | .003** |
| Number Line PAE (combined) | 0.72 | 0.42 | .519 |
| Magnitude Comparison | 0.67 | 0.70 | .403 |
| Arithmetic | 1.33 | 0.32 | .571 |
| Multi-step Arithmetic | 0.94 | 0.04 | .834 |
| Objective Numeracy | 1.27 | 1.13 | .288 |
| Math Anxiety | 0.84 | 1.34 | .246 |
| Math Attitudes | 1.07 | 0.01 | .912 |
| Subjective Numeracy | 0.89 | 0.06 | .812 |
**p < .001.
Table 7
Accuracy on the Four Health Problems With Predictors Entered Simultaneously
| Predictors | b | SE | t-value | p-value |
|---|---|---|---|---|
| Attention to Math | 0.05 | 0.08 | 0.66 | .512 |
| Number Line PAE (Combined) | -0.04 | 2.44 | -0.23 | .822 |
| Magnitude Comparison | 0.35 | 0.94 | 2.08 | .041* |
| Arithmetic (Combined) | -0.17 | 0.19 | -0.86 | .395 |
| Multi-step Arithmetic | 0.27 | 0.11 | 1.49 | .015* |
| Objective Numeracy | -0.03 | 0.08 | -0.41 | .684 |
| Math Anxiety | -0.14 | 0.06 | -2.42 | .018* |
| Math Attitudes | -0.32 | 0.28 | -1.11 | .270 |
| Subjective Numeracy | 0.26 | 0.19 | 1.36 | .177 |
Note. Unstandardized slope parameters (b) are reported.
*p < .05.
Why might these particular math measures be associated with decision-making performance? Magnitude comparison and fraction arithmetic are strongly related to estimation precision (i.e., PAE: Siegler et al., 2011); all three measures are thought to tap a common underlying construct—magnitude understanding—which is purported to be the building block for all of mathematics (Siegler, 2016). Also, recall that the multi-step fraction arithmetic problems were the exact problems that participants needed to solve if they were to correctly calculate the answers for each of the decision-making problems. Specifically, participants needed to find common denominators in two of the problems, add two sets of fractions, and subtract one fraction from another. These steps are more procedural than conceptual in nature (Rittle-Johnson, 2017). The reason multi-step arithmetic may not have been as strongly associated with accuracy on the non-health task is that the problem already contained common denominators, and thus, participants did not need to carry out a transformation to find common denominators to correctly answer the problem.
Math anxiety is likely associated with decision-making performance because it interferes with math reasoning and problem solving. Math anxiety is positively related to PAE (see Table 4). That is, as math anxiety increases, people are less precise at estimating numbers on number lines. Recent research has shown that math anxiety is associated with objective and subjective numeracy (Choi et al., 2020); the higher the math anxiety, the lower the numeracy. Further, we have recently shown that math anxiety is associated with math problem-solving in a real-world health domain: COVID-19 (Thompson et al., invited revision).
Discussion
Overview of Findings
The primary goal of the current study was to test the hypothesis that math-related factors that have been associated with accurate math performance in pure numerical contexts would also be associated with numeric decision-making performance in health contexts. Although health-related decisions are likely to be influenced by a variety of cognitive and affective processes that do not pertain to pure math decisions devoid of context, we anticipated that the same mathematical processes would be engaged when participants made health versus non-health decisions. In the current study, approximately 180 adults were randomly assigned to solve either numeric health or non-health problems. Across both types of problem-solving scenarios, at least one objective math factor (i.e., magnitude comparison accuracy, multi-step fraction arithmetic problem-solving accuracy, and explicit mention of attending to numeric information from the decision-making problem, as coded in participants’ open-ended strategy reports) was associated with performance on either the health or non-health decision-making problems. These measures likely tap the common underlying skill of numerical magnitude understanding (Sidney et al., 2019; Siegler & Thompson, 2014; Siegler et al., 2011). Prior research has shown that math anxiety is also related to tasks that tap magnitude understanding in pure numerical (Sidney et al., 2019) and health-related contexts (Choi et al., 2020; Thompson et al., invited revision).
These results align with previous research that points to the important role of objective mathematical competence for accurate decision making (e.g., Pertl et al., 2017). In the current study, validated measures of objective and subjective numeracy, commonly used in the health decision-making literature (Fagerlin et al., 2007; Weller et al., 2013), did not account for significant variance in decision-making performance in health or non-health contexts when objective and subjective measures from the math cognition literature were also entered as predictors in the models (see Tables 6 and 7Table 7). These findings converge with other recent findings (Thompson et al., invited revision) in which reliable and valid measures of math skills from the math cognition literature predicted COVID-19-related math problem-solving accuracy in a national panel of U.S. adult participants, yet objective and subjective measures of numeracy did not when math cognition measures were included in the same models. Therefore, health decision-making researchers should consider implementing validated measures from the math cognition literature to better understand the reasons that people fail to make mathematically-accurate numerical health-related decisions.
Variables That Are Associated With Health and Non-Health Decision-Making Accuracy
Magnitude comparison and multi-step fraction arithmetic accuracy were significantly associated with health decision-making performance. Magnitude comparison (Schneider et al., 2017; Siegler et al., 2011; Siegler & Thompson, 2014) is strongly correlated with overall math achievement and other measures of math skills, suggesting that it is a good proxy for overall math ability. Comparison of rational numbers is also a necessary component for correctly solving the decision-making problems. Similarly, if participants knew how to employ the multi-step procedures to correctly answer these equations, it meant that they had the prerequisite math knowledge that could be applied to correctly solve the decision-making problems. Finally, about 50% of participants in the non-health condition, and across about 50% of total trials in the health condition, people explicitly told us that they attended to math to solve the decision-making problems (see Table 2). Given that health decision-making problems have been framed as multiple-choice questions in past health decision-making research (cf. Waters et al., 2007; Thompson et al., invited revision), strategy reports provide important information about why participants answer correctly or make the errors that they do. It is also a novel contribution that in our study we requested open-ended strategy reports that allowed participants the ability to detail the steps they went through to answer the question rather than simply ask them to choose a response from a drop-down menu that could have influenced their subsequent strategy reports by suggesting a strategy that they otherwise would not have considered.
Note that only one of the health decision-making questions used in our study was adapted from prior literature (see Supplementary Materials on OSF in which we indicate the measures that accounted for significant variance in this problem adapted from Waters et al., 2007). The other problems in Table 1 were generated specifically for use in this study. These problems should be assessed for reliability and validity for use in future research studies, and any differences in factors associated with health vs. non-health decision-making accuracy should be interpreted with caution. This is especially true given that the non-health problem that we analyzed contained common denominators, whereas two of the health problems contained common denominators and two contained different denominators. Common denominator fraction arithmetic problems are easier to solve than are different denominator fraction arithmetic problems because participants do not have to transform the fractions to combine them (Siegler et al., 2011). Indeed, 82% of participants in the current study (see Table 5) correctly answered the non-health problem which contained common denominators. To correctly answer our health decision-making problems that involved different denominators, participants needed to execute one more procedural step prior to adding the risks together and comparing them (i.e., find common denominators), which may have more thoroughly engaged their math-relevant processes relative to problems with common denominators.
Limitations and Future Directions
Typically, in pure numerical contexts, researchers craft strategy report prompts that are very targeted (i.e., Why did you decide that this was the right place for this number on the number line?). However, in the current experiment, we asked a two-pronged question: Please describe your decisions about risk and willingness. More recent research (Thompson et al., invited revision) included a more targeted strategy prompt question, Please describe in as much detail as possible how you made your decision about which disease (COVID-19 vs. the flu) is more fatal. This latter strategy report prompt resulted in participants providing very rich descriptions of their problem-solving strategies, including explicit mention of specific math misconceptions, such as the whole number bias. Therefore, we would recommend that in the future, health decision-making researchers should include more targeted strategy report prompts to ensure that they can identify any existing math misconceptions that could impede their participants’ health decision-making performance.
As with many psychology experiments, our final sample of over 180 participants was essentially a convenience sample of participants who were willing to sign up for our study through the MTurk platform. We were as vague as possible about the purpose of the study in our IRB-approved consent form. However, it is true that participants knew that our study dealt with numerical decisions, and that may have attracted a certain type of participant, namely participants who were willing to solve math problems. But, we did not disclose that participants would be asked to make decisions about rational numbers. Results from the parent study (Sidney et al., 2021) revealed that participants held more negative attitudes about fractions than other types of numbers, such as whole numbers and percentages. Thus, we interpret this finding to mean that we did not “scare off” participants from completing our study who disliked rational numbers.
As indicated in Table 2, participants mentioned percentages in their strategy reports even when they were randomly assigned to conditions in which the numeric health information was presented as fractions or whole number frequencies. This suggests that some respondents translated the provided numeric information into an easier to comprehend format, a technique that has been shown to improve precision of magnitude estimates in pure numerical contexts (Fazio et al., 2016; Sidney, Thalluri, et al., 2019; Siegler & Thompson, 2014; Siegler et al., 2011). However, this finding should be interpreted cautiously given that we were underpowered to assess the effect of number type on decision-making accuracy. Because it has been recommended that risk information should be expressed as whole number frequencies, such as 2 in 100 (Cameron et al., 2017; Cuite et al., 2008; Nelson et al., 2008; Schwartz et al., 2017; Schwartz et al., 1997; Waters et al., 2016; Waters et al., 2006; Waters et al., 2007; Witteman et al., 2011), additional research is needed to determine how likely participants are to mentally convert whole number frequencies into percentages when interpreting health-related statistics.
Finally, it is an open empirical question as to whether some participants reported non-math strategies in their open-ended reports because they did not want to expend the “mental effort” needed to carry out a mathematical calculation because they were fatigued at the end of the study. Non-math strategies were reported by about half of our participants; however, we did not have a formal way to assess participants’ expended effort on the decision-making task. It is possible that “mental effort” could be operationalized as the amount of time that it takes for a participant to solve each decision-making problem. However, response times may not be an unambiguous measure of mental effort. Previous research (Fazio et al., 2016) does provide evidence that different types of math strategies take differing amounts of time to execute. However, the time it takes to execute a strategy does not always map onto the accuracy of the response. For instance, various correct strategies, such as transforming a fraction into a percentage or visualizing a pie cut into parts, can take a short or long time to execute. Further, if we draw on findings from one of our recently conducted studies on participants’ math misconceptions pertaining to COVID-19 (Thompson et al., invited revision), we have evidence that for the very first health decision-making problem that participants solved at the beginning of the study, participants still mentioned a mixture of math and non-math strategies. Therefore, we argue that it is unlikely that non-math strategies deployed in the decision-making task at the end of the current study were simply the result of fatigue. It is a worthy endeavor for future researchers to investigate how slowly or quickly participants deployed the strategies that they mentioned using in their strategy reports.
Practical Implications
We cannot underscore enough that objective math skills accounted for a significant amount of variance in decision-making performance. Thus, known math misconceptions identified in “cold,” purely numerical contexts should be remediated to facilitate participants’ attempts at comprehending their risks in “hot,” real-world health contexts. For example, consider recent work by Thompson and colleagues (invited revision) in which a multifaceted educational intervention aimed at teaching adults how to calculate COVID-19 case-fatality rates resulted in decreased use of whole number bias strategies (i.e., paying attention to just the number of deaths or the total number of people infected with COVID-19) after engaging with the brief math-based intervention. Not only did participants learn to more accurately calculate rates, they also gave higher ratings across select periods of time within a 10-day daily diary follow up when asked about (a) their own and others’ risk of COVID-19 and (b) worry about themselves and others getting COVID-19. The implication is, when people are asked to make numeric health decisions, researchers should consider whether the participants’ prior math knowledge, or lack thereof, poses a significant barrier to their health risk comprehension (see also Woodbury et al., under review).
Conclusions
With growing interest in precision medicine (Ashley, 2015; Collins & Varmus, 2015), people will increasingly receive numeric disease risk information. Additionally, at the time of writing this, we are in the midst of a global COVID-19 pandemic, and people are receiving numeric health information about this virus at an alarming pace. Understanding the math and non-math related strategies that people use to reason about health-related information could inform future interventions to enhance comprehension of health messaging (e.g., how COVID-19 case-fatality rates decrease in locations with mask mandates). Our data suggest that having participants complete reliable and valid measures of their objective math skills drawn from the math cognition literature as well as having them report the strategies that they employed to solve health-related decision-making problems, as is commonly done in purely numerical contexts (Sidney, Thalluri, et al., 2019), is a fruitful way to advance research on health decision making.