^{1}

^{2}

^{3}

^{1}

^{3}

In this study, we used multivariate decoding methods to study processing differences between canonical (montring and count) and noncanonical finger numeral configurations (FNCs). While previous research investigated these processing differences using behavioral and event-related potentials (ERP) methods, conventional univariate ERP analyses focus on specific time intervals and electrode sites and fail to capture broader scalp distribution and EEG frequency patterns. To address this issue a supervised learning classifier—support vector machines (SVM)—was used to decode ERP scalp distributions and alpha-band power for montring, counting, and noncanonical FNCs (for integers 1 to 4). The SVM was used to test whether the numerical information presented in FNCs can be decoded from the EEG data. Differences in the magnitude and timing of accuracy rates were used to compare the three types of FNCs. Overall, the algorithm was able to predict numerical information presented in FNCs beyond the random chance level accuracy, with higher rates for ERP scalp distributions than alpha-power. Montring had lower peak accuracy compared to counting and noncanonical configurations, likely due to automaticity in processing montring configurations leading to less distinct scalp distributions for the four numerical magnitudes (1 to 4). Paralleling the response time data, the peak decoding accuracy time for montring was earlier for montring (472 ms), compared to counting (577 ms) and noncanonical FNCs (604 ms). The results provide support for montring configurations being processed automatically, somewhat similar to number symbols, and provide additional insights for processing differences across different forms of FNCs. This study also highlights the strengths of decoding methods in EEG/ERP research on numerical cognition.

Numerical cognition, the ability to reason with numbers, is crucial for individual life and career advancement (

Finger-numeral configurations (FNCs) refer to specific ways fingers are raised to enumerate number sequences (finger-counting) and to communicate numbers to other people (finger-montring). While finger-counting has been extensively studied in the literature, in the context of both counting and arithmetic tasks, finger-montring is a relatively new concept (first defined by

Previous research has found both behavioral and neural differences in visually identifying canonical (culturally dominant) and noncanonical FNCs with adult participants (

In the first study that studied processing differences across montring, counting, and noncanonical configurations, participants were asked to identify numerical information in FNCs (

Overall, the limited number of studies on processing differences between canonical and noncanonical FNCs hint at early developmental experiences with FNCs having a lasting effect that can be observed in adulthood, and canonical, particularly montring, configurations being processed in a way that is somehow similar to number symbols. However, how canonical configurations are processed differently than noncanonical ones (e.g., which stages of processing) is still not clear. While ERPs provide the high temporal resolution needed to answer this question, use of conventional ERP analysis requires a priori determination of ERP components to be studied, which are characterized by specific types of processing, associated with a time interval and electrode site. This approach works when the focus of investigation can be narrowed down to specific ERP components (e.g., use of the N400 component in language studies on semantic incongruence), however, it is not ideal when the tested effects are distributed across multiple electrode sites and time intervals.

Machine learning methods provide alternative ways to analyze EEG, and in general neuroimaging, data that allow studying processing differences in wider temporal and spatial scales, and with higher sensitivity (

In this study, we used multivariate decoding methods to study FNCs for the first time. Our overarching goal was to investigate whether sustained potentials (ERP scalp distributions) and alpha-frequency EEG signals can be decoded to predict the numerical information represented in FNCs and to compare prediction accuracy rates across different types of FNCs. The presented analysis is a follow-up to a previous study (

The EEG dataset analyzed in this study was a publicly available dataset previously used in an ERP study, investigating behavioral and ERP differences in processing montring, counting, and noncanonical FNCs, using traditional univariate methods (

Data from thirty-eight adult participants were included in the analysis (20 female,

The stimulus set for the experiment constituted the combination of three types of FNCs—montring (M), counting (C), and noncanonical (NC)—and four numbers: one to four shown by different FNCs, separately for the left and right hands. This added up to 24 pictures of unique FNCs. Because the previous study reported no differences between the FNCs for the left and the right hands, data from both hands were pooled together, generating 12 unique categories (

The experiment was divided into 10 blocks, with 96 trials in each block, including four sets of the 12 configurations, randomly sequenced across each block. There were 960 trials in total, with 80 trials for each FNC. In each trial, an FNC was presented to the participant for 500 ms followed by an Arabic numeral for 1000 ms (

A BrainVision 32 Channel ActiChamp system, with Easy Cap recording caps using Ag/AgCl electrodes (Fp1, Fz, F7, FT9, FC5, FC1, C3, T7, TP9, CP5, CP1, Pz, P3, P7, O1, Oz, O2, P4, P8, TP10, CP6, CP2, C4, T8, FT10, FC6, FC2, F4, F8, Fp2, Cz) was used for data collection. The international 10-20 system was used for electrode locations. The electrode Cz was the recording reference electrode, and the recording sampling rate was 500 Hz.

A custom MATLAB script using EEGLAB (

The EEG signals were epoched between 500 ms before the stimulus onset (FNC presentation) to 1500 ms after (stimulus offset). All epochs were corrected to the 500 ms pre-stimulus baseline. A moving window peak-to-peak threshold algorithm (threshold 60 µV, window size 80 ms, window step 20 ms) was used for detecting eye blinks, and a step-like artifacts algorithm (threshold 50 µV, window size 200 ms, window step 100 ms) was used for detecting eye movements. The results of the automatic artifact detection were inspected visually. Epochs that were marked during the artifact detection step were excluded (20.92% of trials,

After artifact detection, separate pre-processing steps were followed for the ERP-based and the alpha-based decoding. To ensure the decoding of non-overlapping signals for the two analyses, ERP decoding was limited to frequencies less than 6 Hz and alpha-band decoding was limited to frequencies between 8 and 12 Hz. For the ERP-based decoding, a 6 Hz (half-amplitude cutoff) low pass IIRButterworth filter (24 dB/octave) was applied to the epoched data. For the alpha-based decoding, an 8-12 Hz band-pass filter was used on the epoched data. Hilbert Transform was applied to improve the measurement of instantaneous alpha amplitude. The amplitude of the complex analytic signal was calculated and squared at each time point to calculate alpha power.

The decoding approach used in this paper was adapted from

The proposed decoding method by

In this study, for each participant, the shuffling EEG data at Step 2 was repeated 10 times which means ten iterations, and for each iteration, decoding was done separately for 100-time points at the time range of -500 ms to 1500 ms (data points were selected with a frequency of 20 Hz). Therefore, after completing all the iterations and cross-validation procedure, the classification score for each participant was a 4D matrix with a dimension of the number of iterations * number of time points * number of cross-validation * number of classes. Also, the total number of decoding attempts for each participant was the product value of number of iterations * number of time points*number of cross-validation * number of classes (i.e., 10*100*3*12). Accuracy rates for each time point, separately for each participant were calculated and then the point-by-point accuracy rates were then averaged across the participants to get the final decoding accuracy rates for each time point.

The decoding steps for the two types of analyses, ERP-based and alpha-based, were identical, except for the nature of the time series data; instantaneous alpha-power for the alpha-based analysis and EEG amplitudes for the ERP-based analysis.

Since there were 12 classes the chance level accuracy was 8.3% (1/12), meaning that if the signal analyzed (alpha-power or ERP scalp distributions) contained no identifying information about FNC categories, the decoding accuracy would be expected to be around 8.3%. For both analyses, the decoding accuracy went above the chance level, but higher for the ERP-based analysis (

We averaged the decoding accuracies across all 12 FNCs to compare the performance of the two decoding analyses. The decoding accuracy peaked at 220 ms with a 12.3% accuracy rate for the alpha-based decoding. The ERP-based decoding was robust and showed extensive time windows where accuracy was greater than chance. The maximum decoding accuracy across all time points for the ERP-based decoding was 26.7% (at 490 ms), which is more than threefold the chance level accuracy. This is a noticeable result and implies that the decoding approach applies to the domain studied. The results indicated that both ERP-based and alpha-based decoding provided above-chance level accuracy for decoding FNCs. However, overall, the ERP-based accuracy was higher than the alpha-based accuracy; therefore, we implemented only the ERP-based method for the category-specific (montring vs. counting vs. noncanonical) decoding.

An addition to the decoding accuracy, confusion matrices provided for both alpha-based and ERP-based decoding ERP-based decoding (

The decoding of numerical information contained in FNCs showed higher success for the ERP-based decoding, compared to the alpha-based one. This result parallels what was reported by

The average decoding accuracy across all four numbers was calculated separately for each type of FNC (M, C, NC). Then, the average accuracy data over time was smoothed with a five-point moving window Gaussian filter to improve the signal-to-noise ratio (

Averaged accuracy over time, peak accuracy, and time of peak accuracy in the 100 to 1000 ms range were calculated as dependent measures (

FNC | Average accuracy (%) |
Peak accuracy (%) |
Peak accuracy time (ms) |
---|---|---|---|

Montring (M) | 36.5 (6.6) | 47.2 (8.9) | 445.789 (186.629) |

Counting (C) | 44.3 (9.6) | 57.7 (11.4) | 506.316 (184.192) |

Noncanonical (NC) | 46.3 (10.9) | 60.4 (12.6) | 515.263 (194.377) |

The one-way ANOVAs were significant for average accuracy and peak accuracy, but not for peak accuracy time (

Dependent Var. | Mean square | η2 | |||
---|---|---|---|---|---|

Average accuracy | 2 | 0.123 | 24.339 | < .001 | 0.397 |

Peak accuracy | 2 | 0.186 | 27.330 | < .001 | 0.425 |

Peak accuracy time | 2 | 54277.193 | 1.492 | 0.232 | 0.039 |

Post-Hoc Tests | Mean difference | Cohen’s |
_{holm} |
|||
---|---|---|---|---|---|---|

Average accuracy | ||||||

M | C | -0.087 | 0.016 | -5.319 | -0.943 | < .001 |

M | NC | -0.107 | 0.016 | -6.570 | -1.165 | < .001 |

C | NC | -0.020 | 0.016 | -1.251 | -0.222 | 0.215 |

Peak accuracy | ||||||

M | C | -0.106 | 0.019 | -5.582 | -0.951 | < .001 |

M | NC | -0.132 | 0.019 | -6.989 | -1.191 | < .001 |

C | NC | -0.027 | 0.019 | -1.408 | -0.240 | 0.163 |

In addition, confusion matrices for all category-specific ERP-based decoding (i.e., M, C, and NC) were provided to investigate any finger-specific difference in decodability (see

The present study investigated processing differences across three types of FNCs (montring, counting, and noncanonical) using a novel decoding approach. We first tested whether scalp-recorded EEG signals contain decodable information about the numerical information contained in FNCs, and which of the two measures, instantaneous alpha-power vs. sustained EEG potentials (ERP scalp distributions), provided higher decoding accuracies. The results demonstrated above-chance level decoding accuracies for both analyses, however, ERP-based analysis showed higher accuracies across all FNCs, compared to alpha-based. Secondly, ERP-based decoding was used to investigate processing differences across the three types of FNCs.

A comparison of the results reported here with the results reported in a previous study (

A remarkable finding is how the ERP-based decoding analysis informed the timing of when the bulk of the processing associated with the retrieval of numerical information from FNCs happens. The peak decoding accuracy times for both counting and noncanonical FNCs were post-500 ms (577 and 604 ms respectively), while it was pre-500 ms only for montring (472 ms). The accuracy rates at these peaks were more than double the chance level for counting and noncanonical (47.2%, 57.7%, and 60.04% percent respectively), showing that EEG scalp distributions were successfully decoded around these time points to predict the numerical information included in the FNCs. In the previous study,

The comparison of results from the decoding analysis and previous ERP studies on FNCs shows that, in general, decoding analyses can be instrumental in identifying stages of processing that best characterize differences across the conditions studied. This can especially be helpful in areas of numerical cognition research that have not been extensively studied using ERP methods, where researchers cannot rely on previous research to determine ERP components of interest. It should be noted that similar issues with traditional ERP analysis—where time intervals and electrode sites associated with specific ERP components need to be identified a priori—were previously discussed and mass univariate methods were proposed as an alternative (

The results also informed processing differences across the three types of FNCs (montring, counting, and noncanonical). Average decoding accuracy and peak accuracy were significantly lower for montring, while counting and noncanonical did not differ. The behavioral data, reported in the previous ERP study (

The confusion matrix results provided insightful information about the decodability of each FNC compared to all other FNCs and within their categories. These results follow the general pattern in the study where ERP-based decoding offers higher true positive rates than the alpha-based decoding results. While these differences in the true positive rate may be driven by processing salient features of that FNC (e.g., NC2 having a higher true positive rate because of its odd non-numerical gestalt), the percentages of correct classifications are somewhat comparable across all FNCs and within categories.

Overall, these results show the advantage of decoding methods in elucidating the processing of internalized representations of FNCs and in numerical cognition studies in general. Compared to traditional ERP analysis, ERP decoding allows the comparison of processing differences across wider time windows and the entire scalp distribution. These findings show the versatility of ERP decoding methods and point to their relevance for numerical cognition studies.

This study aimed to investigate the neural processing of FNCs using a multivariate decoding method and compare this method with conventional univariate ERP analysis. A support vector machine (SVM) was used as the decoding and classification algorithm. Both ERP scalp distributions and alpha-band power were used for decoding FNCs. The results showed higher accuracy rates during the entire time interval when ERP scalp distributions were used. The results also showed that ERP scalp distribution is best at distinguishing task categories in the early stages of perceptual processing, paralleling previous results.

The results informed processing differences across the three FNCs; montring, counting, and noncanonical. Counting and noncanonical FNCs showed higher averaged and peak accuracy rates, compared to montring. The comparison of the decoding results with the behavioral and conventional ERP analysis results reveals a complementary picture, where recognition of montring FNCs are found to be more automatic and less effortful. Montring configurations were identified faster and more accurately, corroborating the automaticity explanation. Faster and more automatized access to numerical information for montring configurations leads to less distinct ERP scalp distribution signatures, leading to lower decoding accuracy rates.

The decoding analysis better complemented the behavioral results compared to conventional ERP analysis, pointing to the advantages of decoding over conventional ERP analysis. In particular, a priori selection of time intervals and electrode sites associated with specific ERP components constitute challenges with novel domains of research, not previously extensively studied with conventional ERP methods. The decoding approach considers wider aspects of data—the entire scalp distribution instead of specific electrode sites, and wider time windows—when characterizing differences across task conditions.

The Supplementary Materials contain a custom MATLAB script using EEGLAB (

The authors have no funding to report.

The authors have declared that no competing interests exist.

The authors have no additional (i.e., non-financial) support to report.