2.1.

Correction Methods

In this study, we compared two methods for correcting FWE. One is a conventional Bonferroni method used to adjust the significance level ($\alpha$) by dividing it by the number of tests as in Eq. (1) presented above. The other is the $M_{\mathrm{eff}}$ method utilizing eigenvalues of correlation matrices. To reflect the strength of correlation between each data point, the $M_{\mathrm{eff}}$ method adjusts the original multiplicity to yield the effective multiplicity, $M_{\mathrm{eff}}$^26–28 In multichannel fNIRS group analyses with multiple subjects, we derive a correlation matrix from measured signals, such as channel-wise average values of oxy-Hb signals, from multiple subjects and calculate the eigenvalues. We then obtain the $M_{\mathrm{eff}}$ from the eigenvalues and apply it instead of the original $M$ to adjust the $\alpha$ value.

Let us explain the theoretical framework of the $M_{\mathrm{eff}}$ method using a typical fNIRS data structure as an example. Note that the major $M_{\mathrm{eff}}$ methods described below were originally developed for genetic data, but we modified them to fit the fNIRS data scheme. When multiple-channel fNIRS data are obtained from $M$ channels for $N$ subjects, summary data for group analyses can be denoted as ${\beta }^{M\times N}$. Then, the eigenvalues (${\lambda }_i$) are derived from a correlation matrix ($M\times M$) of the measured signals (${\beta }^{M\times N}$) as follows:

Using the eigenvalue variance $V_{\lambda }$, Nyholt²⁶ calculated the $M_{\mathrm{eff}}$ as described below:

In order to present a more accurate estimate of $M_{\mathrm{eff}}$, Li and Ji²⁷ introduced a new calculation for $M_{\mathrm{eff}}$ as described below:

$I(x\ge 1)$ is an indicator function, which yields 1 when $x\ge 1$ and gives 0 otherwise. [$x$] is the floor function, which gives the largest integer less than or equal to $x$.

Galwey²⁸ pointed out that the $M_{\mathrm{eff}}$ by Li and Ji gives more weight to the fractional part than to the integer part and proposed a generalized function as follows:

In this research, we implemented Galwey’s function to calculate $M_{\mathrm{eff}}$ because this method is optimized for multiple signals with strong correlations and can be treated in a continuous way.

2.2.

Experimental Procedures

Experimental data used in this study were obtained from three of our previous, separate studies^29–31 and reanalyzed from different perspectives. Here, we briefly describe the experimental procedures.

Participants in the visual-based oddball task (OBT, task 1) comprised 22 right-handed, healthy children (15 boys, 7 girls; $\text{average}\text{age}=9.8\text{years}$, $\mathrm{SD}=2.0$, and range: 6 to 13 years). Those for the verbal category fluency task (CFT, task 2) comprised 22 right-handed, healthy volunteers (6 participants were excluded; 16 males, 6 females; $\text{average}\text{age}=34.0\text{years}$, $\mathrm{SD}=10.5$, and range: 22 to 57 years). Those for the naming task (NMT, task 3) comprised 26 right-handed, healthy volunteers (4 participants were excluded; 19 males, 7 females; $\text{average}\text{age}=33.7\text{years}$, $\mathrm{SD}=10.7$, and range: 22 to 57 years). Written informed consent was given by all participants and, in the case of the OBT experiment, their parents. The study was approved by the Jichi Medical University ethics committee.

In the OBT, participants were requested to push a button in response to stimuli. The task included detection of and response to infrequent (oddball) target events included in a series of repetitive events. During the session, subjects viewed a series of pictures once every second and responded with a key press to every picture. In the baseline block, subjects were presented one picture and asked to press a blue button for that picture. Following the baseline block, two kinds of pictures were presented sequentially and the participants were instructed to respond to the standard stimuli and target stimuli by pressing a blue and red button, respectively. Each session consisted of six block sets, each containing alternating baseline (25 s) and oddball (25 s) blocks.

In the CFT, participants were requested to overtly generate words for five categories. The task paradigm was a periodic block design with five alternating conditions of rest (30 s) and experimental task (20 s).

In the NMT, participants were requested to overtly name the content of a picture exhibited by an experimenter. The task paradigm was a periodic block design with five alternating conditions of rest (30 s) and experimental task (20 s). During the experimental task, each picture was presented for about 3 s.

2.3.

Evaluation of $M_{\mathrm{eff}}$

Using resampling simulations from experimental data of fNIRS studies, we evaluated the relationship between $M_{\mathrm{eff}}$ values and the number of channels. For the resampling simulations, we selected, from published studies, three kinds of experimental data with different activation profiles: OBT data from children with lateralized focused activation (task 1),²⁹ CFT data from adults with lateralized moderately focused activation (task 2),³⁰ and NMT data from adults with bilateral broad activation (task 3).³¹ The $\beta$-values of a general linear model (GLM) with regression to a hemodynamic response function or the average values of oxy-Hb signals were used as summary data. Because the measured signals of the children contained many movement artifacts, the averaged values of oxy-Hb, which were compatible with $\beta$-values, were selected to analyze the children’s data (note that $\beta$-values match the average when a GLM is performed with regression to a box-car function alone). Specifically, after the blocks with considerable artifacts were removed, for each channel of each subject the averaged differences in oxy-Hb values between each task and the preceding baseline periods were further averaged across the task blocks.²⁹

To find the relationship between the number of channels and the $M_{\mathrm{eff}}$, we randomly resampled a given number channels ($m:(1\le m\le 44)$) from each dataset and calculated $M_{\mathrm{eff}}$. For each $m$, resampling was performed 1000 times, and the average value and standard deviation were calculated.

In order to find the relationship between the overall number of channels and the number of channels detected as active with the $M_{\mathrm{eff}}$ derived for a given dataset, truly active channels must be set. We performed one-sample $t$-tests for the measured signals for each channel, and those channels with $t$-values above a given $t$-value threshold were defined as “truly active channels” for the corresponding task. The $t$-value thresholds were set to 2.0, 2.5, 3.0, and 3.5. Using these obtained truly active channels as a starting point (e.g., 2 channels for task 1, 9 for task 2, and 12 for task 3 at a $t$-value threshold of 3.0), we added additional channels, derived the $M_{\mathrm{eff}}$, and calculated the number of active channels. For each channel number, resampling was performed 1000 times (or for a theoretical maximum when the resampling range was limited), and average values and standard deviations were calculated. For comparison, the numbers of active channels were also obtained for uncorrected data and for Bonferroni correction.

3.1.

Selection of Active Channels as “Truly Active”

For each task, second-level group analyses using one-sample $t$-tests against zero were performed. We defined these channels as “truly active channels” for the resampling simulation. The numbers of truly active channels are shown in Fig. 3. For example, when the $t$-value threshold was 3.0, the number of truly active channels was 2 for task 1 (OBT), 9 for task 2 (CFT), and 12 for task 3 (NMT). These cases are presented in Fig. 1 for each task. The numbers and distributions of truly active channels varied for each task. We confirmed that task 1 had a right-lateralized focused activation pattern, task 2 had a left-lateralized moderate activation pattern, and task 3 had a bilateral diffuse activation pattern.

Fig. 1

$t$-Value color maps for each task and channel. $t$-Values that are greater than 3.5, 3.0, 2.5, and 2.0 are indicated by red, orange, yellow, and green circles, respectively, and those lower than 2.0 by white circles. (a) Task 1 is the visual-based oddball task (OBT) evoking right-laterilized focused activation. (b) Task 2 is the verbal category fluency task (CFT) evoking left-lateralized moderately focused activation. (c) Task 3 is the naming task (NMT) evoking bilateral diffused activation.

3.2.

Evaluation of $M_{\mathrm{eff}}$

We plotted the average $M_{\mathrm{eff}}$ values and SD for each number of channels for each task (Fig. 2). The $M$ values, used for correction in the Bonferroni method, were also plotted for comparison. When the number of measurement channels was less than 10, there was little difference between $M_{\mathrm{eff}}$ and $M$ for all tasks. However, when the number of measurement channels exceeded 10, the value of $M_{\mathrm{eff}}$ became gradually controlled, with tapered response curves in all cases. Eventually at around $m=44$, $M_{\mathrm{eff}}$ converged at a range of approximately 10 to 15.

Fig. 2

$M_{\mathrm{eff}}$ values with SD error bars and $M$ for the total number of channels for each task. Task names are as indicated in Fig. 1.

3.3.

Evaluation of Detected Active Channels

For each task, significantly active channels for which $p$-values were less than 0.05 were calculated for different numbers of channels. The number of active channels was derived with no correction, with Bonferroni correction, and with the $M_{\mathrm{eff}}$ method. The number of active channels for each case is plotted in Fig. 3.

Fig. 3

Number of active channels with and without correction by $M_{\mathrm{eff}}$ or $M$ for all channels measured for each task and for each $t$-value threshold. The horizontal axis indicates the number of channels, and the vertical axis indicates the number of detected active channels as exemplified in the bottom left corner. The dashed red lines indicate the number of active channels without correction. The thin blue lines indicate the number of active channels with Bonferroni correction. The bold green lines indicate the number of active channels with $M_{\mathrm{eff}}$ correction.

As shown in Fig. 3, without correction, the number of active channels increased linearly for each task. For a large number (approximately 40) of channels, the number of detected active channels did not inflate linearly but converged to a certain value, which equaled the number of detected active channels with the $M_{\mathrm{eff}}$ method applied to the whole number of channels. Irrespective of the total number of channels and truly active channels initially given, the numbers of detected active channels with $M_{\mathrm{eff}}$ methods remained larger than or equal to those with Bonferroni methods and smaller than or equal to those without correction.

4.1.

Overview

The current study examined the relationship between the number of channels and detected active channels based on FWE correction with the $M_{\mathrm{eff}}$ approach. We conducted resampling simulations applied to actual multichannel fNIRS data for different cognitive tasks. Compared with the conventional Bonferroni method, the $M_{\mathrm{eff}}$ approach generally yielded sufficiently moderate FWE corrections and increased statistical power.

Since the Bonferroni method regards each channel as independent, its multiplicity linearly increased as the number of channels increased. In contrast, the $M_{\mathrm{eff}}$ approach limited the inflation of $M_{\mathrm{eff}}$ when there were, roughly, more than 10 channels. Accordingly, $M_{\mathrm{eff}}$ converged to a certain value, which was specific to task species. Considering these characteristics, $M_{\mathrm{eff}}$ can be considered an effective FWE correction method for modern fNIRS measurement with a large number of channels.

$M_{\mathrm{eff}}$ methods are expected to have a high affinity to fNIRS data. First, they do not require any assumptions about the spatial configuration of channels, in contrast to RFT, which assumes spatial smoothness and continuity in a good-lattice assumption. Sparsely and irregularly distributed fNIRS channels can be treated effectively using the $M_{\mathrm{eff}}$ method. Second, the $M_{\mathrm{eff}}$ method deals with the spatial correlation structure of data holistically, but is not adjusted by local variations of spatial correlation. Thus, $M_{\mathrm{eff}}$ is applicable to data from all channels without local bias. Third, the $M_{\mathrm{eff}}$ method is applicable to a wide variety of data structures including both single- and multiple-subject analyses and to multivariate analyses such as ANOVA.^26,27

4.2.

Interpretation of Results

$M_{\mathrm{eff}}$ increased with the number of channels, but not in a linear manner. The increase ratio went down and finally converged at a range of 10 to 15. Average $M_{\mathrm{eff}}$ and standard deviation at 20 channels were $12.2\pm 0.4$, $8.8\pm 0.3$, and $11.4\pm 0.5$ for OBT, CFT, and NMT, respectively. Since variability (SD) due to different channel selections was negligible, we suggest that the $M_{\mathrm{eff}}$ estimations in resampling simulations could well represent actual channel number increases. Note that relatively small SD at around channel numbers 1 and 44 simply reflect the limited number of channel selections.

For each task, $M_{\mathrm{eff}}$ reached 80% of maximum values at 20 channels. This observation suggests that a substantial $M_{\mathrm{eff}}$ increase beyond 44 channels is not likely to occur and that $M_{\mathrm{eff}}$ converges to a certain level for each task.

As shown in Fig. 3, the lack of FWE correction resulted in an increased number of false positive detected-active channels. This suggests the necessity for Type I error corrections.

Although application of the Bonferroni method maintained a number of active channels similar to the number of truly active channels with a small number of channels, the number of detected active channels tended to drop below that of truly active channels as the total number of channels increased. Hence, we suggest that the current resampling simulations represent the over-correction of FWE, the so-called “Valley of Bonferroni,” often encountered in actual multichannel fNIRS measurements well.

On the other hand, the $M_{\mathrm{eff}}$ approach yielded a number of detected active channels similar to that of truly active channels at a $t$-value threshold of 3.0 even when the overall number of channels increased. This clearly demonstrates that the $M_{\mathrm{eff}}$ approach can reduce Type II errors compared with the conventional Bonferroni method, while maintaining control over Type I errors compared with uncorrected states.

In other words, consideration of spatial correlation among channels and analysis of their eigenvalue can lead to $M_{\mathrm{eff}}$ estimations that are similar to actual multiplicity for FWE correction. Accordingly, the $M_{\mathrm{eff}}$ approach can provide a simple, data-driven FWE correction method that maintains statistical power for multichannel fNIRS and its increasing number of channels.

4.3.

Consideration of Task Species

To explore the applicability of the $M_{\mathrm{eff}}$ approach, we selected three different experimental tasks from previously published studies with the same numbers of channels. This was in order to cover a wide range of activation patterns from a focused and lateralized activation pattern evoked by an OBT in healthy children,²⁹ to a moderately focused and lateralized activation pattern evoked by a CFT in healthy adults,³⁰ and to a broad and less lateralized activation pattern evoked by an NMT in healthy adults.³¹ Our first expectation was to detect changes in $M_{\mathrm{eff}}$ associated with degrees of focus. However, we did not find any tendency between focus size (the number of truly active channels) and $M_{\mathrm{eff}}$. Thus, we tentatively suggest that $M_{\mathrm{eff}}$ is affected by task species, but not by focus size. However, further exploration is essential to draw discrete conclusions. First, the current samples were from different groups and included both adults and children. The true effect may have been buried by sample heterogeneity. In order to explore the factors determining $M_{\mathrm{eff}}$, we have to examine the effects of focus size and task species using the same group of subjects. Despite this, however, $M_{\mathrm{eff}}$ did not vary much among different tasks and subject groups, with values ranging from 10 to 15 out of 44 channels. This leads us to expect that $M_{\mathrm{eff}}$ is robust to different tasks when similar experimental conditions are adopted. In this sense, the $M_{\mathrm{eff}}$ approach could be regarded as a practical option that can be readily adapted to many fNIRS studies.

4.4.

Technical Considerations

For the current analyses, we selected Galwey’s method²⁸ from among several different $M_{\mathrm{eff}}$ methods used in the realm of genetics.^26,27 As represented in Eq. (5) root-mean-square values, Galwey’s method puts weight on large spatial correlations with eigenvalues over 1. Selection of an appropriate multiplier (currently 2) in Galwey’s equation would require further examination.

One inherent merit of the $M_{\mathrm{eff}}$ approach is its wide applicability. Since its core idea lies in determination of $M_{\mathrm{eff}}$ in a data-driven way, it can be combined with other methods such as the FDR method. The combination of $M_{\mathrm{eff}}$ and FDR has been adopted in genetic studies.²⁷ Indeed, the current study itself may be rather interpreted as a comparison of two Bonferroni correction methods with or without data-driven $M_{\mathrm{eff}}$. Thus, how the combination of FDR and $M_{\mathrm{eff}}$ affects the statistical power would be an interesting topic to explore.

Another aspect of the applicability of $M_{\mathrm{eff}}$ is its extension to various experimental designs beyond one-sample $t$-tests. Thus far, two methods with less lenient statistical thresholds than Bonferroni correction have been adopted for fNIRS studies. The first is the D/AP alpha boundary^22–25 for adjusting the spatial correlation among channels which confers effective $M$ values to each fNIRS channel.²³ However, this method has two major limitations: One is that $M$ values differ among channels so that no unified criterion can be realized for multichannel measurements, and the other is that it is limited to a one-sample $t$-test scheme. Similarly, Singh and Dan explored a resampling-based approach based on the Max-T method,^17,21 but this was also limited to a one-sample $t$-test scheme.

On the other hand, the $M_{\mathrm{eff}}$ approach can be adopted for a wide range of experimental designs categorized into a GLM including two-sample $t$-tests, ANOVA, and regression analysis. The former two designs are important for group studies and the latter is essential for individual studies dealing with temporal as well as spatial correlations.

In addition, although the current study limited its focus to channel-wise analyses, the $M_{\mathrm{eff}}$ approach may be applicable to pixel- or voxel-based methods, namely continuous images created by interpolation, image reconstruction, and DOT. Since these continuous images are constructed from channel-wise data, comparison between the two approaches as to how they affect $M_{\mathrm{eff}}$ would be an intriguing topic to investigate.

In this sense, we would expect the current study to be regarded as a primer with the goal of inspiring a wide range of applications for $M_{\mathrm{eff}}$ in order to take fNIRS studies in various directions.