8 Notes

8.1 Keywords

Connectomics, Fingerprinting, Individual Differences, Electroencephalography, functional Magnetic Resonance Imaging, Lexical Decision Task, Resting State

8.2 Research Question

Broadly: How do individual differences in EEG signal (e.g., power spectra, variability and complexity, graph theory metrics) relate to individual differences in behaviour and cognition?

Specifically: Can we (1) identify and differentiate between individuals in a data set purely from their EEG signal? (2) predict individual differences in performance on a task purely from individual differences in EEG signal from before, during, or after the task? Or perhaps from EEG signal from different sessions? (this likely depends on the method we use for identification) (3) is identification and prediction stable across different sessions, and across different “database” and “identification” pairings (e.g., eyes-open rest to task-state; task-state to eyes-closed rest, etc.)?

8.3 Research Need

8.3.1 How is this research unique?

  1. There does not appear to be much EEG fingerprinting research examining persistence of accurate identification across sessions using task state EEG. So far I’ve only found: Armstrong et al. (2015).

  2. Although there is a fair amount of EEG fingerprinting research, there does not appear to be any (?) EEG fingerprinting research that looks at the link between an individual’s fingerprint and their behaviour or cognition. This is likely due to the focus on using EEG fingerprinting for biometric security, rather than for understanding individual differences in EEG activity and what that means for how neural events give rise to cognition and action.

  3. There does not appear to be any (?) EEG fingerprinting research (or fMRI fingerprinting research for that matter) that uses EEG signal variability or complexity as a marker for fingerprinting.

  4. Most fingerprinting studies (whether EEG or fMRI) only use resting state data, so the present study would help to further identify how EEG activity during a task can improve the ratio of within- to between-subject variability, and, more specifically, would test whether the lexical decision task can draw out variation that is relevant to a phenotype of interest.

8.3.2 Why is this research needed?

“There are two main reasons to study individual differences in the course of neuroimaging research. First, of basic scientific concern, more precise descriptions of brain activity in single subjects moves us closer to a mechanistic understanding of how neural events give rise to cognition and action. Second, of practical concern, mapping from individual brains to individual behaviors is a crucial step in developing imaging-based biomarkers with real-world utility.” (Finn et al., 2017)

“Mapping from individual brains to individual behaviors is a crucial step in developing imaging-based biomarkers with real-world utility” seems like a nice way to phrase the real-world importance of my project, since I’ll be working with healthy subjects, but ultimately this work can be a stepping stone to help clinical populations.

“The persistence of individual characteristics in EEG data has yet to be investigated” (Chan et al., 2018).

Call to use complexity or connectivity for EEG fingerprinting by Chan et al. (2018): “Neural complexity can be regarded as irregularity of activity in the brain, which is related to brain functions and information processing (McDonough and Nashiro, 2014). Therefore, differences in the complexity of data embedded in EEG can be used to differentiate among entities. Entropy is one measure by which to assess the complexity of the brain, and it has also been used in person authentication (Mu et al., 2016). Calculating entropy at various temporal scales makes it possible to formulate a complete description of the non-linearity in EEG signals (Gao et al., 2015). We suggest using both linear and non-linear features of EEG signals with different temporal scales as a means of increasing accuracy. … The second approach is the use of connectivity in the human brain, including functional and effective connectivity (Friston, 2011), to assess the interactions between various regions of the brain.”

“Much previous work has been concerned with separating the functional connectivity signal into state versus trait components: for example, by investigating the test-retest reliability of resting-state connectivity within individuals (depending on the choices of connectivity analysis and reliability metric, it’s generally low to moderate (Birn et al., 2013; Braun et al., 2012; Shehzad et al., 2009; Zuo and Xing, 2014), or comparing group-averaged network organization across brain states (it’s grossly similar between rest and any of several tasks (Cole 10 et al., 2014; Smith et al., 2009)). However, there is little work at the intersection of these lines of research, investigating how brain state affects test-retest reliability of single subjects, or measurements of individual differences across subjects.” (Finn et al., 2017)

“The end goal of most neuroimaging research into individual differences is to relate brain measures to behavior. Ultimately, we should be striving to not simply report correlations between connectivity patterns and phenotypes, but to build predictive models that can take in neuroimaging information from a previously unseen subject and predict something about their present or future behavior, such as cognitive ability, risk for illness, or response to treatment, to name a few (Bach et al., 2013; Gabrieli et al., 2015; Lo et al., 2015; Whelan and Garavan, 2014). With this in mind, the question is not which states maximize the ratio of within- to between-subject variability in and of itself, but rather which states maximize this ratio while drawing out variation that is relevant to a phenotype of interest. To this end, investigators might try varying the input to connectivity-based predictive models (Shen et al., in press) by testing several states and determining which yields the best behavioral prediction, using data sets such as HCP and others that contain multiple scan conditions and behavioral measures for each subject.” (Finn et al., 2017)

“There is growing recognition in human neuroscience that mean-based approaches, in which data are averaged across many individuals, may obscure more than they reveal about brain-behavior relationships (Speelman and McGann, 2013). In fact, the perceived “universality” of functional brain regions and networks may be more an epiphenomenon of how we analyze our data than a reflection of how individual brains actually work. … It is worth exploring alternative paradigms for individual differences research, as other brain states may afford an improved ratio of within- to between-subject variability, and/or enhance the individual connectivity signature in networks of interest. A better understanding of how brain state affects measurements of individual differences is important from a cognitive neuroscience perspective, and may increase our chances of finding imaging-based biomarkers with translational utility." (Finn et al., 2017)

8.5 Fingerprinting Methodology

Identification experiments often require that a given connectivity profile acquired in one session is matched to the same individual’s profile (i.e., you need an index to compare individuals to).

“EEG-based biometrics are applicable to person recognition applications, including identification and authentication systems. Personal identification systems predict the identity of a user from among all enrolled clients, whereas authentication systems validate the identity claimed by a user. Despite differing purposes, both systems make decisions based on the EEG features of the user and all clients in the database and therefore share the following four components: a database, an EEG acquisition system, a signal preprocessing system, and a feature extraction system. … The performance of EEG-based person recognition systems relies on the design of signal acquisition protocols, feature extraction methods, and classification techniques.” (Chan et al., 2018).

“Databases impose four basic challenges: (1) number of users: as the number of subjects increases, it becomes increasingly difficult for the system to accurately classify users. Previous studies have used between 3 and 120 participants. … (3) Variations at the individual level: most previous studies collected data from individual subjects once only. However, as mentioned previously, the brain changes over time and one’s mental state can have a tremendous influence on brain activity.” (Chan et al., 2018).

“Generally, protocols involving tasks are more reproducible than those without tasks, such as resting-state brain signals. Furthermore, protocols that involve simple tasks based on sensory inputs are more reproducible than those with complex tasks requiring cognitive processing. However, there is a tradeoff between reproducibility and distinctiveness. Protocols capable of evoking brain activity patterns with personal characteristics are regarded as particularly suitable. Brain activity patterns generated during cognitive tasks are distinctive between individuals, which makes them useful in biometric systems; however, these tasks also tend to be time-consuming.” (Chan et al., 2018).

In evaluating fingerprinting accuracy it is important that: “(1) Testing data should be independent or nearly independent from the training data. … (2) Additional indices of performance should be reported. … (3) Researchers report the duration of the EEG data used or the information transfer rate (ITR). … (4) In studies with a small sample size, the accuracy of results should be tested using statistical methods that take the size of the sample as well as variations between subjects into account.” (Chan et al., 2018).

“The problem of template aging can be overcome by ensuring the completeness of the representation for each enrolled client prior to the training of classifiers. This means that for each individual, EEG data obtained under various conditions and at different times should be included in the training data sets. … e.g., In recent developments of biometrics in smartphone authentication, the models used for face or fingerprint recognition are adjusted during every login procedure. … We therefore strongly recommend the use of longitudinal EEG acquisition and performance evaluation during training steps in order to improve the temporal persistence of EEG-based biometric systems.” (Chan et al., 2018).

“EEG measurements can be affected by psychological and physiological factors. … Thus, developing a stable and effective EEG-based biometric system requires an understanding of the factors affecting EEG, as well as a means of selecting EEG features with high stability and distinctiveness. … This model also makes it possible to predict changes in features, thereby enabling the biometric system to maintain high recognition accuracy over time. … The accumulation of abundant knowledge concerning the influence of various factors makes it possible for researchers to build a model capable of making accurate predictions of EEG features under varying conditions. The use of a prediction model in conjunction with feature augmentation could greatly reduce the time required to obtain training data.” (Chan et al., 2018).

“Hypothetically, what would be the ideal condition for measuring individual differences? Simply maximizing between-subject variability—i.e., making subjects look as different as possible from one another—is not necessarily the answer. Rather, the optimal condition should make subjects look as different as possible while also retaining the most important features of each individual. … There are two ways to boost identifiability (a metric that is, in this case, subjective, but is a proxy for trait-level variance associated with some phenotype of interest): by exaggerating the most prominent features of each individual (the “caricature” condition), which incidentally makes subjects look more different from one another, or by blurring irrelevant features while retaining and enriching relevant ones (“selective enhancement”)." (Finn et al., 2017)

“Certain in-scanner tasks could act as neuropsychiatric “stress tests” to enhance individual differences in the general population, or, in at-risk individuals, to reveal abnormal patterns of brain activity before they show up at rest. While a variety of tasks might serve this role, naturalistic paradigms—e.g., having subjects watch a movie or listen to a story in the scanner—are especially intriguing candidates. By imposing a standardized yet engaging stimulus on all subjects, naturalistic tasks evoke rich patterns of brain activity. These patterns lend themselves to functional connectivity analysis as well as other data-driven techniques such as inter-subject correlation (ISC) (Hasson et al., 2004) and inter-subject functional connectivity (ISFC) (Simony et al., 2016), which are model-free ways to isolate stimulus-dependent brain activity from spontaneous activity and noise. … While naturalistic tasks are promising, any task that elicits variable brain activity and/or behavior across subjects is a worthwhile candidate." (Finn et al., 2017)

“Much of the variance in functional connectivity is intrinsic to an individual, and not associated with how the brain is engaged during scanning. Similar results have now been reported using other data sets (Airan et al., 2016). … However, our exploratory analysis showed that between-subject variability does, in fact, change with brain state. Overall, individuals tended to look more similar during tasks than during rest. Additionally, in nearly all conditions, females were more similar to other females than males were to other males. … males. These results suggest that investigators should take into account sex when choosing conditions to maximize individual variability: tasks that maximize variability in females may not do the same for males, and vice versa. This may be especially important when studying traits relevant to neuropsychiatric illnesses that disproportionately affect one sex or the other.” (Finn et al., 2017)

“Individual differences do, in fact, change depending on the condition in which they are measured. … Interestingly, conditions that make subjects look more similar to one another also make better databases in identification experiments. … The high correlation between database score and between-subject similarity suggests that task conditions mainly serve to reduce intra-individual variability, while preserving meaningful inter-individual variance (i.e., variance related to true individual differences).” (Finn et al., 2017)

“Of note, linking imaging measures to behavior crucially depends on choosing the right behavior.” (Finn et al., 2017)

8.6 Resting State Studies

8.6.1 EEG Single Session

La Rocca et al. (2014) proposed a person identification system using functional connectivity during eyes-closed (EC) and eyes-open (EO) conditions as features. They achieved 100% recognition accuracy among 108 subjects.

Thomas & Vinod (2018) proposed a person authentication system using the power spectrum density (PSD) of resting-state EEG signals as features. They achieved an equal error rate (EER) of just 0.008 among 70 subjects.

8.6.2 EEG Longitudinal

Marcel & Millan (2007) found that the half total error rate (HTER) of their EEG-based authentication system increased from 7.1 to 36.2 within just 3 days.

Hu, Liu, Zhao, Qi, & Peng (2011), found that the true positive rate (TAR) after a 1-day span was 94.60%; however, this dropped to 83.64% after a span of 1 week and to 78.20% after 6 months.

Kostílek & Št’astný (2012) proposed a person authentication system using movement-related EEG signals during rest. They achieved up to 98% recognition accuracy for a single session, and up to 87.1% accuracy approximately one year later.

Maiorana, La Rocca, & Campisi (2016) proposed a person authentication system using AR modelling, power spectrum density, and coherence of eyes-closed (EC) and eyes-open (EO) resting-state EEG signals as features. They achieved up to recognition accuracy for a single session, one week later, and 34 days later (no percentage given).

8.7 Task State Studies

8.7.1 EEG Single Session

Ruiz-Blondet, Jin, & Laszlo (2016) proposed a person identification system using tasks known to elicit event-related potentials (ERPs) from various functional brain systems. They achieved 100% recognition among 50 subjects.

8.7.2 EEG Longitudinal

Armstrong et al. (2015) proposed a person authentication system using several pattern classifiers applied to event-related potentials (ERPs) representing the response of individuals to a stream of text designed to be idiosyncratically familiar to different individuals. They achieved 97% recognition accuracy among 45 subjects. This dropped to 89% accuracy among 30 subjects after 5-40 days, and rose to 93% accuracy among 15 subjects after 134-188 days.

8.7.3 fMRI Single Session

Finn et al. (2015) proposed a person identification system using functional connectivity during rest and task states across pairs of scans consisting of one “target” and one “database” session, with the requirement that the target and database sessions be taken from different days. They achieved 92.9% identification accuracy among 126 subjects using the whole-brain connectivity matrix.

Amico & Goñi (2018) proposed a person identification system using principle component analysis (PCA) to improve the individual fingerprint in functional connectomes of rest and task state from a group-level perspective. They achieved identification accuracy ranging from 92%-98% among 100 subjects.

8.7.4 fMRI Longitudinal

Zhang, Kranz, & Lee (2019) proposed a person identification system using phase synchrony during resting state. They achieved 86% identification accuracy among 205 subjects scanned after an average of 2.63 years.

8.8 Fingerprinting, Behaviour, and Cognition

Finn et al. (2015) …

Biazoli et al. (2017) proposed a person identification system using single subject connectome information (whole brain functional connectivity) included directly into their model, testing whether the information conveyed by a given connectome fingerprint could predict the intelligence quotient, cognitive function and also emotional and behavioral problems at the individual level. Among 655 subjects, they found that individuals with similar connectomes had (i) similar ages; (ii) similar intelligence quotients; (iii) analogous performance on tests measuring executive function; (iv) and similar levels of behavioral problems.

8.9 Misc

What will the study look like?

How is the study unique?

get a one paragraph description of what my thesis is about. then send that to andrea. focus on big picture and differentiate it from the existing literature—if people have done a lot of work linking fingerprinting to cognition then we have to work around how the project is unique; if there isn’t much work linking it to cognition then we have more room to stake out room for the study. And LDT is a good task to link to performance since it’s well understood. Sending a draft that doesn’t have to be perfect.

tie brain variability/complexity into fingerprinting; ask Amir about his recently published graph theory paper

look at EEG fingerprinting papers and identify the best methods

How does the brain switch from task to resting, and the switch between the two. and we can justify this with Finn’s manipulating brain state paper

EEG fingerprinting:

  • Albuquerque, Damaševičius, Tavares, & Pinheiro (2018);
  • Chan et al. (2018);
  • Demuru & Fraschini (2020);
  • Fraschini, Pani, Didaci, & Marcialis (2019);
  • Kong, Wang, Xu, Babiloni, & Chen (2019);
  • La Rocca, Campisi, & Scarano (2012)

fMRI fingerprinting:

  • Finn et al. (2015)

manipulating brain state:

  • Finn et al. (2017)

References

Albuquerque, V. H. C. de, Damaševičius, R., Tavares, J. M. R. S., & Pinheiro, P. R. (2018). EEG-Based Biometrics: Challenges And Applications. Computational Intelligence and Neuroscience, 2018, e5483921. https://doi.org/https://doi.org/10.1155/2018/5483921

Amico, E., & Goñi, J. (2018). The quest for identifiability in human functional connectomes. Scientific Reports, 8(1), 8254. https://doi.org/10.1038/s41598-018-25089-1

Armstrong, B. C., Ruiz-Blondet, M. V., Khalifian, N., Kurtz, K. J., Jin, Z., & Laszlo, S. (2015). Brainprint: Assessing the uniqueness, collectability, and permanence of a novel method for ERP biometrics. Neurocomputing, 166, 59–67. https://doi.org/10.1016/j.neucom.2015.04.025

Biazoli, C. E., Salum, G. A., Pan, P. M., Zugman, A., Amaro, E., Rohde, L. A., … Sato, J. R. (2017). Commentary: Functional connectome fingerprint: identifying individuals using patterns of brain connectivity. Frontiers in Human Neuroscience, 11, 47. https://doi.org/10.3389/fnhum.2017.00047

Chan, H.-L., Kuo, P.-C., Cheng, C.-Y., & Chen, Y.-S. (2018). Challenges and Future Perspectives on Electroencephalogram-Based Biometrics in Person Recognition. Frontiers in Neuroinformatics, 12. https://doi.org/10.3389/fninf.2018.00066

Demuru, M., & Fraschini, M. (2020). EEG fingerprinting: Subject-specific signature based on the aperiodic component of power spectrum. Computers in Biology and Medicine, 120, 103748. https://doi.org/10.1016/j.compbiomed.2020.103748

Finn, E. S., Scheinost, D., Finn, D. M., Shen, X., Papademetris, X., & Constable, R. T. (2017). Can brain state be manipulated to emphasize individual differences in functional connectivity? NeuroImage, 160, 140–151. https://doi.org/10.1016/j.neuroimage.2017.03.064

Finn, E. S., Shen, X., Scheinost, D., Rosenberg, M. D., Huang, J., Chun, M. M., … Constable, R. T. (2015). Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nature Neuroscience, 18(11), 1664–1671. https://doi.org/10.1038/nn.4135

Fraschini, M., Pani, S. M., Didaci, L., & Marcialis, G. L. (2019). Robustness of functional connectivity metrics for EEG-based personal identification over task-induced intra-class and inter-class variations. Pattern Recognition Letters, 125, 49–54. https://doi.org/10.1016/j.patrec.2019.03.025

Hu, B., Liu, Q., Zhao, Q., Qi, Y., & Peng, H. (2011). A Real-Time Electroencephalogram (EEG) Based Individual Identification Interface for Mobile Security in Ubiquitous Environment. 2011 IEEE Asia-Pacific Services Computing Conference, 436–441. https://doi.org/10.1109/APSCC.2011.87

Kong, W., Wang, L., Xu, S., Babiloni, F., & Chen, H. (2019). EEG Fingerprints: Phase Synchronization of EEG Signals as Biomarker for Subject Identification. IEEE Access, 7, 121165–121173. https://doi.org/10.1109/ACCESS.2019.2931624

Kostílek, M., & Št’astný, J. (2012). EEG biometric identification: Repeatability and influence of movement-related EEG. 2012 International Conference on Applied Electronics, 147–150.

La Rocca, D., Campisi, P., & Scarano, G. (2012). EEG biometrics for individual recognition in resting state with closed eyes. 2012 BIOSIG - Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG), 1–12.

La Rocca, D., Campisi, P., Vegso, B., Cserti, P., Kozmann, G., Babiloni, F., & Fallani, F. D. V. (2014). Human Brain Distinctiveness Based on EEG Spectral Coherence Connectivity. IEEE Transactions on Biomedical Engineering, 61(9), 2406–2412. https://doi.org/10.1109/TBME.2014.2317881

Maiorana, E., La Rocca, D., & Campisi, P. (2016). On the Permanence of EEG Signals for Biometric Recognition. IEEE Transactions on Information Forensics and Security, 11(1), 163–175. https://doi.org/10.1109/TIFS.2015.2481870

Marcel, S., & Millan, J. D. R. (2007). Person Authentication Using Brainwaves (EEG) and Maximum A Posteriori Model Adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4), 743–752. https://doi.org/10.1109/TPAMI.2007.1012

Ruiz-Blondet, M. V., Jin, Z., & Laszlo, S. (2016). CEREBRE: A Novel Method for Very High Accuracy Event-Related Potential Biometric Identification. IEEE Transactions on Information Forensics and Security, 11(7), 1618–1629. https://doi.org/10.1109/TIFS.2016.2543524

Thomas, K. P., & Vinod, A. P. (2018). EEG-Based Biometric Authentication Using Gamma Band Power During Rest State. Circuits, Systems, and Signal Processing, 37(1), 277–289. https://doi.org/10.1007/s00034-017-0551-4

Zhang, R., Kranz, G. S., & Lee, T. M. C. (2019). Functional Connectome from Phase Synchrony at Resting State is a Neural Fingerprint. Brain Connectivity, 9(7), 519–528. https://doi.org/10.1089/brain.2018.0657