Exploring Electroencephalography-Based Affective Analysis and Detection of Parkinson’s Disease

Exploring Electroencephalography-Based Affective Analysis and Detection of Parkinson’s Disease Ravikiran Parameshwara Human-Centred Technology Research Cluster, University of Canberra, Canberra, Australia. https://orcid.org/0000-0003-0605-888X Soujanya Narayana Human-Centred Technology Research Cluster, University of Canberra, Canberra, Australia. https://orcid.org/0000-0002-7972-978X Murugappan Murugappan Department of Electronics and Communication Engineering, Kuwait College of Science and Technology, Kuwait City, Kuwait. Department of Electronics and Communication Engineering, Vels Institute of Sciences, Technology, and Advanced Studies, Chennai, India. Centre for Excellence in Unmanned Aerial Systems, Universiti Malaysia Perlis, Perlis, Malaysia. Ibrahim Radwan Human-Centred Technology Research Cluster, University of Canberra, Canberra, Australia. Roland Goecke Human-Centred Technology Research Cluster, University of Canberra, Canberra, Australia. Ramanathan Subramanian Human-Centred Technology Research Cluster, University of Canberra, Canberra, Australia. https://orcid.org/0000-0001-9441-7074

While Parkinson’s disease (PD) is typically characterized by motor disorder, there is also evidence of diminished emotion perception in PD patients. This study examines the utility of electroencephalography (EEG) signals to understand emotional differences between PD and healthy controls (HCs), and for automated PD detection. Employing traditional machine learning and deep learning methods on multiple EEG descriptors, we explore (a) dimensional and categorical emotion recognition and (b) PD versus HC classification from multiple descriptors characterizing emotional EEG signals. Our results reveal that PD patients comprehend arousal better than valence and, among emotion categories, fear, disgust, and surprise less accurately, and sadness most accurately. Mislabeling analyses confirm confounds among opposite-valence emotions for PD data. Emotional EEG responses also achieve near-perfect PD versus HC recognition. Cumulatively, our study demonstrates that (a) examining implicit responses alone enables (i) discovery of valence-related impairments in PD patients and (ii) differentiation of PD from HC and that (b) emotional EEG analysis is an ecologically valid, effective, practical, and sustainable tool for PD diagnosis vis-à-vis self-reports, expert assessments, and resting-state analysis.
10 17 2024 01 2024 0084 10.34133/icomputing.0084 1 10.34133/aaas_crossmark_01 spj.science.org true 2023-07-20 2023-12-12 2024-02-08 2024-10-17 Australian Research Council http://dx.doi.org/10.13039/501100000923 DP190101294 10.34133/icomputing.0084 https://spj.science.org/doi/10.34133/icomputing.0084 https://spj.science.org/doi/pdf/10.34133/icomputing.0084 10.1016/S0896-6273(03)00568-3 Neurol Sci Trojano L 53 39 1 2018 10.1007/s10072-017-3155-7 Cognitive and behavioral disorders in Parkinson’s disease: An update. II: Behavioral disorders Trojano L, Papagno C. Cognitive and behavioral disorders in Parkinson’s disease: An update. II: Behavioral disorders. Neurol Sci. 2018;39(1):53–61. Bioeng Transl Med Chopade P Article e10367 8 1 2023 10.1002/btm2.10367 Alzheimer’s and Parkinson’s disease therapies in the clinic Chopade P, Chopade N, Zhao Z, Mitragotri S, Liao R, Chandran SV. Alzheimer’s and Parkinson’s disease therapies in the clinic. Bioeng Transl Med. 2023;8(1): Article e10367. Front Aging Neurosci Wang Q 373 12 2020 10.3389/fnagi.2020.587396 Characterization of EEG data revealing relationships with cognitive and motor symptoms in Parkinson’s disease: A systematic review Wang Q, Meng L, Pang J, Zhu X, Ming D. Characterization of EEG data revealing relationships with cognitive and motor symptoms in Parkinson’s disease: A systematic review. Front Aging Neurosci. 2020;12:373. Front Aging Neurosci Chaturvedi M 9 2017 Quantitative EEG (QEEG) measures differentiate Parkinson’s disease (PD) patients from healthy controls (HC) Chaturvedi M, Hatz F, Gschwandtner U, Bogaarts JG, Meyer A, Fuhr P, Roth V. Quantitative EEG (QEEG) measures differentiate Parkinson’s disease (PD) patients from healthy controls (HC). Front Aging Neurosci. 2017;9. Neural Comput Appl Oh SL 10927 32 2020 10.1007/s00521-018-3689-5 A deep learning approach for Parkinson’s disease diagnosis from EEG signals Oh SL, Hagiwara Y, Raghavendra U, Yuvaraj R, Arunkumar N, Murugappan M, Acharya UR. A deep learning approach for Parkinson’s disease diagnosis from EEG signals. Neural Comput Appl. 2020;32:10927–10933. IEEE Trans Affect Comput Koelstra S 18 3 1 2012 10.1109/T-AFFC.2011.15 DEAP: A database for emotion analysis using physiological signals Koelstra S, Muhl C, Soleymani M, Jong-Seok Lee, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I. DEAP: A database for emotion analysis using physiological signals. IEEE Trans Affect Comput. 2012;3(1):18–31. Behav Brain Res Yuvaraj R 248 298 2016 10.1016/j.bbr.2015.10.036 Brain functional connectivity patterns for emotional state classification in Parkinson’s disease patients without dementia Yuvaraj R, Murugappan M, Acharya UR, Adeli H, Ibrahim NM, Mesquita E. Brain functional connectivity patterns for emotional state classification in Parkinson’s disease patients without dementia. Behav Brain Res. 2016;298:248–260. IEEE Trans Affect Comput Abadi MK 209 6 3 2015 10.1109/TAFFC.2015.2392932 DECAF: MEG-based multimodal database for decoding affective physiological responses Abadi MK, Subramanian R, Kia SM, Avesani P, Patras I, Sebe N. DECAF: MEG-based multimodal database for decoding affective physiological responses. IEEE Trans Affect Comput. 2015;6(3):209–222. J Pers Soc Psychol Russell JA 1161 39 6 1980 10.1037/h0077714 A circumplex model of affect Russell JA. A circumplex model of affect. J Pers Soc Psychol. 1980;39(6):1161–1178. 10.1080/13803391003596371 Neuropsychologia Clark US 2300 46 9 2008 10.1016/j.neuropsychologia.2008.03.014 Specific impairments in the recognition of emotional facial expressions in Parkinson’s disease Clark US, Neargarder S, Cronin-Golomb A. Specific impairments in the recognition of emotional facial expressions in Parkinson’s disease. Neuropsychologia. 2008;46(9):2300–2309. Neuropsychologia Miller K 1917 47 8 2009 10.1016/j.neuropsychologia.2009.03.002 Startle reflex hyporeactivity in Parkinson’s disease: An emotion-specific or arousal-modulated deficit? Miller K, Okun M, Marsiske M, Fennell E, Bowers D. Startle reflex hyporeactivity in Parkinson’s disease: An emotion-specific or arousal-modulated deficit? Neuropsychologia. 2009;47(8–9):1917–1927. Philos Trans R Soc Lond B Biol Sci RJR B 561 358 1431 2003 10.1098/rstb.2002.1220 Facial expressions, their communicatory functions and neuro–cognitive substrates RJR B. Facial expressions, their communicatory functions and neuro–cognitive substrates. Philos Trans R Soc Lond B Biol Sci. 2003;358(1431):561–572. 10.1145/3536220.3563685 Narayana S Subramanian R Radwan I Goecke R. To improve is to change: Towards improving mood prediction by learning changes in emotion. Paper presented at: Companion Publication of the 2022 International Conference on Multimodal Interaction; 2022; Bengaluru India. IEEE Trans Affect Comput Subramanian R 147 9 2 2018 10.1109/TAFFC.2016.2625250 ASCERTAIN: Emotion and personality recognition using commercial sensors Subramanian R, Wache J, Abadi MK, Vieriu RL, Winkler S, Sebe N. ASCERTAIN: Emotion and personality recognition using commercial sensors. IEEE Trans Affect Comput. 2018;9(2):147–160. IEEE Trans Affect Comput Shukla A 781 13 2 2022 10.1109/TAFFC.2020.2964549 Recognition of advertisement emotions with application to computational advertising Shukla A, Gullapuram SS, Katti H, Kankanhalli M, Winkler S, Subramanian R. Recognition of advertisement emotions with application to computational advertising. IEEE Trans Affect Comput. 2022;13(2):781–792. IEEE Trans Affect Comput Petrantonakis PC 81 1 2 2010 10.1109/T-AFFC.2010.7 Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis Petrantonakis PC, Hadjileontiadis LJ. Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis. IEEE Trans Affect Comput. 2010;1(2):81–97. J Psychophysiol Balconi M 41 67 1 2008 10.1016/j.ijpsycho.2007.10.002 Consciousness and arousal effects on emotional face processing as revealed by brain oscillations. A gamma band analysis. Int’l Balconi M, Lucchiari C. Consciousness and arousal effects on emotional face processing as revealed by brain oscillations. A gamma band analysis. Int’l. J Psychophysiol. 2008;67(1):41–46. Bashivan P Rish I Yeasin M Codella N. Learning representations from EEG with deep recurrent-convolutional neural networks. Paper presented at: International Conference on Learning Representations; 2016; San Juan Puerto Rico. 10.1016/B978-0-12-814597-5.00006-0 Ai Q Liu Q Meng W Xie SQ. Chapter 6—EEG-based brain intention recognition. In: Ai Q Liu Q Meng W Xie SQ editors. Advanced rehabilitative technology. Cambridge (MA): Academic Press; 2018. p. 135–166. Expert Syst Bhurane AA Article e12472 39 7 2019 10.1111/exsy.12472 Diagnosis of Parkinson’s disease from electroencephalography signals using linear and self-similarity features Bhurane AA, Dhok S, Sharma M, Yuvaraj R, Murugappan M, Acharya UR. Diagnosis of Parkinson’s disease from electroencephalography signals using linear and self-similarity features. Expert Syst. 2019;39(7): Article e12472. Parkinsonism Relat Disord Mantri S 56 67 2019 10.1016/j.parkreldis.2019.09.004 Comparing self-reported and objective monitoring of physical activity in Parkinson disease Mantri S, Wood S, Duda JE, Morley JF. Comparing self-reported and objective monitoring of physical activity in Parkinson disease. Parkinsonism Relat Disord. 2019;67:56–59. Exp Neurol Hsu YT 253 233 1 2012 10.1016/j.expneurol.2011.10.013 The role of the sub-thalamic nucleus in the preparation of volitional movement termination in Parkinson’s disease Hsu YT, Lai HY, Chang YC, Chiou SM, Lu MK, Lin YC, Liu YL, Chen CC, Huang HC, Chien TF, et al. The role of the sub-thalamic nucleus in the preparation of volitional movement termination in Parkinson’s disease. Exp Neurol. 2012;233(1):253–263. Neuropsychology Gray HM 176 24 2 2010 10.1037/a0018104 A meta-analysis of performance on emotion recognition tasks in Parkinson’s disease Gray HM, Tickle-Degnen L. A meta-analysis of performance on emotion recognition tasks in Parkinson’s disease. Neuropsychology. 2010;24(2):176. Cortex Kan Y 623 38 4 2002 10.1016/S0010-9452(08)70026-1 Recognition of emotion from facial, prosodic and written verbal stimuli in Parkinson’s disease Kan Y, Kawamura M, Hasegawa Y, Mochizuki S, Nakamura K. Recognition of emotion from facial, prosodic and written verbal stimuli in Parkinson’s disease. Cortex. 2002;38(4):623–630. 10.1016/j.neuropsychologia.2012.05.020 Neuropsychologia Narme P 3295 49 12 2011 10.1016/j.neuropsychologia.2011.08.002 Understanding facial emotion perception in Parkinson’s disease: The role of configural processing Narme P, Bonnet AM, Dubois B, Chaby L. Understanding facial emotion perception in Parkinson’s disease: The role of configural processing. Neuropsychologia. 2011;49(12):3295–3302. Behav Neurol Lin CY 9287092 2016 2016 Degraded impairment of emotion recognition in Parkinson’s disease extends from negative to positive emotions Lin CY, Tien YM, Huang JT, Tsai CH, Hsu LC. Degraded impairment of emotion recognition in Parkinson’s disease extends from negative to positive emotions. Behav Neurol. 2016;2016:9287092. Neuropsychiatr Dis Treat Jin Y 2891 13 2017 10.2147/NDT.S149227 Altered emotional recognition and expression in patients with Parkinson’s disease Jin Y, Mao Z, Ling Z, Xu X, Zhang Z, Yu X. Altered emotional recognition and expression in patients with Parkinson’s disease. Neuropsychiatr Dis Treat. 2017;13:2891–2902. 10.1145/3136755.3136790 Bilalpur M Kia SM Chawla M Chua TS Subramanian R. Gender and emotion recognition with implicit user signals. Paper presented at: International Conference on Multimodal Interaction; 2017; Glasgow UK. PLOS ONE Wabnegger A Article e0136110 10 8 2015 10.1371/journal.pone.0136110 Facial emotion recognition in Parkinson’s disease: An fMRI investigation Wabnegger A, Ille R, Schwingenschuh P, Katschnig-Winter P, Kögl-Wallner M, Wenzel K, Schienle A. Facial emotion recognition in Parkinson’s disease: An fMRI investigation. PLOS ONE. 2015;10(8): Article e0136110. PLOS ONE Moonen AJ Article e0177085 12 5 2017 10.1371/journal.pone.0177085 An fMRI study into emotional processing in Parkinson’s disease: Does increased medial prefrontal activation compensate for striatal dysfunction? Moonen AJ, Weiss PH, Wiesing M, Weidner R, Fink GR, Reijnders JSAM, Weber WM, Leentjens AFG. An fMRI study into emotional processing in Parkinson’s disease: Does increased medial prefrontal activation compensate for striatal dysfunction? PLOS ONE. 2017;12(5): Article e0177085. Comput Math Methods Med Wu P 427826 2014 2014 Objectifying facial expressivity assessment of Parkinson’s patients: Preliminary study Wu P, Gonzalez I, Patsis G, Jiang D, Sahli H, Kerckhofs E, Vandekerckhove M. Objectifying facial expressivity assessment of Parkinson’s patients: Preliminary study. Comput Math Methods Med. 2014;2014:427826. J Neurosci Tessitore A 9099 22 20 2002 10.1523/JNEUROSCI.22-20-09099.2002 Dopamine modulates the response of the human amygdala: A study in Parkinson’s disease Tessitore A, Hariri AR, Fera F, Smith WG, Chase TN, Hyde TM, Weinberger DR, Mattay VS. Dopamine modulates the response of the human amygdala: A study in Parkinson’s disease. J Neurosci. 2002;22(20):9099–9103. Clin Neurophysiol Mirowski P 1927 120 11 2009 10.1016/j.clinph.2009.09.002 Classification of patterns of EEG synchronization for seizure prediction Mirowski P, Madhavan D, LeCun Y, Kuzniecky R. Classification of patterns of EEG synchronization for seizure prediction. Clin Neurophysiol. 2009;120(11):1927–1940. Artif Intell Med Qureshi MNI 10 98 2019 10.1016/j.artmed.2019.06.003 3D-CNN based discrimination of schizophrenia using resting-state fMRI Qureshi MNI, Oh J, Lee B. 3D-CNN based discrimination of schizophrenia using resting-state fMRI. Artif Intell Med. 2019;98:10–17. Front Neurosci Plis SM 229 8 2014 Deep learning for neuroimaging: A validation study Plis SM, Hjelm DR, Salakhutdinov R, Allen EA, Bockholt HJ, Long JD, Johnson HJ, Paulsen JS, Turner JA, Calhoun VD. Deep learning for neuroimaging: A validation study. Front Neurosci. 2014;8:229. J Integr Neurosci Yuvaraj R 89 13 1 2014 10.1142/S021963521450006X Emotion classification in Parkinson’s disease by higher-order spectra and power spectrum features using EEG signals: A comparative study Yuvaraj R, Murugappan M, Ibrahim NM, Omar MI, Sundaraj K, Mohamad K, Palaniappan R, Satiyan M. Emotion classification in Parkinson’s disease by higher-order spectra and power spectrum features using EEG signals: A comparative study. J Integr Neurosci. 2014;13(1):89–120. Int J Psychophysiol Yuvaraj R 482 94 3 2014 10.1016/j.ijpsycho.2014.07.014 Optimal set of EEG features for emotional state classification and trajectory visualization in Parkinson’s disease Yuvaraj R, Murugappan M, Ibrahim NM, Sundaraj K, Omar MI, Mohamad K, Palaniappan R. Optimal set of EEG features for emotional state classification and trajectory visualization in Parkinson’s disease. Int J Psychophysiol. 2014;94(3):482–495. Cogn Neurodyn Yuvaraj R 225 10 3 2016 10.1007/s11571-016-9375-3 Hemispheric asymmetry non-linear analysis of EEG during emotional responses from idiopathic Parkinson’s disease patients Yuvaraj R, Murugappan M. Hemispheric asymmetry non-linear analysis of EEG during emotional responses from idiopathic Parkinson’s disease patients. Cogn Neurodyn. 2016;10(3):225–234. 10.1109/GlobalSIP45357.2019.8969309 Lee S Hussein R McKeown MJ. A deep convolutional-recurrent neural network architecture for Parkinson’s disease EEG classification. Paper presented at: 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP); 2019; Ottawa Canada. Bradley MM Lang PJ. The International Affective Digitized Sounds: Affective ratings of sounds and instruction manual (Technical Report No. B-3). Gainesville (FL): University of Florida; 2007. NIMH CSEA Lang PJ 3 1 1997 International affective picture system (IAPS): Technical manual and affective ratings Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): Technical manual and affective ratings. NIMH CSEA. 1997;1:3. 10.1016/j.tins.2011.09.003 10.5772/4754 Kim J. Bimodal emotion recognition using speech and physiological changes. In: Robust speech recognition and understanding. Rijeka (Croatia): InTech; 2007. Vol. 265; p. 280. J Neural Eng Roy Y Article 051001 16 5 2019 Deep learning based electroencephalography analysis: A systematic review Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J. Deep learning based electroencephalography analysis: A systematic review. J Neural Eng. 2019;16(5): Article 051001. IEEE Trans Rehabil Eng Ramoser H 441 8 4 2000 10.1109/86.895946 Optimal spatial filtering of single trial EEG during imagined hand movement Ramoser H, Muller-Gerking J, Pfurtscheller G. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng. 2000;8(4):441–446. Clin Neurophysiol Müller-Gerking J 787 110 5 1999 10.1016/S1388-2457(98)00038-8 Designing optimal spatial filters for single-trial EEG classification in a movement task Müller-Gerking J, Pfurtscheller G, Flyvbjerg H. Designing optimal spatial filters for single-trial EEG classification in a movement task. Clin Neurophysiol. 1999;110(5):787–798. 10.1109/TIT.1967.1053964 10.1007/BF00994018 John GH Langley P. Estimating continuous distributions in Bayesian classifiers. arXiv. 2013. https://doi.org/10.48550/arXiv.1302.4964. Breiman L Friedman JH Olshen RA Stone C. Classification and regression trees. Monterey (CA): Wadsworth and Brooks/Cole; 1984. 10.4324/9780203774441 Cohen J Cohen P West SG Aiken LS. Applied multiple regression/correlation analysis for the behavioral sciences. New York (NY): Routledge; 2013. Stud Hist Philos Sci C: Stud Hist Philos Biol Biomed Sci Cramer JS 613 35 4 2004 The early origins of the logit model Cramer JS. The early origins of the logit model. Stud Hist Philos Sci C: Stud Hist Philos Biol Biomed Sci. 2004;35(4):613–626. 10.1145/3394171.3413700 Chugh K Gupta P Dhall A Subramanian R. Not made for each other: Audio-visual dissonance-based deepfake detection and localization. Paper presented at: ACM International Conference on Multimedia; 2020; Seattle WA USA. Front Psychol Shuman V 261 4 2013 10.3389/fpsyg.2013.00261 Levels of valence Shuman V, Sander D, Scherer KR. Levels of valence. Front Psychol. 2013;4:261. Biomed Signal Process Control Yuvaraj R 108 14 2014 10.1016/j.bspc.2014.07.005 Detection of emotions in Parkinson’s disease using higher order spectral features from brain’s electrical activity Yuvaraj R, Murugappan M, Mohamed Ibrahim N, Sundaraj K, Omar MI, Mohamad K, Palaniappan R. Detection of emotions in Parkinson’s disease using higher order spectral features from brain’s electrical activity. Biomed Signal Process Control. 2014;14:108–116. 10.1145/3536221.3556598 Sharma G Pandey P Subramanian R Miyapuram KP Dhall A. Neural encoding of songs is modulated by their enjoyment. Paper presented at: International Conference on Multimodal Interaction; 2022; Bengaluru India. 10.1109/FG57933.2023.10042798 Parameshwara R Radwan I Subramanian R Goecke R. Examining subject-dependent and subject-independent human affect inference from limited video data. Paper presented at: International Conference on Automatic Face and Gesture Recognition; 2023; Puako HI USA. 10.1145/3581783.3613784 Parameshwara R Radwan I Asthana A Abbasnejad I Subramanian R Goecke R. Efficient labelling of affective video datasets via few-shot & multi-task contrastive learning. In: Proceedings of the 31st ACM International Conference on Multimedia. MM ’23. New York (NY): Association for Computing Machinery; 2023. p. 6161–6170. J Int Neuropsychol Soc Smith MC 383 2 5 1996 10.1017/S1355617700001454 Spontaneous and posed facial expression in Parkinson’s disease Smith MC, Smith MK, Ellgring H. Spontaneous and posed facial expression in Parkinson’s disease. J Int Neuropsychol Soc. 1996;2(5):383–391. Dementia Neufeld M 23 5 1 1994 EEG frequency analysis in demented and nondemented Parkinsonian patients Neufeld M, Blumen S, Aitkin I, Parmet Y, Korczyn A. EEG frequency analysis in demented and nondemented Parkinsonian patients. Dementia. 1994;5(1):23–28. Neurology Hoehn M 318 50 2 1998 10.1212/WNL.50.2.318 Parkinsonism: Onset, progression, and mortality Hoehn M, Yahr M. Parkinsonism: Onset, progression, and mortality. Neurology. 1998;50(2):318–334. 10.24963/ijcai.2019/932 Assaf R Schumann A. Explainable deep neural networks for multivariate time series predictions. Paper presented at: International Joint Conference on Artificial Intelligence; 2019; Macao China. Neurology Jacobs DH 1696 45 9 1995 10.1212/WNL.45.9.1696 Emotional facial imagery, perception, and expression in Parkinson’s disease Jacobs DH, Shuren J, Bowers D, Heilman KM. Emotional facial imagery, perception, and expression in Parkinson’s disease. Neurology. 1995;45(9):1696–1702. Cognit Emot Simons G 759 17 2003 10.1080/02699930302280 Disturbance of spontaneous and posed facial expressions in Parkinson’s disease Simons G, Ellgring H, Smith PM. Disturbance of spontaneous and posed facial expressions in Parkinson’s disease. Cognit Emot. 2003;17:759–778. Comput Biol Med Dar MN Article 105327 144 2022 10.1016/j.compbiomed.2022.105327 EEG-based emotion charting for Parkinson’s disease patients using convolutional recurrent neural networks and cross dataset learning Dar MN, Akram MU, Yuvaraj R, Khawaja SG, Murugappan M. EEG-based emotion charting for Parkinson’s disease patients using convolutional recurrent neural networks and cross dataset learning. Comput Biol Med. 2022;144: Article 105327.