Hybrid heuristic mechanism for occlusion aware facial expression recognition scheme using patch based adaptive CNN with attention mechanism

Hybrid heuristic mechanism for occlusion aware facial expression recognition scheme using patch based adaptive CNN with attention mechanism A. Reddy Prasad A. Rajesh

In computer vision, the recognition of expressions from partially occluded faces is one of the serious problems. By the prior recognizing techniques it can solve the issue by various assumptions. A benchmark-guided branch was proposed for detecting and eliminating the manipulated features from the occluded regions since the human visual system is proficient for eliminating the occlusion and the appropriate focal point was obtained on the non-occluded areas. In recent years deep learning has attained a great place in the recognition of facial reactions Still, the precision of facial expression is affected by the occlusion and large skew. In this research work, a deep structure-based occlusionaware facial expression recognition mechanism is introduced to provide superior recognition results. Firstly, the required image is taken from publically provided online sources and the gathered images are subjected to the face extraction method. The face extraction method is done via the ViolaJones method for the extraction of redundant patterns from the original images. Secondly, the extracted face features are given to the pattern recognition stage, where the Adaptive CNN with Attention Mechanism (ACNN-AM) is introduced. This mechanism automatically forms the occluded region of the face and the focal point was on the most discriminative un-occluded regions. Moreover, the hidden patterns in the Occlusion aware facial expressions are identified through the Hybrid Galactic Swarm Yellow Saddle Goatfish Optimization (HGSYSGO). Finally, the overall effectiveness of the developed occlusion aware facial expression recognition model is examined through the comparative analysis of different existing baseline recognition techniques.
07 31 2023 773 797 10.3233/IDT-230047 https://www.medra.org/servlet/aliasResolver?alias=iospress&doi=10.3233/IDT-230047 https://content.iospress.com/download?id=10.3233/IDT-230047 IEEE Transactions on Circuits and Systems for Video Technology 32 4 2226 AP-GAN: Improving Attribute Preservation in Video Face Swapping Zhang 2022 10.1109/TCSVT.2021.3089724 IEEE Transactions on Circuits and Systems for Video Technology 32 2 690 Triplet Loss With Multistage Outlier Suppression and Class-Pair Margins for Facial Expression Recognition Xie 2022 10.1109/TCSVT.2021.3063052 IEEE Transactions on Circuits and Systems for Video Technology 32 3 1681 Joint Expression Synthesis and Representation Learning for Facial Expression Recognition Zhang 2022 10.1109/TCSVT.2021.3056098 IEEE Transactions on Image Processing 16 1 172 Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines Kotsia 2007 10.1109/TIP.2006.884954 IEEE Transactions on Image Processing 29 4445 Geometry Guided Pose-Invariant Facial Expression Recognition Zhang 2020 10.1109/TIP.2020.2972114 IEEE Access 5 19409 Facial Expression Recognition from Image Sequence Based on LBP and Taylor Expansion Ding 2017 10.1109/ACCESS.2017.2737821 IEEE Access 6 4630 Facial Expression Recognition Using Weighted Mixture Deep Neural Network Based on Double-Channel Facial Images Yang 2018 10.1109/ACCESS.2017.2784096 IEEE Transactions on Circuits and Systems for Video Technology 32 3 1443 Local and Global Perception Generative Adversarial Network for Facial Expression Synthesis Xia 2022 10.1109/TCSVT.2021.3074032 IEEE Transactions on Multimedia 24 3327 Identity-Aware Facial Expression Recognition Via Deep Metric Learning Based on Synthesized Images Huang 2022 10.1109/TMM.2021.3096068 IEEE Transactions on Affective Computing 11 2 348 Co-Clustering to Reveal Salient Facial Features for Expression Recognition Khan 2020 10.1109/TAFFC.2017.2780838 IEEE Transactions on Cybernetics 46 4 916 Time-Delay Neural Network for Continuous Emotional Dimension Prediction From Facial Expression Sequences Meng 2016 10.1109/TCYB.2015.2418092 IEEE Transactions on Instrumentation and Measurement 70 1 Facial Expression Recognition Using Local Gravitational Force Descriptor-Based Deep Convolution Neural Networks Mohan 2021 10.1109/TIM.2020.3031835 IEEE Transactions on Affective Computing 4 1 83 Facial Expression Recognition in the Encrypted Domain Based on Local Fisher Discriminant Analysis Rahulamathavan 2013 10.1109/T-AFFC.2012.33 IEEE Access 8 164130 Expression-EEG Based Collaborative Multimodal Emotion Recognition Using Deep AutoEncoder Zhang 2020 10.1109/ACCESS.2020.3021994 IEEE Access 7 32297 Learning Affective Video Features for Facial Expression Recognition via Hybrid Deep Learning Zhang 2019 10.1109/ACCESS.2019.2901521 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 367 Covariance pooling for facial expression recognition Acharya 2018 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 3359 Joint pose and expression modeling for facial expression recognition Zhang 2018 IEEE Transactions on Image Processing 29 6977 Multi-Modal Recurrent Attention Networks for Facial Expression Recognition Kim 2020 10.1109/TIP.2020.2996086 IEEE Access 5 19409 Facial Expression Recognition from Image Sequence Based on LBP and Taylor Expansion Ding 2017 10.1109/ACCESS.2017.2737821 The Visual Computer 37 2315 Facial emotion recognition using subband selective multilevel stationary wavelet gradient transform and fuzzy support vector machine Arumugam 2021 10.1007/s00371-020-01988-1 Journal on Multimodal User Interfaces 15 429 Facial expression and action unit recognition augmented by their dependencies on graph convolutional networks He 2021 10.1007/s12193-020-00363-7 Multidimensional Systems and Signal Processing 32 633 Transformation guided representation GAN for pose invariant face recognition Chikontwe 2021 10.1007/s11045-020-00752-x 10.1007/s00371-022-02413-5 Liang X, Xu L, Zhang W, Zhang Y, Liu J, Liu Z. A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition. The Visual Computer. 2022. Multimedia Tools and Applications 79 30861 Rapid facial expression recognition under part occlusion based on symmetric SURF and heterogeneous soft partition network Hu 2020 10.1007/s11042-020-09566-2 IEEE Transactions on Image Processing 28 5 2439 Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism Li 2019 10.1109/TIP.2018.2886767 International Journal of Science and Research (IJSR) 6 14 Face Detection System Based on Viola – Jones Algorithm Dabhi 2013 Procedia Manufacturing 49 112 Intelligent bearing Fault diagnosis using Attention-based CNN Wang 2020 10.1016/j.promfg.2020.07.005 Multimedia Tools and Applications 73 397 Speech driven photo realistic facial animation based on an articulatory DBN model and AAM features Jiang 2014 10.1007/s11042-013-1610-x 10.1007/s12046-022-01943-x Podder T, Bhattacharya D, Majumdar A. Time efficient real time facial expression recognition with CNN and transfer learning. Sādhanā. 2022; 47(177). 10.1093/comjnl/bxy133 Brammya G, Praveena S, Ninu Preetha NS, Ramya R, Rajakumar BR, Binu D. Deer Hunting Optimization Algorithm: A New Nature-Inspired Meta-heuristic Paradigm. The Computer Journal. 2019. IEEE Access 9 50001 Grasshopper Optimization Algorithm: Theory, Variants, and Applications Meraihi 2021 10.1109/ACCESS.2021.3067597 Applied Soft Computing 38 771 Galactic Swarm Optimization: A new global optimization metaheuristic inspired by galactic motion Muthiah-Nakarajan 2016 10.1016/j.asoc.2015.10.034 2019 Digital Image Computing: Techniques and Applications (DICTA) 1 Facial-Expression Recognition from Video using Enhanced Convolutional LSTM Miyoshi 2019 10.1155/2022/7450637 Ye M, Hu Q, Liu G. CNN-LSTM Facial Expression Recognition Method Fused with Two-Layer Attention Mechanism. Computational Intelligence and Neuroscience. 2022.