Transformer-based Self-supervised Representation Learning for Emotion Recognition Using Bio-signal Feature Fusion
Sawant SS, Erick FX, Arora P, Pahl J, Foltyn A, Holzer N, Götz T (2023)
Publication Type: Conference contribution
Publication year: 2023
Publisher: Institute of Electrical and Electronics Engineers Inc.
Conference Proceedings Title: 2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2023
Event location: Cambridge, MA, USA
ISBN: 9798350327458
DOI: 10.1109/ACIIW59127.2023.10388149
Abstract
In this paper, we present a new emotion recognition framework that utilizes transformer based self-supervised representations from different bio-signals and combines them into a fused representation for the task of emotion recognition. Specifically, we explore a cross-attention based fusion mechanism that can explore mutual features among different bio-signals and learn more meaningful embeddings to estimate emotions effectively. Extensive experiments on a public dataset WESAD outperform the performance of fully supervised method for emotion recognition tasks and demonstrate the benefits of self-supervised features in recognizing different emotions. We also present a series of ablation studies to validate the proposed approach.
Involved external institutions
Germany (DE)How to cite
APA:
Sawant, S.S., Erick, F.X., Arora, P., Pahl, J., Foltyn, A., Holzer, N., & Götz, T. (2023). Transformer-based Self-supervised Representation Learning for Emotion Recognition Using Bio-signal Feature Fusion. In 2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2023. Cambridge, MA, USA: Institute of Electrical and Electronics Engineers Inc..
MLA:
Sawant, Shrutika S., et al. "Transformer-based Self-supervised Representation Learning for Emotion Recognition Using Bio-signal Feature Fusion." Proceedings of the 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2023, Cambridge, MA, USA Institute of Electrical and Electronics Engineers Inc., 2023.
BibTeX: Download