Federated Artificial Resampling for Imbalanced Facial Emotion Recognition

Chatterjee S, Ghosh K, Bhattacharjee S, Das AK, Banerjee S (2024)

Publication Language: English

Publication Type: Journal article

Publication year: 2024

Journal

IEEE Transactions on Affective Computing Institute of Electrical and Electronics Engineers

DOI: 10.1109/TAFFC.2024.3516822

Abstract

Federated Learning (FL) has emerged as an essential tool for computing devices to participate in collaborative training of deep learning models. However, due to the decentralized distribution of data over clients/local computing devices, the class imbalance problem has become evident, causing severe degradation in the performance of the global model. Motivated by the emergence of FL models in emotion recognition, the current study proposes an FL-based facial emotion recognition system by addressing local imbalance data problems encountered in client devices. Firstly, the local imbalance problem is mitigated by utilizing the data-level artificial resampling method on the client side. To address the possibility of an adversarial attack using imbalanced data, the local training is equipped with a pre-Training check to verify if the data being used is imbalanced above a predefined threshold of imbalance ratio. In case of high imbalance, a pre-Training step will balance the data locally without sharing any information with other participants thereby ensuring privacy in the FL framework. Experiments have been conducted by using benchmark facial emotion recognition data with a balanced testing strategy. It indicated that considerable improvement can be achieved by the proposed FL-based facial emotion recognition model.

Involved external institutions

Indian Institute of Engineering Science and Technology, Shibpur (IIEST Shibpur) IN India (IN) University of Alberta CA Canada (CA) Narula Institute of Technology IN India (IN)

How to cite

APA:

Chatterjee, S., Ghosh, K., Bhattacharjee, S., Das, A.K., & Banerjee, S. (2024). Federated Artificial Resampling for Imbalanced Facial Emotion Recognition. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2024.3516822

MLA:

Chatterjee, Sankhadeep, et al. "Federated Artificial Resampling for Imbalanced Facial Emotion Recognition." IEEE Transactions on Affective Computing (2024).

BibTeX: Download