Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech

Baumann I, Wagner D, Schuster M, Riedhammer K, Nöth E, Bocklet T (2024)

Publication Type: Conference contribution

Publication year: 2024

Publisher: International Speech Communication Association

Pages Range: 2430-2434

Conference Proceedings Title: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Event location: Kos Island GR

DOI: 10.21437/Interspeech.2024-2134

Abstract

Cleft lip and palate (CLP) speech presents unique challenges for automatic phoneme analysis due to its distinct acoustic characteristics and articulatory anomalies. We perform phoneme analysis in CLP speech using a pre-trained wav2vec 2.0 model with a multi-head self-attention classification module to capture long-range dependencies within the speech signal, thereby enabling better contextual understanding of phoneme sequences. We demonstrate the effectiveness of our approach in the classification of various articulatory processes in CLP speech. Furthermore, we investigate the interpretability of self-attention to gain insights into the model's understanding of CLP speech characteristics. Our findings highlight the potential of the self-attention mechanisms for improving automatic phoneme analysis in CLP speech, paving the way for enhanced diagnostics, adding interpretability for therapists and affected patients.

Authors with CRIS profile

Elmar Nöth Technische Fakultät

Involved external institutions

Technische Hochschule Nürnberg "Georg Simon Ohm" DE Germany (DE) Ludwig-Maximilians-Universität (LMU) DE Germany (DE)

How to cite

APA:

Baumann, I., Wagner, D., Schuster, M., Riedhammer, K., Nöth, E., & Bocklet, T. (2024). Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (pp. 2430-2434). Kos Island, GR: International Speech Communication Association.

MLA:

Baumann, Ilja, et al. "Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech." Proceedings of the 25th Interspeech Conferece 2024, Kos Island International Speech Communication Association, 2024. 2430-2434.

BibTeX: Download