Sensors, Vol. 24, Pages 2573: Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition

Sensors, Vol. 24, Pages 2573: Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition Sensors doi: 10.3390/s24082573 Authors: Geon Woo Lee Hong Kook Kim This paper addresses a joint training approach applied to a pipeline comprising speech enhancement (SE) and automatic speech recognition (ASR) models, where an acoustic tokenizer is included in the pipeline to leverage the linguistic information from the ASR model to the SE model. The acoustic tokenizer takes the outputs of the ASR encoder and provides a pseudo-label through K-means clustering. To transfer the linguistic information, represented by pseudo-labels, from the acoustic tokenizer to the SE model, a cluster-based pairwise contrastive (CBPC) loss function is proposed, which is a self-supervised contrastive loss function, and combined with an information noise contrastive estimation (infoNCE) loss function. This combined loss function prevents the SE model from overfitting to outlier samples and represents the pronunciation variability in samples with the same pseudo-label. The effectiveness of the proposed CBPC loss function is evaluated on a noisy LibriSpeech dataset by measuring both the speech quality scores and the word error rate (WER). The experimental results reveal that the proposed joint training approach using the described CBPC loss function achieves a lower WER than the conventional joint training approaches. In addition, it is demonstrated that the speech quality scores of t...
Source: Sensors - Category: Biotechnology Authors: Tags: Article Source Type: research