Investigating Salient Representations and Label Variance Modeling in Dimensional Speech Emotion Analysis

Representations from fashions akin to Bidirectional Encoder Representations from Transformers (BERT) and Hidden models BERT (HuBERT) have helped to realize state-of-the-art efficiency in dimensional speech emotion recognition. Each HuBERT, and BERT fashions generate pretty massive dimensional representations, and such fashions weren’t skilled with emotion recognition job in thoughts. Such massive dimensional representations lead to speech emotion fashions with massive parameter measurement, leading to each reminiscence and computational price complexities. On this work, we examine the collection of representations primarily based on their job saliency, which can assist to cut back the mannequin complexity with out sacrificing dimensional emotion estimation efficiency. As well as, we examine modeling label uncertainty within the type of grader opinion variance, and display that such info will help to enhance the mannequin’s generalization capability and robustness. Lastly, we analyzed the robustness of the speech emotion mannequin in opposition to acoustic degradation and noticed that the collection of salient representations from pre-trained fashions and modeling label uncertainty helped to enhance the fashions generalization capability to unseen knowledge containing acoustic distortions within the type of environmental noise and reverberation.