Enhancing Emotion Prediction and Recognition in Conversation through Fine-Grained Emotional Cue Analysis and Cross-Modal Fusion

The architecture of our model

Abstract

The purpose of emotion recognition in conversation (ERC) is to identify the emotion category of an utterance based on contextual information. Previous ERC methods relied on simple connections for cross-modal fusion and ignored the information differences between modalities, resulting in the model being unable to focus on modality-specific emotional information. At the same time, the shared information between modalities was not processed to generate emotions. Information redundancy problem. To overcome these limitations, we propose a cross-modal fusion emotion prediction network based on vector connections. The network mainly includes two stages{:} the multi-modal feature fusion stage based on connection vectors and the emotion classification stage based on fused features. Furthermore, we design a supervised inter-class contrastive learning module based on emotion labels. Experimental results confirm the effectiveness of the proposed method, demonstrating excellent performance on the IEMOCAP and MELD datasets.

Type
Publication
In the Twentieth International Conference on Intelligent Computing
Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.
Haoxiang Shi
Haoxiang Shi
University of Science and Technology of China
Ning Cheng
Ning Cheng
Researcher
Yong Zhang
Yong Zhang
Researcher