Challenges in Detecting Nuanced Sentiment with Advanced Models

Abstract

Sentiment analysis, an essential task in Natural Language Processing (NLP), determines the sentiment expressed in texts. This paper compares six different sentiment analysis models, categorized into three groups based on their underlying techniques: lexicon-based, machine learning-based, and zero-shot learning. The models are evaluated on four publicly available datasets (Movie Reviews, Amazon, Yelp, and Financial), each varying in complexity. The main objective is to assess the efficiency of these models in both binary (positive and negative) and ternary (positive, neutral, and negative) sentiment classification scenarios. Our results indicate that for binary classification, pre-trained large-scale NLP state-of-the-art models outperform other approaches, demonstrating superior results across all evaluated metrics. On average, across all datasets, these models achieved 94% accuracy, 96% precision, 94% recall, and 94% F1-score. However, these pre-trained NLP models face significant challenges in three-class classification tasks, where their performance noticeably declines. Achieving on average, across datasets, 60% accuracy, 66% precision, 60% recall, and 56% F1-score. This study highlights the limitations of current state-of-the-art models in handling more subtle sentiment distinctions. It emphasizes the need for further advancements in sentiment analysis techniques to effectively manage multi-class sentiment categorization that captures and interprets specialized jargon, technical terminology, and nuanced language.