Sentiment classification using gradient modulation and layered attention

Abstract

Sentiment analysis is a technique for evaluating text to ascertain whether a statement is positive, negative, or neutral. Currently, transformer-based models capture the contextual relationships among words in a phrase and accomplish sentiment analysis in a nuanced manner via multi-head attention. This approach, with a fixed number of layers and heads, struggles to find the complex relationships between phrases and their semantic structures. To mitigate this issue, the suggested technique incorporates the graded multi head attention model (GMHA) at the base of the distilled bidirectional encoder representations from transformers (DistilBERT) model. It is employed to augment the layers and heads progressively, capturing the relationships between sentences in a sophisticated manner. By increasing the layers and heads the proposed model extracts long-term and hierarchical relationships from the sentence. Additionally, the attention sentient optimization technique is introduced, which improves model learning by giving more weight to important words in a sentence. During training, the process checks to see which words (“amazing" or "worst") get more attention and gives them more weight in the model update. This makes it easier for the model to understand important emotions. Our suggested model enhances performance in sentiment exploration, with an accuracy of 96.53%. This interpretation includes a comparison analysis with another contemporary framework.