EEBERT: An Emoji-Enhanced BERT fine-tuning on Amazon Product Reviews for Text Sentiment Classification
Date
2024Item Type
ArticleAbstract
Understanding customer sentiment through product reviews is important for businesses aiming to enhance their products and services. Traditional sentiment analysis often struggles to detect the wide range of emotional nuances, particularly those expressed through emojis, in online reviews. To address this challenge, our paper introduces a novel technique called Emoji-Enhanced BERT (EEBERT) based, on the BERT architecture. The Sentiment Adjustment Factor (SAF), created for emoji tokenization with the embeddings layer, is applied to analyze sentiment and emotion within review text content. Sentiments classification is evaluated using three different methods: (1) evaluating the full review text; (2) analyzing the text with emojis; and (3) integrating star ratings and total votes in EEBERT. These methods were fine-tuned using a labeled Amazon product reviews dataset. The approach utilizes a well-vetted dataset comprising approximately 10 million customer reviews from four different categories. This dataset is condensed to 26,818 reviews to only consider emoji contained reviews, covering 3,275 distinct emojis. A systematic approach to classifying reviews and an innovative technique for assessing the emotional semantics of emojis are developed. The proposed model achieves an accuracy of 97.00%, further applies continuous testing via 5-fold cross-validation, achieving an average accuracy of 99.21%, which supports the reliability of the EEBERT model.
Author
Narejo, Komal Rani
Zan, Hongying
Dharmani, Kheem Parkash
Zhou, Lijuan
Alahmadi, Tahani Jaser
Sehito, Nabila
Ghadi, Yazeed Yasin
Assam, Muhammad