Deep learning sentiment analysis with English linguistic context
Sentiment analysis in English text remains difficult when the polarity of an utterance depends on negation, intensification, contrast, aspect scope, and long-distance syntactic relations rather than isolated sentiment words. Existing deep learning models capture contextual semantics, but they often treat contextual tokens as uniformly informative and may assign excessive weight to sentiment words that are syntactically or pragmatically irrelevant to the target expression. To address this problem, this paper proposes context-guided graph RoBERTa (CGG-RoBERTa), a deep learning sentiment analysis algorithm that combines pretrained contextual encoding, the English Linguistic Context Gate (ELCG), dependency-aware graph propagation, and supervised contrastive regularization. The model uses RoBERTa as the sentence encoder, constructs a token graph from dependency relations and sequential adjacency, applies ELCG to emphasize negation, adversative conjunctions, modifiers, and contextual-focus evidence, and optimizes a joint classification and contrastive objective. Experiments are designed on three English sentiment benchmarks: Stanford Sentiment Treebank (SST)-2, IMDB, and SemEval-2017 Twitter sentiment. CGG-RoBERTa achieves 95.6% accuracy and 95.5% macro-F1 on SST-2, 96.2% accuracy and 96.1% macro-F1 on IMDB, and 74.1% AvgRec with 72.9% macro-F1 on SemEval-2017. Compared with the vanilla RoBERTa-base baseline, CGG-RoBERTa improves macro-F1 by 1.3, 0.8, and 1.7 points, respectively. Ablation analysis shows that dependency graph propagation contributes 0.8–1.0 macro-F1 points, while ELCG contributes 0.7–1.2 points depending on the domain. The results indicate that explicitly modeling English linguistic context can improve both accuracy and interpretability in deep sentiment classification.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://doi.org/10.48550/arXiv.2005.14165
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Castro-Ros, A., Pellat, M.,… Wei, J. (2024). Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70), 3381–3433.
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT 2019, June 2-7, 2019, Minneapolis, MN. 4171–4186. https://doi.org/10.18653/v1/N19-1423
Do, H. H., Prasad, P. W. C., Maag, A., & Alsadoon, A. (2019). Deep learning for aspect-based sentiment analysis: A comparative review. Expert Systems with Applications, 118, 272–299. https://doi.org/10.1016/j.eswa.2018.10.003
He, P., Liu, X., Gao, J., & Chen, W. (2021). DeBERTa: Decoding-enhanced BERT with disentangled attention. arXiv. arXiv:2006.03654. https://doi.org/10.48550/arXiv.2006.03654
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Kim, Y. (2014). Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, October 25-29, 2014, Doha, Qatar. 1746–1751. https://doi.org/10.3115/v1/D14-1181
Kumar, D., & Verma, C. (2021). Automatic leaf species recognition using deep neural network. Evolving Technologies for Computing. Communication and Smart World, 694, 13–22. https://doi.org/10.1007/978-981-15-7804-5_2
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2020). ALBERT: A lite BERT for self-supervised learning of language representations. arXiv. arXiv:1909.11942. https://doi.org/10.48550/arXiv.1909.11942
Liu, B. (2012). Sentiment analysis and opinion mining. San Rafael, CA: Morgan & Claypool Publishers. https://doi.org/10.2200/S00416ED1V01Y201204HLT016
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv. arXiv:1907.11692. https://doi.org/10.48550/arXiv.1907.11692
Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. arXiv. arXiv:1711.05101. https://doi.org/10.48550/arXiv.1711.05101
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011). Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, June 19-24, 2011, Portland, OR. 142–150. https://doi.org/10.5555/2002472.2002491
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv. arXiv:1301.3781. https://doi.org/10.48550/arXiv.1301.3781
Mohammad, S. M., Kiritchenko, S., & Zhu, X. (2013). NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. In: Proceedings of the Seventh International Workshop on Semantic Evaluation, June 14-15, 2013, Atlanta, Georgia. 321–327. https://doi.org/10.48550/arXiv.1308.6242
Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135. https://doi.org/10.1561/1500000011
Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, July 6-7, 2002, Philadelphia, PA. 79–86. https://doi.org/10.3115/1118693.1118704
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. https://doi.org/10.5555/1953048.2078195
Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, October 25-29, 2014, Doha, Qatar. 1532–1543. https://doi.org/10.3115/v1/D14-1162
Rosenthal, S., Farra, N., & Nakov, P. (2017). SemEval-2017 Task 4: Sentiment analysis in Twitter. In: Proceedings of the 11th International Workshop on Semantic Evaluation, August 3-4, 2017, Vancouver, Canada. 502–518. https://doi.org/10.18653/v1/S17-2088
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv. arXiv:1910.01108. https://doi.org/10.48550/arXiv.1910.01108
Schouten, K., & Frasincar, F. (2016). Survey on aspect-level sentiment analysis. IEEE Transactions on Knowledge and Data Engineering, 28(3), 813–830. https://doi.org/10.1109/TKDE.2015.2485209
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, October 18-21, 2013, Seattle, WA. 1631–1642. https://doi.org/10.18653/v1/D13-1170
Tang, D., Qin, B., Feng, X., & Liu, T. (2016). Effective LSTMs for target-dependent sentiment classification. In: Proceedings of COLING 2016, December 11-16, 2016, Osaka, Japan. 3298–3307. https://doi.org/10.48550/arXiv.1512.01100
Tenney, I., Das, D., & Pavlick, E. (2019). BERT rediscovers the classical NLP pipeline. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, July 28-August 2, 2019, Florence, Italy. 4593–4601. https://doi.org/10.18653/v1/P19-1452
Turney, P. D. (2002). Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA. 417–424. https://doi.org/10.3115/1073083.1073153
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1706.03762
Wang, Y., Huang, M., Zhu, X., & Zhao, L. (2016). Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, November 1-5, 2016, Austin, TX. 606–615. https://doi.org/10.18653/v1/D16-1058
Wiebe, J., Wilson, T., & Cardie, C. (2005). Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 39(2-3), 165–210. https://doi.org/10.1007/s10579-005-7880-9
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019). BERT post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of NAACL-HLT 2019, June 2-7, 2019, Minneapolis, MN. 2324–2335. https://doi.org/10.18653/v1/N19-1242
Xue, W., & Li, T. (2018). Aspect-based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, July 15-20, 2018, Melbourne, Australia. 2514–2523. https://doi.org/10.18653/v1/P18-1234
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems, 32. https://doi.org/10.48550/arXiv.1906.08237
Yao, L., Mao, C., & Luo, Y. (2019). Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, 33(1), 7370–7377. https://doi.org/10.1609/aaai.v33i01.33017370
Zeng, B., Yang, H., Xu, R., Zhou, W., & Han, X. (2019). LCF: A local context focus mechanism for aspect-based sentiment classification. Applied Sciences, 9(16), 3389. https://doi.org/10.3390/app9163389
Zhang, C., Li, Q., & Song, D. (2019). Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of EMNLP-IJCNLP 2019, November 3-7, 2019, Hong Kong, China. 4567–4577. https://doi.org/10.18653/v1/D19-1464