Human-Centered Emotion Classification Using Classical Machine Learning Models on the GoEmotions Dataset
Abstract
As technology grows more “emotionally aware”, models are expected to deliver a high-level of accuracy, while being extremely quick and easy to interpret. While Transformer-based models offer optimal performance, they complicate human-AI collaboration owing to its “black-box” nature and high computational cost. This study critically evaluated classical machine learning models -LinearSVC, Logistic Regression, and Random Forest against Transformer-based models such as RoBERTa and DistilBERT, on the GoEmotions dataset. While Transformer models established a performance ceiling of 53% accuracy and 45% Macro F1-score, they present a “black-box” nature and significant computational overhead. On the other hand, classical models managed to achieve a peak accuracy of 53% and 43% Macro F1-score. Our research demonstrated that lightweight architectures can achieve comparable accuracy with a significant reduction in execution time, offering a more sustainable approach for real-time, human-centric affective interfaces.References
[1] S.K. Bharti et al., "Text-Based Emotion Recognition Using Deep Learning Approach," Computational Intelligence and Neuroscience, vol. 2022, pp. 1-8, Aug. 2022, Art. no. 2645381. doi: 10.1155/2022/2645381.
[2] K. R. Scherer and H. G. Wallbott, "Evidence for universality and cultural variation of differential emotion response patterning," J. Pers. Soc. Psychol., vol. 66, no. 2, pp. 310–328, Feb. 1994, doi:10.1037/0022-3514.66.2.310.
[3] Demszky, D. Movshovitz-Attias, J. Ko, A. Cowen, G. Nemade, and S. Ravi, "GoEmotions: A dataset of fine-grained emotions," in Proc. 58th Annu. Meeting Assoc. Comput. Linguistics, 2020, pp. 4040–4054, doi: 10.48550/arXiv.2005.00547.
[4] S. Mohammad and F. Bravo-Marquez, "WASSA-2017 shared task on emotion intensity," in Proc. 8th Workshop Comput. Approaches Subjectivity, Sentiment Social Media Anal., Copenhagen, Denmark, 2017, pp. 34–49.
[5] R. Ghazi, D. Inkpen, and S. Szpakowicz, "Detecting emotion stimuli in sentences," in Proc. 11th Int. Conf. Comput. Linguistics Intell. Text Process., Iasi, Romania, 2010, pp. 154–165.
[6] R. Tokuhisa, K. Inui, and Y. Matsumoto, "Emotion classification using massive examples extracted from the web," in Proc. 22nd Int. Conf. Comput. Linguistics (Coling 2008), Manchester, UK, 2008, pp. 881–888.
[7] R. F. P. Pratama and W. Maharani, "Comparative Analysis of Naive Bayes and SVM for Improved Emotion Classification on Social Media," Edumatic: J. Pendidikan Inform., vol. 9, no. 1, pp. 33–42, 2025.
[8] M. R. Rasyad and W. Maharani, "Logistic Regression and Naïve Bayes Comparison in Classifying Emotions on Indonesian X Social Media," Edumatic: J. Pendidikan Inform., vol. 9, no. 1, pp. 43–52, 2025.
[9] T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient estimation of word representations in vector space," arXiv preprint arXiv:1301.3781, 2013.
[10] K. Spärck Jones, "A statistical interpretation of term specificity and its application in retrieval," J. Documentation, vol. 28, no. 1, pp. 11–21, 1972, doi: 10.1108/eb026526.
[11] D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge, U.K.: Cambridge Univ. Press, 2008.
[12] S. S. Shrivatava and S. K. Singh, "A Comprehensive Study on Text-Based Emotion Recognition Using Deep Learning," Eng. Sci., vol. 27, p. 1650, 2024, doi: 10.30919/es1650.
[13] S. Saji, "Emotion Detection in Text: A Comprehensive Analysis Using Classical, Deep Learning, and Transformer-Based Models," M.S. thesis, School of Comput., National College of Ireland, Dublin, Ireland, 2024.
[14] R. Cox, "The regression analysis of binary sequences," J. Roy. Stat. Soc. Ser. B (Methodological), vol. 20, no. 2, pp. 215–242, 1958.
[15] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in Proc. Conf. N. Am. Chapter Assoc. Comput. Linguistics: Hum. Lang. Technol., 2019, pp. 4171–4186.
[16] S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997, doi: 10.1162/neco.1997.9.8.1735.
[17] Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures," Neural Netw., vol. 18, no. 5-6, pp. 602–610, Jul. 2005, doi: 10.1016/j.neunet.2005.06.042.
[18] Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273–297, Sep. 1995.
[19] L. Breiman, "Random forests," Machine Learning, vol. 45, no. 1, pp. 5–32, Oct. 2001.
[20] M. H. Ibnath, K. M. Hasib, M. R. Parvez, and M. F. Mridha, "Enhancing Multi-Emotion Detection in Text: A Comparative Study of Feature Extraction Techniques and Machine Learning Models," in Proc. 2025 Int. Conf. Elect. Comput. Commun. Eng. (ECCE), Feb. 2025, pp. 1–6.
[21] M. Zghoul and A. Shaban, "Performance Evaluation of Classical Machine Learning Models for Emotion Classification," Int. J. Artif. Intell. Appl., vol. 1, no. 2, pp. 1–15, Dec. 2025.
[22] K. M. Suryaningrum, "Comparison of the TF-IDF Method with the Count Vectorizer to Classify Hate Speech," Eng., Math. Comput. Sci. J. (EMACS), vol. 5, no. 2, pp. 79–83, May 2023.
[23] W. Q. A. Saif, M. K. Alshammari, B. A. Mohammed, and A. A. Sallam, "Enhancing Emotion Detection in Textual Data: A Comparative Analysis of Machine Learning Models and Feature Extraction Techniques," Eng. Technol. Appl. Sci. Res., vol. 14, no. 5, pp. 16471–16477, Oct. 2024.
[24] O. Adewumi, M. Okpekwu, S. Adanini, A. O. Olayemi, and A. Joshua, "Multimodal Emotion Recognition and Human-Computer Interaction for AI-Driven Mental Health Support," J. BioMed Res. Rep., vol. 8, no. 5, pp. 1–8, Sep. 2025.
[25] Y. Liu et al., "RoBERTa: A Robustly Optimized BERT Pretraining Approach," arXiv preprint arXiv:1907.11692, 2019. Available: https://arxiv.org/abs/1907.11692
[26] Sanh, L. Debut, J. Chaumond, and T. Wolf, "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter," in Proc. 5th Workshop Energy Efficient Machine Learning and Cognitive Computing - NeurIPS, 2019. Available: https://arxiv.org/abs/1910.01108
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
