Why simple neural networks outperform traditional classifiers in cyberbullying detection

Abstract

Cyberbullying on social media, particularly in Bahasa Indonesia, has become a pressing problem. Detecting abusive language in Bahasa Indonesia poses unique challenges due to the prevalence of informal expressions, slang, code-mixing, and implicit aggression. Recently pre-trained language models, such as Indonesian bidirectional encoder representations from transformers (IndoBERT), can generate contextual word embeddings that have shown strong potential for understanding complex linguistic patterns. However, it remains unclear which classification models can best exploit these rich embeddings. This study aims to enhance the performance of cyberbullying detection and investigate why simple neural networks outperform traditional machine learning classifiers, such as support vector machine (SVM) and na¨ıve Bayes (NB) when applied to contextual embeddings in Bahasa Indonesia. This study conduct both theoretical and empirical comparisons of fully connected neural networks (FCNN), SVM, and NB classifiers using IndoBERT embeddings. The results demonstrate that even shallow neural architectures can leverage contextual semantic information more effectively than traditional methods, yielding higher accuracy and robustness. Besides performance improvement, the study also explains the underlying reasons for this superiority, highlighting the adaptability of neural models to complex embedding spaces. These findings provide insights for developing more effective abusive language detection systems in low-resource languages and contribute to understanding the interaction between modern embeddings and classifier architectures.