Enhancing fake news detection: a hybrid BERT-XGBoost model for improved performance and interpretability

Abstract

The widespread spread of fake news poses a serious threat to the integrity of information. The dominant approach to detection involves end-to-end fine-tuning of large transformer models like bidirectional encoder representations from transformers (BERT), which, despite achieving high accuracy, often function as opaque “black boxes” with limited interpretability. This paper proposes and validates a hybrid, decoupled architecture that proves to be a more practical and powerful alternative. We first fine-tune a DistilBERT model on the full WELFake dataset of 71,537 articles after cleaning to create domain-specific embeddings. These high-dimensional vectors are then used as input features to train a robust extreme gradient boosting (XGBoost) classifier. The results demonstrate that the hybrid model achieves a state-of-the-art accuracy of 99.76%, slightly surpassing the already high performance of a standard end-to-end fine-tuned model. Crucially, this approach provides this top-tier performance while offering significant advantages in model interpretability through feature importance analysis. This work establishes that a decoupled architecture is not just a viable alternative but a superior practical strategy for combating misinformation, successfully balancing state-of-the-art accuracy with essential model transparency.