Enhanced deepfake detection using an ensemble of convolutional neural networks

Abstract

Digital media integrity and authenticity have been seriously challenged with the rise of deepfakes. The challenge is to automatically detect this artificial intelligence (AI) generated manipulations. These manipulations or forgeries can cause harmful consequences such as spreading fake news in politics, scamming people online and invading privacy. Convolutional neural networks (CNN) models are found to be good at classification tasks, but the performance could not reach high accuracy, especially when they were tested on more challenging deepfake datasets. In this paper we present a deepfake detection system based on an ensemble of CNN architectures, ResNet50 and EfficientNet, capable of distinguishing between real and deepfake videos with high accuracy. For the experiment, we have chosen Celeb-DF version 2, as it has emerged to be one of the most challenging deepfake dataset. The ensemble model achieved an F1-score of 94.69% and an accuracy of 90.58%, outperforming the individual CNN models. This study shows that ensemble learning can increase the reliability and accuracy of deepfake detection systems on challenging datasets.