Parallel processing using big data and machine learning techniques for intrusion detection

Abstract

Currently, information technology is used in all the life domains, multiple devices produce data and transfer them across the network, these transfers are not always secured, they can contain new menaces invisible by the current security devices. Moreover, the large amount and variety of the exchanged data cause difficulties related to the detection time. To solve these issues, we suggest in this paper, a new approach based on storing the large amount and variety of network traffic data employing Big Data techniques, and analyzing these data with Machine Learning algorithms, in a distributed and parallel way, in order to detect new hidden intrusions with less processing time. According to the results of the experiments, the detection accuracy of the Machine Learning methods reaches 99.9 %, and their processing time has been reduced considerably by applying them in a parallel and distributed way, which proves that our proposed model is effective for the detection of new intrusions.