Comparison among search algorithms for hyperparameter of support vector machine optimization

Abstract

Support vector machine (SVM) is widely used in machine learning for classification and regression tasks, but its performance is highly dependent on hyperparameter tuning. Therefore, fine-tuning these parameters is key to improving accuracy and generality. Recently, many researchers have focused only on applying different algorithms to optimize these parameters. There is a shortage of studies that compare the performance of these methods. Hence, research is needed to compare the performance of these algorithms for the hyperparameters of the SVM optimization problem. This paper compares five optimization algorithms for tuning SVM hyperparameters: grid search (GS), random search (RS), Bayesian optimization (BO), genetic algorithm (GA), and the novel chemical reaction optimization (CRO) algorithm. Experimental results on benchmark datasets such as iris, digits, wine, breast cancer Wisconsin, and credit card fraud demonstrate that CRO consistently outperforms other methods in terms of classification scoring metrics and computational time. It achieves improvements in accuracy, precision, recall, and F1-score of up to 1% on balanced datasets and up to 10% on highly imbalanced datasets such as credit card fraud. It also reduces computation time by up to 50% compared to GS, BO, and RS. These findings suggest that CRO is a promising approach for hyperparameter optimization (HPO) of SVM models.