Pneumonia classification from chest X-rays using significant feature selection and machine learning

Abstract

The chest X-ray images of normal lungs differ only subtly from those of lungs with pneumonia, making image-based diagnosis highly challenging. To address this issue, we developed a machine learning (ML)-based, lightweight, end-to-end Python package that processes chest X-ray images, implements robust feature selection methods, and classifies the images using various algorithms. While many studies have focused on improving classification accuracy using newer methods, few have addressed the interpretability of the extracted features or the growing computational demands of complex models. We used four publicly available datasets and extracted first-order, textural, and transform-based radiomic features to test our package. Features were selected using the Shapley additive explanations (SHAP) combined with recursive feature elimination (RFE) and stability selection algorithms. Our final solution contains a method that extracts a finite set of features identified by stability selection and feeds them as inputs into classical ML algorithms. Our model achieved 98% accuracy on the primary dataset, and 97%±1, 96%±2, and 94%±2% accuracy on the other three datasets. Our approach is fast, self-contained, and requires only an ideal set of features, making it suitable for resource-constrained clinical environments.