Hybrid of convolutional neural network algorithm and autoregressive integrated moving average model for skin cancer classification among Malaysian

Received Dec 28, 2020 Revised May 14, 2021 Accepted May 29, 2021 Skin cancer is a widely spreading cause of mortality among the people specifically living on or near the equatorial belt. Early detection of skin cancer significantly improves the recovery prevalence and the chance of surviving. Without the assist of computer-aided decision (CAD) system, skin cancer classification is the challenging task for the dermatologist to differentiate the type of skin cancer and provide the suitable treatment. Recently, the development of machine learning and pretrained deep neural network (DNN) shows the tremendous performance in image classification task which also provide the promising performance in medical field. However, these machine learning methods cannot get the deep features from network flow which resulting in low accuracy and the pretrained DNN has the complex network with a huge number of parameters causes the limited classification accuracy. This paper focuses on the classification of skin cancer to identify whether it is basal cell carcinoma, melanoma or squamous cell carcinoma by using the development of hybrid convolutional neural network algorithm and autoregressive integrated moving average model (CNN-ARIMA). The CNNARIMA model was trained and found to produce the best accuracy of 92.25%.


INTRODUCTION
In National Cancer Registry data, Malaysia's reported that skin cancers account for 2.6% cancer cases in the country which considers as minor cases in Malaysia [1]. Based on data collected from Dermatology Clinic in Hospital Kuala Lumpur from 2006 to 2014 [2], the most common type of skin cancer is basal cell carcinoma with 34.9%, followed by cutaneous lymphoma with 25.7% and squamous cell carcinoma in 20.6% cases while melanoma is the least common cases which only 5.4% of the patients diagnosed. Even though there are less skin cancer cases in Malaysia, this disease still needs to be concerned due to skin cancer consider as most predominant type of cancer in worldwide [3].
Dermatologists often adopt various diagnosis guide for skin lesion such as ABCDE rule (A-stands for asymmetrical shape, B-irregular borders of lesion, C-uneven distribution of color, D-diameter, and E-a presence of evolution) [4], [5] and 7-point checklist [6], [7] but need to be further examined via  Garnavi et al. described that there is an improvement of 5-30% in the detection while using ELM as compared to the examination of naked eye [8], [9]. Therefore, it is possible to assist dermatologists via computer-aided diagnosis (CAD) system for analyzing the pigmented skin lesions efficiently with the development of ELM through automatic and quantitative assessment of the skin lesion in clinical practice.
Recently, convolutional neural network (CNN) allowed computers to outperform in the classification tasks of skin cancer as compare to the machine learning due to the CNN has ability to handle the large and complex datasets that produce supremacy performance in terms of accuracy [10]. This can be proved through the study about the artificial intelligence (AI) program that trained on nearly 130,000 images of moles, rashes, and lesions using a technique known as deep learning which provide "at least" 91 percent accuracy [11]. The benefit of CNN is this architecture able to classify the skin lesions according to the highlevel features instead of the conventional method that incorporate the visual information of low-level dermoscopic that needed a segmentation step before the extraction of those features [12].
In addition, autoregressive integrated moving average (ARIMA) model which known as the Box-Jenkins method [13] is one of the machine learning algorithms also show the great performance in medical cases forecasting recently through the statistical analysis [14]. Therefore, the hybrid approach of CNN and ARIMA has been proposed for skin cancer classification problem in this paper. This approach believe that will be performed efficiently on a reliable skin cancer classification with the ability to differentiate between three different common types of skin cancer such as melanoma, basal cell carcinoma and squamous cell carcinoma. Actually, the aim of this paper is to evaluate the hybrid of CNN algorithm and ARIMA model performance for skin cancer classification among Malaysian.

RELATED WORKS
Recently, researchers around the world focus on solving the classification problems in medical field especially skin cancer classification by using the machine learning and deep learning approaches. Codella et al. [15] depicted the combination of deep learning and sparse coding as feature extractor while the fused support vector machine (SVM) algorithm was used for classification task. The beauty of this paper is the adoption of the deep learning in feature extraction task which able to increase time and space complexity of the model. After that, an improvement approach has been introduced by same author in terms of the automated segmentation, multi-contextual analysis of the extracted features and additional machine learning techniques for classification task [16].
In Ozkan and Koklu [17] proposed a skin lesion classification system by applying four machine learning methods to classify the lesion into melanoma, abnormal, and normal. In this work, the highest accuracy percentage of 92.50% was performed by the artificial neural networks (ANN). This obviously shows that machine learning techniques still consider as low accuracy performance when compare to ANN due to the machine learning methods unable to get the deep features from network flow. Oliveira et al. [18] used used the ABCD rule and the Box-Counting approach as attribute extraction descriptors and support vector machine (SVM) as classifier for the identification pigmented skin lesions in macroscopic images. The best performance was achieved by SVM with histogram intersection kernel give the accuracy of 79.01% at the classification between nevi and seborrheic keratosis.
For the utilization of deep learning approach in skin cancer classification, Emara et al. [19] proposed the modified Inception-v4 architecture which was also a CNN architecture to classify skin lesion through imbalance HAM 10000 skin cancer dataset [20]. The enhancement of this work was employing the feature reuse using long residual connection to improve classification performance with the achievement of high accuracy, 94.70% but this pretrained DNN has the complex network with a huge number of parameters that may causes the limited classification accuracy. Mukherjee et al. [21] using CNN methods named as CNN malignant lesion detection (CMLD) architecture to analyze the classification accuracy performance of malignant melanoma. The best performance in this work was achieved with accuracy of 90.58% when using Dermofit dataset for model training and testing. Recently, Kassem et al. [22] used modified GoogleNet by replacing last two layers with multiclass support vector machine (SVM) with transfer learning to classify the skin lesion. This hybrid approach achieved classification performance with accuracy, sensitivity, specificity, and precision are 94.92%, 79.80%, 97.00%, and 80.36%.
Since ARIMA model shows the outstanding performance in forecasting tasks, researchers tend to use hybridisation methods for enhancement due to its outstanding performance in solving a lot of engineering problems. Wang et al. [23] developed the hybrid model of SVM and ARIMA (SVM-ARIMA) to the garlic short-term price and outperformed the ARIMA and SVM models. Ma et al. [24]  for network-wide traffic state prediction. MLP neural network and MSVR were implemented to capture the network-scale co-movement pattern of all traffic flows while ARIMA was utilized to postprocess the residual time series out of neural network in this paper. Suhermi et al. [25] employed the hybrid deep neural network (DNN) and ARIMA model for roll motion prediction. This proposed approach is based on the supremacy performance of ARIMA and artificial neural network (ANN) models in linear and non-linear modelling. In addition, the advance hybrid approach has been yielded by Ji et al. [26]. The authors introduced the hybrid of ARIMA model, CNN and long short-term memory network (LSTM) to forecast the future prices of carbon. The idea behind of this hybrid method is ARIMA used for linear features capturing. CNN utilized for hierarchical data structure capturing while LSTM utilized for long-term dependencies capturing in the data. The beauty of these papers were the empirical results of hybrid model performed better than that of nonhybrid model and proven that the hybrid method quite effective in improvement of forecast accuracy.

METHODOLOGY
In this paper, a new technique has been introduced for skin cancer classification using hybridisation technique by combining CNN algorithm and ARIMA model. Our technique has been implemented and validated. From the CNN-ARIMA model, CNN algorithm will be used to extract the features of skin cancer and ARIMA model behave as classifier to classify the skin cancer based on three types common skin cancer which consists of melanoma, basal cell carcinoma and squamous cell carcinoma [27].

Dataset
The dataset used in this paper was collected from The International Skin Imaging Collaboration (ISIC) via Kaggle website [28]. This dataset consists of 2357 RGB dermoscopic images in JPEG format which categorized in 9 different skin cancer diseases based on the classification taken from ISIC. These images were very clean with high resolution. Therefore, the dataset from ISIC is suitable for deep learning implementation. Besides that, there were also different image resolution from this source since the archive combines many sub datasets. In this paper, there were 995 images which comprised of 3 different classes in Figure 1 such as melanoma, basal cell carcinoma and squamous cell carcinoma were selected due to these three types of skin cancers considered as common in Malaysia. Figure 1 illustrates the selected skin cancer types that used in this work such as Melanoma in Figure 1

Data pre-processing
Data pre-processing is the essential step when dealing with complex skin cancer images and also able to provide the good classification with less testing or training time. Without pre-processing step, there will be the major problem in the feature extraction step as the prominent features unable to produce and consequently also degrades the overall performance. This is because skin cancer images taken from a normal camera may contains the fine hair that may influence the outcome of the study. Therefore, the skin cancer images that contains fine hair removed by using hair removal technique through open-source computer vision library (OpenCV) approach in this paper. Figure 2 illustrates the process of hair removal technique through OpenCV.
Initially, the original skin cancer images in Figure 2(a) that contain fine hair convert to the grayscale Figure 2(b) and then perform the morphological black hat operation. This operation is to enhance the skin cancer image that contain the fine hair which are smaller than the structuring element and darker than their surroundings. Therefore, the fine hair contour will be highlighted Figure 2

Data augmentation
There are two purposes of data augmentation technique utilized in this paper such as to increase the dataset and to avoid overfit or underfit issues that might result in poor accuracy. Initially, there were total of 995 original skin cancer images collected through Kaggle which the data collected was limited but more dataset was needed for better performance. There is no specific requirement for deep learning but the training data set must be in high quality in medical imaging [29]. Based on [30], at least 500 to more than 1000 images per class is good enough for classification task in deep neural network [31]. Therefore, the dataset used in this paper have been increased to the total of 4000 images that consist of the pre-processed images without fine-hair and the augmented images which utilized via the intensity adjustment technique from original image to blur, light or dark in order to generate additional data to establish a class-balanced dataset. The number of images before and after augmentation task was given in Table 1. After that, the further augmented approaches such as width shift, height shift, zoom, stretch and rotation were selected to avoid overfit or underfit issues help the model to generalize properly. This process able to provide the data set with acceptable size. The detail parameters choice for further data augmentation of skin cancer datasets were summarized in Table 2.

CNN-ARIMA architecture
In this paper, the performance for the hybrid (the combined architecture) of CNN and ARIMA will be investigated for skin cancer classification. The idea of this hybridization is CNN algorithm behave as feature extractor while ARIMA model acts as classifier by merging into fully connected layer which performs non-linear transformations of the extracted features as the enhancement for classification. As compare to the traditional CNN algorithm, the fully connected layer in this architecture will be modified by inserting ARIMA model in this layer as shown in Figure 3 to output three skin cancer classes as proposed. Before preprocessed and augmented dataset implement in the CNN-ARIMA model, all of these datasets need to resize to the shape of 32×32 and consequently convert every pixel of images to array which known as feature maps. The input will be the skin cancer image with 2D array that corresponding to the pixel values. According to Figure 3, the proposed CNN-ARIMA model is the multistage architecture consists of three stages that consists of multiple layers and fully connected layers which are connected after these stages. The details of the proposed CNN-ARIMA model are such as:  First stage -Third Stage: The input initially fed into first stage of the CNN-ARIMA model. The first and second convolutional layers in this stage contain 64 filters with the kernel size of 5×5 followed by rectified linear unit (ReLU) as an activation function after each convolutional layer. The output size after pooling layer decreases by half since a stride of 2 have been used with pooling size of 2×2. The dropout layer with the ratio of 0.5 also added after pooling layer to avoid overfitting. The final dropout layer in this stage has an output of 12×12×64 which will behave as input in next stage. The same step will be repeated in next two stages except the 32 filters with kernel size of 3×3 and 16 filters with kernel size of 1×1 are used in second stage and third stage respectively.  Fully connected layer: The output from final dropout layer in third stage convert to a single array through flatten layer which also means convert the 3D array into a 1D array of size 2×2×16= 64. After that, the ARIMA model obtain in the ARIMA layer via the algorithm; (3, ( , , )) (1) Where p is the auto regressive value, d is the differencing value and q is the moving average value. However, whatever value insert in p, d and q, the output parameter still the same which will not affect the performance in this study. In the final last 3 layers, the dense implement according to large 512-unit layers with the dropout ratio of 0.5 followed by the final layer with 3 nodes for classification performance among 3 classes of skin cancer which are basal cell carcinoma, melanoma and squamous cell carcinoma through a softmax activation function.
In addition, there are also the parameters compute after every layer in model except input layer, dropout layer and flatten layer due to there is no backpropagation learning involve in these layers. The number of parameters for convolutional layer, and fully connected layer, are computed by the formula; Where is the shape of width of the filter, is the shape of height of the filter, is the number of filters in the previous layer, is the number of filters in current layer, is the previous layer neurons and is the current layer neurons. The total parameter computed for the proposed CNN-ARIMA model as shown in Table 3 is 136422 which also based on the sum of every single parameter from each layer.

Performance measure
The classification result for the proposed model has been evaluated in four quantitative measures such as test accuracy (ACC), sensitivity (TPR), specificity (TNR) and precision (PPV) via confusion matrix [33]. These measures are determined by the formula [34]; Where is true positive value, is true negative value, is false positive value and is false negative value while , and PPV refers to true positive rate, true negative rate and positive prediction value respectively. Therefore, the classification result performance in this paper consists of overall test accuracy with the average sensitivity, average specificity and average precision according to basal cell carcinoma, melanoma and squamous cell carcinoma.

RESULTS AND DISCUSSION
In this experiment, the test accuracy achieved for the proposed CNN-ARIMA model is 92.25%. From the loss and accuracy performance for training and validation respectively as shown in Figure 4, the flow of training was decrease exponentially for loss performance while increase exponentially for accuracy performance which able to produce high accuracy. After 14 th epoch during training and validating in loss performance, the validation loss is higher than training loss as shown in Figure 4(a). This show that the model was slightly overfit after 14 th epoch due to the model only able to memorize the limited amount of training data even the data augmentation technique has been implemented and testing dataset have been balance distributed randomly in this work. Meanwhile, the validation accuracy is lower than training accuracy after 15 th epoch in accuracy performance as shown in Figure 4(b) due to the slightly underfit issue. This shows that the validation data may perform well under circumstances which the validation data fits better to the model than training data even though the data is in underfitting situation.
Based on the confusion matrix performance as shown in Figure 5, majority of skin cancer types were classified accurately since the test accuracy shows the promising performance. The 1200 testing images (30% of overall dataset) has been balance distributed randomly in three types of skin cancer which 415 images distributed in basal cell carcinoma, 402 images distributed in melanoma and 383 images distribute in squamous cell carcinoma. From the balance distributed testing dataset in each skin cancer, 362 images of basal cell carcinoma, 371 images of melanoma and 374 images of squamous cell carcinoma were categorized accurately in confusion matrix performance as shown in Figure 5.   Table 4, the performance measure of three skin cancer types via confusion matrix as shown in Figure 5 achieve the average sensitivity of 92.39%, average specificity of 96.11% and average precision of 92.40%. In addition, the performance of the proposed CNN-ARIMA model also compared with the existing techniques using the different amount of skin cancer dataset and different classes of skin cancer to classify. Results of comparative study as presented in Table 5 show that the proposed CNN-ARIMA model achieve better performance than the existing tehniques in the term of sensitivity and precision, as it has been able to accurately distinguish skin cancer in various classes of skin cancer. However, the classification accuracy and specifity for the proposed model of CNN-ARIMA is still marginally lower than the previous work proposed by Kassem et al. [22].

CONCLUSION
This research introduced the hybrid algorithm of deep neural network and machine learning approach for skin cancer classification. The hybrid approach was relied on the combination of CNN and ARIMA. The proposed hybrid performance shown significantly higher results in terms of sensitivity, specificity and precision as compared to the state-of-art skin cancer classification approaches that used single machine learning and single deep learning methods. In addition, this work also did not require any prior segmentation approach.