IAES International Journal of Artificial Intelligence (IJ-AI)

Received Sep 5, 2021 Revised Sep 16, 2022 Accepted Oct 16, 2022 To meet customer expectations and remain competitive, industrials try constantly to improve their quality control systems. There is hence increasing demand for adopting automatic defect detection solutions. However, the biggest issue in addressing such systems is the imbalanced aspect of industrial datasets. Often, defect-free samples far exceed the defected ones, due to continuous improvement approaches adopted by manufacturing companies. In this sense, we propose an automatic defect detection system based on one-class classification (OCC) since it involves only normal samples during training. It consists of three sub-models, first, a convolutional autoencoder serves as latent features extractor, the extracted features vectors are subsequently fed into the dimensionality reduction process by performing principal component analysis (PCA), then the reduced-dimensional data are used to train the one-class classifier support vector data description (SVDD). During the test phase, both normal and defected images are used. The first two stages of the trained model generate a low-dimensional features vector, whereas the SVDD classifies the new input, whether it is defect-free or defected. This approach is evaluated on the carpet images from the industrial inspection dataset MVTec anomaly detection (MVTec AD). During training, only normal images were used. The results showed that the proposed method outperforms the state-of-the-art methods.


INTRODUCTION
Efficient quality inspection is one of the cornerstones of successful manufacturing companies. Since human visual inspection is error-prone and subjective, there has been a movement towards automatic defect detection systems [1], more than that, recently we witness the transition to the era of quality 4.0, which we can define as a mature quality system that sought to leverage industry 4.0 technologies. In fact artificial intelligence and machine learning [2] had proven their abilities to perform quality inspection [3]. A plethora of defect detection methods have shown to be promising and efficient [4]. However most of them are based on supervised learning [5], requiring considerable amount of normal and abnormal (i.e. defected) samples to train the model efficiently, which is not always available in real applications. Actually, manufacturing companies adopt increasingly improvement approaches [6] as lean management and six sigma [7], with the attention of managing the fabrication process and reducing defective products by targeting zero defect manufacturing. Defects hence rarely occur in production, and meanwhile there is enormous need to build automatic defect detection system. One-class classification (OCC) algorithms [8] offer the potential to create automatic inspection system in early stage without having to wait for more defected samples to be collected. Consequently industrials could exploit the important amount of data in their possession, despite highly skewed class distribution. The main idea of OCC methods is to distinguish between normal and abnormal class, drew on the knowledge gained of normal samples during training.
The term OCC was first introduced by [9] to denote a category of classification algorithms that address cases where few to none defect samples are available for training; the normal class is well-defined while abnormal one is under-sampled [10] which is quite common in industrial areas [11] ,and with that, defects are seen as a deviation from defect-free class. The OCC concept encompasses several approaches, such as methods based on density [12], distance [13], neural networks [14], [15], and boundary approaches [16] that aims to encircle normal samples by a decision boundary. The work [17] developed adversarially trained deep neural networks, the first component works as image reconstructor, while the second represents the classifier. Autoencoders also were used to address OCC problems [18], [19]. In general, these researchers assumed that autoencoders generates higher reconstruction error for defected samples. Going through research works that address OCC, it is clearly apparent that support vector data description (SVDD) [20] is one of the extensively used algorithms in OCC applications for its satisfactory results [21]. Luo et al. [22] introduced a cost-sensitive SVDD, which is a method that provides different costs to classification errors. The paper [23] developed a new SVDD approach to deal with uncertain data. They trained the SVDD by using the similarity scores between examples and normal class. It is reported to outperforms regular SVDD in terms of sensitivity to noise. Shi et al. [24] introduced improved SVDD by combining relative density weight with SVDD. However, those SVDD based methods have difficulty dealing with large and high-dimensional datasets [25] mainly because of optimization complexity.

METHOD
The intent of this work is to create an automatic defect detection system. We propose a model that could manage imbalanced data and, at the same time yield to interesting results in high-dimensional spaces using SVDD algorithm. In this section, we describe the proposed defect detection system in detail, and we highlight the three components of our model: convolutional autoencoder, principal component analysis algorithm (PCA) and SVDD algorithm.

Overview
Product surface images are used as input for our algorithm and only images of normal class are present during training. The proposed system consists of three main phases. Firstly, we use a convolutional autoencoder that allows extracting image's abstract features. Once this submodel is trained, the decoder part is discarded. Then the bottleneck features vector is later fed into PCA in order to perform dimensionality reduction by inducing efficient and discriminating features representation. So that it serves as training input of the SVDD classifier. The test images are forwarding through the trained model to determine whether the image is normal or defected. The proposed system takes advantage of the convolutional autoencoder ability to extract robust features automatically; meanwhile it alleviates the problem of SVDD of handling highdimensional datasets. Figure 1 shows the overview of the proposed approach, and Figure 2 illustrates the flowchart of the entire system.

Features extraction
As a first stage a convolutional autoencoder (CAE) is used to extract image features. CAE can automatically provide powerful feature learning [26]. Moreover, in [27] it has been proven that the more the architecture of convolutional autoencoder becomes deeper, the more powerful features are extracted through layers , in a way that first layer provides generic features while subsequent layers extract complex features. Basically, autoencoder is a neural network that consists of two connected sub-models: an encoder and a decoder. The symmetric structure of the autoencoder allows the reconstruction of input data through the features vector provided by the bottleneck layer (i.e., encoder output) as illustrated in Figure 1. Considering X the domain of the input data samples and Z the domain of the encodings.
Given a sample x ∈ , z ∈ ℤ the encoded representation of x, and xr the reconstructed input. The encoder-decoder process can be summarized, Where ℊ ℯ : → ℤ is the encoding function and ℊ : ℤ → the decoding function, while , represent the encoder and decoder parameters respectively. A loss function is used to measure reconstruction error; we have chosen ℓ2 loss for its simplicity and computational speed. The loss function is formulated as, In this work, we trained the convolutional autoencoder on defect-free samples. Once the model is trained, we use the bottleneck layer as automatic features extractor. The encoder consists of five convolution layers where each layer is followed by a batch normalization layer. The max-pooling layer is used from the second convolution layer where each layers use the rectifier linear unit (ReLu) acti-vation function, i.
The first two convolutional layers consist of 32 fil-ters of size 3×3, the second one followed by a downsampling (max-pooling) layer. ii. The third convolutional layer consists of 64 filters of size 3×3, followed by another downsampling layer. iii. The fourth convolutional layer consists of 128 fil-ters of size 3×3. iv. The fifth convolutional layer consists of 256 filters of size 3×3.

Dimensionality reduction
Since SVDD has difficulty handling high-dimensional datasets, PCA [28] was applied to reduce the dimensionality of features vectors provided by the CAE. PCA is a statistical procedure that aims to project high-dimensional input data into a lower dimension space while retaining most of the information. Let Z be an n×m data matrix, where the rows represent the n extracted vectors while the features are represented by columns. PCA process is formulates, a) Standardize the m-dimensional data Where denotes the eigenvalues, ν represents the eigenvectors and = 1, … , n. a. Choose k eigenvectors corresponding to the k highest eigenvalues k<m. b. Project the input data into new k-dimensional space. Let, Then, Where z ′ is the low-dimensional features vector.

Defect detection
The defect detection task is done using SVDD. This algorithm attempts to determine a hypersphere with the minimum volume that encircles almost all training data. The spherical boundary is characterized by a center a and a radius R, hence during test, points that fall outside the boundary are considered as abnormal as illustrated in Figure 3. The parameters R and a are defined by (9),  (9) Subject to, ‖z ′ − a‖ 2 ≤ R 2 + ξ ∀ (10) here : are slack variables that allow some points in training data to be outside the sphere and C represents a penalty constant that controls the trade-off between the volume of the hypersphere and rejected points.
Where α are lagrange multipliers, and 〈z ′ , z ′ 〉 denotes the inner product of z ′ and z ′ .The hypersphere boundary is determined by support vectors for which 0 ≤ α ≤ C. The center 'a' is calculated as, The radius R is then computed by selecting an arbitrary support vector ′ , For each new input , the distance between this tested point and the center is computed as, dist 2 (y) = 〈y , y〉 − 2 ∑ α 〈y , z ′ 〉 + ∑ α α 〈z ′ , z ′ 〉 , =1 Thus, points with dist 2 (y) ≥ R 2 are considered as defected. According to [20], replacing the inner product 〈z ′ , z ′ 〉 in (12) with appropriate kernel function ( ′ , ′ )provides more flexibility to define the boundary. We have chosen to use the Gaussian kernel function, given that it is the kernel reported to be yielding satisfactory results in OCC applications [29]. The Gaussian kernel is defined as, Where is the kernel width.

RESULTS AND DISCUSSION
The experimental environment is a computer with an i7-7700HQ CPU, 16GB of RAM, Nvidia RTX 2080 Ti GPU, running Windows 10, and we used the Tensorflow library to implement the proposed approach. To demonstrate the effectiveness of our suggested system for defect detection, we conducted a set of experiments on the carpet images from MVTec anomaly detection (MVTec AD) dataset [30],which is a benchmark of natural images dedicated to unsupervised anomaly detection that mimics real industrial inspection applications. The carpet dataset consists of 390 high-resolution images of 1024×1024 pixels, divided into 72% of normal data and 28% of abnormal data .The training set is composed of defect-free images, while the test set consists of normal images as well as images containing 5 different types of finegrained anomalies on the carpet's surface like threads, cuts, metal contamination, color and holes. The dataset overview is shown in Figure 4. Data augmentation was needed to diversify the training data set and make the model more generalizable. To do so, random flips and rotations strategies were applied generating 1449 image. Finally, both categories were rescaled to 512×512 pixels. For this implementation, we assumed that there are no outliers in training data. We defined C=1, which implied that ξ_(i=)=0 [31]. The kernel width σ is optimized through cross validation. We investigated the performance of our model on the carpet images and compare it with state-of-the-art methods. Furthermore, we conducted comparison with the two-stage model "Convolutional autoencoder-SVDD" (CAE-SVDD), to prove the importance of PCA algorithm in the proposed system.
As shown in Table 1, the proposed approach method outperforms the compared methods on all metrics at detecting the defects in carpet dataset. The AUROC result of CAE-PCA-SVDD model was better, with mean value 0.20% higher than the other algorithms. As can be observed in the comparison with CAE-SVDD algorithm, applying PCA to reduce features vector dimensionality, leads to better performance than the CAE-SVDD structure. Hence the proposed defect detection system could operate well in high dimensional spaces. However, the proposed approach has some limitations. As shown in Figure 5, the system failed to recognize a defect when the contrast of defected part is similar to the texture of the image.  Figure 5. A false negative case

CONCLUSION
The current paper proposes automatic defect detection system that attempts to mitigate the lack of representative samples of defected images. Hence it can be applied in cases where significant amount of normal data is available, while the defected class is characterized by few samples. The suggested approach comprises three components: convolutional autoencoder for image features extraction, then PCA is applied to reduce dimensionality of features vector, and finally SVDD is implemented to classify images. The motivation behind this structure is to build automatic defect detection system that is able to handle effectively imbalanced data. The proposed method has proved to be efficient in experiment developed on carpet dataset from MVTec AD. It is important to note that SVDD parameters (C and the width ) vary from one dataset to another. In this work C was defined as C=1, while was chosen through cross-validation to find best accuracy. As a future work, we plan to evaluate the proposed approach on different datasets, and also utilize an automatic method to determine the parameters: (C, ), which will facilitate the application of our approach. Furthermore we will investigate further techniques to improve the ability of the proposed system to handle texture features.