Automated vision based defect detection using gray level co- occurrence matrix for beverage manufacturing industry

Norhashimah Mohd Saad, A. R. Abdullah, W. H. W. Hasan, N. N. S. A. Rahman, N. H. Ali, I. N. Abdullah Faculty of Electrical and Electronic Engineering Technology, Universiti Teknikal Malaysia Melaka, Malaysia Faculty of Electrical Engineering, Universiti Teknikal Malaysia Melaka, Malaysia Faculty of Electronics and Computer Engineering, Universiti Teknikal Malaysia Melaka, Malaysia Faculty of Engineering and Technology, Multimedia Universiti, Malaysia Stevia Industry Sdn. Bhd, Gurun, Kedah, Malaysia


INTRODUCTION
Inspection process has been widely used in real-world applications to detect defect on the product, especially in color concentration defects [1]. The high demand for the product in the market has made the industry to produce a product in a shorter time with less product quality. The use of human inspectors in production has contributed to many problems such as inconsistent, slower, fatigue and sickness [2]. Moreover, the important purpose of creating digital image processing helps to identify and classify certain subjects automatically [3]. Thus, most industries have substituted manual inspection with automatic inspection which is more consistent, faster and has high production efficiency.
A major challenge of the beverage industry in the global market is producing a high-quality product with low cost and fast product manufacturing. SMEs are the beverages industries that facing problems in ISSN: 2252-8938 Automated vision based defect detection using gray level co-occurrence… (Norhashimah Mohd Saad) 819 maintaining the quality of the product. The growing demand for products with high quality leads to the improvement of inspection process development. However, the lack of capital and the financial problem has made SMEs to produce products with low a specification or without follow the ISO 9001 standard. Besides, several SMEs have implemented manual inspection using human's operator to inspect product quality of various shapes, colors, levels, dimensions, sizes and patterns. The use of manual inspection contributes to human error such as time-consuming, tiredness, sickness, boredom, repetition and low productivity.
Most of the foods and beverage products require a color inspection system as part of their quality control. The color inspection using image processing technique is applied to inspect the defective color on the surface of the product. Previous researchers have developed different systems for different applications in inspecting the color of the product. Previously, a lot of work has been done using different techniques to detect the defect of color concentration. Tang [4] proposed region growing method to segment the color image to increase the color image application. The watershed technique is applied to initialize segmentation process. The results show the proposed technique can obtain better segmentation and can be implemented on color image application. However, the proposed technique required a longer time to process the algorithm and sometimes it over segment the image [4].
In 2011, X. Y. Wang et al. [5] recommended support vector machine (SVM) for color image classification because of the ability of SVM as a classifier and to solve the problems of computer vision. The accuracy of the experimental result is 96%. Nevertheless, the higher number of training sample makes the longer time taken to process the sample [5]. In the study made by [6], K-means clustering technique is suggested for the infected fruit part detection to replace the manual inspection used by industry. In order to classify the colors, k-means clustering technique is used and two different colors are measured by using euclidean distance metric. Nevertheless, unsuitable K-value selection causes under-segmentation or oversegmentation for the image.
In 2015, G. Capizzi et al. [7] proposed automatic fruit defect classification based on gray level cooccurrence matrix (GLCM) and radial basis probabilistic neural network (RBPNN) to be applied in fruit industrial application. Color features are extracted using the conversion of RGB image to HSV image. GLCM technique is applied to the image to extract the texture features. Results from shape, color and texture features are classified using RBPNN classifier technique. Through the proposed technique, 97.25% accuracy is achieved. Due to the high number of samples, large memory size is required and a lot of time is taken to complete all the processes [7].
In 2017, A. Yamin et al. [8] they have worked on blood group detection and classification using color images in medical applications. The morphological operation technique is applied to segment the image by removing the unwanted object and restructure the image. For the classification of blood groups, the rulebased classifier technique is implemented by setting the density threshold value and objects threshold value. High accuracy which is 98% is obtained from the performance verification. However, with the proposed technique, it reduces the quality of the image and is inefficient to be implemented for real-time processing [8]. This morphological technique not only can remove noise but is also able to remove small objects from a binary image due to obvious contrast between the target and background regions [9], [10]. Nevertheless, in segmenting complex detection environment parameters such as illumination, shapes and movement of the sample, the threshold segmentation method is too subjective to adapt [11]. Table 1 summarizes the implementation of the previous techniques. Based on the previous works, this paper proposed gray level co-occurrence matrix (GLCM) technique for beverages quality inspection based on color concentration. GLCM proves to be a good textural technique and is commonly used in computer vision applications. The information provided by GLCM from surrounding or neighboring pixels, makes this technique to be the most relevant technique for texture image segmentation [12]. GLCM has information of gray level frequency and edge transitions, where it is suitable to handle objects with fuzzy textures and boundaries [13]. Another method of boundary computation technique can also be done using semi global boundary detection (SGBD) that requires little parameter to increase computer vision application [14]. The analysis involves in this paper are pre-processing, bottle segmentation using thresholding and morphological operation, RGB conversion, GLCM analysis, cooccurrence histogram computation and classification using Euclidean distance and rule-based classification.

RESEARCH METHOD
The analysis framework for the proposed technique is shown in Figure 1. The samples of beverages dataset are first collected. The image is analyzed using Matlab software in detecting the color concentrations. The digital camera is used to capture the bottle image. Then, the image is pre-processed to normalize and enhance the image quality. Otsu' method and morphological operation are applied to segment the bottle's region of interest (ROI) area. GLCM is then calculated. The GLCM cross-section is then computed. This cross-section provides information about the region and boundary of the image. Thus, an optimal adaptive threshold can be obtained from the matrix to perform the classification. Co-occurrence histogram analysis is performed to get the intensity distribution data. The Euclidian distance is computed to find the parameters as input to the rule-based classifier.

Image acquisition and pre-processing stage
The image acquisition is done in real-time via via IoT process using IP camera and a running conveyer. The image acquired is then stored in Apache server. The whole process can be found in [15]. Figure 2 shows the complete image acquisition system. The total number of samples used for the analysis is 500 data as illustrated in Table 2.
Pre-processing stage is then computed in order to normalize the image [16], improve the image intensity [17], suppress the unwanted distortion and enhance the features [18]. The process involved converting the sample images of RGB components to HSV representations [19].
After that, thresholding technique is computed to segment the bottle [20]. The image intensity value which has above threshold value is characterized as a foreground value and the remaining pixels are a background value. Morphological operation is the process that utilizing of erosion and dilation [21]. The operations of morphological are computed to remove small pixels and to create the same size of the output image. Each output image pixel is compared with the input image pixel of its neighbours. The analysis using the morphological operation technique required conversion of the images in binary and in grayscale form. From the set of mathematical formulation using lattice algebra and integral, the gray levels of an image in pixel between dark and bright regions are separated from the threshold value [22], [23].

Gray level co-occurrence matrix
Gray level co-occurrence matrix (GLCM) is a matrix of relative frequencies in which two neighboring pixels separated by distance d, at angular orientation,  occur in an image [24]. The GLCM,  Figure 3. The matrices are averaged for all nearestneighbour orientations. Therefore, G(u,v) is computed as:

Classification
Euclidian distance is used to compute the threshold value between reference and test data. The formulation can be shown as:  where P1(i) is a reference image and P2(i) is a test image from the histogram bins. The details explanation of the formulation has been discussed in [26]. The rule-based classifier is then used to classify the beverages defect between pass or reject [27]. The "if-then" logic ruleset has fewer rules, thus it is a more compact and simpler representation [28].

RESULTS AND DISCUSSION
The proposed technique has been tested with 500 bottle images where 100 samples are reference image and 400 samples are tested image. Figure 4 illustrates the example of color concentration for reference images used in the experiment. The reference images consist of strawberry as shown in Figure 4(a), fruitade as shown in Figure 4(b), orange as shown in Figure 4(c), grapes as shown in Figure 4(d), and citrus as shown in Figure 4(e). Images of Figures 5 and 6 show the example of tested images for fruitade and orange, respectively. Both figures demonstrate different concentrations, with less-concentration images are shown in Figure 5(a) and Figure 6(a) and over-concentration images are shown in Figure 5(b) and Figure 6(b). Figure 7 illustrates the color analysis process to detect defect in fruitade concentration. The process started with the conversion of RGB image in Figure 7(a) to HSV color space. Only the saturation component is taken as shown in Figure 7(b) and to convert back into RGB in Figure 7(c) without the background image. The RGB image was then transformed into GLCM with the number of resolution level N g =256 to extract the color texture of the image as shown in Figure 7(d). Figure 8(a) shows the RGB image of a green fruitade concentration, and Figure 8(b) illustrates the contour plot of a computed GLCM. Here, the GLCM, G(u,v) is computed for 12-bits length, Ng=256, d=1 and at average orientations for each RGB component in a contour plot. This contour represents the cooccurrence frequencies, or the number of repetitions between each pixel pair, u and v. The red dash-line shows a cross-section at u=v. At this line, the co-occurrence frequency is the highest, and it is diagonally symmetrical. The colour concentrations are higher when the transitions increase off-diagonally with the matrix. GLCM cross-section can be constructed in which each entry at u=v is plotted.    Figure 11(a), less concentration as shown in Figure 11(b), and over-concentration as shown in Figure 11(c). Their respective GLCM contour plots are computed and shown in Figure 12 From the contour plots, the 8-bit of red, green and blue histograms is constructed that consists 10 bins for each color. Figure 13(a)-(c) show the results for red, green and blue component of reference image for fruitade (green). Based on Figure 13(a)-(c), the first bin (highest bin) show the value of black color that indicates as background image. In this study, background image was neglected and only bin 2-10 was taken for further analysis. Then, all of these histograms will be combined into one histogram for the classification process. The combination of red, green and blue histogram is shown in Figures 14(a)-(c). According to these figures, the red, green and blue color are combined and divided into three components. The first nine bins are represented as the red component value. The green component is between bin 10 until bin 18 and lastly, the blue component is placed in the bin 19 until 27. The last three bins are ignored which indicates as background color image.
In determining the distance value of two histograms, bin 1 to bin 27 is taken to be calculated using Euclidian distance formula as indicated in (1). The distance value will differentiate and classify the color of the images either pass or failed. The threshold value for each color has been set that tabulated in Table 3. Table 4 shows the Euclidian distance value for all the colors either pass or failed. The color concentration is classified using rule-based classifier as illustrated in Figure 15. The accuracy of the proposed technique has shown in Figure 16. The experimental result shows that the proposed technique has achieved 100% accuracy using 500 samples image. To evaluate the performance of the proposed technique, different color of the sample image is used, and it also depends on the number of the sample image.   Figure 16. Accuracy of the proposed technique

CONCLUSION
This paper presents the analysis of color concentration detection using gray level co-occurrence matrix (GLCM) technique. The process was started by carried out the intensity normalization during preprocessing stage. In color analysis, the original image is converted into HSV color space. GLCM technique was proposed in this paper to extracts the color information of the image. From GLCM process, GLCM cross-section and GLCM contour plot were constructed to show the color texture of the image. The RGB histogram is plotted from the GLCM contour to calculate the distance between two colors. Euclidean distance classifier was used to classify the color of the image either pass or failed. The results show 100% accuracy has achieved makes the proposed technique is successful to classify the color concentration of the image and can be implemented in the manufacturing industry.