Pineapple maturity classifier using image processing and fuzzy logic

Received Apr 27, 2020 Revised Aug 2, 2021 Accepted Aug 20, 2021 This paper describes the development of a prototype using an image processing system for extracting features and fuzzy logic for classifying the maturity of pineapple fruits depending on the colors of its scales. The standards that the system used are from Philippine National Standards for fresh fruits-pineapple for the 'queen' variant. The prototype automatically classified the maturity of queen pineapple variant grown in Munting Ilog, Silang, Cavite, Philippines. Data gathered are from the images loaded into the system using a camera unit under a controlled environment. The images loaded consist of the three faces of the pineapple sample, each with 120degree coverage to capture the whole 360-degree view of the scale. The images then are sent to the system of the prototype where the features of the images are segmented based on the RGB color reduction. By using the fuzzy logic classifier, the obtained experimental results showed 100% accuracy for both the unripe and overripe maturity and 90% accuracy for the under-ripe and ripe maturity classification. The results obtained show that the developed image processing algorithm and the fuzzy-logic-based classifier could be used as an accurate and effective tool in classifying the maturity of pineapples.


INTRODUCTION
Philippines is known to have fertile soil and its favorable climate makes it a great location for tropical fruit production like the production of pineapple fruit [1]. After Thailand, the Philippines ranks second among the countries with leading exporters and producers of fresh or processed pineapple fruits, and pineapple products is listed third in the top 10 agricultural exports of the Philippines [2]. Based on food engineering researches, it is suggested that the maturity of the fruit, which is related to their texture and color, can be classified by their external appearances and physical characteristics and this can be applied when grading or classifying pineapple maturity [3]. For maturity determination by external appearances, factors included are color, shape, texture, and patterns or relative location of the scales, while some of these features may be apparent other features are not [4]. In marketing the pineapples, quality must be checked and ensured before they are allocated to customers such as huge companies or even local consumers [5]. The grading of a pineapple fruit quality and maturity has to be decided based on the standard criteria set by Philippine National Standards, PNS/BAFPS 09:2004 for Fresh Fruits-Pineapple, where it concerns pineapple size, weight, scales, variety, and eyes [6].
Manual classifying of pineapple fruits could be done incorrectly by farmers due to human error ISSN: 2252-8938 Pineapple maturity classifier using image processing and fuzzy logic (Edwin R. Arboleda) 831 since they base it on personal perspectives and they do not have constant standards to go by; this is why classifiers are used to help give constant assessments for the pineapples and help reduce human error [7]. Manual classifying of fruits can easily be affected due to being costly and the possible shortage in labor, that's why some farmers propose a more cost-effective process or way of classifying fruits by making the process automated [8]. For the automated process of classifying, it could be done through image acquisition where the image of the fruit is processed under different methods and results are given after; this helps provide a better and more accurate classifying method for the farmers [9]. There has been the use of different kinds of classifiers for grading pineapples and an example of this is the use of convolutional neural network (CNN) to classify pineapples based on skin color which gave a 90.77% accuracy after evaluating the system using its training data set [10]. Another study used firmness, soluble solids, and resonant frequency to classify pineapples while artificial neural network (ANN) is used to analyze the data given and it showed 83% accuracy for three classifications of pineapple which are unripe, ripe, and overripe [11]. Raspberry Pi is one of the microcontrollers that can be used easily because it is compatible with most computers [12]. It is used in various automation systems because Raspberry Pi serves as the brain or the main intelligence of the system [13]. For many years, a lot of researchers chose to use Raspberry Pi for their prototypes because it is small and easily stored, it is low cost, and it can easily adapt to different programming languages and modes [14]. This microcontroller is also known to be an example of a single board computer and can run various desktop applications [15]. This prototype uses Raspberry Pi for its GPU, processor, RAM, and a MicroSD card that serves as its server where data can be stored and can make the automation process easier [16].
The use of image segmentation or the use of thresholds in image processing helps get information that can be used to the advantage of the researchers which are from images without human aid [17]. When making any kind of a vision system for a prototype, the image segmentation which is from algorithms is crucial. It shows how efficient the system can be since it is where the image is broken into parts and those segments are analyzed to get the needed data [18]. In doing image segmentation for this paper regarding pineapple maturity, the process aims to divide the image into regions and spot specific details with properties such as texture, color, and gray level, and this process is considered as a key step in doing image analysis for classifying [19].
Color reduction is a method that can be used after image segmentation where each of red, green, and blue pixels detected from the image is then compared to other components so that the detected noise or complexities may be removed [20]. Color information is another algorithm used for computer vision applications where color reduction is an example. This is where color information is used to recognize, detect, track, and different objects from the image to help retrieve helpful information for the process of classifying pineapple maturity [21]. There are studies regarding the classification of fruits using nondestructive methods where image processing is used with RGB color classifications to detect the colors and patterns from the external appearance of a pineapple fruit [22]. To speed up the process of classifying fruits and to ensure the consistency of giving good quality fruits to the consumers, image processing was used in classifying and decision-making to eliminate human error [23].
Fuzzy logic has been implemented for many years now as a help to classify different objects of different species from each other, and this helps the process of classifying the maturity of the pineapple fruits [24]. The use of fuzzy logic classifier in this system helps in the decision-making based on the data set given and the fuzzy-in rules, and it also helps interpret the data easier [25]. In a study, the method of RGB color reduction and fuzzy logic is used to make a more efficient way of grading fruits, and it showed that the system has high accuracy compared to human grading meaning that the system is a huge help in agricultural application [26]. Fuzzy logic was applied and proven accurate in categorization of fruit maturity of apples which utilized its external appearance [27]. In this study external appearances, such as color and pattern of the eyes were used as inputs to fuzzy logic to come up with an automated system. The methods and algorithms mentioned are implemented to make the prototype for pineapple maturity classifier.

RESEARCH METHOD 2.1. Research design
Pineapples need to be graded according to their general appearance, quality, and condition [28]. This study aims to classify pineapple according to its general appearance through the use of image processing and fuzzy logic-based classifier. The pineapple fruits used in this study were of the queen variant and from a farm in Munting Ilog, Silang, Cavite. The pineapple fruit variant used represents the higher pineapple cultivar planted in the Philippines. Through the use of image processing, the RGB content of the sampled image was determined and exported as input to the classifier. The fuzzy logic-based classifier processed and grouped the input features base on the defined parameters. Figure 1 shows the indices assigned to the general appearance of the queen pineapple variant through the color of its scales [2]. Table 1

Research procedure 2.2.1. Image acquisition
The snapshots of the pineapple samples were taken by placing the sample within the scanning box to obstruct unwanted light. The dimension of the scanning box was based on the average size of the queen pineapple which is 20 inches by 15 inches box as stated in [29]. The camera was positioned at the top of the box directly above the sample, the light used in the image acquisition is a ring light. Each sample was captured three times with a 120-degree angle to obtain the full 360-degree view of its scale. The camera used in obtaining the sample images is the Sony IMX219 camera module with 3280 by 2464-pixel static images. The images were stored and processed using a Raspberry Pi v4. Figure 2 shows the image of the pineapple fruit sample's three faces.

Pre-processing of image
The pre-processing of an image is done to enhance the quality of an image before it undergoes the system process [30]. The Python 3.8.0 was used to develop an algorithm to preprocess and extract features from the samples. The images of the faces of the pineapple sample undergoes a background subtraction following the edge, in an elliptical shape, of the fruit. The background of the image contains unnecessary data that may affect the process of color analysis [31]. To return the user's detailed interest, a region-based feature is preferred which is largely dependent on segmentation techniques [32]. Image segmentation is the process where each pixel is assigned with a label, where pixels with the same label have the same characteristics [31]. The image segmentation presented in the study was done through Otsu's two-level thresholding. The main purpose of image segmentation is to divide the image into isolated parts with identical and constant characteristics like intensity, tone, color, and texture [33]. In Otsu's thresholding, it is presumed that an image is composed of two parts the background and the foreground. To attain the maximum separation between the two, Otsu's thresholding gives the best threshold values making their inter-class variance at maximum and their combined spread at a minimum [34]. Figure 3 shows the image segmentation process performed on the sample.

Feature extraction
Once the images of the samples were pre-processed, the features needed were extracted. The feature extracted from each face consist of red, green, and blue pixels. RGB color space is composed of red, green, and blue spectrum components combined to produce different color space models [35]. The components of RGB each represent a value from 0 to 255. A pixel in an image has a color from the combination of these three-color components [36]. Figure 4 shows the RGB representation on a pineapple using Ginifab-Color Picker App. To provide a yellow color space from RGB, the combination of red and green color is enough [35]. In this study, the blue color space is included to produce a lighter color of yellow and further expand the range of definition of the color. The ranges for every color were divided from the original colors (red, green, blue) to the lighter shade (light red, light green, light blue) to the lightest (very light red, very light green, very light blue). These topics are further discussed and visualized in the classification stage.

Classification
Fuzzy logic was used as the classifier of the maturity of the pineapple fruit. A fuzzy logic classifier is a tool that maps the input pattern to give an accurate output [37]. Fuzzy logic is commonly used as a classifier because of its simple way of providing a defined conclusion out of imprecise and indefinite input [38], [39].
The fuzzy logic toolbox in MATLAB was used in developing the fuzzy logic-based classifier. For the classification, the features extracted from 100 samples were used to develop a database and 80 samples were used for testing. The fuzzy logic system that was developed has three inputs with three membership function each. The three input represents the red, green, and blue color spaces extracted from the sample image. The three membership functions define the ranges set for the RGB. There are 18 rules to be the basis for the input pattern and the number for the choices for the output are 4. Figure 5 shows the Mamdani type fuzzy logic system developed for the study. The three inputs are composed of the red, green, and blue color spaces extracted from the image. The output has four decisions: Unripe, Underripe, Ripe, Overripe.
Each input is composed of three membership functions. The membership functions for the input were all based on the graph from the features extracted. The ranges of the membership functions vary from the original color (i.e. red) to light color to very light color. The membership function type used in the inputs was trapezoidal and triangular. Trapezoidal membership function was used for color spaces with high ISSN: 2252-8938 Int J Artif Intell, Vol. 10, No. 4, December 2021: 830 -838 834 recognition percentages, those that can be recognized instantly and can be represented by a wide range of values. Triangular membership functions, on the other hand, was used for color space with definite points of ranges. Triangular membership functions tend to overlap when the values for the color space become too close. Figure 6 shows the three membership functions for each of the color spaces' input.  Pineapple maturity classifier using image processing and fuzzy logic (Edwin R. Arboleda)

835
The centroid method is used in the defuzzification of the output. The set of rules that are the basis of the input pattern to give an output was displayed in the rule viewer. Eighteen rules were developed for the system. Figure 7 shows the rules viewer that is composed of the membership functions combination using the OR operation and the corresponding output.

Figure 7. Rule viewer
The output of the system is shown in Figure 8. It is composed of four membership function namely the Unripe, Underripe, Ripe, and Overripe. The unripe and overripe membership functions were trapezoidal because a part of them has color characteristics that can be distinguished instantly (i.e. the extreme values of green and the extreme values of yellow). The underripe and ripe classification tend to have close values and may share some color characteristics. They were represented by a triangular membership function.

RESULTS AND DISCUSSION
In this experimental setup, the combination of the RGB color spaces extracted from the image sample was used to obtain a corresponding yellow pixel percentage that classifies the maturity of the pineapple sample. The image processing algorithm developed consists of background segmentation and RGB extraction. In recent studies of pineapple maturity classification, [35] the method used in focusing the region of interest involves cropping and resizing of the sample image. This method obtained high accuracy (lowest of 85% and highest of 100%) having classified three maturities of pineapple but the region examined included some part of the background of the sample. Another study developed a mask for the shape of the sample to separate it from the background which also gathered high accuracy (94.29%) but only covered a part of the pineapple scales [31]. The image background subtraction developed in this study consists of capturing three faces of a pineapple sample to cover all of its scales and using Otsu's two-level thresholding to separate the exact shape of each face captured from its background.
Testing data is used to assess the performance of the proposed model during and after training. In the training stage, 100 samples were used, while 80 samples were tested during the testing stage. Table 2 shows the Yellow color space ranges as extracted from the RGB color spaces. The result from Figure 9 shows the normalized data for the test data classification through the fuzzy logic technique. The white boxes represent null or zero, the blue boxes represent the right classification and the pink boxes represent the misclassified samples. The result indicates that for the 20 samples tested under the unripe maturity classification, an average of 100% accuracy was obtained. Likewise, the 20 samples that were tested for the overripe maturity classification, an average of 100% accuracy was also obtained. For the underripe maturity classification, a 90% average accuracy was obtained where 18 out of the 20 samples fall under the range of the classification while the two misclassified samples belong to the ripe maturity classification. The ripe classification also obtained a 90% average accuracy from 18 out of 20 samples that tested and showed a result within the range of the classification. The two misclassified samples from the ripe maturity belong to the underripe maturity classification. The misclassification that happened in the ripe and underripe maturity classification led to a 10% average error for each maturity respectively and an overall average error of 5%. Pineapple maturity classifier using image processing and fuzzy logic (Edwin R. Arboleda) 837 From the test data, it can be said that the fuzzy logic-based classifier has high accuracy in classifying pineapples within the overripe and underripe maturity. Despite the small amount of error obtained from the classification of the ripe and underripe classification, high accuracy is still obtained in classifying the ripe and underripe maturity. The system's overall accuracy is still high, compared to the accuracy obtained from recent studies, at 95%.

CONCLUSION
In this paper, a new image processing algorithm was introduced in categorizing pineapple into four maturity classifications: unripe, under-ripe, ripe, and overripe. The system automatically classifies the maturity of queen pineapple variant grown in Munting Ilog, Silang, Cavite from the images loaded into the system through a camera unit. The experimental results obtained from the classification through the fuzzy logic-based classifier show a 100% classification percentage for both the unripe and overripe maturity classification and 90% for both the underripe and the ripe maturity classification. The overall accuracy of the developed system is 95%. The results obtained from this study with the use of a newly developed image processing algorithm and the fuzzy logic-based classifier in classifying the maturity of pineapple is highly recommendable. Not only did it exhibit a more reliable approach in image processing by covering the 360degree-view of the scales of the pineapple but also a high accuracy in differentiating the four maturity classification of pineapple.