Labeling of an intra-class variation object in deep learning classification

ABSTRACT


INTRODUCTION
Artificial intelligence (AI) technology has been widely applied without users realizing it.All devices are equipped with intelligent drive-by software that can recognize user-environment characteristics and connect to the networks.The global goal of AI is to provide a method to solve problems that humans perform intuitively and near automatically.AI-related to work in inference, planning, heuristics, or automatic machine reasoning [1]- [4].Machine learning (ML) is a subfield of AI technology.ML more specifically focuses on pattern recognition that is learning from a dataset.ML allows a device to take actions or decisions based on optimal analysis results obtained after learning from the trained datasets.This allows users to get results or decisions quickly and are autonomous.ML has been widely applied in various fields to solve complex problems involving large volumes of events or data and requiring fast optimal decisions.One of the main things about ML is the object classification process [5]- [7].Deep learning (DL) is a subfield of ML.DL uses a raw input to self-learning from data and not selected features that have been designed.Features then automatically gain from learning in the training process [8]- [13].The convolutional neural networks (CNNs) is a computer vision subfield that leverage ML to be the most powerful object image classifier [14]- [18].Considering a large of neural networks, the term of deep consists of more than two hidden layers that involve a huge dataset to be trained.DL now is a thriving field with many practical applications and active research topics [19]- [22].Although DL and CNNs have demonstrated that powerful image classification is robust under various challenges, to be accurate and functioning, DL scope object classification as tight and welldefined as primarily possible uses a 2-Class object for instances using mask or no-mask.However, development on the DL object classification faced many challenges, i.e., semantic gap (difference perceives, the human versus computer represented an object), and variation factors (the intra-class variation) [23]- [28].
In promotes deep learning forward to the state-of-the-art, researchers through internet communities develop network models and datasets.The first network model is LENET architecture with modified national institute of standards and technology (MNIST) dataset.Intending to classify handwriting (numbers 0-9) with significant accuracy usually uses as a benchmark of ML algorithms.Another dataset for the image classification algorithm is ImageNet, which consists of more than a thousand objects of everyday lives.The intra-class variation present in every object, its nature of diversity of an object.Every object has a class, for instance, class object dog.If an intra-class variation is used, the class object becomes a name of an object, for instance, a dalmatian (a white-dog with black-spot).Unfortunately, not every object has a specific name, in exceptionally high similarity inside the category.This paper explores this problem in flower plants' taxonomy naming, but the possibility also for intra-class variation of person, despite the fact person already has a given name or ID-number; instead, a visual feature can be a substitute for future works [29]- [31].
Building of image classifier model can be used available dataset through the DL internet communities.These datasets (MNIST, CIFAR, Flowers, Caltech, ImageNet, CVPR, STANFORDCARS) generally consist of a massive image of a noun class, i.e., cat, dog, or panda in the animals' category or others, and uses as a benchmark for machine learning algorithm study.Class labeling and categorizing these datasets are organized according to the WordNet hierarchy called a synonym set or synonym for short.As seen, object labeling with their actual name, i.e., the name species of flower in Flowers-17 dataset or yearmaker-model-car in STANFORDCARS dataset [32]- [35].
The objects that have huge intra-class variation are difficult to have a given name.For the sake of simplicity generally uses their major-class name as a label.Indeed, the DL classification needs the image datasets to have labels associated with them.In the supervised learning algorithm of DL, the process needs to see these labels to teach itself how to recognize each class.The different classes must have unique labels, i.e., incremental integers.However, labels must be meaningful of a word or phrase that humans are familiar [36]- [38].
This study explores the intra-class variation of the flower Plumeria L (local name Bunga Jepun), a genus of flowering plants in the family Apocynaceae.In personal communication, the local botanical claim has collected about 400 varieties of this flower.Few varieties have registered in the standardized taxonomy and nomenclature database (Interagency Taxonomic Information System, itis.org),remain has a no-given name [39].

RESEARCH METHOD 2.1. Method of labeling
This study proposes an ID system for the intra-class variation of flower Plumeria L labeling for the DL classification.The term flower in this study meaning a single piece of flower, not a bunch of flowers bound at their plant stem.Hence, image pre-processing such as image segmentation is out of the scope of this study.The ID label consists of feature extraction from the object flower that can be retrieved using computer vision during the preparation stage in DL classification.Therefore, this method can fully automatically [38], [40], [41].The feature that can be extracted from flower Plumeria L: i) Shape.The flower Plumeria L in general has five petals and clearly can be used to distinguish them from other flowers.In an intra-class variation, the shape of individual petals, their configuration, and the overall shape of the flower are similar.Hence, the shape features are not considering in creating the flower ID label [7], [25], [32], [33].ii) Scale.The scale of flower Plumeria L has variation in petal dimension, varies from has small to a big petal, some flowers have wrapped up the petal and others have blooming petal.Moreover, found in one stem, there is variation in scale as well.The dimension of the flower will vary in viewpoint.For the sake of simplicity, viewpoint only two will be considered, i.e., a natural or non-pose and petal pose [2], [6], [15], [31].iii) Texture.The flower Plumeria L in general has a similar texture of the petal.Some variety has softer fine thin of petal texture.However, it cannot capture detail within a tiny image size needed for the DL classification.Hence, the texture feature is not considering in creating the flower ID label [41]- [43].
Labeling of an intra-class variation object in deep learning classification (Putri Alit Widyastuti Santiary) 181 iv) Color.The main characterizes of the flower Plumeria L is petal color.The dominant petal color is bright white to bright red, dark color scarce, i.e., green to blue.Some variety has dark deep red color.The ID color of the petal will be extracted from the peak of their BGR histogram color [33]- [35], [44].Therefore, the ID label will consist of scale and color features of petals in format ss-peakBGR.
Figure 1 shows four stages of the DL classification in general; at the preparation stage (block number 1), the proposed method of creating ID label is most useful, where the class images must be associated with a label.The remaining stages, i.e., dataset splitters, train networks, and evaluation progress as it is [42]- [44].In the preparation stage, the flower Plumeria L will be processed to extract the perspective dimensions of the petal in cm unit.The extracted peak of blue green red (BGR) color histogram in the 8-bit unit.Creating ID label for each variant or class of flower Plumeria L. Archives hundred of a pose, non-pose photograph of flowers.Creating dataset completely with a label in CSV format.

Dataset collector
Flowers collect directly from the plant and process to the image-workbench not within five hours to ensure petal is fresh to prevent bias in measurement.The image workbench is set up with a camera and light.Exposer light ensures no shadow effect and camera setting with a minimum of lens distortion to minimize interference or noise image output.For consistency of the image output image workbench equipped with a 15×15 cm of green canvas paper ink-jet printing to void reflected light to the camera, finally image workbench operated by python code running on pc with NVidia graphics processing unit (GPU).
Camera position calibrated exactly perpendicular and center above the object.The edge of the green canvas makes a constraint controller for camera position and area to captures.The pixel metric (size of a pixel in cm) was obtained by calibrating the camera with a known dimension object in cm, such as a 7×7 cm glass cap. Figure 2 shows the illustration at image workbench capture progress to feature extraction.Figure 2(a) and Figure 2(b) show the scale or the flower dimension feature extraction in pose and non-pose of the flower petal.The python code will create a rectangle bounding box in cm unit after scale calibration process (in figure mark as dim A and B in cm unit).Note that results will be dynamic according to the viewpoint.Figure 2(c) shows the peak of histogram per channel BGR of flower under evaluation.In axis-x, an 8-bit color resolution, and axis-y is a histogram frequency.In figure, a peak mark as letter x.
Figure 3 shows the Author operates a python code to creates the flower Plumeria L dataset for deep learning on a set of photography workbench that uses in this study.In figure, the photography workbench consists of a webcam installed on a rounded LED light on an adjustable vertical tripod.The height and orientation of the webcam from the table are adjusted using a green canvas as a reference that displays on a pc monitor.Then, the flower is placed over the green canvas for measurement with computer vision.

Accuracy of labeling
The labeling process in this research is limited to the preparation stage in Figure 1 (block 1).So that, the evaluation stage (block 4) in Figure 1 to obtains the accuracy of test and validation of classification results is out of the scope of this research.The flower Plumeria L ID label results from section 2.1.were obtained from computer vision evaluated by comparison with manual measurement of petal flower dimension.This measurement uses a laser distance meter (LDM) tool that has 2.0 mm measuring accuracy.A set of equipment was prepared for that purpose shown in Figure 4.The LDM measures the distance between laser sources with a reflector where the flower being measured is placed.The flower position is adjusted to get 2 flower dimensions i.e., the shortest and longest distance between laser source and reflector.The LDM is set to automatic mode to measure the distance of each in 1 second.The flower petal measured in 5 seconds, result in the dimension maximum, minimum, and averaged value for 5 times measurement.The accuracy of labeling is evaluated using the root mean square error (RMSE) and Pearson's coefficient of correlation (r) of output between the computer vision with LDM labeling [45], [46].

RESULTS AND DISCUSSION
Unfortunately, the flower collectors cannot collect about 400 varieties of flower Plumeria L due to a different blooming session and time-consuming issue.For an initial study step, about 28 varieties of flower Plumeria L have been collected and processing.See section discussion for further, how flower Plumeria L has reacted 400 child varieties and compared with taxonomy databases naming [28], [36], [38].
Table 1 shows the results of scale-color feature extraction of flowers from the image workbench.This table shows four representing the smallest to biggest petal flower collected during the first periods of research.Other 24 remain variant datasets not possible to show in this paper.In Table 1 column header contains Item ID (necessary marks of flower), computer vision (CV) results of scale dimensions of flowers in a perspective way in cm unit (dim A and B), the peak of the color histogram channel (BGR), LDM results of scale measurement in cm unit (dim A and B), RMSE and Accuracy (r).All values are computes and averaged from 200 image samples (n=200) [32], [34], [35].The result shows overall accuracy from the comparison between CV with LDM method has reached 99% with RMSE in 3 mm.So that this proposed method of labeling of intra-class variation of flower Plumeria L can be justified.Using this method for labeling, images will be associated with class ID and their label ID, e.g., class id 6 labels, 4.3-4.4-0-234-244,and so on.After 75% of the flower collection and processing are complete, the scale features of petal flowers will be categorized into five Linkert scales, i.e., biggest, big, medium, small, smallest (bb, b, m, s, ss) for short in labeling [36].Figure 5 shows the histograms of petal flower of the Plumeria L obtained from computer vision.Figure 5(a) for class ID: 6 (the flower variant in Figure 6), Figure 5(b) for class ID: 4 (the flower variant in Figure 7), and histogram Figure 5(c) for class ID:16 (the flower variant in Figure 8).In Figure 5, the peaks of BGR channel shows in black-cross marks (x).The bins in digital number of axis x obtained after traces histogram values with axis x range.The peak values in Table 1 result is a mean peak value computes from 200 image samples.As seen from Figure 5, the flower variant that has high similarity can be distinguished from their peaks of color channel profile [32], [34].
Figures 6-8 show the result of scale feature extraction capture from the image workbench.Figures 6(a)-8(a) are for non-pose of petal flower, while Figures 6(b)-8(b) are for pose of petal flower.In Figures 6-8 show of each image have four red dots that creates a rectangle with green color line.The red dot is computed from the rotated bounding box of the specified contour flower.Then added four blue dots at half of the green line length with the magenta color line where the dim A and dim B are obtained.In Table 1 genus is mask as none that means no associates of taxonomy naming in the database for this variety of flower Plumeria L [39], [44].The world checklist of selected plant's families (wcsp.science.kew.org)registered 160 proposals of genus Plumeria taxonomy names.However, only 21 approved remain not accepted by status.For instance, the Plumeria Tourn.ex L. is approved [18], [23], [31], [41], [42].The Plumeria is a flowering plant that is easy to be cultivated, by grafting technique; hence cross cultivation can be with any varieties, resulting in about a hundred (400 variety, personal communication) varieties of petal flowers.

CONCLUSION
The flower Plumeria L has a huge of intra-class variation, from scale to the color feature of a petal flower.Not of each variety of flower Plumeria L has a taxonomy naming.Few varieties have been approved.In the supervised learning algorithm of DL, the image datasets must be labeling with a meaningful word or phrase that humans are familiar with, the taxonomy naming.Labeling with visual feature extraction brings a fully automatic classification, which can be processed in the initial preparation stage.The flower Plumeria L labeling extracted from the perspective scale of petal dimension and the peaks of BGR color histogram channels of a petal flower.For short, the perspective scale of the petal dimension is categorized into a five of the Linkert scale.This method presents a novelty in building datasets for intra-class variation for the DL classification.

Figure 1 .
Figure 1.Propose method of ID-label extraction, where processes integrated within DL classification in the preparation stage (block 1) The code automatically extracts scale features by a rectangular bound of flower in a perspective way, including flower dimensions.The code also automatically extracts color features by computing the peak of BGR color histogram and return intensity bins value in 8-bit format.The images for one variety of flowers captured in a non-pose and petal pose within 200 times vary in viewpoints and flowers.The image dataset has 340 by 340 pixels with 24-bit color depth and 170 kB file size.The averages of the processing time for labeling of one variety of the flower Plumeria L with 200 images capture approximately 23 minutes 45 seconds or just for one image approximately 7 seconds.

Table 1 .
Tabulation of scale-color feature extraction of Plumeria L