Analysis of machine repair time prediction using machine learning at one of leading footwear manufacturers in Indonesia

ABSTRACT


INTRODUCTION
Based on export data released by the Direktorat Jenderal Bea dan Cukai Indonesia (Indonesian Directorate General of Customs and Excise) taken from export declaration of goods documents in 2020, the value of Indonesia's exports decreased by 2.68% from the previous year, reflecting the impact of COVID-19. The decline in the value of Indonesia's exports in 2020 was caused by a 30.01% decline in commodity exports of oil and gas, as well as a 0.61% decline in non-oil and gas exports [1]. The role of Indonesia's oil and gas and non-oil exports has shifted. Non-oil and gas exports accounted for 94.94% of total exports in 2020, an increase of 1.97% from 2019. One of the non-oil and gas commodity sectors, namely the manufacturing industry sector, has the largest contribution to total exports. Its contribution will be 80.33% by 2020. One of the export commodities that experienced an increase was the footwear sector, especially sporting goods. Export value increased by 31.09% in 2020 compared to the previous year. Indonesia is among the top four countries with the highest number of footwear exports, still behind China, India, and Vietnam [2].
The footwear industry is a labor-intensive industry [2]. Indonesia has about 18,687 footwear business units with a workforce of 795,000 manpower, which are then followed by the required number of machines. This is to meet high demand for shoes, but still limited in technology and capital, so that Indonesia can compete with other exporting countries, especially China and Vietnam. If the focus is on the needs of the machine, then the function or type of machine used is also very diverse, adjusting to each process and model of footwear that must be done. This research will focus on the cutting, stitching, and assembly (CSA) process because most processes and stages of manufacturing and assembling components are carried out in that process. Based on data that has been collected in one of the leading footwear manufacturing industries in Tangerang, Indonesia, it takes about 10,633 machines in the CSA process, which consists of 106 types of machines and/or 375 series of machines that must be managed properly. With many machines that must be managed, the handling and control of machines must also be carried out very well. The effectiveness of several machines in supporting production activities to fulfill exports is closely related to the availability and reliability of machines. Technicians always try to ensure machine availability and reliability are maintained through regular maintenance activities [3], but due to the large number of machines used, machines sometimes break down before maintenance time is completed [4], [5].
In 2021, the number of machine breakdowns that occur in production lines is very high. Currently, all machine breakdowns that occur in the CSA process are well recorded using the Andon system. Besides being used as a tool to call technicians, Andon has also been used as a tool for storing historical information on machine breakdowns [6]. Five machines in particular had a high breakdown rate and made the biggest contribution to the overall breakdown, namely: 1N postbed stitching machine, 2N postbed stitching machine, computer stitching machine-Medium, skiving machine, and computer stitching machine-Large. In addition to availability, the reliability of the machine must also be controlled properly. The average time for repairing a machine breakdown from the five machines above is 27.6 minutes.
The export process will be disrupted if the production process has problems included because of machine breakdown time, so machines that are broken for more than 20 minutes must be worked in the machine warehouse. The next problem is that the technicians cannot predict it. In addition to the uneven distribution of technician expertise, this is also because they still tend to rely on experience and intuition in estimating repair time intervals. Historical data on machine breakdown is recorded digitally through the Andon system, but the data is still not properly utilized to assist decision making. The aim of this research is to analyze historical data on machine breakdown to provide predictions of time intervals for repairing machines with a focus on finding the best algorithm accuracy using a machine learning approach.

METHOD
This study uses the cross-industry standard process for data mining (CRISP-DM) model process for a universal data analysis approach [7], [8]. Several supervised machine learning classification method algorithms were chosen to get the best accuracy value [9], [10]. The algorithms are logistic regression (LR) [11], [12], naive bayes (NB) [13], [14], random forest (RF) [15], [16], k-nearest neighbor (KNN) [17], [18], and support vector machine (SVM) [19], [20]. Selected five classifications supervised machine learning based on each algorithm have on different dimension metrics there are parametric-simple for LR and NB, parametric-complex for SVM, non-parametric-simple for KNN, and non-parametic-complex for RF. Several dimensions of the variables involved in this study include reports of machine breakdown, such as repair time, response time, machine model or type, building location, type of breakdown, causes of breakdown, and repair solutions. Variable dimensions of assets, such as asset number, machine age, machine price, machine arrival date, and machine ownership status. Machine replacement, such as the location of the old and new buildings. Employee variables such as position and years of work experience. The Framework in this study describes how the concept of data utilization using machine learning methods [21], [22]. An overview of the framework can be seen in Figure 1, how data sources get and which variable is used, next go to how data are transformed with several data modeling, performance analysis, and finding the best model to get the output of the prediction accuracy. There are two evaluation metric methods that will be used: confusion metrics [23] and receiver operating characteristic -area under curve (ROC-AUC) [24]. Confusion metrics is an evaluation model to find the values of accuracy, precision, recall, and F1 score by looking at the probability value between actual value and predicted value at one threshold. Meanwhile, ROC-AUC evaluates all possible performances at all thresholds that are under the ROC curve area. At the end of this study, ROC-AUC will be used as the main evaluation [25], while the evaluation of confusion metrics will be used as a supporting evaluation [26]. To be more effective and time efficient all machine learning methods are processed by platform orange data mining and do some setup parameters in some algorithms. LR set up on a ridge with a coefficient score is 11, RF setup with 10 number of trees, SVM setup with cost is 10 and regression loss epsilon is 0.10, and KNN setup with 10 number of neighbor and metric euclidean.
In addition to using a statistical approach, another way of determining variables in addition to using a statistical approach is through a domain expertise approach, with direct brainstorming to the engineering department leaders starting from the supervisory level to the manager level, through focus group discussions (FGD) [27]. After several independent variables have been determined, a correlation test is carried out with the dependent variable, that is, repair time interval, using the chi-squared test [28]. Prior to modeling, data will be divided into two with a ratio of 70:30, 70% as training data and 30% as test data. The flow chart of this research is shown in Figure 2.

RESULTS AND DISCUSSION
Several data sets are already recorded digitally, but there are also those that use worksheets in data collection and require a data transformation process from manual to digital with a computerized system. The following is the flow of data used in this research, as shown in Figure 3. Machine breakdown in Andon, spare parts transactions, and equipment assets data are recorded digitally and stored directly on the server, work order forms need to transform from paper to computerize before store on the server. The dataset in this research is exported directly from the internal server and cloud server. The data used for modeling analysis is in the (.csv) format to make analysis easier because this format can be used for many systems and is compact and straightforward [29]. Data that has been collected is then carried out a thorough depiction of data for each variable, as well as checking whether there is empty or missing data contained in each data set.

Data preparation
Data that has been thoroughly prepared, inspected, and described is still very messy and needs to be preprocessed so that it is ready for use in machine learning modeling. Based on the initial description of each data set as shown in Figure 4, where there is missing data, which will then be processed to handle the missing data. Missing data that is still less than 5% does not need to be removed; it only needs to be imputationed by replacing it with the average or median value for numeric data types, and it is replaced based on the frequency with which it appears for categorical data types [30]. The variable target in this research is the repair time interval. Whether the required repair time is less than 18 minutes or not, the determination of the 18 minutes repair time standard is based on the machine breakdown history during 2021, where the average repair time is 24 minutes and the median time is 18 minutes, as shown in Figure 5.   Figure 6, the distribution of machine breakdown time is a positive skewed distribution. The median value is more appropriate to use for the distribution of positively skewed and negatively skewed data [31]. This is what underlies the determination of the machine repair time in this study, which is 18 minutes based on the median result of the breakdown time data.

Data modeling and analysis
The data set is divided into two parts for modeling evaluation: training data and testing data. The distribution of training and test data from the total data set is 70:30. The entire data utilized for modeling comes from 4,163. Therefore, when employing a 70:30 ratio, 2,915 training data and 1,248 test data will be employed. The cross-validation approach will be used to confirm the model's performance for several evaluations. In this research, the modeling will be evaluated using 5-fold cross validation [32].
As seen in Table 1  Predictions made by test data are the result of learning from training data. Machine learning algorithms will perform modeling based on their respective formulations and techniques and then determine the probability of whether to enter the classification under 18 minutes or above 18 minutes for repair time by considering the variables that have been determined in this study. Table 1 compares the outcomes of the modeling evaluation between the training and test data. From this comparison, the difference between training and test data is not considerably different, hence LR has reasonable modeling accuracy. The results of the test data modeling show that the LR, NB, RF, and KNN algorithms are in the satisfactory category, while the SVM algorithm is in the unsatisfactory category. The results of the confusion metrics between the actual and predicted data can be seen in Figure 7.

Model performance evaluation and quality model result
This research seeks to determine the degree of modeling accuracy, using both training and test data. The gap between training and test data is not a serious issue if the difference is not too large, regardless of whether it is overfitting or underfitting. In accordance with the outcomes of machine learning modeling using five classification algorithms, LR is an algorithm that fits the variables in this study with the difference between the training data and the test data being -0.005, meaning that test data is better at 0.005 than training data, or at a percentage of 0.5%, according to Figure 8. Based on the conclusion of data evaluation on training data and test data, for LR, NB, RF, and KNN algorithms in this study, the modeling quality is in the satisfactory category because the AUC value is around 0.6 to 0.7 [24]. As for the SVM algorithm in this study, the quality of the modeling is in the unsatisfactory category because the AUC value is below 0.6, but the modeling is still accepted because the AUC value is still above 0.5. LR algorithms are the best result since the dependent variable is binary and data structure is simple.

CONCLUSION
This research wants to know the level of accuracy of the modeling that has been done both on the training data and test data. The difference between training data and test data is not a significant problem if the difference between the two is not too great, be it overfitting or underfitting. The results of this study prove that historical machine breakdown data can be optimized to predict machine repair time intervals in the production line within under 18 minutes. The accuracy of LR algorithm is slightly better than the other algorithms. Based on ROC-AUC performance evaluation metric, the quality value of the accuracy of LR model is satisfied with a percentage of 69% with a difference of 0.5% between the train and test data. In further research, the variables used can be enriched, so that the percentage of the results of the analysis of the resulting model will be even better. In addition, it can be developed to the implementation stage and integrated into existing maintenance systems to provide real-time predictions. Int J Artif Intell ISSN: 2252-8938 