A self-adaptation algorithm for quay crane scheduling at a container terminal

Received Feb 24, 2021 Revised Jun 5, 2021 Accepted Aug 20, 2021 Quay cranes scheduling at container terminals is a fertile area of study that is attracting researchers as well as practitioners in different parts of the world, especially in OR and artificial intelligence (AI). This process efficiency may affect the accomplishment and the competitive merits. As such, four local search algorithms (LSs) are utilized in the current work. These are hill climbing (HC), simulated annealing (SA), tabu search (TS), and iterated local search (ILS). The results obtained demonstrated that none of these LSs succeeded to achieve good results on all instances. This is because different QCSP instances have different characteristics with NP-hardness nature. Therefore, it is difficult to define which LS can yield the best outcomes for all instances. Consequently, appropriate LS selection should be governed by the type of problem and search status. The current work proposes to achieve this, the self-adaptation heuristic (self-H). The self-H is composed of two separate stages: The upper (LS-controller) and the lower (QCSP-solver). The LScontroller embeds an adaptive selection mechanism to adaptively select which LS is to be adopted by the QCSP-solver to solve the given problem. The results revealed that the self-H outperformed others as it attained better results over most instances and competitive results.


INTRODUCTION
Container terminals are regarded as one of the most significant modern sea freights means of transportation, as they represent the essential connection between local markets and global customers and contrariwise [1]. Thus, their effectiveness is essential to the achievement of the supply chain. In accordance with the speedy yearly growth of transported container volume, no wonder that container terminal now represents one of the predicaments in the international chain of supply [2], and container terminal efficiency a significant issue for liner transportation corporations as they work on reducing their cost [3]. The primary objective of container terminals is to attain fast movement of containers at the minimum cost possible [4]. Therefore, the time the vessel requires to load or unload is generally the terminal's uppermost precedence; the mooring time of a vessel is its turn time [5].
Quay crane scheduling problem (QCSP) is among the most significant operations related to seaport terminals. Its aim is to identify the order of tasks to be made by each quay crane (QC) [1]. Improving the efficiency of this operation will improve the vessel turn time [6]. Concerning QCSP, there is a common belief that the allocations of cranes to vessels are already accomplished. All QCs are made to move on a railway line that runs in a parallel direction with the vessel where each QC cannot cross over to other QCs. The vessel is divided in a longitudinal way to produce several bays where the containers are stored. Containers that are stored in these bays usually are aligned together with their features and criteria, such as weight, size, and origin and destination ports [1], [5]. Based on Meisel and Bierwirth, QCSP is classified into three basic problems that are most prevalent in the research community [7]:  QCSP with container groups  QCSP with complete bays  QCSP with bay areas The problem addressed in the present work is a QCSP with container groups that involve the most significant complexity among the other two classes [5], [7]. Each task, in such problems, is situated at the vessel bay location. Some pairs of tasks situated in the same bay have precedence relations that should be considered in processing these pairs. Thus, these tasks cannot be processed concurrently [1], [8]. In order to eliminate issues of congestion at the yard blocks, simultaneous processing of certain tasks may be disallowed [7], [8]. Each QC is associated with its initial bay position and ready time. During operations, each QC should not be crossed by other QCs. Moreover, two QCs must not work simultaneously at the same bay place, and a safe distance, usually in units of specified bays, must be preserved [1], [7].

RELATED WORK
The early works of Daganzo [9] initiated the study of QCSP. The work involved an investigation of problems with multiple vessels at a berth with cranes that are allowed to move freely. Exact approaches were suggested for handling small issues. A branch-and-bound algorithm proposal was made by [10] to solve actual size issues. Nevertheless, these studies never made necessary considerations for the interference of the quay cranes [1]. Another formulation was made by Kim and Park [11], which is based on a mixed-integer programming model. The proposal made by that study is a Branch and Bound algorithm that seeks to reduce the search space with a lower object connected to define a function value that will lead to an optimal schedule. The high computation time required in Branch and Bound with large instances results in certain limitations when applied in real situations; Thus, the researchers modified a greedy randomized adaptive search procedure (GRASP) that allows for the attainment of good quality schedules within considerably shorter computational times. Moccia et al. [12] conducted an extended analysis of Kim and Park's [11] model and discovered certain situations whereby there is a possibility of interference between the quay cranes. This work provided a revised mathematical formulation to provide solutions for the weakness demonstrated. At the same time, numerous families with solid inequalities were applied to a branch and cutalgorithmm to provide solutions for large instances. Obtained results showed the positive performance of the suggested branch and cutalgorithmm in comparison with the branch and boundalgorithmm developed by Kim and Park [9]. Kasm and Diabat [13] developed a mixed-integer program and a two-stage exact solution methodology to solve the QCSP. Sun et al. [14] address the QCSP with vessel stability constraints, an exact algorithm based on logic-based benders decomposition.
Sammarra et al. [15] suggested that QCSP has a routing problem and, therefore, unable to fix the tasks performed by each quay crane. It also suffers a scheduling problem to determine when to start a task and when to finish. And then, a tabu search has been developed for solving the routing issue. The neighbourhood structure utilized a highly potential promising movement to minimize the volume of makespan obtainable within the present schedule. Moreover, a descriptive graph was used to evaluate the schedule obtained during the search process. The tabu search's performance can be compared with the branch and cut that is proposed by [12], [15]. Bierwirth and Meisel [8] disclosed that the mathematical model proposed by [11] is not correct because it cannot be used to find out schedules of interference among quay cranes at all times. This paper developed a revised formulation of the QCSP by creating a suitable time distance between the two tasks in the design. It offered heuristics known as unidirectional scheduling (UDS) that is used on tree search. The UDS achieved the best-known schedules necessary for benchmarking suites utilized in previous studies within limited computation time. Chung and Choy [16] suggested a Genetic Algorithm for the QCSP. Starting from an initial randomly generated population sample, varied mutation and crossover operators were utilized to obtain high-quality schedules. Legato et al. [17] offered a new approach for QCSP, which adopted numerous practical issues in the form of quay cranes ready times and due date, and individual cranes processing time. Developments were made in this paper based on the UDS extension that was suggested by [10], which is known as the LTM method. Meisel and Bierwirth [7] offered a platform with the intention of comparing models of optimization and approaches for QCSP by generating a benchmark suite. The suggested generation scheme adopted the comparability principle, reproductively and unbiasedness, with a group of parameters that feature services of container vessels. Izquierdo et al. [18] [4], and it adopted a local search for solving the QCSP. Such a scheme was based on a priori view of the problem to attain high-quality schedules. Nguyen et al. [1] creates a new form of priority-based schedule construction for generating quay crane schedules. Such a procedure also brought about the development of two hybrid evolutionary computation approaches that adopted genetic programming and genetic algorithm formulated for QCSPs with container groups that allow for easy location of near-optimal schedules with reasonable time for computations. Yue et al. [19] proposed the two-phase scheduling optimization model to optimize the joint scheduling problem between dual-trolley quay crane and AGV.
According to the survey conducted by [20]- [22], most of the previous studies conducted to solve QCSP utilized evolutionary or genetic algorithms because these algorithms are simply designed and easily applied. Yet, these algorithms were reported to obtain solutions with limited quality. Currently, understanding QCSP has become much better, hence simplicity of the design and easy application are no longer basic considerations in selecting the QCSP solver. Under the established understanding, this work proposes a self-adaptation heuristic, which has been proven to have high efficacy when it comes to solving different combinational optimization problems to produce more efficient results [23], [24]. The basic thing this work aims to achieve is to design a self-adaptation heuristic (self-H) that adopts the best fitting local search algorithm that can be implemented in the course of search as a remedy to the QCSP problems.

PROBLEM DESCRIPTION
QCSP with container group can be described as [7]: A set of tasks has been assigned to the QCSP together with a set of uniform QCs is used to represent the loading or unloading activities of a given container group which a QC must undertake without any pre-emption. The tasks have distinct processing times pi, and they are situated at bay positions li inside a vessel of b bays. For the tasks within the same bay, the stacking-dependent accessibility can call for the incorporation of relations between precedence. In the case of a QCSP with container groups,  is used to represent precedence-constrained task pairs set, while  is used to represent the set of pair tasks that are permitted for simultaneous processing, for instance, to evade bottleneck at yard blocks used for holding containers for these pair tasks.
Each QC Q k  has a ready time r k and an initial bay location k l 0 . Between two neighbouring bays, all QCs can move in the same travel time t. It is presupposed that two QCs cannot operate simultaneously at the same bay. Moreover, they cannot cross each other and should maintain a safety margin s, estimated in units of bays as shown in Figure 1. The issue is to define the feasible time to complete the task on the cranes in terms of the constraints defined. The main goal is to reduce the vessel handling time, which refers to the completion time for the most recently completed task. This problem is referred to as the NP-hard, see [6]. For more details on QCSP, consider [9], [10].

THE PROPOSED METHOD
Due to the importance of the QCSP, on the one hand, and the inability of current approaches to operate well all over its instances, on the other hand, the need to advance a new algorithm that has the necessary ability to operate well on all available instances is still pressing matter. The meta-heuristic approaches were effectively utilized to address various optimization problems. Of them are university timetabling [25], multi-objective optimization [26], healthcare [27], data mining [28], multi-robot searching system [29], structural design [30] and others [31]- [34]. Recently, self-adaptation heuristics comprise a group of approaches that are motivated by the aim of automating the design of heuristic methods in order to solve difficult computational search problems and across different problem instances [23], [24], [35]. Accordingly, in this paper, a self-adaptation heuristic (self-H) is proposed. The self-H consists of two separate levels: a higher and a lower level. The Multi-armed bandit selection mechanism (MAB) [36]- [37] selects the most appropriate LS to be executed at the current decision point whose basis is the search status. The lower level is the number of LSs that are managed by the MAB to solves the given problem instance as shown in Figure 2.
The self-H is executed for a certain number of iterations (MaxItr), aiming to improve the quality of the QCSP solution iteratively. The self-H will hopefully work effectively during the search. This is due to the ability of the suggested self-H to apply different LSs for a different problem in an online manner).

The higher level of self-H
The high-level heuristic for the self-H is the adaptive selection mechanism (MAB) which attempts to adaptively select different LSs for different problem instances depend on the status of the search. The adaptive selection mechanism has two components which are the local search impact evaluation and the local search selection mechanism [37].

Local search algorithm impact evaluation component
In this work, each LS is connected with empirical quality estimation ( ' ,i ls q ) that indicates the average reward gained by an LS from the first iteration up to the current iteration i of the search process. The empirical quality estimation is computed using (1) [37], and it is updated throughout the search process.
L refers to the number of LSs, and C refers to the scaling factor that attains exploration-exploitation balance during the search process. In this equation, the right-hand term enhances the exploration ability by giving chances to LSs that recorded low improvements history. The left-hand term of this equation enhances the exploitation ability by favouring the LS, which has the best empirical quality. The C factor is very important as it balances between the search exploration and exploitation abilities. Thus, the MAB takes into consideration the improvements achieved by each LS during the search and how many times it is applied. The MAB algorithm process is illustrated in algorithm 1 [37].

The lower level of self-H
In this work, the lower level of self-H is a set of local search algorithms that contains four wellknown local search algorithms used to solve the QCSP. The major difference among the utilized local search algorithms is how they escape local optima. These algorithms are Hill climbing algorithm (HC), simulated annealing (SA), tabu search (TS) and iterated local search (ILS) [38]- [41]. In this study, an initial solution is generated based on the priority-based schedule construction algorithm [1]. The criterion for the termination of local search algorithms is adopted in accordance with the number of non-improved iterations.
In order to generate neighbouring solutions, two neighbourhood structures have been adopted. To provide the self-H with more opportunities to explore the search space, the neighbour solution is generated through the Move neighbourhood structure. In this operator, the last task within a particular crane route is deleted from the current route and then reinserted in the first position of the next route. In case the last route is selected, the first task will be moved to the last position of the previous route. In terms of ILS, perturbation operator is utilized to make a great change in the current solution [40]. Instead of one task, this operator moves the last segment (a group of subsequent tasks) from a route that is selected randomly to the subsequent one. In case the last route is selected, the first segment will be moved to the previous route.

EXPERIMENTAL DESIGN
In this study, Meisel and Bierwirth benchmark [7] is employed to assess the performance of the proposed method. This section examines the aspects of this benchmark and self-H parameter settings.

QCSP benchmark
The adopted Meisel and Bierwirth benchmark contains 400 instances [7] which are generated using QCSPgen generation procedure-which is available at http://prodlog.wiwi.uni-halle.de/qcspgen. The generation process is controlled by the following eight parameters [7]: c. c = Bay's capacity, referring to maximum number of containers per bay. d. f = Handling rate, referring to the percentage of container groups as handled in a service in contrast with the capacity of the vessel. e. loc = Location, in charge of the container groups' distribution over the vessel. One of three supported values are assigned to loc:  uni: the unified distribution of container groups over the vessel's bays.  cl1: practically, the common distribution containers are sometimes stored in the immediate vicinity over a vessel. In order to achieve that, a Gaussian distribution is adopted.  cl2: in this case, the container groups are clustered in two areas of the vessel. Two different Gaussian distributions are adopted for this purpose. f. d = Precedence density, in charge of generating the relations of precedence among the groups of containers. g. g = Density of non-simultaneous, in charge of generating the non-simultaneous relations among the groups of containers. h. seed= Random seed, in charge of initiating a pseudorandom number generator which leads to start at an arbitrary point in a random sequence. These instances are distributed into seven sets according to various crane scheduling issues, such as: number of tasks, number of bays, bays' capacity, handling rate, container groups' distribution, precedence and non-simultaneity relations among container groups and number of cranes. These sets are known as A, B, C, D, E, F and G. The sets A, B, and C refer to small, medium and large vessels with two, four and six cranes, respectively. The rest sets D, E, F and G refer to medium vessels with various container groups' distribution, precedence's density, number of cranes and safety requirements. The features of these sets are illustrated in Table 1.

Experimental setup
The proposed algorithms (HC, SA, TS, ILS and self-H) are all programmed in Java and executed by using PC with windows7, Intel processor Quad CPU 2.33 GHz, RAM 2.00 GB. In this section, preliminary test or as suggested by previous works, are appropriately used to determine the appropriate values for all proposed algorithm parameters. During the preliminary test, the adopted heuristics are executed 10 runs on seven instances-which are selected based on seven groups founded in the benchmark. The parameter settings used in this work are summarized in Table 2.

EXPERIMENTAL RESULTS AND COMPARISONS
To assess the performance of the suggested self-H for tackling QCSP, two experimental tests are carried out in this work. The first one compared the results of the self-H with that of four well-known LSs; HC, SA, TS and ILS in section 6.1. Fourteen instances are selected in this experimental (two instances from each group):  A1: Set A, n=10 A2: Set A, n=10  B1: Set B, n=45 B2: Set B, n=50  C1: Set C, n=75 C2: Set C, n=80  D1: Set D, f=0.2, Loc=cl1 D2: Set D, f=0.2, Loc=cl2  E1: Set E, d=0.8 E2: Set E, d=0.85  F1: Set F, q=2 F2: Set F, q=3  G1: Set G, s=0 G2: Set G, s=1 In the second experimental test, a comparison was carried between the results of self-H and those obtained in former studies, shown in section 6.2.

Comparisons of the self-H with the LSs
This study investigated self-H performance in comparison with that of four well-known LSs; HC, SA, TS and ILS. The obtained results are tabulated in Tables 3 and 4; these results involved Best and Avr. The high-quality solutions are marked in bold. In terms of the best values, the self-H obtained better results than the others. The results tabulated in Table 3 show that the self-H got the best results on thirteen instances out of fourteen, while HC, SA, TS and ILS got the best results on 3, 5, 8 and 9 instances, respectively. Based on the Avr values, Table 4 shows that the self-H outperformed the others as it obtained the best Avr results on eleven instances out of fourteen, while HC and ILS gained the best Avr results on one and two instances.  Instances  HC  SA  TS  ILS  self-H  A1  599  599  599  599  599  A2  561  561  517  561  517  B1  798  863  776  798  768  B2  959  871  786  818  818  C1  1507 1443 1320 1295  1295  C2  1381 1183 1319 1273  1183  D1  373  388  373  373  373  D2  336  339  329  336  329  E1  959  840  840  818  818  E2  959  951  887  818  818  F1  1515 1515 1515 1515  1515  F2  1240 1029 1036 1029  1029  G1  960  820  820  820  820  G2  959  840  818 818 818 For the purpose of assessing the statistical significance of the gained results, a statistical test is conducted as: At the beginning, the Shapiro-Wilk normality test with 0.05 critical level is carried to verify whether the distribution of obtained results is normal or not normal. The test revealed that the obtained results significantly deviate from a normal distribution (the p-value is less than 0.05). Consequently, two  Table 5 provides the ranking of the HC, SA, TS, ILS and self-H in lieu of the Friedman test (where the lower the value, the higher the rank). The self-H ranked first as it had the lowest value. Whilst, ILS, TS, SA and HC ranked 2 nd ,3 rd , 4 th and 5 th , respectively. It can be noticed that the p-value of the Friedman and Iman-Davenport statistical tests demonstrated the existence of a significant difference between the obtained results (p-value < 0.05, i.e. p-value=0.00). Consequently, a post-hoc statistical test with critical level 0.05 is employed to obtain the regulated p-values for each comparison between the control algorithm (self-H) and the others; HC, SA, TS and ILS. Table 6 shows the adjusted p-values of Holm and Hochberg statistical tests. Table 5 makes it clear that the self-H is far better statistically than HC, SA and TS (adjusted p-value < 0.05), yet it is not statistically significant compared to ILS.  Table 7 illustrates the results obtained by the standard self-H for all the instances in the adopted benchmark (sets A to G). In this table, each cell comprises two values, upper and lower. The upper value refers to the average vessel handling times, whereas the lower value refers to the average running time as measured in seconds. The results obtained in this work demonstrate that the suggested self-H is an efficient solution technique for the QCSP. This may result from the ability of the self-H to select different LSs for a different problem in an online manner). By utilizing different LSs during the search, the self-H can deal with various problem instances and cope with any changes that may occur during optimizing a solution. Based on these results, the hypothesis raised above is accepted and proved to be true.

Comparison of self-H with the state of the art methods
In this experiment, self-H results are compared with the best-known existing results in the available works of literature (CPLEX and UDS), which are collected from [1]. There are numerous studies that have been proposed in the literature for QCSP. Three of these recent studies achieved the best-known results, so they were selected for comparison with the proposed method. These studies are:  GA: a genetic algorithm proposed by [5]  HGA and HGP: two hybrid evolutionary computation methods proposed by [1] Table 8 reports the GAP the best results obtained by self-H and the other compared algorithms. The aforementioned tabulated results reveal that self-H attained competitive results as compared to others. This comparison demonstrates that the self-H excelled USD, HGA and HGP on D, F and G datasets. Even though self-H did not match the best-known existing results for all datasets, the obtained results for these datasets remain very competitive.

CONCLUSION
The researchers in this study proposed a Self adaptation heuristic (self-H) to solve QCSP. The self-H involves two separate levels, the upper level (LS-controller) and the lower level (QCSP-solver). This enables the self-H to deal with different problems and the changes that may happen throughout the course of optimizing a solution. The LS-controller is based on an adaptive selection mechanism that adaptively selects which LS will be adopted by the QCSP-solver to solve the given problem. The QCSP-solver embeds a group of adopted local search algorithms. The effectiveness of the self-H was verified by using the standard QCSP benchmark, suggested by Meisel and Bierwirth (2011). The experimental results showed that the f-H is an efficient solution method for the QCSP as it is able to inherit the most important features of the QCSP-solver to select the most suitable local search algorithm to be applied through the search process. In future work, the proposed algorithm (self-H) can be applied to solve more practical problems such as university timetabling and multi-objective optimization of machine learning model. This application will identify the generality characteristic of the proposed algorithm.