IAES International Journal of Artificial Intelligence (IJ-AI)

Mohammed Maree, Mujahed Eleyat, Shatha Rabayah, Mohammed Belkhatir Department of Information Technology, Faculty of Engineering and Information Technology, Arab American University, Jenin, Palestine Department of Computer Systems Engineering, Faculty of Engineering and Information Technology, Arab American University, Jenin, Palestine Department of Computer Science, Faculty of Engineering and Information Technology, Arab American University, Jenin, Palestine Department of Computer Science, Campus de la Doua, University of Lyon, Lyon, France


INTRODUCTION
Constraint programming is closely related to constraint satisfaction theory, which offers a simple formal scheme for representing and solving combinatorial problems of artificial intelligence [1]. Among the tasks solved by constraint programming: checking electronic circuits, calendar planning, schedule planning, as well as many combinatorial tasks [2]- [4]. Constraint programming is a programming paradigm in which relationships between variables are specified in the form of constraints. Formally, the maximum constraint satisfaction problem (Max-CSP) is defined by a set of variables which are linked by a set of constraints following a domain of definition for each variable. Max-CSP solution is an instantiation to satisfy the maximum of constraints [5].
The concept introduction of Max-CSP leads to extensive research for choosing an appropriate resolution method. In addition, exact methods require very high computational time due to the size and complexity of the problem. Whereas approximate methods are necessary for the mission to find an instantiation for the maximum constraint satisfaction problem.
The simulated annealing algorithm, in recent years, has been used to solve real problems especially optimization problems [6]. The peculiarity of the simulated annealing algorithm lies in its flexibility to adapt with any optimization problem [7]. This makes the simulated annealing algorithm more efficient, faster and easier to program to solve many optimization problems [8].
In view of the attention given to the simulated annealing approach to solve many optimization problems, this work adopted this approach as a method of solving Max-CSP. The simulated annealing method is able to avoid local minima to find the optimal solution. The optimal choice of a set of parameters like the cooling model, the initial temperature and the final temperature is essential to ensure good convergence. The limitation of the simulated annealing method lies in the inability of the method to take into account the behavior of the problem during convergence. In contrast, Hopfields neural network has proven its ability in the field of machine learning. In this article, we propose a hybrid approach for solving maximum constraint satisfaction problems. The idea is to improve the simulated annealing algorithm in order to build a powerful system that can be adapted with any type of quadratic problem. To achieve this goal, we adopt the Hopfield network which is capable of taking the system state upon convergence to the simulated annealing algorithm to avoid the local solution. In order to perfect the proposed approach, three cooling models are used for the simulated annealing algorithm.
This work is structured in five sections. The section 2 presents a quadratic model for the maximum constraint satisfaction problem. The section 3 describes the hybrid approach which combines the neural network and simulated annealing. The section 4 allows to implement the proposed approach to solve the Max-CSP. The section 5 gives a conclusion and proposes an alternative avenue of research on another field of application.

MODELLING OF MAX-CSP
The constraint satisfaction problem can be defined as a network of variables that are related to each other. In this network, the assignment of a value to a variable with the satisfaction of all the constraints between each pair of variables is necessary to find a solution for the constraint satisfaction problem [9]. In some cases, satisfaction of all constraints is impossible given the complexity and size of the problem [10]. To deal with this problem, reducing the number of violated constraints is necessary as a partial solution [11]. This paradigm is known in the literature by the maximum constraint satisfaction problem. The Max-CSP consists in assigning a value to a variable for the entire network with the maximum constraint satisfaction [12]. More formally, the maximum constraint satisfaction problem is a form of model that is represented by a set of variables and a set of constraints. The aim of this work is to study the binary constraints of Max-CSP. The maximum constraint satisfaction problem is defined by a tuple = < , , , > such that: The basic idea for solving the maximum constraint satisfaction problem is based on assigning a value to a variable with minimization of number of violated constraints. In this context, a quadratic model under linear constraints is proposed. The modeling phase requires the declaration of the following mathematical notations: is a decision variable, is the size of decision variable , is the value assigned to the decision variable , and is the sum of the size of all variables.
The decision variable is defined by (1): A unique value is selected for each decision variable. This expression is defined by (2): A relation between the variable and the variable makes it possible to define a binary constraint . In this modelization, a matrix of dimension is built starting from the checking of each constraint between the two variables and . An element of matrix of row (i.e. variable ) and column (i.e. variable ) is defined: The constraint is expressed: The objective function is defined: The Max-CSP problem is modeled as a new quadratic programming, which constitutes an objective function subjected to a linear constraint.
The matrix is a symmetric matrix of dimension × that represents the relationship between the decision variables. The matrix is of dimension × , which represents the linear constraint. The vector is of dimension . To solve this proposed model of Max-CSP, a new approach is proposed to solve it.

THE PROPOSED MODEL SOLVED BY NEW APPROACH
This section gives a representation of a hybrid approach that combines the simulated annealing method and the Hopfield neural network to solve the maximum constraint satisfaction problems. In the subsection of the simulated annealing approach, different cooling models are represented. Then Hopfield's neural network is represented as an adaptive approach to solving any quadratic problem. The last subsection presents a new hybrid approach that combines simulated annealing and the Hopfield neural network. First, a detailed description of the simulated annealing approach is shown in the next subsection.

Simulated annealing
The simulated annealing method mimics the physical phenomenon of crystallization [13]- [16]. Crystallization is an operation that allows a substance to transition from a liquid phase to a solid phase. This process was used in the simulated annealing method to solve an optimization problem. The operation of the simulated annealing method is related to a set comprises the initial temperature, final temperature and cooling model. Controlled cooling participates to ensure good convergence. The following notations are used for the simulated annealing method: is a possible solution, ( ) is an energy function, ihe maximum temperature, ihe minimum temperature, is a random transformation function, function generating a new state, and is the transition probability defined by the following expression: The application of the simulated annealing method requires the use of a good cooling model to reduce the temperature of the energy function [17]- [19]. Controlled cooling makes it possible to switch from a high energy level to a low energy level. In the simulated annealing algorithm, choosing a good cooling model is important for better convergence. The following subsection describes the cooling models used in this work.

Geometric model
The geometric cooling model is inspired by an arithmetic-geometric sequence in which each term makes it possible to deduce the next by multiplication by a constant factor [20]. This model is defined by the relation = −1 + . The factor is selected from the interval [0.1]. The relation is an arithmetic sequence when = 1 and is a geometric sequence when = 0. Therefore, the parameter must be different from 0 and 1.

Logarithmic model
The logarithmic model was first proposed by Geman and Geman by the following formula : +1 = × ( ( + 1) ) −1 [21]. The logarithmic model proposes a relationship between the initial temperature and the final temperature. The temperature decreases in two phases: the first phase marks a rapid change in temperature in only a few first iterations. The second phase is characterized by a very slow change in temperature. Therefore, the convergence of this model is very slow and this requires considerable computation time.

Logarithmic model
The Lundy-Mees model is temperature cooling technique described by the following formula +1 = × (1 + ) −1 [22]. The parameter is defined by the following relation: = ( 0 − ) × ( . 0 . ) −1 . The parameter 0 represents the initial temperature, the parameter represents the final temperature, and the parameter is the number of iterations.

Continuous hopfield network
Physicist John Hopfield proposed the Hopfield model in 1982, it was a major breakthrough in the field of neural networks [23]. The Hopfield model not only allows to function as associative memory to help object recognition in image processing domain but also it is able to solve a lot of optimization problem such as the problem of installing a surveillance camera, the traveling salesman and the problems of maximum satisfaction of constraints [24], [25]. Due to the great use of this model, it has become the center of attraction for many researchers. Hopfields neural network is a fully connected network [26]. More formally, it is represented by a symmetrical matrix to guarantee the stability of this network. The Hopfield neural network is composed of n interconnected neurons [27]. The dynamics of the Hopfield neural network is described by the following differential (8): The vector = ( ) is the input vector of neurons and = ( ) is the output vector of neurons with 1 ≤ ≤ and ∈ {0,1}. The weight matrix is given by = ( , ) and is the neuron bias. The hyperbolic function is used to calculate the output of each neuron. Neuron output is expressed: where 0 is a parameter used to control the gain of the activation function. Hopfield proved that the symmetry of the zero-diagonal matrix is a sufficient condition for the existence of the Lyapunov function [28]. Therefore, the existence of the equilibrium point is guaranteed [29]. Continuous Hopfield networks are capable of solving combinatorial problems that have an energy function taking the following form:

Proposed hybrid approach
A hybrid algorithm consists of combining two or more different algorithms in order to arrive at an optimal solution. One of the objectives achieved in this work is to propose a quadratic model for the problem of maximum constraint satisfaction and to solve this model via a robust hybrid algorithm. This hybrid algorithm is a combination of two different approaches: the Hopfield neural network and the simulated annealing algorithm. Hopfield's neural network methodology has been widely used in optimization problems since their arrival. In this work, the Hopfield network was adopted to improve the convergence of the simulated annealing algorithm. This section presents a hybrid algorithm that can solve different problems of maximum constraint satisfaction.

RESULTS AND DISCUSSION
The proposed approach that combines simulated annealing and the Hopfield neural network is used as a solver for the maximum constraint satisfaction problem. To assess the effectiveness of the proposed approach, a series of instances that represent real problems is used in this work. In this section, the basic simulated annealing algorithm is used to solve the maximum constraint satisfaction problem. In addition, the proposed approach is also used to carry out the research process to ensure good convergence.
This section presents the different instances (scens, CNF) used to evaluate the performance of the proposed approach. Software and hardware prerequisites are required to implement the proposed approach. Instances are run on a 3.0 GHs processor desktop and 4 GB RAM. The proposed algorithm is programmed through the use of Java object-oriented programming language. Given the stochastic nature of the proposed approach (the complexity of the algorithm and the structure of the test instance), the experiment was carried out 30 times. When implementing the proposed approach, a number of parameters can help with good convergence. These parameters are determined through preliminary experiment. The preliminary experiment made it possible to set the value of and at 0.99 and 3.5 respectively.

Experiments with scens instance
The scens instance represents that are used to compare the proposed approach with other methods. Figure 1(a) shows the average execution time for instance scenes with a fixed number of variables that is equal to 100 and a number of constraints varies between 1,178 and 1,222. Figure 1(b) shows the same instance scens but relatively large with a number of variables ranging from 82 to 458 and with a number of constraints varying from 382 to 5,286. The hybrid approach that combines the Hopfield neural network and simulated annealing with the Lundy cooling model (HA+ Lundy) has proven to be the most robust solver in terms of quality and runtime.

Experiments with CNF instance
In this experiment, the conjunctive normal form (CNF) instance is used to evaluate the performance of the proposed approach. The first step in this experiment is to extract the data from extensible markup language (XML) file. The second step is to represent the relationships that lie between the variables in a decision function that evaluate the maximum constraint satisfaction problem. Figure 2(a) shows the average execution time of the CNF instance with a number of variables fixed at 40. Figure 2

CONCLUSION
Hopfield's neural network was used in this work to improve the simulated annealing algorithm. This makes it possible to build a new hybrid approach. Hopfield's neural network is a robust algorithm that takes into account the previous information to improve the direction of the algorithm towards a better solution. The simulated annealing is fed by Hopfield's neural network during the research process. The proposed approach gives better results comparing with other conventional methods. The proposed approach has made it possible to solve the scens instance of variable number between 82 and 458 in a better execution time. And also allows to solve the CNF instance of variable number is set to 40 and to 80 in a better execution time better compared to other approaches. Future research should attempt to model and solve the quadratic model associated with the query optimization problem in databases.

BIOGRAPHIES OF AUTHORS
Mohammed El Alaoui is a Doctorate Status in applied computer sciences and mathematics from the Laboratory of Modeling and Scientific computing at the Faculty of Sciences and Technology of Fez, Morocco. He is a member of Artificial intelligence for engineering sciences group in the Laboratory of mathematical modeling, Operational Research and computer sciences. He works on Neural Network, Artificial Intelligence, classification problems, Database, constraint satisfaction problem and machine learning and Data warehouse. He can be contacted at email: md.elalaoui@gmail.com.

Mohamed Ettaouil
is a Doctorate Status in Operational Research and Optimization, FST University Sidi Mohamed Ben Abdellah USMBA, Fez. Ph.D. in Computer Science, University of Paris 13, Galilee Institute, Paris France. He is a professor at the Faculty of Science and technology of Fez FST, and he was responsible for research team in modelization and pattern recognition, operational research and global optimization methods. He was the Director of Unit Formation and Research UFR: Scientific computing and computer science, Engineering Sciences. He is also a responsible for research team in Artificial Neural Networks and Learning, modelization and engineering sciences, FST Fez. He is an expert in the fields of the modelization and optimization, engineering sciences. He can be contacted at email: mohamedettaouil@yahoo.fr.