Motivation assessment model for intelligent tutoring system based on Mamdani inference system

Many educators have used the benefit offer by intelligent tutoring system. To become more personalizing and effective tutoring system, student characteristics need to be considered. One of important student characteristic is motivation. Therefore, in this study a motivation assessment model based on self-efficacy theory was proposed. Refer to the theory, effort, choice of activities, performance and persistence were discussed as motivation attributes. Further, time spend, difficulty level, number of correct answers and number of questions skipped are the parameters was defined for each attribute. The model was designed by taking the advantages of Mamdani inference system as fuzzy logic technique to predict students’ motivation level. The model able to inmates like a human tutor does in the traditional classroom to understand students’ motivation level.


INTRODUCTION
The definition of motivation may take several forms and differ upon its application. According to Keller and Litchfield [1], motivation can be defined as a persons' desire to pursue a goal or accomplish a task. Williams and Burden [2] define motivation as a "A state of cognitive and emotional encouragement, which brings to a firm decision to act, and which gives rise to a period of sustained knowledge and/or physical effort in order to reach a set of aim or aims". Motivation has always been important for learning process and has a great influence [3], [4]. In a real-world classroom, educators easily capture students' motivation level during learning process and adjusts lessons accordingly, in order to maximize the student's interest and participation. Educators usually understand student motivation level from observational cues such as student body language or their behavior.
In e-learning environment mainly in intelligent tutoring system (ITS) the same consideration need to be taken where the tutoring system able to recognize when the student is becoming demotivated. Vicente and Pain [5] and Thinakaran and Ali [6] have argued that motivation components are as important as cognitive components in ITS, and that important benefits would arise from considering techniques that track the students' motivation. Thus, the authors claim that ITS should include a mechanism for detecting the students' motivational level, and appropriately responding to that level. This study tries to address aforesaid issues by proposing a model for motivation assessment in ITS that takes the active and successive environment of motivation into account.

REVIEW RELATED WORK
The capability to assess the students' motivational level in ITS can bring numerous benefits. Since motivation characterizes an important factor in learning process, different researchers have recommended different motivation assessment to examine student motivation level in e-learning. From the literature, different approach was proposed in order to measure and assess students' motivation level and they can be grouped in questionnaire-based approach, interaction-based approach, sentic modulation approach (physical assessment of a persons' emotional changes via sensors) and also hybrid-based approach. The followings are some of the tutoring systems presented base on stated approaches. Vicente and Pain [5] developed motivation diagnosis study (MOODS) for learning Japanese numbers with an added motivation self-report facility. The motivation self-report facility is based on a number of motivational factors consists of trait and state variables. First, student need to answer traits questionnaire before carrying the exercises. In between answering the exercise, the student are required to feedback on their state motivation factor. The state factors can be changed as often as possible since it is necessary for the computer to understand student current motivation level in order to modify the instruction accordingly.
While, M-Ecolab was designed for teaching pupils aged between 9 to 11 years old related to food chains and food-webs. M-Ecolab is the extension of Ecolab developed by Rebolledo-Mendez et al. [7] to provide motivational scaffolding by an on-screen character called Paul at interaction time. The motivational modeling was based on three motivational traits: effort, independence and the confidence. The system provides Paul's spoken feedback and gestures at preand post-activity according to the motivation model's perception. For example, if the motivation model determines a low state of motivation due to the quality of the actions which was poor, Paul's post-activity feedback states: "For the next node try to make fewer errors". Under these situations, Paul's face would reflect concern.
Hurley and Weibelzahl [8] developed a motivational strategy recommender tool known as MotSaRT. Its functionality enables the teacher to specify the students' motivation profile. By observing the students' activities and interaction, teacher would evaluate students' motivation in terms of their self-efficacy, goalorientation, locus of control and perceived task difficulty. In the recommended strategies, depending on the profile entered, a list of strategies will appear. MotSaRT would then classify this situation and sort the strategies in terms of their applicability and plan their interventions according to the recommendations. E-learning with motivational adaptation also known as ELMA developed by Endler et al. [9] presents a fixed number of tasks and measures the student's motivational level during learning process. The system used self-assessed motivation questionnaire. The questionnaire containing 7-point Likert scales with 18 questions covering four motivation factors, anxiety, probability of success, interest, and challenge. In the questionnaire, the student will be ask to report their current motivation based on the previous block of tasks. The complete questionnaire could assess the student's motivation at the beginning and at the end of the program. Motivational questionnaire covering each of the motivational factors was presented several times during the program to make sure that the program always captured the learner's current motivation.
Derbali and Frasson [10] assessed student motivation level in ITS gameplay called Food-Force. To assess student motivation level, physiological sensors which consists heart rate, skin conductance, and electroencephalogram also known as EEG and self-reported scores of the ARCS model consist of attention, relevance, confidence, and satisfaction have been considered. To assess motivation level, galvanic skin resistance (GSR) electrodes and the blood volume pulse (BVP) sensor were attached to the fingers of participant's nondominant hands. GSR used to measure the conductance across the skin and BVP to measure heart rate. An EEG cap fitted on learners' heads to measure brainwaves. Self-reported scores of the ARCS model used to identify four factors of motivation: attention, relevance, confidence, and satisfaction.
The intervention of students' motivation assessment in ITS can bring many benefits but have some drawbacks. MOODS [5] and ELMA [9] assess students' motivation by asking how their feeling was in between their learning process. These self-motivation reports cause interruption in student concentration in the learning process. The interruption also can make student lost interest to continue the learning process. MotSaRT [8] is a motivation strategy recommender tool, where the teacher has to enter students' motivation level according student activity in the tutoring system. Then the tool will suggest appropriate strategies to motivate the student. In this intervention, the teacher still has to evaluate the students' motivation level manually by interpreting students' activates in e-learning. Derbali and Frasson [10] used physiological sensors to assess students' motivation level. Even though the intervention brings new dimension in student motivation assessment but in real world is not applicable. Imagine that, student need to attach the particular devices at their body during in their learning process and again this situation can disturb the student concentration. As conclusion, a motivation assessment in ITS should be construct in the system itself without interruption students' learning process. In the following session, a motivation assessment model was proposed to assess students' motivation level without interruption students' learning process.

METHOD
In this study a deductive approach is used to reach a logical true conclusion [11]. The approach holds a theory and based on it, make a prediction of its consequences. Figure 1 illustrated how the study carried out using the deductive approach.

Figure 1. Deduction approach
The proposed motivation assessment model was design base on a well-known self-efficacy theory by Bandura [12], a Canadian psychologist. He has claimed that self-efficacy beliefs effect on choice of activities a student takes part in; the level of student effort expended in performing a task, persistence in the face of difficulties in completing a task, and student performance in the task. Through research on self-efficacy as learning motivations factor, many scholars have demonstrated their relationship. For example, Emre and Ayverdi [13]; Durak et al. [14]; Gorson and O'Rourke [15], had state that individuals with a high perception of self-efficacy on a particular situation strive to accomplish a task. They do not easily give up and are persistent and patient. While Hattie [16], from 800 meta-analyses, the researcher has identified self-efficacy as the strongest predictor of educational achievement.
Base on self-efficacy theory as motivation factor, choice of activities, effort, performance and persistence were identified as motivation attributes. These motivation attributes were used in this study to determine students' motivation level. Choice of activities is defined as the level of challenging task the student chooses [17]. Difficulty level of tasks such as low, medium, high, has been considered as a parameter to measure choice of activities [18]. Effort define as the amount that the student is employing their self in order to perform the learning activities [19]. To measure effort, the amount of time spent to perform a task [20] has been considered as a parameter. Performance explains the student's achievement on a specific topic [21]. To measure performance, the number of correct answers has been considered as parameter [17], [21]. Persistence, describe as a constant in performing an activity [21]. The number of questions skipped was used as a parameter to measure persistence [17], [20].
Fuzzy logic (FL) as artificial intelligent technique applied to predict the students' motivation level. This technique was introduced by Zadeh [22] and used when conventional logic fails. It is a computational paradigm which is based on human thinking. The aim of using FL technique in this study is to capture the vagueness of effort, performance, choice of activities and persistence, then determine students' self-efficacy which are used together to draw the conclusion of students' motivation level. The main advantage of FL is that it uses reasoning that closely resembles human. Furthermore, motivation is characterized by ambiguity thus difficult to quantify. Consequently, Wang and Hsieh [23] suggested the use of FL technique to help in solving this problem.
In general FL technique consist of [24]: i) fuzzification which translates crisp (real-valued) inputs into fuzzy values; ii) rule evaluation is an engine that applies a fuzzy reasoning mechanism to obtain a fuzzy output; and iii) defuzzification which translates this latter output into a crisp value. There are 3 different inference system which are widely used in FL which are Mamdani inference system [25], Sugeno inference system [26] and Tsukamoto inference system [27]. The most widely used system is Mamdani inference system [28]. This inference system also known as Max Min inference system which was introduced by Professor Ebrahim Mamdani from London University [25]. The advantages are, it is intuitive; it has widespread acceptance, and it is well suited to human input. Hence, in this study Mamdani's Fuzzy inferences system as students' motivation prediction technique was applied.

PROPOSED MODEL
To assess students' motivation level, the authors has applied Mamdani's fuzzy inferences system. The main advantage of Mamdani's fuzzy inferences system is that it uses reasoning that closely resembles the presence of human. The aim of using Mamdani's fuzzy inferences system in this study is to capture the vagueness of effort, performance, choice of activities and persistence which will therefore determines students' self-efficacy to draw a conclusion on students' motivation level. The following are steps describes how the motivation assessment model was developed based on Mamdani fuzzy Inference System.

Determining the linguistic variables and fuzzy sets
Choice of activities (CA) parameter depends on the difficulty of each particular question. This parameter is calculated as a weightage average difficulty of all solved questions by the student as in (1). The weightage value for easy question is 1, medium question is 2 and hard question is 3. The weightage average equation is given (1) where ans will be assigned as 1 if the question is answered correctly or else it will be assigned as 0. The value of weightage average (wa) becomes a crisp value for CA.
Effort (EF) parameters depends on the time (t) taken by a student to answer a set of tutorial questions. The maximum time depends on the time that the teacher has defined for solving a set of questions. For this study an average of 1.2 minutes is given to answer each question. As in (2) is used to calculate time taken by the student for answering the given questions. The time taken becomes a crisp value for EF.

= ( 1 + time2 + ⋯ + )
Performance (PF) parameter depends on the number of correct answers answered by the student on the particular set of tutorial questions. As in (3) is used to calculate total number of correct answers (cAns) answered by the student over the total number of generated questions (numOfQuest) by the system times by 100%. The percentage of correct answers (%cAns) will be the crispy value for PF.
Persistence (PS) parameter depends on the number of skipped questions on a given tutorial. As in (4) is used to calculated as the total number of skipped questions (sQuest) by the student over number of generated questions (numOfQuest) by the system times by 100%. The percentage of skipped questions (%sQuest) will be the crispy value for PS.

Fuzzification
Fuzzification, translates crisp (real-valued) inputs into fuzzy values using a membership function [23]. In this study, triangular and trapezoidal with R-and L-functions were used to translate each linguistic variable value as crisp value into fuzzy values. The membership functions have proven popular with fuzzy logic and have been in use extensively due to their simple formula and computational efficiency [24]. The following are fuzzification for each input linguistic variable.
CA has 3 fuzzy sets shows in Figure 2 with possible values of easy, medium and hard which are denoted as CA(x)={easy, medium, hard}. These distributions are formulated as in (5). EF has 3 fuzzy sets shows in Figure 3 with possible values of short, medium and long which are denoted as EF(x)={short, medium, long}. These distributions are formulated as in (6). PF has 3 fuzzy sets shows in Figure 4 with possible values of poor, good and excellent which are denoted as PF(x)={poor, good, excellent}. These distributions are formulated as in (7).    The output variable which is called as motivation level (ML) of a student is also determined by the fuzzy logic. The motivation level of a student has three fuzzy sets shows in Figure 6 which are low, medium and high and are denoted as ML(x) = {Low, medium, high}. These distributions are formulated as in (9).

Fuzzy inferencing or evaluate rules
The logic for assessing students' motivation level is encoded as a set of if-then rules. The antecedents of the production rules consist of CA, EF, PF, PS and one set of values representing the conclusion and, the rules consequent (motivation level-ML). A rule is defined as every possible combination of antecedents that may occur. In this study, 81 rules were obtained as the combination of each value (difficulty level, time, number of correct answer and number of skipped questions) from CA, EF, PF and PS. However, only 26 rules have been logically accepted. The following shows one of linguistic rule used whereby the inputs (antecedents) are combined logically using the AND operator in order to get students' motivation level as output (consequent). The output of students' motivation level is denoted as ML(x)={low, medium, high}.

Rule
Linguistic rules 1 IF CA is easy AND EF is short AND PF is poor AND PS is low THEN ML is low.

Rules output
The min method is applied as an implication function. It combines each degree of memberships to each if-then rule then truncates the output. For example, a student manages to answer 4 easy questions correctly out of 12 questions within 15 minutes and skips all the medium and hard questions. The following is Rule 1 using min method while Figure 7 illustrates in a graphical view. This method is repeated so that the output membership functions are determined for all 26 rules as shown in Figure 8 in a graphical view.

Rule 1
= IF CA is easy AND EF is short AND PF is poor AND PS is low THEN ML is low. On the other hand, the max method is applied as an aggregation function. The input for the aggregation process is the list of truncated output returned by the implication process for each rule. Figure 9 shows all 26 rules which are displayed to show how the rule outputs are aggregated into a single fuzzy set whose membership function is assigned for every output (motivation) value and are represented in a graphical view.

Defuzzification
Defuzzification functions to convert the fuzzy values into crisp values. The input for the defuzzification process is the aggregate output. In this study, a Centroid method was applied which is one of the most common methods used. The Centroid method which returns the center of area under the curve is shown in Figure 10 in a graphical view. From the example given, the defuzzified value is between 0 and 1. Therefore, it can be concluded that the students' motivation level is recorded to be at 0.452 which is considered to be at a low level.  Figure 10. Defuzzify the motivation level using centroid method Figure 11 display the steps how the motivation assessment model was developed based on Mamdani Fuzzy Inference System. The steps started with deciding linguistic variables and fuzzy sets; translates crisp inputs into fuzzy values using a membership function; Fuzzy inferencing; and defuzzification. Following with motivation assessment algorithm shows in Figure 11 derived from motivation assessment model shows in Figure 12. While Figure 11 is motivation assessment algorithm derived from motivation assessment model which was illustrated in Figure 11. Figure 12 as shown in Appendix. Figure 11. Motivation assessment model based mamdani fuzzy inference system

CONCLUSION AND FUTURE WORK
Predicting student motivation level in holds great promise for ITSs. The proposed model can be used to detect student motivation level during their learning process. This model describes all the steps of inference starting from fuzzification, rule evaluation and defuzzifiction. Future work will involve implementation of the proposed model into ITS. The model will be incorporated with ITS architecture specifically in student or user model. Besides detection of student motivation level, the tutoring system aims some recommendations in automatic manner based on student motivation level, much like in the traditional classroom.