Predicting baccalaureate student result to prevent failure: a hybrid model approach

ABSTRACT


INTRODUCTION
The Moroccan Ministry of National Education has implemented significant reforms over the past twenty years, including implementing the educational information system (MASSAR) for the year 2012, organizing and controlling the absences of teachers and students, the emergency program that extended from 2009 to 2013, and framework law 51.17, which established the strategic vision for the years 2015-2030.According to the Program for International Student Assessment (PISA) 2018 [1], Morocco came in 75th out of 79 countries in the ranking factors, which evaluated the ability to read, mathematics, and science skills of 15-year-old pupils around the world [2].
Ahajjam et al. [3] predict the performance of Moroccan students at the baccalaureate level using an intelligent system that relies on machine learning techniques such as neural networks to extract data to determine the student's basic average.It also aims to design and implement a system to direct students from common streams to one of the technical, literary, or scientific baccalaureate divisions.In their study, the researchers created a methodology to predict student performance using a machine learning algorithm.Student data was exploited from the MASSAR system for a secondary school in Morocco between 2016 and 2018.The researchers concluded that the chosen model could provide good accuracy for students' performance.Qazdar et al. [4] built a model using multivariate regression to predict the performance of students in a Moroccan secondary school over three years of study.Their goal was to combine their model with the "MASAR" system to predict students' early results for the purpose of increasing their performance.In contrast to the percentage obtained, which amounted to 37% according to the results of the predictive model, 42% of the students were at risk of failing.The system indicated that 29% of students completed the academic year successfully, although the actual number was closer to 21%.In order to support students in achieving better Int J Artif Intell ISSN: 2252-8938  Predicting baccalaureate student result to prevent failure: a hybrid model approach (Abdessalam Essayad) 765 academic achievements and raise the level of system outcomes generally, Alamri et al. [5] have attempted to anticipate students' academic success, identify obstacles and problems that affect student outcomes, and give ways to improve the educational system.The research we are presenting is focused in the same direction as previous research, except that it compares the efficiency of the selected models between linear regression and selection, as well as investigates the reasons that prevent achieving good results for success in the baccalaureate.We seek to forecast the results of the current academic year using the baccalaureate results from the previous two academic years.The goal is to predict what will happen at the end of the current academic year immediately after the results of the first semester have been released, exactly six months before the final exam results.The respective departments will receive the anticipated results in order to identify students who require immediate school support.Additionally, pre-intervention can be used to help students who are anticipated to fail by identifying them and letting the administration and teachers know who they are.The remainder of this paper is structured as follows: in the second section, we discuss the justification for the study.The third topic we'll cover is comparable work.In the fourth section, we will focus on data processing and modeling by creating a solid model and attempting to use it on a newly created dataset.The resulting outcome is covered and discussed in Section 5. We will conclude the article with some concepts and recommendations that may be useful in order to improve the success rate of baccalaureate students.

SEARCH GROUNDS
Obtaining a baccalaureate degree is considered the end of secondary education and the gateway to higher education, with credibility at the national and international levels.According to the Moroccan Finance Law of 2021 [6], the budget allocated to the public education sector has increased; it is almost 22% of its budget and seven percent of gross domestic product (GDP).Additionally, Moroccan families have doubled their expenses for their children's education [7].These numbers indicate that students' failure when the academic year has ended is costly for families and the state in terms of financial and human resources.
In Table 1, we note that the baccalaureate pass rate has clearly increased, even though the last few years have been punctuated by the COVID-19 pandemic.The concerned ministry has intensified its efforts to maintain the rhythm of the baccalaureate level.The ministry has allocated lessons on platforms including a public television channel as well as a free-access website.The strengthened regional structure expanded since 2016, which has given regional education and training academies more authority, is one reason for this rise in the success rate.The Supreme Council for Education and Training has created a strategic vision for the 2015-2030 reform of a school of quality and equity to address weaknesses in the Moroccan educational system.Additionally, of the 18 projects of the Framework Laws (51.17), nine were adopted with the aim of improving educational work.The establishment of an information system called "MASSAR" also contributed to the management and investment of school results [2].
Early detection of student failure enables management to offer early coaching and counseling to students in order to improve success rates and student retention.Academic administrators will find it easy to arrange some important interventions to enhance the performance of students who are expected to fail in the middle of the academic year, thus helping to open up prospects for successful completion of their studies [9].This research aims to predict the final results of the baccalaureate in order to enable early decision-making by the administration in cooperation with teachers and to provide school support, especially for students who are close to success, and increase the success rate.

RELATED WORKS
Many studies are concerned with students' performance.In the paper "Prediction of Student Success Using Enrollment Data" explained by Cengiz and Uka [10].According to the researchers, early detection of students who are at a high risk of failing will allow for immediate intervention by educators with the appropriate measures, which will raise the graduation rate.Strecht et al. [11] use socioeconomic characteristics such as age, sex, marital status, nationality, displacement, scholarship, special needs, and kind of admission to forecast students' performance and grade in a class.Mohamed and Husein [12] employed machine learning algorithms for learning management, academics, socioeconomics, and student demographics.Kraft et al. [13] discovered that the school climate can affect students' performances.Mesarić et al. [14] the objectives of this study are to develop a model that efficiently divides students into two groups based on how well they performed at the end of the first school year and to identify the important variables that have a major impact on performance.Xu et al. [15] research was done to determine how to forecast students' future success based on their current academic records.Kavitha et al. [16] attempted to use the analysis's findings to pinpoint at-risk students and offer suggestions for improvement.Berens et al. [17], in their research, they discover student traits that set probable dropouts apart from graduates using regression analyses, neural networks, decision tree methods, and the AdaBoost algorithm.Griffioen et al. [18] examine student satisfaction and performance.Kumari et al. [19] investigates student behavioral features that play an important role in showing the student's interactivity with the e-learning system.Aysha and Khan [20], create a model that admission-seeking students can employ to evaluate their performance in the chosen course, claiming that educational systems require creative approaches to raise educational quality to get the best outcomes and lower the failure rate.Sedova et al. [21], according to their paper, there is a substantial correlation between a student's achievement and the amount of conversation time they had in class, the number of times they used reasoning, and other factors.Suhaimi et al. [22], in their paper, compared interests among those who were interested in studying the performance of students by gender; they discovered that the methodology used in studies involving men and women differs.Imran et al. [23] examines the identification of students at risk based on socioeconomic and cultural factors.Lorenz and Wigger [24] utilizes two methods, including decision trees and logistic regressions, to predict dropping out of school using exam data.

METHOD 4.1. Data collection
Data processing is essential for academics to carry out initial survey screening, followed by editing and coding of the data [25].Data processing allows for the measurement of the data's accuracy, completeness, consistency, timeliness, credibility, and interpretability [26].The framework in Figure 1 is adapted from the cross-industry standard practice for data mining (CRISP-DM) approach.CRISP-DM is a project funded by the European Commission that creates a common paradigm for conducting data mining projects [27].

Figure 1. Framework of data processing and modelization
The data was approved and obtained from El Hajeb's Ministry of National Education Directorate in accordance with the Right to Information Law (No. 31-13) through the Transparency Portal and Access to Information (Chafafiya) [28].The dataset includes two school years' worth of baccalaureate results.It was processed and purified, and only the correct data was kept.This data is described in Table 2 and contains 1,585 rows, 8 features, and 1 target.

Data cleaning
Some of the goals of data cleaning include detecting anomalies and resolving the problem of missing and duplicate values.To ensure that the dataset is reliable and produces accurate predictions.The following tasks were performed prior to using the models: i) Removing unknown or null values from the data; ii) Converting the data in each column from categorical to numerical values; and iii) Selecting only the columns that have a high degree of influence or a strong correlation with the input value (feature selection).

Data analysis
Data analysis aims to gain an in-depth understanding of the data before it can be utilized.It also saves a significant amount of time, especially in determining whether the data is relevant to the problem we are trying to solve.Data analysis is just one of the many steps that must be taken when conducting research; nevertheless, it takes on special relevance when it is used in many different fields to guide businesses and organizations to make wiser financial decisions [29].Researchers and companies are urged to adopt algorithms that process real-time data, analyze it, and deliver highly accurate analytics conclusions due to the complexity of data analysis and interpretation [30].Machine learning, data science, and predictive modeling have become widely distributed in any sector where analyzing data is crucial [31].During the data analysis process, we examine the correlations between different variables to determine which of these correlations is the strongest and has the greatest impact.According to Table 3, there is a strong relationship between the general average and the continuous monitoring rate, which is 81%.However, the relationship between the regional average and the general average is 71%.
Figure 2 shows that the relationship between continuous monitoring and the general average is good; it is stacked between 12 and 15, and most of the points found in the linear regression.Nevertheless, the relationship between the regional average and the general average is not perfect, and we find it stacked between six and 12.In Figure 2, there is a good regression between the cc_average and ga_average, although there are many points that deviate from the regression line between the reg_average and ga_average.4, the mean of the continuous monitoring rates is 14.03, whereas the mean of the regional average is 9.73, which has a favorable impact on the baccalaureate success rate.It is noted that the regional average has a negative impact on the success rate.Therefore, educational officials should reconsider passing the regional exam one year before passing the final baccalaureate exam.The second problem is passing this exam in specializations that will not be part of their future orientations.There is no doubt that this will lead to a decrease in their success rate at the end of the second year of the baccalaureate.In Figure 3, the rate of continuous observation has a positive effect on the pass rate, while the rate of regional examination has a negative effect for the aforementioned reasons.However, the coefficient of continuous monitoring and the regional examination have the same value.This shows that the negative impact should be the subject of discussion with officials to avoid a negative impact on the baccalaureate results.
To succeed at the baccalaureate level, an average of 10 or more is required.Figure 4 shows that about 15% are close to the success rate.This statistic informs the concerned administration about targeting this sample of students with academic support to raise the success rate by 15% to more than 90%.

Modelization
Machine learning is a group of techniques that use data, such as classification or regression, to build models for performance prediction and decision-making.We will use a variety of machine-learning algorithms in our search for the most accurate indicators of the results once the academic year is over.Figure 5 shows the life cycle of managing machine learning models.Recognizing students who are at risk of failure to take preventative measures is one of our objectives, such as providing educational support to help them succeed when the school year comes to a close.Therefore, we need a dataset for the first semester of each school year and work on preparing it to apply the choice model, which, at the end of the school year, displays the students' results.This problem falls under supervised learning and can be represented as a regression or classification model.To achieve this, there are many machine-learning algorithms, each with many hyperparameters to define.

RESULT 5.1. Classification using various evaluation matrix
The categorization problem is a common topic of study for experts in data mining (DM) and machine learning (ML) [32].Based on the values of other attributes, the category attribute's value will be predicted [32].It is important to construct a set of models that explain and classify data categories and concepts in order to use the model for predicting the value of an anonymous class [26].
As shown in Table 5, we compared 11 models using the Pycaret [33] library to determine their performance differences.By analyzing each model's performance, we can select the faster model based on the test results and use it to achieve the best predictive results.The gradient boosting method uses binary decision trees as its main predictors, and after selecting the hyperparameters, the CatBoost Classifier was shown to be the most effective model [34].

Confusion metrics
The efficacy of predictions in a classification model can be evaluated using a confusion matrix [35].They are frequently employed to evaluate the accuracy of several machine learning models.After using the test data with the CatBoost Classifier model, we obtained the following confusion matrix in Figure 6.
The confusion matrix table serves as an example of a tabular display that evaluates the model's accuracy in predicting.The matrix compares the predicted values by machine learning models to the real values.The matrix shows how each model categorizes errors and whether it is better at predicting the proportion of students who succeed or fail.

Evaluation metrics using CatBoost classifier model
The three different forms of evaluation metrics are threshold measurement, likelihood, and ranking metrics [36].As Table 6 shows, the accuracy of the CatBoost Classifier model is high, at 85%, and it predicts the percentage of students who pass well (Precision=89%).The model also has a high rate of sensitivity (Recall), which is about 93%, which means that our model can identify most of the number of students with success.Also, the model predicts the success of students well but predicts less well the percentage of students who fail (specificity=73%).We can increase the percentage of specificity by reducing the outliers, or we can choose another model, like Xgboost, that predicts well the failure of students; in this case, the administration can focus only on the students who the model predicts to fail.
Precision and recall are often used in combination to evaluate the performance of a classifier, and a balance between precision and recall is desired.High precision means that the model accuratly predicts the student's success.However, a low recall means that the model might miss many student successes.Means how often is the model correct.

𝑇𝑃 + 𝑇𝑁 𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁 85% Precision
Determines how well a model predicted positive results, such as student achievement in terms of accuracy.

𝑇𝑃 𝑇𝑃 + 𝐹𝑃 89%
Specificity Means the percent of students whose failure was accurately predicted by the model.

𝑇𝑁 𝑇𝑁 + 𝐹𝑃 73%
Sensitivity Measure the true positive rate or recall (TPR), often known as the precision of a model to recognize student success.

Learning curve
The model has an accuracy of 85%, a recall of more than 90%, and a high precision of 89%.This indicates that the model can accurately predict which students will be accepted at the end of the academic year.A ROC curve, which plots the true positive rates (TPR) and false positive rates (FPR) versus both of them, is a visual representation of the efficacy of a binary classification system.
Figure 7 displays for us the ROC, which is a metric that measures the ability of a binary classifier to distinguish well between successful and failing students.The area under the receiver (AUC) metric is yet another popular way to measure how well a binary classifier performs.The model's accuracy in identifying student outcomes increases with the AUC.We have an AUC of 0.92, which is around the ideal level of 1 for a classifier.

Regression using many functions of evaluation matrix
Regression was primarily used to determine which independent factors were significant predictors of the outcome and whether the independent variables were correct in predicting the result variable [37].The mathematical formula y = mx + c is used in linear regression analysis to determine the line of greatest fitting for the correlation between the dependent variable (y) and the independent variable (x) [38].Gradient Boosting algorithms can optimize different functions, which makes them flexible, and they often provide predictive accuracy [39].After training and testing the data with the Gradient Boosting model using many cost functions, as shown in Table 7, we got MSE 0.7686.Gradient boosting, a powerful machine learning model, successfully handles categorical data and takes advantage of handling it during training rather than during pre-processing time to achieve the greatest results on a variety of practical issues [40].

Evaluation metrics for gradient boosting regressor model
Utilizing evaluation measures, we may gauge the model's efficacy.The evaluation scale that will be utilized will be determined by the unique tasks and objectives of the model.Error measurements can be used to examine the remaining differences between actual and predicted values.The minimal mean residual suggests that the model is successful in predicting our situation.
Outliers are observations in a dataset that are significantly different from most of the data.They can have a significant impact on the results predicted by the model, as they can greatly affect the relationships between variables and the overall fit of the model.Excluding outliers can be a useful approach if they are truly anomalous and do not represent the underlying pattern in the data.As shown in Table 8, the outliers can affect the efficiency of the model.The explanation for this is that the students in these cases succeeded by cheating on the exam, failed due to their absence, or obtained a zero in one of the subjects.The MSE (0.7686 vs. 1.9561) and RMSE (0.8744 vs. 1.3882) are sensitive to outliers, as they involve squaring the variation from the expected and actual values.This means that outliers in the data can significantly affect the value of these metrics.MAE (0.6769 vs. 1.0458), which is the absolute value of the differences between expected and actual, could have a significant impact on outliers if not reduced.
An indicator of how well a model fits the training data is its R-squared value.It represents the percentage of ga_average that the regression model's cc_average and reg_average are able to predict.R 2 calculates the proportion of the change in the target variable that can be assigned to the model's predictor variables.A high R 2 (0.7898) value indicates that the model fits the training data well and explains a large proportion of the variation in the target variable, while a low R 2 (0.5460) value indicates that the model does not fit the data well and explains that the outliers can have a negative effect on the R².The prediction error R², also known as the cross-validated R², is a metric used to assess how well a model predicts fresh data that isn't

DISCUSSION
In this paper, we employed several classification and regression models; however, we still need to compare them to determine which model is most suitable for making accurate predictions.For classification, we didn't take into account how close students were to the success criteria; instead, we solely considered forecasting whether a student's admission would be successful or unsuccessful.However, using linear regression, we were able to quantify each level of participation in the classroom.Students' learning was directly impacted by classroom management and interactions between students and teachers on both an academic and social level [41].
The regression model is useful in comparing the competencies of students and teachers and pinpointing where a large percentage of at-risk students, which is 15%, are located between nine and 10, as shown in Figure 9.Using the regression model, students can be grouped according to their needs for school support in specific subjects.This allows them to receive the appropriate level of support, whether full or partial, to meet their needs.

773
The model's accuracy in predicting academic success rises as the number of outliers decreases.The presence of these values indicates that the results the concerned students obtained were far from those predicted by the model.The explanation for this is either that the students made more efforts in the second semester, which led to their success (these cases are rare), or they cheated on the final exam, which helped them succeed.
Choosing the ideal model that can be applied may not be an easy process, and that model can be applied across different types of models of the same type configured with different hyperparameters.We leave the choice to officials according to whether they should consider the outliers or not.As shown in Table 7, it is better to adopt the models (gradient boosting regressor, CatBoost regressor) in regression or, as shown in Table 5, (CatBoost classifier, linear discriminant analysis) in classification.
We intend to create a web application using the lightweight web framework Flask in order to deploy our model to production.We will extract predictive results from the model through the web application and generate a list of students whose model predicts they are at risk of failing.Officials at the educational institution will be able to identify students who need additional help and reinforcement on specific subjects as a result of this prediction.

CONCLUSION
This article explains how predicting baccalaureate graduation rates based on their academic abilities.To evaluate the efficacy of regression and classification models, we compared them.Informed decisions regarding how to support challenging students can be made by school administrators with the help of our methodology, which has been put into practice.In order to find solutions to lower failure rates, future studies can examine students' socioeconomic and health statuses, as well as the educational procedures that affect their academic achievement.

Figure 2 .
Figure 2. Regression between the input and the output variables

Figure 3 .
Figure 3 .Difference between the effect of the reg average and the cc average

Figure 4 .
Figure 4. Distribution of general average

Int 769 Figure 5 .
Figure 5. Lifecycle of model management


ISSN: 2252-8938 Int J Artif Intell, Vol. 13, No. 1, March 2024: 764-774 772 included in the training set.The prediction error R 2 and the R-squared R 2 are both measures of regression model performance, but they have different interpretations and applications.The prediction error R 2 (0.823) for a gradient boosting regressor, as shown in figure8, is a measure of how well the model predicts new data that is not included in the training set, while the R-squared R 2 (07898) determines exactly how the model fits the training collected data and how much of the variance in the target variable is explained by the predictor variables.It is also valuable for predicting how well the model will perform on new data.

Figure 8 .
Figure 8. Prediction error for gradient boosting regressor model

Figure 9 .
Figure 9. Breakdown of pass averages

Table 4 .
Features description

Table 5 .
Models' performance using Pycaret library

Table 6 .
Description of the different evaluation metrics using CatBoost Classifier model

Table 7 .
Models performance using Pycaret library

Table 8 .
Evaluation metrics description