A general framework for selecting appropriate criteria of student as research assistant using fuzzy delphi method

Received Dec 15, 2020 Revised Jul 22, 2021 Accepted Aug 12, 2021 This research aims to build a general framework for choosing the most appropriate set of criteria for recruiting student as a research assistant in a university research project. University researchers could benefit from such a framework because it could optimize the costs of research while also enhancing students research skills. In the same time, it is also essential that the quality of research ought to measure up to the grants provided by the university. Nevertheless, it is a challenging problem for many research supervisors in the selection of qualified research assistants. In this paper, we attempted to resolve this problem by building a general framework for selecting the appropriate criteria in the evaluation of student performance. We explored earlier studies on the proposed evaluation criteria of the research assistant and identified 47 most impactful criteria criteria. We obtained experts in engineering and information technology fields from two universities to answer questionnaires to identify their commonly used criteria for grant research assistant (GRA). Then, all the identified criteria were evaluated using the fuzzy delphi method (FDM) for finding the best fitting criteria which resulted in 16 most impactful criteria.


INTRODUCTION
The main aim of this paper is to propose a general framework for the selection of the appropriate criteria for evaluating student performance for a grant research assistant (GRA) in the IT and engineering departments at universities. Hence, it is essential to review the related theories, variables, and techniques that could be of support to the main aim of this research. The knowledge possessed by students could be defined as the skills and information acquired from learning [1]. Thus, one of the primary purposes of developing students' skills is to prepare them for job placement once they graduate. Several work tasks could be allocated to students based on their current skills while studying at a university.
It is undeniable that many universities focus on improving the skills and knowledge of students to meet the needs of the industry with human potentials [2]. The transferred skills in classrooms must have an impact on students in order to develop their knowledge in fields, such as doing researches and analytical work [3]. Away from work settings, this research set to emphasize the importance of developing and enhancing student research skills at the university level in order to support and improve university research outputs [4], [5]. This is achieved through the offering of the GRA job scheme in universities.
There are numerous variables available to assess the research skills of GRA [6], [7]. However, the selection of GRA is problematic because these research skills would overlap or intersect within matrices such as the evaluation variables, structure of research group, and the research proprieties. Therefore, computerbased methods are recommended to assist in the process of GRA selection decision making. A selection process must rely on several and interconnected evaluation attributes. The multi-criteria decision making (MCDM) would be a practical approach for effective selection of GRA based on different variables, attributes, and objectives.
It is commonly and practically known that all the members of any evaluation committee would not share a similar view when judging a problem. A panel of experts may come to a disagreement caused by different opinions on the rating of the alternatives or the merit of the criteria. Arriving at the best resolution despite such differences becomes a significant issue in group-based decision making. If experts realize that using a numeric scale for expressing their opinions is convenient, it would be useful to consider averaging the scores as a simple way for aggregating conflicting assessments. Besides, if the opinions of the members of a group do not carry the same weight, then it would be essential to implement a weighted averaging scale to specify their relative importance. However, to solve an IT or engineering evaluation problem, it is more important to arrive at the right level of consensus among the experts; through encouraging them to reconsider their assessments rather than aggregate their scores. This is the core of the delphi method. The delphi method, having a repetitive procedure, aims at making various subjective opinions converge into more widely acceptable viewpoints [8], [9]. The difficulty faced by any research supervisor in selecting a GRA could be attributed to several reasons [10]- [13]: i) the variety of evaluation criteria and characteristics (there is no standard for the GRA evaluation and selection criteria), ii) the process of assessing the skills of researcher performance due to the different type of research activities.
In this case, a supervisor would face difficulties in creating a research group with balanced individual and group work skills. In order to do so, a supervisor must evaluate such skills by employing useful variables. There is a large number of evaluation variables that can be used in this respect. However, this large number makes it harder for supervisors to pinpoint the most relevant variables to evaluate the research skills performance based on different characteristics. To solve this problem, we explored earlier studies pertaining to the evaluation and selection of GRA to identify the commonly used evaluation criteria. The criteria are then distributed to experts to evaluate followed by the application of the fuzzy delphi program.

RESEARCH METHOD
There are two major phases in this research. First is the identification of criteria through literatures reviews. The second phase is the evaluation of the criteria through experts' opinion which are implemented using the fuzzy delphi method (FDM).

Criteria identification
Literatures related to research skills are important primary sources to identify criteria that could be adopted to assess the GRA skills. There are numerous skills which commonly appeared and proposed in literatures as important research skills. These skills could be grouped into corresponding category. Every group would include a collection of criteria to facilitate the process of mathematical calculations [14], [15].

Implementation of the fuzzy delphi method
Such large number of criteria extracted from the literature review would need to be evaluated using the delphi fuzzy to achieve the consensus of experts on the most useful criteria. In other words, the criteria would be examined and tested in the interviews with experts. Figure 1 presents the flow of steps involved in the implementation of the FDM to determine the suitable evaluation criteria of GRA. The steps are further explained in sections 2.2.1 to 2.2.4.

Selection of the experts
In this study, experts are defined as researchers who have wide research experience and supervises many postgraduate students and GRA. In terms of the numbers of experts for the study, a consensus has to be reached in this regards. Previous researchers suggested that the number of experts ranges between 3 to 25 as an optimal number in Delphi method [16]- [20]. Nevertheless, a precondition to the experts is that they must have the aptitude to process information and give decisions. In this study, we chose 23 experts among

Development of questionnaire
We developed a questionnaire to collect the data (opinions of the experts) that are grounded on the criteria inferred from the analysis and combination of existing guidelines. Rahim et al. [21] suggested that in the digital age, researchers may place the questionnaire online to avoid delay and burden. Hence, the questionaires were built and distributed via the use of google form online survey. The questionnaires comprises of two parts; part one is related to the personal information of the expert and part two contains the list of identified criterias to be scored. The assessment was done by using the fuzzy likert with five scale response: strongly agree, agree, neutral, disagree, and strongly disagree.

Data collection
In this questionnaires, an electronic form of answers was presented and submitted to the selected group of professors, assistant professors and professors who have long experience in supervising students in the universities. The responses for the questionnaires were collected through google forms and downloaded as microsoft excel file for ease of analysis, which involved finding the average of threshold value, the average percentage of expert' consensus and average fuzzy score. It is suggested that maximum and minimum method uses a cumulative frequency distribution and fuzzy scoring in order to deal with the opinion of experts with respect to the fuzzy numbers resulting from the FDM [22], [23].

Fuzzy delphi data analysis
The FDM is used to determine the best type of criteria and to set the type of factors appertaining to this study. The following are the steps performed in fuzzy delphi data analysis:  Convert the Linguistic variables to triangular fuzzy numbers. The linguistic variables are for weighting the agreement of the experts. Table 1 as shown in the process of linguistic variables for weighting the agreement of the experts [19], [24]. Calculate the distance between two fuzzy numbers by measuring the deviation between the average fuzzy evaluation data and the experts' evaluation as shown in (1). Data vertex method used to calculate the distance, which is the threshold value ( ) of the two (2) fuzzy numbers = ( 1, 2, 3) and = ( 1, 2, 3), which are then averaged for all the results. The last average represents the total threshold value ( ) for that particular criteria.
 Calculate the percentage for each item, when the threshold value (response) for each criteria with d<= 0.2. The data analysis is based on the triangular fuzzy number where it aims to get threshold value ( ). Therefore, the first requirement to be followed is threshold value ( ) must be less or equal to 0.2. Percentages for certain item if reach an agreement of experts exceeding 75.0%, then this item is accepted. Instead, if it is less than 75.0%, it means that this item need to be rejected.


The average fuzzy score was determined based on the value of α-cut, which is 0.5. If the average fuzzy score ( ) is more than or equal to 0.5, the elements are regarded as has achieved the consensus of the experts. The formula used for defuzzification in (2):

RESULTS AND DISCUSSION
In this study, such findings that represent the FDM and the final set of criteria gathered from the literature review. This is in order to formulate the questionnaire and then select the suitable criteria. To do so, there are numerous methods for the identification of the relationship among various the criteria. This identification is based on the characteristics of such criteria.

Identified criteria
The results of the student classification process are presented in Table 2. Collected from literature review, the criteria are grouped into four categories; namely Human Behavior, Methodology skills, Mental and Personal skills. These categories were derived from relevant literatures and suggested by experts in a close format. Table 3 shows the results of the accepted criteria extracted from the FDM. Consistency among the results of each group of criteria does exist and demonstrated through the results. These accepted criteria where its percentage is greater than or equal to 75%, were divided into coherent categories. This process is to facilitated the understanding and mathematical operations in the future works. The percentage attained for each criteria is approved on the basis of this ratio, as described in the steps of the fuzzy delphi analysis. Other analysis steps were carried out in the fuzzy delphi phase along with drafting abbreviations for each element to facilitate the work of tables and reduce their size.

Data analysis using fuzzy delphi
Two Iraqi universities were selected as a case study to examine GRA skills selection, in particular, the lecturers from Department of IT, the Department of Computer Science and the Departments of Engineering in the University of Anbar and the University of Technology in Baghdad. The assessment of the criteria of GRA skills were done using the five-scale fuzzy likert responses: strongly agree, agree, neutral, disagree, strongly Disagree; each with a score of 5, 4, 3, 2, 1 respectively. The following sections and subsections elaborates on the evaluation and selection of the initial assessment decision matrix. This is in order to have an ideal decision matrix and accordingly exclude dimensions and elements having no impact on the evaluation and selection of GRA. Nevertheless, it is noteworthy that the FDM is used to get an expert agreement on the pillars and the elements of the said decision matrix. The findings of the experts' consensus on the formulation of the proposed decision matrix are summarized in Table 4, comprising of the average of threshold value, the average percentage of expert' consensus and average fuzzy score. Red colored fonts represents the results that are rejected because of the average percentage score that is below 75%, as explained in the previous section.

CONCLUSION
This paper proposed a general framework for the selection of most fitting criteria for GRA based on experts' evaluations. There were 47 initial criteria obtained through literature review. There were 23 experts from IT and engineering related departmens at two universities; University of Anbar and the University of Technology in Baghdad were presented with these 47 criteria in a questionaire form to score using five scale fuzzy likert response. The expert's responses on these criteria were evaluated using the FDM, where in turn 16 criteria that met the condition for acceptance were selected as the best impactful criteria. Future works will attempts to assess these 16 standards criteria with MCDM technique in order to select the best candidate for GRA.