Classification of Tri Pramana learning activities in virtual reality environment using convolutional neural network

Abstract

Tri Pramana as the local genius of Balinese society, is now adopted in the education system. This adaptation results in a Learning Cycle Model which essentially consists of three classes namely Sabda Pramana (theoretical study), Pratyaksa Pramana (direct observation), and Anumana Pramana (practicum). In learning activities, it is difficult for educators to fully observe individuals to find out the most suitable learning model. Through Virtual Environment Technology, educators can observe students more freely through the recording of students' activities. However, in its implementation, manual analysis requires large resources. Deep Learning approach based on Convolutional Neural Network (CNN) is able to automate this analysis process through the classification ability of the image of the recorded learner activity. To produce a robust CNN model, this research compares four of the most commonly used architectures, namely ResNet-50, MobileNetV2, InceptionV3, and Xception. Each architecture is tuned using a combination of learning rate and batch size. Through a 512 x 512 resolution dataset with 70% training subset (4,541 images), 20% validation (1,296 images), and 10% test (652 images), the best ResNet model is obtained with a learning rate configuration of 1e-3 and batch size 64 with an accuracy of 99.39%, precision of 99.37%, and recall of 99.42%.