A novel deep learning-based pollinator species classification using the multimodal species-image regularization framework

Abstract

Pollinators, such as bees, butterflies, and other insects, are essential to maintaining biodiversity and ensuring agricultural productivity, with over 80% of flowering plants and 75% of global food crops relying on them for successful reproduction. However, pollinator populations are facing significant declines due to environmental changes, habitat destruction, and climate change, posing substantial risks to ecosystems and global food security. This paper introduces the multimodal species-image regularization (MSIR) framework for automating the classification of pollinator species using both binary classification (pollinator vs. non-pollinator) and multiclass classification (bee genera). The framework integrates multimodal data, including visual images of pollinators and species details such as genus, family, and environmental factors, to improve accuracy and scalability. The system leverages the multimodal contrastive learning framework (MCLF) to align both image and species-detail features into a unified embedding space, enabling mor effective classification. Additionally, the framework applies image-species prototype regularization (ISPR) and species-detail prototype regularization (SDPR) to further enhance the classification accuracy by regularizing the tunable weights based on prototype alignment. The proposed deep learning (DL) model is evaluated against traditional machine learning (ML) methods, such as random forest (RF), and demonstrates superior performance on key metrics, including accuracy, precision, recall, and F1-score.