Investigating the use of near infrared in household object recognition

Abstract

Household object recognition remains a challenging task in robotics due to environmental disturbances like varying illumination conditions. This study investigates the utilization of standalone near-infrared (NIR) imaging to mitigate household object recognition challenges across 20 distinct classes under diverse lighting scenarios. Crucially, this research focuses on evaluating the isolated, single-channel baseline capacity of the NIR spectrum to explicitly establish its independent robustness prior to introducing complex multimodal architectural fusion frameworks. Based on a total dataset of 73,269 synchronized images (36,631 red, green, blue (RGB) and 36,638 NIR) captured across variable illumination states, this research modeled the recognition process using a convolutional neural network (CNN) with transfer learning based on the efficient network (EfficientNet) architecture. To evaluate the model's capabilities, we tested performance across variations of dim white and RGB light. This research reveals that the NIR-based model significantly outperforms the standard RGB baseline, which achieved an average accuracy of (64.55 ± 24.32)%. In contrast, the investigative NIR model maintained high robustness, achieving an average accuracy of (95.85 ± 3.08)% across all testing scenarios. This research establishes a robust foundation for implementing NIR spectrum analysis in household robotics to enhance operational reliability in real-world environments.