Adaptive deformable feature augmentation and refinement network for scene text detection and recognition

Abstract

Scene text recognition (STR) is the task of detecting and identifying text within images captured from natural scenes, a challenging process due to variations in text appearance, orientation, and background complexity. The proposed methodology, adaptive deformable feature augmentation and refinement network (ADFARN), is designed to address these challenges by combining deformable convolutional networks for robust enhanced feature extraction with a novel deep feature refinement (FRE) that leverages refinement for precise text localization. This approach enhances the differentiation between text and background, significantly improving recognition accuracy. The ADFARN methodology includes a comprehensive process of feature extraction, deep feature augmentation module (DFAM), and the generation of score and threshold maps through differentiable binarization. The adaptive nature of the model allows it to handle low resolution and partially occluded text effectively, further increasing its robustness. Additionally, the proposed method aligns visual and textual features seamlessly. Extensive performance evaluation on the common objects in context (COCO)-Text dataset demonstrates that ADFARN outperforms existing state-of-the-art methods in terms of precision, recall, and F1-scores, establishing it as a highly effective solution for STR in real world applications.