YOLOv11 optimization for tiny object in crowded scenes

Abstract

Small object detection in crowded urban and aerial scenes remains a critical challenge due to limited pixel information and information loss in deep neural networks. This study introduces a novel optimization framework for YOLOv11, specifically engineered for tiny-scale targets by integrating convolutional block attention modules (CBAM), k-means anchor clustering, and an enhanced feature pyramid network (FPN). Evaluated on the TinyPerson and COCO-mini datasets, the YOLOv11-optimized model achieves significant performance breakthroughs, delivering a +7.3% gain in mean average precision (mAP) and a +10.5% increase in recall over the baseline. Notably, the model achieved a recall of 0.072 on the TinyPerson dataset, with double sensitivity of standard YOLOv11. With a high-speed inference rate of 27.3 FPS, this research demonstrates that strategic architectural refinements can drastically improve small object detection reliability without compromising real-time viability on edge devices.