YOLO-based deep learning for tooth detection, segmentation, and numbering in panoramic radiographs

Abstract

Panoramic dental radiographs are essential for diagnosis and treatment planning, but are difficult to interpret due to overlapping anatomical structures and limited image quality. Automating tooth detection, segmentation, and numbering can enhance clinical efficiency and reduce observer variability. This study developed a you only look once version 8 (YOLOv8)-based deep learning model for automatic detection and segmentation of teeth in panoramic radiographs. A dataset of 302 images containing 9,009 annotated teeth was divided into 70% training, 20% validation, and 10% testing sets. Teeth were labeled using the Federation Dentaire Internationale (FDI) two-digit numbering system with bounding boxes and segmentation masks. Model performance was assessed using precision, recall, F1-score, and mean average precision (mAP) at intersection over union (IoU) thresholds of 0.5 and 0.5–0.95. The model achieved bounding box precision, recall, and F1-score of 0.9075, 0.9352, and 0.9212, respectively, and segmentation scores of 0.9078, 0.9344, and 0.9209. Bounding box mAP@0.5 reached 0.9510 and mask mAP@0.5 was 0.9504, while stricter thresholds lowered performance to 0.7763 and 0.6896. Class-wise analysis showed high accuracy overall but reduced performance on posterior teeth due to anatomical overlap and peripheral image quality. YOLOv8 enables fast, accurate, and robust tooth detection and segmentation, supporting real-time computer-aided diagnosis in orthodontics, prosthodontics, and forensic dentistry.