CycleGAN for day-to-night image translation: a comparative study

Abstract

Computer vision tasks often fail when applied to night images, because the models are usually trained using clear daytime images only. This creates the need to augment the data with more nighttime image for training to increase robustness. In this study, we consider day-to-night image translation using both traditional image processing approaches and deep learning models. This study employs a hybrid framework of traditional image processing followed by a CycleGANbased deep learning model for day-to-night image translation. We then conduct a comparative study on various generator architectures in our CycleGAN model. This research compares four different CycleGAN models; i.e., the orginal CycleGAN, feature pyramid network (FPN) based CycleGAN, the original U-Net vision transformer based UVCGAN, plus a modified UVCGAN with additional edge loss. The experimental results show that the orginal UVCGAN obtains an Frechet inception distance (FID) score of 16.68 and structural similarity index ´ measure (SSIM) of 0.42, leading in terms of FID. Meanwhile, FPN-CycleGAN obtains an FID score of 104.46 and SSIM score of 0.44, leading in terms of SSIM. Considering FPN-CycleGAN’s bad FID score and visual observation, we conclude that UVCGAN is more effective in generating synthetic nighttime images.