Photo-realistic photo synthesis using improved conditional generative adversarial networks

Abstract

There are a wide range of potential uses for both the forward (generating face drawings from actual images) and backward (generating photos from synthetic face sketches). However, photo/sketch synthesis is still a difficult problem to solve because of the distinct differences between photos and sketches. Existing frameworks often struggle to acquire a strong mapping among the geometry of drawing and its corresponding photo-realistic pictures because of the little amount of paired sketch-photo training data available. In this study, we adopt the perspective that this is an image-to-image translation issue and investigate the usage of the well-known enhanced pix2pix generative adversarial networks (GANs) to generate high-quality photo-realistic pictures from drawings; we make use of three distinct datasets. While recent GAN-based approaches have shown promise in image translation, they still struggle to produce high-resolution, photorealistic pictures. This technique uses supervised learning to train the generator's hidden layers to produce low-resolution pictures initially, then uses the network's implicit refinement to produce high-resolution images. Extensive tests on three sketch-photo datasets (two publicly accessible and one we produced) are used to evaluate. Our solution outperforms existing image translation techniques by producing more photorealistic visuals with a peak signal-to-noise ratio of 59.85% and pixel accuracy of 82.7%.