HVS-Aware Single-Shot HDR Imaging Using Deep Convolutional Neural Network

Abstract

Recent advances in digital imaging technology have enabled a variety of mobile devices, such as smartphones and tablets, to capture high-quality images and videos. However, despite this advancement, conventional imaging devices have a limited dynamic range to store the scene information and the noise components present in the acquisition process. High dynamic range (HDR) images are capable of capturing the full dynamic range of real scenes, overcoming the limitations of conventional imaging devices. With this advantage, HDR images have been employed in a wide range of practical applications, including in the camera and display industries, and recent display devices support HDR content. However, devices designed to capture HDR images directly are too expensive to be employed in practical applications. Therefore, many researchers are engaged in developing HDR imaging algorithms to synthesize HDR images using conventional imaging devices. In this thesis, we propose a single-shot HDR imaging algorithm using a deep convolutional neural network (CNN) for row-wise varying exposures in a single image. First, we propose to use a deep CNN to restore the missing information resulting from under- and/ or over-exposed pixels in an input image and reconstruct the raw radiance map. To obtain full-color HDR images, we employ conventional demosaicing algorithms to interpolate the color information. We also develop a loss function for the CNN by employing the human visual system (HVS) properties. Moreover, we test a simple tone-mapped function in the L2 domain as the loss function. We then evaluate another metric using the retinex model by decomposing images into illumination and reflectance components. Finally, inspired by the perceptually-motivated metric, we propose an HVS model as the loss function to guide the network for HDR imaging. We choose these among the existing indexes, because the HVS model is a standard function applied in HDR imaging technologies. Experimental results show that the proposed algorithm provides HDR images of higher quality than those of conventional algorithms. Specifically, the proposed model achieves the highest image quality scores, e.g., HDRVDP, pu-PSNR, and log-PSNR, over the test images.