Sensors, Vol. 19, Pages 3727: Deep Visible and Thermal Image Fusion for Enhanced Pedestrian Visibility

Sensors, Vol. 19, Pages 3727: Deep Visible and Thermal Image Fusion for Enhanced Pedestrian Visibility Sensors doi: 10.3390/s19173727 Authors: Ivana Shopovska Ljubomir Jovanov Wilfried Philips Reliable vision in challenging illumination conditions is one of the crucial requirements of future autonomous automotive systems. In the last decade, thermal cameras have become more easily accessible to a larger number of researchers. This has resulted in numerous studies which confirmed the benefits of the thermal cameras in limited visibility conditions. In this paper, we propose a learning-based method for visible and thermal image fusion that focuses on generating fused images with high visual similarity to regular truecolor (red-green-blue or RGB) images, while introducing new informative details in pedestrian regions. The goal is to create natural, intuitive images that would be more informative than a regular RGB camera to a human driver in challenging visibility conditions. The main novelty of this paper is the idea to rely on two types of objective functions for optimization: a similarity metric between the RGB input and the fused output to achieve natural image appearance; and an auxiliary pedestrian detection error to help defining relevant features of the human appearance and blending them into the output. We train a convolutional neural network using image samples from variable conditions (day and night) so that the network learns the appearance of humans in t...
Source: Sensors - Category: Biotechnology Authors: Tags: Article Source Type: research