Deep learning with cinematic rendering: fine-tuning deep neural networks using photorealistic medical images.

Deep learning with cinematic rendering: fine-tuning deep neural networks using photorealistic medical images. Phys Med Biol. 2018 Aug 16;: Authors: Mahmood F, Chen R, Sudarsky S, Yu D, Durr NJ Abstract Deep learning has emerged as a powerful artificial intelligence tool to interpret medical images for a growing variety of applications. However, the paucity of medical imaging data with high-quality annotations that is necessary for training such methods ultimately limits their performance. Medical data is challenging to acquire due to privacy issues, shortage of experts available for annotation, limited representation of rare conditions and cost. This problem has previously been addressed by using synthetically generated data. However, networks trained on synthetic data often fail to generalize to real data. Cinematic rendering simulates the propagation and interaction of light passing through tissue models reconstructed from CT data, enabling the generation of photorealistic images. In this paper, we present one of the first applications of cinematic rendering in deep learning, in which we propose to fine-tune synthetic data-driven networks using cinematically rendered CT data for the task of monocular depth estimation in endoscopy. Our experiments demonstrate that: (a) Convolutional Neural Networks (CNNs) trained on synthetic data and fine-tuned on photorealistic cinematically rendered data adapt better to real medical images and de...
Source: Physics in Medicine and Biology - Category: Physics Authors: Tags: Phys Med Biol Source Type: research