Iterative applications of image completion with CNN-based failure detection

概要

Image completion is a technique to fill missing regions in a damaged or redacted image. A patch-based approach is one of major approaches, which solves an optimization problem that involves pixel values in missing regions and similar image patch search. One major problem of this approach is that it sometimes duplicates implausible texture in the image or overly smooths down a missing region when the algorithm cannot find better patches. As a practical remedy, the user may provide an interaction to identify such regions and re-apply image completion iteratively until she/he gets a desirable result. In this work, inspired by this idea, we propose a framework of human-in-the-loop style image completion with automatic failure detection using a deep neural network instead of human interaction. Our neural network takes small patches extracted from multiple feature maps obtained from the completion process as input for the automated interaction process, which is iterated several times. We experimentally show that our neural network outperforms a conventional linear support vector machine. Our subjective evaluation demonstrates that our method drastically improves the visual quality of resulting images compared to non-iterative application.

発表文献
Journal of Visual Communication and Image Representation
中島悠太
中島悠太
准教授

コンピュータビジョン・パターン認識などの研究。ディープニューラルネットワークなどを用いた画像・映像の認識・理解を主に、自然言語処理を援用した応用研究などに従事。