Video deblurring by utilizing non-local reference frames

Abstract

In this thesis, we study the problem of video deblurring by utilizing nonlocal reference frames. Our key observation is that some frames in a video are much sharper than others, and thus we can transfer the texture information in these sharp reference frames to blurry frames. We first present an internal learning approach that heuristically selects sharp frames from a video and then trains a convolutional neural network on these sharp frames. The trained network often absorbs visual details in sharp reference frames to perform deblurring on all video frames. Such an internal learning approach can avoid the domain gap between synthetic training data and real-world test data, which is an issue for existing video deblurring approaches. While internal learning approaches are generally slow at test time, we also develop an external learning method with our proposed multi-head source reference attention module (MHSRA) to significantly reduce inference time for video deblurring with nonlocal reference frames. Our perceptual user study on real-world videos shows that our methods with nonlocal reference frames can reconstruct clearer and sharper videos than state-of-the-art video deblurring approaches.

Date of Award	2022
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Qifeng CHEN (Supervisor)