Cascadic Multireceptive Learning for Multispectral Pansharpening

Pansharpening refers to the fusion of a panchromatic image with high spatial resolution (PAN) a multispectral image with low spatial resolution (LRMS) image with low spatial resolution to obtain a high spatial resolution multispectral (HRMS) image, which is beneficial to visual display and geographic research. Recently, many deep learning (DL) methods have been proposed to address the pansharpening problem, but still a few examples of DL-based techniques are designed from the perspective of a better receptive field while the scale of features greatly varies among different ground objects. In this article, we mainly focus on designing a cascadic multireceptive learning resblock (CML-resblock) relying on the residual network (ResNet) block, which can efficiently extract multiscale features from both the PAN and LRMS images. Moreover, we propose a novel multiplication network preserving a physical significance, which uses deep neural networks (DNNs) to learn the coefficients of the pixelwise restoration mapping and multiplies the upsampled LRMS image with the learned coefficients to get the HRMS image. The two parts mentioned above constitute our cascadic multireceptive learning network (CMLNet). Extensive experiments on both reduced-resolution and full-resolution images acquired by the WorldView-3 (WV-3), GaoFen-2 (GF-2), and QuickBird (QB) satellites show that the proposed approach outperforms state-of-the-art methods. Furthermore, additional experiments have been conducted to prove the generality of the CML-resblock and multiplication network. The code is available at: https://github.com/wajuda/CML.