Wavelet-based super resolution using pansharpened multispectral images

: Several remote sensing applications require high-spatial-high-spectral resolution multispectral (MS) images. However, most MS sensors provide low-spatial-high-spectral resolution MS images together with high-spatial-low-spectral resolution panchromatic (PAN) bands. In order to increase the spatial resolution of MS bands to the resolution of PAN images and to obtain high-spatial/spectral resolution MS bands, either MS and PAN images are fused (i.e., pansharpening) or super resolution (SR) is performed using MS bands only. Nevertheless, existing methods do not utilize the available temporal and spatial information together. In this paper, we propose a multiframe SR algorithm using high-spatial/spectral resolution MS images (i.e., pansharpened), taking advantage of both spatial and temporal data, in order to exceed the spatial resolution of the available PAN bands. We first employ a wavelet-based pansharpening method on a set of MS and PAN images captured at different times. Then, we utilize these pansharpened MS bands in a wavelet-based multiframe SR scheme. The proposed method reveals the inter-wavelet-subband relationship of multitemporal images for SR. We demonstrate our results with comparisons on a Landsat 7 ETM+ dataset.

PDF

___

[1] Amro I, Mateos J, Vega M, Molina R, Katsaggelos AK. A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP Journal on Advances in Signal Processing 2011; 2011 (1): 1-22. doi: 10.1186/1687-6180-2011-79
[2] Tu TM, Huang PS, Hung CL, Chang CP. A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery. IEEE Geoscience and Remote Sensing Letters 2004; 1 (4): 309-312. doi: 10.1109/LGRS.2004.834804
[3] Laben CA, Brower BV, inventors; Eastman Kodak Co, assignee. Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. United States Patent US 6,011,875. 2000 Jan 4.
[4] Gillespie AR, Kahle AB, Walker RE. Color enhancement of highly correlated images. II. Channel ratio and “chromaticity” transformation techniques. Remote Sensing of Environment 1987; 22 (3): 343-365. doi: 10.1016/0034- 4257(87)90088-5
[5] Schowengerdt RA. Remote sensing: models and methods for image processing. Elsevier, 2006.
[6] Price JC. Combining multispectral data of differing spatial resolution. IEEE Transactions on Geoscience and Remote Sensing 1999; 37 (3): 1199-1203. doi: 10.1109/36.763272
[7] Garzelli A, Nencini F. Interband structure modeling for pan-sharpening of very high-resolution multispectral images. Information Fusion 2005; 6 (3): 213-224. doi: 10.1016/j.inffus.2004.06.008
[8] Kim Y, Lee C, Han D, Kim Y, Kim Y. Improved additive-wavelet image fusion. IEEE Geoscience and Remote Sensing Letters 2010; 8 (2): 263-7. doi: 10.1109/LGRS.2010.2067192
[9] Delleji T, Kallel A, Ben Hamida A. Multispectral image adaptive pansharpening based on wavelet transformation and NMDB approaches. International Journal of Remote Sensing 2014; 35 (19): 7069-7098. doi: 10.1080/01431161.2014.967883
[10] Tian J, Ma KK. A survey on super-resolution imaging. Signal, Image and Video Processing 2011; 5 (3): 329-42. doi: 10.1007/s11760-010-0204-6
[11] Irani M, Peleg S. Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991; 53 (3): 231-239. doi: 10.1016/1049-9652(91)90045-L
[12] Zhou F, Yang W, Liao Q. Interpolation-based image super-resolution using multisurface fitting. IEEE Transactions on Image Processing 2012; 21 (7): 3312-3318. doi: 10.1109/TIP.2012.2189576
[13] Babacan SD, Molina R, Katsaggelos AK. Variational Bayesian super resolution. IEEE Transactions on Image Processing 2010; 20 (4): 984-999. doi: 10.1109/TIP.2010.2080278
[14] Wang Z, Yang Y, Wang Z, Chang S, Han W et al. Self-tuned deep super resolution. In: IEEE 2015 Conference on Computer Vision and Pattern Recognition Workshops; Boston, Massachusetts, USA; 2015. pp. 1-8. doi: 10.1109/CVPRW.2015.7301266
[15] Robinson MD, Toth CA, Lo JY, Farsiu S. Efficient Fourier-wavelet super-resolution. IEEE Transactions on Image Processing 2010; 19 (10): 2669-81. doi: 10.1109/TIP.2010.2050107
[16] Vandewalle P, Sbaiz L, Vandewalle J, Vetterli M. Super-resolution from unregistered and totally aliased signals using subspace methods. IEEE Transactions on Signal Processing 2007; 55 (7): 3687-3703. doi: 10.1109/TSP.2007.894257
[17] Demirel H, Anbarjafari G. Discrete wavelet transform-based satellite image resolution enhancement. IEEE Transactions on Geoscience and Remote Sensing 2011; 49 (6): 1997-2004. doi: 10.1109/TGRS.2010.2100401
[18] Dong W, Zhang L, Shi G, Wu X. Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Transactions on Image Processing 2011; 20 (7): 1838-1857. doi: 10.1109/TIP.2011.2108306
[19] Li F, Jia X, Fraser D. Superresolution reconstruction of multispectral data for improved image classification. IEEE Geoscience and Remote Sensing Letters 2009; 6 (4): 689-693. doi: 10.1109/LGRS.2009.2023604
[20] Aydin VA, Foroosh H. A linear well-posed solution to recover high-frequency information for super resolution image reconstruction. Multidimensional Systems and Signal Processing 2018; 29 (4): 1309-1330. doi: 10.1007/s11045-017- 0499-3
[21] Carper W, Lillesand T, Kiefer R. The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data. Photogrammetric Engineering and Remote Sensing 1990; 56 (4): 459-467.
[22] Chavez Jr PS. Digital merging of Landsat TM and digitized NHAP data for 1: 24 000-scale image mapping. Photogrammetric Engineering and Remote Sensing 1986; 52 (10): 1637-1646.
[23] Fasbender D, Radoux J, Bogaert P. Bayesian data fusion for adaptable image pansharpening. IEEE Transactions on Geoscience and Remote Sensing 2008; 46 (6): 1847-1857. doi: 10.1109/TGRS.2008.917131
[24] Otazu X, González-Audícana M, Fors O, Núñez J. Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods. IEEE Transactions on Geoscience and Remote Sensing 2005; 43 (10): 2376-2385. doi: 10.1109/TGRS.2005.856106
[25] Alparone L, Wald L, Chanussot J, Thomas C, Gamba P et al. Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing 2007; 45 (10): 3012-3021. doi: 10.1109/TGRS.2007.904923
[26] Temizel A, Vlachos T. Wavelet domain image resolution enhancement using cycle-spinning. Electronics Letters 2005; 41 (3): 119-121. doi: 10.1049/el:20057150
[27] Chan RH, Chan TF, Shen L, Shen Z. Wavelet algorithms for high-resolution image reconstruction. SIAM Journal on Scientific Computing 2003; 24 (4): 1408-1432. doi: 10.1137/S1064827500383123
[28] Jiji CV, Joshi MV, Chaudhuri S. Single‐frame image super‐resolution using learned wavelet coefficients. International Journal of Imaging Systems and Technology 2004; 14 (3): 105-112. doi: 10.1002/ima.20013
[29] Patel RC, Joshi MV. Super-resolution of hyperspectral images: Use of optimum wavelet filter coefficients and sparsity regularization. IEEE Transactions on Geoscience and Remote Sensing 2014; 53 (4): 1728-1736. doi: 10.1109/TGRS.2014.2346811
[30] Zhang H, Zhang L, Shen H. A super-resolution reconstruction algorithm for hyperspectral images. Signal Processing 2012; 92 (9): 2082-2096. doi: 10.1016/j.sigpro.2012.01.020
[31] Fernandez-Beltran R, Latorre-Carmona P, Pla F. Single-frame super-resolution in remote sensing: a practical overview. International Journal of Remote Sensing 2017; 38 (1): 314-354. doi: 10.1080/01431161.2016.1264027
[32] Bovolo F, Bruzzone L, Capobianco L, Garzelli A, Marchesi S et al. Analysis of the effects of pansharpening in change detection on VHR images. IEEE Geoscience and Remote Sensing Letters 2009; 7 (1): 53-7. doi: 10.1109/LGRS.2009.2029248
[33] Aydin VA, Foroosh H. In-band sub-pixel registration of wavelet-encoded images from sparse coefficients. Signal, Image and Video Processing 2017; 11 (8): 1527-1535. doi: 10.1007/s11760-017-1116-5
[34] Aydin VA, Foroosh H. Motion compensation using critically sampled dwt subbands for low-bitrate video coding. In: IEEE 2017 International Conference on Image Processing; Beijing, China; 2017. pp. 21-25. doi: 10.1109/ICIP.2017.8296235
[35] Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 2004; 13 (4): 600-612. doi: 10.1109/TIP.2003.819861