Image fusion is the process of combining image data from different sources into a single image. It is a vast subject that encompasses many methods, some of them very complicated [1]. In this post we show a simple and straightforward way of doing it.

By far the most common application in remote sensing is pansharpening, the process by which we can combine multispectral data with a co-registered panchromatic image of higher spatial resolution, normally acquired at the same time by the same satellite. The result is a multispectral image of higher spatial resolution than the original one.

There are of course other important applications, such as the one described in this post, namely the merging of radar (Sentinel SAR) and optical (Landsat VNIR) imagery. This is a very powerful method because it enables us to combine the spectral information of an optical image with the texture of a radar image, creating a single image that is incredibly rich in information.

We will use as example a subset of a Landsat 8 OLI image and one from a Sentinel 1A SAR image, both covering the same region of the northeastern Antarctic Peninsula:

Landsat 8 OLI and Sentinel 1A SAR images. James Ross Island group, northeastern Antarctic Peninsula.
James Ross Island group, northeastern Antarctic Peninsula. Landsat 8 OLI acquired 2017-02-04 (left), and Sentinel 1A SAR, acquired 2017-02-06 (right). The Landsat image is a RGB combination of bands 6, 5 and 4 (mid-IR, near-IR and visible red). The Sentinel image is single band gray scale.

These images have been preprocessed to have them cover exactly the same area, with the same pixel size (30 m in this example), and in the same coordinate reference system (UTM 21S). Both images are in GeoTIFF format and the data are stored as unsigned 16 bit integers, as is usual for these images. In the case of the Landsat 8 OLI image, the scene is already a RGB stack of bands 6, 5 and 4.

The procedure now is extremely simple: we will merely multiply both images, pixel by pixel. Of course, such a simple procedure can be done with any software that is capable of doing band operations. I here will go one step further in simplicity and use the utility program composite, part of the ImageMagick suite:

composite Sentinel-SAR.tif Landsat-654.tif -compose Multiply fusion.tif

ImageMagick operates directly in 16 bit, so no conversion to another data range is necessary and the result will likewise be a 16-bit image. Image multiplication will always darken an image, so the resulting image will always need some enhancement.

The only trouble is that ImageMagick does not understand GeoTIFF and the resulting image will not have any geographic reference information. This is however easily solved using the command line tools listgeo and geotifcp, that come with the GeoTIFF library.

We first extract the georeference information from one of the input images:

listgeo Landsat-654.tif > Landsat-654.geo

We then feed this same information into the fusion image to upgrade it to a GeoTIFF:

geotifcp -g Landsat-654.geo fusion.tif fusion-geo.tif

The results can then be opened in any GIS software as usual. For the sake of this post I made a simple 8-bit PNG version, also using ImageMagick directly from the command line:

convert -normalize -depth 8 -resize 20% -gamma 1.35 -contrast-stretch 2% fusion.tif fusion.png

If you look carefully you will see that there is a lot of processing in this humble line of code:

1. we first normalize the image, to use all available data range
2. then force the data into 8 bit range
3. resize the original image to reasonably fit into a blog post
4. enhance the image with a gamma function
5. cut the lower 2% of pixels

The resulting image is both useful and beautiful:

James Ross Island group, northeastern Antarctic Peninsula. Image fusion of Landsat 8 OLI and Sentinel 1A SAR.
James Ross Island group, northeastern Antarctic Peninsula. Image fusion of Landsat 8 OLI and Sentinel 1A SAR.

One can, for example, see the region of dry snow at the top of the ice dome on James Ross Island, as well as differences in snow facies over the glaciers. The dark line running in the direction NW-SE is a result of the imperfect mosaic compilation of two Sentinel images acquired along the same track.

There are also minor problems inherent to the radar sensor, as the geometric distortion at the cliffs and other rough topography, that account for mismatch between the optical and radar images. The results are nevertheless more than acceptable and the richness of information is readily apparent:

Detail of James Ross Island, northeastern Antarctic Peninsula. Image fusion of  Landsat 8 OLI and Sentinel 1A SAR.
Detail of James Ross Island, northeastern Antarctic Peninsula. Image fusion of Landsat 8 OLI and Sentinel 1A SAR.

Imagemagick is a very sophisticated and powerful image processing package that sometimes goes under the radar of many remote sensing and GIS experts. It can operate from the command-line, either as stand alone of through scripts, and can also work with very large images as it has minimal demands on system memory (no GUI!). It is worth a look.

As far as image processing goes, I always find an inherent joy in using simple methods to achieve useful and interesting results. There is power in simplicity.

References

[1] Pohl, Cle, and John L. Van Genderen. Multisensor image fusion in remote sensing: concepts, methods and applications. International journal of remote sensing 19.5 (1998): 823-854.

Advertisements