Hello, everybody.
Welcome back to Remote Sensing Foundations.
This week, we're gonna learn about some aspects
of image processing.
Specifically, we'll be discussing image restoration,
image enhancement, and image compositing.
At the end of the week, I hope that you'll understand
how and why images are processed,
how to interpret histograms of reflectance values,
how to understand the difference
between a composited and a raw image,
and then finally, how to understand
when you might use image masking.
Let's get started.
First, let's review some things that we learned last week.
So, last week, we covered data types
in GIS and remote sensing.
So, we talked about the difference between vector data
and raster data.
We also covered the difference
between extent and grain size.
So, grain size is kind of similar to a pixel size,
and extent is the entire area
that you are capturing or measuring or studying.
We also talked about band composites within images.
So, images, they can have multiple different bands.
And in order to visualize those bands as a color image,
we use three different bands.
Those bands could be red, green, and blue,
which would make a true-color image.
Or we can make a false color image by compositing
bands that aren't the red, green, and blue band,
and that can help us better interpret features
on the landscape.
Let's dive in and talk about image processing
and what that entails.
Digital image processing includes the steps that we take
to turn satellite data into images.
We're gonna talk about three aspects
of image processing this week: image restoration,
image enhancement, and image compositing.
In future weeks, we're also gonna talk
about image transformation and image classification.
Broadly speaking, image restoration is concerned
with the correction and calibration of images
in order to achieve as faithful of representation
of the Earth's surface as possible,
which is a fundamental goal in remote sensing.
Image enhancement is concerned with the modification
of images to optimize their appearance to our eyes.
So, this is really important in terms of making maps,
because visual analysis is such a key element
in remote sensing.
And then, finally, image compositing is a way
to correct images that have lost or undesirable information.
There are two main types of image restoration,
including radiometric restoration and geometric restoration.
These kinds of restoration are usually referred to
as image pre-processing, because they're done
before doing any other sort of manipulation
or analysis of images.
We'll start off by talking about radiometric restoration,
which is the process where we convert raw DNs
received by the remote sensing sensor,
and convert them into surface reflectance values.
Radiometric restoration can also involve removing
or diminishing distortions in the electromagnetic energy
that is registered by each detector.
I've shown on the right the general process
that might happen in a satellite with satellite data
to convert it from DNs to usable surface reflectance images.
So, the sensor collects information as DNs,
which are also referred to as digital numbers.
And these numbers reflect the intensity
of electromagnetic energy.
DNs range from zero to some higher number,
depending on the sensor's bit size.
Sensors can usually automatically convert DNs to radiance,
given rescaling factors that scientists or engineers
will program into the sensor.
Then, in order to convert sensor radiance
to top-of-atmosphere reflectance,
we have to use information like the Earth-sun distance,
the solar zenith angle and the exo-atmospheric irradiance.
Top-of-atmosphere Images are technically usable,
but because they reflect the view from above the atmosphere,
they can have distortions
due to light scattering in the atmosphere,
which is also called atmospheric haze.
And you can see the effect of atmosphere haze
in the top-of-atmosphere image
at the bottom left of the slide;
you can see that images with haze appear more washed out,
and they don't have a high degree of contrast.
We can correct for this haze
and create surface reflectance images
by having an understanding of atmospheric conditions
when that satellite imagery was collected.
There are also some other reasons
why radiometric restoration is needed.
For example, random noise, which is unpredictable
or unsystematic performances of the sensor
where certain pixels have randomly higher
or lower values than they should.
This can occur due to errors in the transmission of data.
Striping is another radiometric error,
and it's an error where lines in the image appear lighter
or darker than they should,
and this is due to a calibration issue
caused by incorrect calibration
of light detectors within the sensor.
We'll talk more about random noise and striping
and how we can correct those at the end of the presentation
when I talk about image compositing.
The other type of image pre-processing restoration
is called geometric restoration.
It's essential for any form of remotely-sensed imagery
to be accurately geo-referenced.
So, you register pixels with the correct coordinates,
and then you can accurately line up rasters
with features on the ground.
There's a lot of reasons, however, why geometric errors
or distortions occur.
For example, satellites, 'cause they fly
at such high altitudes, they have to consider details
like how the curvature of the Earth
and the rotation of the Earth can distort mapping of pixels.
For aircraft that fly closer to the ground,
they have a lot of other considerations,
including relief displacement,
which is when terrain elevation
distorts their view of the ground.
Variable topography can lead to a really irregular pattern
of distortions, and it can be hard to remove.
Small changes to the aircraft's speed and orientation
can also affect imagery,
like as shown in this bottom figure.
For instance, if a super-strong wind hits the plane,
you might have roll distortion;
or if you don't fly in a perfectly straight line,
you might have crab distortion.
It's important to minimize geometric errors as possible
when collecting remote sensing data,
but you can see that some of these errors
are inherent to the process.
Therefore, scientists and engineers
are really well-versed in using equations
and resampling techniques to geometrically correct images,
which I'll go over in the next slide.
An important place to begin
with the geometric correction process
is using ground control points, or GCPs.
Ground control points are points or features
on the Earth's surface
that you know the coordinates for.
So, those should be features that never change location.
For example, you could use buildings or islands or lakes.
Scientists can find several ground control points
in their imagery, and this will help them map out
the coordinates of all of the pixels in their image.
When we have pixel sizes
or orientations that we have to correct,
we use a process called resampling.
And resampling basically involves generating equations
that equate the ground control points
on the distorted image with where they are
on the corrected image, and then applying
those same equations to the remaining pixels
to correct the entire image.
An important part of resampling is calculating
new pixel values from the old pixel values.
And there's three types of resampling methods
that I'll bring up here.
Those are the nearest-neighbor method,
the bilinear interpolation method,
and the bi-cubic interpolation method.
These are good methods to know and understand
for other types of processing and data manipulation
that we'll cover later in the course.
In the nearest-neighbor resampling method,
the new pixel value will be equal to the value
of the closest or most overlapping old pixel.
Advantages of the nearest-neighbor method
is that it's pretty simple to implement,
but one disadvantage is that the features
in the output image could potentially be offset
by around one half pixel.
And this can cause a disjointed or blocky appearance
in the new image
Bilinear interpolation is where the value
of the new pixel is equal to the distance-weighted average
of the four nearest pixels.
This gives a smoother image appearance
than the nearest-neighbor method,
but you can get new pixel values
that you didn't have before.
And then, finally, in the bicubic interpolation method,
your new pixel value will be the average
of the 16 nearest pixels.
This also leads to images that are smoother in appearance
than the nearest-neighbor method,
and potentially even a little bit smoother
than the bilinear interpolation method.
Again, there are trade-offs between these methods.
For example, you have to be really careful
of using the bilinear and bicubic interpolation methods
when you're working with categorical data.
So, if you have a map where 1 represents forest
and 2 represents agriculture,
if you use the bilinear method,
you could get resampled pixel values of 1.5,
which would obviously be meaningless
and might sort of break your code.
So, therefore you wanna use the nearest-neighbor method
when you're working with categorical or non-continuous data.
I now wanted to give a little bit of context
about different Landsat collections
that you may find within Google Earth Engine,
'cause if you type in Landsat 8,
you'll see that there's a lot of options to choose from,
and this can be a bit confusing.
So, I wanted to lay this out here,
and it helps that we've gone over the radiometric
and geometric correction processes.
Because, for example, one of the Landsat collections
that you might come across is called raw images.
So, raw images basically means
that it has not been
radiometrically, or perhaps even geometrically corrected.
And then, you have top-of-atmosphere collections,
which, as we explained, represents the reflectance values
at the top-of-atmosphere.
And so, those might include a bit of haze,
and they might be a little bit more washed out
than the Earth was at that time.
And then, finally, surface reflectance collections;
that is after it has been
completely radiometrically corrected,
and that represents the reflectance values of the surface.
And then, the Landsat collections,
they also have tiers associated with them.
So, they have Tier 1, Tier 2,
and then you might also see real-time.
So, real-time means that that data was just brought in,
and they haven't checked it yet for potential errors.
Tier 1 is the best quality, and Tier 2 means
that there are some errors within that data
that meant that that data didn't meet
the accuracy standards for Tier 1.
So, as you can see, there's sort of, as you go to the left
of the screen, an increase in image quality.
So, when you're looking at Landsat collections to use,
definitely look for Tier 1,
and I'd recommend using the surface reflectance collections.
The top-of-atmosphere can also work,
but you have to be aware of haze.
We'll now talk about image enhancement.
Image enhancement is concerned with the modification
of images to make them more suited
to the vision capabilities of the human eye.
This is important because visual analysis
plays a really strong role in all aspects of remote sensing,
meaning that we judge and interpret
and even classify images just by looking at them.
Image enhancement techniques
can take a lot of different shapes depending on your image
and your question.
I'm gonna concentrate today on talking
about contrast stretching, which is a technique used
to increase the visual distinction
between features in an image.
Contrast stretching is important
because, even though digital sensors have a wide range
of reflectance values that they can measure,
some environments have a really narrow range of values
that they reflect, and therefore images
of those environments could appear
quite dark or washed out.
Through contrast stretching,
we can stretch the reflectance values in those images
and hopefully see more detail.
The figure in the bottom left of the screen
shows a Landsat image displaying the visible red band
in grayscale, with its related histogram below it.
Note that this image is really dark,
and the reflectance values in the histogram
are quite skewed to the left.
The right panel of the figure shows the same image
after a linear stretch has been applied.
And you can see that there's a lot more detail
in the landscape that you can see now.
So, why do we care about histograms in remote sensing?
Histograms provide a lot of information on images
without having to even look at the images themselves.
For example, you can pick out the presence
or absence of features
and the distribution of those features.
Some histogram shapes for image
include normal, skewed, bimodal, uniform.
You can see those shapes
in the figure at the right of the screen.
And within these different histogram distributions,
you can gather information about the images themselves.
For example, you can tell a little bit
about what's going on.
For instance, an image that has lots of water
and lots of clouds might have a bimodal distribution,
because water has low reflectance values,
and clouds have high reflectance values.
Histograms can also be helpful in image classification
and image segmentation,
which we'll cover later in the course.
Here are a few more examples of images
and their associated histograms.
And as you can see, the shapes of the histograms
reflect what's going on in the image.
For example, normally distributed histograms
mean that the histogram is balanced,
or that there is approximately similar numbers
of low reflectance values
and high reflectance values on either side
of the histogram's peak.
A normally distributed histogram
that has a wide range of reflectance values,
like the histogram on the far left of this figure,
means that the image has good contrast,
and likely has a good visual distinction between features.
If the histogram is normal
but narrow, this may mean that the image has low contrast.
So, again, you can see in this, the image associated
with a narrow histogram here is quite dark,
and you can't see differences
between features on the landscape very well.
You can also see a skewed histogram here,
meaning that the histogram isn't balanced,
or the peak isn't in the middle of the distribution.
For example, this histogram is negatively skewed,
meaning that, on the left side of the peak of the histogram,
there's more reflectance values than there is
on the right side of the peak of the histogram.
And then, finally, we can see a bimodal histogram here
and its associated image.
In this case, this bimodal histogram is made up
of a normal distribution with an additional peak of values
in the low reflectance range.
So, the right peak, the normally distributed histogram
in this image, represents
the reflectance values of the land.
And then, that left peak of values
represents the reflectance values of the water.
So, if you didn't see the images here
and you only saw the histograms,
you can see that you can kind of tell what's going on
in the images, or at least you could make a guess.
The last image processing method
that we'll talk about today is image compositing.
We can talk about composites in a couple
of different ways in remote sensing,
but for right now, let's just stick to the generation
of composites for visual analysis.
We already talked last week about band compositing,
or selecting three bands in an image
to make a color image.
This, week we're gonna talk about the application
of image compositing to fill gaps in your image.
So, first off, there's a variety of reasons
that there may be gaps in your image.
I've shown some of these reasons
at the right of this screen.
For example, line drops, bit errors, and striping
are all radiometric errors that can occur
due to transmission errors
or calibration issues within your sensor.
Atmospheric interferences like clouds or smoke
from volcanic eruptions are also reasons
that you may wanna make an image composite,
'cause they obstruct your view of the Earth's surface.
All right, in order to correct these images
or to make images that don't have errors,
we typically start off by making a temporal composite,
which just means that you have an image
composed of pixels from two or more time periods.
One super way that you can make a temporal composite
is by taking one original image that has errors,
identifying those problem pixels,
and then replacing the pixels with pixels from another image
that is close in time to the original image.
Alternatively, we could build a new error-free image
by making a stack of images of the same location
through time, and then using pixel resampling methods.
Again, there's a lot of different pixel resampling methods
that we can choose from.
I'll just talk about three different ways.
So, first, we can average all the pixel values
of a certain confidence interval through time.
Second, we could take the median pixel value
of the image stack, and that usually gets rid of clouds,
because clouds have really high reflectance values.
Or another way is that we could write some fancy code
to identify and exclude problem areas in the image,
and then only include pixels that meet certain thresholds.
So, I've had to use this technique before
when studying areas in Latin America
that have cloud cover for over half of the time.
I wanted to tell you about a famous line-drop issue
with one of the Landsat satellites.
So, on May 31st, 2003, the scan line corrector
of Landsat 7 failed.
And ever since then, the outputs from Landsat 7
have had line drops, and the images appear stripey,
as you can see.
So, what the scan line corrector does is that it compensates
for the forward movement of the satellite.
In other words, as the satellite moves,
it does so with a slight zigzag motion
due to its momentum in space.
And the scan line corrector is supposed
to keep the image lines straight.
But because it broke on Landsat 7,
the image lines are now zigzaggy,
and they lose a lot of data because of this.
In fact, around 22% of the data is corrupted
from this satellite.
So, if you use images from Landsat 7
in Google Earth Engine,
you'll likely see this striping in the image,
and now you know why.
The images from Landsat 7 aren't totally unusable, though;
we just have to use compositing to fill the gaps.
Finally, I'd like to spend a minute talking about masking
and how it relates to image compositing.
Masking means that you exclude pixels from your analysis,
or make them transparent.
And again, things that we might want to mask
include clouds or smoke, cloud shadows,
shadows from mountains or buildings.
You may also wanna mask out features
like snow or ice in your image.
And in image compositing,
masking is important because it removes those pixels
and allows that space to be filled
with pixels from another image,
or allows you to not do analyses over those pixels.
So, again, masking can be important
when you're analyzing images.
So, for example, if you wanna know
the mean reflectance value of vegetation in a park,
you may wanna first mask out any streams
or water features in that park so that they don't contribute
to your answer and skew your data.
A lot of the images that we use in Google Earth Image
already have things like clouds masked out of them for us.
But if you're working with raw imagery,
you'll have to apply a mask yourself.
This process is actually pretty simple
to do in Google Earth Image,
and you'll learn how to do it in the coming weeks.