- [Amanda] Hi, everyone, welcome to week four
of Remote Sensing Foundations.
This week, we will learn about image processing,
filtering, masking, and classification,
all using Google Earth Engine.
At the end of this week,
I hope you will have accomplished the following.
First, I hope you'll understand
how and why images are processed.
Second, understand the difference between a composited
and a raw image,
then select and filter an image,
apply an image mask to remove water,
understand different classification methods
and when to apply each.
And finally, you will classify an image
and calculate the pixel area for each class.
Readings this week and the included tutorials
will help guide you through this list,
which I know is a tall order.
Let's get started.
Last week, we went into some detail
about how we take satellite measurements
and organize them into the imagery
that we know of as satellite imagery.
We learned that there are several types
of geospatial data sets, but in remote sensing,
roster data sets are the most common format.
Rosters organize information into cells or pixels
where rows and columns of pixels form a grid.
Rosters are great for showing
continually varying information.
The size of pixels in a roster determines
its spatial resolution.
Roster images can contain one or more bands,
each covering the same spatial area,
but containing different spectral information.
When roster data contains bands
from different parts of the electromagnetic spectrum,
they are called multi-spectral images.
We can visualize the multi-spectral image
using any three bands of that image
and displaying them in the red, green, and blue colors.
Single bands can also be visualized in gray scale.
Roster data sets of satellite images are typically stored
in a GeoTIFF, or a stack of roster data sets,
which can be organized into many types of maps,
including surface maps, thematic maps, and attribute maps.
We also discussed in further detail
about what determines spatial resolution
and the trade-off between grain and extent.
The spatial grain of the landscape
is the smallest resolution of data.
Extent is the overall size of the landscape or area
encompassed by the observation.
Finally, we gave an example
of how to display multi-band images
and the difference between true color
and false color image composites.
Okay, so let's dive right in here
to talk about image processing and what that entails.
Digital image processing are the steps that we take
to turn satellite data into images.
We can break them up into four basic operations,
including image restoration, image enhancement,
image classification, and image transformation.
Broadly speaking, image restoration is concerned
with the correction and calibration of images
in order to achieve as faithful a representation
of the Earth's surface as possible,
which is a fundamental goal in remote sensing.
Image enhancement, on the other hand,
is predominantly concerned with the modification of images
to optimize their appearance to the visual system.
So this is really important in terms of making maps.
Visual analysis is a key element,
even in digital image processing,
and the effects of these techniques can be dramatic.
Image transformation, on the other hand,
refers to the derivation of new imagery
as a result of mathematical treatment
of the raw image bands.
And finally, image classification refers
to the computer-assisted interpretation of images.
Today, we will touch on all four types of image processing
and go into some greater detail
about image classification methods.
Okay, so image restoration is often defined
by operations that aim to correct distorted
or degraded image data
to create a more faithful representation
of the original scene.
It's often also termed image pre-processing
because these operations or methods are normally done
before doing any sort of further manipulation
or analysis of the image data
in order to extract information.
And usually they involve correcting the distortions
and degradations that are caused by several factors
during the actual acquisition of the image data,
so during the image acquisition data process.
So there are two main types of image restoration,
radiometric and geometric.
Radiometric restoration, which the schematic is showing
an example of here on the left,
is basically defined as the removal or diminishment
of distortions in the degree of electromagnetic energy
registered by each detector.
So many of these include things that happen
to the sensors themselves
or in the communication of the sensor data
back to the computer on the ground.
There are a variety of things that can cause distortions
in the values recorded for image cells or pixels.
Some of the most common distortions
for which correction procedures exist include things
like what's called uniformly elevated values,
which are due to things like atmospheric haze
and which basically scatter short wavelength bands,
particularly in the blue regions.
So if you're looking at an image that has atmospheric haze,
if you're looking at the raw image of that,
that image is going to sort of uniformly appear
to be more washed out
and not have a high degree of contrast.
A second correction procedure is called striping.
Striping is due to when detectors go out of calibration.
This happened to one of the Landsat sensors
about a decade ago.
A third way that distortions can happen
is with random noise.
So this is due to unpredictable or unsystematic performances
of the sensor itself,
or can be also in the transmission of the data.
And finally we have what's called scan line dropout,
and that is due to signal loss from specific detectors,
which is a sometimes common occurrence
depending on your orbit path
and kind of what's going on with that particular satellite
and where it's located.
So the other type of restoration that I want to talk about
is called geometric restoration.
And by the way, I forgot to put up the examples there.
So those are the four radiometric restorations
there at the bottom, uniformly elevated values,
striping, random noise, and scan line dropout.
Okay, so back to geometric restoration.
So for mapping purposes,
it's essential for any form of remotely sensed imagery
to be accurately registered, sorry, to the map base,
or basically the points on the ground.
So with satellite imagery,
the very high altitude of the sensing platform
results in minimal image displacements
due to topography or relief.
This can definitely be more of an issue
when you're talking about airborne sensors
mounted on aircraft or drones.
As a result, registration of the image
can usually be achieved through a use
of a systematic transformation process
that generally warps the entire image
or changes the entire image
through the use of different types of equations
or other methodologies,
which I'll go over a little bit more of in the next slide.
And these are based on the known positions of a set
of a wide variety of control points.
With airborne sensor data, however,
the process can be also pretty complex.
So not only are there systematic distortions related
to tilt and varying altitude,
which is shown here in the figure at the bottom left,
but also variable topography can lead
to a very irregular pattern of distortions
that can be a lot harder to remove.
So you can think of it a little along the lines
of a 2D versus a 3D problem.
However, even the 2D problem is complex.
With satellites, though we may not have to interact
with topography as much,
we instead have to consider a lot of other details
like the curvature of the earth, the rotation of the earth,
the angle of the sensor or sensors
in relation to those things,
as well as the sensor orbit path.
So needless to say, with respect to translating raw data
from a sensor to an image,
we need to know a lot of both physics and geometry.
Okay, so with both of these types of restoration,
it's important to mention here
about the procedures that we use
to convert these raw, unitless relative reflectance values.
Remember, these are called digital numbers or DNs,
into the true measures of reflective power or radiance
that we know of when we look at imagery.
So let's talk a little about the methods we typically use
to correct images and why correction is often needed.
First, why is it needed?
Well, as we saw from the last slide,
raw digital images usually contain significant radiometric
and geometric distortions,
and these distortions make the raw images unusable,
meaning they cannot be directly applied
as a map base without subsequent processing.
By applying geometric correction procedures,
the distortions introduced by these factors
are compensated for so that the corrected image
will have the highest practical geometric
and radiometric accuracy and integrity,
meaning they alter the image
to make it more approximate what we see on the ground.
At the top of the slide,
I've put a list of why we use correction processes,
many of which I also mentioned on the previous slide.
We apply corrections because of variations in altitude,
attitude, and velocity of the sensor platform,
the Earth's curvature, the Earth's eastward rotation,
atmospheric refraction, and relief displacement.
One way to begin the correction process
is to use ground control points or GCPs.
I've also heard of them referred to as tie points.
GCPs are features of known ground location
that can be accurately located in an image.
Numerous GCPs are located both in terms
of their two image coordinates,
the column and row numbers on the distorted image,
and in terms of their ground coordinates.
Using these GCPs
with what are called transformation equations,
we can correct the map.
One important set of methods use types
of transformation equations to correct the image.
Resampling is the most common
and basically involves generating equations
like the example on the slide
that equate the GCPs on the distorted image
with where they are on the corrected image,
and then applying those same equations
to the remaining pixels to correct the rest of the image.
Once the coefficients for these equations are determined,
the distorted image coordinates
for any map position can be precisely estimated.
Okay, so in the resampling method of correction,
the coordinates of each element
in the undistorted output matrix are transformed
to determine their corresponding location
in the original input or distorted image matrix.
The intensity value or digital number assigned to a cell
in the output matrix is determined based
on the basis of the pixel values
that surround its transformed position
in the original input matrix.
Three types of resampling methods are nearest neighbor
and bilinear and bicubic interpolation.
These are good methods to know and understand
for other types of processing
and data manipulation in remote sensing,
which we'll cover later in this course, as well.
The nearest neighbor, or in the nearest neighbor,
the DN of the transformed or corrected pixel is equal
to the DN of its closest original or distorted pixel.
Advantages of the nearest neighbor method
are that it's really relatively simple to implement,
and it does not involve altering the original pixel values.
One disadvantage, however,
is that the features in the output image
may be offset spatially by up to 1/2 pixel,
which can cause disjointed or blocky appearance
in the output image.
Bilinear interpolation is where the DN
of the transformed or corrected pixel
is equal to the distance weighted average
of the four nearest pixels,
which gives a smoother image appearance
than nearest neighbor, but alters the original DN values.
And finally, bicubic interpolation or cubic convolution
evaluates a block of 16 pixels in the original image
that surrounds the output pixel
to determine the DN of the transformed or corrected pixel.
This also leads to images that are smoother in appearance
than nearest neighbor,
and even probably a little bit more smoother
than bilinear interpolation.
And there's an added benefit
of making the image slightly sharper
than that of interpolation.
But again, the original DN values are altered
from the original image.
So there are trade offs
in which of these methods you decide to use,
between nearest neighbor, bilinear interpolation,
and bicubic interpolation.
And when you're using Google Earth Engine,
oftentimes when you are choosing how to display a map
or how to output a map
that you're interested in working with in another program,
you're going to be having to choose
between nearest neighbor and a couple of other options
for resampling, so that's just good to know.
Image enhancement is concerned with the modification
of images to make them more suited
to the vision capabilities of the human eye.
Regardless of the extent of digital intervention,
visual analysis invariably plays a very strong role
in all aspects of remote sensing.
Enhancement is the process of manipulating an image
so that the result is more suitable than the original
for a specific application.
It should also be noted, though,
that enhancement techniques are very problem specific.
And while the range
of image enhancement techniques is broad,
here I will concentrate on two types,
contrast stretching and composite generation.
Digital sensors have a wide range of output values
to accommodate the strongly varying reflectance values
that can be found in different environments.
However, in any single environment,
it is often the case that only a narrow range of values
will occur over most areas.
Contrast manipulation procedures are thus essential
to most visual analyses.
The figure at the bottom left shows a Landsat TM image
displaying the visible red and gray scale
and its related histogram.
Note that the values of the image are quite skewed
to the left, and the image appears very dark.
The right image of that figure shows the same image band
after a linear stretch has been applied
and its associated histogram.
So by the way,
why do we care about histograms in remote sensing?
Histograms can provide a lot of information on images
even without looking at the images themselves,
such as things like the presence or absence of features,
the distribution of these features.
Histograms also help evaluate images statistically.
So you can look at a histogram, and depending on its shape,
you can evaluate it statistically.
So shapes can be things like normal, skewed, bimodal.
Those are types of distributions,
and I'm showing a number of other ones
at the figure at the right.
And within these different histogram distributions,
you can also gather statistics based
on the shapes of the histograms.
And that can tell you a lot
about what's going on in the image, as well.
So histograms are also used
in individual image enhancements,
like the one that I'm showing at the left.
And they're used in image classification,
which we'll go over more in a bit here.
And they're also used in image segmentation
as well as in helping to match images
across space or through time.
So when you're looking at a time series of different images,
oftentimes the histogram
or the statistics that we evaluate within these histograms
can tell us a lot about what's going on
in the landscape through time.
Okay, so here are a few more examples of images
and their associated histograms.
And as you can see, the shapes of the histogram, again,
can kind of tell us what's going on in the image.
So really tightly or narrow histogram like shown
at the one second from the left
is going to have very low contrast,
and at the low end of the reflectance spectrum
is gonna be very dark,
as you can see in that image.
Something with a normal histogram,
like the one at the far left,
has a better degree of contrast depending
on how thick the histogram is
or how many reflectance bands it covers
or reflectance values it covers.
The one that's being shown here has a pretty good balance.
You can also note two other shapes of histogram,
skewed and bimodal.
Those are very common shapes for histograms, as well.
And for the bimodal one,
the low reflections values there, that first peak,
is indicative of what you see in the image
as basically a whole bunch of dark area,
which in this case is water.
And then with the skewed image,
sometimes a landscape that has a high degree
of human disturbance or human development
can have an impact on the shape of the histogram.
This one is called a skewed histogram,
and as you can see, it's got a very pointy peak.
And that peak is not straight in the middle
as with the normal distribution histogram.
Okay, so the second enhancement technique
that we wanna talk about here is composite generation.
And we can talk about composites
in a couple of different ways in remote sensing.
But for right now, let's stick to generation
of composites for visual analysis.
So for visual analysis, color composites make use
of the fullest capabilities of the human eye.
Depending upon the graphic systems in use,
composite generation ranges
from simply selecting the bands to use
to more involved procedures of band combinations
and associated contrast-stretching techniques.
The figure at the left shows how a three-band composite
is constructed from a multi-band image.
This is the most basic way that we construct composites.
And this is what we do,
what I showed you last week in Google Earth Engine.
When you display a multi-band image
on the Google Earth Engine map,
and you open up that layer dropdown box,
and then you go into the popup associated with that map,
you can pick and choose which bands
you want to visualize in the map.
And then the figures down at the bottom
show several composite examples made
with different band combinations
from the same set of TM images.
So based on what you're interested in looking at,
it's a good idea to go through and kind of play around
with how you visualize your bands
and what order you might visualize them in.
What is an image composite, and why do we make one?
Well, we make composites for a variety of reasons,
such as the examples I'm showing at the right.
Three types caused by center failures
include stripes or banding,
which have the appearance of defective lines
and are due to the variations in the response
of individual detectors,
which result in relatively higher or lower values
along every sixth line,
as is what happened with Landsat's MSS product.
Line drops, a second way,
which is when a number of adjacent pixels along the line
may contain spurious DNs
and is usually caused by data transmission errors.
Or bit errors or shot noise or random noise as a third way,
which can occur in an image
and which can be spiky in character
in terms of the associated histograms
and causes images to have a salt and pepper
or snowy appearance.
We also make image composites for atmospheric interferences,
such as clouds or smoke from fires or volcanic eruptions.
Temporal composites, on the other hand,
involve stacking a bunch of images
of the same area through time
and using pixel resampling methods
to build an uncorrupted image.
There are, again, numerous methods that we can use
to build the composite from the stack of images.
Three examples include, the first way,
we can average all the pixel values
of a certain confidence interval
for a given pixel location through time.
Second, we can take the median
or maybe the mode pixel value.
Or third, we can apply another statistic or sampling method
if, say, all of the pixels in a given pixel location
are clouded for the temporal composite time period,
as can happen over tropical forests sometimes.
The way that we build composites in Google Earth Engine
is to typically create an image collection
by filtering a satellite product collection
to the desired time range.
We then write a function to build the image composite.
We will explore how to do this
in Google Earth Engine more next week.
Let's talk now about image transformation.
Digital image processing offers a limitless range
of possible transformations on remotely sensed data.
Some of these include things like color,
space transformations, texture calculations,
thermal transformations,
and a wide variety of other ad hoc transformations
such as image ratioing
that can all be most effectively accomplished for images.
Because of their spectral significance
in environmental monitoring and applications, however,
for the purposes of this course,
the two that we'll come to know most and focus on most
are vegetation indices and principal components analysis.
There are a wide variety of vegetation indices
that have been developed to help
in the monitoring of vegetation.
Most are based on the way very different interactions
between vegetation and electromagnetic energy
in the red and near infrared wavelengths occur.
In the figure at the top left,
we see a generalized spectral response pattern
for green broad-leaf vegetation.
From the graph, reflectance in the red region
of about 0.6 to 0.7 is low
because of absorption by leaf pigments,
principally chlorophyll.
The infrared region,
which is about from 0.8 to 0.9, however,
characteristically shows high reflectance
because of scattering by the cell structure of the leaves.
A very simple vegetation index
can thus be achieved by comparing the measure
of infrared reflectance to that of red reflectance.
Although a number of variants
of this basic logic have been developed,
the one which has received most attention
is the normalized difference vegetation index, or NDVI.
So it's calculated basically by subtracting the red
from the near infrared
and dividing it by the near infrared plus the red band.
So the image on the bottom left
displays a Landsat 8 surface reflectance on the left panel.
And on the right,
the surface reflectance derived
normalized difference vegetation index for that same image.
As you can see, different vegetation areas
and different vegetation types
have different spectral signatures
that are being shown in green,
with high NDVI values being darker green,
and low NDVI values being blue or whitish.
On the right, I'm showing another type
of image transformation technique,
which is using a spatial principal components analysis
to look at the spatial variation of pixel values
across the North American Tundra Biome.
This map shows an RGB image of PCAs one, two, and three,
where red shows areas where the majority of variance
is explained in PCA one.
Green is PCA two, and blue is PCA three.
Pinks, purples, and browns are areas
where more than one PCA explains the variance.
And this image that I'm showing
was actually calculated by me
using a MODIS composite of images,
and it was done in Google Earth Engine, also.
Finally, I'd like to spend a few minutes talking
about image classification.
Image classification refers
to the computer-assisted interpretation
of remotely sensed images.
Though we will go over methods
for accomplishing image classifications
using Google Earth Engine in more detail
in the coming weeks,
today, I will provide you with a brief overview
of different types of classification methods.
Although some procedures are able to incorporate information
about such characteristics as texture and context,
the majority of image classification is based solely
on the detection of the spectral signatures
of land cover classes.
The success with which this can be done
will depend on two things.
First, the presence of distinctive signatures
for the land cover classes of interest
in the band set being used,
and second, the ability to reliably distinguish
these signatures from other spectral response patterns
that may be present, as well.
There are two general approaches to image classification,
supervised and unsupervised.
They differ in how the classification is performed.
In the case of supervised classification,
Google Earth Engine, or the program being used,
delineates specific land cover types based
on statistical characterization data drawn
from known examples in the image.
With unsupervised classification, however,
instead, clustering algorithms are used
to uncover the commonly occurring land cover types
with the analyst providing interpretations
of those land cover types at a later stage.
For a supervised classification,
like what I'm showing here on the left panel,
the first step is to identify examples
of the information classes
or the land cover types of interest in the image.
These are called training sites.
Embedded applications in Google Earth Engine
will then develop a statistical characterization
of the reflectances for each information class.
This stage is often called signature analysis
and may involve developing a characterization
as simple as the mean
or the range of reflectances on each band,
or as complex as detailed analyses of the mean variances
and covariances over all the bands.
Once a statistical characterization
has been achieved for each information class,
the image is then classified
by examining the reflectances for each pixel
and making a decision about which of the signatures
it resembles most.
There are several techniques
for making these decisions, called classifiers,
which have three types, hard, soft, and hyperspectral.
With hard classifiers,
the distinguishing characteristic
is that they all make a definitive decision
about the land cover class to which any pixel belongs,
using a classifier method like parallelepiped,
minimum distance to means, or maximum likelihood,
the latter of which is the most popular.
Contrary to hard classifiers,
soft classifiers do not make a definitive decision
about the land cover class to which each pixel belongs.
Rather, they develop statements of the degree
to which each pixel belongs
to each of the land cover classes being considered.
Thus, for example, a soft classifier might indicate
that a pixel has a 72% probability of being forest,
a 24% probability of being pasture,
and a 4% probability of being bare ground.
A hard classifier would resolve this uncertainty
by concluding that the pixel was forest.
However, a soft classifier makes this uncertainty
explicitly available for any of a variety of reasons.
For example, the analyst might conclude
that the uncertainty arises because the pixel contains
more than one cover type
and could use the probabilities as indications
of the relative proportion of each.
This is known as subpixel classification.
Alternatively, the analyst may conclude
that the uncertainty arises
because of unrepresented training site data,
and therefore may wish to combine these probabilities
with other evidence before hardening the decision
to a final conclusion.
And finally, there are hyperspectral classifiers.
All of the classifiers mentioned above operate
on multi-spectral imagery, as we know,
images where several spectral bands
have been captured simultaneously
as independently accessible image components.
And extending this logic to many bands
produces what has come to be known as hyperspectral imagery.
Although there is essentially no difference
between hyperspectral and multi-spectral imagery,
i.e., they differ only in degree,
the volume of data and high spectral resolution
of hyperspectral images does lead to differences in the way
that we must go about handling classification methodologies.
So on the other hand, and shown in the panel on the right,
in contrast to supervised classification,
where we tell the system about the character,
i.e. the spectral signature of the information classes
we are looking for,
unsupervised classification requires no advanced information
about the classes of interest.
It examines the data and breaks it into the most prevalent,
natural spectral groupings or clusters present in the data.
The analyst then identifies these clusters
as land cover classes through combination
of familiarity with the region and ground truth visits.
The mathematical logic
by which unsupervised classification works
is known as cluster analysis.
And in Google Earth Engine,
there are a number of different types of cluster methods
that can be tried to investigate
which gives the best outcome for a given site.
It is important to recognize, however,
that the clusters unsupervised classification produces
are not information classes, but spectral classes,
meaning they group together features
with similar reflectance patterns.
It's usually the case that the analyst needs
to reclassify spectral classes into information classes.
For example, the system might identify classes differently
for asphalt and cement.
But the analyst might want to later group those together,
creating an information class called pavement.
So finally, I'd like to spend a minute here
mentioning masking because masking pixels
is also an important component of processing
or pre-processing prior to interpretation
of an image in remote sensing.
So masking pixels basically renders those pixels transparent
and excludes them from an analysis.
A lot of the images that we will use in Google Earth Engine
have already had things like clouds
masked out of them for us.
However, if you were working with raw imagery,
you would want to apply a cloud mask to that image
to remove the clouds.
Other things you might wanna remove would be shadows
from things like mountains or buildings
or other topographical features.
And then also we typically, for terrestrial image analysis,
we wanna remove water.
And that water can be in the form
of rivers and lakes and oceans.
It can also be in the form of snow.
But typically, if we are trying to do some sort
of spectral analysis,
it's important to remove those pixels from our analysis.
And the way that we do that is either
by using somebody else's mask,
or we can also create a classified image
where we classify water
to then essentially remove it eventually.
So we'll show you how to do all that
in Google Earth Engine in the future.
But because I include masking in the list of things
that we do to pre-process or process an image
prior to interpretation, I wanted to mention it here today.
Okay, so we went over a lot today.
We talked a lot about image processing,
and four areas that we focused on
were restoration, enhancement,
classification, and transformation.
And we talked about subcategories,
including radiometric versus geometric,
contrast stretching, composite generation,
supervised versus unsupervised,
and then also vegetation indices,
principal components analysis, and masking.
And though some of these things are accomplished for us
when we're using Google Earth Engine,
I think it's important to know a lot about
what is kind of going on behind the scenes
before we get to that processed image
to use for our interpretation.
Some of the divisions here we do ourselves.
And remember that with image enhancement, a lot of times,
the focus is on rendering an image more visible
for us to be more visibly able to kind of analyze it
as opposed to some of the other ones,
which actually involve using
different mathematical algorithms
to actually translate or transfer the spectral signatures
of each pixels.
And I put this schematic here at the right
just as kind of an example to show you
that there are a lot of different processes
that come behind any sort of model creation
for an image analysis or an interpretation analysis
in remote sensing.
This was just a figure from a scientific article
that was talking about using object
and pixel-based approaches to classifiers
and using them in wetland, to create wetland maps.
So as you can see, a lot of different steps and processes
go into creating these maps and doing this analysis.
And some of the things that we have listed here at the left
are involved in the process stream over there at the right.
Okay, so in terms of the readings this week
and how they apply to what we've talked about today,
you're gonna be doing two readings.
And within those readings,
there are some tutorials to follow
that are going to help you do things
like explore images, filter an image collection,
explore temporal features
and examine pixel quality and image quality,
all using metadata.
And so please let us know if you have any questions,
and I look forward to seeing you all virtually next week.
Take care.