- [Brian] In part two of our lecture,
I'll demonstrate some of the lecture concepts
that we saw in the last slide deck.
We'll look at both vector
and raster geo processing operations.
On the vector side,
we'll use some parcel data to explore the union, merge
and append geo processing tools.
On the raster side,
we'll work with some digital elevation models
to create a mosaic and add rasters.
We'll also explore the append tool for raster data as well.
If we switch gears here and look at the ArcGIS Interface,
we see we've got two parcel datasets,
Waitsfield Parcels Standardized shown in light green,
and Warren Fayston Parcels shown in a darker green.
Remember the end goal here is to produce
a single unified parcel dataset
that we could use for our analysis.
Let's first look at the attribute tables
of the two different datasets we have,
Our Waitsfield Parcels Standardized dataset
contains a number of different attributes
related to the grand list,
Vermont's tax rolls collected at the town level.
If we look at our Warren Fayston Parcels attributes
we see a significantly shorter list.
Looks like just the key items out of that grand list.
Now this won't affect our union operation,
but think about how choosing
the appropriate right first input dataset
might affect the merge and append operations.
And we'll take a look at those both in a minute.
Let's start with the union.
Again, think about the fact that we want to combine
our parcels data into a single cohesive parcel dataset
where all the attributes match up.
And there are no repeat attributes represented in the table.
If I union my two datasets,
we can look at the output.
We've got a dataset now called parcels union,
and if we look at that attribute table, what do we see?
Well, we've got the FID from the Warren Fayston Parcels,
and anywhere that value is greater than zero
indicates that it comes from the original dataset,
that Warren Fayston Parcels,
and anything with a negative value indicates
that that comes from
the Waitsfield Parcel Standardized dataset.
If I continue to scroll over,
we see now that we've got quite a few attributes.
And in fact, we see if we were to interrogate slowly,
we could see the repeat values in the attribute list.
Let's take another look from the field interface
to see if it's a bit easier to see what I'm talking about.
If we scroll down,
we see we've got all these value related-attributes:
Real, Hstead, NRES, Land, Improvements.
And if I continue to scroll down here through the list,
I see I've also got these values,
but they all have underscore one.
This is exactly why the union tool does not work
for assembling these datasets.
We were not able to do any field mapping.
So all of the attributes from the Waitsfield dataset
are written to the end of that attribute table.
In other words, additional attributes were created
in the output dataset.
This is pretty inefficient
and won't really work for what we're trying to do.
Let's take a look at another possibility here,
and we'll use the merge command this time.
So remember how I mentioned previously
that there were different attributes
in each one of the datasets.
Remember also that it really matters
which dataset we select first
because with the merge tool, as we know,
we're going to write out a new feature class,
and that new feature class,
the structure of that attribute table
will be based on the information
that's drawn from the first of your input datasets.
Well, I like the format of the Warren Fayston Parcels data.
So I'm gonna enter that one first.
And now we see all the attributes
in the output fields list here
in the lower left portion of the merge dialog box.
Now, if I add the Waitsfield Parcel Standardized,
I can scroll down the list,
and I see a bunch of twos next to all of my categories.
That's a good sign.
That means that those fields have mapped from both datasets
onto that single Warren Fayston dataset.
If I scroll down further, I can also see
a list of other attributes with no numbers next to it.
All of these came from the second dataset that I entered
into my input list, the Waitsfield Parcels.
We can just get rid of those,
so they don't get written to the output.
If I click on the first one
and click on the last one that I want to get rid of,
click that red X,
and those will not be in the output dataset.
Remember, I didn't delete anything from a dataset.
I just told ArcGIS not to include those attributes
in my output dataset.
Let's call this one parcels merge.
Click run and evaluate our result.
Well, that doesn't look exactly like I was hoping.
It appears that none of the Waitsfield parcels
were written to the output dataset.
Let's investigate to see what we can figure out here.
I received a warning that indicates
that ArcGIS is not able to write the Waitsfield value
to the town attribute.
Let's check out why that might be.
If I click on the town attribute
and the properties of that particular attribute,
I notice that it's a text field, which is good,
but it's only got a length of seven.
Try it in your head to spell out Waitsfield
or count the number of letters, and you'll notice
that it's definitely more than seven.
It's actually 10.
That wasn't a problem for either Fayston or Warren,
but for Waitsfield, the length limit on the text field
means that the whole name of the town can't be written.
ArcGIS gets confused, and as a result,
none of those records were added.
Let's try this again, but this time,
we'll set our length to 10.
Click run.
So far so good with no warnings.
And look here, we've produced a new dataset
with all the parcels for the matter of Valley towns.
If we open the attribute table, we can scroll through
and see that short list of attributes that was included
in that original Warren Fayston Parcels dataset.
So one last thing to check here.
We've got 5,474 records in our parcels merged dataset.
Let's just do a quick little bit of math
to see that we've got 4,471 parcels
in our Warren Fayston parcels dataset
and 1,003 parcels in our Waitsfield parcels dataset.
So if we add those two together,
we get that 5,474 records in the dataset.
We can interpret that to mean
that all of the individual features
of the two datasets were successfully added
to our new output dataset,
the parcels merged that we just created.
Okay, let's look at one more example here
before we move on to our vector analysis.
In this last example, we'll look at the difference
between the merge and the append tool.
So we just ran the merge
and successfully created a unified parcel dataset.
Remember we had two input datasets
and used the merge tool to produce a new output dataset.
What about the append tool?
Locate that in my geo processing pane,
and now I need to add my input datasets here.
Just a couple quick notes before we proceed
with the append tool.
Note that this tool will modify the input data.
We've also got two required parameters,
an input dataset and a target dataset.
Our input dataset contains the data to be appended
to the target dataset.
So that means that in this case,
our Waitsfield parcel standardized
would be our input dataset and that target dataset,
the dataset where the data of the inputs will be appended
is going to be our Warren Fayston Parcels.
With that, I can click run.
Notice that that failed almost immediately.
If I look at the details here,
it indicates that the parcels,
the schema for the attribute tables do not match
between the two datasets,
but that's what I specified for my field matching type.
Let's change that to use the field map
to reconcile my differences.
That takes us back into our familiar field map list,
Where we can see all of the different attributes
from before.
Click run again,
and this time the tool runs to completion
but with some errors.
It looks like it's that same issue that we had before
where we can't write the town output,
town name into the output table.
Let's try to fix that one more time
Remembering that we needed 10 characters, not just seven.
And click run one more time.
It looks like we're still suffering from that same problem.
Surprising that the append operation,
let me change the length of the field,
but it does not appear that that actually
that change actually took hold in the dataset.
Okay, we got one more thing to try.
It's not quite time to give up just yet.
Let's look at our Warren Fayston Parcels dataset,
and check out that attribute table.
If I go into my fields view,
I can see that my town is a text data type
with a length of seven.
Let's change that length here and save those changes.
This doesn't do anything to the data
other than make the container,
that text container for the town attribute
a little bit bigger.
Okay, let's try that merge just one more time
or the append just one more time.
I've made changes to the data.
So it's always good to back
all the way out of the tool and reset your parameters.
Once again, I've got my Waitsfield parcels for my input data
and my Warren Fayston parcels for my target data.
Let's use that field map to reconcile any differences,
and now when I scroll down to town
and switch over to the properties tab,
I see it's got that length of 10.
It seems like things are looking up.
I'll click run, and it looks like my append is completed.
If I open the attribute table for my Warren Fayston Parcels,
I see I've got 5,474 records.
That's the same number that we achieved
when we used the merge tool.
Now takeaway message here,
remember that union, not the right tool for the job.
If you use the merge tool,
that's going to produce a new output feature class.
And if you use the append tool,
that's going to modify one of the input datasets
that you supply to the append tool itself.
So that's it for the vector side of things.
Let's switch over to raster
and take a look there real quick.
To explore the raster example,
I downloaded some data from the USGS national map
one meter digital elevation model.
We can see the two different slices
of the data on the screen here.
So, what are we gonna do here?
We want to put these two pieces of the DEM
into a single piece that we could use in our analysis.
The equivalent of the merge on the vector side
is what's known as creating a Mosaic dataset.
So I need to specify an output location,
and I'll write that information
to my geo database for the project.
I need to give my Mosaic dataset a name.
And lastly I need to give my Mosaic dataset
a coordinate system.
I'm going to use the input coordinate system
of one of my datasets, which I see is the UTM Zone 18 North.
That's it for creating the Mosaic dataset.
Just click run, and ArcGIS will add that
to the database and to the project.
Note, we've got the boundary,
the outer bounds of my Mosaic, the footprints,
the boundary of the individual tiles
that go into the Mosaic, and then the image itself.
Right now the image doesn't show us anything
because we haven't added any data to the Mosaic.
Let's add some data now.
I'll use the add rasters to Mosaic dataset,
specify the Mosaic dataset here
as the one that we just created.
And the last thing we need to do is supply some input data.
If we hover over that information,
we see that we have three different options here,
four different options, actually.
And that the right one for us will be the dataset option
because we're using TIFF files.
So let's locate those files and add them to the Mosaic.
So I add the two TIFF files
to my list of rasters that I want to include
in my Mosaic dataset and click run.
And now we can see our result.
We've got our footprints that define those two slices
of the digital elevation model and our resulting image.
Note the values there in the image itself.
Let's fix that.
If we go back to our geo processing interface,
we can calculate statistics on our raster dataset.
In this case, the Mosaic.
We don't need to skip anything,
and we're not going to specify an area of interest.
We'll just click run.
We note that the software locks down that Mosaic dataset
while the statistics are being computed.
We can see those lock icons,
and once the calculate statistics is done
that lock icon will disappear.
Okay, well, calculate statistics is done running,
and now we see from our image
that the symbology looks much better.
Now, if I had other data
that I wanted to add into my Mosaic,
I could certainly do that at this time.
And I'll note that I have two other tiles here
that are not directly connected to the data
that I've already built into the mosaic.
But instead of using that add rasters to the Mosaic,
I wanna show you one other thing.
I'm going to export
the raster dataset that we just created
and then we'll use the append tool
to actually add those other two slices into the dataset.
I've set up the export raster tool.
And I'll just click export to produce that output.
All right.
The export image ran successfully,
and now I've got that standalone raster dataset
that we see up on the top.
One last thing before we're done.
Let's add in these other two pieces
using the append tool.
Remember that append tool is going to modify
my target dataset.
In this case, the export from the Mosaic I just created.
I've added the other two input datasets from the list.
And now I'll click run to see what I can produce.
Looks like my append operation concluded successfully.
So now let's look over at the map pane to check our results.
And sure enough, I've got those two other pieces
that I appended to the exported Mosaic dataset.
That's it for this demo.
Remember one of the key takeaways here
is the difference between the union and the merge tool
when you're trying to combine vector data.
And that when we're working with raster data,
we take a slightly different approach,
but note that the append tool
can work for both raster and vector data.
The primary difference between the append and the merge
is that the merge tool will produce a new dataset,
and the append will modify an existing dataset.
Keep all these factors in mind
as you think about ways to assemble and overlay
your data for analysis.