- Yes, I got it.
I am the Principal Investigator of the FEMC,
and I know we have fabulous presentations
at these conferences every year,
but I really do appreciate your willingness
to come into one of these working sessions with us,
and I feel like this is really where
we're able to have sort of a tangible amount of time
to be able to work on moving forward
some of the efforts that we have.
So, this working session
is about the Northeastern Forest Inventory Network.
So, this work actually began
as an FEMC project.
We were tasked with looking
at continuous forest inventory data
from across the region and comparing the methods,
so what were the plot layouts,
what were the metrics that were collected,
how many could we identify,
and really just looking to see what types of questions
we might be able to answer with these.
And that work really spurred us
to think about how this could really be
an amazingly rich resource for all of us across the region,
to have access to a larger set
of continuous forest inventory data.
And so, we, FEMC staff, got together
with Tony D'Amato and Aaron Weisskittel,
brought on Soren Donisvitch as a graduate student
to think about how we might build off of what we had started
as that smaller FEMC project,
and the justification that we used,
and I'm going to try and advance my slide here.
Tell me you can see the next slide?
Okay, the justification that we used is that, yes,
there is a lot of forest inventory data out there,
but particularly in the Northeast,
our forests are really complex,
complex in terms of land use history,
in terms of ownership patterns,
in terms of management approaches,
in terms of species composition,
environmental conditions, disturbance patterns,
and so the inventory data really does need
to reflect that complexity.
And it doesn't really seem that any one dataset
is able to capture both the spatial
and temporal resolution that we need
to really be able to understand
both the historical patterns within our forest,
but also then use that information
to project forest change into the future.
And so, what we found in that initial project
is that there really are a lot of people out there
collecting great inventory data,
but a lot of these datasets sit somewhere isolated
and perhaps are only used by the people who collected them.
And so, we argued that if we could aggregate
this information and standardize these different datasets,
and then build a tool that people could use
to easily access and continue to add to those datasets,
that this would be really critical
to help inform forest management in a changing environment.
So, we were fortunate that the USDA did agree to fund that,
and there were three primary objectives.
The first was just to gather that data.
So, can we go out there, find more datasets?
Figured out if we can find an efficient way
to standardize those so that they can be aggregated,
you can search or extract data
beyond just pulling out the individual projects.
Can we build a tool that makes that access easy
and makes that data discoverable?
And also, thinking about how to make this something
that's dynamic, that can continue to be updated
and isn't static.
So, that was the first objective,
but the second objective was to actually start
to use that data.
So, could we demonstrate how this aggregated information
might be of use to the forest community,
and in particular comparing how just having
any one of these in isolation,
what can that tell you versus what additional information
are you able to get only from that aggregation,
so sort of thinking about the benefit,
is there really a tangible benefit to aggregating this?
And then, the third objective
was to do what we're doing right now
and try to bring people together,
both from the contributor side, people who are collecting,
actively collecting forest inventory data,
but also users, potential users
to expand access to this data and information
and really build a more engaged,
collaborative network so that we can answer
a lot of the questions that are still outstanding.
So, that's sort of the project as a whole.
For this workshop in particular,
we really wanted to spend a little bit of time
just making you aware of the work that we have done already,
what datasets we have, what those datasets look like,
what the tool, the web portal, looks like,
and the functionality of that,
but then, we really want to take the time to get feedback
from you about what the current sort of opportunities
and barriers are to really using forest inventory data
to inform forest management,
and then, also, to identify
a potential use case
that we could analyze as an example
of the utility of this dataset.
So, that's sort of what we're going to be doing here today,
and the first part of that is really just to talk
about the data that we have already aggregated.
And so, I am going to pass that over to Soren
to show you what he has been digging into
as a part of his graduate work.
- Okay, so,
first off, it's really good to see everybody here today.
I know I've been able to see the names of all the people,
and I've had a lot of contact with many of the people
in this chat and session today.
It's good to see you, in some instances, only via email,
over the phone, or in some instances, in person.
So, it's good to see here, and everyone else,
I'm excited to kind of show you
where the NEFIN data is currently.
So, again, my name is Soren Donisvitch,
and I am the grad student attached to this project.
So, to go over some basic goals
of this specific presentation,
I wanted to kind of let you know where the aggregation
of the NEFIN data really stands, where that dataset is,
what are the basic overviews of the records
that are in the dataset, so what's the shape of it,
what can you actually use it for perhaps,
but also really what's comprised
within the NEFIN dataset as of now.
This is going to go over, really, the spatial extent,
the geographic regions of the plots that we have
in the NEFIN dataset, as well as the programs
that we have included or are currently trying to include
and some future ones we would like to include in the future.
And then, to really go over some of the basic summaries
of the NEFIN data itself,
I don't really have time to go into, like, very specifics,
but a lot of the maybe plot and tree records by the program,
tree data by program through time,
really showing the value that this dataset actually has
not only in plot data in the present,
but also extending into the past,
the sapling and seedling data,
where that stands within NEFIN currently,
the top 10 tree species that are present
in the records that we have,
and then the top 10 tree mortality species.
This is just to give a basic overview of what the dataset
looks like and might be useful to you for.
So, for the spatial extent,
this map is comprised of all of the datasets
that have been aggregated into one and standardized.
This ranges all the way from Mid-Central New York,
Northern Maine, as well as out to the Cutler Coast
and all the way back down to Southern Connecticut.
So, it is a broad range.
The colors correspond to the plot location
and is differentiated by state.
As you can see, Massachusetts comprises the bulk
of a lot of the plots that we currently have.
We are looking to extend that
to many of the other states and other regions.
Just in case, I don't know everyone's background,
a continuous forest inventory
is a fixed forest inventory plot
that's repetitively sampled through time,
and what this allows you to do is answer a lot of questions,
such as mortality, in-growth, recruitment,
a lot of forest stand dynamic questions
that are more nuanced than maybe
a simple cruise may not be able to actually get at.
As you can see from this map, most of the data resides
within what we traditionally considered the Acadian forest
and the Northeastern coastal forests,
although we are looking to expand that further
with many of the other programs we're trying to incorporate.
This is a lot to look at, but basically this is a list
of all of the datasets that we have incorporated,
are in the process of incorporating,
or hopefully will incorporate in the future.
The second column has the code
that'll be used in this presentation
that corresponds to a given program.
Not all of the programs that are in
that code column are included in this summary
just because the standardization and aggregation
for those haven't all been completed
and aren't fully, you know, done,
so I don't want to put the cart before the horse
when it comes to putting up graphs.
But currently, there are 16 CFI programs
we are in the process of integrating into the dataset,
14 more, hopefully, in the works.
The dataset as a whole, the NEFIN dataset, as of right now,
has more than 60 years of CFI data,
the bulk of it moving from 1960 to the present.
This includes more than 14,000 CFI plots
across that geographic region,
being New York, Maine, Vermont,
New Hampshire, Connecticut, and Rhode Island.
So, there's an enormous number of plots
that have innumerable value just spatially in and of itself.
The tree records are just shy of (mumbles),
considerably shy, but 600,000 individual tree records
spanning that time, over 100,000 seedling records,
16,000 sapling records,
a considerable amount of health records,
47,000 health records attached to those trees,
as well as mortality,
and mortality is something that's very useful for CFI data.
So, spatial distribution of plots,
this is broken up by plot,
the NEFIN data, as it is right now, by state,
and then by county.
And so, something that's really nice about this kind
of a dataset is that it provides regional wealth in data,
but also localized wealth.
So, for forest land managers and researchers alike,
if, for instance, you're a land manager
and you're dealing with a forester
having a white pine stand,
but you don't have enough data to be able to answer
the questions you may be wanting to answer
to introduce good civic culture,
you can go through this dataset and find, hopefully,
data that not only is geographically close to your location,
but also may be structurally similar
and can provide you with a wealth of data
to better manage any number of applications to this data.
But this kind of gives you an overview
of where the data resides, predominantly in Massachusetts,
followed by Maine and New York,
and we are looking to increase the number of programs
you can connect with in Vermont,
New Hampshire, and Connecticut.
So, the graph on the left, again,
this is trying to just give you a basic understanding
of the shape of the NEFIN dataset as it is right now,
is the plot counts.
There are individual plots within each program
that we have access to,
so Massachusetts CFI program, followed by SUNY-ESF,
Maine Ecological Reserves.
At no point is any of this data not valuable.
This is valuable.
Each program has unique value,
and aggregated together, also has increased value.
Relative tree records on the right,
I deal with a lot of circles,
so I thought it'd be interesting to do a square pie chart.
It's called the Forest Chart,
but it's basically the idea that the large square is 100%
and the size of the squares correspond to the records.
The relative tree records, this is the graph on the right,
it's relative because each program
has a different expansion factor for its plot,
and so, I expanded it out to a per acre basis
just so that each of those boxes
would be representative against one another.
And so, this just gives you a basic overview
of where the contributions
to the overall NEFIN dataset come from and which program.
The greatest amount of value coming from CFI data
is that it's repetitive through time, and so you can answer
a lot of different questions from that.
This graph is able to kind of visually show you
how each program was sampled through time,
and if you can imagine collapsing that together
to overall temporal resolution increases,
you are able to look out over a long period,
a greater period of time with more data.
Sapling and seedling data.
So, sapling data,
not every program
had sapling data, which is why there's not as much,
but you're able to see
how each program corresponded to that.
And then, the spatial on the bottom left-hand corner,
you're able to see, from the 1970s to 2021, the present,
you're able to see how those tree saplings
were being reported and by which program.
The same thing for seedling data,
and basics around this is just showing
how this data is valued, not only now,
but also past in time.
Taking of points, you know, past, present,
hopefully we'll be able to figure out things in the future.
So, the species composition,
there are more than 80 recorded species
by genus level and by genus and individual species level,
but one of the things that's interesting
about this kind of a dataset is you're able
to plot the plot comparisons,
being able to find out species' compositional shifts,
things of that nature.
The graph is the top 10 tree species by percentage.
The Y-axis is percentage of overall composition
of the dataset itself.
So, for this instance, red maple comprises roughly 17%
of the overall dataset.
So, for instance, if you're interested in, I don't know,
white pine growth and yield based on carbon,
and you want to be able to find some plots for that kind
of data, this is a resource for you.
So, this also is incorporated by mortality,
so mortality by species, and this is single instance,
so it's not repetitive mortality.
Roughly 60 recorded species, and then, again,
this graph is just really looking
at the overall percentage that comprises
the NEFIN dataset itself.
So, if you're interested in mortality,
there's something here for you as well.
These are some citations for, I don't have too much time,
these are some citations for the data
that I'll be using here today.
I do really want to thank all of the people who are here
as well as the people we hope to incorporate in the future
for these data,
and looking forward to training with all of you.
If there are any questions- - I've got a question.
It's kind of a selfish one,
but I've submitted it out of my own data
to this program before, but I'm not sure
you folks actually have it.
Soren, do you remember seeing any of my data?
I've probably submitted 50 plots by now.
- Nothing that I've come across.
I don't want to make that final,
but I'll definitely keep an eye out for it.
- I submitted the plot data
when I submitted the tree ring data.
Maybe it's somehow linked with that, I don't know.
- No, that's a really good point.
So, FEMC does a lot of, sorry,
I'll make it so you can see me,
FEMC does a lot of data archiving,
and if you had submitted data to a different project,
it wouldn't automatically know to come
and be ingested into this one.
So, you would be someone that we would be reaching out to,
saying, do we have permission
to incorporate your actual inventory data?
So, stating- - Yeah, you have permission.
- (laughing) We will be in touch.
Jim Duncan, the Director of the FEMC,
and Emma Tait, our Data Scientist,
along with Clarke Cooper and other FEMC staff
have been working really hard on the back-end of this
to begin to develop some sort of a tool that would make
this data available, so I am going to turn it over to Jim
to share that work.
It's a little demo.
- This is an overview
of the Northeastern Forest Inventory Network Data System
and the status of where we are
with developing this infrastructure.
So, before I walk through the two components of the website,
the first thing I wanted to highlight
is our network design principles.
We're trying to make sure that we don't require
a change in data storage by programs,
that we automate as much as possible
the input and ingested data,
we share as much as possible with the network,
but enable restrictions when needed,
especially around plot locations,
and provide cross-coded, standardized data
coming out of these various forest inventory programs.
So, those are the things that we were trying
to meet with building this system.
Looking at the workflow on a high level,
the data manager preps their data,
they upload it into the NEFIN system,
and then they go through a structured workflow
to update any metadata that might've changed
since the last time a program was put in.
Once all of that is done,
the NEFIN system will actually process that data
to store with both the original observations
and standardized subset of fields, such as DBH and species.
And finally, users then can access that data by download
or exploring the system through the online interface.
Switching over to the actual NEFIN site,
this is what you'll see at the Home page,
and it reviews the network and key statistics
about the programs already in.
We've put in six programs so far,
and this shows the count of plots, trees,
and species represented as well as a breakdown
of the basal area running off the standardized diameter
that we've calculated for each program's observations.
You can start getting into the data
in the system through the Get Data tab.
It provides an overview of programs, plot data,
and tree data that's associated with the NEFIN system.
We'll eventually be adding regeneration data
and whether they're seedlings or saplings as well.
But the other nice piece of this is that you can use
the filters on the left to actually limit down
what you're interested in looking at.
So, for example, we could look down to sugar maple
and red maple, and we could consider only, say,
the northern New England states
and set our filters this way,
and click to search,
and this limits down the programs, the plots,
and the trees to just those records that are matching.
And if you were to click on something
like view the Original Data,
you can actually see what was stored
in the data file as delivered,
and this shows you the standardized information,
DBH in centimeters, cross-coded tree status
that represents live or dead, and the sample year.
But as you can see in the filters,
we're also going to be including other items,
such as whether there's an overstory tree,
whether there's a presence
of a forest health indicator mentioned,
such as defoliation.
You can limit by DBH range or program.
So, there's many different ways to use
this standardized data to get an extract.
You can click on the program to get
some general information about it.
This is the Maine Ecological Reserves Program,
so some basic details, and then more detailed definitions,
such as the shape of the plot
and a list of files that are available,
such as methods, documents, import packages,
and scripts that have been used to process the data.
So, if you were then to click on the Download tab,
you'd be able to see
both an aggregate of all the information
you're seeing in plots, trees, and programs,
as well as supporting documentation for each
of the programs, including raw files if you want them.
So, you can really access all of the information
underneath that that you would like.
Clicking over to Inventory Programs is a way just to look
at the list of programs that are in there
and get more information specific to them,
so again, like the Maine Ecological Reserves Program,
for example, get those same details,
definitions about how the plots were set up,
key statistics about the species by count and tallies,
the number of observations of trees,
information on the history of changes made
to the data system, as well as downloads.
So, again, more of a program-based exploration,
and some of this information can be tailored
so that certain parts can be downloadable
and certain parts can't.
We provide additional information to help people
learn more about the program through the About
or to contact us if they're interested in participating.
So, as I mentioned, we have
six programs that we've largely processed and put into here,
representing most of the states in the seven state region,
and we have at least four more that are pretty much ready
to go and are working on a number
of others that could also be of interest.
So, now I will turn it over to Emma,
who will give us a rundown of the back-end
and how you put data into this system
from an individual program.
- [Emma] Once a continuous forest inventory program
is in the NEFIN system,
the data management portal allows managers
to add new data for their program.
Once you've selected your program,
you can see some basic information about the program
as well as a summary of the last version
of the metadata for your program.
To create a new version,
first select the years that the data covers.
This could be one year or multiple,
and then select your files.
These files are going to be unique for each program.
In the case of Mass CFI, these are a Plot File, a Tree File,
and a Region File that are direct dumps of tables
from the accessed database.
Once these files are uploaded,
they will be processed
to have a set of unique IDs that are specific to NEFIN,
and to do any drawings required to create three sets
of unique observations, one for plots, one for trees,
and one for regeneration.
Next will be a step to match
the standard fields required for the NEFIN system
to field in the uploaded data.
It will try to match the same fields from the last version
if those fields exist in the newly uploaded data.
If those fields don't exist, it will be blank,
and you'll need to match a field,
but there may also be situations in which a field changes,
but you keep both, and so, in that case,
you would need to indicate what that new field is,
and this will need to happen for any plot information,
any tree information, and any regeneration information.
There's also some program-level information,
and this is unlikely to change all that often,
but it's possible that it might need
to change over the program's lifetime,
and so that can be done as well.
Next, the system will scan
any fields that have lists, such as your Species column,
and it will check to see if the codes in that column
match the species list for your program
or whatever the code list is for your program.
If it finds a code
in your newly uploaded data that doesn't match
the species list, then you'll need to provide
your program definition for that code
as well as
match it to a NEFIN generalized code.
In the case of species, this is going to be a species name,
and that will also link with an ITIS code.
Once you've done all of the matching,
we do some scanning for some quality control checks,
and the first set of checks are critical checks
that look at whether or not you have text data
in your numeric fields,
such as, you know, text in a DBH field.
If those critical checks fail, the system can't progress.
So, the program manager would need
to edit the data and fix those issues,
and then re-upload the data.
The other set of checks are a set of warnings that look
at whether your tree DBHs are within an expected range
or the ratio of your height to DBH is realistic for a tree.
Those don't cause the system to fail,
and so they can be addressed as needed
by the program manager
or can be ignored if they're not relevant.
Lastly, provide a brief description for the new version,
and then review
the summary of the changes made for your program.
This includes the files, the years, the standard metrics,
and any codes added that were not previously in the system.
Once all of that is verified to be complete,
then you submit the version and the metadata gets added
to the NEFIN metadata database,
and a copy of the data you uploaded is standardized
into a set of standard tables,
along with the rest of the programs within the NEFIN system.
This process may take some time, and in the future,
you'll be able to check your progress
as the processing happens,
but once the processing is complete,
then you'll be able to see the data
that was just uploaded in the NEFIN Data Explorer.
- So, I'm happy to answer any questions.
I know there's a fair bit of technical detail there,
but just a kind of a quick walk-through
of both the accessing information at the front-end
and being able to put content in from the back-end.
So, yeah, any questions on anything that was shown there?
- It was great, that's really interesting
to see how this works.
I guess I had two questions.
First, some of us who use large plots
also include the XY coordinates.
Are those retained in these datasets?
In other words, can someone search for all plots
that had XY coordinates for the trees?
- So, you mean specifically you have tree-level coordinates.
- Whoever's looking at the spatial location.
- Yes, this system isn't set up to really,
because that's not very common, we didn't set that up
to be a feature of the tree information.
It will be retained in that original data popup,
so someone will be able to see it,
but as far as searching for trees
that fall within that range, no.
We'd have to rely on the plot location at this point,
but if we did see, it would not be a significant amount
of work to add that to the standardized tree information
if we started having more inventory programs coming in
with tree location data.
- Okay, thanks.
The other question, just to make sure I understand,
a lot of your examples referred to programs,
like state programs, that update.
Would I use the same process as an individual researcher
to upload my data?
- If you wanted to be contributing to NEFIN?
- Yeah. - Yep.
The idea is to have programs that are going to do
repeated measures on permanent plots.
So, as long as that kind of basic requirement is met,
that's the same process for the back-end,
so you get your own access to your own space.
You'd have your own structure for your data
that has been prepared, and that can be put in.
- Okay, thank you.
- We have a couple more months of work to do
just to make sure we've sealed up the cracks,
and right now, it works with kind of an admin user,
but we haven't done the kind of user level features yet.
So, I think there's still a little bit of work to do
on that side of things, and then just handling
some of the other pieces, like regeneration data,
where we want to make sure we have sussed that out
before inviting people to come in.
So, I think, you know, the first step with any program
that's going to go in here is some work to prepare
the system to accept that program data, 'cause again,
we're trying to not force people to use
some sort of standard template to upload data,
but rather upload the data in the system they collected in,
so that would also add time to a new program coming in.
And I think, by spring,
like, spring is kind of my target,
that we're pretty much launching
a fully baked public and private version.
- Thank you all for contributing your ideas.
I really do think it was pretty interesting
to see the diversity of people,
at least that were in our breakout room,
and to hear the different ideas and different perspectives.
Let me share my screen again just to kind of tell you
where we are hoping to go
right now.
We do have a website for this project that is live,
and on that website is a link to be able to get involved.
So, if you
have been somehow inspired and you have data
that you think would be appropriate to contribute,
this link, and Jim, I don't know,
do you have the direct link to that
that you can stick in the chat?
- Yeah, I can do that. - Thank you.
You can go to the Get Involved tab
and it will allow you to just enter your information.
It will come straight to us in the NEFIN project
and we can connect back with you, whether that's, you know,
if you're interested in contributing data
or if you are interested in being a part
of any future analyses or data summaries
or research questions or different ways
of actually utilizing this aggregate data.
So, please do let us know if you're interested
in continuing on.
Thank you all for contributing here
and really helping us figure out where to take this.
So, thanks, all.
- Thanks, everybody, and I looked at the link.
I think that we had a website upgrade
that broke something in that.
So, if you try and fill out the Contact To Contribute form,
it might work for you, it might not,
so if it doesn't, just feel free to email me
and I can connect with you about that.
- Yeah, and I can put my email in the chat as well.