- Am sick, so bear with me.
My brain feels a little slow, and my throat's not great.
Oh, now I can see you, okay.
So I was joking with Anne and Cassandra before we got on
that this webinar's gonna be kind of boring.
So Kim already hyped it up before people got on
that it was gonna be so great,
and now I'm like, oh boy, I don't know.
But hopefully it will at least be useful and helpful,
if not exciting.
I am Amy Wheeler-Sutton.
I'm here today with Anne Dubie and Cassandra Townshend,
who many of you may have known
was previously on the BEST team,
and now is back with us, which is very exciting.
So she and I are the new co-directors of the BEST Project
and Vermont PBIS state team.
So welcome back, Cassandra.
- Thank you, Amy.
- [Amy] Excited to have you.
- Thank you.
I'm excited to be back
and see some familiar faces and names,
so excited to join you.
And also, Amy, I was just thinking,
who cannot get excited about data and analyzing, right?
It's fun, one of my faves.
This is gonna be great.
So I'm excited to be part of this this morning.
Welcome, everyone.
- Awesome.
So we will keep you muted during this session,
but we are a pretty small bunch,
so if you have a question that you want to raise your hand
and unmute and ask out loud,
or you can type into the chat box, we'll be watching it.
Anne, if I can ask you to put the link to the materials
in the chat box.
I was just updating the PowerPoint
a little bit this morning, so it's not...
The one that's on the website is missing a few slides,
but we'll get that updated right after the webinar,
and then there's some other helpful materials.
We are recording this morning,
so if you need to hop off,
we will send a out the link to the recording
and put it on the website.
And if you have other colleagues
that you think might need this information,
feel free to share the recording with them.
And we will get started.
We're gonna be talking today
about the Tiered Fidelity Inventory or the TFI.
Some people call it the Tiffy
'cause that's what we started calling it
'cause that's what we thought the national center
was gonna call it, and they don't.
It's just us and Vermont calling it the Tiffy,
the Self-Assessment Survey, and the School Climate Survey.
So before we get started and launch into things,
we are gonna kind of just acknowledge
the current context of education that we're operating within
and recognize how to challenging things are,
and that this, what we're talking about this morning
may absolutely not be a priority for your school right now
and that that's okay
and remind you of the importance of these assessments
and how they may help in the current context.
Help you kind of come back to that basis
of working from an action planning framework,
then get into the TFI and the SAS, how to access it,
how to understand your results.
Do a little exploration of the School Climate Survey,
which some of you may have completed last year,
and then talk a little bit about putting it all together,
action planning, and disseminating your results.
So in terms of the context,
this quote is now outdated
from Peoples Academy Middle School Principal, Matt Young,
but kind of is still relevant
in that schools that have a strong PBIS framework in place
have really leaned on that framework
during the now almost two years of the pandemic
and it's really helped schools
to have something to come back to
in terms of you already know how to teach
new expectations to students,
to acknowledge when they're meeting those expectations,
to think about your teaming structures
and who you can lean on.
So we're hopeful that these three assessments
will further enhance your framework
and help you identify some key priorities
to hone in on improving rather than being over overwhelmed
by all there is to do in schools right now.
So all of you are probably very familiar,
or many of you are probably very familiar with this visual
that's helping us to see that
when we're talking about PBIS,
we're really talking about systems, data, and practices
and really thinking about the best outcomes
for our all students.
And today, of course, we're really gonna be focusing
on the data aspect of this concept.
So for those of you who have been involved
with PBIS in the past,
you are familiar that we ask schools to complete
the Tiered Fidelity Inventory each year.
And last year, we gave the option of whether you wanted
to complete the Self-Assessment Survey
or the School Climate Survey,
and that will be the same this year.
Some schools did choose to complete all three assessments,
and that's fine too.
They SAS and the School Climate Survey
are not interchangeable.
They do serve different fun functions,
but the Self-Assessment Survey was becoming
a little bit outdated,
and it's much lengthier than the School Climate Survey.
And so in these complicated times
and in a time where it's really important to understand
the status of your school climate,
we wanted to offer the School Climate Survey as an option.
Some schools have or districts have
their own School Climate Survey.
The one that we're gonna talk about today
is the School Climate Survey that's housed within PBISApps.
But if you complete a School Climate Survey
through your district, if you just wanna let us know that,
then we will kind of count you in our count
of how many schools are completing
any School Climate Survey this year.
And we do open the window, usually from January to March.
If you've been around a while,
you may notice we usually give an extension,
and especially during these times,
we likely will give an extension to all,
but if you know right now you're gonna want an extension,
you know you're gonna complete it during, you know,
let's say an in-service day in April or something,
just let Anne know, and she can extend your window
so that it won't close on you.
So like I just was mentioning,
we know that when schools are implementing
PBIS with fidelity,
they've had a little bit easier time navigating this
both when they transitioned to virtual learning
in March of 2020,
when they came back and in-person learning,
and really thinking about the increased mental health needs
of students and staff and families right now
and how your PBIS framework can help you
integrate mental health supports and resources
into your schools.
And we actually have some participants on today
from our Project AWARE schools,
which are doing really intentional work
around the interconnected systems framework,
which is about the really intentional integration
of mental health into PBIS.
So in other words, we don't want this
to seem like something more and extra
that schools are having to do right now.
We want you to be able to use this
as part of your planning process
to better help students access their learning
and improve your mental health supports within your schools.
And basically, now that we know what an, you know,
intense impact there has been over the past few years
on teachers, students, families,
it's really important to expand the data that we're using.
So for instance, something like, you know,
office discipline referrals
or behavior observation and data forms
that you've been using in the past
may not fully represent the picture of what's going on
in your school right now,
but when you're able to look at something
like the School Climate Survey,
that's another data point for you to use
and to help you hone in on how students, families, and staff
are feeling at this time.
Okay, so when we're thinking about data,
we wanna be really careful to make sure
that any data we are collecting is gonna serve a purpose.
So we never want people to feel like
we are surveying them or doing any kind of data collection
that is just gonna sit and not be analyzed
and used and reported back on.
We really wanna be collecting our data with fidelity.
So we talk a lot about this when we're talking about
student outcome data,
but we also wanna think about this
for things like the Tiered Fidelity Inventory.
Some schools, especially when they're in a place of stress
or their, you know, PBIS implementation is faltering,
they might just have the administrator
complete the Tiered Fidelity Inventory,
and that's not how it was intended to be used.
It's intended to be used and completed
by the whole leadership team,
preferably with a coach as well, an external coach,
who's really familiar with the questions
and what they all mean,
so thinking about whether you're using these assessments
in the way that they were intended
and collecting that data with fidelity.
And then, like I mentioned,
you don't just want this data to sit.
So you wanna be able to promptly look at the data,
act upon it, and share back with people
what changes are being need as a result of it.
Again, we wanna be looking at multiple sources of data
so we know that one data point
is not gonna tell us the whole story,
and we wanna be using our data to support
and never used for punishment.
So sometimes we get kind of far into PBIS,
and we may have lost track of why we got into it
in the first place
and what we're hoping PBIS will help us
to improve within our school.
So we can't really say whether it's working or not
unless we know where we're headed
and what we're hoping to achieve.
So part of your Tiered Fidelity Inventory,
Self-Assessment Surveys, School Climate Survey process
might be coming back and making sure that your action plan
has some SMART goal outcomes associated with it.
So I know some schools,
I don't know if Chantel's on today,
they actually keep their outcome statements
right at the top of their agenda
so that every meeting they're coming back to
what our big overarching goals that we're working toward,
and then what's our action plan to help us get there.
So thinking about that SMART goal format,
which I'm sure you all are familiar with,
to set those outcomes both for the year
and then also kind of mini action plans
maybe on a monthly or bimonthly basis.
And thinking about in this new context,
what might those outcomes look like for you?
So in the past, you might have been really relying heavily
upon your behavior observation and data forms or ODRs
to track whether PBIS implementation was successful.
That might still be really useful during this time,
especially now that, well, until this week,
we were back fully in-person for the most part.
However, we wanna be really careful
to make sure that that's not our only outcome
that we're looking for.
So thinking about whether you are seeing
increased rates of bullying
and are putting interventions into place
to improve that, looking at attendance or engagement rates,
school climate, fidelity of interventions, academics,
and then many of us are thinking about
how can we really draw in our community partners
to provide supports that our schools
are maybe lacking right now
or don't have the capacity to provide.
So making sure that these goal statements
have time associated with them,
so maybe by the end of the school year,
by the end of the month,
and are really specific so that you can measure
whether that goal has been accomplished or not.
So thinking about things like suspension/expulsion,
which probably have changed a lot
over the course of the pandemic
in terms of use of those practices,
but you can see they've named
they want to decrease it by 10%
by the end of the school year,
so giving an actual target,
or something more broad like as a result of completing
the School Climate Survey
for those three stakeholder groups,
we will see improvement in the three priority areas
that we've identified.
So they're gonna do the School Climate Survey
at the beginning of the year and the end of the year
and hope to see progress.
So many of you have probably seen this slide before,
and I'm not gonna spend too much time on it right now,
but this is a great one to come back to
if you feel like you are lacking some data sources
that might build out that picture
of how your students, staff, and families
are doing this year.
And you can see different data sources
at the universal, targeted, and intensive levels.
So we're mostly gonna be talking about universal today,
but for those of you who have completed
the TFI and the SAS,
you know that there's also tier two and tier three questions
on the TFI if you have been trained
or are implementing at those levels.
And we just wanna kind of come back to this idea
that data's really helping us ask the right questions.
It's not necessarily gonna provide us with all the answers.
So it provides a jumping off point
to have some rich conversations,
but we need to do some digging
to really get to the answers and the why behind the data.
So before we launch into the Tiered Fidelity Inventory,
it would be great to,
if you could chime in in the chat box
about how familiar you are with the TFI and the SAS already
so I know how much detail to go into or not.
- [Woman] Oh, hey! - [Woman 2] Hi.
- [Woman] How we doing?
- Okay, looks like we have mostly people
on the end of not very familiar,
which is great 'cause that's why you're here.
Tracy is familiar, okay.
And some of you in new roles as PBIS coordinators,
so maybe participated in the past
but now are expected to take more of a leadership role
within these.
Great.
Okay.
That's looking funky for me.
Hopefully it will go back to normal.
There we go.
So I'm gonna do the best I can in the next 40 minutes.
Oh good, my husband's texting me.
My daughter would love to peek in.
No thank you.
However, if you walk away needing additional information
about any of these assessments,
on the PBIS Vermont website,
under the evaluation tab at the top,
there is a page for each of these.
I'm remembering, as I say that,
that the School Climate Survey one,
if you click on evaluation at the top,
School Climate Survey is not gonna show up in this list,
but if you hover over evaluation
and the dropdown list comes,
you will see School Climate Survey, TFI, and SAS all there.
So that gives you all the information
about even how to log in to PBIS assessment,
where they're housed, where to find the assessments,
how to complete them, how to get your results.
All of that is there.
Okay, so the Tiered Fidelity Inventory,
like I just said,
it has the capacity to assess all three tiers,
And at the beginning of the assessment, this year is new,
it's gonna ask you which tier are you assessing today.
So for instance, if your tier one team
is meeting on Tuesday,
and your tier two team is not meeting until Thursday,
that's okay.
You're just gonna put your tier one team
is assessing right now,
and it'll just give you the tier one questions.
And then when your tier two team goes in,
they'll just get the tier two questions.
So that's a nice addition this year.
And at tier one, it's assessing teams,
implementation, and evaluation.
Tier two and tier three, the categories are very similar,
but they change a little bit
for those tiers of implementation.
We know we're completing this so that we can
really look at our PBIS implementation within our school
on a year by year basis
and see where our areas of strength are,
where we need to focus our attention and grow.
And the goal is to be really honest
with yourselves and your team
so that you can identify those areas
that you might need some work in.
If you go through and just select twos for everything,
it's not actually gonna help you identify an action plan
and come out of it with anything.
And for us, what we're really looking for
is just completion.
So we don't necessarily,
not that we don't care what your scores are,
but we want it to be an accurate representation
of your implementation.
There's no like, PBIS police gonna come after you
if you are implementing at 50% fidelity.
Our goal is for all of our schools to be at 70% fidelity
after they've been implementing for a few years,
but there are always implementation dips,
and maybe you really focused on tier two this year,
and your tier two implementation is gonna be at 90%,
but in doing so, your tier one implementation
has dipped a little bit.
So that's just to say please be honest with yourselves,
and don't worry that we're gonna, you know,
be looking at your scores
and thinking that something is going wrong.
And it's a pandemic,
so all of our scores are lower across the board
in every part of our lives.
So like I said, the window is already open.
It's set to close right now, March 31st,
but if you need an extension, please email Anne.
In a typical year, you might complete
the Tiered Fidelity Inventory more than once,
especially if you're really focusing on something
and you wanna see whether your efforts
are helping that implementation score to grow.
And in terms of planning,
you wanna think about 30 minutes for each tier.
So if your team is gonna sit down
and complete all three tiers at once,
that might be around a 90 minute process,
and that's just to complete the assessment,
and then afterward is when you wanna do the action planning.
So you wanna keep the conversation pretty tight
when you're actually completing the assessment
and not get into so much of like,
oh, well, we could send a newsletter out to families,
and we could send it via email,
and we could send a paper version.
You don't wanna get into the details of the action planning
about how you're gonna improve that score.
You just wanna do the scoring and then be able to come back
and identify your areas of strength and growth.
So, and in the past,
we have used the score on the Tiered Fidelity Inventory
and completion of the SAS
to determine merit and exemplar schools.
If you've been around,
you've noticed we haven't done that same process
for the past two years.
It's been more of a qualitative process
where we're asking you to submit a narrative
about how your school's implementation is going
because we recognize that our typical measures
are not necessarily as useful right now
in terms of, you know, holding up one school as a model.
So like I said,
the three areas that are assessed on tier one
are teaming, implementation, and evaluation.
And if you've, you know,
been and through universal training,
that's kind of how the training is structured too, right?
We start at the beginning,
really focusing on who should be on your team,
how can you make that team
representative of your school staff,
how can it be a really strong and well functioning team
with some good norms and agendas and processes?
So that's what the first chunk of questions on the tier one,
it's only two questions, I believe.
The whole tier one is only 15 questions,
but that's what those first two questions are around
is the teaming functions.
And then you'll get into implementation,
thinking about whether your school
has those identified behavioral expectations,
whether they're teaching them really explicitly,
do you have definitions for your problem behaviors,
consistent responses,
and really thinking about that consistency across staff,
across locations within your building
and whether your school wide PBIS implementation
is really well integrated into classroom systems.
And just an example of that that we've been noticing lately
is many of our schools are also
responsive classroom schools,
and sometimes when we go,
and we're walking around and seeing schools,
we'll see that they have their PBIS school wide expectations
posted in the classroom.
So, you know, whether it's
we are safe, respectful, and responsible,
and then they'll also have the shared agreements
that they've come together
in responsive classroom morning meetings to develop,
and those are almost two totally separate things
versus developing those expectations with your students
in responsive classroom way
and then bringing them over
into your school wide expectations and saying,
okay, you know, we agreed that we want to make sure
that only one person is talking at a time
so we can hear everyone's thoughts.
Is that safe, respectful, or responsible?
Where does that fall into our school wide expectations?
So making sure that all those kind of classroom systems
are aligned so that students aren't overwhelmed
by all the kind of social emotional related language
that's going on in the classroom.
Making sure the acknowledgement system
is happening in the classroom
and is connected with the school wide system,
things like that.
And we know that when
there are really great systems in place
and that are very formalized
and consistent across the building,
that when we are acknowledging and reteaching
the behaviors we are looking for,
students are more likely to sustain
those positive behaviors.
And then a big piece where we often fall short
is this component around actively engaging
our families, students, and staff in PBIS implementation,
that two-way communication.
So we often share information out with families
and may not necessarily ask a for their feedback and input
around our school wide implementation of PBIS.
So there's the assessment itself, those 15 questions,
but then you'll see each item
has some additional tools that you might use
to that you're scoring yourself accurately on that item.
So a big one is, for tier one,
is this Tiered Fidelity Inventory walkthrough tool.
And this is best used by somebody
who is not necessarily a school staff.
So if you have a district PBIS coordinator,
if you have a coach,
someone who's a little bit unfamiliar to the staff
is the best kind of person to complete this assessment.
And it basically involves walking around the building
and randomly asking students and staff a few questions,
and you can, maybe you can't see 'cause it's super tiny,
but of three questions for the staff
and two questions for the student.
It's a super quick, like 30 second interaction
with that person,
and it really gives you a sense of whether the staff
are teaching the expectations explicitly
and are acknowledging students
when they meet those expectations
and for students, whether do they know
what the expectations are
and whether they are receiving acknowledgement
when they meet those expectations.
So those inform some of the items on the TFI.
So that's one additional tool.
And then the last section on tier one is evaluation,
so making sure that your data system
is able to give you instantaneous access
to your data in graph form.
So if you're using SWIS, that's kind of an automatic,
you get a two on that.
If you're using a different system,
it helps you really look at what does that system
need to have in order to be able to give you the data
in the way that you need it when you need it.
Are you using your fidelity data annually
and making a plan based on it?
And are you reporting data back to your key stakeholders?
So after you've completed your assessment,
then you have a few different ways
of looking at your reports.
So you can look at,
the first one that comes up on the screen
in PBIS Assessment is your total score report,
and that's all three of the tiers averaged.
I'm curious to see what it will look like
now that they know which tier you were assessing.
So before, if you were only assessing tier one,
you'd have to put in zeros for tier two and tier three,
and that would bring your average way down.
So that first graph was usually useless
if you were only assessing one or two tiers.
I'm curious what'll look like this year
now that they know which tiers you were planning to assess.
The subscale report then breaks it down by tier,
so that's obviously more helpful,
and then going down even further,
breaking it down by components,
and then you also can look at it by your items.
And that's where we can really look at
what do we give ourselves a zero, one, or two on,
really celebrate those twos.
They may not have always been twos
and some hard work went into them.
And where are zeros and ones,
and what do we really wanna hone in on and focus on?
So this is an example of what that scale report
might look like,
and this shows it over the course of two years,
so their 2019 data and their 2020 data side by side.
So you can see this kind of demonstrates what I was saying.
You know, they might have been really focusing
on tier two this year,
and so their tier two implementation improved,
and their tier one implementation
might have suffered a little bit because of that.
But being able to show your staff or your team
a graph like this and being able to ask, you know,
what do you notice, what are you wondering,
what might we prioritized moving forward,
and then even better, looking at your subscale report
gives you much more information.
So you can see in tier one,
their scores decreased in all three sub skills.
So it wasn't just that, you know, their teaming fell apart.
Each of those, they, you know, lost,
gave themselves, you know, a few lower scores in
versus in tier two, you can see both their teaming
and their interventions improved,
their evaluation decreased slightly.
So each level you go to is gonna give you
slightly more information about your results.
And then this is what the items report looks like.
And same with this, you can pull in multiple years.
So you can see they have their 2019 scores
and their 2020 scores side by side
so that you can really look at, you know,
how did we change over the years and what might-
So for instance, in their item one
about their team composition, you know,
they're probably gonna know,
okay, well, last year, you know, we were able to have
our BCBA on the team.
This year, we were not due to scheduling.
You know, what could we plan for for next year
to make sure that our team composition is really strong?
So new this year, like I mentioned,
it's gonna ask you which tiers you're assessing.
You can't leave any responses blank.
And some of you who've done this before might be familiar.
There used to be three questions at the end
that you never actually saw your results from anyway.
Those have been removed.
So there was a question around like,
what percentage of your students are successful right now
behaviorally and academically,
and there was two other questions.
They've removed those questions at the end.
They always were optional,
but now they've been removed.
They also are coming out with a TFI 3.0.
So if you're, I don't know, a magical unicorn school
within our state that is not suffering right now,
and you wanna do some extra work,
they're looking for schools
who wanna help validate that assessment.
And so if you're interested in that,
feel free to reach out to me.
It involves completing the assessment with your coach,
without your coach, and then again with your coach
over a certain timeframe in February and March.
So if that sounds really exciting to you, let me know.
And then when the new Tiered Fidelity Inventory comes out,
we'll obviously do some additional training around that.
And then thinking about
where you're gonna capture this information.
So on the website, we have a link for you
for a Tiered Fidelity Inventory action planning form,
where you can keep track of scores,
what actions you think need to take place
in order to improve your implementation,
who's gonna do it and by when.
I forgot to add a slide on this, but there are also,
some of you might be familiar
with the culturally responsive field guide that has come out
from the national center of PBIS,
and we actually created an action plan template
that has those culturally responsive...
I forget how they refer to them, not add ons, but I forget.
But it has what you would need to do
in order to ensure that your PBIS implementation
is really culturally responsive.
So you can look at the typical item
and then the elaboration, that might be the word,
right below it.
And so that can really help inform your action planning too.
So maybe your PBIS team
is representative of your school staff,
but the student that you've selected to be on your team
is not representative of your student population.
So thinking about those elaborations
to ensure that our PBIS implementation
is really culturally responsive.
So I will add that other action planning form
and the cultural responsiveness field guide
to the materials on the website
for schools that are ready for that step as well.
So I'm gonna pause for a minute
'cause I think that's the last TFI
and see what questions you have about the TFI
and hopefully catch my breath.
Oh, good, Kim, I was right, elaboration.
Okay.
I'm gonna keep going,
but if a TFI question comes up for you,
feel free to put it in the chat box.
And Kim has put the link
to the cultural responsive field guide
in the chat box for us.
Thank you, Kim.
Okay, so I'm gonna move on to the Self-Assessment Survey.
And again, this year you're gonna choose
whether you wanna complete the Self-Assessment Survey
or the School Climate Survey or both.
The purpose of the Self-Assessment Survey
is really for your entire staff to be able to give
some feedback and input about their perceptions
of your PBIS practices and implementation
across your school
and then to develop your action plan outside of that.
Yes, so the TFI is intended to be completed
by your PBIS leadership team.
So whether you just have one team
that oversees all three tiers of implementation,
or whether you have a universal team,
targeted team, intensive team,
whether in your school, you just have a leadership team
that oversees all school, you know,
practices and structures and everything,
then it would be your leadership team completing it.
So it's kind of whoever oversees PBIS implementation
within your school.
We have had schools actually put up the items
of the tier one TFI on like poster paper around, you know,
the library or wherever you have your staff meetings,
virtually, if you're doing them virtually this year,
and have the entire staff give feedback
about the items on the Tiered Fidelity Inventory.
That's an option, too.
Some of the staff, you know,
might not be familiar with all elements
of the Tiered Fidelity Inventory,
so they might not know whether, you know,
data is shared with all stakeholders or something like that,
but it gives them an opportunity to give feedback
on the items that they are connected with,
but it's not a requirement.
It's intended for your leadership team,
ideally with a coach or TA.
So if you don't have a coach,
if you wanna reach out to your TA,
if you're interested in having them join
either virtually or in-person to help your team
complete the TFI.
The Self-Assessment Survey is intended for the full staff,
and that includes people like your, you know,
cafeteria workers, bus drivers if possible,
anybody that you would be expecting
to implement PBIS within your school.
It's a link that gets sent out.
So you log into your PBIS Assessment account,
and you're able to get that link to send out to people
that's specific to your school,
and then it creates the graphs for you.
Okay.
So the two questions that it is asking
are what systems are in place now,
and what systems are most in need of improvement.
So it helps you identify
what might be strengths or challenges
in terms of implementation,
and then how do staff feel about
whether that's important to them that it gets better?
So they might see an area of weakness
that is also a low priority for improvement in your school
or the opposite.
So, like I said, you'll log into PBIS Assessment.
You'll find the Self-Assessment Survey
in the open survey windows section.
For those of you completing the TFI,
that'll also be in the open surveys window,
and I'll talk later about how you get
the School Climate Survey to show up there as well.
But you're gonna locate...
If you are on multiple schools' PBIS assessment accounts,
so maybe if you're the district coordinator,
you're gonna have several schools show up here,
but you'll locate the school you're hoping to talk about
and click link over there.
And then it's gonna give you that link
that you'll send out to your whole staff,
anybody that you want to submit a response.
Nicole is asking, are they already open?
So the TFI and the SAS are already open.
The School Climate Survey you have to email Anne
to ask her to open it,
and I'll let you know what she's gonna be looking for
in that email.
Let's see, did I miss anything else?
And of course, when you send out this link to staff,
you wanna explain the purpose behind this assessment,
how long you're expecting them to spend on it.
And the most ideal arrangement
is that they are completing this
during something like a staff meeting,
where they're already expected...
You know, you're not taking up extra time.
You're asking them to do it during a time
they're already expected to be working.
The other benefit of doing it during a staff meeting
is that you can answer any questions they might have
about certain questions.
When you get your results back,
this is what it's gonna look like.
So this is the, again, like the TFI,
you're gonna get your results in several different formats.
So this is what it looks like overall for your total score.
This is specifically for the school wide current status.
So you can see these graphs are not quite as simple to read
as the TFI graphs.
It takes a little extra minute
to see what exactly you're looking at here,
but the staff have all rated items on
in place, partially in place, or not in place at all.
And so the dark green is in place,
the light green is partially in place,
and the hash lines are not in place at all.
That's on the left hand side,
and that's over the course of two years,
or sorry, two administrations of this.
So they did it in April and then in November.
And so you see that they actually
have fewer of these items in place in November.
So maybe they've had, you know,
administrative turnover or staff turnover,
and their implementation of the school wide practices
has actually gone down from April to November.
And then on the right hand side,
is that school wide improvement priority graph.
So whether they rated that item
as a high priority for improvement, medium priority,
or low priority.
So you can see,
in April when their implementation was pretty good,
they rated that as a low priority for improvement.
51% said, no, we don't really need
to spend too much time on this.
Now that their implementation has declined,
52% of staff are saying
this is a high priority for improvement.
So you can see how this information might be useful.
The challenge of the Self-Assessment Survey
is that it's quite lengthy,
and some of the language is a little clunky.
They actually are also doing,
the national center is also working on
a new Self-Assessment Survey.
I've forget, 4.0 or something.
So if you're interested in being involved
in that validation process,
that's an option right now as well,
and I think that just involves taking
this old Self-Assessment Survey
and then taking the new Self-Assessment Survey
a few months later.
So again, we feel really uncomfortable
asking schools to do anything
more than you're already doing,
so we're not really gonna push that,
but if for some reason you feel really jazzed
about either of these assessments
and want to complete them more than once,
feel free to reach out.
And same with the Self-Assessment Survey,
you wanna share these back to staff
and give them the opportunity to reflect on them.
What are they seeing?
What do they wonder?
What might they prioritize?
And like the TFI, you also can see it by subscale.
So you can see the sections down below
are the different categories
that the different items fall into.
And you can see where some areas
have declined or improved over the years
and identify some areas of prioritization.
And then you also can see it by items.
One thing to really look at when you get this back
is number of responses.
So sometimes you might have sent out
the Self-Assessment Survey to 50 staff to complete,
and you have four responses.
It's not really representative
of your whole staff's perception of PBIS implementation.
So you wanna be careful about making any decisions
based on what four people have said is, you know,
in place or a priority for improvement.
I believe you can also...
No, I think this one, if you wanna view multiple years,
it doesn't go side by side
'cause I think the data might be too confusing.
I think you view it in chunks.
So you see all of school year 2019-20 responses
and then down below, the next year,
I might be wrong on that,
but I can imagine this would be
a really hard visual to look at if it had,
you know, six additional columns as well.
We also provided for you in the materials
a SAS summary, or sorry, a SAS action planning format
for you to be able to dig into this data a little bit more
and determine what are your three areas
in need of development
and your three areas of major strength
and then determine what you might prioritize
for improvement over the course of the year.
So like I said, there's pros and cons to the SAS.
It does really give all staff a chance to give input
on those features specific to PBIS implementation.
So the School Climate Survey is a little bit more broad
and not directly about just PBIS implementation
versus the Self-Assessment Survey really allows you
to hone in on your different elements
of PBIS implementation.
Their perceptions are really important,
and you wanna take it with a grain of salt
because there are some items on the Self-Assessment Survey
that they probably just don't know.
So for instance, there's a question about like, you know,
is PBIS adequately resourced within the school budget.
They might just not know,
or like, is there district support or something like that.
So there's certain items that,
when your team is then looking at the responses,
you might say, okay, we just need to do a better job
of letting staff know that this is happening.
They thought it wasn't happening.
It really is.
How can we get that message out there?
And the most key piece is reporting this back to staff,
that you told us these were the priorities for change.
Here's what our team is gonna do to address that.
And that's what of the biggest indicators
of PBIS sustainability
is how often you share the data back with staff.
So that's student outcome data, but also fidelity data
and really contributes to their buy-in
to know that they were heard and that you're working on it.
So I'll pause there
and see if there's any questions about the SAS.
Okay.
Now, the School Climate Survey.
So this was fairly new, 2018.
And like I just said to Nicole,
the School Climate Survey
is not automatically opened for you
because not all schools decide to end up using it,
so it's easier for us to open it one by one.
And if you would like it to be opened,
you're gonna email Anne Dubie.
So I have a slide at the end that's gonna tell you
what you would need to let her know,
but basically, the School Climate Survey can be opened
for students, staff, and families.
So you would decide which audiences you want
to be able to complete the assessment
and when you want the window to be open for them.
Yes, so like I just mentioned,
there's students, staff, family.
Students is broken out grades three through five
or six through 12.
So it asks slightly different questions
and in slightly different ways based on the grade levels.
If you're interested in doing it for K through two,
we actually have a Google form version of the survey
that Moncton School created
and allowed us to create a template from.
So we would share that with you for you to make a copy of
and send out that link to your K through two teachers
to have them complete with their students.
So it's a simplified version of like, you know,
thumbs up, thumbs sideways, or thumbs down
for them to answer the questions.
For the students, it's either nine questions
for middle school, 11 questions for elementary school.
The staff school class survey is 29 questions,
and the family one is 21 questions.
I will say there are, I think,
two questions on the family survey that we do not love.
One of them is around...
And I think they appear on the staff survey,
and it's slightly different wording,
but one of them is around like family involvement in school.
And it's a pretty outdated vision of what
family involvement is.
So it asks something like, you know,
do you attend open house, PTA,
parent teacher conference, things like that,
and we've kind of moved beyond that vision
of that's how we're hoping
parents will interact with the school.
So depending on how your school population
feels about those questions,
it might be useful to kind of have a disclaimer
when you send out the School Climate Survey
to parents and families just explaining, you know,
question 19 to ask, blah, blah, blah.
Like, either we don't want you to answer
that question like that.
We want you to answer it X, Y, Z way,
or, you know, we recognize this is not how we think about
your involvement in our school, blah, blah, blah, something.
So you might just, you know,
kind of look over the questions before you send it out
and see if there's any that you wanna
provide a disclaimer about.
Kojia's is asking, if they're K through six,
can the sixth graders take the elementary survey?
I would have to go back and look,
but I think that it wouldn't allow them to answer.
So it'd ask them what grade they are in,
and I'm guessing that the sixth grade
might not show up as a choice
if they complete the early elementary one,
but I can look at that and get back to you.
If worse came to worse,
you could have them answer that they were fifth graders,
but then you would know
that the fifth and sixth grade responses
were grouped into fifth grade.
So it's a super easy to use survey, very quick.
We know some schools in the past or districts
have created their own School Climate Surveys,
and sometimes they are like 40 or 50 questions long.
And in some ways,
that's good because you get lots of information,
but we know the longer a survey is, the fatigue sets in,
and the students may just be clicking through
and answering questions in a way that's not authentic
to their real answer.
It also allows that multiple perspective
from staff, students, and families,
and for the students, it asks them their gender,
their grade, their race/ethnicity.
And so you can break down the results based on that,
depending on your numbers.
So if you only have, you know, one or two students
that identify as a different race or ethnicity,
it's not gonna allow you to see it that way,
but if not, you can compare,
and you might notice something like, wow,
our girls really feel safe at school,
and our boys are reporting
that they do not feel safe at school.
What's going on with that?
And that kind of comes back to that, you know,
it doesn't give you the answer.
It helps you ask better questions.
For the student version, you wanna think about
whether you're gonna require parent permission
or an opt out process.
And in that School Climate Survey manual
that I showed the picture of before,
that gives you some example forms
that you could use for that
that explain what the survey is
and how a family would opt out
if they didn't want their student to complete it.
And then you wanna be thinking about, you know,
what team is looking at results of these
and what's your timeline in terms of reviewing results
and action planning and reporting back
to whichever stakeholder group took the survey.
So like I mentioned, the student version,
they do ask them the demographic questions at the beginning.
They can opt out and say they prefer not to answer them.
Okay, so this answers that question, Kojia.
What grade are you in?
Sixth grade is included on the three through five one.
So they probably anticipated that some schools
are K through six.
So the sixth graders can complete that early elementary one.
And then those are the 11 questions for elementary.
I'm not gonna go through them right now,
but you can come back to this PowerPoint,
go back to the manual to review the questions
and see if this is an assessment you would wanna use.
And then just like the TFI and the SAS,
you're gonna get those results graphed for you
in a variety of different ways.
So you can look at them by gender,
by sexual orientation for middle high school,
grade, by item, race/ethnicity, et cetera.
The school personnel and families ones
is a little bit different the way the results come up
but same basic idea.
And the sub skills are slightly different for families,
students, and school personnel,
but in general, looking at things
like school safety, relationships between people,
and the environment.
So this is an example of a report that would come out
from the elementary version.
So you can see, over the two administrations
in March and then in November,
how many students completed it.
So you can see the second administration almost, you know,
more than double the number of students completed it.
So that's something to keep in mind
as you're reviewing the results and the average scores.
Okay, and then when you get to the scores by items,
if you hover over the number of the question,
you can see what that question was.
So it doesn't show you it down here.
You have to hover over it, but then you can see,
okay, so question nine was,
I know an adult I can talk with at school if I need help.
That was a, you know, a higher scoring one.
Question seven, students are frequently recognized
for good behavior was lower.
And make some, you know,
do some analysis around that and create some action items.
And then, just like those other assessments,
you can see the scores by item,
and in this way of viewing it,
you can see it by those subgroups,
so by race/ethnicity, by gender.
So you can actually look and see
are there certain items that a certain subgroup
scored significantly different than another subgroup on?
And then you wanna think about, you know,
what are those that were in the three to four range?
Great news, those are going really well.
Share that out.
Identify three items that are your highest priority,
and then use that item spreadsheet for more information.
So is this really a school wide issue?
Everyone scored this item low,
or is there a specific group that we wanna focus on
and get some more information from?
So for instance, you know,
if there was a subgroup that did not feel safe at school,
could we do a focus group of students from that subgroup
and really get at those feelings of lack of safety
and get to the reasons behind them?
And then you wanna,
just like you would do with your student outcome data,
like your behavior observation and data forms,
get to a really precise problem statement.
So for instance, theirs, in this example,
is students across all subgroups have reported
that students are not frequently recognized
for good behavior.
So what are we gonna do?
What is our action plan around that?
Who's responsible for it, by when?
What's our timeline,
and how are we gonna know that that plan was effective?
So Kim is asking about confidentiality.
So if there were students, you know,
only a few students in your school
who identified by a certain subgroup,
they may not want to identify themselves on the survey
for fear that their responses
will be tied back to them specifically.
And so I think you can talk about that
when you administer it.
There are those options of, you know, prefer not to answer.
So they don't have to identify themselves under a subgroup.
And if there are fewer than,
I think it's 10 students that identify under that subgroup,
you're not gonna be able to see their results
in a way that compares them to other subgroups.
So I don't know exactly how you explain that
in kid friendly language so that they will understand
about like data suppression,
but just help them understand that
if they identify as any kind of group
that is not the majority,
their responses will not be tied back to them
either if they put do not prefer
or if their group is so small.
Okay, questions about the School Climate Survey.
Okay, so we are at 10 o'clock,
so I'm gonna speedily finish up.
And I'll answer your question in one second, Monica.
So before you even look at your data,
you wanna be thinking about
are there key subgroups that you are concerned about,
are there themes or pro albums you're concerned about,
know them going in.
What could you predict based on what you know?
So maybe you know staff morale is really low.
You still decide you're gonna complete
the School Climate Survey to narrow in
on areas for priority,
but recognize kind of your current context going in
and see whether those, you know, fears or suspicions
are confirmed.
Then look at those patterns between the groups,
within the groups, across time,
and think about your PBIS data,
so maybe your attendance rates
or your office discipline referrals,
behavior observation forms
alongside those school climate data points.
So does one data point inform another data points?
So for instance, out of school suspension rates
have increased for English language learning students,
and our English language learning students
report the lowest perceptions of adult peer relationships.
So you may see some connections between data points.
Again, creating those data informed action steps
for each of these assessments,
and then thinking about how you're gonna
share this data back.
What kind of story are you gonna tell
about your PBIS implementation this year?
How are you gonna show that visually?
What graphs or charts might be helpful for people to see?
So after this webinar, think about when might be a good time
for your leadership team to complete
the Tiered Fidelity Inventory.
And like I said, totally fine if you're like, August 5th,
or sorry, not August,
August would be too late, April, April 5th.
And then make the determination,
is your school gonna complete the Self-Assessment Survey
or the School Climate Survey or both.
If you're gonna do the Self-Assessment Survey,
really think about whether you're just gonna
send this link out to staff
and expect them to complete it whenever they want,
or whether you're gonna schedule a time
for the full staff to complete the Self-Assessment Survey.
And then School Climate Survey, like I said,
you're gonna decide
do you wanna do all three of these groups,
staff, students, and families?
Do you just wanna do two or one?
When do you want that survey window to be open for?
And then email Anne and let her know.
Let's see, Kojia is asking if the same link
goes to all students.
So if you were doing just the three through six survey,
the same link would be sent to all of them,
and then they select their grade,
and that's how you know,
that's how it sorts the data into the different grades.
It is a different link for staff, students, and families,
or if your students ranged, you know, K through eight,
you would send a different link for the three through five
and the six through eight.
And Monica, the TFI and the SAS,
because they're already opened,
if you have an account on PBIS Assessment already,
they're gonna show up in your open survey windows.
If you don't already have an account on PBIS Assessment,
if you email Anne, we'll add you.
And for the School Climate Survey, like I said,
you'll email Anne to have them opened up.
Other questions, and then we are ready to close.
So if you don't have questions, feel free to log off.
Anne's going to email you after this webinar
with a link to an evaluation survey
to let us know how it went for you,
and you'll get your certificate of attendance
that way as well.
And happy to stay on and answer any questions
people might have.
Cassandra just put the link
to the School Climate Survey manual in the chatbox too.
Thanks, I'm gonna stop sharing so I can see you guys
if you have any questions,
and if not, good luck this week.
We're feeling for you.
Thanks, Heidi.
Anne, you can stop recording.