Instructor: So today we're gonna talk to,
as Wendy mentioned,
a couple of different components
and we're going to jump between them.
Hopefully I can make a smooth transition between them
and again, offer time in between each tool
for any questions that you may have.
We are recording the session,
so we do wanted to share our legal disclaimer for you.
This just means that you will have access to the recording
outside of this training session,
and we just ask that you don't share it out to the world.
As exciting as it will be, it will share confidential
and proprietary information from the UVM site.
So we do want to ensure
that that is not shared out to the world,
but please, watch it as many times as you need to,
and share it with friends within UVM.
For the agenda today,
we're gonna look at a few different tools,
rubrics, more specifically in terms of building them,
connecting them to tools, how do they work.
We're also going to look at using annotation tools
to provide detailed feedback,
and then we're going to look at a tool
called the Intelligent Agents.
Intelligent Agents is a fantastic tool
built inside of Brightspace where you can set
and go and build as you go as well.
Now, with the Intelligent Agents,
you can use these in a couple of different capacities,
just like the rubrics,
and we'll show you multiple capacities
for each of these tools as we go through our session.
If you've been in one of our sessions before,
I will go through a brief PowerPoint
just to share with you some screenshots,
but we really enjoy jumping into the environment
and working along.
So we do hope that you log into your Brightspace account
and practice the tools as we review
in your practice course or sandbox.
We are together for about an hour today
and we will have time for some questions
once we get through some of the components.
And starting with the PowerPoint, we just, again,
want to describe and express some of the tools
that we're gonna be talking about today,
and we can provide this information
to you after our training.
When we talk about the rubrics tool,
which is what we're gonna jump right into,
inside of Brightspace we have two types of rubrics
that you can build that are available,
however, you can edit the rubrics as you need to.
In terms of naming convention,
we have a holistic type of rubric
and an analytic type of rubric.
With a holistic rubric,
it's seen as a single criterion or one-dimensional.
It's used to assess participants
in their overall achievement on a specific activity or item,
and you can base it on predefined achievement levels.
Now, holistic rubrics can be used for percentage values
or just a text-only scoring method.
We'll show you what a holistic rubric looks like, but again,
once you jump into your course and start to build,
you can use them for multiple types
of evaluations and assessment.
Another rubric that we have available
is the analytic rubric.
Now, an analytic rubric in Brightspace
is a two-dimensional rubric with different levels
of achievement such as columns and assessment criterias
that you can build in a row fashion.
An analytic rubric allows you
to assess participants' achievements
based on a multitude of criterion
rather than using holistic single,
but the great thing is is with an analytic rubric,
you can build a group of criterion
or a single criterion, just like the holistic.
Now you can assign different weight values
or point values to different criteria.
You can customize, you can have no points.
So depending on what you wanna utilize a rubric for,
although we have the two types,
there's a multitude of purposes that you can use them.
Now, just to share with you a quick screenshot,
you can, again, build upon what we have in the system.
So we have a holistic and analytic rubric,
but this is just a snapshot of different ways
that you could customize those rubrics
to meet the need for your items within your course.
Now with the rubrics themselves,
you can also determine how students will view the rubric.
So you can build a rubric,
you can keep it hidden from the students,
or you can display it to the students.
It can be displayed at the very beginning where a student
will see the different levels of criteria,
the values that they need to accomplish
to achieve those levels,
or you can hide certain components of the rubric
from the student where they don't see it at all,
or they only see it once it's graded,
or they only see the feedback,
not the point value or percentage.
So there are many options in terms of rubrics.
With Brightspace,
you can add rubrics to a variety of activities.
So you can add a rubric to an assignment,
you can add it to a discussion,
and you can grade it in those tools themselves.
We'll show you how to do that.
Again, this is just a snapshot to share
where you can find some of these tools
and we can share this snapshot with you after training.
Another topic we wanna talk about which sort of aligns
with the rubrics in terms of evaluation
is an Annotation tool that we find
inside of the Assignment tool set.
When a student submits a file submission,
what that means is you've asked a student to submit a file
for grading in an assignment or assessment tool.
In a file submission,
instructors and teachers can open up
that file submission that they've submitted
and they can annotate directly onto that submission.
Now with the annotation tools, we'll show you how it works.
A file that can be submitted is, again,
up to you when you design
and craft your assignment or assessment.
In terms of the types of files students can submit,
you can submit or ask them to submit a PDF,
a Word document, a PowerPoint, a Docs,
various types of file submissions.
If you can open them on the screen,
you can annotate on them.
Now, that doesn't mean you cannot use a rubric
because you can use those together
with the annotation tools in assignments.
So as you can see on my screen,
there is a rubric available with this particular assignment.
So you can do a multitude
of different feedback functionalities.
Finally, we're gonna talk about the Intelligent Agent tool.
Now, again, we're just in the PowerPoint.
In a moment, we're gonna go out
to your site and start to build,
but an Intelligent Agent is a tool that not only monitors
how your students are progressing within your course,
but it can also give you a report
and you can also send an email to your student
to just let them know or give them a nudge
about certain things that they may have missed
or celebrate them on things
that they have accomplished. (attendee faintly speaking)
Now, some of the uses for Intelligent Agents,
just so you can start to think
about how you would like to use them,
are discovering users who have specific grade values
that may be below or above a certain level.
With the Intelligent Agent, the system will check
to see which students are logging into your course,
which ones have not logged
into their specific activities within the course,
which ones have not viewed specific content, or, again,
you could do the opposite and celebrate students
who have achieved something within their course.
Now, we'll show you how Intelligent Agents work
and how you can begin to build them.
And again, they can be used to monitor students,
they can also be used to inform students.
So we'll show you the difference between that.
With the Intelligent Agents, we use something called
a replace string or replace strings,
and what this does is it allows you
to set up an Intelligent Agent
prior to even students starting your course,
and once they accomplish or do not accomplish a specific
criteria in the Intelligent Agent, you don't have to worry.
The replace strings will put into the email the details
that are required to send a student
that email or information.
So that is the high-level PowerPoint overview
of what we're going to cover today.
I'm just gonna jump into your site
and please let me know that you can see it, just to confirm.
Attendee: Yep, you're all good.
Jackie: Perfect. I'm just gonna check the chat.
Thank you, Sarah,
letting me know that you can see it, perfect.
Now, again, I do encourage you to jump into your environment
and to practice along with us, or, again, if you prefer,
you can just follow along with my screen.
For a demonstration today,
I'm going to use a practice sandbox space,
which may look very similar to your practice sandbox spaces.
We're gonna begin with the rubrics tool.
Now the rubrics tool is built inside of the course.
It can be built in a course and copied to other courses,
if needed, and it may already be in your course.
One of the ways we can find rubrics
is going to our Course Admin tool tab here
and finding the rubrics
under our Course Administration page.
Now, I find the rubrics under the Assessment tool,
but we can also build rubrics and manage them
inside of the individual tools,
such as assignments and discussions.
And I'll show you that as well,
but for now, we're just gonna start
in the rubrics tool itself.
In my example, I have a couple of rubrics
that I've already designed for our training session today
just to share with you what it looks like.
Now, when I'm in my rubrics dashboard, if you will,
I can create a new rubric by just clicking
the blue button, New Rubric.
If I have multiple rubrics available to me,
I could search for them in the Search tab
and I have this table because I already have
some rubrics available to me within my course.
Now, again, rubrics may have
already been shared in your course,
so it's a good practice to come into the rubrics tool
just to see what's available or to begin to build.
Now with the rubric itself,
I've already built this Assignment and Discussion rubric,
and you'll notice there is this little lock here.
The lock represents this has
already been graded within this course.
So the lock represents that it cannot be changed
because it's already been graded against.
Now, to share with you what a rubric will look like,
beside each rubric that you build there is a dropdown
and it shows a pop-up screen,
which allows you to preview and to edit.
Now, again, we're coming from the end in mind,
so I've already built this rubric.
We're gonna click Edit just to share with you
what it looks like in its completion.
So I've previously built this rubric
and this is what we call an analytic type of rubric.
An analytic, if you recall,
has different criterion sets
that you can add point values to,
versus a holistic, which is either percent or text only.
For an analytic rubric, you can see
that I've added a criterion, such as content.
For each of these areas, we call these levels,
I've added a level description: meets all expectations,
meets most expectations, meet some,
and I've added the criteria for each level
within the criterion group or area.
In this example, I'm using a point value.
So for content, I have four levels,
starting at 10 points and going to 2 points.
Now, I can also add initial feedback here,
which will be seen and can be expanded
upon once you are grading the rubric.
Now, as I scroll down, because this is an analytic rubric,
I do have another group called Overall Score.
Now, the overall score in an analytic rubric is
for me to add a different component
or submission for an achievement level.
In this case, I'm using participate versus not participate,
and I'm stating when I build this rubric
that a student must achieve a minimum
of 20 or more points to be able to participate
so that I can gauge the participation in this example.
Now, that's a completed analytic rubric.
So I'm gonna close this and we're gonna start to build.
Now to build, I can click New Rubric
and it gives me this open screen.
Now, again, I can choose between analytic and holistic,
but I also have individual choices
per rubric that I'm building.
Now with an analytic rubric, if I wanna switch to holistic,
I can very easily do that by clicking
the dropdown beside the word type.
We're gonna stick to analytic for a moment.
I also have this scoring dropdown,
which allows me to choose if this analytic rubric
has no score, a point value,
or if I wanna choose custom points for each criterion level.
For the moment, I'm just gonna stick with a point value
and you'll see the check mark, just to verify that.
Wendy: Jackie, this is Wendy,
because I have a question here in the chat
that I think is related to scoring points.
Carolyn asked, "Is there a way to set a point range
for each of those levels?"
Jackie: So in the individual levels,
I can set a one-point value.
However, when I build my criterion,
I can say complete, or add my criterion here,
and then I can use, in parentheses, four to three points.
The system will not allow me
to go four to three here in the box.
It has to be a whole number,
but I can certainly explain to the students
who are viewing the rubric that they can achieve
between a certain point value.
Wendy: Thank you.
Jackie: I see Alex's question there.
"Can a rubric exists in a library that we can pull from?"
You can create a library inside of a course,
and I will refer that to Wendy.
You can create master rubrics
that may be shared between courses, and again,
that's an administrative component that you can do.
Wendy: So Jackie, at this point,
you're referring to the learning object repository?
Jackie: Rubrics, I don't know that you could put there.
You could place them in the the learning objects repository
and bring them back down.
So that is an option, yes.
Wendy: Okay.
I'm not sure if you can see
Susan Marie's comment here about the point value?
Jackie: Yes, so it is confusing,
and I do apologize, Susan Marie.
When you build the rubric,
you can suggest that this level is maybe a four-point value.
Now, when we grade the rubric,
we can override the point value.
So we can give the student
less than four or greater than four,
and that is a more of a personal component.
So in this case, the question was:
"Can you have a point range
within an individual level criterion score?"
And you can if you add the comment to the student,
but in the build of it, in the point value,
I cannot show four to three points or four to two points,
or if they achieve level four,
they're awarded between this range.
So I don't know if that further confuses you or if it helps,
but I will show you when we grade a rubric
where you can adjust the point values.
Now with the analytic rubric,
we do have criterion levels and groups here,
and we have levels across the top.
So depending on how you want to build your rubric,
we do recommend you build it in Brightspace.
If you have a rubric that exists outside of Brightspace,
you can certainly copy components of that rubric
from your laptop or computer into Brightspace
by just doing a simple copy and paste.
If you grab a rubric and upload it into Brightspace,
it's not going to connect to the tools.
So when we ask or when we build a rubric,
we build it with these components.
For this analytic, for a criterion,
this may be something like content,
and you can add any sort of text here that would meet
the need for your criterion grading for this rubric.
It's also editable in terms of the levels.
Some institutions will use levels,
some will use words such as exceeds, meets,
and all of this is editable for you
to add and create your own rubric.
If you don't need four levels,
you could always delete a level
by clicking into this trash can.
If you need more levels,
you can just hit the plus sign
to add more levels to the criterion area.
So it is customizable to what you need
for your particular rubrics.
Now, you can continue to build
and you can change the point value.
So I say that this may be 100 points if they exceed,
this one may be 80 points and so forth.
As I change my point value,
the system automatically updates so that the rubric will add
itself up as you're changing your point value.
Now, I can continue to build my criterion,
add different components,
and if I need to, I could reverse the order.
So if you're more familiar or if your lesson plans require
you to do a reverse level order, meaning if I reverse this,
I'm going to start with the lower level
and work towards the top level of my rubric criterion.
Now, I can also change the point value per criterion area.
So for example, if content is worth 100 points
and topic matter may only be worth 20 points,
I can go to the scoring points dropdown
and select Custom Points.
In that sense, that means that each criterion area
can be worth its own customized points.
So I could say topic matter is only worth 20 points,
whereas content is worth 100 points.
Once I select that, it will then, it should adjust.
Also to note, you can delete the criterions
or add more criterion to this group or criterion group,
or if I needed to, I could add another criterion group
that may deal with a different component
of what I'm grading within this topic
that I'm building the rubric for.
So for example, I could click Add Criterion Group,
and this may be completion in my example.
Again, I can build upon as I need to,
create custom points if required
because it is completely editable,
and build a way.
Now, again, as we're building an analytic type of rubric,
I do have this extra group here or overall score area.
This does not add to the point value listed above here.
This is more of a complete/incomplete,
pass/fail, extra comments,
and it's up to you to design and create as you need.
So perhaps you only need one level
or two level for extra feedback
or two levels to show completion.
Again, it's really up to your own creativity
when building your rubric.
Whether you're building a holistic or an analytic rubric,
each rubric type has its own options.
So each rubric you build,
you can set criteria as to what can be seen by the students,
what you want to reveal to them,
and each rubric has an area for a description.
So if you are collaborating together
or you're building rubrics for an entire department,
adding a description will help your colleagues
just sort of understand what this rubric could be used for,
and where it could best be suited.
Now, this description is not shared with students.
So they will not see this,
but your colleagues or other facilitators or administrators
who are using this rubric will see this description.
If you are copying this rubric from one course to another,
the description will follow this rubric.
So it will always be listed.
So once I click Close,
it'll save the description and it'll be added
to the description area within the dashboard of the rubric.
Now, just like we built an analytic,
you could go back in and build a holistic.
Again, you could give it a name.
I'm just gonna give it a Sample name,
and I could choose Holistic.
Now, when I change the type, again,
holistic is a single criterion type
of rubric that you could utilize.
And just like we saw in the analytic,
I could choose the scoring method.
So in this example, I could choose a percentage rubric,
a single criterion or a no score, which is just text only.
Now, again, I just have the levels
which are completely editable,
and I can add the different levels,
add more levels, delete the levels as you see fit.
Now, you will notice here at the top right of my screen,
I have a dropdown called Published, Archived and in Draft.
In draft means exactly that: it's in draft,
it is saved in draft, and if it's graded against,
you can still change it after that.
Published means that it is published to the course,
and once it's graded, it will be locked as it is.
And archived means it goes into an archived bin,
and it's treated as though it's still available.
You can copy an archived rubric,
but you cannot attach it to an assessment.
Just like the analytic rubric,
we do have our options here with our description.
So again, depending on the type of rubric you wanna build,
you could build it here.
Now it does tell you in your rubric area
the type of rubric that you've built,
the point value or scoring method
and the status of that rubric.
If you do see a rubric that you want
to use again and it is locked,
you can always click this dropdown,
and as we previewed, you could also copy.
So it will allow you to copy the rubric
that's already been graded against,
and then you can edit it.
Now, we also have Viewing Statistics.
Viewing statistics of the rubric allows you to see,
of a rubric that has been graded
and attached to an item within your course,
it'll show you how many students have received
the different levels or criterions or points
based on the rubric that you're viewing.
So viewing statistics just allows you to see:
How are your students engaging?
How are they being graded against within the rubric?
As a facilitator, as an administrator,
viewing the statistics really helps you
understand how students are fairing
and how instructors may be grading the rubric types.
Wendy: Jackie, this is Wendy.
We are getting a couple questions in the chat,
particularly about, before we go on much further,
is to go back to the overall score section.
Carolyn, I don't know whether you wanna ask
a specific question about the overall score?
Carolyn: Well, it looks like it's required.
I'm trying to delete it altogether on my rubric and I can't.
You said it's used for pass/fail?
So I'm just trying to figure out how the main rubric
and the overall score rubric connect to each other
and what that overall score rubric means or does.
Jackie: Yeah, great question.
So the points in this analytic rubric are totaled here
for each of the criterion levels or groups, right?
Now, the overall score itself does not necessarily connect.
So if I were to award this particular student
25 out of 30 in their rubric,
then the system sees the 25 out of 30
and gives the student a participate
because it met, it was more than 20 points.
So the overall score is not necessarily connected
to the scoring of the rubric above it,
it's gauging the student on what they achieved
on the overall rubric.
Did that mean that if they met 20 or more points,
they participated versus having five or less points
in the scoring method in the above rubric
that means they did not participate.
So the scoring is not connected to each other.
It's not automatically updated in terms of,
I'm giving them a 10, it's going to reflect a 10 here.
So I hope that helps or makes sense,
but it's an overall achievement value
that they have achieved in the above rubric
that gives them the overall score at the end.
Wendy: This is Wendy.
I feel like that could be confusing to students,
like, what do they see in their grades?
And to Carolyn's point,
it doesn't feel like you can hide it or not use it?
Jackie: In the analytic,
you cannot hide this overall score.
You could remove all of the levels, but one,
and put your own component here, add whatever this might be,
but unfortunately, we cannot remove this little box.
So it's really dependent on what you want to use it for,
with limitations is the best way I could say that.
Now, I don't know if our teams at D2L
can have this removed or share with you
how to remove this particular overall score,
but you not necessarily have to relate it to a point value.
You could just add comments here.
Wendy: Okay, I don't wanna get too much in the weeds
because we have another question here.
"We tend to use the same discussion rubric
across most non-cross courses.
Can you show us best practices
for sharing rubrics across courses?"
Jackie: Absolutely.
So we've built some rubrics here in this course.
I could go to the Course Admin because there is a rubric
that I've built in another course that I want.
So starting in the course I want to bring the rubric into,
knowing that there's a rubric somewhere else,
I could go to Course Admin
and I could go to Import / Export / Copy Components.
This allows me to search for the course that I wanna grab
the rubric from and bring into my new sandbox space.
So I would click Search for offering, find the course,
click Search.
Once the course is revealed here,
I would just select it, click Add Selected,
and then I could come down to the bottom of the screen
and Select Components, click this tab here.
And this should list if there is
a rubric within my course, the Rubrics.
There are four rubrics
within the Brightspace Resource Course.
I'm gonna Select individual items to copy,
come to the bottom, click Continue,
and it will list all of the rubrics that are available to me
from the Brightspace Resource Course
that I'm going to copy into my sandbox space course.
So I could choose all, or one, or none.
Click Continue, click Finish,
and my generic rubric is now being copied
to my sandbox space.
It should just take a moment.
That is the best practice to add a rubric to another course.
Administrators may be able to build rubrics
at different levels within your organization and share them,
but for an instructor,
you could copy from one course to another.
And here, you can see my generic rubric has been moved.
Now, in terms of that locked assignment rubric,
I can copy by just selecting the rubric
that's been locked and click Copy
and it'll copy inside of my course.
So Copy of Assignment Rubric.
From this component or dropdown
in the Copy, I can then edit,
and in this case, I can edit what I'd like.
Now, in this assignment rubric,
here, you can see where I've put in parenthesis
that if a student achieves the meets all expectations
for structure, they can achieve 10 points.
And this is an example where I've put a parenthesis,
they can achieve anywhere between seven
to nine points for this one.
Now, to share with you why I've done that,
what I'm going to do is I'm going
to go out to an assignment, see if I can find one.
Now, you'll notice, this rubric here is attached
to an assignment when I click into the rubric to grade.
So what I've done is I've gone to my Assignments tool
and I've opened up an assignment
that has been submitted by Jackie Learner.
When I built to the assignment, I added this rubric.
Now, you'll notice, because I've opened up the rubric,
it shows what the rubric has already built into it.
Here are those levels that we built.
So meets all expectations, meets most expectations.
As I'm scoring the rubric, whether I've created it or not,
it does display to me, for structure,
this is worth 10 points overall
if the student meets the expectations.
Now, you'll notice, I have in parentheses 10 points
if all required components are included.
In the rubric itself, when I'm grading,
if I hover over to the next level,
it will show me what the student needs
to meet most expectations and the overall point value.
Now, the reason I put seven to nine points is
because if a student achieves this particular level
of criterion and I'm awarding them a 7 out of 10,
it gives me the opportunity to override that seven,
which means I've informed the student,
but I'm also overriding the overall score of this level
by clicking that number and clicking this arrow up,
which means if a student achieves
a 9 out of 10 in level two,
I've explained that they could achieve
between a seven to nine.
So Susan Marie,
I don't know if that helps with this component,
but it just allows me to have a bit
more flexibility as to how I can score.
For the rubric itself, it is something that I can override
in the point value so I'm not locked in or limited
to a point value that I've built in my rubric.
Now, again, the reason we require you to build the rubric
in the Brightspace environment and attach
is so that you can see that it connects
to all of the components within this assignment.
I also have availability in the rubric to add more comments,
so I'm not, again, limited to the criterion
that I've built when I built my rubric.
So here we have this initial feedback, "Great work!"
I can add to this
my own comments within the rubric itself.
You'll also notice, as I grade my rubric,
it will automatically update the score
or overall score as I'm adding.
So if I do make a change, it automatically updates.
Now, here's what a student would see
with that overall component with the analytic rubric.
So they would see overall score is a check mark,
meaning they have complete.
They have a 10-point minimum to receive a complete value
within this overall score in an analytic rubric.
If they are incomplete, they would see
that they have not met the minimum points for this activity,
and that shows incomplete.
Now, you could tie the overall score value
complete versus incomplete when you build
this grade item in the grade book.
So that is what it looks like to grade a rubric.
Yes, I just noticed Jean's question.
I apologize if I missed another question, but yes,
you could remove the point values.
You would still have the overall grade area
in your feedback section where you're grading the item.
So if you have no point value attached, no percentage value,
you could manually enter the grade value here
and even override it, if you did have points.
So great question, Jean.
Sorry, I'm just looking at Holly's question.
"Using proficiency based grading approach,
we wanna be able to display grades and attach rubrics
to specific learning objectives
rather than to adjust a single assignment."
Yes, Brightspace does has a mastery view,
and that is using standards within your environment.
With the standards,
once you've deployed them in your course,
you can attach the standards to the rubric.
We don't have it deployed at the moment
in this particular example, but, again,
once we deploy the standards,
you can add standards to a rubric
and grade against the standard,
as well as, if you so choose,
give a point value or grade value.
So it works together.
And yes, likely a topic in the future.
What's great about that is you can have a full proficiency
or competency level or mastery skillset
that you can utilize, and again, it attaches to the tools.
Now, in terms of assignments and discussions,
when you're attaching a rubric to an assignment
or discussion, when you build your assignment,
and I'm just gonna use this assignment example,
and select Edit Assignment.
To add a rubric to an assignment,
you would come down to the assignment build.
As we can see, I've started to build this assignment.
And under Evaluation & Feedback,
once you select into that area,
it allows you to add a rubric.
Now, if you were to use objectives in the future,
you could manage your objectives here
in your Evaluation & Feedback,
and again, attach it to the rubric itself.
In our case today, we can just click Add a Rubric.
Now, again, we mentioned that you could
build the rubrics in the rubrics tool,
but you can also build a rubric in the other tools.
So as an example,
when I select into the Evaluation & Feedback area here
and select Add Rubric,
I could create a new rubric directly
from the assignments area for Assignment 2
or add an existing rubric.
If I select Add Existing,
it takes me to the dashboard of rubrics I've already built.
If I have a long list,
I can always click this Preview tab
to see what the rubric looks like before I attach it
to that particular tool, or assignment.
Go back.
So adding a rubric to an assignment
is a simple couple of clicks.
Once I add an existing, again,
I'm just gonna use this generic one that I copied,
click Add Selected,
and it's now added into my assignment build.
Now to grade that,
you'll notice for Assignment 2,
I do have a student who has submitted.
In this particular assignment,
there is a rubric attached now and it is a file submission.
So before we move to that,
I just wanna ensure that we have some rubric questions
answered or some that we could work on.
Wendy: I think there's still some confusion
about total score and overall score.
Jackie: So overall score, there's no scoring connected.
It's up to the instructor who builds the rubric
to add that overall score.
Let me see if I can share or demonstrate.
Wendy: And maybe you could
describe a use case and why that,
like, what's the value, I guess.
I think that's what I'm trying to figure out.
What's the value-add there?
Jackie: So this is an assignment rubric,
and here you can see this is what a student
might see before it's been graded.
So the student sees the level of criterion,
what they need to achieve to get the point value.
The overall score for a student is basically saying,
if you met a specific point value,
we see that as a complete versus an incomplete.
So either you did or you did not.
Now, that's just this example.
In the overall score,
it doesn't connect necessarily to these points.
So you can't add any extra points
to this value using the overall score.
It's just, you can add extra feedback.
If the student is a different level,
you could add it to your overall score.
Perhaps you're using this as an example
where you have your criterion listed here
in your upper portion or your criterion level,
and you're using your overall score for extra feedback
for a proficiency that they need to meet.
There's multiple ways in which you could look at this.
It's not meant to confuse,
it's just meant to say to the student,
you have met or did not meet this, here's where you land.
I'm not sure if that helps or just causes more confusion.
(chuckles) There are different ways
in which you could use it.
Now, I, in my example, just put complete or incomplete.
In your build, if I go to New Rubric,
you can have four levels, 10 levels of an overall score.
Perhaps this is a component that adds to other assignments
to build towards a final result.
So even here, you could put something like Assignment 1
and a score that they've achieved on Assignment 1.
Again, overall score is just to inform
the students of how they did overall
with this particular rubric or assignment.
If you do not want to use the overall score,
you could certainly flip to a holistic.
The difference with the holistic is
there's no criterion listed here.
So you could, I guess, get creative
with how you wanted to use that overall score,
and we could certainly check with our implementation team
at D2L to see if we can remove that.
I haven't heard of it yet, but you never know.
Wendy: Yeah, we'll put that.
We have a standing meeting
with those folks on Thursdays too.
So we can put that on our question log, thank you.
Jackie: So yes, overall score may be a mechanism
for pegging global feedback.
It may be adding extra comments.
It's up to your design to be able to do that.
It doesn't connect directly to the point value
in the above analytic rubric, so.
Yes, you could.
Again, we'll check with our teams at D2L
to see if it's something we can remove,
but you can just leave it to one option,
one level and just use it for comments.
There's many different things that you could do.
Now, very quickly,
if you were attaching this to a discussion,
so a rubric to a discussion,
it works very similarly to the Assignments tool,
however, it's found in a different space.
So if I go to my Discussions area
and I've pre-built a discussion for us today,
I would just go, when I'm building the discussion
or if I wanna go back and assign a rubric to that later,
I would come down to the discussion topic
or, again, in my build,
and in this case, I'll click Edit Topic.
But whether you're editing and adding later
or building from scratch,
you can add the rubric to a discussion
using the Assessment tab here.
So once I click on Assessment,
again, just like we saw in the Assignments tool,
it will give me an option to add a rubric.
When I click the Add Rubric tab,
I could choose the rubric I want to add
for this particular discussion activity
and click Add Selected.
Now with the discussions, although you've added a rubric,
you can still be very specific with how you're going
to be grading individual assessments.
So it can be used together
and you can allow different assessments
for different calculation methods of posts.
The reason I point that out is you may have a student
that you're grading overall for a discussion topic,
but they might be very active,
and every day they're responding to another person.
So you could use this overall assessment
or calculation method to assess separately.
So just before we forget.
Now, the Annotation tool.
Annotation tool is connected
to a file submission type of assignment,
and again, I'm coming to this example
in my sandbox of an Assignment 2
that a student has submitted a file to be graded.
To get to that, I'm just going to go
into my assignment dashboard here
and click on this number one.
I could also obtain this from going to my quick Eval tool
or my grade book or from a variety of different spaces.
For our example,
I'm using the Assignment tool to capture this assignment.
This student has submitted a file assignment.
So I'm going to come to this view or this Feedback view
and it shows me the document is available here.
When I click into this document,
it will open it up because it is a file submission.
So the student has uploaded this particular document to me.
Once I open it up,
you'll notice I still have an availability
to add an overall grade here, add some feedback,
and again, if I had a rubric attached,
I would also be able to grade with the rubric.
Apart from that, this is a file submission
and it does allow me to open it
where I can annotate directly onto this file submission.
When I open up my Annotation screen,
it does allow me to use thumbnails, flip between pages,
use my Settings tab or page layout so I can display this
in pages across the board or fit to the page.
I can make it larger or smaller.
In terms of annotation,
I do have this annotation tool that I can utilize.
When I select this Annotation tool,
a dropdown appears where I could choose
to draw on the document,
highlight, whether freeform or text,
or erase my annotations.
When it comes to drawing,
you would simply click on the Drawing tab
and it will allow you to choose a color
that you want to draw on.
It will allow you to use a point value or a darkness,
or sorry, thickness of that annotation
and a way in which that you could annotate
in different formats.
Now, once you've chosen your color and text size,
you can then begin to annotate.
Now, you may also notice that I'm not a very good artist,
so I'm gonna come down to this Annotation tool,
choose Erase and remove some of these annotations.
As we mentioned, you can also either use
a freeform highlight or text highlight onto the document
that the student has submitted.
So if I choose Text Highlight, again, I can choose a color,
come to the area, click my mouse,
or use my keypad on my computer, highlight what I'd like.
Once I've highlighted, I just select off with my mouse
or click my Enter on my keypad and it is highlighted.
Another tool with annotation is
what we call the Note Feature.
The note feature, when I select it,
allows me to choose a symbol
and add a note to this particular annotation.
So for example, if I select the Note Feature,
I can again choose a color.
The reason we have the colors is
you could highlight something in a color
and then connect a note feature to that,
where it just speaks to that area you've highlighted.
In my example, I'm just gonna use this sticky note,
select it and click onto this file where I want
my sticky note to appear to my student.
When I click onto the document,
it allows me to add a note to my sticky note.
So I'm just gonna add my note,
click the X in my sticky note feature,
and when the student downloads this annotation,
they'll be able to hover over the sticky note or the symbol
and see the comment that I've made.
Again, I can just click on my Annotation tool,
use a text highlighter, perhaps choose a different color,
highlight something.
And then I can choose a symbol,
change the color to match the highlight, add my note.
Now with the annotation tools, you can also draw a text box,
use this drawing feature for lines, arrows, and rectangular.
It's really up to you.
Students as well as instructors will have the availability
to select onto this tab here where they can
download the original uploaded document
or download the document with annotations.
It would be downloaded to their computer screen
that's attached to their D2L system,
so their laptop, wherever they can download,
and they can always go back and refer to this.
Once they've downloaded it,
they can see the annotations that are attached.
You can also add your overall grade, as we mentioned,
and overall feedback.
And if you did have a rubric,
you could open that up and grade it from the rubric itself.
So I'm just gonna pause and see how we're doing.
Any questions on that?
Wendy: The chat seems a bit quieter,
but does anyone wanna speak a question aloud?
Okay, I think we can move on.
Jackie: All right.
So the final tool we wanna share with you,
I silenced everyone with rubrics, I apologize,
now I'm going to wow you with excitement.
So the final tool we wanna share
with you today is the Intelligent Agent.
Now, to get to the Intelligent Agent at the course level,
I'm gonna go back to my Course Admin tool tab
and I'm gonna find Intelligent Agents
under my Communication area.
An Intelligent Agent is an agent that you can create
to capture how your students are doing
and capture them not doing something.
With an Intelligent Agent,
you would just come to your Agent view, and again,
shiny blue button, you can select New Agent.
Now, I'll share with you what it does first,
and we'll build an agent to show you what you can
actually capture or what agents you can build.
When I clicked into New Agent, I can give this agent a name,
and I'm just gonna call this, We Missed You.
Now, the titling and name and convention again is up to you.
Intelligent Agents can also be copied
from course to course to course.
So if you create some Intelligent Agents in your sandbox,
you can certainly move them
to your active courses at a later time.
Now with the agent, we do require a name.
You can add a description.
I do recommend adding descriptions because you may create
several different agents within your course
and you can create categorization or categories
for these agents to be placed and just organized.
Now with the status,
the status is asking us to enable or disable this agent.
Enabling means that as soon as you complete building
your agent and click Save and Close,
the system will automatically look
at the students within your course
and see if any of those students have met
the criteria of this agent that you've built.
My best practice is to not enable it yet,
but you're welcome to test this out.
In terms of the agent,
what they do is they read the data
of the agent that you have created.
They look to see any students within your course
that is attached with this agent
and see if they've met the criteria within the agent.
If they have or have not met the criteria
based on how you've built the agent,
the system will automatically send an email to the student
or build a report about how the agent is running
and which students did not
or did meet the criteria of the agent.
With that being said, you can build a schedule for an agent,
meaning you can set this up and the system will
automatically gather the information and criteria and run
behind the scenes without you having to do anything else.
Scheduling can be one time.
So you're only going to send an agent to the students once
if they meet or do not meet the criteria you've built.
You can run an agent hourly, daily,
weekly, monthly, or even every year.
Now, we set the criteria for the agent.
First and foremost, we'll look at,
which of the people within my class
are we going to build this agent for?
Visible in the classlist means that they are visible
to other students within the course.
So I'm gonna put Learner here.
Visible means that a super admin,
admin, designer, and various roles,
although they are enrolled in every course,
they're not visible.
Typically a visible role would be a learner
or an instructor, a facilitator, course builder or a TA,
but for this, we just wanna capture
the learners who have missed.
Now, we have different criteria that you could set
for different types of agents.
You can take action and build this Intelligent Agent
to gather and inform students if they have not logged in
to the system in Brightspace in a specific amount of days
or have logged in and you wanna celebrate.
So that's the first Intelligent Agent you could build.
The one we're gonna build is around course activity.
So I want to capture and send an email to all those students
using an Intelligent Agent who have not logged in
into my course in the last five days.
You could send an Intelligent Agent
to students that have access your course.
Again, it's about choosing which this Intelligent Agent
is going to be used for.
Now, we're not limited to log in
the system activity or course activity.
You could also create an Intelligent Agent
using those release conditions.
So for example, instead of capturing
all students and sending an email
to those who have not accessed the course,
I may build a separate Intelligent Agent
that has to do with a score on an associated rubric
or I could create an agent for students
that have completed their checklist.
Perhaps they have not visited any of the content.
I could send them an Intelligent Agent to remind them
that they have to go in and actually look at content.
Perhaps I could build an Intelligent Agent based on a score
that a student has received on a quiz.
A good example of that is maybe a student has failed a quiz
and we want to give them information about their next steps.
I could create an Intelligent Agent
that sends an email to a student to say,
these are your next steps
based on the score you've received.
So those are some of the examples of an Intelligent Agent.
However, we're gonna stick to this course activity
where I'm gonna build an Intelligent Agent
to capture and send an email to students
who have not accessed their course in five days.
Once I've decided on the activity
that the Intelligent Agent will take,
I can then choose, how often do I wanna repeat this?
In this example, I'm building an Intelligent Agent
to capture students who have not logged
into my course in five days.
So I have to decide,
do I want the system to send a new agent
every time this happens to an individual student,
or group of students, or just once?
In my case, I'm only gonna take action once
and hope that this agent will help them get back
into the system and back into my course.
You could take action every time
a student does not enter the course on a five-day basis,
but I'm just gonna do it once.
Now, I'm going to use this Intelligent Agent
to send an email to the students within my course
that do not meet the five-day criterion that I've chosen.
For that, I'm gonna select into this area here
called Send an Email and just select Send Email.
This allows me to create an email that's going
to automatically go to any of the students
who have not entered my course in five days.
And here's where I can use what we call replace strings.
A replace string starts with what we call
a banner or squiggly bracket.
In the To field,
I'm going to put a replace string called [InitiatingUser].
The Brightspace system will acknowledge that the student
that does not meet the criteria in the Intelligent Agent
will put the user's name in this replace string.
I can then add a subject.
So I can say, We Missed You,
and I'm gonna use another replace string, [OrgUnitName].
Now, using this replace string
will automatically tell the system,
when they send the email to the student
who does not meet the criteria,
this replace string will place
the name of the course in this field.
So when the student receives this email,
it will say, We Missed You in sandbox space.
If you're curious about other replace strings
that you can use, I will put this list in the chat for you,
but it's also listed in the Intelligent Agents.
Let me know if you receive that.
The nice thing with the replace strings is, again,
you can really customize the email that the student
will receive for missing your course.
So I'm gonna use another one.
You can even use replace strings
such as, when was their last login date?
So what I could do is I could copy
this replace string from this pop-up screen,
paste it here, correct my spelling,
and continue to build.
Now, if you do not choose to send emails to students,
you could still build the replace or, sorry,
the Intelligent Agent without having
to actually send an email.
The system, if you decide not to send an email,
will gather the data from the agent.
So for this email, I've completed it.
I'm gonna Save and Close,
and it's going to bring me back
to my Intelligent Agent list.
Now, you'll notice, I gave it a title.
And I did give it a description, and it is not enabled,
which means the system is not gathering the data
until I enable the agent.
So what I would need to do is just select
on the agent I've created and select Enable.
Once I've enabled it,
what the system is going to do is go to the class
and see which of the students within my class
have not logged in in the last five days.
It's going to run that result for me and give me a report.
If I wanna check to see if students
are missing my class and have not logged in
but I'm not ready to run a report and send emails,
I can come to the agent, click the dropdown beside it,
and I can do something we call a practice run.
Once I select a Practice Run,
it's going to provide me a report without informing
any of the students that this is happening.
In this manner, I can receive the details
without having to send those emails
and I can email or connect with the students myself.
Now, you can build agents for a variety of purposes.
We had just enough time to share with you
how it works and what you could do with it,
but it is a great way for you to build these agents
and just let them take hold
with the data within your course.
Wendy: Jackie, are you able to copy agents
from course to course?
Jackie: Yes.
Wendy: Great.
Jackie: So again, if you go to Course Admin,
Import/Export, if an agent exists,
you should be able to copy it to another course.
Wendy: And might you have time for one more question
that I missed before we went to Intelligent Agents?
Jackie: Mm-hmm.
Wendy: It was from Gene at 123,
and Gene prefers to use letter grades
to avoid equivalent over minor percent variations.
"Will there be an option to hide point values
so that students only see letter grades?"
And then she also has a follow up.
"Is it possible to see the student view,
what they see when they look
at our feedback, annotation, rubrics, grades?"
Jackie: Yes, so to answer the last question first,
if you go into the grade book, so if I go to Grades,
you can go to a student in the grade book,
click the dropdown, and you'll see Preview.
When you open up that Preview view,
it shows you what the student sees.
Now in my example, I do not have any feedback graded yet.
If the student receives feedback
through an annotation in an assignment,
it will list here a link called inline feedback.
As the instructor,
when you're in the Preview view of the student,
you can click the link and it'll show you what the student
sees in terms of those annotations.
If you've graded a rubric for this student,
again in the Preview view,
you can click Open the Rubric to see what the student sees.
So again, I would encourage you to come to the grades,
go to an individual student
and preview to see what they see.
In terms of your letter grades,
the rubric is built on a point value,
a percentage value or text only.
You could create a text only or no score rubric,
either in analytic or holistic and add the A, B, C, or D,
or you could add that scheme
to the grade item in the grade book.
So for example, if you're using
a point value rubric for Assignment 2,
it will grade the point value.
When you build that item in the grade book,
you can choose the scheme of a letter grade.
So the student will see the letter grade
front-facing in their grade book based on the grade item
versus the point value in the rubric.
If you choose to hide the rubric, they won't see it at all.
They'll just see the grade
that they've achieved within the assignment.
And if you build the rubric,
you can also say that the student will not see
the point value or letter grade,
they'll only see the feedback comments.
So when you go to your rubrics,
I'm just gonna go to this one,
you could go to your Options
at the very bottom of your rubric build,
and you can say, hidden from students, include feedback,
hidden until feedback is published and hide scores.
So depending on how you build that
and what your options are.
So hopefully that helped.
We covered a lot of heavy topics today,
so I am very grateful that you stuck it out with me.
Wendy: So thank you all for showing up today,
and Jackie, thanks so much for your time.
Jackie: Of course.