-: Hello and welcome to the video lecture on evaluation
where you will learn about how to use many of the methods
and tools that we're learning in this class
for the specific purpose of evaluating
something like a program to see how it's doing.
Is it worthwhile? How could it be done better, et cetera.
By way of agenda,
I'm going to talk a little bit about evaluation
and two types and four main topics.
And then, give the social context of why,
what you learn and report in an evaluation
may not be used in ways that you think is right,
as well as the use of social indicators,
which are sort of data that can tell us something
about the wellbeing or the status of a community.
And that can also form another set of information or data
which can be used in evaluation or in research in general.
So, this is a research methods class,
but evaluation isn't so much a set of methods
as it's a purpose to understanding
that the benefit of the impact of the effectiveness
of some sort of intervention or program.
There's two main types.
First, summative is overall, what's its merit,
what was its impact in many cases, should it be done or not?
Was it good? Was it worthwhile?
And formative tells how it can be done better.
What changes could be made
in this program that would make it more effective.
So, here are four common topics.
Note that they are not mutually exclusive
and I will talk a bit about each one.
So, first, needs assessment.
And I would also put asset mapping hint to this,
which tells that the beginning state of things,
defining what are the problems,
what are the assets that are in play.
This usually should be done with a community partner.
It's a good way to build relationships
and understand their perspective on things.
And this tends to be formative
in that it says what should be done and how.
Cost benefit analysis looks at,
what are the costs and what are the benefits?
And do the benefits outweigh what it costs?
The cost might be time,
it might be money, maybe be opportunity costs.
And these are often summative.
What's the bang for the buck? Does it pay for itself?
Monitoring provides a flow of information.
What's happening as we're doing this,
not just at the end, but as we do it.
Well, what are we learning here?
This tends to be formative,
what works, and what needs to be improved?
Monitoring can be both the implementation.
Is it happening as it was intended?
And if not, why not? As well as the process.
Are all voices being heard?
Do all key stakeholders have a place?
Are their needs being met?
And last, program assessment.
Does the intervention provide the results?
Did a job training result in employment?
Did a membership drive bring members?
What is the impact? What is the outcome?
And this can be either summative or formative.
Here are a set of social indicators that are often used.
So, this was a well-known study
from the 1940s where, "As You Sow",
where a researcher looked at two communities in California,
two farm, agricultural farmer-based rural towns.
One that had a large number of smaller farms
and one that had a smaller number of larger farms
that mostly had employees.
And looked at a number of these social indicators,
things like all these things that you see here
and looked at how do these two measure?
And not surprisingly,
found that the town with a larger number
of owner-operated smaller farms
had scored much better in these
than the town with just a few large farms.
One of the key indicators
of wellbeing here is income inequality.
And here, you see draft,
the degree of income inequality
and an index of social indicators.
And I think you can see how things stand.
This is a few years ago,
but gives you an idea of how things like this can be used.
Note too that just because that you might give
what you think are amazing results and they're not used.
First, the researcher has more control
for the most part of the parts of the thing
that they're examining than the evaluator does.
That many times, life is more uncontrollable
than just an experiment or a survey.
You cannot often assign treatment and control groups.
The treatment might change in midstream
due to lack of funding, due to social changes.
And in many cases, the results
that you might think you have clear results,
but they're not used, 'cause they're not understood,
because they are a challenge to authority
that the vested interests might block and oppose it.
So, even more than research,
it may not be that just because you could come up
with great findings in an evaluation, they may not be used.
And this is what we did.
Here are the key takeaways and a recap. Thank you.