-: Hello and welcome to the video lecture on experiments
and the use of experiments in social science research.
So we're going to do an introduction to tell you about,
for the purpose of this class,
what are the three elements of a classical experiment.
We're gonna talk a bit about how to select subjects,
some of the validity issues in experiments,
and last, what are their strengths and their weaknesses?
So there's many, many ways
in which we informally do experiment.
We do something and we see what the actions are.
We taste the soup and see if it needs more spice.
We go outside
and see if we need an extra layer or hat or gloves.
But in social science research,
they tend to be very sort of narrow in scope,
that we really want to narrow down on a cause
and an effect.
It's not good for exploratory...
It's good for sort of seeing the effect
of a specific intervention,
like people who are exposed to some things
are more or less likely to take some behavior.
So the three parts,
and we'll go through each one,
are independent and dependent variables,
pre and post testing,
and having experimental and control groups.
So one way of thinking about it
is the independent is the cause.
What is the treatment or the stimulus?
And in some cases it either may happen
or it doesn't happen,
and the dependent variable is the response.
What's the sort of behavior that we are looking at
as a result or as a response.
The second element is pre and post-tests,
where you measure
sort of where someone is via a pretest,
you expose them to some sort of treatment
and then you post-test,
and see if there's a change.
Last is the experimental and the control group,
where the experimental group gets it
and the control doesn't.
Think about, why should the experimental group
resemble the control groups?
Why should these groups sort of resemble each other
in all feasible ways?
And why even have a control group?
Why not just do a test,
do an intervention,
do another test, see if there's a change?
Why do you need an intervention?
What would be an example
of how we could use this in our class project?
What would be each of these things?
And we'll discuss this in class.
In many cases you do a double-blind,
where the researcher doesn't know
what's the control group and what is the experimental group,
and they often do this in medicine,
where one group gets a pill,
say that they think will actually treat the condition,
and the other control group gets a placebo,
and neither group knows who is who and why does that matter?
What would be the effect?
You know, it's clear that if I know I'm getting a pill,
that I'll think I'll get better and I get better,
and if I don't get it, I'll get worse.
But why is it important that even the doctors
or the nurses not know?
When would it be important?
What are some of the behaviors that the doctor
or the nurse say, might take, that would mess things up?
Very often, folks like you,
college students, are used in selecting subjects.
What are the strengths and weaknesses of this?
The strengths are that you're available,
you're probably curious
and it might sound like fun,
and also if we pay you,
many of you would appreciate the money.
The weakness is,
there's no way to generalize to larger populations.
Experiments can have validity issues.
Can you really...
So like internally,
can we really sort of draw the conclusion
that this treatment had an effect?
Sometimes there can be historical events
which sort of blow everything out of the water.
So if had done pre and post tests on contagious diseases
and right in the middle of the treatment
COVID or another pandemic hit, you know,
that sort of blows out of the water
Any sort of like training course.
Same with flood preparedness.
Here in Vermont we've had many, many floods,
like so-called (indistinct) floods.
So the effect of watching a video say,
on flood preparedness would not compare to so many farmers
and others who actually lost their home.
And there's just sort of the natural maturation
of your subjects,
as well as sometimes, the test itself might change,
that we might learn that this just wasn't
a good way to do it,
and we sort of change the course in midstream.
I have used experimental auctions
to measure how much people will pay
for certain product attributes.
And there's sort of the artificiality of it,
that there's like a wealth effect.
Do you spend sort of the funny money
that you get in an experiment
in the same way that you would say, in a grocery store?
And as with an auction,
like, we liked to win and we liked to get the thing
and will this auction really reveal
what somebody will pay?
And not, "I just kept bidding
so that I would win, because it's fun to win."
There also can be a concern
that the pretest itself can be a type of treatment
or stimulus.
And here is that the so-called Solomon Four Group design,
where only half of the group gets a pretest.
So there's sort of two treatments,
the actual sort of pretest itself,
the actual treatment itself, but also the pretest itself.
So for example, if I gave a pretest
on statistical knowledge
and then went and gave one group,
half of the class, some sort of tutorial,
and one group didn't get it,
that if you did very poorly on the pretest,
you might sort of, you know, work a lot harder,
study more, get outside resources,
as opposed to those who did well,
and any effect of the change from pre to post-test
could be attributed to that work
and not this sort of special tutorial.
So breaking it up into four groups
isolates the effect of each one.
So the strengths of experiment
is the ability to tightly control things
and carefully measure the effect of things.
They can be fairly low cost and not too time consuming.
It can be done over and over again,
and it has a sort of logical rigor.
The downside is that's our artificiality,
that we really will wanna know
how will someone behave in the real world,
and by doing it in a sort of closely scripted experiment
that may not exactly measure what we want.
So here are the key takeaways of this video lecture.
Thank you.