[Instructor] Hello, everyone.
And welcome to the first recorded lecture
of CDAE 250 for this fall.
And this class, we're gonna be talking about
how do we know things
and some basic research terminology and vocabulary
that we'll be using throughout the semester.
So what is social science research?
That's what we'll be doing this semester.
And this quotation talks about
how it is the study of human behavior.
And I want to emphasize that it is more
the study of groups of individuals.
It's the study of social units and groups
more than it is the study of individual behavior,
which is more like psychology,
but the idea being to understand patterns of behavior,
norms, rules, expectations, behaviors of groups.
That tends to be the kind of topics
that social science addresses.
So if we're gonna understand things,
hopefully, we do so correctly.
And how do we know things?
How do we know things are true?
How do we learn things?
And how do we know what is credible?
I think it's fair to say
that there's a lot of misinformation, half truths,
outright lies being told
out in the the world all the time.
Sometimes the person who tells you
this believes it true, sometimes not.
So how do we know?
How do we know what is credible?
That's a really big objective of mine for all of you,
is to walk out of this class really having
a very finely-honed,
as I said in the first class, BS detector,
that you can really have the skills and knowledge
to be able to judge the credibility of statements
and of research.
And part of knowing what's credible
is to know what mistakes do we make
and why do we make them?
One way of thinking about how we know things
is to look at these two options.
Sometimes we know things from our own experience,
sometimes we know things from tradition or authority.
And in each case, this is not social science research.
Neither of these is sort of a rigorous process
that we go through to discover information and meaning.
So there are things that you've learned
about UVM for your own experience.
I assume that all of you have been
in face-to-face classrooms,
and the very first time you went to that classroom,
it might have taken you a bit of time,
but soon after you've been there
and probably by the second or third week
through your own experience going there
that you really don't even have to think about it anymore,
that you know where the classroom is,
where class meets, and you just do it.
There are also many things
that we learn by tradition or authority.
So when I think about tradition,
maybe one really simple way
of thinking about this is cultural norms.
So there are certain behaviors
that when you're in a face-to-face classroom
that are acceptable and that are not acceptable.
If you sit in a chair, write notes,
sit fairly quietly and still,
tradition says that that's sort of what the expected
and acceptable behavior of a student is.
If you would stand up and dance,
stand on your seat, turn around wildly,
tap your fellow students on the shoulder,
bump them, do all sort of distracting things like that,
it's doubtful that you learn that by your own experience.
It's doubtful that you ever read a manual
that said don't do those things.
You just sort of learned it through tradition.
In the same way, much of what we learn is by authority.
And frankly, in this class,
I am an authority on this subject matter.
I hold a PhD.
I have 10 years of experience teaching this class.
I have almost 20 years of experience
doing social science research.
So I have earned the right, hopefully,
in your mind to be credible when I tell you
what are effective and credible methods
for carrying out social science research.
But even beyond this, sometimes we make mistakes.
Now I'm going to walk you through some of the common errors
that we make both in our everyday lives
and in conducting research.
The three of them, and I'm going to go through each one,
are inaccurate observation,
selective observation, and illogical reasoning.
And we humans are prone to all three of these things.
The first of these is inaccurate observation.
That we very often just fail to pay attention.
That sometimes the details of things
simply are not important.
So if I were to ask you,
assuming that at some point during the day, yesterday,
you were out and about,
you were walking around town, doing things like that,
I could ask you, how many blue cars did you pass?
How many blue cars did you see while you were outside?
Since this wasn't an important thing,
not anything that you really had your mind on,
I doubt if any of you
could give me anything approaching
an accurate estimate even.
But if I asked you, tomorrow,
go outside and count how many blue cars
that you see during the the day.
I'm not asking you to do this.
This is just an example.
I think that you would by paying attention,
and if I was grading you on it,
you would think it was important,
and you would have a much more accurate number
of the blue cars that you passed.
But many things in life,
we just sort of don't pay attention to 'em
because they don't seem important at the time.
One of the most common one is selective observation.
We humans are very good at seeing patterns,
and that has served us well throughout our evolution.
However, we tend to look for events
that fit a pattern,
that once we have established a pattern,
we're much more likely to see things that fit that pattern.
So the idea of if all my life,
all I've ever seen were white swans,
that I might conclude that all swans are white.
And if I see a white swan, I'm like,
see, look, there's a white swan.
There's a swan, it's white.
That means all swans are white.
And this was actually an example
that was developed by a philosopher
of science named Gunnar Myrdal
who used this black and white swan thing.
But we tend to do that,
that we tend to only notice things
that fit our patterns and we discount things
that don't fit our pattern.
And when we see something that fits the pattern
that we have, we say, aha, see, I'm right.
It's a very natural common way that our brains work.
But it's one thing that we really have to be aware of
and cautious of in social science research.
And this is very closely related to a very common bias
that we all have, social scientists and not,
called confirmation bias.
And I'm gonna speak to you a bit about that now.
So confirmation bias is that we give preferential treatment
to evidence that supports our preexisting belief,
and we tend to look for cases or evidence or examples
that fit what we already think,
and we tend to overweigh them,
that we put a lot of credibility
in things that we think are true,
and we tend to easily discount and dismiss things
that don't fit into this pattern.
And I think it's obvious
in today's highly politicized environment
that if you were to talk to a Donald Trump supporter
who watches Fox News,
and you show them something that shows
that Donald Trump has done a very good job,
that you would be like, yes, see, it's true.
This is true.
This fits what I'm saying.
And this proves, see, Donald Trump is doing a great job.
Whereas if you're more on the left,
if you are not a fan of President Trump,
and you oppose much or all of what he does, you will say,
well, this, you know,
this really isn't true or this isn't important
or this, you know, it's true,
but there's all these other reasons that counteract this.
That you will tend to dismiss some bit of evidence
that claims that Donald Trump is doing a good job.
And we are very, very prone to this.
We all are.
Don't feel like it makes us bad or anything like that.
We all do, but it's very important that we are aware of it.
So here is a paragraph with the URL
if you would like to know more about confirmation bias,
and that is that we seek out things that we seek out,
and we put credibility into things
that support what we already think,
and we dismiss or overlook things
that do not conform to our preexisting beliefs.
And in a few minutes and during our discussion in class,
I wanna dive into why this could be a problem.
So hold that thought.
You're gonna see confirmation bias again, hint, hint.
Another common error is illogical reasoning
like the so-called gambler's fallacy.
That if you've lost five straight hands,
and you're almost out of money
that you think, well,
the odds are that I'm going to win this time.
That you sort of skew the odds to think that, well,
I've lost so many straight times,
now it's my turn, and I should make a big bet here.
Whereas in almost any sort of gambling scenario,
the odds are what they are,
that they don't change whether you have won
a whole bunch in a row or lost a whole bunch in a row.
Your odds of winning lottery ticket do not have anything
to do with whether you have bought a whole lot of ones
that won or a whole bunch of ones that lost or a mixture.
The odds don't change.
In the same way, in a more simple example,
if you take a fair coin and flip it 10 times
and every single time that its heads, you might think,
well, it's been 10 in a row heads, I'm due for a tails now.
But the actual probability of heads
or tails on that particular flip is still 50/50.
I talked about the problems.
Now I'm going to talk about the solutions.
What can we as social scientists do
to overcome these common errors?
So first is intent.
Much of the intent, as I said, count blue cars,
to specifically pay attention,
to really do a good job of paying attention
and to have the intention of doing the next two points here.
First is to use recording devices, to take notes as we go.
So if you were counting blue cars,
you might wanna bring a tablet
and a piece of paper and just make slashes as you go.
In many cases, when you're doing interviews,
you may wish to audio or even video record the subject,
or at very least, take notes as you go,
or maybe all three,
so that, you know, much less is lost.
It's also important to look for deviant cases.
Maybe the example of you wanna know
what the people in a certain area think about a policy.
Maybe what do students think
about the reopening of the university?
So if you go and you keep talking to the same people
and hearing the same thing,
you might be getting into that all swans are white thing.
So you may wanna think about, well,
who is it out there
that might have a different point of view?
Seek out that person, hear their point of view.
Sort of seek out this deviant case
that doesn't fit in with what you've been hearing
most commonly so far.
Another very important part of social science research,
especially when it's done as a profession as I do,
is to use peer review.
And we're gonna talk more about that in a few classes.
But the idea being that I show my work
to other experts in my field, my peers.
They read my work, they think about it,
they look at what I've done, was it well done?
Are my methods sound?
Are my results?
Do they make sense?
Are the conclusions I make on these results
logical and sensible?
And so by having outsiders, sort of,
a set of fresh eyes,
but who are also experts in the field, reviewing my work,
hopefully that means that any work that I would report,
once it has their sort of stamp of approval,
will be much more credible as a result.
And one of the most important things to address,
especially social desirability bias or rather, sorry,
that's another bias that we're gonna talk about.
One of the most important things that we can do
to deal with confirmation bias is reflexivity.
And I'm gonna show you a slide about that.
Here is a slide that talks about reflexivity
or reflexive thinking.
And I encourage you to read it
and go to the cited link as well.
The bottom line, for right now,
is that it's really important for you
to do your best to step outside of yourself
and see yourself sort of from the outside,
and think about what are the biases,
what are the assumptions that I bring?
Well, what am I more likely to think
and see based on my own preconceptions?
Because as this paragraph says,
none of us are completely objective.
And by very deliberately examining and owning our biases,
it makes it much more likely that we will not fall prey
as much to confirmation bias
and will be more open-minded
about evidence that may contradict
our preexisting notions,
and therefore, do a much better job at, you know,
telling a more fair and full and credible account
of the research issue that we are working on.
Here are two cartoons, which I think spell things out well,
especially the one, the top middle.
Like, you see something, it's like click on it,
look, it says what I thought it was, jackpot.
So in the midst of our discussion
when we are next together remotely,
I want to explore a little bit more about confirmation bias.
Why is it a concern?
Why as consumers of research,
why as producers of research is it a concern?
Why is it something that we should really be a aware of
and actively address?
And how do we do that?
How do we guard against it?
The rest of this session,
I would like to walk you through a number of concepts,
vocabulary terms that we will be using
throughout the rest of the class.
Here's a list of them,
and I will be going through each one of them.
So the first two are two basic approaches
of how do you collect and analyze the data?
And I'll go through each one in the next few slides.
The first is the distinction
between idiographic and nomothetic.
And this basically looks at,
it's a function of whether you're looking at
a few cases in great depth or looking
at a narrower, shallower view of many cases.
And hopefully, a few examples and more detail might help.
I imagine both of these terms
are fairly unfamiliar to you.
When you take an idiographic approach,
you are taking a very deep, detailed,
exhaustive description of just a few events or a few cases.
So you're sort of learning a lot about a little.
So if your question
is how do students maintain healthy weight?
The idiographic approach might really focus
on one per person.
Like, this sentence here about my roommate
and all the things that he does.
A very sort of exhaustive, deep,
detailed description of what this one individual does
to maintain healthy weight.
If you were to take a nomothetic approach,
it would be looking at just a couple
of explanatory variables
that you think are of great importance
and look at it at a number,
a larger number of respondents or observation.
And in this way, it's sort of learning a little about a lot,
a little information about a lot of people.
So here, this description here is a more nomothetic,
where there was a survey of 200 students
and found that these few factors here,
fruit and vegetables, no alcohol,
and getting adequate sleep,
that these are the key factors
that were found in maintaining healthy weight.
So hopefully, this example makes sense
of the idiographic is a lot of detailed information
about a small number of individuals say,
and the nomothetic is looking at
just a little bit of information,
a few key variables in a very large population.
The other dialectic or number approach
is more in the analysis of data
and the direction of reasoning.
And how do you sort of derive details
from patterns and vice versa.
So an inductive approach
is where you start with the data,
you start with the information,
the observations that you do,
and you look for patterns and themes
and maybe develop hypotheses
or general principles or theories
that you can then test with later data.
So this is sort of a bottom-up approach,
where you start with the data themselves, the information,
the observation themselves and look at it and say,
what are the patterns emerging?
Well, what are the themes that we're seeing in these data?
And what sort of hypotheses or principles
do these patterns and themes suggest?
Whereas deductive, you're starting
from a general principle
that you are assuming is true and you say,
well, if this is true,
then this is true and this is true, then this is true.
And you're starting with the principle
and deriving observations from that.
So this is a good way of testing hypotheses.
And this tends to be a more top-down approach,
where you start with the principle and see
if the data hold true to the predictions
or the forecasts of that sort of principle would suggest.
Another set of terminology is these two types of analysis,
positive and normative.
So a positive analysis strictly describes, it says what is,
not what should be, what's right or wrong,
it's just what is present.
And it tends to describe a phenomenon.
Whereas a normative approach says what should be.
And in this case, you would have to agree on
what's the goal, what's the direction,
what does better look like.
And it tends to be more prescriptive.
That it says this is what we should do
to attain this goal,
or this action is better or more preferred.
So if I were to ask what teaching techniques do I use?
That might lend itself to a positive approach.
Whereas what teaching techniques should I use
to create an effective space?
Then that would be normative.
The goal being creating effective learning for you all.
So one of the practices that I have used
throughout my career as a researcher
and that I put a lot of faith and credit
and find a lot of value in is using mixed methods.
That thinking that most really complex
and therefore interesting social science questions
tend to have rather complex answers.
And by using a mixed method,
that they can create a more complete
and comprehensive account
of the thing that you are studying.
So a qualitative approach would have non-numerical data.
Usually, you're examining text of some kind,
something that someone said or wrote,
or behaviors that they are doing.
And it tends to seek a much more in-depth understanding
of social phenomena.
It tends to be rather flexible.
So you tend to ask a lot of open-ended questions,
and tell me in your own words what is your experience?
Whereas a quantitative,
it tends to deal much more with numbers,
where you have them choose from a number of options
and then in, therefore, something like a survey,
you can count how many said this versus how many said that,
look at correlations of those who tended
to answer true on this question
were more or less likely to agree or not agree
with this other question.
Tends to be much more structured.
And again, I'm a very firm believer
in the value of mixed methods
and believing that they really compliment each other well,
they tell different sides of the story
and together do a better job
than either one singularly might do.
And not so surprisingly,
this class will deal with mixed methods throughout.
We'll be employing a number of both qualitative
and quantitative methods.
We will learn about them,
and we will be employing them in our research project.
The next set of vocabulary is probably something
you're a little bit more familiar with.
That is variables and attributes.
So variables, they vary.
So, in theory, different people
would give a different answer to each one.
And you can think of it as questions on a survey.
They tend to be measurable.
And some demographic variables that you may be familiar with
include age, occupation, income,
race, gender, ethnicity, household size.
So it's sort of like the questions on a survey.
And if you asked everyone in this class these questions,
the responses would vary among us,
that we would have a variety of answers
around age and race and gender and et cetera.
By contrast, attributes are sort of the answer to a survey.
It's each individual has an attribute for each variable.
So your attribute for the variable of age, for many of you,
may be 21 and your attribute for the variable
of occupation would be student.
Mine would obviously not be those.
I am somewhat older than that, and I am a professor.
I wanna say a bit about variable types,
that sometimes we think about variables
as being independent and dependent.
So the independent variable
is not the subject of the research.
In some cases, it's not the behavior or action
that we're really interested in,
but we're wondering how a certain group
of people might answer that question.
So with the independent,
we often think of it as given or exogenous
and that in some cases, we might think of it as a cause.
So in many cases, demographic variables
may be the independent.
One's age, income, household size, occupation,
race, gender, gender identity, things like that.
The dependent is the subject of the research.
So in many cases, it is the behavior
or the action or the attitude that we are interested in.
And in some cases, we look at the relationship,
there may even be a causal one by the independent.
So again, in many cases, but not all,
the demographic is an independent
and the behavior is the dependent.
So we might graph out how does,
if we took a whole bunch of folks
and made them run 100 meter dash,
and we would graph age and speed
that we would probably see a relationship
where the speed is probably fastest or the lowest time
somewhere in your teens and 20s.
And then as you get older,
the age would probably increase over time
for the majority of folks.
So you can maybe think of another example
of what would be an independent and a dependent.
The last topic is thinking about the quality of measurement
and sort of what are the criteria for the quality
of a measurement that we do?
And I'm gonna talk about precision,
accuracy, reliability, and validity.
And I'm gonna talk about each one.
So first I'm gonna talk about precision and accuracy.
So precision is about the narrowness of detail,
that how sort of narrow does the description go?
So you see a photo of this man here,
probably most of you know that is Tom Brady.
And each of these attributes is true.
So they are all accurate.
So it is true that he is a carbon-based life form,
he's a human, he's a male human,
he plays in the NFL,
plays for the Bucs, plays quarterback.
And you can also see that these are sort of nested,
that if you play quarterback for the Bucs,
then you play for the Bucs.
If you play for the Bucs, you play for the NFL.
If you're in the NFL, you're a male human.
All male humans are humans,
and all humans are carbon-based life forms.
You can see that we're coming from a very,
very broad description to a very, very narrow one,
down to, you know,
one or a very small handful of individuals.
To say Tom Brady plays quarterback for the Buccaneers
is a much more precise statement
than Tom Brady is a carbon-based life form.
Even though they're both accurate.
And accurate basically means it's true,
it correctly reflects the the real world.
And again, all of these statements are accurate,
but each one, as we go from top to bottom,
gets more precise.
So do you think that it's important?
Is one more important than the other?
Is it more important to be accurate?
Is it more important to be precise?
And why?
Or maybe think of it as I frame it there, true or false,
accuracy is more important than precision.
Well, I would say that it is a false statement,
and here's why.
That if you are going to locate me,
so right now I am at my home most likely
in Burlington, Vermont.
So if you were to find me, and on my street,
you would maybe say, you know,
it would be an accurate statement
to say Professor Connor is in the known universe,
or in the solar system, or he's on Earth,
or he is in North America, or he is in Vermont.
And those are all accurate, but they're not very precise.
But if you wanted to find me, you know,
that there was some important reason,
and they were going house to house,
that you could say that I was one house over
than my actual house.
So 1 Main Street instead of 3 Main Street.
And then that even though,
if I'm actually at 1 Main street,
but you say I'm at 3, that it would be a lot better,
it could be a lot easier and quicker
to find me by saying I'm on 3 Main Street,
which is precise and not accurate
than to say I am in North America,
which is accurate but not precise.
In the same way,
if you're baking a cake and you need exactly
one cup of milk,
that an accurate response might be you need between 1/8
and eight cups of milk.
Okay, it's a very accurate statement, but it's not precise.
But if you said 1.00001 cups of milk,
it would not be completely accurate,
but it would be much more precise,
and that would be a much more useful bit of information
to make a good cake.
Finally, I'm going to talk about reliability.
So reliability means that if you do the same procedure
over and over,
that you're going to get the same kinds of information.
So another way of thinking about it is it's predictable.
So a very good example is if you ask someone
to answer a survey, pick one of these five things.
Then you know that they're gonna pick
one of these five things.
Like Professor Connor is an effective professor
and you can do a five point scale,
strongly disagree to strongly agree,
and you know that if they answer it and they do it right,
they're gonna get one of those five things.
Whereas if you said,
what do you think about Professor Connor?
It could be all over the place,
that you could talk about me at at my job.
You could talk about, I don't know,
how I dress or what type of car I drive,
or how funny my jokes are, or any of a number of things.
So reliability means that you're gonna get
the same kind of answers,
and you tend to get a more precise measure from reliability.
Validity means, basically, is it truthful?
Does it credibly measure what you want to?
Does it accomplish the purpose?
So in other words, is it accurate?
So valid means does it really reflect the real world?
So if you're conducting an interview,
instead of saying pick one of these five things, say,
put your experience of this phenomenon
that we're studying into your own words,
and it will much more accurately
and validly reflect what they actually think,
that when because maybe
none of the five things that you ask on a survey
are really right or maybe the survey question that you ask,
they really don't even know or don't have an opinion
or don't find it important.
We're gonna spend a lot of time
with accuracy and reliability over time.
But I wanted to introduce it now.
This slide sort of sums it up.
So when you do nomothetic research,
remember that that is asking just a few variables
and getting the attributes of a large number of people.
It tends to be much more structured,
tend to be quantitative
because we count how many say each thing,
tend to be very reliable
because you only get a narrow range of answers,
tends to be quite predictable.
The downside is it can be rather shallow or superficial
when you're saying
choose one of these five things, and maybe,
you know, again, maybe you're asking the wrong question,
maybe none of those five things apply.
Whereas when you do an idiographic approach,
it's much more flexible.
You get qualitative data
'cause they express it in their own words.
Tends to be very valid.
But it also then tends to be very unpredictable,
that it's, in many ways,
a lot easier to analyze quantitative data
than it is qualitative data
and much more sort of straightforward
because of its reliability.
So both of these approaches are commonly used
in social and economic research,
that in many cases,
when you do an idiographic approach,
you use things like focus groups
and key informant interviews where you talk to people,
you have 'em put their experiences into their own words.
You probably only talk
to a fairly small number of respondents,
whereas as a nomothetic,
you would administer a survey,
and you get lots and lots and lots of responses.
You know, you can get data from lots of people,
but there's only so many sort of questions that you can ask.
Here are some key takeaways,
and I encourage you to look these over
and make sure that they all make sense
and be ready to discuss them
when we come to together in the remote class setting.