-: Hello and welcome to the video lecture on sampling.
In this lecture we will learn about
how do you choose the people or the subjects
from which you are going to collect data.
What are the considerations?
So we're gonna talk about the two main types,
non-probability and probability,
and then talk about some sort of vocabulary
that goes along with that.
Talking about parameter and statistics
and then confidence levels and intervals.
So again, the idea of sampling here
is from whom do we collect data?
How do we choose what's our methods
for choosing the right subjects for the research question
or the research problem that we're dealing with?
So as a motivating example, I wanna make it clear
that depending on who you ask,
you're gonna get very different questions
that if you asked who's your favorite musical artist,
who's your favorite actor?
And you would ask my kids, if you had asked my daughter,
she would say, Taylor Swift.
If you asked me, I'd probably say David Bowie.
If you ask folks older than me,
they would have other answers.
Similarly, if you ask what's the most important policy issue
that would change from say, a red state to a blue state,
and then people from the US versus China
versus Iran versus other nations.
So depending on who you ask,
you're gonna get a different response.
So we're gonna talk about the two main types,
non-probability and probability.
So the first type, non-probability
means that you cannot state the probability
of any individual being selected.
That you can't say there's a one in 10 chance,
there's a one in five chance,
there's a 20% chance, anything like that.
We use these, this is used actually very often
in social science research when the number
of knowledgeable subjects is small or hard to reach.
So I use the example of NFL bands and fans in Belgium,
or might be some well-known musical artist.
Then rather than sort of going out
and doing a representative sample of Belgium or of Vermont
and to find out about this artist,
it might make more sense to go to where this artist
is playing and ask folks there
because they're gonna know things about it
and actually have experiences about that.
In many cases too, they might have specialized knowledge,
and in some cases it's done for convenience.
It can just be the most convenient, the easiest,
the low cost, and sometimes that's the best way.
So here are four types of non-probability sampling.
I note that they are not mutually exclusive,
that they don't like clear boxes of this Os and this,
that there will be overlap among them.
So, first is convenience sampling.
And this is where you just sort of go
and talk to people where they may be,
sort of the person on the street perspective,
go to the library
or go to a dining hall to talk to students,
'cause that is where they are.
If you're interested in market research
on Fair Trade coffee, you might go
to a specialty coffee shop.
You wanna go to where consumers will be
that are likely to want your product.
Next, is purposive or judgmental.
And this is where you as the researcher,
seek out and find subjects that you think have information
that you want that'll be valuable, that'll be useful.
In many cases, we call these key informants,
people who know about a certain subject.
So if you wanna know about, again, a certain artist,
you may wanna go talk to their fans.
If you wanna know about a certain novel kind of food,
go find people who eat it or who make, et cetera.
When we do this, it is important to the extent that we can
to use what we call maximum variability sampling.
And that is to think about what are the dimensions
of variability in the subjects.
And you might wanna think about gender, age, income,
geography, and not just talk to the same types of folks,
so not just talk about, sort of talk to males
who are older and wealthier and live in Shelburne,
that you wanna sort of get a range of gender, age,
income and variables like that.
You also may wish to seek outliers.
Find folks that you think will have a different
or unique opinion than don't see it in that way,
so that you get a much like broader range of views.
Next is snowball, and this is called,
it's like if you make a small snowball
and roll it in the snow,
and that's how you make a snowman of course.
It gets bigger and bigger.
And in this way, once you find somebody who's knowledgeable,
you can ask them, who else knows about this,
who should I talk to?
Many times the last question I ask on an interview is,
are there others that you think that you could recommend
that are knowledgeable, that I could talk to?
This is especially valuable when
those key informants are hard to find.
It's a good way of identifying networks.
And I've used this in some of the supply chain work
where I ask from ask folks, who do you buy from,
and then who do you sell to?
And in that way, that's a good way to find other subjects
that are knowledgeable about this.
The last non-probability method is a quota
where you think about what are the characteristics
that you want and you sort of want a certain percentage
of genders, of ages, of ethnicities,
things like liberals or conservatives,
believers or skeptics.
When you're interviewing businesses,
things like different sizes, different industries,
number of employees, experience and years in business.
You think about I wanna have at least one or at least some
or a certain percentage of my respondents sort of falling
into one or more of these categories
and sort of sampling until you find that,
until you sort of meet those quotas.
So to discuss this, again, it's the idea
of maximum variability sampling that you want to make sure
that you sample from a breadth of folks
that have a diversity of opinion
and experience and viewpoints.
And a good question to begin with
is what are the dimensions of variability
and how do you make sure that you include folks
that sort of represent different sort of points
along those various dimensions.
And when we start to collect data,
I'm going to have you start to think about
what are some of the important dimensions
of variability of UVM students?
The attributes may be their major,
their class rank, where they live, race,
gender, ethnicity, things like that,
that will be important for the research question
and topic at hand.
So to sum up, non-probability, sometimes it's the only way,
sometimes it's the best way.
But one thing that you can't generally do
is you cannot generalize, that you can't say,
"Well, these folks hold this view,
so therefore everybody does."
And that can bring sort of, and by not having a non
or by having a non-generalizable non-probability sample
is there may be selection biases
where you're only talking to a certain group.
When you sort of like go to a farmer's market
and have a table or things like that,
or you have folks to sort of stop by,
more likely you're gonna have folks
that have a strong opinion one way or the other.
Same with those who sort of call a show
or take an online poll,
but those folks tend to have stronger opinions,
and so you really can't generalize.
And certainly if you only talk to viewers of Fox News
or only talk to viewers of MSNBC that you cannot generalize
as to the views and opinions that are expressed there
and say that's how all Americans feel.
I think it's important though,
do not worship at the altar of probability sampling.
Sometimes a probability makes sense because again,
you get to generalize and avoid those selection bias.
But there are many, many times when a non-probability sample
will be more so.
And I invite you to think about when that might be.
And I also really like this quotation,
"All models are wrong, some models are useful,"
so sometimes to get more useful information,
it makes sense to use a non-probability sample.
So the opposite of a non-probability sample
is then a probability sample.
Here you do know the probability
of being selected into the sample.
So if you have a, so if my class has 50 students
and I choose five to take a survey,
sort of put all of your names into a hat
and pull out five, that you have a one in 10 chance.
So, you know the probability.
And that's an example of a random selection.
Just pull names, sort of pulling names out of a hat.
And one way that's very commonly used is this
where everyone has an equal chance of being selected.
And this is called the equal probability
of selection method.
And while no sample is perfect,
it does both eliminate the obvious biases
like selection bias.
And it can also, you can measure the degree
to which the sample looks like the population
and you can generalize with much more
sort of confidence that the views
of your sample reflect the larger population.
When we say that a sample is representative,
it means that it looks like the population,
that it sort of has similar percentages
of attributes as the overall population.
And this does allow for generalizability
that if you have a large enough sample
that you can say with a good amount of confidence
that sort of everybody feels the way
that the sample responded.
One thing that you need here is that you need a list
of all possible subjects.
And here's two very similar ways,
the simple random sampling and systematic sampling.
So in random sampling, you just give everybody a number
and then you use a random number generator from a computer
or from a table and just use those hooks.
And if you're using the systematic method,
you select every Nth unit.
So you make a long numbered list,
you pick the first one, one at random,
and then you see how many that you need.
So if you have a population of 10,000
and you need 1,000 respondents,
you start at a random point and then just sample every 10th
until you get to who you need.
So we will look at this in class, but this website
tells you how large of a subject do you need
to be able to generalize.
And it depends on two things that we're gonna talk about
here in a few minutes.
The confidence level, how certain do you wanna be,
you wanna be 90% sure, you wanna be 95% sure,
you wanna be 99% sure.
And the confidence interval, thinking about like plus
or minus how many percentage points,
sort of how close do we need to be?
And it also depends on population size.
So a larger population needs a larger sample
as well as how homogeneous are the answers.
Now I'm gonna give you some vocabulary
that describe how certain you can be
that your sample is generalizable.
So, first we're gonna talk about parameters.
That is the true value of the entire population.
That if you sampled everyone
and you got everybody's response that you would know
that that is the true response.
And it might be expressed as median, mean or mode,
it might be expressed as the frequencies.
And just as made up example, say that we talked
to every single UVM student
and found that 65% own an iPhone.
Again, totally made up example,
but just to give you an idea.
The statistic is the measure of the sample.
It's the estimated mean or the estimated frequency.
It's the estimate of the true parameter,
the true value that we make
by drawing a sample and asking them.
And let's say that we draw a sample
and say they find a 68% of those
that we talk to own an iPhone.
So in this case it's off by three percentage points.
The sampling error is the difference
between the parameter and the statistic.
And it's the error incurred by not talking to everyone.
And it depends on three things.
The larger the N the less the error,
the more diverse the population,
the sort of the greater range of answers
the greater the error and the confidence level
that the more precision that you need,
the more that you need to narrow in
the greater the chance of error.
So, here's a very simple formula
and it's for a binary yes or no.
Like do you own a bike
or did you vote in the last election?
And it's the square root.
And here p is the proportion who say yes,
and 1-p is the proportion who say no.
So it's the square root of two times p
times 1-p all divided by N.
So you see that as N gets larger and larger,
your sampling error goes down and as,
and if everybody says the same thing,
like are you a UVM student
or are you a carbon based life form,
that the sampling error would be zero.
And the greatest sampling error would be when P equals 0.5,
that when they're sort of evenly distributed,
same numbers across the two responses.
That's the greatest error.
So now we're gonna talk about
how certain can you be that are that the statistic
lies within a certain range of the parameter.
And it depends on how certain you wanna be
and how precise that you wanna be.
So the Vermonter poll, which is an annual sort
of public opinion bold, done by UVM Center for Rural Studies
is a so-called 95/5 poll.
And it's done so that they are 90 for 5% certain
that the statistic that they measure is within plus
or minus five percentage points of the true value.
So if, again, hypothetical example, 65% say
that creating jobs is the greatest priority in the state.
That we're 95% sure that the true value lies between 60
and 70 plus or minus five points.
And the more precise that we wanna be,
if we wanna be 99% sure then it's going to be greater
than five percentage points plus or minus.
But if we only need to be 90% sure it can be less
than five persistent percentage points.
And see if that makes sense
and let's discuss that as needed in class.
So again, we learned about confidence levels
and confidence intervals and in class we'll do an exercise
of how many that we need for say a 95/5 poll.
That's what we just did.
Thank you for watching.
Have a good day.