- [Lecturer] Hello and welcome to this video lecture
on inference and hypothesis testing.
So today, we're gonna be talking
about how to do hypothesis tests.
And specifically, how do we know
if one of the beta hats that we estimate,
or one or more of them are statistically significant?
So can we say with confidence that given our data,
the beta hat has a real impact on Y.
Does a change in X actually have a change in Y?
And how sure are we about what we are saying?
So again, we're talking about hypothesis tests.
So far, we've been talking about our beta hat,
the OLS estimator, or beta hat OLS, more specifically.
And we learned that if these five assumptions hold,
that OLS is BLUE, it's the best linear unbiased estimator.
And we talked a bunch about under and over-specification,
and variance and variance inflation
by adding an irrelevant variable,
and the bias that ensues by excluding a relevant one.
And now we're gonna look at hypothesis tests.
So what do our data say about the statistical significance
of the betas that we have estimated?
And for this, we need to add a sixth assumption.
So so far, the five that we have learned about
and assumed so far lead to OLS is BLUE.
Now we're going to need a sixth one
to be able to do hypothesis test,
and that is the specific distribution of the error terms.
So we know that its expected value is zero,
and its variance is a constant,
both of which are independent of any value of X.
But we haven't really talked
about the distribution of the error.
So what is sort of the exact shape
of the distribution curve?
So now we're going to add a sixth assumption,
which is the normality of the error.
So we assume that our u's,
our disturbances, our error terms,
have a known distribution.
That we know the basic shape of the bell curve specifically,
that they have a normal distribution.
And the easiest way to think about this
is with a bell curve.
That the shape of the distribution is a bell curve.
So here is a bell curve.
You could see that it's centered on zero.
And so the y-axis here is,
how likely are you going to get that value?
And then the x-axis are the various values.
You can see that in a bell curve that it's most likely
that the closer to zero that you get,
the more likely that those are the error terms.
And the farther away from zero that you get,
the less likely that you're going to get
that value for your error terms.
This is another way of thinking about
what I've been talking about,
the bingo ball distribution.
There's a whole lot of values at or near zero.
And the farther away from zero that you get,
the fewer values of that bingo ball you will have.
So here is the sixth assumption, that in the population,
that the error is not only independent of X1 through Xk,
but it has this normal distribution with a mean zero
and a constant variance sigma squared.
And we denote it like this.
u is distributed in a normal distribution
with mean zero, variance sigma squared.
So that's how we would write it.
Note that this is a fairly strong assumption.
The parts of mean zero and variance of sigma squared,
we already assume that.
That's the assumptions above,
so that really doesn't say much.
But that, to assume a specific distribution is
a fairly strong assumption.
But if we do assume that together,
one through six, that these are often lumped together
and called CLR, classic linear regression assumptions.
Note as well that we can make one more leap,
that if all six of them are true,
that OLS is not only BLUE, it's BEUE.
It is the most efficient, unbiased estimator of any.
And another way that we can say that is that Y,
given all the values of X,
has a distribution
of mean of beta 0 plus beta 1 X1 plus dot dot dot beta k Xk,
and a variance of sigma squared.
So if we think about, how robust is this assumption?
How certain can we be?
Do we feel like we are
on pretty solid ground when we make it?
So the good news is that the central limit theorem says
that as sample size increases,
that the distribution will converge
to a normal distribution, approximately.
And in this class, we basically assume,
and for most applications that you'll do,
that we have a large enough sample
that we can sort of call on the central limit theorem
and make this assumption with a fair amount of certainty.
We're now going to talk about testing
the significance of a single parameter,
a single beta hat that we get from our regression analysis.
So again, we start with our familiar model
with Y on the left side,
and k regressors on the right, and an error term.
And we were assuming that all six of our assumptions hold.
We know that our beta hat OLS is unbiased,
but we can never really know the true population value.
We don't know the real beta.
All we know are our beta hats.
They're unknown and unknowable.
So we wanna know, how sure are we that this beta hat
that we got is close to the true value?
How sure do we need to be in order to say,
yes, this is a true relationship.
And most of the time, as we see,
we're going to test it against a null hypothesis
that this beta equals zero.
So if our assumptions one through six are true,
then we know that the,
we can generate a T-statistic for every beta j.
And this T-statistic is beta j hat minus its true value,
beta j, divided by the standard error of our beta j.
So we're gonna usually test it
against the null hypothesis that beta j equals zero.
So the T-stat that you will get an SPSS is beta hat j
divided by the standard error of beta hat j.
And this has a known probability distribution,
which is a T-distribution, which also has a bell curve.
And note that every T-distribution is specific
for a number of degrees of freedom.
So we're gonna, we know that this has this distribution
for n minus k minus one degrees of freedom.
And basically, any time online or in any book, any textbook,
you can find a table with the T-distribution
with n minus k degrees of freedom.
What this T-test does,
so when we're doing one parameter at a time,
we're you're using T-tests.
And one of the things that we ask is,
how likely are we to get a T-stat of that value?
So recall, our T-stat is the ratio between beta hat j
and the standard error of beta hat j.
And we ask, how likely is it that we would get
that value of a T-stat if the null hypothesis is true?
That this is our p-value.
So if our null hypothesis is true,
beta hat j equals zero.
Now, if it's true, and looking at this bell curve here,
it's pretty darn likely
that we're going to get a very small T value.
Because this curve is centered over zero,
if we get a very large number, either negative or positive,
it's going to be far away from the middle of the bell curve.
It's gonna be way out on the tails.
And it's fairly unlikely that we would get that test stat
if the null hypothesis is true.
I'm going to walk you through how to do
a T-test for a single variable.
So we wanna look specifically if a beta hat j
that we get from our analysis is
statistically different from zero.
So as with any hypothesis tests,
we start with a null hypothesis.
Most commonly, again,
we start with our null that beta j equals zero.
Beta j equals zero, if that is true,
that means that the slope of the line is zero.
That it is a horizontal line.
That no matter, that no change in X has any effect on Y.
So to think of our example,
that no change in income has any effect
on local food expenditure.
That changing one's income has no effect
on how much they spend on food.
And we're looking at one at a time, so one of our k betas.
And again, this is a test.
Once all the other X's have been accounted for,
that this Xj has no effect on Y.
So if the null hypothesis is true, Xj has no effect on Y.
When we do SPSS and do a regression, we get a T-stat.
And this T-stat is the ratio of beta hat j
divided by the standard error of beta hat j.
It is its estimated mean
to its estimated standard error.
It is its magnitude divided by its precision.
And since the standard error is always greater than zero,
the T-ratio has the same sign as B.
And a big beta hat
and a small standard error leads to a big T-stat.
So if beta j hat is far away from zero,
and if it's a very precise estimate
along a tall, skinny bell curve,
then we will have a big T-stat.
The beta hat j that we get out of SPSS will rarely,
if ever, be exactly zero, but it might be close to zero.
So basically, what this is asking is,
how likely is it that this beta j hat is far away from zero?
How sure are we that it's far away?
And how sure must we be to reject our null hypothesis?
So to do a test, then we need an alternative hypothesis,
which we will call H1.
We're gonna start with one-sided test.
And not surprisingly,
we're gonna do two-sided test next,
but first, let's do one-sided test.
In a one-sided test,
we say that beta j is strictly greater than zero.
And in this case, we do a one-sided test
if we can assume that beta j less than zero
doesn't make economic sense,
that it's not feasible that you would get this.
It might be in a, there's a lot of examples,
but if some firm never demotes or never gives a pay cut,
then salary will always increase.
The next thing that we need is, how sure do we have to be?
How certain do we have to be to reject the null?
Usually, we choose 5%,
or in some cases, we sort of bat around 1%, 5%, and 10%.
But for right now,
let's assume that we are going to choose
that if there's a 5% chance of a type one error,
anything less than that is good enough.
That for 95% sure or more,
we are sufficiently certain
that we are going to think this is a real relationship.
And what we do is we look at,
we look for a sufficiently large T-stat or T-value
that we can confidently reject this null.
And we call this c, c for, I think, critical value.
So we call c the critical value at the 5% significance value
as the 95th percentile in the T-distribution
with n minus k minus one degrees of freedom.
Another way of thinking of this is,
it's the area under the, it's the part of the curve,
the very tail of the curve
that only has 5% of the area under it.
So in this bell curve,
it's way out on the tail of the curve,
such that only 5% of the area
of that whole curve is under it.
So if our T-stat is greater than c, we reject the null.
Remember, a big T-stat means that beta j hat is big,
and the standard error is small.
If it's less than that critical value, we fail to reject.
Note that the smaller the significance level,
the smaller the critical value,
and the more likely we are to reject it.
So if we reject at the 10% significance value,
we always fail to reject as well at the 1%.
And since we're doing a one-sided test,
any negative T-value will lead to fail to reject the null.
And the rule for our null hypothesis
of beta j being less than zero is the same,
except it's a large negative value,
greater in absolute value than our c.
So way, way out on the left-hand tail of the curve.
More commonly, we do a two-sided test,
where we're honestly not sure what the direction might be.
So an example I thought of was,
does the number of annual trips impact
the total weight of trash?
So if you drive your trash to the dump,
does the annual number of trips impact
how much total weight of trash that you dump?
I think it's fair to say, we wouldn't be sure.
Some people might make a lot of small runs.
Some people might wanna make a few large runs.
It depends on a lot of other factors.
But we have our same null hypothesis,
beta j equals zero.
But our alternative is simply that it does not equal zero.
It could be a positive or a negative number,
and we don't know.
So in this case, we choose our c
such that 2.5% of the curve lies in the two tails.
So it's the 97.5% of each of the two tails.
So that a total of 5% of the area lies in these two tails,
but there's half of it on each side.
And again, a T-ratio of greater than c
or less than negative c results in rejection of the null.
So when we reject the null,
we say we are 95% sure that the true value
of our estimated parameter does not equal zero.
We say that our Xj is
statistically significant at the .05 level.
Note that this T curve is the probability
that you would get a T-stat of that magnitude
if the null hypothesis were true.
And every set of degrees of freedom has its own curve.
And note that as we see here in this diagram,
the greater the degrees of freedom,
the sort of taller and skinnier this curve is.
And also, I had said before, the more degrees of freedom,
the more likely that you're going to find significance.
So you can see that, say our T-value is two.
You could reject the null
with the infinity degrees of freedom,
but not if you only have three degrees of freedom.
So every degrees of freedom value has its own curve.
And the curve in that degree of freedom number is,
how likely is it that you would get
a T-stat of this magnitude if the null is true?
So just to re-emphasize it,
if you get a T-stat near zero,
it's pretty likely that you would get that
if the null is true.
If you get a very large T-stat way out on the tails,
it's very unlikely that you would get that
if the null is true.
If your T-stat is very large,
it means that your beta hat j is large,
it's far away from zero,
and our standard error is small,
that we're pretty darn sure that it lies near that value.
Sometimes we do tests where our null hypothesis is not zero,
but that is beta j equals some constant,
maybe one or some other number.
And then, we simply subtract that from this aj,
some constant that we subtract that from our beta hat j,
divide by the standard error,
and compare that to the critical value.
I wanna emphasize that there's nothing magical,
or certain, or right, or anything about this 5%.
It is, to be honest, arbitrary.
It's sort of, to be honest, it's just a convention.
And it's what we sort of all agree to.
But there's nothing magical about it.
And one of the things that you get
with the SPS output for every beta is a p-value.
And that is, and the p-value measures
what is the smallest significance level
at which we can reject the null?
So if we get a p-value of .04, then we can,
then we're four, we can be 96% sure
that we're not making a type one error.
Let's look at more detail at P, so the p-value.
Again, you always get a p-value in the output of SPSS
or any regressions software, I imagine.
It's a number.
It's a probability.
It is strictly between zero and one,
where zero, of course is, there is no chance,
and one is certain.
When you get the output from SPSS,
the standard is that the null
of that beta equals zero against a two-tailed test,
the two-sided alternative.
And it is the probability where the T
with a n minus k minus one degree of freedom.
So that place on the curve is greater than the test stat.
More broadly, P is the likelihood
of a type one error, of a false positive.
It's the probability
that you reject the null when you shouldn't.
The p-value is also the probability
of seeing a T-stat that large.
So how likely is it that you would get that magnitude
of a T-stat if the null hypothesis were true?
So if you have a p-value of .02,
there's only a 2% chance that the T-stat
that you got would occur if the null is true.
So you can be 98% sure
that there's a real relationship there,
or that your beta does not equal zero.
So a small p-value is evidence against the null.
So having a small p means you are likely to reject it.
And a large p gives you
a little evidence to reject the null.
A very large p-value, you will fail to reject the null.
And just a bit here of nomenclature,
that we would say we fail to reject the null at the 5%,
not we accept the null.
That's just sort of the way that you phrase it.
I wanna make an aside on significance,
that just because something is statistically significant
doesn't mean that it's economically
or socially or otherwise important.
That, again, statistical significance is
strictly a statistical measure.
It's the function of the beta's magnitude,
and its variance or standard error.
And there may be times when something
could be statistically significant,
but not economically important, say.
And an example here is if your Y is
local food expenditure in dollars
and our X1 is annual income.
And if beta is significant and its value is .01,
that means that for every thousand dollars of income,
folks spend one cent more per year.
It might be, clearly, we reject our null,
and it is a statistically significant measure,
but it's not very economically important.
Note as well that as n increases,
so our sample size increases,
our variance decreases
and the likelihood of significance increases.
So some researchers choose as n gets larger
to require a stricter standard.
So they'll use maybe .05 if n is in the hundreds,
but .01 if it's in the thousands,
Some quick rules for empirical work.
First thing you do when you get the result is
you look for significance,
and then you sort of look at its magnitude and think about,
is it economically, or socially important?
Does it really say anything
about the phenomenon that we are studying?
And if it is significant in the wrong sign,
it's often a sign of specification error.
Another thing that we can do is
to create a confidence interval
for a given confidence level for every beta j.
So we've been saying the 95%, so that would be a p of .05.
The confidence interval is beta j plus or minus c
times the standard error of beta j,
where c is the constant at the 97.5 percentile.
So that point out in the two tails of the curve
with 2.5% underneath it,
and the T curve with n minus k minus one degrees of freedom.
So this tells us two things.
First, how precise is our estimate.
And you can imagine that you might have seen
data presented with little error bars around it,
like limits a lot like that.
And if zero lies within it, what does that mean?
And we'll discuss it in class.
So far, we've been talking about T-tests,
and conducting, looking at one parameter at a time,
a single hypothesis that we're looking at,
pulling out a single beta j and testing
the null hypothesis that it's equal to zero.
What about, though, if we want to test more than one?
And when this is often used
in sort of nested models where we wanna see,
is there a group of parameters
that jointly have an effect on our Y?
And for this, we use an F-test.
And this is, the procedure is called
testing multiple restrictions.
So looking at more than one beta j at a time,
and doing a test, if jointly,
they are significantly different than zero.
So here's a model that we are now familiar with.
We have five regressors, k equals five.
And we wonder, do X3, X4,
and X5 jointly have an effect on our Y?
So under this test, we have a null hypothesis
that beta3 equals beta4 equals beta5 equals zero,
or each one is equal to zero.
Our alternative hypothesis is if any of these are not true.
And we call this exclusion restrictions
that we're sort of pulling,
when we do the test, we pull those out of the model.
Or we are restricting their value to zero.
And then we do a multiple hypothesis test,
which is an F-test.
When we test multiple restrictions,
the null is that all of them are jointly equal to zero.
The alternative is that it's not true.
So we reject the null
if any of the three B's is not equal to zero.
Why don't we do all three?
Just do each one singly.
So the problem here, the biggest one is
that three 5% significance tests, that .95 cubed is .86.
So if any of them, it doesn't hold
that the other one is equal to zero,
which is what we're doing in an F-test.
It takes out, that the F-test takes out all three singly,
whereas the T-test only takes out one.
The intuition of the test is,
how much does the goodness of fit change
with and without these regressors in the equation?
Recall that adding regressors
always improves the goodness of fit.
It always increases R squared.
It always decreases the sum of squared residuals.
And taking them out always increases
the sum of squared residuals.
And actually, this is what we use to do the test.
So we look at the SSR.
And we run the two models.
We run the k equals five, which is our full model,
and the k equals two
where we leave in two and take out three.
We run both regressions,
and we note the sum of squared residuals of each one,
which you get in SPSS.
Note that the SSR from the restricted model is
always greater than in the full model,
and think about why that is.
So in our general case, we make queue restrictions.
So we start with k regressors.
We remove q of them.
So in our restricted model, we have k minus q.
And we can just order them so that the ones
that we're going to take out are the last ones.
So here is the F step.
And it is, the best way to think of it is,
it is the relative change of the sum of squared residuals
weighted by the degree of freedom.
So you look at the SSR of the restricted model
and the SSR of the unrestricted model.
And the biggest driver of the magnitude of that stat is,
how much does that change?
How much does the SSR change
when we take those regressors out?
So if there's a big change, we will see that this,
the part of the numerator,
SSR, we restricted minus SSR.
Unrestricted will be a big number,
which leads to a big F-stat.
So under our null hypothesis,
and if our CLR assumptions all hold, then we,
in the same way that we know the distribution of the T-stat,
we now know the distribution of the F-stat,
and it has an F-distribution
with q and n minus k minus one degrees of freedom.
q being the number of restrictions,
how many regressors we take out,
n is the size of the sample,
k is the total regressors in the full model, and one.
So it's q, and n minus k minus one degrees of freedom.
As with a T-test, we reject the null if the F-stat is big.
If it's sufficiently large to be way out in the tail.
and a large F-stat implies a big change
in the residual when we make these restrictions.
Again, we choose a significance level, like 5%.
And we find the critical value
for some c for the F-distribution,
with that number of degrees of freedom.
We reject the null if the F-stat is
bigger than c, the critical value,
same as with a T-test.
This is especially valuable in testing restrictions
of highly correlated regressors,
because it tests them as a group.
And it's like all together, are they significant?
Not, is one significant holding all the others constant?
So we're gonna do an example in SPSS
that looks at major league baseball batting stats.
But I wanna talk to you here about another one.
And I don't have data for this.
This is just kind of a thought experiment,
but I think it would hold true,
of farming practices and thinking
about climate change best management practices.
So three practices that we know are sort of good for soil
and good for keeping phosphorus
out of the lake are cover cropping,
buffer strips around a stream, and reduced tillage.
So if our Y is how much phosphorus is
leaking off of the farm,
and we might have a bunch of regressors
about farmer experience and age
and what they grow and all kinds of things like that.
But looking at these three practices
that maybe each one of them
wouldn't be significant from a T-test,
that having a buffer strip holding cover crop
and reduced tillage constant
might not have a significant effect on phosphorus load,
but the three of them together do.
And I invite you to maybe think
of another example of how this,
where this might come into play.
And again, we'll see it when we do
an SPSS exercise with baseball stats.
The cool thing about T and F-stats
and this sort of intuition of how they work is
I think all test stats work the same.
They all work from this same intuition.
So you start with some sort of hypothesis test of something,
or many somethings equal zero,
that the one thing has no effect on another.
If you get a big stat,
it means it's going to be way out
in the tail of the distribution.
And so you can think of this as being far away from zero,
far away from the null hypothesis.
This test stat is sort of far away from zero,
it's far away from the idea of the null being true.
And if that's true, when we get a big F and a big T-test,
it's very unlikely that the null hypothesis is true.
We have a high confidence to reject our null.
And that is associated with a small p-value
and a small significance.
So a big test stat means
that we can usually reject the null with high confidence,
and it's associated with a small p-value.
Doing one hypothesis test at a time,
one beta j at a time,
it makes sense to just use an F-test.
They're easier to compute,
you get the output automatically.
It will yield the same result
as an F-test in a two-tailed test.
And a one-tailed test is the best,
or a T-test is best for a single restriction,
so there's really no point
in doing the q equals one F-test.
You can simply do a T-test and you'll get the same result.
Last time, we talked at length about specification.
What to include, and what not.
So one way that this can be used,
because you run a full model,
you include things that you really think are important.
You run this full model, and then you look at the output,
and you can look at those with a p of greater than .10.
Or maybe a T-stat of less than one in absolute value,
and use an F-test.
Or also, if you do collinearity diagnostics,
you can do an F-test of the most colinear variables
and use that to determine which is your best model.
There's also a way to use R squared to calculate an F-stat.
You'll get the same results.
It's just a change in R squared
adjusted for degrees of freedom.
This may be a little easier in some stat packages.
Another thing that you get sort of for free in SPSS is
the overall significance of a regression.
So here, your null is, none of the X's matter.
All of them are jointly equal to zero.
The restricted model is just an intercept and an error.
And you could run this.
And since the R squared from the rejected,
from the restricted model is zero, here is the F-stat.
Almost always, you will find
that the overall regression is significant.
If you don't, you really just have a very poor model.
For right now, just keep these in mind.
Here are other hypothesis tests
that you use in other models, not in OLS,
and we're gonna revisit them.
It's just to sort of have them in your mind for right now.
I wanted to touch briefly
on some thoughts that the Kennedy textbook.
And as always, keep in mind that all models are wrong,
some models are useful, as George box said.
So models are just lenses into the world.
And how do we make them more useful?
So recognize first
that the specification can be a challenge,
that you're not always control of data generation.
That in many cases, it's more art than science.
And also in many cases,
there may not be a single universally accepted best way.
So a good place to start is
to use a theory and prior studies,
maybe some common sense,
and it include things, but the goal being,
what the text, what author calls a clean economic story.
And note again that models with a low overall F-stat,
where it's almost all error or noise,
may just be misspecified.
That it's always good to use these kind of tests.
In some cases, it might make sense
to set aside a few observations
and see if your model holds in those.
And if there are many (indistinct),
it's often worthwhile to compare and contrast them,
do it a few ways, see what you get,
and discuss strengths and weaknesses,
and probably choose which one.
The most important point here in this slide is,
be prepared to explain the path that you chose.
That there may not be one single way,
but you really have to make a good case
of this is what I did and why I chose this.
So again, some of the,
this is an approach that I have used.
So starting with one.
Start with a fairly large number of regressors.
Ones that you really think have an impact
based on theory, common sense, and previous studies.
Look at the overall F-test.
And then run a restricted model
with only those variables that are significant,
where T value in absolute value greater than one.
And then use an F-test to choose
sort of what is your final model.
So I hope you have a better understanding
of hypothesis tests now, and thank you.