- [Lecturer] Hello, and welcome to the video
lecture on heteroscedasticity.
I wanted to start with some learning expectations.
So what I expect is for you to be able to
first understand sort of what it means and why it matters,
and what are the implications.
Next, to learn some tests of how to test for it
and then to name some potential solutions
and how to solve it?
What I don't expect is for you to be able to use data
and create these solutions, especially in SPSS.
Frankly SPSS is not particularly
adept at dealing with these issues.
But I do want you to be able to understand it,
to test for it and to know what to do.
And then, if your thesis
or other research requires it,
see me, see your advisor,
you may have to use another software to do so,
but I want you to be able to at least understand it
and know how to deal with it.
Recall back that, one the assumptions
of what I've been calling well behaved data is not,
is spherical errors.
So that's one of the requirements for OLS is BLUE.
Spherical error means that there is a uniform variance
at the variance of the error term
for every individual, regardless of the value of X
is a constant, which we call sigma squared.
And that the errors are not correlated with each other.
That your error and my error
and anybody else's error in the sample
knowing the value of mine doesn't tell anything
about the value of yours.
That they sort of don't have anything to do with each other.
One way of thinking about this mathematically is
to think about the variance, covariance matrix.
This is an N by N matrix.
So N again is the size of your sample.
The diagonals going from the top left
to the bottom right, are the variance.
So with well behaved data, those are sigma squared
and every other value, other than that diagonal is a zero.
So we can say that when we have spherical errors,
the variance, covariance matrix which we call the G matrix,
is sigma squared I, so it has sigma squared
on each of these diagonals going from
top left to bottom right, and zero's everywhere else.
So looking at this G matrix,
if the diagonals are a constant, we have homoscedasticity.
That's good.
That's well behaved data.
OLS is BLUE, we don't have to deal with it.
If it's not a constant,
especially if it's a function of the regressors,
if they change among respondents,
then we have heteroscedasticity and we need to deal with it.
And finally if the off diagonals,
so looking at these three matrices
on the left here,
if there is anything other than zeros in the non diagonals,
then we have what's called auto correlated errors.
And in that, that is where your error
and my error do depend on each other,
that they do have a covariance not equal to zero,
that they are correlated
and will deal with that more down the road, in a few weeks.
If the G matrix is not this sigma squared I,
with a constant on the diagonals and zeros.
Everything else, it's really not OLS anymore.
It's GLS, it's generalized least squares,
and we can come up with a better estimator than OLS.
The good news is that even if you run OLS
with heteroscedasticity and auto correlated errors,
there will be no bias that the OLS estimator
that we've come to use is unbiased,
but there is some bad news, as you may imagine.
The bad news has to do with two things,
first inference, and next efficiency.
The problem with inference
is that the variance of the estimator of OLS,
that formula that we learned is biased.
The variance is biased,
not the actual estimate is biased,
but the variance
and the standard error that you would get in a T-test
in the denominator is biased.
And therefore it's faulty inference that T-tests
and F-tests no longer work.
They no longer give you an accurate representation
of any hypothesis test.
So T-tests and F-tests no longer work
when we have heteroscedasticity.
The other problem is that OLS is no longer blue.
It's not the minimum variance.
And what we do, one of the ways that we fix it
is we try to weigh those residuals with a larger weight,
with a smaller variant
to sort of weigh the outliers small,
weigh the ones with a smaller variance
with a bigger weight.
Note that I said weigh at the beginning,
that OLS weighs everything equally.
So again,
if you use OLS
and the equal weight to every observation,
you're gonna get a bias estimator.
What you do is you weigh them,
which creates a new estimator beta GLS, which is then BLUE.
So there is a more estimator than OLS which is GLS.
So it's important to test.
And again, the reasons is that
you want to be able to do sound inferences
and you in some cases may want a more efficient estimator.
A common approach is to just assume
for now heteroscedasticity.
We assume that we don't have auto correlated errors
that the off diagonals are zero
and we'll deal with this later.
And I gave the example of,
an example would be that as income increases the variance
and an expenditure might also increase.
That someone with a low income,
say on, if we're looking at expenditure of food,
that it doesn't change that much,
that if you can imagine the regression line
with food expenditure on the y-axis
and income on the x-axis for low income folks.
So probably hug the regression line pretty closely
that the amount of money they spend on food
won't change that much,
but for a high income, it might change a great deal.
Some weeks they're buying fancy food and go into restaurants
and eating high off the hog, as we say.
Some weeks or some month,
maybe they made a huge run to the warehouse and stocked up
and have a really well stocked pantry
and don't buy anything that week.
And you can see that maybe the residuals
might be really far away from the line
or the large residuals
and the observations getting farther away from the lines.
So it's important that we test for this.
The first test is just to plot
and is to calculate the residuals
and to plot them versus X.
So is there a relationship,
sort of eyeballing?
It if it's clear that they get bigger,
as X gets bigger or smaller, as X gets bigger,
that's a good test.
And then there are more mathematical tests,
the White tests, and there's a couple version of this
and the Breusch-Pagan test.
And I will talk about each one.
Here are the tests for heteroscedasticity.
So we assume that the first four assumptions hold,
we have this linear model
and our no hypothesis,
with heteroscedasticity is that
the variance of U, given X1 through XK equals a constant
sigma squared.
So if this no holds, we have heteroscedasticity,
then everything is good.
Another way of writing this is that the expected value
of U squared, given X1 through XK equals sigma squared.
And the intuition here is that
the co-variants between U squared
and all the Xs equals zero.
You know what I may get a bold X,
which implies all the Xs or the sort of vector of Xs.
The first test may be the most simple test
is the Breusch-Pagan test.
Known for the, developed by former
Pirates third baseman José Pagán
who then got into econometrics.
Actually that's, obviously not true, but anyway.
So here we test if U squared
is a simple linear function of the Xs.
So we run the regular regression
as you know how to do, you save everybody's U,
which you know how to do and you square it.
So you compute a new variable
UI times UI for every individual.
Then you take that and you do a regression of all of the Xs
in your model, X1 through XK with
U squared as the dependent.
And you have a new here with the,
now we have deltas instead of betas.
Our no hypothesis is that
the delta one equals delta two, equals ...
equals delta K.
And you look at the overall F stat.
So think about what do we hope that we find here?
so ponder that a second.
And basically if we have homoscedasticity,
we hope that these Xs have no effect.
So we're hoping here for a very small all F stat.
And if we do that's good,
we have homoscedasticity as well.
If we have a big F stat,
it means that we have heteroscedasticity
and we have to do something about it.
So this Breush-Pagan is the simplest model.
Next, we have the White test.
Which is of course, named for Walter White
famed high school chemistry
teacher turned methamphetamine and gangster,
and here it's very similar to the Breush-Pagan,
except we square and do all the cross products of the Xs.
So we take not only X1 through XK,
but we have X1 squared,
..XK squared,
and then multiply X1 times X2,
X1 times X3, X2 times three et cetera.
So if you have K, with regressors that's nine in this model,
so each one squared,
and then the three interactions.
One and two, one and three, two and three.
And then you do an LM test,
likelihood test with
the nine restrictions.
Note that you will lose a lot of degrees of freedom here.
The LM is the Lagrange multiplier test
and note that the downside of this is,
it's not just, you're gonna test
for more than just a simple linear function of X.
The downside is you will lose degrees of freedom.
There's also a special case of the White test,
named for the ageless wonder,
Betty White of course,
where you run a regression with U squared hat
on the left side,
but with Y and Y-hat squared,
and you can kind of get all of these cross products anyway.
So first you run the original model,
the Y equals beta not plus
beta 1XK plus ..beta K XK.
And then you save the residuals and the Y-hats,
and you square them
and save them and regress,
the UI-hat squared on Y-hat and Y-hat squared.
And it must be the hat, not the original observations.
And then you can run either
a Lagrange multiplier or NF test.
And again, in every case you don't want significance.
You want it so that our U squared
is not explain all by the Xs.
That once you sort of run the regression once
that you don't get any association there.
Finally note that the White test also
owes great debt to former NFL defensive linemen,
the late great Reggie White,
and note that it's important to run
other specification tests first
and sort of once you're pretty sure
that you have the right model.
If you wanna run those sort of,
the kind of nested model tests that we have so far
come up with the right model and then run these tests.
Once we are satisfied that we have the right model.
So that's a bit on how to test for it.
Once we've done our tests.
And if we find heteroscedasticity, what do we do about it?
What's the cure for the illness?
So there's three things that I wanna walk you through.
The first is if the structure is unknown,
then there's a way to recalculate the variance
and the standard error of each beta.
And we can use this in a T-test.
And then if we want to have a more efficient estimator,
there's both weighted least squares
and generalized least squares.
And one is for when we know the relationship between
X and U squared.
And when we don't.
So the first one is when know the relationship,
the second is when we do not.
Again homoscedasticity, well behaved data,
is the variance of U
given X1 through XK equals a constant.
If it's violated, we have heteroscedasticity
and variance of U is a function.
And we know again, there's no effect on biased,
no effect on our squared or our bar squared,
but the estimate of the variances have beta hat J,
all of them are biased.
So we need to come up with another way
to calculate the standard error.
Because if we just use OLS,
we can no longer use T-tests.
The T-stats, the confidence intervals are wrong.
T-stats no longer have a T distribution.
F-stats are not F distributed.
And this is a case where even a really big end cannot save.
OLS is not BLUE, it's not even efficient in large sample.
So having a big N, doesn't save you year.
The first thing I'll walk you through is
how to recalculate the standard error.
And again, this isn't something that I'm expecting
you to know how to do,
it isn't something you could do on the back of an envelope,
but I wanna walk you through
how you do it, just so you know,
and then if you need to do this,
you may wish to use another software package
because SPSS, I guess it has a macros,
but I haven't learned how to do that.
So as before, let's just start with a K goals Y model,
start with the simplest model.
And now we know that if we test
that the variance of UI is a function of X,
that it's sigma squared on.
Everybody has a different variance for their UI,
and that's a bad thing, and we need to deal with it.
So in this case, the variance would be
the sum of X minus X bar squared,
times sigma squared I, which is every individual's
times SSTX squared.
Where again, SSTX is the total,
variation in X.
So the general form is.
So you could in theory, go and get
each of these through SPSS and do the math by yourself.
So in theory.
So the way that you do this is
run the regression, get the UI and save and square them.
Get the X bar,
subtract X from it, square it, sum it
and then square that for the denominator
and plug everything in.
Again, it's something that you could do,
but it would be rather tedious.
For the K regressor case,
it's a little bit more complicated.
So here you need to do
this weird R here,
this funky R.
IJ which is the residual
for person I, of regressing XJ on all the other Xs
and SSRJ squared,
is the sum of squared residuals
of regressing XJ on all of the Xs.
So this would be a lot more complex.
You would have to do it a whole bunch of times.
I guess there is a macro that can do this in SPSS.
But other software would probably be easier.
So I'm just pulling this out of the textbook slides
that this is how you could do it.
So you see here, that's how you do it.
And then this is called the White
or Huber or Eicker errors.
And if you use these as the variance,
take the square root for the standard error,
use that as the denominator of your T-test.
And usually if you want to do an F-test for nested models,
a more sophisticated software would have this available
and could do it for you.
And it's always nice when they do this for you of course.
So again, the square root of this new way
to calculate the variants
is you take that the square root,
and that is the standard error.
Again, these are called the White, Huber, Eicker errors.
I don't know why they have so many names,
and I can't think of anybody any,
clever associations with them,
but again,
it's hard to do an SPSS.
Once you get those though,
we recall that a T-stat is
the hypothesis value so the beta hat
divided by the standard error.
So you could just use,
this is pretty easy to do
when you have homoscedasticity.
SPSS automatically has it.
But if you have heteroscedasticity,
you can calculate these new kinds of errors.
And in many cases
it's good to do it both ways
and report of what you come up with
and let the reader decide,
but always be clear about what you did
and why and what the results were.
And in many cases sort of let the reader
decide for themselves what they think is best.
So those sort of
new forms of standard errors is what you can do.
So you can do TNF tests.
If you really want a more efficient estimator,
then you can use what's called weighted least squares.
And if you do you get good T-stats, good F-stats,
and you can use those.
And one thing that if you know,
the functional form of your regressors
then you can use what's called weighted least squares.
And here where the variance of U
given X is some function of X, H of X,
and this would probably be guided by theory,
previous studies, et cetera. (Matthew coughs)
Note that HMX must be strictly greater than zero.
So you know from previous studies
that it's been shown that the variance
is this specific function of X.
The easiest case is when it's just a constant times XI.
So I talked about how, as income increases,
the variance might also increase
of the expenditure on some good like food.
So here we just divide through,
divide by H of X, which is in this case divide by income,
and then divide every term by that.
So large income folks with higher variants,
get a smaller weight here.
Small income folks with smaller variants,
get a larger weight.
So you're over weighing those,
that have a smaller variant,
underwing those that have a larger one.
There's some problems here,
with the interpretation,
but if this is the right functional form,
this will be an efficient estimator
and your TNF stats work.
So weighted least squares assumes that we know
the functional form.
And if we don't,
then we can use a very flexible functional form
that sort of works for everything
which is to use the exponential function,
also guarantees that it's strictly greater than zero,
and this is called feasible generalized least squares, FGLS.
These are the how-tos of
Feagles feasible generalized least squares
named for former NF fell punter Jeff Feagles.
So I know this is really mathy,
and I wouldn't really worry about all that much of it.
So you start with the equation of,
U squared as a function of your regressors,
you take the log and then you get the fitted values.
So sort of the Y-hats of this,
the UI-hat squared of once you
run this regress,
and then you rename that the G of I,
and you take the exponential function
and rename that H of I,
and then you weigh each term with one over H-hat of I.
Again, so you run a regression
of your main model, save the residuals,
square and take their log, regress that, on our Xs
to get our fitted values G-hat,
take the exponent of that and call that H-hat
and then weigh every observation with one over H-hat I.
So this is very mathy,
but what you see is
if there's a big U squared that
you will give them a smaller weight.
And if it's a small one, you'll get a big one.
So you're sort of over weighing less
precise estimates and under weighing more precise ones.
In general,
if you have heteroscedasticity,
weighted least squares is more efficient
and in many cases,
it's good to do it both ways and see why.
Sometimes it may be due to misspecification
instead of heteroscedasticity.
So always make sure that you do
your specification test first.
If there's strong heteroscedasticity,
those are robust estimators that we learned about are good.
And last slide, the recap.
So if you have heteroscedasticity, there's no bias,
but OLS is no longer BLUE, the error terms are bias.
The variance of the hat is biased,
and so the (indistinct) T-test don't work.
We learned about the tests, the Breush Pagan
and the White tests.
Of how to test for it, and then three solutions
so you could do these robust to standard errors,
if you're concerned about inference.
And if you want a more efficient estimator,
if the functional form is known,
you can use weighted least squares.
And if it's unknown,
you can use Feagles feasible generalized least squares.
And that's it thank you.