Paper not ready for distribution When working with tax and regulatory variables, it is important to carefully examine the effect of
multicollinearity because when
two explanatory variables in a regression model are positively
correlated, their regression coefficients will be negatively correlated
and one of the OLS coefficients might have the "wrong" sign.
A
strict interpretation of the OLS coefficients would therefore lead the
researcher to conclude that a particular tax or regulatory variable has
no effect or the effect opposite of the true effect on the outcome of
interest. Ridge regression helps diagnose such problems by tracing the
paths of the coefficients as they are shrunk towards zero.
To better understand the effect of multicollinearity, the rest of this page takes a simple dataset from Obenchain (2004) and places it in the context of a public policy analysis.
Suppose
you have data on growth rates and cigarette and alcohol tax rates by
state. Suppose also that there are strong theoretical reasons to
believe that raising tax rates on cigarettes and alcohol increases a
state’s rate of economic growth.
correlation matrix (fictional data)
growth rate
alcohol tax rate
cigarette tax rate
growth rate
1.00
alcohol tax rate
0.94
1.00
cigarette tax rate
0.79
0.87
1.00
Common
sense suggests that higher cigarette tax rates should be associated
with higher growth rates, but when you run the regression:
dep. variable: growth rate
alcohol tax rate
1.03
**
(0.24)
cigarette tax rate
−0.11
(0.24)
Because
the two tax rates are positively correlated with each other, their
regression coefficients are negatively correlated with each other.
Moreover,
common sense suggests that increasing the alcohol tax rate 5 percentage
points won't increase the growth rate 5 percentage points
and increasing the cigarette tax rate won't reduce the growth rate.
Something's wrong here.
If we account for the ill-conditioning
by effectively reducing the number of dimensions along which the
dependent variable is regressed, we obtain biased estimates of the
regression coefficients, but:
the regression coefficients may have lower mean-squared error and
we may correct the "wrong" signs.
Two questions arise when using Ridge Regression:
What is the best path along which to shrink the coefficients to zero?
At what point along that path should you stop shrinking?
There are objective answers to those questions, but the answers are not wholly satisfactory.
Shrinkage Paths -- Obenchain (2004)
Q = 0 in Hoerl and Kennard's (1970) shrinkage path
Q = negative infinity in Principal Components Regression
Notice that in Principal Components Regression:
The cigarette tax coefficient is 0.46.
The alcohol tax coefficient is also 0.46.
Both coefficients are outside their 95 percent confidence intervals (as measured by the OLS estimates)
Obenchain
(1977) proves that the OLS t-statistic equals the Ridge t-statistic and
argues that practitioners should center their confidence intervals on
the OLS estimates. For this reason, I report the OLS estimates and
their standard errors in my regression tables.
I also include graphs of ridge traces in my papers, so that we can examine the stability of the regression coefficients
as they are shrunk towards zero. The ridge traces help diagnose cases
in which the OLS regression coefficients have the "wrong" sign.
Checking the stability of the coefficients can prevent us from concluding that a tax or regulatory variable's effect on the outcome of
interest is not the opposite
of its true effect. In this way, use of ridge regression can help us
provide public policymakers with better guidance on the true effect of
measures that they are considering. References: A. E. Hoerl and R. W. Kennard. "Ridge Regression: Biased Estimation for Nonorthogonal Problems." Technometrics, 12(1):55-67, Feb. 1970. R. L. Obenchain. "Classical F-Tests and Confidence Regions for Ridge Regression." Technometrics, 19(4):429-439, Nov. 1977. R. L. Obenchain. Shrinkage Regression: ridge, BLUP, Bayes, spline and Stein.Mar. 2004.
