| Eric Doviak | Doviak.net
Economics and Public Policy Analysis |
Income Inequality
Health Insurance
Ridge Regression
Graduate MACRO
INTRO
intro lecture notes
email me
Download the paper (PDF)
When working with tax and regulatory variables, it is important to carefully examine the effect of multicollinearity because when two explanatory variables in a regression model are positively correlated, their regression coefficients will be negatively correlated and one of the OLS coefficients might have the "wrong" sign.
A strict interpretation of the OLS coefficients would therefore lead the researcher to conclude that a particular tax or regulatory variable has no effect or the effect opposite of the true effect on the outcome of interest. Ridge regression helps diagnose such problems by tracing the paths of the coefficients as they are shrunk towards zero.
To better understand the effect of multicollinearity, the rest of this page takes a simple dataset from Obenchain (2004) and places it in the context of a public policy analysis.
Suppose you have data on growth rates and cigarette and alcohol tax rates by state. Suppose also that there are strong theoretical reasons to believe that raising tax rates on cigarettes and alcohol increases a state’s rate of economic growth.
| correlation matrix (fictional data) | ||||||||||||||||
|
Common sense suggests that higher cigarette tax rates should be associated with higher growth rates, but when you run the regression:
| dep. variable: growth rate | ||||||||||||
|
Because the two tax rates are positively correlated with each other, their regression coefficients are negatively correlated with each other.
Moreover, common sense suggests that increasing the alcohol tax rate 5 percentage points won't increase the growth rate 5 percentage points and increasing the cigarette tax rate won't reduce the growth rate. Something's wrong here.
If we account for the ill-conditioning by effectively reducing the number of dimensions along which the dependent variable is regressed, we obtain biased estimates of the regression coefficients, but:
- the regression coefficients may have lower mean-squared error and
- we may correct the "wrong" signs.
 |  |
Two questions arise when using Ridge Regression:
- What is the best path along which to shrink the coefficients to zero?
- At what point along that path should you stop shrinking?
| Shrinkage Paths -- Obenchain (2004) | ||
|
- Q = 0 in Hoerl and Kennard's (1970) shrinkage path
- Q = negative infinity in Principal Components Regression
- The cigarette tax coefficient is 0.46.
- The alcohol tax coefficient is also 0.46.
- Both coefficients are outside their 95 percent confidence intervals (as measured by the OLS estimates)
I also include graphs of ridge traces in my papers, so that we can examine the stability of the regression coefficients as they are shrunk towards zero. The ridge traces help diagnose cases in which the OLS regression coefficients have the "wrong" sign.
Checking the stability of the coefficients can prevent us from concluding that a tax or regulatory variable's effect on the outcome of interest is not the opposite of its true effect. In this way, use of ridge regression can help us provide public policymakers with better guidance on the true effect of measures that they are considering.
References:
A. E. Hoerl and R. W. Kennard. "Ridge Regression: Biased Estimation for Nonorthogonal Problems." Technometrics, 12(1):55-�67, Feb. 1970.
R. L. Obenchain. "Classical F-Tests and Confidence Regions for Ridge Regression." Technometrics, 19(4):429-�439, Nov. 1977.
R. L. Obenchain. Shrinkage Regression: ridge, BLUP, Bayes, spline and Stein. Mar. 2004.