Citation:
Anatolyev, Stanislav (2025) "Ridging out many covariates", Communications in Statistics - Theory and Methods, pp. 1-15
Abstract:
The paper considers a conditionally heteroskedastic linear regression setup with few regressors of interest and many nuisance covariates. We propose to subject the parameters corresponding to those nuisance covariates to a generalized ridge shrinkage. We show that under the assumption of dense random effects from the nuisance covariates, the ridge-out estimator of the parameters of interest is conditionally unbiased, and derive the optimal ridge intensity that delivers conditional efficiency. When tight structures on the variance of random effects are imposed, the asymptotic variance of the ridge-out estimator, under the dimension asymptotics, may be arbitrarily smaller than that of the least squares estimator. We also demonstrate how the optimal ridge-out estimator can be implemented under tight structures on the variance of random effects, and run simulation experiments where significant efficiency gains are possible to reach.
Link to paper online:
Paper in accepted version: