Multicollinearity in Multiple Linear Regression: Detection, Consequences, and Remedies

Abstract

Multiple linear regression is a widely used statistical tool for modeling relationships between a dependent variable and multiple explanatory variables. However, it assumes that these explanatory variables are independent, which is not always the case in practical scenarios, leading to a phenomenon known as multicollinearity. Multicollinearity occurs when explanatory variables in a regression model are strongly correlated with each other, causing several issues in regression analysis. This paper discusses the detection and remedies for multicollinearity in detail. Detection methods include examining the determinant of the correlation matrix, inspecting correlation coefficients, using partial regression coefficients, calculating Variance Inflation Factors (VIFs), and assessing the condition number and condition index. These techniques help researchers identify the presence and severity of multicollinearity in their dataset. To address multicollinearity, several remedies are proposed, including obtaining more data, dropping collinear variables, using relevant prior information, employing generalized inverses, and employing principal component regression. Ridge regression, which introduces bias to reduce variance, is also discussed as an effective technique to combat multicollinearity. Understanding multicollinearity and employing appropriate detection and remediation strategies is crucial for obtaining reliable and meaningful results from multiple linear regression models.

Introduction

References

[1] Vatcheva, K.P., Lee, M., McCormick, J.B., and Rahbar, M.H., “Multicollinearity in regression analysis conducted in epidemiologic studies,” Epidemiology (Sunnyvale, Calif.), 6 (2). 227. 2016. [2] \"Applied Multivariate Statistical Analysis\" by Richard A. Johnson and Dean W. Wichern. [3] Gunst, R.F. and Webster, J.T., “Regression analysis and problems of multicollinearity,” Communications in Statistics, 4 (3). 277-292. 1975. [4] Kleinbaum and David G 2008 Applied regression analysis and other multivariable methods (Australia; Belmont, CA: Brooks/Cole) 906 [5] \"Regression Analysis and Its Application: A Data-Oriented Approach\" by Richard F. Gunst and Robert L. Mason. [6] Debbie J Dupuis and Maria-Pia Victoria-Feser 2013 Robust VIF regression with application to variable Selection in large data sets The Annals of Applied Statistics 7 319-341 [7] Jensen D.R and Ramirez D.E. 2012 Variance Inflation in Regression, Advances in Decision Sciences 1-15 2013 [8] \"Linear Regression Analysis\" by George A. F. Seber and Alan J. Lee.

Copyright

Copyright © 2023 Siddamsetty Upendra, Dr. R. Abbaiah, Dr. P. Balasiddamuni. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.