Combating Multicollinearity: A New Two-Parameter Approach

Sıradan en küçük kareler (OLS) tahmincisi, tüm doğrusal regresyon modeli varsayımları geçerli olduğunda En İyi Doğrusal Yansız Tahmin Edicidir (MAVİ). Bununla birlikte, OLS tahmincisi, çoklu bağlantının varlığında verimsiz hale gelir. Çoklu bağlantı problemini aşmak için çeşitli bir ve iki parametreli tahmin ediciler önerilmiştir. Bu makale, Liu-Kibria Lukman Tahmincisi (LKL) tahmincisi olarak adlandırılan yeni bir iki parametreli tahmin edicidir. Teorik ve simülasyon sonuçları, önerilen tahmin edicinin, ortalama kare hata kriteri kullanılarak bazı koşullar altında bu çalışmada ele alınan bazı mevcut tahmin edicilerden daha iyi performans gösterdiğini göstermektedir. Portland çimentosu ve Longley veri kümelerine yapılan gerçek hayattaki bir uygulama, teorik ve simülasyon sonuçlarını destekledi.

Combating Multicollinearity: A New Two-Parameter Approach

The ordinary least square (OLS) estimator is the Best Linear Unbiased Estimator (BLUE) when all linear regression model assumptions are valid. The OLS estimator, however, becomes inefficient in the presence of multicollinearity. To circumvent the problem of multicollinearity, various one and two-parameter estimators have been proposed. This paper a new two-parameter estimator called Liu-Kibria Lukman Estimator (LKL) estimator. The theoretical and simulation results show that the proposed estimator performs better than some existing estimators considered in this study under some conditions, using the mean square error criterion. A real-life application to Portland cement and Longley datasets supported the theoretical and simulation results.

___

  • [1] Chatterjee, S. and Hadi, A. S., Regression Analysis by Example, 4th ed., NJ: John Wiley and Sons, 2006.
  • [2] Chatterjee, S., Hadi, A. S. and Price, B., Regression by example, 3rd ed., New York: John Wiley and Sons, 2000.
  • [3] G. S. Maddala, Introduction to econometrics, 3rd ed., England: John Willey and Sons Limited, 2002.
  • [4] W. H. Greene, Econometric analysis, 5th ed., New Jersey: Prentice-Hall Saddle River, 2003.
  • [5] Lukman A.F. and Ayinde, K., "Review and classifications of the ridge parameter estimation techniques," Hacettepe Math Stat., vol. 46, no. 5, p. 953‐967, 2017.
  • [6] W. F. Massy, "Principal component regression in exploratory statistical research," Journal of the American Statistical Association, vol. 60, p. 234 –246, 1965.
  • [7] D. W. Marquardt, "Generalized inverse, ridge regression, biased linear estimation and non – linear estimation," Technometrics, vol. 12, p. 591–612, 1970.
  • [8] Naes, T., and Marten, H., "Principal component regression in NIR analysis: Viewpoints, background details selection of components," Journal of Chemometrics, vol. 2, p. 155 – 167, 1988.
  • [9] A. E. Hoerl, "Application of ridge analysis to regression problems," Chemical Engineering Progress, vol. 58, p. 54 –59, 1962.
  • [10] Hoerl, A. E., and Kennard, R. W., "Ridge regression biased estimation for non-orthogonal problems," Technometrics, p. 27–51, 1970.
  • [11] I. S. Helland, "On the structure of partial least squares regression," Communication is Statistics, Simulations and Computations, vol. 17, p. 581–607, 1988.
  • [12] I. S. Helland, "Partial least squares regression and statistical methods," Scandinavian Journal of Statistics, vol. 17, p. 97 – 114, 1990.
  • [13] Phatak, A. and Jony, S. D., "The geometry of partial least squares," Journal of Chemometrics, vol. 11, p. 311–338, 1997.
  • [14] Lukman, A. F., K. Ayinde, S. K. Sek, and E. Adewuyi, "A modified new two-parameter estimator in a linear regression model," Modelling and Simulation in Engineering 2019:6342702, 2019.
  • [15] Kibria, B.M.G. and Lukman, A.F., "A New Ridge-Type Estimator for the Linear Regression Model: Simulations and Applications," Scientifica, p. 1–16, 2020.
  • [16] K. Liu, "A new class of biased estimate in linear regression," Commun. Stat.-Theory and Methods, vol. 22, p. 393–402, 1993.
  • [17] Yang, H. and Chang, X., "A new two-parameter estimator in linear regression," Commun. Stat.-Theory and Methods, vol. 39, no. 6, p. 923–934, 2010.
  • [18] Kaciranlar, S., Sakallioglu, S., Akdeniz, F., Styan, G.P.H. and Werner, H.J., "A new biased estimator in linear regression and a detailed analysis of the widely– analyzed dataset on portland cement," Sankhya 61, p. 443–459, 1999.
  • [19] Wang, S.G., Wu, M.X. and Jia, Z.Z., Matrix Inequalities, 2nd ed., Beijing: Chinese Science Press, 2006.
  • [20] R. Farebrother, "Further results on the mean square error of ridge regression," J R Stat Soc. , vol. B38, p. 248‐250, 1976.
  • [21] Trenkler, G. and Toutenburg, H. , "Mean squared error matrix comparisons between biased estimators-an overview of recent results," Stat Pap. , vol. 31, no. 1, p. 165‐179, 1990.
  • [22] A. V. Dorugade, "New Ridge Parameters for Ridge Regression," Journal of the Association of Arab Universities for Basic and Applied Sciences, pp. 1-6, 2016.
  • [23] Saleh, A. K. M. E.; Arashi, M. and Kibria, B. M. G., Theory of Ridge Regression Estimation with Applications., New Jersey: Wiley, Hoboken., 2019.
  • [24] Aslam, M. & Ahmad, S., "The modified Liu-ridge-type estimator: a new class of biased estimators to address multicollinearity," Communications in Statistics - Simulation and Computation, 2020.
  • [25] McDonald, G. C., and Galarneau, D. I., "A Monte Carlo evaluation of some ridge-type estimators," Journal of the American Statistical Association, vol. 70, pp. 407-416, 1975.
  • [26] Wichern, D. and Churchill, G., "A Comparison of Ridge Estimators," Technometrics, vol. 20, p. 301–311, 1978.
  • [27] B. M. Kibria, "Performance of some new ridge regression estimators," Communications in Statistics - Simulation and Computation, vol. 32, pp. 419-435, 2003.
  • [28] Newhouse, J. P., and Oman, S. D., "An evaluation of ridge estimators.," A report prepared for the United States air force project RAND, 1971.
  • [29] Woods, H.; Steinour, H. H., and Starke, H. R., "Effect of composition of Portland cement on heat evolved during hardening," Industrial & Engineering Chemistry, vol. 24, no. 11, p. 1207–1214, 1932.
  • [30] Li, Y. and Yang, H., "A new Liu-type estimator in a linear regression model," Stat. Pap. , vol. 53, p. 427–437, 2012.
  • [31] Ayinde, K.; Lukman, A.F.; Samuel, O.O.; & Ajiboye, S.A., "Some new adjusted ridge estimators of linear regression model," Int. J. Civ. Eng. Technol., vol. 9, no. 11, p. 2838‐2852, 2018.
  • [32] J. Longley, "An appraisal of least-squares programs for electronic computer from the point of view of the user," J. Am. Stat. Assoc., vol. 62, p. 819–841, 1967.
  • [33] Dawoud, I. and Kibria, B. M. G., "A New Biased Estimator to Combat the Multicollinearity of the Gaussian Linear Regression Model," Stats, vol. 3, no. 4, pp. 526-541, 2020.