Çoklu Doğrusallık ve Değişen Varyans Altında Farklı Ridge Parametrelerinin Bir Karşılaştırması

Uygun bir doğrusal regresyon modeli tahmin edilmesi sırasında karşılaşılan ana problemlerden biri bağımsız değişkenler yüksek korelasyona sahip olduğu zaman ortaya çıkan çoklu doğrusallıktır. Bu sorunun giderilmesi için sıradan en küçük karelere bir alternatif yöntem olarak tanıtılan ridge regresyon tahmincisi kullanılmaktadır. Sabit varyanslar varsayımını bozan değişen varyans durumu, regresyon tahmininde diğer ana sorunlardan biridir. Daha sağlam bir doğrusal regresyon eşitliği tahmin edebilmek için bu bozulma sorununa çözüm olarak ağırlıklı en küçük kareler tahmini kullanılır. Ancak, hem çoklu doğrusallık hem de değişen varyans sorunu mevcut olduğunda, ağırlıklı ridge regresyon tahminine başvurulmalıdır. Ridge regresyon, kesin bir hesaplama formülü bulunmayan ridge parametresine bağlıdır. Literatürde önerilen bir çok ridge parametresi bulunmaktadır. Hem çoklu doğrusallık hem de değişen varyans içeren veri için bu ridge parametrelerinin performanslarını analiz etmeye yönelik bir simülasyon çalışması düzenlenmiştir. Farklı örnek hacimleri, farklı bağımsız değişken sayıları ve farklı çoklu doğrusallık dereceleri kullanılmıştır. Ridge parametrelerinin performansları ortalama hata kareleri değerleri göz önüne alınarak karşılaştırılmıştır. Çalışma aynı zamanda, verinin hem çoklu doğrusallık hem de değişen varyansa sahip olduğu durumda, ridge parametrelerinin performanslarının, verinin sadece çoklu doğrusallığa sahip olduğu durumdakinden farklı olduğunu göstermiştir.

Anahtar Kelimeler:

Çoklu doğrusallık, Ridge parametresi, Değişen varyans, Ridge regresyon, Ağırlıklı ridge regresyon

A Comparison of Different Ridge Parameters under Both Multicollinearity and Heteroscedasticity

One of the major problems in fitting an appropriate linear regression model is multicollinearity which occurs when regressors are highly correlated. To overcome this problem, ridge regression estimator which is an alternative method to the ordinary least squares (OLS) estimator, has been used. Heteroscedasticity, which violates the assumption of constant variances, is another major problem in regression estimation. To solve this violation problem, weighted least squares estimation is used to fit a more robust linear regression equation. However, when there is both multicollinearity and heteroscedasticity problem, weighted ridge regression estimation should be employed. Ridge regression depends on the ridge parameter which does not have an explicit form of calculation. There are various ridge parameters proposed in the literature. A simulation study was conducted to compare the performances of these ridge parameters for both multicollinear and heteroscedastic data. The following factors were varied: the number of regressors, sample sizes and degrees of multicollinearity. The performances of the parameters were compared using mean square error. The study also shows that when the data are both heteroscedastic and multicollinear, the estimation performances of the ridge parameters differs from the case for only multicollinear data.

Keywords:

Multicollinearity, Ridge parameter, Heteroscedasticity, Ridge regression, Weighted ridge regression,

PDF

___

[1] Hoerl, A. E., Kennard, R. 1970a. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12(1)(1970a), 55-67.
[2] Hoerl, A.E. and Kennard, R. 1970b. Ridge Regression: Applications to Nonorthogonal Problems. Technometrics 12(1)(1970b), 69-82.
[3] Hoerl, A. E., Kennard, R. and Baldwin, K. 1975. Ridge Regression: Some Simulations. Communications in Statistics. – Simulation and Computation, 4(2)(1975), 105-123.
[4] Lawless, J., Wang, P. A. 1976. Simulation Study of Ridge and Other Regression Estimators. Communications in Statistics – Theory and Methods, 5(4)(1976), 307-323.
[5] Schaeffer, R.L., Roi, L.D., Wolfe, R. A. 1894. A Ridge Logistic Estimator. Communications in Statistics - Theory and Methods, 13(1)(1984), 99-113.
[6] Nomura, M. 1988. On The Almost Unbiased Ridge Regression Estimator. Communications in Statistics - Simulation and Computation, 17(3)(1988), 729-743.
[7] Kibria, B. M. G. 2003. Performance of Some New Ridge Regression Estimators. Communications in Statistics - Simulation and Computation, 32(2)(2003), 419-435.
[8] Khalaf, G., Shukur, G 2005. Choosing Ridge Parameter for Regression Problems. Communications in Statistics - Theory and Methods, 34(5)(2005), 1177-1182.
[9] Norliza, A., Maizah, H. A., Ahmad, R. A. 2006. A Comparative Study On Some Methods for Handling Multicollinearity Problems. Mathematika, 22(2)(2006), 109-119.
[10] Alkhamisi, M. A., Shukur, G. 2007. A Monte Carlo Study of Recent Ridge Parameters. Communications in Statistics - Simulation and Computation, 36(3)(2007), 1177-1182.
[11] Batah, F. S., Ramnathan, T., Gore, S. D. 2008. The Efficiency of Modified Jackknife and Ridge Type Regression Estimators: A Comparison. 24(2)(2008), 111-122.
[12] Muniz, G., Kibria, B. M. G. 2009. On Some Ridge Regression Estimators: An Empirical Comparisons. Communications in Statistics - Simulation and Computation 38(3)(2009), 621-630.
[13] Kibria, B. M. G., Månsson, K., Shukur, G. 2011. Performance of Some Logistic Ridge Regression Estimators. Comput Econ., 40.4 (2011), 401-414.
[14] Dorugade, A. V. 2014. On Comparison of Some Ridge Parameters in Ridge Regression. Sri Lankan Journal of Applied Statistics, 15(1)(2014), 31-46.
[15] Asar, Y., Karaibrahimoğlu, A., Genç, A. 2014. Modified Ridge Regression Parameters: A Comparative Monte Carlo Study. Hacettepe Journal of Mathematics and Statistics, 43(5)(2014), 827-841.
[16] Göktaş, A., Sevinç, V. 2016. Two New Ridge Parameters and A Guide for Selecting an Appropriate Ridge Parameter in Linear Regression. Gazi University Journal of Science, 29(1)(2016), 201-211.
[17] Macedo, P., Scotto, M., Silva, E. 2010. On the Choice of the Ridge Parameter: A Maximum Entropy Approach. Communications in Statistics - Simulation and Computation, 39(8)(2010), 1628-1638.
[18] Månsson, K., Shukur, G., Kibria, B. M. G. 2010. A Simulation Study of Some Ridge Regression Estimators under Different Distributional Assumptions. Communications in Statistics - Simulation and Computation, 39(8)(2010), 1639-1670.
[19] Månsson, K., Shukur, G. 2011. On Ridge Parameters in Logistic Regression. Communications in Statistics - Theory and Methods, 40(18)(2011), 3366-3381.
[20] Salam, M. E. F. A. E. 2015. Alternative Ridge Robust Regression Estimator for Dealing with Collinear Influential Data Points. International Journal of Contemporary Mathematical Sciences, 10(2015), 119-130.
[21] Khalaf, G. 2012. A Proposed Ridge Parameter to Improve the Least Square Estimator. Journal of Modern Applied Statistical Methods, 11(2)(2012), 443-449.
[22] Hamed, R., Hefnawy, A. E., Farag, A. 2013. Selection of the Ridge Parameter Using Mathematical Programming. Communications in Statistics - Simulation and Computation, 42(6)(2013), 1409-1432.
[23] Månsson, K., Shukur, G., Sjölander, P. 2013. A New Ridge Regression Causality Test in the Presence of Multicollinearity. Communications in Statistics - Theory and Methods, 43(2)(2013), 235-248.
[24] Dorugade, A. 2015. Correlation Based Ridge Parameters in Ridge Regression with Heteroscedastic Errors and Outliers. Journal of Statistical Theory and Applications, 14(4)(2015), 413-424.
[25] Wong, K. Y., Chiu, S. N. 2015. An Iterative Approach to Minimize the Mean Squared Error in Ridge Regression. Computational Statistics, 30(2)(2015), 625-639.
[26] Somahi, A. A., Mousa, S., Turk, L. I. 2015. Some New Proposed Ridge Parameters for the Logistic Regression Model. International Journal of Research in Applied, Natural and Social Sciences, 3(1)(2015), 67-82.
[27] Duzan, H., Shariff, N. S. 2016. Solution to the Multicollinearity Problem by Adding some Constant to the Diagonal. Journal of Modern Applied Statistical Methods, 15(1)(2016), 752-773.
[28] Kibria, B. M. G., Banik, S. 2016. Some Ridge Regression Estimators and Their Performances. Journal of Modern Applied Statistical Methods, 15(1)(2016), 206-238.
[29] Alibuhtto, M. C. 2016. Relationship Between Ridge Regression Estimator and sample Size When Multicollinearity Present Among Regressors. World Scientific News, 59(2016), 12-23.
[30] Lukman, A. F., Ayinde, K. 2016. Some Improved Classification-Based Ridge Parameter of Hoerl and Kennard Estimation Techniques. İstatistik: Journal of the Turkish Statistical Association, 9(3)(2016), 93-106.
[31] Bhat, S., Raju, V. 2016. A Class of Generalized Ridge Estimators. Communications in Statistics - Simulation and Computation, 46(7)(2016), 5105-5112.
[32] Uzuke, C.A., Mbegbu, J.I., Nwosu C. R. 2017. Performance of Kibria, Khalaf, and Shurkurs Methods When the Eigenvalues are Skewed. Communications in Statistics - Simulation and Computation, 46(3)(2017), 2071-2102.
[33] Macedo, P. 2017. Ridge Regression and Generalized Maximum Entropy: An Improved Version of the Ridge–GME Parameter Estimator. Communications in Statistics - Simulation and Computation, 46(5)(2017), 3527-3539.
[34] Lukman, A. F., Ayinde, K., Ajiboye, A. S. 2017. Monte Carlo Study of Some Classification-Based Ridge Parameter Estimators. Journal of Modern Applied Statistical Methods, 16(1)(2017), 428-451.
[35] Giacalone, M., Panarello, D., Mattera, R. 2017. Multicollinearity in Regression: An Efficiency Comparison Between Lp-Norm and Least Squares Estimators. Quality & Quantity, 52(4)(2017), 1831–1859.