An information-theoretic alternative to maximum likelihood estimation method in ultrastructural measurement error model

In this paper, a data constrained generalized maximum entropy (GME) estimator for the general linear measurement error model is proposed. GME estimation, as developed by (A. Golan, G. Judge and D. Miller A Maximum Entropy Econometrics: Robust Estimation with limited data (Wiley, New York, 1996)), was formulated as a convex mixed-integer nonlinear optimization problem. Shannon entropy measures and its generalization, namely ‘entropy of order r’ by Tsallis and R´enyi are briefly discussed. A Monte Carlo comparison is made with the classical maximum likelihood estimation (MLE) method. The results show that, with moderate sample size; the GME outperforms the MLE estimators in terms of mean squared error.

PDF

___

[1]Al-Nasser, A. Measuring Customer Satisfaction: An Information - Theoretic Approach (Lambert Academic Publishing AG & Co. KG., Germany, 2010).
[2]Al-Nasser, A. Entropy type estimator to simple linear measurement error models, Austrian Journal of Statistics 34 (3), 283-294, 2005.
[3]Al-Nasser, A. Estimation of multiple linear functional relationships, Journal of Modern Applied Statistical Methods 3(1), 181-186, 2004.
[4]Al-Nasser, A. Customer satisfaction measurement models: Generalized maximum entropy approach, Pakistan Journal of Statistics 19(2), 213-226, 2003.
[5]Caputo, M and Paris, Q. Comparative statics of the generalized maximum entropy estimator of the general linear model, European Journal of Operational Research 185(1), 195-203, 2008.
[6]Carroll, R. J., Ruppert, D. and Stefanski, L. A. Measurement Error in Nonlinear Models (Chapman and Hall, London, 1995).
[7]Cheng, C. -L. and Van Ness, J. W. Statistical Regression with Measurement Error (Arlond, New York, 1999).
[8]Ciavolino, E and Al-Nasser, A. Comparing generalized maximum entropy and partial least squares methods for structural equation models, Journal of Nonparametric Statistics 21 (8), 1017-1036, 2009.
[9]Ciavolino, E and Dahlgaard, J. Simultaneous equation model based on the generalized maximum entropy for studying the effect of management factors on enterprise performance, Journal of Applied Statistics 36(7), 801-815, 2009
[10]Csiszar, I. Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems, The Annals of Statistics 19, 2032-2066, 1991.
[11]Dolby, G. R. The ultra-structural model: A synthesis of the functional and structural relations, Biometrika 63, 39-50, 1976.
[12]Donho, D. L, Johnstone, I. M, Hoch, J. C, and Stern, A. S. Maximum entropy and nearly black object, J. Royal, Statistical Society, Ser B 54, 41-81, 1992.
[13]Golan, A. Information and entropy econometrics - A review and synthesis, Foundations and Trends in Econometrics 2(1-2), 1-145, 2008.
[14]Golan, A., Judge, G. and Miller, D. A Maximum Entropy Econometrics: Robust Estimation with limited data (Wiley, New York, 1996).
[15]Golan, A., Judge, G. and PerlofF, J. Estimation and inference with censored and ordered multinomial response data, J. Econometrics 79, 23-51, 1997.
[16]Gleser, L.J. A note on G.R. Dolby's unreplicated ultrastructural model, Biometrika 72, 117-124, 1985.
[17]Havrada, J. H. and Charvat, F. Quantification methods of classification process: Concept of structural a-entropy, Kybernetika 3, 30-35, 1967.
[18]Jaynes, E. T. Information and Statistical Mechanics I, Physics Review 106, 620-630, 1957.
[19]Jaynes, E. T. Information and Statistical Mechanics II, Physics Review 108, 171-190, 1957.
[20]Jaynes, E. T. Information Theory and Statistical Mechanics, in Statistical Physics, K. Ford (ed.), (Benjamin, New York, 181, 1963).
[21]Kapur J. N. Maximum Entropy Models in Science and Engineering (John Wiley & Sons, New York, 1989)
[22]Paris, Q. Multicollinearity and maximum entropy estimators, Economics Bulletin 3(11), 1-9, 2001.
[23]Peeters, L. Estimating a random-coefficients sample-selection model using generalized maximum entropy, Economics Letters 84, 87-92, 2004.
[24]Pukelsheim, F. The three sigma rule, The American Statistician 48(2), 88-91, 1994.
[25]Renyi, A. On measures of information and entropy (Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, 1960), 547-561, 1961.
[26]Srivastava, A. and Shalabh, K. Consistent estimation for the non-normal ultrastructural model, Statist. Probab. Lett. 34, 67-73, 1997.
[27]Shannon, C,E. A mathematical theory of communication, Bell System Technical Journal 27, 379-423, 1948.
[28]Taneja, I. J. Generalized Information Measures and Their Applications, On-line book: http://www.mtm.ufsc.br/~taneja/book/book.html, 2001.
[29]Tsallis, C. Possible generalization of Boltzmann-Gibbs statistics, J. Statistical Physics 52, 479-487, 1988.