Determining Differential Item Functioning with the Mixture Item Response Theory

Purpose: Studies in the literature have generallydemonstrated that the causes of differential itemfunctioning (DIF) are complex and not directlyrelated to defined groups. The purpose of this studyis to determine the DIF according to the mixture item response theory (MixIRT) model, based on the latent group approach, as well as the Mantel-Haenszel method, based on the observed group approach, compare the results, and determine the possiblecauses of the DIF. Research Methods: As this studyis contributing to the production of information todevelop the theory, it is considered basic research. Inaccordance with the purposive sampling method, theresearch sample consisted of 1166 fourth-grade level students from Singapore, Kuwait, andTurkey who participated in the Trends in International Mathematics and Science Studymathematics application and took the sixth booklet. During the data analysis, the model thatadapted the data according to MixIRT was determined. Then, the status of the itemsdisplaying DIF was determined according to the adaptive model. Findings: According to theMixIRT, the two latent class models fit best to the data. No significant difference by genderwas observed in either class or any country. This finding suggests that the gender variable,which is frequently used as the observed group in DIF studies, should not be dealt with alone.Implications for Research and Practice: Since it is difficult to state whether an item isadvantageous for a subgroup when DIF is determined in accordance with known groups, it isrecommended to employ the latent class approach to determine DIF.

PDF

___

Camilli, G., & Shepard, L. A. (1994). MMSS: Methods for identifying biased test items. Thousand Oaks, CA: Sage.
Cho, S. J., & Cohen, A. S. (2010). A multilevel mixture IRT model with an application to DIF. Journal of Educational and Behavioral Statistics, 35(3), 336-370. doi: 10.3102/1076998609353111.
Choi, Y., Alexeev, N., & Cohen, A. S. (2015). Differential item functioning analysis using a mixture 3-parameter logistic model with a covariate on the TIMSS 2007 mathematics test. International Journal of Testing, 15(3), 239-253. doi: 10.1080/15305058.2015.1007241.
Clark, S. L. (2010). Mixture modeling with behavioral data. Unpublished doctoral dissertation. University of California, Los Angeles.
Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17(1), 31-44. doi: 10.1111/j.1745-3992.1998.tb00619.x
Cohen, A. S., & Bolt, D. M. (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42(2), 133–148. doi: 10.1111/j.1745-3984.2005.00007.
Cohen, A. S., Gregg, N., & Deng, M. (2005). The role of extended time and item content on a high-stakes mathematics test. Learning Disabilities Research & Practice, 20(4), 225–233. doi: 10.1111/j.1540-5826.2005.00138.x.
De Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: Guilford Press.
De Ayala, R. J., Kim, S. H., Stapleton, L. M., & Dayton, C. M. (2002). Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2(3&4), 243-276. doi: 10.1080/15305058.2002.9669495.
De Ayala, R. J. & Santiago, S. Y. (2017). An introduction to mixture item response theory models. Journal of School Psychology, 60, 25–40. doi: 10.1016/j.jsp.2016.01.002.
De Mars, C. E., & Lau, A. (2011). Differential item functioning detection with latent classes: How accurately can we detect who is responding differentially? Educational and Psychological Measurement, 71(4), 597–616. doi: 10.1177/0013164411404221.
Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel haenszel and standardization. In P. W. Holland, and H. Wainer, (Eds.), Differential item functioning (p. 35–66), New Jersey: USA.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.
Finch, W. H. & Finch, M. E. H. (2013). Investigation of specific learning disability and testing accommodations based differential item functioning using a multilevel multidimensional mixture item response theory model. Educational and Psychological Measurement, 73(6) 973–993. doi: 10.1177/0013164413494776.
Finch, W. H. & French, B. F. (2012). Parameter estimation with mixture item response theory models: A monte carlo comparison of maximum likelihood and bayesian methods. Journal of Modern Applied Statistical Methods, 11(1), 167-178. doi: 10.22237/jmasm/1335845580.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and application. Boston, MA: Kluwer Academic Publishers Group.
Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the mantel- haenszel procedure. In H. Wainer, and H. I. Brown (Eds.), Test validity (p. 129- 145). Hillsdale, NJ: Lawrence Erlbaum Associates.
Holland, P. W., & Wainer, H. (1993). Differential item functioning. Hillsdale, NJ: Erlbaum.
Kelderman, H., & Macready, G. B. (1990). The use of loglinear models for assessing differential item functioning across manifest and latent examinee groups. Journal of Educational Measurement, 27(4), 307–327.
Li, F., Cohen, A. S., Kim, S., & Cho, S. (2009). Model selection methods for mixture dichotomous IRT models. Applied Psychological Measurement, 33(5), 353–373. doi: 10.1177/0146621608326422.
Magis, D., Béland, S., Tuerlinckx, F., & De Boeck, P. (2015). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847-862. doi:10.3758/BRM.42.3.847.
Maij-de Meij, A. M., Kelderman, H. & van der Flier, H. (2010). Improvement in detection of differential item functioning using a mixture item response theory model. Multivariate Behavioral Research, 45(6), 975-999. doi:10.1080/00273171.2010.533047.
Mellenbergh, G. J. (1989). Item bias and item response theory. International Journal of Educational Research, 13(2), 127-143. doi: 10.1016/0883-0355(89)90002-5.
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 international results in mathematics. Retrieved from Boston College, TIMSS & PIRLS International Study Center. Retrieved December 12, 2017, from http://timssandpirls.bc.edu/timss2015/international-results/
Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (Eighth Edition). Los Angeles, CA: Muthén & Muthén.
Ong, Y. M., Williams, J., & Lamprianou, I. (2013). Exploring differential bundle functioning in mathematics by gender: The effect of hierarchical modelling. International Journal of Research & Method in Education, 36(1), 82-100. doi: 10.1080/1743727X.2012.675263.
Osterlind, S. J., & Everson, H. T. (2009). Differential item functioning. Los Angeles: Sage.
Samuelsen, K. M. (2008). Examining differential item functioning from a latent mixture perspective. In G. R. Hancock & K. M. Samuelsen (Eds.), Advances in latent variable mixture models (p. 177-197). Charlotte, NC: Information Age.
Skrondal, A. & Rabe-Hesketh, S. (2007). Latent variable modelling: A survey. Scandinavian Journal of Statistics, 34(4), 712–745. doi: 10.1111/j.1467- 9469.2007.00573.x.
Tay, L., Newman, D. A., & Vermunt, J. K. (2011). Using mixed-measurement item response theory with covariates (MM-IRT-C) to ascertain observed and unobserved measurement equivalence. Organizational Research Methods, 14(1), 147-176.
Uyar, S., Kelecioglu, H., & Dogan, N. (2017). Comparing differential item functioning based on manifest groups and latent classes. Educational Sciences: Theory & Practice, 17(6), 1977–2000. doi: 10.12738/estp.2017.6.0526.
Vermunt, J. K., & Magidson, J. (2002). Latent class cluster analysis. In J. A. Hagenaars, & A. L. McCutcheon, Applied latent class analysis (p. 89-107). New York: Cambridge University Press.
von Davier, M., & Rost, J. (2017). Logistic mixture-distribution response models. In W. J. van der Linden (Ed.), Handbook of item response theory, volume one: Models (p. 393-406). Boca Raton: Chapman and Hall/CRC.
von Davier, M., & Yamamoto, K. (2004). Partially observed mixtures of IRT models: An extension of the generalized partial credit model. Applied Psychological Measurement, 28(6), 389-406.
Yalcin, S. (2017). The effect of latent classes formed according to the affective characteristics of students on differential item functioning based on sex.Elementary Education Online, 16(4), 1917-1931. doi: 10.17051/ilkonline.2017.364505.
Yuksel, S. (2012). Olceklerde saptanan madde islev farklılıgının karma rasch modelleri ile incelenmesi (Analyzing differential item functioning by mixed rasch models which stated in scales). Yayımlanmamıs Doktora tezi. Ankara Universitesi Saglik Bilimleri Enstitusu, Ankara.