Kübra ATALAY, Bilge GÖK, Nihan ARSAN, Hülya KELECİOĞLU

Değişen madde fonksiyonunun belirlenmesinde kullanılan farklı yöntemlerin karşılaştırılması: Bir simülasyon çalışması

Bu araştırmanın amacı değişen madde fonksiyonunun (DMF) belirlenmesinde kııllanilan gözlenen puan yöntemlerinden Mantel-Haenszel ve lojistik regresyotı yöntemleri ile örtük puan yöntemlerinden MTK-00 ve SİBTFST yöntemlerinin karşılaştırılmasıdır. Yöntemlerin karşılaştırılması simulasyon çalışmasıyla yapılmıştır. Simıılasyon koşulları önıeklem büyüklüğü, yetenek dağılımı ve testteki DMF'li madde oranıdır. Araştırmada kullanılan örtük puan yöntemlerinin gözlenen puan yöntemlerine göre DMF'li maddeleri belirlemede daha duyarlı ve etkili olduğu görülmüştür. Araştırmada. UVnin diğer yöntemlere göre DMF belirleme oranının daha diişiik olduğu, MTK-00'nun ise daha yükseli olduğu sonucuna varılmıştır Tek biçimli DMF'yi belirlemede MH, SİBTEST ve MTK-00 yöntemleri; tek biçimli olmayan DM'F'yi belirlemede ise U SIBTEST ve MTK-00 yöntemleri birbiriyle uyumlu sonuçlar vermiştir.

Comparing different diffential item functioning methods: A simulation study

The primary purpose of this research is to compare and evaluate the effectiveness of observed score methods -Mantel-Haenszel. logistic regression- and latent score methods -IRT-LR, SIBTEST- which used to determine DIF under variety conditions. These methods were compared by simulation study. Sample sizes, ability distribution, proportion of items with DIF were considered for data simulation conditions. Results of this research revealed that latent score methods were more sensitive and effective in determining items with DIF rather than observed score methods. Latent score methods were more liberal and observed score methods were more conservative in identifying items with DIF. As a result, MH, SIBTEST and 1RT-LR methods present consistent result in determining uniform DIF in all conditions, l^urthermore. consistent results were found in identifying non-uniform DIF with LR, SIBTEST and IRT-LR methods.

PDF

___

Agresti, A. (1984). Analysis of ordinal categorical data. Newyork: John Wiley & Sons.
Atar B. (2007). Differential item functioning analyses for mixed response data using hi likelihood raiiotest, logistic regression tiial gllamnt procedures. Unpublished doctoral thesis. The Florida State University, Florida State.
Bakan Kalaycioğlu, D. (2009). Öğrenci Seçme Sınavı'run madde yanlılığı açısından incelenmesi. Yayınlanmamış doktora tezi. Hacettepe Üniversitesi, Sosyal Bilimler Enstitüsü, Ankara.
Bekci, B. (-007). Orta Öğretim Kurumları Öğrenci Seçme re Yerleştirme Sınavı 'run değişen madde fonksiyonlarının cinsiyete ve oku! türüne göre incelenmesi. Yiiksek lisans tezi. Hacettepe Üniversitesi, Sosyal Bilimler Enstitüsü, Ankara.
Berbcroğlu, G., ve Kalaycioğlu Bakan, D. (2010, in Press}. Differential item functioning analysis of the science and mathematics items in the university entrance examinations in Turkey. Journal of Psychoeducational Assessment.
Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational measurement (4liled.,Vol. 4, pp. 221-256). Westport: American Council on Education & Praeger Publishers.
Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. London: Sage Publications.
Chang. Mazzeo, J., & Roussos, L. (1996). Detecting DIP for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33(3), 333-353.
Cheng (2005). A study on differential item functioning of the basic mathematical competence test for junior high schools in Taiwian. Unpublished doctoral thesis, Michigan State University.
Clauser, B. E.. & Mazor, IC. M. (1998). Using statistical procedures to identify differential item functioning test items. Educational Measurement: Issues and Practice, .17, 31-44.
Clauser, E. B., Mazor, K., & Hambleton, ICR. (1993). The effects of purification of the matching criterion on the identification of DIP using t h e Mantel - Haenszel procedure. Applied Measurement in Education, 6(4), 269-279.
Dorans, N. J., & Holland, P.W. (1993). DIF detection and description: Mantel-FIaenszel and Standardization. Dijferantial item functioning, in P.W. Holland & H.Wainer (Eds.), Differential item functioning (pp. 35-66). Hillsdale. NJ: Lawrence Erlhaum Associates.
Fidalgo. A. M., Ferreres, D., & Muniz, .1. (2004). Liberal and conservative differential item functioning detection using Mantel-Haenszel and SIBTEST: implications for Type I and Type II error rates. Journal of Experimental Education, 75(1), 23-29.
Finch, W. H., & French, B.F. (2007). Detection of crossing differential item functioning: Comparison of four methods. Educational and Psychological Measurement, 67. 4, 565-582.
Gno, L„ & Wang, C. (2005). Using jive procedures to detect DIF with passage-based testlets. A paper prepared for the Poster Presentation at the Graduate Student Poster Session at the Annual Meeting of the National Council of Measurement in Education, Montreal, Quebec.
Gierl, M. J.. Jodoin, M., & Ackerman T. (2000). Performance of niantei-haenszel, simultaneous item bias test and logistic regression when the proportion of dif items is large. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, Louisiana, USA.
Goniez-Benito, J.. & Navas-Ara, M. J. (2000). A comparison of Chi-square, RFA and IRT based procedures in the detection of DIF. Quality and Quantity. 34, 17-31.
Güler. N.. & Penfıeld, R. (2009). A comparison of the logistic regression and contingency table methods for the simultaneous detectionb of uniform a nonuniform differential item functioning. Journal of Educational MeasurementA63.
Hambleton, R. K., Swaminathan, II., & Rogers, H. J. (1991). Fundamentals of item response theory. California: Sage Publications
Han, K. T. (2007). WinGen2: Windows software that generates IRT parameters and item responses [computer program]. Amherst, MA: Center for Educational Assessment, University of Massachusetts Amherst. 10 Aralık 2010 tarihinde http://people.umass.edu/kha/wingen/ adresindenjndirilmİştir.
Hidalgo. M. D., & Lopez-Pina, J. A. (2004). Differential item functioning detection an effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64(6), 903-915.
Jodoin, M. G., & Gierl, M. J. (2001). Evaluating Type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329-349.
Li, H.-H., & Stout, W. (1996). A new procedure for detection of crossing DfF. Psychometrika, 61, 647-677.
Miller, T. R., & Spray, J. A. (1993). Logistic discrimnant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement, 30(2), 107-122.
Narayanan, P., & Swaminathan, FI. (1994). Performance of the Mantel-FIaenszel and simultaneous item bias procedures lor detecting differential. Applied Psychological Measurement, J8 (4). 3 15-328.
Narayanan, P., & Swaminathan, H. (1996). Identification of items that non-uniform DIF. Applied PsychologicalMeasurement. 2(7(3), 257-274.
Padilla, J. L., Hidalgo, M. D., Benitez, I. & Gomez-Benito, J. (2011). Comparison of three software programs for evaluating DIF by means of the Mantel-FIaenszel procedure: EASY-DIF, DIFAS and EZDIF. International journal of methodology and experimental psychology, preprints. http:www.es/psicok)gica/preprints/Padilla.pdf adresinden 16.11.2011 tarihinde alınmıştır. 1
Penlieid. R. D. (2003a). Application of the Breslow-Day test of trend in odds heterogenity to the detection of nonuniform nftr ALberta JOurnal of Educational Researcli.49.231-243
Pitoniak, M., Cook. L,. Cline, F„ & Cahalan-Laitusis. C. (2006). Using differential iıeın functioning to investigate the impi accomodations on the scores of students with disabilities on english-language arts assessments. Educational Testink Service, NCME Presentation.
Polenza. T. M.. & Dorans. J. N. (1995). DIE assessment for polytomously scored items: A framework for classification evaluation. Applied Psychological Measurement, 19{ I). 23-37.
Roussos, L.L.. & Stout, W. F. (1996). Simulation studies of the effects of small sample size and studied item parameters SIBTEST and Mantel-Haenszel type 1 error performance. Journal of Educational Measurement, 33(2), 215-230.
Rogers. H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures lor detectinc; differential item functioning. Applied Psychological Measurement, 11, 105- 116.
Shepard. L. A., CamiHi. G.. & Williams. D. M. (1984). Accounting for statistical artifacts in item bias research. Journal Educational Statistics, 9,93-128.
Shealy, R. T., & Stout, W. F. (1993). An item response theory model for test bias and differentia! test functioning, In P. W Holland & H. Wainer (Eds.), Differential item functioning (pp.!97-239). Hillsdale, NJ: Erlbaum.
Stout, W. (2005). SIBTEST Software (1.7 sürümü) [Bilgisayar Yazılımı]. Institute for Measurement: Assessment System Corporations.
Swaminathan, H., & Rogers. H. J. (1990). Detecting differentia! item functioning using logistic regression procedures. Journal of Educational Measurement, 2 7(4), 361 - 3 70.
Thissen, D. (2001). IRTLRDIF (2.0b Sürümü) [Computer software]. Chapel Mill: L. L. Thurstone Psychometric Laboratory. University of North Carolina.
Thissen, D., Steinberg, L., &. Gerrard, M. (1986). Beyond group-mean differences: The concept of item bias. Psvcholo^cal Bulletin, 99, 118-128.
Thissen. D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace linos. In H.Wainer&H. Braun (Eds.), Test validity (pp. 147-169). Hillsdale, NJ: Lawrence Erlbaum.
Thissen, D., Steinberg, L., & Wainer, H. (İ993). Detection of differential item functioning using parameters of itemresponsemodels. In PAY. Holland & H.Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale.
Waller, N. G. (1998). EZD1F: Detection of uniform and nonuniform differential item functioning with the Mantel-Haenszel and logistic regression procedures. Applied Psychological measurement, 22, 391.
Wiberg, M. (2009). Differential item limctioning in mastery tests: A comparison of three methods using real data. International Journal o f Testing, 9, 41-59.
Yıldtrım. S. (2008). Farklı işleyen maddelerin belirlenmesinde sınırlandırılmış faktör çözümlenmesinin olabilirlik-oraın ve maıUel-haenszel yöntemleriyle karşılaştırılması. H.Ü. Eğitim Fakültesi Dergisi, 34. 297-307.
Yurdugül, H. (2003). Orta öğretim kurumları öğretici seçme ve yerleştirme sınavının madde yanlılığı açısından incelenmesi, Doklara tezi, Hacettepe Üniversitesi, Ankara.
Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. Hoîland& H. Wainer (Eds.), Differential item functioning, (pp. 337-347). Hillsdale, NJ: Erlbaum
Zumbo. B. D. (1999).handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and likert-type (Ordinal) item scores. Ottawa, ON": Directorate of Human Resources Research and Evaluation, Department of National Defense.