Eren Can AYBEK, Metin YAŞAR, Seval KARTAL

Öğretmen Yapımı Bir Testteki Maddelerin Değişen Madde Fonksiyonu Bağlamında İncelenmesi

Bu araştırmanın amacı öğretmen yapımı bir testte yer alan maddelerin klasik test kuramı ve madde tepki kuramı temelli değişen madde fonksiyonu (DMF) gösterip göstermeme durumunu farklı DMF belirleme yöntemlerine göre incelemektir. Bu amaç doğrultusunda 435 üniversite öğrencisinin ölçme ve değerlendirme dersi dönem sonu sınavı puanları kullanılmıştır. Dönem sonu sınavı 50 çoktan seçmeli maddeden oluşmaktadır. Veriler uzaktan eğitim sistemi üzerinden çevrimiçi olarak toplanmıştır. Veri analizi R üzerinde ShinyItemAnalysis paketi kullanılarak gerçekleştirilmiştir. Araştırma sonucunda, bir maddenin cinsiyete göre dört farklı yönteme göre DMF gösterdiği bulunmuştur. Üniversiteye göre yapılan inceleme sonucunda ise delta plot yöntemi hiçbir maddede DMF bulunamazken, Mantel-Haenszel, Lojistik Regresyon ve Lord χ2 yöntemlerine göre beş maddede DMF’ye rastlanmıştır. Hem cinsiyete hem de üniversiteye göre DMF bulunan maddeler incelendiğinde DMF’ye yol açabilecek ifadelere rastlanmamıştır.

Anahtar Kelimeler:

öğretmen yapımı test, yanlılık, değişen madde fonksiyonu

Examination of Items in a Teacher-Made Test in the context of Differential Item Functioning

The present study aims to examine whether the items in a teacher-made test show a classical test theory and item response theory-based differential item functioning (DIF) with different DIF determination methods. For this purpose, final exam scores of 435 students regarding the course of measurement and evaluation were used. The final exam consisted of 50 multiple-choice items. Data were collected online through the learning management system. Data analysis was performed using the ShinyItemAnalysis package on R. As a result of the study, it was found that one item showed DIF according to four different methods by gender. As a result of the analysis made by the university, on the other hand, none of the items showed DIF according to the delta plot method, while five items showed DIF according to Mantel-Haenszel, Logistic Regression, and Lord χ2 techniques. When the statements in the items showing DIF by both gender and university are examined, on the other hand, it was observed that there was no statement that can create DIF.

Keywords:

teacher-made test, bias, differential item functioning,

PDF

___

Angoff, W.H. (1982). Use of difficulty and discrimination indices for detecting item bias. In R.A. Berk (Ed.), Handbook of methods for detecting test bias (pp.96-116). Baltimore: Johns Hopkins University.
Ayva Yörü, F.G. ve Atar, H.Y. (2019). Determination of differential item functioning (DIF) according to SIBTEST, Lord’s χ2, Raju’s area measurement and Breslow-Day methods. Journal of Pedagogical Research, 3(3), 139-150. doi: 0.33902/jpr.v3i3.137
Crocker, L. ve Algina, J. (2008). Introduction to classical and modern test theory. Ohio: Cengage Learning.
Çıkrıkçı Demirtaşlı, N. ve Ulutaş, S. (2015). A study on detecting of differential item functioning of PISA 2006 science literacy items in Turkish and American samples. Eurasian Journal of Educational Research, 58. 41-60.
de Ayala, R.J. (2009). The theory and practice of item response theory. New York: The Guilford Press.
Doğan, N. ve Öğretmen, T. (2008). Değişen madde fonksiyonu belirlemede Mantel-Haenszel, ki-kare ve lojistik regresyon tekniklerinin karşılaştırılması. Eğitim ve Bilim, 33(148). 100-112.
Feldt, L.S., Woodruff, D.J. ve Salih, F.A. (1987). Statistical inference for coefficient alpha. Applied psychological measurement, 11(1), 93-103.
Gamerman, D., Goncalves, F. B. ve Soares, T. M. (2018). Differential item functioning. In W. J. van der Linden (Ed.), Handbook of item response theory (Vol. 3, pp. 67–86). Boca Raton, FL: CRC Press.
Gierl, M., Khaliq, S. N., & Boughton, K. (1999, June). Gender differential item functioning in mathematics and science: Prevalence and policy implications. In Annual Meeting of the Canadian Society for the Study of Education, Canada.
Gök, B., Kelecioğlu, H. ve Doğan, N. (2010). Değişen madde fonksiyonunu belirlemede Mantel-Haenszel ve lojistik regresyon tekniklerinin karşılaştırılması. Eğitim ve Bilim, 35(156). 3-16.
Hambleton, R.K. ve Swaminathan, H. (1985). Item response theory: Principles and applications. New York: Springer Science+Bussiness Media.
Karami, H. (2012). An introduction to differential item functioning. The International Journal of Educational and Psychological Assessment, 11(2), 59–76.
Koyuncu, İ., Aksu, G. ve Kelecioğlu, H. (2018). Comparison of Mantel-Haenszel, logistic regression and likelihood ratio methods to evaluate differential item functioning by using different computer software. Elementary Education Online, 17(2). 909-925. doi: 10.17051/ilkonline.2018.41933
Lord, M.D. (1980). Applications of item response theory to practical testing problems. New Jersey: Lawrance Erlbaum Associates.
Martinková, P. ve Drabinová, A. (2018). ShinyItemAnalysis for teaching psychometrics and to enforce routine analysis of educational tests. The R Journal, 10(2), 503-515. doi: 10.32614/RJ-2018-074
McNamara, T. ve Roever, C. (2006). Language testing: The social dimension. Massachusetts: Blackwell Publishing.
Millsap, R.E. (2011). Statistical approaches to measurement invariance. New York: Routledge.
Narayanan, P. ve Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18(4). 315-328.
Osterlind, S.J. (1983). Test item bias. London: Sage Pub.
Popham, W.J. (2014). Classroom assessment: What teachers need to know. Boston: Pearson.
R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
Reise, S.P. ve Revicki, D.A. (2015). Handbook of item response theory modeling. New York: Routledge.
Rogers, H. J. ve Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17(2), 105-116.
Şahin, A. ve Anıl, D. (2017). The effects of test length and sample size on item parameters in item response theory. Educational Sciences: Theory & Practice, 17(1). 321-335. doi: 10.12738/estp.2017.1.0270
Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense