Kübra KABASAKAL ATALAY, Hülya KELECİOĞLU

Evaluation of attitude items in PISA 2006 student questionnaire in terms of differential item functioning

Bu çalışma kapsamında PISA 2006 öğrenci anketinde yer alan fen bilimlerine ilişkin tutum maddeleri, cinsiyete göre ve kültüre göre değişen madde fonksiyonu gösterip göstermediğini incelenmektedir. Değişen madde fonksiyonu (DMF)’nun cinsiyetler arası farklılıklardan kaynaklanabileceği gibi, kültürel farklılıklardan ve testlerin başka bir dile çevrilmesinde karşılaşılan sorunlardan da kaynaklanabilmektedir. Bu nedenle araştırmanın çalışma grubu PISA uygulamasına katılan ABD’den 5611 öğrenci ve Türkiye’den 4942 öğrenci olarak belirlenmiştir. Cinsiyete göre yapılan analizlerde her iki yöntemde ortak ve önemli düzeyde DMF gösteren madde sayısı Türkiye’de 7, ABD’ de 13’tür. Đki ülke arasında kültüre göre her iki yöntemde yapılan analizlerde 30 maddede önemli düzeyde DMF görülmüştür. Analizler sonucu iki yöntem karşılaştırıldığında, yöntemlerin DMF belirlemede bir uyum gösterdiği; ancak düzeylerinde farklılıklar olabileceği sonucuna varılmıştır.

PISA 2006 öğrenci anketinde yer alan maddelerin değişen madde fonksiyonu açısından incelenmesi

Under the scope of this study, attitude items towards sciences were examined in PISA 2006 Student Survey is used to determine whether differential item functioning was observed in respect to gender and culture with ordinal logistic regression and poly- SIBTEST. Cultural differences and translational restrictions stand among the possible reasons of the observation of differential item functioning (DIF). Due to this reason, the research group of this study was determined as 5611 students from USA and 4942 students from Turkey whom participated in PISA aplication. In both methods, the number of items that showed DIF significantly and common is 7 in Turkey and 13 in the USA. In 30 items, with respect to culture, DIF was observed at a significant level through the analyses carried out in both countries utilizing both methods. When the above-mentioned two methods employed in the analyses, in terms of culture and gender, the data from USA and Turkey were compared, these two methods were found to be in compliance with each other, also with possible variations of their levels.

PDF

___

Allaouf, A., Hambleton, R. & Sireci, S. (1999). Identifying the cause of DIF in translated verbal items. Journal of Educational Measurement, 36 (3),185–198.
Anıl, D. (2009). Uluslararası Öğrenci Başarılarını Değerlendirme Programı (PISA)’nda Türkiye’deki öğrencilerin fen bilimleri başarılarını etkileyen faktörler. Eğitim ve Bilim, 35 (152), 87-100.
Atar, B. (2010). Basit doğrusal regresyon analizi ile hiyerarşik doğrusal modeller analizinin karşılaştırılması. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 1(2), 78-84.
Asil, M. (2010). Uluslararası Öğrenci Değerlendirme Programı (PISA) 2006 Öğrenci Anketinin Kültürler Arası Eşdeğerliğinin Đncelenmesi. Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, yayınlanmamış doktora tezi.
Barış, F. (2010). TIMMS-R ve TIMSS-2007 sınavlarının öğrenci başarısını yordayan değişkenler açısından incelenmesi. Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, yayınlanmamış yüksek lisans tezi.
Bloom, B. S. (1989). Đnsan Nitelikleri ve Okulda Öğrenme (Çev. D. Ali Özçelik), Ankara: Milli Eğitim Basımevi.
Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational Measurement (4th ed., Vol. 4, pp. 221-256). Westport: American Council on Education & Praeger Publishers.
Camili, G., & Shepard, L. A. (1994). Methods for identifying biased test items. London: Sage publications.
Chang, H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33(3), 333-353.
Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differential item functioningtest items. Educational Measurement: Issues and Practice, 17, 31-44.
Dancer, L. S. , Anderson, A. J. , Derlin, R. L. (1994). Use of log-lineer models for assesing differential item functioning in a measure of psychological functioning. Journal of consulting and clinical psychology,62(4),710-717.
Dorans, N. J., & Schmitt, A. P. (1993). Constructed response and differential item functioning: A pragmatic approach. (ETS-RR-91-47). Princeton, NJ: Educational Testing Service.
Jodion, M. G. & Gierl, M. J. (2001). Evaluating type I error and power rates using an effect size measure ith the logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329-349.
Johanson, G. A., & Dodeen, H. (2003). An Analysis of Sex-related Differential Item Functioning in Attitude Assessment. Assesment &Evaluation in Higher Education, 28(2).
Fang, T. (1999). Detecting DIF in polytomous item responses. Unpublished doctor dissertation, University of Ottawa, Canada.
Fang, T. (1999). Detecting DIF in polytomous item responses. Unpublished doctoral dissertation, University of Ottawa, Canada.
Feng, Y. (2008). Difference in gender differential item functioning patterns across item format and subject area on diploma examinations after change in administration procedure. Unpublished doctoral dissertation, University of Alberta, Edmonton, Alberta, Canada.
French, A. W., & Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33(3), 315-332.
Gelin, M. N., & Zumbo, B. D. (2003). DIF results may change depending on how an item is scored: An illustration with the Center for Epidemiological Studies Depression (CES-D) scale. Educational and Psychological Measurement, 63, 65-74.
Henderson, D. L. (1999). Investigation of differential item functioning in exit Examinations across item format and subject area. Unpublished doctor dissertation, University of Alberta, Edmonton, Alberta, Canada.
Kristjansson, E., Aylesworth, R., McDowell, I, & Zumbo, B. D. (2005). A comparison of four methods for detecting DIF in ordered response items. Educational and Psychological Measurement, 65, 935-953.
Le, L.T. (2009). Investigation gender differential item functioning across countries abd test languages for PISA science items. International Journal of Testing, 9(2), 122-133.
Mellor, T. L. (1995). A comparison of four differantial item functioning methods for polytomously scored items. Unpublished doctoral dissertation. The university of Texas Austin.
Miller, T. R., & Spray, J. A. (1993). Logistic discriminant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement, 30(2), 107-122.
Özdemir, F. (2009). PISA 2003’de genel lise öğrencileri ve Kanuni Lisesi öğrencilerinin matematik başarısını etkileyen faktörlerin incelenmesi. Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, yayınlanmamış yüksek lisans tezi.
Smith, F. W. (2009). Language-related differential item functioning in the WASL mathematics test. Unpublished doctor dissertation. The university of Washington.
Sireci, G. S. & Khaliq, K. N. (2002). An analysis of the psychometric properties of dual language test forms. Annual Meeting of the National Council on Measurement in Education, New Orleans, LA, April 2-4.
Shealy, R. T., & Stout, W. F. (1993). An item response theory model for test bias and differential test functioning. Đn P. W. Holland & H. Wainer (Eds.), Differential itemfunctioning (pp.197-239). Hillsdale, NJ: Erlbaum.
Swaminathan, H. & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361- 370.
Yıldırım, H.H. ve Berberoğlu, G. (2006). Judgementel and statistical analyses of the PISA 2003 mathematics literacy items. International Journal of Testing, 9(2),108-121.
Welch, C., & Hoover, H. D. (1993). Procedures for extending item bias techniques to polytomously scored items. Applied Measurement in Education, 6, 1-19.
Zumbo, B. D. (1999). A Handbook on the Theory and Methods of Differential Đtem Functioning (DIF): Logistic Regression Modeling as a Unitary Framework for Binary and Likert-Type (Ordinal) Đtem Scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.
Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, Where it is now, and Where It Is Going. Language Assessment Quarterly, 4(2), 223–233.
Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessing differential item functioning for performance tests. Journal of Educational Measurement, 30(3), 233-251.
Zwick, R., Thayer, D. T., & Mazzeo, J. (1997).Descriptive and inferential procedures for assesing Differential Item Functioning in Polytomous Items. Applied Measurement in Education,10.