Hülya KELECİOĞLU, Dilara Bakan KALAYCIOĞLU

Öğrenci Seçme Sınavı'nın madde yanlılığı açısından incelenmesi

Bu çalışmada, 2005 yılı Öğrenci Seçme Sınavı'nda (ÖSS) yer alan maddelerin cinsiyete göre Değişen Madde Fonksiyonu (DMF) içerip içermediği incelenmiştir. DMF içerdiği belirlenen maddelerin yanlı olup olmadığına karar vermek için uzman görüşüne başvurulmuştur. DMF belirleme yöntemlerinden Mantel-Haenszel (MH) ve Lojistik Regresyon (LR) kullanılmıştır. Analizler 2005 yılında ÖSS'ye giren 599,330 lise son sınıf öğrencisinin her biri 45 maddeden oluşan Türkçe, sosyal bilimler, matematik ve fen bilimleri alt testlerine verdikleri cevaplar kullanılarak yapılmıştır. Araştırmanın sonucunda, cinsiyete göre Türkçe alt testini oluşturan maddelerin DMF içermediği sonucuna varılmıştır. Sosyal bilimler alt testinde yedi, matematik ve fen bilimleri alt testlerinde de üç maddenin DMF içerdiği ve bu maddelerden fen bilimleri testinde yer alan bir tanesinin yanlılık gösterdiği belirlenmiştir.

Item bias analysis of the University Entrance Examination

This study investigated the gender related Differential Item Functioning (DIF) of the 2005 University Entrance Examination (UEE) in order to decide whether a DIF item is biased. In this study, measurement specialists' opinions were gathered and Mantel-Haenszel (MH) and Logistic Regression (LR) DIF detection methods were used. The analyses were based on the answers of 599.330 high school seniors to the Turkish, social sciences, mathematics and natural science subtests of the 2005 UEE. It was found that no items were flagged for gender DIF in the Turkish subtest. However, seven social sciences items and three mathematics and natural sciences items displayed DIF. Among these items, only one natural sciences item was identified as biased.

PDF

___

Azen, R., Bronner, S., & Gafni, N. (2002). Examination of Gender Bias in University Admissions.Applied Measurement in Education, 15, 75-94.
Balluerka, N., Gorostiaga, A., Gomez-Benito, J. & Hidalgo, M. D. (2010). Use of Multilevel Logistic Regression to Identify the Causes of Differential Item Functioning. Psicothema, 22,4,1018-1025.
Benito J.G. & Ara M.J.N. (2000). A Comparison of X2, RFA and IRT Based Procedures in the Detection of DIF, Quality & Quantity, 34,17-31.
Berberoğlu, G. (1996). The University Entrance Examinations in Turkey. Studies in Educational Evaluation, 22, 4, 363-373.
Camilli, G. & Shepard, L.A. (1994). Methods for Identifying Biased Test Items. Sage Publications, California.
Coley, R. J. (2001). Differences in the Gender Gap: Comparison Across Racial/Ethnic Groups in Education and Work. Policy Information Report, Educational Testing Service.
Doolittle, A. E. & Cleary, T. A (1987). Gender-Based Differential Item Performance in Mathematics Achievement Items. Journal ofEducational Measurement, 24, 2, 157-166.
Educational Testing Service, (2009). ETSGuidelinesfor Fairness Review ofAssessments. Princeton, NJ: ETS.
Gök, B., Kalecioğlu, H. Doğan, N. (2010). Değişen Madde Fonksiyonu Belirlemede Mantel - Haenzel ve Lojistik Regresyon Tekniklerinin Karşılaştırılması. Eğitim ve Bilim, 35 (156), 3-16.
Haladyna, T. M. (1997). Writing Test Items to Evaluate Higher Order Thinking. Allyn & Bacon.
Hambleton, R.K., Swaminathan, H. & Rogers, H.J. (1991). Fundamentals of Item Response Theory. Sage Publications, California.
Harris, A. M. & Carlton, S. T. (1993). Patterns of Gender Differences on Mathematics Items on the Scholastic Aptitude Test. AppliedMeasurement in Education, 6, 2,137-151.
Hidalgo, M. D. & Lopez-Pina, J. A. (2004). Differential item functioning detection and effect size: a comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64, 6, 903-915.
Jodoin M. G. & Gierl, M. J. (2001).Evaluating Type I Error and Power Using an Effect Size Measure with the Logistic Regression Procedure for DIF Detection. Applied Measurement in Education, 14,4, 329-349.
Kristanjansonn E., R. Aylesworth, I. McDowell & B.D. Zumbo (2005). A Comparison of Four Methods for Detecting Differeantial Item Functioning In Ordered Response Model. Educational and Psychological Measurement. 65,6. 935-953.
Le, V. (1999). Identifying Differential Item Functioning on the NELS: 88 History Achievement Test. Center for the Study of Evaluation Technical Report. (ED443879)
Mendes-Barnett, S. & Ercikan, K. (2006). Examining sources of gender DIF in mathematics assessments using a confirmatory multidimensional model approach. Applied Measurement in Education, 19,4, 289-304.
Oshima T. C. & Morris, S. B. (2008). Raju's Differential Functioning of Items and Tests (DFIT). Educational Measurement: Issues and Practice, 27, 3, 43-50.
Öğretmen, T. (1995). Differential Item Functioning Analysis of the Verbal Ability Section of the First Stage of the University Entrance Examination in Turkey. Yayımlanmamış Yüksek Lisans Tezi. Orta Doğu Teknik Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
Pitoniak, M., Cook, L., Cline, F. & Cahalan-Laitusis, C. (2006). Using Differential Item Functioning to Investigate the Impact of Accomodations on the Scores of Students with Disabilities on English-Language Arts Assessments. Educational Testing Service, NCME Presentation.
Swaminathan, H. & Rogers, H. J. (1990). Detecting Differential Item Functioning Using Logistic Regression Procedures. Journal ofEducational Measurement, 27, 4, 361-370.
Yenal, E. (1995). Differential Item Functioning Analysis of the Quantitative Ability Section of the First Stage of the University Entrance Examination in Turkey. Yayımlanmamış Yüksek Lisans Tezi. Orta Doğu Teknik Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
Yıldırım, H. H., Berberoglu, G. (2009). Judgmental and Statistical DIF Analyses of the PISA-2003 Mathematics Literacy Items. International Journal ofTesting, 9,108-121.
Yurdugül, H. & Aşkar, P. (2004). Ortaöğretim Kurumlan Öğrenci Seçme ve Yerleştirme Sınavı'nın Cinsiyete Göre Madde Yanlılığı Açısından İncelenmesi. Eğitim Bilimleri ve Uygulama, 3,5,3-20.
Waller, N. G. (2005) EZDIF: A computer Program for Detecting Uniform and Non uniform Differential Item Functioning with the Mantel-Haenszel and Logistic Regression Procedures.
Zenisky, A.L., Hambleton, R.K. ve Robin, F. (2003). DIF Detection and Interpretation in Large-Scale Science Asssessments: Informing Item Writing Practices. Educational Assessment, 9 (1&2), 61-78.
Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. Holland & H. Wainer (Eds.), Differential item functioning pp. 337-347. Hillsdale, NJ: Erlbaum.
Zumbo, B. D., & Thomas, D. R. (1996). A measure of DIF effect size using logistic regression procedures. Paper presented at the National Board of Medical Examiners, Philadelphia.
Zumbo, B. D. (1999). A Handbook on the Theory and Methods of Differential Item Functioning (DIF): Logistic Regression Modeling as a Unitary Framework for Binary and Likert-Type (Ordinal) Item Scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.
Zumbo, B. D., & Gelin, M.N. (2005). A Matter of Test Bias in Educational Policy Research: Bringing the Context into Picture by Investigating Sociological Community Moderated (or Mediated) Test and Item Bias. Journal ofEducational Research and Policy Studies, 5,1-23.