Değişen Madde Fonksiyonunun Belirlenmesinde “difR” R Paketinin Kullanımı: Ortaöğretime Geçiş Sınavı Fen Alt Testi

Değişen Madde Fonksiyonunun (DMF) belirlenmesi, bir testin ve testten alınan puanlarıngeçerliğine ilişkin önemli göstergeler sunmaktadır. difR paketi ise farklı DMF belirlemeyöntemlerinin uygulanmasına izin vererek araştırmacılara ve uygulayıcılara büyük kolaylıksağlayan R paketidir. Bu araştırmanın temel amacı örnek bir araştırma verisi üzerinden, difRpaketinde farklı DMF belirleme yöntemlerine ilişkin; yazılım kurulumu, varsayımlarınincelenmesi, analiz adımları ve analiz sonuçlarının yorumlanması için izlenen sürecinresmedilmesidir. Bu temel amaç doğrultusunda Türkiye'de 8. sınıf öğrencilerine uygulananOrtaöğretime Geçiş Sınavı 2018 uygulamasında yer alan fen maddelerinin, madde sıra etkisibakımından DMF gösterme durumları incelenmiştir. Bu yönüyle araştırma tarama modelindebir araştırmadır. Araştırmada sıklıkla kullanılan DMF belirleme yöntemlerinden Klasik TestKuramına dayalı Mantel-Haenszel, Lojistik Regresyon ve SIBTEST ile Madde Tepki Kuramınadayalı Olabilirlik Oran yöntemlerine ilişkin adımlar ele alınmıştır. DMF analizleri sonucu eldeedilen bulgulara göre fen maddelerinin madde sıra etkisi bakımından çoğunlukla DMFgöstermediği ya da ihmal edilebilir düzeyde DMF gösterdiği sonucuna ulaşılmıştır.

Using R to Detect Differential Item Functioning: Science sub-test of Secondary School Entrance Examination

Differential Item Functioning (DIF) analyses provide critical information about validity of atest. R; an open source software, that comprises all of the DIF detection methods, has animportant role in DIF research. Therefore, conducting a guiding study for measurementinvariance or DIF analyses by following scientific methods and procedures will be very usefulfor researchers and practitioners. In this research, it is aimed to illustrate the proceduresfollowed in different DIF detection methods in R, beginning from the installation of the Rsoftware to the interpretation of the analysis results, using a sample test (science sub-test ofSecondary School Entrance Examination) and data. Four DIF detection methods, which arecommonly used in DIF analyses, Mantel-Haenszel, Logistic Regression, SIBTEST andLikelihood Ratio methods are handled in this study. According to the analysis results, no itemsindicate DIF or indicate negligible DIF

___

  • Adedoyin, O. (2010). An investigation of the effects of teachers’ classroom questions on the achievements of students in mathematics: Case study of Botswana community junior secondary schools. European Journal of Educational Studies, 2(3), 313-329. Retrieved from http://connection.ebscohost.com/c/articles/67192996/investigation-effectsteachers-classroom-questions-achievements-students-mathematics-case-studybotswana-community-junior-secondary-schools
  • Akalın, Ş. (2014). Kamu personeli seçme sınavı genel yetenek testinin madde yanlılığı açısından incelenmesi (Yayınlanmamış doktora tezi). https://tez.yok.gov.tr/UlusalTezMerkezi/’nden erişilmiştir. (Tez No. 381796).
  • Allaire, J. J. (2011, August). RStudio: Integrated development environment for R. Paper presented at the R User Conference, useR! University of Warwick,
  • Coventry, UK. Retrieved from https://www.r-project.org/conferences/useR2011/abstract_booklet.pdf Asil, M. ve Gelbal, S. (2012). PISA öğrenci anketinin kültürler arası eşdeğerliği. Eğitim ve Bilim, 37(166). 236-249 http://egitimvebilim.ted.org.tr/index.php/EB/article/view/1501/462 adresinden erişilmiştir.
  • American Educational Research Association, American Psychological Association, National Council on Measurement in Education [AERA/APA/NCME]. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
  • Avcu, A., Tunç, E. B., and Uluman, M. (2018). How the order of the items in a booklet affects item functioning: Empirical findings from course level data. European Journal of Education Studies,4(3), 227-239. doi:10.5281/zenodo.1199695
  • Bakan-Kalaycıoğlu, D., and Kelecioğlu, H. (2011). Item bias analysis of the university entrance examination. Education and Science, 36(161), 3-13. Retrieved from http://egitimvebilim.ted.org.tr/index.php/EB/article/view/143
  • Balta, E., and Ömür-Sünbül, S. (2017). An investigation of ordering test items differently depending on their difficulty level by differential item functioning. Eurasian Journal of Educational Research, 72(2017), 23-42. doi: 10.14689/ejer.2017.72.2.
  • Barcikovski, R. S., and Olsen, H. (1975). Test item arrangement and adaptation level. The Journal of Psychology, 90(1), 87-93. doi: 10.1080/00223980.1975.9923929.
  • Beaujean. A. A. (2013). Factor analysis using R. Practical Assessment: Research & Evaluation, 18(4), 1-11. https://doi.org/10.7275/z8wr-4j42.
  • Bolt, D. (2000). A SIBTEST approach to testing DIF hypothesis using experimentally designed test items. Journal of Educational Measurement, 37(4), 307-327. Retrieved from https://www.jstor.org/stable/1435243?seq=1
  • Bolt, S. E., and Ysseldyke, J. (2008). Accommodating students with disabilities in large-scale testing: A comparison of differential item functioning (DIF) identified across disability types. Journal of Psychoeducational Assessment, 26(2), 121-138. doi:10.1177/0734282907307703
  • Büyüköztürk, Ş. (2004). Veri analizi el kitabı. Ankara: PegemA Yayıncılık. Camilli, G., and Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
  • Cattell, R. B. (1966). The data box: Its ordering of total resources in terms of possible relational systems. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (pp. 67–128). Chicago, IL: Rand-McNally
  • Cheung, G. W., and Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-cultural Psychology, 31(2), 188-213. Retrieved from https://doi.org/10.1177/0022022100031002003
  • Chiu, P. (2012, April). The effect of item position on state mathematics assessment. Paper presented at the Annual Meeting of the American Educational Research Association, Kansas. Retrieved from https://aai.ku.edu/sites/cete.ku.edu/files/docs/Presentations/2012_04_Chiu_Mat h_Item_Or dering_AERA_2012.pdf
  • Chiu, P. C., and Irwin, P. M. (2011, April). Chronological item ordering: Does it make a difference on a state history and government assessment? Paper presented at the Annual Meeting & Training Sessions of the National Council on Measurement in Education, New Orleans, Louisiana, USA. Retrieved from https://docplayer.net/131592073-Chronological-item-ordering-does-it-make-adifference-on-a-state-history-and-government-assessment-pui-chi-chiu-andpatrick-m.html
  • Crocker, L. M., and Algina, J. (1986). Introduction to classical and modern test theory. Orlando, FL: Holt, Rinehart and Winston.
  • Çepni, Z. (2011). Değişen madde fonksiyonlarının SIBTEST, Mantel-Haenzsel, lojistik regresyon ve madde tepki kuramı yöntemleriyle incelenmesi (Yayınlanmamış doktora tezi). https://tez.yok.gov.tr/UlusalTezMerkezi/’nden erişilmiştir (Tez No. 308512).
  • Çokluk, Ö., Gül, E., and Doğan-Gül, Ç. (2016). Examining differential item functions of different item ordered test forms according to item difficulty levels. Educational Sciences: Theory & Practice, 16, 319-330. http://dx.doi.org/10.12738/estp.2016.1.0329
  • DeMars, C. E. (2010). Type I error inflation for detecting DIF in the presence of impact. Educational and Psychological Measurement, 70(6), 961-972. doi: 10.1177/0013164410366691
  • Doğan, C. D. ve Uluman, M. (2016). İstatistiksel veri analizinde R yazılımı ve kullanımı, İlköğretim Online, 15(2), 615-634, 2016. doi: 10.17051/io.2016.24991.
  • Doğan Gül, Ç. ve Çokluk Bökeoğlu, Ö. (2018). Madde güçlüklerine göre farklı sıralanan testlerde düşük ve yüksek kaygılı öğrencilerin akademik başarılarının karşılaştırılması. İnönü Üniversitesi Eğitim Fakültesi Dergisi, 19 (3) , 252-265. doi: 10.17679/inuefd.341477.
  • Dorans, N. J., and Holland, P. W. (1993). DIF detection and description: MantelHaenszel and standardization. In P. W. Holland and H. Wainer (Eds.), Differential item functioning (pp. 35–66). New Jersey: Lawrence Erlbaum Associates Publishers.
  • Er, F. ve Sönmez, H. (2005). Temel istatistik eğitiminde R for Windows paket programı kullanımı. Eskişehir Osmangazi Üniversitesi Mühendislik ve Mimarlık Fakültesi Dergisi, 18(2), 71-86. https://dergipark.org.tr/tr/download/articlefile/321059 adresinden erişilmiştir.
  • Fraenkel, J. R., and Wallen, N. E. (2006). How to design and evaluate research in education. New York, NY: McGraw Hill Higher Education.
  • Gierl, M. J., Jodoin, M. G., and Ackerman, T. A. (2000, April). Performance of Mantel-Haenszel, Simultaneous Item Bias Test and Logistic Regression when the proportion of DIF items is large. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.
  • Gierl, M. J. (2000). Construct equivalence on translated achievement tests. Canadian Journal of Education, 25(4), 280-296. doi:10.2307/1585851
  • Gotzmann, A., Wright, K., and Rodden, L. (2006, April). A comparison of power rates for items favoring the reference and focal group for the Mantel-Haenszel and SIBTEST procedures. Paper presented at the American Educational Research Association (AERA) in San Francisco, California.
  • Gök, B., Kelecioğlu, H. ve Doğan, N. (2010). Değişen madde fonksiyonunu belirlemede Mantel–Haenszel ve Lojistik Regresyon tekniklerinin karşılaştırılması. Eğitim ve Bilim, 35(156), 3-16. http://egitimvebilim.ted.org.tr/index.php/EB/article/view/19/28 adresinden erişilmiştir.
  • Greer, T. G. (2004). Detection of differential item functioning (dif) on the satv: A comparison of four methods: Mantel-Haenszel, Logistic Regression, simultaneous item bias and likelihood ratio test (Unpublished doctoral dissertation). Retrieved from www.proquest.com (Tez No. 3122349).
  • Grover, R. K., and Ercikan, K. (2017). For which boys and which girls are reading assessment items biased against? Detection of differential item functioning in heterogeneous gender populations. Applied Measurement in Education, 30(3), 178–195. doi: 10.1080/08957347.2017.1316276.
  • Hahne J. (2008). Analyzing position effects within reasoning items using the LLTM for structurally incomplete data. Psychology Science Quarterly, 50(3), 379-390. Retrieved from http://journaldatabase.info/articles/analyzing_position_effects_within.html
  • Hambleton, R. K., and Traub, R. E. (1974). The effects of item order on test performance and stress. Journal of Experimental Education, 43(1), 40- 46. doi:10.1080/00220973.1974.10806302
  • Hambleton, R. K., and Swaminathan, H. (1989). Item response theory: Principles and applications. USA: Kluwer Nijhoff Publishing.
  • Hambleton, R.K., Swaminathan, H. and Rogers, H. J. (1991). Fundamentals of Item Response Theory. Newbury Park, CA: Sage.
  • Hambleton, R. K. (2006). Good practices for identifying differential item functioning. Medical Care, 44(11), 182- 188. doi:10.1097/01.mlr.0000245443.86671.c4
  • Herrera, A.-N., and Gómez, J. (2008). Influence of equal or unequal comparison group sample sizes on the detection of differential item functioning using the MantelHaenszel and Logistic Regression techniques. Quality & Quantity: International Journal of Methodology, 42(6), 739–755. doi: 10.1007/s11135-006-9065-z
  • Horgan, J. M. (2012). Programming in R. WIREs Comp Stat, 4(1), 75-84. doi: 10.1002/wics.183 International Test Commission (2005). International test commission guidelines for test adaptation. London: Author. Retrieved from https://www.intestcom.org/files/guideline_test_adaptation.pdf
  • Jodoin, M. G., and Gierl, M.J. (2001). Evaluating type I error and power rates using an effect size measure with logistic regression procedure or DIF detection. Applied Measurement in Education, 14(4), 329-349. doi:10.1207/S15324818AME1404_2
  • Johnstone, C. J., Thompson, S. J., Moen, R. E., Bolt, S., and Kato, K. (2005). Analyzing results of large-scale assessments to ensure universal design (Technical Report 41). Retrieved from http://education.umn.edu/NCEO/OnlinePubs/Technical41.htm
  • Karakoç-Alatlı, B. ve Çokluk-Bökeoğlu, Ö. (2018). Investigation of measurement invariance of literacy tests in the Programme for International Student Assessment (PISA-2012). Elementary Education Online, 17(2), 1096-1115. doi: 10.17051/ilkonline.2018.419357
  • Kingston, N. M., and Dorans, N. J. (1984). Item location effects and their implications for IRT equating and adaptive testing. Applied Psychological Measurement, 8(2), 147–154. doi: 10.1177/014662168400800202
  • Kleinke, D. J. (1980). Item order, response location, and examinee sex and handedness on performance on multiple-choice tests. Journal of Educational Research, 73(4), 225–229. doi:10.1080/00220671.1980.10885240
  • Koyuncu, İ., Aksu, G. ve Kelecioğlu, H. (2018). Mantel-Haenszel, Lojistik Regresyon ve Olabilirlik Oranı değişen madde fonksiyonu inceleme yöntemlerinin farklı yazılımlar kullanılarak karşılaştırılması. İlköğretim Online,17(2), 902-925. doi 10.17051/ilkonline.2018.419339
  • Luo, L., Arizmendi, C., and Gates K. M. (2019). Exploratory Factor Analysis (EFA) programs in R. Structural Equation Modeling: A Multidisciplinary Journal, 26(5), 819-826. doi: 10.1080/10705511.2019.1615835
  • Loehlin, J. C. (1987). Latent variable models: An introduction to factor, path, and structural analysis. Hillsdale, NJ: Erlbaum.
  • Lyons-Thomas, J., Sandilands, D. D., and Ercikan, K. (2014). Gender differential item functioning in mathematics in four international jurisdictions. Education and Science, 39(172), 20-32. Retrieved from http://egitimvebilim.ted.org.tr/index.php/EB/article/view/2873/625
  • Magis, D., Beland, S., and Raiche, G. (2018). Collection of methods to detect dichotomous Differential Item Functioning (DIF). Retrieved from https://cran.rproject.org/web/packages/difR/difR.pdf
  • Magis, D., Beland, S., Tuerlinckx, F., and De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847-862. doi:10.3758/BRM.42.3.847
  • Mertler, C. A. and Vannatta, R. A. (2005). Advanced and multivariate statistical methods: Practical application and interpretation. Los Angeles, LA: Pyrczak.
  • Milli Eğitim Bakanlığı (2018). Sınavla öğrenci alacak ortaöğretim kurumlarına ilişkin merkezî sınav başvuru ve uygulama kılavuzu. Ankara. https://www.meb.gov.tr/meb_iys_dosyalar/2019_04/03134315_Kilavuz2019.p df adresinden erişilmiştir.
  • Osadebe, P. U., and Agbure, B. (2018). Assessment of differential item functioning in social studies multiple choice questions in basic education certificate examination. European Journal of Education Studies, 4(9), 236-257. doi: 10.5281/zenodo.1301272
  • Osterlind, S. J., and Everson, H., T. (2009). Differential item functioning. Thousand Oaks, CA: SAGE Publications, Inc.
  • Raju, N. S., Laffitte, L. J., and Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87(3), 527-529. doi:10.1037//0021- 9010.87.3.517
  • RStudio (2020). Take control of your R code. Retrieved from https://rstudio.com/products/rstudio/
  • Ryan, K. E., and Chiu, S. (2001). An examination of item context effects, DIF, and gender DIF. Applied Measurement in Education, 14(1), 73–90. doi:10.1207/S15324818AME1401_06
  • Shealy, R.,and Stout, W. F. (1993). A model-based standardization approach that separates true bias/ DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58(2), 159–194. doi:0033- 3123/93/0600-91330500.75/0
  • Stark, S., Chernyshenko, O. S., and Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292-1306. doi:10.1037/0021-9010.91.6.1292
  • Stiglic, G., Watson, R., and Cilar, L. (2019). R you ready? Using the R program for statistical analysis and graphics. Research in Nursing and Health, 42(6), 494- 499. doi:10.1002/nur.21990
  • Stoneberg, B. D. (2004). A study of gender-based and ethnic-based Differential Item Functioning (DIF) in the Spring 2003 Idaho Standards Achievement tests applying the Simultaneous Bias Test (SIBTEST) and the Mantel-Haenszel chisquare Test. (Yayın No. Mart, 2004). Retrieved from http://elearndesign, http://files.eric.ed.gov/ fulltext/ED489949.pdf
  • Stout, W., and Roussos, L. (1995). SIBTEST user manual. Urbana: University of Illinois.
  • Swaminathan, H., and Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361–370. https://doi.org/10.1111/j.1745-3984.1990.tb00754.x
  • Tabachnick, B. G., and Fidell, L. S. (2007). Using multivariate statistics. Boston, MA: Pearson.
  • Tan, Ş. (2009). Soru sırasının madde güçlüğü ve ayırıcılık gücüne etkisi. e-Journal of New World Sciences Academy, 4(2), 486-493. http://www.newwsa.com/download/gecici_makale_dosyalari/NWSA-1091-3- 2.pdf adresinden erişilmiştir.
  • Thissen, D. (2001) OOADIF v.2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. Retrieved from http://www.unc.edu/~dthissen/dl.html Using Rtools4.0 on Windows (2020). Retrieved from https://cran.rstudio.com/bin/windows/Rtools/
  • Ünsal-Özberk, E. B. ve Koç, N. (2017). WÇZÖ-IV maddelerinin cinsiyet ve sosyoekonomik düzey açısından işlev farklılığının belirlenmesinde kullanılan yöntemlerin karşılaştırılması. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 8(1), 112-127. doi:10.21031/epod.287577
  • Van Buuren, S. (2012). Flexible imputation of missing data. Boca Raton, FL: Chapman and Hall/CRC.
  • Vandenberg, R. J., and Lance, C. E. (2000). A review and synthesis of the MI literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-69. doi:10.1177/109442810031002
  • Walzebug, A. (2014). Is there a language-based social disadvantage in solving mathematical items? Learning, Culture and Social Interaction, 3(2), 159-169. doi:10.1016/j.lcsi.2014.03.002
  • Yen, W. M. (1980). The extent, causes and importance of context effects on item parameters for two latent trait models. Journal of Educational Measurement, 17(4), 297–311. doi:10.1111/j.1745- 3984.1980.tb00833.x
  • Yıldırım, H. (2015). 2012 yılı seviye belirleme sınavı matematik alt testinin madde yanlılığı açısından incelenmesi (Yayınlanmamış yüksek lisans tezi). https://tez.yok.gov.tr/UlusalTezMerkezi/’nden erişilmiştir. (Tez No. 388230).
  • Zumbo, B. D., and Thomas, D. R. (1996). A measure of DIF effect size using logistic regression procedures. Paper presented at National Board of Medical Examiners, Philadelphia, PA. Retrieved from http://www.educ.ubc.ca/faculty/zumbo/cv.htm.
  • Zumbo, B. D. (1999). A Handbook on the Theory and Methods of Differential Item Functioning (DIF): Logistic Regression Modeling as a Unitary Framework for Binary and Likert-Type (Ordinal) Item Scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.
Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi-Cover
  • ISSN: 1301-3718
  • Yayın Aralığı: Yılda 3 Sayı
  • Başlangıç: 1968
  • Yayıncı: ANKARA ÜNİVERSİTESİ (EĞİTİM BİLİMLERİ FAKÜLTESİ)