Gruplararası Karşılaştırmalarda Ölçek Eşdeğerliğinin İncelenmesi: Madde ve Test Fonksiyonlarının Farklılaşması

Gerek kültürlerarası karşılaştırmalarda, gerek belirli bir kültür içerisindeki gruplararası karşılaştırmalarda ölçme eşdeğerliğinin sağlanması temel metodolojik problemlerden birisini oluşturmaktadır. Ölçme eşdeğerliğinin incelenmesinde son yıllarda giderek yaygınlaşan madde-cevap kuramına dayalı modellerden yararlanılmaktadır. Madde ve Test Fonksiyonlarının Farklılaşması (Differential Item-Test Functioning, DIF-DTF) genel başlığı altında ele alınan bu modeller, gözlenen test puanları ile bunların altında yatan örtük özellik arasındaki ilişkinin, karşılaştırma grupları açısından eşit olup olmadığının incelenmesine dayalıdır. Ölçülen özellik bakımından aynı düzeyde bulunan, fakat farklı gruplara ait kişilerin maddeyi anahtarlanan yönde cevaplama olasılıkları farklılaştığında, madde fonksiyonlarının farklılaşması ya da maddenin karşılaştırma grupları için yanlı olması söz konusudur. Araştırmamızda 1807 kişilik bir öğrenci örneklemi üzerinde, bir kişilik alt ölçeğinin (Yumuşak Başlılık) 16 maddesinin kız ve erkek öğrenciler için madde fonksiyonlarının farklılıkları incelenmiştir. Yapılan analizler sonucunda 16 maddenin hepsinin iki parametreli modele uyum sağladığı, beş maddenin kız ve erkek öğrenciler arasında farklılık gösterdiği görülmüş, bu maddelerin özellikleri incelenmiş, toplam ölçek bazında nasıl ele alınabileceği tartışılmıştır.

Measurement Equivalence in Group Comparisons: Differential Item and Test Functioning

Both in within and cross-cultural settings, measurement equivalence is one of the most important methodological problems in comparisons of group differences. Models based on Item Response Theory are being widely used in recent years in holding measurement equivalence. These models (generally take place under the title of Differential Item-Test Functioning-DIF, DTF) refer to the methods analyzing the relations between observed scores and the latent attribute measured by the test, across comparison groups. The existence of DIF-DTF is evidenced when these relations are different across comparison groups. DIF is defined as differences in the probability of endorsing an item between members of the reference and focal groups having the same latent trait level. In this study, DIF analyses of 16 items of a personality scale (agreeableness) were performed on a student sample (1807 subjects). According to the results, all of the 16 items fitted to the two-parameter logistic model, but 5 items of the agreeableness scale showed differential item functioning between girls and boys. The properties of these DIF items and how to handle them are discussed.

PDF

___

Ackerman, T. A. (1989). Unidimensional IRT calibration ofcompensatory and non-compensatory multidimensionalitems. Applied Psychological Measurement, 13(2),113-117.
Byrne, B. M., Shavelson, R. J., & Muthen, B. (1989). Testing theequivalence of factor covariance and mean structures:the issue of partial measurement invariance.Psychological Bulletin, 105 (3), 456-466.
Camilli, G., & Shepard, L. A. (1994). Methods for IdentifyingBiased Test Items. California: Sage Pub. Inc.
Collins, C. C., Raju, S. N., & Edwards, J. E. (2000). Assessingdifferential functioning in a satisfaction scale. Journal ofApplied Psychology,85(3), 451-461.
Ellis, B. B., & Mead A. D. (2000). Assessment of measurementequivalence of a Spanish translation of the 16PFQuestioanaire. Educational and PsychologicalMeasurement,60(5), 787-807.
Ferrando, P. J. (2001). The measurement of neuroticism usingMMQ, MPI, EPI and EPQ items: a psychometricanalysis based on item response theory. Personality andIndividual Differences,30, 641-656.
Geisinger, K. F. (1994). Cross-cultural normative assesment:translation and adaptation issues influencing thenormative interpretation of assesment instruments.Psychological Assesment, 6(4), 304-312.
Hambleton, R. K., Frederic, R., & Xing, D. (2000). ItemResponse Models for the analysis of educational andpsychological test data Personality Research. In (Eds.) H.E. A. Tinsley & S. D. Brown, Handbook of AppliedMultivariate Statistics and Mathematical Modeling.(pp. 553-581). San Diago: Academic Press.
Hambleton, R. K., & Swaminathan, H. (1989). Item ResponseTheory, Principles and Applications. Kluwer NijhoffPublishing, Boston.
Hambleton, R. K., Swaminathan, H., & Rogers. H. J. (1991).Fundamentals of Item Response Theory. Sage Pub. CA.
Hayduk, L. A. (1987). Structural Equation Modeling withLISREL, Essentials and advances. The John HopkinsPress Ltd., London..
Hoijtink, H., Rooks, G., & Wilmink, F. W. (1999). ConfirmatoryFactor analysis of items with a dichotomous responseformat using the multidimensional Rasch Model.Psychologicl Methods, 4 (3), 300-314.
Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). ItemResponse Theory: Application to PsychologicalMeasurement. Illinois: Dow Jones-Irwin.
Kim, S-H., & Cohen, A. S. (1995). A comparison of Lord’sChi-Square, Raju’s Area Measures, and Likelihood RatioTests on detection of Differential Item functioning.Applied Measurement in Education, 8(4), 291-312
Lambert M. C., Schmitt, N., Vaughan, m. E. S., An, J. S.,Fairclough, M., & Nutter, C. A. (2003). Is it prudent toadminister all items for Each Child Behavior Checklistcross-informant syndrome? Evaluating the psychometricProperties of the Youth Self-Report Dimensions withconfirmatory factor analysis and item response theory.Psychological Assessment, 15 (4), 550-568.
Meijer, R. R. (2003). Diagnosing item score patterns on a testusing item response theory-based person-fit statistics.Psychological Methods, 8(1), 72-87.
Mellenbergh, G. J. (1994). Generalized lineer Item ResponseTheory. Psychological Bulletin, 115(2), 300-307.
Millsap, R. E. (1997). Invariance in measurement and prediction:Their relationship in the Single-factor case.Psychological Methods, 2(3), 248-260.
Muraki, E., & Bock, R. D. (2002). PARSCALE: Parameterscaling of rating data(Version 4. 1) (Software Manual).Chicago: Scientific Software Inc.
Muthen, B., & Christoffersson, A. (1981). Simultaneous factoranalysis of dichotomous variables in several groups.Psychometrika, 4674,407-419.
Orlando, M., & Rand, G. N. M. (2002). Differential itemfunctioning in a spanish translation of the PTSCChecklist: Detection and evaluation of impact.Psychological Assesment, 1(1), 50-59.
Raju, N. S. (1990). Determining the significance of estimatedsigned and and unsigned areas between two itemresponse functions. Applied PsychologicalMeasurement, 14(2), 197-207.
Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurementequivalance: a comparison of methods based onconfirmatory factor analysis and item response theory.Journal of Applied Psychology, 87(3), 517-529.
Raju, N. S., van der Linden, W. J., & Fleer, P. F. (1995).IRT-based internal measures of differential functioningof items and tests. Applied Psychological Measurement,19 (4), 353-368.
Reise, S. P. (1999). Personality measurement issues viewedthrough the eyes of IRT. In (Eds.) S. E Embretson., S. L.Hershberger The New Rules of Measurement: WhatEvery Psychologist Should Know(pp. 219-241).Mahwah, New Jersey: L. Earlbaum Associates,Publishers.
Reise, S. P., & Waller, N. G. (2003). How many IRT parametersdoes it take to model psychopathology items?Psychological Methods, 8(2), 164-184.
Reise, S. P., Waller, N. G., & Comrey, A. L. (2000). Factoranalysis and scale revision. Psychological Assessment,12(3), 287-297.
Reise, S. P., & Widaman, K. F. (1999). Assessing the fit ofmeasurement models at the individual level: Amresponse theory and covariance structure approach.Psychological Methods, 4 (1), 3-21.
Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatoryfactor analysis and Item Response Theory, twoapproaches for exploring measurement invariance.Psychological Bulletin, 114(3), 552-566.
Roznowski, M., & Reith, J. (1999). Examining the measurementquality of tests containin differentially functioning items:do biased items result in poor measurement. Educationaland Psychological Measurement, 59, 248-270.
Schumacker, R. E., & Lomax, R. G. (1996). A beginner’s guide toStructural Equation Modeling. Lawrencw ErlbaumAssociates Inc., New Jersey.
Somer, O. (1999). Çok Kategorili (Polytomous) Maddelerde,Klasik ve Modern Test Kuramlar› ‹le Madde Analizleri,Güvenirlik ve Bilgi Kavramlar›n›n Karfl›laflt›r›lmas›.Türk Psikoloji Dergisi, 14(44), 63-78.
Somer, O. (1998). Kiflilik Testlerinde Klasik ve Modern TestKuramlar› ‹le Madde Analizi. Türk Psikoloji Dergisi, 13(41), 1-17.
Somer, O., Korkmaz, M., & Tatar, A. (2002). Befl Faktör KiflilikEnvanteri’nin Gelifltirilmesi I: Ölçek ve Alt ÖlçeklerinOluflturulmas›. Türk Psikoloji Dergisi, 17 (49), 21-33.
Smith, L. L. (12002). On the usefulness of item bias analysis topersonality psychology. Society for Personality andSocial Psychology, 28(7), 754-763.
Steinberg, L. (2001). The consequences of pairing questions:Context effects in personality measurement. Journal ofPersonality and Social Psychology, 81 (2), 332-343.
Steinberg, L., & Thissen, D. (1995). Item Response Theory inPersonality Research. In (Eds.) P. E. Shrout & T. Fiske,Personality Research, Methods, and Theory.(pp. 161-181). Hillsdale, N. J. : Earlbaum.
Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection ofdifferential item functioning using the parameters of ItemResponse Theory. In (Eds.) P. H. Holland & H. Wainer,Differential Item Functioning, (pp. 67-113). Hillsdale,NJ: Earlbaum.
Traub, R. E. (1983). A priori considerations in choosing an itemresponse model. In R.K. Hambleton (Ed.), Applicationsof Item Response Theory. Vancouver. BC: EducationalRessearch Institude of British Columbia.
Van de Vijver, Fons J. R., & Leung, K. (2000). Methodologicalissues in psychological research. Journal ofCross-Cultural Psychology, 31(1), 19-33.
Waller, N. G., Thompson, J. S., & Wenk, E. (2000). Using IRT toseperate measurement bias from true group differenceson homogeneous and heterogeneous Scales: Anillustration with the MMPI. Psychological Methods, 5(1), 125-146.