Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models

Bu çalışmada, pozitif duygu durumlarının (mutlu, huzurlu gibi) ölçülmesinde kullanılmak üzere geliştirilen 14 maddelik Pozitif Duygu Durum Ölçeği’nde yer alan likert tipi maddelerin beşli tepki kategorileri için maddeler-içi ve maddeler-arası geçişlilik özellikleri Aşamalı Tepki Modeli, Kısmi Puanlama Modeli ve Dereceli Ölçekleme Modeli kullanılarak karşılaştırmalı olarak incelenmiştir. Her bir model için, model veri uyumu, ölçekte yer alan maddelerin tepki kategorileri, kategori eşik (threshold) değerleri ve kategori sıraları hesaplanmış, maddeler arası eşdeğerlik varsayımının ne kadar sağlandığı incelenmiştir. Model veri uyumu, model kestirimlerinden elde edilen -2log likelihood ki-kare değerleri, AIC ve BIC değerleri kullanılarak değerlendirilmiştir. Uygulama verisinde 339 gönüllü katılımcı yer almaktadır ve ölçek için hesaplanan Cronbach α güvenirlik katsayısı 0.92’dir. Analizler ölçeğin tek boyutlu bir yapıya sahip olduğunu göstermektedir. Model-veri uyumu değerlendirmesi sonucunda veriye en iyi uyum sağlayan modelin Aşamalı Tepki Modeli olduğu sonucu elde edilmiştir. Aşamalı Tepki Modeli ile parametre kestirimi yapılan maddeler için gözlenen kategori eşik parametrelerinin hem madde içlerinde beklendiği gibi neredeyse eşit aralıklar ile sıralı olduğu hem de bu aralıkların ölçeği oluşturan tüm maddeler için karşılaştırılabilir olduğu sonucu elde edilmiştir.

The purpose of this study was to investigate within- and between-threshold parameter invariance for items of a fourteen-item Positive Affect Scale developed to assess positive moods (like happy, peaceful, etc.) of university students. To test whether the estimated threshold parameters were as expected (1 to 5, with increments of 1) across all the 14 items, Graded Response, Partial Credit, and Rating Scale Models were fit the response data collected from 326 students. A comparison of the model fit statistics, such as the negative 2log likelihood and chi-square values, revealed that the Graded Response Model had the best fit and that the thresholds estimates for all the items in the Positive Affective Scale were reasonably close to the expected 1 to 5 values with increments of 1. The study illustrates how polytomous response models can be used to test the psychometric quality of items with ordinal rating scales.

Kaynakça

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561-573. doi: 10.1007/BF02293814

Baker, F. B. (2001). The basis of item response theory. USA: ERIC Clearing house on Assessment and Evaluation.

Baker, J. G., Rounds, J. B., & Zevon, M. A. (2000). A comparison of graded response and Rasch partial credit models with subjective well-being. Journal of Educational and Behavioral Statistics, 25(3), 253-270. doi: 10.3102/10769986025003253

Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York, NY: Guilford.

Caycho-Rodríguez, T., Vilca, L. W., Carbajal-León, C., White, M., Vivanco-Vidal, A., Saroli-Araníbar, D., ..., Moreta-Herrera, R. (2021). Coronavirus anxiety scale: New psychometric evidence for the Spanish version based on CFA and IRT models in a Peruvian sample. Death Studies. doi: 10.1080/07481187.2020.1865480

Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. doi: 10.18637/jss.v048.i06

Chernyshenko, O. S., Stark, S., Chan, K. Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: issues and insights. Multivariate Behavioral Research, 36(4), 523-562. doi: 10.1207/S15327906MBR3604_03

Cho, S., Drasgow, F., & Cao, M. (2015). An investigation of emotional intelligence measures using item response theory. Psychological Assessment, 27(4), 1241-1252. doi: 10.1037/pas0000132

de Ayala, R. J., Dodd, B. G., & Koch, W. R. (1990, April). A comparison of the partial credit and graded response model in computerized adaptive testing. Paper presented at the AERA Annual Meeting. Boston.

DeMars, C. (2010). Item response theory: Understanding statistics measurement. Oxford: Oxford University Press.

Demirtaşlı, N., Yalçın, S., & Ayan, C. (2016). The development of irt based attitude scale towards educational measurement course. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 1(7), 133-144. doi: 10.21031/epod.43804

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey: LEA publishers.

Ferrando, P., Lorenzo, U., & Molina, G. (2001). An item response theory analysis of response stability in personality measurement. Applied Psychological Measurement, 25(1), 3-17. doi: 10.1177/01466216010251001

Flannery, W. P., Reise, S. P., & Widaman, K. F. (1995). An item response theory analysis of the general and academic scales of the self-description questionnaire II. Journal of Research in Personality, 29(2), 168-188. doi: 10.1006/jrpe.1995.1010

Glass, G. V., & Hopkins, K. D. (1984). Statistical methods in education and psychology. Englewood Cliffs, NJ: Prentice Hall.

Gray-Little, B., Williams, V., & Hancock, T. (1997). An item response theory analysis of the Rosenberg self - esteem scale. Personality and Social Psychology Bulletin, 23(5), 443-451. doi: 10.1177/0146167297235001

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. New York: Springer Science and Business Media.

Hattie, J. (1992). Self-concept. Hillsdale, NJ: Erlbaum.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. doi: 10.1080/10705519909540118

Kahraman, N., Akbaş, D., & Sözer, E. (2019). Bilişsel-olmayan öğrenme durum ve süreçlerini ölçme ve değerlendirmede boylamsal yaklaşımlar: Duygu Cetveli Alan Uygulaması örneği. Bolu Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 19(1), 257-269. doi: 10.17240/aibuefd.2019.19.43815-459831

Kaptan, S. (1995). Bilimsel araştırma ve istatistik teknikleri (10. Basım). Ankara: Rehber Yayınevi.

Koch, W. R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7(1), 15-32. doi: 10.1177/014662168300700104

Köse, İ. A. (2015). Aşamalı tepki modeli ve klasik test kuramı altında elde edilen test ve madde parametrelerinin karşılaştırılması. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 184-197. Retrieved from https://dergipark.org.tr/tr/download/article-file/17439

Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85-106. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.424.2811&rep=rep1&type=pdf

Lord, F. M. (1980). Applications of item response theory practical testing problems. Hillsdale, NJ: Erlbaum.

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. doi: 10.1002/j.2333-8504.1994.tb01618.x

Min, S., & Aryadoust, V. (2021). A systematic review of item response theory in language assessment: implications for the dimensionality of language ability. Studies in Educational Evaluation, 68. doi: 10.1016/j.stueduc.2020.100963

Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159-176. doi: 10.1177/014662169201600206

R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/

Roskam, E. E. (1985). Current issues in item response theory. In E. E. Roskam (Ed.), Measurement and personality assessment (pp. 3-19). Amsterdam: North Holland.

Rubio, V. J., Aguado, D., Hontangas, P. M., & Hernandez, J. M. (2007). Psychometric properties of an emotional adjustment measure: an application of the Graded Response Model. European Journal of Psychological Assessment, 23(1), 39-46. doi: 10.1027/1015-5759.23.1.39

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monography No. 17). Retrieved from https://www.psychometricsociety.org/sites/main/files/file-attachments/mn17.pdf?1576606975

Samejima, F. (1996). Evaluation of mathematical models for ordered polychotomous responses. Behaviormetrika, 23, 17-35. doi: 10.2333/bhmk.23.17

Silvia, P. J. (2021). The self-reflection and insight scale: applying item response theory to craft an efficient short form. Current Psychology. doi: 10.1007/s12144-020-01299-7

Steiger, J. H., & Lind, J. M. (1980, May). Statistically based tests for the number of common factors. Paper presented in Psychometric Society. Iowa City.

Wang, W. C., Wilson, M., & Shih, C. L. (2006). Modeling randomness in judging rating scales with a random-effects rating scale model. Journal of Educational Measurement, 43(4), 335-353. doi: 10.1111/j.1745-3984.2006.00020.x

Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063-1070. doi: 10.1037/0022-3514.54.6.1063

Wu, M., & Adams, R. (2006). Modelling mathematics problem solving item responses using a multidimensional IRT model. Mathematics Education Research Journal, 18(2), 93-113. doi: 10.1007/BF03217438

Yaşar, M., & Aybek, E. C. (2019). Üniversite öğrencileri için bir yılmazlık ölçeğinin geliştirilmesi: Madde tepki kuramı temelinde geçerlilik ve güvenilirlik çalışması. İlköğretim Online, 18(4), 1687-1699. Retrieved from https://ilkogretim-online.org/fulltext/218-1597121020.pdf?1618815938

Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145. doi: 10.1177/014662168400800201

Kaynak Göster

Bibtex @araştırma makalesi { epod819927, journal = {Journal of Measurement and Evaluation in Education and Psychology}, issn = {1309-6575}, eissn = {1309-6575}, address = {}, publisher = {Eğitimde ve Psikolojide Ölçme ve Değerlendirme Derneği}, year = {2021}, volume = {12}, pages = {129 - 146}, doi = {10.21031/epod.819927}, title = {Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models}, key = {cite}, author = {Sözer, Esra and Kahraman, Nilüfer} }
APA Sözer, E , Kahraman, N . (2021). Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models . Journal of Measurement and Evaluation in Education and Psychology , 12 (2) , 129-146 . DOI: 10.21031/epod.819927
MLA Sözer, E , Kahraman, N . "Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models" . Journal of Measurement and Evaluation in Education and Psychology 12 (2021 ): 129-146 <
Chicago Sözer, E , Kahraman, N . "Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models". Journal of Measurement and Evaluation in Education and Psychology 12 (2021 ): 129-146
RIS TY - JOUR T1 - Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models AU - Esra Sözer , Nilüfer Kahraman Y1 - 2021 PY - 2021 N1 - doi: 10.21031/epod.819927 DO - 10.21031/epod.819927 T2 - Journal of Measurement and Evaluation in Education and Psychology JF - Journal JO - JOR SP - 129 EP - 146 VL - 12 IS - 2 SN - 1309-6575-1309-6575 M3 - doi: 10.21031/epod.819927 UR - Y2 - 2021 ER -
EndNote %0 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models %A Esra Sözer , Nilüfer Kahraman %T Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models %D 2021 %J Journal of Measurement and Evaluation in Education and Psychology %P 1309-6575-1309-6575 %V 12 %N 2 %R doi: 10.21031/epod.819927 %U 10.21031/epod.819927
ISNAD Sözer, Esra , Kahraman, Nilüfer . "Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models". Journal of Measurement and Evaluation in Education and Psychology 12 / 2 (Haziran 2021): 129-146 .
AMA Sözer E , Kahraman N . Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models. Journal of Measurement and Evaluation in Education and Psychology. 2021; 12(2): 129-146.
Vancouver Sözer E , Kahraman N . Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models. Journal of Measurement and Evaluation in Education and Psychology. 2021; 12(2): 129-146.
IEEE E. Sözer ve N. Kahraman , "Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models", Journal of Measurement and Evaluation in Education and Psychology, c. 12, sayı. 2, ss. 129-146, Haz. 2021, doi:10.21031/epod.819927