A Comparison of Two Standard-Setting Methods for Tests Consisting of Constructed-Response Items

A Comparison of Two Standard-Setting Methods for Tests Consisting of Constructed-Response Items

Purpose: This study aimed to compare the cut scoresobtained by the Extended Angoff and ContrastingGroups methods for an achievement test consistingof constructed-response items.Research Methods: This study was based on surveyresearch design. In the collection of data, the studygroup of the research consisted of eight mathematicsteachers for the Extended Angoff method, 75 eighthgrade students, and a mathematics teacher for theContrasting Groups method. Data were collectedthrough math achievement test consisting ofconstructed-response items, scoring rubrics, expert opinion form for Extended Angoff method and student classification form for ContrastingGroups method

___

  • Akhun, I. (1982). Iki yuzde arasindaki farkin manidarliginin test edilmesi [Testing the significance of the difference between two percentages]. Ankara University Journal of Educational Sciences, 15(1), 240-259. doi: 10.1501/Egifak_0000000817
  • Baykul, Y. (1992). Egitim sisteminde degerlendirme [Evaluation in education system]. Hacettepe University Journal of Education (7), 85-94.
  • Baykul, Y. (2000). Egitimde ve psikolojide olcme: klasik test teorisi ve uygulamasi [Measuring in education and psychology: classical test theory and practice.]. Ankara: OSYM Yayinlari.
  • Brandon, P. R. (2002). Two versions of the contrasting-groups standard-setting method: A review. Measurement and Evaluation in Counseling and Development, 35(3), 167-181. doi: 10.1080/07481756.2002.12069061
  • Buyukozturk, S., Kilic Cakmak, E., Akgun, O. E., Karadeniz, S., & Demirel, F. (2013). Bilimsel arastirma yontemleri [Scientific research methods] (14 b.). Ankara: Pegem Akademi.
  • Cizek, G. J. (1993). Reconsidering Standards and Criteria. Journal of Educational Measurement, 30(2), 93-106. doi: 10.1111/j.1745-3984.1993.tb01068.x
  • Cizek, G. J., & Bunch, M. B. (2007). Standart Setting: A Guide to Establishing and Evaluating Performance Standards on Tests. Sage Publications.
  • Cetin, S., & Gelbal, S. (2010). Farkli standart belirleme yontemlerinin gecme puanlari uzerine etkisi [Impact of standard setting methodologies over passing scores]. Ankara University Journal of Educational Sciences, 43(1), 79-95. doi: 10.1501/Egifak_0000001191
  • Demir, O., & Kose, İ. A. (2014). Angoff, Nedelsky ve Ebel standart belirleme yontemleri ile belirlenen kesme puanlarinin karsilastirilmasi [A comparison of cutting points determined by Angoff, Nedelsky and Ebel standard setting methods]. Mersin University Journal of the Faculty of Education, 10(2), 14-27. doi: 10.17860/efd.18119
  • Gundeger, C., & Dogan, N. (2014). Angoff, Yes/No ve Ebel standart belirleme yontemlerinin karsilastirilmasi [A comparison of Angoff, Yes/No and Ebel standard setting methods]. Journal of Measurement and Evaluation in Education and Psychology, 5(1), 53-60. doi: 10.21031/epod.47091
  • Hambleton, R. K., & Plake, B. S. (1995). Using an extended Angoff procedure to set standards on complex performance assessments. Applied Measurement in Education, 8(1), 41-55. doi: 10.1207/s15324818ame0801_4
  • Hambleton, R. K., Jaeger, R. M., Plake, B. S., & Mills, C. (2000). Setting performance standards on complex educational assessments. Applied psychological measurement, 24(4), 355-366. doi: 10.1177/01466210022031804
  • Jaeger, R. M. (1989). Certification of student competence. R. L. Linn (Ed). Educational measurement (s. 485-514). New York: Macmillan.
  • Kane , M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64(3), 425-651. doi: 10.3102/00346543064003425
  • Konge, L., Clementsen, P., Larsen, K. R., Arendrup, H., Bunchwald, C., & Ringsted, C. (2012). Establishing pass/fail criteria for Bronchoscopy performance. Respiration, 83, 140-146. doi: 10.1159/000332333
  • Korkmaz, S. (2015). Evet/Hayir, Ebel ve Isaretleme standart belirleme yontemlerinin karsilastirilmasi [Comparing Yes/No, Ebel and Bookmark standard setting methods]. Retrieved from http://hdl.handle.net/11655/1849
  • Livingston, S. A., & Zieky, M. J. (1982). Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. Princeton, NJ: Educational Testing Service. Retrieved from https://files.eric.ed.gov/fulltext/ED227113.pdf
  • Livingston, S. A., & Zieky, M. J. (1989). A comparative study of standard-setting methods. Applied Measurement in Education, 2(2), 121-141. doi: 10.1207/s15324818ame0202_3
  • Näsström, G., & Nyström, P. (2008). A comparison of two different methods for setting performance standards for a test with constructed-response items. Practical Assessment, Research, and Evaluation, 13(1), 9. doi: 10.7275/bhb9-8t88
  • Omur, S., & Selvi, H. (2010). Angoff, Ebel ve Nedelsky yontemleriyle belirlenen kesme puanlarinin siniflama tutarliliklarinin karsilastirilmasi [Comparison of classification consistency of cutting scores obtained from Angoff, Ebel and Nedelsky methods]. Journal of Measurement and Evaluation in Education and Psychology, 1(2), 109-113. Retrieved from https://dergipark.org.tr/tr/pub/epod/issue/5807/77246
  • Saracbasi, O., Karaagaoglu, E., & Saka, O. (1986). Basic programlama ve istatistiksel yontemler [Basic programming and statistical methods]. Ankara: Unalan Ofset
  • Tasdelen, G., Kelecioglu, H., & Guler, N. (2010). Nedelsky ve Angoff standart belirleme yontemleri ile elde edilen kesme puanlarinin genellenebilirlik kurami ile karsilastirilmasi [A comparison of cut points obtained from Nedelsky and Angoff standard setting methods with Generalizability Theory]. Journal of Measurement and Evaluation in Education and Psychology, 1(1), 22-28. Retrieved from https://dergipark.org.tr/tr/pub/epod/issue/5808/77250
  • Tekin, H. (2017). Egitimde Olcme ve Degerlendirme [Measurement and evaluation in education] (Yirmi besinci b.). Yargi Yayinevi.
  • Turgut, M. F., & Baykul, Y. (2014). Egitimde Olcme ve Degerlendirme [Measurement and evaluation in education] (Altinci b.). Ankara: Pegem Akademi.
  • Tulubas, G. (2009). Psikolojik testlerde Angoff ve sinir grup yontemleri ile kesme puanlarinin belirlenmesi [To determine cut off points with Angoff and Borderline Group Tecniques in psychological testing] (Unpublished master's thesis). Hacettepe University, Ankara.