Osman DEMİR, Hasan DAĞMURA, Emin DALDAL, Fatih DAŞIRAN, İsmail OKAN, Hüseyin BAKIR, Ali İhsan SAĞLAM, Murat YILDIRIM, Ertan BÜLBÜLOĞLU

Does the performance assessment of students during surgery clerkship in a medical faculty jeopardize the objectivity of unbiased structured clinical examination?

Aim: Objective structured clinical examination (OSCE) has become an important assessment method in medicine which is more reliable than traditional exams. Faculty members make an opinion about the students during their clerkship. We aimed to investigate whether their judgments affect the scoring in the OSCE application.Material and Methods: The study was planned prospectively. 4th-year students participating in the OSCE was identified as the working group. At the end of the clerkship, the faculty members gave a performance assessment (PA) score including the professional attitude of the students. PA and OSCE scores were compared. Two other faculty members participated all OSCE stations as external evaluators.Results: There was a difference between OSCE and PA scores. Fewer students were successful at OSCE than PA (p=0.002). While the mean PA scores of three of the five faculty members were statistically similar, other two were different. The scores given by the responsible faculty members and the external evaluators from the faculty of internal medicine were similar (p>0.05). The evaluators from the surgical faculty gave lower scores statistically different from both groups (p

PDF

___

1. Tabish SA. Assessment methods in medical education. Int J Health Sci, 2010;2.
2. Mohammad AH, Gominda P, Zubair A. The OSCE in the context of a holistic assessment, Hamza Mohammad Abdulghani, Essential Guide To Developing, Implementing, And Evaluating Objective Structured Clinical Examination, Singapore, World Scientific 2014:9-21.
3. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. Br Med J 1975;1:447-51.
4. McAleer S, Walker R. Objective structured clinical examination (OSCE). Occas Pap R Coll Gen Pract 1990:39.
5. Falcone JL, Claxton RN, Marshall GT. Communication skills training in surgical residency: a needs assessment and metacognition analysis of a difficult conversation objective structured clinical examination. J Surg Educ 2014;71:309-15.
6. Gopalakrishnan M. Objective Structured Clinical Examinations (OSCEs) as an Assessment Tool in Undergraduate Medical Education. J Natl Med Assoc 2018;110:553.
7. Khan KZ, Ramachandran S, Gaunt K, et al. The objective structured clinical examination (OSCE): AMEE guide no. 81. Part I: an historical and theoretical perspective. Med Teach 2013;35:1437-46.
8. Mazor KM, Zanetti ML, Alper EJ, et al. Assessing professionalism in the context of an objective structured clinical examination: an in-depth study of the rating process. Med Educ 2007;41:331-40.
9. Arnold L. Assessing professional behavior: yesterday, today, and tomorrow. Acad Med 2002;77:502-15.
10. Patrício M, Julião M, Fareleira F, et al. A comprehensive checklist for reporting the use of OSCEs. Med teach, 2009;31:112-24.
11. Simon SR, Bui A, Day S, Berti D, Volkan K. The relationship between second-year medical students’ OSCE scores and USMLE Step 2 scores. J Eval Clin Pract 2007;13:901-5.
12. Van Der Vleuten CPM, The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ Theory Pract 1996; 1:41-67.
13. Ginsburg S, Regehr G, Hatala R, et al. Context, conflict, and resolution: a new conceptual framework for evaluating professionalism. Acad Med 2000;75:6-11.
14. Jha V, HL Bekker HL, Duffy SRG, et al. Perceptions of professionalism in medicine: a qualitative study. Med Educ 2006;40:1027-36.
15. Stroud L, Herold J, Tomlinsonet G, et al. Who you know or what you know? Effect of examiner familiarity with residents on OSCE scores. Acad Med 2011;86:8-11.
16. Iramaneerat C, Yudkowsky R. Rater errors in a clinical skills assessment of medical students. Eval Health Prof 2007;30:266-83.
17. Wood TJ. Exploring the role of first impressions in rater-based assessments. Adv Health Sci Educ Theory Pract 2014;19:409-27.
18. McKinstry BH, Cameron HS, Elton RA, et al. Leniency and halo effects in marking undergraduate short research projects. BMC Med Educ 2004;4:28.
19. Malouff JM, Stein SJ, Bothma LN, et al. Preventing halo bias in grading the work of university students. Cogent Psychol 2014;1: 988937.
20. Malouff JM, Emmerton AJ, Schutte NS. The risk of a halo bias as a reason to keep students anonymous during grading. Teach Psychol 2013;40:233-7.
21. Jefferies A, Simmons B, Regehr G. The effect of candidate familiarity on examiner OSCE scores. Med educ 2007;41:888-91.
22. McManus IC, Thompson M, Mollon J. Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP (UK) clinical examination (PACES) using multi-facet Rasch modelling. BMC Med Educ 2006;6:42.
23. Finn Y, Cantillon P, Flaherty G. Exploration of a possible relationship between examiner stringency and personality factors in clinical assessments: a pilot study. BMC Med Educ 2014;14:1052.
24. Gormley G. Summative OSCEs in undergraduate medical education. Ulster Med J 2011;80:127.