Managıng task-related trouble ın l2 oral profıcıency tests: contrastıng ınteractıon data and rater assessment

Bu çalışma deneysel bir çıkış noktası olarak ikinci dil konuşma testlerindeki etkileşimin doğasını temel almaktadır. Öğrencilerin sözlü yeterlilik testlerindeki performansı ile ‘test konuşması’ yönetimi sosyal uygulaması arasındaki ilişkiyi incelemekteyiz. Temel odak noktamız konuşma çözümlemesi kullanarak öğrecilerin akranlarıyla yürüttükleri test etkileşimlerinde aktiviteye bağlı problemlerle nasıl başa çıktıklarıdır. Öğrencilerin sadece dilbilimsel yeteneklerinin değil ayni zamanda etkileşimsel yeteneklerinin ve kendilerine verilen konuyu ele alışlarının da ölçüldüğünü göz önünde bulundurarak, etkinlik yönetimi üzerine olan vurgumuz ‘test etkinliğine yönelim genel test neticesine derinlemesine bağlıdır’ varsayımından yola çıkmaktadır. Aktivite-ilintili Sorunların (AİS) farklı çeşitlerinin muhtelif test aktivitesi anlayışlarını ortaya çıkardığını ve ‘iyi bir aktivite yöneticisi olabilmenin’ aktiviteye ve test biçimine ölçülü bir yaklaşıma bağlı olduğunu göstermekteyiz. Bu aktivite yönetme stratejilerini uygulayabildiğini gösteren öğrenciler aynı zamanda yüksek derecede yetkin olarak ölçülmüşlerdir, ancak çalışmamızda belirlediğimiz diğer aktivite yönetim stratejileri düşük notlar ile ilişkilendirilmiştir. Fakat, alt-beceri değerlendirmeleri ile görev yönetimi arasındaki ilişki her zaman çok açık olmamıştır. Öğrencilerin gösterdiği farklı test anlayışlarının nasıl değerlendirildiklerinin bir parçası olduğunu ve belirli aktivite yönetim stratejilerinin daha az uygun olarak ölçüldüğünü savunmaktayız. Çalışmamızın sözlü yeterlilik alt-beceri ölçülerinin geçerliğini sağladığına ve öğrenci performansı ve değerlendirmesi ile alakalı sonuçları olduğu için de test görevlerinin tasarlanması konusunu sorguladığına inanmaktayız.

The present study takes as an empirical point of departure the nature of interaction in second language speaking tests. We examine the relationship between ratings of students’ performance in an oral proficiency test and the social practice of conducting ‘test talk’. Using conversation analysis, our focal point is how students in peer- driven test interactions manage trouble related to the task-at-hand. Given that students were assessed not only on their linguistic skills, but also on their interactional ability and treatment of topics assigned, our emphasis on task management stems from a hypothesis that orientation to the test task is intimately connected to overall test outcome. We demonstrate that different types of task-related trouble (TRT) reveal diverse understandings of the test task and that ‘doing-being a successful task manager’ is connected to a moderate orientation to the task and test format. Students displaying such task management strategies were also assessed as highly proficient, whereas other task management strategies identified in our study correlated with low scores and grades. However, the relationship between subskill ratings and task management was not always clear-cut. We argue that the diverging understandings of the test task that learners display become part of how they are assessed and that certain task management strategies are rated less favorably than others. Our study holds promise for the fine-tuning of oral proficiency subskill ratings and raises questions as to the framing of test tasks, since this appears to have implications for student performance and evaluation.

PDF

___

Bernsten, S. G. (2002). Using Conversation Analysis to evaluate pre-sequences in invitation, offer and request dialogues in ESL textbooks. University of Illinois, Urbana- Champaign.
Bowles, H. (2006). Bridging the gap between conversation analysis and ESP- an applied study of the opening sequences of NS and NNS service telephone calls. English for Specific Purposes. 25, 332-357.
Bowles, H. (2011). The contribution of CA to the study of literary dialogue. Novitas-ROYAL (Research on Youth and Language). 5(1), 161-168. Retrieved from
http://www.novitasroyal.org/Vol_5_1/bowles.pdf
Brown, A. (2003). Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20(1), 1-25.
Brown, T.P. and Lewis, M. (2003). An ESP project: Analysis of an authentic workplace conversation. English for Specific Purposes. 22, 93-98.
Brouwer, C. (2003). Word searches in NNS-NS interaction: Opportunities for language learning? The Modern Language Journal, 87(4), 534-545.
Brouwer, C. E. and Wagner, J. (2004). Developmental issues in second language conversation. Journal of Applied Linguistics. 1(1), 30-47.
Carroll, D. (2005). Learning through interactive talk: A school-based mentor teacher study group as a context for professional learning. Teacher and Teacher Education. 21, 457- 473.
Cekaite, A. (2007). A child’s development of interactional competence in a Swedish L2 classroom. The Modern Language Journal. 91(1), 45-62.
Christodoulidou, M. (2011). Lexical markers within the university lecture . Novitas-ROYAL (Research on Youth and Language). 5(1), 143-160. Retrieved from http://www.novitasroyal.org/Vol_5_1/chistodoulidou.pdf
Cook, V. (2007). The nature of the L2 user. In Roberts, L., Gurel, A., Tatar, S. and Marti, L. (eds). EUROSLA Yearbook. 7, 205-20.
Durus, N., Ludwig, M., Sert, O., Steinkraus, R. and Ziegler, G. (2010). Turn-initials as learners’ participation devices: “turning out” in ELF discourse. New Insights into the Study of Conversation: Applications to the Language Classroom. May 26-28, 2010. University of Granada, Granada, Spain.
Egbert, M. (1998). Miscommunication in language proficiency interviews of first-year
German students: comparison with natural conversation. In R. Young & A. He (Eds.), Talking and Testing: Discourse Approaches to the Assessment of Oral Proficiency (pp. 147-69). Amsterdam: Benjamins.
Evnitskaya, N. and Morton, T. (2011). Knowledge construction, meaning-making and interaction in CLIL science classroom communities of practice. Language and Education. 25(2), 109-127.
Firth, A. and Wagner, J. (1997). On discourse, communication, and (some) fundamental concepts in SLA research. The Modern Language Journal. 81, 285-300.
Firth, A. and Wagner, J. (2007). Second/Foreign language learning as a social accomplishment: Elaborations on a ‘reconceptualised’ SLA. The Modern Language Journal. 91, 800-819.
Firth, A. (2009). Doing not being a foreign language learner: English as a lingua franca in the workplace and (some) implications for SLA. International Review of Applied Linguistics in Language Teaching. 47(1), 127-156.
Galaczi, E., D. (2008). Peer-peer interaction in a speaking test: the case of the First Certificate in English examination. Language Assessment Quarterly. 5(2), 89-119.
Gan, Z. (2010). Interaction in group oral assessment: A case study of higher- and lower- scoring students. Language Testing. 27(4), 585-602.
Gan, Z., Davison, C., and Hamp-Lyons, L. (2008). Topic negotiation in peer group oral assessment situations: A conversation analytic approach. Applied Linguistics. 30(3), 315-344.
Garfinkel, H. (1964). Studies in the routine grounds of everyday activities. Social Problems. 11, 225-50.
Garfinkel, H. (1967). Studies in Ethnomethodology. Cambridge: Polity.
Hall, G. (2001). Relationships in a professional development school: Teachers and academics learning together through their talk. ATEA Conference-Teacher Education: Change of
Heart, Mind and Action. Melbourne 24-26 September 2001, 1-9.
Hellermann, J. (2003). The interactive work of prosody in the IRF exchange: Teacher repetition in feedback moves. Language in Society. 32, 79-104.
Hellermann, J. (2005). The sequential and prosodic co-construction of a ‘quiz game’ activity in classroom talk. Journal of Pragmatics. 37, 919-944.
Hellermann, J. (2006). Classroom interactive practices for literacy. Applied Linguistics. 27, 377-404.
Hellermann, J. (2007). The development of practices for action in classroom dyadic interaction: Focus on task openings. The Modern Language Journal. 91(1), 83-96.
Hellermann, J. (2008). Social Actions for Classroom Language Learning. Clevedon: Multilingual Matters.
Hellermann, J. (2009). Practices for dispreferred responses using no by a learner of English.
International Review of Applied Linguistics in Language Teaching. 47(1), 95-126.
Hellermann, J., and Pekarek Doehler, S. (2010). On the Contingent Nature of Language- Learning-Tasks. Classroom Discourse, 1(1), 25-45.
Heritage, J. (1984). Garfinkel and ethnomethodology. Cambridge: Polity Press.
Hutchby, I. and Wooffitt, R. (2008). Conversation Analysis (2nd Edition). Cambridge: Polity Press.
Jacknick, C. M. (2011). “But this is writing”: post-expansion in student-initiated sequences. Novitas-ROYAL (Research on Youth and Language). 5(1), 39-54. Retrieved from http://www.novitasroyal.org/Vol_5_1/jacknick.pdf
Jenks, C. J. (2010). Adaptation in online voice-based chat rooms: implications for language learning in Applied Linguistics. In Seedhouse, P., Walsh, S. and Jenks, C. (eds.). Conceptualising Learning in Applied Linguistics. (pp. 147-162), Basingstoke: Palgrave MacMillan.
Kasper, G., and Ross, S. (2001). 'Is drinking a hobby, I wonder': Other-initiated repair in language proficiency interviews. Paper presented at AAAL, St. Louis, MS.
Kasper, G., and Ross, S. (2003). Repetition as a source of miscommunication in oral proficiency interviews. In J. House, G. Kasper & S. Ross (Eds.), Misunderstanding in social life. Discourse approaches to problematic talk, (pp. 82-106). Harlow, UK: Longman/Pearson Education.
Kasper, G., and Ross, S. (2007). Multiple questions in oral proficiency interviews. Journal of Pragmatics. 39, 2045-2070.
Kääntä, L. (2010). Teacher Turn-allocation and Repair Practices in Classroom Interaction: A Multisemiotic Perspective. Unpublished PhD Thesis. University of Jyväskylä, Jyväskylä.
Koshik, I. (2010). Questions that convey information in teacher-student conferences. In,
Freed, A.F. and Ehrlich, S. “Why do you ask?” The function of questions in institutional discourse. 159-186. New York: Oxford University Press.
Kupetz, M. (2011). Multimodal resources in students’ explanations in CLIL interaction. Novitas-ROYAL (Research on Youth and Language). 5(1), 121-141. Retrieved from http://www.novitasroyal.org/Vol_5_1/ kupetz.pdf
Lazaraton, A. (1997). Preference organization in oral proficiency interviews: The case of language ability assessments. Research on Language and Social Interaction. 30(1), 53-72.
Lazaraton, A. (2002). A qualitative approach to the validation of oral language tests. Cambridge: Cambridge University Press.
Lazaraton, A. (2004). Gesture and speech in the vocabulary explanations of one ESL Teacher: A microanalytic inquiry. Language Learning. 54(1), 79-117.
Lazaraton, A. and Ishihara, N. (2005). Understanding second language teacher practice using microanalysis and self-reflection: A collaborative case study. The Modern Language Journal. 89(4), 529-542.
Lee, Y. (2007). Third turn position in teacher talk: contingency and the work of teaching. Journal of Pragmatics. 39, 180-206.
Liddicoat, A. J. (2007). An introduction to Conversation Analysis. London: Continuum.
Markee, N. (2000). Conversation Analysis. New Jersey: Routledge.
Markee, N. (2008). Toward a learning behavior tracking methodology for CA-for-SLA. Applied Linguistics. 29, 404-427.
Markee, N. and Kasper, G. (2004). Classroom talks: An introduction. The Modern Language Journal. 88, 491–500.
Mashford-Scott, A. and Church, A. (2011). Promoting children’s agency in early childhood education. Novitas-ROYAL (Research on Youth and Language). 5(1), 15-38. Retrieved from http://www.novitasroyal.org/Vol_5_1/mashford-scott_church.pdf
McCarten, J. (2007). Teaching Vocabulary: Lessons from the corpus, lessons from the classroom. New York: Cambridge University Press.
McHoul, A. (1978). The Organization of Turns at Formal Talk in the Classroom. Language in Society. 7, 183-213.
Mondada, L. and Pekarek Doehler, S. (2004). Second language acquisition as situated practice: Task accomplishment in the French second language classroom. The Modern Language Journal. 88(4), 501-518.
Moreno Jaen, M., and Perez Basanta, C. (2009). Developing conversational competence through language awareness and multimodality: the use of DVDs. ReCALL. 21(3), 283-301.
Mori, J. (2005). Why Not Why? The Teaching of Grammar, Discourse, Sociolinguistic and Cross-Cultural Perspectives. Japanese Language and Literature. 39(2), 255-289.
Mori, J. and Markee, N. (2009). Language learning, cognition, and interactional practices: An introduction. International Review of Applied Linguistics in Language Teaching. 47(1), 1-9.
Mortensen, K. (2008). Selecting next-speaker in the second language classroom: How to find a willing next-speaker in planned activities. Journal of Applied Linguistics. 5(1), 55- 79.
Mortensen, K. (2009). Establishing recipiency in pre-beginning position in the second language classroom. Discourse Processes. 46(5), 491-515.
Mortensen, K and Hazel, S. (2011). Initiating round robins in the L2 classroom – preliminary observations . Novitas-ROYAL (Research on Youth and Language). 5(1), 55-70. Retrieved from http://www.novitasroyal.org/Vol_5_1/mortensen_hazel.pdf
Pappamihiel, N. E. (2002). English as a second language students and English language anxiety: Issues in the mainstream classroom. Research in the Teaching of English. 36(3), 327-355.
Pekarek Doehler, S. (2006). Compètence et langage en action. Bulletin Suisse de Linguistique Appliquèe. 84, 9-45.
Pekarek Doehler, S. (2010). Conceptual changes and methodological challenges: on language, learning and documenting learning in conversation analytic SLA research. In Seedhouse, P., Walsh, S. and Jenks, C. (eds.). Conceptualising Learning in Applied Linguistics. (pp. 105-126), Basingstoke: Palgrave MacMillan.
Pochon-Berger, E. (2011). A participant’s perspective on tasks: from task instruction, through pre-task planning, to task accomplishment. Novitas-ROYAL (Research on Youth and Language). 5(1), 71-90. Retrieved from http://www.novitasroyal.org/Vol_5_1/pochon-berger.pdf
Psathas, G. (1995). Conversation Analysis. Thousand Oaks, CA: Sage.
Ross, S. (2007). A Comparative Task-in-Interaction Analysis of OPI Backsliding. Journal of Pragmatics. 39, 2017-2044.
Sandlund, E. and Sundqvist, P. (2011). Managing task-related trouble in L2 oral proficiency tests: contrasting interaction data and rater assessment. Novitas-ROYAL (Research on Youth and Language). 5(1), 91-120. Retrieved from http://www.novitasroyal.org/Vol_5_1/sandlund_sundqvist.pdf
Saraç, H.S. (2007). Öğretmen adaylarının 8. Sınıf İngilizce ders kitabındaki diyalogların etkinliği konusunda inançları: toplumedim sorunları. Kastamonu Eğitim Dergisi (Kastamonu Journal of Education). 15(1), 401-410.
Schegloff, E. A. and Sacks, H. (1973). Opening up closings. Semiotica. 7, 289-327.
Schegloff, Emanuel A. (1992). Repair after next turn: The last structurally provided defense of intersubjectivity in conversation. American Journal of Sociology. 97(5), 1295-1345.
Schegloff, E.A., Koshik, I., Jacoby, S. and Olsher, D. (2002). Conversation Analysis and Applied Linguistics. Annual Review of Applied Linguistics. 22, 3-31.
Seedhouse, P. (2004). The interactional architecture of the language classroom: A conversation analysis perspective. Malden, MA: Blackwell.
Seedhouse, P. (2005). Conversation Analysis and language learning. Language Teaching. 38(4), 165-187.
Seedhouse, P. (2008). Learning to Talk the Talk: Conversation Analysis as a Tool for Induction of Trainee Teachers. Chapter in Garton, S. & Richards, K. (Eds.) Professional Encounters in TESOL (pp. 42-57) Basingstoke: Palgrave Macmillan.
Seedhouse, P. (2010). How research methodologies influence findings. Novitas-ROYAL (Research on Youth and Language). 4(1), 1-15. Retrieved from http://www.novitasroyal.org/Vol_4_1/seedhouse.pdf
Seedhouse, P. (2011). Conversation Analytic Research into Language Teaching and Learning. In Hinkel, E. (Ed.) The Handbook of Research in Second Language Teaching and Learning, Volume II (pp. 345-363). Routledge.
Seedhouse, P. and Egbert, M. (2006). The interactional organisation of the IELTS speaking test. IELTS Research Reports.6, 161-206.
Seedhouse, P. and Harris, A. (in preparation). Topic Development in the IELTS Speaking Test. IELTS Research Reports.
Seedhouse, P. and Walsh, S. (2010). Learning a second language through classroom interaction. In Seedhouse, P., Walsh, S. and Jenks, C. (eds.). Conceptualising Learning in Applied Linguistics. (pp. 127-146), Basingstoke: Palgrave MacMillan.
Sert, O. (2009). Developing Interactional Competence by Using TV Series in "English as an Additional Language" Classrooms. Enletawa Journal. 2, 23-50.
Sert, O. (2010). A Proposal for a CA-Integrated English Language Teacher Education Program in Turkey. Asian EFL Journal (Special Issue on English Language Teacher Education and Development: Issues and Perspectives in Asia, ed. Eva Bernat). 12(3), 62-97.
Sidnell, J. (2010). Conversation Analysis- an introduction. West Sussex, UK: Wiley- Blackwell.
Sinclair, J.M. and Coulthard, R.M. (1975). Towards an analysis of discourse: The English used by teachers and pupils. London: Oxford University Press.
Skidmore, D. and Murakami, K. (2010). How prosody marks shifts in footing in classroom discourse. International Journal of Educational Research. 49(2-3), 69-77.
Strong, M. and Baron, W. (2004). An analysis of mentoring conversations with beginning teachers: suggestions and responses. Teaching and Teacher Education. 20, 47-57. ten Have, P. (2007). Doing Conversation Analysis (2nd Edition). London: Sage.
Walsh, S. (2002). Construction or Obstruction: teacher talk and learner involvement in the EFL classroom. Language Teaching Research. 6(1), 3-23.
Walsh, S. (2003). Developing interactional awareness in the second language classroom. Language Awareness. 12, 124-42.
Walsh, S. (2006). Investigating Classroom Discourse. London: Routledge.
Walsh, S. and O’Keeffe, A. (2010). Investigating higher education seminar talk. Novitas-
ROYAL (Research on Youth and Language). 4(2), 141-158. Retrieved from
http://www.novitasroyal.org/Vol_4_2/walsh_okeeffe.pdf
Walsh, S. (2011). Exploring Classroom Discourse: Language in action. Oxon: Routledge.
Walters, F. S. (2007). A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence. Language Testing. 27(2), 155-183.
Walters, F. S. (2009). A Conversation Analysis-Informed Test of L2 Aural Pragmatic Comprehension. TESOL Quarterly. 43(1), 29-54.
Waring, H. Z. (2009). Moving out of IRF (Initiation-Response-Feedback): A single case analysis. Language Learning. 59(4), 796-824.
Wong, J. (2002). Applying Conversation Analysis in applied linguistics: Evaluating dialogue in English as a second language textbooks. International Review of Applied Linguistics in Language Teaching. 40, 37-60.
Wong, J. (2007). Answering my call: A look at telephone closings. In Bowles, H. & P. Seedhouse (Eds.), Conversation Analysis and language for specific purposes, (pp. 271-304), Bern: Peter Lang.
Wright, T. (2002). Doing language awareness: Issues for language study in language teacher education. In Trappes-Lomax, H. and Ferguson, G. (Eds.), Language in Language Teacher Education. (pp. 113-130). London: John Benjamin.
Young, R. F. (2008). Language and interaction: a resource book. Oxon: Routledge.
Young, R.F. and He, A. (eds.) (1998). Talking and testing: discourse approaches to the assessment of oral proficiency. Amsterdam: Benjamins.
Young, R. F. and Miller, E. R. (2004). Learning as changing participation: Discourse roles in ESL writing conferences. The Modern Language Journal. 88, 519–535.
Zemel, A. and Koschmann, T. (2011). Pursuing a question: reinitiating IRE sequences as a method of instruction. Journal of Pragmatics. 43(2), 475-488.