Comparing Learners’ General Proficiency Levels with Their Writing Productive Ability: How Correlated are They?

Since complexity, accuracy and fluency (CAF) have been suggested as measures of language development (Larsen-Freeman, 1978), a heated debate has surrounded the issue of whether they can sufficiently capture the multi-dimensional facets of language proficiency and be reliable indices of language proficiency levels. Contributing to this debate, the present study investigates the correlation between L3 English proficiency and CAF measures in writing. It recruits 88 semester-one 2nd year Baccalaureate students who were divided into nine groups of general proficiency using the 9-point Stanine scale based on their scores in a general English proficiency test. The scores for the proficiency test were obtained based on the holistic scoring method. We, then, assigned the participants a writing task which we evaluated in terms of CAF. We used the number of dependent clauses per T-unit to measure complexity, the ratio of error-free T-units to the total number of T-units to measure accuracy and number of words per minute to measure fluency. The results showed a strong correlation between proficiency levels and CAF measures, thereby proving that CAF measures serve as a framework suitable for benchmarking language proficiency development. In light of the results, some implications are made for future research targeting language development.

Keywords:

Language proficiency, CAF measures writing,

PDF

___

Ahmadian, M. J., & Tavakoli, M. (2011). The effects of simultaneous use of careful online planning and task repetition on accuracy, complexity, and fluency in EFL learners' oral production. Language Teaching Research, 35-59.
Bachman, L.F. (1990). Fundamental Considerations in Language Testing. Oxford: OUP.
Bachman, L.F. & Palmer, A.S. (1996). Language Testing in Practice: Designing and Developing Useful Language Tests. Oxford: OUP.
Bardovi-Harlig, K., & Bofman, T. (1989). Attainment of syntactic and morphological accuracy by advanced language learners. Studies in Second Language Acquisition, 11, 17-34.
Benevento, C., Storch, N. (2011). Investigating writing development in secondary school learners of French. Assessing Writing, 16(2), 97-110.
Bulté, B., & Housen, A. (2015). Evaluating short-term changes in L2 complexity development. Círculo de Lingüística Aplicada a la Comunicación, 63, 42-76.
Canale, M & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1, 1-47.
Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K. & James, M. (2005). Differences in writing discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10, 5-43.
Ellis, R. (2003). Task-based language learning and teaching. Oxford, UK: Oxford University Press.
Ellis, R., & Barkhuizen, G. (2005). Analysing learner language. Oxford: Oxford University Press.
Flahive, D. E., & Snow, B. G. (1980). Measures of syntactic complexity in evaluating ESL compositions. In J. W. Oller, & K. Perkins (Eds.), Research in language testing (pp. 171– 176). Rowley, MA: Newbury House.
Geisler, G. & Pardiwalla, M. (2010). Socialization patterns and boys’ underperformance in Seychellois schools. Journal Statistique Africain, 11, 62-84.
Godfrey, L., Treacy, C., & Tarone, E. (2014). Change in French second language writing in study abroad and domestic contexts. Foreign Language Annals, 47(1), 48-65.
Harley, B., Allen, P., Cummins, J., & Swain, M. (1990). The development of second language proficiency. Cambridge: Cambridge University Press
Henry, K. (1996). Early L2 writing development: A study of autobiographical essays by university-level students of Russian. The Modern Language Journal, 80, 309-26.
Hirano, K. (1991). The effect of audience on the efficacy of objective measures of EFL proficiency in Japanese university students. Annual Review of English Language Education in Japan, 2, 21-30.
Housen, A., & Kuiken, F. (2009). Complexity, accuracy, and fluency in second language acquistion. Applied Linguistics, 30(4), 461-473.
Hunt, K. W. (1965). Grammatical structures written at three grade levels. Champaign, IL: National Council of Teachers of English.
Hymes, D. (1972). On communicative competence. In J. B. Pride & J. Holmes (Eds.), Sociolinguistics: Selected readings (pp. 269-293). Hantiondsworth: Penguin Books.
Ishikawa, S. (1995). Objective measurement of low-proficiency EFL narrative writing. Journal of Second Language Writing, 4, 51–70.
Kennedy, C., & Thorp, D. (2002). A corpus-based investigation of linguistic responses to an IELTS academic writing task: University of Birmingham.
Kenney-Benson, G. A., Pomerantz, E. M., Ryan, A. M., & Patrick, H. (2006). Sex differences in math performance: The role of children’s approach to schoolwork. Developmental Psychology, 42, 11–26.
Larsen-Freeman, D. (1978). An ESL Index of development. TESOL Quarterly, 12, 439-48.
Larsen-Freeman, D. (1983). Assessing global second language proficiency. In H. W. Seliger & M. H. Long (Eds), Classroom Oriented Research in Second Language Acquisition (pp. 287- 304). Rowley, MA: Newbury House.
Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied Linguistics, 27(4), 590- 619.
Michel, M. C., Kuiken, F., & Vedder, I. (2007). The influence of complexity in monologic versus dialogic tasks in Dutch L2. IRAL, 241-259.
Myles, F. (2012). Complexity, accuracy, and fluency: The role played by formulaic sequences in early interlanguage development. In A. Housen, F. Kuiken & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 71-94). Amsterdam/Philadelphia: John Benjamins Publishing Company.
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555-578.
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24, 492–518.
Pennington, M. C., & So, S. (1993). Comparing writing process and product across two languages: A study of six Singaporean university student writers. Journal of Second Language Writing, 2, 41–63.
Robinson, P. (2001). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 22(1), 27–57.
Robinson, P. (2005). Cognitive complexity and task sequencing: A review of studies in a Componential Framework for second language task design. International Review of Applied Linguistics in Language Teaching, 43(1), 1–33.
Seo, S. J. (2009). Study on the interlanguage development of Korean language learners by syntactic proficiency assessment: Based on analyzing syntactic features of writings. (Unpublished master’s thesis). Yonsei University, Seoul, South Korea.
Seo, S. J., & Eo, J. H. (2011). Study on the accuracy variation of connective endings by Korean proficiency level. Journal of Korean Language Education, 22(1), 123–143.
Schmidt, R. (1992.) Psychological mechanisms underlying second language fluency. Studies in Second Language Acquisition, 14, 357-385.
Skehan, P. (1996). Second language acquisition research and task-based instruction. In J. Willis & D. Willis (Eds.), Challenge and change in language teaching (pp. 17–30). Oxford: Heinemann.
Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press. Skehan, P. (2009). Modeling second language development: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics 30, 510-532.
Skehan P. (2009b). Lexical performance by native and non-native speakers on language- learning tasks. In Richards B., Daller H., Malvern D.D., Meara P. (Eds.), Vocabulary studies in first and second language acquisition: The interface between theory and application (pp. 107–24). London: Palgrave Macmillan.
Skehan, P. & Foster, P. (1999). The influence of task structure and processing conditions on narrative retellings. Language Learning, 49, 93-120.
Skehan, P. & Foster, P. (2001). Cognition and tasks. In P. Robinson (Ed.), Cognition and Second Language Instruction (pp. 183-205). Cambridge: Cambridge University Press.
Verspoor, M. H., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21(3), 239-263.
Verspoor, M. H., & Smiskova, H. (2012). Foreign language writing development from a dynamic usage based perspective. In R. M. Manchón (Ed.), L2 writing development: Multiple perspectives (pp. 17–46). Boston/Berlin: Walter de Gruyter.
Widdowson, H. G. (1983). Learning Purpose and Language Use. Oxford: Oxford University Press.
Wolfe-Quintero, K., Inagaki, S., & Kim, H.Y. (1998). Second Language Development in Writing: Measures of Fluency, Accuracy & Complexity. Honolulu, HI: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa.
Wu, S.-L., & Ortega, L. (2013). Measuring global oral proficiency in SLA research: A new elicited imitation test of L2 Chinese. Foreign Language Annals, 46(4), 680-704.
Zyad, H., Rguibi, S., & Bouziane, A. (2016). The relationship between L2 writing quality and objective measures of linguistic and lexical complexity. Paper presented at the 4th International conference on “Cultures and Languages in Contact” organised by the Faculty of Letters and Humanities, Chouaib Doukkali University, El Jadida, 14-15 December, 2016.