Using Videos in ESL Listening Achievement Tests: Effects on Difficulty

The question as to whether an assessment construct of second language (L2) listening comprehension should include the decoding of visual information remains unanswered (see Buck, 2001; Ockey, 2007). This study aimed to fill this gap by investigating how audio-only and video-enhanced delivery formats of listening passages compared in terms of difficulty for English as a second language (ESL) students. It utilized students’ performance on listening achievement tests developed by the researcher. The participants were 60 low- and high-proficiency ESL students enrolled in an American intensive English program. The participants’ scores on the achievement tests were used to compare the difficulty of items of different formats and determine whether this difficulty related to video type (context versus content) and students’ proficiency level. The findings suggested that, at least for higher-level students, listening testlets enhanced with videos containing mostly content-related visuals were significantly easier than their audio-only counterparts were. On the contrary, the inclusion of videos with mostly context visuals did not affect the difficulty of testlets in any proficiency category. The findings are discussed in terms of their practical significance for ESL teachers as well as theoretical implications for the field of ESL listening assessment.


Baltova, I. (1994). The impact of video on the comprehension skills of core French students. Canadian Modern Language Review, 50, 507–531.

Batty, A. O. (2015). A comparison of video- and audio-mediated listening tests with many-facet Rasch modeling and differential distractor functioning. Language Testing, 32, 3-20.

Bejar, I., Douglas, D., Jamieson, J., Nissan, S., & Turner, J. (2000). TOEFL 2000 listening framework: A working paper (TOEFL Monograph Series Report No. 19). Retrieved from

Buck, G. (2001). Assessing listening. Cambridge, UK: Cambridge University Press.

Chan, C., Wang, L., & Xu, L. (2014). A study of video effect of English listening comprehension. Studies in Literature and Language, 8, 53-58.

Copyright Law of the United States and Related Laws Contained in title 17 of the United States Code. (2011). Retrieved from

Cubilo, J., & Winke, P. (2013). Redefining the L2 listening construct within an integrated writing task: Considering the impacts of visual-cue interpretation and note-taking. Language Assessment Quarterly, 10, 371–397.

Ginther, A. (2002). Context and content visuals and performance on listening comprehension stimuli. Language Testing, 19, 133–167.

Gruba, P. (1993). A comparison study of audio and video in language testing. JALT Journal, 15, 85–88.

Gruba, P. (2004). Understanding digitized second language videotext. Computer Assisted Language Learning, 17, 51-82.

Kintsch, W. (1998). Comprehension. Cambridge: Cambridge University Press.

Lado, R. (1961). Language Testing: The construction and use of foreign language tests. London: Longman.

Latifi, M., Tavakoli, M., & A’lipour, J. (2013). Investigating the effect of video materials on testing foreign language learners’ listening performance. Middle-East Journal of Scientific Research, 13, 1197-1201. Retrieved from

Londe, Z. C. (2009). The effects of video media in English as a second language listening comprehension test. Issues in Applied Linguistics, 17, 41-50. Retrieved from

Mendelsohn, D. J. (1994). Learning to listen: A strategy-based approach for the second language learner. SanDiego: Dominie Press.

Ockey, G. J. (2007). Construct implications of including still image or video in computer-based listening tests. Language Testing, 24, 517–537.

Shin, D. (1998). Using videotaped lectures for testing academic listening proficiency. International Journal of Listening, 12, 57–80.

Stallings, W. M. (1972). A Comparison of television and audio presentations of the MLA French Listening Examination. The Journal of Educational Research, 65, 472-474.

Sueyoshi, A., & Hardison, D. M. (2005). The role of gestures and facial cues in second language listening comprehension. Language Learning, 55, 661-699.

Suvorov, R. (2009). Context visuals in L2 listening tests: The effects of photographs and video vs. audio-only format. In C. A. Chapelle, H. G. Jun, & I. Katz (Eds.), Developing and evaluating language learning materials (pp. 53-68). Ames, IA: Iowa State University.

Suvorov, R. (2015a). The use of eye tracking in research on video-based second language (L2) listening assessment: A comparison of context videos and content videos. Language Testing, 21, 1-21.

Suvorov, R. (2015b). Interacting with visuals in L2 listening tests: An eye-tracking study. In V. Berry (Ed.), ARAGs research reports online (Report #AR-A/2015/1). Retrieved from sts_suvorov.pdf

Underwood, M. (1989). Teaching listening. New York: Longman.

Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language Learning & Technology, 11, 67–86. Retrieved from

Wagner, E. (2010a). Test-takers’ interaction with an L2 video listening test. System, 38, 280- 291.

Wagner, E. (2010b). The effect of the use of video texts on ESL listening test-taker performance. Language Testing, 27, 493-513.

Wagner, E. (2013). An investigation of how the channel of input and access to test questions affect L2 listening test performance. Language Assessment Quarterly, 10, 178-195.

Witkin, B. R., & Trochim, W. W. K. (1997). Toward a synthesis of listening constructs: A concept map analysis. International Journal of Listening, 11, 69-87.

Wolvin, A., & Coakley, C. G. (1996). Listening (5th ed.). Dubuque, IA: Brown & Benchmark.

Kaynak Göster