Does Modality Matter? On the Modality of Visuals in Computerized Tests of Listening and Its Implications

Abstract

Inspired by the controversial findings of studies suggesting that the modality or redundancy effect is produced when information presented to learners is available in multiple modalities, the present study examined whether the integration of context video or static images in multimedia tests of listening would privilege the participants or adversely impact their listening comprehension in English. To this aim, three study groups were assigned to listen to five aural passages on everyday themes under one of three conditions: an audio-only condition where only auditory information was available; a video-plus-audio condition where the participants listened to the passages while watching video clips of the interlocutors and the settings; and an image-plus-audio condition where both aural information and screenshots of the video counterparts of context visuals were available. The result revealed that both types of context visuals had proved fruitful in aiding the participants’ comprehension of the texts, and that while the modality effect was present, the redundancy effect was not produced. Yet, the study found a significant difference in the performance levels of the two experimental groups in favor of context video. Among the proposed explanations is the idea that higher levels of engagement, increased noticing, reduced cognitive load, and in-depth processing and decoding of aural input associated with the use of video modality may demonstrate a preference for the inclusion of video modality in computerized tests of listening.