Sequential Use of <scp>L1</scp> and <scp>L2</scp> Captions: Exploring the Benefits for Vocabulary Acquisition

There has been a proliferation in recent years of studies on second language (L2) vocabulary acquisition from watching audiovisual materials, such as TV programs and movies. A recurring theme in this line of inquiry has been the extent to which captioning facilitates vocabulary acquisition (Montero Perez, 2022; Teng, 2021). Studies have examined and compared the benefits of captions in the learners' first language (L1 captions, often referred to in the literature as subtitles) and of captions in the language that is used in the video's audio-track (L2 captions) (e.g., Peters, Heynen, & Puimège, 2016; Pujadas & Muñoz, 2019). It stands to reason that L1 captions are especially helpful for learners to comprehend the content of the video, while L2 captions are helpful to segment L2 speech into words and to match spoken words to their written representations. L1 and L2 captions thus complement one another, and the benefits of providing both simultaneously have recently been investigated as well (Wang & Pellicer-Sánchez, 2022). A concern with this simultaneous provision is that learners find it hard to allocate their attention to both sources of input. A scenario that, to our knowledge, has not been given consideration in the research literature is the sequential use of L1 and L2 captions, where learners first watch a video assisted by L1 captions and then watch it again with L2 captions. This brief report explores this scenario. A growing body of research demonstrates that vocabulary acquisition from audio-visual input is made more likely when on-screen text is available (e.g., Majuddin, Siyanova-Chanturia, & Boers, 2021; Montero Perez, Peters, Clarebout, & Desmet, 2014; Peters, 2019; Teng, 2022). Both L1 captions and L2 captions have been found to be beneficial in this regard (Peters et al., 2016). L1 captions help learners to comprehend the content of the audio-visual material (Baranowska, 2020). On the downside, learners may feel little need to process the L2 audio-recording because the L1 captions suffice to understand the content of the video. If they do try to attend also to the L2 speech, they may find it hard to connect the L1 words in the captions to specific lexical items in the L2 audio-track because of word-order differences between the two languages and because it is far from easy to discern word boundaries in connected speech. L2 captions, on the other hand, help learners to connect what they hear to how it is written in the target language, and so L2 captions foster a facet of word knowledge (written form) where L1 captions may fail (Bird & Williams, 2002). While L2 captions do not give direct access to the meaning of lexical items, they do help learners to segment the speech stream into distinct words, whose meaning they can then try to guess (Vanderplank, 1993). L2 captions are also to some degree helpful for content comprehension, especially for learners who are better at decoding written discourse than spoken discourse. Because L1 and L2 captions benefit vocabulary acquisition in complementary ways, their combined use is likely to be particularly beneficial. One way of doing this is to provide both types of captions together. A study by Wang and Pellicer-Sánchez (2022) has confirmed that this leads to greater vocabulary gains than providing no onscreen text. Moreover, it brought about better results in their study than watching only with L2 captions, at least at the level of acquiring the meaning of words. When it came to the acquisition of the written form of the L2 words, it was the learners who watched the video only with L2 captions who fared the best. It must be challenging for learners to take in two lines of captions (L1 and L2) as they watch a video in real time. Eye-movement data collected by Wang and Pellicer-Sánchez showed that the L1 captions attracted most attention, and this at the expense of attention that L2 captions tend to receive when only they are available. An alternative way of reaping the rewards of the combination of L1 and L2 captioning, explored in the present study, is to present the two types of captions sequentially. This entails that learners watch the same video at least twice. Several studies of vocabulary acquisition from audio-visual materials involved repeated viewing of the same video (e.g., Peters, 2019; Peters et al., 2016; Winke, Gass, & Syodorenko, 2010). It stands to reason that repeated viewing is beneficial because, when the content of the material has become familiar to the learners after a first viewing, it will be easier for them to attend to the language used to express the content. The benefits for vocabulary acquisition of watching a video with L2 captions twice instead of once were demonstrated by Majuddin et al. (2021). Despite repeated viewing, the meaning of lexical items may remain elusive to learners if insufficient contextual support is available. This is a crucial part of the rationale for a viewing sequence that starts with the support of L1 captions because this helps learners to establish meanings. Viewing the same material again but with the support of L2 captions will complement the learning of meaning as it helps to establish the connection between spoken and written forms of the L2 words. It also provides an incentive to recall the word meanings that were supplied through the L1 captions. The learners may subsequently view the material once more but without any captions, in which case they will need to attend more carefully to the audio-track and retrieve word meanings based on their spoken forms. The benefit of effortful retrieval for the consolidation of knowledge is well recognized (Karpicke, Lehman, & Aue, 2014; Roediger III & Butler, 2011), and this holds true also for L2 vocabulary learning (e.g., Barcroft, 2015; Candry, Decloedt, & Eyckmans, 2020). In the exploratory study reported below, vocabulary acquisition from this viewing sequence was compared to that of another sequence where learners also watched a video three times, but consistently with L2 captions. A 5-minute TED Talk video entitled A brie(f) History of Cheese was selected (Paul S. Kindstedt: A brie(f) history of cheese | TED Talk), consisting of 688 running words. Knowledge of the 4000 most frequent word families of English provides close to 95% lexical coverage of this text, according to Cobb's (n.d.) lexical profiler on https://www.lextutor.ca/vp/. This suggests that, at least as far as lexical demand is concerned, this TED Talk was suitable for the participants (e.g., van Zeeland & Schmitt, 2013). At the same time, there were likely words with which they were not yet familiar and that could thus serve as targets for learning. Eleven such words were selected (see Table 1). A meaning recall test was adopted as a pretest, a posttest, and a delayed posttest. The test presented the participants with 20 items of which 11 were the target words and 9 were fillers (higher-frequency words likely to be known). The participants were asked to write down the meaning of the words, either in English or in their L1. Half of the test prompts were presented in writing, and the other half were presented twice aurally (see Jelani & Boers, 2018, and Sydorenko, 2010, for a similar approach). Although the test was primarily meant to assess the participants' ability to supply the meaning of target words, the participants were also asked to write down the aurally presented words. This was done to examine whether learners who were exposed to the written forms repeatedly (in the L2–L2–L2 condition) reproduced these forms more accurately than those who saw the L2 captions only once during their viewing sequence. A content comprehension test was administered after the first viewing of the video. This was announced beforehand as an incentive for the participants to focus on the content of the video. The test consisted of six questions: two short-answer questions, three true/false questions, and one multiple-choice question. None of the target words were included in this comprehension test. No other tasks or tests were announced beforehand. After the delayed posttest, the participants were invited to complete an online anonymous questionnaire inquiring about their habitual ways of using L2 audiovisual materials outside the classroom. The questions of interest were whether they tend to view the same material more than once, and what type of textual support (if any) they prefer. The study was conducted over a 3-week period. In the first session, the participants gave their informed consent and then completed the vocabulary pretest. In the second session, the following week, the participants watched the TED Talk video three times in the treatment condition they had randomly been assigned to. The participants were informed that they would watch the same video three times and that there would be questions about the content after the first viewing, followed by two more viewings to give them the opportunity to refine or rectify their understanding. Shortly after the third viewing, they completed the (unannounced) vocabulary posttest. In the third week, the participants took the delayed posttest. After all the tests had been collected, the participants were given the meaning of the target words. They were then invited to complete the online questionnaire. Responses on the vocabulary meaning recall test were scored dichotomously (either 1 or 0). This was done twice: first using strict criteria where the proposed meaning needed to be exact, and then more leniently where credit was given when the response approximated the meaning of the word. Because the statistical trends were the same for both scoring systems, only the results from the strict scoring will be reported here for the sake of conciseness. The responses were assessed by two raters who were unaware of the treatment condition of the participants. Any cases of disagreement (mostly in the lenient scoring) were resolved after discussion. We also compared the two groups' performance on the content comprehension questions presented to them after the first viewing. This was considered useful to ascertain that L1 captions provided more assistance than L2 captions with content comprehension and to examine if better content comprehension after the first viewing was associated with vocabulary gains. Given the format of the comprehension questions (see above), all the responses were coded as either correct or incorrect (no partial credit given). Mixed-effects logistic regression was adopted to compare the vocabulary gains by the two participant groups, using the GAMLj module in jamovi (The Jamovi Project, 2021). There were 660 observations. The fixed effects were condition (the two viewing sequences), time (pre, immediate, delayed test), test modality (written versus spoken prompts), and content comprehension scores (covariates). We were primarily interested in an interaction effect between time (pre vs. posttests) and viewing condition, which would indicate greater vocabulary gains from one viewing condition than the other. The random factors were participants and target words. Alpha for the significance level was set at .05. According to the descriptive statistics (Table 2), the group of students assigned to the viewing sequence with L1 captions followed by L2 captions and finally no captions had slightly poorer prior knowledge of the set of target words than their peers in the comparison condition. However, their scores on the posttests were noticeably better than the comparison group's, showing gains between the pretest and the posttests that were twice as large. (Neither group showed attrition in the delayed posttest, with the L1–L2–none group even doing marginally better on the delayed test, but this may well reflect the benefit of re-taking the same test.) 2.09 (1.61) [1.80, 2.38] 4.91 (1.78) [4.59, 5.23] 5.18 (1.59) [4.90, 5.47] 2.56 (1.78) [2.20, 2.91] 3.89 (2.44) [3.41, 4.37] 3.89 (2.39) [3.42, 4.36] Recall that half of the words were presented aurally, and the participants were asked to write them down before supplying their meanings. Because the comparison condition repeatedly exposed the learners to the written form of the L2 words, it seemed likely that this condition would lead to a greater accuracy in the learners' production of these written forms. There was no evidence for such an effect, however. Both conditions brought about a score increase from median 3 in the pretest to median 4 in both the immediate and the delayed posttest. It needs to be acknowledged, however, that the number of these target words (n = 5) was very small, and their written form may have been relatively guessable from the phonological form. Part of the rationale for the L1 captions–L2 captions–no captions sequence was that the L1 captions available in the first viewing would support content comprehension, which would subsequently free up attentional resources to deal with unfamiliar vocabulary. It is therefore worth examining the learners' performance on the content-comprehension questions presented after the first viewing. As expected, the group of learners assisted by L1 captions obtained a better score (M = 4.64, SD = 1.21, Mdn = 5) than the group who watched the video with L2 captions (M = 3.56, SD = 1.81, Mdn = 3). The mixed effects logistic model (see Table 3) shows that time of vocabulary test was a significant predictor of test performance [χ2 (2) = 30.944, p < .001], but that the interaction between time of test and viewing condition fell short of significance despite the considerable odds ratios (OR = 2.33 and OR = 2.71) in favor of the L1–L2–none condition. In other words, the greater learning gain in the L1–L2–none condition compared to that in the L2–L2–L2 condition did not reach statistical significance. It is of course worth bearing in mind that this was an exploratory study, with small participant groups. However, the model did reveal that performance on the content comprehension questions was a significant predictor of performance on the vocabulary test [χ2 (1) = 5.646, p < .017]. The participants with higher scores on this comprehension test had a better chance of learning word meanings, with the odds of a correct response increasing by 31.93% as a participant's score on the comprehension test increased by 1 unit. It is worth recalling in this regard that comprehension tended to be better after viewing with L1 captions. The mixed effects logistic model detected no other significant fixed factors or significant interactions. The total amount of variance explained by the model was 47.6% (conditional R2 = .476), with 13.4% of the variance (marginal R2 = .134) attributed to the fixed effects and leaving 34.2% of the variance associated with participants and target items. The question might be asked how realistic it is for learners to engage repeatedly with the same L2 audiovisual material. Responses to the online questionnaire suggested that, at least for the learners who participated in this study, this is a realistic scenario. Eighteen participants responded to the questionnaire. Seventeen indicated that they habitually watch a video more than once for the purpose of understanding the content better and for improving their vocabulary knowledge. Fourteen respondents additionally indicated that they usually seek support from both L1 captions and L2 captions, although not necessarily in the sequence that was of interest here. For example, some learners wrote that they like challenging themselves by watching first without any captions and only then seeking assistance from captions. The results indicate that repeated viewing of the TED Talk promoted vocabulary acquisition and retention significantly at the level of meaning recall. We also found that the sequential use of L1 captions and L2 captions is a promising way of enhancing the vocabulary learning gains obtained from repeated viewing. While the difference fell short of statistical significance, the vocabulary knowledge gained in the L1 captions–L2 captions–no captions condition was twice as large as the gain observed for the 3 × L2 captions condition. This finding was in line with our hypothesis that better content comprehension assisted by L1 captions at the first viewing helped the participants to interpret vocabulary during the subsequent viewings. Logistic regression showed that content comprehension after the first viewing was a significant factor in word learning, and the scores on the comprehension test indeed suggested better comprehension of the TED Talk after watching it with L1 captions. Any conclusions drawn from this need to remain tentative, however, considering the small sample size, which is an obvious limitation of the study. Having no control condition and only one comparison condition are additional limitations. For example, it would be interesting to compare the sequential use and the simultaneous use of L1 captions and L2 captions that was tried by Wang and Pellicer-Sánchez (2022). It may also be worth exploring other viewing sequences, including ones mentioned in the responses to the questionnaire (for instance, viewing first without captions). This preliminary study was the first—to our knowledge—to explore the effect of the sequential use of L1 captions and L2 captions on L2 vocabulary learning through repeated viewing. The results are promising, although further research, with more participants, more target items, and tests of other facets and degrees of vocabulary knowledge in addition to meaning recall, is clearly necessary to confirm the effectiveness of the tried procedure. It is somewhat surprising that the sequential use of L1 and L2 captions has not yet attracted more attention from researchers because, according to our questionnaire, it is something that language learners appear to resort to spontaneously outside the language classroom when they engage with audiovisual materials that they find useful for L2 learning. While acknowledging that the findings are preliminary, we believe they may nonetheless prompt materials designers, teachers, and learners to consider more diverse and strategic ways of using audiovisual materials. This research project received no funding. Neither of the authors declares any conflict of interest. Ethics approval was obtained from the Research Ethics Board of the University of Western Ontario: REB# 116673. Injung Wi is a doctoral student in Applied Linguistics at Western University. She obtained an MA degree in TESOL and Applied Linguistics from the same university. She has 9 years of experience teaching English to EFL learners. Her research interests include L2 vocabulary and grammar learning through audiovisual input. Frank Boers is a professor in Applied Linguistics and TESOL at Western University. His early publications were in the field of lexicology, and focused on figurative language, but in recent years he has published mostly about second or foreign language education, often with a focus on vocabulary and phraseology.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.