Actively managed products: Think-aloud data and methods in applied linguistics research
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Abstract Verbal reports, specifically in the form of concurrent verbalizations (i.e., think-alouds [TAs]), have played a foundational role in the production of knowledge in applied linguistics. Most often drawn upon because the talk they generate is deemed to accurately reflect individual learners’ thought or cognitive processes as they complete an L2 task, concurrent verbalization methods have been central to investigations of and claims about the learning, use, and assessment of L2 vocabulary, listening, speaking, reading, and writing (among others). And although critical discussion concerning the quality of spoken data obtained through concurrent verbalization methods continues among L2 researchers (e.g., Cohen, Andrew D. 1987. Using verbal reports in research on language learning. In Claus Færch & Gabriele Kasper (eds.), Introspection in second language research , 82–95. Philadelphia: Multilingual Matters; Cohen, Andrew D. 1996. Verbal reports as a source of insights into second language learner strategies. Applied Language Learning 7(1–2). 5–24; Cohen, Andrew D. 2013. Verbal report. In Carol A. Chapelle (ed.), The encyclopedia of applied linguistics . Oxford: Wiley-Blackwell), the majority of this discussion has focused primarily on how best to generate talk which “more accurately reflect[s] the actual thought processes” of L2 users (Cohen, Andrew D. 2013. Verbal report. In Carol A. Chapelle (ed.), The encyclopedia of applied linguistics . Oxford: Wiley-Blackwell: 1). The result has been to further naturalize approaches to concurrent verbalizations which treat language as a neutral means for accessing cognition, and similarly, which treat the verbalizations themselves as individually accomplished events. In this article, my aim is to diversify the critical discussion by describing how discursive psychology (e.g., Edwards, Derek & Jonathan Potter. 1992. Discursive psychology . New York: Sage; Potter, Jonathan. 2006. Cognition and conversation. Discourse Studies 8(1). 131–140) and a conversation analytic perspective (e.g., Kasper, Gabriele. 2009. Locating cognition in second language interaction and learning: Inside the skull or in public view? International Review of Applied Linguistics 47. 11–36; Markee, Numa & Mi-Suk Seo. 2009. Learning talk analysis. International Review of Applied Linguistics 47. 37–63) can be combined to present an alternative to both ‘naturalized’, as well as sociocultural, understandings of concurrent verbalization data and methods. To this end, after establishing some of the key differences between information processing, sociocultural, and discursive approaches, I draw on data from two recently published TA-based studies in an attempt to accomplish two goals: the first is to shift critical discussion towards issues of epistemology, methodology, and research representation, and the second is to identify methodological issues about which researchers working from different conceptual orientations might engage in cross-paradigmatic dialogue.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.023 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.003 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it