MétaCan
Menu
Back to cohort
Record W2616245153 · doi:10.1515/applirev-2017-0028

Actively managed products: Think-aloud data and methods in applied linguistics research

2017· article· en· W2616245153 on OpenAlex

Why this work is in the frame

A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.

affAt least one author lists a Canadian institution in the pinned OpenAlex snapshot.

Bibliographic record

VenueApplied Linguistics Review · 2017
Typearticle
Languageen
FieldArts and Humanities
TopicLanguage, Discourse, Communication Strategies
Canadian institutionsUniversity of British Columbia
Fundersnot available
KeywordsPsychologyVocabularyApplied linguisticsLinguisticsActive listeningReading (process)Cognitive linguisticsEncyclopediaCognitionCognitive scienceCommunication

Abstract

fetched live from OpenAlex

Abstract Verbal reports, specifically in the form of concurrent verbalizations (i.e., think-alouds [TAs]), have played a foundational role in the production of knowledge in applied linguistics. Most often drawn upon because the talk they generate is deemed to accurately reflect individual learners’ thought or cognitive processes as they complete an L2 task, concurrent verbalization methods have been central to investigations of and claims about the learning, use, and assessment of L2 vocabulary, listening, speaking, reading, and writing (among others). And although critical discussion concerning the quality of spoken data obtained through concurrent verbalization methods continues among L2 researchers (e.g., Cohen, Andrew D. 1987. Using verbal reports in research on language learning. In Claus Færch & Gabriele Kasper (eds.), Introspection in second language research , 82–95. Philadelphia: Multilingual Matters; Cohen, Andrew D. 1996. Verbal reports as a source of insights into second language learner strategies. Applied Language Learning 7(1–2). 5–24; Cohen, Andrew D. 2013. Verbal report. In Carol A. Chapelle (ed.), The encyclopedia of applied linguistics . Oxford: Wiley-Blackwell), the majority of this discussion has focused primarily on how best to generate talk which “more accurately reflect[s] the actual thought processes” of L2 users (Cohen, Andrew D. 2013. Verbal report. In Carol A. Chapelle (ed.), The encyclopedia of applied linguistics . Oxford: Wiley-Blackwell: 1). The result has been to further naturalize approaches to concurrent verbalizations which treat language as a neutral means for accessing cognition, and similarly, which treat the verbalizations themselves as individually accomplished events. In this article, my aim is to diversify the critical discussion by describing how discursive psychology (e.g., Edwards, Derek & Jonathan Potter. 1992. Discursive psychology . New York: Sage; Potter, Jonathan. 2006. Cognition and conversation. Discourse Studies 8(1). 131–140) and a conversation analytic perspective (e.g., Kasper, Gabriele. 2009. Locating cognition in second language interaction and learning: Inside the skull or in public view? International Review of Applied Linguistics 47. 11–36; Markee, Numa & Mi-Suk Seo. 2009. Learning talk analysis. International Review of Applied Linguistics 47. 37–63) can be combined to present an alternative to both ‘naturalized’, as well as sociocultural, understandings of concurrent verbalization data and methods. To this end, after establishing some of the key differences between information processing, sociocultural, and discursive approaches, I draw on data from two recently published TA-based studies in an attempt to accomplish two goals: the first is to shift critical discussion towards issues of epistemology, methodology, and research representation, and the second is to identify methodological issues about which researchers working from different conceptual orientations might engage in cross-paradigmatic dialogue.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.006
metaresearch head score (Gemma)0.023
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Meta-epidemiology (narrow), Science and technology studies
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Theoretical or conceptual · Consensus signal: none
GenreCandidate signal: Other · Consensus signal: none
Teacher disagreement score0.957
Threshold uncertainty score1.000

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0060.023
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0010.000
Bibliometrics0.0000.000
Science and technology studies0.0010.001
Scholarly communication0.0010.000
Open science0.0030.002
Research integrity0.0000.001
Insufficient payload (model declined to judge)0.0000.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.336
GPT teacher head0.501
Teacher spread0.165 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it