MétaCan
Menu
Back to cohort
Record W2616245153 · doi:10.1515/applirev-2017-0028

Actively managed products: Think-aloud data and methods in applied linguistics research

2017· article· en· W2616245153 on OpenAlexaff
Ryan Deschambault

Bibliographic record

VenueApplied Linguistics Review · 2017
Typearticle
Languageen
FieldArts and Humanities
TopicLanguage, Discourse, Communication Strategies
Canadian institutionsUniversity of British Columbia
Fundersnot available
KeywordsPsychologyVocabularyApplied linguisticsLinguisticsActive listeningReading (process)Cognitive linguisticsEncyclopediaCognitionCognitive scienceCommunication

Abstract

fetched live from OpenAlex

Abstract Verbal reports, specifically in the form of concurrent verbalizations (i.e., think-alouds [TAs]), have played a foundational role in the production of knowledge in applied linguistics. Most often drawn upon because the talk they generate is deemed to accurately reflect individual learners’ thought or cognitive processes as they complete an L2 task, concurrent verbalization methods have been central to investigations of and claims about the learning, use, and assessment of L2 vocabulary, listening, speaking, reading, and writing (among others). And although critical discussion concerning the quality of spoken data obtained through concurrent verbalization methods continues among L2 researchers (e.g., Cohen, Andrew D. 1987. Using verbal reports in research on language learning. In Claus Færch & Gabriele Kasper (eds.),Introspection in second language research, 82–95. Philadelphia: Multilingual Matters; Cohen, Andrew D. 1996. Verbal reports as a source of insights into second language learner strategies.Applied Language Learning7(1–2). 5–24; Cohen, Andrew D. 2013. Verbal report. In Carol A. Chapelle (ed.),The encyclopedia of applied linguistics. Oxford: Wiley-Blackwell), the majority of this discussion has focused primarily on how best to generate talk which “more accurately reflect[s] the actual thought processes” of L2 users (Cohen, Andrew D. 2013. Verbal report. In Carol A. Chapelle (ed.),The encyclopedia of applied linguistics. Oxford: Wiley-Blackwell: 1). The result has been to further naturalize approaches to concurrent verbalizations which treat language as a neutral means for accessing cognition, and similarly, which treat the verbalizations themselves as individually accomplished events. In this article, my aim is to diversify the critical discussion by describing how discursive psychology (e.g., Edwards, Derek & Jonathan Potter. 1992.Discursive psychology. New York: Sage; Potter, Jonathan. 2006. Cognition and conversation.Discourse Studies8(1). 131–140) and a conversation analytic perspective (e.g., Kasper, Gabriele. 2009. Locating cognition in second language interaction and learning: Inside the skull or in public view?International Review of Applied Linguistics47. 11–36; Markee, Numa & Mi-Suk Seo. 2009. Learning talk analysis.International Review of Applied Linguistics47. 37–63) can be combined to present an alternative to both ‘naturalized’, as well as sociocultural, understandings of concurrent verbalization data and methods. To this end, after establishing some of the key differences between information processing, sociocultural, and discursive approaches, I draw on data from two recently published TA-based studies in an attempt to accomplish two goals: the first is to shift critical discussion towards issues of epistemology, methodology, and research representation, and the second is to identify methodological issues about which researchers working from different conceptual orientations might engage in cross-paradigmatic dialogue.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.139
metaresearch head score (Gemma)0.225
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesnone
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Qualitative · Consensus signal: none
GenreCandidate signal: Empirical · Consensus signal: none
Teacher disagreement score0.139
Threshold uncertainty score0.736

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.1390.225
Meta-epidemiology (narrow)0.0010.001
Meta-epidemiology (broad)0.0010.001
Bibliometrics0.0080.008
Science and technology studies0.0030.007
Scholarly communication0.0100.007
Open science0.0030.007
Research integrity0.0020.004
Insufficient payload (model declined to judge)0.0060.003

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.336
GPT teacher head0.501
Teacher spread0.165 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

The models applied no category: nothing in the taxonomy fit this work.
Study designQualitative
Domainnot available
GenreEmpirical

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations5
Published2017
Admission routes1
Has abstractyes

Explore more

Same venueApplied Linguistics ReviewSame topicLanguage, Discourse, Communication StrategiesFrench-language works237,207