Comparing the Lexical Features of EAP Students’ Essays by Prompt and Rating
Bibliographic record
Abstract
Previous research has shown that high frequency lexical items, such as AWL words and formulaic expressions, may differentiate between texts written by ex- pert and novice writers (Chen & Baker, 2010; Hancioğlu, 2009), and that lexical features related to breadth, depth, and accessibility differentiate among texts from L2 writers of different proficiency levels (Crossley & McNamara, 2009, 2012; Crossley, Weston, McLain Sullivan, & McNamara, 2011). The current study compared the essays written by EAP students in response to either a cause or an effect writing prompt. As part of their EAP writing class, the students (N = 94) had two weeks to read six source texts and take notes to prepare for an integrative-writing exam. Students’ essays were assessed by three raters using a holistic rubric, and five lexical features of their essays were analyzed: percentage of AWL word use, content word frequency, word familiarity, imagability, and lexical diversity. The results indicated that responses to the effect prompt were rated sig- nificantly lower than cause essays, contained more frequent and familiar words, and had a lower percentage of AWL words. However, there was no significant correlation between essay ratings and lexical features. Potential explanations for the findings and pedagogical implications are discussed. Des recherches antérieures ont révélé que les items lexicaux à haute fréquence, tels la liste des mots académiques et les formules rigides, peuvent varier selon que le texte soit écrit par un expert ou un débutant (Chen & Baker, 2010; Hancioğlu, 2009), et que les éléments lexicaux liés à l’envergure, la profondeur et l’accessi- bilité varient dans les textes écrits par des auteurs L2 de compétences différentes (Crossley & McNamara, 2009, 2012; Crossley, Weston, McLain Sullivan, & McNamara, 2011). La présente étude a comparé des rédactions écrites par des élèves d’anglais académique où la tâche d’écriture évoquait des causes ou des effets. Le cours d’anglais académique exigeait que les élèves (N=94) lisent, en deux semaines, six textes originaux et qu’ils prennent des notes pour se préparer à un examen écrit intégratif. Trois évaluateurs se sont appuyés sur une rubrique globale pour analyser cinq éléments lexicaux : pourcentage de l’utilisation des mots de la liste des mots académiques, fréquence des mots lexicaux, capacité à évoquer des images mentales et diversité lexicale. Les résultats démontrent que les évaluations des rédactions basées sur la tâche d’écriture évoquant des effets étaient nettement inférieures aux évaluations des rédactions basées sur la tâche d’écriture évoquant des causes. Ces premières comptaient également plus de mots familiers et répandus, et moins de mots figurant dans la liste des mots académiques. Les auteurs présentent des hypothèses pour expliquer ces résultats, ainsi que les impli- cations pédagogiques de leur recherche.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.029 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".