Short answers to deep questions: supporting teachers in large‐class settings
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Abstract In large class settings, individualized student–teacher interaction is difficult. However, teaching interactions (e.g., formative feedback) are central to encouraging deep approaches to learning. While there has been progress in automatic short‐answer grading, analysing student responses to support formative feedback at scale is arguably some way from being widely applied in practice. However, analysing student written responses can provide insights into student conceptions, thus directly informing teacher actions. Indeed, we argue that analysing student responses to provide feedback directly to teachers is as worthy a goal as providing individualized feedback to students and is achievable given the current state‐of‐the‐art in natural language processing. In this paper, we analyse student written responses to short‐answer questions posed in the context of a large first year health sciences course. Each question was designed to elicit deep responses. Our qualitative analysis illustrates the variability in student responses and reveals multiple relationships between these responses, course materials and the questions posed. Such information can be invaluable for teacher praxis. We conclude with a conceptual ‘dashboard’ that categorizes student responses and reveals relationships between responses, course resources and the questions. Such a dashboard could provide timely, actionable insights for teachers and help foster deep learning approaches for students.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it