Are we offering science students sufficient authentic assessments?
Bibliographic record
Abstract
Authentic assessments are commonly promoted to assess higher-order thinking, encourage deep learning, and strengthen ties between classroom content and real-world problems. However, while many institutions highlight a commitment to authentic assessment in strategic documents, a clear definition of what constitutes authenticity and objective measures of the patterns and prevalence of these assessments is lacking. In a multi-year project, we compiled an inventory of all assessments across a complete BSc curriculum at a large Canadian comprehensive university and documented their authenticity to better understand student assessment experiences and facilitate discussion between instructors. Based on Villarroel's (2018) core dimensions of authenticity: realism, cognitive challenge, and evaluative judgement, we developed a rubric-style tool to score individual assessments as low, moderate, or high on each dimension. The tool has been applied to over 1000 assessments from face-to-face and remote settings, uncovering patterns in authenticity by class size, year level and assessment type. The prevalence of authentic assessment in our BSc program was low (<2%), with evaluative judgement being the weakest dimension across contexts. Small, 4th year courses were more authentic than large, early-year core courses, and assignments were consistently more authentic than tests. Curriculum-level authenticity didn’t change from face-to-face to remote settings, although nearly equal number of courses improved authenticity as decreased authenticity. This work presents a tangible tool and process that can be used to critically review individual assessments or complete curriculums and offers a representative data set for comparison. We will share practical strategies participants can consider at course, curriculum, or institutional levels to promote authenticity with an open call for collaboration.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.045 | 0.211 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.004 | 0.005 |
| Scholarly communication | 0.012 | 0.011 |
| Open science | 0.002 | 0.009 |
| Research integrity | 0.004 | 0.004 |
| Insufficient payload (model declined to judge) | 0.009 | 0.006 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".