Course Evaluations ‐ Are We Gaining Insight, or Feeding Egos? Using Novel Evaluations to Uncover Value
Bibliographic record
Abstract
Assessment is generally the evaluation of student learning and competency. But, when the roles are reversed, and we ask students for their perspective, assessment has the potential to improve a course. Unfortunately in practice however, there are some limitations. In particular, asking students to rate their experience on ubiquitous Likert scale evaluations imposes a ceiling effect, which limits utility. So while top notch evaluations feel great, and often support tenure, promotion and hiring decisions, constructive change seldom results. Over the past 7 years the Education Program in Anatomy at McMaster University has focussed on understanding learner profiles in an interprofessional education (IPE)/anatomy dissection course using multiple evaluation strategies. By employing Likert‐scales, qualitative ratings and Q method, students were asked about what they valued in the course. Specifically through Q method's factor analysis, individuals were grouped based on similarities in their rankings of common preferences/viewpoints: three previously unknown independent learner groups (Anatomy IPE Enthusiasts, Practical IPE Advocates and Skeptical IPE Anatomists) were identified for which common challenges could be addressed. Conversely, qualitative and Likert‐based analyses provided less granular information which was not directly useful for course evolution. With the above study serving as an exemplary template, we will challenge value of current course evaluation practices, and further explore Q methodology as a viable and valuable alternative. In all, session attendees can expect to learn methods for better understanding the types of students in our classrooms which will support strategies for more refined student‐centred learning. This abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal .
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.032 | 0.133 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.002 | 0.008 |
| Scholarly communication | 0.015 | 0.021 |
| Open science | 0.001 | 0.005 |
| Research integrity | 0.001 | 0.003 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".