An Automated Feedback System Framework to Facilitate Instructor Self-Assessment
Bibliographic record
Abstract
Instructors in higher education face challenges getting feedback about their instructional artifacts. It could impact instructors' performance and students' learning outcomes. Exploring a better way to assist instructors to self-examine their work is important. This research investigated the creation of a framework to automatically generate individualized feedback for instructors regarding their instructional artifacts. It limited the investigation to use one instructional artifact, the course outline preparation, as the proof-of-concept experiment to demonstrate the potential of automatically providing instructors with feedback about communication, organizational, and planning instructional skills.Limited studies investigated instructors' skills. This research conducted a thorough literature review to identify the skills of successful instructors and the process used to evaluate those skills. It used a mixed approach looking for specific skills that appear more frequently and indicate clear impacts on success. The investigation concluded with a clear identification of the essential skills possessed by good instructors that can help to understand how to improve the quality of their instructions.Furthermore, this research conducted a single exploratory survey using closed-ended questions. The goal of the survey's questions is to gather opinions from participants in different departments and colleges from the University of Guelph. The results investigated the essential components in the course outlines and how they impact instructional skills. The outcomes from the survey demonstrated the data components used in data analysis. The data analysis outcomes identified the way to design the proposed framework, which can help university instructors. This research made three major contributions: a) the identification of essential skills for successful instructors; b) the necessary components to design the proposed Automated Feedback System for Instructors (AFSI) framework; and c) the exploration of data to demonstrate that feedback topics can be automatically determined. The objective of the AFSI feedback is not to judge the instructors' performance but to provide private and immediate feedback that can help instructors to adjust their work as the semester progresses.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.010 | 0.020 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.003 |
| Open science | 0.003 | 0.002 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.008 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".