Faculty preparation for academic evaluation
Bibliographic record
Abstract
At most universities in North America, the evaluation of a faculty member’s academic activities is generally performed by a committee consisting of senior academics and elected representatives. The Department of Medicine at the University of Alberta developed an academic promotions workshop for its faculty members, who include clinician-teachers/educators, clinician-investigators and basic scientists. A number of faculty members, both junior and senior, are poorly informed regarding the requirements for academic evaluation, the role of the Faculty Evaluation Committee (FEC), and how they should document their scholarly activities for promotion and tenure purposes. A half-day promotion workshop was developed and implemented. Workshop participants are provided with explicit information regarding the Faculty of Medicine and Dentistry’s evaluation promotion and tenure guidelines. Templates for both a ‘clinical dossier’ and an ‘administrative dossier’ demonstrate how academic scholarship in these domains might be presented. The workshop participants then assume the role of an FEC to review three fictitious faculty applications, of which two are assessed for tenure and promotion to associate professorship and one for promotion to full professorship. Of the tenure applicants, one is a clinician-teacher and the other is a basic scientist. The professorial candidate is a clinician-investigator. The cases presented are not model applications, but, rather, applications likely to evoke discussion. In addition to the workshop participants, the mock FEC includes senior faculty ‘plants’ who provide critical assessments of the documentation. Existing chairs and senior administrators play the roles of the presenting department. After discussing the applications, the mock FEC, using secret ballots, makes a recommendation for or against promotion. Since 2000 the Department of Medicine has offered this workshop once or twice yearly. Increasingly, requests from other departments have been accommodated and the workshop has evolved to a Faculty-wide endeavour. To date, 148 individuals from 19 departments have participated, with a 70 : 30 ratio of assistant : associate professors. Ratings have been uniformly positive, with all evaluating participants recommending the workshop. Comments from participants reflect the quality and relevance of the workshop: ‘…very useful to hear the “planted” criticisms…’ ‘…reinforced how critical documentation is … emphasised how scholarly work is viewed by others...’ ‘…should be [a] mandatory part of the introduction of new faculty…’ ‘…wish I had done this sooner…’ and ‘…I have a much better idea how to prepare junior divisional members. I would recommend they all go through this before their third year assessment.’ This workshop has proven to be useful to faculty members, divisional directors and departmental chairs. It has allowed participants to more objectively assess the scholarly contributions required for academic promotion and to understand how to present their contributions for assessment by others.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.096 | 0.247 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.003 | 0.002 |
| Bibliometrics | 0.011 | 0.005 |
| Science and technology studies | 0.008 | 0.003 |
| Scholarly communication | 0.011 | 0.005 |
| Open science | 0.006 | 0.012 |
| Research integrity | 0.003 | 0.007 |
| Insufficient payload (model declined to judge) | 0.212 | 0.096 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".