Cluster Analysis and Rankings of Canadian Universities: Misadventures with Rank-based Data and Implications for the Welfare of Students
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
We present a data-based perspective concerning the Maclean’s magazine (November 17, 2003) rankings of Canadian universities, including two cluster analyses and other nonparametric analyses. These data are similar to those in recent university ranking exercises conducted by other magazines, such as U.S. News. In many cases, the cluster procedure showed that universities actually resemble and relate to each other in a manner different from their formal classification and final rank ordering by Maclean’s. Several pitfalls in ranking procedures, related to unreliable relationships among specific indices underlying the final ranks, are outlined. Comparisons are made also with the most recent student satisfaction rankings for 47 Canadian universities, published in November, 2003, by the Toronto Globe and Mail. The latter rankings do not reliably reflect the general results of the Maclean’s data. In their present format, and although they have become increasingly publicized and promoted, it remains difficult for the Maclean’s data to be consistently or empirically useful to students. Ranking exercises have unintended, though increasingly predictable, consequences, which likely bear heavily upon the intellectual and personal well being of students.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it