Cluster Analysis and Rankings of Canadian Universities: Misadventures with Rank-based Data and Implications for the Welfare of Students
Bibliographic record
Abstract
We present a data-based perspective concerning the Maclean’s magazine (November 17, 2003) rankings of Canadian universities, including two cluster analyses and other nonparametric analyses. These data are similar to those in recent university ranking exercises conducted by other magazines, such as U.S. News. In many cases, the cluster procedure showed that universities actually resemble and relate to each other in a manner different from their formal classification and final rank ordering by Maclean’s. Several pitfalls in ranking procedures, related to unreliable relationships among specific indices underlying the final ranks, are outlined. Comparisons are made also with the most recent student satisfaction rankings for 47 Canadian universities, published in November, 2003, by the Toronto Globe and Mail. The latter rankings do not reliably reflect the general results of the Maclean’s data. In their present format, and although they have become increasingly publicized and promoted, it remains difficult for the Maclean’s data to be consistently or empirically useful to students. Ranking exercises have unintended, though increasingly predictable, consequences, which likely bear heavily upon the intellectual and personal well being of students.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.055 | 0.213 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.012 | 0.029 |
| Science and technology studies | 0.013 | 0.015 |
| Scholarly communication | 0.010 | 0.004 |
| Open science | 0.003 | 0.004 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".