Use of <i>Z</i>-scores to Rank Applicants to Professional Degree Programs
Bibliographic record
Abstract
Criteria for assessing suitability of applicants for professional degree programs such as veterinary medicine are usually treated as distinct components of a composite scoring procedure that determines applicant ranking. Some components are valued more than others, which is reflected in the relative weights assigned to each component. However, the patterns of dispersal of individual components have the potential to alter the assigned relative weights. Components with larger variances can have greater influences on composite scores than intended. Such unintended altered weighting can be avoided through standardization. Yet non-standardized approaches continue to be used for admissions ranking in several programs. In this study, we documented the potential for differential selection of applicants when non-standardized scoring approaches are applied to admissions assessment components. At our medical school, applicants' component scores with differing variances are standardized by determining Z-scores with a mean of 0 and standard deviation of 1 before mathematically combining to calculate composite scores and admissions ranking. We retrospectively and hypothetically ranked one applicant cohort using non-standardized methods and identified differences in ranking between the standardized and non-standardized approaches. Most differences were observed for applicants in the second, third, and fourth quintiles of the admissions rank list, that is, those for whom admissions cut-off decisions make a marked difference. Observations were supported by lower Spearman's rank correlation coefficients in these quintiles. Although standardization of component scores is not a novel topic, we document the implications of using non-standardized scoring approaches for applicant ranking and underscore the importance of standardization of component scores.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.013 | 0.042 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.007 | 0.006 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".