Artificial Intelligence Use in Academic Applicant Screening: A Systematic Review
Bibliographic record
Abstract
Background: Escalating application volumes challenge holistic admissions review; artificial intelligence (AI) offers potential screening efficiencies. This systematic review examined AI algorithm use in academic admissions, categorizing types and evaluating objectives, performance, and ethical considerations. Methods: A systematic search of 5 databases through October 2024 identified original research on AI screening in undergraduate to medical residency programs. Data extraction covered study characteristics, AI types, performance metrics, and ethical considerations. Bias risk was assessed using the Newcastle-Ottawa Scale, and objective outcomes were compared statistically. Results: Eighteen studies (61,327 applicants) were included. The primary uses of AI were interview selection (n = 7), admissions decisions (n = 4), and reviewer scoring (n = 2). Algorithms included traditional machine learning (TML) and natural language processing (NLP). In the subset of studies reporting area under the receiver operating characteristic metrics, TML models averaged 0.89 versus 0.77 for NLP models. Ethical concerns, notably bias, were reported in 50% of studies. Conclusions: TML algorithms excelled with structured data; NLP showed value for unstructured data despite challenges (eg, transparency and data needs). Ethical concerns highlight the need for transparency and bias mitigation in AI adoption. Findings emphasize continued research to address ethical and methodological challenges.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.023 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.007 | 0.001 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".