Predicting which cancer patients will see a psychiatrist or counsellor from their initial oncology consultation document using natural language processing.
Bibliographic record
Abstract
e13544 Background: Patients with cancer often have unmet psychosocial needs. Early detection of who might require referral to a counsellor or psychiatrist may help improve their care. In this work, we investigated whether natural language processing (NLP) can predict which patients will see a counsellor or psychiatrist from a patient’s initial oncologist consultation document. NLP is the branch of artificial intelligence that uses language models to accomplish tasks using written text. A different application of NLP has recently been popularized through the ChatGPT online question-answering system. In this work, we used traditional and neural language models to predict whether patients will see a psychiatrist or counsellor based on their initial oncologist consultation document. Methods: This retrospective prognostic study used data from 47,625 of 59,800 patients who received initial cancer care at any of the 6 BC Cancer sites located in the province of British Columbia between April 1, 2011 and December 31, 2016. We excluded patients with multiple cancer diagnoses, or those that did not see a medical or radiation oncologist within 180 days of their cancer diagnosis. We trained neural language models to predict which patients would see a psychiatrist or counsellor in the twelve months following their initial oncologist consultation. We used the non-neural language model bag-of-words, as well as three neural models: convolutional neural networks (CNN), long-short term memory, and bidirectional encoder representations from transformers. Results: The models performed similar to or better than previous applications of artificial intelligence in predicting psychosocial needs. Our best-performing models, utilizing CNNs, achieved a balanced accuracy above 75% and a receiver-operator area-under-curve (AUC) above 0.80 when predicting which patients would see a psychiatrist. Performance was lower for predicting which patients would see a counsellor, with balanced accuracy and AUC exceeding 70% and 0.75, respectively. We examined what words or phrases our models used to make the predictions, finding symptom burden, certain cancers, and family history of cancer were positive predictors of future referral to psychosocial services. Conclusions: Our results suggest NLP can be used to predict which cancer patients require referral to a psychiatrist or counsellor from their initial oncologist consultation. Future research may be able to extend our work to predict other needs of cancer patients such as palliative care, or other cancer outcomes more generally.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.007 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".