P563 MSM predictive modeling within a large, linked database of laboratory, surveillance, and administrative healthcare records
Bibliographic record
Abstract
<h3>Background</h3> Enumeration or measurement of populations of men who have sex with men (MSM) is critical to developing and evaluating sexually transmitted and bloodborne infection (STBBI) prevention and treatment programs. However, there is a lack of data sources in which sexual orientation or behaviour is measured. In this study, we present the development and validation of a novel model (i.e., ‘computational phenotype’) to predict MSM status using multiple data sources. <h3>Methods</h3> Three disease case surveillance databases (HIV, hepatitis B and C, and syphilis), a public health laboratory database (which performs ≥95% of all HIV, hepatitis C and syphilis tests in British Columbia), and five administrative health record databases were linked and aggregated, resulting in a retrospective cohort of 727,091 adult men from 1990 to 2013. Self-reported MSM status (‘gold-standard’) from the three disease case surveillance databases was used to develop a multivariable prediction model for identifying MSM in the larger cohort. Models were selected using ‘elastic-net’ (combination of lasso and ridge regression), implemented through the GLMNet package in R, and a final model optimized area under the receiver operating characteristics curve (AUC). <h3>Results</h3> History of gonorrhea and syphilis diagnoses, HIV tests in the past year, history of visit to identified gay and bisexual men’s clinics, and residence in MSM-dense neighborhoods (based on self-reported MSM) were all positively associated with being MSM. The selected model had a sensitivity of 72%, specificity of 94%, and AUC of 92%. Combining self-reported MSM (n=6,280) and predicted MSM (n=85,521), a total of 91,801 men (13% of the cohort) were classified as MSM. <h3>Conclusion</h3> Applying a computational phenotyping method to administrative data yielded a cohort of 85,521 MSM, which may be used to monitor and evaluate health outcomes and healthcare utilization. Sensitivity and specificity of this model were comparable to interviewer-administered self-report measures of sexual orientation. <h3>Disclosure</h3> No significant relationships.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.009 | 0.014 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".