(096) REAL WORLD UTILITY OF CHATGPT IN PRE-VASECTOMY COUNSELLING IN AN OFFICE-BASED SETTING: A PILOT STUDY
Bibliographic record
Abstract
Abstract Introduction There is increasing interest among the medical community on to developing new ways to utilize ChatGPT. New applications have included writing scientific manuscripts, drafting correspondents, as well as testing its ability on standardized medical licensing examinations. Despite ChatGPT demonstrating good medical knowledge, it is still unclear if this translates to real world clinical practice. A potential application of this technology is to provide patient counseling and to improve patient flow in one’s practice. Objective Our goal is to assess if pre-vasectomy counseling with ChatGPT safely streamlines the consultation process by reducing visit times and increasing patient satisfaction through the consultation process. Methods A single institutional prospective, randomized single-blinded pilot study was conducted to evaluate the safety and efficacy for the use of ChatGPT (v4.0) for pre-vasectomy counselling. All eligible patients who are interested in undergoing vasectomy were included in the study. Patients were excluded if unwilling to participate in the study or did not have a home computer/internet connection to interact with the ChatGPT software. Patient were randomized 1:1 to pre-appointment counselling with ChatGPT + standard in-person consultation to in-person consultation alone. Baseline demographic information was collected including age, level of education. Outcomes collected at that time were length of visit, number of questions, as well as a Likert scale (/10) questionnaire assessing participant satisfaction in the ChatGPT counselling. Chat logs were reviewed by 2 men’s health specialists to assess the accuracy and safety of ChatGPT responses. Descriptive statistics were performed as well as comparative analysis with the independent sample t-test method. Results Total of 18 patients were analyzed with a mean age of 35.8 (±) 5.4 (n = 9) in the intervention arm and 36.9 (±) 7.4 (n = 9) in the control arm. Overall pre-vasectomy counselling with ChatGPT was associated with provider perceived improved understanding of the procedures (8.8 ± 1.0 vs 6.7 ± 2.8; p = 0.047) and decreased length of in-person consultation (7.7 ± 2.3 min vs 10.6 ± 3.4 min; p = 0.05). Patients however reported no difference in how they rated the quality of their clinical encounter, with excellent scores seen in all domains. Patient experiences with ChatGPT in respect to pre-vasectomy counseling was also surveyed. Quality of information provided by ChatGPT, ease of access/use, and overall experience were rated highly at 8.3 (±) 1.9, 9.1 (±) 1.5, and 8.6 (±) 1.7 respectively. Some participants did express concern regarding the confidentiality of information at 4.9 (±) 2.9. Of the 19 total questions and responses provided by ChatGPT, 95% (n = 18/19) were considered to be accurate responses. Conclusions ChatGPT for pre-vasectomy counselling did show to improve the efficiency of encounters and provider perceived patient understanding of the procedure. ChatGPT was also able to provide accurate responses to questions regarding the vasectomy in 95% of cases. Disclosure Any of the authors act as a consultant, employee or shareholder of an industry for: Boston Scientific.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".