Can Large Language Models Aid Caregivers of Pediatric Cancer Patients in Information Seeking? A Cross-Sectional Investigation
Bibliographic record
Abstract
Abstract Background and Objectives In pediatric oncology, caregivers seek detailed, accurate, and understandable information about their child’s condition, treatment, and side effects. The primary aim of this study was to assess the performance of four publicly accessible large language model (LLM)- supported knowledge generation and search tools in providing valuable and reliable information to caregivers of children with cancer. Methods This cross-sectional study evaluated the performance of the four LLM-supported tools — ChatGPT (GPT-4), Google Bard (Gemini Pro), Microsoft Bing Chat, and Google SGE- against a set of frequently asked questions (FAQs) derived from the Children’s Oncology Group Family Handbook and expert input. Five pediatric oncology experts assessed the generated LLM responses using measures including Accuracy (3-point ordinal scale), Clarity (3-point ordinal scale), Inclusivity (3-point ordinal scale), Completeness (Dichotomous nominal scale), Clinical Utility (5-point Likert-scale), and Overall Rating (4-point ordinal scale). Additional Content Quality Criteria such as Readability (ordinal scale; 5- 18th grade of educated reading), Presence of AI Disclosure (Dichotomous scale), Source Credibility (3- point interval scale), Resource Matching (3-point ordinal scale), and Content Originality (ratio scale) were also evaluated. We used descriptive analysis including the mean, standard deviation, median, and interquartile range. We conducted Shapiro-Wilk test for normality, Levene’s test for homogeneity of variances, and Kruskal-Wallis H-Tests and Dunn’s post-hoc tests for pairwise comparisons. Results Through expert evaluation, ChatGPT showed high performance in accuracy (M=2.71, SD=0.235), clarity (M=2.73, SD=0.271), completeness (M=0.815, SD=0.203), Clinical Utility (M=3.81, SD=0.544), and Overall Rating (M=3.13, SD=0.419). Bard also performed well, especially in accuracy (M=2.56, SD=0.400) and clarity (M=2.54, SD=0.411), while Bing Chat (Accuracy M=2.33, SD=0.456; Clarity M=2.29, SD=0.424) and Google SGE (Accuracy M=2.08, SD=0.552; Clarity M=1.95, SD=0.541) had lower overall scores. The Presence of AI Disclosure was less frequent in ChatGPT (M=0.69, SD=0.46), which affected Clarity (M=2.73, SD=0.266), whereas Bard maintained a balance between AI Disclosure (M=0.92, SD=0.27) and Clarity (M=2.54, SD=0.403). Overall, we observed significant differences between LLM tools (p < .01). Conclusions LLM-supported tools potentially contribute to caregivers’ knowledge of pediatric oncology on related topics. Each model has unique strengths and areas for improvement, suggesting the need for careful selection and evaluation based on specific clinical contexts. Further research is needed to explore the application of these tools in other medical specialties and patient demographics to assess their broader applicability and long-term impacts, including the usability and feasibility of using LLM- supported tools with caregivers.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.038 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".