Developing expert consensus for specialist e-consult response: A delphi study to inform graduate medical education
Bibliographic record
Abstract
Abstract Introduction To meet the demands of a growing number of specialty referrals, outpatient electronic consultations (e-consults) have emerged as a rapid access strategy for specialist consultation, yet no framework currently exists to standardize specialist response to e-consults. With growing implementation of e-consult platforms throughout North America, it is essential to establish criteria for high-quality correspondence between specialists and primary care providers (PCP) and develop a formal e-consult curriculum to integrate into Accreditation Council for Graduate Medical Education (ACGME) subspecialty training programs. Methods We used a systematic, consensus-building methodology (modified Delphi) to develop expert recommendations for key elements of specialist response to e-consults. An expert group consisting of 14 clinicians (13 physicians, 1 physician assistant) from across the U.S. and Canada was selected to form the Delphi panel. Panelists were purposefully chosen to balance clinical expertise, practice location, and setting. Selection criteria included recognition as an expert in e-consults based on academic and/or clinical contributions. Panelists participated in an initial synchronous meeting to review and discuss a set of objectives identified through comprehensive literature review using MEDLINE/PubMed, followed by two rounds of anonymous and iterative voting. Consensus was determined a priori as ≥ 80% of panelists agreeing that an objective was essential. All objectives that achieved consensus were mapped to ACGME core competencies, including Patient Care (PC), Medical Knowledge (MK), Professionalism (P), Interpersonal and Communication Skills (ICS), Practice-Based Learning and Improvement (PBLI), and Systems-Based Practice (SBP). Results Two PCP and 12 specialty providers (7 hematology, 1 infectious diseases, 2 endocrinology, 1 gastroenterology, 1 cardiology) representing a range of geographic regions in the U.S. and Canada (7 East, 1 Central, and 6 West) and practice settings (12 academic, 1 private, and 1 integrated healthcare) were included in the Delphi panel. Three panelists serve in leadership roles for their e-consult program and 6 panelists have five or more e-consult peer-reviewed publications. After two survey rounds, 8 essential objectives of specialist e-consult responses were identified and achieved consensus (≥ 80%), including:Briefly summarize patient-specific descriptives and pertinent workup (e.g., key labs, imaging studies, procedures, etc.), specifying the time period of data reviewed and any pertinent missing data (PC).Review the differential diagnosis and suspected etiology, if pertinent (MK).Communicate specific recommendations (e.g., additional tests, monitoring, and/or treatment, including duration and administration), and explicitly state what the specialist will order or arrange if applicable (PC, ICS).Include a brief rationale for recommendations applied to the clinical scenario to improve educational value and encourage guideline adherence (e.g., cite guidelines, relevant data, etc.) (PBLI).Provide a clear contingency plan based on expected results (e.g., if results are positive/negative, proceed with treatment X) and document when a face-to-face referral or recontacting the specialist would be indicated/necessary (SBP, ICS).Communicate in a professional and supportive tone, acknowledging the referring provider's efforts and recognizing that the patient may review this communication (P, ICS).Delineate if/how the referring provider can communicate with the specialist, especially to ask an additional question or provide clarity about a patient (e.g., in basket message, chat function, repeat e-consult, etc.) (SBP, ICS).Understand the local context, including test and treatment availability and ordering, timeliness of e-consult completion, and clear role-delineation between referring and specialty providers (SBP). Conclusions An expert panel of PCP and medical specialists established consensus on a set of key components of effective specialist e-consult correspondence. These objectives align with ACGME core competencies and should inform future medical education curricula aimed to build competence in providing e-consult services.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.263 | 0.262 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.009 | 0.005 |
| Science and technology studies | 0.006 | 0.005 |
| Scholarly communication | 0.004 | 0.005 |
| Open science | 0.004 | 0.015 |
| Research integrity | 0.004 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".