Effectiveness of Clinician-selected Electronic Information Resources for Answering Primary Care Physicians' Information Needs
Bibliographic record
Abstract
OBJECTIVE: To determine if clinician-selected electronic information resources improve primary care physicians' abilities to answer simulated clinical questions. DESIGN: Observational study using hour-long interviews in physician offices and think-aloud protocols. PARTICIPANTS: answered 23 multiple-choice questions and chose 2 to obtain further information using their own information resources. We established which resources physicians chose, processes used, and results obtained when looking for information to support their answers. MEASUREMENTS: Correctness of answers before and after searching, resources used, and searching techniques. RESULTS: 23 physicians sought answers to 46 questions using their own information resources. They spent a mean of 13.0 (SD 5.5) minutes searching for information for the two questions using an average of 1.8 resources per question and a wide variety of searching techniques. On average 43.5% of the answers to the original 23 questions were correct. For the questions that were searched, 18 (39.1%) of the 46 answers were correct before searching. After searching, the number of correct answers was 19 (42.1%). This difference of 1 correct answer was attributed to 6 questions (13.0%) going from an incorrect to correct answer and 5 (10.9%) questions going from a correct to incorrect answer. We found differences in the ability of various resources to provide correct answers. CONCLUSION: For the primary care physicians studied, electronic information resources of choice did not always provide support for finding correct answers to simulated clinical questions and in some instances, individual resources may have contributed to an initially correct answer becoming incorrect.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.014 | 0.206 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".