Putting the ghost in the machine: exploring human-machine hybrid virtual patient systems for health professional education
Bibliographic record
Abstract
Background: Virtual patient authoring tools provide a simple means of creating rich and complex online cases for health professional students to explore. However, the responses available to the learner are usually predefined, which limits the utility of virtual patients, both in terms of replayability and adaptability. Using artificial intelligence or natural language processing is expensive and hard to design. This project description lays out an alternative approach to making virtual patients more adaptable and interactive. Methods: Using OpenLabyrinth, an open-source educational research platform, we modified the interface and functionality to provide a human-computer hybrid interface, where a human facilitator can interact with learners from within the online case scenario. Using a design-based research approach, we have iteratively improved our case designs, workflows and scripts and interface designs. The next step is to robustly test this new functionality in action. This report describes the piloting and background as well as the rationale, objectives, software development implications, learning designs, and educational intervention designs for the planned study. Results: The costs and time required to modify the software were much lower than anticipated. Facilitators have been able to handle text input from multiple concurrent learners. Learners were not discouraged waiting for the facilitator to respond. Discussion: The implementation and use of this new technique seems very promising and there are a great many ways in which it might be used for training and assessment purposes. This report also explores the provisional implications arising from the study so far.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.008 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.001 | 0.004 |
| Scholarly communication | 0.005 | 0.004 |
| Open science | 0.002 | 0.005 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".