Assessing Performance and Engagement on a Computer-Based Education Platform for Pharmacy Practice
Bibliographic record
Abstract
A computer-based education platform was developed using a theory-based approach to help Canadian pharmacy professionals adopt their full scope of practice. Data from the platform were used to identify factors that impacted user performance and engagement. A de-identified dataset included response data for 21 unique modules, including quiz responses and self-reflection questions. Outcome measures included user performance (mean quiz score) and engagement (completion rate for attempted modules). Analysis of variance (ANOVA), multivariate regression modelling, and machine learning cluster analysis were used to analyze the data. Of the 5290 users, 68% were pharmacists, 11% were technicians, 13% were pharmacy students, and 8% were pharmacy technician students. Four clusters were identified separately for pharmacists and technicians. Clusters with the higher performance and engagement tended to have more users practicing in community pharmacies while the lower performing clusters tended have more internationally trained users. In the regression modelling, pharmacists performed better than technicians and students while students were more engaged (p < 0.0001). Further, internationally trained pharmacists had slightly lower scores but similar engagement compared to domestically trained pharmacists (p < 0.0001). Users demonstrated higher performance on modules related to scope of practice than on clinical topics, and were most engaged with topics directly impacting daily practice such as influenza vaccinations and new and emerging subjects such as cannabis. The cluster analysis suggests that performance and engagement with a computer-based educational platform in pharmacy may be more related to place of practice than to personal demographic factors such as age or gender.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".