Evolving a conceptual framework and developing a new questionnaire for usability evaluation of blended learning programs in health professions education
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Background: Blended learning programs (BLPs) have been widely adopted across health professions education (HPE). To bolster their impact on learning outcomes, the usability of BLPs should be rigorously evaluated. However, there is a lack of reliable and validated tools to appraise this dimension of BLPs within HPE. The purpose of this investigation was to evolve a conceptual framework for usability evaluation in order to initially develop the Blended Learning Usability Evaluation – Questionnaire (BLUE-Q). Methods: After the completion of a scoping review, we conducted a qualitative descriptive study with seven purposefully selected international experts in usability and learning program evaluation. Individual interviews were conducted via videoconferencing, transcribed verbatim, and analyzed through thematic analysis. Results: Three themes were identified: (1) Consolidation of the multifaceted ISO definition of usability in BLPs within HPE; (2) Different facets of usability can assess different aspects of BLPs; (3) Quantitative and qualitative data are needed to assess the multifaceted nature of usability. The first theme adds nuance to a previously established HPE-focused usability framework, and introduces two new dimensions: ‘pedagogical usability’ and ‘learner motivation.’ The latter two provide guidance on structuring BLP evaluations within HPE. From this followed the development of the BLUE-Q, a new questionnaire that includes 55 Likert scale items and 6 open-ended questions. Conclusions: Usability is an important dimension of BLPs and must be examined to improve the quality of these interventions in HPE. As such, we developed a new questionnaire, solidly grounded in theory and the expertise of international scholars, currently under validation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.014 | 0.012 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it