The Use of Web-Based Patient Reviews to Assess Medical Oncologists’ Competency: Mixed Methods Sequential Explanatory Study
Bibliographic record
Abstract
BACKGROUND: Patients increasingly use web-based evaluation tools to assess their physicians, health care teams, and overall medical experience. OBJECTIVE: This study aimed to evaluate the extent to which the standardized physician competencies of the CanMEDS Framework are present in web-based patient reviews (WPRs) and to identify patients' perception of important physician qualities in the context of quality cancer care. METHODS: The WPRs of all university-affiliated medical oncologists in midsized cities with medical schools in the province of Ontario (Canada) were collected. Two reviewers (1 communication studies researcher and 1 health care professional) independently assessed the WPRs according to the CanMEDS Framework and identified common themes. Comment scores were then evaluated to identify κ agreement rates between the reviewers, and a descriptive quantitative analysis of the cohort was completed. Following the quantitative analysis, an inductive thematic analysis was performed. RESULTS: This study identified 49 actively practicing university-affiliated medical oncologists in midsized urban areas in Ontario. A total of 473 WPRs reviewing these 49 physicians were identified. Among the CanMEDS competencies, those defining the roles of medical experts, communicators, and professionals were the most prevalent (303/473, 64%; 182/473, 38%; and 129/473, 27%, respectively). Common themes in WPRs include medical skill and knowledge, interpersonal skills, and answering questions (from the patient to the physician). Detailed WPRs tend to include the following elements: experience and connection; discussion and evaluation of the physician's knowledge, professionalism, interpersonal skills, and punctuality; in positive reviews, the expression of feelings of gratitude and a recommendation; and in negative reviews, discouragement from seeking the physician's care. Patients' perception of medical skills is less specific than their perception of interpersonal qualities, although medical skills are the most commented-on element of care in WPRs. Patients' perception of interpersonal skills (listening, compassion, and overall caring demeanor) and other experiential phenomena, such as feeling rushed during appointments, is often specific and detailed. Details about a physician's interpersonal skills or "bedside manner" are highly perceived, valued, and shareable in an WPR context. A small number of WPRs reflected a distinction between the value of medical skills and that of interpersonal skills. The authors of these WPRs claimed that for them, a physician's medical skills and competence are more important than their interpersonal skills. CONCLUSIONS: CanMEDS roles and competencies that are explicitly patient facing (ie, those directly experienced by patients in their interactions with physicians and through the care that physicians provide) are the most likely to be present and reported on in WPRs. The findings demonstrate the opportunity to learn from WPRs, not simply to discern physicians' popularity but to grasp what patients may expect from their physicians. In this context, WPRs can represent a method for the measurement and assessment of patient-facing physician competency.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.055 | 0.102 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".