Examining Methods to Assess Core Knowledge Competencies: A Canadian Perspective
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Click to increase image sizeClick to decrease image size Notes The first author is the JCB Fellowship Director, the second author is the Fellowship Evaluation Coordinator, and the remaining authors are rotation supervisors and/or past fellows. See HEC Forum 24(3) for a special issue exploring the professionalization of practicing healthcare ethicists in Canada. The ASBH Core Competencies, 2nd edition (ASBH 2011), is one of the key reference documents informing the JCB Fellowship program, along with the Canadian Bioethics Society Taskforce on Working Conditions for Bioethics’ Model Role Description for Ethicists (Chidwick et al. Citation2010), which focuses on the ideal role and eight areas of responsibility of a PHE. The JCB Fellowship program is also informed by the emerging work of PHEEP (Practicing Healthcare Ethicists Exploring Professionalization), which anticipates the development of Canadian health care ethics practice standards (see Reel Citation2012). We are considering an expanded set of indicators as well a matrix summary approach to identifying incrementally demonstrated knowledge and skill acquisition. The matrix of required knowledge and skill competencies (see, e.g., Bingham et al. Citation2005) would be completed by fellows over the course of the fellowship, and domains/elements would be signed off by rotation supervisors and/or the fellowship director, attesting to the fellow having reached the thresholds expected in each domain. Such a matrix approach emphasizes formative assessment and allows for clear focus on those areas not yet achieved in order to ensure they are addressed within the year of the program. This approach has the advantage of being able to accommodate the variation in knowledge and skills among incoming JCB Fellows with typically interdisciplinary backgrounds.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.005 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it