Implementation Evaluation of a Critical Online Resource Evaluation (CORE) Program for High School Students in Quebec
Bibliographic record
Abstract
Being literate in the 21st century goes beyond the abilities of reading and writing; new skills and methods are required to evaluate the credibility of the information we find online. It is frequently assumed that young students, the so-called 'digital born generation,’ can discern credible information from mis/dis information or biased information; however, evidence suggests they have limited critical evaluation skills. Aware of this gap, researchers from the University of Concordia and practitioners in three high schools in Quebec implemented a Critical Online Resource Evaluation (CORE) intervention program to improve students' ability to evaluate online resources. With the vision to scale up the intervention in Canada and worldwide, it was essential to understand how the program was implemented and whether it was delivered as intended by the research team, to make informed decisions about modifications, adaptations, or refinements to improve the program's effectiveness. \nImplementation fidelity was evaluated using a pragmatic mixed methods design, concurrently collecting and analyzing quantitative and qualitative data from three sources: teachers, researchers and students. The CORE intervention was implemented with high fidelity from both teacher and researcher perspectives (79.9%, 89.0% ). While teachers adhered to the program content as planned, they adapted it to meet their classroom context. Teachers conducted the program with a high level of quality, with a mean score of 90.3% from the teacher’s perspective and 96.7% from the researcher’s perspective, suggesting they were well-prepared and confident. Students' engagement and motivation varied from the different views. From the teacher's perspective, students were highly engaged, with a mean score of 80.9%, while researchers and students perceived they were moderately engaged and motivated (77%, 69.6%). Teachers, researchers, and students concurred that the program was long, complex and the COVID topic fatiguing. Finally, the intervention highlighted group work and interactivity as the most engaging and motivating factors.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".