Do authors of research funded by the Canadian Institutes of Health Research comply with its open access mandate?: A meta-epidemiologic study
Bibliographic record
Abstract
BACKGROUND: Since 2008, the Canadian Institutes of Health Research (CIHR) has mandated that studies it funds either in whole or in part are required to publish their results as open access (OA) within 12 months of publication using either online repositories and/or OA journals. Yet, there is evidence that authors are poorly compliant with this mandate. Specifically, there has been an apparent decrease in OA publication after 2015, which coincides with a change in the OA policy during the same year. One particular policy change that may have contributed to this decline was lifting the requirement that authors deposit their article in an OA repository immediately upon publication. We investigated the proportion of OA compliance of CIHR-funded studies in the period before and after the policy change of 2015 with manual confirmation of both CIHR funding and OA status. METHODS AND FINDINGS: We identified CIHR-funded studies published between the years 2014 to 2017 using a comprehensive search in the Web of Science (WoS). We took a stratified random sample from all four years (i.e. 2014 to 2017), with 250 studies from each year. Two authors independently reviewed the final full-text publications retrieved from the journal web page to determine to confirm CIHR funding, as indicated in the acknowledgements or elsewhere in the paper. For each study, we also collected bibliometric data that included citation count and Altmetric attention score Statistical analyses were conducted using two-tailed Fisher's exact test with relative risk (RR). Among the 851 receiving CIHR funding published from 2014 to 2017, the percentage of CIHR-funded studies published as OA significantly decreased from 79.6% in 2014 to 70.3% in 2017 (RR = 0.88, 95% CI: 0.79-0.99, P = 0.028). When considering all four years, there was no significant difference in the percentage of CIHR-funded studies published as OA in both 2014 and 2015 compared to both 2016 and 2017 (RR = 0.97, 95% CI: 0.90-1.05, P = 0.493). Additionally, OA publications had significantly higher citation count (both in year of publication and in total) and higher attention scores (P<0.05). CONCLUSIONS: Overall, we found that there was a significant decrease in the proportion of CIHR funded studies published as OA from 2014 compared to 2017, though this difference did not persist when comparing both 2014-2015 to 2016-2017. The primary limitation was the reliance of self-reported data from authors on CIHR funding status. We posit that this decrease may be attributable to CIHR's OA policy change in 2015. Further exploration is warranted to both validate these studies using a larger dataset and, if valid, investigate the effects of potential interventions to improve the OA compliance, such as use of a CIHR publication database, and reinstatement of a policy for authors to immediately submit their findings to OA repositories upon publication.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.076 | 0.205 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.007 | 0.017 |
| Bibliometrics | 0.016 | 0.034 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.006 | 0.004 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.003 | 0.002 |
| Insufficient payload (model declined to judge) | 0.004 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".