A Systematic Evaluation of Payback of Publicly Funded Health Research in Hong Kong Since 1993
Bibliographic record
Abstract
Rationale: There is a need to demonstrate accountability of public funding for health research. Objectives: To quantify the outcomes of completed research projects supported by the Health and Health Services Research Fund (HHSRF), and to explore factors associated with the impact of research outcomes on health policy and provider behaviour Methods: Based on the widely adopted payback evaluation framework, we sent questionnaires to the principal investigators of all projects completed (n=205/235; 87%, total funding USD9.4 million; since the fund inception in 1993. The questionnaire gathered information in 6 outcome areas: a) knowledge production, b) use of research in the research system, c) use of research project findings in health system policy/decision making d) application of the research findings through changed behaviour, e) factors influencing the utilisation of research and f) health /health service/ economic benefits. The number of publications, promotions and post-graduate qualifications attained, policies influenced, and behaviours changed were counted. Logistic regression was used to identify factors associated with the uptake of research to inform policy, or that led to behavioural change and health service benefit. To account for over dispersion in the data, a negative binomial regression was used to estimate the impact of the funding award, project duration, years from project completion and administering institution on publication of peer-reviewed papers. Results: Of the 205 questionnaires sent, 178 (86.7%) were completed and returned for analysis. Among projects with completed questionnaires, 86% resulted in research publications (mean number of publication = 5.4). Career advancement, acquisition of higher qualifications and subsequent research were attributed to 50%, 38% and 34% of the projects, respectively. 35% of projects reported use of findings in policy making and product development, 49% in changing the behaviour of health service providers or general public, 42% in producing health service benefits. These outcomes compared favourably with those of research funds of similar nature in the UK and Canada. The number of peer-reviewed papers published per project was positively associated with the amount of funding award but not with the project duration or the type of research institution. Liaison with potential users of the research and participation in health related policy/advisory committees were significantly associated with reporting of health services benefit (ORparticipation=2.9, 95% CI 1.3-6.4; ORliaison=2.0, 95% CI 1.0-4.0), impact on policy and decision-making (ORparticipation=10.7, 95% CI 4.2-27.5; ORliaison=2.5, 95% CI 1.2-5.3), and on change in behaviour (ORparticipation=3.7, 95% CI 1.5-8.8). A significantly higher number of peer-reviewed papers published were associated with the funding award but not with the project duration, time from project completion or administering institution. Conclusions: Outcomes of the HHSRF compared favourably with those of overseas research funds of similar nature, demonstrating the 'value for money' investment in the fund. Liaison with potential users of the research and participation in related policy/advisory committees increase the chance of impacting on health policy and behaviour. Further studies are needed to better understand factors and to develop interventions that may facilitate the knowledge translation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.286 | 0.399 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.004 | 0.004 |
| Bibliometrics | 0.019 | 0.020 |
| Science and technology studies | 0.003 | 0.004 |
| Scholarly communication | 0.008 | 0.005 |
| Open science | 0.003 | 0.008 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".