Value-Based Framework for Evaluating Pre-Commercial Procurement: Case Study of Value-Based Key Performance Indicators
Bibliographic record
Abstract
Background: The demographic shift toward older populations is placing increasing pressure on health care systems, and only 20% of patients with chronic issues in the industrial world's rural areas have guaranteed access to adequate health care services. This stresses the health care systems, emphasizing the need for innovative solutions. The Horizon 2020 Pre-Commercial Procurement (PCP) project, Crane, addresses these needs by facilitating the procurement of a digital self-management system for treating patients with chronic issues at home. Three rural European regions are participating in the project: Västerbotten (Sweden), Extremadura (Spain), and Agder (Norway). Objective: This study aims to explore and identify key design criteria and value-based key performance indicators (VB-KPIs) to support the development and evaluation of digital health care solutions for patients with chronic issues in rural areas within the Crane PCP process. Methods: A 3-iteration process was used to identify and prioritize the VB-KPIs in the Crane project. First, user needs were investigated based on stakeholder analyses in the participating rural regions. The early health technology assessment tool, Step Up, was used in 5 workshops (2 in Agder, 2 in Extremadura, and 1 in Västerbotten). Participants included patients and health care professionals. Second, post workshop, stakeholders were asked to comment on the summarized results, which were accordingly adjusted. Third, following the workshops, VB-KPIs were identified and prioritized, and discussions among representatives from the 3 buyer regions were conducted. Results: Thirty-five VB-KPIs across 5 domains were identified. User-related (9 VB-KPIs), employee-related (9 key performance indicators), clinical (4 VB-KPIs), organizational (6 VB-KPIs), and economic (8 VB-KPIs) outcomes from the workshops and the subsequent discussions emphasized regional differences in terms of user needs and priorities. While Agder (Norway) and Västerbotten (Sweden) emphasized privacy, digital trust, and physical interaction as important, Extremadura (Spain) prioritized negotiation and shared decision-making. Despite differences, shared values were identified, including empowerment, flexibility, preventative care, and improved quality of life. Conclusions: The identified and prioritized VB-KPIs are likely to provide a need-based foundation for the development and subsequent evaluation of the digital PCP, Crane, although regional socioeconomic and cultural differences may necessitate local adaptations.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.072 | 0.070 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.016 | 0.015 |
| Science and technology studies | 0.005 | 0.010 |
| Scholarly communication | 0.012 | 0.011 |
| Open science | 0.003 | 0.007 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".