A Three-Way Approach to Evaluating Partnerships: Partnership Survey, Integration Measure and Social Network Analysis
Bibliographic record
Abstract
An increasing number of initiatives are delivered through cross-sectoral or cross-jurisdictional partnerships. Methods to evaluate partnerships have also proliferated, but tend to focus only on particular aspects of partnerships. None alone provide a comprehensive picture of how a partnership is working. Working on commissioned evaluations of partnership initiatives, we have faced the challenge of selecting appropriate methods to gather information on partnership processes—from program or community-level partnerships to partnerships across the highest levels of government. While many partnership evaluations rely on analysis of participants’ views on how effectively the partnership works, more powerful evaluation demands methods to collect systematic quantifiable data on the actual behaviour of the partnership and how it changes over time. We reviewed a number of quantitative methods for assessing partnership processes and outcomes. As a result, we have been adapting and trialling three data collection tools to assess different aspects of partnerships: 1 A partnership survey (adapted from the Nuffield Partnership Assessment Tool): to collect systematic feedback from participating stakeholders on key partnership dimensions. 2 An integration measure (based on the Human Services Integration Measure developed by Browne and colleagues in Canada): to assess the level of cooperation between participating partners. 3 Social network analysis (employing a sociocentric approach and analysing data using UCINET 1 ): to collect information on interactions between individuals in the partnership. We have found that the complementarity of these methods provides a robust and more complete picture of the processes and outcomes of partnership initiatives and can be used in combination with qualitative methods to enhance the robustness of partnership assessments.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.230 | 0.241 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.004 | 0.003 |
| Bibliometrics | 0.034 | 0.032 |
| Science and technology studies | 0.005 | 0.006 |
| Scholarly communication | 0.009 | 0.014 |
| Open science | 0.004 | 0.019 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".