Development and testing of a brief EBP implementation intentions scale using Rasch analysis
Bibliographic record
Abstract
BackgroundIn 2016, the Department for Family and Community Services in New South Wales, Australia selected Multisystemic Therapy -Emerging Adults (MST-EA) as a potentially suitable intervention for clients in a leaving care program with high and complex support needs emerging from challenging behaviour, mental health problems, involvement with the criminal justice system, intellectual disabilities, and alcohol and other drug use.MST-EA was originally developed in the U.S. for young people aged 17 -21 with a serious mental health condition and involvement in the justice system [1].The program is an adaptation of standard MST [2] and had not been tested with a population with intellectual disabilities before.In the Australian MST-EA trial, its potential to be effective for people aged 16 -26 with a mild to moderate disability and at high risk for poor outcomes was explored.The first year of MST-EA implementation took place in a complex policy environment that was dominated by one of the most comprehensive social reforms in Australiathe introduction of the National Disability Insurance Scheme (NDIS).Its national roll-out began in July 2016.The NDIS follows a market-style system where government funding will no longer go directly to disability service providers, but instead to the client, who can choose the providers they want.This reform created substantial barriers to the implementation of MST-EA in New South Wales. Materials and MethodsBased on the Consolidated Framework for Implementation Research [3], a semi-structured questionnaire was developed for use with 15 key stakeholders to the MST-EA Implementation.It was administered with clinicians, managers, partner organisations, consultants and program developers to explore the perceived barriers that contributed most substantially to the lack of success in adapting, transferring and implementing this evidence-based program to the Australian context. ResultsData are currently being collected.Data collection will finish in May, and data analysis commence in June.Data will undergo thematic analysis guided by the Consolidated Framework for Implementation Research (CFIR).Of particular interest will be to understand in what way respondents suggest addressing the challenges that were perceived as substantial barriers to MST-EA adaptation, transport and implementation. ConclusionsToo few examples of challenged implementation projects are being documented, analysed and utilised for learning.Our understanding of complex policy contexts and how to manage them during implementation requires further development.The Australian MST-EA trial mirrors an implementation experience that is shared by many other projects initiated by government or non-government organisations and providers.It should be used to inform future implementation practice and decision-making.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.016 | 0.033 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.005 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".