Adaptation of the First Episode Psychosis Services -- Fidelity Scale for use in New and Small Programs
Bibliographic record
Abstract
Introduction: Fidelity assessments can support healthcare services to deliver care consistent with best practices. However, early psychosis (EP) fidelity assessment tools typically require a volume of service data that is often unavailable to small or new programs. In this study, we pilot a formative fidelity assessment approach to address these challenges. Methods: A formative assessment approach to using the First Episode Psychosis Services – Fidelity Scale (FEPS-FS) was developed to enable the assessment of small and new EP programs. Over 48 months, EPI-CAL EP learning health care network programs completed standard FEPS-FS fidelity assessments, formative assessments for new programs, or formative assessments for small programs, depending upon program eligibility. Results: Of 27 remote fidelity assessments completed with EP programs across California, nine (33.3%) had insufficient service data to complete a standard FEPS-FS assessment. Utilizing the proposed formative assessment approach, one program met the criteria for a new program assessment, and seven a small program assessment. In the new program assessment approach, 34 of 36 items could be assessed. In the small program assessments, a median of 19 items was scored, with a mean FEPS-FS score range from 3.45 to 4.12. Conclusion: These findings suggest that a formative approach to fidelity assessment can generate a meaningful amount of data, capture known variability between programs, and potentially identify areas for service improvement to enhance quality for small and new programs. However, critical data regarding the delivery of pharmacological and psychosocial care were not captured, highlighting the limitations of the approach.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.033 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".