Multisite Agricultural Veterans Affairs Farming and Recovery Mental Health Services (VA FARMS) Pilot Program: Protocol for a Responsive Mixed Methods Evaluation Study
Bibliographic record
Abstract
BACKGROUND: Veterans Affairs Farming and Recovery Mental Health Services (VA FARMS) is an innovative pilot program to provide supportive resources for veterans with interests in agricultural vocations. Implemented at 10 pilot sites, VA FARMS will provide mental health services and resources for veterans while supporting training in gardening and agriculture. As each pilot site project has unique goals, outreach strategies, and implementation efforts based on the local environment and veteran population, evaluating the pilot program provides a unique challenge for evaluators. This paper describes the protocol to evaluate VA FARMS, which was specifically designed to enable site variation by providing both site-specific and cross-site understanding of site implementation processes and outcomes. OBJECTIVE: The objectives of this paper are to (1) describe the protocol used for evaluating VA FARMS, as an innovative Department of Veterans Affairs (VA) agriculturally based, mental health, and employment pilot program serving veterans at 10 pilot sites across the Veterans Health Administration enterprise; and (2) provide guidance to other evaluators assessing innovative programs. METHODS: This evaluation uses the context, inputs, process, product (CIPP) model, which evaluates a program's content and implementation to identify strengths and areas for improvement. Data collection will use a concurrent mixed methods approach. Quantitative data collection will involve quarterly program surveys, as well as three individual veteran participant surveys administered upon the veteran's entrance and exit of the pilot program and 3 months postexit. Quantitative data will include baseline descriptive statistics and follow-up statistics on veteran health care utilization, health care status, and agriculture employment status. Qualitative data collection will include participant observation at each pilot site, and interviews with participants, staff, and community stakeholders. Qualitative data will provide insights about pilot program implementation processes, veterans' experiences, and short-term participation outcomes. RESULTS: Evaluation efforts began in December 2018 and are ongoing. Between October 2018 and September 2020, 494 veterans had enrolled in VA FARMS and 1326 veterans were reached through program activities such as demonstrations, informational presentations, and town-hall discussions. A total of 1623 community members and 655 VA employees were similarly reached by VA FARMS programming during that time. Data were collected between October 2018 and September 2020 in the form of 336 veteran surveys, 30 veteran interviews, 27 staff interviews, and 11 community partner interviews. Data analysis is expected to be completed by October 2022. CONCLUSIONS: This evaluation protocol will provide guidance to other evaluators assessing innovative programs. In its application to the VA FARMS pilot, the evaluation aims to add to existing literature on nature-based therapies and the rehabilitation outcomes of agricultural training programs for veterans. Results will provide programmatic insights on the implementation of pilot programs, along with needed improvements and modifications for the future expansion of VA FARMS and other veteran-focused agricultural programs. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/40496.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.104 | 0.075 |
| Meta-epidemiology (narrow) | 0.004 | 0.004 |
| Meta-epidemiology (broad) | 0.004 | 0.003 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.006 | 0.003 |
| Scholarly communication | 0.005 | 0.004 |
| Open science | 0.004 | 0.004 |
| Research integrity | 0.005 | 0.008 |
| Insufficient payload (model declined to judge) | 0.056 | 0.012 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".