The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care
Bibliographic record
Abstract
BACKGROUND: The onset of the COVID-19 pandemic precipitated a rapid shift to virtual care in health care settings, inclusive of mental health care. Understanding clients' perspectives on virtual mental health care quality will be critical to informing future policies and practices. OBJECTIVE: This study aimed to outline the process of redesigning and validating the Virtual Client Experience Survey (VCES), which can be used to evaluate client and family experiences of virtual care, specifically virtual mental health and addiction care. METHODS: The VCES was adapted from a previously validated telepsychiatry survey. All items were reviewed and updated, with particular attention to the need to ensure relevance across mental health care sectors and settings. The survey was then revalidated using the 6 domains of health care quality of the Institute of Medicine (IOM) as a guiding framework. These 6 domains include being safe, effective, patient-centered, efficient, timely, and equitable. The VCES was piloted with a convenience sample of clients and family members accessing outpatient care at the Centre for Addiction and Mental Health (CAMH) in Toronto, Ontario, through video or telephone. A confirmatory factor analysis (CFA) was conducted in MPlus and used to test the factorial structures of the VCES, with minor respecification of the model based on modification indices, factor loadings, reliability, and item-total correlation. The respecifications were checked for alignment with the construct definitions and item interpretation. The reliability of the constructs was estimated by the Cronbach α coefficient. RESULTS: The survey was completed 181 times. The construct reliability was generally high. Timely was the only subscale with an α lower than 0.7; all others were above 0.8. In all cases, the corrected item-total correlation was higher than 0.3. For the CFA, the model was adjusted after multiple imputations with 20 datasets. The mean chi-square value was 437.5, with df=199 (P<.001). The mean root mean square error of approximation (RMSEA) was 0.08 (SD 0.002), the mean confirmatory fit index (CFI) was 0.987 (SD 0.001), the mean Tucker-Lewis Index (TLI) was 0.985 (SD 0.001), and the mean standardized root mean square residual (SRMR) was 0.04 (SD 0.001). CONCLUSIONS: This study describes the validation of the VCES to evaluate client and family experiences of virtual mental health and addictions care. Given the widespread uptake of virtual care, this survey has broad applicability across settings that provide mental health and addiction care. The VCES can be used to guide targeted quality improvement initiatives across health care quality domains. By effectively addressing challenges as they emerge, it is anticipated that we will continue to move toward hybrid modalities of practice that leverage the strengths and benefits of telephone, video, and in-person care to effectively respond to unique client and family needs and circumstances.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.020 | 0.022 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".