Validation of a Web-Based Platform for Online Training in Point-of-Care Diaphragm Ultrasound

Diaphragm dysfunction occurs frequently in patients on mechanical ventilation (MV) and is associated with long-term morbidity and mortality (1, 2). Transthoracic diaphragm ultrasound (DUS) has emerged as a feasible and reproducible noninvasive technique to assess diaphragm structure and function during MV (3). DUS measurements can inform prognosis and may facilitate diaphragm-protective ventilation (4). Technical training remains an important hurdle to widespread dissemination of the technique (5). Although online learning platforms are increasingly employed in medical education and might facilitate dissemination, it is uncertain whether competency in DUS can be acquired via this medium. We set out to determine whether an online training module with expert feedback provided remotely can achieve competency in the DUS technique in trainees with no prior experience with DUS (DUS naïve) or no prior experience with any form of ultrasound (ultrasound naïve). A prospective observational study was conducted in a quaternary intensive care unit from May to October 2019 following institutional ethics approval (Institutional Review Board #18-1192). Five trainees (from 43 potential candidates) were voluntarily selected based on their response to a survey and placed in one of two categories: 1) DUS naïve (n = 3, postgraduate year 5–6 pulmonary and critical care medicine fellows with experience in general critical care ultrasound [>30 studies] but no prior experience with DUS) and 2) ultrasound naïve (n = 2, research coordinators with no prior ultrasound experience). Each trainee completed an online DUS course (https://michener.blackboard.com) consisting of a 90-minute didactic curriculum on diaphragm anatomy, physiology, and DUS technique, including probe positioning, and an online test with 38 questions requiring an 80% correct response rate to pass. The ultrasound-naïve group was also given a 1-hour hands-on in-person ultrasound knobology session by an expert sonographer (S.D.) to familiarize them with the ultrasound machine (GE Venue, General Electric), probe manipulation, image acquisition, and archiving. Next, each trainee independently performed 17 B-mode and M-mode examinations (2 in healthy volunteers and 15 in MV patients). No hands-on training was provided by experts on the performance of DUS. Each exam consisted of two tidal breaths and one maximal inspiratory effort breath with measurement of diaphragm thickness at end-expiration (Tdi,ee), diaphragm thickening fraction during inspiration (TFdi), and maximal thickening fraction (TFdi,max). Measurements were made in the eighth or ninth intercostal space between the anterior and midaxillary lines using M-mode according to a previously published technique (3). During the training program, trainees received feedback on the quality of the acquired images and identification of zone of apposition through the online platform from two experts in DUS (J.W. and E.C.G.). Each trainee completed the course over the allotted 3 months. In the validation step, all trainees and experts (S.D. and E.C.G. together) measured Tdi,ee, TFdi, and TFdi,max in 10 MV patients after obtaining informed consent. Each observer repeated the measurement a few minutes after the initial measurement. The observer order was randomized for each patient, with experts last to perform the ultrasound. Observers were blinded to each other’s findings, and all measurements were performed independently. The location of probe was marked during the first examination in each patient, and this mark was used as a guide for probe positioning in the same patient by subsequent examiners, as this method has been shown to be necessary to attain adequate reproducibility (3). The competency of the trainees (and hence the effectiveness of the training methodology) was established by comparing measurement agreement between the reference standard (measurement obtained by S.D. and E.C.G.) and each trainee using the method of Bland and Altman (6, 7). Intraobserver repeatability coefficients were computed from random effect models. Agreement on the diagnosis of diaphragm dysfunction (TFdi,max < 20%) was assessed by the multirater kappa. Assuming a standard deviation for the difference in Tdi,ee between observers of 0.2 mm (larger than that previously observed) (3), a sample size of 10 patients was computed to be required to achieve 95% confidence intervals of ±0.2 mm for limits of agreement for Tdi,ee (6). Analyses were performed using R statistical software and SAS 9.4 software (SAS Institute). All trainees successfully completed the online module and the online test and uploaded 17 examinations for online review and feedback. In the validation phase, 10 MV patients were enrolled with mean (±standard deviation) duration of MV of 25 (±43) days (Table 1). All trainees were able to obtain DUS measurements on all 10 patients. TFdi,max could not be measured in three patients, as they were unable to tolerate a spontaneous mode of ventilation to make vigorous respiratory efforts. Bias and limits of agreement between trainees and experts varied slightly between trainees but were generally acceptable and similar to previously reported values for reproducibility of Tdi,ee and TFdi (Table 2, Figure 1). The coefficient of repeatability suggested acceptable within-observer reproducibility. Results were similar for DUS-naïve and ultrasound-naïve groups. Limits of agreement for TFdi,max were wide and variable between trainees (Table 2). In a sensitivity analysis excluding ultrasound-naïve trainees, the reproducibility parameters were generally very similar for all measurements. Characteristics of critically ill study population for validation phase Statistics are presented as mean ± standard deviation or n (column %). Agreement between trainees and the reference standard after completing the online diaphragm ultrasound course Definition of abbreviation: N/A = not applicable. * The repeatability coefficient is the largest difference one would observe in repeated measurements in the same patient by the same observer 95 times out of 100 repeated measurements. †The reproducibility coefficient is the largest difference one would observe in repeated measurements in the same patient between two different observers 95 times out of 100 paired measurements. Bland-Altman plot comparing the average end-expiratory diaphragm thickness measured by each trainee to the value obtained by the experts. DUS = diaphragm ultrasound; LOA = limit of agreement; US = ultrasound. We found that, upon completion of an online DUS training platform, trainees with varying prior ultrasound experience and no prior DUS experience obtained DUS measurements of Tdi,ee and TFdi with acceptable agreement to measurements obtained by experts. The observed measurement precision was sufficient to detect clinically relevant changes in diaphragm thickness (±0.2 mm) and diaphragm thickening fraction (±15%) based on thresholds established in previous outcome studies (2, 4). However, the reproducibility of TFdi,max measurements was comparatively inadequate to distinguish the presence or absence of diaphragm weakness (TFdi,max < 20–30%), which might result in misdiagnosis of diaphragm weakness (8). We conclude that competency in measurements of Tdi,ee and TFdi can be achieved using a web-based training platform and that further work is required to develop the platform to disseminate competency in TFdi,max measurements. Several approaches have been used to evaluate online learning in bedside ultrasound. A combination of web-based didactic training with either hands-on training or self-guided assessment has been found to be noninferior to traditional training methods (9, 10). For example, a combined approach of video tutorials and expert-guided hands-on training in DUS was more effective in obtaining windows and performing DUS measurements than video tutorial alone (11). Competency is often defined qualitatively, such as the ability to perform a procedure or to obtain an ultrasound view (10). A strength of the present study was that competency was assessed quantitatively rather than qualitatively, providing evidence of competency to obtain measurements in clinical practice or research. The web-based training approach described in the present study provides a potential model for disseminating competency in other ultrasound techniques. Future work to improve dissemination of competency in measuring TFdi,max is required, possibly by increasing the number of measurements obtained by a single operator to reduce measurement variability. Our findings suggest that the online DUS training platform can be deployed to efficiently disseminate competency in DUS measurements of diaphragm thickness and TFdi at a level sufficient for application in research and clinical practice. Future research is required to improve training in the measurement of TFdi,max through the platform. Author Contributions: S.D. and E.C.G. are the guarantors of the paper. S.D., A.D, and E.C.G. conceived and designed the study. S.D., O.M., A.H., M.T.S., R.V., N.T. J.W., S.V., and E.C.G. acquired the data. S.D., M.L., and E.C.G. conducted the analysis. S.D. drafted the manuscript. All authors revised it critically for important intellectual content and gave final approval of the version to be published. All authors agree to be accountable for all aspects of the work, ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Author disclosures are available with the text of this article.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.