Proficiency Testing Customization in Clinical Trials: How the pSMILE Project Ensures High-Quality Proficiency Testing Coverage for International Laboratories

The Johns Hopkins University Patient Safety Monitoring in International Laboratories (pSMILE) project is a National Institutes of Health contract resource designed to evaluate and develop the capability of laboratories to participate in National Institute of Allergy and Infectious Diseases supported prevention, vaccine, and therapeutic clinical studies conducted outside the United States (1, 2). One of the core functions of this project is to facilitate appropriate proficiency testing (PT), also known as external quality assessment (EQA), coverage for every specified analyte according to sponsor requirements. As a key quality assurance contractor, pSMILE is tasked to identify PT providers and assess the suitability of their available PT panels. Examples of criteria to take into consideration when evaluating suitability for an analyte include but are not limited to (3–5): instrument/method type available peer group shipping logistics frequency of survey events type and number of samples per event clinical relevance of samples evaluation scheme. When standard panels are not available, our team sources alternatives by working with suppliers and clinical trials network partners or developing our own (virtual) panels. Once panels have been selected, pSMILE facilitates the ordering and carefully monitors shipment to ensure that samples arrive in good condition. After testing by the laboratories and evaluation by the PT providers, pSMILE reviews each laboratory’s survey, grades and monitors their performance, and provides written reviews and summaries on a monthly basis. When performance failures or trends are identified, we assist the laboratories in troubleshooting by providing guidance and training in areas needing improvement. Additionally, a web-based investigation report procedure is used to systematically review each step in the testing process and lead laboratories through a detailed investigation to find the root cause of the error. Ensuring appropriate PT coverage commonly consists of reviewing the laboratory test menu and selecting a PT product that covers each analyte in each discipline. Laboratories may select their PT provider based on their accreditation status, as some accrediting bodies require a specific PT provider. In the United States, the Centers for Medicare & Medicaid Services website provides a list of Clinical Laboratory Improvement Amendments Approved Proficiency Testing Programs. After almost 2 decades of facilitating, reviewing, and evaluating PT, pSMILE has discovered that one size does not fit all. Additional challenges are met by international laboratories that may be using non-FDA-approved methods. Although the US-based sponsor prefers the use of FDA-approved methods, there are many cases where these methods are not locally available to international laboratories. Additionally, research studies may use new and developing technologies or methods that have not been approved or cleared by the FDA. In such cases, the use of US-based PT providers becomes problematic since many do not provide panels or do not have a sufficient peer group for non-FDA-approved methods. It is incumbent to evaluate each PT panel before purchase and again after each round of PT to ensure that the panel is appropriate for the analyte, the test methodology, the instrument, the assay range, and the clinical use of the analyte. In Table 1, we present a selection of case studies that demonstrate the need for PT customization and pSMILE’s approach. To summarize, PT ordering was tailored to specific laboratory needs in the fields of mycobacteriology, urine pregnancy, HIV serology, and microscopy by identifying PT providers with surveys containing an increased number of challenges, providing advantageous shipping, testing new methodologies, employing materials covering pre- and postanalytical testing phases, and addressing critical clinical cutoffs as well as by developing our own custom virtual survey. Examples of proficiency testing customization. Abbreviations: MTB, Mycobacteria tuberculosis; DST, drug susceptibility testing; NIH, National Institutes of Health; CAP, College of American Pathologists; IQLS, Integrated Quality Laboratory Services; IGRA, interferon gamma release assay; QFT, QuantiFERON; QFT®-Plus, QuantiFERON®-MTB Gold Plus; IGRA MTB, interferon gamma release assays Mycobacterium tuberculosis; LPA, line probe assay; DCS, dried culture spots; hCG, human chorionic gonadotropin; OWA, One World Accuracy; RVSS, retrovirus and syphilis serology; EIA, enzyme immunoassay; MTS, Medical Training Solution. Over the course of the project, initially funded in 2004, we have experienced the need for PT customization based on knowledge gained from assisting over 285 laboratories with proficiency testing. We have facilitated selecting and ordering PT panels covering more than 320 analytes in the areas of chemistry, hematology, microbiology, immunology, clinical microscopy, and others. We have additionally reviewed results from over 12 460 PT panels, revealing trends and patterns among laboratories and surveys. We have utilized more than 12 different PT providers from the United States, Canada, South Africa, Brazil, Germany, France, and the United Kingdom. Developing a strong working relationship with PT providers, beyond the customer service representative level, can help laboratories facilitate improvements to panel offerings. Providing data to show analytical and clinical relevance can help providers appreciate the impact of their panels and results. Additionally, offering constructive feedback and a desire to work with the provider to make changes that can be beneficial to all parties can foster a spirit of collaboration. In summary, a one-size-fits-all approach to PT can lead to uninformative or insufficient coverage of an analyte. It is important for laboratories to fully investigate all aspects of these PT panels before purchasing and to continually reevaluate panels for suitability. Participating in external PT programs is a vital part of any laboratory to uphold an excellent standard of care. Tailoring PT panels to the needs of the specific laboratory adds value to the overall improvement of the testing site. Additionally, customization of PT contributes to the success of research studies, making the data more robust, reliable, and valuable. It ensures the quality of the data required for regulatory compliance, which in turn facilitates approval of drug and treatment regimens. While this approach was initially used by pSMILE to meet study-specific needs, the overall safety of the local patient population may also benefit from utilizing this customized approach. Nonstandard Abbreviations: pSMILE, Patient Safety Monitoring in International Laboratories; PT, proficiency testing; EQA, external quality assessment; UK NEQAS, United Kingdom National External Quality Assessment Service. Author Contributions:The corresponding author takes full responsibility that all authors on this publication have met the following required criteria of eligibility for authorship: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Nobody who qualifies for authorship has been omitted from the list. Anne Leach (Writing—original draft-Lead), Kristin Murphy (Writing—review & editing-Lead), Mandana Godard (Writing—original draft-Supporting, Writing—review & editing-Supporting), Matthew Schwartz (Writing—original draft-Supporting, Writing—review & editing-Supporting), and Lori Sokoll (Funding acquisition-Lead, Resources-Lead, Writing—review & editing-Equal). Authors’ Disclosures or Potential Conflicts of Interest:Upon manuscript submission, all authors completed the author disclosure form. Research Funding: This work was supported in whole or in part with federal funds from the National Institute of Allergy and Infectious Diseases, National Institutes of Health, Department of Health and Human Services, under Contract No. 75N93020C00001. Disclosures: L.J. Sokoll, Associate Editor for The Journal of Applied Laboratory Medicine, ADLM. Acknowledgments: The authors thank Daniella Livnat, Division of AIDS, National Institute of Allergy and Infectious Diseases, National Institutes of Health, for her leadership and support of the pSMILE program and for critical review of the manuscript.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.