Controlled testing of next generation leak detection and quantification solutions to evaluate performance and develop consensus assessment metrics
Bibliographic record
Abstract
Reducing methane emissions, a potent short-term climate forcer, is critical for mitigating global warming. The oil and gas (O&G) industry is a major source of anthropogenic methane emissions, and regulations in the U.S. and Canada mandate leak detection and repair (LDAR) programs to mitigate these emissions. Traditional LDAR methods, which includes manually surveying O&G assets with handheld optical gas imaging (OGI) cameras or portable organic vapor analyzers, can be costly and labor-intensive given the vast spatial extent of O&G facilities. However, emerging, next-generation leak detection and quantification (LDAQ) solutions promise a more cost-effective alternative but must demonstrate equal or superior emissions mitigation potential to gain regulatory approval. Standardized controlled testing is essential for verifying this equivalence, yet no widely accepted framework currently exists to achieve this goal. This study evaluates and improves the first known standardized controlled testing protocols designed to address this gap. Two test protocols were developed for the two broad categories of LDAQ solutions: continuous monitors, which operate autonomously over extended durations, and survey solutions, which function over shorter durations with human supervision. These protocols, developed through multi-stakeholder collaboration, were used to test 29 LDAQ solutions (some tested multiple times) at the Methane Emissions Technology Evaluation Center (METEC). METEC is an 8-acre outdoor controlled testing facility that simulates methane emissions from North American onshore O&G equipment. Each survey solution and continuous monitor was tested for a minimum of 3 days and 11 weeks, respectively. Tested controlled release rates were up to 7100 g CH4/h for continuous monitors and 2100 g CH4/h for survey solutions. Key performance metrics, including probability of detection (POD), localization accuracy and precision, quantification accuracy, and survey times, were assessed. Seven solutions were retested 3 to 13 months after their initial tests to examine performance changes over time. Results showed that no single LDAQ solution or solution category achieved optimal performance across all the metrics evaluated. For continuous monitors, only two solutions achieved 90% POD within the tested range, failed to detect ≤ 40% of the controlled releases, and had ≤ 40% of their reported detections classified as false alerts. Camera- and laser-based continuous monitors demonstrated the highest emissions source localization accuracy, with most of them attributing ≥49% of their detection reports to the correct emission source. Quantification uncertainty varied widely, with solutions underestimating and overestimating actual emission rates by factors up to 50 and 46, respectively. For survey solutions, handheld OGI cameras exhibited better accuracy and repeatability in detecting and localizing small fugitive emissions compared to mobile (automobile-/drone-based) survey solutions, although the latter completed emission surveys faster. Additionally, performance improvements were observed with repeated testing, emphasizing the likely importance of regular, independent, and comprehensive evaluations in advancing LDAQ solutions. Findings from these controlled tests, combined with stakeholders feedback and insights from parallel field testing, informed the revision of the protocols to better reflect the application of LDAQ solutions at real O&G facilities. Study findings demonstrates that integrating multiple solutions can complement the limitations of any individual or category of LDAQ solutions. Continuous monitors and automobile-/drone-based survey LDAQ solutions can rapidly detect and narrow-down sources of emissions, enabling targeted follow-up investigations with handheld LDAQ survey solutions. In general, this work contributes significantly to efforts aimed at accelerating regulatory approval and adoption of next-generation LDAQ solutions for methane emissions mitigation through transparent and rigorous controlled testing.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".