Lessons from Building a Large, Public, HIV-Related Database in Support of Ending the HIV Epidemic Initiative (Preprint)

The HIV epidemic remains a national priority in the United States, and the Ending the HIV Epidemic initiative has renewed the call for expanded prevention and treatment strategies capable of reducing new HIV infections by 90% by 2030. Achieving this goal requires robust, integrated data for understanding HIV-related needs, barriers to care, and the effectiveness of interventions. However, despite the existence of numerous publicly available datasets, few integrate multiple domains such as HIV outcomes, social determinants of health, and community-level factors. The lack of unified data and difficulty linking datasets hampers efforts for meaningful cross-domain analyses to tailor HIV management and treatment strategies. The resulting fragmentation constitutes a methodological gap: implementation teams lack replicable guidance for constructing unified HIV and contextual databases from public sources. In this viewpoint, we describe our experience building a unified compilation of publicly available HIV and community data to identify factors influencing HIV outcomes and interventions. The completed database comprises 242 variables drawn from 8 public sources mapped across clinic, zip code, county, and state levels of geography. Rather than simply reporting what we built, we position four core decisions as transferable methodological advances: (1) treating source identification as a bounded phase before construction begins, (2) adopting automated data engineering tools from the outset rather than manual entry, (3) establishing a shared data dictionary before the first variable is entered, and (4) integrating quality control throughout the workflow rather than as a final phase. The build required approximately 350 total project hours and revealed an initial spot-check error rate of approximately 33%, which we attribute primarily to manual data entry. By sharing the approach used to develop this database and making the final resource publicly accessible through the Yale Center for Methods in Implementation and Prevention Science, we aim to reduce barriers to data access and encourage similar data integration efforts. The methodological framework described in this paper is intentionally designed to be replicable with modest resources, and we present it as a practical model for research teams operating without specialized infrastructure. Consolidating HIV, social determinants of health, and contextual variables into a unified data source is a critical step toward enabling deeper, more comprehensive analysis and supporting ongoing efforts to end the HIV epidemic in the US.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.