Privacy Preservation Embedding-Based Clustering for Population Stratification Using Large Language Models
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Addressing population stratification is crucial in Genome wide association Studies (GWAS) since genetic differences tied to ancestry can confound such studies and introduce bias if not adequately managed. In this context, clustering methods are essential, enabling the accurate grouping of genetically similar individuals. However, due to the complexity and volume of genomic data, traditional clustering struggles and dimensionality reduction techniques often fail to retain key biological insights for proper analysis. This work uses a genome-specific large language model, DNABERT-S, to create embeddings for the 1000 Genomes population dataset to assess the effectiveness of the embedding when used in the clustering. The embeddings preserve the critical biological features, contextual meaning, and relationships encoded in the genomic sequences, which were clustered using K-means to analyze population-level patterns. Reconstruction attacks target embeddings produced by large language models in the natural language processing (NLP) domain. These attacks can also target genomic embeddings, which poses significant privacy concerns. To investigate this, we analyze how genomic embeddings are susceptible to reconstruction attacks, and we use differential privacy (DP) on mean embedding to mitigate such risk. Our findings demonstrate that differential privacy helps reduce the risk of reconstruction attacks while preserving utility; however, it does not entirely eliminate the attack.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it