Privacy Preservation Embedding-Based Clustering for Population Stratification Using Large Language Models
Bibliographic record
Abstract
Addressing population stratification is crucial in Genome wide association Studies (GWAS) since genetic differences tied to ancestry can confound such studies and introduce bias if not adequately managed. In this context, clustering methods are essential, enabling the accurate grouping of genetically similar individuals. However, due to the complexity and volume of genomic data, traditional clustering struggles and dimensionality reduction techniques often fail to retain key biological insights for proper analysis. This work uses a genome-specific large language model, DNABERT-S, to create embeddings for the 1000 Genomes population dataset to assess the effectiveness of the embedding when used in the clustering. The embeddings preserve the critical biological features, contextual meaning, and relationships encoded in the genomic sequences, which were clustered using K-means to analyze population-level patterns. Reconstruction attacks target embeddings produced by large language models in the natural language processing (NLP) domain. These attacks can also target genomic embeddings, which poses significant privacy concerns. To investigate this, we analyze how genomic embeddings are susceptible to reconstruction attacks, and we use differential privacy (DP) on mean embedding to mitigate such risk. Our findings demonstrate that differential privacy helps reduce the risk of reconstruction attacks while preserving utility; however, it does not entirely eliminate the attack.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.022 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.002 | 0.005 |
| Open science | 0.001 | 0.004 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".