Containers for the Borzoi Model (Linder et al. 2025) using the Genomic API for Model Evaluation (GAME) Framework
Bibliographic record
Abstract
The Predictor container (borzoi_human_predictor.sif) includes: Predictor script for sequence processing and error handling. Integrated Borzoi model with its dependencies and borzoi-gpu conda environment created using borzoi_gpu_environment.yml. Latest Baskerville and Borzoi repositories (as of 2025-02-25), which also contain helper scripts and 4 replicate model weights. Support scripts like: api_preprocessing_utils.py, error_message_functions_updated.py, predictor_help_message.json. NOTE: This container requires a GPU for execution because the Borzoi model relied on TensorFlow's GPU-accelerated operations. Running on CPU may lead to excessive memory usage and thread allocation failures. The (sample) Evaluator container (borzoi_evaluator.sif) includes: - Evaluator API script for genomic sequence evaluation. - Installed Python dependencies required by the Evaluator. Evaluator data directory (evaluator_data.tar.gz): This compressed file contains sample JSON data for testing the Evaluator container -- evaluator_message_more_complex.json. Running the containers: Always run the Predictor first, so it can listen for incoming connections: apptainer run --nv borzoi_human_predictor.sif HOST_IP HOST_PORT Then run the Evaluator: apptainer run \ -B absolute/path/to/evaluator_data:/evaluator_data \ -B absolute/path/to/predictions:/predictions \ evaluator.sif PREDICTOR_HOST PREDICTOR_PORT /predictions Important Notes: Input JSON file restriction: Currently, the script in the (sample) evaluator container can only process evaluator_message_more_complex.json as input. If a different JSON needs to be tested, please rename the JSON file to evaluator_message_more_complex.json to allow for the script to be able to find the mounted path in evaluator_data/ Predictions directory must be created: Before running the Evaluator container, please also create and mount predictions/ directory. Additional information can be found on GitHub: Genomic API for Model Evaluation Borzoi Model-specific information can be found within the same repository: Borzoi
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.007 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.001 | 0.003 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.003 |
| Open science | 0.005 | 0.004 |
| Research integrity | 0.001 | 0.004 |
| Insufficient payload (model declined to judge) | 0.103 | 0.060 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".