Abstract B007: TheBlueScrubs-v1: A Large-Scale Curated Dataset with ∼11 Billion Oncology Tokens for AI-Driven Cancer Research

Abstract Large language models (LLMs) are increasingly pivotal in cancer research, yet current public datasets offer insufficient scale and diversity to capture the complexity of oncology. To address this gap, we created TheBlueScrubs-v1, a 25-billion-token corpus of medical texts curated from the SlimPajama dataset. Approximately one-third of these tokens (∼11 billion) are annotated as cancer-related, making this one of the largest public, domain-focused text collections available for training and benchmarking oncology LLMs. Our two-stage pipeline first applied a high-speed logistic regression classifier (trained on a balanced set of 60,000 medical vs. non-medical documents) to label texts by medical relevance. This process extracted ∼4% of SlimPajama, yielding documents with at least 0.8 probability of containing medical content. Next, a 70B-parameter open-source LLM (Llama 3.1) evaluated each text’s medical scope, factual precision, and safety on 1–5 scales. Validation by clinicians and GPT-4o found strong concordance, confirming the reliability of these automated assessments. We further developed a specialized cancer classifier using logistic regression with TF-IDF features, trained on 60,000 examples, to identify oncology-related texts. This yielded a high-quality oncology subset (∼11 billion tokens) spanning topics such as cancer diagnosis, therapeutics, and real-world clinical notes. Detailed safety metrics enable red-teaming to mitigate misinformation and promote ethical use in oncology research. Potential applications include (1) fine-tuning LLMs for oncology-focused tasks such as treatment recommendation, clinical trial matching, and patient education, (2) building safety classifiers to detect harmful or misleading content, and (3) synthetic data generation to expand training sets while preserving privacy. Early experiments demonstrate that LLMs fine-tuned on TheBlueScrubs-v1 achieve performance on par with or exceeding models trained on smaller, specialized medical corpora. By releasing this large-scale, annotated dataset under an open license, we aim to accelerate innovation in AI-driven cancer research and foster collaborative efforts toward safer, more accurate clinical language models. Citation Format: Luis Felipe, Gilmer Valdes. TheBlueScrubs-v1: A Large-Scale Curated Dataset with ∼11 Billion Oncology Tokens for AI-Driven Cancer Research [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Artificial Intelligence and Machine Learning; 2025 Jul 10-12; Montreal, QC, Canada. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(13_Suppl):Abstract nr B007.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.