Abstract B041: A Cellular Network-Aware Foundation Model Improves Single-Cell Level Predictions

Abstract Large-scale, single-cell transcriptomics datasets present an opportunity to develop foundation models for universal cell and gene representations. Yet, single-cell foundation models are often trained or evaluated based on abstract rather than biologically relevant benchmarks, such as predicting the specific perturbations that may induce a desired state transition in a tumor or TME cells, which are especially critical in oncology. Critically, current foundation models fail to effectively leverage the increasingly understanding of regulatory and signaling networks (gene regulator networks or GRN for short), which would serve as a biological inductive bias and significantly constrain the potential solution search space. These networks can also enhance language model performance by introducing meaningful relative positional information and long-range dependencies between gene tokens. While current transformer-based models should be able, in theory, to learn GRN structure during training, the very large number of pairwise and three-ways interactions—combined with the non-local and loopy structure of GRNs—makes this task exceedingly challenging for models. We propose that leveraging graph models, representing the molecular interactions governing cell behavior, can significantly improve single cell foundation model performance. For this purpose, we introduce a foundation model that leverages large-scale graph topologies and graph diffusion approaches to support Transformers’ attention mechanisms, thus effectively learning generalizable single cell level gene embeddings that are consistent with the cell’s underlying regulatory logic. By enforcing subpopulation-specific, GRN-consistent solutions—which support modeling and quantifying the heterogeneity of both tumor and TME-related cells in-silico—such an approach also addresses critical issues arising from lack of context specificity. This supports biologically relevant tasks, such as predicting gene expression distributions and treatment effects induced by unseen genetic perturbations. To validate the computational integrity of our model, we performed comprehensive benchmarking studies on tasks such as cell type prediction/annotation, GRN structure understanding, and gene expression predictions. The cellular network-aware model outperformed state-of-the-art foundation model baselines. By capturing regulatory dependencies between gene products, the model reduces the solution search space and facilitates biologically grounded predictions. We propose that such GRN-enabled models are better suited to address biologically relevant questions—such as elucidating mechanistic determinants ranging from oncogenesis and progression to immune evasion and exhaustion. Citation Format: Mingxuan Zhang, Vinay Swamy, Léo Dupire, Rowan Cassius, Charilaos Kanatsoulis, Evan Paull, Theofanis Karaletsos, Andrea Califano. A Cellular Network-Aware Foundation Model Improves Single-Cell Level Predictions [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Artificial Intelligence and Machine Learning; 2025 Jul 10-12; Montreal, QC, Canada. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(13_Suppl):Abstract nr B041.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.