Current opinions on large cellular models

Glossary of some machine learning conceptsPretraining: Pretraining is a process where a model learns from large amounts of data before being used on specific tasks.This initial phase allows the model to capture general patterns and semantics, enhancing its performance and adaptability across various specific applications.Self-supervised learning: Self-supervised learning is a strategy of machine learning that does not use extra labels to supervise the learning procedure, but instead uses the task of predicting certain parts of the data from other parts as the objective of learning.This approach eliminates the need for labeled data, as the model generates its own supervision signal during training.Embedding: Embedding in the general sense is to represent objects such as text, images, genes, or amino acids using mathematical vectors in certain continuous space where the locations of the vectors represent meanings or semantics of the objects with regard to their relations.In LCMs, embeddings are compact representations of the transcriptomic data in a mathematical space, encoding gene symbols and expression value information.LCMs can also obtain embeddings for cells and cell states.Loss function: Loss function is one of the key elements in the design of a machine learning model and algorithm.It quantifies the difference between predicted and true values, guiding the model to adjust its parameters to minimize this difference.Transformer: Transformer is a neural network architecture that uses self-attention mechanisms to capture relationships between embeddings of the input objects (tokens).It is widely used in large artificial intelligence (AI) models nowadays due to its scalability and effectiveness.Masked modeling: In masked modeling, parts of the input data are replaced by a specific mask embedding, and the model learns to predict these parts based on the remaining information.It is like a "cloze test": When the model learns to predict the masked data, it "understands" the data.Autoregressive generative modeling: In autoregressive generative modeling, the model generates new data points given the preceding ones in a sequence.In LCMs, gene expression rank can be regarded as sequential information.Zero-shot learning: Zero-shot learning is a machine-learning paradigm where a model is trained to perform tasks it has not been explicitly trained on.In LCMs, it usually refers to extracting cell or gene embeddings for downstream tasks from the pretrained model without the need for extra training for specific tasks.Fine-tuning: Fine-tuning is a process where a pretrained model is further trained on a specific task or dataset to improve its performance.This is particularly useful when the pretrained model has learned useful representations that can be adapted to the target task, saving time and computational resources.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.