Bibliographic record
Abstract
One key factor in training deep learning models is having abundant training data. This often requires collecting data from multiple sources (clients), which may violate data protection regulations. Federated learning (FL) provides a training paradigm where clients collaboratively train a global model while keeping data decentralized. Despite its potential, current FL frameworks rely on restrictive assumptions, limiting their performance and applicability in real-world scenarios. In practice, client data originate from different distributions (i.e., data heterogeneity), and local models are optimized for their respective data, leading to poor performance and slow convergence. To address data heterogeneity, we first adopt a domain-invariant feature learning perspective to capture task-relevant features, and propose a generalized FL framework. We extend this framework to a label-scarce setting where only a few samples are labeled for each client, thereby reducing annotation burden. We then tackle a more challenging scenario involving both labeled and unlabeled clients, indicating that data distributions differ across the two groups. To facilitate effective knowledge transfer, we introduce a generalized FL framework that combines pseudo-labeling with a dual-selection strategy which selects pseudo-labeled samples and model components for updating. Next, we reformulate the global objective to produce personalized models for each client, addressing data heterogeneity from a new perspective. We propose a Bayesian-enhanced personalized FL framework which incorporates Bayesian learning into FL to mitigate overfitting, and designs high-quality personalized priors for each client to guide local training. We then continue our investigation of personalized FL, focusing on an underexplored yet critical form of data heterogeneity: concept drift across clients. We also explore the potential of multimodal data. To improve model performance, we propose a multimodal-enhanced personalized FL framework with personalized modules to capture individual understanding of each input and effective fusion strategies to integrate features from diverse modalities. We finally explore the feasibility of allowing clients to customize their model structures and propose a heterogeneous FL framework that tackles dual heterogeneity. In this framework, a designed 'bridge' enables collaboration between clients by representing local knowledge as logits on the bridge and a similarity-based knowledge distillation strategy supports effective cross-client knowledge absorption.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.010 | 0.033 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.001 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.003 |
| Scholarly communication | 0.003 | 0.007 |
| Open science | 0.004 | 0.005 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".