Cross Domain Model Adaptation and Generalization

Deep learning models often experience performance degradation when applied to data that differs from their training distribution. This thesis addresses the challenge of data distribution discrepancies through two main aspects: adapting source models to target domains without data annotation and improving model generalization to unseen domains. For source-free domain adaptation (SFDA), where source data is inaccessible, this thesis proposes two innovative methods to address key challenges. The first method generates labeled surrogate source training data by optimizing inputs while keeping the source model fixed. Gradient-based global fitting constraints are introduced to ensure the surrogate data accurately reconstructs the complete source distribution. These surrogate data can then be utilized by existing unsupervised domain adaptation methods. The second method focuses on source-free domain adaptation under the inductive setting. A semi-supervised fine-tuning approach is introduced, which partitions the unlabeled target training set into a confident pseudo-labeled subset and a less-confident unlabeled subset based on prediction confidence from the source model. A moving-average prototypical classifier updates soft labels for the unlabeled subset, enabling incremental adaptation of the source model to the target domain. Complementary to SFDA, domain generalization focuses on training models that are capable of generalizing to unseen testing domains. This thesis introduces two methods to address standard and federated domain generalization. The first method employs a two-stage training approach for standard domain generalization that simulates an approximate meta-generalization scenario and incorporates a self-adaptation module to adjust pretrained meta-source models to the meta-target domains. The core concept of self-adaptation involves leveraging contextual information as domain knowledge to automatically adapt a model trained in the first stage to new contexts in the second stage. The second method tackles federated domain generalization, where isolated client data is used to train a unified and generalizable model. The proposed approach incorporates local and global flatness regularizations to avoid sharp minima and encourage convergence to the global optimum. These regularizations leverage adversarial parameter perturbations, with two perturbation methods proposed at the level of weights and singular values. All proposed methods are evaluated on standard benchmarks, demonstrating their effectiveness in addressing data distribution discrepancies in deep learning.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.