Bilinear Multimodal Discriminator for Adversarial Domain Adaptation with Privileged Information
Bibliographic record
Abstract
Over the past few years, deep Convolutional Neural Networks have shown outstanding performance on semantic segmentation, which is an essential tool needed by self-driving cars to understand their environments. However, their training relies on large datasets with pixel-level ground truth annotations, which are costly and tedious to produce on real data, making application to new situations difficult. In this context, Unsupervised Domain Adaptation (UDA) from synthetic data is an approach of great interest since it leverages cost-free labeled synthetic datasets to help generalizing to unlabeled real ones. In this paper, we propose a new adversarial training strategy for UDA that uses additional privileged information on the synthetic domain during training to improve transfer to the real one. Our method introduces a multimodal discriminator for adversarial training, featuring a bilinear fusion between representations of segmentation and privileged information to exploit at best alignment between modalities. We evaluate our approach on real-world Cityscapes dataset, using synthetic labeled data with depth as privileged information from SYNTHIA dataset and show competitive results.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.003 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.001 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".