Data and codes for "Disentangling Multi-view Representations Beyond Inductive Bias"
Bibliographic record
Abstract
This record contains the data and codes for this paper:Guanzhou Ke, Yang Yu, Guoqing Chao, Xiaoli Wang, Chenyang Xu, and Shengfeng He. 2023. "Disentangling Multi-view Representations Beyond Inductive Bias." In Proceedings of the 31st ACM International Conference on Multimedia (MM '23), October 29–November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3581783.3611794dmrib-weights is the file for pre-trained weights. DMRIB-main is a copy of the project's GitHub Repository at https://github.com/Guanzhou-Ke/DMRIBThe official repos for ""Disentangling Multi-view Representations Beyond Inductive Bias"" (DMRIB)Status: Accepted in ACM MM 2023.Training stepWe show that how DMRIB train on the EdgeMnist dataset.Before the training step, you need to set the CUDA_VISIBLE_DEVICES, because of the faiss will use all gpu. It means that it will cause some error if you using tensor.to() to set a specific device.set environment. export CUDA_VISIBLE_DEVICES=0 train the pretext model. First, we need to run the pretext training script src/train_pretext.py. We use simclr-style to training a self-supervised learning model to mine neighbors information. The pretext config commonly put at configs/pretext. You just need to run the following command in you terminal: python train_pretext.py -f ./configs/pretext/pretext_EdgeMnist.yaml train the self-label clustering model. Then, we could use the pretext model to training clustering model via src/train_scan.py. python train_scan.py -f ./configs/scan/scan_EdgeMnist.yaml After that, we use the fine-tune script to train clustering model scr/train_selflabel.py. python train_selflabel.py -f ./configs/scan/selflabel_EdgeMnist.yaml training the view-specific encoder and disentangled. Finally, we could set the self-label clustering model as the consisten encoder. And train the second stage via src/train_dmrib.py. python train_dmrib.py -f ./configs/dmrib/dmrib_EdgeMnist.yaml ValidationNote: you can find the pre-train weights in the file dmrib-weights. And put the pretrained models into the following folders path to/{config.train.log_dir}/{results}/{config.dataset.name}/eid-{config.experiment_id}/dmrib/final_model.pth, respectively. For example, if you try to validate the EdgeMnist dataset, the default folder is ./experiments/results/EdgeMnist/eid-0/dmrib. And then, put the pretrained model edge-mnist.pth into this folder and rename it to final_model.pth.If you do not want to use the default setting, you have to modify the line 58 of the validate.py. python validate.py -f ./configs/dmrib/dmrib_EdgeMnist.yaml CreditThanks: Van Gansbeke, Wouter, et al. "Scan: Learning to classify images without labels." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X. Cham: Springer International Publishing, 2020.Citation Guanzhou Ke, Yang Yu, Guoqing Chao, Xiaoli Wang, Chenyang Xu, and Shengfeng He. 2023. Disentangling Multi-view Representations Be- yond Inductive Bias. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29–November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3581783.3611794
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.026 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.004 | 0.006 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.004 | 0.006 |
| Open science | 0.005 | 0.008 |
| Research integrity | 0.003 | 0.005 |
| Insufficient payload (model declined to judge) | 0.383 | 0.369 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".