Functional Alignment-Auxiliary Generative Adversarial Network-Based Visual Stimuli Reconstruction via Multi-Subject fMRI
Bibliographic record
Abstract
Functional Magnetic Resonance Imaging (fMRI) provides more precise spatial and temporal information to reconstruct stimulus images than other technologies that can be used to measure the human brain's neural responses. The fMRI scans, however, generally show heterogeneity among different subjects. The majority of the existing methods aim primarily at mining correlations between stimuli and evoked brain activity, disregarding the heterogeneity among subjects. Therefore, this heterogeneity will impair the reliability and applicability of multi-subject decoding results, leading to sub-optimal results. The present paper proposes the functional alignment-auxiliary generative adversarial network (FAA-GAN) as a novel multi-subject approach for visual image reconstruction that employs functional alignment to alleviate the heterogeneity between subjects. Our proposed FAA-GAN includes three key components: 1) a generative adversarial network (GAN) module for reconstructing visual stimuli, which consists of a visual image encoder as the generator that uses a nonlinear network to convert stimuli images into an implicit representation and a discriminator that generates the images comparable to the original images in detail; 2) a multi-subject functional alignment module, which is used to precisely align the individual fMRI response space of each subject in a common space to reduce the heterogeneity among different subjects; and 3) a cross-modal hashing retrieval module used for similarity retrieval of two modalities of data, i.e., the visual images and the evoked brain responses. Experiments on real-world datasets show that our FAA-GAN method does better than other state-of-the-art deep learning-based reconstruction methods with fMRI.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".