FiLM: Visual Reasoning with a General Conditioning Layer
Why is this work in the frame?
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Machine scores (provisional)
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
- Teacher spread
- 0.280 · how far apart the two teachers sit on this one work
- Validation status
score_only:v0-immature-baseline· verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it
Abstract
We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - answering image-related questions which require a multi-step, high-level process - a task which has proven difficult for standard deep learning methods that do not explicitly model reasoning. Specifically, we show on visual reasoning tasks that FiLM layers 1) halve state-of-the-art error for the CLEVR benchmark, 2) modulate features in a coherent manner, 3) are robust to ablations and architectural modifications, and 4) generalize well to challenging, new data from few examples or even zero-shot.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
The record
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Topic
- Multimodal Machine Learning Applications
- Field
- Computer Science
- Canadian institutions
- Canadian Institute for Advanced ResearchUniversité de Montréal
- Funders
- Fonds de recherche du Québec – Nature et technologiesCHIST-ERANvidia
- Keywords
- Affine transformationComputer scienceBenchmark (surveying)Artificial intelligenceFeature (linguistics)ComputationTransformation (genetics)Artificial neural networkSimple (philosophy)Layer (electronics)Visual reasoningImage (mathematics)Task (project management)Process (computing)Pattern recognition (psychology)Machine learningAlgorithmMathematics
- Has abstract in OpenAlex
- yes