Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Why is this work in the frame?
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Abstract
Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-the-art performance on three benchmark datasets: Flickr8k, Flickr30k and MS COCO.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
The record
- Venue
- arXiv (Cornell University)
- Topic
- Multimodal Machine Learning Applications
- Field
- Computer Science
- Canadian institutions
- Canadian Institute for Advanced ResearchUniversity of TorontoUniversité de Montréal
- Funders
- —
- Keywords
- Computer scienceBenchmark (surveying)Artificial intelligenceVisualizationGazeObject (grammar)BackpropagationSalientSequence (biology)Object detectionMachine translationImage (mathematics)Artificial neural networkMachine learningComputer visionPattern recognition (psychology)
- Has abstract in OpenAlex
- yes