Theoretical Assessment of the SOILIE Model of the Human Imagination - eScholarship

Theoretical Assessment of the SOILIE Model of the Human Imagination Michael O. Vertolli (michaelvertolli@gmail.com) Vincent Breault (breault_vincent@gmail.com) Sebastien Ouellet (sebouel@gmail.com) Sterling Somers (sterling@sterlingsomers.com) Jonathan Gagne (gagne.jonathan@gmail.com) Jim Davies (jim@jimdavies.org) Institute of Cognitive Science, Carleton University 1125 Colonel By Drive, Ottawa, Ontario K1S 5B6 Canada things should be located, the mental scene is passed on for further processing—perhaps mental imagery. Abstract We describe the overall theory of the SOILIE model of the human imagination. In this description, we outline cognitive capacities for learning and storage, image component selection and placement, as well as analogical reasoning. The guiding theory behind SOILIE is that visual imagination is constrained by regularities in visual memories. Keywords: imagination; spatial cognition; analogy; visualization; cognitive model. creativity; Introduction The cognitive literature on imagination involves two related capacities: general creativity and the ability to generate mental simulations of possible worlds, often using sensory data from memory or the environment. The current focus is on the latter, particularly in the visual modality. This type of imagination is implicated in a number of cognitive activities, including reading a novel, planning future actions, recalling previous experiences, fantasizing about the future, and dreaming (Davies, Atance, & Martin Ordas, 2011). Although imagination of visual phenomena is often thought to be identical with pictographic, mental imagery, the view described here sees the rendering of a mental image as a final, optional stage. The process of rendering an imagined scene into neural “pixels” (colors at particular locations) is usually preceded by processes that determine what is to be placed in the image and where. For example, if one is asked to picture “a computer and a mouse,” one is likely to also picture a keyboard, desk, and related objects in an office or similar environment. The question is how does a mind know to combine these particular objects in their appropriate spatial configurations? To address this question, we chose to model a task in which a given agent takes a single word (e.g., “computer”) as the trigger to engage in the act of imagination. The task of the agent is to imagine a “computer” in a realistic scene. Using visual and spatial long-term memories, the agent populates the scene with elements that are likely to appear in an image with the triggering word (such as a keyboard). Once the underlying cognitive processes of the agent have selected what should appear in the image and where those Figure 1: SOILIE’s imagined output given the query ‘mouse’ and the returned labels: ‘computer’, ‘keyboard,’ ‘monitor,’ and ‘screen.’ The Model The Science of Imagination Laboratory Imagination Engine (SOILIE) is a computational model composed of multiple subsystems that together create the informational precursors of a 2D visual scene from an environmental trigger or query. In its current implementation, the engine takes a single word as input and returns a collection of object labels and their relative positions. The over-arching goal is for SOILIE to create visually imagined scenes in the same way that humans do. Many of SOILIE’s underlying subsystems have been discussed in previous work (Breault, Ouellet, Somers, & Davies, 2013; Davies & Gagne, 2010; Somers, Gagne, Astudillo & Davies, 2011; Vertolli & Davies, 2013). In what follows, we will take a step back and look at the entire model as a whole, including parts that are not explicitly used to determine SOILIE’s output. These elements contribute to the overall theory and include what is currently being worked on or extended in the model. Each of the parts will be addressed in chronological order as they might occur in an act of imagination. This chronological account will outline the following processes and structures. The first area is the agent-world interface, or the point at which information in the

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.