General Instruction Following in a Large-Scale Biologically Plausible Brain Model

General Instruction Following in a Large-Scale Biologically Plausible Brain Model Xuan Choo (fchoo@uwaterloo.ca) Chris Eliasmith (celiasmith@uwaterloo.ca) Center for Theoretical Neuroscience, University of Waterloo Waterloo, ON, Canada N2L 3G1 Abstract Y”, “add 1 to the value in WM area X”). Apart from motor and cognitive commands, actions can also be utilized to change the values of the model’s states. Rules are conditional statements typically of the form “IF X, THEN Y” (e.g. “If you see a 1, then push button A”) where X is a set of conditions which have to be met for the set of actions Y to be executed. More generally, in Spaun, rules are statistical maps between cortical states and actions. An instruction is a combination of rules or actions that can be executed sequentially (e.g. “Remember the number 1; add 1 to that number; write the result”) or in any order (e.g. “If you see a 1, then push button A; If you see a 2, then push button B”). We present a spiking neuron brain model implemented in 318,870 LIF neurons organized with distinct cortical modules, a basal ganglia, and a thalamus, that is capable of flexibly fol- lowing memorized commands. Neural activity represents a structured set of rules, such as “If you see a 1, then push button A, and if you see a 2, then push button B”. Synaptic connec- tions between these neurons and the basal ganglia, thalamus, and other areas cause the system to detect when rules should be applied and to then do so. The model gives a reaction time difference of 77 ms between the simple and two-choice reac- tion time tasks, and requires 384 ms per item for sub-vocal counting, consistent with human experimental results. This is the first biologically realistic spiking neuron model capable of flexibly responding to complex structured instructions. Keywords: neural engineering; spiking neuron model; in- struction following; instruction processing; cognitive con- trol; cognitive architectures Spaun The architecture of Spaun (the Semantic Pointer Architecture, or SPA) is composed of 9 distinct but interconnected modules (see Figure 1A). Of interest to this paper is how the action selection module interacts with the rest of the model. Fun- damentally, the action selection module of Spaun is identical to the basal ganglia (BG) based production system described in (Stewart, Bekolay, & Eliasmith, 2012), and functions sim- ilarly to the action selection component of production system models (e.g. (Anderson, 1996)). In these systems, action selection is hard-coded by a pre- defined set of rules. To select an action, the BG monitors internal cortical state variables and executes a rule whose an- tecedent best matches the values of the internal state variables (see Figure 1B). Critically, to encode instructions, the tran- sitions between each rule in the instruction has to be hard- coded into the BG as well. For example, if the instruction was to perform ACTION-A followed by ACTION-B, and then ACTION-C, the following rules would have to be encoded into the BG: Introduction One of the hallmarks of complex cognition is the ability to perform a multitude of tasks using the same underlying ar- chitecture. When given an instruction, the human brain is capable of processing and executing the instruction without the need for extensive rewiring of the underlying neural con- nections. As far as we are aware, no neural model to date has been shown to exhibit this ability. Eliasmith et al. (2012) describes what is currently the world’s largest functional brain model. While the model, called Spaun (for Semantic Pointer Architecture Unified Net- work), is able to perform 8 different cognitive tasks with- out necessitating changes to its architecture, the knowledge needed to complete these 8 tasks is hard-coded into the ac- tion selection mechanism (the basal ganglia) of the model, making it unable to perform any task other than the prede- fined 8. In this paper, we propose an extension to the Spaun action selection component making it capable of processing generic instructions. IF INIT, THEN state = ACTION-A Terminology IF state = ACTION-A, THEN state = ACTION-B Four key concepts are discussed in this paper: states, actions, rules, and instructions. States are internal variables that the action selection sys- tem monitors to figure out what is the best action to perform. States can be both internal (e.g. goal memories, working memories (WM)) and external (e.g. visual input) to the sys- tem. Actions are atomic commands within the architecture, and are typically motor commands (e.g. “write the number X”, “push the X button”) or cognitive commands (e.g. “remember the word X”, “route information from WM area X to WM area IF state = ACTION-B, THEN state = ACTION-C Several ACT-R models (e.g. (Taatgen & Lee, 2003), (Taatgen, 1999)) able to follow instructions, however no neu- ral implementation has been previously discussed. Aside from its architecture, Spaun is also unique in the way information is represented. Information is encoded and rep- resented using semantic pointers (Eliasmith, In Press). These representations are used in the SPA to define a type of vec- tor symbolic architecture (VSA). In typical VSAs, the vector

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.