The overfitted brain hypothesis

What is the purpose of dreaming? Many scientists have postulated a role for dreaming in learning, often with the aim of improving generative models. In this issue of Patterns, Erik Hoel proposes a novel hypothesis, namely, that dreaming provides a means to reduce overfitting. This hypothesis is interesting both for neuroscience and for the development of new machine-learning systems. What is the purpose of dreaming? Many scientists have postulated a role for dreaming in learning, often with the aim of improving generative models. In this issue of Patterns, Erik Hoel proposes a novel hypothesis, namely, that dreaming provides a means to reduce overfitting. This hypothesis is interesting both for neuroscience and for the development of new machine-learning systems. The reasons for how and why we dream are poorly understood; however, the suppression of sleep stages most closely associated with dreaming have been known to impair learning in mammals for some time.1Walker M.P. Stickgold R. Sleep-dependent learning and memory consolidation.Neuron. 2004; 44: 121-133Abstract Full Text Full Text PDF PubMed Scopus (617) Google Scholar Given the limits of experimental techniques in cognitive and systems neuroscience, theories and experiments that address the role of sleep in learning often do not consider the impact that dreaming specifically may have. In a new perspective in this issue of Patterns, titled “The overfitted brain: Dreams evolved to assist generalization”, Erik Hoel explores this question using concepts from machine learning.2Hoel E. The Overfitted Brain: Dreams evolved to assist generalization.Patterns. 2021; 2https://doi.org/10.1016/j.patter.2021.100244Abstract Full Text Full Text PDF Scopus (3) Google Scholar Interestingly, within machine learning, there is actually a long history of algorithmic techniques that use dream-like processes for learning.3Dayan P. Hinton G.E. Neal R.M. Zemel R.S. The Helmholtz machine.Neural Comput. 1995; 7: 889-904Crossref PubMed Scopus (657) Google Scholar, 4Hinton G.E. Sejnowski T.J. Learning and relearning in Boltzmann machines.in: McLellan J. Parallel distributed processing: Explorations in the microstructure of cognition. MIT Press, 1986: 2Google Scholar, 5Hinton G.E. Osindero S. Teh Y.-W. A fast learning algorithm for deep belief nets.Neural Comput. 2006; 18: 1527-1554Crossref PubMed Scopus (10227) Google Scholar, 6Hinton G.E. Dayan P. Frey B.J. Neal R.M. The “wake-sleep” algorithm for unsupervised neural networks.Science. 1995; 268: 1158-1161Crossref PubMed Scopus (562) Google Scholar Many of these were motivated by the challenges of training probabilistic generative models. Specifically, machine learning researchers often wrestle with the dilemma of needing to find a model that maximizes the marginal likelihood or evidence of observed data, despite the fact that they are unable to evaluate this intractable quantity. A large amount of machine learning research is devoted to circumventing this problem by instead maximizing a suitable lower bound on this quantity, referred to as the evidence lower bound (ELBO) and sometimes as negative variational free energy.7Kingma D.P. Welling M. Auto-encoding variational bayes.arXiv. 2013; arXiv:1312.6114v10Google Scholar A canonical example is the expectation-maximization (EM) algorithm,8Dempster A.P. Laird N.M. Rubin D.B. Maximum Likelihood from Incomplete Data via the EM Algorithm.J. R. Stat. Soc. B. 1977; 39: 1-38Google Scholar the iterative two-stage procedure used in fitting Gaussian mixture models and other hierarchical probabilistic models without closed-form marginal likelihoods. This involves an expectation, E-step, in which we fix model parameters and compute the expected posterior mixture assignments, and a maximization, M-step, in which we update model parameters to maximize the ELBO while keeping the posterior mixture assignments fixed. One can also think of EM as performing two steps of maximization of the ELBO, because during the E-step one is finding a posterior distribution that maximizes the negative variational free energy.9Neal R.M. Hinton G.E. A view of the EM algorithm that justifies incremental, sparse, and other variants.in: Jordan M.I. Learning in graphical models. Springer, 1998: 355-368Crossref Google Scholar Iterative two-step training procedures such as these are ubiquitous in approaches to solving the problem of fitting hierarchical probabilistic models. Possibly the most relevant in relation to Hoel’s theory is the wake-sleep algorithm for training Helmholtz machines.3Dayan P. Hinton G.E. Neal R.M. Zemel R.S. The Helmholtz machine.Neural Comput. 1995; 7: 889-904Crossref PubMed Scopus (657) Google Scholar,5Hinton G.E. Osindero S. Teh Y.-W. A fast learning algorithm for deep belief nets.Neural Comput. 2006; 18: 1527-1554Crossref PubMed Scopus (10227) Google Scholar The two phases of the wake-sleep algorithm5Hinton G.E. Osindero S. Teh Y.-W. A fast learning algorithm for deep belief nets.Neural Comput. 2006; 18: 1527-1554Crossref PubMed Scopus (10227) Google Scholar correspond to an input-driven “wake” phase, akin to the M-step of the EM algorithm, and an internal representation-driven “sleep” phase, akin to the E-step of the EM algorithm. During the wake phase, layers in a deep neural network are activated by fixed feedforward connections, while feedback connections are updated to maximize the probability of reconstructing the input (the M-step). The sleep phase plays out in reverse: layers are activated by fixed feedback connections, while feedforward connections are updated to find a posterior that better matches the generative distribution (the E-step). In this case, hidden layer activations during the sleep phase are interpreted as “dreams” because they are internally generated data constructed during a time when the network is cut off from sensory inputs. According to this theory, dreams are a way of aligning our recognition pathways with our generative pathways, and so, over time, dreams should come more and more to resemble that which is experienced during waking. Within neuroscience and psychology, recent theories inspired by these machine-learning algorithms have attempted to explain learning and the role that sleep and dreaming might contribute in a similar light.10Friston K. The free-energy principle: a unified brain theory?.Nat. Rev. Neurosci. 2010; 11: 127-138Crossref PubMed Scopus (2751) Google Scholar However, Hoel proposes a different role for dreams, also inspired by machine learning. Specifically, Hoel’s hypothesis is that dreams help to prevent overfitting. Specifically, he proposes that the purpose of dreaming is to aid generalization and robustness of learned neural representations obtained through interactive waking experience. Dreams, Hoel theorizes, are augmented samples of waking experiences that guide neural representations away from overfitting waking experiences. Hoel argues that the properties of these augmentations explain why dreams are both less detailed and more fantastic than waking experience but retain the sequential ordering that we are familiar with. This proposal is different from EM-style proposals because the dream phase is not used to improve the match between a generative model and a recognition model, but rather to regularize a single model. This essentially proposes that brains use a secondary training phase in order to engage in some regularization process courtesy of using corrupted versions of the original data. This is akin, to some degree, to the use of augmented data for training machine-learning models to improve robustness and generalization. Hoel’s proposal is interesting from a neuroscience perspective, as it provides a normative theory of dreaming that, unlike the EM-style proposals, can explain why dreams do not become more realistic over time. But it is also interesting from a pure machine-learning perspective. If Hoel is correct, there should be ways to incorporate the phenomenology of dreaming to algorithm design for training and regularizing artificial neural networks (ANNs). Given that a fundamental flaw of current deep neural networks is their inability to learn representations from data that generalize to out-of-distribution samples, this is an interesting proposal. Cognitive scientists and machine-learning researchers alike have argued that this ability is key to learning causal world models,11Schölkopf B. Locatello F. Bauer S. Ke N.R. Kalchbrenner N. Goyal A. Bengio Y. Towards Causal Representation Learning.arXiv. 2021; arXiv:2102.11107v1Google Scholar and it relies on being able to decompose sensory experiences into disjoint hierarchies of discrete objects with locally continuous features. For example, the ability to recognize a coworker in an unfamiliar location relies on the ability to separate the coworker from the workplace and attire one usually perceives them in. In this case the disjoint hierarchical objects are the coworker, attire, and environment, which themselves can be decomposed into further sub-objects (facial features, items of clothing, workplace furniture, etc.). While solutions to this problem in deep-learning research are still in their infancy, the most common paradigm is to include a mechanism that guides learned representations away from the training data, either explicitly through regularization in the objective function or implicitly by injecting simple independent noise to the training data. More complex strategies try to generate synthetic training examples that exploit counterfactuals to observations, thereby supporting learning of causal relations between discrete objects. Hoel’s proposals for the role of dreaming in human and animal learning can be interpreted as exploiting each of these strategies. Finally, Hoel speculates on the use of dream substitutions—dream-like stimuli generated to aid learning during wakefulness or to ameliorate the effects of sleep deprivation. This offers clear empirical predictions for human studies and implications for the use of augmented reality technologies. Forming our own speculations, other waking experiences such as the use of psychedelics may also offer alternative dream substitutions. Recent work shows how the use of psychedelics alters variability in neural activity.12Schartner M.M. Carhart-Harris R.L. Barrett A.B. Seth A.K. Muthukumaraswamy S.D. Increased spontaneous MEG signal diversity for psychoactive doses of ketamine, LSD and psilocybin.Sci. Rep. 2017; 7: 46421Crossref PubMed Scopus (102) Google Scholar Should this variability impact learning in the way Hoel’s theory suggests, we might also predict that psychoactive substances or other consciousness-altering experiences might promote more robust learning. The overfitted brain: Dreams evolved to assist generalizationErik HoelPatternsMay 14, 2021In BriefWhy do we dream? While it is known that dreams must be important for learning, it is unknown precisely how or why this is. This paper explores the many hypotheses around why organisms dream, eventually proposing that the evolved purpose of dreams can be identified from research on artificial neural networks. Specifically, the overfitted brain hypothesis claims that in our daily lives the brain learns its tasks too well, and dreams are necessary to stop this “overfitting.” Full-Text PDF Open Access

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.