Skip to content

2017 NeuroAI 1

Generated by DeepSeek

Neuroscience-Inspired Artificial Intelligence

  • 2017, Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, Matthew Botvinick

Summary

  • Past: Deep Learning

    • Origins in Neuroscience
      • ANN (1940s): Inspired by logical functions (McCulloch and Pitts, 1943)
      • Learning: Supervised (Rosenblatt, 1958), Unsupervised (Hebb, 1949)
    • Backpropagation & PDP
      • Backpropagation: Multi-layered learning (Rumelhart et al., 1985; Werbos, 1974)
      • PDP: Distributed cognition, error/reward optimization (Rumelhart et al., 1986)
    • PDP Influence
      • Distributed representations: Vectors for words/sentences (St. John and McClelland, 1990)
      • Models human behaviors (Hinton et al., 1986)
    • CNNs
      • Features: Nonlinear transduction, divisive normalization, max-pooling (Hubel and Wiesel, 1959)
      • Hierarchical organization (Fukushima, 1980; LeCun et al., 1989; Krizhevsky et al., 2012)
    • Advancements
      • Deep Belief Networks (Hinton et al., 2006)
      • Large datasets (Deng et al., 2009)
      • Dropout: Stochastic regularization (Hinton et al., 2012)

  • Past: Reinforcement Learning (RL)

    • Origins
      • Animal conditioning (Sutton and Barto, 1981)
      • TD Learning: Second-order conditioning (Sutton and Barto, 1998)
    • Impact
      • Maps states to actions (Sutton and Barto, 1998)
      • Applications: Robotics (Hafner and Riedmiller, 2011), Backgammon (Tesauro, 1995), Go (Silver et al., 2016)

  • Present: Attention

    • Modular Brain Structure
      • Subsystems: Memory, language, cognitive control (Anderson et al., 2004)
    • Visual Attention
      • Sequential processing (Koch and Ullman, 1985)
      • Neurocomputational models (Olshausen et al., 1993)
    • AI Architectures
      • Sequential glimpses (Larochelle and Hinton, 2010; Mnih et al., 2014)
      • Selective attention: Cluttered classification (Mnih et al., 2014)
      • Multi-object recognition (Ba et al., 2015)
      • Image-to-caption (Xu et al., 2015)
    • Internal Attention
      • Memory focus (Summerfield et al., 2006)
      • Machine translation (Bahdanau et al., 2014)
      • Content-addressable retrieval (Hopfield, 1982)
    • Generative Models
      • DRAW: Incremental image synthesis (Gregor et al., 2015)

  • Present: Episodic Memory

    • Neuroscience Basis
      • Episodic memory: Hippocampal one-shot encoding (Tulving, 1985)
    • DQN
      • Experience replay: Hippocampal-neocortical interaction (Mnih et al., 2015)
      • Reward-based replay (Schaul et al., 2015)
    • Complementary Learning
      • Hippocampal rapid encoding, neocortical consolidation (O’Neill et al., 2010)
    • Episodic Control
      • Rapid behavioral changes (Blundell et al., 2016)
      • One-shot learning (Vinyals et al., 2016)

  • Present: Working Memory

    • Human Basis
      • Prefrontal cortex, central executive (Baddeley, 2012)
    • RNN
      • Attractor dynamics (Elman, 1990)
    • LSTM
      • Gated memory maintenance (Hochreiter and Schmidhuber, 1997)
      • Query response (Zaremba and Sutskever, 2014)
    • DNC
      • External memory matrix (Graves et al., 2016)
      • Complex reasoning (Graves et al., 2014)

  • Present: Continual Learning

    • Challenges
      • Catastrophic forgetting (French, 1999)
    • Neuroscience Insights
      • Dendritic spine dynamics (Nishiyama and Yasuda, 2015)
      • Synaptic protection (Cichon and Gan, 2015)
    • AI Solutions
      • EWC: Weight consolidation (Kirkpatrick et al., 2017)

  • Future: Achievements

    • Object recognition (Krizhevsky et al., 2012)
    • Games: Atari (Mnih et al., 2015), Go (Silver et al., 2016), Poker (Moravčík et al., 2017)
    • Image/speech synthesis (Lake et al., 2015)
    • Multilingual translation (Wu et al., 2016)
    • Neural art (Gatys et al., 2015)

  • Future: Intuitive Understanding

    • Human Cognition
      • Innate physical concepts (Spelke and Kinzler, 2007)
    • AI Approaches
      • Scene decomposition (Battaglia et al., 2016)
      • Deep RL commonsense (Denil et al., 2016)
      • Deep generative models: Disentangled representations (Higgins et al., 2016)

  • Future: Efficient Learning

    • Human Efficiency
      • Few-shot learning (Lake et al., 2016)
    • AI Progress
      • Probabilistic models (Lake et al., 2015)
      • DRAW-based one-shot learning (Rezende et al., 2016b)
      • Meta-learning (Santoro et al., 2016)

  • Future: Transfer Learning

    • AI Architectures
      • Progressive Networks: Knowledge transfer (Rusu et al., 2016a)
      • Compositional representations (Higgins et al., 2016)
    • Neuroscience Basis
      • Grid codes: Abstract reasoning (Constantinescu et al., 2016)

  • Future: Imagination & Planning

    • Limitations
      • Model-free RL inefficiency (Daw et al., 2005)
    • Human/Animal Planning
      • Simulation-based (Daw et al., 2005)
      • Hippocampal preplay (Johnson and Redish, 2007)
    • AI Techniques
      • Dyna: Hypothetical experiences (Sutton, 1991)
      • Model-based RL, MCTS (Silver et al., 2016)
      • Deep generative models: Coherent sequences (Eslami et al., 2016)

  • Future: Virtual Brain Analytics

    • Neuroscience Tools
      • Dimensionality reduction (Zahavy et al., 2016)
      • Receptive field mapping (Nguyen et al., 2016)
      • Linearized network analysis (Saxe et al., 2013)

  • From AI to Neuroscience

    • Applications
      • Neuroimaging: fMRI/MEG analysis (Cichy et al., 2014)
      • RL: TD prediction errors (Schultz et al., 1997)
      • CNN: Ventral visual stream (Yamins and DiCarlo, 2016)
      • LSTM: Prefrontal gating (O’Reilly and Frank, 2006)
      • Memory-augmented networks: Hippocampal querying (Kumaran and McClelland, 2012)
      • Meta-RL: Prefrontal RL (Duan et al., 2016)
      • Backpropagation: Biologically plausible approximations (Lillicrap et al., 2016)

Past: Deep Learning

  • Origins in Neuroscience:

    • ANN, inspired by neuroscience, were developed in the 1940s to compute logical functions (McCulloch and Pitts, 1943).
    • Learning mechanisms proposed include supervised incremental learning (Rosenblatt, 1958) and unsupervised encoding of environmental statistics (Hebb, 1949).

  • Backpropagation and Parallel Distributed Processing (PDP):

    • Backpropagation algorithm enabled learning in multi-layered networks (Rumelhart et al., 1985; Werbos, 1974).
    • PDP movement, led by neuroscientists, proposed cognition emerges from distributed interactions in neuron-like units, tuned by learning to minimize error or maximize reward (Rumelhart et al., 1986).
    • Contrasted with symbolic AI, PDP emphasized stochastic, parallelized processing inspired by brain function.

  • PDP Influence on AI:

    • PDP models introduced distributed representations (e.g., vectors for words/sentences), foundational for modern machine translation (St. John and McClelland, 1990; LeCun et al., 2015).
    • Successfully modeled diverse human behaviors using small-scale problems (Hinton et al., 1986).

  • CNNs:

    • CNNs incorporate neuroscience-inspired features: nonlinear transduction, divisive normalization, and max-pooling, derived from V1 simple and complex cell recordings (Hubel and Wiesel, 1959; Yamins and DiCarlo, 2016).
    • Replicate hierarchical cortical organization with convergent/divergent information flow (Fukushima, 1980; LeCun et al., 1989; Krizhevsky et al., 2012).
    • Successive non-linear computations enable invariant object recognition across pose, illumination, and scale.

  • Advancements in Deep Learning:

    • Deep belief networks advanced NN capabilities (Hinton et al., 2006).
    • Large datasets, inspired by human language research, improved training (Deng et al., 2009).
    • Dropout regularization, motivated by stochastic neuronal firing (Poisson-like statistics), enhances generalization (Hinton et al., 2012).

Past: Reinforcement Learning

  • Origins in Neuroscience:

    • RL emerged from studies of animal learning, particularly conditioning experiments (Sutton and Barto, 1981).
    • Temporal-difference (TD) methods, a core RL component, were inspired by animal behavior in second-order conditioning, where a conditioned stimulus gains significance through association with another conditioned stimulus.

  • TD Learning Mechanism:

    • TD methods learn from differences between successive predictions in real-time, without waiting for actual rewards.
    • Provides a natural explanation for second-order conditioning and broader neuroscience findings.

  • Impact on AI:

    • RL maps environmental states to actions to maximize future rewards, widely used in AI research (Sutton and Barto, 1998).
    • TD methods and related techniques enabled advances in robotic control (Hafner and Riedmiller, 2011), expert backgammon play (Tesauro, 1995), and Go (Silver et al., 2016).

Present: Attention

  • Modular Brain Structure:

    • Biological brains are modular, with distinct subsystems for memory, language, and cognitive control (Anderson et al., 2004; Shallice, 1988).
    • This modularity influences AI architectures, incorporating specialized subsystems.

  • Visual Attention in Primates:

    • Primate visual systems prioritize processing through strategic shifts of attention, focusing on specific regions sequentially (Koch and Ullman, 1985; Moore and Zir imperfections, 2017; Posner and Petersen, 1990).
    • Neurocomputational models show this approach isolates relevant information, enhancing behavioral outcomes (Olshausen et al., 1993; Salinas and Abbott, 1997).

  • Attention in AI Architectures:

    • AI models mimic visual attention by taking sequential "glimpses" of input images, updating internal states, and selecting subsequent sampling locations (Larochelle and Hinton, 2010; Mnih et al., 2014).
    • Selective attention enables networks to ignore irrelevant objects, improving performance in cluttered object classification tasks (Mnih et al., 2014).
    • Attention reduces computational cost, scaling efficiently with input image size (Mnih et al., 2014).
    • Extended attention mechanisms outperform conventional CNNs in multi-object recognition, improving accuracy and efficiency (Ba et al., 2015).
    • Attention enhances image-to-caption generation (Xu et al., 2015).

  • Internal Attention and Memory:

    • Attention can focus on internal memory contents, inspired by neuroscience (Summerfield et al., 2006).
    • AI architectures use attention to selectively read from internal memory, advancing machine translation (Bahdanau et al., 2014) and memory/reasoning tasks (Graves et al., 2016).
    • Implements content-addressable retrieval, a concept from neuroscience (Hopfield, 1982).

  • Attention in Generative Models:

    • Deep generative models incorporate attention to synthesize realistic images (Hong et al., 2015; Reed et al., 2016).
    • The DRAW model uses attention to incrementally build images by focusing on portions of a "mental canvas" (Gregor et al., 2015).

Present: Episodic Memory

  • Multiple Memory Systems in Neuroscience:

    • Intelligent behavior relies on reinforcement-based and instance-based (episodic) memory systems (Tulving, 1985).
    • Episodic memory enables rapid, one-shot encoding of experiences in a content-addressable store, primarily in the hippocampus (Squire et al., 2004; Tulving, 2002).

  • Experience Replay in Deep Q-Networks (DQN):

    • DQN integrates RL with deep learning, achieving expert-level performance in Atari 2600 games by transforming image pixels into action policies (Mnih et al., 2015; Silver et al., 2016).
    • Experience replay, inspired by hippocampal-neocortical interactions, stores and replays training data offline to enhance data efficiency and prevent destabilization from correlated experiences (Kumaran et al., 2016; McClelland et al., 1995).
    • Replay of highly rewarding events improves DQN performance, mirroring hippocampal replay of rewarding experiences (Schaul et al., 2015; Singer and Frank, 2009).

  • Hippocampal-Neocortical Complementary Learning:

    • The hippocampus encodes novel information rapidly, which is consolidated to the neocortex during sleep or rest via replay of neural activity patterns (O’Neill et al., 2010; Skaggs and McNaughton, 1996).
    • This mechanism prevents catastrophic forgetting in NN by mitigating interference from sequential tasks (McClelland et al., 1995).

  • Episodic Control in AI:

    • Episodic control, inspired by hippocampal function, enables rapid behavioral changes by re-enacting rewarded action sequences from memory (Gershman and Daw, 2017).
    • AI architectures implementing episodic control store experiences (e.g., actions, rewards, game states) and select actions based on similarity to past events, improving early learning performance (Blundell et al., 2016).
    • Episodic control excels in one-shot learning tasks where traditional deep RL struggles (Blundell et al., 2016).

  • Broader Episodic-Like Memory Systems:

    • Episodic-like memory enables rapid learning of new concepts from few examples (Vinyals et al., 2016).
    • Future architectures may integrate rapid episodic memory with incremental learning, mirroring complementary learning systems in the mammalian brain.

Present: Working Memory

  • Human Working Memory:

    • Maintains and manipulates information in an active store, primarily in the prefrontal cortex and interconnected areas (Goldman-Rakic, 1990).
    • Classic models propose a central executive interacting with domain-specific buffers (e.g., visuo-spatial sketchpad) (Baddeley, 2012).

  • RNN:

    • Inspired by neuroscience, early RNNs with attractor dynamics model sequential behavior and working memory (Elman, 1990; Hopfield and Tank, 1986; Jordan, 1997).
    • Enable detailed modeling of human working memory (Botvinick and Plaut, 2006; Durstewitz et al., 2000).

  • Long Short-Term Memory (LSTM) Networks:

    • Evolved from RNNs, LSTMs gate information into a fixed activity state for maintenance and retrieval (Hochreiter and Schmidhuber, 1997).
    • Achieve state-of-the-art performance in tasks like responding to queries about latent variable states after training on computer code (Zaremba and Sutskever, 2014).
    • Intertwine sequence control and memory storage, unlike separate modules in classic human working memory models.

  • Differential Neural Computer (DNC):

    • Inspired by neuroscience’s separation of control and storage, DNC uses a NN controller to read/write to an external memory matrix (Graves et al., 2016).
    • Performs complex memory and reasoning tasks (e.g., finding shortest paths in graphs, manipulating blocks in Tower of Hanoi) via end-to-end optimization (Graves et al., 2014, 2016; Weston et al., 2014).
    • Overcomes limitations of traditional NN, previously thought to require symbol processing and variable binding (Fodor and Pylyshyn, 1988; Marcus, 1998).

  • Long-Term Memory Potential:

    • LSTMs and DNC can retain information over thousands of training cycles, suitable for long-term memory tasks like understanding book contents.

Present: Continual Learning

  • Continual Learning in Biological and Artificial Agents:

    • Both biological and artificial agents require continual learning to master new tasks without forgetting prior ones (Thrun and Mitchell, 1995).
    • NN suffer from catastrophic forgetting, where learning a new task overwrites parameters needed for previous tasks (French, 1999; McClelland et al., 1995).

  • Neuroscience Insights on Continual Learning:

    • Advanced neuroimaging (e.g., two-photon imaging) enables visualization of dendritic spine dynamics during learning at single-synapse resolution (Nishiyama and Yasuda, 2015).
    • Neocortical plasticity studies reveal mechanisms protecting prior task knowledge, including reduced synaptic lability via persistent dendritic spine enlargements (Cichon and Gan, 2015; Yang et al., 2009).
    • These structural changes correlate with task retention over months; erasing them with synaptic optogenetics causes forgetting (Hayashi-Takagi et al., 2015).
    • Theoretical models suggest memories are protected by synapses transitioning through states with varying plasticity levels (Fusi et al., 2005).

  • AI Developments Inspired by Neuroscience:

    • Elastic Weight Consolidation (EWC) algorithm, inspired by neuroscience, slows learning in weights critical to prior tasks, anchoring them to previous solutions (Kirkpatrick et al., 2017).
    • EWC enables deep RL networks to support continual learning without expanding network capacity, efficiently sharing weights across related tasks.

Future

  • AI Performance Achievements:

    • Artificial systems match human performance in object recognition tasks (Krizhevsky et al., 2012).
    • Outperform expert humans in Atari video games (Mnih et al., 2015), Go (Silver et al., 2016), and heads-up poker (Moravčík et al., 2017).
    • Generate near-realistic synthetic images and human speech simulations (Lake et al., 2015; van den Oord et al., 2016).
    • Enable multilingual translation (Wu et al., 2016) and create "neural art" mimicking famous painters (Gatys et al., 2015).

  • Neuroscience Advancements:

    • New tools in brain imaging and genetic bioengineering provide detailed insights into neural circuit computations (Deisseroth and Schnitzer, 2013).
    • Promise a deeper understanding of mammalian brain function.

Future: Intuitive Understanding of the Physical World

  • Core Human Cognitive Abilities:

    • Human infants possess innate knowledge of physical world concepts like space, number, and objectness, enabling compositional mental models for inference and prediction (Gilmore et al., 2007; Gopnik and Schulz, 2004; Spelke and Kinzler, 2007; Battaglia et al., 2013).

  • NN Architectures for Scene Reasoning:

    • Novel architectures decompose scenes into objects and their relations, mimicking human-like reasoning (Battaglia et al., 2016; Chang et al., 2016; Eslami et al., 2016).
    • Achieve human-level performance on complex reasoning tasks (Santoro et al., 2017).

  • Deep RL for Commonsense:

    • Deep RL models simulate how children develop commonsense through interactive experiments (Denil et al., 2016).

  • Deep Generative Models for Object Representation:

    • Models construct rich object models from raw sensory inputs, leveraging neuroscience-inspired constraints like redundancy reduction (Barlow, 1959; Higgins et al., 2016).
    • Learned latent representations are disentangled (e.g., shape, position) and exhibit compositional properties, enabling flexible transfer to new tasks (Eslami et al., 2016; Higgins et al., 2016; Rezende et al., 2016a).

Future: Efficient Learning

  • Human Cognitive Efficiency:

    • Humans rapidly learn new concepts from few examples, using prior knowledge for flexible inductive inferences.
    • Excel in tasks like the "characters challenge," distinguishing novel handwritten characters after one example (Lake et al., 2016).

  • AI Progress in Few-Shot Learning:

    • Structured probabilistic models enable inferences and sample generation from single examples (Lake et al., 2015).
    • Deep generative models, based on DRAW, support one-shot concept learning and sample generation (Rezende et al., 2016b).
    • Networks that "learn to learn" leverage prior experience for one-shot learning and faster RL task acquisition (Santoro et al., 2016; Vinyals et al., 2016; Wang et al., 2016).

  • Neuroscience Roots:

    • "Learning to learn" concept originates from animal learning studies (Harlow, 1949).
    • Further explored in developmental psychology (Adolph, 2005; Kemp et al., 2010; Smith, 1995).

Future: Transfer Learning

  • AI Architectures for Generalization:

    • Compositional representations enable zero-shot inferences for novel shapes outside training distributions (Higgins et al., 2016).
    • Progressive networks transfer knowledge from one video game to another, supporting rapid learning and far transfer (Rusu et al., 2016a).
    • Progressive networks reduce training time by transferring knowledge from simulated robotic environments to real-world robot arms (Rusu et al., 2016b).

  • Relation to Human Task Learning:

    • Progressive network architecture resembles computational models of sequential task learning in humans (Collins and Koechlin, 2012; Donoso et al., 2014).
    • Deep networks solve visual analogies, indicating progress in relational reasoning (Reed et al., 2015).

  • Neural Coding for Abstract Knowledge:

    • Conceptual representations may encode invariant, relational information across sensory domains (Doumas et al., 2008).
    • Grid codes in the entorhinal cortex, which encode allocentric space with hexagonal patterns, may support abstract reasoning (Constantinescu et al., 2016; Rowland et al., 2016).
    • Functional neuroimaging suggests grid-like codes during abstract categorization tasks, indicating periodic encoding in human knowledge organization (Constantinescu et al., 2016).
    • Grid codes may decompose state spaces efficiently, aiding subgoal discovery and hierarchical planning (Stachenfeld et al., 2014).

Future: Imagination and Planning

  • Deep RL Limitations:

    • Deep RL (e.g., DQN) operates reactively, mapping perceptual inputs to actions to maximize future value (model-free RL).
    • Drawbacks: data inefficiency (requires extensive experience) and inflexibility to outcome value changes (Daw et al., 2005).

  • Human and Animal Planning:

    • Humans use simulation-based planning, forecasting long-term outcomes via internal environmental models (Daw et al., 2005; Dolan and Dayan, 2013; Tolman, 1948).
    • Animals (e.g., scrub jays, rats) exhibit planning: scrub jays consider future food recovery conditions (Raby et al., 2007); rats use cognitive maps for navigation and one-shot learning (Daw et al., 2005; Tolman, 1948).

  • AI Planning Techniques:

    • Early AI planning (e.g., Dyna) inspired by mental models for generating hypothetical experiences (Sutton, 1991; Craik, 1943).
    • Model-based RL and Monte Carlo tree search (MCTS) enable forward search, contributing to expert-level Go performance (Silver et al., 2016; Browne et al., 2012).

  • Hippocampal Role in Planning:

    • Hippocampus supports simulation-based planning across species, creating spatially and temporally coherent imagined experiences (Hassabis and Maguire, 2007, 2009; Schacter et al., 2012).
    • Rat hippocampal "preplay" activity during navigation resembles imagined trajectories (Johnson and Redish, 2007; Ólafsdóttir et al., 2015; Pfeiffer and Foster, 2013).
    • Human non-spatial planning involves similar hippocampal processes (Doll et al., 2015; Kurth-Nelson et al., 2016).

  • Deep Generative Models:

    • Generate temporally consistent sequences reflecting realistic environments’ geometric layouts (Eslami et al., 2016; Rezende et al., 2016a, 2016b; Gemici et al., 2017; Oh et al., 2015).
    • Parallel hippocampal function in binding components for coherent imagined experiences (Hassabis and Maguire, 2007).

  • Neuroscience Insights for AI:

    • Hippocampus instantiates an internal environmental model; goal-contingent valuation occurs in orbitofrontal cortex or striatum (Redish, 2016).
    • Prefrontal cortex may initiate hippocampal model roll-forward, paralleling AI proposals of bidirectional controller-model interactions (Schmidhuber, 2014).
    • AI architectures separate controller and environmental model for simulation-based planning in physical object interactions (Hamrick et al., 2017).

  • Human Imagination Characteristics:

    • Constructive: humans recombine familiar elements into novel scenarios using compositional/disentangled representations (Eslami et al., 2016; Higgins et al., 2016; Rezende et al., 2016a).
    • Hierarchical and "jumpy": planning spans multiple temporal scales, considering terminal goals, interim choices, and steps (Balaguer et al., 2016; Solway et al., 2014; Huys et al., 2012).

Future: Virtual Brain Analytics

  • Neuroscience Tools for AI Analysis:

    • Neuroscience techniques (e.g., single-cell recording, neuroimaging, lesion studies) applied to AI systems enhance understanding of computations and representations in complex tasks.
    • Termed "virtual brain analytics" for interpreting AI "black boxes."

  • Dimensionality Reduction:

    • Common in neuroscience, dimensionality reduction visualizes brain states and has been adapted to analyze NN states (Zahavy et al., 2016).

  • Receptive Field Mapping:

    • Used to determine response properties of NN units.
    • Activity maximization generates synthetic images by maximizing specific unit activity (Nguyen et al., 2016; Simonyan et al., 2013).

  • Linearized Network Analysis:

    • Neuroscience-inspired analyses of linearized networks reveal principles for optimizing learning, understanding network depth, and representational structure (McClelland and Rogers, 2003; Saxe et al., 2013).

  • Advantages in AI Research:

    • AI researchers have complete system knowledge and can causally manipulate components, enabling precise analysis and hypothesis-driven experiments (Jonas and Kording, 2017; Krakauer et al., 2017).

From AI to Neuroscience

  • Machine Learning in Neuroimaging:

    • Machine learning enhances analysis of fMRI and MEG data through multivariate techniques (Cichy et al., 2014; Çukur et al., 2013; Kriegeskorte and Kievit, 2013).
    • Promises advancements in connectomic analysis (Glasser et al., 2016).

  • RL and Neuroscience:

    • RL concepts, inspired by animal psychology, align with neural signals in midbrain dopaminergic neurons, resembling temporal difference (TD) prediction errors (O’Doherty et al., 2003; Schultz et al., 1997).
    • Suggests the brain implements TD learning.

  • CNN and Visual Processing:

    • CNN architectures explain neural representations in the ventral visual stream of humans and monkeys (Khaligh-Razavi and Kriegeskorte, 2014; Yamins and DiCarlo, 2016).
    • Deep supervised networks account for increased coding of object properties (e.g., position, size) higher up the ventral visual stream (Hong et al., 2016).

  • Long Short-Term Memory (LSTM) and Working Memory:

    • LSTM properties inform models of working memory, supporting gating-based maintenance of task-relevant information in the prefrontal cortex (Lloyd et al., 2012; O’Reilly and Frank, 2006).

  • NN with External Memory:

    • Memory-augmented networks enable iterative querying, critical for reasoning over multiple inputs (Sukhbaatar et al., 2015).
    • Suggests potential hippocampal mechanisms for similar cognitive processes (Kumaran and McClelland, 2012).

  • Meta-RL:

    • RL optimizes recurrent network weights to create a faster-learning emergent RL algorithm (Duan et al., 2016; Wang et al., 2016).
    • Connects to prefrontal cortex roles in RL alongside dopamine-based mechanisms (Tsutsui et al., 2016; Schultz et al., 1997).

  • Backpropagation and Biological Plausibility:

    • Random backward connections enable effective backpropagation without symmetric connectivity, relaxing biological implausibility (Liao et al., 2015; Lillicrap et al., 2016).
    • Hierarchical auto-encoder and energy-based networks approximate backpropagation using local information, aligning with spike-timing dependent plasticity (Scellier and Bengio, 2016; Whittington and Bogacz, 2017).
    • Local learning rules in supervised networks generate high-level invariances, such as mirror-symmetric tuning (Leibo et al., 2017).