Person

Demis Hassabis

CEO

Chief executive of Google DeepMind and a leading researcher in reinforcement learning, planning, and general AI systems.

Google DeepMind Website

Papers

openalex-author · RePEc: Research Papers in Economics

Mastering Atari, Go, chess and shogi by planning with a learned model

Abstract Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess1 and Go2, where a perfect simulator is available. However, in real-world problems, the dynamics governing the environment are often complex and unknown. Here we present the MuZero algorithm, which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. The MuZero algorithm learns an iterable model that produces predictions relevant to planning: the action-selection policy, the value function and the reward. When evaluated on 57 different Atari games3—the canonical video game environment for testing artificial intelligence techniques, in which model-based planning approaches have historically struggled4—the MuZero algorithm achieved state-of-the-art performance. When evaluated on Go, chess and shogi—canonical environments for high-performance planning—the MuZero algorithm matched, without any knowledge of the game dynamics, the superhuman performance of the AlphaZero algorithm5 that was supplied with the rules of the game.

openalex-author · Nature

Advancing regulatory variant effect prediction with AlphaGenome

Deep learning models that predict functional genomic measurements from DNA sequences are powerful tools for deciphering the genetic regulatory code. Existing methods involve a trade-off between input sequence length and prediction resolution, thereby limiting their modality scope and performance1-5. We present AlphaGenome, a unified DNA sequence model, which takes as input 1 Mb of DNA sequence and predicts thousands of functional genomic tracks up to single-base-pair resolution across diverse modalities. The modalities include gene expression, transcription initiation, chromatin accessibility, histone modifications, transcription factor binding, chromatin contact maps, splice site usage and splice junction coordinates and strength. Trained on human and mouse genomes, AlphaGenome matches or exceeds the strongest available external models in 25 of 26 evaluations of variant effect prediction. The ability of AlphaGenome to simultaneously score variant effects across all modalities accurately recapitulates the mechanisms of clinically relevant variants near the TAL1 oncogene6. To facilitate broader use, we provide tools for making genome track and variant effect predictions from sequence.

openalex-author · ArXiv.org

SIMA 2: A Generalist Embodied Agent for Virtual Worlds

We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a significant step toward active, goal-directed interaction within an embodied environment. Unlike prior work (e.g., SIMA 1) limited to simple language commands, SIMA 2 acts as an interactive partner, capable of reasoning about high-level goals, conversing with the user, and handling complex instructions given through language and images. Across a diverse portfolio of games, SIMA 2 substantially closes the gap with human performance and demonstrates robust generalization to previously unseen environments, all while retaining the base model's core reasoning capabilities. Furthermore, we demonstrate a capacity for open-ended self-improvement: by leveraging Gemini to generate tasks and provide rewards, SIMA 2 can autonomously learn new skills from scratch in a new environment. This work validates a path toward creating versatile and continuously learning agents for both virtual and, eventually, physical worlds.

openalex-author · Nature

Olympiad-level formal mathematical reasoning with reinforcement learning

A long-standing goal of artificial intelligence (AI) is to build systems capable of complex reasoning in vast domains, a task epitomized by mathematics with its boundless concepts and demand for rigorous proof. Recent AI systems, often reliant on human data, typically lack the formal verification necessary to guarantee correctness. By contrast, formal languages such as Lean1 offer an interactive environment that grounds reasoning, and reinforcement learning (RL) provides a mechanism for learning in such environments. Here we present AlphaProof, an AlphaZero-inspired2 agent that learns to find formal proofs through RL by training on millions of auto-formalized problems. For the most difficult problems, it uses test-time RL, a method of generating and learning from millions of related problem variants at inference time to enable deep, problem-specific adaptation. AlphaProof substantially improves state-of-the-art results on historical mathematics competition problems. At the 2024 International Mathematical Olympiad competition, our AI system, with AlphaProof as its core reasoning engine, solved three out of the five non-geometry problems, including the competition's most difficult problem. Combined with AlphaGeometry 23, this performance, achieved with multi-day computation, resulted in reaching a score equivalent to that of a silver medallist, marking the first time an AI system achieved any medal-level performance, to our knowledge. Our work demonstrates that learning at scale from grounded experience produces agents with complex mathematical reasoning strategies, paving the way for a reliable AI tool in complex mathematical problem solving.

openalex-author · bioRxiv

AlphaGenome: advancing regulatory variant effect prediction with a unified DNA sequence model

Deep learning models that predict functional genomic measurements from DNA sequence are powerful tools for deciphering the genetic regulatory code. Existing methods trade off between input sequence length and prediction resolution, thereby limiting their modality scope and performance. We present AlphaGenome, which takes as input 1 megabase of DNA sequence and predicts thousands of functional genomic tracks up to single base pair resolution across diverse modalities – including gene expression, transcription initiation, chromatin accessibility, histone modifications, transcription factor binding, chro- matin contact maps, splice site usage, and splice junction coordinates and strength. Trained on human and mouse genomes, AlphaGenome matches or exceeds the strongest respective available external models on 24 out of 26 evaluations on variant effect prediction. AlphaGenome’s ability to simultaneously score variant effects across all modalities accurately recapitulates the mechanisms of clinically-relevant variants near the TAL1 oncogene. To facilitate broader use, we provide tools for making genome track and variant effect predictions from sequence.

openalex-author · Proceedings of the National Academy of Sciences

Bridging the human–AI knowledge gap through concept discovery and transfer in AlphaZero

AI systems have attained superhuman performance across various domains. If the hidden knowledge encoded in these highly capable systems can be leveraged, human knowledge and performance can be advanced. Yet, this internal knowledge is difficult to extract. Due to the vast space of possible internal representations, searching for meaningful new conceptual knowledge can be like finding a needle in a haystack. Here, we introduce a method that extracts new chess concepts from AlphaZero, an AI system that mastered chess via self-play without human supervision. Our method excavates vectors that represent concepts from AlphaZero's internal representations using convex optimization, and filters the concepts based on teachability (whether the concept is transferable to another AI agent) and novelty (whether the concept contains information not present in human chess games). These steps ensure that the discovered concepts are useful and meaningful. For the resulting set of concepts, prototypes (chess puzzle-solution pairs) are presented to experts for final validation. In a preliminary human study, four top chess grandmasters (all former or current world chess champions) were evaluated on their ability to solve concept prototype positions. All grandmasters showed improvement after the learning phase, suggesting that the concepts are at the frontier of human understanding. Despite the small scale, our result is a proof of concept demonstrating the possibility of leveraging knowledge from a highly capable AI system to advance the frontier of human knowledge; a development that could bear profound implications and shape how we interact with AI systems across many applications.

openalex-author · Nature

Addendum: Accurate structure prediction of biomolecular interactions with AlphaFold 3

In our original article, we provided an extensive

openalex-author · Nature

Learning high-accuracy error decoding for quantum processors

Building a large-scale quantum computer requires effective strategies to correct errors that inevitably arise in physical quantum systems1. Quantum error-correction codes2 present a way to reach this goal by encoding logical information redundantly into many physical qubits. A key challenge in implementing such codes is accurately decoding noisy syndrome information extracted from redundancy checks to obtain the correct encoded logical information. Here we develop a recurrent, transformer-based neural network that learns to decode the surface code, the leading quantum error-correction code3. Our decoder outperforms other state-of-the-art decoders on real-world data from Google's Sycamore quantum processor for distance-3 and distance-5 surface codes4. On distances up to 11, the decoder maintains its advantage on simulated data with realistic noise including cross-talk and leakage, utilizing soft readouts and leakage information. After training on approximate synthetic data, the decoder adapts to the more complex, but unknown, underlying error distribution by training on a limited budget of experimental samples. Our work illustrates the ability of machine learning to go beyond human-designed algorithms by learning from data directly, highlighting machine learning as a strong contender for decoding in quantum computers.

openalex-author · Nature

Scalable watermarking for identifying large language model outputs

Large language models (LLMs) have enabled the generation of high-quality synthetic text, often indistinguishable from human-written content, at a scale that can markedly affect the nature of the information ecosystem1-3. Watermarking can help identify synthetic text and limit accidental or deliberate misuse4, but has not been adopted in production systems owing to stringent quality, detectability and computational efficiency requirements. Here we describe SynthID-Text, a production-ready text watermarking scheme that preserves text quality and enables high detection accuracy, with minimal latency overhead. SynthID-Text does not affect LLM training and modifies only the sampling procedure; watermark detection is computationally efficient, without using the underlying LLM. To enable watermarking at scale, we develop an algorithm integrating watermarking with speculative sampling, an efficiency technique frequently used in production systems5. Evaluations across multiple LLMs empirically show that SynthID-Text provides improved detectability over comparable methods, and standard benchmarks and human side-by-side ratings indicate no change in LLM capabilities. To demonstrate the feasibility of watermarking in large-scale-production systems, we conducted a live experiment that assessed feedback from nearly 20 million Gemini6 responses, again confirming the preservation of text quality. We hope that the availability of SynthID-Text7 will facilitate further development of watermarking and responsible use of LLM systems.

openalex-author · arXiv (Cornell University)

De novo design of high-affinity protein binders with AlphaProteo

Computational design of protein-binding proteins is a fundamental capability with broad utility in biomedical research and biotechnology. Recent methods have made strides against some target proteins, but on-demand creation of high-affinity binders without multiple rounds of experimental testing remains an unsolved challenge. This technical report introduces AlphaProteo, a family of machine learning models for protein design, and details its performance on the de novo binder design problem. With AlphaProteo, we achieve 3- to 300-fold better binding affinities and higher experimental success rates than the best existing methods on seven target proteins. Our results suggest that AlphaProteo can generate binders "ready-to-use" for many research applications using only one round of medium-throughput screening and no further optimization.

openalex-author · Nature

Accurate structure prediction of biomolecular interactions with AlphaFold 3

The introduction of AlphaFold 21 has spurred a revolution in modelling the structure of proteins and their interactions, enabling a huge range of applications in protein modelling and design2-6. Here we describe our AlphaFold 3 model with a substantially updated diffusion-based architecture that is capable of predicting the joint structure of complexes including proteins, nucleic acids, small molecules, ions and modified residues. The new AlphaFold model demonstrates substantially improved accuracy over many previous specialized tools: far greater accuracy for protein-ligand interactions compared with state-of-the-art docking tools, much higher accuracy for protein-nucleic acid interactions compared with nucleic-acid-specific predictors and substantially higher antibody-antigen prediction accuracy compared with AlphaFold-Multimer v.2.37,8. Together, these results show that high-accuracy modelling across biomolecular space is possible within a single unified deep-learning framework.

openalex-author · arXiv (Cornell University)

Capabilities of Gemini Models in Medicine

Excellence in a wide variety of medical applications poses considerable challenges for AI, requiring advanced reasoning, access to up-to-date medical knowledge and understanding of complex multimodal data. Gemini models, with strong general capabilities in multimodal and long-context reasoning, offer exciting possibilities in medicine. Building on these core strengths of Gemini, we introduce Med-Gemini, a family of highly capable multimodal models that are specialized in medicine with the ability to seamlessly use web search, and that can be efficiently tailored to novel modalities using custom encoders. We evaluate Med-Gemini on 14 medical benchmarks, establishing new state-of-the-art (SoTA) performance on 10 of them, and surpass the GPT-4 model family on every benchmark where a direct comparison is viable, often by a wide margin. On the popular MedQA (USMLE) benchmark, our best-performing Med-Gemini model achieves SoTA performance of 91.1% accuracy, using a novel uncertainty-guided search strategy. On 7 multimodal benchmarks including NEJM Image Challenges and MMMU (health & medicine), Med-Gemini improves over GPT-4V by an average relative margin of 44.5%. We demonstrate the effectiveness of Med-Gemini's long-context capabilities through SoTA performance on a needle-in-a-haystack retrieval task from long de-identified health records and medical video question answering, surpassing prior bespoke methods using only in-context learning. Finally, Med-Gemini's performance suggests real-world utility by surpassing human experts on tasks such as medical text summarization, alongside demonstrations of promising potential for multimodal medical dialogue, medical research and education. Taken together, our results offer compelling evidence for Med-Gemini's potential, although further rigorous evaluation will be crucial before real-world deployment in this safety-critical domain.

openalex-author · arXiv (Cornell University)

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve excellent performance on language. It has a fixed-sized state, which reduces memory use and enables efficient inference on long sequences. We provide two sizes of models, containing 2B and 9B parameters, and provide pre-trained and instruction tuned variants for both. Our models achieve comparable performance to similarly-sized Gemma baselines despite being trained on fewer tokens.

openalex-author · Nature Communications

TacticAI: an AI assistant for football tactics

Identifying key patterns of tactics implemented by rival teams, and developing effective responses, lies at the heart of modern football. However, doing so algorithmically remains an open research challenge. To address this unmet need, we propose TacticAI, an AI football tactics assistant developed and evaluated in close collaboration with domain experts from Liverpool FC. We focus on analysing corner kicks, as they offer coaches the most direct opportunities for interventions and improvements. TacticAI incorporates both a predictive and a generative component, allowing the coaches to effectively sample and explore alternative player setups for each corner kick routine and to select those with the highest predicted likelihood of success. We validate TacticAI on a number of relevant benchmark tasks: predicting receivers and shot attempts and recommending player position adjustments. The utility of TacticAI is validated by a qualitative study conducted with football domain experts at Liverpool FC. We show that TacticAI's model suggestions are not only indistinguishable from real tactics, but also favoured over existing tactics 90% of the time, and that TacticAI offers an effective corner kick retrieval system. TacticAI achieves these results despite the limited availability of gold-standard data, achieving data efficiency through geometric deep learning.

openalex-author · arXiv (Cornell University)

Gemma: Open Models Based on Gemini Research and Technology

This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. We release two sizes of models (2 billion and 7 billion parameters), and provide both pretrained and fine-tuned checkpoints. Gemma outperforms similarly sized open models on 11 out of 18 text-based tasks, and we present comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development. We believe the responsible release of LLMs is critical for improving the safety of frontier models, and for enabling the next wave of LLM innovations.

openalex-author · The Keio Journal of Medicine

Accelerating Scientific Discovery with AI

This lecture will explore Google DeepMind’s groundbreaking AI research and its transformative impact on the trajectory of scientific progress. It traces the path from AlphaGo, which mastered the complex game of Go through self-learning and innovative problem-solving, to AlphaFold, which revolutionized the field of protein structure prediction. Then, looking ahead to further AI-led advancements in medicine, materials science, climate science, and more, this lecture emphasizes Google DeepMind’s commitment to safety and responsibility while pursuing advances in AI that benefit humanity.

openalex-author · Nucleic Acids Research

AlphaFold Protein Structure Database in 2024: providing structure coverage for over 214 million protein sequences

The AlphaFold Database Protein Structure Database (AlphaFold DB, https://alphafold.ebi.ac.uk) has significantly impacted structural biology by amassing over 214 million predicted protein structures, expanding from the initial 300k structures released in 2021. Enabled by the groundbreaking AlphaFold2 artificial intelligence (AI) system, the predictions archived in AlphaFold DB have been integrated into primary data resources such as PDB, UniProt, Ensembl, InterPro and MobiDB. Our manuscript details subsequent enhancements in data archiving, covering successive releases encompassing model organisms, global health proteomes, Swiss-Prot integration, and a host of curated protein datasets. We detail the data access mechanisms of AlphaFold DB, from direct file access via FTP to advanced queries using Google Cloud Public Datasets and the programmatic access endpoints of the database. We also discuss the improvements and services added since its initial release, including enhancements to the Predicted Aligned Error viewer, customisation options for the 3D viewer, and improvements in the search engine of AlphaFold DB.

openalex-author · arXiv (Cornell University)

Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero

Artificial Intelligence (AI) systems have made remarkable progress, attaining super-human performance across various domains. This presents us with an opportunity to further human knowledge and improve human expert performance by leveraging the hidden knowledge encoded within these highly performant AI systems. Yet, this knowledge is often hard to extract, and may be hard to understand or learn from. Here, we show that this is possible by proposing a new method that allows us to extract new chess concepts in AlphaZero, an AI system that mastered the game of chess via self-play without human supervision. Our analysis indicates that AlphaZero may encode knowledge that extends beyond the existing human knowledge, but knowledge that is ultimately not beyond human grasp, and can be successfully learned from. In a human study, we show that these concepts are learnable by top human experts, as four top chess grandmasters show improvements in solving the presented concept prototype positions. This marks an important first milestone in advancing the frontier of human knowledge by leveraging AI; a development that could bear profound implications and help us shape how we interact with AI systems across many AI applications.

openalex-author · Frontiers for Young Minds

AI Helping Science: The ‘Shape’ of Things to Come

When we started working with artificial intelligence (AI) more than a decade ago, people were skeptical about whether this technology would develop enough in the foreseeable future to do anything useful. But we held on to our faith in AI’s potential to benefit humanity. We used games like chess, Go and Atari to train and test our AI systems to become smarter and more capable. In 2016, we decided to use our smart systems to try to solve a 50-year-old fundamental problem in biology, called the protein-folding problem. This was the birth of AlphaFold, our AI system that predicts the three-dimensional structures of proteins based on their amino acid sequence. In this article, you will learn about AlphaFold’s achievements, which demonstrate the power of AI to dramatically accelerate scientific discovery and benefit society.

openalex-author · arXiv (Cornell University)

Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network

Quantum error-correction is a prerequisite for reliable quantum computation. Towards this goal, we present a recurrent, transformer-based neural network which learns to decode the surface code, the leading quantum error-correction code. Our decoder outperforms state-of-the-art algorithmic decoders on real-world data from Google's Sycamore quantum processor for distance 3 and 5 surface codes. On distances up to 11, the decoder maintains its advantage on simulated data with realistic noise including cross-talk, leakage, and analog readout signals, and sustains its accuracy far beyond the 25 cycles it was trained on. Our work illustrates the ability of machine learning to go beyond human-designed algorithms by learning from data directly, highlighting machine learning as a strong contender for decoding in quantum computers.

openalex-author · JAMA

The Protein Structure Prediction Revolution and Its Implications for Medicine

In this Viewpoint, 2023 Lasker award winners John Jumper and Demis Hassabis describe their invention, the artificial intelligence–based system AlphaFold, which is able to predict protein structure with great accuracy.

openalex-author · Zenodo (CERN European Organization for Nuclear Research)

Predictions for AlphaMissense

This repository provide AlphaMissense predictions. Please see the README for more details. For questions about AlphaMissense or the prediction Database please email [email protected].

openalex-author · Science

Accurate proteome-wide missense variant effect prediction with AlphaMissense

The vast majority of missense variants observed in the human genome are of unknown clinical significance. We present AlphaMissense, an adaptation of AlphaFold fine-tuned on human and primate variant population frequency databases to predict missense variant pathogenicity. By combining structural context and evolutionary conservation, our model achieves state-of-the-art results across a wide range of genetic and experimental benchmarks, all without explicitly training on such data. The average pathogenicity score of genes is also predictive for their cell essentiality, capable of identifying short essential genes that existing statistical approaches are underpowered to detect. As a resource to the community, we provide a database of predictions for all possible human single amino acid substitutions and classify 89% of missense variants as either likely benign or likely pathogenic.

openalex-author · arXiv (Cornell University)

Diversifying AI: Towards Creative Chess with AlphaZero

In recent years, Artificial Intelligence (AI) systems have surpassed human intelligence in a variety of computational tasks. However, AI systems, like humans, make mistakes, have blind spots, hallucinate, and struggle to generalize to new situations. This work explores whether AI can benefit from creative decision-making mechanisms when pushed to the limits of its computational rationality. In particular, we investigate whether a team of diverse AI systems can outperform a single AI in challenging tasks by generating more ideas as a group and then selecting the best ones. We study this question in the game of chess, the so-called drosophila of AI. We build on AlphaZero (AZ) and extend it to represent a league of agents via a latent-conditioned architecture, which we call AZ_db. We train AZ_db to generate a wider range of ideas using behavioral diversity techniques and select the most promising ones with sub-additive planning. Our experiments suggest that AZ_db plays chess in diverse ways, solves more puzzles as a group and outperforms a more homogeneous team. Notably, AZ_db solves twice as many challenging puzzles as AZ, including the challenging Penrose positions. When playing chess from different openings, we notice that players in AZ_db specialize in different openings, and that selecting a player for each opening using sub-additive planning results in a 50 Elo improvement over AZ. Our findings suggest that diversity bonuses emerge in teams of AI agents, just as they do in teams of humans and that diversity is a valuable asset in solving computationally hard problems.

openalex-author · Zenodo (CERN European Organization for Nuclear Research)

Source code for AlphaMissense

This package provides the AlphaMissense model implementation. This implementation is provided for reproducibility of the AlphaMissense 2023 publication.

openalex-author · Nature

Faster sorting algorithms discovered using deep reinforcement learning

Fundamental algorithms such as sorting or hashing are used trillions of times on any given day1. As demand for computation grows, it has become critical for these algorithms to be as performant as possible. Whereas remarkable progress has been achieved in the past2, making further improvements on the efficiency of these routines has proved challenging for both human scientists and computational approaches. Here we show how artificial intelligence can go beyond the current state of the art by discovering hitherto unknown routines. To realize this, we formulated the task of finding a better sorting routine as a single-player game. We then trained a new deep reinforcement learning agent, AlphaDev, to play this game. AlphaDev discovered small sorting algorithms from scratch that outperformed previously known human benchmarks. These algorithms have been integrated into the LLVM standard C++ sort library3. This change to this part of the sort library represents the replacement of a component with an algorithm that has been automatically discovered using reinforcement learning. We also present results in extra domains, showcasing the generality of the approach.

openalex-author · Pour la Science

« L’IA va accélérer les avancées scientifiques »

No abstract available from the OpenAlex source record.

openalex-author · Science

Mastering the game of Stratego with model-free multiagent reinforcement learning

We introduce DeepNash, an autonomous agent that plays the imperfect information game Stratego at a human expert level. Stratego is one of the few iconic board games that artificial intelligence (AI) has not yet mastered. It is a game characterized by a twin challenge: It requires long-term strategic thinking as in chess, but it also requires dealing with imperfect information as in poker. The technique underpinning DeepNash uses a game-theoretic, model-free deep reinforcement learning method, without search, that learns to master Stratego through self-play from scratch. DeepNash beat existing state-of-the-art AI methods in Stratego and achieved a year-to-date (2022) and all-time top-three ranking on the Gravon games platform, competing with human expert players.

openalex-author · Proceedings of the National Academy of Sciences

Acquisition of chess knowledge in AlphaZero

We analyze the knowledge acquired by AlphaZero, a neural network engine that learns chess solely by playing against itself yet becomes capable of outperforming human chess players. Although the system trains without access to human games or guidance, it appears to learn concepts analogous to those used by human chess players. We provide two lines of evidence. Linear probes applied to AlphaZero's internal state enable us to quantify when and where such concepts are represented in the network. We also describe a behavioral analysis of opening play, including qualitative commentary by a former world chess champion.

openalex-author · Zenodo (CERN European Organization for Nuclear Research)

Figure Data for the paper "Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning"

Data Release for Article: Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning This package releases a Python notebook reproducing the quantitative figures featured in the research article "Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning". Usage The notebook can be uploaded to and executed using the [Colab](https://colab.research.google.com) runtime service. The Python notebook is tested against Python `3.7`. License and disclaimer Copyright 2022 DeepMind Technologies Limited All software is licensed under the Apache License, Version 2.0 (Apache 2.0); you may not use this file except in compliance with the Apache 2.0 license. You may obtain a copy of the Apache 2.0 license at: https://www.apache.org/licenses/LICENSE-2.0 All other materials are licensed under the Creative Commons Attribution 4.0 International License (CC-BY). You may obtain a copy of the CC-BY license at: https://creativecommons.org/licenses/by/4.0/legalcode Unless required by applicable law or agreed to in writing, all software and materials distributed here under the Apache 2.0 or CC-BY licenses are distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the licenses for the specific language governing permissions and limitations under those licenses. This is not an official Google product.

openalex-author · Nature

Discovering faster matrix multiplication algorithms with reinforcement learning

Improving the efficiency of algorithms for fundamental computations can have a widespread impact, as it can affect the overall speed of a large amount of computations. Matrix multiplication is one such primitive task, occurring in many systems-from neural networks to scientific computing routines. The automatic discovery of algorithms using machine learning offers the prospect of reaching beyond human intuition and outperforming the current best human-designed algorithms. However, automating the algorithm discovery procedure is intricate, as the space of possible algorithms is enormous. Here we report a deep reinforcement learning approach based on AlphaZero1 for discovering efficient and provably correct algorithms for the multiplication of arbitrary matrices. Our agent, AlphaTensor, is trained to play a single-player game where the objective is finding tensor decompositions within a finite factor space. AlphaTensor discovered algorithms that outperform the state-of-the-art complexity for many matrix sizes. Particularly relevant is the case of 4 × 4 matrices in a finite field, where AlphaTensor's algorithm improves on Strassen's two-level algorithm for the first time, to our knowledge, since its discovery 50 years ago2. We further showcase the flexibility of AlphaTensor through different use-cases: algorithms with state-of-the-art complexity for structured matrix multiplication and improved practical efficiency by optimizing matrix multiplication for runtime on specific hardware. Our results highlight AlphaTensor's ability to accelerate the process of algorithmic discovery on a range of problems, and to optimize for different criteria.

openalex-author · arXiv (Cornell University)

Improving alignment of dialogue agents via targeted human judgements

We present Sparrow, an information-seeking dialogue agent trained to be more helpful, correct, and harmless compared to prompted language model baselines. We use reinforcement learning from human feedback to train our models with two new additions to help human raters judge agent behaviour. First, to make our agent more helpful and harmless, we break down the requirements for good dialogue into natural language rules the agent should follow, and ask raters about each rule separately. We demonstrate that this breakdown enables us to collect more targeted human judgements of agent behaviour and allows for more efficient rule-conditional reward models. Second, our agent provides evidence from sources supporting factual claims when collecting preference judgements over model statements. For factual questions, evidence provided by Sparrow supports the sampled response 78% of the time. Sparrow is preferred more often than baselines while being more resilient to adversarial probing by humans, violating our rules only 8% of the time when probed. Finally, we conduct extensive analyses showing that though our model learns to follow our rules it can exhibit distributional biases.

openalex-author · Science

Response to Comment on “Pushing the frontiers of density functionals by solving the fractional electron problem”

Gerasimov et al. claim that the ability of DM21 to respect fractional charge (FC) and fractional spin (FS) conditions outside of the training set has not been demonstrated in our paper. This is based on (i) asserting that the training set has a ~50% overlap with our bond-breaking benchmark (BBB) and (ii) questioning the validity and accuracy of our other generalization examples. We disagree with their analysis and believe that the points raised are either incorrect or not relevant to the main conclusions of the paper and to the assessment of general quality of DM21.

openalex-author · Scientific Reports

Multiagent off-screen behavior prediction in football

In multiagent worlds, several decision-making individuals interact while adhering to the dynamics constraints imposed by the environment. These interactions, combined with the potential stochasticity of the agents' dynamic behaviors, make such systems complex and interesting to study from a decision-making perspective. Significant research has been conducted on learning models for forward-direction estimation of agent behaviors, for example, pedestrian predictions used for collision-avoidance in self-driving cars. In many settings, only sporadic observations of agents may be available in a given trajectory sequence. In football, subsets of players may come in and out of view of broadcast video footage, while unobserved players continue to interact off-screen. In this paper, we study the problem of multiagent time-series imputation in the context of human football play, where available past and future observations of subsets of agents are used to estimate missing observations for other agents. Our approach, called the Graph Imputer, uses past and future information in combination with graph networks and variational autoencoders to enable learning of a distribution of imputed trajectories. We demonstrate our approach on multiagent settings involving players that are partially-observable, using the Graph Imputer to predict the behaviors of off-screen players. To quantitatively evaluate the approach, we conduct experiments on football matches with ground truth trajectory data, using a camera module to simulate the off-screen player state estimation setting. We subsequently use our approach for downstream football analytics under partial observability using the well-established framework of pitch control, which traditionally relies on fully observed data. We illustrate that our method outperforms several state-of-the-art approaches, including those hand-crafted for football, across all considered metrics.

openalex-author · Nature

Magnetic control of tokamak plasmas through deep reinforcement learning

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable1,2, including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and 'snowflake' configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained 'droplets' on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.

openalex-author · OPUS 4 (Zuse Institute Berlin)

Tackling Climate Change with Machine Learning

Climate change is one of the greatest challenges facing humanity, and we, as machine learning experts, may wonder how we can help. Here we describe how machine learning can be a powerful tool in reducing greenhouse gas emissions and helping society adapt to a changing climate. From smart grids to disaster management, we identify high impact problems where existing gaps can be filled by machine learning, in collaboration with other fields. Our recommendations encompass exciting research questions as well as promising business opportunities. We call on the machine learning community to join the global effort against climate change.

openalex-author · Communications of the ACM

Reimagining chess with AlphaZero

AI is driving the next evolution of chess, giving players a glimpse into the game's future.

openalex-author · Nature Methods

Protein structure predictions to atomic accuracy with AlphaFold

No abstract available from the OpenAlex source record.

openalex-author · GigaScience

3D-Beacons: decreasing the gap between protein sequences and structures through a federated network of protein structure data resources

While scientists can often infer the biological function of proteins from their 3-dimensional quaternary structures, the gap between the number of known protein sequences and their experimentally determined structures keeps increasing. A potential solution to this problem is presented by ever more sophisticated computational protein modeling approaches. While often powerful on their own, most methods have strengths and weaknesses. Therefore, it benefits researchers to examine models from various model providers and perform comparative analysis to identify what models can best address their specific use cases. To make data from a large array of model providers more easily accessible to the broader scientific community, we established 3D-Beacons, a collaborative initiative to create a federated network with unified data access mechanisms. The 3D-Beacons Network allows researchers to collate coordinate files and metadata for experimentally determined and theoretical protein models from state-of-the-art and specialist model providers and also from the Protein Data Bank.

openalex-author · Science

Pushing the frontiers of density functionals by solving the fractional electron problem

Density functional theory describes matter at the quantum level, but all popular approximations suffer from systematic errors that arise from the violation of mathematical properties of the exact functional. We overcame this fundamental limitation by training a neural network on molecular data and on fictitious systems with fractional charge and spin. The resulting functional, DM21 (DeepMind 21), correctly describes typical examples of artificial charge delocalization and strong correlation and performs better than traditional functionals on thorough benchmarks for main-group atoms and molecules. DM21 accurately models complex systems such as hydrogen chains, charged DNA base pairs, and diradical transition states. More crucially for the field, because our methodology relies on data and constraints, which are continually improving, it represents a viable pathway toward the exact universal functional.

openalex-author · arXiv (Cornell University)

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world. In this paper, we present an analysis of Transformer-based language model performance across a wide range of model scales -- from models with tens of millions of parameters up to a 280 billion parameter model called Gopher. These models are evaluated on 152 diverse tasks, achieving state-of-the-art performance across the majority. Gains from scale are largest in areas such as reading comprehension, fact-checking, and the identification of toxic language, but logical and mathematical reasoning see less benefit. We provide a holistic analysis of the training dataset and model's behaviour, covering the intersection of model scale with bias and toxicity. Finally we discuss the application of language models to AI safety and the mitigation of downstream harms.

openalex-author · Nature

Advancing mathematics by guiding human intuition with AI

The practice of mathematics involves discovering patterns and using these to formulate and prove conjectures, resulting in theorems. Since the 1960s, mathematicians have used computers to assist in the discovery of patterns and formulation of conjectures1, most famously in the Birch and Swinnerton-Dyer conjecture2, a Millennium Prize Problem3. Here we provide examples of new fundamental results in pure mathematics that have been discovered with the assistance of machine learning-demonstrating a method by which machine learning can aid mathematicians in discovering new conjectures and theorems. We propose a process of using machine learning to discover potential patterns and relations between mathematical objects, understanding them with attribution techniques and using these observations to guide intuition and propose conjectures. We outline this machine-learning-guided framework and demonstrate its successful application to current research questions in distinct areas of pure mathematics, in each case showing how it led to meaningful mathematical contributions on important open problems: a new connection between the algebraic and geometric structure of knots, and a candidate algorithm predicted by the combinatorial invariance conjecture for symmetric groups4. Our work may serve as a model for collaboration between the fields of mathematics and artificial intelligence (AI) that can achieve surprising results by leveraging the respective strengths of mathematicians and machine learning.

openalex-author · Nature Communications

Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons

In order to better understand how the brain perceives faces, it is important to know what objective drives learning in the ventral visual stream. To answer this question, we model neural responses to faces in the macaque inferotemporal (IT) cortex with a deep self-supervised generative model, β-VAE, which disentangles sensory data into interpretable latent factors, such as gender or age. Our results demonstrate a strong correspondence between the generative factors discovered by β-VAE and those coded by single IT neurons, beyond that found for the baselines, including the handcrafted state-of-the-art model of face perception, the Active Appearance Model, and deep classifiers. Moreover, β-VAE is able to reconstruct novel face images using signals from just a handful of cells. Together our results imply that optimising the disentangling objective leads to representations that closely resemble those in the IT at the single unit level. This points at disentangling as a plausible learning objective for the visual brain.

openalex-author · Research-Technology Management

DeepMind: From Games to Scientific Discovery

It’s a great honor to be awarded the IRI Medal and to join so many luminaries from the past, some of whom are scientific and innovation heroes of mine.Today I’m going to talk about the potential to...

openalex-author · Nucleic Acids Research

AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models

Abstract The AlphaFold Protein Structure Database (AlphaFold DB, https://alphafold.ebi.ac.uk) is an openly accessible, extensive database of high-accuracy protein-structure predictions. Powered by AlphaFold v2.0 of DeepMind, it has enabled an unprecedented expansion of the structural coverage of the known protein-sequence space. AlphaFold DB provides programmatic access to and interactive visualization of predicted atomic coordinates, per-residue and pairwise model-confidence estimates and predicted aligned errors. The initial release of AlphaFold DB contains over 360,000 predicted structures across 21 model-organism proteomes, which will soon be expanded to cover most of the (over 100 million) representative sequences from the UniRef90 data set.

openalex-author · bioRxiv

Protein complex prediction with AlphaFold-Multimer

While the vast majority of well-structured single protein chains can now be predicted to high accuracy due to the recent AlphaFold [1] model, the prediction of multi-chain protein complexes remains a challenge in many cases. In this work, we demonstrate that an AlphaFold model trained specifically for multimeric inputs of known stoichiometry, which we call AlphaFold-Multimer, significantly increases accuracy of predicted multimeric interfaces over input-adapted single-chain AlphaFold while maintaining high intra-chain accuracy. On a benchmark dataset of 17 heterodimer proteins without templates (introduced in [2]) we achieve at least medium accuracy (DockQ [3] ≥ 0.49) on 13 targets and high accuracy (DockQ ≥ 0.8) on 7 targets, compared to 9 targets of at least medium accuracy and 4 of high accuracy for the previous state of the art system (an AlphaFold-based system from [2]). We also predict structures for a large dataset of 4,446 recent protein complexes, from which we score all non-redundant interfaces with low template identity. For heteromeric interfaces we successfully predict the interface (DockQ ≥ 0.23) in 70% of cases, and produce high accuracy predictions (DockQ ≥ 0.8) in 26% of cases, an improvement of +27 and +14 percentage points over the flexible linker modification of AlphaFold [4] respectively. For homomeric inter-faces we successfully predict the interface in 72% of cases, and produce high accuracy predictions in 36% of cases, an improvement of +8 and +7 percentage points respectively.

openalex-author · Proteins: Structure, Function, and Bioinformatics

Applying and improving <scp>AlphaFold</scp> at <scp>CASP14</scp>

Abstract We describe the operation and improvement of AlphaFold, the system that was entered by the team AlphaFold2 to the “human” category in the 14th Critical Assessment of Protein Structure Prediction (CASP14). The AlphaFold system entered in CASP14 is entirely different to the one entered in CASP13. It used a novel end‐to‐end deep neural network trained to produce protein structures from amino acid sequence, multiple sequence alignments, and homologous proteins. In the assessors' ranking by summed z scores (>2.0), AlphaFold scored 244.0 compared to 90.8 by the next best group. The predictions made by AlphaFold had a median domain GDT_TS of 92.4; this is the first time that this level of average accuracy has been achieved during CASP, especially on the more difficult Free Modeling targets, and represents a significant improvement in the state of the art in protein structure prediction. We reported how AlphaFold was run as a human team during CASP14 and improved such that it now achieves an equivalent level of performance without intervention, opening the door to highly accurate large‐scale structure prediction.

openalex-author · Paper

Author response for "Applying and improving AlphaFold at CASP14"

No abstract available from the OpenAlex source record.

openalex-author · Nature

Highly accurate protein structure prediction for the human proteome

Protein structures can provide invaluable information, both for reasoning about biological processes and for enabling interventions such as structure-based drug development or targeted mutagenesis. After decades of effort, 17% of the total residues in human protein sequences are covered by an experimentally determined structure1. Here we markedly expand the structural coverage of the proteome by applying the state-of-the-art machine learning method, AlphaFold2, at a scale that covers almost the entire human proteome (98.5% of human proteins). The resulting dataset covers 58% of residues with a confident prediction, of which a subset (36% of all residues) have very high confidence. We introduce several metrics developed by building on the AlphaFold model and use them to interpret the dataset, identifying strong multi-domain predictions as well as regions that are likely to be disordered. Finally, we provide some case studies to illustrate how high-quality predictions could be used to generate biological hypotheses. We are making our predictions freely available to the community and anticipate that routine large-scale and high-accuracy structure prediction will become an important tool that will allow new questions to be addressed from a structural perspective.

openalex-author · Nature

Highly accurate protein structure prediction with AlphaFold

Proteins are essential to life, and understanding their structure can facilitate a mechanistic understanding of their function. Through an enormous experimental effort1-4, the structures of around 100,000 unique proteins have been determined5, but this represents a small fraction of the billions of known protein sequences6,7. Structural coverage is bottlenecked by the months to years of painstaking effort required to determine a single protein structure. Accurate computational approaches are needed to address this gap and to enable large-scale structural bioinformatics. Predicting the three-dimensional structure that a protein will adopt based solely on its amino acid sequence-the structure prediction component of the 'protein folding problem'8-has been an important open research problem for more than 50 years9. Despite recent progress10-14, existing methods fall far short of atomic accuracy, especially when no homologous structure is available. Here we provide the first computational method that can regularly predict protein structures with atomic accuracy even in cases in which no similar structure is known. We validated an entirely redesigned version of our neural network-based model, AlphaFold, in the challenging 14th Critical Assessment of protein Structure Prediction (CASP14)15, demonstrating accuracy competitive with experimental structures in a majority of cases and greatly outperforming other methods. Underpinning the latest version of AlphaFold is a novel machine learning approach that incorporates physical and biological knowledge about protein structure, leveraging multi-sequence alignments, into the design of the deep learning algorithm.

openalex-author · Journal of Artificial Intelligence Research

Game Plan: What AI can do for Football, and What Football can do for AI

The rapid progress in artificial intelligence (AI) and machine learning has opened unprecedented analytics possibilities in various team and individual sports, including baseball, basketball, and tennis. More recently, AI techniques have been applied to football, due to a huge increase in data collection by professional teams, increased computational power, and advances in machine learning, with the goal of better addressing new scientific challenges involved in the analysis of both individual players’ and coordinated teams’ behaviors. The research challenges associated with predictive and prescriptive football analytics require new developments and progress at the intersection of statistical learning, game theory, and computer vision. In this paper, we provide an overarching perspective highlighting how the combination of these fields, in particular, forms a unique microcosm for AI research, while offering mutual benefits for professional teams, spectators, and broadcasters in the years to come. We illustrate that this duality makes football analytics a game changer of tremendous value, in terms of not only changing the game of football itself, but also in terms of what this domain can mean for the field of AI. We review the state-of-the-art and exemplify the types of analysis enabled by combining the aforementioned fields, including illustrative examples of counterfactual analysis using predictive models, and the combination of game-theoretic analysis of penalty kicks with statistical learning of player attributes. We conclude by highlighting envisioned downstream impacts, including possibilities for extensions to other sports (real and virtual).

openalex-author · Nature Protocols

Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records

No abstract available from the OpenAlex source record.

openalex-author · arXiv (Cornell University)

Atomistic graph networks for experimental materials property prediction

Machine Learning (ML) has the potential to accelerate discovery of new materials and shed light on useful properties of existing materials. A key difficulty when applying ML in Materials Science is that experimental datasets of material properties tend to be small. In this work we show how material descriptors can be learned from the structures present in large scale datasets of material simulations; and how these descriptors can be used to improve the prediction of an experimental property, the energy of formation of a solid. The material descriptors are learned by training a Graph Neural Network to regress simulated formation energies from a material's atomistic structure. Using these learned features for experimental property predictions outperforms existing methods that are based solely on chemical composition. Moreover, we find that the advantage of our approach increases as the generalization requirements of the task are made more stringent, for example when limiting the amount of training data or when generalizing to unseen chemical spaces.

openalex-author · arXiv (Cornell University)

Alchemy: A structured task distribution for meta-reinforcement learning.

There has been rapidly growing interest in meta-learning as a method for increasing the flexibility and sample efficiency of reinforcement learning. One problem in this area of research, however, has been a scarcity of adequate benchmark tasks. In general, the structure underlying past benchmarks has either been too simple to be inherently interesting, or too ill-defined to support principled analysis. In the present work, we introduce a new benchmark for meta-RL research, which combines structural richness with structural transparency. Alchemy is a 3D video game, implemented in Unity, which involves a latent causal structure that is resampled procedurally from episode to episode, affording structure learning, online inference, hypothesis testing and action sequencing based on abstract domain knowledge. We evaluate a pair of powerful RL agents on Alchemy and present an in-depth analysis of one of these agents. Results clearly indicate a frank and specific failure of meta-learning, providing validation for Alchemy as a challenging benchmark for meta-RL. Concurrent with this report, we are releasing Alchemy as public resource, together with a suite of analysis tools and sample agent trajectories.

openalex-author · New Scientist

“With AI, you might unlock some of the secrets about how life works”

No abstract available from the OpenAlex source record.

openalex-author · bioRxiv

A model of egocentric to allocentric understanding in mammalian brains

In the mammalian brain, allocentric representations support efficient self-location and flexible navigation. A number of distinct populations of these spatial responses have been identified but no unified function has been shown to account for their emergence. Here we developed a network, trained with a simple predictive objective, that was capable of mapping egocentric information into an allocentric spatial reference frame. The prediction of visual inputs was sufficient to drive the appearance of spatial representations resembling those observed in rodents: head direction, boundary vector, and place cells, along with the recently discovered egocentric boundary cells, suggesting predictive coding as a principle for their emergence in animals. Strikingly, the network learned a solution for head direction tracking and stabilisation convergent with known biological connectivity. Moreover, like mammalian representations, responses were robust to environmental manipulations, including exposure to novel settings. In contrast to existing reinforcement learning approaches, agents equipped with this network were able to flexibly reuse learnt behaviours —adapting rapidly to unfamiliar environments. Thus, our results indicate that these representations, derived from a simple egocentric predictive framework, form an efficient basis-set for cognitive mapping.

openalex-author · Nature

Addendum: International evaluation of an AI system for breast cancer screening

No abstract available from the OpenAlex source record.

openalex-author · arXiv (Cornell University)

Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess

It is non-trivial to design engaging and balanced sets of game rules. Modern chess has evolved over centuries, but without a similar recourse to history, the consequences of rule changes to game dynamics are difficult to predict. AlphaZero provides an alternative in silico means of game balance assessment. It is a system that can learn near-optimal strategies for any rule set from scratch, without any human supervision, by continually learning from its own experience. In this study we use AlphaZero to creatively explore and design new chess variants. There is growing interest in chess variants like Fischer Random Chess, because of classical chess's voluminous opening theory, the high percentage of draws in professional play, and the non-negligible number of games that end while both players are still in their home preparation. We compare nine other variants that involve atomic changes to the rules of chess. The changes allow for novel strategic and tactical patterns to emerge, while keeping the games close to the original. By learning near-optimal strategies for each variant with AlphaZero, we determine what games between strong human players might look like if these variants were adopted. Qualitatively, several variants are very dynamic. An analytic comparison show that pieces are valued differently between variants, and that some variants are more decisive than classical chess. Our findings demonstrate the rich possibilities that lie beyond the rules of modern chess.

openalex-author · arXiv (Cornell University)

Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons

Deep supervised neural networks trained to classify objects have emerged as popular models of computation in the primate ventral stream. These models represent information with a high-dimensional distributed population code, implying that inferotemporal (IT) responses are also too complex to interpret at the single-neuron level. We challenge this view by modelling neural responses to faces in the macaque IT with a deep unsupervised generative model, beta-VAE. Unlike deep classifiers, beta-VAE "disentangles" sensory data into interpretable latent factors, such as gender or hair length. We found a remarkable correspondence between the generative factors discovered by the model and those coded by single IT neurons. Moreover, we were able to reconstruct face images using the signals from just a handful of cells. This suggests that the ventral visual stream may be optimising the disentangling objective, producing a neural code that is low-dimensional and semantically interpretable at the single-unit level.

openalex-author · Nature Medicine

Predicting conversion to wet age-related macular degeneration using deep learning

No abstract available from the OpenAlex source record.

openalex-author · Nature Physics

Author Correction: Unveiling the predictive power of static structure in glassy systems

No abstract available from the OpenAlex source record.

openalex-author · Bulletin of the American Physical Society

Unveiling the predictive power of static structure in glassy systems

Despite decades of theoretical studies, the nature of the glass transition remains elusive and debated, while the existence of structural predictors of its dynamics is a major open question. Recent approaches propose inferring predictors from a variety of human-defined features using machine learning. Here we determine the long-time evolution of a glassy system solely from the initial particle positions and without any handcrafted features, using graph neural networks as a powerful model. We show that this method outperforms current state-of-the-art methods, generalizing over a wide range of temperatures, pressures and densities. In shear experiments, it predicts the locations of rearranging particles. The structural predictors learned by our network exhibit a correlation length that increases with larger timescales to reach the size of our system. Beyond glasses, our method could apply to many other physical systems that map to a graph of local interaction. The physics that underlies the glass transition is both subtle and non-trivial. A machine learning approach based on graph networks is now shown to accurately predict the dynamics of glasses over a wide range of temperatures, pressures and densities.

openalex-author · arXiv (Cornell University)

MEMO: A Deep Network for Flexible Combination of Episodic Memories

Recent research developing neural network architectures with external memory have often used the benchmark bAbI question and answering dataset which provides a challenging number of tasks requiring reasoning. Here we employed a classic associative inference task from the memory-based reasoning neuroscience literature in order to more carefully probe the reasoning capacity of existing memory-augmented architectures. This task is thought to capture the essence of reasoning -- the appreciation of distant relationships among elements distributed across multiple facts or memories. Surprisingly, we found that current architectures struggle to reason over long distance associations. Similar results were obtained on a more complex task involving finding the shortest path between nodes in a path. We therefore developed MEMO, an architecture endowed with the capacity to reason over longer distances. This was accomplished with the addition of two novel components. First, it introduces a separation between memories (facts) stored in external memory and the items that comprise these facts in external memory. Second, it makes use of an adaptive retrieval mechanism, allowing a variable number of "memory hops" before the answer is produced. MEMO is capable of solving our novel reasoning tasks, as well as match state of the art results in bAbI.

openalex-author · Nature

Improved protein structure prediction using potentials from deep learning

No abstract available from the OpenAlex source record.

openalex-author · Nature

A distributional code for value in dopamine-based reinforcement learning

No abstract available from the OpenAlex source record.

openalex-author · Nature

International evaluation of an AI system for breast cancer screening

No abstract available from the OpenAlex source record.

openalex-author · Nature

Grandmaster level in StarCraft II using multi-agent reinforcement learning

No abstract available from the OpenAlex source record.

openalex-author · Proteins: Structure, Function, and Bioinformatics

Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13)

We describe AlphaFold, the protein structure prediction system that was entered by the group A7D in CASP13. Submissions were made by three free-modeling (FM) methods which combine the predictions of three neural networks. All three systems were guided by predictions of distances between pairs of residues produced by a neural network. Two systems assembled fragments produced by a generative neural network, one using scores from a network trained to regress GDT_TS. The third system shows that simple gradient descent on a properly constructed potential is able to perform on par with more expensive traditional search techniques and without requiring domain segmentation. In the CASP13 FM assessors' ranking by summed z-scores, this system scored highest with 68.3 vs 48.2 for the next closest group (an average GDT_TS of 61.4). The system produced high-accuracy structures (with GDT_TS scores of 70 or higher) for 11 out of 43 FM domains. Despite not explicitly using template information, the results in the template category were comparable to the best performing template-based methods.

openalex-author · Research Square

Developing Deep Learning Continuous Risk Models for Early Adverse Event Prediction in Electronic Health Records: an AKI Case Study

No abstract available from the OpenAlex source record.

openalex-author · Nature

A clinically applicable approach to continuous prediction of future acute kidney injury

No abstract available from the OpenAlex source record.

openalex-author · Science

Human-level performance in 3D multiplayer games with population-based reinforcement learning

Reinforcement learning (RL) has shown great success in increasingly complex single-agent environments and two-player turn-based games. However, the real world contains multiple agents, each learning and acting independently to cooperate and compete with other agents. We used a tournament-style evaluation to demonstrate that an agent can achieve human-level performance in a three-dimensional multiplayer first-person video game, Quake III Arena in Capture the Flag mode, using only pixels and game points scored as input. We used a two-tier optimization process in which a population of independent RL agents are trained concurrently from thousands of parallel matches on randomly generated environments. Each agent learns its own internal reward signal and rich representation of the world. These results indicate the great potential of multiagent reinforcement learning for artificial intelligence research.

openalex-author · Nature Human Behaviour

Slow escape decisions are swayed by trait anxiety

No abstract available from the OpenAlex source record.

openalex-author · Trends in Cognitive Sciences

Reinforcement Learning, Fast and Slow

Deep reinforcement learning (RL) methods have driven impressive advances in artificial intelligence in recent years, exceeding human performance in domains ranging from Atari to Go to no-limit poker. This progress has drawn the attention of cognitive scientists interested in understanding human learning. However, the concern has been raised that deep RL may be too sample-inefficient - that is, it may simply be too slow - to provide a plausible model of how humans learn. In the present review, we counter this critique by describing recently developed techniques that allow deep RL to operate more nimbly, solving problems much more quickly than previous methods. Although these techniques were developed in an AI context, we propose that they may have rich implications for psychology and neuroscience. A key insight, arising from these AI methods, concerns the fundamental connection between fast RL and slower, more incremental forms of learning.

openalex-author · Science

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

The game of chess is the longest-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. By contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go by reinforcement learning from self-play. In this paper, we generalize this approach into a single AlphaZero algorithm that can achieve superhuman performance in many challenging games. Starting from random play and given no domain knowledge except the game rules, AlphaZero convincingly defeated a world champion program in the games of chess and shogi (Japanese chess), as well as Go.

openalex-author · Neuron

Big-Loop Recurrence within the Hippocampal System Supports Integration of Information across Episodes

No abstract available from the OpenAlex source record.

openalex-author · Nature Medicine

Clinically applicable deep learning for diagnosis and referral in retinal disease

No abstract available from the OpenAlex source record.

openalex-author · Science

Neural scene representation and rendering

Scene representation-the process of converting visual sensory data into concise descriptions-is a requirement for intelligent behavior. Recent work has shown that neural networks excel at this task when provided with large, labeled datasets. However, removing the reliance on human labeling remains an important open problem. To this end, we introduce the Generative Query Network (GQN), a framework within which machines learn to represent scenes using only their own sensors. The GQN takes as input images of a scene taken from different viewpoints, constructs an internal representation, and uses this representation to predict the appearance of that scene from previously unobserved viewpoints. The GQN demonstrates representation learning without human labels or domain knowledge, paving the way toward machines that autonomously learn to understand the world around them.

openalex-author · Nature Neuroscience

Prefrontal cortex as a meta-reinforcement learning system

No abstract available from the OpenAlex source record.

openalex-author · Nature

Vector-based navigation using grid-like representations in artificial agents

No abstract available from the OpenAlex source record.

openalex-author · arXiv (Cornell University)

Unsupervised Predictive Memory in a Goal-Directed Agent

Animals execute goal-directed behaviours despite the limited range and scope of their sensors. To cope, they explore environments and store memories maintaining estimates of important information that is not presently available. Recently, progress has been made with artificial intelligence (AI) agents that learn to perform tasks from sensory input, even at a human level, by merging reinforcement learning (RL) algorithms with deep neural networks, and the excitement surrounding these results has led to the pursuit of related ideas as explanations of non-human animal learning. However, we demonstrate that contemporary RL algorithms struggle to solve simple tasks when enough information is concealed from the sensors of the agent, a property called "partial observability". An obvious requirement for handling partially observed tasks is access to extensive memory, but we show memory is not enough; it is critical that the right information be stored in the right format. We develop a model, the Memory, RL, and Inference Network (MERLIN), in which memory formation is guided by a process of predictive modeling. MERLIN facilitates the solution of tasks in 3D virtual reality environments for which partial observability is severe and memories must be maintained over long durations. Our model demonstrates a single learning agent architecture that can solve canonical behavioural tasks in psychology and neurobiology without strong simplifying assumptions about the dimensionality of sensory input or the duration of experiences.

openalex-author · Proceedings of the National Academy of Sciences

How cognitive and reactive fear circuits optimize escape decisions in humans

Flight initiation distance (FID), the distance at which an organism flees from an approaching threat, is an ecological metric of cost-benefit functions of escape decisions. We adapted the FID paradigm to investigate how fast- or slow-attacking "virtual predators" constrain escape decisions. We show that rapid escape decisions rely on "reactive fear" circuits in the periaqueductal gray and midcingulate cortex (MCC), while protracted escape decisions, defined by larger buffer zones, were associated with "cognitive fear" circuits, which include posterior cingulate cortex, hippocampus, and the ventromedial prefrontal cortex, circuits implicated in more complex information processing, cognitive avoidance strategies, and behavioral flexibility. Using a Bayesian decision-making model, we further show that optimization of escape decisions under rapid flight were localized to the MCC, a region involved in adaptive motor control, while the hippocampus is implicated in optimizing decisions that update and control slower escape initiation. These results demonstrate an unexplored link between defensive survival circuits and their role in adaptive escape decisions.

openalex-author · arXiv (Cornell University)

Memory-based Parameter Adaptation

Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the training distribution shifts, the network is slow to adapt, and when it does adapt, it typically performs badly on the training distribution before the shift. Our method, Memory-based Parameter Adaptation, stores examples in memory and then uses a context-based lookup to directly modify the weights of a neural network. Much higher learning rates can be used for this local adaptation, reneging the need for many iterations over similar data before good predictions can be made. As our method is memory-based, it alleviates several shortcomings of neural networks, such as catastrophic forgetting, fast, stable acquisition of new knowledge, learning with an imbalanced class labels, and fast learning during evaluation. We demonstrate this on a range of supervised tasks: large-scale image classification and language modelling.

openalex-author · Proceedings of the National Academy of Sciences

Reply to Huszár: The elastic weight consolidation penalty is empirically valid

In our recent work on elastic weight consolidation (EWC) (1) we show that forgetting in neural networks can be alleviated by using a quadratic penalty whose derivation was inspired by Bayesian evidence accumulation. In his letter (2), Dr. Huszar provides an alternative form for this penalty by following the standard work on expectation propagation using the Laplace approximation (3). He correctly argues that in cases when more than two tasks are undertaken the two forms of the penalty are different. Dr. Huszar also shows that for a toy linear regression problem his expression appears to be better. We would like to thank Dr. Huszar for pointing out … [↵][1]1To whom correspondence should be addressed. Email: [email protected]. [1]: #xref-corresp-1-1

openalex-author · Paper

Learning Dynamic State Abstractions for Model-Based Reinforcement Learning

A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed models that learn predictive and compact state representations, also called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment (ALE) from raw pixels. Furthermore, RL agents that use Monte-Carlo rollouts of these models as features for decision making outperform strong model-free baselines on the game MS_PACMAN, demonstrating the benefits of planning using learned dynamic state abstractions.

openalex-author · In: Bengio, Y and LeCun, Y, (eds.) Proceedings of the Sixth International Conference on Learning Representations (ICLR 2018). International Conference on Lea

SCAN: Learning Hierarchical Compositional Visual Concepts

The seemingly infinite diversity of the natural world arises from a relatively small set of coherent rules, such as the laws of physics or chemistry. We conjecture that these rules give rise to regularities that can be discovered through primarily unsupervised experiences and represented as abstract concepts. If such representations are compositional and hierarchical, they can be recombined into an exponentially large set of new concepts. This paper describes SCAN (Symbol-Concept Association Network), a new framework for learning such abstractions in the visual domain. SCAN learns concepts through fast symbol association, grounding them in disentangled visual primitives that are discovered in an unsupervised manner. Unlike state of the art multimodal generative model baselines, our approach requires very few pairings between symbols and images and makes no assumptions about the form of symbol representations. Once trained, SCAN is capable of multimodal bi-directional inference, generating a diverse set of image samples from symbolic descriptions and vice versa. It also allows for traversal and manipulation of the implicit hierarchy of visual concepts through symbolic instructions and learnt logical recombination operations. Such manipulations enable SCAN to break away from its training data distribution and imagine novel visual concepts through symbolically instructed recombination of previously learnt concepts.

openalex-author · arXiv (Cornell University)

Learning and Querying Fast Generative Models for Reinforcement Learning

A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed generative models that learn and operate on compact state representations, so-called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment from raw pixels. The computational speed-up of state-space models while maintaining high accuracy makes their application in RL feasible: We demonstrate that agents which query these models for decision making outperform strong model-free baselines on the game MSPACMAN, demonstrating the potential of using learned environment models for planning.

openalex-author · arXiv (Cornell University)

Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents

Psychlab is a simulated psychology laboratory inside the first-person 3D game world of DeepMind Lab (Beattie et al. 2016). Psychlab enables implementations of classical laboratory psychological experiments so that they work with both human and artificial agents. Psychlab has a simple and flexible API that enables users to easily create their own tasks. As examples, we are releasing Psychlab implementations of several classical experimental paradigms including visual search, change detection, random dot motion discrimination, and multiple object tracking. We also contribute a study of the visual psychophysics of a specific state-of-the-art deep reinforcement learning agent: UNREAL (Jaderberg et al. 2016). This study leads to the surprising conclusion that UNREAL learns more quickly about larger target stimuli than it does about smaller stimuli. In turn, this insight motivates a specific improvement in the form of a simple model of foveal vision that turns out to significantly boost UNREAL's performance, both on Psychlab tasks, and on standard DeepMind Lab tasks. By open-sourcing Psychlab we hope to facilitate a range of future such studies that simultaneously advance deep reinforcement learning and improve its links with cognitive science.

openalex-author · arXiv (Cornell University)

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

openalex-author · arXiv (Cornell University)

Parallel WaveNet: Fast High-Fidelity Speech Synthesis

The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system. However, because WaveNet relies on sequential generation of one audio sample at a time, it is poorly suited to today's massively parallel computers, and therefore hard to deploy in a real-time production setting. This paper introduces Probability Density Distillation, a new method for training a parallel feed-forward network from a trained WaveNet with no significant difference in quality. The resulting system is capable of generating high-fidelity speech samples at more than 20 times faster than real-time, and is deployed online by Google Assistant, including serving multiple English and Japanese voices.

openalex-author · arXiv (Cornell University)

Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017

We agree with Lake and colleagues on their list of key ingredients for building humanlike intelligence, including the idea that model-based reasoning is essential. However, we favor an approach that centers on one additional ingredient: autonomy. In particular, we aim toward agents that can both build and exploit their own internal models, with minimal human hand-engineering. We believe an approach centered on autonomous learning has the greatest chance of success as we scale toward real-world complexity, tackling domains for which ready-made formal models are not available. Here we survey several important examples of the progress that has been made toward building autonomous agents with humanlike abilities, and highlight some outstanding challenges.

openalex-author · Proceedings of the Annual Meeting of the Cognitive Science Society, vol 39, iss 0

A model of structure learning, inference, and generation for scene understanding.

Humans possess rich knowledge of the structure of the world, including co-occurrences among entities, and co-variation among their discrete and continuous features. But how people learn, infer and predict this structure is not wellunderstood. Here we explore everyday scene understanding as a case study of people’s structural knowledge and reasoning.We introduce a probabilistic model over scene graphs that can learn the relational structure of objects and their arrangementsand support inference and generation. Our model was able to learn the underlying structure of real-world scenes, and use it forinference and compression. In two human psychophysical experiments we found that a corresponding computational cognitivemodel was able to explain how people learn novel scene distributions and use it for classification and construction. Our workrepresents the first computational theory of human scene understanding that can account for people’s rich capacity for learningand reasoning about structure.

openalex-author · Nature

Mastering the game of Go without human knowledge

No abstract available from the OpenAlex source record.

openalex-author · arXiv (Cornell University)

Imagination-Augmented Agents for Deep Reinforcement Learning

We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a policy, I2As learn to interpret predictions from a trained environment model to construct implicit plans in arbitrary ways, by using the predictions as additional context in deep policy networks. I2As show improved data efficiency, performance, and robustness to model misspecification compared to several strong baselines.

openalex-author · arXiv (Cornell University)

SCAN: Learning Abstract Hierarchical Compositional Visual Concepts

The natural world is infinitely diverse, yet this diversity arises from a relatively small set of coherent properties and rules, such as the laws of physics or chemistry. We conjecture that biological intelligent systems are able to survive within their diverse environments by discovering the regularities that arise from these rules primarily through unsupervised experiences, and representing this knowledge as abstract concepts. Such representations possess useful properties of compositionality and hierarchical organisation, which allow intelligent agents to recombine a finite set of conceptual building blocks into an exponentially large set of useful new concepts. This paper describes SCAN (Symbol-Concept Association Network), a new framework for learning such concepts in the visual domain. We first use the previously published beta-VAE (Higgins et al., 2017a) architecture to learn a disentangled representation of the latent structure of the visual world, before training SCAN to extract abstract concepts grounded in such disentangled visual primitives through fast symbol association. Our approach requires very few pairings between symbols and images and makes no assumptions about the choice of symbol representations. Once trained, SCAN is capable of multimodal bi-directional inference, generating a diverse set of image samples from symbolic descriptions and vice versa. It also allows for traversal and manipulation of the implicit hierarchy of compositional visual concepts through symbolic instructions and learnt logical recombination operations. Such manipulations enable SCAN to invent and learn novel visual concepts through recombination of the few learnt concepts.

openalex-author · Neuron

Neuroscience-Inspired Artificial Intelligence

No abstract available from the OpenAlex source record.

openalex-author · arXiv (Cornell University)

Noisy Networks for Exploration

We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration. The parameters of the noise are learned with gradient descent along with the remaining network weights. NoisyNet is straightforward to implement and adds little computational overhead. We find that replacing the conventional exploration heuristics for A3C, DQN and dueling agents (entropy reward and $ε$-greedy respectively) with NoisyNet yields substantially higher scores for a wide range of Atari games, in some cases advancing the agent from sub to super-human performance.

openalex-author · arXiv (Cornell University)

Grounded Language Learning in a Simulated 3D World

We are increasingly surrounded by artificially intelligent technology that takes decisions and executes actions on our behalf. This creates a pressing need for general means to communicate with, instruct and guide artificial agents, with human language the most compelling means for such communication. To achieve this in a scalable fashion, agents must be able to relate language to the world and to actions; that is, their understanding of language must be grounded and embodied. However, learning grounded language is a notoriously challenging problem in artificial intelligence research. Here we present an agent that learns to interpret language in a simulated 3D environment where it is rewarded for the successful execution of written instructions. Trained via a combination of reinforcement and unsupervised learning, and beginning with minimal prior knowledge, the agent learns to relate linguistic symbols to emergent perceptual representations of its physical surroundings and to pertinent sequences of actions. The agent's comprehension of language extends beyond its prior experience, enabling it to apply familiar language to unfamiliar situations and to interpret entirely novel instructions. Moreover, the speed with which this agent learns new words increases as its semantic knowledge grows. This facility for generalising and bootstrapping semantic knowledge indicates the potential of the present approach for reconciling ambiguous natural language with the complexity of the physical world.

openalex-author · Nature

Artificial Intelligence: Chess match of the century

No abstract available from the OpenAlex source record.

openalex-author · Proceedings of the National Academy of Sciences

Overcoming catastrophic forgetting in neural networks

The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on a hand-written digit dataset and by learning several Atari 2600 games sequentially.

openalex-author · arXiv (Cornell University)

Neural Episodic Control

Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.

openalex-author · Behavioral and Brain Sciences

Building machines that learn and think for themselves

We agree with Lake and colleagues on their list of "key ingredients" for building human-like intelligence, including the idea that model-based reasoning is essential. However, we favor an approach that centers on one additional ingredient: autonomy. In particular, we aim toward agents that can both build and exploit their own internal models, with minimal human hand engineering. We believe an approach centered on autonomous learning has the greatest chance of success as we scale toward real-world complexity, tackling domains for which ready-made formal models are not available. Here, we survey several important examples of the progress that has been made toward building autonomous agents with human-like abilities, and highlight some outstanding challenges.

openalex-author · arXiv (Cornell University)

DeepMind Lab

DeepMind Lab is a first-person 3D game platform designed for research and development of general artificial intelligence and machine learning systems. DeepMind Lab can be used to study how autonomous artificial agents may learn complex tasks in large, partially observed, and visually diverse worlds. DeepMind Lab has a simple and flexible API enabling creative task-designs and novel AI-designs to be explored and quickly iterated upon. It is powered by a fast and widely recognised game engine, and tailored for effective use by the research community.

openalex-author · Neuron

Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information

Knowledge about social hierarchies organizes human behavior, yet we understand little about the underlying computations. Here we show that a Bayesian inference scheme, which tracks the power of individuals, better captures behavioral and neural data compared with a reinforcement learning model inspired by rating systems used in games such as chess. We provide evidence that the medial prefrontal cortex (MPFC) selectively mediates the updating of knowledge about one's own hierarchy, as opposed to that of another individual, a process that underpinned successful performance and involved functional interactions with the amygdala and hippocampus. In contrast, we observed domain-general coding of rank in the amygdala and hippocampus, even when the task did not require it. Our findings reveal the computations underlying a core aspect of social cognition and provide new evidence that self-relevant information may indeed be afforded a unique representational status in the brain.

openalex-author · Nature

Hybrid computing using a neural network with dynamic external memory

No abstract available from the OpenAlex source record.

openalex-author · Proceedings of the National Academy of Sciences

Semantic representations in the temporal pole predict false memories

Recent advances in neuroscience have given us unprecedented insight into the neural mechanisms of false memory, showing that artificial memories can be inserted into the memory cells of the hippocampus in a way that is indistinguishable from true memories. However, this alone is not enough to explain how false memories can arise naturally in the course of our daily lives. Cognitive psychology has demonstrated that many instances of false memory, both in the laboratory and the real world, can be attributed to semantic interference. Whereas previous studies have found that a diverse set of regions show some involvement in semantic false memory, none have revealed the nature of the semantic representations underpinning the phenomenon. Here we use fMRI with representational similarity analysis to search for a neural code consistent with semantic false memory. We find clear evidence that false memories emerge from a similarity-based neural code in the temporal pole, a region that has been called the "semantic hub" of the brain. We further show that each individual has a partially unique semantic code within the temporal pole, and this unique code can predict idiosyncratic patterns of memory errors. Finally, we show that the same neural code can also predict variation in true-memory performance, consistent with an adaptive perspective on false memory. Taken together, our findings reveal the underlying structure of neural representations of semantic knowledge, and how this semantic structure can both enhance and distort our memories.

openalex-author · Scientific Reports

Retrieval-Based Model Accounts for Striking Profile of Episodic Memory and Generalization

A fundamental theoretical tension exists between the role of the hippocampus in generalizing across a set of related episodes, and in supporting memory for individual episodes. Whilst the former requires an appreciation of the commonalities across episodes, the latter emphasizes the representation of the specifics of individual experiences. We developed a novel version of the hippocampal-dependent paired associate inference (PAI) paradigm, which afforded us the unique opportunity to investigate the relationship between episodic memory and generalization in parallel. Across four experiments, we provide surprising evidence that the overlap between object pairs in the PAI paradigm results in a marked loss of episodic memory. Critically, however, we demonstrate that superior generalization ability was associated with stronger episodic memory. Through computational simulations we show that this striking profile of behavioral findings is best accounted for by a mechanism by which generalization occurs at the point of retrieval, through the recombination of related episodes on the fly. Taken together, our study offers new insights into the intricate relationship between episodic memory and generalization, and constrains theories of the mechanisms by which the hippocampus supports generalization.

openalex-author · Trends in Cognitive Sciences

What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated

No abstract available from the OpenAlex source record.

openalex-author · arXiv (Cornell University)

Model-Free Episodic Control

State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and its capacity for episodic memory. Here we investigate whether a simple model of hippocampal episodic control can learn to solve difficult sequential decision-making tasks. We demonstrate that it not only attains a highly rewarding strategy significantly faster than state-of-the-art deep reinforcement learning algorithms, but also achieves a higher overall reward on some of the more challenging domains.

openalex-author · Neuron

Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network

Planning allows actions to be structured in pursuit of a future goal. However, in natural environments, planning over multiple possible future states incurs prohibitive computational costs. To represent plans efficiently, states can be clustered hierarchically into "contexts". For example, representing a journey through a subway network as a succession of individual states (stations) is more costly than encoding a sequence of contexts (lines) and context switches (line changes). Here, using functional brain imaging, we asked humans to perform a planning task in a virtual subway network. Behavioral analyses revealed that humans executed a hierarchically organized plan. Brain activity in the dorsomedial prefrontal cortex and premotor cortex scaled with the cost of hierarchical plan representation and unique neural signals in these regions signaled contexts and context switches. These results suggest that humans represent hierarchical plans using a network of caudal prefrontal structures. VIDEO ABSTRACT.

openalex-author · 2016 AAAS Annual Meeting (February 11-15, 2016)

The Technology of Artificial Intelligence

No abstract available from the OpenAlex source record.

openalex-author · Nature

Mastering the game of Go with deep neural networks and tree search

No abstract available from the OpenAlex source record.

openalex-author · Cell Systems, 2016, 3 (5), pp.406-410

Principles of Systems Biology, No. 11

This month: AI that learns patterns and facts, new protein-RNA and protein-protein relationships, engineering signaling and metabolism, and more variants of Cas9.

openalex-author · Neural Information Processing Systems

The Forget-me-not Process

We introduce the Forget-me-not Process, an efficient, non-parametric meta-algorithm for online probabilistic sequence prediction for piecewise stationary, repeating sources. Our method works by taking a Bayesian approach to partition a stream of data into postulated task-specific segments, while simultaneously building a model for each task. We provide regret guarantees with respect to piecewise stationary data sources under the logarithmic loss, and validate the method empirically across a range of sequence prediction and task identification problems.

openalex-author · arXiv (Cornell University)

Approximate Hubel-Wiesel Modules and the Data Structures of Neural Computation

This paper describes a framework for modeling the interface between perception and memory on the algorithmic level of analysis. It is consistent with phenomena associated with many different brain regions. These include view-dependence (and invariance) effects in visual psychophysics and inferotemporal cortex physiology, as well as episodic memory recall interference effects associated with the medial temporal lobe. The perspective developed here relies on a novel interpretation of Hubel and Wiesel's conjecture for how receptive fields tuned to complex objects, and invariant to details, could be achieved. It complements existing accounts of two-speed learning systems in neocortex and hippocampus (e.g., McClelland et al. 1995) while significantly expanding their scope to encompass a unified view of the entire pathway from V1 to hippocampus.

openalex-author · AI Magazine

Letters to the Editor

Artificial intelligence (AI) research has explored a variety of problems and approaches since its inception, but for the last 20 years or so has been focused on the problems surrounding the construction of intelligent agents — systems that perceive and act in some environment. In this context, "intelligence" is related to statistical and economic notions of rationality — colloquially, the ability to make good decisions, plans, or inferences. The adoption of probabilistic and decision-theoretic representations and statistical learning methods has led to a large degree of integration and cross-fertilization among AI, machine learning, statistics, control theory, neuroscience, and other fields. The establishment of shared theoretical frameworks, combined with the availability of data and processing power, has yielded remarkable successes in various component tasks such as speech recognition, image classification, autonomous vehicles, machine translation, legged locomotion, and question-answering systems. As capabilities in these areas and others cross the threshold from laboratory research to economically valuable technologies, a virtuous cycle takes hold whereby even small improvements in performance are worth large sums of money, prompting greater investments in research. There is now a broad consensus that AI research is progressing steadily, and that its impact on society is likely to increase. The potential benefits are huge, since everything that civilization has to offer is a product of human intelligence; we cannot predict what we might achieve when this intelligence is magnified by the tools AI may provide, but the eradication of disease and poverty are not unfathomable. Because of the great potential of AI, it is important to research how to reap its benefits while avoiding potential pitfalls. The progress in AI research makes it timely to focus research not only on making AI more capable, but also on maximizing the societal benefit of AI. Such considerations motivated the AAAI 2008–09 Presidential Panel on Long-Term AI Futures and other projects on AI impacts, and constitute a significant expansion of the field of AI itself, which up to now has focused largely on techniques that are neutral with respect to purpose. We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial: our AI systems must do what we want them to do. The attached research priorities document [see page X] gives many examples of such research directions that can help maximize the societal benefit of AI. This research is by necessity interdisciplinary, because it involves both society and AI. It ranges from economics, law and philosophy to computer security, formal methods and, of course, various branches of AI itself. In summary, we believe that research on how to make AI systems robust and beneficial is both important and timely, and that there are concrete research directions that can be pursued today.

openalex-author · eLife

Hippocampal place cells construct reward related sequences through unexplored space

Dominant theories of hippocampal function propose that place cell representations are formed during an animal's first encounter with a novel environment and are subsequently replayed during off-line states to support consolidation and future behaviour. Here we report that viewing the delivery of food to an unvisited portion of an environment leads to off-line pre-activation of place cells sequences corresponding to that space. Such 'preplay' was not observed for an unrewarded but otherwise similar portion of the environment. These results suggest that a hippocampal representation of a visible, yet unexplored environment can be formed if the environment is of motivational relevance to the animal. We hypothesise such goal-biased preplay may support preparation for future experiences in novel environments.

openalex-author · Paper

Author response: Hippocampal place cells construct reward related sequences through unexplored space

Article Figures and data Abstract eLife digest Introduction Results Discussion Materials and method References Decision letter Author response Article and author information Metrics Abstract Dominant theories of hippocampal function propose that place cell representations are formed during an animal's first encounter with a novel environment and are subsequently replayed during off-line states to support consolidation and future behaviour. Here we report that viewing the delivery of food to an unvisited portion of an environment leads to off-line pre-activation of place cells sequences corresponding to that space. Such ‘preplay’ was not observed for an unrewarded but otherwise similar portion of the environment. These results suggest that a hippocampal representation of a visible, yet unexplored environment can be formed if the environment is of motivational relevance to the animal. We hypothesise such goal-biased preplay may support preparation for future experiences in novel environments. https://doi.org/10.7554/eLife.06063.001 eLife digest As an animal explores an area, part of the brain called the hippocampus creates a mental map of the space. When the animal is in one location, a few neurons called ‘place cells’ will fire. If the animal moves to a new spot, other place cells fire instead. Each time the animal returns to that spot, the same place cells will fire. Thus, as the animal moves, a place-specific pattern of firing emerges that scientists can view by recording the cells' activity and which can be used to reconstruct the animal's position. After exploring a space, the hippocampus may replay the new place-specific pattern of activity during sleep. By doing so, the brain consolidates the memory of the space for return visits. Recent evidence now suggests that these mental rehearsals—or internal simulations of the space—may begin even before a new space has been explored. Now, Ólafsdóttir, Barry et al. report that whether an animal's brain simulates a first visit to a new space depends on whether the animal anticipates a reward. In the experiments, rats were allowed to run up to the junction in a T-shaped track. The animals could see into each of the arms, but not enter them. Food was then placed in one of the inaccessible arms. Ólafsdóttir, Barry et al. recorded the firing of place cells in the brain of the animals when they were on the track and during a rest period afterwards. The rats were then allowed onto the inaccessible arms, and again their brain activity was recorded. In the rest period after the rats first viewed the inaccessible arms, the place cell pattern that would later form the mental map of a journey to and from the food-containing arm was pre-activated. However, the place cell pattern that would become the mental map of the other inaccessible arm was not activated before the rat explored that area. Therefore, Ólafsdóttir, Barry et al. suggest that the perception of reward influences which place cell pattern is simulated during rest. An implication of these findings is that the brain preferentially simulates past or future experiences that are deemed to be functionally significant, such as those associated with reward. A future challenge will be to determine whether this goal-related simulation of unvisited spaces predicts and is needed for behaviour such as successful navigation to a goal. https://doi.org/10.7554/eLife.06063.002 Introduction We investigated whether the presence of an inaccessible goal in an unvisited portion of an environment was sufficient to elicit pre-activation (‘preplay’) of hippocampal place cell sequences that will subsequently represent runs through the unvisited environment. To this end, we recorded from ensembles of place cells (O'Keefe and Dostrovsky, 1971) (4 rats, 37–66 place cells each, 212 cells in total) while rats ran along a T-shaped track (Figure 1—figure supplement 1, Table 1) with visible yet inaccessible arms (Figure 1A)—RUN1. One arm (counter balanced between animals) was subsequently cued with food while the animal remained on the track—GOAL-CUE. During a rest period before RUN1 (REST1) and after GOAL-CUE (REST2), spiking events—periods of 300 ms or less, where at least 15% of cells were active (Foster and Wilson, 2006; Diba and Buzsaki, 2007)—were analysed. These spiking events were associated with significantly higher power in the ripple spectrum (80–250 Hz) than other comparable periods (Figure 1—figure supplement 2). To investigate whether paths on the cued and uncued arms were preplayed we assessed the match between the order in which cells fired during spiking events and during future runs on the arms (RUN2, Figure 1—figure supplement 3). Specifically, we computed the rank-order correlations between spiking events and sequences of place cells active on the arms, referred to as templates (Lee and Wilson, 2002; Foster and Wilson, 2006; Diba and Buzsaki, 2007; Dragoi and Tonegawa, 2011) (Figure 1—figure supplement 4). Preplay events were identified as those with either a significant positive or negative correlation—a two-tailed test, each tail tested at the 97.5% level. These preplay events were found to exhibit higher power in the ripple spectrum than non-significant spiking events (Figure 1—figure supplement 2). To establish significance at the population level, the proportion of preplay events measured was compared to a null distribution generated by calculating correlations between place cell templates and shuffled sequences from events (see Figure 1B–C). Table 1 Experimental parameters https://doi.org/10.7554/eLife.06063.003 R1838R505R584R504All rats (mean)Cue bias (dwell time) RUN10.330.32−0.05−0.200.10 GOAL-CUE0.460.200.310.320.33Cue bias (looking time) RUN1−0.09−0.020.01−0.36−0.11 GOAL-CUE0.060.310.090.050.13RUN2 arm bias RUN21.00.841.01.00.96Session duration (min) SLEEP16088757474 RUN1131013911 GOAL-CUE1017121113 SLEEP26067716065 RUN23419353130Template length (number of cells) Up cued arm2036534338 Down cued arm1535453332 Up uncued arm1926454033 Down uncued arm1532414333 Figure 1 with 4 supplements see all Download asset Open asset Preferential preplay of a behaviourally relevant, unvisited environment. (A) Experimental protocol. (i) Prior to running on the track, the animals rested for at least an hour (REST1). (ii) Following REST1, animals ran 20 laps on the stem (RUN1). Access to the arms was blocked by a barrier at the end of the stem which the animals could see through but not pass. (iii) Following RUN1, the experimenter baited one arm so to provoke the animals' interest in that arm (GOAL-CUE). (iv) Following goal-cueing, the animal rested for at least another hour (REST2). (v) Following REST2, the barrier was removed and the animals traversed the extent of the track, in alternate L-shaped laps (RUN2). (B) Left: an example template for a run to the cued arm. x-axis shows location on the track and y-axis cell IDs. Right: Example raster plots of preplay events—the title shows the correlation between the preplay event and the template sequence. C same as B but for the uncued template. https://doi.org/10.7554/eLife.06063.004 Results During GOAL-CUE, all four animals displayed more interest in the cued arm than the uncued arm (as indexed by the difference in time spent on the cued side of the stem vs the uncued side, divided by the total time spent on either side, mean bias = 0.33). In contrast, prior to goal-cueing, two animals spent more time on the uncued side of the stem (mean bias = 0.10, see Table 1 for results for individual animals). Moreover, during GOAL-CUE all animals also spent more time looking towards the cued arm than the uncued arm (mean bias = 0.13), again this bias was not observed prior to goal-cueing when only one animal spent more time looking towards the cued arm (mean bias = −0.11, see Table 1 for results of individual animals). Furthermore, during RUN2 when the barrier was first removed and the food cue was no longer present, all four animals initially turned towards the cued arm and spent more time on the cued rather than the uncued arm (mean bias = 0.96, see Table 1 for results for individual animals). Consistent with the behavioural bias, in REST2 we found significant preplay of the yet unvisited cued arm (7.37% preplay events, p < 0.001, binomial test vs chance, Figure 2A,D). Conversely, the uncued arm was not significantly preplayed (4.41% preplay events p = 0.33, vs cued arm: p < 0.001, Figure 2B,D). Similarly significant effects were found when animals were analysed individually (see Figure 2D, Table 2), although the results for one animal were based on a relatively small sample (number of cued preplay events = 9, R1838), it still showed significant preplay of the cued arm. Moreover, the results were corroborated by a distribution-based analysis; namely, comparing the area under the curve (AUC) of bootstrapped cumulative distributions of absolute correlations for each arm to that of their shuffle distribution (Figure 2A,B, cued: p < 0.001, uncued: p = 0.22, cued vs uncued: p = 0.0044, Figure 2—figure supplement 1). Preplay events of the cued arm were equally likely to represent paths to and from the cued arm (7.34% vs 7.40% p = 0.49) and to run towards (‘forward’) and away (‘reverse’) from the ends of the arms (6.93% vs 7.80%, p = 0.18, Figure 3A). Moreover, the amount of preplay exhibited by each animal appeared to be predicted by the interest they displayed for the cued arm during GOAL-CUE (Figure 3—figure supplement 3). Importantly, preferential preplay of the cued arm could not be explained by differences in the number of cells with fields on the arms (Figure 3—figure supplement 2A–B), spike-sorting quality (cells with neighbouring place fields were as well separated in cluster-space as those with distant fields, p = 0.45, 2-sample Kolmogorov–Smirnov test), place field stability on the two arms (cued arm stability r = 0.54 vs uncued arm stability r = 0.49, p = 0.15) or the location of place fields on the cued arm (Figure 3B, p = 0.22 two-sample Kolmogorov–Smirnov test). In sum, we found during rest after goal-cueing, significant and preferential preplay of an unvisited and motivationally relevant portion of the environment. Figure 2 with 2 supplements see all Download asset Open asset Preplay is a function of goal-cueing. (A) Bootstrapped cumulative distribution of (absolute) correlations between spiking events and the cued template in REST2 (red = data, black = shuffle). Lighter areas of the curve show 1 standard deviation of the mean. Inset: difference between the data and shuffle distributions. If there are more high correlations in the data compared to the shuffle then the data distribution will deviate below the shuffle distribution. (B–C) same as A but for the uncued template in REST2 and the cued template in REST1, respectively. (D) Proportion of spiking events categorised as preplay events in REST2 for the cued and uncued arms. Bars show mean for all animals, and the black lines show the result for each animal. The grey dashed line shows the proportion of preplay events expected by chance. (E) Same as D but comparing proportion of preplay events for the cued template in REST1 and REST2. https://doi.org/10.7554/eLife.06063.009 Table 2 REST period results https://doi.org/10.7554/eLife.06063.012 AnimalArm# spiking events# preplay events% preplay% chancep-valueREST2 R1838Cued44920.456.866.56 × 10−4Uncued26311.546.770.10 R505Cued631386.024.570.037Uncued860293.373.820.72 R584Cued437327.324.736.20 × 10−3Uncued398225.534.740.19 R504Cued516417.955.150.0027Uncued373195.094.400.21 All ratsCued16281207.374.864.12 × 10−6Uncued1657734.414.220.33REST1 R1838Cued6334.767.050.66Uncued3525.717.370.48 R505Cued664395.874.340.025Uncued1215383.133.450.70 R584Cued269145.204.670.28Uncued247218.504.720.0035 R504Cued31161.934.440.99Uncued173116.364.210.063 All ratsCued1307624.744.560.34Uncued1670724.313.800.12 Summary results from REST1 and REST2 for the cued and uncued arms for individual animals. # Spiking events = total number of spiking events recorded. # preplay events = number of significant spiking events. % preplay = Proportion of the spiking events that qualified as preplay events (i.e., that were significant), expressed as a percentage. % chance = proportion of spiking events from the shuffled data that qualified as preplay events, expressed as a percentage. p-value = probability, derived from a binomial test, of obtaining the observed number of preplay events for each template given the chance level calculated from the shuffled data. Figure 3 with 4 supplements see all Download asset Open asset Spatial and temporal dynamics of preplay. (A) The proportion of preplay events when negative (‘reverse’) and positive (‘forward’) spiking event correlations are analysed separately. Bars show means for all data and black lines the results for each animal. (B) Frequency of preplay events vs location on the cued (red) and uncued (blue) arms normalised by the density of place field centres—100% indicates the expected number of preplay events under an even distribution across each arm. No bias towards particular sections of the arms was evident (cued p = 0.22, uncued p = 0.15, based on a two-sample Kolmogorov–Smirnov test). Lighter areas show standard error of the mean (SEM) and the black line the expected distribution. (C) Bootstrapped cumulative distribution of (absolute) correlations between spiking events and the cued template during GOAL-CUE (red = data, black = shuffle). Lighter areas of the curve show 1 standard deviation of the mean. Inset: difference between the data and shuffle distributions. (D) Ratio of activity levels between cued and uncued arm cells (cued/uncued) during events for the first and second half of each experimental period. Red line shows mean ratio, derived from bootstrapped data, obtained for each period, and the shaded areas 1sd of the bootstrapped data. The black horizontal line indicates equal rates for the two arms. * = significantly different from 1 based on 95% confidence intervals. https://doi.org/10.7554/eLife.06063.013 Does goal-cueing trigger preplay? If so, there should be a greater number of significant pre-play events in REST2 compared to REST1 which was recorded before animals had visited or seen any part of the environment. Preplay of the cued arm was higher in REST2 than REST1 (7.37% vs 4.74%, p < 0.001, Figure 2C,E), an effect that was seen for all animals (Table 2). Indeed, the cued arm was not significantly preplayed during REST1 (4.74%, p = 0.34). Again, the result was corroborated using an AUC analysis (Figure 2C, Figure 2—figure supplement 1). Thus, we find preplay only occurs during rest periods recorded after goal-cueing. However, it is possible that the frequency of preplay might decrease as a function of the temporal gap between rest and behaviour. As such our failure to detect preplay in REST1 might be due to the greater delay between REST1 and RUN2 than between REST2 and RUN2. To address this we analysed preplay of the stem (i.e., RUN1) during REST1. We did not find preplay of the stem (4.12% preplay events, p = 0.44, AUC analysis p = 0.053, Figure 2—figure supplement 2, Table 3, RUN1 REST1 vs cued REST2: p < 0.001). Consequently, these results imply that the preplay of the unvisited, yet visible, environment we observed in REST2 was driven by behavioural cueing of that environment. Table 3 REST1 stem results https://doi.org/10.7554/eLife.06063.018 Animal# spiking events# preplay events% preplay% chancep-valueR18386834.416.010.59R505980353.574.030.74R584329175.174.530.24R504396184.553.500.11All rats1773734.124.080.44 Summary results from REST1 analysing preplay of the stem. # Spiking events = total number of spiking events recorded. # Preplay events = number of significant spiking events. % preplay = Proportion of the spiking events that qualified as preplay events (i.e., that were significant), expressed as a percentage. % chance = proportion of spiking events from the shuffled data that qualified as preplay events, expressed as a percentage. p-value = probability, derived from a binomial test, of obtaining the observed number of preplay events for each template given the chance level calculated from the shuffled data. At what point does preferential preplay of the cued arm emerge? Plausibly preplay might be initiated immediately when the cued arm is baited (start of GOAL-CUE) and simply persist into the subsequent REST2 period, alternatively the bias may only arise during rest. Due to the short duration of the goal-cueing period (∼10 min) a relatively small number of spiking events were recorded for the two arms during this period (172 and 170 for the cued and uncued arm respectively). However, based on a bootstrapped comparison of the AUC for absolute correlations from the cued and uncued arm vs shuffled distributions, we found that the cued but not the uncued arm was preplayed (p = 0.02, p = 0.24 respectively, Figure 3C, Figure 3—figure supplement 1). A direct comparison of the proportion of preplay events for the cued vs uncued arm was marginally not significant (6.4% vs 4.12%, p = 0.052, see Table 4 for results for individual animals). Finally, to validate the results from this smaller dataset we carried out a further, more inclusive, analysis. Specifically, we tracked the temporal evolution of the bias in preplay by comparing the activity of cells from the cued and uncued arms at different points during the experiment. For every spiking event we computed the mean rate for cells that would subsequently have fields on the cued arm compared to those with fields on the uncued arm. During REST1 and RUN1 the future cued and uncued arm cells did not differ in activity, this was true for both the first and second half of these periods (mean cued/uncued rate ratio: REST1 early ratio = 0.96, p = 0.88, REST1 late ratio = 1.04 p = 0.09, RUN1 early ratio = 1.09 p = 0.32, RUN1 late ratio = 1.18 p = 0.22, Figure 3D). However, during GOAL-CUE cued arm cells were significantly more active than uncued arm cells, an effect that was most pronounced during the first half (5 min) of the cueing period (GOAL-CUE early ratio = 1.78, p = 0.01, late = 1.46, p < 0.01). Finally, the difference between the two groups persisted through the subsequent rest period, albeit attenuating with time (REST2 early ratio = 1.30 p < 0.001, REST2 late ratio = 1.10 p < 0.01, Figure 3D). Importantly, control analyses showed that the bias in sequential preplay is not a mere product of differing activity levels of the cells for the two arms (Figure 3—figure supplement 2C–D). Together, these findings indicate that biased pre-activation of future experiences is instantiated at the point when an environment becomes motivationally-relevant. Table 4 GOAL-CUE results https://doi.org/10.7554/eLife.06063.019 AnimalArm# spiking events# preplay events% preplay% chancep-valueR1838Cued50070.30Uncued50070.30R505Cued4536.674.020.11Uncued4812.083.570.52R584Cued11187.214.810.087Uncued11254.464.820.46R504Cued11004.360.39Uncued7114.295.000.044All ratsCued172116.404.640.11Uncued17074.124.490.50 Summary results from GOAL-CUE analysis # Spiking events = total number of spiking events recorded. # Preplay events = number of significant spiking events. % preplay = Proportion of the spiking events that qualified as preplay events (i.e., that were significant), expressed as a percentage. % chance = proportion of spiking events from the shuffled data that qualified as preplay events, expressed as a percentage. p-value = probability, derived from a binomial test, of obtaining the observed number of preplay events for each template given the chance level calculated from the shuffled data. Finally, to corroborate the results obtained from the rank-order analysis of spike sequences, we applied a Bayesian spatial reconstruction algorithm (Davidson et al., 2009; Bendor and Wilson, 2012) to the data from the two rest sessions. In contrast to the rank-order method, which utilised only the first spike emitted by each cell, the Bayesian decoding approach used all spikes emitted during an event. The Bayesian decoding approach uses the spiking activity of all simultaneously recorded place cells to calculate the posterior probability of an animal being at any position in the environment, based on a Poisson spiking framework. (See Materials and methods; R1838 was excluded from this analysis due to low cell yield). The Bayesian decoder performed equally well on the cued and uncued arms (Figure 4A,B, median error for both arms = 10.0 cm). Next, we applied the Bayesian decoding to spiking events during the rest sessions. First, we calculated the posterior probabilities for 5 ms non-overlapping bins, which generated a posterior probability matrix for each event (Figure 4C,D). A spiking event that reflects a constant speed run through the environment will show an increased posterior along a line in the decoded posterior matrix. Therefore, for each spiking event we fit a line that accounted for the maximum variance and calculated its goodness of fit (Figure 4C,D). To assess if that event represented a significant preplay event: we generated 1000 posterior probability matrices by shuffling the identities of cells included in the event, fit lines on all matrices, and calculated their goodness of fits. Events whose goodness of fits exceeded the 95th percentile of the shuffled distributions were labelled as preplay events. Again, during REST2, we found preplay of the cued arm (7.64% of events, p < 0.001, binomial test) but not of the uncued arm (4.78% of events, p = 0.55, vs cued = p < 0.001, Figure 4E). Moreover, neither the cued nor the uncued arm were significantly preplayed in REST1 (cued = 5.04%, p = 0.43, uncued = p = 0.55, Figure Thus, this more analysis the same pattern of results as by our analysis; that pre-activation of future place cell sequences is a and Figure 4 Download asset Open asset Bayesian reconstruction preferential cued arm preplay. matrices based on RUN2 data decoding for the cued (A) and uncued arms show the mean posterior probability distribution across the arm by the true position of the rat bins, data for as power away from the both arms of the animal's position at all points on the track. preplay events for the cued (C) and uncued (D) arms the line that fits the decoded event indicates time (5 the event, y-axis position on title indicates probability of obtaining a by the null distribution of fits obtained by shuffling the cell identities for each event quality of proportion of shuffled (E) Proportion of spiking events categorised as preplay events for the cued and uncued arms in REST2. Bars show mean for all data with Same as but comparing preplay in REST1 and REST2 for the cued arm. Discussion Preplay of a visible, unvisited environment that has motivational relevance to an animal with from and a for the hippocampus in future and and future et al., 2007; et al., Moreover, these findings with replay can be by reward and and 2009; and but this to preplay of In our data, sequences of place cells to and away from the goal were preplayed with similar frequency and in both and (i.e., positive and negative of sequences towards goal has been with the of future and Plausibly the proportion of preplay events we observed to the cued goal might also given the animals' subsequent for the cued arm. However, preplay of sequences from the cued goal towards the stem are one it is possible that these sequences simply for a return to the of the stem after of the However, replay has been as a to the temporal (Foster and Wilson, 2006; Foster and a to a of that to a successful In the of when the cued arm has not been an is that paths towards the goal might be simulated using preplay then and from during in a similar to that to for periods of and rest (Foster and Wilson, 2006; and et al., have of that have yet to be and Tonegawa, is to the activity of cell ensembles et al., that subsequently become to in the environment. We did not preplay in the preplay of stem during the of this is we that other also not find preplay for a novel environment and and Wilson, However, it likely that other but in the preplay and Tonegawa, such as the and visible but inaccessible arms, might have this Indeed, the AUC analysis comparing spiking events recorded during REST1 the stem template Figure 2—figure supplement was not significant (p = that with more data or a environment we might have found preplay for the stem. In a similar one might also have expected to a rate of preplay for the uncued arm during REST2. it possible that the of the environment such an effect in that the motivationally relevant cued arm was preplayed at the of the uncued arm. to be is if this bias reflects an active associated with the or if the relevance of the cued arm simply it to more cell In sum, our data indicates that preplay of an unvisited environment can be initiated in response to viewing that environment if it is relevant to future that were in the of hippocampal representations for novel spaces and and Wilson, 2002; et al., Furthermore, although our data does not direct support for preplay it also does not and is with the that the hippocampus the of novel spatial experiences as by other example see Dragoi and These results are with the that place cells form or et al., active but being associated with the cued arm during GOAL-CUE, but not the that place cell firing is driven in a et al., We that these findings support the of preplay for preparation for future experiences and Buzsaki, 2007; and Foster and and but it to future experiences in yet to be explored. to be seen if the of the hippocampus to future spatial sequences is to the of more temporal et al., and Materials and method and a rats were used in this All were by the to the and in the of rats at two each four of and to the and One animal a Finally, one animal one four of to the and one four

openalex-author · Nature

Human-level control through deep reinforcement learning

No abstract available from the OpenAlex source record.

openalex-author · Paper

Executive Summary: Success in the quest for articial intelligence has the potential to bring unprecedented benets to humanity, and it is therefore worthwhile to research how to maximize these benets while avoiding potential pitfalls. This document gives numerous examples (which should by no means be construed as an exhaustive list) of such worthwhile research aimed at ensuring that AI remains robust and benecial.

No abstract available from the OpenAlex source record.

openalex-author · Current Biology

A Goal Direction Signal in the Human Entorhinal/Subicular Region

Navigating to a safe place, such as a home or nest, is a fundamental behavior for all complex animals. Determining the direction to such goals is a crucial first step in navigation. Surprisingly, little is known about how or where in the brain this "goal direction signal" is represented. In mammals, "head-direction cells" are thought to support this process, but despite 30 years of research, no evidence for a goal direction representation has been reported. Here, we used fMRI to record neural activity while participants made goal direction judgments based on a previously learned virtual environment. We applied multivoxel pattern analysis to these data and found that the human entorhinal/subicular region contains a neural representation of intended goal direction. Furthermore, the neural pattern expressed for a given goal direction matched the pattern expressed when simply facing that same direction. This suggests the existence of a shared neural representation of both goal and facing direction. We argue that this reflects a mechanism based on head-direction populations that simulate future goal directions during route planning. Our data further revealed that the strength of direction information predicts performance. Finally, we found a dissociation between this geocentric information in the entorhinal/subicular region and egocentric direction information in the precuneus.

openalex-author · 1st Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2013)

Decoding future state representations during planning

Planning enables humans and animals to use their knowledge of the structure of the world to anticipate the consequences of their actions, even when these consequences have never been experienced. Yet little is known about the algorithm used by the brain for planning. A possible neural basis for planning is hinted at in recordings in rodents that have revealed “preplay”, or explicit sequential neural representation of future states, at decision points. In humans, neuroimaging studies have also identified neural correlates of future state values, but a direct observation of a neural representation of future states during planning has re- mained elusive. Directly observing these representations would allow us to disambiguate different possible planning algorithms. In the present study, we asked subjects to perform a 5-step planning task in a complex maze. To prevent habitization, one unavailable transition was cued to subjects at the beginning of each trial. Fitting computational models with different maximum search depths to behavioral data suggested a wide range of depth between subjects, with some maximizing only immediate rewards, and others taking into ac- count deep future contingencies. We took advantage of the fast time resolution of magnetoencephalography (MEG), along with multivariate pattern classification, to study neural representations of future states during planning. Dimensionality of MEG time-frequency data was reduced with principal components analysis, and a linear classifier was applied to the low-dimensional data. The classifier was trained by recording MEG activity while presenting stimuli in random order before the task began. In leave-one-out cross-validation on the training data, this classifier performed significantly above chance for all subjects. We then applied this classifier to neural data acquired at choice points during the task, and using this approach we seek to identify future states represented during planning.

openalex-author · Journal of Neuroscience

Foraging under Competition: The Neural Basis of Input-Matching in Humans

Input-matching is a key mechanism by which animals optimally distribute themselves across habitats to maximize net gains based on the changing input values of food supply rate and competition. To examine the neural systems that underlie this rule in humans, we created a continuous-input foraging task where subjects had to decide to stay or switch between two habitats presented on the left and right of the screen. The subject's decision to stay or switch was based on changing input values of reward-token supply rate and competition density. High density of competition or low-reward token rate was associated with decreased chance of winning. Therefore, subjects attempted to maximize their gains by switching to habitats that possessed low competition density and higher token rate. When it was increasingly disadvantageous to be in a habitat, we observed increased activity in brain regions that underlie preparatory motor actions, including the dorsal anterior cingulate cortex and the supplementary motor area, as well as the insula, which we speculate may be involved in the conscious urge to switch habitats. Conversely, being in an advantageous habitat is associated with activity in the reward systems, namely the striatum and medial prefrontal cortex. Moreover, amygdala and dorsal putamen activity steered interindividual preferences in competition avoidance and pursuing reward. Our results suggest that input-matching decisions are made as a net function of activity in a distributed set of neural systems. Furthermore, we speculate that switching behaviors are related to individual differences in competition avoidance and reward drive.

openalex-author · Cerebral Cortex

Imagine All the People: How the Brain Creates and Uses Personality Models to Predict Behavior

The behaviors of other people are often central to envisioning the future. The ability to accurately predict the thoughts and actions of others is essential for successful social interactions, with far-reaching consequences. Despite its importance, little is known about how the brain represents people in order to predict behavior. In this functional magnetic resonance imaging study, participants learned the unique personality of 4 protagonists and imagined how each would behave in different scenarios. The protagonists' personalities were composed of 2 traits: Agreeableness and Extraversion. Which protagonist was being imagined was accurately inferred based solely on activity patterns in the medial prefrontal cortex using multivariate pattern classification, providing novel evidence that brain activity can reveal whom someone is thinking about. Lateral temporal and posterior cingulate cortex discriminated between different degrees of agreeableness and extraversion, respectively. Functional connectivity analysis confirmed that regions associated with trait-processing and individual identities were functionally coupled. Activity during the imagination task, and revealed by functional connectivity, was consistent with the default network. Our results suggest that distinct regions code for personality traits, and that the brain combines these traits to represent individuals. The brain then uses this "personality model" to predict the behavior of others in novel situations.

openalex-author · The Journal of Neuroscience

Detecting Representations of Recent and Remote Autobiographical Memories in vmPFC and Hippocampus

How autobiographical memories are represented in the human brain and whether this changes with time are questions central to memory neuroscience. Two regions in particular have been consistently implicated, the ventromedial prefrontal cortex (vmPFC) and the hippocampus, although their precise contributions are still contested. The key question in this debate, when reduced to its simplest form, concerns where information about specific autobiographical memories is located. Here, we availed ourselves of the opportunity afforded by multivoxel pattern analysis to provide an alternative to conventional neuropsychological and fMRI approaches, by detecting representations of individual autobiographical memories in patterns of fMRI activity. We examined whether information about specific recent (two weeks old) and remote (10 years old) autobiographical memories was represented in vmPFC and hippocampus, and other medial temporal and neocortical regions. vmPFC contained information about recent and remote autobiographical memories, although remote memories were more readily detected there, indicating that consolidation or a change of some kind had occurred. Information about both types of memory was also present in the hippocampus, suggesting it plays a role in the retrieval of vivid autobiographical memories regardless of remoteness. Interestingly, we also found that while recent and remote memories were both represented within anterior and posterior hippocampus, the latter nevertheless contained more information about remote memories. Thus, like vmPFC, the hippocampus too respected the distinction between recent and remote memories. Overall, these findings clarify and extend our view of vmPFC and hippocampus while also informing systems-level consolidation and providing clear targets for future studies.

openalex-author · Neuron

The Future of Memory: Remembering, Imagining, and the Brain

No abstract available from the OpenAlex source record.

openalex-author · The Journal of Neuroscience

Scene Construction in Amnesia: An fMRI Study

In recent years, there has been substantial interest in how the human hippocampus not only supports recollection of past experiences, but also the construction of fictitious and future events, and the leverage this might offer for understanding the operating mechanisms of the hippocampus. Evidence that patients with bilateral hippocampal damage and amnesia cannot construct novel or future scenes/events has been influential in driving this line of research forward. There are, however, some patients with hippocampal damage and amnesia who retain the ability to construct novel scenes. This dissociation may indicate that the hippocampus is not required for scene construction, or alternatively, there could be residual function in remnant hippocampal tissue sufficient to support the basic construction of scenes. Resolving this controversy is central to current theoretical debates about the hippocampus. To investigate, we used fMRI and a scene construction task to test patient P01, who has dense amnesia, ∼50% bilateral hippocampal volume loss, and intact scene construction. We found that scene construction in P01 was associated with increased activity in a set of brain areas, including medial temporal, retrosplenial, and posterior parietal cortices, that overlapped considerably with the regions engaged in control participants performing the same task. Most notably, the remnant of P01's right hippocampus exhibited increased activity during scene construction. This suggests that the intact scene construction observed in some hippocampal-damaged amnesic patients may be supported by residual function in their lesioned hippocampus, in accordance with theoretical frameworks that ascribe a vital role to the hippocampus in scene construction.

openalex-author · Nature

Is the brain a good model for machine intelligence?

No abstract available from the OpenAlex source record.

openalex-author · Frontiers in Human Neuroscience

Multi-voxel pattern analysis in human hippocampal subfields

A complete understanding of the hippocampus depends on elucidating the representations and computations that exist in its anatomically distinct subfields. High-resolution structural and functional MRI scanning is starting to permit insights into hippocampal subfields in humans. In parallel, such scanning has facilitated the use of multi-voxel pattern analysis (MVPA) to examine information present in the distributed pattern of activity across voxels. The aim of this study was to combine these two relatively new innovations and deploy MVPA in the hippocampal subfields. Delineating subregions of the human hippocampus, a prerequisite for our study, remains a significant challenge, with extant methods often only examining part of the hippocampus, or being unable to differentiate CA3 and dentate gyrus (DG). We therefore devised a new high-resolution anatomical scanning and subfield segmentation protocol that allowed us to overcome these issues, and separately identify CA1, CA3, DG, and subiculum (SUB) across the whole hippocampus using a standard 3T MRI scanner. We then used MVPA to examine fMRI data associated with a decision-making paradigm involving highly similar scenes that had relevance for the computations that occur in hippocampal subfields. Intra- and inter-rater scores for subfield identification using our procedure confirmed its reliability. Moreover, we found that decoding of information within hippocampal subfields was possible using MVPA, with findings that included differential effects for CA3 and DG. We suggest that MVPA in human hippocampal subfields may open up new opportunities to examine how different types of information are represented and processed at this fundamental level.

openalex-author · Learning & Memory

Decoding overlapping memories in the medial temporal lobes using high-resolution fMRI

The hippocampus is proposed to process overlapping episodes as discrete memory traces, although direct evidence for this in human episodic memory is scarce. Using green-screen technology we created four highly overlapping movies of everyday events. Participants were scanned using high-resolution fMRI while recalling the movies. Multivariate pattern analysis revealed that the hippocampus supported distinct representations of each memory, while neighboring regions did not, demonstrating that the human hippocampus maintains unique pattern-separated memory traces even when memories are highly overlapping. The hippocampus also contained representations of spatial contexts that were shared across different memories, consistent with a specialized role in processing space.

openalex-author · Hippocampus

Decoding representations of scenes in the medial temporal lobes

Recent theoretical perspectives have suggested that the function of the human hippocampus, like its rodent counterpart, may be best characterized in terms of its information processing capacities. In this study, we use a combination of high-resolution functional magnetic resonance imaging, multivariate pattern analysis, and a simple decision making task, to test specific hypotheses concerning the role of the medial temporal lobe (MTL) in scene processing. We observed that while information that enabled two highly similar scenes to be distinguished was widely distributed throughout the MTL, more distinct scene representations were present in the hippocampus, consistent with its role in performing pattern separation. As well as viewing the two similar scenes, during scanning participants also viewed morphed scenes that spanned a continuum between the original two scenes. We found that patterns of hippocampal activity during morph trials, even when perceptual inputs were held entirely constant (i.e., in 50% morph trials), showed a robust relationship with participants' choices in the decision task. Our findings provide evidence for a specific computational role for the hippocampus in sustaining detailed representations of complex scenes, and shed new light on how the information processing capacities of the hippocampus may influence the decision making process.

openalex-author · Proceedings of the National Academy of Sciences

Role of the hippocampus in imagination and future thinking

Proceedings of the National Academy of Sciences (PNAS), a peer reviewed journal of the National Academy of Sciences (NAS) - an authoritative source of high-impact, original research that broadly spans the biological, physical, and social sciences.

openalex-author · Neuropsychologia

Imagining fictitious and future experiences: Evidence from developmental amnesia

Patients with bilateral hippocampal damage acquired in adulthood who are amnesic for past events have also been reported to be impaired at imagining fictitious and future experiences. One such patient, P01, however, was found to be unimpaired on these tasks despite dense amnesia and 50% volume loss in both hippocampi. P01 might be an atypical case, and in order to investigate this we identified another patient with a similar neuropsychological profile. Jon is a well-characterised patient with developmental amnesia and 50% volume loss in his hippocampi. Interestingly both Jon and P01 retain some recognition memory ability, and show activation of residual hippocampal tissue during fMRI. Jon's ability to construct fictitious and future scenarios was compared with the adult-acquired cases previously reported on this task and control participants. In contrast to the adult-acquired cases, but similar to P01, Jon was able to richly imagine both fictitious and future experiences in a comparable manner to control participants. Moreover, his constructions were spatially coherent. We speculate that the hippocampal activation during fMRI noted previously in P01 and Jon might indicate some residual hippocampal function which is sufficient to support their preserved ability to imagine fictitious and future scenarios.

openalex-author · Current Biology

Decoding Individual Episodic Memory Traces in the Human Hippocampus

In recent years, multivariate pattern analyses have been performed on functional magnetic resonance imaging (fMRI) data, permitting prediction of mental states from local patterns of blood oxygen-level-dependent (BOLD) signal across voxels. We previously demonstrated that it is possible to predict the position of individuals in a virtual-reality environment from the pattern of activity across voxels in the hippocampus. Although this shows that spatial memories can be decoded, substantially more challenging, and arguably only possible to investigate in humans, is whether it is feasible to predict which complex everyday experience, or episodic memory, a person is recalling. Here we document for the first time that traces of individual rich episodic memories are detectable and distinguishable solely from the pattern of fMRI BOLD signals across voxels in the human hippocampus. In so doing, we uncovered a possible functional topography in the hippocampus, with preferential episodic processing by some hippocampal regions over others. Moreover, our results imply that the neuronal traces of episodic memories are stable (and thus predictable) even over many re-activations. Finally, our data provide further evidence for functional differentiation within the medial temporal lobe, in that we show the hippocampus contains significantly more episodic information than adjacent structures.

openalex-author · Neuropsychologia

Differential engagement of brain regions within a ‘core’ network during scene construction

Reliving past events and imagining potential future events engages a well-established "core" network of brain areas. How the brain constructs, or reconstructs, these experiences or scenes has been debated extensively in the literature, but remains poorly understood. Here we designed a novel task to investigate this (re)constructive process by directly exploring how naturalistic scenes are built up from their individual elements. We "slowed-down" the construction process through the use of auditorily presented phrases describing single scene elements in a serial manner. Participants were required to integrate these elements (ranging from three to six in number) together in their imagination to form a naturalistic scene. We identified three distinct sub-networks of brain areas, each with different fMRI BOLD response profiles, favouring specific points in the scene construction process. Areas including the hippocampus and retrosplenial cortex had a biphasic profile, activating when a single scene element was imagined and when 3 elements were combined together; regions including the intra-parietal sulcus and angular gyrus steadily increased activity from 1 to 3 elements; while activity in areas such as lateral prefrontal cortex was observed from the second element onwards. Activity in these sub-networks did not increase further when integrating more than three elements. Participants confirmed that three elements were sufficient to construct a coherent and vivid scene, and once this was achieved, the addition of further elements only involved maintenance or small changes to that established scene. This task offers a potentially useful tool for breaking down scene construction, a process that may be key to a range of cognitive functions such as episodic memory, future thinking and navigation.

openalex-author · The Journal of Neuroscience

From Threat to Fear: The Neural Organization of Defensive Fear Systems in Humans

Postencounter and circa-strike defensive contexts represent two adaptive responses to potential and imminent danger. In the context of a predator, the postencounter reflects the initial detection of the potential threat, whereas the circa-strike is associated with direct predatory attack. We used functional magnetic resonance imaging to investigate the neural organization of anticipation and avoidance of artificial predators with high or low probability of capturing the subject across analogous postencounter and circa-strike contexts of threat. Consistent with defense systems models, postencounter threat elicited activity in forebrain areas, including subgenual anterior cingulate cortex (sgACC), hippocampus, and amygdala. Conversely, active avoidance during circa-strike threat increased activity in mid-dorsal ACC and midbrain areas. During the circa-strike condition, subjects showed increased coupling between the midbrain and mid-dorsal ACC and decreased coupling with the sgACC, amygdala, and hippocampus. Greater activity was observed in the right pregenual ACC for high compared with low probability of capture during circa-strike threat. This region showed decreased coupling with the amygdala, insula, and ventromedial prefrontal cortex. Finally, we found that locomotor errors correlated with subjective reports of panic for the high compared with low probability of capture during the circa-strike threat, and these panic-related locomotor errors were correlated with midbrain activity. These findings support models suggesting that higher forebrain areas are involved in early-threat responses, including the assignment and control of fear, whereas imminent danger results in fast, likely "hard-wired," defensive reactions mediated by the midbrain.

openalex-author · Neuron

Tracking the Emergence of Conceptual Knowledge during Human Decision Making

Concepts lie at the very heart of intelligence, providing organizing principles with which to comprehend the world. Surprisingly little, however, is understood about how we acquire and deploy concepts. Here, we show that a functionally coupled circuit involving the hippocampus and ventromedial prefrontal cortex (vMPFC) underpins the emergence of conceptual knowledge and its effect on choice behavior. Critically, the hippocampus alone supported the efficient transfer of knowledge to a perceptually novel setting. These findings provide compelling evidence that the hippocampus supports conceptual learning through the networking of discrete memories and reveal the nature of its interaction with downstream valuation modules such as the vMPFC. Our study offers neurobiological insights into the remarkable capacity of humans to discover the conceptual structure of related experiences and use this knowledge to solve exacting decision problems.

openalex-author · Neuropsychologia

Autobiographical memory in semantic dementia: A longitudinal fMRI study

Whilst patients with semantic dementia (SD) are known to suffer from semantic memory and language impairments, there is less agreement about whether memory for personal everyday experiences, autobiographical memory, is compromised. In healthy individuals, functional MRI (fMRI) has helped to delineate a consistent and distributed brain network associated with autobiographical recollection. Here we examined how the progression of SD affected the brain's autobiographical memory network over time. We did this by testing autobiographical memory recall in a SD patient, AM, with fMRI on three occasions, each one year apart, during the course of his disease. At the outset, his autobiographical memory was intact. This was followed by a gradual loss in recollective quality that collapsed only late in the course of the disease. There was no evidence of a temporal gradient. Initially, AM's recollection was supported by the classic autobiographical memory network, including atrophied tissue in hippocampus and temporal neocortex. This was subsequently augmented by up-regulation of other parts of the memory system, namely ventromedial and ventrolateral prefrontal cortex, right lateral temporal cortex, and precuneus. A final step-change in the areas engaged and the quality of recollection then preceded the collapse of autobiographical memory. Our findings inform theoretical debates about the role of the hippocampus and neocortical areas in supporting remote autobiographical memories. Furthermore, our results suggest it may be possible to define specific stages in SD-related memory decline, and that fMRI could complement MRI and neuropsychological measures in providing more precise prognostic and rehabilitative information for clinicians and carers.

openalex-author · Psychological Science

Choking on the Money

A pernicious paradox in human motivation is the occasional reduced performance associated with tasks and situations that involve larger-than-average rewards. Three broad explanations that might account for such performance decrements are attentional competition (distraction theories), inhibition by conscious processes (explicit-monitoring theories), and excessive drive and arousal (overmotivation theories). Here, we report incentive-dependent performance decrements in humans in a reward-pursuit task; subjects were less successful in capturing a more valuable reward in a computerized maze. Concurrent functional magnetic resonance imaging revealed that increased activity in ventral midbrain, a brain area associated with incentive motivation and basic reward responding, correlated with both reduced number of captures and increased number of near-misses associated with imminent high rewards. These data cast light on the neurobiological basis of choking under pressure and are consistent with overmotivation accounts.

openalex-author · OpenGrey (Institut de l'Information Scientifique et Technique)

Neural processes underpinning episodic memory

Episodic memory is the memory for our personal past experiences. Although numerous functional magnetic resonance imaging (fMRI) studies investigating its neural basis have revealed a consistent and distributed network of associated brain regions, surprisingly little is known about the contributions individual brain areas make to the recollective experience. In this thesis I address this fundamental issue by employing a range of different experimental techniques including neuropsychological testing, virtual reality environments, whole brain and high spatial resolution fMRI, and multivariate pattern analysis. Episodic memory recall is widely agreed to be a reconstructive process, one that is known to be critically reliant on the hippocampus. I therefore hypothesised that the same neural machinery responsible for reconstruction might also support ‘constructive’ cognitive functions such as imagination. To test this proposal, patients with focal damage to the hippocampus bilaterally were asked to imagine new experiences and were found to be impaired relative to matched control participants. Moreover, driving this deficit was a lack of spatial coherence in their imagined experiences, pointing to a role for the hippocampus in binding together the disparate elements of a scene. A subsequent fMRI study involving healthy participants compared the recall of real memories with the construction of imaginary memories. This revealed a fronto-temporo-parietal network in common to both tasks that included the hippocampus, ventromedial prefrontal, retrosplenial and parietal cortices. Based on these results I advanced the notion that this network might support the process of ‘scene construction’, defined as the generation and maintenance of a complex and coherent spatial context. Furthermore, I argued that this scene construction network might underpin other important cognitive functions besides episodic memory and imagination, such as navigation and thinking about the future. It is has been proposed that spatial context may act as the scaffold around which episodic memories are built. Given the hippocampus appears to play a critical role in imagination by supporting the creation of a rich coherent spatial scene, I sought to explore the nature of this hippocampal spatial code in a novel way. By combining high spatial resolution fMRI with multivariate pattern analysis techniques it proved possible to accurately determine where a subject was located in a virtual reality environment based solely on the pattern of activity across hippocampal voxels. For this to have been possible, the hippocampal population code must be large and non-uniform. I then extended these techniques to the domain of episodic memory by showing that individual memories could be accurately decoded from the pattern of activity across hippocampal voxels, thus identifying individual memory traces. I consider these findings together with other recent advances in the episodic memory field, and present a new perspective on the role of the hippocampus in episodic recollection. I discuss how this new (and preliminary) framework compares with current prevailing theories of hippocampal function, and suggest how it might account for some previously contradictory data.

openalex-author · Philosophical Transactions of the Royal Society B: Biological Sciences

The construction system of the brain

The ability to construct a hypothetical situation in one's imagination prior to it actually occurring may afford greater accuracy in predicting its eventual outcome. The recollection of past experiences is also considered to be a reconstructive process with memories recreated from their component parts. Construction, therefore, plays a critical role in allowing us to plan for the future and remember the past. Conceptually, construction can be broken down into a number of constituent processes although little is known about their neural correlates. Moreover, it has been suggested that some of these processes may be shared by a number of other cognitive functions including spatial navigation and imagination. Recently, novel paradigms have been developed that allow for the isolation and characterization of these underlying processes and their associated neuroanatomy. Here, we selectively review this fast-growing literature and consider some implications for remembering the past and predicting the future.

openalex-author · Current Biology

Decoding Neuronal Ensembles in the Human Hippocampus

These results show that highly abstracted representations of space are expressed in the human hippocampus. Furthermore, our findings have implications for understanding the hippocampal population code and suggest that, contrary to current consensus, neuronal ensembles representing place memories must be large and have an anisotropic structure.

openalex-author · NeuroImage

Cortical midline involvement in autobiographical memory

Recollecting autobiographical memories of personal past experiences is an integral part of our everyday lives and relies on a distributed set of brain regions. Their occurrence externally in the real world ('realness') and their self-relevance ('selfness') are two defining features of these autobiographical events. Distinguishing between personally experienced events and those that happened to other individuals, and between events that really occurred and those that were mere figments of the imagination, is clearly advantageous, yet the respective neural correlates remain unclear. Here we experimentally manipulated and dissociated realness and selfness during fMRI using a novel paradigm where participants recalled self (autobiographical) and non-self (from a movie or television news clips) events that were either real or previously imagined. Distinct sub-regions within dorsal and ventral medial prefrontal cortex, retrosplenial cortex and along the parieto-occipital sulcus preferentially coded for events (real or imagined) involving the self. By contrast, recollection of autobiographical events that really happened in the external world activated different areas within ventromedial prefrontal cortex and posterior cingulate cortex. In addition, recall of externally experienced real events (self or non-self) was associated with increased activity in areas of dorsomedial prefrontal cortex and posterior cingulate cortex. Taken together our results permitted a functional deconstruction of anterior (medial prefrontal) and posterior (retrosplenial cortex, posterior cingulate cortex, precuneus) cortical midline regions widely associated with autobiographical memory but whose roles have hitherto been poorly understood.

openalex-author · The Journal of Neuroscience

Using Imagination to Understand the Neural Basis of Episodic Memory

Functional MRI (fMRI) studies investigating the neural basis of episodic memory recall, and the related task of thinking about plausible personal future events, have revealed a consistent network of associated brain regions. Surprisingly little, however, is understood about the contributions individual brain areas make to the overall recollective experience. To examine this, we used a novel fMRI paradigm in which subjects had to imagine fictitious experiences. In contrast to future thinking, this results in experiences that are not explicitly temporal in nature or as reliant on self-processing. By using previously imagined fictitious experiences as a comparison for episodic memories, we identified the neural basis of a key process engaged in common, namely scene construction, involving the generation, maintenance and visualization of complex spatial contexts. This was associated with activations in a distributed network, including hippocampus, parahippocampal gyrus, and retrosplenial cortex. Importantly, we disambiguated these common effects from episodic memory-specific responses in anterior medial prefrontal cortex, posterior cingulate cortex and precuneus. These latter regions may support self-schema and familiarity processes, and contribute to the brain's ability to distinguish real from imaginary memories. We conclude that scene construction constitutes a common process underlying episodic memory and imagination of fictitious experiences, and suggest it may partially account for the similar brain networks implicated in navigation, episodic future thinking, and the default mode. We suggest that additional brain regions are co-opted into this core network in a task-specific manner to support functions such as episodic memory that may have additional requirements.

openalex-author · Science

When Fear Is Near: Threat Imminence Elicits Prefrontal-Periaqueductal Gray Shifts in Humans

Humans, like other animals, alter their behavior depending on whether a threat is close or distant. We investigated spatial imminence of threat by developing an active avoidance paradigm in which volunteers were pursued through a maze by a virtual predator endowed with an ability to chase, capture, and inflict pain. Using functional magnetic resonance imaging, we found that as the virtual predator grew closer, brain activity shifted from the ventromedial prefrontal cortex to the periaqueductal gray. This shift showed maximal expression when a high degree of pain was anticipated. Moreover, imminence-driven periaqueductal gray activity correlated with increased subjective degree of dread and decreased confidence of escape. Our findings cast light on the neural dynamics of threat anticipation and have implications for the neurobiology of human anxiety-related disorders.

openalex-author · Trends in Cognitive Sciences

Deconstructing episodic memory with construction

No abstract available from the OpenAlex source record.

openalex-author · Proceedings of the National Academy of Sciences

Patients with hippocampal amnesia cannot imagine new experiences

Amnesic patients have a well established deficit in remembering their past experiences. Surprisingly, however, the question as to whether such patients can imagine new experiences has not been formally addressed to our knowledge. We tested whether a group of amnesic patients with primary damage to the hippocampus bilaterally could construct new imagined experiences in response to short verbal cues that outlined a range of simple commonplace scenarios. Our results revealed that patients were markedly impaired relative to matched control subjects at imagining new experiences. Moreover, we identified a possible source for this deficit. The patients' imagined experiences lacked spatial coherence, consisting instead of fragmented images in the absence of a holistic representation of the environmental setting. The hippocampus, therefore, may make a critical contribution to the creation of new experiences by providing the spatial context into which the disparate elements of an experience can be bound. Given how closely imagined experiences match episodic memories, the absence of this function mediated by the hippocampus, may also fundamentally affect the ability to vividly re-experience the past.

openalex-author · Neuropsychologia

Impaired spatial and non-spatial configural learning in patients with hippocampal pathology

The hippocampus has been proposed to play a critical role in memory through its unique ability to bind together the disparate elements of an experience. This hypothesis has been widely examined in rodents using a class of tasks known as "configural" or "non-linear", where outcomes are determined by specific combinations of elements, rather than any single element alone. On the basis of equivocal evidence that hippocampal lesions impair performance on non-spatial configural tasks, it has been proposed that the hippocampus may only be critical for spatial configural learning. Surprisingly few studies in humans have examined the role of the hippocampus in solving configural problems. In particular, no previous study has directly assessed the human hippocampal contribution to non-spatial and spatial configural learning, the focus of the current study. Our results show that patients with primary damage to the hippocampus bilaterally were similarly impaired at configural learning within both spatial and non-spatial domains. Our data also provide evidence that residual configural learning can occur in the presence of significant hippocampal dysfunction. Moreover, evidence obtained from a post-experimental debriefing session suggested that patients acquired declarative knowledge of the underlying task contingencies that corresponded to the best-fit strategy identified by our strategy analysis. In summary, our findings support the notion that the hippocampus plays an important role in both spatial and non-spatial configural learning, and provide insights into the role of the medial temporal lobe (MTL) more generally in incremental reinforcement-driven learning.

Demis Hassabis

Papers

Mastering Atari, Go, chess and shogi by planning with a learned model

Advancing regulatory variant effect prediction with AlphaGenome

SIMA 2: A Generalist Embodied Agent for Virtual Worlds

Olympiad-level formal mathematical reasoning with reinforcement learning

AlphaGenome: advancing regulatory variant effect prediction with a unified DNA sequence model

Bridging the human–AI knowledge gap through concept discovery and transfer in AlphaZero

Addendum: Accurate structure prediction of biomolecular interactions with AlphaFold 3

Learning high-accuracy error decoding for quantum processors

Scalable watermarking for identifying large language model outputs

De novo design of high-affinity protein binders with AlphaProteo

Accurate structure prediction of biomolecular interactions with AlphaFold 3

Capabilities of Gemini Models in Medicine

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

TacticAI: an AI assistant for football tactics

Gemma: Open Models Based on Gemini Research and Technology

Accelerating Scientific Discovery with AI

AlphaFold Protein Structure Database in 2024: providing structure coverage for over 214 million protein sequences

Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero

AI Helping Science: The ‘Shape’ of Things to Come

Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network

The Protein Structure Prediction Revolution and Its Implications for Medicine

Predictions for AlphaMissense

Accurate proteome-wide missense variant effect prediction with AlphaMissense

Diversifying AI: Towards Creative Chess with AlphaZero

Source code for AlphaMissense

Faster sorting algorithms discovered using deep reinforcement learning

« L’IA va accélérer les avancées scientifiques »

Mastering the game of Stratego with model-free multiagent reinforcement learning

Acquisition of chess knowledge in AlphaZero

Figure Data for the paper "Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning"

Discovering faster matrix multiplication algorithms with reinforcement learning

Improving alignment of dialogue agents via targeted human judgements

Response to Comment on “Pushing the frontiers of density functionals by solving the fractional electron problem”

Multiagent off-screen behavior prediction in football

Magnetic control of tokamak plasmas through deep reinforcement learning

Tackling Climate Change with Machine Learning

Reimagining chess with AlphaZero

Protein structure predictions to atomic accuracy with AlphaFold

3D-Beacons: decreasing the gap between protein sequences and structures through a federated network of protein structure data resources

Pushing the frontiers of density functionals by solving the fractional electron problem

Scaling Language Models: Methods, Analysis &amp; Insights from Training Gopher

Advancing mathematics by guiding human intuition with AI

Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons

DeepMind: From Games to Scientific Discovery

AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models

Protein complex prediction with AlphaFold-Multimer

Applying and improving <scp>AlphaFold</scp> at <scp>CASP14</scp>

Author response for "Applying and improving AlphaFold at CASP14"

Highly accurate protein structure prediction for the human proteome

Highly accurate protein structure prediction with AlphaFold

Game Plan: What AI can do for Football, and What Football can do for AI

Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records

Atomistic graph networks for experimental materials property prediction

Alchemy: A structured task distribution for meta-reinforcement learning.

“With AI, you might unlock some of the secrets about how life works”

A model of egocentric to allocentric understanding in mammalian brains

Addendum: International evaluation of an AI system for breast cancer screening

Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess

Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons

Predicting conversion to wet age-related macular degeneration using deep learning

Author Correction: Unveiling the predictive power of static structure in glassy systems

Unveiling the predictive power of static structure in glassy systems

MEMO: A Deep Network for Flexible Combination of Episodic Memories

Improved protein structure prediction using potentials from deep learning

A distributional code for value in dopamine-based reinforcement learning

International evaluation of an AI system for breast cancer screening

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13)

Developing Deep Learning Continuous Risk Models for Early Adverse Event Prediction in Electronic Health Records: an AKI Case Study

A clinically applicable approach to continuous prediction of future acute kidney injury

Human-level performance in 3D multiplayer games with population-based reinforcement learning

Slow escape decisions are swayed by trait anxiety

Reinforcement Learning, Fast and Slow

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Big-Loop Recurrence within the Hippocampal System Supports Integration of Information across Episodes

Clinically applicable deep learning for diagnosis and referral in retinal disease

Neural scene representation and rendering

Prefrontal cortex as a meta-reinforcement learning system

Scaling Language Models: Methods, Analysis & Insights from Training Gopher