Paper

Author response: Hippocampal place cells construct reward related sequences through unexplored space

Article Figures and data Abstract eLife digest Introduction Results Discussion Materials and method References Decision letter Author response Article and author information Metrics Abstract Dominant theories of hippocampal function propose that place cell representations are formed during an animal's first encounter with a novel environment and are subsequently replayed during off-line states to support consolidation and future behaviour. Here we report that viewing the delivery of food to an unvisited portion of an environment leads to off-line pre-activation of place cells sequences corresponding to that space. Such ‘preplay’ was not observed for an unrewarded but otherwise similar portion of the environment. These results suggest that a hippocampal representation of a visible, yet unexplored environment can be formed if the environment is of motivational relevance to the animal. We hypothesise such goal-biased preplay may support preparation for future experiences in novel environments. https://doi.org/10.7554/eLife.06063.001 eLife digest As an animal explores an area, part of the brain called the hippocampus creates a mental map of the space. When the animal is in one location, a few neurons called ‘place cells’ will fire. If the animal moves to a new spot, other place cells fire instead. Each time the animal returns to that spot, the same place cells will fire. Thus, as the animal moves, a place-specific pattern of firing emerges that scientists can view by recording the cells' activity and which can be used to reconstruct the animal's position. After exploring a space, the hippocampus may replay the new place-specific pattern of activity during sleep. By doing so, the brain consolidates the memory of the space for return visits. Recent evidence now suggests that these mental rehearsals—or internal simulations of the space—may begin even before a new space has been explored. Now, Ólafsdóttir, Barry et al. report that whether an animal's brain simulates a first visit to a new space depends on whether the animal anticipates a reward. In the experiments, rats were allowed to run up to the junction in a T-shaped track. The animals could see into each of the arms, but not enter them. Food was then placed in one of the inaccessible arms. Ólafsdóttir, Barry et al. recorded the firing of place cells in the brain of the animals when they were on the track and during a rest period afterwards. The rats were then allowed onto the inaccessible arms, and again their brain activity was recorded. In the rest period after the rats first viewed the inaccessible arms, the place cell pattern that would later form the mental map of a journey to and from the food-containing arm was pre-activated. However, the place cell pattern that would become the mental map of the other inaccessible arm was not activated before the rat explored that area. Therefore, Ólafsdóttir, Barry et al. suggest that the perception of reward influences which place cell pattern is simulated during rest. An implication of these findings is that the brain preferentially simulates past or future experiences that are deemed to be functionally significant, such as those associated with reward. A future challenge will be to determine whether this goal-related simulation of unvisited spaces predicts and is needed for behaviour such as successful navigation to a goal. https://doi.org/10.7554/eLife.06063.002 Introduction We investigated whether the presence of an inaccessible goal in an unvisited portion of an environment was sufficient to elicit pre-activation (‘preplay’) of hippocampal place cell sequences that will subsequently represent runs through the unvisited environment. To this end, we recorded from ensembles of place cells (O'Keefe and Dostrovsky, 1971) (4 rats, 37–66 place cells each, 212 cells in total) while rats ran along a T-shaped track (Figure 1—figure supplement 1, Table 1) with visible yet inaccessible arms (Figure 1A)—RUN1. One arm (counter balanced between animals) was subsequently cued with food while the animal remained on the track—GOAL-CUE. During a rest period before RUN1 (REST1) and after GOAL-CUE (REST2), spiking events—periods of 300 ms or less, where at least 15% of cells were active (Foster and Wilson, 2006; Diba and Buzsaki, 2007)—were analysed. These spiking events were associated with significantly higher power in the ripple spectrum (80–250 Hz) than other comparable periods (Figure 1—figure supplement 2). To investigate whether paths on the cued and uncued arms were preplayed we assessed the match between the order in which cells fired during spiking events and during future runs on the arms (RUN2, Figure 1—figure supplement 3). Specifically, we computed the rank-order correlations between spiking events and sequences of place cells active on the arms, referred to as templates (Lee and Wilson, 2002; Foster and Wilson, 2006; Diba and Buzsaki, 2007; Dragoi and Tonegawa, 2011) (Figure 1—figure supplement 4). Preplay events were identified as those with either a significant positive or negative correlation—a two-tailed test, each tail tested at the 97.5% level. These preplay events were found to exhibit higher power in the ripple spectrum than non-significant spiking events (Figure 1—figure supplement 2). To establish significance at the population level, the proportion of preplay events measured was compared to a null distribution generated by calculating correlations between place cell templates and shuffled sequences from events (see Figure 1B–C). Table 1 Experimental parameters https://doi.org/10.7554/eLife.06063.003 R1838R505R584R504All rats (mean)Cue bias (dwell time) RUN10.330.32−0.05−0.200.10 GOAL-CUE0.460.200.310.320.33Cue bias (looking time) RUN1−0.09−0.020.01−0.36−0.11 GOAL-CUE0.060.310.090.050.13RUN2 arm bias RUN21.00.841.01.00.96Session duration (min) SLEEP16088757474 RUN1131013911 GOAL-CUE1017121113 SLEEP26067716065 RUN23419353130Template length (number of cells) Up cued arm2036534338 Down cued arm1535453332 Up uncued arm1926454033 Down uncued arm1532414333 Figure 1 with 4 supplements see all Download asset Open asset Preferential preplay of a behaviourally relevant, unvisited environment. (A) Experimental protocol. (i) Prior to running on the track, the animals rested for at least an hour (REST1). (ii) Following REST1, animals ran 20 laps on the stem (RUN1). Access to the arms was blocked by a barrier at the end of the stem which the animals could see through but not pass. (iii) Following RUN1, the experimenter baited one arm so to provoke the animals' interest in that arm (GOAL-CUE). (iv) Following goal-cueing, the animal rested for at least another hour (REST2). (v) Following REST2, the barrier was removed and the animals traversed the extent of the track, in alternate L-shaped laps (RUN2). (B) Left: an example template for a run to the cued arm. x-axis shows location on the track and y-axis cell IDs. Right: Example raster plots of preplay events—the title shows the correlation between the preplay event and the template sequence. C same as B but for the uncued template. https://doi.org/10.7554/eLife.06063.004 Results During GOAL-CUE, all four animals displayed more interest in the cued arm than the uncued arm (as indexed by the difference in time spent on the cued side of the stem vs the uncued side, divided by the total time spent on either side, mean bias = 0.33). In contrast, prior to goal-cueing, two animals spent more time on the uncued side of the stem (mean bias = 0.10, see Table 1 for results for individual animals). Moreover, during GOAL-CUE all animals also spent more time looking towards the cued arm than the uncued arm (mean bias = 0.13), again this bias was not observed prior to goal-cueing when only one animal spent more time looking towards the cued arm (mean bias = −0.11, see Table 1 for results of individual animals). Furthermore, during RUN2 when the barrier was first removed and the food cue was no longer present, all four animals initially turned towards the cued arm and spent more time on the cued rather than the uncued arm (mean bias = 0.96, see Table 1 for results for individual animals). Consistent with the behavioural bias, in REST2 we found significant preplay of the yet unvisited cued arm (7.37% preplay events, p < 0.001, binomial test vs chance, Figure 2A,D). Conversely, the uncued arm was not significantly preplayed (4.41% preplay events p = 0.33, vs cued arm: p < 0.001, Figure 2B,D). Similarly significant effects were found when animals were analysed individually (see Figure 2D, Table 2), although the results for one animal were based on a relatively small sample (number of cued preplay events = 9, R1838), it still showed significant preplay of the cued arm. Moreover, the results were corroborated by a distribution-based analysis; namely, comparing the area under the curve (AUC) of bootstrapped cumulative distributions of absolute correlations for each arm to that of their shuffle distribution (Figure 2A,B, cued: p < 0.001, uncued: p = 0.22, cued vs uncued: p = 0.0044, Figure 2—figure supplement 1). Preplay events of the cued arm were equally likely to represent paths to and from the cued arm (7.34% vs 7.40% p = 0.49) and to run towards (‘forward’) and away (‘reverse’) from the ends of the arms (6.93% vs 7.80%, p = 0.18, Figure 3A). Moreover, the amount of preplay exhibited by each animal appeared to be predicted by the interest they displayed for the cued arm during GOAL-CUE (Figure 3—figure supplement 3). Importantly, preferential preplay of the cued arm could not be explained by differences in the number of cells with fields on the arms (Figure 3—figure supplement 2A–B), spike-sorting quality (cells with neighbouring place fields were as well separated in cluster-space as those with distant fields, p = 0.45, 2-sample Kolmogorov–Smirnov test), place field stability on the two arms (cued arm stability r = 0.54 vs uncued arm stability r = 0.49, p = 0.15) or the location of place fields on the cued arm (Figure 3B, p = 0.22 two-sample Kolmogorov–Smirnov test). In sum, we found during rest after goal-cueing, significant and preferential preplay of an unvisited and motivationally relevant portion of the environment. Figure 2 with 2 supplements see all Download asset Open asset Preplay is a function of goal-cueing. (A) Bootstrapped cumulative distribution of (absolute) correlations between spiking events and the cued template in REST2 (red = data, black = shuffle). Lighter areas of the curve show 1 standard deviation of the mean. Inset: difference between the data and shuffle distributions. If there are more high correlations in the data compared to the shuffle then the data distribution will deviate below the shuffle distribution. (B–C) same as A but for the uncued template in REST2 and the cued template in REST1, respectively. (D) Proportion of spiking events categorised as preplay events in REST2 for the cued and uncued arms. Bars show mean for all animals, and the black lines show the result for each animal. The grey dashed line shows the proportion of preplay events expected by chance. (E) Same as D but comparing proportion of preplay events for the cued template in REST1 and REST2. https://doi.org/10.7554/eLife.06063.009 Table 2 REST period results https://doi.org/10.7554/eLife.06063.012 AnimalArm# spiking events# preplay events% preplay% chancep-valueREST2 R1838Cued44920.456.866.56 × 10−4Uncued26311.546.770.10 R505Cued631386.024.570.037Uncued860293.373.820.72 R584Cued437327.324.736.20 × 10−3Uncued398225.534.740.19 R504Cued516417.955.150.0027Uncued373195.094.400.21 All ratsCued16281207.374.864.12 × 10−6Uncued1657734.414.220.33REST1 R1838Cued6334.767.050.66Uncued3525.717.370.48 R505Cued664395.874.340.025Uncued1215383.133.450.70 R584Cued269145.204.670.28Uncued247218.504.720.0035 R504Cued31161.934.440.99Uncued173116.364.210.063 All ratsCued1307624.744.560.34Uncued1670724.313.800.12 Summary results from REST1 and REST2 for the cued and uncued arms for individual animals. # Spiking events = total number of spiking events recorded. # preplay events = number of significant spiking events. % preplay = Proportion of the spiking events that qualified as preplay events (i.e., that were significant), expressed as a percentage. % chance = proportion of spiking events from the shuffled data that qualified as preplay events, expressed as a percentage. p-value = probability, derived from a binomial test, of obtaining the observed number of preplay events for each template given the chance level calculated from the shuffled data. Figure 3 with 4 supplements see all Download asset Open asset Spatial and temporal dynamics of preplay. (A) The proportion of preplay events when negative (‘reverse’) and positive (‘forward’) spiking event correlations are analysed separately. Bars show means for all data and black lines the results for each animal. (B) Frequency of preplay events vs location on the cued (red) and uncued (blue) arms normalised by the density of place field centres—100% indicates the expected number of preplay events under an even distribution across each arm. No bias towards particular sections of the arms was evident (cued p = 0.22, uncued p = 0.15, based on a two-sample Kolmogorov–Smirnov test). Lighter areas show standard error of the mean (SEM) and the black line the expected distribution. (C) Bootstrapped cumulative distribution of (absolute) correlations between spiking events and the cued template during GOAL-CUE (red = data, black = shuffle). Lighter areas of the curve show 1 standard deviation of the mean. Inset: difference between the data and shuffle distributions. (D) Ratio of activity levels between cued and uncued arm cells (cued/uncued) during events for the first and second half of each experimental period. Red line shows mean ratio, derived from bootstrapped data, obtained for each period, and the shaded areas 1sd of the bootstrapped data. The black horizontal line indicates equal rates for the two arms. * = significantly different from 1 based on 95% confidence intervals. https://doi.org/10.7554/eLife.06063.013 Does goal-cueing trigger preplay? If so, there should be a greater number of significant pre-play events in REST2 compared to REST1 which was recorded before animals had visited or seen any part of the environment. Preplay of the cued arm was higher in REST2 than REST1 (7.37% vs 4.74%, p < 0.001, Figure 2C,E), an effect that was seen for all animals (Table 2). Indeed, the cued arm was not significantly preplayed during REST1 (4.74%, p = 0.34). Again, the result was corroborated using an AUC analysis (Figure 2C, Figure 2—figure supplement 1). Thus, we find preplay only occurs during rest periods recorded after goal-cueing. However, it is possible that the frequency of preplay might decrease as a function of the temporal gap between rest and behaviour. As such our failure to detect preplay in REST1 might be due to the greater delay between REST1 and RUN2 than between REST2 and RUN2. To address this we analysed preplay of the stem (i.e., RUN1) during REST1. We did not find preplay of the stem (4.12% preplay events, p = 0.44, AUC analysis p = 0.053, Figure 2—figure supplement 2, Table 3, RUN1 REST1 vs cued REST2: p < 0.001). Consequently, these results imply that the preplay of the unvisited, yet visible, environment we observed in REST2 was driven by behavioural cueing of that environment. Table 3 REST1 stem results https://doi.org/10.7554/eLife.06063.018 Animal# spiking events# preplay events% preplay% chancep-valueR18386834.416.010.59R505980353.574.030.74R584329175.174.530.24R504396184.553.500.11All rats1773734.124.080.44 Summary results from REST1 analysing preplay of the stem. # Spiking events = total number of spiking events recorded. # Preplay events = number of significant spiking events. % preplay = Proportion of the spiking events that qualified as preplay events (i.e., that were significant), expressed as a percentage. % chance = proportion of spiking events from the shuffled data that qualified as preplay events, expressed as a percentage. p-value = probability, derived from a binomial test, of obtaining the observed number of preplay events for each template given the chance level calculated from the shuffled data. At what point does preferential preplay of the cued arm emerge? Plausibly preplay might be initiated immediately when the cued arm is baited (start of GOAL-CUE) and simply persist into the subsequent REST2 period, alternatively the bias may only arise during rest. Due to the short duration of the goal-cueing period (∼10 min) a relatively small number of spiking events were recorded for the two arms during this period (172 and 170 for the cued and uncued arm respectively). However, based on a bootstrapped comparison of the AUC for absolute correlations from the cued and uncued arm vs shuffled distributions, we found that the cued but not the uncued arm was preplayed (p = 0.02, p = 0.24 respectively, Figure 3C, Figure 3—figure supplement 1). A direct comparison of the proportion of preplay events for the cued vs uncued arm was marginally not significant (6.4% vs 4.12%, p = 0.052, see Table 4 for results for individual animals). Finally, to validate the results from this smaller dataset we carried out a further, more inclusive, analysis. Specifically, we tracked the temporal evolution of the bias in preplay by comparing the activity of cells from the cued and uncued arms at different points during the experiment. For every spiking event we computed the mean rate for cells that would subsequently have fields on the cued arm compared to those with fields on the uncued arm. During REST1 and RUN1 the future cued and uncued arm cells did not differ in activity, this was true for both the first and second half of these periods (mean cued/uncued rate ratio: REST1 early ratio = 0.96, p = 0.88, REST1 late ratio = 1.04 p = 0.09, RUN1 early ratio = 1.09 p = 0.32, RUN1 late ratio = 1.18 p = 0.22, Figure 3D). However, during GOAL-CUE cued arm cells were significantly more active than uncued arm cells, an effect that was most pronounced during the first half (5 min) of the cueing period (GOAL-CUE early ratio = 1.78, p = 0.01, late = 1.46, p < 0.01). Finally, the difference between the two groups persisted through the subsequent rest period, albeit attenuating with time (REST2 early ratio = 1.30 p < 0.001, REST2 late ratio = 1.10 p < 0.01, Figure 3D). Importantly, control analyses showed that the bias in sequential preplay is not a mere product of differing activity levels of the cells for the two arms (Figure 3—figure supplement 2C–D). Together, these findings indicate that biased pre-activation of future experiences is instantiated at the point when an environment becomes motivationally-relevant. Table 4 GOAL-CUE results https://doi.org/10.7554/eLife.06063.019 AnimalArm# spiking events# preplay events% preplay% chancep-valueR1838Cued50070.30Uncued50070.30R505Cued4536.674.020.11Uncued4812.083.570.52R584Cued11187.214.810.087Uncued11254.464.820.46R504Cued11004.360.39Uncued7114.295.000.044All ratsCued172116.404.640.11Uncued17074.124.490.50 Summary results from GOAL-CUE analysis # Spiking events = total number of spiking events recorded. # Preplay events = number of significant spiking events. % preplay = Proportion of the spiking events that qualified as preplay events (i.e., that were significant), expressed as a percentage. % chance = proportion of spiking events from the shuffled data that qualified as preplay events, expressed as a percentage. p-value = probability, derived from a binomial test, of obtaining the observed number of preplay events for each template given the chance level calculated from the shuffled data. Finally, to corroborate the results obtained from the rank-order analysis of spike sequences, we applied a Bayesian spatial reconstruction algorithm (Davidson et al., 2009; Bendor and Wilson, 2012) to the data from the two rest sessions. In contrast to the rank-order method, which utilised only the first spike emitted by each cell, the Bayesian decoding approach used all spikes emitted during an event. The Bayesian decoding approach uses the spiking activity of all simultaneously recorded place cells to calculate the posterior probability of an animal being at any position in the environment, based on a Poisson spiking framework. (See Materials and methods; R1838 was excluded from this analysis due to low cell yield). The Bayesian decoder performed equally well on the cued and uncued arms (Figure 4A,B, median error for both arms = 10.0 cm). Next, we applied the Bayesian decoding to spiking events during the rest sessions. First, we calculated the posterior probabilities for 5 ms non-overlapping bins, which generated a posterior probability matrix for each event (Figure 4C,D). A spiking event that reflects a constant speed run through the environment will show an increased posterior along a line in the decoded posterior matrix. Therefore, for each spiking event we fit a line that accounted for the maximum variance and calculated its goodness of fit (Figure 4C,D). To assess if that event represented a significant preplay event: we generated 1000 posterior probability matrices by shuffling the identities of cells included in the event, fit lines on all matrices, and calculated their goodness of fits. Events whose goodness of fits exceeded the 95th percentile of the shuffled distributions were labelled as preplay events. Again, during REST2, we found preplay of the cued arm (7.64% of events, p < 0.001, binomial test) but not of the uncued arm (4.78% of events, p = 0.55, vs cued = p < 0.001, Figure 4E). Moreover, neither the cued nor the uncued arm were significantly preplayed in REST1 (cued = 5.04%, p = 0.43, uncued = p = 0.55, Figure Thus, this more analysis the same pattern of results as by our analysis; that pre-activation of future place cell sequences is a and Figure 4 Download asset Open asset Bayesian reconstruction preferential cued arm preplay. matrices based on RUN2 data decoding for the cued (A) and uncued arms show the mean posterior probability distribution across the arm by the true position of the rat bins, data for as power away from the both arms of the animal's position at all points on the track. preplay events for the cued (C) and uncued (D) arms the line that fits the decoded event indicates time (5 the event, y-axis position on title indicates probability of obtaining a by the null distribution of fits obtained by shuffling the cell identities for each event quality of proportion of shuffled (E) Proportion of spiking events categorised as preplay events for the cued and uncued arms in REST2. Bars show mean for all data with Same as but comparing preplay in REST1 and REST2 for the cued arm. Discussion Preplay of a visible, unvisited environment that has motivational relevance to an animal with from and a for the hippocampus in future and and future et al., 2007; et al., Moreover, these findings with replay can be by reward and and 2009; and but this to preplay of In our data, sequences of place cells to and away from the goal were preplayed with similar frequency and in both and (i.e., positive and negative of sequences towards goal has been with the of future and Plausibly the proportion of preplay events we observed to the cued goal might also given the animals' subsequent for the cued arm. However, preplay of sequences from the cued goal towards the stem are one it is possible that these sequences simply for a return to the of the stem after of the However, replay has been as a to the temporal (Foster and Wilson, 2006; Foster and a to a of that to a successful In the of when the cued arm has not been an is that paths towards the goal might be simulated using preplay then and from during in a similar to that to for periods of and rest (Foster and Wilson, 2006; and et al., have of that have yet to be and Tonegawa, is to the activity of cell ensembles et al., that subsequently become to in the environment. We did not preplay in the preplay of stem during the of this is we that other also not find preplay for a novel environment and and Wilson, However, it likely that other but in the preplay and Tonegawa, such as the and visible but inaccessible arms, might have this Indeed, the AUC analysis comparing spiking events recorded during REST1 the stem template Figure 2—figure supplement was not significant (p = that with more data or a environment we might have found preplay for the stem. In a similar one might also have expected to a rate of preplay for the uncued arm during REST2. it possible that the of the environment such an effect in that the motivationally relevant cued arm was preplayed at the of the uncued arm. to be is if this bias reflects an active associated with the or if the relevance of the cued arm simply it to more cell In sum, our data indicates that preplay of an unvisited environment can be initiated in response to viewing that environment if it is relevant to future that were in the of hippocampal representations for novel spaces and and Wilson, 2002; et al., Furthermore, although our data does not direct support for preplay it also does not and is with the that the hippocampus the of novel spatial experiences as by other example see Dragoi and These results are with the that place cells form or et al., active but being associated with the cued arm during GOAL-CUE, but not the that place cell firing is driven in a et al., We that these findings support the of preplay for preparation for future experiences and Buzsaki, 2007; and Foster and and but it to future experiences in yet to be explored. to be seen if the of the hippocampus to future spatial sequences is to the of more temporal et al., and Materials and method and a rats were used in this All were by the to the and in the of rats at two each four of and to the and One animal a Finally, one animal one four of to the and one four

Authors: H Freyja Ólafsdóttir · Caswell Barry · Aman B Saleem · Demis Hassabis · Hugo J Spiers

Related discussions