• Three ways an agent might learn, through sequential memory access, the relationship between actions and rewards. Left: when reward is first encountered, through reverse reactivation; center: during sleep or rest through "offline" reactivation of the sequence; right: prior to choice, by prospective (forward) activation. The latter case is the most commonly envisioned in theories of model-based deliberation, but replay of all three sorts exists, and human neuroimaging evidence suggests that all can support decisions.

  • From the Daw Lab: Prioritized memory access explains planning and hippocampal replay

    Making good decisions requires candidate options to be evaluated by accessing memories of relevant experiences. This deliberation is essential for goal-directed behavior, and not engaging in it underlies habits. In a new publication in Nature Neuroscience, the Daw Lab considers a more granular question: which specific memories are considered or ignored during deliberation and what is their ultimate effect on future decisions?

    To address this question, the authors start with the idea that accessing memories helps us make better choices and harvest more rewards. This allowed them to quantify the utility of each memory in terms of how much extra reward would be earned following its access, suggesting a simple account of which memories are best. Through simulations, they investigated the patterns of memory access that would be obtained if an animal could access spatial memories sequentially, ordered by utility. These simulations showed that the patterns of reactivation predicted by the theory resemble, in many ways, the replay sequences measured in the rat hippocampus with electrophysiological recordings, including sequences that extend in forward and backward directions, and even patterns of memory reactivation during sleep. This simple theory provides a unifying explanation for numerous findings about place cells, and suggests a mechanism whose dysfunction could underlie pathologies like rumination and craving.