FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2979619911> ?p ?o ?g. }

• W2979619911 endingPage "77" @default.
• W2979619911 startingPage "63" @default.
• W2979619911 abstract "Animals infer when and where a reward is available from experience with informative sensory stimuli and their own actions. In vertebrates, this is thought to depend upon the release of dopamine from midbrain dopaminergic neurons. Studies of the role of dopamine have focused almost exclusively on their encoding of informative sensory stimuli; however, many dopaminergic neurons are active just prior to movement initiation, even in the absence of sensory stimuli. How should current frameworks for understanding the role of dopamine incorporate these observations? To address this question, we review recent anatomical and functional evidence for action-related dopamine signaling. We conclude by proposing a framework in which dopaminergic neurons encode subjective signals of action initiation to solve an internal credit assignment problem. Animals infer when and where a reward is available from experience with informative sensory stimuli and their own actions. In vertebrates, this is thought to depend upon the release of dopamine from midbrain dopaminergic neurons. Studies of the role of dopamine have focused almost exclusively on their encoding of informative sensory stimuli; however, many dopaminergic neurons are active just prior to movement initiation, even in the absence of sensory stimuli. How should current frameworks for understanding the role of dopamine incorporate these observations? To address this question, we review recent anatomical and functional evidence for action-related dopamine signaling. We conclude by proposing a framework in which dopaminergic neurons encode subjective signals of action initiation to solve an internal credit assignment problem. Dopamine-releasing neurons of the vertebrate midbrain (mDA neurons) fire at continuous low rates at rest but occasionally exhibit transient (∼0.2 s) increases and decreases in activity (Grace and Bunney, 1984aGrace A.A. Bunney B.S. The control of firing pattern in nigral dopamine neurons: single spike firing.J. Neurosci. 1984; 4: 2866-2876Crossref PubMed Google Scholar, Grace and Bunney, 1984bGrace A.A. Bunney B.S. The control of firing pattern in nigral dopamine neurons: burst firing.J. Neurosci. 1984; 4: 2877-2890Crossref PubMed Google Scholar). These “phasic” bursts of activity are thought to be a critical modulatory teaching signal that underlies the essential act of learning to pursue reward (Palmiter, 2008Palmiter R.D. Dopamine signaling in the dorsal striatum is essential for motivated behaviors: lessons from dopamine-deficient mice.Ann. N Y Acad. Sci. 2008; 1129: 35-46Crossref PubMed Scopus (201) Google Scholar). The information encoded in phasic mDA neuron activity has thus been a subject of substantial interest for the past several decades. Phasic mDA neuron activity has most commonly been examined in relation to external sensory stimuli that convey information about reward (Schultz, 2016Schultz W. Dopamine reward prediction-error signalling: a two-component response.Nat. Rev. Neurosci. 2016; 17: 183-195Crossref PubMed Scopus (176) Google Scholar). However, it has long been appreciated that the responses are subjective; i.e., the magnitude of phasic responses are dependent upon the motivational state, learning, and perception (Phillips and Olds, 1969Phillips M.I. Olds J. Unit activity: motivation-dependent responses from midbrain neurons.Science. 1969; 165: 1269-1271Crossref PubMed Google Scholar, Schultz, 2015Schultz W. Neuronal reward and decision signals: from theories to data.Physiol. Rev. 2015; 95: 853-951Crossref PubMed Scopus (250) Google Scholar, Wise, 2004Wise R.A. Dopamine, learning and motivation.Nat. Rev. Neurosci. 2004; 5: 483-494Crossref PubMed Google Scholar). Given that mDA neurons lie within the basal ganglia, a structure involved in motor learning and execution, it is reasonable to consider whether the subjectivity of their signals results from a participation in actions emitted by an animal in response to an external stimulus (a process also determined by motivational state, learning, etc.). Influential early studies concluded from recordings of putative mDA neurons in monkeys that movement-related activity was secondary to a subjective encoding of sensory stimuli (Romo and Schultz, 1990Romo R. Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements.J. Neurophysiol. 1990; 63: 592-606Crossref PubMed Scopus (264) Google Scholar, Schultz and Romo, 1990Schultz W. Romo R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions.J. Neurophysiol. 1990; 63: 607-624Crossref PubMed Scopus (226) Google Scholar, Schultz et al., 1983Schultz W. Ruffieux A. Aebischer P. The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation.Exp. Brain Res. 1983; 51: 377-387Crossref Google Scholar). This original description has led to quantitative theoretical frameworks for mDA neuron function, including the reward prediction error (RPE; Schultz et al., 1997Schultz W. Dayan P. Montague P.R. A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599Crossref PubMed Scopus (4669) Google Scholar) and incentive salience (Berridge, 2007Berridge K.C. The debate over dopamine’s role in reward: the case for incentive salience.Psychopharmacology (Berl.). 2007; 191: 391-431Crossref PubMed Scopus (1302) Google Scholar) theories, as well as those that address salience or aversive signaling (Bromberg-Martin et al., 2010Bromberg-Martin E.S. Matsumoto M. Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting.Neuron. 2010; 68: 815-834Abstract Full Text Full Text PDF PubMed Scopus (953) Google Scholar, de Jong et al., 2019de Jong J.W. Afjei S.A. Pollak Dorocic I. Peck J.R. Liu C. Kim C.K. Tian L. Deisseroth K. Lammel S. A neural circuit mechanism for encoding aversive stimuli in the mesolimbic dopamine system.Neuron. 2019; 101: 133-151.e7Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar, Redgrave and Gurney, 2006Redgrave P. Gurney K. The short-latency dopamine signal: a role in discovering novel actions?.Nat. Rev. Neurosci. 2006; 7: 967-975Crossref PubMed Scopus (449) Google Scholar, Watabe-Uchida and Uchida, 2019Watabe-Uchida M. Uchida N. Multiple dopamine systems: weal and woe of dopamine.Cold Spring Harb. Symp. Quant. Biol. 2019; (Published online February 20, 2019)https://doi.org/10.1101/sqb.2018.83.037648Crossref Scopus (1) Google Scholar). Despite the early conclusion that sensory encoding predominated mDA neuron activity, only a few direct quantitative comparisons existed. For example, a single direct comparison came from a study in which monkeys executed both self-initiated and cued reaches for possible food rewards; 16% of putative substantia nigra pars compacta (SNc)-dopamine (DA) neurons exhibited peri-reach activation, as compared to 77% and 84% of putative SNc-DA neurons responding to action-inducing sensory stimuli (Romo and Schultz, 1990Romo R. Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements.J. Neurophysiol. 1990; 63: 592-606Crossref PubMed Scopus (264) Google Scholar). From this observation, a profound, although subsequently replaced, conclusion was reached: mDA neurons are excited by “stimuli eliciting immediate behavioral reactions.” This original description placed mDA activity squarely in the middle of sensorimotor transformations rather than as a more passive qualifier of sensory stimuli. Over the last several years, many studies have begun to restore a focus on how actions initiated by an animal are reflected in the activity of identified mDA neurons. Here, we review this literature with a special focus on how mDA neuron activity is poised anatomically to integrate information about action, encode properties of action, and contribute to reinforcement learning. This review draws upon recent evidence in mammals due to space considerations, but we note that increasing evidence in insects (Cohn et al., 2015Cohn R. Morantte I. Ruta V. Coordinated and Compartmentalized Neuromodulation Shapes Sensory Processing in Drosophila.Cell. 2015; 163: 1742-1755Abstract Full Text Full Text PDF PubMed Scopus (97) Google Scholar) and birds (Gadagkar et al., 2016Gadagkar V. Puzerey P.A. Chen R. Baird-Daniel E. Farhang A.R. Goldberg J.H. Dopamine neurons encode performance error in singing birds.Science. 2016; 354: 1278-1282Crossref PubMed Scopus (41) Google Scholar, Xiao et al., 2018Xiao L. Chattree G. Oscos F.G. Cao M. Wanat M.J. Roberts T.F. A Basal Ganglia Circuit Sufficient to Guide Birdsong Learning.Neuron. 2018; 98: 208-221.e5Abstract Full Text Full Text PDF PubMed Scopus (6) Google Scholar) further implicates action-related signaling in the function of dopamine. We conclude with a proposal for incorporating action encoding with sensory stimulus responses into a theoretical account of the phasic activity of mDA neurons and its role as a teaching signal for reinforcement. Central control of movement is the consequence of multiple descending pathways thought to contribute to at least partially distinct aspects of voluntary action (Donkelaar and ten Donkelaar, 2009Donkelaar H.J.T. ten Donkelaar H.J. Evolution of motor systems: corticospinal, reticulospinal, rubrospinal and vestibulospinal systems.in: Binder M.D. Hirokawa N. Windhorst U. Encyclopedia of Neuroscience. Springer, 2009: 1248-1254Crossref Google Scholar). Direct descending projections to the vertebrate spinal cord arise from several major midbrain motor areas, including the reticular formation (reticulospinal), red nucleus (rubrospinal), and superior colliculus (tectospinal) (Kandel et al., 2012Kandel E.R. Schwartz J.H. Jessell T.M. Siegelbaum S.A. Hudspeth A.J. Principles of Neural Science.Fifth Edition. McGraw Hill Professional, 2012Google Scholar). In addition, the neocortex directly projects to spinal cord via deep layer pyramidal tract (PT) neurons (corticofugal; Figure 1A). mDA neurons receive direct, monosynaptic input from each of these descending motor pathways (Figure 1B; Geisler et al., 2007Geisler S. Derst C. Veh R.W. Zahm D.S. Glutamatergic afferents of the ventral tegmental area in the rat.J. Neurosci. 2007; 27: 5730-5743Crossref PubMed Scopus (295) Google Scholar, Watabe-Uchida et al., 2012Watabe-Uchida M. Zhu L. Ogawa S.K. Vamanrao A. Uchida N. Whole-brain mapping of direct inputs to midbrain dopamine neurons.Neuron. 2012; 74: 858-873Abstract Full Text Full Text PDF PubMed Scopus (495) Google Scholar). Projection neurons that coordinate activity across telencephalic hemispheres, intratelencephalic (IT) neurons (Greig et al., 2013Greig L.C. Woodworth M.B. Galazo M.J. Padmanabhan H. Macklis J.D. Molecular logic of neocortical projection neuron specification, development and diversity.Nat. Rev. Neurosci. 2013; 14: 755-769Crossref PubMed Scopus (300) Google Scholar), are critical for coordinated movement. Polysynaptic input arising from the IT pathway also shapes the activity of mDA neurons via disynaptic projections from striatum (Dudman and Gerfen, 2015Dudman J.T. Gerfen C.R. The Basal Ganglia.Fourth Edition. The Rat Nervous System, 2015Google Scholar). Other primary afferents of mDA neurons do not have direct spinal projections but are considered to shape behavioral output through encoding of action or action-inducing stimuli, including the lateral hypothalamus (Jennings et al., 2015Jennings J.H. Ung R.L. Resendez S.L. Stamatakis A.M. Taylor J.G. Huang J. Veleta K. Kantak P.A. Aita M. Shilling-Scrivo K. et al.Visualizing hypothalamic network dynamics for appetitive and consummatory behaviors.Cell. 2015; 160: 516-527Abstract Full Text Full Text PDF PubMed Scopus (187) Google Scholar), prefrontal cortices (Otis et al., 2017Otis J.M. Namboodiri V.M.K. Matan A.M. Voets E.S. Mohorn E.P. Kosyk O. McHenry J.A. Robinson J.E. Resendez S.L. Rossi M.A. Stuber G.D. Prefrontal cortex output circuits guide reward seeking through divergent cue encoding.Nature. 2017; 543: 103-107Crossref PubMed Scopus (69) Google Scholar, Simon et al., 2015Simon N.W. Wood J. Moghaddam B. Action-outcome relationships are represented differently by medial prefrontal and orbitofrontal cortex neurons during action execution.J. Neurophysiol. 2015; 114: 3374-3385Crossref PubMed Scopus (6) Google Scholar), ventral tegmental area (VTA)-GABA neurons (Hughes et al., 2019Hughes R. Watson G. Petter E. Kim N. Bakhurin K. Precise coordination of 3-dimensional rotational kinematics by ventral tegmental area GABAergic neurons.bioRxiv. 2019; https://doi.org/10.1101/678391Crossref Google Scholar, Su et al., 2019Su X.-Y. Chen M. Yuan Y. Li Y. Guo S.-S. Luo H.-Q. Huang C. Sun W. Li Y. Zhu M.X. et al.Central processing of itch in the midbrain reward center.Neuron. 2019; 102: 858-872.e5Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar, Zhou et al., 2019Zhou Z. Liu X. Chen S. Zhang Z. Liu Y. Montardy Q. Tang Y. Wei P. Liu N. Li L. et al.A VTA GABAergic neural circuit mediates visually evoked innate defensive responses.Neuron. 2019; 103: 473-488.e6Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar), dorsal striatum (DS) (Panigrahi et al., 2015Panigrahi B. Martin K.A. Li Y. Graves A.R. Vollmer A. Olson L. Mensh B.D. Karpova A.Y. Dudman J.T. Dopamine is required for the neural representation and control of movement vigor.Cell. 2015; 162: 1418-1430Abstract Full Text Full Text PDF PubMed Google Scholar), nucleus accumbens (Taha et al., 2007Taha S.A. Nicola S.M. Fields H.L. Cue-evoked encoding of movement planning and execution in the rat nucleus accumbens.J. Physiol. 2007; 584: 801-818Crossref PubMed Scopus (0) Google Scholar), globus pallidus (Stephenson-Jones et al., 2016Stephenson-Jones M. Yu K. Ahrens S. Tucciarone J.M. van Huijstee A.N. Mejia L.A. Penzo M.A. Tai L.-H. Wilbrecht L. Li B. A basal ganglia circuit for evaluating action outcomes.Nature. 2016; 539: 289-293Crossref PubMed Google Scholar), ventral pallidum (Richard et al., 2018Richard J.M. Stout N. Acs D. Janak P.H. Ventral pallidal encoding of reward-seeking behavior depends on the underlying associative structure.eLife. 2018; 7: e33107Crossref PubMed Scopus (3) Google Scholar), and periaqueductal gray (Evans et al., 2018Evans D.A. Stempel A.V. Vale R. Ruehle S. Lefler Y. Branco T. A synaptic threshold mechanism for computing escape decisions.Nature. 2018; 558: 590-594Crossref PubMed Scopus (2) Google Scholar). Thus, mDA neurons receive diverse sources of input carrying information about the planning, coordination, and kinematics of voluntary actions. mDA neurons exert the largest part of their influence through reciprocal interconnections with nuclei of the basal ganglia, a telencephalic (forebrain) to mesencephalic (midbrain) descending projection system conserved in its basic cell types and functional architecture throughout vertebrates (Dudman and Gerfen, 2015Dudman J.T. Gerfen C.R. The Basal Ganglia.Fourth Edition. The Rat Nervous System, 2015Google Scholar, Stephenson-Jones et al., 2012Stephenson-Jones M. Ericsson J. Robertson B. Grillner S. Evolution of the basal ganglia: dual-output pathways conserved throughout vertebrate phylogeny.J. Comp. Neurol. 2012; 520: 2957-2973Crossref PubMed Scopus (70) Google Scholar). Thus, it is important to understand the position of the basal ganglia within action generation circuits. IT and PT neurons provide motor information monosynaptically to the telencephalic input nuclei of basal ganglia-dorsal and ventral striatum (Dudman and Gerfen, 2015Dudman J.T. Gerfen C.R. The Basal Ganglia.Fourth Edition. The Rat Nervous System, 2015Google Scholar). PT neurons also provide monosynaptic input to multiple mesencephalic nuclei within basal ganglia, including the globus pallidus, subthalamic nucleus, ventral tegmental area, and substantia nigra (Kita and Kita, 2012Kita T. Kita H. The subthalamic nucleus is one of multiple innervation sites for long-range corticofugal axons: a single-axon tracing study in the rat.J. Neurosci. 2012; 32: 5990-5999Crossref PubMed Scopus (102) Google Scholar, Levesque et al., 1996Levesque M. Charara A. Gagnon S. Parent A. Deschenes M. Corticostriatal projections from layer V cells in rat are collaterals of long-range corticofugal axons.Brain Res. 1996; 709: 311-315Crossref PubMed Scopus (99) Google Scholar, Winnubst et al., 2019Winnubst J. Bas E. Ferreira T.A. Wu Z. Economo M.N. Edson P. Arthur B.J. Bruns C. Rokicki K. Schauder D. et al.Reconstruction of 1,000 Projection Neurons Reveals New Cell Types and Organization of Long-Range Connectivity in the Mouse Brain.Cell. 2019; 179 (in press. Published online September 5, 2019)https://doi.org/10.1016/j.cell.2019.07.042Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar). Basal ganglia output does not directly modulate spinal activity but instead monosynaptically and disynaptically modulates corticospinal, tectospinal, and reticulospinal pathways (Deniau et al., 2007Deniau J.M. Mailly P. Maurice N. Charpier S. The pars reticulata of the substantia nigra: a window to basal ganglia output.Prog. Brain Res. 2007; 160: 151-172Crossref PubMed Scopus (0) Google Scholar, Dudman and Gerfen, 2015Dudman J.T. Gerfen C.R. The Basal Ganglia.Fourth Edition. The Rat Nervous System, 2015Google Scholar, Dudman and Krakauer, 2016Dudman J.T. Krakauer J.W. The basal ganglia: from motor commands to the control of vigor.Curr. Opin. Neurobiol. 2016; 37: 158-166Crossref PubMed Google Scholar, Parent et al., 1983Parent A. Mackey A. Smith Y. Boucher R. The output organization of the substantia nigra in primate as revealed by a retrograde double labeling method.Brain Res. Bull. 1983; 10: 529-537Crossref PubMed Scopus (83) Google Scholar). The majority of dorsal basal ganglia output descends onto tectospinal and reticulospinal pathways (Dudman and Gerfen, 2015Dudman J.T. Gerfen C.R. The Basal Ganglia.Fourth Edition. The Rat Nervous System, 2015Google Scholar), while roughly one-third of the output is ascending to higher-order thalamic nuclei (Anderson and Turner, 1991Anderson M.E. Turner R.S. Activity of neurons in cerebellar-receiving and pallidal-receiving areas of the thalamus of the behaving monkey.J. Neurophysiol. 1991; 66: 879-893Crossref PubMed Google Scholar, Phillips et al., 2018Phillips J.W. Schulmann A. Hara E. Liu C. Wang L. Shields B. Korff W. Lemire A. Dudman J. Nelson S.B. et al.A single spectrum of neuronal identities across thalamus.bioRxiv. 2018; https://doi.org/10.1101/241315Crossref Google Scholar) that in turn modulate cortical activity, although the extent of that modulation is still unclear (Goldberg and Fee, 2012Goldberg J.H. Fee M.S. A cortical motor nucleus drives the basal ganglia-recipient thalamus in singing birds.Nat. Neurosci. 2012; 15: 620-627Crossref PubMed Scopus (50) Google Scholar, Schwab et al., 2019Schwab B.C. Kase D. Zimnik A. Rosenbaum R. Rubin J.E. Turner R.S. Weak modulation of thalamic discharge by basal ganglia output in association with a reaching task.bioRxiv. 2019; https://doi.org/10.1101/546598Crossref Google Scholar). Central control of action is both parallel and degenerate (Leonardo, 2005Leonardo A. Degenerate coding in neural systems.J. Comp. Physiol. A Neuroethol. Sens. Neural Behav. Physiol. 2005; 191: 995-1010Crossref PubMed Scopus (0) Google Scholar). This is reflected in the oft-replicated observation that during extensive training, different regions of the cortex and striatum become involved in behavioral control despite apparently similar actions being executed (Gremel and Costa, 2013Gremel C.M. Costa R.M. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions.Nat. Commun. 2013; 4: 2264Crossref PubMed Scopus (196) Google Scholar, Smith and Graybiel, 2013Smith K.S. Graybiel A.M. A dual operator view of habitual behavior reflecting cortical and striatal dynamics.Neuron. 2013; 79: 361-374Abstract Full Text Full Text PDF PubMed Scopus (71) Google Scholar, Yin et al., 2008Yin H.H. Ostlund S.B. Balleine B.W. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks.Eur. J. Neurosci. 2008; 28: 1437-1448Crossref PubMed Scopus (274) Google Scholar). These changes in the underlying functional anatomy are correlated with changes in the flexibility of behavior, such that behavior often becomes less sensitive to changing contingencies with extended training (Balleine and Dickinson, 1998Balleine B.W. Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates.Neuropharmacology. 1998; 37: 407-419Crossref PubMed Scopus (815) Google Scholar, Barnes et al., 2005Barnes T.D. Kubota Y. Hu D. Jin D.Z. Graybiel A.M. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories.Nature. 2005; 437: 1158-1161Crossref PubMed Scopus (377) Google Scholar, Yin et al., 2004Yin H.H. Knowlton B.J. Balleine B.W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning.Eur. J. Neurosci. 2004; 19: 181-189Crossref PubMed Scopus (678) Google Scholar). Thus, although a lever press in an instrumental task may be accomplished with similar movements early and late in training, control is achieved by combinations of distinct underlying neural circuits. Given the anatomy discussed above, the activity of mDA neuron populations should provide a summary of the circuit context under which an action was initiated. This has recently been described under the organizing principle of synchrony, such that mDA neuron activity provides a scalar estimate of consensus across its varied afferent inputs (Beeler and Kisbye Dreyer, 2019Beeler J.A. Kisbye Dreyer J. Synchronicity: the role of midbrain dopamine in whole-brain coordination.eNeuro. 2019; 6 (ENEURO.0345-18.2019)Crossref PubMed Scopus (1) Google Scholar). We further note that feedback on the dynamics responsible for a current state can be considered as “internal credit assignment,” a process considered to be a powerful tool for setting learning rates within networks (Rumelhart et al., 1986Rumelhart D.E. Hinton G.E. Williams R.J. Learning representations by back-propagating errors.Nature. 1986; 323: 533-536Crossref Google Scholar). We review mounting evidence that phasic mDA neuron activity is mechanistically tied to action initiation, followed by an overview of the functions demonstrated for phasic mDA neuron activity, before returning to the idea of mDA neuron signals as components of internal credit assignment (Figure 5). It is often unclear how action accompanies stimuli that recruit phasic mDA neuron activity, as behavioral measurements during studies of phasic mDA neuron activity have been largely limited to single discretized assays of learning (e.g., the rate of instrumental actions, the presence of anticipatory behaviors, and time spent near reward delivery areas). This amounts to quantifying the tip of an iceberg of complex motivated behavioral responses to reward-predictive stimuli (Hearst and Jenkins, 1974Hearst E. Jenkins H.M. Sign-Tracking: The Stimulus-Reinforcer Relation and Directed Action. Psychonomic Society, 1974Google Scholar, Zener, 1937Zener K. The significance of behavior accompanying conditioned salivary secretion for theories of the conditioned response.Am. J. Psychol. 1937; 50: 384-403Crossref Google Scholar). Meanwhile, subjective stimulus encoding has been studied almost exclusively in very well-trained animals, meaning that stimulus presentation marks the initiation of motivated action within some short latency. This is the case whether stimuli explicitly cue actions with reward contingency, such as arm reaches (Schultz et al., 1997Schultz W. Dayan P. Montague P.R. A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599Crossref PubMed Scopus (4669) Google Scholar), nose pokes (Roesch et al., 2007Roesch M.R. Calu D.J. Schoenbaum G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards.Nat. Neurosci. 2007; 10: 1615-1624Crossref PubMed Scopus (371) Google Scholar), saccades (Bromberg-Martin and Hikosaka, 2009Bromberg-Martin E.S. Hikosaka O. Midbrain dopamine neurons signal preference for advance information about upcoming rewards.Neuron. 2009; 63: 119-126Abstract Full Text Full Text PDF PubMed Google Scholar, Lak et al., 2016Lak A. Stauffer W.R. Schultz W. Dopamine neurons learn relative chosen value from probabilistic rewards.eLife. 2016; 5: e18044Crossref PubMed Google Scholar, Nomoto et al., 2010Nomoto K. Schultz W. Watanabe T. Sakagami M. Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli.J. Neurosci. 2010; 30: 10692-10702Crossref PubMed Scopus (85) Google Scholar), or lever presses (Roitman et al., 2004Roitman M.F. Stuber G.D. Phillips P.E.M. Wightman R.M. Carelli R.M. Dopamine operates as a subsecond modulator of food seeking.J. Neurosci. 2004; 24: 1265-1271Crossref PubMed Scopus (458) Google Scholar), or whether they implicitly cue actions, such as anticipatory licking (Waelti et al., 2001Waelti P. Dickinson A. Schultz W. Dopamine responses comply with basic assumptions of formal learning theory.Nature. 2001; 412: 43-48Crossref PubMed Scopus (674) Google Scholar) or reward area or cue approach (Flagel et al., 2011Flagel S.B. Clark J.J. Robinson T.E. Mayo L. Czuj A. Willuhn I. Akers C.A. Clinton S.M. Phillips P.E.M. Akil H. A selective role for dopamine in stimulus-reward learning.Nature. 2011; 469: 53-57Crossref PubMed Scopus (480) Google Scholar, Steinberg et al., 2013Steinberg E.E. Keiflin R. Boivin J.R. Witten I.B. Deisseroth K. Janak P.H. A causal link between prediction errors, dopamine neurons and learning.Nat. Neurosci. 2013; 16: 966-973Crossref PubMed Scopus (358) Google Scholar). Such “conditioned responses” constitute the evidence for learning in classical conditioning, even though they are not explicitly required for the attainment of reward. While canonical mDA neuron reward signaling may be well aligned to stimulus presentations, some nuance is in order before a wholesale discard of its contribution to a representation of action. Given that there is substantial jitter involved in motor preparation (Haith et al., 2016Haith A.M. Pakpoor J. Krakauer J.W. Independence of movement preparation and movement initiation.J. Neurosci. 2016; 36: 3007-3015Crossref PubMed Google Scholar) and inherent noise in many aspects of motor execution (Todorov and Jordan, 2002Todorov E. Jordan M.I. Optimal feedback control as a theory of motor coordination.Nat. Neurosci. 2002; 5: 1226-1235Crossref PubMed Scopus (1517) Google Scholar), there are key limits on an observer’s ability to identify the initial moment of action preparation from behavior. Motor preparatory activity develops with flexible timing relative to the initiation of the action to which it is causally related (Churchland et al., 2012Churchland M.M. Cunningham J.P. Kaufman M.T. Foster J.D. Nuyujukian P. Ryu S.I. Shenoy K.V. Neural population dynamics during reaching.Nature. 2012; 487: 51-56Crossref PubMed Scopus (416) Google Scholar, Moran and Schwartz, 1999Moran D.W. Schwartz A.B. Motor cortical representation of speed and direction during reaching.J. Neurophysiol. 1999; 82: 2676-2692Crossref PubMed Google Scholar). A further source of noise is introduced when action is measured according to the readout of some manipulandum, such as a lever (Jin and Costa, 2010Jin X. Costa R.M. Start/stop signals emerge in nigrostriatal circuits during sequence learning.Nature. 2010; 466: 457-462Crossref PubMed Scopus (275) Google Scholar), joystick (Panigrahi et al., 2015Panigrahi B. Martin K.A. Li Y. Graves A.R. Vollmer A. Olson L. Mensh B.D. Karpova A.Y. Dudman J.T. Dopamine is required for the neural representation and control of movement vigor.Cell. 2015; 162: 1418-1430Abstract Full Text Full Text PDF PubMed Google Scholar), or running wheel (Dodson et al., 2016Dodson P.D. Dreyer J.K. Jennings K.A. Syed E.C.J. Wade-Martins R. Cragg S.J. Bolam J.P. Magill P.J. Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism.Proc. Natl. Acad. Sci. USA. 2016; 113: E2180-E2188Crossref PubMed Scopus (30) Google Scholar, Howe and Dombeck, 2016Howe M.W. Dombeck D.A. Rapid signalling in distinct dopaminergic axons during locomotion and reward.Nature. 2016; 535: 505-510Crossref PubMed Scopus (110) Google Scholar). Given these limitations, we will concentrate below on experiments that either recorded continuous quantifications of action or offered some other path to dissociating action and stimulus encoding. We review first the evidence for mDA neuron action signaling independent of explicit reward learning and then the evidence for action signaling within the context of rewards and predictive cues. Correlations between movement and mDA neuron activity have been reported in rodents, with speed of movement and mDA neuron activity being generally positively correlated in the VTA (Puryear et al., 2010Puryear C.B. Kim M.J. Mizumori S.J.Y. Conjunctive encoding of movement and reward by ventral tegmental area neurons in the freely navigating rodent.Behav. Neurosci. 2010; 124: 234-247Crossref PubMed Scopus (34) Google Scholar, Wang and Tsien, 2011Wang D.V. Tsien J.Z. Conjunctive processing of locomotor signals by the ventral tegmental area neuronal population.PLoS ONE. 2011; 6: e16528Crossref PubMed Scopus (0) Google Scholar) and SNc (Barter et al., 2015Barter J.W. Li S. Lu D. Bartholomew R.A. Rossi M.A. Shoemaker C.T. Salas-Meza D. Gaidis E. Yin H.H. Beyond reward prediction errors: the role of dopamine in movement kinematics.Front. Integr. Nuerosci. 2015; 9: 3" @default.
• W2979619911 created "2019-10-18" @default.
• W2979619911 creator A5018585616 @default.
• W2979619911 creator A5076889746 @default.
• W2979619911 date "2019-10-01" @default.
• W2979619911 modified "2023-10-11" @default.
• W2979619911 title "Learning from Action: Reconsidering Movement Signaling in Midbrain Dopamine Neuron Activity" @default.
• W2979619911 cites W1273318020 @default.
• W2979619911 cites W1498436455 @default.
• W2979619911 cites W1571140265 @default.
• W2979619911 cites W1596586683 @default.
• W2979619911 cites W1604039963 @default.
• W2979619911 cites W1631357241 @default.
• W2979619911 cites W1633847368 @default.
• W2979619911 cites W1642699591 @default.
• W2979619911 cites W1810663112 @default.
• W2979619911 cites W1870304014 @default.
• W2979619911 cites W1875506666 @default.
• W2979619911 cites W1884568409 @default.
• W2979619911 cites W1925686653 @default.
• W2979619911 cites W1933363359 @default.
• W2979619911 cites W1944839424 @default.
• W2979619911 cites W1949404224 @default.
• W2979619911 cites W1964812893 @default.
• W2979619911 cites W1965922839 @default.
• W2979619911 cites W1967344144 @default.
• W2979619911 cites W1973229581 @default.
• W2979619911 cites W1973747047 @default.
• W2979619911 cites W1974803102 @default.
• W2979619911 cites W1975401347 @default.
• W2979619911 cites W1979441832 @default.
• W2979619911 cites W1980440818 @default.
• W2979619911 cites W1980737345 @default.
• W2979619911 cites W1983327875 @default.
• W2979619911 cites W1984652306 @default.
• W2979619911 cites W1989442300 @default.
• W2979619911 cites W1989895344 @default.
• W2979619911 cites W1991603311 @default.
• W2979619911 cites W1993407225 @default.
• W2979619911 cites W1997128521 @default.
• W2979619911 cites W2007203345 @default.
• W2979619911 cites W2007910638 @default.
• W2979619911 cites W2007959075 @default.
• W2979619911 cites W2011451766 @default.
• W2979619911 cites W2016033342 @default.
• W2979619911 cites W2020902468 @default.
• W2979619911 cites W2027852160 @default.
• W2979619911 cites W2027898754 @default.
• W2979619911 cites W2032738876 @default.
• W2979619911 cites W2034708714 @default.
• W2979619911 cites W2037457092 @default.
• W2979619911 cites W2037500812 @default.
• W2979619911 cites W2040418090 @default.
• W2979619911 cites W2043292527 @default.
• W2979619911 cites W2045821490 @default.
• W2979619911 cites W2052011218 @default.
• W2979619911 cites W2053955820 @default.
• W2979619911 cites W2054466443 @default.
• W2979619911 cites W2055197689 @default.
• W2979619911 cites W2055963704 @default.
• W2979619911 cites W2055969447 @default.
• W2979619911 cites W2060218354 @default.
• W2979619911 cites W2066954672 @default.
• W2979619911 cites W2069060695 @default.
• W2979619911 cites W2069090835 @default.
• W2979619911 cites W2069166530 @default.
• W2979619911 cites W2069258922 @default.
• W2979619911 cites W2074370892 @default.
• W2979619911 cites W2075994330 @default.
• W2979619911 cites W2076457657 @default.
• W2979619911 cites W2078731619 @default.
• W2979619911 cites W2079215847 @default.
• W2979619911 cites W2083537735 @default.
• W2979619911 cites W2084738519 @default.
• W2979619911 cites W2085553767 @default.
• W2979619911 cites W2090290990 @default.
• W2979619911 cites W2091094596 @default.
• W2979619911 cites W2092720178 @default.
• W2979619911 cites W2094037121 @default.
• W2979619911 cites W2094843557 @default.
• W2979619911 cites W2094933282 @default.
• W2979619911 cites W2108023589 @default.
• W2979619911 cites W2112175160 @default.
• W2979619911 cites W2114520249 @default.
• W2979619911 cites W2114586089 @default.
• W2979619911 cites W2116049552 @default.
• W2979619911 cites W2117726420 @default.
• W2979619911 cites W2117782909 @default.
• W2979619911 cites W2118830830 @default.
• W2979619911 cites W2119717200 @default.
• W2979619911 cites W2120222754 @default.
• W2979619911 cites W2123053862 @default.
• W2979619911 cites W2123575723 @default.
• W2979619911 cites W2128290890 @default.
• W2979619911 cites W2129717318 @default.
• W2979619911 cites W2136748963 @default.
• W2979619911 cites W2137480418 @default.
• W2979619911 cites W2139495058 @default.

• next