.Neuroscience of Rule-Guided Behavior Phần 4 doc

O’Reilly RC, Noelle DC, Braver TS, Cohen JD (2002) Prefrontal cortex and dynamic categorization tasks: representational organization and neuromodulatory control. Cerebral Cortex 12:246–257. Owen AM (1997) The functional organization of working memory processes within human lateral frontal cortex: the contribution of functional neuroimaging. Euro- pean Journal of Neuroscience 9:1329–1339. Owen AM, Herrod NJ, Menon DK, Clark JC, Downey SP, Carpenter TA, Minhas PS, Turkheimer FE, Williams EJ, Robbins TW, Sahakian BJ, Petrides M, Pickard JD (1999) Redefining the functional organization of working memory processes within human lateral prefrontal cortex. European Journal of Neuroscience 11:567–574. Paivio A, Yuille JC, Madigan SA (1968) Concreteness, imagery, and meaningfulness values for 925 nouns. Journal of Experimental Psychology 76 (Supplement):1–25. Pandya DN, Barnes CL (1987) Architecture and connections of the frontal lobe. In: The frontal lobes revisited (Perecman E, ed.), pp 41–72. New York: IRBN Press. Petrides M (1996) Specialized systems for the processing of mnemonic information within the primate frontal cortex. Philosophical Transactions of the Royal Society of London B 351:1455–1461. Plato (360 BCE/2003) The republic, 2nd edition. New York: Penguin Books. Sakai K, Passingham RE (2003) Prefrontal interactions reflect future task operations. Nature Neuroscience 6:75–81. Semendeferi K, Armstrong E, Schleicher A, Zilles K, Van Hoesen GW (2001) Prefrontal cortex in humans and apes: a comparative study of area 10. American Journal of Physical Anthropology 114:224–241. Shallice T (1982) Specific impairments of planning. Philosophical Transactions of the Royal Society of London B 298:199–209. Simoncelli EH, Heeger DJ (1998) A model of neuronal responses in visual area MT. Vision Research 38:743–761. Smith R, Keramatian K, Smallwood J, Schooler J, Luus B, Christoff K (2006) Mind- wandering with and without awareness: an fMRI study of spontaneous thought processes. In: Proceedings of the 28th Annual Conference of the Cognitive Science Society (Sun R, ed.), pp 804–809. Vancouver: Lawrence Erlbaum Associates. Thompson RKR, Oden DL, Boysen ST (1997) Language-naive chimpanzees (Pan trog- lodytes) judge relations between relations in a conceptual matching-to-sample task. Journal of Experimental Psychology: Animal Behavior Processes 23:31–43. Wallis JD, Anderson KC, Miller EK (2001) Single neurons in prefrontal cortex encode abstract rules. Nature 411:953–956. Wandell B (1995) Foundations of vision. Sunderland, MA: Sinauer Associates. Wilson MD (1988) The MRC psycholinguistic database: machine readable dictionary, version 2. Behavioural Research Methods, Instruments and Computers 20:6–11. 126 Rule Representation II RULE IMPLEMENTATION This page intentionally left blank 7 Ventrolateral and Medial Frontal Contributions to Decision-Making and Action Selection Matthew F. S. Rushworth, Paula L. Croxson, Mark J. Buckley, and Mark E. Walton The frontal cortex has a central role in the selection of actions, and in many circumstances, action selection is likely to be the consequence of activity dis- tributed across a swathe of frontal lobe areas. Evidence from lesion and oth er interference techniques, such as transcranial magnetic stimulation (TMS ), however, suggests that a useful distinction may be drawn between the roles of ventrolateral prefrontal cortex (PFv) and dorsomedial frontal cortex areas (Fig. 7–1), including the pre-supplementary motor area (pre-SMA) and the anterior cingulate cortex (ACC). The PFv region is centered on cytoarchitec- tonic region 47/12 (200 2a) [see Fig. 7–5], but the lesions that are used to in- vestigate this area often include adjacen t lateral orbital areas 11 and 13 (PFvþo lesion) [for example, Bussey et al., 2001, 2002]. Cells in these areas have some similar responses to those in the PFv (Wallis et al., 2001). The pre-SMA is situated in an anterior division of area 6, whereas the ACC region under dis- cussion in this chapter is in cytoarchitecton ic areas 24c and 24c 0 (Matsuzaka et al., 1992; Luppino et al., 1993; Vogt, 1993). A series of experiments have all suggested that the PFv has a central role in the selection of actions in response to external stimuli and according to learned arbitrary rules. However, it has been more difficult to describe how the con- tribution of the PFv differs from that made by premotor areas in more poste- rior parts of the frontal lobe. Recent results suggest that the PFv is particularly concerned with the selection of the behaviorally relevant stimulus information on which action selection will, in turn, be contingent, and the deployment of prospective coding strategies that facilitate rule learning. Once behavioral rules for action selection have been learned, it is often necessary to switch quickly between one set of rules and another as the context changes. The pre-SMA is known to be important at such times. The role of the ACC appears to be quite distinct. Both lesion investigations and neuroimaging implicate the ACC most 129 strongly when choices are made on the basis of the recent reward history rather than on the basis of learned conditional cue-action associations. The ACC may be important for representing the reinforcement values associated with actions rather than the stimulus conditional selection rules associated with actions. In both humans and macaques, the PFv is distinguished by a pattern of strong anatomical connection with the temporal lobe, whereas the ACC is unusual in being closely con nected with reward processing areas and the motor system. Such differences in anatomical connectivity may underlie the different specializations of the areas. pre-SMA ACC PMd PFv PFo pre-SMA ACC PMd PFv PFo Figure 7–1 Medial (left) and lateral (right) views of magnetic resonance images of a human brain (top) and photographs of a macaque brain (bottom). The ventral and orbital prefrontal regions PFv and PFo, respectively, have a central role in learning conditional rules for response selection, perhaps because of their roles in identifying behaviorally relevant stimuli and guiding efficient learning strategies. More dorsal and medial areas, such as the anterior cingulate cortex (ACC), pre-supplementary motor area (pre-SMA), and dorsal premotor cortex (PMd), may also be active when condi- tional rules are used, but their functional contributions are distinct. Although PMd may use conditional rules to select actions, pre-SMA may be concerned with the se- lection of sets of responses rather than individual responses. The ACC is more con- cerned with representing the reinforcement value of actions and their reinforcement outcome associations than with representing the learned conditional associations of actions with sensory cues. 130 Rule Implementation VENTRAL PREFRONTAL CORTEX Ventral Prefrontal Cortex and the Use of Conditional Rules for Action Selection Discussions of prefrontal function have often focused on its role in working memory (Goldman-Rakic, 1996). This is consistent with the delay dependency of the deficits that are seen after some prefrontal lesions. For example, Fu- nahashi and colleagues (1993) showed that macaques with lesions in the dor- solateral prefrontal cortex (PFdl) surrounding the principle sulcus were in- accurate when they made saccades in the absence of visible targets to locations that were held in memory. The same animals, however, were able to make vi- sually instructed saccades in a relatively normal manner. The deficits that follow PFv lesions are different, and are not delay- dependent in the same way (Rushworth and Owen, 1998). In one study, ma- caques were taught to select one of two colored shapes, A or B, at the bottom of a touch-screen monitor (Rushworth et al., 1997).The correct choice was con- ditional on the identity of a ‘‘sample’’ stimulus shown at the top of the screen at the beginning of the trial. If the macaque saw stimulus A as the sample at the beginning of the trial, then the rule was to select a matching copy of stimulus A when subsequently given a choice between it and stimulus B. Similarly, the macaques also learned to choose the matching stimulus B when the sample was stimulus B. At the beginning of each trial, the macaques touched the sample stimulus to indicate that they had seen it. On ‘‘simultaneous’’ trials, the sample stayed on the screen even after it was touched, and it was still present at the time of the response choice. In the delay version of the task, the sample stimulus disap- peared from the screen before the macaque could choose between the response options. After PFv lesions were made, the animals were first tested on the si- multaneous version of the task, and their performance was found to be sig- nificantly impaired. After retraining, the animals with lesions eventually over- came their impairments on the simultaneous matching task. Notably, once the relearning of the simultaneous matching task was complete, the subsequent imposition of a delay between sample and choice periods did not cause them additional difficulty. Such a pattern of results suggests that the PFv lesion did not cause a delay-dependent deficit analogous to the one seen after PFdl le- sions; the PFv lesion impaired the use of the matching rule that guided correct responding, but it did not selectively impair the retention in memory of which sample stimulus was presented at the beginning of each trial. Although the ability to associate a sample stimulus with a matching stim- ulus when making a choice might seem like a trivial one, it is important to re- member that from the macaque’s perspective, using the matching rule is as arbitrary as using a nonmatching rule. The results of the experiment by Rush- worth and colleagues (1997) suggest that it is the learning and use of the ar- bitrary rule for which the PFv is necessary. Once the rule is acquired, however, Frontal Cortex and Action Selection 131 memory for which sample stimulus has been recently shown may rely on distinct brain structures. Several studies have confirmed that the learning of conditional rules that link stimuli to responses is a critical aspect of PFv function. Bussey and col- leagues (2001) taught macaques to select joystick movements in response to the presentation of visual stimuli. Conditional rules linked the presentation of each stimulus to the retrieval of a particular response. The conclusion that the PFv was especially concerned with conditional rules was based on the finding that animals with lesions of the PFv and the adjacent lateral orbital prefrontal region (referred to as ‘‘PFvþo lesions’’) were impaired on the conditional visuomotor task, bu t could still learn visual discrimination problems well. In visual discrimination tasks, the correct choice is consistently associated with reinforcement, whereas the incorrect choice is never associated with reinforce- ment. In the conditional tasks, all of the responses are partially and equally well associated with reinforcement, and which one is correct varies from trial to trial in a manner that is conditional on the presence of the stimulus that is also presented. Related accounts of the PFv have also emphasized its importance in me- diating otherwi se difficult associations (Petrides, 2005). Rather than empha- sizing the conditional nature of the association, Petrides and others (Wagner et al., 2001) have emphasized the role of the PFv in the active nonautomatic retrieval of associations from memory. Active retrieval is needed when the as- sociation is arbitrary or learned, and activation of the representation does not occur aut omatically as the result of the arrival of matching sensory input in posterior cortex. It has been argued that, when human participants follow instructions, they are essentially employing conditional rules linking certain stimuli, or more generally, any arbitrary antecedent, with subsequent action choices (Murray et al., 2000, 2002; Passingham et al., 2000; Wise and Murray, 2000). Petrides and Pandya (2002a) have identified a number of similarities between human and macaque PFv cytoarchitecture, and human PFv is active when human participants learn cue-conditional instructions for selecting actions (Toni et al., 2001; Bunge et al., 2003; Grol et al., 2006; see also Chapter 3). Routes for Conditional Association: Interactions between Ventrolateral Prefrontal Cortex and Temporal Lobe Conditional rule learning does not depend on PFv in isolation, but on its in- teraction with other brain areas, especially the temporal lobe. PFv is densely interconnected with the temporal lobe (Webster et al., 1994; Carmichael and Price, 1995; Petrides and Pandya, 2002a). Within PFv, area 12/47 is particu- larly well connected with visual association areas in the inferior temporal cortex, whereas the slightly more posterior area 45 may be more strongly con- nected with the auditory association cortex in the superior temporal lobe. The connections not only convey sens ory information about visual and auditory 132 Rule Implementation object identity to PFv but also provide a route by which PFv is able to exert a top-down influence over temporal lobe activity (Tomita et al., 1999). The interaction between PFv and the temporal lobe during visual stimulus conditional learning can be examined by making a ‘‘crossed’’ disconnection lesion. A PFvþo lesion is made in one hemisphere and in the inferior temporal lobe cortex in the other hemisphere. Because most interareal connections are intrahemispheric, the crossed lesion prevents the possibility of direct, intra- hemispheric communication between PFv and the temporal lobe. Like PFvþo lesions, PFvþo-temp oral disconnection lesions impair visual conditional tasks, but leave visual discrimination learning relatively intact (Parker and Gaffan, 1998; Bussey et al., 2002). It is also po ssible to study frontotemporal interactions by directly trans- ecting the fibe rs that connect the two lobes. In the macaque, many of the direct connections between the visual association cortex in the inferior temporal lobe and PFvþo travel in a fibe r bundle called the ‘‘uncinate fascicle’’ (Ungerleider et al., 1989; Schmahmann and Pandya, 2006). Connections with the auditory association cortex in the superior temporal gyrus, and perhaps more posterior parts of the inferior temporal cortex, run more dorsally in the extreme cap- sule (Petrides and Pandya, 1988, 2002b; Schmahmann and Pandya, 2006). Al- though the roles of the extreme capsule and auditory conditional associations have received little attention, a number of experiments have considered the effects of uncinate fascicle transection on visual conditional associations. As is the case with the disconnection lesions, the ability to follow rules that are con- ditional on visual stimuli is impaired if the uncinate fascicle is cut (Eacott and Gaffan, 1992; Gutnikov et al., 1997). Unlike the disconnection lesion, which disrupts all intrahemispheric communication between PFvþo and the inferior temporal lobe, uncinate fascicle transection only disrupts direct monosynaptic connections. Macaques with uncinate fascicle transection are still able to use conditional rules to select actions if the rule is based on the presentation of reinforcement, as opposed to visual stimuli. Eacott and Gaffan (1992) gave macaques one of two free rewards at the beginning of each trial. If animals received a free reward A, they were taught to select action 1 to earn an additional reward A. If, on the other hand, the trial started with free delivery of reward B, then the condi- tional rule meant that animals were to select action 2 to earn an additional reward B. Surprisingly, macaques with uncinate fascicle transection were still able to perform this task, even though they were impaired at selecting actions in response to conditional visual instructions. The discrepancy can be un- derstood if the frontal lobe is not interacting with inferior temporal corte x in the case of reinforcement conditional action, but if the relevant information that the frontal lobe needs to access comes from elsewhere—perhaps an area such as the amygdala or the striatum, both of which are known to encode reinforcement information (Schultz, 2000; Yamada et al., 2004; Samejima et al., 2005; Paton et al., 2006). Frontal Cortex and Action Selection 133 Figure 7–2 Quantitative results of probabilistic tractography from the human extreme capsule (A), uncinate fascicle (B), and amygdala (C) to the prefrontal regions. The prob- ability of connection with each prefrontal region as a proportion of the total connec- tivity with all prefrontal regions is plotted on the y-axis. The majority of connections from the posterior and superior temporal lobe areas running in the extreme capsule are with areas ventral to the dorsal prefrontal cortex (PFdlþdm). High connection prob- abilities were found for the ventrolateral prefrontal areas (PFvl) and the lateral, central, and medial orbital regions (PFol, PFoc, and PFom, respectively). Connections from the anterior and inferior temporal lobe via the uncinate fascicle are more biased to orbital areas. The amygdala connections are most likely to be with even more medial regions, for example, PFom. The high diffusion levels in the corpus callosum distort connection estimates in the adjacent anterior cingulate cortex, but nevertheless, it is clear that there is still some evidence for connectivity between the amygdala and the cingulate gyral and sulcal regions (CG and CS, respectively). The right side of each part of the figure shows three sagittal sections depicting the estimated course taken by each connecting tract for a sample single participant. (Reprinted with permission from Croxon et al., Journal of Neuroscience, 25, 8854–8866. Copyright Society for Neuroscience, 2005.) Frontostriatal connections take a course that differs from those running between the inferior temporal cortex and PFvþo. Outputs from the amygdala run ventral to the striatum, rather than in the more lateral parts of the un- cinate fascicle affecte d by the transection (Schmahmann and Pandya, 2006). Indeed, anatomical tracing studies show that there is still evidence of connec- tion between the frontal lobe and the amygdala, even after the uncinate fascicle has been cut (Ungerleider et al., 1989). Reinforcement conditional action selection may, there fore, depend on distinct inputs into the frontal lobe; it may even depend on additional frontal regions. Later in this chapter, it is argued that, in many situations, when action selection is guided not by well-defined conditional rules, but by the history of reinforcement associated with each action, then ACC, and not just PFv, is essential for selecting the correct action. Diffusion weighted magnetic resonance imaging (DWI) and probabilistic tractography have recently been used to compare the trajectories of white matter fiber tracts, such as the uncinate fascicle, in vivo in the human and macaque. DWI provides information on the orientation of brain fiber path- ways (Basser and Jones, 2002; Beaulieu, 2002). Such data can be analyzed with probabilistic tractography techniques that generate estimates on the likelihood of a pathway existing between two brain areas (Behrens et al., 2003b; Hag- mann et al., 2003; Tournier et al., 2003). Using the method developed by Behrens et al. (2003a), Croxson and colleagues (2005) were able to show, in the macaque, that the extreme capsule was interconnected with more dorsal PFv regions (Fig. 7–2A), whereas the uncinate fascicle was interconnected with the more ventral PFv and the orbitofrontal cortex (Fig. 7–2B). Consistent with the tracer injection studies indicating that amygdala connections with the fron- tal lobe take a distinct course, the highest connectivity estimates for the amyg- dala were more medially displaced across a wider area of the orbital surface and extended onto the medial frontal cortex (Fig. 7–2C). A similar pattern was also observed in human participants. The extreme capsule and uncinate fas- cicle connection estimates within the human frontal lobe include the same regions that have been identified in human neuroimaging studies when con- ditional rules are used during action selection (Toni and Passingham, 1999; Toni et al., 1999, 2001; Walton et al., 2004; Crone et al., 2006; Grol et al., 2006). STRATEGY USE AND ATTENTION SELECTION Attention and Stimulus Selection during Conditional Rule Learning A number of single-neuron recording studies have identified PFv activity related to the encoding of conditional rules linking stimuli and responses (Boussaoud and Wise, 1993a, b; Wilson et al., 1993; Asaad et al., 1998; White and Wise, 1999; Wallis et al., 2001; Wallis and Miller, 2003; see also Chapter 2). Another important aspect of PFv activity, however, concerns the encoding of the attended stimulus and its features. Many neurons in PFv exhibit distinct Frontal Cortex and Action Selection 135 [...]... Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions Experimental Brain Research 97 :42 3 43 6 Sakagami M, Tsutsui K-i, Lauwereyns J, Koizumi M, Kobayashi S, Hikosaka O (2001) A code for behavioral inhibition on the basis of color, but not motion, in ventrolateral prefrontal cortex of macaque monkey Journal of Neuroscience 21 :48 01– 48 08 Samejima... of the orbital and medial prefrontal cortex of macaque monkeys Journal of Comparative Neurology 363: 642 –6 64 Chen Y, Thaler D, Nixon PD, Stern C, Passingham RE (1995) The functions of the medial premotor cortex (SMA), II The timing and selection of learned movements Experimental Brain Research 102 :46 1 47 3 Crone EA, Wendelken C, Donohue SE, Bunge SA (2006) Neural evidence for dissociable components of. .. Rushworth MFS, Bogdanovic MD, Kischka U, Wimalaratna S, Matthews PM (2002) The role of ipsilateral premotor cortex in hand movement after stroke Proceedings of the National Academy of Sciences U S A 99: 145 18– 145 23 Kennerley SW, Sakai K, Rushworth MFS (20 04) Organization of action sequences and the role of the pre-SMA Journal of Neurophysiology 91:978–993 Kennerley SW, Walton ME, Behrens TE, Buckley MJ,... negative value of visual stimuli during learning Nature 43 9:865–870 Petrides M (1982) Motor conditional associative-learning after selective prefrontal lesions in the monkey Behavioral Brain Research 5 :40 7 41 3 Petrides M (1986) The effect of periarcuate lesions in the monkey on performance of symmetrically and asymmetrically visual and auditory go, no-go tasks Journal of Neuroscience 6:20 54 2063 Petrides... Journal of Neuroscience 17 :48 29 48 38 Rushworth MFS, Owen AM (1998) The functional organization of the lateral frontal cortex: conjecture or conjuncture in the electrophysiology literature? Trends in Cognitive Science 2 :46 –53 Rushworth MFS, Walton ME, Kennerley SW, Bannerman DM (20 04) Action sets and decisions in the medial frontal cortex Trends in Cognitive Sciences 8 :41 0 41 7 Sakagami M, Niki H (19 94) Encoding... frontal cortex: effects of stimulus and movement Experimental Brain Research 95:28 40 Brass M, von Cramon DY (2002) The role of the frontal cortex in task preparation Cerebral Cortex 12:908–9 14 Brass M, von Cramon DY (20 04) Selection for cognitive control: a functional magnetic resonance imaging study on the selection of task-relevant information Journal of Neuroscience 24: 8 847 –8852 Browning PG, Easton... switch between one set of conditional action rules and another, so it is important when participants switch between one set of rules for sequencing actions and another Kennerley and colleagues showed that 140 0 Block 1 Block 2 Block 3 600 TMS No TMS 600 Inter-response Time (ms) 1200 1000 RT (ms) Response Time (ms) 800 40 0 800 600 40 0 200 200 0 A 2 4 6 8 Movement Number 10 12 B 40 0 300 200 100 0 0 0 TMS... of Neuroscience 9:15 24 1529 Hadland KA, Rushworth MFS, Gaffan D, Passingham RE (2003) The anterior cingulate and reward-guided selection of actions Journal of Neurophysiology 89:1161–11 64 Hagmann P, Thiran JP, Jonasson L, Vandergheynst P, Clarke S, Maeder P, Meuli R (2003) DTI mapping of human brain connectivity: statistical fiber tracking and virtual dissection Neuroimage 19: 545 –5 54 Halsband U, Passingham... instances of positive reinforcement, after making the correct action, subsequent to an earlier error The histogram at the bottom part of each graph indicates the number of instances of each trial type (± standard error of the mean) White and gray bars indicate control and ACCs lesion data, respectively, whereas hatched bars indicate data from the postoperative session Estimates of the influence of the... Nature Neuroscience 9: 940 – 947 Koechlin E, Ody C, Kouneiher F (2003) The architecture of cognitive control in the human prefrontal cortex Science 302:1181–1185 Kunishio K, Haber SN (19 94) Primate cingulostriatal projection: limbic striatal versus sensorimotor striatal input Journal of Comparative Neurology 350:337–356 Lebedev MA, Messinger A, Kralik JD, Wise SP (20 04) Representation of attended versus . Anthropology 1 14: 2 24 241 . Shallice T (1982) Specific impairments of planning. Philosophical Transactions of the Royal Society of London B 298:199–209. Simoncelli EH, Heeger DJ (1998) A model of neuronal. Specialized systems for the processing of mnemonic information within the primate frontal cortex. Philosophical Transactions of the Royal Society of London B 351: 145 5– 146 1. Plato (360 BCE/2003) The republic,. 12: 246 –257. Owen AM (1997) The functional organization of working memory processes within human lateral frontal cortex: the contribution of functional neuroimaging. Euro- pean Journal of Neuroscience