Neuronal correlates of perceptual salience in spike trains from the primary visual cortex

NEURONAL CORRELATES OF PERCEPTUAL SALIENCE IN SPIKE TRAINS FROM THE PRIMARY VISUAL CORTEX BONG JIT HON NATIONAL UNIVERSITY OF SINGAPORE 2012 NEURONAL CORRELATES OF PERCEPTUAL SALIENCE IN SPIKE TRAINS FROM THE PRIMARY VISUAL CORTEX BONG JIT HON B.Eng.(Hons.), NUS A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY OF ENGINEERING DEPARTMENT OF ELECTRICAL & COMPUTER ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2012 Dedication To my family and friends, for their endless care, love and support. i Acknowledgements I will never forget my time at NUS. It was the best part of my life. I learned a lot and find inspiration and motivation for my life. It is with the assistance, companionship and kindness of the numerous people listed here, that I have completed my PhD study and this dissertation. Here, I would like to express my deepest gratitude and appreciation for the following people. First, I would like to thank my supervisor, Dr. Yen Shih-Cheng for introducing me to the world of neuroscience, and for his continuous support, trust and help in making this study possible. Not to mention his excellence in research and teaching, he supported and guided me in every aspect of this project, including giving me the freedom to help foster my own independence. He inspired me to move forward, trusted me and showed great patience throughout my years in graduate school. I am also grateful to Dr. Charles M. Gray and Dr. Rodrigo Salazar at Montana State University for their advice in my research work. Both of them have been heavily involved in my PhD work and contributed valuable insights and comments into this project. The work presented in this dissertation was supported by grants from the National Eye Institute and the Singapore Ministry of Education Academic Research Fund. All the work shown in this dissertation was the result of collaboration between the lab ii of Dr. Yen Shih-Cheng at NUS and the lab of Dr. Charles M. Gray at Center for Computational Biology, Montana State University. I am also grateful to have many good lab mates who help me and from whom I learned a lot, they are Roger, Yasamin, Omer, Seetha, Esther and Ido Amihai. I very much appreciate their constant source of companionship and encouragement. Without them, my time in graduate school will be much dull and difficult. Finally, I would like to give the biggest appreciation to my family for their patience, support and understanding during this time period. They have been always the source of motivation and encouragement in my life. iii Contents Dedication i Acknowledgements ii Summary viii List of Tables xi List of Figures xii List of Symbols xviii Introduction Literature Review 2.1 Firing Rate Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Temporal Correlation Hypothesis . . . . . . . . . . . . . . . . . . . . 2.3 Response Latency Hypothesis . . . . . . . . . . . . . . . . . . . . . . 15 Materials and Methods 3.1 18 Subjects and Surgical Procedures . . . . . . . . . . . . . . . . . . . . iv 18 3.2 Behavioral Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Recording Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.4 Visual Stimuli . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.5 Spike Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.6 Multi-Unit Activity (MUA) . . . . . . . . . . . . . . . . . . . . . . . 27 3.7 Envelope Multi-Unit Activity (eMUA) . . . . . . . . . . . . . . . . . 28 3.8 Response Onset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.9 Eye Jitter and Reaction Time . . . . . . . . . . . . . . . . . . . . . . 31 3.10 Behavioral Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.11 Orientation Tuning Curve . . . . . . . . . . . . . . . . . . . . . . . . 33 3.12 Receiver Operating Characteristics (ROC) analysis . . . . . . . . . . 34 3.13 Raw Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Firing Rate Hypothesis 39 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.2 Methods of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.2.1 Test for Bimodality of Neuronal Responses . . . . . . . . . . . 40 4.2.2 Population Analysis - Modulation Index (MI) . . . . . . . . . 43 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.3.1 Single Neuron Firing Rate - ROC Analysis . . . . . . . . . . . 45 4.3.2 Single Neuron Firing Rate - Raw Data Analysis . . . . . . . . 61 4.3.3 MUA Firing Rate - ROC Analysis . . . . . . . . . . . . . . . . 62 4.3.4 Dependence on other experimental variables . . . . . . . . . . 63 4.3.5 Population eMUA Analysis . . . . . . . . . . . . . . . . . . . 64 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.3 4.4 v Temporal Correlation Hypothesis 73 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.2 Methods of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 5.2.1 Rate-Covariation . . . . . . . . . . . . . . . . . . . . . . . . . 74 5.2.2 Paired Synchrony Analysis . . . . . . . . . . . . . . . . . . . . 75 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.3.1 Rate-Covariation . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.3.2 Paired Synchrony Analysis . . . . . . . . . . . . . . . . . . . . 82 5.3.3 Dependence on other experimental variables . . . . . . . . . . 86 5.4 Different Types of Paired Synchrony Analysis . . . . . . . . . . . . . 90 5.5 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.3 Response Latency Hypothesis 94 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.2 Methods of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.2.1 First Spike Latency Analysis . . . . . . . . . . . . . . . . . . . 95 6.2.2 Relative Response Latency Analysis . . . . . . . . . . . . . . . 95 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.3.1 First Spike Latency - ROC Analysis and Raw Data Analysis . 96 6.3.2 Relative Response Latency - ROC Analysis and Raw Data 6.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.3.3 MUA First Spike Latency & Relative Response Latency - ROC Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.3.4 6.4 Dependence on other experimental variables . . . . . . . . . . 105 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 vi Conclusion 109 7.1 Thesis conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 7.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 A Using MUA pairs to compute the cross-correlation function 113 B List of publications 115 B.1 Peer-reviewed journal publication . . . . . . . . . . . . . . . . . . . . 115 B.2 Conference publication . . . . . . . . . . . . . . . . . . . . . . . . . . 115 References 116 vii Summary In this thesis, we examined the representation of visual saliency in the responses of neurons in the primary visual cortex. We investigated this by recording from the primary visual cortex of macaque monkeys while they performed a contour detection task. The visual stimuli consisted of an array of randomly drifting Gabor patches, with a subset aligned to form a coherently drifting closed contour. The orientations of the Gabor patches on the contour were jittered to create contours with high, intermediate, and low saliency. The neurons under study were stimulated by Gabor patches belonging either to part of the contour (contour condition), or part of the background (control condition). Recordings of single, as well as pairs of cortical cells, were analyzed. Using methods from signal detection theory, we identified neurons in which the firing rate in the high-salience contour and control conditions were significantly different (44 out of 181 neurons, or 24.3%), and neurons in which at least one contour salience condition was significantly different from the other salience conditions (29/181, or 16%). Interestingly, we found neurons that exhibited differences between the contour and control condition much earlier (approximately 40 ms after stimulus onset) than previously reported. We also computed the correlation coefficients between the neurometric and psychometric performance curves, and found the activity of the 29 viii CHAPTER 6. RESPONSE LATENCY HYPOTHESIS 108 first spike (Thorpe et al., 1996, 2001; Van Rullen and Thorpe, 2001; Van Rullen et al., 2005; Foffani et al., 2008; Gollisch and Meister, 2008; Gollisch, 2009). Similar to the firing rate study, we found that some cells exhibited significantly shorter latencies when they were part of the contour, while others, albeit a small fraction, exhibited longer latencies. This is probably not surprising since latencies are highly correlated to firing rates, which we have shown earlier, also exhibits some heterogeneity. To account for the heterogeneity, we also computed the behavioral bias, tuning curve, curvature of the stimulus, and the depth of the recording. As can be seen in Figure 6.7, these limited variables were unable to explain the heterogeneity that we found. Recording with more simultaneously recorded neurons will be required to fully understand the origin of this heterogeneity. Chapter Conclusion 7.1 Thesis conclusions In this thesis, three hypotheses (firing rate, temporal correlation and response latency) have been put forth to study the role of primary visual cortex in perceptual grouping. First, for the firing rate hypothesis, we found 44 out of 181 neurons in which the firing rate in the high-salience contour and control conditions were significantly different, and 29 out of 181 neurons in which at least one contour salience condition was significantly different from the other salience conditions. The activity of these 29 neurons seems to be well correlated with the behavior of the animal. Inspection at the time course, we found neurons that exhibited differences between the contour and control condition as early as 40 ms after stimulus onset. These results suggest that striate cortex may be the site of origin for the neuronal correlates of perceptual grouping rather than merely representing feedback signals from extra-striate cortex. Interestingly, we also observed considerable heterogeneity in the differences in firing rate. In one case, we found neurons that exhibited significantly higher firing rates 109 CHAPTER 7. CONCLUSION 110 when they were part of the contour, while others exhibited lower firing rates in the contour condition. This form of heterogeneity has not been reported before in figureground segmentation tasks. For the temporal correlation hypothesis, we found that that there was a higher rate-covariation for the high-salience contour condition compared to the control condition. This result is consistent with the finding of Roelfsema et al. (2004). Moreover, we found that the difference in rate-covariation was mainly due to the drop in rate-covariation for the control condition after the stimulus onset, while the ratecovariation for the high-salience contour condition was not significantly different before and after the stimulus onset. We also found the spike synchrony, to be very dynamic, with higher synchrony observed in the control condition for the windows from -30 to 30 ms when compared to the contour condition, and lower synchrony observed in the control condition for the windows from 50 to 100 ms when compared to the contour condition. However, the effect of spike synchronization for perceptual salience was pretty weak. For the response latency hypothesis, we found that 28 out of 181 cells exhibited significant differences in their latencies when they were activated by part of a contour compared to when they were activated by part of the background. Among these 28 cells, 20 exhibited significantly different responses across salience conditions. The activity of these 20 neurons appeared to be well correlated with the behavior of the animal. We also found that the first-spike latencies were highly correlated with the firing rates for most of the neurons, which lends support to the idea that there may be early firing rate differences for some neurons. However, due to this high correlation between the firing rates and the response latencies of neurons, it is still CHAPTER 7. CONCLUSION 111 unclear whether contour integration is represented by a latency code or a firing rate code, since there is no simple way to separate the two. In conclusion, the firing rate, rate-covariation, and the response latency of neurons are all possible coding methods that the visual system could use to represent visual saliency. We found little evidence for spike synchronization to play a role in representing perceptual salience, but studies recording simultaneously from a larger number of single units need to be carried out. 7.2 Future work As we pointed out previously, we need simultaneous recordings with more neurons to understand the heterogeneity in the neuronal responses that we observed in this study. This is currently still quite challenging, but the situation should improve as multi-channel electrodes with higher performance and densities become available. The contour detection task in this study is pretty simple, with only one contour present at a time. However, in natural scenes, the visual system often has to perform detection and discrimination of multiple objects. Therefore, a more difficult contour discrimination task may be necessary to understand some of the mechanisms utilized by the brain in natural environments. In this study, we kept the contrast of the Gabor stimulus present in the receptive field identical across stimulus conditions. However, other investigators have found firing rate and latency differences due to different stimulus contrasts (Tolhurst, 1989; Gawne et al., 1996b; Raiguel et al., 1999; Reich et al., 2001), so it might be interesting to see how contrast would affect the results obtained. CHAPTER 7. CONCLUSION 112 Since the reaction times in our study were very short, this suggests that the feedback signal may arrive too late to influence the decision of the animal. It will be interesting to see what, if any, behavioral and neuronal consequences can be observed if we were to eliminate the feedback signal. In addition, to check if early response differences had a causal influence on the behavior of the monkey, it would be interesting to perform microstimulation experiments during and after the early phase to look at how they would affect behavior. Appendix A Using MUA pairs to compute the cross-correlation function There is a simple relationship between a MUA-MUA cross-correlation function and the cross-correlation functions between SUAs at two recording sites. Let us assume that n1 single units contribute to MUAs at site 1, and n2 units contribute to MUAs at site 2. Thus, n1 T rial SM U A1 (t) n1 T rial S1i (t), PM U A1 (t) = = i=1 n2 T rial SM U A2 (t) = P1i (t) (A.1) P2j (t) (A.2) i=1 n2 T rial S2j (t), PM U A2 (t) = j=1 j=1 T rial where SM U A1 (t) is the MUA at time t in trial Trial at site 1, which equals the T rial sum of all SUAs S1i (t) at this site. PM U A1 (t) is the MUA PSTH, which equals the sum across P1i (T ), the single-unit PSTHs. Hence, the shift-predictor corrected 113 APPENDIX A. 114 MUA-MUA cross-correlation should be defined as: CM U A1,M U A2 (τ ) = = N T rial N T rial n1 N T rial T T rial T rial SM U A1 (t)SM U A2 (t + τ ) − T rial=1 t=−T N T rial T n1 n2 T rial=1 t=−T i=1 n2 PM U A1 (t)PM U A2 (t + τ ) t=−T T rial S1i (t) = ( N T rial i=1 j=1 n1 T N T rial T j=1 n2 P1i (t) t=−T i=1 T P2j (t + τ ) j=1 T T rial T rial S1i (t)S2j (t T rial=1 t=−T n1 T rial S2j (t + τ ) − + τ) − P1i (t)P2j (t + τ )) t=−T n2 C1i,2j (τ ) = i=1 j=1 Thus, the MUA-MUA cross-correlation function CM U A1,M U A2 (τ ) equals the sum of all n1 × n2 between-electrode SUA-SUA cross-correlation functions C1i,2j (τ ). This implies that the shape of the MUA-MUA cross-correlation function provides an unbiased estimate of the average shape of the contributing SUA-SUA cross-correlation functions. Appendix B List of publications B.1 Peer-reviewed journal publication [1] Bong J. H., Salazar R. F., Gray C. M., Yen S. C., “Neuronal correlates of perceptual salience in the primary visual cortex, Journal of Neuroscience (2012) (submitted) B.2 Conference publication [1] Bong J. H., Yen S. C., Salazar R. F., Gray C. M., “Neuronal representations of visual salience in primary visual cortex, BMC Neuroscience, 9(Suppl 1):P129 (11 July 2008) 115 References Adrian, E. D., & Zotterman, Y. (1926). The impulses produced by sensory nerveendings: Part II. the response of a single end-organ. J Physiol (Lond), 61 (2), 151–71. Aertsen, A. M., Gerstein, G. L., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal firing correlation: modulation of “effective connectivity”. J Neurophysiol , 61 (5), 900–17. Aggelopoulos, N. C., Franco, L., & Rolls, E. T. (2005). Object perception in natural scenes: encoding by inferior temporal cortex simultaneously recorded neurons. J Neurophysiol , 93 (3), 1342–57. Angelucci, A., & Bullier, J. (2003). Reaching beyond the classical receptive field of V1 neurons: horizontal or feedback axons? J Physiol , 97 (2-3), 141–154. Angelucci, A., Levitt, J. B., Walton, E. J. S., Hup´e, J.-M., Bullier, J., & Lund, J. S. (2002). Circuits for local and global signal integration in primary visual cortex. J Neurosci , 22 (19), 8633–8646. Angelucci, A., & Sainsbury, K. (2006). Contribution of feedforward thalamic afferents and corticogeniculate feedback to the spatial summation area of macaque V1 and LGN. J Comparative Neurology, 498 (3), 330–351. Bair, W., Zohary, E., & Newsome, W. T. (2001). Correlated firing in macaque visual area mt: time scales and relationship to behavior. J Neurosci , 21 (5), 1676–97. Bradley, D. R., & Petry, H. M. (1977). Organizational determinants of subjective contour: the subjective necker cube. Am J Psychol , 90 (2), 253–62. Brody, C. (1999a). Correlations without synchrony. Neural computation, 11 (7), 1537–51. Brody, C. (1999b). Disambiguating different covariation types. Neural computation, 11 (7), 1527–35. Caputo, G. (1998). Texture brightness filling-in. Vision Research, 38 (6), 841–51. Castelo-Branco, M., Goebel, R., Neuenschwander, S., & Singer, W. (2000). Neural synchrony correlates with surface segregation rules. Nature, 405 (6787), 685–9. Celebrini, S., Thorpe, S., Trotter, Y., & Imbert, M. (1993). Dynamics of orientation coding in area v1 of the awake primate. Vis Neurosci , 10 (5), 811–25. 116 References 117 Chelaru, M. I., & Dragoi, V. (2008). Efficient coding in heterogeneous neuronal populations. Proc Natl Acad Sci USA, 105 (42), 16344–9. Cohen, M. R., & Maunsell, J. H. R. (2009). Attention improves performance primarily by reducing interneuronal correlations. Nat Neurosci , 12 (12), 1594–600. Cuf´ı, X., Mu˜ noz, X., Freixenet, J., & Mart´ı, J. (2002). A review on image segmentation techniques integrating region and boundary information. In Advances in imaging and electron physics (Vol. 120, p. - 39). Academic Press. Driver, J., Baylis, G. C., & Rafal, R. D. (1992). Preserved figure-ground segregation and symmetry perception in visual neglect. Nature, 360 (6399), 73–5. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., et al. (1988). Coherent oscillations: a mechanism of feature linking in the visual cortex? multiple electrode and correlation analyses in the cat. Biol Cybern, 60 (2), 121–30. Engel, A. K., Kreiter, A. K., K¨onig, P., & Singer, W. (1991). Synchronization of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat. Proc Natl Acad Sci USA, 88 (14), 6048–52. Engel, A. K., Knig, P., Gray, C. M., & Singer, W. (1990). Stimulus-dependent neuronal oscillations in cat visual cortex: Inter-columnar interaction as determined by cross-correlation analysis. Euro J Neurosci , (7), 588–606. Farin, G. E. (1990). Curves and surfaces for computer aided geometric design: a practical guide (2nd ed.). Boston: Academic Press. Field, D. J., Hayes, A., & Hess, R. F. (1993). Contour integration by the human visual system: evidence for a local “association field”. Vision Res, 33 (2), 173–93. Foffani, G., Chapin, J. K., & Moxon, K. A. (2008). Computational role of large receptive fields in the primary somatosensory cortex. J Neurophysiol , 100 (1), 268–80. Freiwald, W. A., Kreiter, A. K., & Singer, W. (1995). Stimulus dependent intercolumnar synchronization of single unit responses in cat area 17. Neuroreport, (17), 2348–52. Fries, P., Neuenschwander, S., Engel, A. K., Goebel, R., & Singer, W. (2001). Rapid feature selective neuronal synchronization through correlated latency shifting. Nat Neurosci , (2), 194–200. Fries, P., Roelfsema, P. R., Engel, A. K., K¨onig, P., & Singer, W. (1997). Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proc Natl Acad Sci USA, 94 (23), 12699–704. Furukawa, S., & Middlebrooks, J. C. (2002). Cortical representation of auditory space: information-bearing features of spike patterns. J Neurophysiol , 87 (4), 1749–62. Gawne, T. J., Kjaer, T. W., Hertz, J. A., & Richmond, B. J. (1996a). Adjacent visual cortical complex cells share about 20% of their stimulus-related information. Cereb Cortex , (3), 482–9. References 118 Gawne, T. J., Kjaer, T. W., & Richmond, B. J. (1996b). Latency: another potential code for feature binding in striate cortex. J Neurophysiol , 76 (2), 1356–60. Gawne, T. J., & Richmond, B. J. (1993). How independent are the messages carried by adjacent inferior temporal cortical neurons? J Neurosci , 13 (7), 2758–71. Gilbert, C. D. (1992). Horizontal integration and cortical dynamics. Neuron, (1), 1–13. Girard, P., Hup´e, J. M., & Bullier, J. (2001). Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities. J Neurophysiol , 85 (3), 1328–1331. Gochin, P. M., Miller, E. K., Gross, C. G., & Gerstein, G. L. (1991). Functional interactions among neurons in inferior temporal cortex of the awake macaque. Exp Brain Res, 84 (3), 505–16. Gollisch, T. (2009). Throwing a glance at the neural code: rapid information transmission in the visual system. HFSP J , (1), 36–46. Gollisch, T., & Meister, M. (2008). Rapid neural coding in the retina with relative spike latencies. Science, 319 (5866), 1108–11. Gray, C. M. (1999). The temporal correlation hypothesis of visual feature integration: still alive and well. Neuron, 24 (1), 31-47, 111–25. Gray, C. M., Goodell, B., & Lear, A. (2007). Multichannel micromanipulator and chamber system for recording multineuronal activity in alert, non-human primates. J Neurophysiol , 98 (1), 527–36. Gray, C. M., K¨onig, P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338 (6213), 334–7. Gray, C. M., & Singer, W. (1989). Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc Natl Acad Sci USA, 86 (5), 1698–702. Gray, C. M., & Viana di Prisco, G. (1997). Stimulus-dependent neuronal oscillations and local synchronization in striate cortex of the alert cat. J Neurosci, 17 (9), 3239–53. Greenberg, D. S., Houweling, A. R., & Kerr, J. N. D. (2008). Population imaging of ongoing neuronal activity in the visual cortex of awake rats. Nat Neurosci, 11 (7), 749–51. Gr¨ un, S., Diesmann, M., & Aertsen, A. (2002a). Unitary events in multiple singleneuron spiking activity: I. detection and significance. Neural computation, 14 (1), 43–80. Gr¨ un, S., Diesmann, M., & Aertsen, A. (2002b). Unitary events in multiple singleneuron spiking activity: II. nonstationary data. Neural computation, 14 (1), 81–119. Harris, K. D., Henze, D. A., Csicsvari, J., Hirase, H., & Buzs´aki, G. (2000). Accuracy of tetrode spike separation as determined by simultaneous intracellular and References 119 extracellular measurements. J Neurophysiol , 84 (1), 401–14. Hatsopoulos, N., Geman, S., Amarasingham, A., & Bienenstock, E. (2003). At what time scale does the nervous system operate? Neurocomputing, 5254 (0), 25 29. Hess, R. F., & Dakin, S. C. (1997). Absence of contour linking in peripheral vision. Nature, 390 (6660), 602–604. Hess, R. F., & Dakin, S. C. (1999). Contour integration in the peripheral field. Vision Res, 39 (5), 947–959. Hess, R. F., Dakin, S. C., & Field, D. J. (1998). The role of ”contrast enhancement” in the detection and appearance of visual contours. Vision Res, 38 (6), 783–7. Hirabayashi, T., & Miyashita, Y. (2005). Dynamically modulated spike correlation in monkey inferior temporal cortex depending on the feature configuration within a whole object. J Neurosci , 25 (44), 10299–307. Hup´e, J. M., James, A. C., Girard, P., & Bullier, J. (2001a). Response modulations by static texture surround in area v1 of the macaque monkey not depend on feedback connections from v2. J Neurophysiol , 85 (1), 146–63. Hup´e, J. M., James, A. C., Girard, P., Lomber, S. G., Payne, B. R., & Bullier, J. (2001b). Feedback connections act on the early part of the responses in monkey visual cortex. J Neurophysiol , 85 (1), 134–145. Hup´e, J. M., James, A. C., Payne, B. R., Lomber, S. G., Girard, P., & Bullier, J. (1998). Cortical feedback improves discrimination between figure and background by v1, v2 and v3 neurons. Nature, 394 (6695), 784–7. Jagadeesh, B., Gray, C. M., & Ferster, D. (1992). Visually evoked oscillations of membrane potential in cells of cat visual cortex. Science, 257 (5069), 552–4. Judge, S. J., Richmond, B. J., & Chu, F. C. (1980). Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res, 20 (6), 535–8. Julesz, B. (1984). A brief outline of the texton theory of human vision. Trends Neurosci , (2), 41 - 45. Kapadia, M. K., Ito, M., Gilbert, C. D., & Westheimer, G. (1995). Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in v1 of alert monkeys. Neuron, 15 (4), 843–56. Kapadia, M. K., Westheimer, G., & Gilbert, C. D. (2000). Spatial distribution of contextual interactions in primary visual cortex and in visual perception. J Neurophysiol , 84 (4), 2048–62. Koffka, K. (1935). Principles of gestalt psychology. Harcourt Brace, New York. Kohn, A., Zandvakili, A., & Smith, M. A. (2009). Correlations and brain states: from electrophysiology to functional imaging. Curr Opin Neurobiol , 19 (4), 434 - 438. Lamme, V. A. (1995). The neurophysiology of figure-ground segregation in primary References 120 visual cortex. J Neurosci , 15 (2), 1605–15. Lamme, V. A., Rodriguez-Rodriguez, V., & Spekreijse, H. (1999). Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cereb Cortex , (4), 406–13. Lamme, V. A., & Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci , 23 (11), 571–9. Lamme, V. A., & Spekreijse, H. (1998). Neuronal synchrony does not represent texture segregation. Nature, 396 (6709), 362–6. Lamme, V. A., Van Dijk, B. W., & Spekereijse, H. (1993a). Organization of texture segregation processing in primate visual cortex. Vis Neurosci, 10 (5), 781–90. Lamme, V. A., Van Dijk, B. W., & Spekreijse, H. (1993b). Contour from motion processing occurs in primary visual cortex. Nature, 363 (6429), 541–3. Lamme, V. A., Zipser, K., & Spekreijse, H. (1998). Figure-ground activity in primary visual cortex is suppressed by anesthesia. Proc Natl Acad Sci USA, 95 (6), 3263– 8. Landy, M. S., & Bergen, J. R. (1991). Texture segregation and orientation gradient. Vision Research, 31 (4), 679–91. Lee, D., Port, N. L., Kruse, W., & Georgopoulos, A. P. (1998a). Variability and correlated noise in the discharge of neurons in motor and parietal areas of the primate cortex. J Neurosci , 18 (3), 1161–70. Lee, T. S., & Mumford, D. (2003). Hierarchical bayesian inference in the visual cortex. J Opt Soc Am A Opt Image Sci Vis, 20 (7), 1434–48. Lee, T. S., Mumford, D., Romero, R., & Lamme, V. A. (1998b). The role of the primary visual cortex in higher level vision. Vision Res, 38 (15-16), 2429–54. Leopold, D. A., Murayama, Y., & Logothetis, N. K. (2003). Very slow activity fluctuations in monkey visual cortex: implications for functional brain imaging. Cereb Cortex , 13 (4), 422–33. Li, W., Pi¨ech, V., & Gilbert, C. D. (2006). Contour saliency in primary visual cortex. Neuron, 50 (6), 951–62. Li, W., Pi¨ech, V., & Gilbert, C. D. (2008). Learning to link visual contours. Neuron, 57 (3), 442–51. Li, Z. (1998). A neural model of contour integration in the primary visual cortex. Neural computation, 10 (4), 903–40. Livingstone, M. S. (1996). Oscillatory firing and interneuronal correlations in squirrel monkey striate cortex. J Neurophysiol , 75 (6), 2467–85. Maldonado, P. E., Friedman-Hill, S., & Gray, C. M. (2000). Dynamics of striate cortical activity in the alert macaque: Ii. fast time scale synchronization. Cereb Cortex , 10 (11), 1117–31. Maunsell, J. H., & Gibson, J. R. (1992). Visual response latencies in striate cortex of the macaque monkey. J Neurophysiol , 68 (4), 1332–44. References 121 May, K. A., & Hess, R. F. (2008). Effects of element separation and carrier wavelength on detection of snakes and ladders: implications for models of contour integration. J Vision, (13), 4, 1–23. Mitchell, J. F., Sundberg, K. A., & Reynolds, J. H. (2009). Spatial attention decorrelates intrinsic activity fluctuations in macaque area v4. Neuron, 63 (6), 879–88. Mumford, D. (1992). On the computational architecture of the neocortex. ii. the role of cortico-cortical loops. Biological Cybernetics, 66 (3), 241–51. N´adasdy, Z., Hirase, H., Czurk´o, A., Csicsvari, J., & Buzs´aki, G. (1999). Replay and time compression of recurring spike sequences in the hippocampus. J Neurosci , 19 (21), 9497–507. Nelken, I., Chechik, G., Mrsic-Flogel, T. D., King, A. J., & Schnupp, J. W. H. (2005). Encoding stimulus information by spike numbers and mean response time in primary auditory cortex. J Comput Neurosci , 19 (2), 199–221. Nothdurft, H. C. (1985). Orientation sensitivity and texture segmentation in patterns with different line orientation. Vision Research, 25 (4), 551–60. Nothdurft, H. C. (1992). Feature analysis and the role of similarity in preattentive vision. Percept Psychophys, 52 (4), 355–75. Nothdurft, H. C. (1994). Common properties of visual segmentation. Ciba Found Symp, 184 , 245-59; discussion 260–71. Oram, M. W. (2011). Visual stimulation decorrelates neuronal activity. J Neurophysiol , 105 (2), 942–57. Panzeri, S., Petersen, R. S., Schultz, S. R., Lebedev, M., & Diamond, M. E. (2001). The role of spike timing in the coding of stimulus location in rat somatosensory cortex. Neuron, 29 (3), 769–77. Petersen, R. S., Panzeri, S., & Diamond, M. E. (2001). Population coding of stimulus location in rat somatosensory cortex. Neuron, 32 (3), 503–14. Pettet, M. W., McKee, S. P., & Grzywacz, N. M. (1998). Constraints on long range interactions mediating contour detection. Vision Res, 38 (6), 865–79. Pettet, M. W., & Verghese, P. (1997). Enhanced contour detection is not mediated by apparent contrast. 27th Annual Meeting of the Society for Neuroscience 1997, New Orleans, LA, USA, 23 , 176. Pipa, G., Wheeler, D. W., Singer, W., & Nikoli´c, D. (2008). Neuroxidence: reliable and efficient analysis of an excess or deficiency of joint-spike events. J Comput Neurosci , 25 (1), 64–88. Polat, U., Mizobe, K., Pettet, M. W., Kasamatsu, T., & Norcia, A. M. (1998). Collinear stimuli regulate visual responses depending on cell’s contrast threshold. Nature, 391 (6667), 580–4. Polat, U., & Sagi, D. (1993). Lateral interactions between spatial channels: suppression and facilitation revealed by lateral masking experiments. Vision Res, 33 (7), 993–9. References 122 Raiguel, S. E., Xiao, D. K., Marcar, V. L., & Orban, G. A. (1999). Response latency of macaque area MT/V5 neurons and its relationship to stimulus parameters. J Neurophysiol , 82 (4), 1944–56. Reich, D. S., Mechler, F., & Victor, J. D. (2001). Temporal coding of contrast in primary visual cortex: when, what, and why. J Neurophysiol , 85 (3), 1039–50. Roelfsema, P. R., Lamme, V. A. F., & Spekreijse, H. (2004). Synchrony and covariation of firing rates in the primary visual cortex during contour grouping. Nat Neurosci , (9), 982–91. Samonds, J. M., Zhou, Z., Bernard, M. R., & Bonds, A. B. (2006). Synchronous activity in cat visual cortex encodes collinear and cocircular contours. J Neurophysiol , 95 (4), 2602–16. Sawitzki, G. (1996). The excess mass approach and the analysis of multi-modality. In From data to knowledge: Theoretical and practical aspects of classification, data analysis and knowledge organization (p. 203 - 211). Springer. Shadlen, M. N., Britten, K. H., Newsome, W. T., & Movshon, J. A. (1996). A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J Neurosci , 16 (4), 1486–510. Shadlen, M. N., & Movshon, J. A. (1999). Synchrony unbound: a critical evaluation of the temporal binding hypothesis. Neuron, 24 (1), 67-77, 111–25. Shamir, M., & Sompolinsky, H. (2006). Implications of neuronal diversity on population coding. Neural computation, 18 (8), 1951–86. Shmiel, T., Drori, R., Shmiel, O., Ben-Shaul, Y., Nadasdy, Z., Shemesh, M., et al. (2006). Temporally precise cortical firing patterns are associated with distinct action segments. J Neurophysiol , 96 (5), 2645–52. Singer, W., & Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. Annu Rev Neurosci , 18 , 555–86. Smith, A. T., Singh, K. D., Williams, A. L., & Greenlee, M. W. (2001). Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. Cereb Cortex , 11 (12), 1182–1190. Stark, E., & Abeles, M. (2009). Unbiased estimation of precise temporal correlations between spike trains. J Neurosci Methods, 179 (1), 90–100. Stopfer, M., Bhagavan, S., Smith, B. H., & Laurent, G. (1997). Impaired odour discrimination on desynchronization of odour-encoding neural assemblies. Nature, 390 (6655), 70–4. Sugihara, T., Qiu, F. T., & von der Heydt, R. (2011). The speed of context integration in the visual cortex. J Neurophysiol , 106 (1), 374–385. Sup`er, H., & Lamme, V. A. F. (2007). Altered figure-ground perception in monkeys with an extra-striate lesion. Neuropsychologia, 45 (14), 3329–34. Sup`er, H., Spekreijse, H., & Lamme, V. A. (2001). Two distinct modes of sensory processing observed in monkey primary visual cortex (v1). Nat Neurosci , (3), References 123 304–10. Tamura, H., Kaneko, H., Kawasaki, K., & Fujita, I. (2004). Presumed inhibitory neurons in the macaque inferior temporal cortex: visual response properties and functional interactions with adjacent neurons. J Neurophysiol , 91 (6), 2782–96. Thorpe, S., Delorme, A., & Van Rullen, R. (2001). Spike-based strategies for rapid processing. Neural Netw , 14 (6-7), 715–25. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381 (6582), 520–2. Tolhurst, D. J. (1989). The amount of information transmitted about contrast by neurones in the cat’s visual cortex. Vis Neurosci, (4), 409–13. Ts’o, D. Y., & Gilbert, C. D. (1988). The organization of chromatic and spatial interactions in the primate striate cortex. J Neurosci , (5), 1712–27. Ts’o, D. Y., Gilbert, C. D., & Wiesel, T. N. (1986). Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by cross-correlation analysis. J Neurosci , (4), 1160–70. Usrey, W. M., & Reid, R. C. (1999). Synchronous activity in the visual system. Annu Rev Physiol , 61 , 435–56. Van der Togt, C., Kalitzin, S., Spekreijse, H., Lamme, V. A. F., & Sup`er, H. (2006). Synchrony dynamics in monkey v1 predict success in visual detection. Cereb Cortex , 16 (1), 136–48. Van Kan, P. L., Scobey, R. P., & Gabor, A. J. (1985). Response covariance in cat visual cortex. Exp Brain Res, 60 (3), 559–63. Van Rullen, R., Guyonneau, R., & Thorpe, S. J. (2005). Spike times make sense. Trends Neurosci , 28 (1), - 4. Van Rullen, R., & Thorpe, S. J. (2001). Rate coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex. Neural computation, 13 (6), 1255–83. Voytenko, S., & Galazyuk, A. (2008). Timing of sound-evoked potentials and spike responses in the inferior colliculus of awake bats. Neurosci , 155 (3), 923 - 936. Wiener, M. C., & Richmond, B. J. (2003). Decoding spike trains instant by instant using order statistics and the mixture-of-poissons model. J Neurosci, 23 (6), 2394–406. Yen, S. C., Baker, J., & Gray, C. M. (2007). Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex. J Neurophysiol , 97 (2), 1326–41. Yen, S. C., & Finkel, L. H. (1998). Extraction of perceptually salient contours by striate cortical networks. Vision Res, 38 (5), 719–41. Zhou, H., Friedman, H. S., & von der Heydt, R. (2000). Coding of border ownership in monkey visual cortex. J Neurosci , 20 (17), 6594–6611. Zipser, K., Lamme, V. A., & Schiller, P. H. (1996). Contextual modulation in primary visual cortex. J Neurosci , 16 (22), 7376–89. [...]... differences in some of neurons that we observed in this study Such early firing rate differences in our data suggest that striate cortex may be the site of origin for the neuronal correlates of visual salience rather than merely representing feedback signals from extra-striate cortex x List of Tables 4.1 Summary of the single neuron firing rate ROC analysis 57 4.2 Summary of the single neuron firing rate... for interactions between cells with non-overlapping receptive fields As the distance between the two cells increased, the overlap of the receptive fields of the cells participating in the interactions gradually diminished Several groups in Germany (Gray and Singer, 1989; Eckhorn et al., 1988; Engel et al., 1990) looked at temporal patterns in the firing of single cells and groups of cells in cat visual cortex, ... Introduction The primary visual cortex (V1) is the best studied visual area in the brain However, our understanding of how cortical neurons encode and process the visual stimulus is still extremely limited One question that has received considerable attention is the role of V1 in the scene segmentation process The neurons in V1 have the smallest receptive fields and are thus capable of representing and processing... hypotheses that have the potential to account for this CHAPTER 2 LITERATURE REVIEW 2.1 6 Firing Rate Hypothesis The concept of firing rate has been successfully applied during the last 80 years It dates back to the pioneering work of Adrian in 1926 (Adrian and Zotterman, 1926), who showed that the firing rate of stretch receptor neurons in the muscles is related to the force applied to the muscle In the. .. compared to the contour condition, and lower synchrony observed in the control condition for the windows from 50 to 100 ms when compared to the contour condition Finally, we also investigated the response latencies of the neurons Again, using methods from the signal detection theory, we found that 28 out of 181 cells exhibited significant differences in their latencies when they were activated by part of a... perceptual grouping has yet to be clearly determined Therefore, in this study, three general hypotheses will be put forth to account for these functions These three hypotheses are the firing rate, the temporal correlation, and the response latency hypotheses In Chapter 2, we will briefly review some of the studies in this area In Chapter 3, we will describe the experimental setup and some of the analysis... that involves a lot of active interpretation of the world One example would be that it emphasizes areas of difference (or contrast) within the visual stimulus, and minimizes areas of uniformity Even though our understanding of the visual system has improved tremendously in the past few decades due to the advancement of neural recording technologies, we are still largely ignorant about how distributed neuronal. .. indicating an important role of stimulus-driven, bottom-up processes in contour integration In their subsequent study (Li et al 2008), they found that contour integration in V1 depended strongly on perceptual learning and top-down in uences that are specific to contour detection They came to this conclusion because they observed that the effect of contour integration in V1 disappeared under anesthesia and in. .. direction deviates from that preferred by the cell, and decreases with increasing stimulus contrast or luminance For example in the study presented by Gawne et al (1996b), they recorded the responses of striate cortical complex cells in fixating monkeys while presenting a set of oriented stimuli that varied in contrast Their results showed that the firing rate defines the stimulus orientation, while the latency... result suggesting some form of facilitation A more interesting finding by Li et al (2006) provides direct evidence that, in monkeys performing a contour detection task, there was a close correlation between the responses of V1 neurons and the perceptual saliency of contours Their receiver operating characteristic analysis showed that single neuronal responses encode the presence or absence of a contour . NEURONAL CORRELATES OF PERCEPTUAL SALIENCE IN SPIKE TRAINS FROM THE PRIMARY VISUAL CORTEX BONG JIT HON NATIONAL UNIVERSITY OF SINGAPORE 2012 NEURONAL CORRELATES OF PERCEPTUAL SALIENCE IN SPIKE. 116 vii Summary In this thesis, we examined the representation of visual saliency in the responses of neurons in the primary visual cortex. We investigated this by recording from the primary visual cortex of. IN SPIKE TRAINS FROM THE PRIMARY VISUAL CORTEX BONG JIT HON B.Eng.(Hons.), NUS A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY OF ENGINEERING DEPARTMENT OF ELECTRICAL & COMPUTER ENGINEERING NATIONAL