Cognitive learning and memory systems using spiking neural networks

COGNITIVE LEARNING AND MEMORY SYSTEMS USING SPIKING NEURAL NETWORKS HU JUN NATIONAL UNIVERSITY OF SINGAPORE 2014 COGNITIVE LEARNING AND MEMORY SYSTEMS USING SPIKING NEURAL NETWORKS HU JUN B Eng., Nanjing University of Aeronautics and Astronautics A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2014 Acknowledgments Acknowledgments This work was done in the computational intelligence group led by Dr Tan Kay Chen at the Department of Electrical and Computer Engineering, National University of Singapore and financially supported by Agency for Science, Technology and Research (A*STAR) and National University of Singapore First of all, I would like to express deepest appreciation to my supervisor Dr Tan Kay Chen for introducing me into the splendid research field of computational intelligence His valuable guidance and support help me to accomplish my research I wish to thank Dr Tang Huajin for his patient and consistent technical advisory and encouraging support His enthusiasm for studying and dedication to research have inspired me throughout my Ph.D course I would like to express gratitude to Dr Tan Chin Hiong, Dr Yu Jiali, Dr Huang Weiwei, and Dr Cheu Eng Yeow in Institute for Infocomm Research, A*STAR, with whom I worked together and from whom I learned how to work professionally as a researcher My thanks also go to my colleagues of our Computational Intelligence Research Group Dr Shim Vui Ann for being my senior who kindly shared his research experience and encouraged me from time to time, Yu Qiang for accompanying me during the last three and half years, Gee Sen Bong for sharing his excellent coding skills, Willson Amalraj A for demonstrating how to convert research achievements into applications, Arrchana Muruganantham for teaching me website design, Lim Pin for sharing his work experience, Qiu Xin for keeping i Acknowledgments our lab full of joy, Zhang Chong and Goh Sim Kuan for being the replacements I also want to thank lab officers of Control & Simulation Lab, Mr Zhang Hengwei and Ms Sara K for their continuous assistance I would like to thank A/Prof Dipti Srinivasan and A/Prof Xiang Cheng at National University of Singapore, who provide me suggestive critiques and encouraging support Last but not least, I would like to dedicate this thesis to my parents for their constant support and unconditional love ii Contents Acknowledgments i Contents iii Summary vii List of Tables x List of Figures xi Nomenclature xiv Introduction 1.1 Background and Basic Concepts 1.1.1 Cognitive Learning and Memory in the Brain 1.1.2 Artificial Neural Networks 1.2 Research Scope and Contributions 1.3 Organization of the Thesis Literature Review 2.1 12 Spiking Neuron Models iii 12 Contents 2.2 Spiking Neural Networks 15 2.2.1 Neural Coding in Spiking Neural Networks 16 2.2.2 Learning in Spiking Neural Networks 20 2.2.3 Memory Models Using Spiking Neural Networks 27 A Spike-Timing Based Integrated Model for Pattern Recognition 30 3.1 Introduction 30 3.2 The Integrated Model 35 3.2.1 Neuron Model and General Structure 35 3.2.2 Latency-Phase Encoding 35 3.2.3 Supervised Spike-Timing Based Learning 39 Numerical Simulations 42 3.3.1 Network Architecture and Encoding of Grayscale Images 42 3.3.2 Learning Performance 44 3.3.3 Generalization Capability 45 3.3.4 Parameters Evaluation 48 3.3.5 Capacity of the Integrated System 52 3.4 Related Works 54 3.5 Conclusion 57 3.3 A Computationally Efficient Associative Memory Model of Hippocampus CA3 by Spiking Neurons 59 4.1 Introduction 59 4.2 CA3 Model 63 iv Contents 4.2.1 Spike Response Neurons 64 4.2.2 SRM Based Pyramidal Cells and Interneurons 65 4.3 Synaptic Modification 66 4.4 Experimental Results and Discussions 69 4.4.1 Associative Memory Storage and Recall 69 4.4.2 Computational Efficiency 75 Discussion and Conclusion 79 4.5 A Hierarchical Organized Memory Model with Temporal Population Codes 81 5.1 Introduction 81 5.2 The Hierarchical Organized Memory Model 85 5.2.1 Pyramidal Cells and Theta/Gamma Oscillations 86 5.2.2 Temporal Population Coding 87 5.2.3 The Spike-timing Based Learning and NMDA Channels 90 Numerical simulation 93 5.3.1 94 5.3 5.4 Network Behavior Discussion 107 5.4.1 5.4.2 Storage, Recall and Organization of Memory 108 5.4.3 Temporal Compression and Information Binding 109 5.4.4 5.5 Information Flow and Emergence of Neural Cliques Related Works 110 Conclusion 107 113 Hierarchical Organized Memory Model with Spike-driven Learn- v Contents ing of Visual Features 115 6.1 Introduction 115 6.2 The Hierarchical Organized Memory Model 118 6.2.1 Network Architecture 118 6.2.2 Temporal Population Coding in Encoding Layer 119 6.2.3 Spike-timing Based Learning 120 6.3 Numerical Simulation 124 6.4 Discussion 126 6.5 Conclusion and Future Works 127 Conclusions and Future Works 128 7.1 Conclusions 128 7.2 Future Works 131 Bibliography 133 Appendix: Author’s Publications 147 vi Summary Summary Neural networks have been studied for many years in efforts to mimic many aspects of biological neural systems Remarkable progress has been made in solving problems such as vehicle control navigation, decision making, financial applications, and data mining using neural networks However, humans can thoroughly defeat artificial intelligence with little difficulty when facing with cognitive tasks such as pattern recognition Moreover, with the increasing demand of our modern life, cognitive function becomes more and more important in intelligent systems Rate coding is a traditional coding scheme used in neural networks However, the behavioral response of a neuron may be too fast that makes it is impossible to describe its activity relying on the firing rate With the development of instruments and experimental techniques, increasing findings suggest that spike times make sense in encoding information The idea that information could be encoded by precisely timed spikes has drawn increasing attention over the past 20 years By incorporating the concept of time, spiking neural networks (SNNs) is compatible with the temporal code rather than the rate code The goal of this thesis is to investigate aspects of theories of spiking neural networks in an attempt to develop cognitive learning and memory models for computational intelligence Firstly, a spike-timing-based integrated model is devised for solving pattern recognition problem We attempt to build an integrated model based on SNNs, which performs sensory neural encoding and supervised learning with precisely vii Bibliography Bi, G Q., & Poo, M M (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, & postsynaptic cell type Journal of Neuroscience, 18 (24), 10464 – 10472 Bi, G Q., & Poo, M M (2001) Synaptic modification by correlated activity: Hebb’s postulate revisited Annual Review of Neuroscience, 24, 139 – 166 Bialek, W., Rieke, F., De Ruyter van Steveninck, R R., & Warland, D (1991) Reading a neural code Science, 252, 1854 – 1857 Blumenfeld, B., Preminger, S., Sagi, D., & Tsodyks, M (2006) Dynamics of memory representations in networks with novelty-facilitated synaptic plasticity Neuron, 52, 383 – 394 Bohte, S M., Kok, J N., & Poutr´, H L (2002) Error-backpropagation in e temporally encoded networks of spiking neurons Neurocomputing, 48, 17 – 37 Bohte, S M., Poutr´, H L., & Kok, J N (2002) Unsupervised clustering e with spiking neurons by sparse temporal coding and multilayer RBF networks IEEE Transaction on Neural Networks, 13, 426 – 435 Bourlard, H & Morgan, N (1993) Continuous speech recognition by connectionist statistical methods IEEE Transaction on Neural Networks, 4, 893 – 909 Brader, J M., Senn, W., & Fusi, S (2007) Learning real-world stimuli in a neural network with spike-driven synaptic dynamics Neural Computation, 19, 2881 – 2912 Bragin, A., Jando, G., Nadasdy, Z., Hetke, J., Wise, K., & Buzsaki, G (1995) Gamma (40-100 hz) oscillation in the hippocampus of the behaving rat The Journal of Neuroscience, 15 (1), 47 – 60 Brembs, B., Lorenzetti, F D., Reyes, F D., Baxter, D A., & Byrne, J H (2002) Operant learning in aplysia: neuronal correlates and mechanisms Science, 296 (5573), 1706 – 1709 Brody, C D., & Hopfield, J J (2003) Simple networks for spike-timing-based computation, with application to olfactory processing Neuron, 37 (5), 843 – 852 Burkitt, A N (2006) A review of the integrate-and-fire neuron model: II inhomogeneous synaptic input and network properties Biological Cybernetics, 95 (1), – 19 Burkitt, A N (2006) A review of the integrate-and-fire neuron model: Ii inhomogeneous synaptic input and network properties Biological Cybernetics, 95 (2), 97 – 112 Buzs´ki, G (2002) Theta oscillations in the hippocampus Neuron, 33 (3), 325 a – 340 134 Bibliography Cantero, J L., Atienza, M., Stickgold, R., Kahana, M J., Madsen, J R., & Kocsis, B (2003) Sleep-dependent θ oscillations in the human hippocampus and neocortex The Journal of Neuroscience, 23 (34), 897 – 903 Carr, C E (1993) Processing of temporal information in the brain Annual Review of Neuroscience, 16, 223 – 243 Chapelle, O., Haffner, P., & Vapnik, V N (1999) Support vector machines for histogram-based image classification IEEE Transaction on Neural Networks, 10 (5), 1055 – 1064 Celebrini, S., Thorpe, S., Trotter, Y., & Imbert, M (1993) Dynamics of orientation coding in area V1 of the awake primate Visual Neuroscience, 10, 811 – 825 Cheu, E Y., Yu, J., Tan, C H., & Tang, H (2012) Synaptic conditions for autoassociative memory storage and pattern completion in Jensen et al.’s model of hippocampal area CA3 Journal of Computational Neuroscience, 33 (3), 435 – 447 Chrobak, J J., & Buzsaki, G (1998) Gamma oscillations in the entorhinal cortex of the freely behaving rat Journal of Neuroscience, 18, 388 – 398 Clopath, C., Longtin, A., & Gerstner, W (2008) An online Hebbian learning rule that performs independent component analysis In: Platt J, Koller D, Singer Y, Roweis S, Eds, Advances in Neural Information Processing Systems, 20, 321 – 328 Cutsuridis, V., & Wennekers, T (2009) Hippocampus, microcircuits and associative memory Neural Networks, 22 (8), 1120 – 1128 Cutsuridis, V., Cobb, S., & Graham, B P (2010) Encoding and retrieval in a model of the hippocampal CA1 microcircuit Hippocampus, 20 (3), 423 – 446 Deadwyler, S A., & Hampson, R E (1997) The significance of neural ensemble codes during behavior and cognition Annual Review of Neuroscience, 20, 217 – 44 deCharms, R C (1998) Information coding in the cortex by independent or coordinated populations PNAS, 95, 15166 – 15168 Delorme, A., Perrinet, L., & Thorpe, S J (2001) Networks of integrate-and-fire neurons using Rank Order Coding B: Spike timing dependent plasticity and emergence of orientation selectivity Neurocomputing, 38-40, 539 – 545 DeVries, S H (1999) Correlated Firing in Rabbit Retinal Ganglion Cells Journal of Neurophysiology, 81, 908 – 920 Du Bois-Reymond, E (1848) Berlin: Verlag Untersuchungen uber Thierische Elektricităt ă a Durstewitz, D., Seamans, J K., & Sejnowski, T J (2000) Neurocomputational models of working memory Nature neuroscience, 3, 1184 – 1191 135 Bibliography Eichenbaum, H (2000) A cortical-hippocampal system for declarative memory Nature Reviews Neuroscience, 1, 41 – 50 Farries, M., & Fairhall, A (2007) Reinforcement learning with modulated spike timing-dependent synaptic plasticity Journal of Neurophysiol, 98, 3648 – 3665 Fetz, E E., & Baker, M A (1973) Operantly conditioned patterns of precentral unit activity and correlated responses in adjacent cells and contralateral muscles Journal of Neurophysiol, 36 (2), 179 – 204 Florian, R V (2007) Reinforcement learning through modulation of spiketiming-dependent synaptic plasticity Neural Computation, 19 (6), 1468 – 1502 Florian, R V (2012) The chronotron: A neuron that learns to fire temporally precise spike patterns PLoS ONE, (8) Frankland, P W., & Bontempi, B (2005) The organization of recent and remote memories Nature Reviews Neuroscience, 6, 119 – 130 Freedman, D J, Riesenhuber M., Poggio T., & Miller E K (2001) Categorical representation of visual stimuli in the primate prefrontal cortex Science, 291, 312 – 331 Freedman, D J, Riesenhuber M., Poggio T., & Miller E K (2003) A comparison of primate prefrontal and inferior temporal cortices during visual categorization The Journal of Neuroscience, 23, 5235 – 5246 Fleischer, J G., & Krichmar, J L (2007) Sensory integration and remapping in a model of the medial temporal lobe during maze navigation by a brain-based device Journal of Integrative Neuroscience, (3), 403 – 431 Fusi, S., & Abbott, L F (2007) Limits on the memory storage capacity of bounded synapses Nature Neuroscience, 10 (4), 485 – 493 Gawne, T J., Kjaer, T W., & Richmond, B J (1996) Latency: another potential code for feature binding in striate cortex Journal of Neurophysiology, 76, 1356 – 1360 Georgopoulos, A P., Schwartz, A B., & Kettner, R E (1986) Neuronal population coding of movement direction Science, 233 (4771), 1416 – 1419 Gers, F A., Schraudolph, N N., & Schmidthuber, J (2002) Learning precise timing with LSTM recurrent networks Journal of Machine Learning Research, 3, 115 - 143 Gerstner, W (1995) Time structure of the activity in neural network models Physical Review E, 51, 738 – 758 Gerstner, W., & Kistler, W (2002) Spiking neuron models: single neurons, pupulations, plasticity Cambridge, MA: Cambridge University Press Gerstner, W., & van Hemmen, L J (1992) Associative memory in a network of spiking neurons Network: Computation in Neural Systems, 3, 139 – 164 136 Bibliography Gollisch, T., & Meister, M (2008) Rapid neural coding in the retina with relative spike latencies Science, 319, 1108 – 1111 Greschner, M., Thiel, A., Kretzberg, J., & Ammermăller, J (2006) Complex u spike event pattern of transient ON/OFF retinal ganglion cells Journal of Neurophysiology, 96, 2845 – 2856 Gurden, H., Takita, M., & Jay, T M (2000) Essential role of D1 but not D2 receptors in the NMDA receptor-dependent longterm potentiation at hippocampal-prefrontal cortex synapses in vivo The Journal of Neuroscience, 20 :RC106 Gătig, R., & Sompolinsky, H (2006) The tempotron: a neuron that learns spike u timing-based decisions Nature Neuroscience, (3), 420 – 428 Haruhiko, T., Masaru, F., Hiroharu, F., Shinji, T., Hidehiko, K., & Terumine, H (2009) Obstacle to training SpikeProp networks: cause of surges in training process IEEE IJCNN 2009, 1225 – 1229 Hasselmo, M E., Bodel´n, C., & Wyble, B P (2002) A proposed function for o Hippocampal theta rhythm: separate phases of encoding and retrieval enhance reversal of prior learning Neural Computation, 14 (4), 793 – 817 Hawkins, J., & Blakeslee, S (2004) On intelligence New York: Henry Holt and Company Haykin, S (1998) Neural Networks: A Comprehensive Foundation, (2nd ed.) Prentice Hall He, H (2011) Self-Adaptive Systems for Machine Intelligence Wiley Hebb, D O (1949) The organization of behavior Wiley Heiligenberg, W F (1991) Neural Nets in Electic Fish (pp 51-60) MIT Press Heil, P (1997) Auditory cortical onset responses revisited I First spike timing Journal of Neurophysiology, 77 (5), 2616 – 2641 Hinton, G., & Sejnowski, T J (1999) Unsupervised Learning: Foundations of Neural Computation MIT Press Hopfield, J J (1995) Pattern recognition computation using action potential timing for stimulus representation Nature, 376, 33 – 36 Hopfield, J J., & Brody, C D (2000) What is a moment? “Cortical” sensory integration over a brief interval PNAS, 97, 13919 – 13924 Hopfield, J J., & Brody, C D (2001) What is a moment? Transient synchrony as a collective mechanism for spatiotemporal integration PNAS, 98, 1282 – 1287 Hu, J., Tang, H., Tan, K C., Li, H & Shi, L (2013) A spike-timing based integrated model for pattern recognition Neural Computation, 25 (2), 450 – 472 137 Bibliography Hughes, J R (2008) Gamma, fast, and ultrafast waves of the brain: Their relationships with epilepsy and behavior Epilepsy & Behavior, 13 (1), 25 – 31 Ito, M (2000) Mechanisms of motor learning in the cerebellum Brain Research, 886, 237 – 245 Ito, M (2008) Control of mental activities by internal models in the cerebellum Nature Reviews Neuroscience, 9, 304 – 313 Izhikevich, E M (2004) Which model to use for cortical spiking neurons? IEEE Transaction on Neural Networks, 15 (5), 1063 – 1070 Jazayeri, M, & Movshon, J (2006) Optimal representation of sensory information by neural populations Nature Neuroscience, (5), 690 – 696 Johansson, R S & Birznieks, I (2004) First spikes in ensembles of human tactile afferents code complex spatial fingertip events Nature Neuroscience, 7, 170 – 177 Jensen, M S., Azouz, R., & Yaari, Y (1996) Spike after-depolarization and burst generation in adult rat hippocampal CA1 pyramidal cells The Journal of Physiology, 492, 199 – 210 Jensen, O., Idiart, M A., & Lisman, J E (1996) Physiologically realistic formation of autoassociative memory in networks with theta/gamma oscillations: Role of fast NMDA channels Learn & Memory, 3, 243 – 256 Jensen, O., & Lisman, J E (1996a) Novel lists of +/- known items can be reliably stored in an oscillatory short-term memory network: interaction with long-term memory Learn & Memory, 3, 257 – 263 Jensen, O., & Lisman, J E (1996b) Theta/gamma networks with slow NMDA channels learn sequences and encode episodic memory:Role of NMDA channels in recall Learn & Memory, 3, 264 – 278 Jensen, O., & Lisman, J E (1996c) Hippocampal CA3 region predicts memory sequences: accounting for the phase precession of place cells Learn & Memory, 3, 279 – 287 Jensen, O (2001) Information transfer between rhythmically coupled networks: reading the hippocampal phase code Neural Computation, 13, 2743 – 2761 Jensen, O., & Lisman, J E (2005) Hippocampal sequence-encoding driven by a cortical multi-item working memory buffer Trends Neuroscience, 28 (2), 67 – 72 Kami´ski, J., Brzezicka, A., & Wr´bel, A (2011) Short-term memory capacn o ity (7±2) predicted by theta to gamma cycle length ratio Neurobiology of Learning and Memory, 95 (1), 19 –23 Kasi´ski, A., & Ponulak, F (2006) Comparison of supervised learning methods n for spike time coding in spiking neural networks International Journal of Applied Mathematics and Computer Science, 16 (1), 101 – 113 138 Bibliography Kayser, C., Montemurro, M A., Logothetis, N K., & Panzeri, S (2009) Spikephase coding boosts and stabilizes information carried by spatial and temporal spike patterns Neuron, 61, 597 – 608 Keat, J (2001) Predicting every spike: a model for the responses of visual neurons Neuron, 30, 803 – 817 Kiani, R., Esteky, H., Mirpour, K., & Tanaka, K (2007) Object category structure in response patterns of neuronal population in monkey inferior temporal cortex Journal of Neurophysiology, 97 (6), 4296 – 4309 Kirson, E D., & Yaari, Y (1996) Synaptic NMDA receptors in developing mouse hippocampal neurones: functional properties and sensitivity to ifenprodil The Journal of Physiology, 497 (Pt 2), 437 – 455 Klampfl, S., Legenstein, R., & Maass, W (2009) Spiking neurons can learn to solve information bottleneck problems and to extract independent components Neural Computation, 21, 911 – 959 Klin’shov, V V., & Nekorkin, V I (2005) Model of a neuron with afterdepolarization and short-term memory Radiophysics and Quantum Electronics, 48 (3), 203 – 211 Knerr, S., Personnaz, L., & Dreyfus, G (1992) Handwritten digit recognition by neural networks with single-layer training IEEE Transaction on Neural Networks, 3, 962 – 968 Knudsen, E I (1994) Supervised learning in the brain Journal of Neuroscience, 14 (7), 3985 – 3997 Koepsell, K., Wang, X., Vaingankar, V., Wei, Y., Wang, Q., Rathbun, D L., Usrey, W M., Hirsch, J., & Sommer, F T (2009) Retinal oscillations carry visual information to cortex Frontiers in Systems Neuroscience, (4) Krichmar, J., Seth, A., Nitz, D., Fleischer, J., & Edelman, G (2005) Spatial navigation and causal analysis in a brain-based device modeling corticalhippocampal interactions Neuroinformatics, (3), 197 – 221 Krăger, J., & Aiple, F (1988) Multimicroelectrode investigation of monkey u striate cortex: spike train correlations in the infragranular layers Journal of Neurophysiology, 60 (2), 798 – 828 Kunec, S., Hasselmo, M E., & Kopell, N (2005) Encoding and retrieval in the CA3 region of the hippocampus: a model of theta-phase separation Journal of Neurophysiology, 94 (1), 70 – 82 Landis, F., Ott, T., & Stoop, R (2010) Hebbian self-organizing integrate-andfire networks for data clustering Neural Computation, 22, 273 – 288 Lavenex, P., & Amaral, D G (2000) Hippocampal-neocortical interaction: A hierarchy of associativity Hippocampus, 10 (4), 420 – 430 139 Bibliography Lee, C., Rohrer, W H., & Sparks, D L (1988) Population coding of saccadic eye movements by neurons in the superior colliculus Nature, 332, 357 – 360 Lega, B C., Jacobs, J., & Kahana, M (2012) Human hippocampal theta oscillations and the formation of episodic memories Hippocampus, 22 (3), 748 – 761 Legenstein, R., Naeger, C., & Maass, W (2005) What can a neuron learn with spike-timing-dependent plasticity? Neural Computation, 17, 2337 – 2382 Legenstein, R., Pecevski, D., & Maass, W (2008) A learning theory for rewardmodulated spike-timing-dependent plasticity with application to biofeedback PLoS Computational Biology, 4, 127 Leibold, C., Gundlfinger, A., Schmidt, R., Thurley, K., Schmitz, D., & Kempter, R (2008) Temporal compression mediated by short-term synaptic plasticity PNAS, 105 (11), 4417 – 4422 Leshno, M., & Spector, Y (1996) Neural network prediction analysis: The bankruptcy case Neurocomputing, 10 (2), 125 – 147 Leutgeb, S., Leutgeb, J K., Moser, M B., & Moser, E I (2005) Place cells, spatial maps and the population code for memory Current Opinion in Neurobiology, 15 (6), 738 – 746 Lin, L., Osan, R., Shoham, S., Jin, W., Zuo, W., & Tsien, J Z (2005) Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus PNAS, 102 (17), 6125 – 6130 Lin, L., Osan, R., & Tsien, J Z (2006) Organizing principles of real-time memory encoding: neural clique assemblies and universal neural codes Trends in Neurosciences, 29 (1), 48 – 57 Lisman, J E., & Idiart, M A (1995) Storage of +/- short-term memories in oscillatory subcycles Science, 267 (5203), 1512 – 1515 Ljungberg, T., Apicella, P., & Schultz, W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions Journal of Neurophysiology, 67(1), 145 – 163 Llinas, R R., Grace, A A., & Yarom, Y (1991) In vitro neurons in mammalian cortical layer exhibit intrinsic oscillatory activity in the 10- to 50-Hz frequency range PNAS, 88 (3), 897 – 901 Lippmann, R P (1989) Review of Neural Networks for Speech Recognition Neural Computation, (1), – 38 Litvak, V., Sompolinsky, H., Segev, I., & Abeles, M (2003) On the transmission of rate code in long feed-forward networks with excitatory-inhibitory balance Journal of Neuroscience, 23, 3006 – 3015 Maass, W (1997) Networks of spiking neurons: The third generation of neural network models Neural Networks, 10 (9), 1659 – 1671 140 Bibliography Maass, W., & Bishop, C M (1998) Pulsed neural networks MIT Press Maass, W., Natschlaeger, T., & Markram, H (2002) Real-time computing without stable states: A new framework for neural computation based on perturbations Neural Computation, 14 (11), 2531 – 2560 Mainen, Z F., & Sejnowski, T J (1995) Reliability of spike timing in neocortical neurons Science, 268, 1503 – 1506 Malenka, R C., & Bear, M F (2004) LTP and LTD: an embarrassment of riches Neuron, 44 (1), – 21 Manette, O F., & Maier M A (2004) Temporal processing in primate motor control: relation between cortical and EMG activity IEEE Transaction on Neural Networks, 15 (5), 1260 – 1267 Markram, H., Helm, P J., & Sakmann, B (1995) Dendritic calcium transients evoked by single back-propagating action potentials in rat neocortical pyramidal neurons The Journal of Physiology, 485 (1), – 20 Martin, G L., & Pitman, J A (1991) Recognizing hand-printed letter and digits using backpropagation learning Neural Computation, (2), 258 – 267 Masquelier, T., & Thorpe, S J (2007) Unsupervised learning of visual features through spike timing dependent plasticity PLoS Computational Biology, (2), e31 Masquelier, T., Guyonneau, R., & Thorpe, S J (2009) Competitive STDPbased spike pattern learning Neural Computation, 21 (5), 1259 – 1276 McCulloch, Warren, & Pitts, Walter (1943) A logical calculus of the ideas immanent in nervous activity The bulletin of mathematical biophysics, (4), 115 – 133 McKennoch, S., Voegtlin, T., & Bushnell, L G (2009) Spike-timing error backprop in theta neuron networks Neural Computation, 21 (1), – 45 Mehta, M R., Lee, A K., & Wilson, M A (2002) Role of experience and oscillations in transforming a rate code into a temporal code Nature, 417, 741 – 746 Meister, M., Lagnado, L., & Baylor, D A (1995) Concerted signaling by retinal ganglion cells Science, 270, 1207 – 1210 Meister, M., & Berry, M J (1999) The Neural code of the retina Neuron, 22, 435 – 450 Meng, Y., Zhang, Y., & Jin, Y (2011) Autonomous self-reconfiguration of modular robots by evolving a hierarchical mechanochemical model IEEE Computational Intelligence Magazine, (1), 43 – 54 Minsky, M & Papert, S (1969) Perceptrons: An introduction to computational geometry MIT Press 141 Bibliography Mongillo, G., Barak, O., & Tsodyks, M (2008) Synaptic theory of working memory Science, 319 (5869), 1543 – 1546 Montgomery, J., Carton, G., & Bodznick, D (2002) Error-driven motor learning in fish Biological Bulletin, 203 (2), 238 – 239 Monyer, H., Burnashev, N., Laurie, D J., Sakmann, B., & Seeburg, P H (1994) Developmental and regional expression in the rat brain and functional properties of four NMDA receptors Neuron, 12 (3), 529 – 540 Moser, E T., Kropff, E., & Moser, M B (2008) Place cells, grid cells, and the brain’s spatial representation system Annual Review of Neuroscience, 31, 69 89 Măler-Putz, G R., Scherer R., Pfurtscheller G., & Neuper, C (2010) Tempou ral coding of brain patterns for direct limb control in humans Frontiers in Neuroscience, (34) Nadasdy, Z (2009) Information encoding & reconstruction from the phase of action potentials Frontiers in Systems Neuroscience, (6) Nicolelis, M A., Baccala, L A., Lin, R C., & Chapin, J K (1995) Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system Science, 268 (5215), 1353 – 1358 O’Keefe, J., & Burgess, N (2005) Dual phase and rate coding in hippocampal place cells: theoretical significance and relationship to entorhinal grid cells Hippocampus, 15, 853 – 866 O’Keefe, J., & Dostrovsky, J (1971) The hippocampus as a spatial map: Preliminary evidence from unit activity in the freely-moving rat Brain Research, 34, 171 – 175 Otani, S., Blond, O., Desce, J M., & Crepel, F (1998) Dopamine facilitates long-term depression of glutamatergic transmission in rat prefrontal cortex Neuroscience, 85, 669 – 676 Otmakhova, N A., & Lisman, J E (1996) D1/D5 dopamine receptor activation increases the magnitude of early long-term potentiation at CA1 hippocampal synapses The Journal of Neuroscience, 16, 7478 – 7486 Ozawa, S., Kamiya, H., & Tsuzuki, K (1998) Glutamate receptors in the mammalian central nervous system Progress in Neurobiology, 54 (5), 581 – 618 Pan, X., & Tsukada, M (2006) A model of the hippocampal-cortical memory system Biological Cybernetics, 95 (2), 159 – 167 Panzeri, S., Brunel N., Logothetis, N K., & Kayser, C (2010) Sensory neural codes using multiplexed temporal scales Trends in Neurosciences, 33 (3), 111 – 120 142 Bibliography Perez-Orive, J., Mazor, O., Turner, G C., Cassenaer, S., Wilson, R I., & Laurent, G (2002) Oscillations and sparsening of odor representations in the mushroom body Science, 297, 359 – 365 Perlovsky, L (2011) Computational intelligence applications for defense IEEE Computational Intelligence Magazine, (1), 20 – 29 Perrett, D.I., Rolls, E.T., & Caan, W (1982) Visual neurones responsive to faces in the monkey temporal cortex Experimental Brain Research, 47 (3), 329 – 342 Pfister, J P., Toyoizumi, T., Barber, D., & Gerstner W (2006) Optimal spiketiming dependent plasticity for precise action potential firing Neural Computation, 18, 1318 – 1348 Ponulak, F., & Kasinski, A (2010) Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting Neural Computation, 22 (2), 467 – 510 Poucet, B., & Save, E (2005) Attractors in memory Science, 308 (3), 799 – 800 Quiroga, R Q., Reddy L., Kreiman G., Koch C., & Fried I (2005) Invariant visual representation by single neurons in the human brain Nature, 435, 1102 – 1107 Reich, D S., Mechler, F., & Victor, J D (2001) Temporal coding of contrast in primary visual cortex: when, what, and why? Journal of Neurophysiology, 85, 1039 – 1050 Riesenhuber, M & Poggio, T (1999) Hierarchical Models of Object Recognition in Cortex Nature Neuroscience, 2, 1019 – 1025 Rolls, E T., & Kesner, R P (2006) A computational theory of hippocampal function, and empirical tests of the theory Progress in Neurobiology, 79 (1), – 48 Rolls, E T (2008) Computational models of hippocampal functions In J H Byrne (Eds.), Learning and Memory: A Comprehensive Reference, 641 – 665 Academic Press Rolls, E T (2010) A computational theory of episodic memory formation in the hippocampus Behavioural Brain Research, 215 (2), 180 – 196 Ros, E., Carrillo, R., Ortigosa, E M., Barbour, B., & Ag´ R (2006) Eventıs, driven simulation scheme for spiking neural networks using lookup tables to characterize neuronal dynamics Neural Computation, 18, 2959 – 2993 Rosenblatt, F (1958) The perceptron: a probabilistic model for information storage and organization in the brain Psychological Review, 65, 386 – 408 Ruf, B., & Schmitt, M (1998) Self-organization of spiking neurons using action potential timing IEEE Transaction on Neural Networks, 9, 575 – 578 143 Bibliography Russell, B C., Torralba, A., Murphy, K P., & Freeman, W P (2008) Labelme: a database and web-based tool for image annotation International Journal of Computer Vision, 77, 157 – 173 Samonds, J M., Zhou, Z.,Bernard, M R., & Bonds, A B (2006) Synchronous activity in cat visual cortex encodes collinear and cocircular contours Journal of Neurophysiology, 95 (4), 2602 – 2616 Sato, N., & Yamaguchi, Y (2005) Online formation of a hierarchical cognitive map for object-place association by theta phase coding Hippocampus, 15 (7), 963 – 978 Sato, N., & Yamaguchi, Y (2009) Spatial-area selective retrieval of multiple object-place associations in a hierarchical cognitive map formed by theta phase coding Cognitive Neurodynamics, (2), 131 – 140 Savin, C., Joshi, P., & Triesch, J (2010) Independent component analysis in spiking neurons PLoS Computational Biology, 6, 533 – 536 Schrader, S., Gewaltig, MO., Kărner U., & Kărner E (2009) Cortext: A columo o nar model of bottom-up and top-down processing in the neocortex Neural Networks, 22 (8), 1055 – 1070 Schreiber, S., Fellous, J., Whitmer, D., Tiesinga, P., & Sejnowski, T (2003) A new correlation-based measure of spike timing reliability Neurocomputing, 52-54, 925 – 931 Schultz, W (2002) Getting formal with dopamine and reward Neuron, 36, 241 – 263 Seung, H S (2003) Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission Neuron, 40 (6), 1063 – 1073 Shadlen, M N., & Newsome, W T (1994) Noise, neural codes and cortical organization Current Opinion in Neurobiology, 4, 569 – 579 Shimizu, E., Tang, Y P., Rampon, C., & Tsien, J Z (2000) NMDA receptordependent synaptic reinforcement as a crucial process for memory consolidation Science, 290 (5494), 1170 – 1174 Sigala N., & Logothetis N K (2002) Visual categorization shapes feature selectivity in the primate temporal cortex Nature, 415, 318 – 320 Singer, W., & Gray, C M (1995) Visual feature integration and the temporal correlation hypothesis Annual Review of Neuroscience, 18, 555 586 Sjăstrăm, P G., & Nelson, S B (2002) Spike timing, calcium signals and o o synaptic plasticity Current Opinion in Neurobiology, 12 (3), 305 – 314 Softky, W R (1995) Simple codes versus efficient codes Current Opinion in Neurobiology, 5, 239 – 247 Sommer, F T., & Wennekers, T (2001) Associative memory in networks of spiking neurons Neural Networks, 14 (6-7), 825 834 144 Bibliography Sporea, I., & Grăning, A (2013) Supervised Learning in Multilayer Spiking u Neural Networks Neural Computation, 25 (2), 473 – 509 Starzyk, J A., & He, H (2009) Spatio-Temporal Memories for Machine Learning: A Long Term Memory Organization IEEE Transactions on Neural Networks, 20 (5), 768 – 780 Tang, H., Tan, K C., & Teoh, E J (2006) Dynamics analysis and analog associative memory of networks with LT neurons IEEE Transactions on Neural Networks, 17 (2), 409 – 418 Tang, H., Li, H., & Yan, R (2010) Memory dynamics in attractor networks with saliency weights Neural Computation, 22, 1899 – 1926 Thach, W T (1996) On the specific role of the cerebellum in motor learning and cognition: Clues from PET activation and lesion studies in man Behavioral and Brain Sciences, 19, 411 – 431 Thorpe, S J., & Imbert, M (1989) Biological constraints on connectionist modelling Connectionism in Perspective, Elsevier Treves, A., & Rolls, E (1994) Computational analysis of the role of the hippocampus in memory Hippocampus, 4, 374 – 391 Tsodyks, M., & Sejnowski, T (1995) Associative memory and hippocampal place cells International Journal of Neural Systems, 6, 81 – 86 Tsodyks, M V., Skaggs, W E., Sejnowski, T J., & Mcnaughton, B L (1996) Population dynamics and theta rhythm phase precession of hippocampal place cell firing: a spiking neuron model Hippocampus, 6, 271 – 280 Usrey, W., & Reid, R (1999) Synchronous activity in the visual system Annual Review of Physiology, 61, 435 – 456 VanRullen, R., & Thorpe, S J (2001) Rate-coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex Neural Computation, 13, 1255 – 1283 van Wyk, M., Taylor, W R., & Vaney, D I (2006) Local edge detectors: a substrate for fine spatial vision at low temporal frequencies in rabbit retina The Journal of Neuroscience, 26 (51), 13250 – 13263 Victor, J D (2000) How the brain uses time to represent and process visual information Brain Research, 886, 33 – 46 Wagatsuma, H., & Yamaguchi, Y (2007) Neural dynamics of the cognitive map in the hippocampus Cognitive Neurodynamics, (2), 119 – 141 Werbos, P (1974) Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences PhD thesis, Harvard University Widrow, B., & Hoff, M E (1960) Adaptive switching circuits IRE WESCON, part 4, 96 – 104 145 Bibliography Wills, T J., Lever, C., Cacucci, F., Burgess, N., & O’Keefe, J (2005) Attractor dynamics in the hippocampal representation of the local environment Science, 308 (3), 873 876 Wyss, R., Kănig, P., & Verschure, P (2003) Invariant representations of visual o patterns in a temporal population code PNAS, 100 (1), 324 – 329 Yan, R., Tee, K P., Chua, Y., Li, H., & Tang, H (2012) Gesture recognition based on localist attractor networks with application to robot control IEEE Computational Intelligence Magazine, (1), 64 – 74 Vanderwolf, C (1969) Hippocampal electrical activity and voluntary movement in the rat Electroencephalography and Clinical Neurophysiology, 26 (4), 407 – 418 VanRullen, R., Guyonneau, R., & Thorpe, S J (2001) Spike times make sense Trends in Neurosciences, 28 (1), – Vasilaki, E., Fremaux, N., Urbanczik, R., Senn, W., & Gerstner, W (2009) Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail PLoS Computational Biology, : e1000586 Veredas, F J., Mesa, H., & Martinez, L A (2008) Imprecise correlated activity in self-organizing maps of spiking neurons Neural Networks, 6, 810 – 816 Widrow, B., Rumelhard, D E., & Lehr, M A (1994) Neural networks: applications in industry, business and science Communications of the ACM, 37, 93 – 105 Wiltgen, B J., Brown, R A., Talton, L E., & Silva, A J (2004) New circuits for old memories: the role of the neocortex in consolidation Neuron, 44, 101 – 108 Yamaguchi, Y., Sato, N., Wagatsuma, H., Wu, Z., Molter, C., & Aota, Y (2007) A unified view of theta-phase coding in the entorhinal-hippocampal system Current Opinion in Neurobiology, 17 (2), 197 – 204 Zhang, G., Hu, M Y., Patuwo, E B., & Indro D (1999) Artificial neural networks in bankruptcy prediction: general framework and cross-validation analysis European Journal of Operational Research, 116 (1), 16 – 32 146 Appendix: Author’s Publications Journal Papers Jun Hu, Huajin Tang, K C Tan and Haizhou Li A hierarchical organized memory model with spike-driven learning of visual features In preparation Jun Hu, Huajin Tang, K C Tan and Haizhou Li A hierarchical organized memory model with temporal population codes Submitted Jun Hu, Huajin Tang, K C Tan, Haizhou Li and Luping Shi A spike-timing based integrated model for pattern recognition Neural Computation, 25 (2), 450 – 472, 2013 Conference Papers Jun Hu, Huajin Tang and K C Tan A spiking neural network model for associative memory using temporal codes The 18th Asia Pacific Symposium on Intelligent and Evolutionary Systems (IES), Singapore, November 10-12, 2014 Accepted 147 Appendix Jun Hu, Huajin Tang and K C Tan A hierarchical organized memory model using spiking neurons IEEE International Joint Conference on Neural Networks (IJCNN), Dallas, US, August 4-9, 2013 C H Tan, Huajin Tang, E Y Cheu and Jun Hu A computationally efficient associative memory model of hippocampus CA3 by spiking neurons IEEE International Joint Conference on Neural Networks (IJCNN), Dallas, US, August 4-9, 2013 Jun Hu, Huajin Tang and K C Tan Spiking-timing based pattern recognition with real-world visual stimuli IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB), Singapore, April 16-19, 2013 C H Tan, E Y Cheu, Jun Hu, Qiang Yu and Huajin Tang Associative memory model of hippocampus CA3 using spike response neurons 18th International Conference on Neural Information Processing (ICONIP), Shanghai, China, November 14-17, 2011 148 ... to develop learning and memory models using spiking neural networks in solving cognitive tasks We focus on memory models using spiking neural networks Traditional neural networks and other AI... existing theories and developing innovative cognitive learning and memory models using spiking neural networks 1.1 1.1.1 Background and Basic Concepts Cognitive Learning and Memory in the Brain... artificial memory systems In the following sections, encoding approaches, learning algorithms and memory models in spiking neural networks will be reviewed successively 2.2.1 Neural Coding in Spiking Neural