Computing with simulated and cultured neuronal networks

COMPUTING WITH SIMULATED AND CULTURED NEURONAL NETWORKS JU HAN NATIONAL UNIVERSITY OF SINGAPORE 2013 COMPUTING WITH SIMULATED AND CULTURED NEURONAL NETWORKS JU HAN (B. Eng. (Hons.), NUS) A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY NUS GRADUATE SCHOOL FOR INTEGRATIVE SCIENCE AND ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2013 Declaration I hereby declare that this thesis is my original work and it has been written by me in its entirety. I have duly acknowledged all the sources of information which have been used in the thesis. This thesis has also not been submitted for any degree in any university previously. Nov 2013 Ju Han Name Signature Date Acknowledgements When I looked at forums in the internet, lots of PhD students complained how miserable they felt about their projects. Now at the end of my four years’ time as a PhD student, looking back to what I have experienced, I would say, everyone’s life is full of both misery and joy. Whether it is really miserable or joyful depends on how you perceive it and whom you have met. I would like to take this opportunity to thank those who gave me the joys during my hard times. The first person is my supervisor, Professor Antonius VanDongen. I remember at my second year when I was pessimistic about my results and future works, discussing with him about those unexpected negative results, he encouraged me with tons of new ideas. “No need to be depressed.” Though it is a very simple sentence from him, you can imagine what a desperate PhD student feels when this sentence is spoken out from his supervisor who just advised many promising ideas that can potentially reverse the negative results to positive. Indeed, encouraging students by plenty of ideas is very effective and pragmatic. He always puts student’s work to high priority. Nearly every time when I walk to his door with a handful of paper, he will stop his work, smile, and turn around to discuss the results – even though sometimes he did not have lunch yet. For every manuscript I sent to him, he edited it sentence by sentence correcting all the grammar errors, and taught me how to write proper English. Thanks for his encouragement, valuable supervision, and great patience. I learnt a lot from him, not only research, but also other skills like how to clearly express a complicated thought, but the most important thing learnt from him is how to be an effective supervisor. I will carry all the things i learnt from him for my whole life, and brand these as “learnt from Tony.” Moreover, I would like to express my gratitude to his wife, Margon. She is very considerate to all the lab members in both research and life, making us feel the lab like home. Sometimes she shared her own experiences with me when I was confused about what to choose for the next step in life. As what I wrote in the Christmas card: You are the best principal investigator and princess investigator (this is the words on Margon’s office door) I have ever seen! I would also like to thank Professor Xu Jianxin for his supervision on my PhD work. Actually, I met him in 2nd year of my undergraduate study, and he supervised me doing an undergraduate research project on neural networks. This experience enlightened my interests in neural network research. Then Tony and he supervised me together for my PhD study. Although my PhD direction is not his main research field, he still discussed with me in all details of my work and gave valuable suggestions on how to improve results. I was greatly touched when I realized he was in a condition that was really not convenient to work, yet he had already read my manuscript. I was deeply moved when he said sorry to me, face to face, telling me the reason why he was not able to closely supervise me in the past year. I was truly touched when he smiled to me before I left his office, saying “We will continue the discussion next time…” This is his attitude to students, earning all my respects. I will never forget his efforts and time spent on my work, as well as all his characteristics as a responsible supervisor. It was also a pleasure to work with all the people in the lab. Thank Dr. Mark Dranias, for teaching me experimental skills and giving a hand ii whenever I needed. Without him, some of my work in this thesis was impossible to finish. Besides, we had lots of chats on diverse topics, including daily news, financial markets, and many others. These made me feel temporarily and mentally out of the boring lab. Thanks to Nicodemus Oey, for helping me on the rehabilitation project. He is a very kind and helpful person, except when he asked “How is the rehabilitation game?”, because nearly every time when he asked me this, I had no idea how to answer as I did nothing on improving that game. However, if he did not ask, I may not have that game in the current shape at all. Then I realized that I am, in fact, a stress-driven person. Thank Nico for chasing me up about the project! I also would like to thank Edmund Chong for those long-time and valuable discussions on the projects. Without him, I would not be able to get things done quickly in the first year of my PhD study. Thanks to Leung How Wing, Rajaram Ezhilarasan, and Niamh Higgins for helping me on the neuronal cultures, and Annabel Tan Xulin, Gokulakrishna Banumurthy, as well as everyone else who helped me during the four years’ time! Ju Han (莒涵) June 27, 2013. iii Table of Contents Summary . ix List of Figures . xi List of Abbreviations . xiv Chapter I Introduction . 1.1 Background 1.2 The LSM Architecture 1.3 Neural networks in silico 1.4 Neural networks in vitro . 1.5 Dissertation Overview 11 Chapter II Application – Music Classification 51 2.1 Introduction 52 2.2 Experiment Setup . 54 2.2.1 Architectural Design of the LSM . 55 2.2.2 Dataset and Spike Encoding . 57 2.2.3 Music and Non-Music Classification . 59 2.2.4 Music Style Classification 62 2.3 Results and Discussion . 64 2.3.1 Music and Non-music Classification 64 2.3.2 Music Style Classification 67 2.4 Comparison with Other Methods . 69 2.5 Conclusions 70 Chapter III Effects of Synaptic Connectivity on LSM Performance 14 3.1 Introduction 14 3.2 Simulation Models . 18 3.2.1 Neuron Model . 18 iv 3.2.2 Dynamic Synapse . 19 3.2.3 Readout . 20 3.3 Connection Topologies . 21 3.3.1 Original Connection Topology . 21 3.3.2 Small World Network . 22 3.3.3 Axon Model 24 3.4 Methods 26 3.4.1 Classification Tasks Description . 26 3.4.2 Separation, Generalization, and the Kernel Quality . 27 3.5 Results and Discussion . 29 3.5.1 Regions with Satisfactory Performance . 32 3.5.2 Regions with Optimal Performance . 34 3.5.3 Music and Non-music Classification 36 3.6 Evolving Synaptic Connectivity . 38 3.6.1 Network Settings 40 3.6.2 Parameter Settings 41 3.6.3 Create/Delete Synaptic Connections 42 3.6.4 Fitness Calculation . 43 3.6.5 Simulation Results 45 3.7 Conclusions 49 Chapter IV Computing with Cultured Cortical Networks: Implementing the Liquid State Machine with Dissociated Neuronal Cultures . 51 4.1 Introduction 73 4.2 Methods 75 4.2.1 Cell Culture and Optogenetic Transfection 75 4.2.2 Recording System . 75 4.2.3 Stimulation System . 76 v 4.2.4 Stimuli Protocol 77 4.2.5 Decoding Responses . 80 4.2.6 Control and simulation setup 80 4.2.7 Metric for selecting best MEA channels . 81 4.2.8 Simulation Parameters 82 4.3 Results 83 4.3.1 Classification of Jittered Spike Train Templates 83 4.3.2 Classification of Random Music 88 4.3.3 Separation Property 90 4.3.4 Real-time State Dependent Computation: Switches . 91 4.3.5 Improving Performance through Drug Treatment 92 4.3.6 Short-term Synaptic Plasticity is Necessary for the Memory 94 4.4 Discussion 95 Chapter V Intrinsic Classification Properties of Spiking Neural Networks 100 5.1 Introduction 101 5.2 Approach 103 5.2.1 Readout schemes 103 5.2.2 Stimulus Protocols 105 5.3 5.3.1 Results 107 Discriminative Information in Membrane Potential Traces 108 5.3.2 Discriminative Information in Spike Response 109 5.3.3 Identification of the Discriminative Neurons . 115 5.4 Discussion and Conclusion 117 5.5 Models used in simulations and methods . 119 5.5.1 Simulation of the cortical circuits . 119 vi 5.5.2 Model Multiple-timescale Adaptive Threshold (MAT) Neuron 120 5.5.3 Dynamic Spiking Synapse 121 5.5.4 STDP Synapse 121 5.5.5 Reward Modulated STDP synapses 122 5.5.6 Linear Discriminant Analysis on Membrane Potential Traces 124 5.5.7 Spike Train Distance . 125 5.5.8 Mutual Information . 126 Chapter VI Reinforcement Learning in Neural Networks 128 6.1 Spiking neural networks in open-loop and closed-loop systems 128 6.2 Introduction to Neurorehabilitation 130 6. The Proposed Model for Motor Rehabilitation 133 6.3.1 Hebbian Learning and the Strength of Anatomical Pathway 133 6.3.2 External Guidance and Somatosensory Response in M1 . 134 6.3.3 Pharmacotherapies and Modulatory Signals 135 6.3.4 Neural Plasticity . 137 6. Simulation 138 6.5 Results 140 6.5.1 Before Focal Lesions 140 6.5.2 After Focal Lesions 141 6.5.3 Behavioral Outcomes . 143 6.6 Discussion 144 6.7 The Proposed Treatment . 146 6.8 Moving to Clinical Trials . 146 Chapter VII Conclusion and Future Work 149 vii predators and preys. In the perspective of pattern recognition theory, familiarity/novelty detector can be thought as binary classification, in which the class of familiar objects is finite while the novelty class is infinite. In the perspective of memory, familiar objects are stored in the memory, and whenever a stimulus is presented, the brain can quickly tell whether the stimulus was in the memory or not. Output can be in various forms such as human language spelling out the word “familiar” or “novel”, or an action of pushing down a button, or even increased activity in networks. It has been shown that there are Novelty Neurons in the perirhinal cortex (Brown and Xiang, 1998). Various neural network models of familiarity detection have been proposed and verified through simulations. The Hopfield network is a famous memory model that can store and recall information. A familiarity detector was designed based on this model (Bogacz et al., 2001). Other models based on the mechanism of Hebbian learning were developed (Bogacz et al., 2001; Yakovlev et al., 2008), in which high network activity is observed when a familiar stimulus is presented. However, they all implemented in the analog neural networks regime and spiking neural network models have not been widely studied yet. Any network that implements familiarity detectors must have long-term memory to store information. STDP mechanism has been widely observed in in vitro neural circuits (Bi and Poo, 1998; Sjostrom et al., 2001; Froemke and Dan, 2002; Cassenaer and Laurent, 2007). Moreover, classifiers using STDP learning have also been proposed (Legenstein et al., 2008). However, using STDP to construct a familiarity detector with spiking neural networks has yet been explored. It is certainly feasible to design a cortical microcircuit model 154 that employs STDP mechanism to detect familiar and novel stimulus. STDP requires hundreds of repetitive stimulus presentations in order to obtain expected network behavior. This results in a very slow learning rate. In order to speed up learning, some other models, e.g. NMDA receptor model, can be implemented in simulations to allow neural microcircuits memorizing input patterns. I have implemented NMDA receptor models (Shouval et al., 2002) into synapses, and preliminary results have been obtained by an undergraduate Final Year Project student after proper parameter tuning. Neural microcircuits with NMDARs are able to detect face characteristics, e.g. beards. Higher activity is observed when face images with beard were presented to the network compared to other faces without beards. Simulations are limited by computer hardware and knowledge of neuron models, but neuron culture has more than what we need. If the simulation results are able to demonstrate satisfactory performance in familiarity detection using spiking neural networks, we should then move to neuronal network in vitro to verify the model. Some preliminary investigations should be performed beforehand to evaluate long-term memory in cultures. For example, will repetitive presentation of high frequency stimuli (with sufficiently long inter-stimulus interval) that are generated from a rule (e.g. the jittered Poisson spike trains or the directionally moving dots stimuli) yield long-term changes in culture responses? One approach to address this question is to observe trajectories of the MEA data during each stimulus’ presentation (For each trial of stimulus presentation, dimensionality reduction technics can be used, e.g. PCA, to reduce MEA data from 252 to dimensions, such that we can visualize the trajectories of the principle components over time). If the 155 trajectories change along with stimulus presentations in a nonrandom way, or even settle down to an attractor state, we may conclude that the dissociated neuronal cultures have memorized something about the rule of the stimuli. 156 Appendix List of Publications 1. Ju H., Xu J.X., & VanDongen, A. M. J. (2010). Classification of musical styles using liquid state machines. In IEEE International Joint Conference on Neural Networks (IJCNN'2010) (pp. 1-7). 2. Ju H., Xu J.X., Chong E., VanDongen AM (2013) Effects of synaptic connectivity on liquid state machine performance. Neural Networks 38:39-51. 3. Dranias MR, Ju H., Rajaram E, VanDongen AM (2013) Short-term memory in networks of dissociated cortical neurons. J Neurosci 33:1940-1953. 4. Dranias MR, Ju H., VanDongen AM (2013) Computation and Memory in Optogenetic Neuronal Networks. Manuscript submitted. 5. Ju H., Dranias MR, Xu J.X., VanDongen AM (2013) Computing with cultured cortical networks. Manuscript is ready to submit. 6. Ju H., Xu J.X., VanDongen AM (2013) Intrinsic classification properties of spiking neural networks. Manuscript is ready to submit. 7. Ju H., Oey N., Xu J.X., VanDongen AM (2013) A neural network model for motor rehabilitation. Manuscript in preparation. 8. Ma DL, Yoon SI, Yang CH, Marcy G., Zhao N., Leong WY, Ganapathy V., Ju H., VanDongen AM, Hsu KS, Song H., Augustine GJ, and Goh EL (2013) Rescue of MeCP2 dysfunction-induced defects in newborn neurons by modulating GABA signalling, Manuscript submitted. Poster Presentations 1. Ju H., Xu J-X and VanDongen AM (2011) “Neurons in a Spiking Neural Network Model of Cortical Microcircuits Intrinsically Discriminate Input Patterns”, poster presentation in society for neural science 2011 annual meeting, Washington D.C. 2. Dranias MR, Ju H., Rajaram E, VanDongen AM (2011) Spatiotemporal and fading memory properties in an in vitro model of information processing. Society for Neuroscience 2011 annual meeting, Washington D.C. Abstracts 37:239.06. 3. Ju H., Oey N., Xu J-X and VanDongen AM (2012) “A Neural Network Model for Motor Rehabilitation”, Poster presentation in Society for Neuroscience 2012 annual meeting, New Orleans. 4. Leung HW Ju H., Dranias M, Van Dongen AM (2013) “Classification of drugs for neurological disorders using liquid state machine”, poster in Singapore General Hospital Annual Scientific Meeting 2013. 157 References Achard S, Salvador R, Whitcher B, Suckling J, Bullmore E (2006) A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs. J Neurosci 26:63-72. Amaral LA, Scala A, Barthelemy M, Stanley HE (2000) Classes of smallworld networks. Proc Natl Acad Sci U S A 97:11149-11152. Ang KK, Guan C, Chua KS, Ang BT, Kuah C, Wang C, Phua KS, Chin ZY, Zhang H (2009) A clinical study of motor imagery-based braincomputer interface for upper limb robotic rehabilitation. Conf Proc IEEE Eng Med Biol Soc 2009:5981-5984. Ang KK, Guan C, Chua KS, Ang BT, Kuah C, Wang C, Phua KS, Chin ZY, Zhang H (2010) Clinical study of neurorehabilitation in stroke using EEG-based motor imagery brain-computer interface with robotic feedback. Conf Proc IEEE Eng Med Biol Soc 2010:5549-5552. Arabzadeh E, Panzeri S, Diamond ME (2006) Deciphering the spike train of a sensory neuron: counts and temporal patterns in the rat whisker pathway. J Neurosci 26:9216-9226. Arias-Carrion O, Poppel E (2007) Dopamine, learning, and reward-seeking behavior. Acta Neurobiol Exp (Wars) 67:481-488. Baker JL (2003) Is there a support vector machine hiding in the dentate gyrus? Neurocomputing 52-4:199-207. Bakker A, Kirwan CB, Miller M, Stark CE (2008) Pattern separation in the human hippocampal CA3 and dentate gyrus. Science 319:1640-1642. Barbour B, Brunel N, Hakim V, Nadal JP (2007) What can we learn from synaptic weight distributions? Trends Neurosci 30:622-629. Batardiere A, Barone P, Knoblauch K, Giroud P, Berland M, Dumas AM, Kennedy H (2002) Early specification of the hierarchical organization of visual cortical areas in the macaque monkey. Cereb Cortex 12:453465. Beeler JA, Cao ZF, Kheirbek MA, Ding Y, Koranda J, Murakami M, Kang UJ, Zhuang X (2010) Dopamine-dependent motor learning: insight into levodopa's long-duration response. Ann Neurol 67:639-647. Bi GQ, Poo MM (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:10464-10472. Bogacz R, Brown MW, Giraud-Carrier C (2001) Model of familiarity discrimination in the perirhinal cortex. J Comput Neurosci 10:5-23. 158 Bonifazi P, Ruaro ME, Torre V (2005) Statistical properties of information processing in neuronal networks. Eur J Neurosci 22:2953-2964. Braitenberg V (1991) Anatomy of the cortex : statistics and geometry / V. Braitenberg, A. Schuz. Berlin ; New York :: Springer-Verlag. Brown JA, Lutsep HL, Weinand M, Cramer SC (2006) Motor cortex stimulation for the enhancement of recovery from stroke: a prospective, multicenter safety study. Neurosurgery 58:464-473. Brown MW, Xiang JZ (1998) Recognition memory: neuronal substrates of the judgement of prior occurrence. Prog Neurobiol 55:149-189. Brunel N, Hakim V, Isope P, Nadal JP, Barbour B (2004) Optimal information storage and the distribution of synaptic weights: perceptron versus Purkinje cell. Neuron 43:745-757. Buccino G, Solodkin A, Small SL (2006) Functions of the mirror neuron system: implications for neurorehabilitation. Cogn Behav Neurol 19:55-63. Buch ER, Johnen VM, Nelissen N, O'Shea J, Rushworth MF (2011) Noninvasive associative plasticity induction in a corticocortical pathway of the human brain. J Neurosci 31:17669-17679. Buonomano DV, Merzenich MM (1995) Temporal Information Transformed into a Spatial Code by a Neural-Network with Realistic Properties. Science 267:1028-1030. Buonomano DV, Maass W (2009) State-dependent computations: spatiotemporal processing in cortical networks. Nat Rev Neurosci 10:113-125. Cabib S, Puglisi-Allegra S (1994) Opposite responses of mesolimbic dopamine system to controllable and uncontrollable aversive experiences. J Neurosci 14:3333-3340. Caporale N, Dan Y (2008) Spike timing-dependent plasticity: a Hebbian learning rule. Annu Rev Neurosci 31:25-46. Cassenaer S, Laurent G (2007) Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature 448:709-713. Chang C-C, Lin C-J (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2:1-27. Chao ZC, Bakkum DJ, Potter SM (2008) Shaping embodied neural networks for adaptive goal-directed behavior. PLoS Comput Biol 4:e1000042. Cisek P, Kalaska JF (2004) Neural correlates of mental rehearsal in dorsal premotor cortex. Nature 431:993-996. 159 Daly JJ, Wolpaw JR (2008) Brain-computer interfaces in neurological rehabilitation. Lancet Neurol 7:1032-1043. Dan Y, Poo MM (2004) Spike timing-dependent plasticity of neural circuits. Neuron 44:23-30. Dannenberg RB, Thom B, Watson D (1997) A Machine Learning Approach to Musical Style Recognition. Proc International Computer Music Conference:344-347. de Leon PJP, Inesta JM (2007) Pattern recognition approach for music style identification using shallow statistical descriptors. Ieee T Syst Man Cy C 37:248-257. Deshpande LS, Lou JK, Mian A, Blair RE, Sombati S, DeLorenzo RJ (2007) In vitro status epilepticus but not spontaneous recurrent seizures cause cell death in cultured hippocampal neurons. Epilepsy Res 75:171-179. Dinse HR, Ragert P, Pleger B, Schwenkreis P, Tegenthoff M (2003) Pharmacological modulation of perceptual learning and associated cortical reorganization. Science 301:91-94. Dobkin BH (2005) Clinical practice. Rehabilitation after stroke. N Engl J Med 352:1677-1684. Dockendorf KP, Park I, He P, Principe JC, DeMarse TB (2009) Liquid state machines and cultured cortical networks: the separation property. Biosystems 95:90-97. Dong Y, Winstein CJ, Albistegui-DuBois R, Dobkin BH (2007) Evolution of FMRI activation in the perilesional primary motor cortex and cerebellum with rehabilitation training-related motor gains after stroke: a pilot study. Neurorehabil Neural Repair 21:412-428. Dranias MR, Ju H, Rajaram E, VanDongen AM (2013) Short-term memory in networks of dissociated cortical neurons. J Neurosci 33:1940-1953. Eerola T, Toiviainen P (2004) MIDI Toolbox: MATLAB Tools for Music Research. Eliasmith C, Anderson CH (2002) Neural Engineering (Computational Neuroscience Series): Computational, Representation, and Dynamics in Neurobiological Systems: MIT Press. Eliasmith C, Stewart TC, Choo X, Bekolay T, DeWolf T, Tang Y, Rasmussen D (2012) A large-scale model of the functioning brain. Science 338:1202-1205. Ertelt D, Small S, Solodkin A, Dettmers C, McNamara A, Binkofski F, Buccino G (2007) Action observation has a positive impact on rehabilitation of motor deficits after stroke. Neuroimage 36 Suppl 2:T164-173. 160 Eytan D, Brenner N, Marom S (2003) Selective adaptation in networks of cortical neurons. J Neurosci 23:9349-9356. Fadiga L, Craighero L, Olivier E (2005) Human motor cortex excitability during the perception of others' action. Curr Opin Neurobiol 15:213218. Fadiga L, Fogassi L, Pavesi G, Rizzolatti G (1995) Motor facilitation during action observation: a magnetic stimulation study. J Neurophysiol 73:2608-2611. Floel A, Hummel F, Breitenstein C, Knecht S, Cohen LG (2005) Dopaminergic effects on encoding of a motor memory in chronic stroke. Neurology 65:472-474. Florian RV (2007) Reinforcement learning through modulation of spiketiming-dependent synaptic plasticity. Neural Computation 19:14681502. Foote J (1999) An overview of audio information retrieval. Multimedia Syst 7:2-10. Freiwald WA, Tsao DY, Livingstone MS (2009) A face feature space in the macaque temporal lobe. Nature Neuroscience 12:1187-1196. Fremaux N, Sprekeler H, Gerstner W (2010) Functional requirements for reward-modulated spike-timing-dependent plasticity. J Neurosci 30:13326-13337. Freund Y, Schapire RE (1995) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55:119-139. Frey U, Matthies H, Reymann KG (1991) The effect of dopaminergic D1 receptor blockade during tetanization on the expression of long-term potentiation in the rat CA1 region in vitro. Neurosci Lett 129:111-114. Froemke RC, Dan Y (2002) Spike-timing-dependent synaptic modification induced by natural spike trains. Nature 416:433-438. Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119 ( Pt 2):593-609. Garrison KA, Winstein CJ, Aziz-Zadeh L (2010) The mirror neuron system: a neural substrate for methods in stroke rehabilitation. Neurorehabil Neural Repair 24:404-412. Ghosh-Dastidar S, Adeli H (2009) Spiking neural networks. Int J Neural Syst 19:295-308. Godschalk M, Lemon RN, Kuypers HG, Ronday HK (1984) Cortical afferents and efferents of monkey postarcuate area: an anatomical and electrophysiological study. Exp Brain Res 56:410-424. 161 Gold JI, Shadlen MN (2001) Neural computations that underlie decisions about sensory stimuli. Trends in Cognitive Sciences 5:10-16. Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535-574. Goldstein LB (1999) Amphetamine-facilitated poststroke recovery. Stroke 30:696-698. Grefkes C, Nowak DA, Wang LE, Dafotakis M, Eickhoff SB, Fink GR (2010) Modulating cortical connectivity in stroke patients by rTMS assessed with fMRI and dynamic causal modeling. Neuroimage 50:233-242. Gross CG (2000) Coding for visual categories in the human brain. Nature Neuroscience 3:855-856. Gross CG (2002) Genealogy of the "grandmother cell". Neuroscientist 8:512518. Gupta A, Wang Y, Markram H (2000) Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex. Science 287:273-278. Hadj Tahar A, Blanchet PJ, Doyon J (2004) Motor-learning impairment by amantadine in healthy volunteers. Neuropsychopharmacology 29:187194. Han J, Jian-Xin X, VanDongen AMJ (2010) Classification of musical styles using liquid state machines. In, pp 1-7. The 2010 International Joint Conference on Neural Networks (IJCNN). Harley CW (2004) Norepinephrine and Dopamine as Learning Signals. Neural Plasticity 11:191-204. Hatsopoulos NG, Suminski AJ (2011) Sensing with the motor cortex. Neuron 72:477-487. Hesse S, Werner C, Schonhardt EM, Bardeleben A, Jenrich W, Kirker SG (2007) Combined transcranial direct current stimulation and robotassisted arm training in subacute stroke patients: a pilot study. Restor Neurol Neurosci 25:9-15. Hosp JA, Pekanovic A, Rioult-Pedotti MS, Luft AR (2011) Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J Neurosci 31:2481-2487. Hull CL (1943) Principles of Behavior: An Introduction to Behavior Theory. New York: Appleton-Century-Crofts. Hummel F, Cohen LG (2005) Improvement of motor function with noninvasive cortical stimulation in a patient with chronic stroke. Neurorehabil Neural Repair 19:14-19. 162 Isope P, Barbour B (2002) Properties of unitary granule cell-->Purkinje cell synapses in adult rat cerebellar slices. J Neurosci 22:9668-9678. Izhikevich EM (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17:2443-2452. Jaeger H (2001) The "echo state" approach to analysing and training recurrent neural networks. GMD Report 148, German National Research Center for Information Technology. Jagadeesh B (2009) Recognizing grandmother. Nature Neuroscience 12:10831085. Jones EG, Coulter JD, Hendry SH (1978) Intracortical connectivity of architectonic fields in the somatic sensory, motor and parietal cortex of monkeys. J Comp Neurol 181:291-347. Joshi P (2007) From memory-based decisions to decision-based movements: a model of interval discrimination followed by action selection. Neural networks 20:298-311. Joshi P, Maass W (2004) Movement generation and control with generic neural microcircuits. Lect Notes Comput Sc 3141:258-273. Kaiser M, Hilgetag CC, van Ooyen A (2009) A simple rule for axon outgrowth and synaptic competition generates realistic connection lengths and filling fractions. Cereb Cortex 19:3001-3010. Kaiser M, Martin R, Andras P, Young MP (2007) Simulation of robustness against lesions of cortical networks. Eur J Neurosci 25:3185-3192. Kim S-J, Magnani A, Boyd S (2006) Optimal kernel selection in Kernel Fisher discriminant analysis. In: Proceedings of the 23rd international conference on Machine learning, pp 465-472. Pittsburgh, Pennsylvania: ACM. Kobayashi R, Tsubo Y, Shinomoto S (2009) Made-to-order spiking neuron model equipped with a multi-timescale adaptive threshold. Front Comput Neurosci 3:9. Koepp MJ, Gunn RN, Lawrence AD, Cunningham VJ, Dagher A, Jones T, Brooks DJ, Bench CJ, Grasby PM (1998) Evidence for striatal dopamine release during a video game. Nature 393:266-268. Krebs HI, Volpe B, Hogan N (2009) A working model of stroke recovery from rehabilitation robotics practitioners. J Neuroeng Rehabil 6:6. Kuczenski R, Segal DS (1997) Effects of methylphenidate on extracellular dopamine, serotonin, and norepinephrine: comparison with amphetamine. J Neurochem 68:2032-2037. Lago-Fernandez LF, Huerta R, Corbacho F, Siguenza JA (2000) Fast response 163 and temporal coherent oscillations in small-world networks. Phys Rev Lett 84:2758-2761. Legenstein R, Maass W (2007) Edge of chaos and prediction of computational performance for neural circuit models. Neural networks 20:323-334. Legenstein R, Pecevski D, Maass W (2008) A learning theory for rewardmodulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol 4:e1000180. Legenstein R, Chase SM, Schwartz AB, Maass W (2010) A reward-modulated hebbian learning rule can explain experimentally observed network reorganization in a brain control task. J Neurosci 30:8400-8410. Lidzba K, Staudt M, Zieske F, Schwilling E, Ackermann H (2012) Prestroke/poststroke fMRI in aphasia: perilesional hemodynamic activation and language recovery. Neurology 78:289-291. Liebetanz D, Nitsche MA, Tergau F, Paulus W (2002) Pharmacological approach to the mechanisms of transcranial DC-stimulation-induced after-effects of human motor cortex excitability. Brain 125:2238-2247. Lytton WW, Stark JM, Yamasaki DS, Sober SJ (1999) Computer Models of Stroke Recovery: Implications for Neurorehabilitation. The Neuroscientist 5:100-111. Maass W, Markram H (2004) On the computational power of circuits of spiking neurons. J Comput Syst Sci 69:593-616. Maass W, Natschlager T, Markram H (2002) Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 14:2531-2560. Maass W, Joshi P, Sontag ED (2007) Computational aspects of feedback in neural circuits. PLoS Comput Biol 3:e165. Markram H (2006) The blue brain project. Nat Rev Neurosci 7:153-160. Markram H, Wang Y, Tsodyks M (1998) Differential signaling via the same axon of neocortical pyramidal neurons. Proc Natl Acad Sci U S A 95:5323-5328. Markram H, Lubke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275:213-215. Martinsson L, Wahlgren NG, Hardemark HG (2003) Amphetamines for improving recovery after stroke. Cochrane Database Syst Rev:CD002090. McEntee WJ, Mair RG, Langlais PJ (1987) Neurochemical specificity of learning: dopamine and motor learning. Yale J Biol Med 60:187-193. 164 Micheloyannis S, Pachou E, Stam CJ, Vourkas M, Erimaki S, Tsirka V (2006) Using graph theoretical analysis of multi channel EEG to evaluate the neural efficiency hypothesis. Neurosci Lett 402:273-277. Mika S, Ratsch G, Weston J, Scholkopf B, Smola A, Muller KR (2003) Constructing descriptive and discriminative nonlinear features: Rayleigh coefficients in kernel feature spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 25:623-628. Molina-Luna K, Pekanovic A, Rohrich S, Hertler B, Schubring-Giese M, Rioult-Pedotti MS, Luft AR (2009) Dopamine in motor cortex is necessary for skill learning and synaptic plasticity. PLoS One 4:e7082. Mongillo G, Barak O, Tsodyks M (2008) Synaptic theory of working memory. Science 319:1543-1546. Naito E (2004) Sensing limb movements in the motor cortex: how humans sense limb movement. Neuroscientist 10:73-82. Natschlager T, Markram H, Maass W (2003) Computer models and analysis tools for neural microcircuits. In: Neuroscience Databases: A Practical Guide, pp 123-138: Kluwer Academic Publishers. Nikolic D, Hausler S, Singer W, Maass W (2009) Distributed fading memory for stimulus properties in the primary visual cortex. PLoS Biol 7:e1000260. Nitsche MA, Fricke K, Henschke U, Schlitterlau A, Liebetanz D, Lang N, Henning S, Tergau F, Paulus W (2003) Pharmacological modulation of cortical excitability shifts induced by transcranial direct current stimulation in humans. J Physiol 553:293-301. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381:607-609. Olshausen BA, Field DJ (2004) Sparse coding of sensory inputs. Curr Opin Neurobiol 14:481-487. Olson VG, Heusner CL, Bland RJ, During MJ, Weinshenker D, Palmiter RD (2006) Role of noradrenergic signaling by the nucleus tractus solitarius in mediating opiate reward. Science 311:1017-1020. Ortman RL, Venayagamoorthy K, Potter SM (2011) Input separability in living liquid state machines. In: Proceedings of the 10th international conference on Adaptive and natural computing algorithms - Volume Part I, pp 220-229. Ljubljana, Slovenia: Springer-Verlag. Pape L, Gruijl J, Wiering M (2008) Democratic Liquid State Machines for Music Recognition. In: Speech, Audio, Image and Biomedical Signal Processing using Neural Networks (Prasad B, Prasanna SRM, eds), pp 191-215: Springer Berlin Heidelberg. 165 Phillips PJ, Hyeonjoon M, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22:1090-1104. Pilly PK, Seitz AR (2009) What a difference a parameter makes: A psychophysical comparison of random dot motion algorithms. Vision Res 49:1599-1612. Pleger B, Ragert P (2006) Pharmacological support of Neurorehabilitation. Current Drug Therapy 1:17-22. Polleux F (2005) Genetic mechanisms specifying cortical connectivity: let's make some projections together. Neuron 46:395-400. Purushothaman G, Bradley DC (2005) Neural population code for fine perceptual decisions in area MT. Nature Neuroscience 8:99-106. Quian Quiroga R, Panzeri S (2009) Extracting information from neuronal populations: information theory and decoding approaches. Nat Rev Neurosci 10:173-185. Quiroga RQ, Kreiman G, Koch C, Fried I (2008) Sparse but not 'grandmothercell' coding in the medial temporal lobe. Trends in Cognitive Sciences 12:87-91. Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I (2005) Invariant visual representation by single neurons in the human brain. Nature 435:11021107. Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169-192. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65:386-408. Ruaro ME, Bonifazi P, Torre V (2005) Toward the neurocomputer: image processing and pattern recognition with neuronal cultures. IEEE Trans Biomed Eng 52:371-383. Sara SJ (2009) The locus coeruleus and noradrenergic modulation of cognition. Nat Rev Neurosci 10:211-223. Scheidtmann K, Fries W, Muller F, Koenig E (2001) Effect of levodopa in combination with physiotherapy on functional motor recovery after stroke: a prospective, randomised, double-blind study. Lancet 358:787790. Schmid M (2009) Reinforcing Motor Re-Training and Rehabilitation through Games: A Machine-Learning Perspective. Front Neuroeng 2:3. Schrauwen B, D'Haene M, Verstraeten D, Campenhout JV (2008) Compact hardware liquid state machines on FPGA for real-time speech 166 recognition. Neural networks 21:511-523. Seung HS (2003) Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40:1063-1073. Shahaf G, Marom S (2001) Learning in networks of cortical neurons. J Neurosci 21:8782-8788. Shouval HZ, Bear MF, Cooper LN (2002) A unified model of NMDA receptor-dependent bidirectional synaptic plasticity. Proc Natl Acad Sci U S A 99:10831-10836. Simard D, Nadeau L, Kröger H (2005) Fastest learning in small-world neural networks. Physics Letters A 336:8-15. Sjostrom PJ, Turrigiano GG, Nelson SB (2001) Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32:1149-1164. Sombati S, Delorenzo RJ (1995) Recurrent spontaneous seizure activity in hippocampal neuronal networks in culture. J Neurophysiol 73:17061711. Sonde L, Lokk J (2007) Effects of amphetamine and/or L-dopa and physiotherapy after stroke - a blinded randomized study. Acta Neurol Scand 115:55-59. Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 3:919-926. Song S, Sjostrom PJ, Reigl M, Nelson S, Chklovskii DB (2005) Highly nonrandom features of synaptic connectivity in local cortical circuits. PLoS Biol 3:e68. Sporns O, Tononi G, Edelman GM (2000) Theoretical neuroanatomy: relating anatomical and functional connectivity in graphs and cortical connection matrices. Cereb Cortex 10:127-141. Squire LR (2008) Fundamental neuroscience, 3rd Edition. Amsterdam ; Boston: Elsevier / Academic Press. Stam CJ (2004) Functional connectivity patterns of human magnetoencephalographic recordings: a 'small-world' network? Neurosci Lett 355:25-28. Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10:99-127. Stark CEL, Bakker A, Kirwan CB, Miller M (2008) Pattern separation in the human hippocampal CA3 and dentate gyrus. Science 319:1640-1642. 167 Stefan K, Wycislo M, Classen J (2004) Modulation of associative human motor cortical plasticity by attention. J Neurophysiol 92:66-72. Stefan K, Kunesch E, Cohen LG, Benecke R, Classen J (2000) Induction of plasticity in the human motor cortex by paired associative stimulation. Brain 123 Pt 3:572-584. Szentagothai J (1990) "Specificity versus (quasi-) randomness" revisited. Acta Morphol Hung 38:159-167. Teoh EJ, Tan KC, Xiang C (2006) Estimating the number of hidden neurons in a feedforward network using the singular value decomposition. IEEE Trans Neural Netw 17:1623-1629. Thom B (2000) Unsupervised learning and interactive jazz/blues improvisation. Seventeenth National Conference on Artificial Intelligence (Aaai-2001) / Twelfth Innovative Applications of Artificial Intelligence Conference (Iaai-2000):652-657. Tkach D, Reimer J, Hatsopoulos NG (2007) Congruent activity during action and action observation in motor cortex. J Neurosci 27:13241-13250. Ungless MA, Magill PJ, Bolam JP (2004) Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303:2040-2042. Urgesi C, Moro V, Candidi M, Aglioti SM (2006) Mapping implied body actions in the human motor system. J Neurosci 26:7942-7949. van Rossum MC (2001) A novel spike distance. Neural Computation 13:751763. Verstraeten D, Schrauwen B, Stroobandt D, Van Campenhout J (2005) Isolated word recognition with the Liquid State Machine: a case study. Inform Process Lett 95:521-528. Verstraeten D, Schrauwen B, D'Haene M, Stroobandt D (2007) An experimental unification of reservoir computing methods. Neural networks 20:391-403. Vertes P, Duke T (2009) Neural networks with small-world topology are optimal for encoding based on spatiotemporal patterns of spikes. BMC Neuroscience 10:O11. Walker-Batson D, Smith P, Curtis S, Unwin H, Greenlee R (1995) Amphetamine paired with physical therapy accelerates motor recovery after stroke. Further evidence. Stroke 26:2254-2259. Watts DJ, Strogatz SH (1998) Collective dynamics of 'small-world' networks. Nature 393:440-442. Waydo S, Koch C (2008) Unsupervised learning of individuals and categories 168 from images. Neural Computation 20:1165-1178. Yakovlev V, Amit DJ, Romani S, Hochstein S (2008) Universal memory mechanism for familiarity recognition and identification. J Neurosci 28:239-248. Yamamoto N, Tamada A, Murakami F (2002) Wiring of the brain by a range of guidance cues. Prog Neurobiol 68:393-407. Yamazaki T, Tanaka S (2007) The cerebellum as a liquid state machine. Neural Netw 20:290-297. Yano H, Kasai K, Saitou H, Iwata H (2003) Development of a gait rehabilitation system using a locomotion interface. The Journal of Visualization and Computer Animation 14:243-252. Ziemann U, Ilic TV, Pauli C, Meintzschel F, Ruge D (2004) Learning modifies subsequent induction of long-term potentiation-like and long-term depression-like plasticity in human motor cortex. J Neurosci 24:16661672. 169 [...]... investigated through simulated and cultured neuronal networks in this thesis The LSM employs a biological (neuronal culture) or biologically-plausible (simulated) neuronal network to nonlinearly transform input stimuli into high dimensional space Outputs are produced by trained readouts (typically linear classifiers) that extract information from the neuronal microcircuit This thesis begins with a demonstration... Rajaram E, VanDongen AM (2013) Short-term memory in networks of dissociated cortical neurons J Neurosci 33:1940-1953 5 Dranias MR, Ju H., VanDongen AM (2013) Optogenetic Stimulation of Cultured Neuronal Networks: Computation, Memory and Disease manuscript to be submitted 6 Ju H., Xu J.X., VanDongen AM (2013) Intrinsic classification properties of spiking neural networks Manuscript to be submitted 12 and learning... best learning rate and lowest learning error compared to ordered or random networks (Simard et al., 2005) In networks build with Hodgkin and Huxley neurons, small-world topology is required for fast responses and coherent oscillations (Lago-Fernandez et al., 2000) It has also been suggested that small-world networks are optimal for information coding via poly-synchronization (Vertes and Duke, 2009) As... dissociated neuronal cultures The liquid filter in silico consists of hundreds of spiking neurons and thousands of synaptic connections that are 5 randomly initialized and connected Due to the limitation of computational power, the size of the liquid filter cannot be as large as a neuronal culture with around 0.1 million neurons and millions of synapses As each neuron has its own nonlinear dynamics and synaptic... system A simulated animal (animat) was created with a spiking neural network as the brain, living in a 3D virtual world 2 Published as: Ju H., Xu J.X., & VanDongen, A M J (2010) Classification of musical styles using liquid state machines In IEEE International Joint Conference on Neural Networks (IJCNN'2010) (pp 1-7) 3 Ju H., Dranias MR, Xu J.X., VanDongen AM (2013) Computing with cultured cortical networks. .. neuronal networks is not radial, but displays directionality Models with directional connectivity have not yet been explored for LSMs Previous study (Verstraeten et al., 2007) investigated the relation between reservoir parameters and network dynamics with a focus of Echo State Networks, which is a computational framework that shares similar structure with LSM and is built from analog neurons ESN and. .. in the world, in silico, in vitro, and in vivo, has randomness This raises many questions: as the ultimate goal of neuroscience is to understand the brain, bearing the large amount of uncertainty and randomness in brain networks, how does the brain maintain the normal function? Does the randomness contribute to or harm brain functions? Or more fundamentally, is a random neural network capable to process... However, FPGAs are expensive and have low capacity, and thus are not realistic for implementing large scale networks 1.4 Neural networks in vitro It is impossible to build an in silico brain model possessing every aspects of the biological neural networks, because current recording technology only allows the measurement of a tiny part of living neuronal and synaptic dynamics, and there are lots of unknown... to fully understand brain function Neural microcircuit simulations in the LSM paradigm enable us to closely investigate these links: effects of neuronal and synaptic dynamics on network behavior and on the performance of the LSM; the capability of a random network to 7 process spatiotemporal patterns; how short-term memory is encoded in the networks, etc The LSM was originally proposed and verified through... of information in parallel and in real-time The main structure of the brain is nearly deterministic, while cortical circuitry in a local area is rather random This raises a fundamental question: are random networks capable to process information? In light of the liquid state machine (LSM) paradigm that is a real-time computing framework, the ability of random spiking neural networks to process information . COMPUTING WITH SIMULATED AND CULTURED NEURONAL NETWORKS JU HAN NATIONAL UNIVERSITY OF SINGAPORE 2013 COMPUTING WITH SIMULATED AND CULTURED NEURONAL NETWORKS. that is a real-time computing framework, the ability of random spiking neural networks to process information is investigated through simulated and cultured neuronal networks in this thesis Chapter IV Computing with Cultured Cortical Networks: Implementing the Liquid State Machine with Dissociated Neuronal Cultures 51 4.1 Introduction 73 4.2 Methods 75 4.2.1 Cell Culture and Optogenetic