A hubel wiesel model of early concept generalization based on local correlation of input features

A Hubel Wiesel Model of Early Concept Generalization Based on Local Correlation of Input Features SEPIDEH SADEGHI (B. Sc., Iran University of Science & Technology) A THESIS SUBMITTED FOR THE DEGREE OF MASTER OF ENGINEERING DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2011 ii Acknowledgements I would like to express my genuine gratitude to Dr. Kiruthika Ramanathan, from Data Storage Institute (D.S.I), for her support and encouragement in the research and the preparation of this thesis. Through her leadership, insightful advice and excellent judgment, I was able to increase my basic knowledge of analysis and commit to research in the area of my interest. I would like to express my gratitude to Professor Chong Tow Chong - my supervisor from National University of Singapore (N.U.S) -, and Dr. Shi Luping - my supervisor from D.S.I - for reviewing the progress of my project. I am also thankful to Singapore International Graduate Award (S.I.N.G.A) and D.S.I for providing me such wonderful project opportunity and the financial support throughout the course of the project. Appreciation is also extended to Electrical and Computer Engineering department at National University of Singapore. I also thank all my friends from N.U.S and D.S.I for the excellent company they gave during the course of the project. I would also like to thank all my friends in Singapore who made my stay a wonderful experience. Last, but not least, I am grateful to my parents, and sisters, whose devotion, support, and encouragement have inspired me and been my source of motivation for graduate school. iii iv Table of Contents 1. Introduction 1.1 On Concepts and Generalization 1.2 Background and Related Studies 1.2.1 Concept acquisition and generalization 1.2.2 Hubel Wiesel models of memory 1.2.3 Hubel Wiesel models of concept representation 1.3 Objective of the Thesis 1.4 Summary of the Model 1.5 Organization of the Thesis 11 2. Methodology 14 2.1 System Architecture 15 2.1.1 Architecture 15 2.1.2 Bottom up hierarchical learning 19 2.2 Hypothesis 22 2.3 Local Correlation Algorithm 27 2.3.1 Marking features/modules as general or specific v 31 2.3.2 Generalization 33 2.3.2.1 Input management 33 2.3.2.2 Prioritization 35 2.3.3 The effect of local correlation model on the categorization of single modules 3. Results and Discussions 36 39 3.1 Two Types of Input Data 39 3.2 Generalization 46 3.3 Local Correlation Operations and Computational Parameters 49 3.4 Building Hierarchical Structures of Data 55 4. Conclusion 61 4.2 Concluding Remarks 61 4.3 Future Works 63 Bibliography 66 Appendix A-1: Dataset A - List of Entities 73 Appendix A-2: Dataset B - List of Entities 75 Appendix A-3: Dataset C - List of Entities 77 vi A Hubel Wiesel Model of Early Concept Generalization Based on Local Correlation of Input Features Sepideh Sadeghi Submitted on JAN 21, 2011 In Partial Fulfillment of the Requirements for the Degree of Master of Engineering in Electrical and Computer Engineering Abstract Hubel Wiesel models, successful in visual processing algorithms, have only recently been used in conceptual representation. Despite the biological plausibility of a Hubel-Wiesel like architecture for conceptual memory and encouraging preliminary results, there is no implementation of how inputs at each layer of the hierarchy should be integrated for processing by a given module, based on the correlation of the features. If we assume that the brain uses a unique Hubel Wiesel like architecture to represent the input information of any modality, it is important to account for the local correlation of conceptual inputs as an equivalent to the existing local correlation of visual inputs in the visual counterpart models. However, there is no intuitive local correlation among the conceptual inputs. The key contribution of this thesis is the proposal of an input integration framework that accounts for the local correlation of the conceptual inputs in a Hubel Wiesel like architecture to facilitate the achievement of broad and coherent concept categories at the top of the hierarchy. The building blocks of our model are two algorithms: 1) Bottom-up hierarchical learning algorithm, and 2) Input integration framework. The first vii algorithm handles the process of categorization in a modular and hierarchical manner that benefits from competitive unsupervised learning in its modules. The second algorithm consists of a set of operations over the input features or modules to weigh them as general or specific to specify how they should be locally correlated within the modules of the hierarchy. Furthermore, the input integration framework interferes with the process of similarity measurement applied by the first algorithm such that, high-weighted features would count more than the low-weighted features towards the similarity of conceptual patterns. Simulation results on benchmark data admit that implementing the proposed input integration framework facilitates the achievement of the broadest coherent distinctions of conceptual patterns. Achieving such categorizations is a quality that our model shares with the process of early concept generalization. Finally, we applied our proposed model of early concept generalization iteratively over two sets of data, which resulted in the generation of finer grained categorizations, similar to progressive differentiation. Based on our results, we conclude that the model can be used to explain how humans intuitively fit a hierarchical representation for any kind of data. Keywords: Early Concept Generalization, Hubel Wiesel Model, Local Correlation of Inputs, Categorization, General Features, Specific Features. Thesis Supervisors: 1. Prof. Chong Tow Chong, National University of Singapore, and Singapore University of Technology and Design. 2. Dr. Shi Luping, Senior Scientist, Data Storage Institute. viii ix List of Tables Table 3.1 Features and their values sorted in a decreasing order…………………………….43 Table 3.2 Features and their weights…………………………………………………………….44 Table 3.3 Datasets used in the simulations………………………………………………………46 Table 3.4 The effect of growth threshold on the quality of categorization biasing, using maxweight operation over dataset B (7 modules at the bottom layer)……………………….……….52 Table 3.5 The effect of growth threshold on the quality of categorization biasing, using sumweights operation over dataset B (7 modules at the bottom layer)……………………….…… .52 Table 3.6 Summary of the experiments… …………………………………………………… .54 Table 4.1 The effect of decreasing growth threshold on the categorization of the local correlation model …………………………………………………………………………………………….63 x 65 Bibliography [1] Margolis, E., & Laurence, S., Concepts: Core Readings, MIT Press, 1999. [2] Stanford encyclopedia of philosophy: Cognitive Science, http://plato.stanford.edu/entries/concepts/. [3] McClelland, J. L., & Rogers, T. T., The parallel distributed processing approach to semantic cognition, Nature reviews Neuroscience, 4, 310-322, 2003. [4] Rogers, T. T., & McClelland, J. L., Semantic Cognition, A parallel distributed processing approach. MIT press, 2004. [5] Rogers, T. T., & McClelland, J. L., Précis of Semantic Cognition, A parallel distributed processing approach, Brain and Behavioral Sciences, 31, 689-749, 2008. [6] Vygotsky, L. S., Thought and Language, MIT Press, 1986. [7] Inhelder, B., & Piaget, J., The Early growth of logic in a child, Classification and serialization, London: Routeledge Kegal Paul, 1964. [8] Murphy, G. L., The big book of concepts, MIT Press, 2002. [9] Rumelhart, D. E., An introduction to Neural and Electronic Networks, NewYork: Academic Press, pp 405-420, 1990. [10] Jones, S. S., & Smith, L. B., How children know the relevant properties or generalizing object names, Developmental Science, 5, 219-232, 2002. 66 [11] Gelman, S. A. & Markman, E., Categories and Induction in young children, Cognition, 23, 183-209, 1986. [12] Rumelhart, D. E. & Todd, P. M., Learning and Connectionist Representations. in Attention and Performance XIV, Synergies in Experimental Psychology, Artificial Intelligence, and Cognitive Neuroscience, MIT Press, pp 330, 1993. [13] Sloutsky, V. M., Lo, Y.-F., & Fisher, A., How much does a shared name make things similar, Linguistic labels, Similarity and the development of inductive inference, Child Development, 72, 1695-1709, 2001. [14] Sloutsky, V. M., The role of similarity in the development of categorization, Trends in Cognitie Sciences, 7, 246-251, 2003. [15] Sloutsky, V. M., Kloos. H., & Fisher, A. V., When looks are everything: appearance similarity versus kind information in early induction, Psychological Science, 18 (2), 179-185, 2007. [16] Quillian, M. R., Semantic Information Processing, MIT Press, pp 227-270, 1965. [17] Mountcastle, V., The columnar organization of the neocortex. Brain, 120, 701-722, 1997. [18] Mountcastle, V., An Organizing Principle for Cerebral Function: The Unit Model and the Distributed System, The Mindful Brain (Gerald M. Edelman and Vernon B. Mountcastle, eds.) Cambridge, MA: MIT Press, 1978. 67 [19] Cadieu, C., Kouh, M., Pasupathy, A., Conner, C., Riesenhuber, M., & Poggio, T. A., A Model of V4 Shape Selectivity and Invariance, Journal of Neurophysiology, 98, 1733-1750, 2007. [20] Reisenhuber, M., & Poggio. T., Hierarchical models of object recognition in cortex, Nature Neuroscience, 2(11), 1019-1025, 1999. [21] Van Essen, D. C., Anderson, C. H., & Felleman, D. J., Information Processing in the primate visual system: An integrated systems perspective, Science, 255 (5043), 419-423, 1992. [22] Felleman, D. J., & Van Essen, D. C., Distributed hierarchical processing in the primate cerebral cortex, Cerebral Cortex, 1(1), 1-47, 1991. [23] Anderson, R. J., Snowden, R. J., Treue, S., & Grazia, M., Hierarchical Processing of motion in the visual cortex of a monkey, Cold Spring Harbor Symposia on quantitative Biology, 55, 741-748, 1990. [24] Wessinger, C. M., Vanmeter, J., Tian, B., Van Lare, J., Pekar, J., & Rauschecker, J. P., Hierarchical organization of the human auditory cortex revealed by functional magnetic resonance imaging, Journal of Cognitive Neuroscience, 13(1), 1-7, 2001. [25] Read, H L., Winer, J. A., & Schreiner, C. E., Functional Architecture of Auditory Cortex, Current Opinions in Neurobiology, 12(4), 433-40, 2002. 68 [26] Iwamura, Y., Dynamic and hierarchical processing in the monkey somatosensory cortex, International symposium on the dynamics of perception, 14, 107-111, 1993. [27] Toda, T., & Taoka, M., Hierarchical somethetic processing of tongue inputs in the post central somatosensory cortex of conscious macaque monkeys, Experimental brain research, 147(2), 243-251, 2002. [28] Hubel, D., & Weisel, T., Receptive fields and functional architecture in two non striate visual areas (18 and 19) of a cat, Journal of Neurophysiology, 28, 229289, 1965. [29] Lampl, I., Ferster, D., Poggio, T., & Riesenhuber, M., Intracellular Measurements of Spatial Integration and the MAX Operation in Complex Cells of the Cat Primary Visual Cortex, Journal of Neurophysiology, 92, 2704-2713, 2004. [30] Fukushima, K., Neocognitron for handwritten digit recognition 1, Neurocomputing , 51C, 161-180, 2003. [31] Fukushima, K., Neocognitron capable of incremental learning, Neural networks, 17, 37-46, 2004. [32] Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T., Robust Object Recognition with Cortex-like Mechanisms, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3), 411-426, 2007. 69 [33] LeCun, Y., Lowe, D. J., Malik, J., Mutch, J., Perona, P., & Poggio, T., Object Recognition, Computer Vision, and the Caltech 101: A Response to Pinto et al., PLoS Computational Biology, Posted Online March 2008. [34] Mel, B. W., SEEMORE: Combining Color, Shape and Texture Histogramming in a neurally inspired approach to visual object recognition, Neural Computation, 9, 777-804, 1997. [35] LeCun, Y., Huang, F. J., & Bottou, L., Learning methods for Generic object recognition with invariance to pose and lighting, IEEE Conference on Computer Vision and pattern Recognition, 2004. [36] Wallis, G., & Rolls, E., A model of invariant object recognition in the visual system, Progress in neurobiology 51, 167-194, 1997. [37] Rolls, E. T., & Stringer, S. M., Invariant Global Motion Recognition in the Dorsal Visual System: A Unifying Theory, Neural Computation, 19(1), 139-169, 2007. [38] Stringer, S. M., & Rolls, E. T., Invariant Object Recognition in the Visual System with Novel Views of 3D Objects, Neural Computation, 14(11), 25852596, 2001. [39] Wersing, H., & Koener, E., Learning Optimized features for hierarchical models of invariant recognition, Neural Computation, 15 (7), 1559-1588, 2003. 70 [40] Wolf, L., Bileschi, S., & Meyers, E., Perception strategies in hierarchical vision systems, IEEE Conference on Computer Vision and Pattern Recognition, 2006. [41] Ramanathan, K., & Shi, S., & Chong, T. C., A Hubel Weisel model for hierarchical representation of concepts in textual documents, The Annual Meeting of the Cognitive Society (COGSCI), 1106-1111, 2010. [42] Murphy, G. L., & Medin, D. L., The role of theories in conceptual coherence. Psychological review, 92, 289- 316, 1985. [43] Alahakhoon, D., Halgamuge, S. K., & Srinivasan, B., Dynamic Self Organizing maps with controlled growth for Knowledge discovery, IEEE Transactions on neural networks, 11(3), 601-614, 2000. [44] Kemp, C., & Tenenbaum, J. B., The discovery of structural form, Proceedings of the National Academy of Science, 105(31), 10687-10692, 2008. [45] Rosch, E., Principles of categorization. Cognition and Categorization, eds Rosch E, LIoyd BB (Lawrence Erlbaum, New York), pp 27-48, 1978. 71 72 Appendix A-1: Dataset A - List of Entities Following are the complete lists of patterns, features in Dataset A from Rogers and McClelland Corpus (2004). It includes 21 patterns and 26 features. List of Patterns Pine, Oak, Maple, Birch, Rose, Daisy, Tulip, Sunflower, Robin, Canary, Sparrow, Penguin, Sunfish, Salmon, Flounder, Cod, Cat, Dog, Mouse, Goat and Pig. List of Features Pretty, Big, Living, Green, Red Yellow, White, Twirly, Grow, Move, Swim, Fly, Walk, Sing, Leaves, Roots, Skin, Legs, Bark, Branches, Petals, Wings, Feathers, Scales, Gills and Fur. 73 Dataset A – Input Matrix, columns 1-12: pine Living pretty green big red yellow white twirly grow move swim fly walk sing bark branches petals wings feathers scales gills leaves roots skin legs fur oak 1 0 0 0 0 1 0 0 0 0 maple 0 0 0 0 0 1 0 0 1 0 birch 0 1 0 1 0 0 1 0 0 1 0 rose daisy 0 0 1 0 0 1 0 0 1 0 tulip 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 0 1 0 sunflower robin 1 1 0 0 0 0 0 0 1 0 canary 0 0 1 0 0 1 0 0 1 sparrow 1 0 0 1 1 0 1 0 0 1 penguin 0 0 0 1 0 0 1 0 0 1 Dataset A – Input Matrix, columns 12-21: sunfish Living pretty green big red yellow white twirly grow move swim fly walk sing bark branches petals wings feathers scales gills leaves roots skin legs fur salmon 0 0 0 1 0 0 0 0 1 0 0 flounder 0 1 0 1 0 0 0 0 1 0 0 cod 0 0 0 1 0 0 0 0 1 0 0 dog 0 0 1 1 0 0 0 0 1 0 0 74 cat 0 0 0 1 0 0 0 0 0 0 1 mouse 1 0 0 0 1 0 0 0 0 0 0 1 goat 0 0 0 1 0 0 0 0 0 0 1 pig 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 0 0 1 0 0 1 Appendix A-2: Dataset B - List of Entities Following are the complete lists of patterns, features in Dataset B from Rogers and McClelland Corpus (2004). It includes 13 patterns and 14 features. List of Patterns Robin, Canary, Sparrow, Penguin, Sunfish, Salmon, Flounder, Cod, Cat, Dog, Mouse, Goat and Pig. List of Features Pretty, Big, Red, Yellow, White, Swim, Fly, Walk, Sing, Wings, Feathers, Scales, Gills and Fur. 75 Dataset B – Input Matrix robin pretty big red yellow white swim fly walk sing wings feathers scales gills fur canary 0 0 0 1 0 0 0 1 1 0 sparrow penguin 0 0 0 0 1 0 sunfish 0 0 1 1 0 salmon 0 1 0 0 1 1 0 0 0 1 flounder cod 0 0 0 0 1 76 dog 0 0 1 0 0 1 cat 0 0 0 0 mouse 0 0 0 0 0 goat 0 0 0 0 0 pig 0 0 0 0 0 0 0 0 0 Appendix A-3: Dataset C - List of Entities Following are the complete lists of patterns, features in Dataset C from Kemp and Tenenbaum Corpus (2008). It includes 33 patterns and 102 features. List of Patterns Elephant, Rhino, Horse, Cow, Camel, Giraffe, Chimp, Gorilla, Mouse, Squirrel, Tiger, Lion, Cat, Dog, Wolf, Seal, Dolphin, Robin, Eagle, Chicken, Salmon, Trout, Bee, Iguana, Alligator, Butterfly, Ant, Finch, Penguin, Cockroach, Whale, Ostrich, and Dear. List of Features Lungs, Large brain, Spinal cord, Warm blooded, Teeth, Feet, legs, legs, Tongue, Visible ears, Nose, Paws, Lives in groups, Tough skin, Long neck, Fins, Long legs, Fish, Snout, Antennae, Eats rodents, Travels in groups, Long, Large, Roars, Claws, Wings, Green, Tusks, Carnivore, Slender, Dangerous, Eats grass, Tall, Beak, Slow, Fast, Lives in trees, eats leaves, Smooth, Lizard, Eats seeds, Poisonous, Soft, Bird, Black, Hunts, Howls, Gills, Feline, Stripes, Lives in the forest, legs, Strong, Predator, Rodent, Lives in hot climates, Webbed feet, Eats mice, Lives in lakes, Squawks, Ferocious, Lives in cold climates, Yellow, Lives in the ocean, Hooves, Feathers, Makes loud noises, Eats bugs, Runs, Bites, Crawls, Swims, Flies, Insect, Lives in water, Sings, Horns, Eats nuts, Brown, Eats fish, Lays eggs, Scaly, Eats animals, Furry, Smart, Blue, Tail, Flippers, Reptile, Lives on land, Colorful, Lives in houses, Digs holes, Lives in grass, Mammal, White, Canine, Womb, Subcutaneous fat, red blood, and Bones. 77 Dataset C – Input Matrix, columns 1-12 elephant lungs large brain spinal cord warm blooded teeth feet legs legs tongue visible ears nose paws lives in groups tough skin long neck fins long legs fish snout antennae eats rodents travels in groups long large roars claws wings green tusks carnivore slender dangerous eats grass tall beak slow fast lives in trees eats leaves smooth lizard eats seeds poisonous soft bird black hunts howls gills feline stripes lives in the forest legs strong predator rodent lives in hot climates webbed feet eats mice lives in lakes squawks ferocious lives in old climates yellow lives in the ocean hooves feathers makes loud noises eats bugs runs bites crawls swims flies insect lives in water sings horns eats nuts brown eats fish lays eggs scaly eats animals furry smart blue tail flippers reptile lives on land colorful lives in houses diges holes lives in grass mammal white canine womb subcutaneous fat red blood bones rhino 1 1 0 1 0 0 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 1 1 horse 1 1 0 1 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 1 cow 1 1 0 1 1 0 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 camel 1 1 0 1 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 1 1 1 giraffe 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 78 chimp 1 1 0 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 1 1 gorilla 1 1 1 1 1 0 0 0 0 1 0 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 1 mouse 1 1 1 1 1 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 squirrel 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 tiger 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 lion 1 1 1 0 1 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 1 1 0 0 1 0 0 1 0 0 0 0 0 0 1 1 0 1 1 0 1 1 1 1 1 0 1 1 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 1 1 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 1 0 1 1 Dataset C – Input Matrix, columns 12-24 cat lungs large brain spinal cord warm blooded teeth feet legs legs tongue visible ears nose paws lives in groups tough skin long neck fins long legs fish snout antennae eats rodents travels in groups long large roars claws wings green tusks carnivore slender dangerous eats grass tall beak slow fast lives in trees eats leaves smooth lizard eats seeds poisonous soft bird black hunts howls gills feline stripes lives in the forest legs strong predator rodent lives in hot climates webbed feet eats mice lives in lakes squawks ferocious lives in old climates yellow lives in the ocean hooves feathers makes loud noises eats bugs runs bites crawls swims flies insect lives in water sings horns eats nuts brown eats fish lays eggs scaly eats animals furry smart blue tail flippers reptile lives on land colorful lives in houses diges holes lives in grass mammal white canine womb subcutaneous fat red blood bones dog 1 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 0 0 1 0 0 0 0 1 0 1 1 0 0 1 0 1 1 wolf 1 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 1 1 0 1 1 1 0 1 0 1 0 0 0 0 1 0 1 1 0 0 1 1 1 1 seal 1 1 1 0 1 1 0 0 1 0 0 1 0 0 0 0 0 0 1 0 1 1 0 0 1 0 0 1 0 0 0 0 1 0 1 1 0 0 1 1 1 1 dolphin 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 1 robin 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 1 79 eagle 1 1 1 0 0 0 0 0 0 0 1 0 0 1 0 1 1 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 1 chicken 1 1 1 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 1 1 1 0 0 0 0 0 1 salmon 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 1 trout 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 1 0 1 0 0 0 0 0 1 bee 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 iguana 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 1 1 0 1 1 0 0 0 0 1 Dataset C – Input Matrix, columns 24-33 alligator lungs large brain spinal cord warm blooded teeth feet legs legs tongue visible ears nose paws lives in groups tough skin long neck fins long legs fish snout antennae eats rodents travels in groups long large roars claws wings green tusks carnivore slender dangerous eats grass tall beak slow fast lives in trees eats leaves smooth lizard eats seeds poisonous soft bird black hunts howls gills feline stripes lives in the forest legs strong predator rodent lives in hot climates webbed feet eats mice lives in lakes squawks ferocious lives in old climates yellow lives in the ocean hooves feathers makes loud noises eats bugs runs bites crawls swims flies insect lives in water sings horns eats nuts brown eats fish lays eggs scaly eats animals furry smart blue tail flippers reptile lives on land colorful lives in houses diges holes lives in grass mammal white canine womb subcutaneous fat red blood bones butterfly 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 0 1 0 1 0 0 0 0 1 ant 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 finch 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 penguin 1 1 1 0 0 0 0 0 0 1 0 0 1 0 1 1 0 0 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 0 0 1 80 1 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 1 cockroach whale 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ostrich 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 1 0 0 0 0 1 1 dear 1 1 1 0 0 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 [...]... specific and broader categories 3 Early concept generalization is the early stage of progressive differentiation of concepts [3], in which children acquire broad semantic distinctions 1 Concept generalization is one of the primary tasks of human cognition Generalization of new concepts (conceptual patterns) based on prior features (conceptual features) leads to categorization judgments that can be used... our model shares this quality with early concept generalization The flow chart of our model of early concept generalization is presented in Figure 1.1 Based on our knowledge about concept generalization, first it facilitates acquiring of broad distinctions and only as a matter of time leads to acquiring of the finer distinctions This flow is called progressive differentiation of concepts which can also... state of art in the field of concept acquisition and generalization, and the second part describes research in the field of Hubel Wiesel models of memory 1.2.1 Concept acquisition and generalization The idea of feature based concept acquisition and generalization has been well studied in the psychological literature Vygotsky [6], Inhelder and Piaget [7] first 2 proposed that the representation of categories... hierarchy are dependent on the input integration framework of the hierarchy Hence, we argue one possible metric based on which a local correlation model among conceptual features can be 7 achieved Then, we propose an input integration framework to maintain such correlation through hierarchy Interestingly, it was observed that the proposed correlation model along with its corresponding input integration framework... they be abstract entities?” [1] In our thesis, we define a concept as a mental representation which partially corresponds to the words of the language We further assume that a concept can be defined as a set of typical features [2] We adopt the following definitions 1 Concept categorization is the process by which the concepts are differentiated 2 Concept generalization is the categorization of concepts... Hierarchical learning algorithm, and 2) Input integration algorithm corresponding to the proposed local correlation model – we may use local correlation algorithm /model or input integration algorithm‟ interchangeably to refer to this algorithm Local correlation algorithm extracts the correlated input features (at the bottom layer) and the correlated input child modules (at the intermediate layers) and... coherent a local correlation of inputs preserved all over the hierarchy On the other hand, in the conceptual Hubel Wiesel model proposed by Ramanathan et al [41], there is no provision to account for the local correlation of inputs and how it should be preserved through the hierarchy 1.3 Objectives of the Thesis The objective of this dissertation is to capture the quality of early concept generalization and... from immature representations that are based on accidental features (appearance similarities) Recent theoretical and practical developments in the study of mature categorization indicate that generalization is grounded on perceptual mechanisms capable of detecting multiple similarities [3, 8-10] Tests such as the trial task [11] show the role of feature similarity in the generation of categorization Further... succeed to facilitate the achievement of coherent categorization - which admits our prior assumption in this regard The proposed model not only effectively captures coherent categorization but also ensures revealing of the broadest differentiation of its conceptual inputs Based on our literature survey, revealing the broadest differentiation is one of the qualities of early concept generalization Therefore,... Hubel Wiesel model of early concept generalization The highlighted rectangles demonstrate local correlations operations 10 Figure 1.2: The flow chart of the top-down algorithm – to model progressive differentiation 1.5 Organization of the Thesis The rest of the thesis is organized as follows:  Chapter 2 presents the methodology to enable Hubel Wiesel model to obtain coherent broad categorization of . local correlation among the conceptual inputs. The key contribution of this thesis is the proposal of an input integration framework that accounts for the local correlation of the conceptual. differentiation of concepts [3], in which children acquire broad semantic distinctions. 2 Concept generalization is one of the primary tasks of human cognition. Generalization of new concepts (conceptual. acquisition and generalization, and the second part describes research in the field of Hubel Wiesel models of memory. 1.2.1 Concept acquisition and generalization The idea of feature based concept