Ways of seeing the scope and limits of visual cognition

WAYS OF SEEING: The Scope and Limits of Visual Cognition Authors: Pierre Jacob, Marc Jeannerod Oxford University Press eBook created (03/01/‘16): QuocSan CONTENTS: Acknowledgement Introduction: What is human visual cognition? Part I The purposes of vision: perceiving, thinking and acting The representational theory of the visual mind A teleosemantic account of visual percepts Visual intentionalism, sense-data and disjunctivism 2.1 Sense-data and the argument from illusion 2.2 Disjunctivism and the rejection of an interface between mind and world 2.3 The challenge of the homunculus Conceptual and non-conceptual content 3.1 The productivity and systematicity of thoughts 3.2 The fine-grainedness and informational richness of visual percepts Elements of cognitive dynamics 4.1 Cognitive engagement and the detachment constraint on thoughts 4.2 Unarticulated constituency Actions and the intentionality of intentions 5.1 The distinctive role of intentions in the etiology of actions 5.2 The intentionality of intentions Part II Empirical evidence for the duality of visual processing Foreword to Part II Multiple pathways in the primate visual system The where and the what: two visual systems Two cortical visual systems Neural mechanisms for object discrimination: the encoding of intrinsic object properties Neural mechanisms for space perception: the encoding of spatial relationships in the posterior parietal lobe Neural mechanisms for acting in space: the visuomotor functions of posterior parietal areas 5.1 Neural mechanisms for coordinate transformations in the posterior parietal cortex 5.2 Neural mechanisms for transforming object geometric properties into motor commands Conclusion Dissociations of visual functions by brain lesions in human patients Introduction Visual impairments following lesions in the primary visual cortex Impairment in visual perception and the recognition of objects following occipito-temporal lesions 3.1 Associative agnosia 3.2 Apperceptive agnosia 3.3 The discovery of preserved visuomotor capacities in apperceptive agnosic patients 3.3.1 Patient DF 3.3.2 Patient SB 3.3.3 Comparing DF and SB Impairments in visually guided behavior following lesion of the dorsal system 4.1 Optic ataxia, a specific disorder of the visuomotor transformation 4.2 Patient AT: a visuomotor impairment with normal object recognition Visuospatial disorders following lesions in the parietal lobes 5.1 Misperception of the orientation of objects 5.2 Dorsal simultagnosia 5.3 Unilateral spatial neglect 5.4 Do visuomotor and visuospatial disorders correspond to dissociable functions of the parietal lobes? The varieties of normal human visual processing Pointing to an unperceived target Temporal properties of perceptual and visuomotor processings Time and awareness in perceptual and visuomotor tasks Frames of reference Do size-contrast illusions deceive pointing? Do size-contrast illusions deceive grasping? Disentangling the pictorial from the motoric role of annuli The interplay between perceptual judgment and visuomotor processing Concluding remarks Part III Perceiving objects and grasping them Visual perception Introduction Visual perception, identification and recognition 2.1 The reliability of visual perception 2.2 Visual attributes and visual modes of perception 2.3 Recognitional concepts and prototypes The interaction of visual and non-visual knowledge 3.1 Seeing and knowing 3.2 Primary and secondary epistemic seeing The scope and limits of visual knowledge 4.1 The justificatory role of vision in secondary epistemic seeing 4.2 Is visual knowledge closed under deduction? How intelligent are perceptual processes? 5.1 Filling-in and constructivism 5.2 The ecological tradition and the modularity of perception Is seeing believing? 6.1 Seeing, seeing that and seeing as 6.2 First-person and third-person perceptual reports 6.3 What must one believe and/or notice in order to perceive? The phenomenology of visual experience 7.1 Phenomenal realism and the explanatory gap 7.2 Dismissing the explanatory gap 7.3 Visual experience and binding Visuomotor representations Introduction Seeing affordances Evidence for dual visual processing in primates 3.1 Electrophysiological evidence for dual visual processing 3.2 Psychophysical evidence for dual visual processing What is it like to see with a dorsal pathway? The perceptual individuation of visual objects by location The motoric encapsulation of visuomotor representations Are there visuomotor representations at all? The role of visuomotor representations in the human cognitive architecture 8.1 Visuomotor representations serve as inputs to motor intentions 8.2 Visuomotor representations serve as inputs to causal indexicals Part IV The perception of action Seeing humans act Introduction From grasping objects to manipulating tools 2.1 The praxic system: a high-level visual pragmatic processing of objects 2.2 Neuropsychological evidence: apraxia The primary level of the visual processing of actions 3.1 Seeing biological motion 3.2 The kinetic cues of animacy Seeing object-oriented actions 4.1 The discovery of mirror neurons in the brain of macaque monkeys 4.2 The mirror system, mimicry, resonance and imitation in humans 4.3 Infants’ visual sensitivity to reaching and grasping The social perception system 5.1 Elementary visual cues of proto-social intentions 5.2 Processing the social significance of human bodily movements 5.3 The face perception system Epilogue: The two visual systems revisited The complexities of pragmatic processing The contribution of the parietal lobes to human vision References Acknowledgement We are very grateful to Jean Bullier, Ruth Garrett Millikan and Chris Peacocke, who read and sent us comments on our manuscript We are also grateful to Chris Peacocke for having nurtured our project and provided many advices from the very beginning In addition, we wish to thank Roberto Casati, Jérôme Dokic, Jean-René Duhamel, Mel Goodale, John Marshall, Franỗois Michel, Alva Noở, Elisabeth Pacherie, Giacomo Rizzolatti, Georges Rey, Yves Rossetti, Charles Travis, Anne Tüscher and Semir Zeki for useful reactions and conversations Much of this book originated from interactions between the two authors at the Institut des Sciences Cognitives in Lyon (France) Introduction: What is human visual cognition? Humans can see a great variety of things They can see tables, trees, flowers, stars, planets, mountains, rivers, substances, tigers, people, vapors, rain, snow, gases, flames, clouds, smoke, shadows, flashes, holes, pictures, signs, movies, events, actions (including people seeing any of the preceding) They can see properties of things such as the color, texture, orientation, shape, contour, location, motion of objects They can see facts, such as the fact that a given object exemplifies a set of visual attributes and/or stands in some visual relation to other objects Sight, visual experience or visual perception, is both a particular kind of human experience and a fundamental source of human knowledge of the world Furthermore, it interacts in multiple ways with human thought, human memory and the rest of human cognition Many of the things humans can see they can also think about Many of the things they can think about, however, they cannot see For example, they can think about, but they cannot see at all, prime numbers Nor can they see atoms, molecules and cells without the aid of powerful instruments Arguably, while atoms, molecules and cells are not visible to the naked eye, unlike numbers, they are not invisible altogether: with powerful microscopes, they become visible Unlike numerals, however, numbers—whether prime or not—are simply not to be seen at all Similarly, humans can entertain the thought, but they cannot see, that many of the things they can think about they cannot see Museums are institutions purposefully designed to promote the exercise of human sight and the enjoyment of visual experience If you visit the Louvre in Paris, for example, you can see a famous painting by the late eighteenthcentury French painter, Jean Siméon Chardin, called Le gobelet d’argent (‘The silver goblet’) (Fig 0.1) Facing this picture, you will see three red and yellow apples, a silvery beaker, a large brown dish with a silver spoon in it and two brown chestnuts lying on a brown table Of course, what you call ‘brown’ in English is not one but many different color shades: although you call them ‘brown’, your visual experiences of the colors of the table, the dish and the chestnuts are all different Nor you see the full spoon: you merely see a tip of the handle emerging from the dish, but you take it that what you see is the handle of a spoon the rest of which is being hidden by the dish in which it is resting The central apple partly occludes the other two The apple on the right partly occludes the brown dish As the light is coming from the top left corner of the canvas, it is reflected in the silvery beaker and falling sideways onto the apples, the chestnuts and the top of the spoon The apples cast their shadows on the table So the chestnuts The dish casts a shadow on the wall If you look closely, you will discover incredibly subtle reflections of the apples in the silvery beaker You will also see a rich network of spatial relationships between the objects: the silvery beaker stands to the left and slightly behind the apples The large brown dish with a spoon in it stands to the right of the beaker and behind the apples The chestnuts are to the right of everything else Everything lies on the table Fig.0.1 Le Gobelet d’argent by Chardin, Jean Baptiste Siméon (1699– 1779), Louvre © Photo RMN – Hervé Lewandowski Your visual experience of the painting raises a number of fascinating issues for cognitive science First of all, you will not perceive the red and yellow apples, the silvery beaker, the brown dish, the silvery spoon, the brown chestnuts and the table, unless your visual system is in good working condition Some brain lesions can result in visualform agnosia, i.e the inability to perceive visually the shapes of objects Others can result in achromatopsia, i.e the inability to visually perceive the colors of objects In the presence of the painting, neither a visual-form agnosic patient nor an achromatopsic patient will see what a normal human being can see Similarly, there is evidence that akinetopsic patients, who, as a result of a brain lesion, cannot visually perceive motion, will not react like normal subjects to the perception of static images of human bodily postures that convey dynamic information about the movement of, for example, an athlete throwing a disc Unlike normal human subjects, even though a prosopagnosic patient may know the person who served as a model for a portrait, she might not be able to recognize, in seeing the painting, who the portrait portrays Second, it soon emerges that the normal visual experience of the shapes, contours, orientations, textures and colors of objects depicted by a painting far outstrips the power to impose conceptual and/or linguistic labels or categories.[1] Third, however, the human power to parse and categorize the visual scene in terms of concepts of objects is striking One can count the objects depicted If so, then in Chardin’s painting, one will find nine objects: the beaker, the three apples, the dish, the spoon, the pair of chestnuts and the table Arguably, unlike the other three artifacts (the beaker, the dish and the spoon), the table is not so much an object as it is part of the background, together with the wall standing behind everything else and perpendicular to the top of the table Furthermore, one can easily group the objects into meaningful sets or classes: the three apples and the two chestnuts together make five fruits Unlike ‘fruit’ and ‘artifact’, ‘apple’ and ‘spoon’ are names of what psychologists call basic concepts Whether ‘apple’ and ‘chestnut’ are names of natural kinds, ‘spoon’, ‘beaker’ and ‘dish’ are certainly not Furthermore, what ‘apple’ and ‘chestnut’ name are living things These fascinating issues arise on the assumption that one is indeed perceiving a visual array consisting of nine objects with their shapes, contours, orientations, textures, colors and intricate spatial relationships But, of course, none of this is literally true Your visual system with the rest of your brain is playing a trick on you: there is no apple, no chestnut, no silvery beaker, no brown dish, no spoon, no table, no wall All there is is a canvas with two-dimensional shapes and patches of colors drawn on it How on earth does one see three apples, two chestnuts, a silvery beaker, a brown dish and a spoon when there are no such things to be seen? Or are there after all? This is the puzzle of visual art There is evidence that non-human animals (e.g monkeys) can see things (e.g plants or animals) in pictures (i.e in photographs) However, there is also evidence that when they do, they believe that there is a plant or an animal in front of them They not think that what they are seeing is a representation They react to the plant or the animal represented, not to a representation of a plant or an animal Human beings are different They too can be fooled by trompe-l’œil representations, but they have the joint ability to see what is in a picture and to see a picture as a picture In seeing [69] Even though neurons in the posterior parietal cortex of anesthesized animals receive visual inputs [70] See the end of the Epilogue for the expression of our doubts about the exclusive role of the ventral stream in conscious visual processing [71] Recent experiments by Bonatti et al (2002) suggest that 10-month old infants might make use of visual featural information to individuate and/or identify objects if the stimuli are human faces or human face-like [72] For discussion of Ungerleider and Mishkin’s (1982) distinction, see Chapter [73] It is important to notice the difference beetween Pylyshyn’s (2000a, 2000b) multiple-tracking task and Pisella et al.’s (2000) and Rossetti and Pisella’s (2000) pointing task The former is a perceptual task: it requires coding of objects’ locations in allocentric coordinates The latter is a visuomotor task: it requires coding of the target’s location in egocentric coordinates [74] For further elaboration on the dual role of the parietal lobes, see the Epilogue [75] Clearly, the processing and monitoring of information about eyemovement and head-movement occurs at a sub-personal level [76] In other words, the constraint of contrastive identification is satisfied by perceptual modes of presentation of such properties as size and shape, not by visuomotor modes of presentation of the same properties [77] See Chapter for elaboration [78] An intention cannot represent an action performed by another even if, as we shall argue in Chapter 7, the visual perception of an action performed by another can trigger in the observer an action plan isomorphic to the plan of the action whose execution is being perceived [79] As we noted above, Poincaré (1902/1968, 1905/1958) insightfully anticipates empirical psychophysical work on motor imagery when he wrote, e.g that one can represent one’s own movements by representing the muscular sensations necessary to execute them [80] The egocentricity of indexical concepts should not be confused with the egocentricity of an egocentric frame of reference in which the visual system codes the location of a target The former is a property of concepts The latter is a property of visual representations [81] Since the orbito-frontal cortex is not really part of visual cognition, we will not talk about it anymore [82] Arguably, there is a trade-off between the accuracy of a visually guided action and the richness of the content of the visual representation of the target The poverty of visuomotor representations might promote the accuracy of the action [83] Arguably, mindreading also involves psychological self-knowledge and the attribution of mental states to oneself Whether knowledge of one’s own mind and knowledge of other minds are acquired by the same path is a matter of intense scrutiny and discussion among psychologists and philosophers ... identification and recognition 2.1 The reliability of visual perception 2.2 Visual attributes and visual modes of perception 2.3 Recognitional concepts and prototypes The interaction of visual and non -visual. .. knowledge 3.1 Seeing and knowing 3.2 Primary and secondary epistemic seeing The scope and limits of visual knowledge 4.1 The justificatory role of vision in secondary epistemic seeing 4.2 Is visual. .. silvery beaker and falling sideways onto the apples, the chestnuts and the top of the spoon The apples cast their shadows on the table So the chestnuts The dish casts a shadow on the wall If you