Face Recognition Face Recognition Edited by Kresimir Delac and Mislav Grgic I-TECH Education and Publishing IV Published by the I-Tech Education and Publishing, Vienna, Austria Abstracting and non-profit use of the material is permitted with credit to the source. Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those of the editors or publisher. No responsibility is accepted for the accuracy of information contained in the published articles. Publisher assumes no responsibility liability for any damage or injury to persons or property arising out of the use of any materials, instructions, methods or ideas contained inside. After this work has been published by the Advanced Robotic Systems International, authors have the right to republish it, in whole or part, in any publication of which they are an author or editor, and the make other personal use of the work. © 2007 I-Tech Education and Publishing www.ars-journal.com Additional copies can be obtained from: publication@ars-journal.com First published June 2007 Printed in Croatia A catalog record for this book is available from the Austrian Library. Face Recognition, Edited by Kresimir Delac and Mislav Grgic p. cm. ISBN 3-86611-283-1 1. Face Recognition. 2. Face sinthesys. 3. Applications. V Preface Face recognition is a task humans perform remarkably easily and successfully. This appar- ent simplicity was shown to be dangerously misleading as the automatic face recognition seems to be a problem that is still far from solved. In spite of more than 20 years of extensive research, large number of papers published in journals and conferences dedicated to this area, we still can not claim that artificial systems can measure to human performance. Automatic face recognition is intricate primarily because of difficult imaging conditions (lighting and viewpoint changes induced by body movement) and because of various other effects like aging, facial expressions, occlusions etc. Researchers from computer vision, im- age analysis and processing, pattern recognition, machine learning and other areas are working jointly, motivated largely by a number of possible practical applications. The goal of this book is to give a clear picture of the current state-of-the-art in the field of automatic face recognition across three main areas of interest: biometrics, cognitive models and human-computer interaction. Face recognition has an important advantage over other biomet- ric technologies - it is a nonintrusive and easy to use method. As such, it became one of three identification methods used in e-passports and a biometric of choice for many other security applications. Cognitive and perception models constitute an important platform for inter- disciplinary research, connecting scientists from seemingly incompatible areas and enabling them to exchange methodologies and results on a common problem. Evidence from neuro- biological, psychological, perceptual and cognitive experiments provide potentially useful insights into how our visual system codes, stores and recognizes faces. These insights can then be connected to artificial solutions. On the other hand, it is generally believed that the success or failure of automatic face recognition systems might inform cognitive and percep- tion science community about which models have the potential to be candidates for those used by humans. Making robots and computers more "human" (through human-computer interaction) will improve the quality of human-robot co-existence in the same space and thus alleviate their adoption into our every day lives. In order to achieve this, robots must be able to identify faces, expressions and emotions while interacting with humans. Hopefully, this book will serve as a handbook for students, researchers and practitioners in the area of automatic (computer) face recognition and inspire some future research ideas by identifying potential research directions. The book consists of 28 chapters, each focusing on a certain aspect of the problem. Within every chapter the reader will be given an overview of background information on the subject at hand and in many cases a description of the au- thors' original proposed solution. The chapters in this book are sorted alphabetically, ac- cording to the first author's surname. They should give the reader a general idea where the VI current research efforts are heading, both within the face recognition area itself and in inter- disciplinary approaches. Chapter 1 describes a face recognition system based on 3D features, with applications in Ambient Intelligence Environment. The system is placed within a framework of home automation - a community of smart objects powered by high user-friendliness. Chapter 2 addresses one of the most intensely researched problems in face recognition - the problem of achieving illumination invariance. The authors deal with this problem through a novel framework based on simple image filtering techniques. In chapter 3 a novel method for pre- cise automatic localization of certain characteristic points in a face (such as the centers and the corners of the eyes, tip of the nose, etc) is presented. An interesting analysis of the rec- ognition rate as a function of eye localization precision is also given. Chapter 4 gives a de- tailed introduction into wavelets and their application in face recognition as tools for image preprocessing and feature extraction. Chapter 5 reports on an extensive experiment performed in order to analyze the effects of JPEG and JPEG2000 compression on face recognition performance. It is shown that tested recognition methods are remarkably robust to compression, and the conclusions are statisti- cally confirmed using McNemar's hypothesis testing. Chapter 6 introduces a feed-forward neural network architecture combined with PCA and LDA into a novel approach. Chapter 7 addresses the multi-view recognition problem by using a variant of SVM and decomposing the problem into a series of easier two-class problems. Chapter 8 describes three different hardware platforms dedicated to face recognition and brings us one step closer to real-world implementation. In chapter 9 authors combine face and gesture recognition in a human- robot interaction framework. Chapter 10 considers fuzzy-geometric approach and symbolic data analysis for modeling the uncertainty of information about facial features. Chapter 11 reviews some known ap- proaches (e.g. PCA, LDA, LPP, LLE, etc.) and presents a case study of intelligent face recog- nition using global pattern averaging. A theoretical analysis and application suggestion of the compact optical parallel correlator for face recognition is presented in chapter 12. Im- proving the quality of co-existence of humans and robots in the same space through another merge of face and gesture recognition is presented in chapter 13, and spontaneous facial ac- tion recognition is addressed in chapter 14. Based on lessons learned from human visual system research and contrary to traditional practice of focusing recognition on internal face features (eyes, nose, and mouth), in chapter 15 a possibility of using external features (hair, forehead, laterals, ears, jaw line and chin) is explored. In chapter 16 a hierarchical neural network architecture is used to define a com- mon framework for higher level cognitive functions. Simulation is performed indicating that both face recognition and facial expression recognition can be realized efficiently using the presented framework. Chapter 17 gives a detailed mathematical overview of some tradi- tional and modern subspace analysis methods, and chapter 18 reviews in depth some near- est feature classifiers and introduces dissimilarity representations as a recognition tool. In chapter 19 the authors present a security system in which an image of a known person is matched against multiple images extracted from a video fragment of a person approaching a protected entrance Chapter 20 presents recent advances in machine analysis of facial expressions with special attention devoted to several techniques recently proposed by the authors. 3D face recogni- tion is covered in chapter 21. Basic approaches are discussed and an extensive list of refer- VII ences is given, making this chapter an ideal starting point for researchers new in the area. After multi-modal human verification system using face and speech is presented in chapter 22, the same authors present a new face detection and recognition method using optimized 3D information from stereo images in chapter 23. Far-field unconstrained video-to-video face recognition system is proposed in chapter 24. Chapter 25 examines the results of research on humans in order to come up with some hints for designs of artificial systems for face recognition. Frequency domain processing and rep- resentation of faces is reviewed in chapter 26 along with a thorough analysis of a family of advanced frequency domain matching algorithms collectively know as the advanced corre- lation filters. Chapter 27 addresses the problem of class-based image synthesis and recogni- tion with varying illumination conditions. Chapter 28 presents a mixed reality virtual sys- tem with a framework of using a stereo video and 3D computer graphics model. June 2007 Kresimir Delac Mislav Grgic University of Zagreb Faculty of Electrical Engineering and Computing Department of Wireless Communications Unska 3/XII, HR-10000 Zagreb, Croatia E-mail: kdelac@ieee.org IX Contents Preface V 1. 3D Face Recognition in a Ambient Intelligence Environment Scenario 001 Andrea F. Abate, Stefano Ricciardi and Gabriele Sabatino 2. Achieving Illumination Invariance using Image Filters 015 Ognjen Arandjelovic and Roberto Cipolla 3. Automatic Facial Feature Extraction for Face Recognition 031 Paola Campadelli, Raffaella Lanzarotti and Giuseppe Lipori 4. Wavelets and Face Recognition 059 Dao-Qing Dai and Hong Yan 5. Image Compression Effects in Face Recognition Systems 075 Kresimir Delac, Mislav Grgic and Sonja Grgic 6. PCA and LDA based Neural Networks for Human Face Recognition 093 Alaa Eleyan and Hasan Demirel 7. Multi-View Face Recognition with Min-Max Modular Support Vector Machines 107 Zhi-Gang Fan and Bao-Liang Lu 8. Design, Implementation and Evaluation of Hardware Vision Systems dedicated to Real-Time Face Recognition 123 Ginhac Dominique, Yang Fan and Paindavoine Michel 9. Face and Gesture Recognition for Human-Robot Interaction 149 Md. Hasanuzzaman and Haruki Ueno X 10. Modelling Uncertainty in Representation of Facial Features for Face Recognition 183 Hiremath P.S., Ajit Danti and Prabhakar C.J. 11. Intelligent Global Face Recognition 219 Adnan Khashman 12. Compact Parallel Optical Correlator for Face Recognition and its Application 235 Kashiko Kodate and Eriko Watanabe 13. Human Detection and Gesture Recognition Based on Ambient Intelligence 261 Naoyuki Kubota 14. Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face 275 Simon Lucey, Ahmed Bilal Ashraf and Jeffrey F. Cohn 15. Measuring External Face Appearance for Face Classification 287 David Masip, Agata Lapedriza and Jordi Vitria 16. Selection and Efficient Use of Local Features for Face and Facial Expression Recognition in a Cortical Architecture 305 Masakazu Matsugu 17. Image-based Subspace Analysis for Face Recognition 321 Vo Dinh Minh Nhat and SungYoung Lee 18. Nearest Feature Rules and Dissimilarity Representations for Face Recognition Problems 337 Mauricio Orozco-Alzate and German Castellanos-Dominguez 19. Improving Face Recognition by Video Spatial Morphing 357 Armando Padilha, Jorge Silva and Raquel Sebastiao 20. Machine Analysis of Facial Expressions 377 Maja Pantic and Marian Stewart Bartlett 21. 3D Face Recognition 417 Theodoros Papatheodorou and Daniel Rueckert 22. Multi-Modal Human Verification using Face and Speech 447 Changhan Park and Joonki Paik [...]... and Facial Expression Recognition workflow 4 Face Recognition 3.1 Face Capturing As the proposed method works on 3D polygonal meshes we firstly need to acquire actual faces and to represent them as polygonal surfaces The Ambient Intelligence context, in which we are implementing face recognition, requires fast user enrollment to avoid annoying waiting time Usually, most 3D face recognition methods work... Handbook of Fingerprint Recognition, Springer, New York Medioni,G & Waupotitsch R (2003) Face recognition and modeling in 3D Prooceding of IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG 2003), pages 232-233, October 2003 Pan, G.; Han, S.; Wu, Z & Wang, Y (2005) 3D face recognition using mapped depth images, Proceedings of IEEE Workshop on Face Recognition Grand Challenge... Strintzis, M G (2003) Use of depth and color eigenfaces for face recognition, Pattern Recognition Letters, vol 24, No 9-10, pp 1427-1435, Jan2003 Xu, C.; Wang, Y.; Tan, t & Quan, L (2004) Automatic 3D face recognition combining global geometric features with local shape variation information, Proceedings of Sixth International Conference on Automated Face and Gesture Recognition, May 2004, pp 308–313 Wang, Y.;... ac.uk/~oa214/ 2 See http: / /synapse vit lit nrc ca/db/video/ faces /cvglab 24 Face Recognition variations in pose and motion, acquired at 25fps and 160 x 120 pixel resolution (face size 45 pixels), see Figure 9 (c) Faces96 the most challenging subset of the University of Essex face database, freely available from http://cswww.essex.ac.uk/mv/allfaces/ faces96 html It contains 152 individuals, most 18-20 years... variation in 3D face recognition, Proceedings of IEEE Workshop on Face Recognition Grand Challenge Experiments, June 2005 Enciso, R.; Li, J.; Fidaleo, D.A.; Kim, T-Y; Noh, J-Y & Neumann, U (1999) Synthesis of 3D Faces, Proceeding of International Workshop on Digital and Computational Video, DCV'99, December 1999 Hester, C.; Srivastava, A & Erlebacher, G (2003) A novel technique for face recognition using... Ramamurthy Bhagavatula, Yung-hui Li and Ramzi Abiantun 27 From Canonical Face to Synthesis An Illumination Invariant Face Recognition Approach 527 Tele Tan 28 A Feature-level Fusion of Appearance and Passive Depth Information for Face Recognition 537 Jian-Gang Wang, Kar-Ann Toh, Eric Sung and Wei-Yun Yau 1 3D Face Recognition in a Ambient Intelligence Environment Scenario Andrea F Abate,... 2005 Papatheodorou, T & Rueckert, D (2004) Evaluation of Automatic 4D Face Recognition Using Surface and Texture Registration, Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp 321-326, May 2004, Seoul, Korea Perronnin, G & Dugelay, J.L (2003) An Introduction to biometrics and face recognition, Proceedings of IMAGE 2003: Learning, Understanding, Information... neutral face plus ten expression variations and one face with beard In Figure 10-b are shown the results of one-to-many comparison between subject with beard and an answer set 12 Face Recognition composed of one neutral face and ten expressive variations Finally for the test reported in Figure 10-c the query was an expression variation or a face with beard, while the answer set could contain a neutral face. .. Person Authentication (AVBPA 2003), LCNS 2688, J Kittler and M.S Nixon, 62-70,2003 14 Face Recognition Bowyer, K.W.; Chang, K & Flynn P.A (2004) Survey of 3D and Multi-Modal 3D+2D Face Recognition, Proceeding of International Conference on Pattern Recognition, ICPR, 2004 Chang, K.I.; Bowyer, K & Flynn, P (2003) Face Recognition Using 2D and 3D Facial Data, Proceedings of the ACM Workshop on Multimodal... Deformation The two captured face images are aligned, combined and blended resulting in a color texture precisely fitting the reconstructed face mesh through the feature points previously extracted The prototype face mesh used in the dataset has about 7K triangular facets, and even if it is possible to use mesh with higher level of detail we found this resolution to be adequate for face recognition This is . Austrian Library. Face Recognition, Edited by Kresimir Delac and Mislav Grgic p. cm. ISBN 3-86611-283-1 1. Face Recognition. 2. Face sinthesys. 3. Applications. V Preface Face recognition is. for Face Recognition 031 Paola Campadelli, Raffaella Lanzarotti and Giuseppe Lipori 4. Wavelets and Face Recognition 059 Dao-Qing Dai and Hong Yan 5. Image Compression Effects in Face Recognition. Features for Face Recognition 183 Hiremath P.S., Ajit Danti and Prabhakar C.J. 11. Intelligent Global Face Recognition 219 Adnan Khashman 12. Compact Parallel Optical Correlator for Face Recognition