computer and machine vision theory, algorithms, practicalities (4th ed ) davies 2012 03 19 Cấu trúc dữ liệu và giải thuật

CuuDuongThanCong.com Computer and Machine Vision: Theory, Algorithms, Practicalities CuuDuongThanCong.com This book is dedicated to my family To my late mother, Mary Davies, to record her never-failing love and devotion To my late father, Arthur Granville Davies, who passed on to me his appreciation of the beauties of mathematics and science To my wife, Joan, for love, patience, support, and inspiration To my children, Elizabeth, Sarah, and Marion, the music in my life To my grandson, Jasper, for reminding me of the carefree joys of youth CuuDuongThanCong.com Computer and Machine Vision: Theory, Algorithms, Practicalities Fourth Edition E R DAVIES Department of Physics Royal Holloway, University of London, Egham, Surrey, UK AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO Academic Press is an imprint of Elsevier CuuDuongThanCong.com Academic Press is an imprint of Elsevier 225 Wyman Street, Waltham, 02451, USA The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK First edition 1990 Second edition 1997 Third edition 2005 Fourth edition 2012 Copyright r 2012 Elsevier Inc All rights reserved No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher Details on how to seek permission, further information about the Publisher’s permissions policies and arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein) Notices Knowledge and best practice in this field are constantly changing As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library ISBN: 978-0-12-386908-1 For information on all Elsevier publications visit our website at elsevierdirect.com Typeset by MPS Limited, a Macmillan Company, Chennai, India www.macmillansolutions.com Printed and bound in United States of America 12 11 10 CuuDuongThanCong.com Contents Foreword xxi Preface xxiii About the Author xxvii Acknowledgements .xxix Glossary of Acronyms and Abbreviations xxxiii CHAPTER Vision, the Challenge 1.1 Introduction—Man and His Senses 1.2 The Nature of Vision 1.2.1 The Process of Recognition 1.2.2 Tackling the Recognition Problem 1.2.3 Object Location 1.2.4 Scene Analysis 1.2.5 Vision as Inverse Graphics 1.3 From Automated Visual Inspection to Surveillance 10 1.4 What This Book is About 12 1.5 The Following Chapters 13 1.6 Bibliographical Notes 14 PART LOW-LEVEL VISION 15 CHAPTER Images and Imaging Operations 17 2.1 Introduction 18 2.1.1 Gray Scale Versus Color 19 2.2 Image Processing Operations 23 2.2.1 Some Basic Operations on Grayscale Images 24 2.2.2 Basic Operations on Binary Images 28 2.3 Convolutions and Point Spread Functions 32 2.4 Sequential Versus Parallel Operations 34 2.5 Concluding Remarks 36 2.6 Bibliographical and Historical Notes 36 2.7 Problems 36 CHAPTER Basic Image Filtering Operations 38 3.1 3.2 3.3 3.4 3.5 Introduction 38 Noise Suppression by Gaussian Smoothing 40 Median Filters 43 Mode Filters 45 Rank Order Filters 52 v CuuDuongThanCong.com vi Contents 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 CHAPTER Reducing Computational Load 54 SharpÀUnsharp Masking 55 Shifts Introduced by Median Filters 56 3.8.1 Continuum Model of Median Shifts 57 3.8.2 Generalization to Grayscale Images 59 3.8.3 Problems with Statistics 60 Discrete Model of Median Shifts 62 Shifts Introduced by Mode Filters 65 Shifts Introduced by Mean and Gaussian Filters 67 Shifts Introduced by Rank Order Filters 68 3.12.1 Shifts in Rectangular Neighborhoods 69 The Role of Filters in Industrial Applications of Vision 74 Color in Image Filtering 74 Concluding Remarks 76 Bibliographical and Historical Notes 77 3.16.1 More Recent Developments 78 Problems 79 Thresholding Techniques 82 4.1 4.2 4.3 Introduction 83 Region-Growing Methods 83 Thresholding 84 4.3.1 Finding a Suitable Threshold 85 4.3.2 Tackling the Problem of Bias in Threshold Selection 86 4.3.3 Summary 88 4.4 Adaptive Thresholding 88 4.4.1 The Chow and Kaneko Approach 91 4.4.2 Local Thresholding Methods 92 4.5 More Thoroughgoing Approaches to Threshold Selection 93 4.5.1 Variance-Based Thresholding 95 4.5.2 Entropy-Based Thresholding 96 4.5.3 Maximum Likelihood Thresholding 97 4.6 The Global Valley Approach to Thresholding 98 4.7 Practical Results Obtained Using the Global Valley Method 101 4.8 Histogram Concavity Analysis 106 4.9 Concluding Remarks 107 4.10 Bibliographical and Historical Notes 108 4.10.1 More Recent Developments 109 4.11 Problems 110 CHAPTER 5.1 5.2 CuuDuongThanCong.com Edge Detection 111 Introduction 112 Basic Theory of Edge Detection 113 Contents 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 CHAPTER 6.1 6.2 6.3 6.4 6.5 6.6 6.7 CuuDuongThanCong.com The Template Matching Approach 115 Theory of 3 Template Operators 116 The Design of Differential Gradient Operators 117 The Concept of a Circular Operator 118 Detailed Implementation of Circular Operators 120 The Systematic Design of Differential Edge Operators 122 Problems with the Above Approach—Some Alternative Schemes 123 Hysteresis Thresholding 126 The Canny Operator 128 The Laplacian Operator 131 Active Contours 134 Practical Results Obtained Using Active Contours 137 The Level Set Approach to Object Segmentation 140 The Graph Cut Approach to Object Segmentation 141 Concluding Remarks 145 Bibliographical and Historical Notes 146 5.18.1 More Recent Developments 147 Problems 148 Corner and Interest Point Detection 149 Introduction 150 Template Matching 150 Second-Order Derivative Schemes 151 A Median Filter-Based Corner Detector 153 6.4.1 Analyzing the Operation of the Median Detector 154 6.4.2 Practical Results 156 The Harris Interest Point Operator 158 6.5.1 Corner Signals and Shifts for Various Geometric Configurations .161 6.5.2 Performance with Crossing Points and Junctions .162 6.5.3 Different Forms of the Harris Operator 165 Corner Orientation 166 Local Invariant Feature Detectors and Descriptors 168 6.7.1 Harris Scale and Affine-Invariant Detectors and Descriptors 171 6.7.2 Hessian Scale and Affine-Invariant Detectors and Descriptors 173 6.7.3 The SIFT Operator 173 6.7.4 The SURF Operator 174 6.7.5 Maximally Stable Extremal Regions 176 6.7.6 Comparison of the Various Invariant Feature Detectors 177 vii viii Contents 6.8 Concluding Remarks 180 6.9 Bibliographical and Historical Notes 181 6.9.1 More Recent Developments 184 6.10 Problems 184 CHAPTER 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 CHAPTER 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 CuuDuongThanCong.com Mathematical Morphology 185 Introduction 185 Dilation and Erosion in Binary Images 186 7.2.1 Dilation and Erosion .186 7.2.2 Cancellation Effects 186 7.2.3 Modified Dilation and Erosion Operators 187 Mathematical Morphology 187 7.3.1 Generalized Morphological Dilation 187 7.3.2 Generalized Morphological Erosion 188 7.3.3 Duality Between Dilation and Erosion .189 7.3.4 Properties of Dilation and Erosion Operators 190 7.3.5 Closing and Opening 193 7.3.6 Summary of Basic Morphological Operations .195 Grayscale Processing 197 7.4.1 Morphological Edge Enhancement .198 7.4.2 Further Remarks on the Generalization to Grayscale Processing 199 Effect of Noise on Morphological Grouping Operations 201 7.5.1 Detailed Analysis 203 7.5.2 Discussion .205 Concluding Remarks 205 Bibliographical and Historical Notes 206 7.7.1 More Recent Developments 207 Problem 208 Texture 209 Introduction 209 Some Basic Approaches to Texture Analysis 213 Graylevel Co-occurrence Matrices 213 Laws’ Texture Energy Approach 217 Ade’s Eigenfilter Approach 220 Appraisal of the Laws and Ade Approaches 221 Concluding Remarks 223 Bibliographical and Historical Notes 223 8.8.1 More Recent Developments 224 Contents PART INTERMEDIATE-LEVEL VISION CHAPTER 227 Binary Shape Analysis 229 9.1 Introduction 230 9.2 Connectedness in Binary Images 230 9.3 Object Labeling and Counting 231 9.3.1 Solving the Labeling Problem in a More Complex Case 235 9.4 Size Filtering 238 9.5 Distance Functions and Their Uses 240 9.5.1 Local Maxima and Data Compression 243 9.6 Skeletons and Thinning 244 9.6.1 Crossing Number 247 9.6.2 Parallel and Sequential Implementations of Thinning 248 9.6.3 Guided Thinning 251 9.6.4 A Comment on the Nature of the Skeleton 251 9.6.5 Skeleton Node Analysis 251 9.6.6 Application of Skeletons for Shape Recognition 253 9.7 Other Measures for Shape Recognition 254 9.8 Boundary Tracking Procedures 257 9.9 Concluding Remarks 257 9.10 Bibliographical and Historical Notes 259 9.10.1 More Recent Developments 260 9.11 Problems 261 CHAPTER 10 Boundary Pattern Analysis 266 10.1 10.2 10.3 10.4 Introduction 266 Boundary Tracking Procedures 269 Centroidal Profiles 269 Problems with the Centroidal Profile Approach 270 10.4.1 Some Solutions 271 10.5 The (s, ψ) Plot 274 10.6 Tackling the Problems of Occlusion 276 10.7 Accuracy of Boundary Length Measures 279 10.8 Concluding Remarks 280 10.9 Bibliographical and Historical Notes 281 10.9.1 More Recent Developments 282 10.10 Problems 282 CHAPTER 11 11.1 11.2 11.3 CuuDuongThanCong.com Line Detection 284 Introduction 284 Application of the Hough Transform to Line Detection 285 The Foot-of-Normal Method 288 11.3.1 Application of the Foot-of-Normal Method 290 ix Subject Index A Abstract pattern matching See Pattern matching Active contours See Segmentation Active vision, Advanced driver assistance system (ADAS) See In-vehicle vision systems Agriculture example, egomotion and, 653À654 Algorithm design criteria, 76 accuracy, 279À280 adaptability, 760 cost, 542, 682À683 detection sensitivity, 368 reliability, 760 robustness, 315À316 speed, 314À315 tradeoffs, 354 Algorithmic parallelism, 744, 745 Animal tracking, 631 Articulated objects, 662 Artificial neural networks (ANNs), 701 Back-propagation algorithm, 705À708 credit assignment problem, 705 cross validation, 711 fixed increment rule, 702, 703, 704 Hebbian learning, 715 multilayer perceptron (MLP) networks, 708À709 noise suppression using, 40 self-organizing map (SOM), 670 spatiotemporal attention (STA), 665 validation set, 711, 712 WidrowÀHoff delta rule, 703À704 Automated visual inspection, 10À12 applications, 10 categories of, 530À532 of cereal grains, 553 of circular products, 533 using radial histograms, 533 color, importance of, 546 design of inspection systems, 757À760 in factories, 527 of food products, 528 inspection process, 527 optimized algorithm, 351 of precision components, 528À529 of printed circuit boards, 530 of products with high levels of variability, 539À542 shape deviations relative to standard template, 532À533 size measurement requirements, 529À530 of steel strips, 538À539 of three-dimensional objects, 530 of wood, 538À539 using X-rays, 546 Autonomous mobile robots See Robots, autonomous mobile B Backpropagation algorithm See Artificial neural networks Bayes’ decision theory, 676À679 Binary images and shape analysis, 229 boundary tracking procedures, 257 circularity, 229 compactness, 254 complexity, 254 concavity trees, 255 connectedness in, 230À231 convex deficiency, 255 convex hull, 255 crossing number χ, 247À248 distance functions, 240À243 image processing operations on, 257 measures for shape recognition, 253À254 metric properties, 238 modified crossing number χskel, 253, 259 moment approximations, 255 object labeling and counting, 231À235 sigma function σ, 248 size filtering, 238 skeletons and thinning, 245À254 Binocular images, 393À395 Blob size filtering, 659 Boundary pattern analysis, 266 accuracy of length measures, 279À280 centroidal profiles, 269À270 chain code, 281 Fourier descriptor method, 276 occlusion problems, 266 (r, θ) plot, 272, 274 (s, κ) plot, 272, 276 (s, ψ) plot, 274À276 Boundary tracking procedures, 269 Bubble sort, 44, 79 861 CuuDuongThanCong.com 862 Subject Index C Cameras, 732 digitization and, 732À735 line-scan, 731 Canny operator, 128 use of hysteresis thresholding, 128, 129 Centroidal profile See Boundary pattern analysis Cereal grains, inspection of, 553 dark contaminants, 555À560 high-speed grain location, 566À572 insects, 560À566 linear feature detection, 560 rodent droppings, 555À557 using sets of template masks, 572À575 ChordÀtangent method See Ellipse detection Circle detection, 314 accurate center location, 311À314 applications, 304À305 Hough transform and, 305À308 speed problem, overcoming, 314À320 unknown radius problem, 308À311 Circles, egomotion and centers of, 460À462 Circular operators, 118À119 Circular products, inspection of, 533À537 Cluster analysis See Statistical pattern recognition Color, 19 channel, 22, 52, 75, 590, 739 Color processing, 38 color bleeding, 52, 76 conversion to HSI, 547 distance-weighted median filter, 77 image filtering, 74À76 mode filter, 65À67 principal components analysis (PCA), 695À699 use in inspection, 546À548 value of, 20, 21, 22 vector median filter, 76, 78 Computational load, calculating, 339À342 generalized Hough transform, 334À335 maximal cliques, 361, 371À373 reducing, 54À55 Computer vision, 13 Connectedness, in binary images, 230À231 Convolutions, 32À34 Corner detection, 149 See also Feature, invariant Beaudet operators, 152 DET operator, 152 determining orientation, 166À168 DreschlerÀNagel (DN) operator, 153 generalized Hough transform, 369 CuuDuongThanCong.com Harris operator, 158À166 KitchenÀRosenfeld (KR) operator, 153 median-based operator, 153À158 Plessey operator, 165 second-order derivative schemes, 151À153 SUSAN operator, 181 template matching, 150À151 ZunigaÀHaralick (ZH) operator, 153 Corner properties, 149 bluntness, 151 contrast, 154 location, 166À167 orientation, 166À168 pointedness, 151 sharpness, 151 Counting, object, 231À238 Cross ratio, 439 Chasles’ theorem, 450À452 cross ratio spectra, 475 cross ratio functions, symmetric, 454À456 5-point configuration, 447À449 ratio of ratios, 441À445 Crossing number χ, 247À248 D Data parallelism, 744, 745 Detection, 15 See also Circle detection; Corner Detection; Edge detection; Ellipse detection; Eye detection; Facial feature detection; Insect detection; In-vehicle vision systems; Iris detection; Line detection; Surveillance contaminant, 555 crack, 92 defect, 109, 193 foreign object, 539À541 hole, 327À328 interest point, 149 laparoscopic tool, 297À298 line segment, 300, 560, 561 linear feature, 560À563 optimal, 353 parabola, 330, 640 people, 662 polygon, 335 salient feature, 378À379 shadow, 721À724 vanishing point, 458À460 Diameter bisection method See Ellipse detection Differential invariants, 452À454 Digitization, cameras and, 732À735 Subject Index Dilation, 186 duality between erosion and, 189À190 inspection of cereal grains, 556, 557 properties of operators, 190À193 Discrete model of median shifts, 62À65 Discrete model of rank order filters, 52À53 Discrete relaxation, 376 Distance functions, 240À244 Distortion, 733 barrel, 488 edge shift, 78 foreshortening, 469 optical, 769 perspective, 359À360 pincushion, 489 radial, 488À490 Driver assistance system See In-vehicle vision systems comparison of methods, 347À348, 349 determining parameters, 323À325 diameter bisection method, 320À322 generalized Hough transform method, 343À347 reducing computational load for, 347À348 superellipse, 321, 322, 329 triple bisection algorithm, 570À571 Ellipses, perspective and centers of, 460À462 Epipolar lines, 396 generalized epipolar geometry, 491À492 Erosion, 186 duality between dilation and, 189À190 inspection of cereal grains, 556, 557 properties of operators, 190À193 Essential matrix, 492À495 Euclidean metric, 245 Extrinsic camera parameters, 486 Eye detection, 463À465, 475 E F Edge, 111 planar, 112, 113 roof, 113 step, 112, 113 Edge detection, 111 advantages of, 111 alternative schemes, 123À126 basic theory of, 113À115 in binary images, 230 Canny operator, 128À134 difference of Gaussians (DoG), 173À174 differential gradient (DG), 117À118 integrated directional derivative (IDD), 125 Kirsch operator, 111 Laplacian of Gaussian (LoG), 171 Laplacian operator, 134À135 MarrÀHildreth operator, 125À126, 146 morphological gradient operator, 201 non-maximum suppression, 153, 157, 158 orientation, 111, 114 Prewitt operator, 117À118, 124 Reeves moment-based operator, 124 Roberts operator, 117 Robinson 3-level operator, 115 Robinson 5-level operator, 115 Sobel operator, 117À118, 121, 129, 135 template matching (TM), 115À116 YuilleÀPoggio operator, 146 Egomotion See In-vehicle vision systems; Robots, autonomous mobile Ellipse detection, 303, 320À325 chordÀtangent method, 322À323 Face recognition, 462À464, 475 Facial feature detection, 224, 325À326, 330, 463À465, 645À647 Fast Fourier transform (FFT), 39, 754 Feature, invariant, 168 affine invariant, 173 gradient location and orientation histogram (GLOH), 177 Harris-based, 170À173 Hessian-based, 173 intensity extrema-based region detector (IBR), 177 maximally stable extremal region (MSER), 176À177 scale invariant, 178 scale-invariant feature operator (SFOP), 177, 179 scale invariant feature transform (SIFT), 173À174 speeded-up robust features (SURF), 174À176 use for wide baseline matching, 519À521 Feature collation, 369À371 Feature detection, 560 corner, 149 edge, 113 hole, 327À328 interest point, 149 line segment, 560 salient, 378À379 Filters, 38 anisotropic diffusion, 552 applications, 74 CuuDuongThanCong.com 863 864 Subject Index Filters (Continued) color, 74À76 color bleeding and, 56, 76 computational load, reducing, 54À55 distance-weighted median, 77 edge-preserving smoothing, 552 Gaussian, 41, 54 hybrid median, 77 Kalman, 517À519 limit, 43 low-pass, 39À43 matched, 336 maximum, 53 mean, 68 median, 43À45 corner detector, based on, 81 minimum, 53 mode, 45À52 noise suppression by Gaussian smoothing, 40À42 particle See Tracking moving objects rank order, 52À53 sharpÀunsharp masking, 55À56 shifts introduced by, 67 mean and Gaussian filters, 67À68 median filters, 56À62 discrete model of, 62À65 mode filters, 65À67 rank order filters, 68À74 spatial, 336 spatial frequency, 40À41 switched, 78 truncated median, 48, 50 vector median, 52, 76 Feature location See Feature detection Focus of contraction, 509 Focus of expansion (FoE), 509, 510, 511À512, 521 Food products, inspection of, 528 cereal grains, 553 color, importance of, 546À548 nematode worms in fish, 739 Foot-of-normal method, 288À290 Frame store, 23À24, 35 Full perspective projection, 418, 429, 435, 436, 461 Fundamental matrix, 495À496 G Gaussian distributions, 47, 88, 296À297, 548, 779, 780, 783 Gaussian filters See Filters CuuDuongThanCong.com Gaussian smoothing, 40À42 Gaussian sphere, vanishing point detection and, 457À458 Generalized Hough transform (GHT), 334À335 basic, 334À335 computational load, 370À371 feature collation and, 369À370 gradient versus uniform weighting, 339À342 line detection and, 285À288 polygon detection and, 335 problems and setting up, 336 reducing computational load, 54À55 sensitivity and computational load, calculating, 339À342 spatial matched filtering, 336À338 Genetic algorithms (GAs), 109, 207 Gradient weighting versus uniform weighting, 339À342 Gray-level co-occurrence matrices, 202 Gray-tone (gray-scale) images, 18 discrete model of median shifts, 62À65 generalized morphology, 59À60 image processing operations on, 23À32 versus color, 19À22 H Hamming distance, 7, 674, 675 Hardware, 11 design of inspection systems, 757À760 digital signal processing (DSP) chip, 754, 756 field programmable gate array (FPGA), 754, 756 dynamic reconfigurability, 764 Flynn’s classification, 748À750 graphics processing unit (GPU), 757, 765 Kinect human motion capture system, 766 multiple instruction, multiple data (MIMD) stream, 748À749 multiple instruction, single data (MISD) stream, 748À749 optimal implementation, 750À754 options, 754À755 pipelined processor, 747À748 N processors, speed gain using, 747À748 real-time, 754À755 single instruction, multiple data (SIMD) stream, 745À747 single instruction, single data (SISD) stream, 746, 748 specification and design, 751À752 very large scale integration (VLSI), 757, 763, 764 Subject Index Harris interest point detector, 158À166 Hole detection, 327 Homogeneous coordinates, 478, 481 Homography, 466, 467, 475, 497, 611À613, 617, 618, 795 Hough transform (HT), 285 See also Generalized Hough transform agriculture application, 658 fast, 351 GerigÀKlein back-projection technique, 352À353 nature of, 333À342 uses, 304 circle detection, 304À308, 328 corner detection, 166À168 ellipse detection, 320À325, 328 line detection, 285À288 superellipse detection, 329 vanishing point detection, 456À458 xy-grouping, 290À291 HumanÀcomputer interaction, HCI, 765 Human gait analysis, 626À628 Hyperspectral cube, 738 Hyperspectral imaging, 577, 738À739 I Illumination schemes, 719À732 infinite parallel strip lights example, 726À729 line-scan cameras, 730À731 overview of uniform illumination, 729À730 producing uniform illumination, 724À726 shadows, eliminating, 721À724 Image acquisition, 718 cameras and digitization, 732À735 illumination schemes, 719À732 sampling theorem, 735À738 Image differencing, 506, 521 Image filters (filtering) See Filters Image parallelism, 745À746 Image processing, 6, 13, 15 applying convolutions, 32À34 applying logical operations, 287 brightening, 17 clearing, 24 displaying, 24 expanding, 29 inverting, 24, 27 sequential versus parallel operations, 34À35 shifting, 17 shrinking, 28 size filtering, 238À240 CuuDuongThanCong.com suppressing noise, 38 on binary images, 38 on gray-scale images, 39 Image segmentation See Segmentation Imaging modalities, 718 color, 547À548 HSI, 547 RBG, 547 hyperspectral, 577, 738À739 infra-red (NIR), 576, 577, 635, 666, 698, 738À739 multispectral, 698, 738 thermal, 577, 634, 635, 666, 668 visible, 397 X-ray, 540 dual-emission X-ray absorptiometry (DEXA), 546 In-vehicle vision systems, 636 See also Robots, autonomous mobile; Surveillance; Tracking moving objects advanced driver assistance system (ADAS), 663À671 all hoursÀall weathers, 666 catadioptric cameras, 669 convoy, 638 global positioning system (GPS), 638, 670 ground plane, location and use of, 653À654 licence plate location, 647À649 omnidirectional cameras, 669 pedestrian location, 650À653 chamfer matching, 650 skin color, 652 road lane marking location, 640À641 RANSAC, 641 road sign, location, 641À644 chamfer matching, 644 matched filter, 643 roadway location, 638À640 use of vanishing points (VPs), 458À460 vehicle location, 644À645 under-vehicle shadow, 645 Industrial parts, location of, 415À417 Insect detection, 576 Inspection See Automated visual inspection Intrinsic camera parameters, 486 Invariant feature See Feature, invariant Invariants, 439 See also Feature, invariant cross ratio, 441À445 functions, symmetric, 454À456 5-point configuration, 654 spectra, 475 865 866 Subject Index Invariants (Continued) defined, 441 differential and semidifferential, 452À454 for noncollinear points, 445À449 for points on conics, 449À452 reasons for using, 439 Inverse graphics, 9À10 Iris detection, 224, 325À326, 330, 463, 645 use to estimate eye Gaze direction, 325 J Junction orientation technique, 411À415 K Kalman filter See Tracking moving objects L Labeling, 231À238 object, 231À238 relaxation, 379 Laparoscopic tools, 297À298 location of tips, 298 location using RANSAC, 297À298 Laplacian operator, 134À135 Laws’ texture energy approach, 217À220 Learning See Artificial neural networks; Statistical pattern recognition Least median of squares, 787À790 Least squares analysis, 779 Light emitting diode (LED) light sources, 731À732 Light striping, 396 Line detection, 284 final line fitting, 292À293 foot-of-normal method, 288À290 generalized Hough transform and, 285À288 Hough transform and, 285À288 longitudinal localization, 290À292 RANSAC, 293À297 slope-intercept equation, 285À286 Line-scan cameras, 730À731 Linear feature detection, 560À563 Local maximum operation, 293 Local minimum operation, 199, 201 Longitudinal line localization, 290À292 M Machine vision, 12 See also Automated visual inspection applications, 769 CuuDuongThanCong.com defined, 768 future for, 768À769 importance of, 768À770 tradeoffs, 770À772 Mathematical morphology, 187 closing, 193À195 connectivity-based analysis, 195À196 dilation generalized, 187À188 duality between dilation and erosion, 189À190 erosion generalized, 188À189 gray-scale processing, 197À201 maximum, 199 minimum, 199 morphological analysis, 537, 552, 555, 558, 576, 577, 589, 590, 592, 619, 663, 670, 764, 776 morphological gradient, 201À205 noise, effects of, 201À205 opening, 193À195 residue function, 193 template matching, 206 top hat operator, 193 black, 193 white, 193 umbra homomorphism theorem, 199À200 Matrix (matrices), 497 essential, 492À495, 496À497 fundamental, 495À497 gray-level co-occurrence, 214À217 Maximal clique, 355, 361 algorithm, 367 computational load, 370À371 concept, 361À362 generalizing, 371À373 Mean filters See Filters Median filters See Filters Metric properties, in digital images, 393 Mobile robots See Robots, autonomous mobile Mode filters See Filters Moment approximations, 255 Moore’s law, 754, 772À773 Morphology See Mathematical morphology Motion, 504 See also Surveillance; Tracking moving objects aperture problem, 506 focus of expansion (FOE), 511À512 human gait analysis, 626À628 Kalman filters, 517À519 optical flow, 506À509 snakes, 633 Subject Index stereo from, 515À517 time-to-adjacency analysis, 513À514 traffic flow monitoring, 614À618 Multiple-view vision, 490À491 N Nasty realities, 768 clutter, 768 glint, 769 noise, 768 Gaussian, 769 impulse, 769 white, 336À337 occlusion, 621 shadow, 721À722 Navigation, robots See Robots, autonomous mobile Near infra-red (NIR), 576 Nearest neighbor algorithm, 674À676 Neighborhood parallelism, 745À746 Noise, 40 morphological grouping operations and effects of, 201À205 spike, 42 white, 336À337 Noise suppression, 40 artificial neural networks and, 701À705 Gaussian smoothing, 40À42 median filters, 43À45 mode filters, 45À52 rank order filters, 52À53 Noncollinear points invariants for, 445À449 O Object labeling and counting, 231À238 Object location See Detection Object recognition schemes, 3-D, 410À411 Occlusion, 606 apparent occlusion, 620 dynamic occlusion, 620 occlusion reasoning, 616 problems, 276À279, 616 scene occlusion, 620 Optical flow, 506À509 interpretation of, 509À511 problems with, 514À515 Optimizing network architecture, 764 Overfitting training data, 709À712 P Parallel image processing operations, 34À35 Parallel thinning, 250 CuuDuongThanCong.com Particle filter See Tracking moving objects Pattern matching, 358 affine matching, 380 Hausdorff distance, 381 cream biscuits example, 363À366 feature collation and use of generalized Hough transform, 354 local-feature-focus (LFF), method, 368 graph matching, 358 maximal clique algorithm, 367 maximal clique concept, 371 reproducible kernel Hilbert space (RKHS), 380 spectral graph theory, 380 relational descriptors, 373À376 relaxation labeling, 376 search space, 376À377 similarity measures, 379, 380 Bhattacharyya coefficient, 604 Pattern recognition See Statistical pattern recognition People tracking, 579 applications, 579 basic techniques, 620 from vehicles, 620 Performance measures, 177À179, 183, 686 accuracy, 688 area under curve (AUC), 688, 717 discriminability, 688 F-measure, 688, 716 false negative (FN), 687 false negative rate (fnr), 687 false positive (FP), 687 false positive rate (fpr), 687 precision, 688 recall, 687 receiver operating characteristic (ROC), 147, 667, 688, 716 sensitivity, 687 sorting optimization curve (SOC), 716 specificity, 688 true negative (TN), 686 true negative rate (tnr), 686 true positive (TP), 686 true positive rate (tpr), 686 Personal computers, 756 Perspective, 466 in art, 466À472 vanishing point, 456À458 Perspective inversion, 425À427 Perspective projection, 392 full, 429, 431 3-point problem, 433À434 867 868 Subject Index Perspective projection (Continued) symmetric trapezia problem, 434 weak, 425, 427À429, 431 Phong model, 402 Photometric stereo, 402À405 Plan view of ground plane, 654 constructing, 654 PointÀline duality, 285 Point pattern matching See Pattern matching, graph matching Point spread functions (PSFs), 32À34 Principal components analysis (PCA), 695À699 Principal point, 486 Probabilistic relaxation, 376 Probability and image analysis, 699 Projection schemes, 3-D, 392À398 PROLOG, 376 Propagation, 231 R Radial distortions, 479, 495 correcting, 488À490 Rank order filters, 52À53 shifts introduced by, 68À74 RANSAC (random sample consensus) approach, 292À297, 301, 640, 641, 668, 791À792 Raw pixel measurements, 495 Real-time operation, 523, 602, 670 Receiver operating characteristic (ROC), 684À688 Recognition See Statistical pattern recognition Region-growing methods See Segmentation Region of interest (RoI), 325 Relational descriptors, 373À376 Relaxation labeling, 376, 379 (r, θ) plot See Boundary pattern analysis Road, 640, 668 lane markings, 640, 641, 642, 766 location, 640À644 signs, 641À644 Robots, autonomous mobile, 653 active vision, agriculture application example, 656 applications, 74 centers of circles and ellipses, 460 cross ratios, 474 distance function navigation for, 479 plan view of ground plane, constructing, 654 safety issues, 579 vanishing point detection, 474 vehicle guidance, 656 Robust estimator See Robust statistics CuuDuongThanCong.com Robust statistics, 778 N adjacent points sample consensus (NAPSAC), 794 beta [distribution] sampling consensus (BetaSAC), 795 breakdown point, 780À782 features from accelerated segment test (FAST), 177 group sampling consensus (GroupSAC), 795 Hough transform and, 778 importance sampling consensus (IMPSAC), 794À795 influence function, 783À787 inlier, 782, 791 L-estimator, 782, 789 least median of squares (LMedS), 787À790 least squares regression, 782, 787, 790 M-estimator, 782, 789 outlier, 782, 783, 784, 785, 793, 795 progressive sample consensus (PROSAC), 795 R-estimator, 782, 789 random sample consensus (RANSAC), 791À792 relative efficiency, 781 Robustness, 179, 315À316, 790 S Salt and pepper noise, 30, 31 Sampling theorem, Nyquist, 735À738 Scaled orthographic projection, 425, 427, 435 Scene analysis, 8À9 Schmitt trigger, 126 Search space, 269, 310 Semidifferential invariants, 452À454 Segmentation, 82 See also Thresholding (threshold) active contour, 136 maximum a posteriori (MAP) modeling, 147À148 deformable contour, 136 graph cuts, 421 flowÀmax cut theorem, 142 residual network, 144 level set, 140À141 fast marching method, 141 region-growing, 83À84 scattergrams, use of, 86À87 snake, 136, 614 greedy algorithm, 138 Sequential image processing operations, 34À35 Sequential labeling, 235 Subject Index Sequential pattern recognition, 699 Sequential thinning, 249 Shading, shape from, 398À402 Shadows, 474 detecting, 719 eliminating, 721À724 Shape recognition, 254 See also Binary images and shape analysis from angle, 411 moment approximations, 255 from shading, 398À402 simple measures for, 229 skeletons and, 253À254 from texture, 407À408 (s, κ) plot See Boundary pattern analysis (s, ψ) plot See Boundary pattern analysis Simplex algorithm, 630 Singular value decomposition (SVD), 380, 497 Skeletons and thinning, 244 crossing number χ, 247À248 defined, 244 guided, 251 modified crossing number χskel, 253 nature of skeleton, 251 node analysis, 251À253 shape analysis using, 253 sigma function σ, 248 thinning implementations, 248À251 Snakes See Active contours Spatial matched filtering, 336À337 Speed gain using N processors, 747À748 Statistical pattern recognition (SPR), 672 See also Artificial neural networks; Performance measures AdaBoost, 715 bag-of-words, 521 bagging, 714 Bayes’ decision theory, 676À678 boosting, 714 cluster analysis, 691, 692 agglomerative algorithms, 693, 694 divisive algorithms, 693, 694 iterative self-organizing data analysis (ISODATA), 694, 713 MacQueen’s k-means algorithm, 695 noniterative clustering algorithms, 695 conditional risk, 682 cost functions, 682À683 distinct class based splitting measure (DCSM), 716 error-reject tradeoff, 682À683 face recognition, 462463 CuuDuongThanCong.com multiple classifiers, 688691 naăve Bayes, 678679 nearest neighbor (NN) algorithm, 674À676 optimum number of features, 681À682 overfitting to training data, 709À712 principal components analysis, 695À699 probability, relevance of, 699 supervised learning, 691À692 support vector machine, 700À701 undertraining, 710 unsupervised learning, 691À692 Stereo from motion, 515À517 Stitching photographs, 439, 441, 470À472, 475 Straight edge detection, 288 Stretching image contrast, 114, 156 Structured lighting, 408 SubgraphÀsubgraph isomorphism, 360, 378 Sudden step-edge response, 112, 113 Support vector machine (SVM), 700À701 Surveillance, 10, 578 See also In-vehicle vision systems; Tracking moving objects articulated bodies, analyzing motions of, 634 iterative parsing, 634 background modeling, 585 expectation maximization (EM) algorithm, 589 fluttering vegetation, 586, 587, 589 Gaussian mixture model (GMM), 588 non-parametric model, 593 parametric model, 590À593 field of view (FoV), 583, 609 foreground detection, 584 ghost, 586, 590, 591, 632 ground plane, location and use of, 609, 610, 611 in-plane rotation, 600 licence plate location, 618À620 monitoring traffic flow, 614À618 motion distillation, 623 rigidity parameter, 624, 625 multiple cameras, 609 non-overlapping fields of view, 613À614 overlapping fields of view, 613 transition probability, 595 occlusion reasoning, 607À609, 620À623 out-of-plane rotation, 600 pedestrian location, 651, 652, 662 chamfer matching, 607À609 histogram of orientated gradients (HOG), 668 869 870 Subject Index Surveillance (Continued) human gait analysis, 626À628 minimum description length (MDL) approach, 607 people location, 613 shadow suppression, 590 stationary background problem, 590 traffic flow monitoring, 614 Bascle method, 614À615 Koller method, 615À618 transient background problem, 590 use of color, 599 chromaticity coordinates, 592 chrominance parameters, 601 color histograms, 599À603 color indexing, 599À600 vehicle location, 644À646 Symmetry, 115, 322 mirror symmetry, 356 reflection symmetry, 645 rotation symmetry, 360 symmetric object, 366 symmetry detection, 645 System design, 451, 638 inspection systems, 538, 757 optimization, 742 T Template matching (TM), 7, 8, 112, 115 boundary pattern analysis and, 269 coarseÀfine, 273À274 corner detection and, 150À151 design of directional masks, 575 edge detection and, 115À116 equal area rule, 576 hole detection and, 327 matched filter, 378 multistage, 774 2-stage, 771À772, 775 tradeoffs, 771À772 Texture, 201À202, 209 defined, 210 fractal-based measures of, 223 Markov random field models of, 223 shape from, 223 texel, 210À211 Texture analysis Ade’s eigenfilter approach, 220À221 autocorrelation approach, 213 gray-level co-occurrence matrices, 214À217 Laws’ texture energy approach, 217À220 CuuDuongThanCong.com spatial gray-level dependence matrix (SGLDM) approach, 214 structural approaches to, 221À222 Thermal imaging, 526, 577 Thinning, 244 crossing number χ, 247À248 guided, 251 implementations, 248À251 modified crossing number χskel, 253 sigma function σ, 248 Three-dimensional (3-D) analysis, 389 ambiguity, 390 BallardÀSabbah method, 420 camera calibration, 418 eight-point algorithm, 497 essential matrix, 492 fundamental matrix, 495 generalized epipolar geometry, 491À492 homogeneous coordinates, 481, 483 homography, 611 Horaud’s junction orientation technique, 411À415 image reconstruction, 500 image rectification, 498À499 image transformations, 479 industrial parts, location of, 415À417 intrinsic and extrinsic camera parameters, 486 methods for studying, 359 multiple-view vision, 490À491 object recognition schemes, 410À411 perspective n-point (PnP) problem, 301, 436 photometric stereo, 402À405 pose estimation, 437, 794 use of coplanarity, 493 projection schemes, 392À393 radial distortions, correcting, 488À490 shape from angle, 411 shape from shading, 398À402 bidirectional reflectance distribution function (BRDF), 421 smoothness, surface, 405À407 shape from texture, 407À408 Silberberg method, 420 structure from motion, 514, 521 structured lighting, 408À410 surface smoothness, 405À407 transformation parameters, 484 triangulation, 500 Three-dimensional (3-D) objects, 530 inspection of, 410 Thresholding (threshold), 82 See also Segmentation Subject Index adaptive, 88 between-class variance method (BCVM), 95À96, 106 bias when selecting, 86À87 Chow and Kaneko approach, 91 dynamic, 88, 91 entropy-based, 96 finding a suitable, 85À86 global valley method (GVM), 98À101 hysteresis, 93 images, 83, 91 local, 92À93 maximum likelihood, 97 in unimodal distributions, 91 concavity analysis, 106 variance-based, 95 Time-to-adjacency analysis, 513À514 Top-hat operator, 193 Tracking moving objects, 517 See also Surveillance animal tracking, 631 Kalman filter, 517À519 mean shift algorithm, 599 monitoring traffic flow, 578 particle filter, 594À599 auxiliary particle filter (APF), 597 Condensation, 597 cumulative distribution function (CDF), 597 iCondensation, 597 iterated likelihood weighting (ILW), 598 kernel particle filter, 598 Epanechnikov kernel, 599 sample impoverishment, 596 sampling importance resampling (SIR), 596 sequential importance sampling (SIS), 595 people tracking, 579 Leeds people tracker, 579 SiebelÀMaybank tracker, 633 from vehicles, 580 Traffic flow monitoring See Surveillance Training data, overfitting to, 709À712 Transform (Transformation), 479 See also Hough transform affine, 498 maintenance of convexity, 380 degrees of freedom (DoF), 390, 410 CuuDuongThanCong.com Euclidean, 452 Fourier, 39À40, 736, 754 Radon, 300 similarity, 170 trace, 355À356 U Umbra homomorphism theorem, 199, 200 V Vanishing point, 458À460 See also In-vehicle vision systems detection, 456À458 use to find circle and ellipse centers, 460À462 Vehicle guidance See In-vehicle vision systems; Robots, autonomous mobile Video analytics See Surveillance Vision, See also Automated visual inspection; Machine vision active, human, 1À2, 663 multiple-view, 490À491 Vision, nature of, inverse graphics, 9À10 object location, 6À8 recognition, scene analysis, 8À9 W Weak perspective projection, 427À429 White noise, 336 Wide baseline, 519 matching, 519 use of invariant feature detector, 521 views, 520 Window operation, 26, 35, 38, 147, 196 X X-ray inspection, 542À546 dual-emission X-ray absorptiometry (DEXA), 546, 552 871 PLATE (FIG 23.15) Value of color in agricultural applications In agricultural scenes such as this, color helps with segmentation and with recognition It may be crucial in discriminating between weeds and crops if selective robot weedkilling is to be carried out Source: r World Scientific 2000 PLATE (FIG 2.2) Value of color for segmentation and recognition In natural outdoor scenes such as this, color helps with segmentation and with recognition While it may have been important to the early human when discerning sources of food in the wild, robot drones may benefit by using color to aid navigation CuuDuongThanCong.com PLATE (FIG 2.3) Value of color in the built environment Color plays an important role for the human in managing the built environment In a vehicle, a plethora of bright lights, road signs and markings (such as yellow lines) are coded to help the driver: they may likewise help a robot to drive more safely by the provision of crucial information PLATE (FIG 2.4) Value of color for food inspection Much food is brightly colored, as with this Japanese meal While this may be attractive to the human, it could also help the robot to check quickly for foreign bodies or toxic substances CuuDuongThanCong.com (a) (b) (c) (d) PLATE (FIG 3.12) Color filtering of brightly colored objects (a) Original color image of some sweets (b) Vector median filtered version (c) Vector mode filtered version (d) Version to which a mode filter has been applied to each color channel separately Note that (b) and (c) show no evidence of color bleeding, though it is strongly evident in (d) It is most noticeable as isolated pink pixels, plus a few green pixels, around the yellow sweets For further details on color bleeding, see Section 3.14 Source: r RPS 2004 (a) (b) (c) PLATE (FIG 3.13) Color filtering of images containing substantial impulse noise (a) Version of the Lena image containing 70% random color impulse noise (b) Effect of applying a vector median filter, and (c) effect of applying a vector mode filter While the mode filter is designed more for enhancement than for noise suppression, it has been found to perform remarkably well at this task when the noise level is very high Source: r RPS 2004 CuuDuongThanCong.com (a) (b) (c) (e) (d) (f) PLATE (FIG 23.11) Another approach to pedestrian location via skin color detection (a) and (b) show that a lot can be achieved via skin color detection, detecting not only faces but also neck, chest, arms and feet: see also the detail in (c) and (d) With proper color classifier training, even more can be achieved, as shown in (e) and (f) CuuDuongThanCong.com ... Bibliographical and Historical Notes 502 18.16.1 More Recent Developments 503 18.17 Problems 504 CHAPTER 19 19.1 19. 2 19. 3 19. 4 19. 5 19. 6 19. 7 19. 8 19. 9 19. 10 19. 11 19. 12 PART Motion... detection He has published more than 200 papers and three books? ?Machine Vision: Theory, Algorithms, Practicalities (199 0), Electronics, Noise and Signal Recovery (199 3), and Image Processing for... papers from Image and Vision Computing as text in Chapters and 14; as Tables 5.1À5.5; and as Figures 3.29, 5.2, 14.1, 14.2, 14.6: Davies (198 4b, 198 7c) Davies, E.R (199 1) Image and Vision Computing