Geometrical environment understanding by building recognition for the intelligent transportation and robot systems

GEOMETRICAL ENVIRONMENT UNDERSTANDING BY BUILDING RECOGNITION FOR THE INTELLIGENT TRANSPORTATION AND ROBOT SYSTEMS By Hoang-Hon Trinh SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY AT UNIVERSITY OF ULSAN ULSAN, KOREA DECEMBER 2008 c Copyright by Hoang-Hon Trinh, 2008 ° UNIVERSITY OF ULSAN DEPARTMENT OF GRADUATE SCHOOL OF ELECTRICAL ENGINEERING The undersigned hereby certify that they have read and recommend to the Faculty of Graduate Studies for acceptance a thesis entitled “Geometrical Environment Understanding by Building Recognition for the Intelligent Transportation and Robot Systems ” by Hoang-Hon Trinh in partial fulfillment of the requirements for the degree of Doctor of Philosophy Dated: December 2008 ii Committee Vice-chair Dr.: Research Supervisor: Kang-Huyn Jo Committee Member Dr.: iii UNIVERSITY OF ULSAN Date: December 2008 Author: Hoang-Hon Trinh Title: Geometrical Environment Understanding by Building Recognition for the Intelligent Transportation and Robot Systems Department: Graduate School of Electrical Engineering Degree: Ph.D Convocation: December Year: 2008 Permission is herewith granted to University of Ulsan to circulate and to have copied for non-commercial purposes, at its discretion, the above title upon the request of individuals or institutions Signature of Author THE AUTHOR RESERVES OTHER PUBLICATION RIGHTS, AND NEITHER THE THESIS NOR EXTENSIVE EXTRACTS FROM IT MAY BE PRINTED OR OTHERWISE REPRODUCED WITHOUT THE AUTHOR’S WRITTEN PERMISSION THE AUTHOR ATTESTS THAT PERMISSION HAS BEEN OBTAINED FOR THE USE OF ANY COPYRIGHTED MATERIAL APPEARING IN THIS THESIS (OTHER THAN BRIEF EXCERPTS REQUIRING ONLY PROPER ACKNOWLEDGEMENT IN SCHOLARLY WRITING) AND THAT ALL SUCH USE IS CLEARLY ACKNOWLEDGED iii iv Table of Contents Table of Contents v List of Tables viii List of Figures ix Abstract xiv Acknowledgements xvi Introduction 0.1 Introduction of ITRS 0.2 Building and Environment (A Good Landmark for ITRS) 0.3 Building Recognition for Localization 0.4 3D Reconstruction Environment for Navigation, Mapping and Exploring 0.5 Proposed Method for ITRS 0.6 Data Sets 0.6.1 ZuBuD Data Set 0.6.2 UlBuD01 data set 0.6.3 UlBuD02 data set 10 0.7 Unification of Words, Phrases and Definitions in This Dissertation 10 Building Detection 1.1 Introduction 1.2 Line Segment Detection 1.2.1 Detecting Line Segment 1.2.2 Model of Line Segment (MLS) 1.3 MSAC-based Calculation of Dominant Vanishing Points (DVPs) v 12 12 14 14 15 19 1.4 1.5 1.3.1 MSAC Algorithm 1.3.2 Vertical Line Segment Processing 1.3.3 Horizontal Line Segment Processing Line Segment Verification 1.4.1 Density of Distribution 1.4.2 Co-existing of Line Segments Building Facet Detection 1.5.1 Empirical Assumptions and Definitions 1.5.2 Rough Detection of Building Facet 1.5.3 Accuracy of Facet’s Boundaries 20 24 26 29 29 30 33 33 34 37 Building Recognition 2.1 Introduction 2.2 Area of Building Facet 2.2.1 Wall Color Histogram (WCH) 2.2.2 Localized Color Histogram [81] 2.3 Local Features of Building Facet 2.3.1 SIFT Description 2.3.2 Rectangular Shape and Local Features of Building Facet 2.4 Data Training 2.4.1 Matching and Constraints 2.4.2 Canonical RANSAC and Hough Transform-based Verification of Correspondences of Image Pairs 2.4.3 Cross Ratio-based Verification of Correspondences of Image Pairs 2.4.4 Geometric Normalization 2.4.5 SVD-based Method for Calculating the Approximate Vectors 2.4.6 A Common Model 40 40 43 43 47 50 51 53 56 57 Geometric Analysis for 3D Reconstruction of Building 3.1 Introduction 3.2 Principal Component (PCs) Detection 77 77 78 Experiments 4.1 Experimental Building Detection 4.2 Experimental Building Recognition 4.2.1 Experimental Recognition of 4.2.2 Experimental Recognition of 4.2.3 Experimental Recognition of 81 81 86 87 89 93 vi ZuBuD data UlBuD01 data UlBuD02 data 58 64 67 71 74 Conclusions 96 Bibliography 98 vii List of Tables 1.1 Results of DVP’s detection 29 1.2 The values of threshold N1 and N2 34 1.3 The condition of horizontal boundaries 37 1.4 The conditions for rejecting the ambiguous partial face 39 2.1 The distances of histograms (di , i = 1, 2, , 5) between the test and the stored images from left to right, respectively 50 2.2 Corresponding parameters for estimating DVP and homography matrix 59 2.3 Explanation for Fig.2.17 67 2.4 Selected γ for updating wall color histogram and local features 72 3.1 Estimated size of buildings in Fig.3.2 80 4.1 Test conditions for detecting building’s facets 83 4.2 Summary of building detection 86 4.3 Explanation for each sub-image in Fig.4.7 89 4.4 Results of building recognition 90 4.5 Comparing the size of database for each building 95 viii List of Figures Examples of challenges of building and environment analysis Building detection and feature extraction Scheme for training database and recognition 1.1 Illustration of line segment detection 15 1.2 An example of line segment detection: (a) Original image in ZuBuD data set; (b) 554 detected line segments, the red lines overwrite on the original image; for easy vision, in (c), the black lines are overwritten on the blurred image with linear transformation [min, max] (values of pixels) → [125, 255]; 1.3 16 MLS: Example image is taken from ZuBuD data set; (a) Neighbored regions; (b) line segment detection; (c) building and non-building segments are selected by handling 1.4 17 Distribution of 4I, σm 200 first sampled segments and selected thresholds 18 1.5 Using MLS to reduce the noise from natural object regions or images 19 1.6 The angle between the segment and the line where lies through the segment’s middle point and vanishing point 1.7 24 Retrieval of broken edges: (a) the vertical segments which create an acute angle 200 in maximum with y-axis; (b) the right edge split into four segments replacing by the blue one ix 25 ... hierarchical system for understanding of intelligent transportation and robot systems The first part of this thesis is for detecting landmark The buildings are classified with other objects like... December 2008 Author: Hoang-Hon Trinh Title: Geometrical Environment Understanding by Building Recognition for the Intelligent Transportation and Robot Systems Department: Graduate School of Electrical... ENGINEERING The undersigned hereby certify that they have read and recommend to the Faculty of Graduate Studies for acceptance a thesis entitled ? ?Geometrical Environment Understanding by Building Recognition