E-DTS 2.0: A NEXT-GENERATION OF A DISTRIBUTED TRACKING SYSTEM

*UDGXDWH6FKRRO(7')RUP 5HYLVHG    385'8(81,9(56,7< *5$'8$7(6&+22/ 7KHVLV'LVVHUWDWLRQ$FFHSWDQFH   7KLVLVWRFHUWLI\WKDWWKHWKHVLVGLVVHUWDWLRQSUHSDUHG  %\   (QWLWOHG    )RUWKHGHJUHHRI    ,VDSSURYHGE\WKHILQDOH[DPLQLQJFRPPLWWHH    &KDLU              7RWKHEHVWRIP\NQRZOHGJHDQGDVXQGHUVWRRGE\WKHVWXGHQWLQWKH5HVHDUFK,QWHJULW\DQG &RS\ULJKW'LVFODLPHU*UDGXDWH6FKRRO)RUPWKLVWKHVLVGLVVHUWDWLRQDGKHUHVWRWKHSURYLVLRQVRI 3XUGXH8QLYHUVLW\¶V³3ROLF\RQ,QWHJULW\LQ5HVHDUFK´DQGWKHXVHRIFRS\ULJKWHGPDWHULDO   $SSURYHGE\0DMRU3URIHVVRUVBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB  $SSURYHGE\    +HDGRIWKH*UDGXDWH3URJUDP 'DWH   Ryan Thomas Rybarczyk e-DTS 2.0: A Next-Generation of a Distributed Tracking System Master of Science Dr. Rajeev Raje Dr. Mihran Tuceryan Dr. Panos Linos Dr. Rajeev Raje Dr. Rajeev Raje 11/17/2010   *UDGXDWH6FKRRO)RUP 5HYLVHG   385'8(81,9(56,7< *5$'8$7(6&+22/  5HVHDUFK,QWHJULW\DQG&RS\ULJKW'LVFODLPHU     7LWOHRI7KHVLV'LVVHUWDWLRQ     )RUWKHGHJUHHRI &KRRVH\RXUGHJUHH  ,FHUWLI\WKDWLQWKHSUHSDUDWLRQRIWKLVWKHVLV,KDYHREVHUYHGWKHSURYLVLRQVRI3XUGXH8QLYHUVLW\ ([HFXWLYH0HPRUDQGXP1R&6HSWHPEHU3ROLF\RQ,QWHJULW\LQ5HVHDUFK  )XUWKHU,FHUWLI\WKDWWKLVZRUNLVIUHHRISODJLDULVPDQGDOOPDWHULDOVDSSHDULQJLQWKLV WKHVLVGLVVHUWDWLRQKDYHEHHQSURSHUO\TXRWHGDQGDWWULEXWHG  ,FHUWLI\WKDWDOOFRS\ULJKWHGPDWHULDOLQFRUSRUDWHGLQWRWKLVWKHVLVGLVVHUWDWLRQLVLQFRPSOLDQFHZLWKWKH 8QLWHG6WDWHV¶FRS\ULJKWODZDQGWKDW,KDYHUHFHLYHGZULWWHQSHUPLVVLRQIURPWKHFRS\ULJKWRZQHUVIRU P\XVHRIWKHLUZRUNZKLFKLVEH\RQGWKHVFRSHRIWKHODZ,DJUHHWRLQGHPQLI\DQGVDYHKDUPOHVV 3XUGXH8QLYHUVLW\IURPDQ\DQGDOOFODLPVWKDWPD\EHDVVHUWHGRUWKDWPD\DULVHIURPDQ\FRS\ULJKW YLRODWLRQ    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 3ULQWHG1DPHDQG6LJQDWXUHRI&DQGLGDWH   BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 'DWHPRQWKGD\\HDU       /RFDWHGDWKWWSZZZSXUGXHHGXSROLFLHVSDJHVWHDFKBUHVBRXWUHDFKFBKWPO  e-DTS 2.0: A Next-Generation of a Distributed Tracking System Master of Science Ryan Thomas Rybarczyk 11/11/2010 E-DTS 2.0: A NEXT-GENERATION OF A DISTRIBUTED TRACKING SYSTEM A Thesis Submitted to the Faculty of Purdue University by Ryan Thomas Rybarczyk In Partial Fulfillment of the Requirements for the Degree of Master of Science December 2010 Purdue University Indianapolis, Indiana ii To my parents, my sisters, and my wife iii ACKNOWLEDGMENTS I would like to start off by thanking my professor, advisor, and mentor, Dr. Rajeev Raje, for giving me the opportunity to be research assistant under his guidance. His support and guidance has allowed me to grow and become a much more capable researcher. I am forever grateful for his support and encouragement through the process of preparing this thesis. I would also like to thank my co-advisor, Dr. Mihran Tuceryan, as his guidance and knowledge of augmented reality allowed him to provide helpful insight into many problems that I encountered throughout my study. Finally, I would like to Dr. Panos Linos for agreeing to serve as my third member of my graduate committee. While his role was not large he provided support and guidance with his helpful knowledge and insight into Software Engineering concepts and practices. I would like to thank my family, most notably my parents – for without their everlasting love and support I would not have been able to achieve this goal in my life. I would also like to thank my wife, Amy, for her support, guidance, and patience throughout the entire process. Often times my work consumed my time but she was always there for me – and for that I forever grateful. Finally, I would like to thank the entire Computer Science Department at IUPUI. They helped to provide an environment where I could strive and provide positive contributions to the field of Computer Science; all the while gaining an invaluable education. In addition to this they allowed me to server many roles including teaching assistant, tutor, and instructor which allowed me to grow as a teacher and an individual. iv TABLE OF CONTENTS Page LIST OF TABLES vi LIST OF FIGURES vii ABBREVIATIONS viii ABSTRACT ix CHAPTER 1 INTRODUCTION 1 1.1 Need for Dynamic Discovery in a DTS 2 1.2 Need for a Coordinate System Handoff in a DTS 3 1.3 Need for Improved Calibration in a DTS 4 1.4 Need for Clock Synchronization in a DTS 5 1.5 Need for Quality of Service Parameters in a DTS 5 1.6 Issues to be Resolved 6 1.7 Problem Statement 7 1.8 Hypothesis 8 1.9 Contributions 8 1.10 Organization 9 CHAPTER 2 RELATED WORK 10 2.1 Related Work in Camera Calibration 11 2.2 Related Work in Dynamic Discovery 13 2.3 Related Work in Clock Synchronization 14 2.4 Related Work in Coordinate Systems and Spatial Relation Graphs 15 2.5 Related Work in Tracking Systems 17 CHAPTER 3 DESIGN AND IMPLEMENTATION 19 3.1 e-DTS Architecture 19 3.2 e-DTS Components 21 3.2.1 Camera and Camera Service 21 3.2.2 Discovery Service 22 3.2.3 Tracking Markers 23 3.2.4 Visibility Filters (VF) 24 3.2.5 Kalman Filter 25 3.2.6 Tracking Client 26 3.3 Implementation 27 3.4 e-DTS 2.0 28 3.4.1 Reasons for Enhancements 28 3.4.2 Proposed Enhancements to the e-DTS 30 3.4.3 Enhanced Features of the e-DTS 2.0 31 v Page CHAPTER 4 EXPERIMENTATION AND VALIDATION 40 4.1 Experiments to Test Dynamic Discovery 40 4.1.1 Experiments to test the functionality of the Multicast Protocol 41 4.2 Camera Calibration 43 4.3 Environmental Worlds 44 4.4 Clock Synchronization 46 4.5 Fusion and Tracking 48 4.6 Coordinate System Handoff and Transformation 50 CHAPTER 5 CONCLUSION AND FUTURE WORK 53 5.1 Future Extensions 53 5.2 Conclusion 53 LIST OF REFERENCES 55 APPENDICES Appendix A. User Manual 60 Appendix B. Figures 64 vi LIST OF TABLES Table Page Table 4.1 Average EEDT 42 Table 4.2 Average EERT 43 Table 4.3 Average Calibration Error 44 Table 4.4 Service Check Frequency (Dead Service) 45 Table 4.5 Clock Drift Averages 47 Table 4.6 Domain Time II Clock Drift 47 Table 4.7 Estimated Error in Averaging Fusion (Millimeters) 49 Table 4.8 Estimated Error in Averaging Fusion (Millimeters) 49 Table 4.9 Handoff Transformation Error – Experiment #1 (Millimeters) 51 Table 4.10 Handoff Transformation Error – Experiment #2 (Millimeters) 51 vii LIST OF FIGURES Figure Page Figure 3.1 e-DTS Architecture [2] 20 Figure 3.2 Fiducial Markers - pattHiro and pattKanji [3] 24 Figure 3.3 Federated Kalman Filter [19] 25 Figure 3.4 e-DTS 2.0 System Architecture 32 Figure 3.5 Tracking Process in the e-DTS 2.0 37 Figure 3.6 Graph for determining world transformation in e-DTS 2.0 38 Figure 4.1 Transformation and Handoff Experiment 52 Appendix Figure Figure B-1 e-DTS 2.0 Use Case 64 Figure B-2a UML Class Diagram 65 Figure B-2b Class Diagram 66 Figure B-2c Class Diagram 67 Figure B-2d Class Diagram 68 Figure B-2e Class Diagram 69 Figure B-2f Class Diagram 70 Figure B-2g Class Diagram 71 Figure B-3 Average EEDT Chart 72 Figure B-4 Clock Synchronization Tool Comparison Graph 73 Figure B-5 Average EERT (Unicast vs. Multicast) Chart 74 Figure B-6 Calibration Pattern 75 viii ABBREVIATIONS DTS Distributed Tracking System e-DTS Enhanced Distributed Tracking System EEDT End-To-End Discovery Time EERT End-To-End Response Time SRG Spatial Relation Graph [...]... SRG Spatial Relation Graph ix ABSTRACT Rybarczyk, Ryan Thomas M.S., Purdue University, December, 2010 e-DTS 2.0: A Next-Generation of a Distributed Tracking System Major Professor: Rajeev Raje A key component in tracking is identifying relevant data and combining the data in an effort to provide an accurate estimate of both the location and the orientation of an object marker as it moves through an environment... calibration Because of this, there is a need for an accurate and efficient method of calibrating the camera There are many tools that allow such a calibration but most are not intended for calibration or accuracy on a scale for wide area tracking Calibration is a two-step process that first involves preparing the camera by gathering its extrinsic and intrinsic parameters 5 [2] The second step of camera... quality of camera and the subsequent calibration of the camera Because of the goal and mission of the e-DTS to provide tracking using inexpensive tracking devices, in the form of web cameras, the calibration is critical to the overall tracking accuracy of the system Therefore alternative methods, from that provided with ARToolkit, and 7 techniques to camera calibration could yield better camera calibration... usefulness and ability for the e-DTS 2.0 to utilize QoS parameters in camera selection and tracking of an object Finally, this thesis will demonstrate the ability of a distributed tracking system to be able to handle and utilize a shared and global coordinate system and provide coordinate system handoff and transformation as an object moves from one coordinate system to another 9 1.10 Organization This... estimate location in a real-time manner is required Because of this a need of high accuracy is required as poor calibration leading to poor estimation could lead to catastrophic results for the robot 2.1 Related Work in Camera Calibration In vision based tracking systems quality camera calibration is key Because of this, there are various techniques and methods available that provide camera calibration In... idea of a federation of tracking devices allows for broader tracking of an object as it moves through a space The results of the various tracking devices can then be combined together in an effort to provide a more accurate estimate as to the location of an object; this process is known as data fusion [2] A well-defined and agile tracking framework is desired to serve as the basis for tracking research... be made such that the e-DTS is able to properly handle coordinate system transformation and the subsequent handoff from one coordinate system to another during the tracking process The fundamental key aspect of any tracking application is the accuracy of the estimated calculation of an object during any moment in time In vision based tracking systems this is typically dominated by the type and quality... calibration pattern and the camera itself increases They also show that the accuracy of the calibration is affected by the angle of the camera with respect to that of the calibration pattern They make a suggestion that correction filters for detecting the angle and distance of the camera from the calibration pattern could help resolve some of the accuracy issues that they discovered As a result, camera... in the area of distributed tracking and outline the importance of creating dynamic and agile tracking systems using vision based sensors, or cameras 19 CHAPTER 3 DESIGN AND IMPLEMENTATION The previous chapter provided a background of related work in the area of Distributed Tracking This chapter will take a closer look at the architecture and design behind the e-DTS [2] Finally, it will provide a discussion... pose of the camera They state that one of the primary shortcomings of the ARToolkit API‟s calibration method is that of correctly recognizing the calibration pattern and properly estimating the pose Because of this shortcoming the overall accuracy of the calibration will be negatively impacted as well as the overall tracking estimation provided Thus, their work shows the importance of camera calibration . Science Ryan Thomas Rybarczyk 11/11/2010 E-DTS 2.0: A NEXT-GENERATION OF A DISTRIBUTED TRACKING SYSTEM A Thesis Submitted to the Faculty of Purdue University by Ryan Thomas Rybarczyk In. preparing this thesis. I would also like to thank my co-advisor, Dr. Mihran Tuceryan, as his guidance and knowledge of augmented reality allowed him to provide helpful insight into many problems that. Clock Synchronization 14 2.4 Related Work in Coordinate Systems and Spatial Relation Graphs 15 2.5 Related Work in Tracking Systems 17 CHAPTER 3 DESIGN AND IMPLEMENTATION 19 3.1 e-DTS Architecture