Development of machine vision systems on object classification and measurement for robot manipulation

國國國國國國國國 國國國國國 國國國 國國國國 國國國國國國國國國國國國國國國國國國國國國國國 Development of machine vision systems on object classification and measurement for robot manipulation 國國國國國國國 國國國國國國國國 國國 Graduate student: Ngo Ngoc Vu Advisor: Prof Quang-Cherng Hsu 國國國國 108 國 國 i 國國國國國國國國國國國國國國國國國國國國國國國 Development of machine vision systems on object classification and measurement for robot manipulation 國國國國國國國 Graduate student: Ngo Ngoc Vu 國國國國國國國國 國國 Advisor: Prof Quang-Cherng Hsu 國國國國國國國國 國國國國 國 國國國國 A Dissertation Submitted to Department of Mechanical Engineering National Kaohsiung University of Science and Technology in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Mechanical Engineering January, 2019 Kaohsiung, Taiwan, Republic of China 國國國國 108 國 國 ii iii 國國國國國國國國國國國國國國國國國國國國國國國 國國國國國國國 國國國國國國國國 國國 國國國國國國國國 國國國國國 國國國 國國國國 國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國 國國國國國國國國國 國國國國國DOF國國國國國國國國國國國國國國國國國國國 國國國 國國國國國國國國國國 CMOS 國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國 國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國 國國國國國國國國 國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國 國國國國國國國 X 國 Y 國國國國國國國國國國 0.48 mm 國 0.38 mm 國國國國國 國國 國國-0.34 mm 國-0.43 mm國 國國國國國國國國國國國國國國國國國國國國(DOF)國國國國國國國國國國國 國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國國國 國國國國國國國國國國(2-D)國國國(3-D)國國國國國國國國國國國國國國國國國國國 國國 國國國國國國國國國國國國國國國國國國國國國國國國國國 X 國 Y 國國國國國 國國國國國國 1.29 mm 國 1.12 mm國 國國國國國國國國國-1.48 mm 國-0.97 mm國國 國國國國國國國國X國Y 國 Z 國國國國國國國國 0.07國-0.418 國-0.063 mm國國國國 國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國 國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國 i Development of machine vision systems on object classification and measurement for robot manipulation Graduate student: Ngo Ngoc Vu Advisor: Prof Quang-Cherng Hsu Department of Mechanical Engineering National Kaohsiung University of Science and Technology ABSTRACT This research presents development of machine vision systems on object classification and measurement for robot manipulation Firstly, a machine vision system for the automatic metal part classification and measurement process is developed under different lighting conditions, and has been applied to the operation of a robot arm with degrees of freedom (DOF) In order to obtain accurate positioning information, the overall image is captured by a CMOS camera which is mounted above the working platform The effects of back-lighting and frontlighting conditions to the proposed system were investigated With the frontlighting condition, four different conditions were performed For each condition, global and local threshold operations were used to obtain good image quality The relationship between the image coordinates and the world coordinates was determined through Zhang’s method, the linear transformation and the quadratic transformation during the calibration process Experimental results show that in a back-lighting environment, the image quality is improved, such that the positions of the centers of objects are more accurate than in a front-lighting environment According to the calibration results, the quadratic transformation is more accurate ii than other methods By calculating the calibration deviation using the quadratic transformation, the maximum positive deviation is 0.48 mm and 0.38 mm in the X and Y directions, respectively The maximum negative deviation is -0.34 mm and -0.43 mm in X and Y directions, respectively The proposed system is effective, robust, and can be valuable to industry The second, a machine vision system for color object classification and measurement process for robot arm with six degree of freedom (DOF) is developed In order to obtain accurate positioning information, the overall image is captured by a double camera C615 and a camera C525 which are mounted above the working platform The relationship between the image coordinate and the world coordinate is performed through calibration procedure The quadratic transformation and generalized perspective transformation algorithms were used to transform coordinates in 2-D and 3-D calibration process, respectively According to calibration results, with 2-D calibration, the positive maximum deviation is 1.29 mm and 1.12 mm in X and Y directions, respectively The negative maximum deviation is -1.48 mm and -0.97 mm in X and Y directions, respectively With 3-D calibration, the deviation is 0.07 mm, -0.418 mm, and -0.063 mm in X, Y and Z directions, respectively The proposed system can catch the three dimensional coordinates of the object and perform classification and assembly automatic operations by the data from visual recognition system Keywords: machine vision, robot arm, camera calibration, image analysis, object recognition, lighting source ACKNOWLEDGMENTS The fulfillment of over three years of study at National Kaohsiung University of Science and Technology (NKUST) has brought me into closer relations with many enthusiastic people who wholeheartedly devoted their time, energy, and support to help me during my studies Therefore, this is my opportunity to acknowledge my great debt of thanks to them I wish to express my thanks and gratitude to my academic supervisor, Prof Dr Quang-Cherng Hsu, for his continuous guidance, valuable advice, and helpful supports during my studies He has always been supportive of my research work and gave me the freedom to fully explore the different research areas related with my study I wish to acknowledge my deepest thanks to Vietnam Ministry of Education and Taiwan Ministry of Education for giving me a great opportunity, necessary scholarships to study at NKUST via VEST500 scholarship which is corporation between Vietnam Government and Taiwan Government, and many enthusiastic helps during my time in NKUST I am also particularly grateful to Thai Nguyen University of Technology (TNUT) provided me unflagging encouragement, continuous helps and support to complete this course My gratitude also goes to all of the teachers, Dean and staffs of Department of Mechanical Engineering at NKUST for their devoted teaching, great helping and thoughtful serving during my study I would also like to express my sincere gratitude to all of my colleagues at the Precision and Nano Engineering Laboratory (PANEL), Department of Mechanical Engineering, NKUST I want to express my sincere thanks to all my Vietnamese friends in NKUST for their helpful sharing and precious helping me over the past time I also wish to express my gratitude to all those who directly or indirectly helped me during my study in NKUST Finally, my special thanks to my dad Ngo The Long and my mom Vu Thi Hai, to my older sister Ngo Thi Phuong, to my adorable wife Duong Thi Huong Lien, to two lovely little daughters Ngo Duong Anh Thu and Ngo Phuong Linh, who are the most motivation for me over years in Taiwan! CONTENTS 國國國國…………………………………………………………………………….i ABSTRACT……………………………………………………………………… ii ACKNOWLEDGMENTS……………………………………………………… iv CONTENTS……………………………………………………………………… vi LIST OF FIGURES…………………………………………………………… xii LIST OF TABLES…………………………………………………………… xvi NOMENCLATURE…………………………………………………………… xvii Chapter Introduction…………………………………………………………….1 1.1 Motivation of the research……………………………………………………1 1.2 Scopes of the research……………………………………………………… 1.3 Contributions…………………………………………………………………9 1.4 Organization of the dissertation…………………………………………… Chapter Theory of image processing and machine vision……………………12 2.1 Image processing system……………………………………………………12 2.1.1 Basics in image processing…………………………………………… 13 2.1.1.1 Pixels…………………………………………………………… 13 2.1.1.2 Resolution of image…………………………………………… 13 2.1.1.3 Gray level……………………………………………………… 14 2.1.1.4 Histogram……………………………………………………… 15 2.1.1.5 Image presentation……………………………………………….16 2.1.1.6 Color models…………………………………………………… 17 2.1.1.7 Neighbors of pixels………………………………………………18 2.1.2 Morphological image processing …………………………………… 19 2.1.2.1 Erosion operation……………………………………………… 19 2.1.2.2 Dilation operation……………………………………………… 19 2.1.2.3 Opening and closing operations …………………………………20 2.1.3 Blob analysis………………………………………………………… 21 2.1.3.1 Goal of blob analysis ……………………………………………21 2.1.3.2 Feature extraction ………………………………………………21 2.1.3.3 Steps to perform blob analysis …………………………………22 2.2 Machine vision system …………………………………………………….22 2.2.1 Lighting design ………………………………………………… .23 2.2.2 Lens …………………………………………………………… … 25 2.2.3 Image sensors ………………………………………………… … 27 Chapter Coordinate calibration methods and camera calibration………….30 3.1 Two-Dimensional coordinate calibration………………………………… 30 vii cover are suitable to use for improving uniformity of light for all objects (bolts, nuts and washers) With the proposed system, all components were classified accurately 100% Positions of them in the world coordinate system were determined for controlling robot arm during working process 6.1.2 Conclusion for the classification and measurement system for color objects In this study, a color object classification and measurement system for a robotic arm was developed, using a machine vision system With this system, the robotic arm could accurately recognize and classify different color objects The conclusions drawn from this research are summarized as follows: For the 2-D object classification and calibration work, the CMOS camera C525 was able to successfully capture the data related to location The quadratic transformation, which is a plane-to-plane transformation, was used to determine the relationship between the image coordinates and the world coordinates Calibration errors of this method is in the range of resolution of the proposed 2-D system For the 3-D object recognition and calibration work, an automatic determination of a position of interest on the calibration pattern was performed successfully The perspective transformation was used to determine the relationship between the image coordinates and the world coordinates Concerning the image calibration, the conversion process from the world coordinate system to the image coordinate system, as well as the reverse process, of five calibration points (from point to point 5) was successful The error was small and within a suitable range of accuracy for the proposed system Algorithms were developed which could ascertain the orientations of different geometries, allowing the robotic arm to position the triangles into their respective holes accordingly By using the RGB colour model, and choosing suitable threshold values for each colour, including red, blue, green, and yellow, all the assembly modules and their associated parts, were recognised The program was able to present the ID, name, and centre lines of the objects and holes accurately All assembly parts and models were classified accurately 100% Positions of them in the world coordinate system were determined for controlling robot arm during working process 6.2 Future works and suggestions In addition to the work realized in this study, there are several points which need to be further investigated and improved in the future, as follows: a Machine vision has been applied in many fields to increase productivity and decrease product price In mechanical engineering, machine vision systems can be developed for defect inspection of online complex geometries or hazardous environment with humans It can be used to replace for non-contact measurement devices which is high cost such as ATOS or contact measurement devices such as CMM (Coordinate Measuring Machine) b New calibration methods and new calibration patterns will be developed to improve accuracy of machine vision based systems c Advanced open software packages will be used such as C# with image library ++ Emgu CV, C , Python with image library OpenCV or close software packages such as Matlab or Halcon for complex geometry recognition and measurement List of publications Journal papers (SCI) Ngo, N V; Hsu, Q C; Hsiao, W L; Yang, C J (2017) Development of a simple three-dimensional machine-vision measurement system for in-process mechanical parts Advances in Mechanical Engineering, 9(10), IF: 0.848, DOI 1687814017717183 Hsu, Q C; Ngo, N V; Ni, R H (2018) Development of a Faster Classification System for Metal Environments Parts The Using Machine International Vision Journal of under Different Advanced Lighting Manufacturing Technology, Vol 76, No 1-4, pp 247-254, IF: 2.601, DOI: 10.1007/s00170-018-2888-7 Ngo, N V; Glen, A P; Hsu, Q C (2019) Development of a Color Object Classification and Measurement System using Machine Vision International Journal of Precision Engineering and Manufacturing (Under review) Journal papers (EI) Ngo, N V; Hsu, Q C; Li, W H; Huang, P J (2017) Optimizing Design of Two-dimensional Forging Preform by Bi-directional Evolutionary Structural Optimization Method Procedia Engineering, 207, 520-525 DOI: https://doi.org/10.1016/j.proeng.2017.10.815 International Conference Papers Hsu, Q C; Ngo, N V; Chen, K M (2016) Development of Positioning System for Edge-milling of Different Wind Deflector by Machine Vision IEEE ICASI 2016, International Conference on Applied System Innovative, Okinawa, Japan Hsu, Q C; Ngo, N V; Lin, H W; Jui, Y C (2016) Development of Simple Equipped 3D Measurement System for in-process Mechanical Parts IEEE ICASI 2016, International Conference on Applied System Innovative, Okinawa, Japan Wu, N J; Hsu, Q C; Do, T V; Ngo, N V (2017) Study on Three Dimensional Metrology System with Automatic Calibration and Spot Guidance Technology for Metal Product Measurement The 14th International Conference on Automation Technology, Kaohsiung, Taiwan Ngo, N V; Hsu, Q C; Ni, R H (2017) Development of Smart Faster Classification and Assembly System for Robot Arm under Non-Control Lighting Environment The 14th International Confer on Automation Technology, Kaohsiung, Taiwan Hsu, Q C; Jian, Y L; Ngo, N V (2018) Development on Automatic Optical Measurement System for Ring-Shaped Workpiece Contour with Economical Cost and High Performance The 6th International Symposium on Sensor Science and 4th SPINTECH Technology Thesis Award, Taiwan References [1] Leemans, V., Magein, H., & Destain, M F (2002) AE-automation and emerging technologies: On-line fruit grading according to their external quality using machine vision Biosystems Engineering, 83(4), 397-404 [2] Blasco, J., Aleixos, N., & Moltó, E (2003) Machine vision system for automatic quality grading of fruit Biosystems engineering, 85(4), 415-423 [3] Cubero, S., Aleixos, N., Moltó, E., Gómez-Sanchis, J., & Blasco, J (2011) Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables Food and Bioprocess Technology, 4(4), 487-504 [4] Xiang, R., He, W., Zhang, X., Wang, D., & Shan, Y (2018) Size measurement based on a two-camera machine vision system for the bayonets of automobile brake pads Measurement, 122, 106-116 [5] Balasundaram, M K., & Ratnam, M M (2014) In-process measurement of surface roughness using machine vision with sub-pixel edge detection in finish turning International journal of precision engineering and manufacturing, 15(11), 2239-2249 [6] Mishra, A., Sainul, I A., Bhuyan, S., Deb, S., Sen, D., & Deb, A K (2018) Development of a Flexible Assembly System Using Industrial Robot with Machine Vision Guidance and Dexterous Multi-finger Gripper In Precision Product-Process Design and Optimization (pp 31-71) Springer, Singapore [7] Stone, K., Cintron-Aponte, A., & Simons, B (2018) U.S Patent Application No 15/288,245 [8] Huang, S H., & Pan, Y C (2015) Automated visual inspection in the semiconductor industry: A survey Computers in industry, 66, 1-10 [9] Najrabi, D., Hamghalam, M., & Ayatollahi, A (2018) Diagnosis of astrocytoma and globalastom using machine vision In Fuzzy and Intelligent Systems (CFIS), 2018 6th Iranian Joint Congress on IEEE, 152-155 [10] Možina, M., Tomaževič, D., Pernuš, F., & Likar, B (2013) Automated visual inspection of imprint quality of pharmaceutical tablets Machine vision and applications, 24(1), 63-73 [11] Chauhan, V., & Surgenor, B (2017) Fault detection and classification in automated assembly machines using machine vision The International Journal of Advanced Manufacturing Technology, 90(9-12), 2491-2512 [12] Kumar, G S., Natarajan, U., & Ananthan, S S (2012) Vision inspection system for the identification and classification of defects in MIG welding joints The International Journal of Advanced Manufacturing Technology, 61(9-12), 923-933 [13] Huang, C K., Liao, C W., Huang, A P., & Tarng, Y S (2008) An automatic optical inspection of drill point defects for micro-drilling The International Journal of Advanced Manufacturing Technology, 37(11-12), 1133-1145 [14] Park, C., Choi, S., & Won, S (2010) Vision-based inspection for periodic defects in steel wire rod production Optical Engineering, 49(1), 017202 [15] Dworkin, S B.; Nye, T J (2006) Image processing for machine vision measurement of hot formed parts Journal of Materials Processing Technology, 174(1), 1-6 [16] Derganc, J., Likar, B., Pernuš, F (2003) A machine vision system for measuring the eccentricity of bearings Computers in Industry, 50(1), 103-111 [17] Shuxia, G., Jiancheng, Z., Xiaofeng, J., Yin, P., Lei, W (2011) Mini milling cutter measurement based on machine vision Procedia Engineering, 15, 1807-1811 [18] Hsu, Q C., Lin, C W., Chen, J Y (2012) Development of an automatic optical inspection system for defect detection of dental floss picks In Advanced Intelligent Mechatronics (AIM), 2012 IEEE/ASME International Conference, 444-449 [19] Ngo, N V., Hsu, Q C., Hsiao, W L., Yang, C J (2017) Development of a simple three-dimensional machine-vision measurement system for in-process mechanical parts Advances in Mechanical Engineering, 9(10), 1-11 [20] Ali, M H., Kurokawa, S., & Uesugi, K (2014) Camera based precision measurement in improving measurement accuracy Measurement, 49, 138-147 [21] Rejc, J., Kovačič, F., Trpin, A., Turk, I., Štrus, M., Rejc, D., & Munih, M (2011) The mechanical assembly dimensional measurements with the automated visual inspection system Expert Systems with Applications, 38(8), 10665-10675 [22] Martínez, S S., Vázquez, C O., García, J G., & Ortega, J G (2017) Quality inspection of machined metal parts using an image fusion technique Measurement, 111, 374-383 [23] Bhuvanesh A., Mani MR (2007) Automatic detection of stamping defects in leadframes using machine vision: Overcoming translational and rotational misalignment The International Journal of Advanced Manufacturing Technology, 32, 1201-1210 [24] Quinn, M K., Spinosa, E., & Roberts, D A (2017) Miniaturisation of pressure-sensitive paint measurement systems using low-cost, miniaturised machine vision cameras Sensors, 17(8), 1708 [25] Zhang, X., Zhang, J., Ma, M., Chen, Z., Yue, S., He, T., & Xu, X (2018) A High Precision Quality Inspection System for Steel Bars Based on Machine Vision Sensors, 18(8), 2732 [26] Cubero, S.; Aleixos, N.; Moltó, E.; Gómez-Sanchis, J.; Blasco, J (2011) Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables Food and Bioprocess Technology, 4(4), 487-504 [27] Jarimopas, B.; Jaisin, N (2008) An experimental machine vision system for sorting sweet tamarind Journal of Food Engineering, 89(3), 291-297 [28] Blasco, J.; Aleixos, N.; Roger, J M.; Rabatel, G.; Molto, E (2002) AE-automation and emerging technologies: robotic weed control using machine vision Biosystems Engineering, 83(2), 149-157 [29] Di Fulvio, G., Frontoni, E., Mancini, A., and Zingaretti, P (2014) A stereovision system for dimensional measurements in industrial robotics applications In Mechatronic and Embedded Systems and Applications (MESA), 2014 IEEE/ASME 10th International Conference, IEEE, 1-6 [30] Phansak, N., Pichitra, U., Kontorn, C (2016) Using machine vision for flexible automatic assembly system The 20th International Conference on Knowledge Based and Intelligent Information and Engineering Systems, KES2016, York, United Kingdom [31] Tsarouchi, P., Matthaiakis, S A., Michalos, G., Makris, S., Chryssolouris, G (2016) A method for detection of randomly placed objects for robotic handling CIRP Journal of Manufacturing Science and Technology, 14, 20-27 [32] Iscimen, B., Atasoy, H., Kutlu, Y., Yildirim, S., Yildirim, E (2015) Smart robot arm motion using computer vision Elektronika ir Elektrotechnika, 21(6), 3-7 [33] Rai, N., Rai, B., Rai, P (2014) Computer vision approach for controlling educational robotic arm based on object properties In Emerging Technology Trends in Electronics, Communication and Networking (ET2ECN), The 2nd International Conference, 1-9 [34] Tsarouchi, P., Michalos, G., Makris, S., Chryssolouris, G (2013) Vision system for robotic handling of randomly placed objects Procedia CIRP, 9, 61-66 [35] Juang, J G., Tsai, Y J., Fan, Y W (2015) Visual recognition and its application to robot arm control Applied Sciences, 5(4), 851-880 [36] Shin, K Y., and Mun, J H (2012) A multi-camera calibration method using a 3-axis frame and wand International Journal of Precision Engineering and Manufacturing, 13(2), 283-289 [37] Ji, Y.; Yamashita, A.; Asama, H (2017) Automatic calibration of camera sensor networks based on 3D texture map information Robotics and Autonomous Systems, 87, 313-328 [38] Deng, L., Lu, G., Shao, Y., Fei, M., and Hu, H (2016) A novel camera calibration technique based on differential evolution particle swarm optimization algorithm Neurocomputing, 174, 456-465 [39] You, X., and Zheng, Y (2016) An accurate and practical calibration method for roadside camera using two vanishing points Neurocomputing, 204, 222-230 [40] Ivo, S., Musić, J., Zanchi, V (2013) Improved structured light 3D scanner with application to anthropometric parameter estimation Measurement, 46(1), 716-726 [41] Davies E.R (1998) Automated Visual Inspection, Machine Vision Second ed., Academic Press, New York, Chapter 19, 471–502 [42] Sanz, J.L.C., Petkovic, D (1988) Machine vision algorithm for automated inspection of thin-film disk heads IEEE Transactions on PAMI 10, 830-848 [43] Bahlmann, C., Heidemann, G., Ritter, H (1999) Artificial neural networks for automated quality control of textile seams Pattern Recognition, 32, 1049-1060 [44] Tucker, J.W (1989) Inside beverage can inspection: an application from start to finish Proceecings of the Vision’89 Conference [45] Novini, A.R (1990) Fundamentals of machine vision inspection in metal container glass manufacturing Vision’90 Conference [46] Ker, J., Kengskool, K (1990) An efficient method for inspecting machine parts by a fixtureless machine vision system Vision’90 Conference [47] Torres, T., Sebastian, J.M., Aracil, R., Jimenez, L.M., Reinoso, O (1998) Automated real-time visual inspection system for high-resolution superimposed printings Image and Vision Computing, 16, 947-958 [48] Shafarenko, L., Petrou, M., Kittler, J (1997) Automatic watershed segmentation of randomly textured color images IEEE Transactions on Image Processing, 6, 1530-1543 [49] Li, H., Lin, J.C (1994) Using fuzzy logic to detect dimple defects of polisted wafer surfaces IEEE Transactions on Industry Applications, 30, 1530-1543 [50] Frank, Y.S (2009) Image processing and mathematical morphology fundamental and applications CRC Press, Taylor & Francis Group [51] Scott, E.U (1998) Computer vision and image processing: a practical approach using CVIP tools Prentice-Hall International, Inc [52] Rafael, C.G., Richard, E.W (2002) Digital Image Processing Prentice-Hall, Upper Saddle, New Jersey [53] Ramesh, J., Rangachar, K., Brian G.S (1995) Machine Vision McGraw-Hill [54] Matrox Image Library, version 8, User guide, Manual no 10513-301-0880, June 1, 2005 [55] David, F.R., Alan, J A (1990) Methematical Elements for Computer Grathics McGraw-Hill [56] Erwin, K (1999) Advanced Engineering Mathematics John Wiley & Sons, Inc [57] Fu, K.S., Gonza´lez, R.C., Lee, C.S.G (1997) Robotics: control, sensing, vision, and intelligence New York: McGraw-Hill [58] Oliver, F (2001) Three-Dimension Computer Vision: A geometry viewpoint The MIT Press, Cambridge, Massachusetts, London, England [59] Gary, B., Adrian, K (2008) Learning OpenCV: Computer vision with the OpenCV library O’REILLY [60] Peter, C (2013) Robotics, Vision and Control: Fundamental Algorithms in Matlab Springer [61] http://www.hiwin.tw/download/Tech_doc/Robot/RA605_Manipulator_manual (EN).pdf [62] Tsai, R.Y (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses IEEE Journal of Robotics and Automation, 3, 323-344 [63] Zhang, Z (2000) A flexible new technique for camera calibration IEEE Transactions on pattern analysis and machine intelligence, 22,1330-1334 [64] Zhang, W., Yuchen, B., Feng, D (2012) Comparision of calibration methods for bubbly flow video image Control and Decision Conference (CCDC), 2012 24th Chinese IEEE [65] https://www.mathworks.com/help/vision/examples/measuring planar objects with a calibrated camera html [66] Chen, H.T (2016) Geometry-based camera calibration using five point correspondences from a single image IEEE Transactions on Circuits and Systems for Video Technology, 27, 2555-2566 [67] Kang, D.J., Wang, H.L (2010) Automatic circle pattern extraction and camera calibration using fast adaptive binarization and plane homography International Journal of Precision Engineering and Manufacturing, 11,13-21 [68] Hsu, Q.C (2003) Comparison of different analysis models to measure plastic strains on sheet metal forming parts by digital image processing International Journal of Machine Tools and Manufacture ,43, 515-521 [69] John, C.R (2007) The Image Processing Handbook Taylor&Francis Group [70] Pérez, L., Rodríguez, Í., Rodríguez, N., Usamentiaga, R., & García, D F (2016) Robot guidance using machine vision techniques in industrial environments: A comparative review Sensors, 16(3), 335 [71] Kopparapu, S K (2006) Lighting design for machine vision application Image and Vision Computing, 24(7), 720-726 [72] Malamas, E N., Petrakis, E G., Zervakis, M., Petit, L., & Legat, J D (2003) A survey on industrial vision systems, applications and tools Image and vision computing, 21(2), 171-188 [73] Martin, D (2007) A practical guide to machine vision lighting Retrieved, 11(05), 2013 [74] Di Fulvio, G., Frontoni, E., Mancini, A., & Zingaretti, P (2014) A stereovision system for dimensional measurements in industrial robotics applications In Mechatronic and Embedded Systems and Applications (MESA) 2014 IEEE/ASME 10th International Conference on (pp 1-6) IEEE [75] Tsarouchi, P., Michalos, G., Makris, S., & Chryssolouris, G (2013) Vision system for robotic handling of randomly placed objects Procedia CIRP, 9, 61-66 [76] https://www.microsoft.com/en-us/research/project/a flexible new technique for camera calibration [77] Kang, D J., Ha, J E., & Jeong, M H (2008) Detection of calibration patterns for camera calibration with irregular lighting and complicated backgrounds International Journal of Control, Automation, and Systems, 6(5), 746-754 [78] Park, J.H., et al (2009) Easy calibration method of vision system for in-situ measurement of strain of thin films Transactions of Nonferrous Metals Society of China, 19, 243-249 [79] Mersch, S (1987) Overview of machine vision lighting techniques In Optics, Illumination, and Image Sensing for Machine Vision International Society for Optics and Photonics, 728, 36-39 [80] Cowan, C K (1991) Automatic camera and light-source placement using CAD models In Automated CAD-Based Vision, 1991, Workshop on Directions in IEEE, 22-32 [81] Hsu, Q C., Ngo, N V., & Ni, R H (2018) Development of a faster classification system for metal parts using machine vision under different lighting environments The International Journal of Advanced Manufacturing Technology, 76(1-4), 247-254 [82] Tan, W., Xuan L , Chen Y., & Erfu Y (2018) A novel method of 3D motion object’s position perception MATEC Web of Conferences, ISWSO 2018, 246, 1-6 [83] Gao, M., Yan, Y., Yang, Y., Huang, J., & He, Z (2016) A positioning system based on monocular vision for industrial robots In Information Science and Control Engineering (ICISCE), 2016 3rd International Conference on IEEE, 784-788 [84] Mustafa, A A., Shapiro, L G., & Ganter, M A (1999) 3-D object identification with color and curvature signatures Pattern recognition, 32(3), 339-355 [85] Lampert, C H., & Peters, J (2012) Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components Journal of RealTime Image Processing, 7(1), 31-41 [86] Caleiro, P M., Neves, A J., & Pinho, A J (2007) Color-spaces and color segmentation for real-time object recognition in robotic applications Electrúnica e Telecomunicaỗoes, 4(8), 940-945 [87] in Zhang, H., & Cao, Q (2017) Texture-less object detection and 6D pose estimation RGB-D images Robotics and Autonomous Systems, 95, 64-79 Appendices Program for camera calibration using Matlab tool M = [1746.1 425.8665;0 1744.9 464.9518;0 1]: Intrinsic matrix RT = [0.9999 -0.0103 -0.0086 -40.0882;0.0103 0.9999 -0.0027 -126.1172; 0.0086 0.0026 1001.6]: Rotation matrix and Translation vector num = csvread('2.csv'); size_num = size(num); for i = : size(num); a(i,1) = M(1,1)*RT(1,1)+M(1,2)*RT(2,1)+M(1,3)*RT(3,1) - num(i,1)*RT(3,1); %x b(i,1) = M(1,1)*RT(1,2)+M(1,2)*RT(2,2)+M(1,3)*RT(3,2) - num(i,1)*RT(3,2); %x c(i,1) = M(2,2)*RT(2,1)+M(2,3)*RT(3,1) - num(i,2)*RT(3,1); %y d(i,1) = M(2,2)*RT(2,2)+M(2,3)*RT(3,2) - num(i,2)*RT(3,2); %y H(1,1) = a(i,1); H(1,2) = b(i,1); H(2,1) = c(i,1); H(2,2) = d(i,1) e(i,1) = -(M(1,1)*RT(1,4)+M(1,2)*RT(2,4)+M(1,3)*RT(3,4) num(i,1)*RT(3,4)); %x f(i,1) = -(M(2,2)*RT(2,4)+M(2,3)*RT(3,4) - num(i,2)*RT(3,4)); %y G(1,1) = e(i,1); G(2,1) = f(i,1); K = inv(H); L(i,1) = K(1,1)*G(1,1)+K(1,2)*G(2,1); L(i,2) = K(2,1)*G(1,1)+K(2,2)*G(2,1); end csvwrite('NEW_WORLDCOOR.csv', L); A User interface of the classification and measurement system for metal parts B User interface of the classification and measurement system for color objects ... National Kaohsiung University of Science and Technology ABSTRACT This research presents development of machine vision systems on object classification and measurement for robot manipulation Firstly,... 國國國國國國國國國國國國國國國國國國國國國國國國國國國國國國 i Development of machine vision systems on object classification and measurement for robot manipulation Graduate student: Ngo Ngoc Vu Advisor: Prof Quang-Cherng Hsu Department of Mechanical... computer vision system for detection and autonomous object manipulation placed randomly on a target surface and controls an educational robotic arm with degree of freedom (DOF) to pick it up and place