Deploying delta robot to a product classifying and sorting system applying image processing graduation project of automation and cotnrol engineering technology

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION THESIS AUTOMATION AND CONTROL ENGINEERING DEPLOYING DELTA ROBOT TO A PRODUCT CLASSIFYING AND SORTING SYSTEM APPLYING IMAGE PROCESSING ADVISOR : TRAN MANH SON, PHD STUDENTS: PHAM THANH HIEP PHAN THIEN BAO SKL010850 Ho Chi Minh City, June 2023 MINISTRY OF EDUCATION & TRAINING HO CHI MINH UNIVERSITY OF TECHNOLOFY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT Major: Automation and Control Engineering Technology DEPLOYING DELTA ROBOT TO A PRODUCT CLASSIFYING AND SORTING SYSTEM APPLYING IMAGE PROCESSING Advisor: TRAN MANH SON, Ph.D Students: PHAM THANH HIEP 19151056 PHAN THIEN BAO 19151071 Ho Chi Minh City, June 2023 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, June 30, 2023 GRADUATION PROJECT TASKS Full name of student 1: PHAM THANH HIEP Student ID: 19151056 Full name of student 2: PHAN THIEN BAO Student ID: 19151071 Major: Automation and Control Engineering Technology Forms of training: Formal training Class of: 2019 – 2023 Class: 19151CLA3 I THESIS NAME: DEPLOYING DELTA ROBOT TO A PRODUCT CLASSIFYING AND SORTING SYSTEM APPLYING IMAGE PROCESSING II TASKS Original documents: - Documentation about PLC controller, Arduino - Documentation on image processing and libraries in Python - Documentation of Delta robot model - Documentation of servo motor and driver Content of implementation: - Learn the theoretical basis and connect the electrical part - Improved Delta robot hardware - Write an image processing program to detect color and rotation angle - Write a program to control Delta robot - Finalize hardware and verify the model - Test run many times, edit the system - Comments and conclusions III DATE OF DELIVERY: 02/2023 IV TASK COMPLETE DATE: 07/2023 V INSTRUCTOR'S FULLNAME: MR TRAN MANH SON INSTRUCTORS (Sign and write full name) AUTOMATIC CONTROL DEPARTMENT (Signed & stamped) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, June 30, 2023 SCHEDULE OF IMPLEMENTATION OF THE GRADUATION THESIS PROJECT Full name of student 1: PHAM THANH HIEP Student ID: 19151056 Full name of student 2: PHAN THIEN BAO Student ID: 19151071 Thesis name: DEPLOYING DELTA ROBOT TO A PRODUCT CLASSIFYING AND SORTING SYSTEM APPLYING IMAGE PROCESSING Week/date Instructor confirm Content Register for project topics Contact to exchange instructor Instructor assigns topics Learn about Delta robots, come up with ideas to improve hardware Design, improve hardware Learn image processing theory, Delta robot model Write an image processing program to recognize colors and angles Write a program to control Delta robot 9, 10 Write a program for a swivel 11, 12 Design the interface for the model 13 14, 15 16 Finish the model Test many times, record clips, write reports Adjust the model and finalize the report Ho Chi Minh City, June 2023 INSTRUCTORS (Sign and write full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, June 30, 2023 ASSESSMENT FORM OF INSTRUCT LECTURER Full name of student 1: PHAM THANH HIEP Student ID: 19151056 Full name of student 2: PHAN THIEN BAO Student ID: 19151071 Major: Automation and Control Engineering Technology Forms of training: Formal training Class of: 2019 – 2023 Class: 19151CLA3 Full name Instructor: Mr TRAN MANH SON Thesis name: DEPLOYING DELTA ROBOT TO A PRODUCT CLASSIFYING AND SORTING SYSTEM APPLYING IMAGE PROCESSING COMMENT: Regarding the topic content and implementation volume: Pros: Cons: Recommend for defend graduation thesis or not? Type rating: Score: (In words:) Ho Chi Minh City, June 2023 INSTRUCTORS (Sign and write full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, June 30, 2023 ASSESSMENT FORM OF DECIPATION LECTURER Full name of student 1: PHAM THANH HIEP Student ID: 19151056 Full name of student 2: PHAN THIEN BAO Student ID: 19151071 Major: Automation and Control Engineering Technology Forms of training: Formal training Class of: 2019 – 2023 Class: 19151CLA3 Full name Instructor: Mr TRAN MANH SON Thesis name: DEPLOYING DELTA ROBOT TO A PRODUCT CLASSIFYING AND SORTING SYSTEM APPLYING IMAGE PROCESSING COMMENT: Regarding the topic content and implementation volume: Pros: Cons: Recommend for defend graduation thesis or not? Type rating: Score: (In words:) Ho Chi Minh City, June 2023 REVIEW LECTURER (Sign and write full name) ACKNOWLEDGEMENT During the hard work of years studying at Ho Chi Minh City University of Technology and Education, we have always received enthusiastic support and valuable knowledge from our teachers Each lesson is invaluable baggage for our life path With deepest gratitude, we thank the teachers who always create the most favorable conditions in the learning and training process As a student of Ho Chi Minh City University of Technology and Education, the graduation project is a testament to the knowledge acquired after four years of study In the process of completing the graduation project, in addition to our efforts, we also received the enthusiastic guidance of Dr Tran Manh Son We would like to express our deepest gratitude to Mr Besides each subject, in each lecture, we always receive the attention, guidance, and dedicated help of the subject teachers in the Faculty of High-Quality Training and the Faculty of Electrical and Electronics Engineering We are very grateful We would also like to express our deep gratitude to our family, relatives, and friends who have supported and encouraged us to complete this graduation thesis Once again, we sincerely thank! Executed students Pham Thanh Hiep Phan Thien Bao GUARANTEE We hereby commit that the research presented in this graduate thesis is done by ourselves Excerpts and images used in the thesis are attached to their references, which are made public following the rules Project executor STUDENT STUDENT Pham Thanh Hiep Phan Thien Bao SUMMARY Thesis name: CLASSIFICATION OF PRODUCTS APPLICATION BY 4-DOF DELTA ROBOT ARM AND IMAGE PROCESSING EXECUTED STUDENTS: PHAM THANH HIEP 19151056 CLASS: 19151CLA3 PHAN THIEN BAO 19151071 CLASS: 19151CLA3 CONTENTS: Improved Delta robot model for vibration-reduced operation, requiring the use of color grading image processing and rotational processing so that the robot correctly classifies and delivers products to the same fixed angle After the implementation of the project, the group's model worked following the requirements set out by the group Besides, we are equipped with knowledge about image processing, controlling AC servo motors, using PLC as well as using Arduino controllers ABSTRACT PROJECT TITLE: CLASSIFICATION PRODUCTS APPLIED ROBOT AND IMAGE PROCESSING STUDENT NAMES: PHAM THANH HIEP 19151056 CLASS: 19151CLA3 PHAN THIEN BAO 19151071 CLASS: 19151CLA3 CONTENT: Improved Delta robot model for vibration-free operation, requires the use of color grading image processing so that the robot correctly classifies and delivers products at the same fixed angle After the implementation of the project, the group's model worked following the requirements set out by the group Besides, the group is equipped with knowledge about image processing, controlling AC servo motors, using PLC as well as using Arduino controllers FACULTY FOR HIGH QUALITY TRAINING Figure 4.10 Product identification in error circle Figure 4.11 No product recognition in yellow rectangle 61 FACULTY FOR HIGH QUALITY TRAINING Figure 4.12 No product recognition in blue circle All of the above results are that we have tested in Normal Lighting conditions, we have tested on 100 samples equally divided for each object and the success rate is 98% Then, we also tested in Glare condition and the success rate is rate 60% In which, Yellow, Error and Red products that affect the most Figure 4.13 Red Rectangle product identification in glare condition 62 FACULTY FOR HIGH QUALITY TRAINING Figure 4.14 Yellow Rectangle product identification in glare condition Figure 4.15 Blue Rectangle product identification in glare condition 63 FACULTY FOR HIGH QUALITY TRAINING Figure 4.16 Error Rectangle product identification in glare condition Similarly, the success rate in Low light condition is 50% In which, Blue products is the most influential because this color is close to the conveyor color Figure 4.17 Red Rectangle product identification in low light condition 64 FACULTY FOR HIGH QUALITY TRAINING Figure 4.18 Yellow Rectangle product identification in low light condition Figure 4.19 Blue Rectangle product identification in low light condition 65 FACULTY FOR HIGH QUALITY TRAINING Figure 4.20 Blue Rectangle product identification in low light condition We have tested the Robot operating in normal lighting conditions, to obtain the best possible results During the process of running the program, we observed the robot sucking and dropping objects exactly as originally intended Figure 4.21 The robot starts to suck things 66 FACULTY FOR HIGH QUALITY TRAINING Figure 4.22 The robot sucks and rotates discovered objects to arrange it in a neat box Figure 4.23 Robot drops objects into the 2nd sorting box 67 FACULTY FOR HIGH QUALITY TRAINING Figure 4.24 The robot returns to the home position after dropping the object Similar to other colors, a situation with a blue circle: Figure 4.25 The robot starts to suck things 68 FACULTY FOR HIGH QUALITY TRAINING Figure 4.26 Objects are dropped into the 3th place sorting box If the conveyor speed is increased too fast or too slow, the camera will still recognize and transmit pulses to the PLC to the pick-up position, but there will be a deviation in time of folding the object, which leads to the Robot not being able to suck at the center of the object We can adapt it in the PLC code Figure 4.27 The robot folds the object off-center due to the conveyor speed deviation However, the Robot's swivel joint is not really effective and errors often occur, because the hardware and code are not optimized 69 FACULTY FOR HIGH QUALITY TRAINING The object is rotated at a deviation angle with respect to the position of the cell due to the influence of the conveyor belt and the rotational resistance of the traction force of the air duct, which also leads to an uncontrolled rotation feedback error Figure 4.28 Red Rectangle product is rotated at an wrong angle from the drop box Figure 4.29 Yellow Rectangle product is rotated at an wrong angle from the drop box 70 FACULTY FOR HIGH QUALITY TRAINING Figure 4.30 Blue Rectangle product is rotated at an wrong angle from the drop box 71 Chapter CONCLUSIONS AND DEVELOPMENT ORIENTATIONS 5.1 Conclusion 5.1.1 Objectives Accomplished During nearly months of implementing the project, the group has achieved the following specific results: - Design and construction of hardware systems to ensure aesthetics and electrical safety - Calculating kinematics for Delta robot arm - Using PLC to connect and control devices in the system - Successful Applying Image Processing to product identifying, classifying, and sorting by color and size - The robot arm system is relatively stable and very solid, no longer shaking 5.1.2 Limitations of the Topic With a relatively short project implementation time, after the system was completed, the group also found several shortcomings that could not be overcome as follows: - Picking and dropping objects have not reached absolute accuracy - The devices that are reused suffer from a lot of other hardware problems, so it takes a lot of time just to find and replace similar devices, and products with relatively high prices - The fourth joint is inefficient due to many hardware influences - The camera works best in normal lighting conditions, when conditions are not satisfactory, there are errors because of reduced accuracy - 5.2 There is no manual mode yet Development With the potential of this topic, the system still has many development directions that can be achieved in the future, such as: - Positions that not change should be properly calibrated to avoid deviations 72 - The conveyor should maintain the speed to determine the exact position, as well as calculate and accurately measure the position of the deviation from the robot axis to the camera axis position, to achieve the most accuracy - At the fourth joint , it is recommended to use a motor with a gearbox to combat the rotational deviation processes affected by external factors - Adding manual mode to deploy given locations, to simulate execution 73 REFERENCES [1] The Delta Parallel Robot: Kinematics Solutions Robert L Williams II, Ph.D., Mechanical Engineering, Ohio University, October (2016) [2] https://www.marginallyclever.com/other/samples/fk-ik-test.html [3] Hoai Nam LE and Xuan Hoang LE, Geometrical design of a RUU type Delta robotbased on the predescribed Workspace, 4th International Conference on Green Technology and Sustainable Development (GTSD), HCMUTE (2018) [4] Robot công nghiệp, PGS.TS Nguyễn Trường Thịnh, Đại học SPKT Tp HCM [5] https://plcmitsubishi.com [6] https://thanhphan.vn/so-do-ket-noi-servo-mitsubishi-mr-j2s-10a [7] https://biquyetxaynha.com/huong-dan-cai-dat-driver-servo-mitsubishi [8] https://blog.paperspace.com/yolov7 [9] Train YOLOv7, nhận diện YOLOv7 GPU Colab, https://www.youtube.com/watch?v=TJ4o5QB7Wqg [10] Giao tiếp Python với PLC (FX-5UJ) Mitsubitshi || Communicating Python with PLC (FX-5UJ) Mitsubitshi, https://www.youtube.com/watch?v=Ubd3ImXOrYc 74 S K L 0