Research on developing digital twins based application for industrial robots nghiên cứu xây dựng bộ đôi số ứng dụng cho robot công nghiệp

Research on developing digital twins based application for industrial robots nghiên cứu xây dựng bộ đôi số ứng dụng cho robot công nghiệp Research on developing digital twins based application for industrial robots nghiên cứu xây dựng bộ đôi số ứng dụng cho robot công nghiệp Research on developing digital twins based application for industrial robots nghiên cứu xây dựng bộ đôi số ứng dụng cho robot công nghiệp Research on developing digital twins based application for industrial robots nghiên cứu xây dựng bộ đôi số ứng dụng cho robot công nghiệp Research on developing digital twins based application for industrial robots nghiên cứu xây dựng bộ đôi số ứng dụng cho robot công nghiệp

MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Doan Thanh Xuan RESEARCH ON DEVELOPING DIGITAL TWINS- BASED APPLICATION FOR INDUSTRIAL ROBOTS Major: Mechanical Engineering Code: 9520103 DOCTORAL DISSERTATION IN MECHANICAL ENGINEERING Hanoi - 2024 MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Doan Thanh Xuan RESEARCH ON DEVELOPING DIGITAL TWINS- BASED APPLICATION FOR INDUSTRIAL ROBOTS Major: Mechanical Engineering Code: 9520103 DOCTORAL DISSERTATION IN MECHANICAL ENGINEERING SUPERVISORS: Prof Dr Vu Toan Thang Assoc Prof Dr Nguyen Thanh Hung Hanoi - 2024 GUARANTEE I hereby confirm that this is my own scientific research work The contents and data used for analysis in the dissertation have clear sources and have been published in accordance with regulations The research results in the dissertation were independently researched and analyzed by me in an honest, objective, and suitable manner for the conditions in Vietnam These results have not been published by any other author in any other research Hanoi, 23rd January 2024 Science instructor Postgraduate Prof Dr Vu Toan Thang Doan Thanh Xuan Assoc Prof Dr Nguyen Thanh Hung ACKNOWLEDGMENT When completing this dissertation, I received dedicated guidance from the academic advisors, the support of the Training Department, and School of Mechanical Engineering at the Hanoi University of Science and Technology Additionally, the Robotics Group and the Department of Mechatronics at the Hanoi University of Science and Technology offered the necessary conditions for my research To facilitate my research, I had the opportunity to work in the Laboratory of Smart Digital Factory at the School of Mechanical Engineering - Hanoi University of Science and Technology, which allowed me to conduct measurements and experiments related to the UR3 robot in the lightbulb assembly system The leaders of the School of Mechanical Engineering, the Department of Mechatronics, and the Robotics Group at the Hanoi University of Science and Technology have provided me with the necessary conditions for my scientific research, assisting me in accessing the equipment required for experiments related to the topic of my research I received valuable contributions and advice from professors, associate professors, doctors, and colleagues, who also provided relevant materials related to the topic of my research Moreover, I obtained support and encouragement from the graduate students at the School of Mechanical Engineering during the process of completing the procedures and the content of my PhD dissertation I want to express my heartfelt gratitude to all the individuals and groups, who provided me with guidance, assistance, and resources throughout this journey I especially want to thank my dissertation supervisors, Prof Dr Vu Toan Thang and Assoc Prof Dr Nguyen Thanh Hung, for their invaluable support I would also like to extend my thanks to my colleagues, friends, and family for their encouragement, sharing, and creating favorable conditions during my research Sincerely, Doan Thanh Xuan TABLE OF CONTENT LIST OF ABBREVIATIONS AND ACRONYMS LIST OF FIGURES LIST OF TABLES INTRODUCTION The Reasons for Choosing The Dissertation Topic Goal, Objectives, Scope, and Research Methodology The Scientific and Practical Significance of the Research Topic 10 The new findings 10 Dissertation structure 11 Chapter 1: OVERVIEW OF DIGITIAL TWINS AND THEIR APPLICATIONS 12 1.1 Introduction 12 1.2 The History of Digital Twin Development and Milestones 12 1.2.1 The History of Development 12 1.2.2 Key Milestones in the Development of Digital Twin Technology 13 1.3 Digital twin concept and classification 14 1.3.1 Digital twin concept 14 1.3.2 Integration-based classification of digital twins 18 1.3.3 Technologies associated to digital twin 19 1.4 Digital twin application 20 1.4.1 General aspects 20 1.4.2 Digital twin application for robotic path planning 22 1.4.3 Digital twin application for human-robot collaboration 23 1.5 The research status both domestically and internationally 23 1.5.1 International research status 24 1.5.2 National research status 27 1.6 Conclusion of Chapter 28 Chapter 2: DEVELOPING A DIGITAL TWIN FOR THE INDUSTRIAL ROBOT IN THE LIGHTBULB ASSEMBLY SYSTEM 29 INTRODUCTION 29 2.1 Lightbulb assembly system 29 2.2 Kinematic calculations and development of digital twin for the UR3 robot 32 2.2.1 Kinematic calculations of the UR3 robot 32 2.2.2 Modelling the UR3 robot 49 2.3 Conclusion of Chapter 52 Chapter 3: APPLICATION OF DIGITAL TWINS IN PATH FINDING FOR ROBOTS 54 INTRODUCTION 54 3.1 Robot path planning problem 54 3.1.1 Path planning problem for robot 54 3.1.2 End-effector motion of the robot 55 3.2 The A* pathfinding algorithm 57 3.2.1 The A* pathfinding algorithm 57 3.2.2 The improved A* algorithm 59 3.3 Application combining digital twins and A* algorithm in robot pathfinding 62 3.3.1 System description 62 3.3.2 Experimental results 64 3.4 Application of combining the digital twin and improved A* algorithm in robot pathfinding problem 68 3.4.1 Using the digital twin method 68 3.4.2 Using the A* algorithm 69 3.4.3 Adding obstacles with the same height 72 3.4.4 Adding obstacles with varying heights 74 3.4.5 Discussion 77 3.5 Conclusion of Chapter 77 Chapter 4: APPLICATION OF DIGITAL TWINS IN HUMAN-ROBOT COLLABORATION 79 INTRODUCTION 79 4.1 Theoretical basis 79 4.1.1 Collaboration between humans and robots 79 4.1.2 Genetic algorithm 81 4.2 Applying a combination of the digital twin method and genetic algorithm in the problem of human-robot collaboration 85 4.2.1 Overall framework diagram of the digital twin method for human-robot collaboration 85 4.2.2 Description of the human-robot collaboration system 87 4.2.3 Application of digital twins in human-robot collaboration 88 4.2.4 Application of genetic algorithms in human-robot collaboration 89 4.2.4.1 Initialization of the initial population 89 4.2.4.2 Objective evaluation function 91 4.2.4.3 Fitness function 93 4.2.4.4 Crossover function 93 4.2.4.5 Mutation function 95 4.2.4.6 Results and discussion 96 4.3 Conclusion of Chapter 101 CONCLUSION 102 FUTURE RESEARCH DIRECTIONS 103 REFERENCES 104 LIST OF PUBLICATIONS 111 LIST OF ABBREVIATIONS AND ACRONYMS No Short form Full form BMI Body Mass Index CAD Computer Aided Design DE Differential Evolution Denavit-Hartenberg DH Digital Twin DT Experimentable Digital Twins EDT Enterprise Resource Planning ERP Genetic Algorithm GA Human-Robot Cooperation HRC Information & Communications Technology 10 ICT International Organization for Standardization 11 ISO Jupiter Tessellation 12 JT Multi-Adaptive Genetic Algorithm 13 MGA Message Queueing Telemetry Transport 14 MQTT National Aeronautics and Space Administration 15 NASA Open Platform Communications-Unified Architecture 16 OPC-UA Programmable Logic Controller 17 PLC Product Lifecycle Management 18 PLM Particle Swam Optimization 19 PSO Source 20 SRC Transmission Control Protocol/Internet Protocol 21 TCP/IP Universal Robot 22 UR Page LIST OF FIGURES Figure 1 Key Milestones in the development of digital twin technology 14 Figure Digital twin framework for production 18 Figure Classification of digital twins based on the degree of integration 19 Figure Overall model 29 Figure 2 Robot algorithm flowchart 30 Figure Algorithm flowchart for supplying lightbulb caps, sockets and completing lightbulb assembly 31 Figure Algorithm flowchart for product return cycle 31 Figure UR3, UR5 and UR10 industrial robot product line 32 Figure Kinematic parameters of the UR3 32 Figure Motion model of the UR robot 33 Figure The angle limits and joint speeds of the UR3 robot's joints 49 Figure Vertical Projection (Left) and Side Projection (Right) of the workspace of the UR3 robot 50 Figure 10 Steps for building a digital model of the UR3 robot 50 Figure 11 A digital model of the lightbulb assembly system created by Tecnomatix Process Simulate software 51 Figure 12 Real time connection between a real robot and a virtual robot 52 Figure Target position and trajectory 56 Figure Tool Center Point (TCP) of the gripper 56 Figure 3 Comparison of toolpath with and without TCP 56 Figure Base coordinate system and coordinate system attached to the tool 57 Figure The flowchart of the A* algorithm 58 Figure The flowchart of the improved A* algorithm 58 Figure A simple example of a robot task 58 Figure A* algorithm-based path planning 58 Figure Optimal result of the first step on the left, the second step on the right 58 Figure 10 The optimal result of the final step 58 Figure 11 An actual image of the complete lightbulb assembly system 63 Figure 12 A sequence of tasks in the lightbulb assembly system 64 Figure 13 Collision detection in Tecnomatix software 65 Figure 14 A visual depiction of the robotic path using Python programming 66 Figure 15 Coordinates of the motion points along the robot's path determined through the implementation of the A* algorithm in Python 67 Figure 16 Robotic path found by the digital twin method 68 Figure 17 Robotic path found by the original A* algorithm (the left) and by the improved A* algorithm (the right) 71 Figure 18 Robotic path found by the improved A* algorithm when a 30-mm obstacle added 72 Figure 19 Robotic path found by the improved A* algorithm when a 50-mm obstacle added 73 Figure 20 Real image of the pairs of obstacles 73 Figure 21 Representation of the pairs of obstacles 75 Figure Genetic algorithm diagram 81 Page Figure A chart illustrating the Roulette selection method 83 Figure Overall framework diagram of the digital twin method for human-robot collaboration 86 Figure 4 Real image (on the left) and schematic diagram (on the right) of the miniature lightbulb assembly system 87 Figure The real human and robot (displayed on the left) and their digital representations within Tecnomatix software (featured on the right) 89 Figure Chromosome structure 90 Figure The crossover operation of two chromosomes 95 Figure The chart illustrates the best fitness indices of a population across each generation 97 Figure The chart comparing Eb, Es, and M across generations 98 Figure 10 The chart displaying entropy across generations 98 Figure 11 The best working process can be obtained in the form of encoding 99 Figure 12 The graph of the best_fitness value across generations 100 Figure 13 The graph of the mean_fitness value across generations 100 LIST OF TABLES Table 1 Definitions of digital twins in studies from 2019 and earlier 15 Table Technologies and tools for DT 20 Table D-H parameter table of the UR3 robot on the lightbulb assembly table 33 Table 2 Analytical solutions of the inverse dynamics problem 15 Table Coordinate parameters (A) and durations (B) for robotic movement in the phase devoid of large-sized obstacles 65 Table Robotic movement times before and after the A* algorithm applied 67 Table 3 Coordinate parameters (A) and durations (B) for robotic movement according to AA1F1F 69 Table Coordinate parameters (A) and durations (B) for robotic movement according to ABCDEF 70 Table Coordinate parameters (A) and durations (B) for robotic movement according to ACDF 71 Table Coordinate parameters (A) and durations (B) for robotic movement according to ACD1F 72 Table Coordinate parameters (A) and durations (B) for robotic movement according to ACD2F 74 Table Coordinate parameters (A) and durations (B) for robotic movement when more pairs of obstacles added in a 50-mm decreasing order of height 76 Table Gene encoding on chromosomes 90 Page INTRODUCTION The Reasons for Choosing The Dissertation Topic Simulating a process or a system helps humans gain a deeper understanding of the system, allowing for flexibility to adjust and change device parameters as well as to perform optimization testing during the initial planning phase Due to the advantages mentioned above, simulation has been widely researched and applied However, it has mostly been limited to static models and one-way information exchange from physical objects to digital entities The concept of the digital twin, aiming to create a digital counterpart that faithfully describes a physical device and can exchange data with it, has been gaining increasing attention from researchers Along with the strong development of digital technology and new-generation information, the ability to collect more data and process it efficiently has provided a strong foundation for the development of digital twins The digital factory has become an inevitable trend in various industries, with the digital twin being a fundamental unit; synthesizing digital twins forms a digital factory Research on building digital twins for robots is a relatively new field, meeting practical industrial demands and promising strong development prospects in the coming years, both domestically and internationally This research is based on theoretical studies and experiments to construct a digital model that faithfully describes a robot and establishes communication connections between the physical entity and the digital model The experiments are conducted on the UR3 robot within a lightbulb assembly system The digital twin of the UR3 robot, once created, continues to be applied in tasks such as pathfinding and optimizing collaboration between humans and robots The application of robots is becoming increasingly popular, in line with the development of the automation industry Current research in robotics continues to grow due to high market demand, a wide range of applications, and technical development potential Particularly, robot path planning has been extensively studied and applied across various generations of algorithms Optimizing path planning for robots helps reduce time and costs in robot operation processes One of the widely applied algorithms for robot path planning is the A* algorithm The A* algorithm was invented, and many improvements have been made to enhance its efficiency in the original path planning algorithm In industrial production lines, we often encounter many collaborative work scenarios between humans and robots In these collaborative systems, the advantages of robots, such as speed and precision, can be combined with human qualities like dexterity and adaptability in different situations Through an overall analysis of the digital twin and industrial robotics problem, the postgraduate has recognized that the research and development of digital twins for industrial robots is a promising research direction that can still be explored in various aspects; in particular, applying digital twins to industrial robots to enhance performance in the operations of digital factories This is the reason why the postgraduate chose the topic "Research on Developing Digital Twins-Based Application for Industrial Robots" In this dissertation, the postgraduate focuses on Page