Tóm tắt research on developing digital twins based application for industrial robots

Trang 1 MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Doan Thanh Xuan RESEARCH ON DEVELOPING DIGITAL TWINS-BASED APPLICATION FOR INDUSTRIAL ROBOTS Major:

MINISTRY OF EDUCATION AND TRAINING

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

Doan Thanh Xuan

RESEARCH ON DEVELOPING DIGITAL TWINS-BASED APPLICATION FOR INDUSTRIAL ROBOTS

Major: Mechanical Engineering Code:9520103

DOCTORAL ABSTRACT IN MECHANICAL ENGINEERING

Hanoi – 2024

The work was completed at Hanoi University of Science and Technology

Supervisors: Prof Dr Vu Toan Thang

Assoc Prof Dr Nguyen Thanh Hung

The dissertation can be found at:

1 Ta Quang Buu library - Hanoi University of Science and Technology

2 Vietnam National Library

LIST OF PUBLICATIONS

1 Đoàn Thanh Xuân, Lê Giang Nam, (2021) “Sự phát triển của công nghệ bộ đôi số và ứng dụng trong các lĩnh vực” Tạp chí

Cơ Khí Việt Nam, số 9, tr 36-45

2 Đoàn Thanh Xuân, Vũ Toàn Thắng, Đặng Thái Việt, Vũ Tiến Dũng, (2022) “Xây dựng bộ đôi số cho rô bốt UR3 trong hệ thống lắp ráp bóng đèn” Tạp chí Cơ khi Việt Nam, số 291, tr 14-17

3 Doan Thanh Xuan, Le Giang Nam, Dang Thai Viet, Vu Toan

Thang “A-star Algorithm for Robot Path Planning Based on Digital Twin” (2022) In: Le, AT., Pham, VS., Le, MQ., Pham, HL (eds) The AUN/SEED-Net Joint Regional Conference in Transportation, Energy, and Mechanical Manufacturing Engineering RCTEMME 2021 Lecture Notes in Mechanical Engineering Springer, Singapore https://doi.org/10.1007/978-981-19-1968-8_8

4 Doan Thanh Xuan, Vu Toan Thang, (2022) “Investigation on

the influence of obstacle size in path planning by a hybrid model combining an improved A-star algorithm and digital twin” The Third International Conference on Material, Machines, and Methods for Sustainable Development 2022, pp 2195-4364 (accepted for publication)

5 Doan Thanh Xuan, Tran Van Huynh, Nguyen Thanh Hung,

Vu Toan Thang “Applying Digital Twin and Multi-Adaptive Genetic Algorithms in Human–Robot Cooperative Assembly Optimization” Applied Sciences 2023; 13(7):4229 https://doi.org/10.3390/app13074229

INTRODUCTION

1 The Reasons for Choosing The Dissertation Topic

Simulating a process or a system helps humans gain a deeper understanding of the system, allowing for flexibility to adjust and change device parameters as well as to perform optimization testing during the initial planning phase Due to the advantages mentioned above, simulation has been widely researched and applied However,

it has mostly been limited to static models and one-way information exchange from physical objects to digital entities The concept of the digital twin, aiming to create a digital counterpart that faithfully describes a physical device and can exchange data with it, has been gaining increasing attention from researchers Along with the strong development of digital technology and new-generation information, the ability to collect more data and process it efficiently has provided

a strong foundation for the development of digital twins

The digital factory has become an inevitable trend in various industries, with the digital twin being a fundamental unit; synthesizing digital twins forms a digital factory Research on building digital twins for robots is a relatively new field, meeting practical industrial demands and promising strong development prospects in the coming years, both domestically and internationally This research is based on theoretical studies and experiments to construct a digital model that faithfully describes a robot and establishes communication connections between the physical entity and the digital model The experiments are conducted on the UR3 robot within a lightbulb assembly system The digital twin of the UR3 robot, once created, continues to be applied in tasks such as pathfinding and optimizing collaboration between humans and robots

The application of robots is becoming increasingly popular, in line with the development of the automation industry Current research in robotics continues to grow due to high market demand, a wide range

of applications, and technical development potential Particularly, robot path planning has been extensively studied and applied across various generations of algorithms Optimizing path planning for

robots helps reduce time and costs in robot operation processes One

of the widely applied algorithms for robot path planning is the A* algorithm The A* algorithm was invented, and many improvements have been made to enhance its efficiency in the original path planning algorithm In industrial production lines, we often encounter many collaborative work scenarios between humans and robots In these collaborative systems, the advantages of robots, such as speed and precision, can be combined with human qualities like dexterity and adaptability in different situations

Through an overall analysis of the digital twin and industrial robotics problem, the postgraduate has recognized that the research and development of digital twins for industrial robots is a promising research direction that can still be explored in various aspects; in particular, applying digital twins to industrial robots to enhance performance in the operations of digital factories This is the reason

why the postgraduate chose the topic "Research on Developing Digital Twins-Based Application for Industrial Robots" In this

Dissertation, the postgraduate focuses on researching methods for developing digital twins-based application for industrial robots with reference to solving problems related to robot path planning and human-robot collaboration

2 Goal, Objectives, Scope, and Research Methodology

Goal

The goal of the Dissertation is to gradually master the construction

of industrial robots, develop software to enable the application of digital twins in optimizing robot paths and coordinating activities between humans and robots

Specific objectives include:

- Developing digital twins-based application for industrial robots

- Proposing improvements to some digital twins-based applications for industrial robots with reference to solving problems related to pathfinding and human-robot collaboration

Objectives

The research subjects of the Dissertation are: Digital twin technology, industrial robots, methods for developing digital twins-based application for industrial robots, methods for robot pathfinding and human-robot collaboration

Scope

The scope of the Dissertation research is focused on developing digital twins and their applications for the UR3 robot within the lightbulb assembly system at the Digital Factory Laboratory of the Hanoi University of Science and Technology

Research Methodology

The research methodology involves a combination of theoretical study and practical experimentation Based on the theoretical research, including literature and previous works conducted both nationally and internationally on digital twins, industrial robotics, robot pathfinding methods, and human-robot collaboration, the study aims to understand the development of digital twins-based application for the UR3 robot Using the findings from the theoretical research, the study proceeds to implement digital twins in a real-world setting It continues to explore proposed improvements in applications related to robot pathfinding and human-robot collaboration The research involves conducting experiments, comparing theoretical and practical results, and publishing research outcomes in scientific journals and conferences, both nationally and internationally

The Scientific and Practical Significance of the Research Topic

In this research, a combination of the improved A* algorithm and digital twin technology has been used to plan the robot's path in the lightbulb assembly system The postgraduate highlights the advantages of the A* algorithm and the application of digital twin technology in mapping out the motion stages of the robot, tailored to propose an effective solution for the entire operational process Furthermore, the impact of obstacle sizes is evaluated in terms of the efficiency of one of the two methods to enhance path planning for the

robot when applied to a real-world system with obstacles of varying sizes

Furthermore, digital twins are applied to optimize the collaboration between humans and robots in the lightbulb assembly production line Firstly, digital twin technology is used to find the motion trajectory of the robot The digital twin of the robot and the human is created by combining cameras to track the position and activities of the human workers in the workspace This helps prevent collisions between humans and robots in the shared workspace Subsequently, an adaptive genetic algorithm is applied to calculate the optimal kinematics and movement schedule for the human workers To ensure uninterrupted operations and avoid material shortages, human workers need to observe and move to the material input conveyor and pallet input conveyor, supplying materials to the assembly system This is done to provide a continuous input of raw materials to the assembly line while allowing workers to perform their tasks in parallel with robot assembly operations The algorithm is designed to minimize the number of movements required for material retrieval, ensuring that the robot always has enough materials to follow the defined trajectory This results in labor savings and process optimization The combination of digital twin technology and the adaptive genetic algorithm optimizes the robotic movement and reduces the number of movements performed by human operators on the system

4 The new findings

The new findings achieved in the Dissertation are as follows:

- Development of Digital Twins-based application for the UR3 Robot in the lightbulb assembly system

- Application of digital twins in conjunction with the improved A* algorithm for robot pathfinding, comparing the cases with obstacles of constant height and varying height; consequently, proposing the use of a combination of digital twins and the improved A* algorithm suitable for each case

- Application of digital twins to optimize the coordination of human and robot activities in the lightbulb assembly system

The results show a reduction in the number of human movements to supply input materials to the system, ensuring that the robot can operate continuously without interruption due to a lack of materials such as lamp cap and bulb socket at the input of the assembly line

5 Dissertation structure

The Dissertation is divided into 4 chapters: Chapter 1: An Overview of Digitial Twins and Their Applications; Chapter 2:

Developing Digital Twins-based Application for Industrial Robots in

the Lightbulb Assembly System; Chapter 3: Application of Digital Twins in Pathfinding for Robots; Chapter 4: Application of Digital Twins in Human-Robot Collaboration; Conclusion; Future work

Chapter 1: OVERVIEW OF DIGITIAL TWINS AND THEIR APPLICATIONS

1.1 Introduction

The technology of Digital Twins has been in existence since the 2000s In recent years, the rapid development of information technology (such as big data, cloud computing, artificial intelligence, etc.) has created favorable conditions for the robust growth of Digital Twin technology, attracting significant attention from researchers with

a substantial increase in the number of research papers Digital Twins are not only used in a single phase but also have the ability to be used throughout the lifecycle of a system: from (i) planning, designing, optimizing parameters, testing, (ii) creating the actual system, (iii) operation, fault prediction, maintenance planning, lifespan prediction,

to (iv) ceasing operation, eliminating from the operational system Digital Twins have wide-ranging applications across various industries (information technology, transportation, aviation, mechanical engineering, construction, healthcare, etc.), helping optimize system operations and reduce costs Alongside this, there is

an increasingly widespread application of robotics in industrial production systems The problem of robot pathfinding and the

coordination between humans and robots are topics of significant interest with numerous practical applications

1.2 The History of Digital Twins Development and Milestones

Figure 1.1 summarizes the important milestones in the development process of Digital Twins according to the report [13]

Figure 1 1 Key Milestones in the development of digital twins technology

1.3 Digital twins concept and classification

On July 27, 2020, ISO standards for digital twins were published under the standard number ISO 23247, marking the emergence of standardized documentation for digital twins This standard builds upon all previous research However, this standard is still a framework and not a detailed protocol standard, focusing primarily on the production phase in factories

Based on the level of integration of digital twins, Kritzinger and colleagues in their research [46] have categorized digital twins into three types:

• Digital Model: In this type of digital twin, data between the physical object and the digital object is manually exchanged Therefore, any changes in the state of the physical object are not directly reflected in the digital one, and vice versa

• Digital Shadow: Data from the physical object is automatically transferred to the digital twin, but this is still done manually in the reverse direction As a result, any changes in the physical object can

be seen in its digital counterpart, but not automatically in the other direction

• Digital Twin: In this type of digital twin, there is a bidirectional automatic data stream between the physical object and the digital object Therefore, real-time changes in either the physical or digital object can lead to changes in the other object

1.4 Digital twin application

Digital twins can be applied in various industries and fields

Manufacturing is the most common research area for digital twins and represents a significant portion of digital twin research Most studies focus on optimizing production planning, simulating manufacturing, monitoring and predicting product performance, and aiming for sustainable production Another popular field is smart buildings and cities, which concentrate on monitoring the sustainability of structures, building management and control, optimizing project planning, and predicting maintenance Information and Communication Technology (ICT) is also a prominent focus in digital twin research and has wide applications in edge computing systems, communication security, and cloud service monitoring Next, energy-related digital twin research focuses on power systems, fault diagnosis, and optimizing power plant operations Research topics related to automotive engineering, aerospace, healthcare, and medical care have a similar proportion, mainly focusing on product monitoring, prediction, testing, and simulation Finally, educational applications of digital twins have been on the rise in recent years due

to the trend of online teaching and learning Various research fields are grouped together, including mining, agriculture, chemistry, etc [47]

Among these applications, there are two applications that have attracted significant attention from researchers and have many

practical applications in production operations: the use of digital twins

in robot path planning and in human-robot collaboration

Chapter 2: DEVELOPING A DIGITAL TWINS FOR THE INDUSTRIAL ROBOT IN THE LIGHTBULB ASSEMBLY SYSTEM

2.1 Lightbulb assembly system

The operational cycle of the system is described through an algorithm flowchart, as presented in Figure 2.2

Figure 2 2 Robot algorithm flowchart

2.2 Kinematic calculations and development of digital twin for the UR3 robot

Using the Denavit-Hartenberg (DH) matrix method [5,19,20], we establish coordinate systems for robots in the shape shown in Figure 2.23 After establishing these coordinate systems, we determine the table of DH dynamic parameters

By which, we obtain the dynamic equations system for the UR3 robot

as follows:

variables q 1 , q2, q3, q4, q5, and q6, as provided in the table below:

Modelling the UR3 robot

A simulation model (digital space in digital twin) is a dynamic environment built by incorporating 3D computer-aided design (CAD) objects into Tecnomatix Process Simulate software Tao [5] suggested that the virtual model consists of four layers, i.e geometry (generating 3D CAD objects), physical location (position of CAD objects in the scene), behavior (robotic kinetics) and rules (sequence of an assembly process) 3D kinetic CAD objects of the robot are available at the online library maintained by the Siemens Support Center

The creation of 3D objects is applied to all other parts and components of a system to be included in the Tecnomatix environment Tecnomatix can import CAD data in JT (Jupiter

Tessellation) format Building a digital model of the Robot UR3 (to simulate robotic geometry and dynamics) includes the following steps

as shown in Figure 2.10

Figure 2 10 Steps for building a digital model of the UR3 robot TCP/IP protocol is used to connect real-time communication connection between a real robot and a virtual robot via Ethernet port The “Live Mode” in Tecnomatix Process Simulate software is used to connect real and virtual robots as shown in Figure 2.12

Figure 2 12 Real time connection between a real robot and a virtual robot

Chapter 3: APPLICATION OF DIGITAL TWINS IN PATH FINDING FOR ROBOTS

3.1 Robot path planning problem

The role of appropriate path planning in manufacturing is crucial When robot path planning is done correctly, industrial robots can efficiently perform their subsequent tasks Path planning for robots plays a significant role in the following aspects: The accuracy of robots, Task repeatability, Product quality

3.2 The improved A* algorithm

The first improvement of the enhanced method is the local path between the current node and the target node, pre-planned in advance,