(Luận văn thạc sĩ) research, design, manufacture of distributed control service robot system

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT MECHATRONICS ENGINEERING TECHNOLOGY RESEARCH, DESIGN, MANUFACTURE OF DISTRIBUTED CONTROL SERVICE ROBOT SYSTEM h ADVISOR: ASSOC PROF NGUYEN TRUONG THINH STUDENT: NGUYEN THI TUONG VY HO THANH TUNG BUI CHI TOAN SKL010417 Ho Chi Minh City, January, 2023 HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF MECHANICAL ENGINEERING DEPARTMENT OF MECHATRONICS BACHELOR THESIS RESEARCH, DESIGN, MANUFACTURE OF DISTRIBUTED CONTROL SERVICE ROBOT SYSTEM h SUPERVISOR: ASSOC PROF NGUYEN TRUONG THINH STUDENT’S NAME: NGUYEN THI TUONG VY STUDENT’S ID STUDENT: 19146430 STUDENT’S NAME: HO THANH TUNG STUDENT’S ID STUDENT: 19146421 STUDENT’S NAME: BUI CHI TOAN STUDENT’S ID STUDENT: 19146405 Ho Chi Minh City, January 2023 HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF MECHANICAL ENGINEERING DEPARTMENT OF MECHATRONICS BACHELOR THESIS RESEARCH, DESIGN, MANUFACTURE OF DISTRIBUTED CONTROL SERVICE ROBOT SYSTEM h SUPERVISOR: ASSOC PROF NGUYEN TRUONG THINH STUDENT’S NAME: NGUYEN THI TUONG VY STUDENT’S ID STUDENT: 19146430 STUDENT’S NAME: HO THANH TUNG STUDENT’S ID STUDENT: 19146421 STUDENT’S NAME: BUI CHI TOAN STUDENT’S ID STUDENT: 19146405 Ho Chi Minh City, January 2023 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF MECHANICAL ENGINEERING CAPSTONE PROJECT EVALUATION FORM (FOR ADVISOR USE ONLY) Title of thesis: Research, Design and Manufacture of Distributed Control Service Robot System Major: Mechatronics Committee Number Student’s name 01 (SN 01): Nguyen Thi Student’s ID: 19146430 Tuong Vy Student’s name 02 (SN 02): Bui Chi Toan Student’s ID: 19146405 Student’s name 03 (SN 03): Ho Thanh Tung Student’s ID: 19146421 Advisor: Assoc.Prof Nguyen Truong Thinh Name of Institute: University of Technology and Education COMMENTS COMMENTS ON ATTITUDE AND BEHAVIOR OF STUDENTS h COMMENTS ON RESULTS OF CAPSTONE PROJECT 2.1 Structure of the capstone project 2.2 Main contents 2.3 Results of capstone project i 2.4 Capstone strengths and weaknesses EVALUATION No CONTENT MAX ACHIEVED POINT POINT Structure of the capstone project 30 Student follows exactly the format for capstone project given by 10 FME The motivation of the project is clearly provided in the thesis 10 The NEED of project is clearly showed in the thesis 10 Main contents (demonstration that students have ability to): 50 Apply knowledge of math, engineering, and science Analyze and interpret data 10 Design and manufacturing the system, component or process to 15 meet needs h Improvement and development in future 15 Use the software and technical tool to solve the problem Real-life applications of capstone project 10 Products of capstone project 10 Total 100 CONCLUSIONS  Accept  Reject HCMC, dd/mm/yy: Advisor (Signature and Name) ii HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF MECHANICAL ENGINEERING CAPSTONE PROJECT EVALUATION FORM (FOR REVIEWER USE ONLY) Title of thesis: Research, Design and Manufacture of Distributed Control Service Robot System Major: Mechatronics Committee Number Student’s name 01 (SN 01): Nguyen Thi Student’s ID: 19146430 Tuong Vy Student’s name 02 (SN 02): Bui Chi Toan Student’s ID: 19146405 Student’s name 03 (SN 03): Ho Thanh Tung Student’s ID: 19146421 Advisor: Assoc.Prof Nguyen Truong Thinh Name of Institute: University of Technology and Education COMMENTS Structure of the capstone project h Main contents Results of capstone project Capstone strengths and weaknesses Questions and Suggestions iii EVALUATION No CONTENT MAX ACHIEVED POINT POINT Structure of the capstone project 30 Student follows exactly the format for capstone project given by 10 FME The motivation of the project is clearly provided in the thesis 10 The NEED of project is clearly showed in the thesis 10 Main contents (demonstration that students have ability to): 50 Apply knowledge of math, engineering, and science 10 Design and manufacturing the system, component or process to 15 h Analyze and interpret data meet needs Improvement and development in future 15 Use the software and technical tool to solve the problem Real-life applications of capstone project 10 Products of capstone project 10 Total 100 CONCLUSIONS  Accept  Reject HCMC, dd/mm/yy: REVIEWER (Signature and Name) iv ACKNOWLEDGEMENT First of all, we would like to express our sincere thanks to Ho Chi Minh City University of Technology and Education and Openlab laboratory for creating opportunities and favorable conditions during the learning and training process , so that we can absorb more useful knowledge in the classroom learning process as well as valuable experiences in practice Next, we would like to express our deepest gratitude Assoc Prof Nguyen Truong Thinh, who has imparted to us many useful specialized knowledge and social knowledge The team is extremely grateful to the teacher's enthusiastic guidance, thanks to his close monitoring and all-time support, which has promoted the progress of the group's project, the team was able to complete the proposed schedule out initially We would also like to thank the Open Lab members for their support during the implementation of the project However, our knowledge still has certain limitations Therefore, it is inevitable that there will be shortcomings in the process of completing this thesis We are looking h forward to receiving sincere comments from the teachers so that we can gain more valuable experience and improve our project Once again, our team sincerely thanks! v TÓM TẮT ĐỀ TÀI Hiện nay, robot dần trở nên phổ biến áp dụng rộng rãi nhiều lĩnh vực khác giúp cho sống người trở nên tiện lợi Các loại robot công nghiệp nhà máy, doanh nghiệp sử dụng nhiều, góp phần cải tiến quy trình sản xuất giảm thiểu sức lao động người tăng độ xác cho hệ thống Về lĩnh vực dịch vụ, phương pháp dùng để thu hút khách hàng hiệu áp dụng cơng nghệ đại, tiên tiến vào quy trình dịch vụ, điều gợi tò mò khách hàng thúc đẩy đến trải nghiệm mẻ Áp dụng robot vào quy trình phục vụ khách hàng ứng dụng áp dụng nhiều robot lĩnh vực Trong dự án ban đầu đề ra, chủ yếu tập trung nghiên cứu, thiết kế, chế tạo robot dịch vụ mang thức ăn, nước uống, tương tác với khách hàng di chuyển tự động quỹ đạo lập trình sẵn Nhận thấy cần thiết tính sáng tạo đổi cao hơn, dự án phát triển thêm hệ phân tán robot Ở dự án này, tập trung nghiên cứu phương pháp điều khiển phân tán đa robot, sử dụng robot dịch vụ đảm nhiệm vai trị riêng h mơi trường làm việc nhà hàng để phục vụ cho mục tiêu nghiên cứu Các robot với chức năng: lễ tân, nhân viên gọi món, nhân viên phục vụ Các robot giao tiếp phối hợp với làm việc, tiếp nhận liệu, thông tin khách hàng truyền đạt với hoàn toàn tự chủ Đặc biệt hơn, robot biết ghi nhớ vị trí nhau, vơ tình ngang tự tránh né cách dễ dàng Ngồi ra, hệ robot phân tán lập trình chỉnh sửa chức cho làm việc môi trường khác bệnh viện, khách sạn,… Kết thực nghiệm cho thấy, robot làm việc với chức đặt ra, hệ thống hoạt động tốt, truyền thông tin đầy đủ xác, khách hàng trải nghiệm đánh giá cao, thích thú, đầu tư phát triển để đưa vào vận hành thực tế vi ABSTRACT IN ENGLISH Currently, robots are gradually becoming popular and widely applied in many different fields to make human life more convenient Industrial robots are being used a lot by factories and businesses, it contributes to improving the production process as well as reducing human labor and increasing the accuracy of the system In the service sector, one of the most effective methods to attract customers today is to apply modern and advanced technologies to the service process, which has aroused the curiosity of customers and promote new experiences Applying robots to the customer service process is one of the most applied applications of robots in this field In the initial project, mainly focused on researching, designing and manufacturing service robots that can carry food, drink, interact with customers and can move automatically on the orbits that have been determined pre-programmed Realizing the need and higher innovation creativity, the project further develops the distributed robot system In this project, focus on researching multi-robot distributed control methods, using service robots to take on separate roles in a restaurant working environment to serve the main research h objective Robots with functions: reception, order staff, food service staff The robots can communicate and coordinate with each other to work, receive data, customer information and communicate with each other completely autonomously More specifically, the robots can know and remember each other's positions, if they accidentally pass each other, they can dodge themselves easily In addition, the distributed robot system can be programmed to edit functions so that it can work in other environments such as hospitals, hotels, etc Experimental results show that the robots work with the correct functions performance, the system works well, transmits complete and accurate information, is highly appreciated and enjoyed by customers, and can be invested in more development to put into practice vii Motion kinematics of the differential wheels, selfpedestal selecting wheels Active wheel diameter D=145mm Robot operating time hours (45Ah) Charging time hours 7.3 Mechanical system Because the operating environment of the robot is mainly crowded places, with food and water, it is necessary to ensure the safety of users as well as to ensure that the electrical and induction systems are not affected by external factors Because the base, like the brain of the robot, should be carefully wrapped, each food compartment is emptied inside to stabilize food and water to avoid breakage As for the display screen, the design team placed it in an elevated position in order to minimize the influence of children's curiosity 7.4 Distributed service robot system 7.4.1 Convert coordinates h Camera calibration data and results Figure 7.2 Camera calibration results After we have done the camera calibration, we have the result as shown below 71 Figure 7.3 Results before and after camera calibration Conclusions: Results after calibration are not completely correct ratio between pixels The value is at a relative level It is necessary to re-align the pixel ratio in the positions Calibration matrix applies only to stationary and a fixed angle camera When the camera angle is shifted, recalibration must be performed 7.4.2 Experiment with pixel ratio Once we have calibrated the camera, we align the pixel ratio at the positions and conduct experiments Determine the coordinate angle under the camera viewing area, from which the pixel value is used to convert to the actual distance according to the coordinate angle h At the same time, we also check the actual values to compare with the converted value After conducting, we get the actual distance value measured from the camera and the real distance value measured shown in the following chart: Figure 7.4 Diagram showing the camera position and the actual location 72 Comment on result: - From the origin -> 300cm: The position deviation from reality is negligible, the error is about 0.1 -> (cm) - From 300 -> 500 cm : This deviation increases from 1-3 cm ⇒ The position from the coordinate angle back after pixel scaling has an error, so the overall position is deviation from 0.1-> cm 7.4.3 YOLO’s training results The training process is performed for about 10000 epochs with a total time about hours mAP during training also increases with each epoch up to 80% and log loss converge from epoch 9000 h Figure 7.5 YOLO Loss and mAP 73 7.4.4 Possibility of positioning Figure 7.6 Diagram showing the position of the robot moving from Home to table Table 7.2 Error value through experiments when the robot moves from Home to table ∆X(mm) 0.495 1.02 20 25 0.495 1.015 15 20 0.5 0.5 1.022 22 22 0.5 0.495 1.02 20 25 0.5 0.5 1.02 20 20 Xset Yset XMeasured 0.5 0.5 h YMeasured n Average error value ∆Y(mm) ∆Error(mm) 22.4 With: ∆X = |Xset – XMeasured| ∆Y = |Yset – YMeasured| ∆Error = ∆X + ∆Y 74 Figure 7.7 The graph shows the error when the robot moves from home to table position h Figure 7.8 Diagram showing the position of the robot move from Table to Table Table 7.3 Error value through experiments when the robot moves from table to table XMeasured YMeasured ∆X(mm) ∆Y(mm) ∆Error(mm) n Xset Yset 1 0.968 1.998 32 37 2 0.983 17 17 0.975 1.991 25 34 0.987 2.01 13 10 23 75 0.977 1.98 23 20 Average error value 43 30.8 With: ∆X = |Xset – XMeasured| ∆Y = |Yset – YMeasured| ∆Error = ∆X + ∆Y Figure 7.9 The graph shows the error when the robot moves from Table to Table h Conclusions on the robot's movement: - The error is less than 0.05(m) for the desired position when reaching the predetermined points - Errors have many causes: • Rounding error during calculation • Accumulated pixel ratio errors while system is running • The signal from the camera is slow due to the network signal, this affects the positioning process 76 7.4.5 Communication and data transmission between robots Figure 7.10 Path planning robots h Figure 7.11 Data transmission of robots Conclusions: the robot was able to create the most optimal path using the A* algorithm Since the system can communicate by transferring data between robots, helping robots to have status and position information of other robots, thereby preventing collisions between robots during operation In the figure 7.10, it is easy to see that the most optimal path for robot is to go straight from the point (30,40) to (30,10) But then there will be a collision with robot 1, so robot creates another optimal path to move 77 7.4.6 Robot - Human Interactive System Figure 7.12 LSTM model accuracy after training 500 epochs The process of training the LSTM model does not take too much time because the model structure is quite simple and the actual amount of data is not much We can see that the model converges with 100% accuracy after only 330 epochs h Figure 7.13 Naive Bayes predict on example Figure 7.14 Naive Bayes predict on example 78 Figure 7.15 LSTM model predict on example h Figure 7.16 LSTM model predict on example As we can see in Figures 7.13 - 7.16 the Naive Bayes classifier predicted different examples both of which are class: “plus_chicken_1” Because basically Naive Bayes classifier predicts based on the number of occurrences of words in a sentence Considering the two examples, we can see that the number of times the words appear in the sentence is the same Obviously, we can see that LSTM has outperformed Naive Bayes classifier completely because LSTM predicts semantically, not simply counting the number of occurrences of each word like Naive Bayes 79 CHAPTER 8: CONCLUSIONS AND RECOMMENDATIONS 8.1 Conclusions The project presented an overview of Distributed Service Robot System along with related research works, building a distributed control robot system, solving the robot positioning problem, creating the optimal trajectory, the ability identify customer intent After the experiment, the group obtained the following results: • The system operates relatively stably, the data transmission rate between robots is relatively high Transmitted data will always be full due to TCP Socket's loss detection and retransmission • The robot system always has information of other robots: what the customer ordered, the robot's trajectory, etc to help with the robot movement and system operation • Ability to recognize customer intent with high accuracy with trained data set But still not able to recognize the untrained speech • Ability to detect and calculate robot position with high accuracy Although there h are still many shortcomings such as the control algorithm is still basic, the processing speed of the program is not high, but in general, the program has achieved the requirements of the proposed project 8.2 Recommendations Based on the work done, the project implementation team found that the system can further develop the following contents: - Improve the object detection by increase more data and try to use different model’s architecture to increase the efficiency of the positioning process Research using other positioning algorithms such as: ROS, to compare and choose a more effective method - Research solutions that the system can be used in many different spaces - Design facial recognition check-in software, create a database to store each customer's information such as name, favorite dishes, from there to suggest suitable dishes 80 VIETNAMESE REFERENCES [5] Trịnh Tuấn Dương (2019), “Tìm đường cho Mobile robots dựa việc tìm đường biên áp dụng thuật tốn A*”, Cơ khí Việt Nam, số [25] Bong-Su Cho, Woo-sung Moon, Woo-Jin Seo and Kwang-Ryul Baek , “A dead reckoning localization system for mobile robots using inertial sensors and wheel revolution encoding”, Bài báo khoa học đại học Hàn Quốc [26] Le Duc Hanh & Nguyen Duy Anh, “HOẠCH ĐỊNH VÀ BÁM ĐƯỜNG ĐI CHO MOBILE ROBOT ỨNG DỤNG ĐIỀU KHIỂN HỒI TIẾP ẢNH” Báo khoa học đại học Bách Khoa TP.HCM [27] Nguyễn Trường Thịnh, “Giáo trình kỹ thuật robot”, Nhà xuất ĐHQG [28] Trịnh Chất, Lê Văn Uyển Tính tốn thiết kế hệ dẫn động khí tập 1, NXB Giáo dục h 81 ENGLISH REFERENCES [1] Chen, Cheng, et al "The adoption of self-driving delivery robots in last mile logistics." Transportation research part E: logistics and transportation review 146 (2021): 102214 [2] Seo, Kyung Hwa, and Jee Hye Lee "The emergence of service robots at restaurants: Integrating trust, perceived risk, and satisfaction." Sustainability 13.8 (2021): 4431 [3] Pinillos, Roberto, et al "Long-term assessment of a service robot in a hotel environment." Robotics and Autonomous Systems 79 (2016): 40-57 [4] “How to measure distance using Ultrasonic sensor”, https://mbedgeek.blogspo t.com/2018/12/ultrasonic-sensor-arduino-HC-SR04.html [Accessed 9/12/2022] [6] Ferguson, D.; Likhachev, M.; Stentz, A A Guide to Heuristic-based Path Planning In Proceedings of the International Workshop on Planning under Uncertainty for Autonomous Systems, International Conference on Automated Planning and Scheduling (ICAPS), Monterey, CA, USA, 5–10 June 2005; pp 9–18 h [7] Gass, S.I.; Harris, C.M Near-optimal solution In Encyclopedia of Operations Research and Management Science; Gass, S.I., Harris, C.M., Eds.; Springer: New York, NY, USA, 2001; p 555 [8] Hernández, C.; Baier, J.; Achá, R Making A* run faster than D*-lite for pathplanning in partially known terrain In Proceedings of the International Conference on Automated Planning and Scheduling, ICAPS, Portsmouth, NH, USA, 21–26 June 2014; Volume 2014, pp 504–508 [9] Cui, X.; Shi, H A*-based Pathfinding in Modern Computer Games Int J Comput Sci Netw Secur 2010, 11, 125–130 [10] Gunawan, S.A.; Pratama, G.N.P.; Cahyadi, A.I.; Winduratna, B.; Yuwono, Y.C.H.; Wahyunggoro, O Smoothed A-star Algorithm for Nonholonomic Mobile Robot Path Planning In Proceedings of the 2019 International Conference on Information and Communications Technology (ICOIACT), 2019; pp 654–658 Available online: https://icoiact.org/ (accessed on October 2020) [11] Zhou, Y.; Cheng, X.; Lou, X.; Fang, Z.; Ren, J Intelligent Travel Planning System based on A-star Algorithm In Proceedings of the 2020 IEEE 4th Information 82 Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China, 12–14 June 2020; Volume 1, pp 426–430 [12] Belhaous, S.; Baroud, S.; Chokri, S.; Hidila, Z.; Naji, A.; Mestari, M Parallel implementation of A* search algorithm for road network In Proceedings of the 2019 Third International Conference on Intelligent Computing in Data Sciences (ICDS), Marrakech, Morocco, 28–30 October 2019; pp 1–7 [13] Dong, G.; Yang, F.; Tsui, K.L.; Zou, C Active Balancing of Lithium-Ion Batteries Using Graph Theory and A-Star Search Algorithm IEEE Trans Ind Inform 2021, 17, 2587–2599 [14] Liu, H.; Shan, T.; Wang, W Automatic Routing Study of Spacecraft Cable based on A-star Algorithm In Proceedings of the 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 12–14 June 2020; pp 716–719 [15] Kider, J.T.; Henderson, M.; Likhachev, M.; Safonova, A High-dimensional planning on the GPU In Proceedings of the 2010 IEEE International Conference on h Robotics and Automation, Anchorage, AK, USA, 3–7 May 2010; pp 2515–2522 [16] Cheng, L.; Liu, C.; Yan, B Improved hierarchical A-star algorithm for optimal parking path planning of the large parking lot In Proceedings of the 2014 IEEE International Conference on Information and Automation (ICIA), Hailar, Hulun Buir, China, 28–30 July 2014; pp 695–698 [17] Yao, J.; Lin, C.; Xie, X.; Wang, A.; Hung, C.C Path Planning for Virtual Human Motion Using Improved A* Star Algorithm In Proceedings of the 2010 Seventh International Conference on Information Technology: New Generations, Las Vegas, NV, USA, 12–14 April 2010; pp 1154–1158 [18] Chen, T.; Zhang, G.; Hu, X.; Xiao, J Unmanned aerial vehicle route planning method based on a star algorithm In Proceedings of the 2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA), Wuhan, China, 31 May–2 June 2018; pp 1510–1514 [19] Baker, Z.K.; Gokhale, M On the Acceleration of Shortest Path Calculations in Transportation Networks In Proceedings of the 15th Annual IEEE Symposium on 83 Field-Programmable Custom Computing Machines (FCCM 2007), Piscataway, NJ, USA, 23–25 April 2007; pp 23–34 [20] Seo, W.; Ok, S.; Ahn, J.; Kang, S.; Moon, B An Efficient Hardware Architecture of the A-star Algorithm for the Shortest Path Search Engine In Proceedings of the 2009 Fifth International Joint Conference on INC, IMS and IDC, Seoul, Korea, 25–27 August 2009; pp 1499–1502 [21] “A* Search Algorithm”, https://www.geeksforgeeks.org/a-search-algorithm/ [Accessed 9/12/2022] [22] “A*’s Use of the Heuristic”, http://theory.stanford.edu/~amitp/GameProgrammi ng/Heuristics.html [Accessed 9/12/2022] [23] “Understanding LSTM Networks”, https://colah.github.io/posts/2015-08- Understanding-LSTMs/ [Accessed 9/12/2022] [24] “LSTM for Text Classification in Python”, https://www.analyticsvidhya.com /blog/2021/06/lstm-for-text-classification/ [Accessed 9/12/2022] h 84 h S K L 0