Báo cáo nghiên cứu khoa học: Design of two-wheeled self-balancing robot with computer vision

4 Table of Contents DESIGN OF TWO-WHEELED SELF-BALANCING ROBOT .... 7 DESIGN OF TWO-WHEELED SELF-BALANCING ROBOT CHAPTER 1 INTRODUCTION 1.1 Problem Statement Automation is becoming a

1

- Program: Automation and Informatics

- Address: Dong Da, Hanoi

- Phone no /Email: 22070160@vnu.edu.vn

II Academic results (from the first year to now)

III Other achievements:

- Robotacon WRO 2023 – Consolation prize

- UET Makerthon 2023 – 1st prize

Hanoi, April 17, 2024

Advisor Team Leader

(Sign and write full name) (Sign and write full name)

Bùi Thanh Tùng Nguyễn Thu Hương

3

1 Project Code:

CN.NC.SV.23_31

2 Member List:

3 Advisor:

Full name: Msc.Bui Thanh Tung

Faculty of Applied Sciences – International school, VNU

4

Table of Contents

DESIGN OF TWO-WHEELED SELF-BALANCING ROBOT 7

CHAPTER 1 7

INTRODUCTION 7

1.1 Problem Statement 7

1.2 Research Objectives 7

1.3 Research Methods 7

1.4 Structure of the Thesis 8

CHAPTER 2 8

THEORETICAL BASIS 8

2.1 Inverted Pendulum 8

2.2 PID Controller 8

2.3 Models and Symbols 9

2.4 Dynamic model of DC motor 10

2.5 Kinematic model of self-balancing 2-wheeled robot system 10

CHAPTER 3 14

3.1 Declaration, System Analysis, and Controller Design 14

3.1.1 System Parameters 15

3.1.2 System Stability Analysis 15

3.1.3 Analysis of System Controllability Based on the principles of automatic control for a linear system with a state-space equation as follows: 16

BALL DETECTION AND TRACKING USING RASPBERRY PI 19

CHAPTER 1 19

INTRODUCTION 19

CHAPTER 2 20

RESEARCH METHODOLOGY 20

2.1 Receiving Image From Camera 20

2.2 HSV Color Setting 21

2.3 Thresholding 21

5

2.4 Morphological Transformation 22

2.5 Draw Contour 22

2.6 Find the center 22

2.7 Draw Circle 22

CHAPTER 3 23

SOFTWARE REQUIREMENT 23

CHAPTER 4 23

HARDWARE REQUIREMENT 23

4.1 Raspberry Pi: 24

4.2 Webcam: 24

4.3 DC Motor: 25

4.4 Battery: 25

4.5 L298N Motor Driver: 25

CHAPTER 5 27

RESULT AND DISCUSSION 27

5.1 Problems solved 28

5.2 Problems unsolved 28

CHAPTER 6 28

CONCLUSION 28

6

LIST OF FIGURES

Figure 1 Inverted pendulum 8

Figure 2.PID controller 9

Figure 3.Models two wheels self balancing robot 9

Figure 4.Analyze the forces acting on the two wheels 10

Figure 5.Pole-Zero Map 16

Figure 6.Image processing techniques flowchart 20

Figure 7 HSV Trackbar 21

Figure 8.Block Diagram of Proposed System 23

Figure 9.Structure of Raspberry Pi 24

Figure 10.Webcam SWC-02 25

Figure 11.L298N Motor Driver 26

Figure 12.Results 27

7

DESIGN OF TWO-WHEELED SELF-BALANCING ROBOT

CHAPTER 1 INTRODUCTION

1.1 Problem Statement

Automation is becoming an indispensable development trend of the modern world, gradually replacing human labor with automated machinery in production lines In this context, the research and development of robots, especially mobile robots, have become a significant focus in the automation industry One of the most interesting topics is the self-balancing two-wheeled vehicle—a design that not only presents technical challenges but also has wide-ranging potential applications The self-balancing two-wheeled vehicle needs to be designed with a controller that can maintain balance and stability on all terrains, under all weather conditions, and in various situations This requires a deep understanding

of mechanics, electronics, and control algorithms, particularly the PID control algorithm,

to process signals from sensors and respond swiftly from the control system The success

of the self-balancing two-wheeled vehicle can help reduce accidents due to loss of control, increase efficiency in transporting goods in factories, and expand applications in everyday life Recognizing the importance and challenges of researching self-balancing two-wheeled vehicles, we have chosen the topic "Design of a Self-Balancing Two-Wheeled Vehicle with

a PID Controller" as the focal point for our scientific research project Although we still have limitations in knowledge and experience, we hope to receive understanding and support from our teachers and peers

1.2 Research Objectives

Gain a basic understanding of the concepts related to two-wheeled balancing robots Apply knowledge from various courses to successfully design a controller for a two-wheeled balancing robot

Understand the design process of a system and simulate it using Matlab

1.3 Research Methods

Review literature and online articles about self-balancing two-wheeled robots

Study the PID control algorithm

8

Validate theories through simulation and proceed with the actual product development

1.4 Structure of the Thesis

Chapter 1: Introduction to the Topic

Chapter 2: Theoretical Basis of PID Controllers

Chapter 3: Simulation

Chapter 4: Conclusion

CHAPTER 2 THEORETICAL BASIS

2.1 Inverted Pendulum

The inverted pendulum is a system where an object is allowed to rotate around an axis A distinctive feature is that the center of gravity of this object is located above the pivot point, creating an inherently unstable system Without precise control, the pendulum will freely fall to one side under the influence of gravity

We have a diagram that describes the inverted pendulum system as follows:

Figure 1 Inverted pendulum

2.2 PID Controller

An effective system is one that operates optimally based on a specific standard, achieving

a desired maximum or minimum value Achieving and maintaining this optimal state depends on quality requirements, external influences, and the operating conditions of the system One of the control tools that helps establish optimal operation for the system is the PID controller

9

Figure 2.PID controller

To design a PID controller, a negative feedback system must be established, where the error value between the actual output and the desired setpoint is measured This error is input into the PID controller, which then calculates and provides the necessary voltage to adjust the motors, aiming to bring the system back to the desired equilibrium position

2.3 Models and Symbols

We have the following model of a two-wheeled self-balancing vehicle:

Figure 3.Models two wheels self balancing robot

Here are the symbols for the two-wheeled self-balancing robot:

Kí Hiệu Đơn vị Ý nghĩa

𝜃𝑝 rad Tilt angle of the robot body

𝜃𝑊̇ 𝑟𝑎𝑑 𝑠⁄ Angular velocity of the wheel

𝑘𝑓 𝑁𝑚𝑠 𝑟𝑎𝑑⁄ Friction coefficient

10

𝑘𝑚 𝑁𝑚 𝐴⁄ Moment coefficient

𝐾𝑒 𝑉𝑠 𝑟𝑎𝑑⁄ Back emf coefficient of the motor

α 𝑟𝑎𝑑 𝑠⁄ 2 Angular acceleration of the engine

𝐻𝐿, 𝐻𝑅, 𝑃𝐿, 𝑃𝑅 Force acting between the wheel and the robot

chassis

l Distance from the center of the wheel to the

center of gravity of the vehicle body

𝐶𝐿, 𝐶𝑅 Motor torque acting on the wheel

𝐻𝑓𝐿, 𝐻𝑓𝑅, Friction force between the wheel and the

∑ 𝑀𝑜 = 𝐼𝛼 (4)

𝐼𝑤𝜃̈𝑤 = 𝐶𝑅 − 𝐻𝑓𝑅𝑟 (5) From the kinematic equation of the DC motor, we obtain the motor torque

𝜏𝑚= 𝐼𝑅𝑑𝜔

𝑑𝑡 + 𝜏𝑎 (6) From the kinematic equation and the differential equation of the DC motor, we obtain the output torque of the motor

𝑀𝑤𝑥̈ =−𝑘𝑚𝑘𝑒

𝑅𝑟 𝜃̇𝑤 +𝑘𝑚

𝑅𝑟 𝑉𝑎 −𝐼𝑤

𝑟 𝜃̈𝑤− 𝐻𝐿 (10) Equation for the right wheel

𝑀𝑤𝑥̈ =−𝑘𝑚𝑘𝑒

𝑅𝑟 𝜃̇𝑤 +𝑘𝑚

𝑅𝑟 𝑉𝑎 −𝐼𝑤

𝑟 𝜃̈𝑤− 𝐻𝑅 (11) Since linear motion exerts force on the motor shaft, angular velocity can be converted into linear velocity according to the following equation:

𝜃̈𝑤𝑟 = 𝑥̈ ⟹ 𝜃̈𝑤 =𝑥̈

𝑟 𝜃̇𝑤𝑟 = 𝑥̇ ⟹ 𝜃̇𝑤 =𝑥̇

𝑟

𝑟 − 𝐻𝑅Calculate the sum of the two equations for the wheels:

2(Mw+𝐼𝑤

𝑟2)𝑥̈ =−2𝑘𝑚𝑘𝑒

𝑅𝑟2 𝑥̇ +2𝑘𝑚

𝑅𝑟2 𝑉𝑎 − (𝐻𝐿 + 𝐻𝑅) (12)

Analysis of Forces Acting on the Wheel Body

Apply Newton's law to calculate the total force acting on the wheel in the horizontal direction

∑ 𝐹𝑥 = 𝑀𝑝𝑥̈ (𝐻𝐿+ 𝐻𝑅) − 𝑀𝑝𝑙𝜃̈𝑝𝑐𝑜𝑠𝜃𝑝+ 𝑀𝑝𝑙𝜃̇2𝑝𝑠𝑖𝑛𝜃𝑝 = 𝑀𝑝𝑥̈

We have

(𝐻𝐿 + 𝐻𝑅) = 𝑀𝑝𝑥̈ + 𝑀𝑝𝑙𝜃̈𝑝𝑐𝑜𝑠𝜃𝑝+ 𝑀𝑝𝑙𝜃̇2𝑝𝑠𝑖𝑛𝜃𝑝

13

Perpendicular force acting on the vehicle body:

∑ 𝐹𝑥𝑝= 𝑀𝑝𝑥̈𝑐𝑜𝑠𝜃𝑝(𝐻𝐿 + 𝐻𝑅)𝑐𝑜𝑠𝜃𝑝 + (𝑃𝐿 + 𝑃𝑅)𝑠𝑖𝑛𝜃𝑝 − 𝑀𝑝𝑔𝑠𝑖𝑛𝜃𝑝 − 𝑀𝑝𝑙𝜃̈𝑝 = 𝑀𝑝𝑥̈𝑐𝑜𝑠𝜃𝑝 (14) Total torque acting on the center of gravity of the vehicle body:

𝐼𝑝𝜃̈𝑝 −2𝑘𝑚 𝑘𝑒

𝑅𝑟 𝑥̇ +2𝑘𝑚

𝑅 𝑉𝑎 + 𝑀𝑝𝑔𝑙𝑠𝑖𝑛𝜃𝑝 + 𝑀𝑝𝑙2𝜃̈𝑝 = −𝑀𝑝𝑙𝑥̈𝑐𝑜𝑠𝜃𝑝 (18) Substituting equation 13 into equation 12, we obtain:

2(Mw+𝐼𝑤

𝑟 2)𝑥̈ = −2𝑘𝑚𝑘𝑒

𝑅𝑟 2 𝑥̇ +2𝑘𝑚

𝑅𝑟 2𝑉𝑎 − 𝑀𝑝𝑥̈ − 𝑀𝑝𝑙𝜃̈𝑝𝑐𝑜𝑠𝜃𝑝 + 𝑀𝑝𝑙𝜃̇2𝑝𝑠𝑖𝑛𝜃𝑝 (19) Equations (18) and (19) represent the nonlinear system of equations for the system To linearize the model above, we 𝜃𝑝 = 𝜋 + 𝜙 where ϕ is a small angle oriented vertically upward From this, we have:

𝑐𝑜𝑠𝜃𝑝 = −1 𝑠𝑖𝑛𝜃𝑝 = −𝜙

𝑀𝑝𝑔𝑙𝑅(Ip+ Mpl2)𝜙

𝑅𝑟2𝛼

0

𝑀𝑝𝑔𝑙𝛽𝛼

1

0 ]

[

𝑥𝑥̇

𝜙𝜙̇

] +[

02𝑘𝑚(𝐼𝑝+ 𝑀𝑝𝑙2−𝑀𝑝𝑙𝑟)

𝑅𝑟2𝛼02𝑘𝑚(𝑀𝑝𝑙 − 𝑟𝛽)

CHAPTER 3

3.1 Declaration, System Analysis, and Controller Design

To build a controller, the first thing we need to do is declare the system values and assess whether the system is controllable, observable, and stable

15

3.1.1 System Parameters

3.1.2 System Stability Analysis

Transfer Function

16

We can see the roots of the transfer function and its poles, from which we can conclude that according to the Routh-Hurwitz criterion, the system is unstable

Figure 5.Pole-Zero Map

3.1.3 Analysis of System Controllability Based on the principles of automatic control for a linear system with a state-space equation as follows:

𝑥̇ = 𝐴𝑥 + 𝐵𝑢

𝑦 = 𝐶𝑥 + 𝐷𝑢 With A ∈ ℝnxn, B ∈ ℝnxu, C ∈ ℝrxn, D ∈ ℝrxm

Construct the controllability matrix:

P = [B, AB, A2B, , An−1B]

The necessary and sufficient condition for a system described by state-space equations to

be controllable is that the rank of the controllability matrix P equals, where n is the number

of state variables in the system

We use MATLAB to verify the controllability of the system

17

The rank of matrix P equals the number of state variables of the system

Analysis of System Observability

We select the matrix : C = [1 0 0 0

0 0 1 0]

We have C.x(t) = [1 0 0 0

0 0 1 0] [

𝑥𝑥̇

𝜙𝜙̇

18

The rank of the matrix P is equal to the number of state variables in the system Comments: From the two verifications, we can conclude that the system is controllable and all four state variables of the system can be observed By designing the matrix C to observe two state variables—the position of the vehicle and the tilt angle of the vehicle—if these two variables are stable, then the entire system will be stable This approach ensures that

by monitoring these critical variables, we can gauge the overall system stability

19

BALL DETECTION AND TRACKING USING RASPBERRY PI

CHAPTER 1 INTRODUCTION

New technology is developing quickly as the future gets closer the development of new age robotics, artificial intelligence, and the internet of things As is customary, robots are crucial to industry since they are assigned certain tasks Robots are employed extensively

in many industries these days, including aerospace, defense, medical, and space travel Current robots are costly due to their sophisticated mechanisms and specialized nature Our suggested work is reasonably priced and comes with standard sensors like a camera, servo motor, and DC motor, among others Numerous applications, including video surveillance, human-computer interaction, vehicle navigation, and robot control, rely heavily on ball tracking The technology is designed to accurately identify and track the item The item is detected by the webcam Information is transmitted to Raspberry pi 3 after detection The item is tracked with the aid of a DC motor It is employed to track several items with various sizes, colors, and structural configurations But in real-world applications, a variety of elements, including changes in lighting, appearance, shape, partial occlusion, and camera motion, complicate the issue

A robot with legs is more flexible than one with four legs It can maneuver in many directions and across a variety of terrains, including confined areas These days, a lot of companies are replacing human labor with this kind of robot to carry out jobs like carrying items and doing activities in dangerous areas As a result, there are many possible applications Numerous applications in fields like sports, health, etc., that facilitate communication between humans and computers, assess patient motions during recuperation, operate robots, and guide automobiles mostly rely on ball tracking The purpose of this technology is to precisely identify and track objects

20

CHAPTER 2 RESEARCH METHODOLOGY

Figure 6.Image processing techniques flowchart

The procedures for software setting are:

2.1 Receiving Image From Camera

The raspberry will use the aforementioned software script to collect picture data from its internal camera and save it in the camera variable If the camera does not provide any data,

it will take a break

21

2.2 HSV Color Setting

By giving the HSV color detection limit value, one may set the HSV color space The following is the script:

The HSV color picker program is used to find the ball color's HSV value An illustration

of how to use the color picker program to select colors is as follows:

Figure 7 HSV Trackbar

2.3 Thresholding

The method of thresholding transforms a hsv color picture into a binary image in which the detected color image turns white if it matches a preset range value and black if it does not The software script that was utilized in this investigation is as follows:

The previously defined hsv color space is utilized as the range value The masking process

is another name for the thresholding procedure

2.6 Find the center

The following program script used in this study is as follows:

2.7 Draw Circle

The following step involves creating a circle with the parameters of the previously discovered contour findings The cv2.circle function is used to do this, and the cv2.minEnclosingCircle function is used to determine the circle's radius and center point position The study employed the subsequent software script, which is as follows:

23

CHAPTER 3 SOFTWARE REQUIREMENT

Python is an interpreted, dynamic, high-level, general-purpose programming language It makes it possible to create programs with an object-oriented design It is easy to use and understand, and it provides a wide range of high-level data structures It is a compelling scripting language for application development since it is strong but easy to learn

OpenCV is a massive open-source library for image processing, computer vision, and machine learning In essence, it plays a crucial part in real-time processes, which are fundamental to contemporary systems It may be used to search for people, objects, and even a person's genuine handwriting by looking through photos and videos Combining Python with additional libraries, such NumPy, makes handling the array of OpenCV structure for analysis simple

CHAPTER 4 HARDWARE REQUIREMENT

A robot with a camera installed on a Raspberry Pi will be part of the proposed system To identify the item, the recorded photos will be analyzed The robot will circle around the object in search of it until it is detected Once an item is recognized, the robot will track it

by controlling its motors The code on the Raspberry Pi will operate the robot's motors and camera mount to identify the same To identify the item, the recorded photos will be analyzed Instead of using the current system, we suggest using this one

Figure 8.Block Diagram of Proposed System