Development of autonomous delivery vehicle in university campus

AN OVERALL PERSPECTIVE ON THE TOPIC

Introduction

In recent years, the integration of transportation and technology has led to groundbreaking advancements, particularly in the development of autonomous vehicles These driverless vehicles can navigate and carry out essential tasks independently, utilizing advanced sensors to perceive their environment, transforming what once felt like science fiction into reality.

Autonomous vehicles utilize advanced remote-sensing technologies such as radar, GPS, cameras, and Lidar to monitor their surroundings and generate a 3-D map This mapping includes critical elements like street infrastructure, other vehicles, pedestrians, traffic signals, and road signs Equipped with powerful computer systems, these vehicles process the collected data to make real-time decisions, continuously adjusting steering, speed, acceleration, and braking in response to dynamic changes in their environment.

Figure 1.1 Some essential components on a self-driving vehicle Source: from “The role of artificial intelligence in autonomous vehicles”, by Anton

Machine learning and artificial intelligence are essential components of autonomous vehicle systems These technologies allow vehicles to analyze complex data and enhance their operational algorithms, improving their navigation capabilities Additionally, artificial intelligence empowers vehicle systems to make autonomous decisions, enabling them to respond effectively to various driving scenarios without requiring detailed instructions for each situation.

The ongoing evolution of Level 4 and Level 5 autonomous vehicles is a significant trend in the automotive industry Level 4 vehicles can drive autonomously in designated conditions or areas, while Level 5 vehicles strive for full autonomy in all driving situations These technological advancements are advancing self-driving capabilities, moving us closer to a future where human drivers may no longer be necessary.

As of 2023, vehicles equipped with partially automated systems like lane-keeping assist, adaptive cruise control, and traffic jam assist are available on the market, but fully autonomous cars remain unavailable for purchase Experts predict that stage 4 automation technology may be accessible to consumers by 2030 to 2035.

1.1.2 The urgency of the topic

Imagine the vibrant atmosphere of a university campus, alive with students, faculty, and staff engaged in various activities In this bustling environment, the need for food, textbooks, and other essentials often arises, but busy schedules can make it challenging to obtain them personally Autonomous delivery vehicles offer an innovative solution, enhancing both efficiency and convenience for the campus community.

Envision a future where students can easily order items through a user-friendly mobile app, with requests integrated into a campus delivery system This system utilizes autonomous vehicles to efficiently collect and deliver items directly to students, enhancing convenience and saving time Additionally, this innovative delivery method supports sustainability by reducing individual transportation and optimizing delivery routes.

The integration of autonomous delivery vehicles on university campuses promotes innovation and collaboration, enabling students and faculty to engage in research projects focused on autonomous technologies This initiative fosters interdisciplinary studies, encouraging students from diverse fields to collaborate and tackle real-world challenges such as navigation, route optimization, and pedestrian interaction in autonomous vehicle systems.

The emergence of autonomous delivery vehicles marks a significant advancement in transportation and logistics, particularly on university campuses These innovative vehicles have the potential to revolutionize the delivery of goods and services, providing enhanced convenience for students and faculty alike By adopting this state-of-the-art technology, universities can improve operational efficiency and meet the growing demand for seamless delivery solutions.

3 efficiency of their services but also empower their students to be at the forefront of groundbreaking advancements in transportation

1.1.3 Types of Autonomous Delivery Vehicles

1.1.3.1 Categorized by the vehicle’s loading types

Towing refers to the process of linking various objects so they can be pulled by a designated power source A towing vehicle, often referred to as a tow vehicle, is specifically engineered to tow other items efficiently.

The MiR250 Hook solution enhances autonomous cart towing capabilities, offering improved efficiency and versatility for various industrial applications This innovative technology allows for seamless integration into existing workflows, significantly reducing manual labor and optimizing logistics processes The autonomous towing vehicle is designed to navigate complex environments, ensuring safe and reliable operation in diverse settings With its advanced features, the MiR250 revolutionizes material handling, making it a valuable asset for modern warehouses and production facilities.

A unit load vehicle (ULV) is a specialized vehicle engineered for the efficient transportation of unit loads, which are collections of individual items or cargo bundled together as a single unit These vehicles are specifically designed to handle unitized loads effectively, ensuring streamlined logistics and enhanced cargo management.

Figure 1.3 Unit load autonomous vehicle

Source: from “Fully automated driving under certain conditions allowed in Japan”, by the Japan times, 2023 (https://www.japantimes.co.jp/news/2023/04/02/national/level-4- fully-automated-driving-allowed-japan/)

AGVs are engineered for seamless integration with conveyor systems, autonomously navigating to receive delivery packs placed on conveyor belts After loading, these AGVs transport packages to their designated locations within the facility, making them ideal for high-throughput environments where conveyor systems are already established.

Figure 1.4 Conveyor integrated AGV Source: from Intech group

(https://intech-group.vn/it-agv-c-agv-with-a-conveyor-belt-on-its-back-sp96.htm)

Automated Guided Vehicles (AGVs) that integrate forklift capabilities are designed to autonomously lift and transport palletized or unitized loads These advanced AGVs navigate to specified locations, self-position, and efficiently load or unload delivery packages using their forklift mechanisms Ideal for environments that demand flexible forklift operations, these automated systems offer the significant benefits of autonomous navigation and streamlined material handling.

Figure 1.5 Forklift AGV Source: from Asseco CEIT

1.1.3.2 Categorized by the vehicle’s path planning system

Automated Guided Vehicles (AGVs) operate along predetermined paths within a facility, which are often marked by physical indicators like magnetic tapes or embedded wires These fixed path AGVs are ideal for settings with clearly defined and consistent routes, such as assembly lines or production floors.

Scientific and practical meanings

Exploring autonomous delivery vehicles on university campuses is both scientifically and practically significant, attracting the interest of researchers and students This initiative provides a unique opportunity to engage with advanced transportation technology while addressing real-world campus logistics challenges, effectively linking innovative research with practical applications.

Exploring autonomous delivery vehicles on university campuses reveals the intricate interplay of artificial intelligence, machine learning, and robotics This interdisciplinary environment fosters collaboration among experts, enhancing our understanding of self-navigation, path planning, and human-vehicle interaction Such scientific inquiry not only advances autonomous technologies but also expands the horizons of innovation in the field.

Implementing autonomous delivery vehicles on campus enhances the daily lives of students, professors, and staff by providing seamless access to essential items through user-friendly mobile apps This innovation transforms the academic experience into a more convenient and efficient journey Additionally, these vehicles serve as a practical testing ground, collecting valuable data as they navigate campus pathways and interact with pedestrians This data aids researchers in refining algorithms, ultimately improving the performance, safety, and reliability of autonomous systems in real-world scenarios.

Objectives of the thesis

The objectives of studying the field of autonomous delivery vehicles within a university campus are:

• Research and development technologies including image processing, machine learning, embedded systems which are essentially used in autonomous vehicles

• Develop guidelines and best practices for the deployment and operation of autonomous delivery vehicles within a university campus, considering regulatory compliance, privacy concerns, and ethical considerations.

Research scope

This graduation project is designed to be completed within a limited timeframe, necessitating a significant reduction in its scope to ensure its feasibility and successful execution.

This project focuses on developing an autonomous vehicle capable of navigating a university campus Additionally, it includes the creation of a user-friendly website where users can input their desired destinations for the vehicle's delivery service.

• The vehicle must be able to detect people and objects on the road and detect the road lane to move on the road

• The operating time is during the daytime and the weather is assumed to be sunny

• The operating environment is assumed to be clean and clear, which is not close to any tall buildings or trees.

Research method

In this project, we leverage scientific research to explore the distinct characteristics and trends of our subject, enabling us to draw meaningful conclusions The study primarily employs three key scientific research methods to achieve its objectives.

• Theoretical research methods: Theoretical research method primarily involves collecting information through documents or texts Using this method, we refer to and search for information in relevant texts or documents

• Practical research methods: we directly interact with the research subjects As a result, the research subjects exhibit and manifest their essence and laws of motion

• Mathematical research methods: we use mathematical models to support statistical analysis of research findings, data gathered during the research process

THEORETICAL BASIS

GPS (Global Positioning System)

GPS, or Global Positioning System, is a sophisticated navigation technology that delivers accurate positioning and timing information It operates by utilizing signals from satellites to calculate users' precise locations, speeds, and altitudes By measuring the time delay of these satellite signals, GPS receivers can effectively determine the exact coordinates of a user.

GPS technology is utilized across various sectors, including navigation, mapping, surveying, and outdoor activities It enables users to pinpoint their exact locations, create efficient routes, and navigate seamlessly between destinations Additionally, GPS provides valuable data such as speed, direction, and distance traveled.

GPS has revolutionized navigation and exploration worldwide with its reliable global coverage and continuous operation Its positioning capabilities remain effective in various weather conditions, significantly enhancing safety, efficiency, and convenience in both transportation and outdoor activities.

GPS satellites orbit the Earth twice daily, each transmitting unique signals and orbital data that enable GPS devices to accurately determine their location By employing trilateration, GPS receivers measure the distance to satellites based on signal transmission times, allowing them to pinpoint a user's exact position With data from multiple satellites, these receivers can electronically display locations, facilitating activities such as tracking running routes, mapping golf courses, or navigating home.

Utilizing multiple satellites significantly improves GPS accuracy by allowing for precise location determination through a process known as trilateration When a GPS receiver relies on a single satellite, it can only calculate distance relative to that satellite, resulting in an ambiguous position within a circular area of coverage However, by incorporating additional satellites, the receiver can define its location more accurately through the overlapping circles of distance from each satellite, pinpointing its exact position at the intersection of these circles.

Figure 2.1 Trilateration Source: from “GPS”, by National Geographic,

(https://education.nationalgeographic.org/resource/gps/)

GPS satellite signals are weak when they reach the Earth's surface and travel via line of sight, allowing them to pass through clouds, glass, and plastic, but not most solid objects like buildings and mountains Nevertheless, modern receivers are more sensitive and can often track signals even through houses.

A GPS signal contains three different types of information: [15]

• Pseudorandom code is an I.D code that identifies which satellite is transmitting information You can see which satellites you are getting signals from on your device's satellite page

• Ephemeris data is needed to determine a satellite's position and gives important information about the health of a satellite, current date and time

Almanac data provides GPS receivers with essential information about the expected positions of GPS satellites over several months, along with their orbital details.

Bearing angle

The bearing angle, also known as azimuth angle or simply bearing, refers to the horizontal angle between a reference direction and the direction of a specific point or object

It helps us describe the orientation or direction of an object in relation to a reference point There are two main types of bearing angles:

Absolute bearing refers to the clockwise angle measured from a reference direction, typically North, to the direction of an object or point of interest This measurement provides a precise angle relative to the reference direction, such that if an object is located directly east, its absolute bearing is 90 degrees.

Relative bearing refers to the angle between the forward direction of a vehicle, vessel, or observer and the direction of an object of interest from the observer's position It quantifies the angle between the observer's line of sight and the target For instance, a relative bearing of 0 degrees indicates that the object is directly in front of the observer, whereas a relative bearing of 180 degrees signifies that the object is located behind the observer.

2.2.1 Formula to find bearing two given points

In geospatial calculations, let 'R' represent the Earth's radius, 'L' signify longitude, 'θ' denote latitude, and 'β' indicate bearing Consider two distinct points, A and B, where 'La' and 'θa' correspond to the longitude and latitude of point A, respectively, and similarly for point B Bearings are measured from the North direction, with 0° indicating North, 90° representing East, 180° denoting South, and 270° indicating West.

Bearing from point A to B, can be calculated as, [17]

𝛽 = 𝑎𝑡𝑎𝑛2(𝑋, 𝑌) (1) where, X and Y are two quantities and can be calculated as:

2.2.2 An example of bearing angle calculation

Let us take an example to calculate bearing between the two points with the formula:

So, X and Y can be calculated as,

This means, from Kansas City if we move in 96.51° bearing direction, we will reach St Louis.

PID controller

A PID (Proportional-Integral-Derivative) controller is a vital feedback control algorithm widely used in engineering and automation systems It effectively adjusts a control variable by calculating the error between the desired setpoint and the measured process variable By utilizing its proportional, integral, and derivative components, the PID controller continuously minimizes this error, ensuring that the process variable remains close to the setpoint Common applications include temperature control, motor speed regulation, robotics, and process control By dynamically adjusting the control variable, the PID controller improves system stability and ensures precise regulation Optimizing the controller gains through tuning enhances its performance for specific applications.

The principle of feedback involves adjusting the manipulated variable based on the process variable's relation to the setpoint; it is increased when the process variable is below the setpoint and decreased when it exceeds it This mechanism is known as negative feedback, as the manipulated variable reacts in the opposite direction to the changes in the process variable.

Figure 2.2 Block diagram of a process with a feedback controller Source: from “PID Controllers, Theory, Design and Tuning” (2 nd ed., p.65), by K

The block diagram illustrates the relationship between the process and the controller, represented as boxes connected by arrows indicating inputs and outputs Additionally, a special symbol is used to signify the summation of signals, highlighting the interaction between these components.

13 connected in a closed feedback loop The presence of the sign-reversing signal indicates that the feedback elements are negative

The reason why feedback systems are of interest is that feedback makes the process variable close to the setpoint despite disturbances and variation of the process characteristics

Mathematical form of PID algorithm can be described as: [4]

𝑑𝑡 (4) Where u is the control variable and e is the control error (e = 𝑦 𝑠𝑒𝑡𝑝𝑜𝑖𝑛𝑡 - y)

The control variable consists of three components: the P-term, which relates to the error; the I-term, which corresponds to the integral of the error; and the D-term, which is associated with the derivative of the error The parameters of the controller are defined by the proportional gain.

K, integral time 𝑇 𝑖 , and derivative time 𝑇 𝑑

In the case of pure proportional control, the control law of Equation (4) reduces to:

The control action is defined by the equation \( u(t) = K_p e(t) + u_b \), where the control output is directly proportional to the control error \( e(t) \) The term \( u_b \) represents a bias or reset value, which ensures that when the control error is zero, the control variable equals \( u_b \) Typically, the bias \( u_b \) is set to the average of the maximum and minimum control values, \( (u_{max} + u_{min})/2 \), but it can be manually adjusted to achieve a stationary control error of zero at a specific setpoint.

𝐾 𝑝 to the process transfer function 𝐺(𝑠) = (𝑠 + 1) −3 with setpoint 1 [4]

Figure 2.3 The effect of 𝐾 𝑝 Source: from “PID Controllers, Theory, Design and Tuning” (2 nd ed., p.66), by K

The proportional term in a PID controller delivers an immediate response to the discrepancy between the desired setpoint and the actual process variable As the most straightforward element of the PID controller, it is directly proportional to the error, playing a crucial role in system performance Key characteristics of the proportional term include its ability to enhance stability and reduce the time taken to reach the setpoint.

The proportional term in control systems is directly linked to the error, generating a control signal that corresponds to the size of the discrepancy between the setpoint and the process variable Consequently, a greater error results in a more substantial control signal, enhancing system response.

• Instantaneous Response: The proportional term provides an immediate response to the error It has no memory or integration of past errors and only considers the current error value

As a result, the proportional term can respond quickly to changes in the error and contribute to fast initial correction

• Steady-State Error: The proportional term alone cannot eliminate steady-state error

A non-zero steady-state error leads to a continuous control signal from the proportional term, potentially causing a residual error To effectively reduce steady-state error, it is essential to combine the proportional term with the integral term.

The proportional gain (𝐾𝑝) is crucial for determining the amplification of the proportional term in a control system A higher proportional gain results in a stronger control signal in response to errors, leading to a more aggressive system response However, if the proportional gain is set too high, it can cause instability and oscillations, negatively impacting the control system's performance.

The integral component of a PID controller computes the cumulative error between the desired setpoint and the actual process variable over time It consistently aggregates errors by factoring in both their duration and magnitude By accumulating past errors, the integral term generates a control signal that reflects the total error, enhancing system stability and performance.

The formula of the integral term is given by:

The integral term in control systems is crucial for mitigating steady-state error, which arises when there is a consistent discrepancy between the desired setpoint and the actual process variable, particularly when the proportional term alone fails to resolve it By accumulating this error over time, the integral term effectively reduces the difference, guiding the process variable closer to the setpoint Figure 2.4 illustrates the impact of the integral time constant (Ti) on the process transfer function G(s) = (s + 1)⁻³ with a setpoint of 1 and a constant proportional gain (Kp) of 1.

Figure 2.4 The effect of 𝑇 𝑖 Source: from “PID Controllers, Theory, Design and Tuning” (2 nd ed., p.68), by K

Some properties of the Integral term are:

The integral term effectively eliminates steady-state error by continuously accumulating and integrating the error over time This process enables it to address systematic biases and long-term discrepancies between the setpoint and the process variable, ultimately driving the error towards zero.

The integral term in control systems keeps a continuous record of past errors, allowing it to remember the cumulative error history By integrating the error, this term generates a control signal that reflects the total accumulated error over time, enhancing system performance and stability.

The integral term in control systems responds more slowly to small errors than the proportional term, as it requires time to accumulate significant values This delayed response means that the integral term becomes increasingly influential on the control signal when errors persist over time.

High settings of the integral term or system saturation can cause oscillatory behavior and integral windup, where the accumulated error grows excessively This situation makes it difficult for the controller to regain stability To address integral windup, various anti-windup techniques are commonly employed, helping to reduce its adverse effects on system performance.

Theoretical foundations of images and image processing

The origin of a natural image is characterized by continuous variations in space and brightness For digital processing, this continuous image must be digitized, which involves transforming it into a set of discrete points that accurately represent the original image's spatial and brightness characteristics The distance between these pixels is carefully determined to ensure that the human eye cannot perceive the boundaries between them Each point, known as a pixel (or Picture Element), is defined by its coordinates (x, y) and possesses a specific grayscale or color value The size and spacing of the pixels are optimized to maintain the perception of continuity in both spatial and color dimensions, making the digital image closely resemble the real image In this context, each element within the pixel matrix is referred to as an image element.

Figure 2.6 Pixels of Image Source: Stanford Edu

Image resolution refers to the pixel density of a digital image, which is crucial for ensuring that the human eye perceives a continuous and clear image To achieve this, the spacing between pixels must be carefully selected, creating a density distribution that defines the resolution in two dimensions along the x and y axes.

Figure 2.7 Common image resolution Source: from “Display Resolution Explained: What is FHD, QHD, UHD, 4K, 5K, 8K?” by Vivek, 2022 (https://www.droidviews.com/screen-resolution-sizes-hd-fhd-qhd-uhd-4k-

The resolution of a CGA (Color Graphic Adapter) screen is defined by a grid of pixels, with dimensions of 320 pixels vertically and 200 pixels horizontally (320x200) Notably, a 12" CGA screen offers a smoother visual experience compared to a larger 17" CGA screen, despite both having the same resolution.

200 resolution Reason: same density (resolution) but wider screen area, less smoothness (continuity of points) The commonly used resolutions are shown in Figure 2.7

2.4.3 Gray level of the image

A pixel (pixel) has two basic characteristics: the position (x, y) of the pixel and its grayness Below we review some commonly used concepts and terms in image processing

- Definition: The gray level of a pixel is its luminous intensity assigned a numerical value at that point

- Common gray level value scales: 16, 32, 64, 128, 256 (Level 256 is common Since computer engineering uses 1 byte (8 bits) to represent gray level: gray level uses 1 byte represents: 28%6 levels, i.e 0 to 255)

- Black and white image: an image with two colors black and white (no other colors) with gray levels at the pixels that may be different

A binary image consists of only two distinct levels, black and white, utilizing just 1 bit to represent each pixel This means that each pixel in a binary image can only be either 0 or 1, resulting in a simplified representation of images.

In the realm of the three-color theory, which encompasses Red, Blue, and Green, the creation of a vibrant color world is achieved through the use of 3 bytes to represent color levels This method allows for a total of 28 different color values, showcasing the depth and variety of colors that can be generated.

- RGB color space: (RGB for Red Green Blue)

The Commission Internationale d'Eclairage (CIE) establishes various standards for color representation, including the CIE-RGB color standard, which utilizes the primary colors red, green, and blue (R, G, B) along with the unique RGBCIE notation to differentiate it from other color systems.

As stated above, a color is a combination of primary colors in some proportion Thus, Px is each pixel of the color image, T is the displacement symbol

People use the three-color coordinate system R-G-B (corresponding to the x-y-z coordinate system) to represent colors as follows:

Figure 2.8 RGB Coordinate System Source: from “CHOBS: Color Histogram of Block Statistics for Automatic Bleeding

Detection in Wireless Capsule Endoscopy Video”, by Tonmoy Ghosh, 2018

In this representation we have the formula: red + green + blue = 1

Maxwell's formula is illustrated by the triangle formed by three dashed lines, known as Maxwell's triangle The white area within this coordinate system is determined through specific calculations.

- HSL color space: (HSL for hue, saturation, lightness)

The HSL color space, illustrated in Figure 2.9, is extensively utilized in image editing, image analysis, and computer vision This color space is defined by three key parameters that effectively describe colors.

Figure 2.9 HSL color space Source: from “Color Spaces Advantages and Disadvantages in Image Color Clustering

Segmentation”, by Erik Cuevas, 2018 https://www.researchgate.net/publication/320410000_Color_Spaces_Advantages_and_

Disadvantages_in_Image_Color_Clustering_Segmentation) The HSL color space describes colors based on three main components:

Hue (H) refers to the dominant wavelength of light perceived as color, measured in degrees on a color wheel from 0° to 360°, where 0° and 360° signify red, 120° indicates green, and 240° represents blue, with other colors in between Saturation (S) reflects the purity or intensity of a color, indicating the amount of gray mixed with the hue, and is expressed as a percentage from 0% (completely gray, desaturated) to 100% (fully saturated, vibrant color).

Lightness, or luminance, measures the brightness of a color on a scale from 0% (black) to 100% (white) A value of 50% indicates the original color without any modifications, while values below 50% result in darker shades and values above 50% produce lighter shades.

The HSL color space offers a more intuitive approach to color manipulation than traditional models like RGB and CMYK, allowing for independent adjustments in hue, saturation, and lightness This makes HSL particularly valuable for color selection, image editing, and achieving color harmonization Its ability to convert seamlessly between HSL and other color spaces facilitates integration with diverse software and devices Due to its simplicity and perceptual relevance, HSL is extensively utilized in graphic design, web development, and visual arts.

2.4.5 Black-white photograph and digital image

A digital image is a collection of pixels with a suitable gray level used to describe an image that is close to the real image

Black and white photos are composed solely of two colors: black and white These images can be divided into L levels of gray When using 8 bits (B = 8 bits) for encoding, the number of gray levels (L) is determined accordingly.

If L equals 2, B = 1, that means there are only 2 levels: level 0 and level 1, also known as binary image Level 1 corresponds to light colors, and level 0 corresponds to dark colors

In imaging, when L is greater than 2, it indicates a grayscale image, where each pixel in a binary image is encoded using 1 bit, while a 256-level image encodes each pixel with 8 bits For black and white images, utilizing 8 bits (1 byte) allows for the representation of 256 gray levels, with each level represented as an integer from 0 to 255 Here, 0 corresponds to the darkest intensity, and 255 signifies the brightest intensity.

A binary image consists of elements that can be interpreted as logical values, primarily utilized to differentiate objects within an image from the background or to identify edge points from other points.

Figure 2.10 Gray image (right) and binary image (left)

Source: from “Python dlib: face landmarks detection”, by techtutorialsx, 2023

(https://techtutorialsx.com/2021/04/07/python-dlib-face-landmarks-detection/#respond)

According to Thomas's theory, a color image is a composite of three primary colors: red (R), green (G), and blue (B), typically captured across different frequency bands Unlike black and white images, color images represent each pixel using three separate values for the primary colors, requiring a total of 24 bits, divided into three 8-bit segments Furthermore, colors are categorized into L different grades, usually expressed as L%6, with each segment indicating the luminous intensity of one primary color.

Hough Transform

Hough transform is a feature extraction algorithm that is used to detect lines in images

The lane detection algorithm is highly effective, yet it faces certain limitations, including slower processing speeds when dealing with images rich in edges and heightened sensitivity to noise within the images.

The Hough transform remains a widely-used algorithm for lane detection in self-driving cars, effectively identifying lane lines under challenging conditions like poor weather and low light.

As we learned from quite early school classes, the straight line can be represented by two parameters which correspond to slope and intercept:

In polar coordinates, a line can be clearly defined using the pair (ρ, θ), where ρ represents the shortest distance from the origin to the line, measured perpendicularly, and θ indicates the angle between the x-axis and the distance line This representation offers a distinct advantage, as it allows for the description of vertical lines using ρ and θ, a capability not available when using the (a, b) parameters in the Cartesian system.

Figure 2.12 Vertical lines described by ρ and θ

We can determine specific ρ and θ for each specific line and then, the following equation is satisfied for each xi, yi point belonging to this line:

𝜌 = 𝑥 cos 𝜃 + 𝑦 sin 𝜃 (10) The next step is mapping from Image space to Hough space using this line described in polar system

- A line in image space represented by some ρ and θ can be drawn such point in (ρ, θ) coordinates which will be later called a Hough space (Figure 2.13)

Figure 2.13 Transformation of a line to a point in Hough space

When multiple lines intersect at a single point in image space, each line can be represented by parameters ρ and θ These lines are then mapped into Hough space, as illustrated in Figure 2.14.

Figure 2.14 Transformation of multiple lines intersect at 1 point

In Hough space, drawing an infinite number of lines that intersect at a specific point generates a continuous sinusoid This illustrates that a point in image space corresponds to a sinusoidal curve in Hough space (see Figure 2.15).

Figure 2.15 Transformation of a point to a sinusoid in Hough space

When points that create a line in the image space are plotted, they result in two intersecting sinusoids in Hough space, converging at a single point (Figure 2.16).

Figure 2.16 A line represented as an interest point in Hough space

By analyzing intersections in Hough space, we can identify lines in an image The greater the number of sinusoids that intersect at a point, the higher the likelihood that this point accurately represents a line in the image space.

Artificial Intelligence

Artificial Intelligence (AI) is a prevalent topic in mainstream media, sparking discussions about its role in the 4.0 industry and its potential to enhance our lives Numerous films depict scenarios where AI takes control, showcasing robots that utilize this technology to mimic human behavior and communication But what does "Artificial Intelligence" truly mean?

Artificial intelligence refers to a computer's or an automated robot's ability to do activities that would typically need human intelligence and judgment Artificial intelligence

25 is computer software that may be installed into machines to assist in activities that would ordinarily need human intelligence

AI applications are prevalent across various fields, including speech recognition, decision-making, and visual perception For instance, Google Translate can instantly translate words from images without manual input, while Facebook utilizes facial recognition to connect users with family members Additionally, Spotify analyzes your music preferences to recommend the latest songs that align with your favorite genres.

Machine Learning, a subset of Artificial Intelligence, enables systems to learn from provided data and make accurate predictions By analyzing specific datasets, ML models can forecast optimal outcomes based on defined objectives Moreover, these models can generate predictions autonomously, without the need for explicit programming.

In today's world, machine learning techniques are increasingly integrated into our daily lives, enhancing various applications such as e-commerce purchase recommendation systems, virtual assistants, facial recognition technology, and cancer diagnosis tools These advancements significantly improve efficiency and provide valuable assistance to individuals and businesses alike.

In this project of ours, machine learning was used to be able to recognize people and lanes line

Machine learning can be primarily classified into two main categories: supervised learning and unsupervised learning Additionally, it encompasses other categories such as semi-supervised learning, deep learning, and reinforcement learning.

Machine learning workflow will show you how to work with machine learning Take a look at the diagram below:

Specifically, each step in the machine learning workflow is as follows:

• Data collection: for the computer to learn you need a data set, you can collect them yourself or get previously published data sets Note that you must collect from an

26 official source, so the new data is accurate, and the machine can learn properly and achieve higher efficiency

Preprocessing is a crucial phase in data analysis that involves normalizing data, eliminating irrelevant attributes, assigning labels, encoding features, extracting relevant information, and reducing the dataset while maintaining accuracy This stage is often the most time-consuming, accounting for over 70% of the total execution time, particularly in relation to the volume of data being processed.

To ensure a smooth training process, it is essential to select an appropriate algorithm for model training and evaluation This involves systematically testing various algorithms to identify the most suitable one for our specific needs.

• Fine-tune your models: we simply need to make our code looks clean and remove those uncertainty part

To ensure the effectiveness of your solution, it's essential to test and analyze it using a test dataset This evaluation process helps determine the quality and performance of the implemented model within the project.

• Launch, monitor, and maintain your system: we need to make sure that the project is runnable and maintain the sustainability of the program

Deep Learning, a specialized branch of Machine Learning, enables computers to autonomously learn and enhance their performance through advanced algorithms This field relies on intricate artificial neural networks designed to replicate the cognitive processes of the human brain.

A neural network is structured with multiple layers, and a greater number of layers indicates a "deeper" network Each layer contains nodes that connect to adjacent layers, with each connection assigned a specific weight The weight of a connection determines its influence on the overall functioning of the neural network, with higher weights indicating a stronger impact.

Each neuron will have an activation function, which is basically responsible for

The output from the neuron is "normalized" as user data passes through all layers of the neural network, ultimately producing results in the final layer known as the output layer.

Neural Network is an artificial neural network This is a series of algorithms to search for relationships in a set of system data based on how the human brain works

Artificial Neural Networks (ANNs) are systems that emulate the connections between artificial neurons, enabling them to adapt to changes in input for optimal output Originating from artificial intelligence, ANNs are increasingly utilized in modern electronic transaction systems.

Neural Network has the following specific and detailed characteristics:

• Artificial neural networks act like neurons in the human brain In it, each neuron is a mathematical function, which collects and classifies data and information according to a detailed structure

• Neural Network is similar to statistical methods according to curve graphs or regression analysis For the simplest explanation, imagine a Neural Network consisting of interconnected network nodes

Each node functions as a collection of perceptions, resembling a multi-linear regression model, and is interconnected with other nodes These layers gather information, classify it, and generate the appropriate output signal.

• Each artificial neural network is a multilayer Perceptron, a Neural Network usually includes 3 specific layers as follows:

• Input Layer (input layer): Located on the left side of the system, including input information data

• Output Layer: Located on the right side of the system, including output information data

• Hidden Layer: Located between the input and output layers, representing the inference and information processing process of the system

Figure 2.17 Neural Network Architecture Source: from “Neural Network là gì? Đặc điểm và ứng dụng của Neural Network”, by

Nguyễn Hưng, 2022 (https://vietnix.vn/neural-network/)

Each network node in the Neural Network is a different activation function but currently applying a uniform algorithm to make it easier to operate

In neural network architecture, the quantity of sigmoid neurons in each layer is determined by the data processing requirements Experts typically incorporate hidden layers with varying numbers of neurons to optimize performance during operations.

Neurons in various layers interconnect to create a dense and comprehensive network This structure allows for an understanding of the network's scale, determined by the number of layers and neurons involved.

MECHANICAL SYSTEM AND HARDWARE CONFIGURATION

Chapter introduction

Mechanical design is a crucial element in the development of machines and robots, particularly in creating self-driving vehicles This chapter outlines the essential steps involved in designing mechanical components, including design calculations, component modeling, and motor selection.

Technical requirements

• Working environment: On the road in university campus, assume that the weather is good, and enough light conditions

• Load capacity: Up to 30 Kg (including weight of the vehicle)

Vehicle mechanism structure

• Option 1: 3 wheels mechanism structure - 1 omnidirectional wheel in the front, 2 driving wheels at the back (Figure 3.1)

- The 3 wheels mechanism always contacts the ground in one plane, meaning that 3 wheels always touch the ground

- Easy to make a turn and control

- The vehicle might get flipped easily when braking

- When carrying, the vehicle may not be balanced leading to wrong movement

• Option 2: 4 wheels mechanism structure - 2 omnidirectional wheels at the front, 2 driving wheels at the back (Figure 3.2)

- Not easy to get flipped because of the 4 wheels support at 4 ends of the vehicle, sharing the weight of it

- No guarantee for 4 wheels to be always on the ground

- May get hard to make a turn with 2 free wheels at the front

• Option 3: 4 wheels mechanism structure - 1 omnidirectional wheel in the front, 2 driving wheels in the middle, 1 omnidirectional wheel at the back (Figure 3.3)

- Not easy to get flipped because of the 4 wheels support at 4 ends of the vehicle, sharing the weight of it

- Easy to control and make a turn with only 1 freewheel in the front

- Maybe a little swinging when running on the road because either of the free wheels may not touch the ground

For this project, we selected a three-wheel mechanism featuring two front driving wheels and one rear free wheel This design ensures consistent ground contact, addressing road imbalances commonly found on university campuses and manufacturing inaccuracies due to budget constraints Additionally, this configuration enhances vehicle control and maneuverability.

Figure 3.4 Chosen 3 wheels mechanism structure

Wheels and motors

As we are using batteries for autonomous vehicles to operate outdoors, DC motors are obviously the most efficient type to consider

Permanent Magnet DC (PMDC) Motor:

• Utilizes a permanent magnet to generate a magnetic field

• Compact size, high starting torque, and good speed control

• Often used in household appliances, power tools, robotics, and automotive applications

• Armature and field windings connected in series

• Provides high starting torque and can achieve remarkably high speeds

• Suitable for applications requiring high torque, such as electric vehicles, locomotives, and industrial machinery

• Armature and field windings connected in parallel

• Offers good speed regulation and steady operation

• Widely used in conveyor belts, printing presses, milling machines, and pumping systems

• Uses electronic commutation instead of brushes for improved efficiency

• Higher power density, smoother operation, and longer lifespan compared to brushed motors

• Commonly found in drones, electric vehicles, computer cooling fans, and industrial automation

• Brushed DC motors have mechanical brushes that contact the commutator to switch the current direction

• They are simple, cost-effective, and offer good torque

• Commonly found in home appliances, toys, and automotive applications

Considering the operating environment and the features of various motor types, our team recommends brushless motors for autonomous vehicles These motors offer a longer lifespan, require less maintenance, produce less noise, and have a higher power density compared to brushed motors.

The Brushless DC motor, or Permanent Magnet Synchronous Motor (PMSM), is a type of synchronous AC motor that aligns the rotor with the rotating magnetic field using permanent magnets This technology allows for a stable magnetic field in the rotor PMSMs can be categorized into two types based on the waveform of the back electromotive force (EMF) produced by the stator.

- Sinusoidal PMSM: This particular PMSM generates a sinusoidal back EMF waveform Its stator windings are designed to create a smooth and efficient sinusoidal magnetic field, ensuring seamless motor operation

The trapezoidal Permanent Magnet Synchronous Motor (PMSM) features a trapezoidal back EMF waveform, achieved through specially designed stator windings that create a trapezoidal magnetic field This design enables the motor to provide high torque output, particularly at elevated speeds, making it an efficient choice for various applications.

The Brushless DC motor is classified as a trapezoidal PMSM, which provides superior torque density and enhanced performance, particularly at high speeds

Figure 3.5 Trapezoidal back EMF waveform Source: from “Permanent Magnet Synchronous Machine”, by MathWorks

(https://ww2.mathworks.cn/help/sps/powersys/ref/permanentmagnetsynchronousmachin e.html)

A Brushless DC (BLDC) motor is an electric motor that functions similarly to a traditional DC motor but eliminates the need for brushes and a commutator Instead, it employs electronic commutation via a motor controller to regulate the current flow in the stator windings, enabling precise control of the rotor's rotation.

BLDC motors are available in various physical configurations, primarily categorized by their stator windings into single-phase, two-phase, or three-phase types Among these, three-phase BLDC motors featuring permanent magnet rotors are the most widely utilized.

Figure 3.6 Brushless DC motors components Source: from “BLDC Engines Enable the Future of Motors”, By Rüdiger Laschewski-

Grossbaier, 2019 (https://www.fierceelectronics.com/components/bldc-engines-enable- future-motors)

Figure 3.7 Brushless DC motor’s stator Source: from “An Introduction to Brushless DC Motor Control”, by Steven Keeping,

2013 (https://www.digikey.be/nl/articles/an-introduction-to-brushless-dc-motor-control)

The stator, a stationary component encasing the rotor, consists of a stator core, windings, and teeth that are meticulously designed and assembled The windings, made of conductive material, are arranged in a specific pattern around the teeth When current flows through these windings, it creates a magnetic field that interacts with the rotor's magnets, facilitating rotation.

Figure 3.8 Several types of rotors Source: from “Classification of Electric Motors - Part Two”, by Electrical Knowhow,

The rotor, a crucial component of electric motors, is the moving part that typically contains magnets made from materials with strong magnetic properties These magnets generate a magnetic field that interacts with other motor components, leading to the initiation of rotational movement.

The Hall sensor is vital for synchronizing stator armature excitation with rotor position in a BLDC motor Since the motor's commutation is electronically controlled, energizing the stator windings in a specific sequence is crucial for smooth rotation Before activating a stator winding, it is essential to know the rotor's position, which is achieved through the embedded Hall Effect sensor in the stator that detects and relays rotor position information.

Figure 3.9 A brushless DC motor with three Hall effect sensors Source: from “How to Select Hall-Effect Sensors for Brushless DC Motors”, 2016

(https://www.assemblymag.com/articles/94866-how-to-select-hall-effect-sensors-for- brushless-dc-motors)

Most BLDC motors feature three embedded Hall sensors in the stator, which produce low and high signals as the rotor poles approach By evaluating the outputs from these sensors, the exact commutation sequence for the stator winding can be accurately established.

When a rotor passes a sensor, it generates a signal that indicates its position as either a North (N) or South (S) pole By tracking the signal transitions of these Hall effect sensors, which switch between high and low, the rotor's position can be accurately determined at intervals of 60 degrees.

Figure 3.10 Six-step commutation of Hall sensors Source: from “A simple brushless sensored motor driver for AVR Atmega”, by Ibrar

Ayyub, 2017 (https://atmega32-avr.com/simple-brushless-sensored-motor-driver-avr- atmega/)

In the six-step commutation process, the three windings (U, V, W) are energized in positive, negative, or off states, determined by the high or low signals from each of the three Hall effect sensors.

A motor controller is an essential electronic device that regulates and manages motor functions by receiving feedback to determine the optimal currents for the stator windings By precisely controlling the timing and sequence of current switching, it enables the motor to achieve the desired speed and torque effectively.

3.4.3 Calculation for choose driving wheels

To enable a vehicle's movement, the motor must generate sufficient torque to turn the wheels, a process influenced by the vehicle's weight, load capacity, and the friction between the wheels and the road surface.

Based on the forces acting on the vehicle:

- F: force needed for the vehicle to move

- Ffriction: friction between wheels and the ground

Table 3.1 Technical data of autonomous vehicle m Load capacity 30 kg

R Wheel’s radius 6.5 inch à Coefficient of sliding friction 0.8 g Gravitational acceleration 9.81 m/s 2 a Acceleration 0.1 m/s 2 v Maximum speed 1 m/s

With the given data, we can then calculate the force needed:

𝐹 = 30 × 0.1 + 0.8 × 9.81 × 30 = 238.44(𝑁) Power needed to drive the vehicle:

𝑃 = 𝐹 𝑣 = 238.44 × 1 = 238.44 (𝑊) Then we can choose a brushless motor that meets the above requirement from hoverboard with the specification: 6.5″ Hub Motor with Encoder

Figure 3.13 Hub Motor Dimension Table 3.2 Technical data of Hub Motor

Maximum speed without load 350 RPM (12VDC)

Hall-effect Sensor Encoder voltage requirements

Current requirements 2.5–15.0+ amps depending on terrain and load

Tire composition Solid rubber, tubeless

Operating temperature range +32 to +120 °F (0 to +49 °C)

Payload capacity per wheel 240 lb (100 kg)

- Small structure and easy to install

- Good adaptation when making a turn

- Can be able to withstand heavy loads

With these requirements, our team had chosen the wheel with the following specifications:

Figure 3.14 Heavy duty omnidirectional wheel

Hardware configurations

Released on June 24, 2019, the Raspberry Pi 4 Model B is a compact and affordable computer that supports dual display connectivity It has gained popularity in automation and robotics due to its significant enhancements in processor speed, multimedia capabilities, memory, and connectivity, while remaining compatible with previous models and maintaining similar power consumption levels.

In the scope of this project, Raspberry pi 4 is used because of its processing power is suitable for computer vision task, algorithm computing task and its affordability

Figure 3.15 Raspberry Pi 4 ports Table 3.4 Raspberry pi 4 Specification

Connectivity • 2.4 GHz and 5.0 GHz IEEE 802.11b/g/n/ac wireless

• 2 × USB 2.0 ports GPIO Standard 40-pin GPIO header

Video & sound • 2 × micro-HDMI ports (up to 4Kp60 supported)

• 2-lane MIPI DSI display port

• 2-lane MIPI CSI camera port

• 4-pole stereo audio and composite video port

Input power • 5V DC via USB-C connector (minimum 3A1)

• 5V DC via GPIO header (minimum 3A1)

• Power over Ethernet (PoE)–enabled

The Arduino Mega 2560 is a microcontroller board renowned for its extensive capabilities and enhanced performance, providing an ideal platform for a wide range of

50 projects Developed by Arduino, the Mega 2560 offers expanded resources and compatibility beyond standard Arduino boards

The Arduino Mega 2560 stands out due to its ability to manage complex tasks and handle large volumes of data With enhanced memory and RAM, it supports the implementation of advanced algorithms and the storage of extensive datasets, making it an ideal choice for intricate projects.

The Mega 2560 offers complete compatibility with Arduino software and programming language, making it ideal for both beginners and experienced users Its seamless integration with existing code and a vast library of examples accelerates project development.

The Arduino Mega 2560 offers flexible power options, allowing it to be powered via USB or an external supply, and can be programmed through either method This adaptable board is capable of operating independently or integrating smoothly into larger systems and projects.

Based on the need of more pins in this project, Arduino mega 2560 is used because of its expanded part and its compatibilities

Digital I/O Pins 54 (of which 15 provide PWM output)

DC Current per I/O Pin 20 mA

DC Current for 3.3V Pin 50 mA

Flash Memory 256 KB of which 8 KB used by bootloader

The Raspberry Pi Camera Module v2 is a remarkable add-on for the Raspberry Pi, enabling users to capture stunning photos and videos Featuring an 8-megapixel Sony IMX219 image sensor and a fixed focus lens, it delivers high-resolution images at 3280 x 2464 pixels The module supports various video formats, including 1080p30, 720p60, and 640x480p60/90 Connecting the camera is simple, as it plugs into one of the designated sockets on the board through a specialized CSi interface designed for camera connections, ensuring a hassle-free setup.

Figure 3.18 Raspberry Pi Camera Module v2

The Raspberry Pi Camera Module v2 has been chosen for this project due to its compatibility with the Raspberry Pi 4 and superior specifications compared to its predecessor Its advanced features make it particularly suitable for human detection, which is a critical requirement for the project's success.

Table 3.6 Specifications of the two Camera Module

Size Around 25 × 24 × 9 mm Around 25 × 24 × 9 mm

Sensor OmniVision OV5647 Sony IMX219

Sensor image area 3.76 × 2.74 mm 3.68 x 2.76 mm (4.6 mm diagonal)

Pixel size 1.4 àm ì 1.4 àm 1.12 àm x 1.12 àm

Depth of field Approx 1 m to ∞ Approx 10 cm to ∞

NoIR version available? Yes Yes

3.5.4 Brushless DC Motor Controller Driver Board

This RioRand motor speed controller board is designed to work with a three-phase brushless DC motor with hall sensors The part number printed on the board is ZS-Z11H

Figure 3.19 Brushless DC Motor Controller Driver Board Connection Source: from “Easy Inexpensive Hoverboard Motor Driver”, by MAD-EE, 2021

(https://mad-ee.com/easy-inexpensive-hoverboard-motor-controller/) The ZS-X11H board with hall sensors works perfectly for driving a hoverboard motor

The board features comprehensive connections for powering the motor, as well as for controlling speed, braking, and direction Additionally, it allows for seamless integration with Arduino, Raspberry Pi, or other microcontroller devices for effective control.

Table 3.7 Specifications of Brushless DC Motor Controller

Product size length 63mm* width 45mm* height 28mm

Photoelectric sensors are essential devices used to identify objects and detect changes in surface conditions by analyzing their optical properties They consist of two primary components: an emitter that emits light and a receiver that captures it When an object obstructs or reflects the emitted light, the intensity of the light reaching the receiver changes, allowing the sensor to convert this variation into an electrical signal.

This Photoelectric switch uses Diffuse-reflective Sensors, which has the Emitter and Receiver installed in the same housing and light normally does not return to the Receiver

Figure 3.21 Diffuse-reflective Sensors operation Source: from “Photoelectric Sensors”, by Omron

(https://www.ia.omron.com/support/guide/43/introduction.html)

When light emitted from the Emitter hits the sensing object, it reflects back and reaches the Receiver, where its intensity is amplified This amplified light intensity is crucial for detecting the presence of the object.

Table 3.8 Photoelectric switch E18-D80NK specifications

Sensing range 3cm to 80cm (depends on obstacle surface)

Type DC 3 Wire NPN-NO (Normal Open)

Control signal level High 2.3V ≤ Vin ≤ 5V

The MPU-9250 is an advanced 9-axis motion tracking sensor that integrates a 3-axis gyroscope, 3-axis accelerometer, and 3-axis magnetometer, along with a Digital Motion Processor™ (DMP) in a compact 3x3x1mm design As a pin-compatible upgrade from the MPU6515, it features a dedicated I2C sensor bus, enabling it to deliver comprehensive 9-axis Motion Fusion™ output directly.

Figure 3.22 Module IMU MPU-9250 components Source: from “MPU9250 9-DOF MEMS Sensor Module”, by Components101, 2021

The MPU-9250 Motion Tracking device features advanced 9-axis integration, on-chip Motion Fusion™, and runtime calibration firmware, allowing manufacturers to significantly reduce costs associated with motion sensing applications This versatile sensor module is ideal for enhancing the performance of various electronic designs, making it a valuable component for engineers in the field.

56 and complex selection, qualification, and system-level integration of discrete devices, guaranteeing optimal motion performance for consumers

Operating current 3.5mA when all nine motion sensing axes and the DMP are enabled

Gyroscope full-scale range ±250, ±500, ±1000, and ±2000°/sec

Accelerometer full-scale range ±2g, ±4g, ±8g, and ±16g

Pin Number Pin Name Description

The GPS receiver module utilizes advanced Ublox chip technology, providing low power consumption and high sensitivity for optimal performance With the capability to receive signals from 56 satellite channels, it surpasses traditional GPS receivers by delivering enhanced accuracy and faster positioning due to its superior signal strength.

This module features a built-in receiving antenna and a high-precision positioning chip, manufactured to industrial-grade standards It is designed to fulfill the positioning needs for both industrial applications and personal use.

The GPS receiver features an internal battery that retains crucial satellite data, including signal status, final position, and time This stored information greatly improves the positioning speed when the module is powered on again.

Figure 3.23 TEL0138 USB GPS Receiver

Furthermore, the receiver is designed with a USB interface, making it convenient to connect to this project’s main controllers Raspberry Pi 4

Table 3.11 TEL0138 USB GPS Receiver specifications

Baud Rate 4800, 9600, 19200, 38400, 57600, 115200bps Receiving Channel 56 channels

Accuracy horizontal position accuracy < 2.5m Outline Dimension 38×49×16mm

Timing accuracy 30ns Operating Temperature -40℃~85℃

For effective lane detection in autonomous vehicles, a high-definition camera with a wide range and auto-focus capabilities is essential, as the commonly used Raspberry camera is primarily suited for human detection Therefore, the Rapoo C200 webcam has been selected for this purpose.

Some main features of the webcam are listed below:

• It adopts a high-definition photosensitive chip, with 5-layer coating lens, supports 1280*720 resolution, and the maximum frame rate is 30 frames

Schematic design

3.6.1 General connection diagram of the autonomous vehicle

A BLDC motor driver is essential for regulating the speed of a BLDC motor, and the wiring connections between the driver and the motor are illustrated in the accompanying figure.

Figure 3.26 Schematic of motor driver setup

THE VEHICLE CONTROL ALGORITHM

Flow of controlling autonomous vehicle

The process of controlling the robot is described in Figure 4.1

Figure 4.1 Robot control Flow Chart

We control the robot using fixed geographic points, with their latitude and longitude stored in a database and displayed on a web-based map This system allows us to monitor autonomous vehicles effectively Once the destination is set, the autonomous vehicle begins to orient itself and move towards it If an obstacle is detected, the vehicle will stop The movements of these autonomous vehicles are primarily categorized into two phases: orientation and straight movement.

Leaflet is the premier open-source JavaScript library for creating mobile-friendly interactive maps, weighing only 42 KB while providing essential mapping features that meet the needs of most developers.

Leaflet is crafted for simplicity, performance, and usability, ensuring efficient functionality on all major desktop and mobile platforms It offers extensive plugin support, features a user-friendly and well-documented API, and boasts a clean, readable source code that encourages contributions.

Figure 4.2 Map built by Leaflet Source: from “Leaflet an open-source JavaScript library for mobile-friendly interactive maps)”, by open source, 2023 (https://leafletjs.com/)

After acquiring the map, we develop functions to save the coordinates along the designated routes The autonomous vehicle will navigate from the starting point to the endpoint for each of the two landmarks as illustrated on the map.

To reach our destination, simply select the starting point and endpoint on the map and save the coordinates These parameters are then sent to the database, enabling the autonomous vehicle to accurately determine its position and navigate independently.

Figure 4.3 Map showing the path for autonomous vehicles to move

To effectively regulate the speed of a DC motor, a power source capable of adjusting voltage is essential When a 12V motor is powered on, it does not instantly achieve its maximum speed due to a brief delay in response time If the power is turned off before the motor reaches full speed, it begins to decelerate Rapidly toggling the power on and off can influence the motor's acceleration and deceleration, allowing for better speed control.

A PWM controller enables precise motor speed control, allowing operation between zero and maximum speed It activates the motor using a series of pulses, adjusting the pulse width to regulate speed, which is why it is known as Pulse Width Modulation.

The average output voltage when controlling by PWM can be calculated as followed:

𝑇 (V) or 𝑈 = 𝑈 𝑚𝑎𝑥 × 𝐷 With Umax: maximum voltage output

Ton: the time pulse is On

The general mathematical form of PID algorithm can be described as:

In sampling time, the integral term is discretized as:

The derivative term is discretized in sampling time as:

∆𝑡 With these terms discretized, we can now build a PID controller program:

Error = Setpoint – Output Integral = Integral + (Error * Δt) Derivative = (Error – Previous Error) / Δt Output = (Kp * Error) + (Ki * Integral) + (Kd * Derivative) Where:

- Setpoint: the desired or target value that the system being controlled should achieve

- Output: the control signal or action that is applied to the system being controlled

The inputs for the robot to orient are the current position and the destination location From there we can calculate the bearing angle between those two positions

Figure 4.5 shows how autonomous vehicles are oriented

Figure 4.5 The process of Orientation

To determine the heading of an autonomous vehicle, we utilize the offset angle between the compass and the bearing as the error for the PID controller, which subsequently adjusts the speed of the left and right wheels For this purpose, we employ the MPU 9250 sensor, as discussed in the previous chapter The process of obtaining the heading from the sensor is illustrated in Figure 4.6.

Figure 4.6 The process of Heading angle

The move block of the robot is described as Figure shown below

The robot will navigate towards its target position by continuously moving in a straight line while tracking the lane line If the autonomous vehicle has not yet reached the desired location, it will rely on a PID controller to maintain its course based on the deviation from the left lane line Details of the lane line tracking process will be discussed in section 4.2.2.

THE EXPERIMENT RESULT AND ANALYSIS

Practical model

Figure 5.1 shows our developed autonomous vehicle This vehicle is equipped with advanced technologies:

• Control System: The control system executes the planned actions by adjusting the vehicle's acceleration, braking, and steering

Autonomous vehicles utilize an array of sensors to effectively perceive their environment, including photoelectric sensors and cameras These technologies are essential for detecting obstacles and identifying lane lines, ensuring safe navigation on the roads.

• Localization techniques, such as GPS, inertial measurement units (IMUs), and visual odometry, help determine the vehicle's position and orientation within the map.

Wheels speed control

The SC speed pin is linked to the Arduino to measure the speed pulse output signal from the wheels Each time a hall sensor changes state, the output signal also changes, resulting in 90 state changes for a single wheel rotation.

To measure wheel speed, first determine the time between two transitions (w) and multiply it by 90 to find the duration of a full revolution Next, calculate revolutions per second (Rps) by taking the inverse of the rotation time, and then multiply by 60 to convert this value into revolutions per minute (Rpm).

𝑤 × 90 Then convert into revolutions per minute:

We can also have the speed calculated in kilometers per hour:

𝐾 𝑝ℎ = (𝑑 × 𝜋 × 𝑅 𝑝𝑚 × 60)/1000(𝐾𝑚/ℎ) The wheels have diameter of 6.5 inch, which is 0.165 when converted to meter Then the speed in kilometers per hour can be calculated as:

When a model-based approach is unfeasible for an object, the empirical method becomes the preferred design strategy Conducting experiments is contingent upon the system achieving a stable working state, ensuring that all signal values within the control system remain within acceptable limits.

To achieve optimal tuning results, we integrate manual tuning with the Ziegler-Nichols method Begin by setting both Ki and Kd to zero, then gradually increase Kp until the system begins to oscillate, identifying this Kp as the ultimate gain Next, set the integral time to match the oscillation period and readjust Kp as necessary If oscillation persists, fine-tune the Kd value Following Ziegler-Nichols guidelines further enhances the tuning process.

Table 5.1 Ziegler Nichols Tuning Chart [1]

PID type Kp Ki Kd

PI Ku/2.2 Pu/1.2 - PID Ku/1.7 Pu/2 Pu/8 With Ku: Ultimate gain

After fine tuning the PID controller of the wheels, we get the parameters as: Kp = 0.225;

Ki = 2.5; Kd = 0 The response of the PID controller can be seen below

Tracking heading angle

In this section, we are going to use the gyroscope in the IMU We will use it to approximate where the robot is facing in three dimensions

The preceding graph contains three lines The Y-axis shows a movement rate in degrees per second, while the X-axis shows the time in seconds since the program started

The displayed graphs feature red, yellow, and blue lines representing gyroscope data: red for the y-axis, blue for the x-axis, and yellow for the z-axis This gyroscope data indicates the rate of turn rather than direction, highlighting the need for a method to approximate the robot's heading for enhanced navigation.

To effectively utilize the raw data from the gyroscope, we must first calibrate the device and then integrate the data, as illustrated in the accompanying diagram.

Figure 5.4 The gyroscope data flow

Figure 5.5 The direction of robot in Quaternion

This is the direction of robot in Quaternion after calibration

Transmission result

The Firebase real-time database serves as a crucial intermediary between the web application and the autonomous vehicle, as illustrated in Figure 5.6 The vehicle's current position is indicated by its latitude and longitude coordinates, while the destination is represented by the value "0."

Figure 5.7 The location of autonomous vehicle

All features were thoroughly coded, tested, and debugged to ensure satisfactory functionality, with the exception of GPS navigation During code integration, any compiler errors were resolved, resulting in an error-free code that meets project requirements Testing of the mobile robot sub-system revealed errors influenced by environmental factors, particularly concerning GPS navigation, where the signal accuracy was inadequate However, the localization of the autonomous vehicle is successfully updated to the cloud server every 10 seconds, allowing for real-time monitoring of the navigation process to be viewed remotely, as illustrated in Figure 5.7.

Image processing and tracking results

To identify individuals, we utilize a Pi camera in conjunction with two photoelectric sensors The photoelectric sensors have a detection range of 0 to 50 cm, while the camera operates within a limited field of view of approximately 62.2 degrees If the camera detects any individuals within this specified area, the autonomous vehicle will come to a halt.

The way to identify a person and return the person's distance to the vehicle will be described in section 4.2.1

Figure 5.8 Viewpoint of Camera in Oxy plane

To effectively capture a full-body image from a distance of 40cm, we determined the optimal camera height and angle, positioning the camera at 0.5 meters with a 5-degree inclination from the x-axis.

Figure 5.9 Camera tilt relative to the x-axis

Using the Raspberry Camera, we implemented YOLOv4-tiny for effective people detection, which outputs bounding boxes around identified individuals As illustrated in Figure 5.10, the distance measurements reveal that one person is positioned 94 cm from the camera, while another is located 194 cm away.

Figure 5.10 Image captured by Raspberry Camera

The algorithm achieves a high level of accuracy in distance measurement, with a deviation of only ±10cm When an individual enters the camera's detection range, the robot will automatically halt, as illustrated in Figure 5.11.

Figure 5.11 Camera detects people in range

In this section we will present the results of the image processing:

- Apply mask for the image:

- Detect line with Hough-Transform:

Figure 5.17 Image with line detect

The results obtained from the others frame are shown in the following figures

CONCLUSION

Collaborative path AGVs are commonly used in complex logistics and distribution operations where multiple AGVs operate in the same space

Adaptive path AGVs leverage machine learning and artificial intelligence to enhance their path planning capabilities over time By analyzing historical data, environmental changes, and performance metrics, these AGVs continuously refine their navigation strategies This adaptability makes them ideal for dynamic environments where ongoing optimization of performance is essential.

Exploring autonomous delivery vehicles on university campuses is both scientifically and practically significant, engaging researchers and students This initiative provides a unique opportunity to delve into advanced transportation technology while addressing real-world campus logistics challenges, effectively connecting innovative research with practical applications.

Exploring autonomous delivery vehicles on university campuses highlights the intricate relationship between artificial intelligence, machine learning, and robotics This interdisciplinary approach fosters collaboration among experts, enhancing our understanding of self-navigation, path planning, and human-vehicle interaction Such scientific inquiry not only broadens our knowledge of autonomous technologies but also drives significant advancements in the field.

Implementing autonomous delivery vehicles on campus enhances the daily lives of students, professors, and staff by providing easy access to essential items through user-friendly mobile apps This innovation streamlines the academic experience, paving the way for future advancements in connected communities Additionally, the university campus serves as an ideal testing ground for these technologies, allowing autonomous vehicles to navigate complex pathways and interact with pedestrians The data collected during these operations is invaluable for researchers, as it aids in refining algorithms and improving the performance, safety, and reliability of autonomous systems in real-world scenarios.

The objectives of studying the field of autonomous delivery vehicles within a university campus are:

• Research and development technologies including image processing, machine learning, embedded systems which are essentially used in autonomous vehicles

• Develop guidelines and best practices for the deployment and operation of autonomous delivery vehicles within a university campus, considering regulatory compliance, privacy concerns, and ethical considerations

This graduation project is designed to be completed within a strict timeframe, necessitating a significant reduction in its scope to ensure feasibility and successful execution.

This project focuses on developing an autonomous vehicle capable of navigating a university campus Users will be able to input their desired destination through a dedicated website, allowing for efficient and convenient transportation across the campus.

• The vehicle must be able to detect people and objects on the road and detect the road lane to move on the road

• The operating time is during the daytime and the weather is assumed to be sunny

• The operating environment is assumed to be clean and clear, which is not close to any tall buildings or trees

In this project, we leverage scientific research to explore the specific characteristics, nature, and trends of our subject, allowing us to draw meaningful conclusions that align with our study's objectives The study primarily employs three key scientific research methods to achieve these goals.

• Theoretical research methods: Theoretical research method primarily involves collecting information through documents or texts Using this method, we refer to and search for information in relevant texts or documents

• Practical research methods: we directly interact with the research subjects As a result, the research subjects exhibit and manifest their essence and laws of motion

• Mathematical research methods: we use mathematical models to support statistical analysis of research findings, data gathered during the research process

The topic will be presented in parts from theoretical application to application Each issue in the article will be presented through each chapter as follows:

• Chapter 1: AN OVERALL PERSPECTIVE ON THE TOPIC

The first chapter is a brief introduction about the urgency of this topic, the scope, limitations of the project and the scientific research methods used in this project

The second chapter focuses on the theories and mathematical models which are used to support the study of this project

• Chapter 3: THE APPLICATION DESIGN AND HARDWARE DESIGN

This chapter consists of the hardware configuration of the autonomous vehicle and the software design for controlling the vehicle

• Chapter 4: THE VEHICLE CONTROL ALGORITHM

The fourth chapter discusses the workflow and algorithms for vehicle operation

• Chapter 5: THE EXPERIMENT RESULT AND ANALYSIS

This chapter includes the experimentation of developing the autonomous vehicle, results analysis and conclusion

CHAPTER 2: THEORETICAL BASIS 2.1 GPS (Global Positioning System)

GPS, or Global Positioning System, is a navigation technology that delivers accurate positioning and timing information by using satellite signals It calculates users' precise locations, velocities, and altitudes by measuring the time delay of these signals, allowing GPS receivers to determine coordinates with high accuracy.

GPS technology is widely utilized in various sectors, including navigation, mapping, surveying, and outdoor activities It enables users to pinpoint their locations with precision, plan effective routes, and navigate seamlessly between destinations Additionally, GPS provides valuable data such as speed, heading, and distance traveled, enhancing the overall user experience.

GPS has revolutionized navigation and exploration worldwide with its extensive coverage and constant functionality Its dependable positioning, available in all weather conditions, greatly enhances safety, efficiency, and convenience in both transportation and outdoor activities.

GPS satellites orbit the Earth twice daily, each transmitting unique signals and orbital data that enable GPS devices to determine their precise locations By utilizing trilateration, GPS receivers measure the distance to multiple satellites based on the time it takes for signals to arrive, allowing them to pinpoint a user's exact location This technology is essential for various applications, including tracking running routes, mapping golf courses, and navigating to desired destinations.

Using multiple satellites significantly improves GPS accuracy A GPS receiver relying on just one satellite can only determine its distance relative to that satellite, without a clear direction This is similar to a flashlight creating a circular area of light on the ground; the receiver could be anywhere within that circle By incorporating two additional satellites, two more circles are created, and the overlap of these three circles pinpoints the exact location of the GPS receiver This method of location determination is called trilateration.

Figure 2.1 Trilateration Source: from “GPS”, by National Geographic,

(https://education.nationalgeographic.org/resource/gps/)

GPS satellite signals are extremely weak when they reach the Earth's surface, traveling via line of sight While these signals can penetrate clouds, glass, and plastic, they struggle to pass through solid objects like buildings and mountains Fortunately, advancements in modern receivers have enhanced their sensitivity, allowing them to typically track signals even through houses.

A GPS signal contains three different types of information: [15]

• Pseudorandom code is an I.D code that identifies which satellite is transmitting information You can see which satellites you are getting signals from on your device's satellite page

• Ephemeris data is needed to determine a satellite's position and gives important information about the health of a satellite, current date and time

Almanac data provides GPS receivers with essential information about the expected positions of GPS satellites over several months, including their orbital details.

The bearing angle, also known as azimuth angle or simply bearing, refers to the horizontal angle between a reference direction and the direction of a specific point or object

It helps us describe the orientation or direction of an object in relation to a reference point There are two main types of bearing angles:

Absolute bearing refers to the angle measured in a clockwise direction from a reference point, typically North, to a specific object or point of interest This measurement provides a precise angle relative to the reference direction For instance, an object located directly east would have an absolute bearing of 90 degrees.