Thông tin tài liệu
ACKNOWLEDGEMENT
I would like to express my sincere gratitude to my supervisor Associate Professor
ZHANG Yunfeng, for his invaluable guidance throughout this project. From literature survey
methodology at the beginning, to the crucial steps taken throughout the project, his advices kept
me on the right direction and inspired me greatly.
In addition, thanks are given to PhD candidates Mr. GENG Lin and Dr. WANG Yifa,
who had been very supportive for my research.
I would also like to show my appreciation for the financial support in the form of a
research scholarship from the National University of Singapore.
i
TABLE OF CONTENTS
ACKNOWLEDGEMENT ............................................................................................................... i
TABLE OF CONTENTS ................................................................................................................ ii
ABSTRACT .................................................................................................................................. vii
LIST OF FIGURES ....................................................................................................................... ix
LIST OF TABLES ........................................................................................................................ xii
CHAPTER 1
INTRODUCTION .......................................................................................................................... 1
1.1 The History of UAVs ............................................................................................................ 1
1.1.1 World War I ................................................................................................................... 1
1.1.2 World War II .................................................................................................................. 2
1.1.3 Post-World War II, through pre-Vietnam war ............................................................... 2
1.1.4 From Vietnam War to Gulf war ..................................................................................... 2
1.1.5 From Gulf war to present ............................................................................................... 3
1.2
Modern Research Areas Concerning UAVs .................................................................... 3
1.2.1 Sensor technologies ....................................................................................................... 3
1.2.2 Control technologies ...................................................................................................... 4
1.2.3 Mission management system ......................................................................................... 4
1.2.4 Mission planning ............................................................................................................ 4
1.3 UAV Path Planning .............................................................................................................. 5
1.3.1 Multiple travel salesman problems with time window constraints (mTSPTW) model . 6
1.3.2 Dots-on-coordinate model ............................................................................................. 7
1.3.3 Polygon-obstacles model ............................................................................................... 8
ii
1.3.4 Three-dimensional (3D) mission model ........................................................................ 9
1.4 Integrated Task Assignment and Path Optimization .......................................................... 12
1.4.1 Dubins‟ vehicle ............................................................................................................ 14
1.4.2 The dynamic network flow model ............................................................................... 15
1.4.3 The tree representation model ...................................................................................... 16
1.4.4 The graph representation model................................................................................... 16
1.5 Surveillance in Complex Urban Environment .................................................................... 16
1.6 Research Objectives ............................................................................................................ 17
1.7 Limitations and Assumptions of Research ......................................................................... 18
1.8 Organization of the Thesis .................................................................................................. 18
CHAPTER 2
RESEARCH MOTIVATIONS ..................................................................................................... 20
2.1 UAV Path Planning in Complex and Realistic Environment ............................................. 20
2.2 UAV Cooperative Task Assignment .................................................................................. 21
2.3 UAV Surveillance in Complex Urban Environment .......................................................... 21
2.4 Genetic Algorithms as a Solution Tool............................................................................... 22
2.5 Java Monkey Engine (JME) as System Implementation .................................................... 22
CHAPTER 3
INTRODUCTION TO GENETIC ALGORITHM ....................................................................... 24
3.1 Encoding of the GA Chromosome...................................................................................... 26
3.2 The Initialization ................................................................................................................. 27
3.3 The Fitness Function ........................................................................................................... 27
3.4 The Genetic Operators ........................................................................................................ 28
iii
3.5 The Selection Process ......................................................................................................... 29
CHAPTER 4
A GA-BASED UAV PATH PLANNER
IN COMPLEX REALISTIC ENVIRONMENT .......................................................................... 31
4.1 The Proposed Model for UAV Path Planner ...................................................................... 32
4.1.1 The description of terrain ............................................................................................. 32
4.1.2 The description of the UAV path ................................................................................. 35
4.1.3 The description of threats ............................................................................................. 39
4.2 The Design of Genetic Algorithm ...................................................................................... 41
4.2.1 The encoding of solution ............................................................................................. 41
4.2.2 Initialization ................................................................................................................. 41
4.2.3 The fitness function...................................................................................................... 43
4.2.4 The selection mechanism ............................................................................................. 47
4.2.5 The genetic operators ................................................................................................... 47
4.3 Simulation Experiments ...................................................................................................... 50
4.3.1 Experiments on different fitness functions .................................................................. 50
4.3.2 Experiments on functions of different GA operators ................................................... 55
4.3.3 Experiments on magnitude of probabilities for the GA operators ............................... 61
4.3.4 Experiments with different UAV and UAV path characters ....................................... 62
4.4 Discussions ......................................................................................................................... 67
CHAPTER 5
A GENETIC ALGORITHM
FOR UAV COOPERATIVE TASK ASSIGNMENT .................................................................. 69
iv
5.1 The Problem Description .................................................................................................... 69
5.2 Dubins Vehicle ................................................................................................................... 70
5.3 Graph Representation ......................................................................................................... 73
5.3.1 Discrete UAV heading angle on target ........................................................................ 73
5.3.2 The graph representation.............................................................................................. 74
5.4 The Solution Encoding ....................................................................................................... 76
5.5 The Initialization ................................................................................................................. 77
5.6 The Fitness Function ........................................................................................................... 79
5.7 The Genetic Algorithm Operators ...................................................................................... 81
5.8 The Simulation Experiments .............................................................................................. 83
5.8.1 Diversity of the random solution generator ................................................................. 84
5.8.2 Experiments on different probabilities of genetic operators ........................................ 84
5.8.3 Experiments with different fitness functions ............................................................... 87
5.8.4 Experiments with different UAV flight dynamics ....................................................... 89
5.8.5 Stability of the solution algorithm ............................................................................... 96
5.9 Summary ............................................................................................................................. 97
CHAPTER 6
AN OCCLUSION-AWARE MODEL FOR UAV SURVEILLIANCE
IN COMPLEX URBAN ENVIRONMENT ................................................................................. 99
6.1 Problem Description ........................................................................................................... 99
6.1.1 The environment .......................................................................................................... 99
6.1.2 The sensor capability of UAVs .................................................................................. 100
6.1.3 The objective function ............................................................................................... 100
v
6.2 The Two-Stage Optimization............................................................................................ 101
6.3 The Occlusion Aware Model ............................................................................................ 101
6.4 System Implementation of the Occlusion Aware Model .................................................. 102
6.5 Limitations of the Two-Stage Method .............................................................................. 104
6.6 Discussions ....................................................................................................................... 105
CHAPTER 7
CONCLUSIONS AND FUTURE WORK ................................................................................. 106
7.1 Conclusions on the Study of the UAV Path Planning Problem ........................................ 106
7.2 Conclusions on the Study of the Multiple UAV Task Assignment Problem ................... 107
7.3 Conclusions on the Study of the UAV Surveillance Problem .......................................... 108
7.4 Recommendation for Future Work ................................................................................... 108
REFERENCES ........................................................................................................................... 110
vi
ABSTRACT
The research work reported in this thesis addresses the fixed-wing UAV mission planning
problem. More specifically, the study focuses on 3 sub-problems: single UAV path planning,
multiple UAV cooperative task assignment, and UAV surveillance in an urban environment. The
study for each specific problem includes the development of problem modeling method and
optimization solution method, followed by simulation experiments for verification.
The first part considers the problem of finding an optimal path for a UAV in a realistic
environment. The realistic model used in this project includes terrain, radar zone, flight
prohibited zones, and UAV capability limits. The flight path is represented by piecewise cubic
Bezier curves. A genetic algorithm (GA) is developed to find the optimal solutions. The unique
design of the GA, including solution encoding, GA operators, and fitness function, allows the
search to escape from those local minimums and manage to find the optimal or near-optimal
solution in a robust manner.
The second part considers the problem of multiple UAV coordination to accomplish
assigned tasks on different targets. To accommodate the fact that UAVs can be heterogeneous, a
multiple graph representation scheme is proposed to model the solution map for each UAV. A
GA with unique design of encoding scheme and GA operators is developed to find optimal
solutions. The testing results from various simulation experiments show that the GA is able to
solve this coordination problem in an effective and efficient manner.
The third part considers the problem of UAV surveillance in an urban environment. To
overcome the limitation of previously reported models, an occlusion-aware model is developed
to simulate the environment under surveillance (terrain and buildings). An efficient ray-casting
based collision checking algorithm is also developed to determine the visibility of any given
vii
observation position of the UAV. Again, heterogeneous UAVs can be modeled. Due to the
limitation of time, however, the solution method to find the optimal set of observation points for
UAVs is yet to be developed.
In summary, the study on the 3 different UAV mission planning problems represent a
major step towards developing advanced modeling and solution tools to deal with close-to-reality
problems. Although there are plenty of challenges to be tackled before the final goal can be
achieved, this effort has built solid foundations for future extension.
viii
LIST OF FIGURES
Figure 1.1 Dot-on-coordinate mission planning model and Voronoi Diagram solution
Figure 1.2 Interface of Polygon-obstacles model
Figure 3.1 The evolution process of giraffes
Figure 3.2 Illustrations of gene and chromosome
Figure 3.3 Example of solution encoding for path planning problems
Figure 4.1 Grayscale height map representation of Oahu Island
Figure 4.2 Rendered 3D representation of Oahu Island
Figure 4.3 Illustration of UAV flight constraints
Figure 4.4 Cubic Bezier interpolation
Figure 4.5 Illustrative example of mission planner over Oahu Island, Hawaii
Figure 4.6 A chromosome with two intermediate waypoints
Figure 4.7 The cross-over operator
Figure 4.8 The local mutation operator
Figure 4.9 The strong mutation operator
Figure 4.10 The deletion operator
Figure 4.11 Test 1: simple obstacle evasion test for static fitness
Figure 4.12 Test 2: concave obstacle evasion test for static fitness
Figure 4.13 Different solutions with same static fitness value
Figure 4.14 Test 3: concave obstacle evasion test for dynamic fitness
Figure 4.15 Test 4: fitness plot for random generation
Figure 4.16 Test 5: fitness plot for cross-over operator
Figure 4.17 Test 6: comparative study of local and strong mutation operators
Figure 4.18 Test 7: simple test without deletion/smoothening
ix
Figure 4.19 Test 8: simple test with deletion/smoothening operator
Figure 4.20 Test 9: aggressive UAV maneuverability
Figure 4.21 Test 10: conservative UAV maneuverability
Figure 4.22 Test 11: fitness evolution for different values of d
Figure 5.1 The minimum path problem
Figure 5.2 Illustration of a feasible path
Figure 5.3 Dubins path is one of the four possible choices when
Figure 5.4 Two additional choices of Dubins path at close starting and ending distance
Figure 5.5 Illustrative example of a mission involving two UAVs and two targets
Figure 5.6 Discrete headings of UAV
Figure 5.7 Two examples of paths with different heading angle on target
Figure 5.8 The graph representations with 3 part discretization
Figure 5.9 One solution of the example
Figure 5.10 Algorithm for the random solution generation
Figure 5.11 Cross-over operator on same solution across different UAVs
Figure 5.12 Mutation operator changes heading angle
Figure 5.13 Illustration of swap operator
Figure 5.14 Fitness evolution for different GA operators
Figure 5.15 Best fitness evolution for different GA operator probabilities
Figure 5.16 Average fitness evolution of the different GA probabilities
Figure 5.17 Result comparison for different fitness functions
Figure 5.18 Fitness value evolutions for different fitness functions
Figure 5.19 Simulation output for turning radius 1
Figure 5.20 Fitness evolution of the solution of experiment with turning radius 1
Figure 5.21 Simulation output for turning radius 3
x
Figure 5.22 Fitness evolution of the solution of experiment with turning radius 3
Figure 5.23 Simulation output for turning radius 7
Figure 5.24 Fitness evolution of the solution of experiment with turning radius 7
Figure 5.25 Simulation output for turning radius 10
Figure 5.26 Fitness evolution of the solution of experiment with turning radius 10
Figure 5.27 Simulation output for heterogeneous turning radius
Figure 5.28 Fitness evolution of the solution of experiment with heterogeneous turning radius
Figure 5.29 The best fitness of 10 same-parameter simulations
Figure 5.30 The average fitness of 10 same-parameter simulations
Figure 6.1 Visibility test with analytical method
Figure 6.2 Illustration of ray detection
Figure 6.3 Occlusion aware model
xi
LIST OF TABLES
Table 1.1 3D models by components
Table 1.2 Models and solution algorithms in previous works
Table 4.1 Parameters defining the piecewise cube Bezier curve
Table 4.2 Static fitness value assignment
Table 4.3 Dynamic fitness value assignment
Table 4.4 Symbols for probability of each EA operator
Table 4.5 Controlled constants for experiments on fitness function
Table 4.6 Homogeneous assignment of fitness multipliers
Table 4.7 Final assignment of fitness multipliers
Table 4.8 Controlled variables for experiments on EA operators
Table 4.9 EA operator probabilities for immigration testing
Table 4.10 EA operator probabilities for cross-over testing
Table 4.11 EA operator probabilities for local mutation testing
Table 4.12 EA operator probabilities for local and strong mutation testing
Table 4.13 EA operator probabilities for deletion/smoothening testing
Table 4.14 Fitness values after 50 iterations with different EA probabilities
Table 4.15 Final adopted EA operator probabilities
Table 4.16 Aggressive UAV maneuverability
Table 4.17 Conservative UAV maneuverability
Table 4.18 Different distances between two consecutive waypoints
Table 5.1 Constant parameters for genetic operators‟ test
Table 5.2 Solution quality of the different GA operators
Table 5.3 Constant parameters for different fitness function test
xii
Table 5.4 Constant parameters for UAV flight dynamics tests
Table 5.5 Result statistics
Table 5.6 Parameters used for repeated simulations testing stability
xiii
CHAPTER 1
INTRODUCTION
1.1 The History of UAVs
This section firstly introduces the definitions UAVs and differentiates them from
conventional aircrafts. It then outlines the essential technologies required for them. A revision of
the historical development of UAVs is also given in detail.
There are two ways to categorize aerial vehicles. The first way is according to their
method of guidance and the second is according to whether they are expendable. In terms of
guidance technology, an aerial vehicle can be either manned, remote piloted or auto piloted.
Expendable aerial vehicles are not reusable and are categorized as missiles. The UAVs discussed
in this thesis are auto or remote piloted and recoverable. James [1] summarizes three important
technologies in order to make UAVs successful for operational use: (1) an aerial platform
capable to maneuver to an appropriate objective; (2) a guidance system that would permit overthe-horizon unmanned aerial vehicle operations; (3) a payload that can perform a useful mission
once the platform gets to its objective.
1.1.1 World War I
Aerial balloon was a primitive form of UAV, used in the siege of Venice by the Austrian
troops in August 1849. In early 20th century, the invention of airplane by Wright brothers was a
major breakthrough in technology that brings UAVs into practical use. The directional mobility
of an airplane is a capability that balloons do not have. Thereafter, the first essential technology
is met. Aerial vehicles are able to reach their objectives. However, the guidance of such vehicles
cannot be achieved without onboard pilots. In 1916, the first pilotless aircraft - the “aerial
1
torpedoes” appeared. Designed by the U.S. and Britain, these are aircrafts filled with explosives
designed to fly for a set distance and crash. However, due to the inaccuracy of the guidance
system, none of the attempts during WWI resulted in operational application of the UAVs.
1.1.2 World War II
Each side had significant development in UAV technology during the war. The most
significant and notorious was the German V-1 [1]. The V-1 was a self-guided monoplane filled
with explosives that would fly a pre-set heading and time, at which time the engines would cut
off and the aircraft would go into a dive, exploding on impact. Hitler‟s V-1 campaign marked the
first large-scale operational employment of unmanned aircraft. Studies have shown that the cost
of the Germen was only one forth of that of Britain, thanks to the use of UAVs. On the other
hand, the allies achieved radio control of the unmanned aircrafts. However, without a computer,
automatic control is not feasible.
1.1.3 Post-World War II, through pre-Vietnam war
During this period, there were some significant improvements in guidance technologies
and flight endurance, which is a direct consequence of the escalating cold war between the US
and the Soviet Union. Therefore, cruise missiles and photo-reconnaissance aircrafts were
developed. However, the low reliability of such aircrafts makes them not practical for
operational use.
1.1.4 From Vietnam War to Gulf war
In terms of capabilities, the UAVs in this period improved in both flight capacities and
accuracies. Unmanned target drones increased in performance with speeds all the way up to
mach 4 and service ceilings of nearly 30000 meters. Their payload also increased, allowing the
2
carriage of bombs. In terms of guidance technology, they began as programmed drones. Later,
they were capable to receive order when in flight. This period was an important one during
which the technology gap between UAVs and the manned aircrafts had been removed.
1.1.5 From Gulf war to present
The increasing computer processing capabilities, data transmission rates and
miniaturization technology were the driving forces behind the development of UAV capabilities
during this period. Computer processing power allowed for smaller chips to be installed onto the
aircraft, increasing the payload. It also enabled auto-piloting by assimilating all the decision
variables in an efficient way. Fast data transmission rate ensures minimum delay in the
information obtained from the controller on the ground. This ensured the real-time decision
making by commanders. Miniaturization decreased the probability that a UAV will be detected
in its mission. This greatly ensured the secrecy of the mission. Moreover, miniaturization widens
the scope of missions that UAVs are capable to carry out.
1.2 Modern Research Areas Concerning UAVs
Due to the widening scope and increasing complexity of UAV missions, the researches
on UAVs span a very broad range. Some of the important research areas include sensor and
control technology, mission management system and UAV mission planning.
1.2.1 Sensor technologies
The development in sensor technologies aims at enhancing the UAV capabilities and
broadening the mission types a UAV is capable of. For the reconnaissance UAVs, the
complexities of the sensors decide the UAV‟s operational altitude, which is pertinent to its
survivability issues [2]. The operational efficiency of small scale UAVs heavily depend on the
3
balance between weight and complexity of its sensors [3]. The equipment of chemical sensor, for
example, enables the Raven UAV – a hand launched close range reconnaissance platform - to
detect and track a chemical plume in a variety of atmospheric conditions [4].
1.2.2 Control technologies
Researches in control technologies look at how to better take charge of the UAVs while
they are a certain distance away. There are two aspects associated with the control technology
challenges. The first is how to send orders to the aircrafts while they are distant from the control
center. The second is how to achieve real-time positioning of the aircraft while they are over the
horizon. Cooperative control investigates the distributed control of multiple UAVs over the
horizon [5]. Integration of onboard and ground camera is a proposed method for real-time
positioning of UAVs [6]. Another study proposes UAV control by means of a single axis rate
gyro, an absolute pressure sensor and a GPS receiver [7].
1.2.3 Mission management system
Researches on mission management system target at facilitating the battlefield
commanders in high-level control and decision making. Cassandras and Wei proposed a
simulated battle space and a dynamic target assignment scheme which achieves the commander‟s
order with optimal performances [8]. Another example is the airspace integration through
network among the UAV, the mission management station and the air traffic control [9].
1.2.4 Mission planning
UAV mission planning, which is also the focus of this study, considers optimization of
flight paths in a dynamic environment in which numerous mission objectives are defined. There
are two kinds of problems in this area of research. The first considers the optimization of the
4
UAV flight paths, which is called UAV Path Planning. The second considers the cooperation of
different UAVs in achieving a collection of mission objectives which is called UAV Cooperative
Task Assignment. More details on these two problems are discussed in the following section.
1.3 UAV Path Planning
UAV mission planning is in itself a very broad research area in which the mission
coordination among multiple UAVs and the path planning of UAVs are two fundamental areas.
These two areas are chosen in this study as the results can be used for future research on mission
planning, e.g., multiple UAV mission planning, real-time, and variable velocity UAV path
planning. For example, multiple UAV mission planning requires a target allocation step before
applying the results of this study. Real-time planner runs parallel threads simultaneously, of
which the primary threads are to be developed in this study. Variable velocity path planner
makes the flight constraint a new variable – which is assumed to be constant throughout a single
UAV take-off.
The previously reported work on UAV path planning is studied from the following two
different perspectives:
(1) The robustness of the model. This examines the UAV path planning model in terms of its
accuracy, complexity and authenticity.
(2) The soundness of solution algorithms. This concerns whether the solution algorithm is
fast enough in terms of finding the optimum solutions.
At the same time, these two perspectives are highly correlated. A very complicated model
can describe the real-life situation well, yet it is more demanding on the solution algorithms. On
the other hand, a simple model gives only a heuristic of the real-life yet the optimal solution
5
could be found much easier. It is such trade off that makes the researches in UAV path planning
challenging, yet interesting.
In general, finding the optimal solution to the route-planning problem is
nondeterministic-polynomial-time complete (NP-complete). Moreover, the UAV path planning
problems are characterized by its huge solution space. Early researchers focused on modeling the
problem in two dimensional (2D) spaces. More solution-demanding three dimensional (3D)
models appeared only after the 2D case studies were mature enough and powerful solution
algorithms were developed. Even today, 3D models still have potential for further developments.
Some important modeling methods reported in the literatures are briefly reviewed in the
following sections.
1.3.1 Multiple travel salesman problems with time window constraints (mTSPTW) model
The first modeling approach on the UAV mission planning problem derives from the
famous mTSPTW [10]. In this model, the UAV has predefined starting and ending positions and
a number of targets to be visited. The route connecting any pair of targets is pre-defined. Each
target is to be visited within a specified time window. Tabu search was proposed to be the
solution method for such model [11, 12]. Tabu search enhances the performance of a local search
method by using memory structures: once a potential solution has been determined, it is marked
as tabu so that the algorithm does not visit that possibility repeatedly. Such local search
procedure is used iteratively to move to better solutions until the stopping criterion is reached.
This model links the UAV mission planning problem to the mature problem of mTSPTW,
which allows for the application of various previous results. However, mTSPTW is more of a
mathematical problem than a practical model. It hardly describes any real world UAV mission
6
situations. Therefore, the path planning result is of little practical use. On the algorithm side, tabu
search is not capable of solving more complex models proposed subsequently.
1.3.2 Dots-on-coordinate model
This proposed model (see Figure 1.1a) describes the mission setting on a 2D coordinate.
It is called “dots-on-coordinate” in this study. One or several UAVs (black triangles) with their
fixed initial positions are to visit single or multiple targets (squares). The constraints are threats
in the form of dots. The closer the UAV is to a dot, the greater the probability of destruction. The
objective is to find a path or multiple paths that visits the targets while minimizing the
probability of destruction of the UAV.
(a)
(b)
Figure 1.1. Dot-on-coordinate mission planning model and Voronoi diagram solution [13]
Studies had targeted both single [14] and multiple [13, 15, 16] UAV mission planning in
such setting. One proposed solution algorithm is the two-stage Voronoi diagram (see Figure 1.2b)
and virtual forces [13]. Voronoi diagram is a way of space repartition by straight lines such that a
point on any given line has equal distance to the two dot-threats closest to it. Hence, finding the
optimal path is transformed to a discrete problem of finding a combination of edges on the
7
Voronoi diagram. In the first stage, graph search is applied to find the best combination of
Voronoi edges as a rough solution. Virtual forces are imaginary repelling forces received by the
paths from the dot-threats. The closer a path is to a threat, the greater the force. Better paths are
subject to smaller repelling forces. The second stage includes the trimming of the rough solution
with virtual forces. A gradient decent method is applied here to arrive at the path that has the
smallest repelling force. The solution algorithm proposed for target assignment is the satisfying
and social welfare paradigms [17]. Multiple UAVs are first decomposed into teams with
assigned targets to each team. The welfare of each team is then optimized in parallel, allowing
for re-allocation of targets in the solution process. Another solution method proposed for solving
the Voronoi diagram is the A-star algorithm. Genetic algorithms are also proposed for the second
step of virtual forces [18].
1.3.3 Polygon-obstacles model
Another popular problem is similar to the Dots-on-coordinate model. But instead of
having dots as the threats, polygons are used to enclose areas as restricted zones (see Figure 1.2).
The objectives are waypoints on the 2D map to be visited and the constraints are the shaded
areas that should not be visited. Two improvements in the model were made as compared to the
Dots-on-coordinate model. First, the dimension of the search space is greatly increased from the
Voronoi edges to the entire 2D space (Except for the prohibited zones). Second, the coupling of
the UAV missions are taken into account [19]. This means that, for instance, the elimination of a
hostile surface to air missile site will have a positive effect on all the UAVs concerned.
8
Figure 1.2. Interface of the Polygon-obstacles model [20]
One proposed solution method is the Mixed-integer Linear Programming [19-23]. However,
being a linear programming method, the computation becomes very costly when the complexity
of the problem increases. Another proposed method is the A-star algorithm [24-26]. Being a
heuristic search algorithm, its efficiency is not robust to the increase in complexity in the model.
1.3.4 Three-dimensional (3D) mission model
Recently, evolutionary algorithm (EA) becomes a popular choice for solving route
planning problem. Since it greatly increases the computational power, complex 3D models are
widely used for the UAV mission planning problems. The major components of a 3D model are
terrain, UAV path, and threats.
For terrain modeling, one method is to artificially generate a terrain surface [27-30]. The
terrain elevation of a point z(x, y) is given by:
z ( x, y) sin( y a) b sin( x) c cos(d y 2 x 2 ) e cos( y) f sin( f y 2 x 2 ) g cos( y)
(1.1)
where a, b, c, d, e, f, g are parameters defining the terrain shape. Such method is not demanding
on the memory and guarantees the smoothness of the terrain surface. However, manually
9
generated terrains lack authenticity as compared to real terrain. Yet, it is not certain whether
memory overflow issues will arise when real terrain is applied.
In other reported works, real-world elevation data is used for terrain representation [3136]. Digital Elevation Model (DEM) is commonly used terrain model in this case. It is a
continuous representation of the ground surface landform through a matrix that contains the
elevation data at the corresponding coordinates. The merit of such representation is that the DEM
data is a common form of geographic data representation. As a result, DEM data of different
parts of the world is widely available. However, the rendering of large area of 3D terrain from
DEM data is memory-intensive. Moreover, DEM may lead to discontinuity of the terrain.
Although highly unusual, this will result in inaccuracy of the path planning procedure.
The choice of path model directly affects the solution algorithm and the complexity of the
entire mission planning model. Paths can be in the form of point-wise linear segments
determined by a list of 3D absolute Cartesian points (x, y, z) [32, 36]. However, such paths are
not smooth and hence hard to be followed by UAVs. In other works, Bezier curve [31] or Bspline curve [37] is used to represent the path to be followed by the UAV. Other curve models,
such as cubic spline and 3rd degree B-spline curves, were also used for path representation [3335]. In this study, piecewise cubic Bezier curve is used for the representation of the UAV path
due to computational efficiency.
In terms of the constraints imposed on path planning, some focused only on the collision
avoidance with the terrain [27-31], while others introduced additional simple circular threat
model (representing enemy radar zone) that the UAV path must avoid [32, 36]. However, such
radar model differs with any real-world radar. For real radar, its probability of object detection
decreases with range. Some took a further step to incorporate a realistic model of both the enemy
10
radar and the flight prohibited zones (FPZs) [33-35]. Yet with increased complexity, the
computational load increases significantly, thus affecting computational efficiency.
As for the solution method to work with the 3D model, EA is typically the optimization
method in which the coding of the points controlling the smooth path is critical. Some EAs
codify the 3D points with absolute Cartesian coordinate [27, 30, 31]. Such coding scheme suits
the EA operators well; however, the search space would be enormous with such setting. Others
use relative polar coordinates for (x, y) [29, 34, 35], which would greatly reduce the search space.
However, with such representation, changing any intermediate point will induce the change in all
the subsequent points. Relative polar coordinate encoding is thus hard to handle for local
modifications. Table 1.1 summarizes the merits and disadvantages of the various modeling
approaches for different components of the 3D UAV path planning model.
Table 1.1. Various 3D modeling approaches for UAV path planning
3D Component
Terrain
Model
Parametric artificial
terrain
Digital Elevation Model
Linear Segments
UAV Path
Threats
EA Coding
Advantages
Continuous, not
demanding on
memory
Abundant realistic
data
Straight forward and
requires less
calculation
Disadvantages
Unrealistic
Memory intensive, may
have discontinuity
Sharp turns is hard to
be followed by UAVs
B-spline curve
Local control
More computations
Single Bezier curve
Easier to compute
Non-local control
Circular Radar Zone
Combined realistic radar
and FPZs
Absolute 3D Cartesian
coordinate
Relative polar
coordinate
Easy to calculate
Unrealistic
Demanding on the
solution algorithm
The search space is
large
Local changes are hard
to handle
Realistic
Easy to be used for the
EA operators
Reduced Search Space
11
Table 1.2 summarizes the models and solution algorithms adopted in some of the
important previous works on UAV mission planning.
Table 1.2. Models and solution algorithms in previous works
Model
mTSPTW
Dots-oncoordinate
Solution Algorithm
Tabu search
Voronoi
Diagram
Graph search, A-star
Virtual
Forces
Gradient Descent, genetic
Algorithms
Polygon-obstacles & coupled
UAV missions
Three-dimensional mission
Related Literatures
[11, 12]
[13-18]
Mixed-integer linear
programming,
[19-23]
A-star
[24-26]
Evolutionary Algorithms
[27-36]
1.4 Integrated Task Assignment and Path Optimization
In a generic UAV task assignment problem, the UAVs are required to perform three tasks
(classify, attack, and verify) on each of the targets in the battlefield. The requirement of flyable
trajectories dictates a lower bound on the turn radius and speed of the aerial vehicles. Thus, the
generic problem is denoted bounded speed task assignment problem (BSTAP) [38]. This generic
problem was further defined more specifically in [39] and is adopted in this research. It is
assumed that the terrain has already been searched by other means and several targets have been
found. Let T = {1, 2, …, Nt } be the set of targets found, V = {1, 2, …, Nv } be the UAVs
performing tasks on these targets. The set of tasks that need to be performed by the UAV team
on each target is M = {Classify, Attack, Verify} and we denote Nm as the number of such tasks.
Target classification, consisting of maximizing the correct target recognition under given
observation ability, can be performed only if the vehicle follows a trajectory that places its sensor
12
footprint on the target. After a target has been successfully classified, one or more UAVs attack
it by releasing appropriate weapons. Following target attack, cooperative damage verification is
performed. In general, the objective is to minimize the total length of the paths of the UAVs
combined.
In the previous works, some assumptions had been used in the description of the basic
problem. With these assumptions, the problem is presented in its simplest form. Relaxing
assumptions will lead to more complicated problem formulation. The implications for relaxing
the assumptions will be discussed further. The assumptions for the basic type of generic
problems are as follows.
(1) Each UAV flies at its own, fixed altitude so that there are no collisions among the UAVs.
(2) All UAVs are homogenous and fly at a constant speed. They hence have the same
minimum turning radius.
(3) Each UAV is capable of carrying out all three tasks on each target.
(4) Each UAV has only three motions: (i) flying straight ahead; (ii) turning left; or (iii)
turning right at the minimum turning radius.
(5) There is no constraint of fuel and weapon system. Therefore, each UAV can travel for as
long as it is assigned and visit as many targets as it can.
(6) It takes one UAV only one time to put its sensor footprint above (to fly over) a target in
order to classify the target.
(7) It takes one UAV only one time to fly over a target in order to destroy it.
(8) It takes one UAV only one time to fly over a target in order to verify that the target is
destroyed.
(9) All the UAVs are deployed simultaneously at known initial positions.
13
(10) The targets are homogenous. There is no priority in destroying any of them.
(11) Each target can be destroyed at any time. There is no time window within which a target
is to be destroyed.
(12) There is no communication delay or interruptions among the UAVs. In the case that one
UAV detects a target; the target information is shared immediately with all other UAVs
on the same battlefield.
While other assumptions are easy to understand, assumption (4) is further discussed in
detail here. It is the so called “Dubins vehicle” problem [40], which explicitly defines the
minimum-length path given the starting and ending positions and tangents of a UAV path.
1.4.1 Dubins’ vehicle
In 1957, Dubins proved the existence and uniqueness of a minimal-length curve with a
constraint on average curvature, and with prescribed initial and terminal positions and tangents
[40]. Let a particle pursue a continuously differentiable path from an initial point u to a terminal
point v. Its speed is unity and its velocity vectors at u and v are U and V, respectively.
Furthermore, if X is a curve in real n-dimensional Euclidean space, parameterized by arc length
(s), for which X’’(s) exists everywhere, then the curvature, defined by X ' ' ( s) , is less than or
equal to R 1 everywhere, if and only if,
X ' (s1 ) X ' (s2 ) R 1 s1 s2
(1.2)
for all s1 and s 2 in the interval of definition of X.
A curve X in real Euclidean n-space parameterized by arc length has average curvature
always less than or equal to R 1 provided that its first derivative X ' exists everywhere and
satisfies the Lipschitz condition in Eq. (1.2). Such path of minimum length between points u and
14
v with starting and ending conditions of U and V is called an R-geodesic. The following theorem
is the main theorem of Dubins‟ research [40]:
Theorem 1 – Dubin’s Vehicle
Every planar R-geodesic is necessarily a continuously differentiable curve which is either:
(1) an arc of a circle of radius R, followed by a line segment, followed by an arc of a
circle of radius R; or
(2) a sequence of three arcs of circles of radius R; or
(3) a sub-path of a path of type (1) or (2).
If applied to UAV paths, the theorem basically says that given the starting and ending positions
and directions, there exists only one minimum-length path, which is among the six options.
Among the six possible paths, one is the minimum length path according to the theorem.
In the following sections, three variations of the generic problem, reported in the previous
works, are discussed. One or many assumptions mentioned above are relaxed in each variation.
As a result, different problem formulations and solution algorithms are proposed.
1.4.2 The dynamic network flow model
In [41, 42], the air vehicle resource allocation problem is solved using dynamic network
flow optimization models. The assumptions (5), (6) and (10) are relaxed. The UAVs have only
sufficient fuel to fly for 30 minutes. Differences in onboard weapons and the attack angle also
restrict the attack of a target. The classification of target is a probabilistic event that depends on
the direction of the sensor on top of the target. Only when the classification probability of a
target is greater than 90% can the target be attacked. Otherwise it has to be classified again.
Moreover, targets are not homogenous and each target is assigned a value associated with its
importance. As for the solution method, Nygard et al. [41] proposed a network optimization
15
model for solving the problem and Schumacher et al. [42] employed a linear program based on
the capacitated trans-shipment problem to solve the task allocation problem.
In [38, 39], the problem of cooperative multiple task assignments is discussed. Further to
the relaxing of assumptions (5) as in [41, 42], assumption (7) is also relaxed. A second attack
must be performed on a target if its destruction is not verified or if multiple attacks are required
on the target. However, assumptions (6) and (10) are still used. As for solution method, Shima et
al. [39] employed genetic algorithm and Rasmussen [38] used a state-space search algorithm.
1.4.3 The tree representation model
Rasmussen and Shima [43] proposed a tree search algorithm for finding the solution for
this classic problem. There are two drawbacks of the tree representation. First, the exhaustive
expansion of all the branches of the tree is very memory-intensive. Usually, a heuristic search is
applied for tree models to avoid exhaustive search. This, however, could easily lead to local
minimums. Second, the tree structure is very hard to use for the representation of target visiting
sequence, which is an important factor that will affect the solution qualities.
1.4.4 The graph representation model
The graph representation has certain advantages as compared to the tree representation.
In such representation scheme, each solution is composed of a combination of edges of the graph.
The construction of the graph is not difficult and requires limited memory space. The
implementation of the graph representation will be discussed in detail in Chapter 5.
1.5 Surveillance in Complex Urban Environment
The integrated task assignment problem discussed above considers the coordination
among multiple UAVs to complete pre-defined tasks on spot targets. Another class of problems
16
is to find an optimal way to cover an area with complex surface conditions. For example, in the
urban areas, skyscrapers usually block the line of sight of the sensors in the sky.
The scenario is best described by the work of [44]. A team of UAVs with onboard sensor
is the given resource, the sensors have limited covering angles. In other words, the sensor
covering area is a cone-like space below the UAV. The team of UAVs is to survey a given urban
area with tall skyscrapers, with the objective to minimize the aggregate non-exposure time on the
ground. The problem formulation in [44] will be described in Chapter 7 in detail.
The drawback of [44] is two folded. First, it proposed a two-stage solution which
decouples the problem into two separate optimization problems. This greatly impedes the ability
of the algorithm to find global optimum. Second, its occlusion aware model allows only for flat
terrain and fixed shapes of buildings, making the problem unrealistic. This study focuses on
developing an occlusion aware model that is both efficient and allows for more generalized
problem definition.
1.6 Research Objectives
This thesis covers three different aspects of the researches on UAV mission planning.
The first is the trajectory and path planning of single UAVs. This study has gone very deep and
realistic into this area as compared to previous researches. The objective is to design a realistic
and complex optimization model and a robust solution algorithm efficient in solving the
optimization problem. The second objective is the coordination among multiple UAVs, in which
reasonably simple assumptions on the UAV trajectories are made so as to focus the
computational resources on the task allocation to different UAVs. Finding the optimal paths of
multiple heterogeneous UAVs is the problem to be solved. The third objective is to model an
17
urban environment in which the line of sight of UAV sensors can be blocked by skyscrapers.
This lays a foundation for the future application of path planning and cooperative task
assignment.
1.7 Limitations and Assumptions of Research
This research project considers the mission planning of the aircraft on a macro scale. The
problem studied does not include the design of the aircraft itself. All experiments are done
through computer simulations, with no actual fly-testing using the physical aircrafts. In terms of
solution algorithm, this study focuses only on the genetic algorithm for optimization. Although
three different topics are studied in this thesis, the general limitation and assumptions of the
research are listed below.
(1) The UAV is assumed to be a point, rather than a mass that has volume. Therefore, the
collision detection is on a single point.
(2) The UAV is assumed to fly at constant velocity. As a result, the turning radius
constraint is constant in each simulation.
The assumptions specific to each topic are described in detail in each section.
1.8 Organization of the Thesis
The thesis is organized into 7 chapters. The current chapter is an overview of the thesis
and the literature reviews. Chapter 2 explains the motivations of conducting the research. It
points to the weakness and room for further development in the current research. It also provides
reason for the use of genetic algorithm as a solution tool. Chapter 3 explains the principles and
mechanisms of Genetic Algorithm. Chapter 4 provides a detailed description of the
implementation of the path planning model. Moreover, the merit of the model and the Genetic
18
Algorithm as a solution is discussed in correlation with the results from the simulation
experiments. Chapter 5 starts by introducing the model for the problem of UAV cooperative task
assignment. It then furthers the discussion by justifying the model and solution through
simulation experiment. Chapter 6 introduces the implementation of an occlusion-aware model in
the urban surveillance problem. The generation of a vantage point set will later be used as the
target waypoints for multiple UAVs in the coordinate task assignment problem. Chapter 7
summarizes the major findings from the thesis and points to the possible future directions of
research.
19
CHAPTER 2
RESEARCH MOTIVATIONS
This chapter starts by describing the motivation of conducting research in the three areas
studied. The motivations concern more on the modeling and formulation of the problem.
Following this, the genetic algorithm (GA) as a solution tool is discussed.
2.1 UAV Path Planning in Complex and Realistic Environment
Despite much effort that has been put on the 3D UAV mission planning and some
success has been achieved, there are plenty of rooms for improvement. The following
observations from the previous reported works serve as the motivations for the current study:
(1) Among the reported optimization methods for path planning, there is no constraint on the
solution search space initially. Instead, random solutions are generated and evolved from
a very large space. The constraints are added during the evolution process as penalties for
unqualified individuals. Although genetic algorithms are powerful, the large initial search
space still adversely affects the path planning efficiency. This study seeks to reduce the
initial search space by forcing each individual in the initial generation to obey the flight
constraints.
(2) The previously reported methods have limitations in solving path planning problems with
concave obstacles. This study attributes it to the problem in evaluating each individual
solution. A new method is proposed to escape concave cases.
(3) Most of the previously reported methods are “result-driven”. They are eager to achieve
good results. This study, however, aims also at fully understanding the path planning
20
problem. Different parameters of the path planner are studied in detail so as to understand
how they affect the solution and algorithm convergence.
2.2 UAV Cooperative Task Assignment
For multiple UAV cooperative task assignment, the following issues are not fully
addressed in previous literatures.
(1) The efficiency of graph representation as compared to tree representation of the problem
is not demonstrated in the previous literatures. This study will demonstrate through
simulation experiments the merits of such.
(2) The UAVs considered in previous literatures all have homogenous flight dynamics.
However, this thesis considers UAVs with different flight dynamics through the
application of multiple graph structures.
Apart from the above two aspects, the development of a 3D display system for the
solution is also a motive for the current research. This is because all the previous literatures
could only represent the solution in 2D coordinates, which prevents the further development of
the problem in the future.
2.3 UAV Surveillance in Complex Urban Environment
The occlusion aware model is an important part of this class of problem. This is a model
that checks the visible areas on the ground from a sensor positioned in the sky. In the literatures,
the check for each sensor would require iteratively going through all the buildings concerned.
Moreover, the ground must be flat for such model. This gives raise to motivations to develop a
model that utilizes the “ray detection” function of the JME3 package. The efficiency is greatly
21
increased with the application of the “ray detection”. Moreover, the direct use of the 3D package
makes the result-rendering easier. This is an advantage for result-visualization.
2.4 Genetic Algorithms as a Solution Tool
Genetic algorithms (GA) are a type of stochastic and non-gradient search optimization
method. It enables searching efficiently a decision state space of possible solutions, as long as
the search space is not extremely rugged. As an optimization tool, GA has the following
characteristics.
(1) It does not require exhaustive search of the whole solution space in order to arrive at an
optimal or near optimal solution.
(2) It does not require the calculation of gradient for each iteration.
(3) It is exceptionally suitable for solving multi-objective optimization problems.
This study is motivated to use GA as a solution tool due to the nature of mission planning
problems. Firstly, the solution space of this type of problems is usually infinite and discontinuous.
Therefore, the exhaustive search algorithms are not suitable in this case. Secondly, it is hard to
measure the improvement on solution quality through gradient, rendering the gradient descent
methods unfeasible. Thirdly, there are usually multiple objective functions to be optimized,
which favor the use of GA.
2.5 Java Monkey Engine (JME) as System Implementation
Java is the main programming language for implementing the methods developed in this
study. As an object-orientated programming language, it is simple to use. All the unnecessary
complexities of the languages such as C++ are removed. Java enables the development of robust
applications on multiple platforms in heterogeneous, distributed networks. Moreover, due to the
22
3D nature of such project, the package JME is used in rendering the results. The JME package
has the following merits.
(1) Robust terrain package makes it easy to render the terrain from data. It also has functions
for height measurements and scaling. These are used in terrain collision detection.
(2) Compatibility with other Java packages.
(3) Plug-in of Java-swing components that allows for the programming of Graphical User
Interface (GUI) with JME.
23
CHAPTER 3
INTRODUCTION TO GENETIC ALGORITHM
The idea of GA comes from the natural evolution process. The famous reasoning behind
the long neck length of giraffes can be used here as an illustration. Giraffes with longer necks are
capable to reach higher branches of trees. They therefore have access to more abundant food
supplies as compared their shorter-necked peers. Fig. 3.1 illustrates the evolution process of two
generations of giraffes. The parent generation is constituted of giraffes of great diversity – long
and short neck ones. The offspring of that generation are therefore also of diversity. However,
due to the scarcity of food, some offspring may die out. According to the assumption – shorterneck ones have less food, the survival rate for longer-necked giraffes is much higher than that of
shorter-neck ones. This step in the evolution process is called “selection”. The environment
selects the most suitable individuals and therefore the most suitable genes. After generations of
evolution, shorter-neck giraffes died out and only the longer-neck giraffes survived and hence
their genes passed down.
The GA, which is also called evolutionary algorithm, is designed according to such
process. It involves five essential steps.
(1) The encoding of the solution – the mathematical representation of the solution. This
resembles the construction of the genes of the giraffes.
(2) Initialization – randomly generates a first generation with diversity. This compares to the
first generation of giraffes as shown in Fig. 3.1. The first generation should consist of
solutions with diversity.
24
(3) The fitness function – the mathematical way of quantifying how suitable is the solution to
the problem. In the giraffe example, giraffes with longer necks are “more fit” to the
environment. They hence have a larger fitness value as compared to shorter-necked ones.
(4) The genetic operators – this resemble the process of producing offspring. The common
operators include crossover, mutation, elitism, and immigration.
(5) The selection – this is the analogy of the natural selection process that retains the superior
individuals and kills the inferior ones.
Figure 3.1. The evolution process of giraffes
Each iteration of the algorithm produces a new generation. The algorithm stops when
certain convergence criteria are met. Thereafter, the individual with the best fitness will be
retained as the final solution.
25
Although a particular GA must be designed for each specific problem. The principle
mechanisms of this class of algorithms are the same. This section introduces the important
components of GAs.
3.1 Encoding of the GA Chromosome
The encoding is the representation of solutions by a list of data resembling that of the
genes on a chromosome. As shown in Fig. 3.2, a chromosome is an organized structure of DNA
and protein found in cells. It is a single piece of coiled DNA containing many genes. Each gene
can be in control of a particular biological feature of the cell, and hence some features of the
entire biological entity. If the chromosome and its interpretation mechanism of a biological entity
are known, this entity can hence be created.
Figure 3.2. Illustrations of Gene and Chromosome [45]
Derived from these biological facts, in GAs, both the chromosome interpretation process
and its inverse process are important. The encoding process involves designing the genes and
chromosome to represent the solutions. The decoding process involves interpreting the
chromosome into a solution. In some of the previously reported works, GA is widely used to
solve path planning problems. Using the waypoints‟ coordinates as genes for the chromosome is
a common way of solution encoding for the path planning problems. For the example path shown
in Fig. 3.3a, the encoding by its waypoints is shown in Fig. 3.3b.
26
(a) Path example
(b) Path encoding
Figure 3.3. Example of solution encoding for path planning problems
3.2 The Initialization
The initialization is a process that randomly generates a set of solutions, which is called
the “first generation”. Usually, each individual solution of the first generation satisfies a number
of constraints. However, the solutions of the first generation do not have to satisfy all the
constraints. This is because some constraints can later be met through evolutionary process. For a
well designed GA, the initialization process must be able to produce a first generation of great
diversity. Yet, the production of subsequent generations from the first one must also be efficient.
3.3 The Fitness Function
The fitness function is a mapping that maps the solution space into positive real numbers.
It basically describes how “optimal” the solution is, given the problem setting. The fitness
function design must accurately reflect the solution quality. In the giraffe problem, it is obvious
that bigger fitness values should be assigned to longer-neck giraffes. However, higher
dimensional and multiple objective problems do not have obvious fitness function assignment.
The fitness function assignment is usually the result of trial and error. In general, a good fitness
function must distinguish better solutions.
27
3.4 The Genetic Operators
Although the genetic operator is problem-specific, the principles behind some important
operators are the same across different problems. Four operators are discussed here: crossover,
mutation, immigration, and elitism. The details of these operators will be discussed in later
chapters according to the specific problems.
Crossover
Just as children retain the characters of both their parents, the crossover offspring
chromosomes contain the characteristics of both their parents. In other words, the crossover
operator swaps certain parts of two parent chromosomes to produce two offspring chromosomes.
Mutation
As the name suggests, the mutation operator drastically changes a single gene or a part of
the chromosome, according to certain pre-defined rules. It is designed to give big changes to the
solutions. Therefore, the principle application of mutation is to escape local minimums.
Immigration
Immigration operator completely replaces an existing chromosome with a new randomly
generated one. Without immigration, the whole generation will eventually become very similar
individuals which could lead to generating local minimums. With the presence of immigration
operator in each generation, the diversity of the solution can be maintained. There is always
some “fresh blood” in the new generation.
Elitism
28
Elitism is just the operator that retains the best solution of each generation. Since the
selection of offspring is probabilistic, without elitism, the best solution is not necessarily selected.
Therefore, elitism makes sure that the best solution is non-decreasing.
3.5 The Selection Process
The selection process is analogue to the natural selection process in the giraffe example
that kills the shorter-neck ones and retains the longer-neck ones. Before selection, the fitness
value for each individual in the current generation is calculated. Thereafter, there are mainly two
ways to select the individuals, the roulette-wheel selection and the tournament selection.
Roulette-wheel selection
In this study, the fitness proportionate selection, also known as the roulette-wheel
selection, is applied. In a generation of population N, let c i be the fitness of individual i in the
population. The probability that such individual will be selected for the next generation is
pi
ci
(3.1)
N
c
j 1
j
Tournament selection
Tournament selection involves running several "tournaments" among a few individuals
chosen at random from the population. The tournament selection pseudo code can be written as:
choose k (the tournament size) individuals from the population at random
choose the best individual from pool/tournament with probability p
choose the second best individual with probability p*(1-p)
choose the third best individual with probability p*((1-p)^2)
and so on...
29
This study applies the roulette-wheel selection for all the three problems. This is because
it involves only one sorting algorithm at the beginning of each iteration, whereas tournament
selection involves multiple sorting processes. Sorting algorithm is known to be an expensive type
of algorithm. Therefore, choosing roulette-wheel algorithm greatly reduces the computational
burden of the solution algorithm.
30
CHAPTER 4
A GA-BASED UAV PATH PLANNER
IN COMPLEX REALISTIC ENVIRONMENT
As a major part of this study, a UAV path planning system will be developed for complex
realistic environment. The focus is mainly on the following two aspects.
(1) the modeling of UAV mission planning problem in a 3D environment
(2) the design of a suitable solution algorithm for the model
The modeling method must satisfy the following requirements.
(1) To model the terrain on which the UAV missions are to be carried out. First, the terrain
model must be able to describe the real-world terrain. Second, the terrain model must be
convenient to use for common data structures available online.
(2) To model the UAV and its path. First, the flight dynamics particular to the aircrafts (as
opposed to ground vehicles) must be taken into account. Second, the UAV paths must be
modeled in a way that is simple to be encoded into the GA planner.
(3) To model the threats. This takes into account the threat from enemies and the flight
restrictions from the friendly commander.
There are some common challenges to the three modeling objectives. They must be
realistic, suitable for encoding into the GA and must not put too much burden on the computer‟s
computational resources.
Each optimization problem has its unique demand on solution algorithms. The following
points describe the requirements particular to this study.
31
(1) The solution algorithm must be specifically designed to suit the model of the problem, as
mentioned above.
(2) The solution algorithm must be robust to different situations. In the experiments
presented in this thesis, a same algorithm is tested with different terrain and obstacle
settings to check the robustness of the algorithm.
(3) The solution algorithm must be efficient even when the search space is greatly increased.
The above three optimization requirements cannot be attained in one shot. Hence,
repeated experiment and testing is required in order to arrive at a satisfactory model and solution
algorithm. From this perspective, this study is also a learning process, in which the
characteristics of the model and solution algorithm is fully explored and understood in great
depth through the fine-tuning of different parameters. Such in-depth understanding of the
problem will lay a solid foundation for future research when multiple UAVs and more
complicated situations are involved. Therefore, an important objective is to fully understand the
mechanism, the sensitivity and the characters of the model and solution algorithm through testing
and experiments.
4.1 The Proposed Model for UAV Path Planner
The model incorporates various components. The choice of each component in the
modeling process is the result of comparison among a number of alternatives from literatures.
4.1.1 The description of terrain
The environment in which the path planner works is an area of convex shape with surface
of varying height. Such surface is called terrain. Every point on the terrain can be described by
its position – the x, y coordinates, and the height – the z coordinate. The terrain is hence a 2D
32
function T ( x, y) . Mathematically, the area of concern A, the height map T and the terrain M can
be defined as:
Area: A ℝ2
Height map: T : A ℝ+
Terrain: M { x, y, z ℝ3 | x, y A, z T ( x, y)}
Further, the space S in which UAVs can operate without collision with terrain is defined
as:
UAV operating space: S { x, y, z ℝ3 | x, y A, z T ( x, y)}
In practice, the input terrain data is discrete, but the rendered terrain is continuous.
Among those previous effort for UAV path planning, the mTSPTW [11, 12] and the
Dots-on-coordinate [13-17, 26] are 2D models. Therefore, they could not encompass elevation
data. The 2D Polygon-obstacles [19-26] could model the mountains as obstacles. Yet, real-world
terrains are not blocks of vertically standing rocks. The parameterized artificial terrain [27-30] is
not realistic. The digital elevation model (DEM) [31-36] has been chosen in this study. Firstly,
geographical data is widely available in this form, allowing for easy use of real-world terrain.
Secondly, it is easy to obtain the height information at any given point of the terrain. Thirdly,
when a test on the parametric artificial terrain is desired, a DEM matrix can be easily generated
from the parametric terrain equation and used.
The raw data can be provided in two forms. The first form is a square matrix, of which
each entry represents the height information at the corresponding map coordinate. The data is
33
stored in Comma-Separated Values (CSV) file format, which can be easily edited by excel. This
way of data representation is straightforward, but it is difficult to visualize the terrain pattern just
by observing the data structure with eyes. Also, each terrain must be normalized in order to be
used effectively and comparatively by the mission planner. Another way is the 8-bit Greyscale
height map (see Fig. 4.1). On the 8-bit greyscale, the different colors between black and white
are divided into 256 levels, with 0 representing black and 255, white. On such height map, each
data point is represented by a pixel of color on the greyscale, with black representing the
minimum altitude. In Fig. 4.1a, the Hawaiian island „Oahu‟ is represented by the Greyscale
height map with the actual terrain pattern in Fig. 4.1b. In this way, given a 2D coordinate (x, y)
within the terrain boundary, the elevation at such point can be found easily. Fig. 4.2 shows the
rendered 2D representation of the Greyscale image shown in Fig. 4.1a. The camera is placed at
sea-level and heads in a horizontal direction with the highest point of the island in sight.
(a)
(b)
Figure 4.1. Grayscale height map representation of Oahu Island
Fig. 4.2 illustrates three important parameters for every terrain.
(1) The base altitude – the altitude in reality represented by the black color in the height map.
In the case of Oahu island, the black area in Fig. 4.1a – which represents an actual sealevel altitude – is the lowest altitude area. The base altitude in this case is hence 0.
34
(2) The vertical scale – actual increase in elevation in unit of meters represented by 1 unit
increase in the grayscale. In the case of Oahu island, an increase by 1 on the grayscale
represents 4.8 meters‟ increase in elevation in reality.
(3) The horizontal scale – actual distance in meters represented by each resolution on the
height map. In the case of Oahu island, each pixel in the height map represents a square
area of 60m×60m = 3600m2.
Figure 4.2. Rendered 2D representation of Oahu Island
All 3 parameters for terrain are encoded in the name of the data file. For example, the file
name of Fig. 4.1a is “OAHU(0_4.8_60).png”. It means that it is a Portable Network Graphics
(PNG) format image which has a base altitude of 0, vertical scale of 4.8, and horizontal scale of
60.
4.1.2 The description of the UAV path
The terrains used in this study are massive enough as compared to the size of the aircraft.
The UAV is hence modeled as a point. The magnitude of the UAV velocity is assumed to be
constant throughout each mission. Besides, the following flight dynamic constraints are imposed
on the UAV.
(1) Maximum Climbing Angle (MCA) – the biggest angle at which the UAV is able to climb
(see Fig. 4.3a). In reality, if the UAV climbs at an excess angle, it will soon lose its speed
and enter into stall.
35
(2) Minimum Gliding Angle (MGA) – the minimum angle at which the UAV is able to glide
(see Fig. 4.3b). In reality, gliding lower than such angle will result in loss of control of
the aircraft.
(3) Minimum Turning Radius (MTR) – The minimum radius at which the UAV is able to
make a turn (see Fig. 4.3c). In reality, an aircraft will lose its speed if it enters into a
sharp turn greater than the maximum turning angle.
(a)
(b)
(c)
Figure 4.3. Illustration of UAV flight constraints
In summary, the following factors need to be considered in modeling the UAV path.
(1) The path must satisfy the constraints illustrated by Fig. 4.3.
(2) It must be easy to check the path‟s condition throughout. The section of the path within
the terrain boundary, inside radar zones or inside flight prohibited zones can be identified
efficiently.
(3) The path is a continuous curve. But the evaluation of the path has to be done on discrete
points or segments. Therefore, the path must be suitable for proper numerical methods to
achieve point (2).
(4) The path representation and encoding must be suitable for the future use by evolutionary
algorithms.
36
In some of the previous research effort [27, 29, 30], the path is represented by B-spline
curves. It is then divided into equal segments to check for their collision with terrain and flight
constraint violations. This approach requires intensive computations, due to the large search
space without flight constraint and the calculation for each segment. Another way is to represent
the path by a single Bezier curve [31]. As discussed in the literature review, this is not a good
choice since local change in the curve will result in change in all the control points.
In this study, the UAV path is modeled as a piecewise C2 continuous cubic Bezier
interpolation of a series of consecutive waypoints. The terminal points are the starting and
ending position of the UAV. The curve between any two waypoints is a single cubic Bezier
curve. For a path with N+1 waypoints (including the starting and ending points), there are N
segments of cubic Bezier curves. Hence, the problem of finding a path has changed to finding the
intermediate waypoints and then interpolates them. The merits of such formulation are.
(1) A Bezier curve automatically has its first and last control points the same as the two
waypoints that it connects. In the case of cubic Bezier, only 2 control points remains to be
found. This greatly reduces the computational burden as compared to using B-spline.
(2) When the intermediate points are randomly generated, it is ensured that the path does not
violate the flight constraint by checking the properties at the two consecutive points. This
greatly reduces search space and increases accuracy of the path for the solution algorithm.
Fig. 4.4 shows the mechanism of piecewise cubic Bezier interpolation through a simple
example where only the x and y coordinates are shown. Fig. 4.4a shows a path found by the
mission planner from sPt to ePt, through finding two intermediate waypoints. The actual
rendered path in the mission planner is shown in Fig. 4.4b. The four points are interpolated by
three piecewise cubic Bezier curves, C2 continuous at the two waypoints. Fig.4.2a shows the
37
terminal points, waypoints, Bezier curves and control points in a Cartesian frame. Section k (1, 2
k
k
or 3) of the entire path is described by its four control points: C 0k , C1 , C 2 and C 3k by:
3
3
Bk (t ) (1 t ) 3i t i Cik
i 0 i
(4.1)
The data for the points describing the curve in Fig. 4.4 is shown in Table 4.1.
(a)
(b)
Figure 4.4. Cubic Bezier interpolation
Table 4.1 Parameters defining the piecewise cube Bezier curve
Control Points
C 01 (sPt)
x
50
y
50
C11
50
48.7
52.1
44.5
54.7
42.2
C
57.3
39.9
C
60.2
39.2
63.8
38.2
C
67.4
37.1
C
71.6
35.7
72
36
C
1
3
1
2
2
0
C , C , (waypoint 1)
2
1
2
2
2
3
3
0
C , C , (waypoint 2)
3
3
3
1
3
21
C , (ePt)
38
4.1.3 The description of threats
Section 1.3.1 describes past work of modeling radar zones as simple circular regions [32,
36], and flight prohibited zones (FPZ) as shaded area on 2D plan [19-23]. These two
representation schemes do not well reflect the realistic situation. The two types of model in this
study are described as follows.
(1) FPZ – They are 3D rectangular volumes defined by the user, these regions are not to be
visited by the UAV. On the other hand, flight ceiling, commercial aircraft flight zones
and foreign country‟s territories are examples of FPZ. Each FPZ is defined by its center
C x , C y , C z and
x, y, z extents
E x , E y and E z as:
Center: C x , C y , C z ℝ3
Extents: E x ℝ+, E y ℝ+, E z ℝ+
FPZ: FPZ { x, y, z ℝ3 | x [C x E x , C x E x ],
y [C y E y , C y E y ], z [Cz Ez , Cz Ez ]}
While a single FPZ thus defined is of simple rectangular shape, the combination of
FPZs can form complex concave regions.
(2) Enemy Radar – They are spherical volumes whose centers are on the terrain. Enemy
radars pose a threat to the UAV because they disturb the mission secrecy and for those
radar site equipped with missiles, possible destruction of UAV might be induced.
However, instead of modeling the radar zones as areas to be avoided, the probability of
detection is measured which depends on the distance from the radar centre, such model is
39
hence closer to the real radars. Since radars are always positioned on terrain surface, the
radar center CR , influence radius R and the radar zone Ra are defined as:
Radar center: CR S
Radar radius: r ℝ+
Radar zone: Ra {x ℝ3 | CR x r}
The norm denotes the Euclidean distance.
Fig. 4.5 shows an example of FPZ, radar and UAV path in a mission planner setting. The
Oahu island height map in Fig. 4.1a is used. The height equal or below zero is marked with blue
color. The color changes from green to yellow and eventually red as elevation increases. A
simple mission is designed in which a UAV is approaching the Pearl Harbor from the north. Its
path is actually characterized by many waypoints. Between the UAV and the Pearl Harbor are
enemy radar zone on the left (black transparent), and a FPZ on the right (red transparent).
Figure 4.5. An illustrative example of mission planner over Oahu Island, Hawaii
40
4.2 The Design of Genetic Algorithm
As described in Chapter 2, the genetic algorithm must be designed to suit the particular
problem considered. This section illustrates the design of the genetic algorithm for the problem
of path planning in realistic environment.
4.2.1 The encoding of solution
The solution encoding is done by representing the solutions with gene-like structures. As
explained in Chapter 3, a gene is a sequence of chromosomes which describes the features of the
solutions. For this problem, each gene of a given chromosome is the 3D coordinate of a waypoint
on the path. A chromosome is an array of such 3D points, with its first gene corresponding to the
starting point (sPt) and the last gene the destination point (ePt) of the UAV. For example, Fig.
4.6 shows the chromosome representing the path in Fig. 4.4. For illustration purpose, 2D
coordinate is used. In the mission planner, instead, 3D coordinate is applied.
Gene 0 (sPt)
Gene 1
Gene 2
Gene 3 (ePt)
(50.0, 50.0)
(54.7, 42.2)
(63.8, 38.2)
(72.0, 36.0)
Figure 4.6. A Chromosome with two intermediate waypoints
The problem of finding an optimal path is thus changed to finding the intermediate
waypoints to connect (sPt) and (ePt) and interpolate them.
4.2.2 Initialization
For the first generation of solutions, every individual is randomly generated with the
naissance operator. For later generations, there is a fixed portion of newly generated immigrants
which help retain the diversity of the population. The mechanism for randomly generating a
single individual is explained in the following section.
41
The UAV flight constraints, (sPt) and (ePt) are given before the iterative method. If the
starting direction is not given, a random starting direction (sDir) within the flight constraint will
be used. The distance between two consecutive waypoints (d) is defined by the user. All
positional and directional variables in this method are in 3D. The iterative method is described as
follows.
(1) The variable „current waypoint‟ (currtPt) is set to be the point in the direction of (sDir)
and a distance (d) away from (sPt). The variable „previous waypoint‟ (prevPt) is set to be
the starting point.
(2) The variable „previous direction‟ (prevDir) is defined as the direction from (prevPt) to
(currtPt). The directional limits of the next direction (nextDir) can then be worked out
according to flight constraints MTR and (prevDir)
(3) A (nextDir) is randomly generated direction within the limit worked out in Step (2).
(4) The (prevPt) is updated to the old (currtPt). Subsequently, the new (currtPt) is updated
as a point in the direction of (nextDir) from the old (currtPt) and a distance (d) from the
old (currtPt).
(5) Steps (2) – (4) are repeated until the (ePt) can be visited in the next iteration.
The direction generation random process in Step (3) must follow a probability
distribution that is more likely to produce directions in line with (ePt) and less likely to produce
directions away from the direction of (ePt). If this is not the case, the random individual will be
extremely hard to find. Because in that case, no force is pushing the UAV towards (ePt) and the
path will keep wandering about without getting close to (ePt) purposely.
42
4.2.3 The fitness function
In all the references that apply GAs, the path is divided into small sections using
numerical methods and the total fitness is an aggregation of each section. In this study, the
fitness of a chromosome is an aggregate description of the fitness of each of its genes. For a
chromosome of N genes, let f i be the fitness value of gene-i. The fitness value for the whole
chromosome, denoted as F, is calculated as:
F
10000
N
( f i ) d ( N 1)
(4.2)
i 1
Further, a large value of F is defined to indicate better solution. Hence, large value of f i
penalizes gene-i and small value is desirable. The term d N penalizes path length. For a single
gene, there are two ways to calculate f i , static and dynamic. For either method, f i is an
aggregation of the fitness values in Tables 4.2 or 4.3. That is, for a point i that is both under
terrain and Inside FPZ, f i is the sum of the fitness value for being under terrain and for being
inside FPZ.
In most literatures on genetic algorithms, violation of constraints is treated in a constant
way [28, 29, 31, 35]. This is called “static fitness value assignment” in this study.
Static fitness value assignment
Static fitness values are constant fitness value assignments that define a single fitness
value for a particular region in the space. There are in total 5 penalty fitness multipliers defined
for this study.
43
f OM - Penalty value for sections outside the map boundary
fUT - Penalty value for sections under the terrain
f FPZ - Penalty value for sections inside the FPZ
f radar - Penalty value for sections inside radar
f AT - Penalty value for sections not violating any constraints, which is very small.
Table 4.2 shows the static fitness value assignment particularly for this study.
Table 4.2. Static fitness value assignment
Situation
Outside Map
Under
Terrain
Inside FPZ
Inside
Radar
Above
Terrain
Fitness Value
d f OM
d fUT
d f FPZ
d f radar
d f AT
Take the path shown in Fig. 4.4a as an example. Suppose that.
(1) waypoint 1 is inside a FPZ but does not violate any other constraint, and
(2) all other waypoints, sPt and ePt do not violate any constraint, and
(3) f FPZ 5, d 10, f AT 1
Then, the path has a total fitness value of:
F
1000
N
( f i ) d ( N 1)
(10 f AT 10 f FPZ
1000
9.10
10 f AT 10 f AT ) 10 (4 1)
i 1
(4.3)
The path has a total fitness value of 9.10. Further, suppose that the path in Fig. 4.4 does not
violate any constraint, its fitness value is 14.29 according to Eq. (4.2).
44
F
1000
14.29
(10 1) 4 10 (4 1)
(4.4)
It is clear that the path is penalized if it has one waypoint inside the FPZ.
Dynamic fitness value assignment
While most literatures adopt the static fitness assignment, this study proposes a dynamic
way of fitness value assignment. Before introducing the dynamic way of assigning fitness values,
the following variables are defined for a given waypoint.
(1) Dvt – Vertical distance between the waypoint and terrain: positive value when the point is
above the terrain, negative otherwise.
(2) Dht – Horizontal distance between the waypoint and closest terrain boundary, it equals 0
if the waypoint is within the terrain boundary, non-zero otherwise.
(3) Df(i) – Distance between the waypoint and centre of the ith FPZ, zero if the waypoint is
not in the ith FPZ.
(4) Ef(i) – Extent of the ith FPZ. It is the largest value of x, y and z extents of the ith FPZ.
(5) Dr(i) - The distance from the centre of the ith radar. It equals zero if the waypoint is not
within the influence area of radar i
(6) N FPZ - The total number of FPZs
(7) N radar - The total number of radars
Table 4.3 shows the dynamic fitness value assignment of the solution individuals. As compared
to the static fitness assignment, the dynamic method takes into account the relative distance of
45
violation between a waypoint and constraints. When there is a violation of constraint, not only is
the particular gene penalized for having violation, but also penalized according to how “serious”
such violation is. In other words, a waypoint deeply inside a FPZ is going to be penalized more
seriously than a waypoint inside the same FPZ but is closer to the boundary.
Table 4.3. Dynamic fitness value assignment
Situation
Outside the Map
Under Terrain
Fitness Value
d f OM Dht
d fUT abs(Dvt)
Inside Radar
Above Terrain without
collision
Situation
N radar
Fitness Value
(d f
i 1
radar
Dr(i ))
Inside FPZ
N FPZ
(d f
i 1
FPZ
( Ef (i) Df (i)))
d f AT abs(Dvt)
For illustration, a similar example to that in the static fitness value assignment is given
here. Take the path in Fig. 4.4 as an example, suppose that.
(1) waypoint 1 is inside a FPZ. As an example, the maximum extent of the FPZ is set to be
20 ( Ef (1) 20 ), waypoint 1 is x unit of distance from the FPZ centre ( Df (1) x ).
Waypoint 1 does not violate any other constraint, and
(2) all other waypoints, sPt and ePt do not violate any constraint, and
(3) all the waypoints are 10 units above the terrain ( Dvt 10 ), and
(4) f FPZ 5, d 10, f AT 1
For the case that x 5 :
46
F
(10 f AT Dvt 10 f FPZ
1000
0.93 (4.5)
( Ef (1) Df (1)) 10 f AT Dvt 10 f AT Dvt) 10 (4 1)
For the case that x 10 (closer to the FPZ boundary):
F
(10 f AT Dvt 10 f FPZ
1000
1.20 (4.6)
( Ef (1) Df (1)) 10 f AT Dvt 10 f AT Dvt) 10 (4 1)
A simple comparison between equations (4.5) and (4.6) shows that, for a path that has a
waypoint inside the FPZ, the further away it is from the centre of the FPZ (i.e. closer to the
boundary), the bigger the fitness value of the whole path. Similar conclusions can be drawn for
points under terrain and inside radar zone.
From equations (4.5) and (4.6), it can be noted that the fitness value is inversely affected
by Dvt , the distance from terrain of each point. This models the fact that lower flying altitude of
the UAV is more desirable. This is because the terrain masking effect facilitates radar evasion,
hence maintaining the mission secrecy.
4.2.4 The selection mechanism
The selection mechanism follows that of roulette-wheel selection described in Section 3.5.
The preference of roulette-wheel selection over tournament is purely based on computational
efficiency concerns.
4.2.5 The genetic operators
There are in total 5 operators designed particularly to suit the UAV path planning
problem. Their usefulness will be fully studied by experimental study. However, their
mechanisms are explained in detail here.
Crossover
47
A child inherits the characteristics of both his parents. The two offspring produced by
crossing-over two individuals have common parts from both sides. Fig. 4.7 illustrates the crossover operator. C1 and C2 are the parent chromosomes, with X, Y and a, b the respective crossover points randomly chosen. ci and cj are the offspring after cross-over. The cross-over retains
and swaps the head and tail genes of C1 and C2. The middle gap in blue is generated using the
same mechanism as naissance for a single gene. The two offspring contain characteristics of both
their parents.
Figure 4.7. The cross-over operator
Local mutation
Fig. 4.8 shows the mechanism of a local mutation operator. A mutation point X is
randomly chosen on a given chromosome. The mutation operator will retain its structure for
genes before X. For genes after X, a new series of random genes (blue) are generated. This
operator is expected to gradually lead the solutions out of local minimums.
Figure 4.8. The local mutation operator
Strong mutation
Unlike local mutations, strong mutation has the capability to escape a local minimum in
just one generation. As illustrated in Fig. 4.9, two random mutation points X and Y are chosen. A
third point Z within the terrain boundary is randomly generated and the section between X and Z
48
is changed to random genes such that the middle section (blue) pass the intermediate point Z.
This operator is extremely useful in getting around difficult obstacles.
Figure 4.9. The strong mutation operator
Deletion/smoothing
The deletion operator has the purpose of smoothing a path that has undesirable turnings
of irregularities. As illustrated by Fig. 4.10, it is a simple deletion of a gene Z.
Figure 4.10. The deletion operator
For each chromosome in a given generation, there is a certain probability that such
chromosome is going to undergo one of the above 5 operations in the evolution process. The five
probabilities are defined respectively in Table 4.4.
Table 4.4. Symbols for probability of each GA operator
Symbol
Probability represented
PIM
immigration
PXO
cross-over
PLM
local mutation
PSM
strong mutation
PDel
deletion/smoothing
49
4.3 Simulation Experiments
A large number of simulation experiments have been conducted with different variables.
They can be classified into the following three categories.
(1) Experiments on different fitness functions
(2) Experiments on different GA operators
(3) Experiments on different UAV and UAV path characters
For each category, tests are undertaken under several different terrain scenarios.
4.3.1 Experiments on different fitness functions
This section examines the pros and cons of the static and dynamic fitness value
assignment through simulation experiments. For these experiments, the variables shown in Table
4.5 are controlled to be constants, while variables concerning the fitness functions are varied.
Table 4.5. Controlled constants for experiments on fitness function
sPt
ePt
sdir
edir
MaxTurnRad
MaxClimbAngle
MinGlidingAngle
(50, 50)
(400,400)
(0,0,-1)
(1,0,1)
π/4
π/4
- π/4
mindistance
d
PIM
PXO
PLM
PSM
PDel
3
10
0.05
0.05
0.05
0.05
0.05
Two maps are used for the testing: a flat map for the radars/FPZ – evading testing and a
rough terrain map for terrain evading testing. In both cases, the (sPt) and (ePt) are both 10 units
above terrain.
Tests with Static fitness functions
Tests 1&2 have the same magnitude of penalty fitness multiplier throughout. The fitness
value for above terrain is 1. The values in Table 4.6 are adopted, in which GenSize defines the
50
size of each generation. No terrain is used in Figure 4.11a, b and c. An easy, relatively flat but
rough terrain is used in Figure 4.11d.
Table 4.6. Homogeneous assignment of fitness multipliers
f OM
fUT
f FPZ
f radar
f AT
GenSize
10
10
10
10
1
1000
(a)
(b)
(c)
(d)
Figure 4.11. Test 1: simple obstacle evasion test for static fitness
51
The GA with the static fitness value assignment managed to find a valid path in all the
four cases shown in Fig. 4.11. In Fig. 4.11a, a simple valid path is found in a case without
obstacles. Figs. 4.11b and 4.11c show the path avoiding a FPZ and enemy radar, respectively.
Fig. 4.11d shows the path not colliding with a rough terrain. However, observing that both the
fitness value for the best and the generation average did not evolve much in 100 iterations (the
flat profile in the charts), it is reasonable to assume that the success of Test 1 is due to the lack of
difficulty in the problem. Moreover, the flat profile of fitness values makes it hard to define a
convergence criterion to make the algorithm to a halt, when manual stopping is not desirable.
Test 2 involves difficult terrain patterns and concave FPZ formation. Fig. 4.12 illustrates
that the static fitness assignment is not robust for concavity in obstacles. In Fig. 4.12a, the
bottom-right corner is a V-shape concave FPZ formed by two connected rectangular FPZs,
behind which the destination is hidden. A flat terrain is used so that the only obstacles come from
the FPZs. Mission planner finds a path through the concave FPZs rather than around them after
more than 100 iterations. In Fig. 4.12b, the destination is at the bottom, which can only be
reached after flying over the mountain. The planner finds a path through the mountain, rather
than over it. Hence, the planner has failed in both cases. It means that the static fitness is not
robust for either vertical (such as FPZs) or horizontal (high mountains) concavity.
(a)
(b)
Figure 4.12. Test 2: concave obstacle evasion test for static fitness
52
Such testing results can be interpreted by the fact of static fitness value assignment.
Suppose the area encircled by the red lines in Fig. 4.13 is the FPZ and a valid solution goes
around it. Intuitively, the solution in Fig. 4.13b is better than that of Fig. 4.13a due to its vicinity
to the valid solution. However, both solutions have the same static fitness value because they are
both inside the FPZ. Hence, the solution on the right was not preferred during the selection
process of the GA. Therefore, in order to escape concavities, the two solutions in Fig. 4.13 must
be differentiated by means of the dynamic fitness functions.
(a)
(b)
Figure 4.13. Different solutions with same static fitness value
Tests with Dynamic fitness functions
The dynamic fitness function assignment is defined by Table 4.3. Test 3 is conducted
with all the fitness multiplier having the same magnitude as in Table 4.6. Test 3 is conducted in
the same terrain and FPZ setting as in Test 2. The result of Test 3 is shown in Fig. 4.14. As
expected, the dynamic fitness values have led the UAV path around the concave objects.
Moreover, the fitness value plot for both the best and average has an increasing tendency as a
larger number of iterations occurred and has a tendency to converge after 100 and 200 iterations,
respectively.
53
(a)
(b)
Figure 4.14. Test 3: concave obstacle evasion test for dynamic fitness
A remaining shortcoming of the dynamic fitness function assignment as in Table 4.3 is
that the penalty value for any constraint violation is only 10 times greater than the normal value.
This does not give enough positive incentives for the paths that really escaped the obstacles.
According to the relative importance of the constraints, Table 4.7 shows the fitness multiplier
that is adopted for all the experiments in the following section.
Table 4.7. Final assignment of fitness multipliers
f OM
fUT
f FPZ
f radar
f AT
GenSize
100000
1000
100
100
1
1000
The reason for the adoption of Table 4.7 is quite straight forward. It is most serious for a
path to go out of the map boundary, because there is unpredictability out of the scope of path
planner. It is also serious for a path to go under terrain which results in collision. It is less serious
for a path to go into a FPZ or radar detection zone, which puts the aircraft in potential danger but
not necessarily leads to destruction. In some cases, the UAV may have to go through a radar area
so as to avoid terrain collision.
54
4.3.2 Experiments on functions of different GA operators
In order to study the power of each GA operator, the experiments started with the random
generator process of naissance and added more GA operators one by one. Table 4.8 shows the
control variables.
Table 4.8. Controlled variables for experiments on GA operators
(i)
sPt
ePt
sdir
edir
MaxTurnRad
MaxClimbAngle
MinGlidingAngle
(50, 50)
(400,400)
(0,0,-1)
(1,0,1)
π/4
π/4
- π/4
mindistance
d
f OM
fUT
f FPZ
f radar
f AT
3
10
100000
1000
100
100
1
Random generator – immigration/naissance
Immigration is a random process that links the starting point and then end point of the
path planner through finding the intermediate waypoints and interpolating them with piecewise
cubic Bezier curves. In this case, the probabilities are set as in Table 4.9. The test result is given
in Fig. 4.15 in which Fig. 4.15b shows that the situation is a simple FPZ block (red) that goes
between the starting (black) and ending position (grey).
Table 4.9. GA operator probabilities for immigration testing
PIM
PXO
PLM
PSM
PDel
1
0
0
0
0
55
(a)
(b)
Figure 4.15. Test 4: fitness plot for random generation
According to Fig. 4.15a, the average fitness value for all 175 random generations remains
almost the same. This shows that the random generator is stable for each generation by having
consistent average fitness values. For the best fitness, from time to time, it has some lucky shot,
but such lucky ones are not retained overtime. Hence, the following conclusions can be drawn
for the immigration/naissance operator.
(1) From a collective perspective, the immigration operator is consistent in that each random
generation has the same average fitness function.
(2) Yet, the high volatility in the fitness of the best solution shows that the random process
generates a population with diversity.
(3) The non-persistence of the peaks in Fig. 4.15a shows that without the selection
mechanism, there is no way to keep the good solutions.
(ii)
Cross-over
The probabilities in Table 4.10 are adopted for the testing of the cross-over operator. The
first generation relies on the random process studied previously. Each individual in all the
following generations have 10% probability to go through a cross-over with another individual in
56
the same population. The cross over points on either individual is a uniformly distributed random
variable. The situations to be tested are shown in Figs. 4.16b and 4.16d, respectively.
Table 4.10. GA operator probabilities for cross-over testing
PIM
PXO
PLM
PSM
PDel
0
0.1
0
0
0
(a)
(b)
(c)
(d)
Figure 4.16. Test 5: fitness plot for cross-over operator
Looking as the testing results shown in Figs. 4.16a and 4.16c, respectively, the following
observations can be concluded for the power of cross-over operator in its own right.
(1) It inherits the merits of ancestors and leads the population towards a greater aggregate
fitness when the situation is not concave. In fact, in the situation shown in Fig. 4.16b, the
planner did indeed find a way around the FPZ.
57
(2) However, for the concave situations shown in Fig. 4.16d, neither the generation fitness
nor the best fitness has any improvement over time.
(iii)
Cross-over and local/strong mutation
Figs. 4.16c and 4.16d do not show any satisfactory solutions for concave cases. This is
why the mutation operator is introduced. The local and strong mutation operators are studied in a
comparative way. For the tests on local mutation, the probabilities in Table 4.11 are used.
Similarly, for the local and strong mutation, the probability values shown in Table 4.12 are used.
Table 4.11. GA operator probabilities for local mutation testing
PIM
PXO
PLM
PSM
PDel
0
0.05
0.05
0
0
Table 4.12. GA operator probabilities for local and strong mutation testing
PIM
PXO
PLM
PSM
PDel
0
0.05
0.05
0.05
0
The comparative test result is shown in Fig. 4.17 and the following characteristics can be
observed.
(1) Both the local and strong mutation had been useful in making the path planner escaping
the local minimum created by the concave FPZ. Hence, the test shows the usefulness of
the mutation operator as a good tool in escaping the local minimum of the optimization
objective. This is in line with the characteristics of such operator described in literature.
(2) The strong mutation is much more effective in making “big jumps” as compared to the
local mutation. As shown in Fig. 4.17a for the local mutation and 4.17c for both local and
strong mutation, the fitness values reach the regime of magnitude of 0.01 between 30 and
58
40 iterations for local mutation operator. Whereas, the fitness values reach the same
regime in just 5 to 10 iterations. The test with strong mutation converges after 60
iterations. However, the test with local mutation does not exhibit an evident sign of
convergence even after 110 iterations.
(3) On the other hand, a steady increase in fitness values can be observed for the test with
only local mutation. For the test with both local and strong mutation, there are several
stages of different fitness increment velocities.
In summary, the mutation operator is useful for the GA due to its effectiveness in evading
local minimums.
(a)
(b)
(c)
(d)
Figure 4.17. Test 6: comparative study of local and strong mutation operators
59
(iv)
Deletion/smoothening operator
Fig. 4.18 shows the optimization result of a simple planner on a flat terrain without
obstacles with cross-over, local and strong mutation operators. The same result is shown from 3
different perspectives. An obvious drawback of such results is the irregularities and fluctuations
in the path. Such fluctuations are direct results of the diversity created by the random path
generator. However, in practical applications, such fluctuations are to be avoided for stability
issues in a real flight. Hence, the smoothing operator is applied. Test 8 is conducted using the
probabilities shown in Table 4.13 and the result is shown in Fig. 4.19.
Figure 4.18 – Test 7: simple test without deletion/smoothening
Table 4.13. GA operator probabilities for deletion/smoothening testing
PIM
PXO
PLM
PSM
PDel
0
0.05
0.05
0.05
0.05
Figure 4.19. Test 8: simple test with deletion/smoothing operator
60
A direct qualitative comparison between Figs. 4.18 and 4.19 shows the power of the
smoothening operator. The irregularities and fluctuations in the path are greatly reduced after the
application of the smoothening operator.
(v)
Argument for immigration in each generation
While it is hard to demonstrate the power of immigration operator through experiment,
the argument is relatively obvious. Without the immigration operator, only the first generation is
a random one. All the following generations are constituted of either the inherited individuals
from the previous generation or the offspring of crossover and mutation. Of course, when the GA
design is correct, the generations are heading towards better solutions. However, without
immigration into each generation that maintains the diversity, it is very hard to pull back the
generations heading towards a local minimum. For example, in the situation of Test 4 (see Fig.
4.15b), if the generations are heading towards the solution that goes around the FPZ from right,
the immigration operator will keep the door open to the solutions around the left of the FPZ.
4.3.3 Experiments on magnitude of probabilities for the GA operators
So far, the usefulness of each GA operator is demonstrated. It remains to determine the
magnitude of the GA probabilities. Experiments have therefore been conducted with the
mountainous setting of Test 3 (see Fig. 4.14b). Several combinations of GA probabilities were
tested and their best and average fitness were recorded at 50 iterations (see Table 4.14).
By comparing the average fitness across 5 cases, table 4.14 shows that choosing 0.05 or
0.1 as the probability for each GA operator results in the best optimization performance after 50
iterations. Greater value of GA probabilities only resulted in delaying the optimization progress.
With the choice of probability of 0.2, in average, 100% of the individuals of each generation go
61
through at least one of the five GA operators, hence the good individuals from the past
generation are no way to be retained. The final probabilities for the GA operators are going to
adopt the values in Table 4.15. This is because the algorithm performs equally well for uniform
probabilities of both 0.05 and 0.1. Yet, the choice of probability of 0.05 will leave in average 75%
of the individuals of each generation unchanged. This allows for a gradual convergence of the
optimization algorithm and avoids big jumps in the average fitness value of the generation.
Table 4.14 – Fitness values after 50 iterations with different GA probabilities
PIM
PXO
PLM
PSM
PDel
Average Fitness
Best Fitness
0.05
0.05
0.05
0.05
0.05
0.023
0.043
0.1
0.1
0.1
0.1
0.1
0.025
0.052
0.2
0.2
0.2
0.2
0.2
0.015
0.062
0.3
0.3
0.3
0.3
0.3
0.005
0.039
0.05
0.1
0.15
0.2
0.25
0.01
0.0325
Table 4.15 – Final adopted GA operator probabilities
PIM
PXO
PLM
PSM
PDel
0.05
0.05
0.05
0.05
0.05
4.3.4 Experiments with different UAV and UAV path characters
The variables MaxTurnRad, MaxClimbAngle, MinGlidingAngle, mindistance and d are to
be studied in this section. Although the determination of the first 4 should depend on the actual
aircraft modeled, it is worth studying how the changes in these variables affect the result.
Different UAV characters
Two tests with different UAV maneuverability have been conducted. The parameters for
Test 9 (see Table 4.16) are considered aggressive for an aircraft. And, the parameters for Test 10
(see Table 4.17) are relatively conservative for an aircraft. The quantitative and qualitative
62
results are shown in Figs. 4.20 and 4.21, respectively. In Figs b, c, d and e of Figs. 4.20 and 4.21,
the same terrain is viewed from the perspectives of top, left, bottom and right.
Table 4.16. Aggressive UAV maneuverability
MaxTurnRad MaxClimbAngle MinGlidingAngle mindistance
π/4
π/4
π/4
3
d
10
Table 4.17. Conservative UAV maneuverability
MaxTurnRad
MaxClimbAngle
MinGlidingAngle
mindistance
d
π/6
π/6
π/6
3
10
63
(a)
(b)
(c)
(d)
(e)
Figure 4.20. Test 9: aggressive UAV maneuverability
64
(a)
(b)
(c)
(d)
(e)
Figure 4.21. Test 10: conservative UAV maneuverability
Firstly, the aggressive test converged before 150 iterations, whereas the conservative test
converged after 200 iterations. Secondly, the fitness values for the aggressive test reach material
levels much earlier than in the conservative test. The reason for such quantitative results is that
the more stringent the flight constraints, the smaller the valid search space without constraint
65
violation (in this case, the constraints are terrain only). Since the GAs work in a way that
searches both the valid and non-valid spaces, the convergence is hence delayed for stricter flight
constraints.
Qualitatively, the conservative path had no choice but to go around the high-raising
mountain in the middle, yet the aggressive path is able to somehow climb over the mountain – an
evidence of larger search space for the aggressive case. In both cases, the path has a tendency to
stick close to the terrain, which is in accordance with the setting of the fitness function. In the
following section, the aggressive UAV maneuverability is applied and different values of the
distance between any two consecutive points are tested.
Different UAV path characters
The tests in Test 11 were conducted with the configurations shown in Table 4.18, and the
testing results in terms of fitness values are shown in Fig. 4.22. The following observations are
made.
(1) For all values of d, the convergence consistently occurs at 200 – 225 iterations. Hence,
the magnitude of d does not affect the convergence of the GA.
(2) With larger values of d, it took less time for the computation of each generation. This is
because the total number of waypoints required is smaller for larger d, hence taking up
less memory.
(3) Too large a d is not desirable. Since the sections between any two waypoints are not
checked against collision, the unlikely collision between two waypoints is therefore not
penalized. Large value of d will increase the chance of happening of such misjudgment.
This is especially the case when turning at sharp corners.
66
Table 4.18. Different distances between two consecutive waypoints
MaxTurnRad
MaxClimbAngle
MinGlidingAngle
mindistance
d
π/4
π/4
π/4
3
15, 20, 25, 30
(a)
(b)
(c)
(d)
Figure 4.22. Test 11: fitness evolution for different values of d
4.4 Discussions
The aim of the study on UAV path planning is not merely to find the best solution or the
most efficient path planning algorithm. Moreover, it is a thorough study of the single UAV path
planning problem that seeks to understand how different fitness functions, different GA
operators and different UAV flight constraints affect the path planning process. In the end,
according to this study, a high-quality combination of parameters and GA operators is proposed.
67
According to the tests conducted in this study, constant assignment of fitness values for
each gene is able to solve simple problems without any concavity. However, constant values
differentiate valid solutions from non-valid ones but do not differentiate among the invalid
solutions. As a result, it is hard to get around concave terrain and obstacle structures. On the
other hand, dynamic assignment of fitness values differentiates among the invalid solutions and
could tell those solutions that are “close” to the valid solutions, hence moving out of the
concavity gradually by means of the selection mechanism of the developed GA.
The random generator for each generation is a process that provides diversity. Cross-over
is useful in combining the good characteristics of different individuals in previous generation.
Local and strong mutations are important operators that help the solution evade local minimum.
Deletion is a smoothing operator on the path that reduces unnecessary fluctuations. All of the
five GA operators are indispensible for the path planner.
The computation speed of the path planner is reduced with more conservative UAV
maneuverability. Also, more generations are required before the convergence for conservative
UAV characteristics. The change in the distance between any two consecutive waypoints does
not affect the GA planner; however, a long distance may result in judgment errors especially at
sharp obstacle corners.
In summary, Chapter 4 starts by modeling the single UAV path planning problem and
designing a GA to solve it. Experiments have been conducted diligently for different fitness
functions, GA operators, and UAV maneuverability. As a result, a thorough understanding of the
problem is achieved. Chapter 4 will also provide a solid foundation for numerous future
extensions, which are to be presented in the following sections.
68
CHAPTER 5
A GENETIC ALGORITHM
FOR UAV COOPERATIVE TASK ASSIGNMENT
In addition to the single UAV path planning problem, another important aspect of the
UAV mission planning problem involves the coordination among multiple UAVs to achieve
certain objectives. In this chapter, this particular problem is clearly defined and modeled.
Effective solution algorithms have also been proposed and implemented. Simulation experiments
have been conducted to show the effectiveness of the developed methods.
5.1 The Problem Description
The UAV mission planning is going to take place in a 3D space represented by Cartesian
coordinates. Mathematically, a horizontal plane H h (c) is the set defined by H h (c) { x, y, z
ℝ3 | z c} , where c is a constant. The timeframe for the mission T is the subset of positive real
number. T ℝ+ {0} . Therefore, a specific time t during the mission is t T . A UAV
characteristics is a set {l , Rmin } , where l ℝ+ denotes the UAV cruising altitude and Rmin
ℝ+ denotes the UAV minimum turning radius. The UAV velocity is not defined here, since all
the UAVs are assumed to have the same velocity. With such definitions, a UAV Path Pl for a
UAV flying at altitude l is a subset of H h (l ) : Pl H h (l ) . The following properties exist for a
valid UAV Path:
Property 1: There exists a mapping Pl (t ) : T Pl such that:
a. Pl (t ) is two times differentiable with respect to t in both x and y directions. This is the
smoothness condition for the UAV paths.
69
b.
1
''
Pl (t )
Rmin . This is the minimum turning radius constraint on the UAV path. The
norm applied here is the Euclidian distance.
Property 2 (Dubin‟s Vehicle): Pl is constituted of segments of straight lines and arcs of Rmin .
The targets are dots on the plane H h (0) . Predefined tasks are assigned for each target. The
UAVs are to fly over these targets to complete the tasks.
5.2 Dubins Vehicle
As discussed in Chapter 1, the building brick of this class of cooperative mission
planning problem is Dubins Vehicle. It assigns a minimum-length path for given starting and
terminal positions u, v and starting and terminal directions U, V (see Fig. 5.1a). Although there
have been well-known algorithms for constructing Dubins path, such as MultiUAV or
MultiUAV2, it is still not convenient and efficient to use them in the Java environment [46]. In
this study, the algorithm for the calculation of Dubins‟ path is developed in the Java environment.
This problem can be formulated as proposed by Dubins [40].
Fig. 5.1b starts by
constructing two tangent circles for u and v respectively. The radius R of the circles equals to the
minimum turning radius of the UAV. The circles are directional. Each circle‟s direction is
defined in such a way that its direction at the starting (ending) point is collinear with the starting
(ending) direction. The purpose of the circle construction can be illustrated by Fig. 5.2, in which
a feasible minimum-length path can be found [40]. It is highlighted by a thick line.
70
(a) Specified end conditions
(b) The directional circles tangent to the end conditions
Figure 5.1. The minimum-length path problem
Figure 5.2. Illustration of a feasible path
According to [40], the minimum-length path is one of the 6 cases, shown in Figs. 5.3 and
5.4, respectively. Each case is constituted of 3 piecewise continuous arcs. Denoting “R” as a
right turn, “L” as a left turn and “S” as going straight, a path can be represented by 3 letters. For
example, RSL means that the UAV starts by turning right, followed by a straight line segment
and finishes by turning left. With such notation, the set of 6 feasible paths can be defined as
{RSL, RSR, LSL, LSR, RLR, LRL} . For example, Fig. 5.2 shows the case of RLR as a solution. It
can be seen that the starting and ending arcs must be a turn, whereas the arc in the middle can be
a straight line or a turn. Further, a feasible path has the following variables: (i) 1 : the angle
turned for the first arc, (ii) 2 : the angle turned for the second arc (in the case that the second arc
71
is not a straight line), (iii) t 2 : the length of the second part (in the case that it is a straight line),
and (iv) 3 : the angle turned for the third arc. The ranges of these 4 variables are given as:
1 [0,2 )
[ ,2 )
2
t 2 [0,)
3 [0,2 )
(5.1)
Fig. 5.2 illustrates a case of RLR and the 4 variables. In the case that the Euclidian
distance l between the starting and ending points satisfies l 4R , only the 4 cases
{RSL, RSR, LSL, LSR} , as shown in Fig. 5.3, are possible.
RSL
RSR
LSL
LSR
Figure 5.3. Dubins‟ path is one of the four possible choices when l 4R
When l 4R , two additional cases, i.e., RLR and LRL, are possible.
LRL
RLR
Figure 5.4. Two additional choices of Dubins path when l 4R
72
After all the possible paths are found analytically for given starting and ending conditions,
the minimum-length path between the starting and ending positions can be found by enumerating
all the possibilities.
5.3 Graph Representation
The weaknesses of the network flow model and the tree representation was discussed in
Chapter 1. This section first explains the computation complexity of the graph representation.
Following that, the graph implementation is discussed in detail. The example task shown in Fig.
5.5 will be used throughout this section for illustration purpose. In this example, there are 2
targets, each to be visited 3 times and 2 UAVs with specified starting positions and directions.
Figure 5.5. An illustrative example of a mission involving two UAVs and two targets
5.3.1 Discrete UAV heading angle on target
In order to implement the graph representation of the problem, the UAV headings on
each target are discretized. Fig. 5.6 illustrates the discrete headings of UAV. In this case, 360
degrees are divided into 8 portions, with each accounting for 45 degrees. The direction towards
the positive y axis is defined as 0 degree. The heading angle increases clockwise.
73
Figure 5.6. Discrete headings of UAV
The example shown in Fig. 5.5 is used to illustrate the 8 part discretization. UAV1 has 8
different ways to fly from its initial position with its initial heading to Target1, according to the
required heading angle when UAV1 is on Target1. Fig. 5.7 shows 2 out of the 8 cases with
minimum-length path. Figs. 5.7a and b show the 2 cases of 0°and 180°heading on Target1,
respectively. The UAV has a minimum turning radius of 0.5 units. It can be seen that different
heading angles on the target will lead to different UAV paths.
(a) 0°heading on target
(b) 180°heading on target
Figure 5.7. Two examples of minimum-length paths with different heading angles
5.3.2 The graph representation
One way to represent the entire solution space is through the use of graph. Define N t be
the number of targets, N d be the number of discrete heading angles of the UAVs and N u the
74
number of UAVs. Fig. 5.8a shows the scenario in which N t 2, N d 3, N u 2 . There are in
total 6 target nodes and 2 UAV nodes, representing the combination of possible UAV heading
angles over targets and the initial position of UAVs. Edges connecting the nodes are directional.
Each edge represents the cost of going from one node to another for a UAV. There are edges
connecting each pair of the target nodes in two directions. Finding the optimal solution is in fact
finding an optimal combination of edges that minimizes the sum of cost.
However, representing all the possible combinations by a single graph does not allow for
the consideration of heterogeneous UAVs. In a single graph as in Fig. 5.8a, the cost of going
from one target node to another is assumed to be the same for both UAVs. This is because the
cost calculation is based on a same edge. This is not realistic because the cost for UAVs with
different turning radius is usually different, even for identical trajectories. Therefore, in this
study, a particular graph is constructed for each UAV, allowing the consideration of
heterogeneous cost functions. The number of edges for the graph of UAV i, N Ei is,
N Ei N t N d ( N t N d ) 2
(5.2)
where the term N t N d represents the number of edges pointing from the UAVi’s starting position
towards each of the nodes, and the term ( N t N d ) 2 represents the bi-directional edges pointing
from each target node to every target node in the graph, including itself. In the particular
example of Fig. 5.8b, where N t 2, N d 3 , N Ei 42 .
Since the solution is a combination of edges, each edge is quantified to have a cost. This
is defined later when the fitness function of the genetic algorithm is discussed. The use of edges
as a solution representation is discussed in the next section.
75
(a)
(b)
Figure 5.8. The graph representations with 3 part discretization
5.4 The Solution Encoding
The new graph representation provides a foundation for the solution encoding. A solution
can be found by finding a combination of the edges connecting the nodes for each UAV. An
example is given in Fig. 5.9. Fig. 5.9a illustrates the data structure of a solution. Figs. 5.9b and
5.9c highlights the chosen edges of the graphs for UAV1 and UAV2. It represents the solution in
which UAV1 completes the first task on Target1 at 0°heading, followed by the second task on
Target2 at 120°heading and finishes the job by visiting Target1 again at 240°heading. On the
other hand, UAV2 starts by visiting Target2 at 0°heading, followed by Target1 at 120°, and
finishes by visiting Target2 at 240°heading. This solution maintains that each target is visited
exactly 3 times (Classify, Attack, and Verification).
76
(a)
(b)
(c)
Figure 5.9. A solution from the graph representation
5.5 The Initialization
The initialization process generates the first generation of solutions. Based on the first
generation, future generations can be produced by applying genetic operators. The first
generation must be as diversified as possible and meets all the constraints. This section describes
the mechanism of generating the first generation. Without loss of generality, the process of
randomly generating a single solution is discussed. The process for the whole generation is just a
repetition of the single solution generation process.
77
Again, let N t be the number of targets on which at least one task is to be performed. N d
be the number of discrete heading angles in which a UAV can approach a target, as described by
Fig. 5.6. Further, let N mi be the number of tasks to be carried out on target i . This models the fact
that some targets are classified and requires only to be destroyed (1 task). There are other targets
which need to be verified as hostile before an attack can be performed (2 tasks). Finally, let N u
the number of UAVs that carries out the mission. Therefore, the total number of visits N visit on all
targets by UAVs can be expressed as the sum of the required tasks on each target:
Nt
N visit N mi
(5.3)
i 1
Figure 5.10 shows the algorithm used to generate a random solution. At the beginning,
two variables V1 and V2 are initialized. V1 is the solution variable which will contain the chosen
nodes of the graph in sequence. V1 is empty at the beginning. V2 is the objective variable which
contains the unassigned tasks on each target. Initially, V2 contains all the required tasks on
targets. At each iteration, the algorithm randomly selects a remaining target from V2 and assigns
it randomly to a UAV at a random heading angle, hence updating V1. Thereafter such task is
removed from V2 for the next iteration. The iterative process continues to assign tasks in V2 to
UAVs and record the assignment in V1. The iteration stops until V2 is empty. Then, V1 is a
random solution. It contains the sequential target assignment for each UAV and their
approaching angle on each of their assigned target. Essentially, each UAV is to visit the nodes in
its graph sequentially. The suitability of the solution can be calculated by the sum of the edges
connecting those nodes.
78
Figure 5.10. Algorithm for the random solution generation
5.6 The Fitness Function
Section 5.2 describes how to generate the minimum-length path for arbitrary starting and
ending positions and directions (Dubins vehicle). Based on that, the minimum-length path
between any two nodes in the graph can be obtained easily. Therefore, every edge connecting a
pair of nodes is assigned a cost value that equals to the length of the Dubins‟ path connecting the
two nodes. Since a solution is a combination of edges, the fitness of a solution is a function F (x)
that maps the space of edge combination to a positive real value:
F ( x) : E R
(5.4)
Where E is the space of all possible combination of edges, and R is the set of positive real
numbers.
79
In this study, two kinds of fitness function assignments are used. The first is the inverse
of the simple sum of the edge weights. Let S be a solution and ei an edge belonging to S , the
first fitness function is defined by:
F1 ( S )
10000
e
ei S
(5.5)
i
Therefore, solutions with shorter total distance have greater fitness value as compared to
those with longer total distance.
The second kind of fitness function assignment essentially derives the fitness from the
mission completion time. Let S be a solution, the total distance traveled by UAV i, l i , can be
calculated. Further, let v i be the velocity of UAV i . The second kind of fitness function can be
expressed as:
F2 ( S )
10000
l
max( i )
vi
(5.6)
The mission completion time is defined by the maximum of the completion time of all
UAVs. In this way, the solution which has a smaller mission completion time has a bigger fitness
value as compared to those with longer completion time.
It will be demonstrated through experiment that F2 is a more suitable fitness function.
The application of F1 usually leads to a solution which assigns all the tasks to the UAV with
smallest turning radius. This is because it travels less distance turning around, hence saving the
total path length as compared to bigger turning radius UAVs. However, making only one UAV
work and leaving others idle is not optimal. Moreover, the mission completion time will be
80
enormous if all the tasks are carried out by only one UAV. In contrast, F2 results in more
reasonable solutions in that the targets are evenly distributed among UAVs. By minimizing the
maximum mission completion time, UAVs are forced to visit targets in a way that minimizes the
mission completion time.
5.7 The Genetic Algorithm Operators
The design of GA operators for this problem is different from that for the single UAV
path planner. The most important principle is that the quantity of tasks to be completed on each
target must be kept constant after the operators. The design of these operators is based on the
important factors that affect a solution. These factors are the assignment of targets to each UAV,
the visiting sequences of targets for each UAV, and the heading angle of UAV on each target.
(i)
Crossover
Unlike the crossover operator in the traditional sense which combines two parent
solutions, the crossover applied here is done between tasks of different UAVs in the same
solution. This arrangement maintains the total number of tasks completed by UAVs a constant.
In essence, the crossover operator changes partially the target assignment for two different UAVs
in the same solution. Fig. 5.11 shows the crossover operator on UAVs i and j in a same solution.
The crossover points are gene r for UAV i and gene s for UAV j.
81
Figure 5.11. Crossover operator on same solution across different UAVs
(ii)
Mutation
The mutation operator changes only the heading angle of a UAV on a target. The
example in Fig. 5.12 shows the mechanism in which gene r of UAV i, which represents heading
angle of on target k, is mutated.
Figure 5.12. Mutation operator changes heading angles
(iii)
Swap
The swap operator is similar to crossover. However, instead of changing all the genes
after the crossover point, the swap operator only swaps a single gene with that from a different
UAV. Fig. 5.13 shows the swap operator on UAVs i and j in a same solution. The genes swapped
are gene r for UAV i and gene s for UAV j. The swap is a more delicate operator as compared to
the crossover. While crossover mainly deals with the assignment of targets to UAVs, the swap
deals with both the assignment and visiting sequence of the targets.
82
Figure 5.13. An illustration of the swap operator
(iv)
Immigration
The immigration operator in this problem is similar to the usual immigration operator. It
simply replaces a solution with a new random solution.
(v)
Elitism
Elitism is the operator that retains the best solution of each generation. With elitism, the
fitness value for the best solution is always going upwards.
Besides, the algorithm is deemed to have converged if the best solution remains
unchanged for 50 iterations.
5.8 The Simulation Experiments
The parameters that may affect the result and algorithm convergence are listed below.
Targets – an array of 2D coordinates representing the targets to be visited.
Tasks – an array of integer whose length equals that of the variable Targets. It defines the
number of tasks to be carried out on each target.
UAV Initial Positions – an array of 2D coordinates that define the starting positions of all UAVs.
83
UAV Initial Directions – an array of two dimensional coordinates which define the starting
direction of all UAVs.
Discrete Heading – an integer that defines the number of discrete headings for the UAV.
Turning Radius – an array of float point numbers that defines the turning radius of each UAV.
Generation Size – The size of generation for each iteration.
XOProba, MutProba, SwapProba, ImmigProba – The four variables represent the probability for
crossover, mutation, swap and immigration respectively.
Convergence Iteration – The minimum number of iterations for which the best solution remains
unchanged such that the algorithm is deemed to have converged.
5.8.1 Diversity of the random solution generator
The success of GA depends partly on the capability to generate random solution of great
diversity. For every experiment, 1000 random solutions are generated for a given problem.
5.8.2 Experiments on different probabilities of genetic operators
The first set of simulation experiments were conducted to examine the usefulness of the
genetic operators. The parameters in Table 5.1 are kept constant throughout the experiments.
Table 5.1. Constant parameters for genetic operators‟ test
Discrete
Heading
Generation
Size
Convergence
Iteration
Minimum
Turning Radius
Fitness
Function
36
1000
50
10 meters
Minimum
Time
The first set of 4 experiments were used to test the capability of each of the EA operators
independently. The testing results are shown in Fig. 5.14, respectively, in which vector P is
defined by: P = (XOProba, MutProba, SwapProba, ImmigProba). A summary for performance
comparison is shown in Table 5.2. Except for immigration, all the other 3 operators have positive
84
contributions towards the improvement of the best solution. The crossover is able to produce
better solutions by combining the desirable characteristics of parent solutions. Yet, the
convergence rate of the experiment with crossover is almost 2 times of that with mutation and
swap. This is because the crossover operator is not able to produce enough diversity for the
population. This also explains why the final solution is worse than the ones with mutation and
swap. What is remarkable about mutation is the sharp increase of average fitness value. This
shows that the visiting angle is essential in determining the solution quality and that mutation is
capable of arriving at good solutions by targeting at the right heading angle on targets. For the
swap operator, both the convergence plot and the result are better than that of mutation and
crossover. This shows that in this problem, the visiting sequence is another important factor in
19
20
18
19
17
18
16
15
Best
14
Average
Fitness Value
Fitness Value
determining good solutions.
17
16
15
Best
14
Average
13
13
12
12
11
11
1
51
101
1
151
51
101
151
201
251
301
Generation
Generation
(a) P = (0.1, 0, 0, 0)
(b) P = (0, 0.1, 0, 0)
18
23
17
21
17
Best
Average
15
Fitness Value
Fitness Value
16
19
15
Best
14
Average
13
13
12
11
11
1
51
101
151
201
251
Generation
(c) P = (0, 0, 0.1, 0)
1
51
Generation
(d) P = (0, 0, 0, 0.1)
Figure 5.14. Fitness evolution for different GA operators
85
Table 5.2. Solution quality of the different GA operators
P = (0.1, 0, 0, 0)
P = (0, 0.1, 0, 0)
P = (0, 0, 0.1, 0)
P = (0, 0, 0, 0.1)
Mission Completion
Time
545.85 sec
531.90 sec
462.26 sec
590.47 sec
Total UAV Path
Length
1626.77 meters
1561.86 meters
1380.22 meters
1674.03 meters
While the first 4 experiments proved the necessity in having all 4 GA operators, another 6
experiments were conducted to find a good combination of GA operators‟ probabilities P. The 6
cases are described by: P = (a, a, a, a); a {0.05,0.1,0.15,0.2,0.25,0.3} . For the convenience of
comparison, the fitness plots (the best fitness and average fitness) for the 6 cases are put on the
same graph in Figs. 5.15 and 5.16, respectively. It is obvious that the case of a = 0.15 out
performs others in terms of the best fitness. In terms of average fitness, lower values of a tends to
converge too fast without reaching high values. This is because the effect of GA operators is too
small to contribute to the total improvement of the whole generation. Higher values of a tend to
have small improvements. This is because in each generation, almost every individual goes
through some sort of GA operators. Therefore, the good solutions are hardly retained.
0.05
0.1
101
151
0.15
0.2
0.25
0.3
23
22
Fitness
21
20
19
18
17
16
1
51
201
251
301
351
401
451
Generation
Figure 5.15. The best fitness evolution for different GA operator probabilities
86
0.05
0.1
101
151
0.15
0.2
0.25
0.3
17
16
Fitness
15
14
13
12
11
1
51
201
251
301
351
401
451
Generation
Figure 5.16. The average fitness evolution of the different GA probabilities
5.8.3 Experiments with different fitness functions
Simulation experiments were also conducted with the two fitness functions. The variables
in Table 5.3 were kept constant throughout these experiments. Fig. 5.17 shows the solution to the
problem with 9 targets and 3 tasks on each target. The UAVs in this problem are homogenous.
Their turning radius is in the same magnitude as compared to the distance among the targets
(represented by the black squares). There are 3 UAVs to complete the missions and their
trajectories are marked with cyan, green and blue colors. The mission completion times for the 2
cases are 444.84 sec and 827.00 sec , respectively. The total distances traveled are 1327.13
meters and 1079.14 meters, respectively. Fig. 5.17a shows that the tasks are well distributed to
the three UAVs. Fig. 5.17b on the other hand, shows that the UAV in blue colors has taken the
majority of the tasks whereas the other two UAVs completed much fewer tasks. It produces
smaller total path length but requires a lot more time to complete. Figure 5.17 demonstrates that
the minimum mission time is a much stronger and more reasonable fitness function as compared
to the minimum distance traveled.
87
Table 5.3. Constant parameters for different fitness function test
XOProba
MutProba
SwapProba
ImmigProba
0.15
0.15
0.15
0.15
Discrete Heading
Generation Size
Convergence Iteration
36
1000
50
Minimum Turning
Radius
10 meters
(a) Minimum Mission Time
(b) Minimum Total Length
Figure 5.17. Result comparison for different fitness functions
88
On the other hand, Fig. 5.18 shows the fitness value evolution for the two different cases
of fitness functions. However, the minimum length fitness produces a result that heavily assigns
the tasks to a single UAV rather than allocating them to different UAVs.
25
10
Minimum Mission Time Fitness
23
9
21
8.5
19
Best
17
Average
15
Fitness Value
Fitness Value
Minimum Total Length Fitness
9.5
8
7.5
Best
7
Average
6.5
6
13
5.5
11
5
1
51
101
151
201
251
301
351
401
451
1
51 101 151 201 251 301 351 401 451 501 551
Generation
Generation
(a) Minimum Mission Time
(b) Minimum Total Length
Figure 5.18. Fitness value evolutions for different fitness functions
5.8.4 Experiments with different UAV flight dynamics
The most prominent results come from the different UAV flight dynamics cases capable
to be solved by the multi-graph representation. This section first examines the homogenous
turning radius cases, following which the heterogeneous cases are studied. The variables in Table
5.4 are kept constant throughout this section.
Table 5.4. Constant parameters for UAV flight dynamics tests
XOProba
MutProba
SwapProba
ImmigProba
0.15
0.15
0.15
0.15
Discrete
Heading
Generation
Size
Convergence
Iteration
Fitness
Function
36
1000
50
Minimum
Time
The first experiment considers a problem for which the UAVs‟ turning radius is small
r 1 meter as compared to the distance among different clusters of targets. The objective to be
89
minimized is the mission completion time. Similar to the case in Fig. 5.17, there are 9 targets and
3 tasks on each target. Fig. 5.19 shows the result, which is intuitive in this case. Since the turning
radius is too small as compared to the distance among targets, each UAV will visit the three
targets closest to its starting position. Moreover, each UAV revisits a target twice and completes
all the three tasks in a row, before reaching for the next target. This is also because the distance
between two targets is longer than the circle turn at a radius of 1 meter. Fig. 5.20 further shows
the evolution of the best and average solution fitness values.
Figure 5.19. Simulation output for Turning Radius of 1
Turning Radius = 1
110
100
Fitness Value
90
80
70
Best
60
Average
50
40
30
20
1
51
101
151
201
251
301
351
401
Generation
Figure 5.20. Fitness evolution of the experiment with Turning Radius of 1 meter
The mission completion time is 89.4 sec and the total distance traveled by three UAVs is
262.06 meters. As a first experiment, it may not have much practical significance. However, this
experiment result does show that the algorithm is capable of generating reasonable solutions.
First, the turning radius of 1 is small as compared to the distance between the targets. Therefore,
90
it is reasonable to have optimal results in which each UAV completes tasks on the targets which
are at close vicinity of its starting positions. This is because traveling among different clusters
will be too time-consuming requires unnecessary extension of the path length. Second, taking a
close look at Fig. 5.19, it is not hard to observe that each UAV tends to re-visit a target until all
three missions on the same target are completed, before it flies towards the next target. This can
be seen by observing the two circles of radius 1 near each target. The circles show that the target
is revisited twice. This is also in line with intuition. Since the distance between any two targets
within the same cluster is larger than 2 at turning radius 1. It is better for a UAV to finish all
three tasks on a target before flying to the next, than to fly back and forth between targets.
Actually, when the turning radius tends to 0, the problem is changed to a multiple travelling
salesmen problem.
In the next experiment, the turning radius of the UAVs is increased to 3. All other
parameters remain the same. Figs. 5.21 and 5.22 show the result output and fitness evolution of
this experiment.
Figure 5.21. Simulation output for turning radius of 3
91
65
Turning Radius = 3
60
55
Fitness Value
50
45
40
Best
35
Average
30
25
20
15
1
51
101
151
201
251
301
351
401
451
Generation
Figure 5.22. Fitness evolution of the experiment with turning radius of 3 meters
The completion time in this case is 160.20 sec and the total distance travelled by UAVs is
478.26 meters. What is interesting is the similarity and distinction between the experiments for
radius of 1 and 3. It can be observed that in both cases, each UAV stays with completing
missions on the three targets at its close vicinity. However, in the case of radius of 3, both revisit and coming back and forth between multiple targets can be observed. This is because the
turning radius of 3 makes visiting among clusters costly, yet within each cluster of targets, each
UAV has comparable cost between revisiting and flying back and forth among targets.
The next experiment is on the turning radius of 7. The result, shown in Figs. 5.23 and
5.24, in this case is less intuitive. The mission completion time is 327.80 sec and the total
distance travelled by three UAVs is 981.19 meters. The UAVs are cooperating in completing the
tasks on different clusters of targets. UAVs are not constrained to only visit targets close to their
starting positions.
92
Figure 5.23. Simulation output for turning radius of 7 meters
Turning Radius = 7
32
Fitness Value
27
Best
22
Average
17
12
1
51
101
151
201
251
301
351
Generation
Figure 5.24. Fitness evolution of the experiment with turning radius of 7 meters
The turning radius is then increased to 10 and the result is shown in Figs. 5.25 and 5.26,
respectively. The completion time is 444.84 sec and the total distance traveled is 1327.13 meters.
A number of features can be observed from the result. First, the cases of re-visit are greatly
reduced. This is because making a turn and coming back to a target requires longer travel
distance as compared to visiting a target nearby. Second, as a result of reduced re-visits, all 3
UAVs visit targets from all 3 clusters. The UAVs are not constrained to their own clusters but fly
in a cooperative manner.
93
Figure 5.25. Simulation output for turning radius of 10 meters
Turning Radius = 10
24
22
Fitness Value
20
18
Best
16
Average
14
12
10
1
51
101
151
201
251
301
351
401
451
Generation
Figure 5.26. Fitness evolution of the experiment with turning radius of 10 meters
The next test deals with the case of heterogeneous UAV turning radius. However, their
speeds are still assumed to be 1 meter. The turning radii for the 3 UAVs are 1 meter, 5 meters
and 10 meters, respectively. The result is shown in Figs. 5.27 and 5.28, respectively. The
observations are summarized in Table 5.5.
94
Figure 5.27. Simulation output for heterogeneous turning radius
54
Heterogeneous Turning Radius
49
Fitness Value
44
39
34
Best
29
Average
24
19
14
1
51
101
151
201
251
301
351
401
451
501
551
Generation
Figure 5.28. Fitness evolution of the experiment with heterogeneous turning radius
Table 5.5. The observation summary
UAV
No. of re-visits
Total Tasks Completed
Left (Cyan, radius = 10 meters)
1
5
Middle (Green, radius = 5 meters)
1
8
Right (Blue, radius = 1 meters)
7
14
It can be seen that the UAV re-visit rate and number of total tasks completed have
increased with decreasing turning radius. This result demonstrates that in a realistic situation
95
with time constraints, agile UAVs are responsible for heavier work load as compared to their less
agile teammates. This is because less agile UAVs relatively waste more time and distance on
making turns.
5.8.5 Stability of the solution algorithm
The final set of experiments is done with the parameters shown in Table 5.6. There are 9
targets and 3 tasks to be completed on each target. There are 3 UAVs to complete the tasks. The
optimization objective is to minimize the mission completion time. The same experiment is
repeated for 10 times to see if similar result is obtained each time. The diversity of the starting
generation was demonstrated before. Therefore, through these experiments, it is to demonstrate
that the algorithm is capable of arriving at similar optimums each time it is run, despite starting
with different and diversified first generations. The parameters in Table 5.6 are adopted for all
the 10 experiments.
The results from the 10 simulation runs, in terms of the best fitness and average fitness
evolution, are shown in Figs. 5.29 and 5.30, respectively. It can be concluded that the algorithm
is quite consistent every time it is run. This also shows that the algorithm is able to find similar
optimal solutions with good stability, which is a very desirable characteristic of the solution
algorithm.
Table 5.6. Parameters used for repeated simulations testing stability
XOProba
MutProba
SwapProba
ImmigProba
Discrete Heading
Generation Size
0.15
0.15
0.15
0.15
36
1000
Convergence
Iteration
Fitness
Function
Turning
Radius
50
Minimum
Time
10 meters
96
1
2
3
4
5
6
7
8
301
351
401
9
10
25
24
23
22
Fitness
21
20
19
18
17
16
15
1
51
101
151
201
251
451
501
551
Generations
Figure 5.29. The evolution of the best fitness of 10 simulations
1
2
3
4
5
6
7
8
301
351
401
9
10
17
16
Fitness
15
14
13
12
11
1
51
101
151
201
251
451
501
551
Generations
Figure 5.30. The evolution of average fitness of 10 simulations
5.9 Summary
In this chapter, a multiple graph representation has been proposed to describe the multiple
UAV mission assignment problem that allows for the consideration of heterogeneous UAV flight
dynamics. A GA based solution algorithm has also been developed to find the optimal solution.
97
The robustness of the solution algorithm is demonstrated through simulations with different
turning radius. The stability test justifies the performance consistency of the solution algorithm.
98
CHAPTER 6
AN OCCLUSION-AWARE MODEL FOR UAV SURVEILLIANCE
IN COMPLEX URBAN ENVIRONMENT
This chapter addresses the problem of multi-UAV surveillance in complex urban
environments with occlusions. The objective is to allocate the UAVs such that the coverage and
the information from last visit about a designated area are maximized. A generic approach to the
problem is a two-stage optimization [44]. The first stage finds the set of vintage observation
points in the sky. Each point on the ground can be seen by at least one observation point. The
second stage is a multiple travelling salesmen problem. It allocates the UAVs to visit these
observation points. In this chapter, the problem is firstly defined, followed by the development of
a more generalized occlusion-aware model compared with the model in [44]. Besides, the flaws
of the two-stage method are also discussed. This model forms the basis to develop a truly
integrated optimization method to find the optimal solution to the UAV surveillance problem in a
realistically complex urban environment.
6.1 Problem Description
This study generalizes the problem description in [44]. In order to fully describe the
problem, the environment, UAV and the objective function must be defined.
6.1.1 The environment
The environment in which the UAV task is to be accomplished consists of a surface of
varying height and all the space above the surface. The surface includes the terrain surface and
rooftops. It is assumed that every point on the surface can be described by its x, y coordinates
99
and a function H ( x, y) giving its z coordinate. More formally, this set of information is
described as follows.
Area: A ℝ2
Height map: H : A ℝ+
Terrain: M { x, y, z ℝ3 | x, y A, z H ( x, y)}
Compared with the model in [44], the definition of terrain here does not restrain the
buildings to be on a same horizontal surface. Instead, the baseline can be of varying elevations,
thanks to the occlusion aware model developed in this study. Then, the space in which the UAVs
can operate (the air) is defined as everything above the surface as:
Air: A3D { x, y, z ℝ3 | x, y A, z H ( x, y)}
6.1.2 The sensor capability of UAVs
Particular to this problem is the model for the sensors on UAVs. The sensors are modeled
to look vertically downwards (see Fig. 6.1), and have a cone-shape coverage space below the
UAV. The surface that lies within the cone-shape and not blocked by any buildings can be seen.
The occlusion aware model to be discussed in section 6.2 is essential for the analysis of such
blocked situations.
6.1.3 The objective function
The mission to be completed by a set of UAVs is to maintain surveillance over an area.
Therefore, it is desirable to have as long an exposure time for each point on the ground as
possible. Let T (0,] be the total mission time, for a point ( x, y) on the ground, define
100
Expo ( x, y, t ) 1 if point ( x, y) is exposed in the sensor of at least one UAV at time t . Otherwise,
Expo ( x, y, t ) 0 . In light of this, the objective function to be maximized can be written as [44]:
O
T M
Expo ( x, y, t )dxdydt
(6.1)
6.2 The Two-Stage Optimization
In [44], the maximization of Eq. (6.1) is achieved over two stages. The objective of the
first stage is to find a set of observation points in the sky such that each point on the entire
surface is visible by at least one observer in the sky. The minimum set of such observation points
is called the vantage set, and are to be visited by a group of UAVs. In the second stage, the
objective is to maximize the aggregate exposure of the UAVs and ensure that they visit all the
points. This study develops an occlusion-aware model, which is the basis for finding the vantage
set of observation points.
6.3 The Occlusion Aware Model
In [44], the surface consists of only a horizontal base plane defined by z 0 and
quadrangular prisms lying of the base plane. Therefore, it does not consider the variation in
terrain elevation and the different shapes of the buildings. This leads to easy computation of
visibility using analytical methods (see Fig. 6.1). However, not every city is built upon a flat
surface. Moreover, real world architectures can be of all kinds of shapes. To overcome this
oversimplification limit, an occlusion-aware model has been developed in this study that can be
potentially applied to a situation in which complex terrain environment and building shapes are
taken into consideration.
101
Figure 6.1. Visibility test with analytical method [44]
Thanks to JME as a robust 3D rendering and computing package, visibility detection can
be achieved through collision checking between a ray (connecting the observation point and the
point to be observed) and a 3D object (the building). Each building is defined as an enclosed 3D
space. As an illustrative example shown in Fig. 6.2, A is a point on the ground whose visibility is
to be checked. A ray shoots from the observation point in the sky to A. Fig. 6.2a shows that the
ray is intercepted by a 3D object half-way. In this case, A cannot be observed. Fig. 6.2b shows A
on a flat terrain, which can be observed from the observation point in sky. However, in a rugged
terrain situation, the line of sight from the sky may be blocked by the terrain. Therefore, the
occlusion detection model to be developed will check the ray collision with both the terrain and
the buildings.
6.4 System Implementation of the Occlusion Aware Model
102
(a)
(b)
Figure 6.2. Illustration of ray detection
In the occlusion-aware model, the surface is reasonably treated as a discrete set of points.
In such setting, given an observation point in the sky and the sensor angle, the observable area on
the surface can be easily marked. Fig. 6.3 illustrates the implementation of the occlusion
awareness model using ray detection. Whenever the ray detection finds an obstacle between the
observation point and a point of the surface, such point on the surface is marked as being
obstructed. Otherwise, the point of surface is visible.
Figure 6.3. The occlusion-aware model
103
Fig. 6.3 is a screenshot from a 3D occlusion aware model simulation. The ground and
rooftops are represented by discrete blue points. The buildings are represented by grey 3D
objects. The observation point in the sky is represented by the green square. Its altitude is 225
meters, whereas the average height of the buildings in this case is 100 meters. The apex angle of
the sensor is 45 degrees, looking vertically downwards. The camera in this case is looking
downward with a 10 degree angle from the vertical axis. The actual observable points after
applying the occlusion aware model by means of the ray casting checking algorithm are colored
red. All other unobservable points remain blue in color. It can be seen that without the two
buildings whose contours are outlined, all points inside the orange circle can be observed.
However, due to the buildings, only a small part of the ground and rooftops can be observed.
Fig. 6.3 also indicates the validity of the occlusion-aware model of this study. Rather than
using an analytical method, the function describing the surface does not need to be known
beforehand. Only the discrete points describing the surface need to be known. This greatly
generalizes the occlusion detection method. It is also capable for occlusion detection on a rough
terrain, whose surface is difficult to be expressed in an analytical way. However, there are two
major drawback of this method. The discretized surface leads to inaccuracy of the occlusion
detection, in particular for corners or edges. On the other hand, rendering the points requires a
large amount of computational resources. This might make the method impractical for largescale applications.
6.5 Limitations of the Two-Stage Method
The two stages are in fact inter-related. Therefore, the shortcoming of such two-stage
optimization is that two different objective functions are used in sequence, instead of a single,
104
integrated one. This may easily lead to local optimum. Nevertheless, the two-stage method
greatly reduces the solution space and hence enhances the efficiency of the algorithm.
6.6 Discussions
In this chapter, an occlusion-aware model has been developed to generalize the problem
definition in [44] for realistic visibility detection. This is done through applying the built-in
functions of JME. Regardless of the optimization method applied, the occlusion-aware model is
an essential part to solve the UAV surveillance problem. Therefore, this study provides a solid
foundation for possible future development in the surveillance by UAVs in urban environment.
105
CHAPTER 7
CONCLUSIONS AND FUTURE WORK
7.1 Conclusions on the Study of the UAV Path Planning Problem
In this study, the single UAV path planning problem is modeled in a realistic mode in
terms of terrain and UAV capability. A new GA based solution algorithm has been developed
and tested. The model and the solution algorithm form a new path planner. The aim of this part
of the study is not merely to find the best solution or the most efficient path planning algorithm.
Moreover, it is a thorough study of the single UAV mission planning problem that seeks to
understand how different fitness functions, different GA operators and different UAV flight
constraints affect the path planning process. In the end, according to this study, a sound
combination of parameters and GA operators has been proposed.
According to the tests conducted in this study, constant assignment of fitness values for
each gene is able to solve simple problems without any concavity. However, constant values
differentiate valid solutions from non-valid ones but do not differentiate among the invalid
solutions. As a result, it is hard to negotiate around concave terrain and obstacle structures. On
the other hand, dynamic assignment of fitness values differentiates among the invalid solutions
and could tell those solutions that are “close” to the valid solutions, hence moving out of the
concavity gradually by means of the selection mechanism of the GA.
The random generator for each generation is a process that provides diversity. Cross-over
is useful in combining the good characteristics of different individuals in the previous generation.
Local and strong mutations are important operators that help the solution evade local minimums.
106
Deletion is a smoothing operator on the path that reduces unnecessary fluctuations. All of the
five GA operators are indispensible for the path planner.
The calculation speed of the path planner is reduced with more conservative UAV
maneuverability. Also, more generations are required before the convergence for conservative
UAV characteristics. The change in the distance between any two consecutive waypoints does
not affect the GA planner; however, the long distance may result in judgment errors especially at
sharp obstacle corners.
In summary, this study started by modeling the single UAV path planning problem and
designing a GA based algorithm to solve it. Experiments have been conducted diligently for
different fitness functions, GA operators and UAV maneuverability. As a result, a thorough
understanding of the problem is achieved.
7.2 Conclusions on the Study of the Multiple UAV Task Assignment Problem
A thorough research on the heterogeneous UAV mission coordination problem has been
conducted. The design of the multiple graph representation allows for the consideration of
heterogeneous UAV flight dynamics. A GA based algorithm has been developed as the solution
method. Simulation experiments demonstrate that the solution algorithm is able to solve both
intuitive problem cases as well as complicated problem cases.
The small turning radius cases provide solution output similar to that of the multiple
traveling salesmen problem. This demonstrates the robustness of the model and solution. When
the turning radius increases, the result output is less intuitive and cannot be imagined by human
mind. This is when the usefulness of the solution algorithm gets illustrated.
107
The stability test in the end demonstrates the consistency in both the convergence speed
and the final solution reached. Hence, it further justifies the choice of model and solution
algorithm in this problem.
7.3 Conclusions on the Study of the UAV Surveillance Problem
For the UAV surveillance problem, a generalized occlusion-aware model has been
proposed that finds the observable area from a given sensor in the sky. Previous literatures only
achieved this in an analytical way on flat terrain with buildings in the shape of quadrangular
prisms. However, the model in this study could potentially be applied to any terrain condition
and all sorts of building shapes, so long as the buildings are described as closed 3D objects. In
this way, this study provides a solid foundation for possible future development in the
surveillance by UAVs in urban environment in terms of solution efficiency and problem
generalization.
7.4 Recommendation for Future Work
In this study, a thread is dedicated to the single UAV path planner described in this thesis.
There are other threads that take charge of the user-interface and the display of the mission
planner. Such multi-thread design environment provides a good foundation for various future
development. Some possible implementable extensions are described in the following section.
Multiple UAV Mission Planning
The following steps can be taken to extend the planner from single UAV to multiple
UAV mission planning.
(1) Each UAV has its pre-defined targets. Several threads run in parallel to calculate the
paths for UAVs. There is no interference among the threads.
108
(2) Based on step (1), there are communications among the threads for coordination and
collision avoidance.
(3) Based on step (2), the targets are not assigned to the UAVs initially. The UAVs will have
to coordinate among themselves for target assignment and path planning.
Another important aspect is to combine the methods for UAV task assignment with that
of single UAV path planning. In this way, the path planning and the coordination among UAVs
are both done in an integrated manner.
Variable UAV Velocity
The UAV velocity is assumed to be constant throughout this study. When variable UAV
velocity is considered, greater flexibility is given to the model. For example, UAVs will have to
turn at greater turning radius when velocity is larger. Variable UAV velocity also gives better
collision avoidance solutions for multiple UAV case.
Online Mission Planner
So far, the mission planner works in a way that the path is planned before the UAVs take
off. However, the multi-thread structure of the planner will allow for the modification of the path
even after the UAV takes off. This mode is called “online” planning since the planner must deal
with unforeseen situations such as “pop-up” radar or addition of targets.
109
REFERENCES
1.
James, G.K., Unmanned Aerial Vehicles and Special Operations: Future Directions. 2000: PhD
Thesis, Naval Postgraduate School, Monterey, California, United States. p. 130p.
2.
Canavan, G.H., UAV sensor and survivability issues. 1996: United States. p. 22p.
3.
Larroque, C., K.S. Thompson, and D. Hickman. UAV sensor systems for close-range operations.
in Infrared and Passive Millimeter-wave Imaging Systems: Design, Analysis, Modeling, and
Testing, 3-5 April 2002. 2002. USA: SPIE-Int. Soc. Opt. Eng.
4.
Altenbaugh, R., et al. Application of the Raven UAV for Chemical and Biological Detection. in
Chemical, Biological, Radiological, Nuclear, and Explosives (CBRNE) Sensing XI, 6-8 April
2010. 2010. USA: SPIE - The International Society for Optical Engineering.
5.
Ryan, A., et al. An overview of emerging results in cooperative UAV control. in 2004 43rd IEEE
Conference on Decision and Control (CDC), 14-17 Dec. 2004. 2004. Piscataway, NJ, USA: IEEE.
6.
Martinez, C., et al. On-board and ground visual pose estimation techniques for UAV control.
2011. Van Godewijckstraat 30, Dordrecht, 3311 GZ, Netherlands: Springer Netherlands.
7.
Pisano, W.J., D.A. Lawrence, and P.C. Gray. Autonomous UAV control using a 3-sensor
autopilot. in 2007 AIAA InfoTech at Aerospace Conference, May 7, 2007 - May 10, 2007. 2007.
Rohnert Park, CA, United states: American Institute of Aeronautics and Astronautics Inc.
8.
Cassandras, C.G. and L. Wei. A receding horizon approach for dynamic UAV mission
management. in Enabling Technologies for Simulation Science VII, 22-25 April 2003. 2003. USA:
SPIE-Int. Soc. Opt. Eng.
9.
Koeners, G.J.M., et al. Exploring network enabled airspace integration functions for a UAV
mission management station. in 2006 IEEE/AIAA 25th Digital Avionics Systems Conference, 1519 Oct. 2006. 2006. Piscataway, NJ, USA: IEEE.
10.
Reinelt, G., TSPLIB-a traveling salesman problem library. ORSA Journal on Computing, 1991.
3(Copyright 1992, IEE): p. 376-84.
110
11.
Sisson, M.R., Applying Tabu Heuristic to Wind Influenced, Minimum Risk and Maximum
Expected Coverage Routes. 1997: United States. p. 83p.
12.
Ryan, J.L., et al. Reactive Tabu Search in unmanned aerial reconnaissance simulations. in
Proceedings of IEEE Winter Simulation Conference, 13-16 Dec. 1998. 1998. Piscataway, NJ,
USA: IEEE.
13.
Beard, R.W., T.W. McLain, and M. Goodrich. Coordinated target assignment and intercept for
unmanned air vehicles. in 2002 IEEE International Conference on Robotics adn Automation,
May 11, 2002 - May 15, 2002. 2002. Washington, DC, United states: Institute of Electrical and
Electronics Engineers Inc.
14.
Bortoff, S.A. Path planning for UAVs. in 2000 American Control Conference, June 28, 2000 June 30, 2000. 2000. Chicago, IL, USA: IEEE.
15.
McLain, T.W., et al. Cooperative control of UAV rendezvous. in 2001 American Control
Conference, June 25, 2001 - June 27, 2001. 2001. Arlington, VA, United states: Institute of
Electrical and Electronics Engineers Inc.
16.
Chandler, P.R., M. Pachter, and S. Rasmussen. UAV cooperative control. in Proceedings of
American Control Conference, 25-27 June 2001. 2001. Piscataway, NJ, USA: IEEE.
17.
Stirling, W.C. and M.A. Goodrich. Conditional preferences for social systems. in Proceedings of
IEEE International Conference on Systems, Man & Cybernetics, 7-10 Oct. 2001. 2001.
Piscataway, NJ, USA: IEEE.
18.
Yao-hong, Q., P. Quan, and Y. Jian-guo. Flight path planning of UAV based on heuristically
search and genetic algorithms. in IECON 2005. Thirty-First Annual Conference of the IEEE
Industrial Electronics Society, 6-10 Nov. 2005. 2005. Piscataway, NJ, USA: IEEE.
19.
Alighanbari, M., Y. Kuwata, and J.P. How. Coordination and Control of Multiple UAVs with
Timing Constraints and Loitering. in 2003 American Control Conference, June 4, 2003 - June 6,
2003. 2003. Denver, CO, United states: Institute of Electrical and Electronics Engineers Inc.
111
20.
Bellingham, J.S., et al. Cooperative path planning for multiple UAVs in dynamic and uncertain
environments. in 41st IEEE Conference on Decision and Control, December 10, 2002 December 13, 2002. 2002. Las Vegas, NV, United states: Institute of Electrical and Electronics
Engineers Inc.
21.
Kuwata, Y. and J. How. Three dimensional receding horizon control for UAVs. in Collection of
Technical Papers - AIAA Guidance, Navigation, and Control Conference, August 16, 2004 August 19, 2004. 2004. Providence, RI, United states: American Institute of Aeronautics and
Astronautics Inc.
22.
Richards, A. and J.P. How. Aircraft trajectory planning with collision avoidance using mixed
integer linear programming. in Proceedings of 2002 American Control Conference, 8-10 May
2002. 2002. Danvers, MA, USA: American Automatic Control Council.
23.
Ruz, J.J., et al. Using MILP for UAVs trajectory optimization under radar detection risk. in 2006
IEEE Conference on Emerging Technologies and Factory Automation, ETFA, September 20,
2006 - September 22, 2006. 2006. Hamburg, Germany: Institute of Electrical and Electronics
Engineers Inc.
24.
Melchior, P., et al., Consideration of obstacle danger level in path planning using A* and fastmarching optimisation: comparative study. Signal Processing, 2003. 83(Copyright 2004, IEE): p.
2387-96.
25.
Szczerba, R.J., et al., Robust algorithm for real-time route planning. Ieee Transactions on
Aerospace and Electronic Systems, 2000. 36(Copyright 2000, IEE): p. 869-78.
26.
Qu, Y.H., Q. Pan, and H.G. Yan, Flight path planning of UAV based on heuristically search and
genetic algorithms. Iecon 2005: Thirty-First Annual Conference of the Ieee Industrial Electronics
Society, Vols 1-3, 2005: p. 45-49
27.
Mittal, S. and K. Deb. Three-dimensional offline path planning for UAVs using multiobjective
evolutionary algorithms. in 2007 IEEE Congress on Evolutionary Computation, 25-28 Sept. 2007.
2007. Piscataway, NJ, USA: IEEE.
112
28.
Hasircioglu, I., H.R. Topcuoglu, and M. Ermis. 3-D path planning for the navigation of
unmanned aerial vehicles by using evolutionary algorithms. in 10th Annual Genetic and
Evolutionary Computation Conference, GECCO 2008, July 12, 2008 - July 16, 2008. 2008.
Atlanta, GA, United states: Association for Computing Machinery.
29.
Nikolos, I.K. and N.C. Tsourveloudis. Path planning for cooperating unmanned vehicles over 3-d
terrain. in International Conference on Informatics in Control Automation and Robotics, ICINCO
2007, May 9, 2007 - May 12, 2007. 2009. Angers, France: Springer Verlag.
30.
Nikolos, I.K., et al., Evolutionary algorithm based offline/online path planner for UAV
navigation. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 2003.
33(Copyright 2003, IEE): p. 898-912.
31.
Pehlivanoglu, Y.V. and A. Hacioglu. Vibrational genetic algorithm based path planner for
autonomous UAV in spatial data based environments. in 3rd International Conference on Recent
Advances in Space Technologies, RAST 2007, June 14, 2007 - June 16, 2007. 2007. Istanbul,
Turkey: Inst. of Elec. and Elec. Eng. Computer Society.
32.
Changwen, Z., et al., Evolutionary route planner for unmanned air vehicles. Ieee Transactions on
Robotics, 2005. 21(Copyright 2005, IEE): p. 609-20.
33.
Besada-Portas, E., et al., Evolutionary Trajectory Planner For Multiple UAVs In Realistic
Scenarios. Ieee Transactions on Robotics, 2010. 26(Copyright 2010, The Institution of
Engineering and Technology): p. 619-34.
34.
De La Cruz-Garcia, J.M., et al. Multiobjective path planner for UAVs based on genetic
algorithms. in Computational Intelligence in Decision and Control - 8th International FLINS
Conference, September 21, 2008 - September 24, 2008. 2008. Madrid, Spain: World Scientific
Publishing Co. Pte. Ltd.
35.
De La Cruz, J.M., et al. Evolutionary path planner for UAVs realistic environments. in 10th
Annual Genetic and Evolutionary Computation Conference, GECCO 2008, July 12, 2008 - July
16, 2008. 2008. Atlanta, GA, United states: Association for Computing Machinery.
113
36.
Ruoding, Z., Z. Changwen, and Y. Ping. Route planning for unmanned air vehicles with multiple
missions using an evolutionary algorithm. in 2007 3rd International Conference on Natural
Computation, 24-27 Aug. 2007. 2007. Piscataway, NJ, USA: IEEE.
37.
Farin, G.E., Curves and surfaces for computer aided geometric design : a practical guide.
Computer science and scientific computing. 1988, Boston: : Academic Press. xv, 334 p., [4] p. of
plates.
38.
Rasmussen, S.J., et al. State-space search for improved autonomous UAVs assignment algorithm.
in Decision and Control, 2004. CDC. 43rd IEEE Conference on. 2004.
39.
Shima, T., S.J. Rasmussen, and A.G. Sparks. UAV cooperative multiple task assignments using
genetic algorithms. in Proceedings of the 2005 American Control Conference, 8-10 June 2005.
2005. Piscataway, NJ, USA: IEEE.
40.
Dubins, L.E., ON CURVES OF MINIMAL LENGTH WITH A CONSTRAINT ON AVERAGE
CURVATURE, AND WITH PRESCRIBED INITIAL AND TERMINAL POSITIONS AND
TANGENTS. American Journal of Mathematics, 1957. 79(3): p. 497-516.
41.
Nygard, K.E., P.R. Chandler, and M. Pachter. Dynamic network flow optimization models for air
vehicle resource allocation. in Proceedings of American Control Conference, 25-27 June 2001.
2001. Piscataway, NJ, USA: IEEE.
42.
Schumacher, C., P.R. Chandler, and S.R. Rasmussen. Task allocation for wide area search
munitions. in American Control Conference, 2002. Proceedings of the 2002. 2002.
43.
Rasmussen, S.J. and T. Shima, Tree search algorithm for assigning cooperating UAVs to multiple
tasks. International Journal of Robust and Nonlinear Control, 2008. 18(Compendex): p. 135-153.
44.
Semsch, E., et al. Autonomous UAV Surveillance in Complex Urban Environments. in Web
Intelligence and Intelligent Agent Technologies, 2009. WI-IAT '09. IEEE/WIC/ACM International
Joint Conferences on. 2009.
45.
Bioinformatics and Interaction Design. 2010 23 September 2010 [cited 2011 19 September];
Available from: http://biointeraction.blogspot.com/2010/09/dna-and-chromosome.html.
114
46.
Rasmussen, S.J., et al. Introduction to the MultiUAV2 simulation and its application to
cooperative control research. in American Control Conference, 2005. Proceedings of the 2005.
2005.
115
[...]... section 1.3 UAV Path Planning UAV mission planning is in itself a very broad research area in which the mission coordination among multiple UAVs and the path planning of UAVs are two fundamental areas These two areas are chosen in this study as the results can be used for future research on mission planning, e.g., multiple UAV mission planning, real-time, and variable velocity UAV path planning For... Concerning UAVs Due to the widening scope and increasing complexity of UAV missions, the researches on UAVs span a very broad range Some of the important research areas include sensor and control technology, mission management system and UAV mission planning 1.2.1 Sensor technologies The development in sensor technologies aims at enhancing the UAV capabilities and broadening the mission types a UAV is... targets while minimizing the probability of destruction of the UAV (a) (b) Figure 1.1 Dot-on-coordinate mission planning model and Voronoi diagram solution [13] Studies had targeted both single [14] and multiple [13, 15, 16] UAV mission planning in such setting One proposed solution algorithm is the two-stage Voronoi diagram (see Figure 1.2b) and virtual forces [13] Voronoi diagram is a way of space repartition... terrain and fixed shapes of buildings, making the problem unrealistic This study focuses on developing an occlusion aware model that is both efficient and allows for more generalized problem definition 1.6 Research Objectives This thesis covers three different aspects of the researches on UAV mission planning The first is the trajectory and path planning of single UAVs This study has gone very deep and. .. conducting research in the three areas studied The motivations concern more on the modeling and formulation of the problem Following this, the genetic algorithm (GA) as a solution tool is discussed 2.1 UAV Path Planning in Complex and Realistic Environment Despite much effort that has been put on the 3D UAV mission planning and some success has been achieved, there are plenty of rooms for improvement The... decision making Cassandras and Wei proposed a simulated battle space and a dynamic target assignment scheme which achieves the commander‟s order with optimal performances [8] Another example is the airspace integration through network among the UAV, the mission management station and the air traffic control [9] 1.2.4 Mission planning UAV mission planning, which is also the focus of this study, considers optimization... in the model 1.3.4 Three-dimensional (3D) mission model Recently, evolutionary algorithm (EA) becomes a popular choice for solving route planning problem Since it greatly increases the computational power, complex 3D models are widely used for the UAV mission planning problems The major components of a 3D model are terrain, UAV path, and threats For terrain modeling, one method is to artificially generate... hard to handle for local modifications Table 1.1 summarizes the merits and disadvantages of the various modeling approaches for different components of the 3D UAV path planning model Table 1.1 Various 3D modeling approaches for UAV path planning 3D Component Terrain Model Parametric artificial terrain Digital Elevation Model Linear Segments UAV Path Threats EA Coding Advantages Continuous, not demanding... Demanding on the solution algorithm The search space is large Local changes are hard to handle Realistic Easy to be used for the EA operators Reduced Search Space 11 Table 1.2 summarizes the models and solution algorithms adopted in some of the important previous works on UAV mission planning Table 1.2 Models and solution algorithms in previous works Model mTSPTW Dots-oncoordinate Solution Algorithm... Integration of onboard and ground camera is a proposed method for real-time positioning of UAVs [6] Another study proposes UAV control by means of a single axis rate gyro, an absolute pressure sensor and a GPS receiver [7] 1.2.3 Mission management system Researches on mission management system target at facilitating the battlefield commanders in high-level control and decision making Cassandras and Wei proposed ... used for future research on mission planning, e.g., multiple UAV mission planning, real-time, and variable velocity UAV path planning For example, multiple UAV mission planning requires a target... addresses the fixed-wing UAV mission planning problem More specifically, the study focuses on sub-problems: single UAV path planning, multiple UAV cooperative task assignment, and UAV surveillance in... mission management system and UAV mission planning 1.2.1 Sensor technologies The development in sensor technologies aims at enhancing the UAV capabilities and broadening the mission types a UAV
Ngày đăng: 12/10/2015, 17:34
Xem thêm: UAV mission planning problem modeling and solution methods
