UAV mission planning problem modeling and solution methods

ACKNOWLEDGEMENT I would like to express my sincere gratitude to my supervisor Associate Professor ZHANG Yunfeng, for his invaluable guidance throughout this project. From literature survey methodology at the beginning, to the crucial steps taken throughout the project, his advices kept me on the right direction and inspired me greatly. In addition, thanks are given to PhD candidates Mr. GENG Lin and Dr. WANG Yifa, who had been very supportive for my research. I would also like to show my appreciation for the financial support in the form of a research scholarship from the National University of Singapore. i TABLE OF CONTENTS ACKNOWLEDGEMENT ............................................................................................................... i TABLE OF CONTENTS ................................................................................................................ ii ABSTRACT .................................................................................................................................. vii LIST OF FIGURES ....................................................................................................................... ix LIST OF TABLES ........................................................................................................................ xii CHAPTER 1 INTRODUCTION .......................................................................................................................... 1 1.1 The History of UAVs ............................................................................................................ 1 1.1.1 World War I ................................................................................................................... 1 1.1.2 World War II .................................................................................................................. 2 1.1.3 Post-World War II, through pre-Vietnam war ............................................................... 2 1.1.4 From Vietnam War to Gulf war ..................................................................................... 2 1.1.5 From Gulf war to present ............................................................................................... 3 1.2 Modern Research Areas Concerning UAVs .................................................................... 3 1.2.1 Sensor technologies ....................................................................................................... 3 1.2.2 Control technologies ...................................................................................................... 4 1.2.3 Mission management system ......................................................................................... 4 1.2.4 Mission planning ............................................................................................................ 4 1.3 UAV Path Planning .............................................................................................................. 5 1.3.1 Multiple travel salesman problems with time window constraints (mTSPTW) model . 6 1.3.2 Dots-on-coordinate model ............................................................................................. 7 1.3.3 Polygon-obstacles model ............................................................................................... 8 ii 1.3.4 Three-dimensional (3D) mission model ........................................................................ 9 1.4 Integrated Task Assignment and Path Optimization .......................................................... 12 1.4.1 Dubins‟ vehicle ............................................................................................................ 14 1.4.2 The dynamic network flow model ............................................................................... 15 1.4.3 The tree representation model ...................................................................................... 16 1.4.4 The graph representation model................................................................................... 16 1.5 Surveillance in Complex Urban Environment .................................................................... 16 1.6 Research Objectives ............................................................................................................ 17 1.7 Limitations and Assumptions of Research ......................................................................... 18 1.8 Organization of the Thesis .................................................................................................. 18 CHAPTER 2 RESEARCH MOTIVATIONS ..................................................................................................... 20 2.1 UAV Path Planning in Complex and Realistic Environment ............................................. 20 2.2 UAV Cooperative Task Assignment .................................................................................. 21 2.3 UAV Surveillance in Complex Urban Environment .......................................................... 21 2.4 Genetic Algorithms as a Solution Tool............................................................................... 22 2.5 Java Monkey Engine (JME) as System Implementation .................................................... 22 CHAPTER 3 INTRODUCTION TO GENETIC ALGORITHM ....................................................................... 24 3.1 Encoding of the GA Chromosome...................................................................................... 26 3.2 The Initialization ................................................................................................................. 27 3.3 The Fitness Function ........................................................................................................... 27 3.4 The Genetic Operators ........................................................................................................ 28 iii 3.5 The Selection Process ......................................................................................................... 29 CHAPTER 4 A GA-BASED UAV PATH PLANNER IN COMPLEX REALISTIC ENVIRONMENT .......................................................................... 31 4.1 The Proposed Model for UAV Path Planner ...................................................................... 32 4.1.1 The description of terrain ............................................................................................. 32 4.1.2 The description of the UAV path ................................................................................. 35 4.1.3 The description of threats ............................................................................................. 39 4.2 The Design of Genetic Algorithm ...................................................................................... 41 4.2.1 The encoding of solution ............................................................................................. 41 4.2.2 Initialization ................................................................................................................. 41 4.2.3 The fitness function...................................................................................................... 43 4.2.4 The selection mechanism ............................................................................................. 47 4.2.5 The genetic operators ................................................................................................... 47 4.3 Simulation Experiments ...................................................................................................... 50 4.3.1 Experiments on different fitness functions .................................................................. 50 4.3.2 Experiments on functions of different GA operators ................................................... 55 4.3.3 Experiments on magnitude of probabilities for the GA operators ............................... 61 4.3.4 Experiments with different UAV and UAV path characters ....................................... 62 4.4 Discussions ......................................................................................................................... 67 CHAPTER 5 A GENETIC ALGORITHM FOR UAV COOPERATIVE TASK ASSIGNMENT .................................................................. 69 iv 5.1 The Problem Description .................................................................................................... 69 5.2 Dubins Vehicle ................................................................................................................... 70 5.3 Graph Representation ......................................................................................................... 73 5.3.1 Discrete UAV heading angle on target ........................................................................ 73 5.3.2 The graph representation.............................................................................................. 74 5.4 The Solution Encoding ....................................................................................................... 76 5.5 The Initialization ................................................................................................................. 77 5.6 The Fitness Function ........................................................................................................... 79 5.7 The Genetic Algorithm Operators ...................................................................................... 81 5.8 The Simulation Experiments .............................................................................................. 83 5.8.1 Diversity of the random solution generator ................................................................. 84 5.8.2 Experiments on different probabilities of genetic operators ........................................ 84 5.8.3 Experiments with different fitness functions ............................................................... 87 5.8.4 Experiments with different UAV flight dynamics ....................................................... 89 5.8.5 Stability of the solution algorithm ............................................................................... 96 5.9 Summary ............................................................................................................................. 97 CHAPTER 6 AN OCCLUSION-AWARE MODEL FOR UAV SURVEILLIANCE IN COMPLEX URBAN ENVIRONMENT ................................................................................. 99 6.1 Problem Description ........................................................................................................... 99 6.1.1 The environment .......................................................................................................... 99 6.1.2 The sensor capability of UAVs .................................................................................. 100 6.1.3 The objective function ............................................................................................... 100 v 6.2 The Two-Stage Optimization............................................................................................ 101 6.3 The Occlusion Aware Model ............................................................................................ 101 6.4 System Implementation of the Occlusion Aware Model .................................................. 102 6.5 Limitations of the Two-Stage Method .............................................................................. 104 6.6 Discussions ....................................................................................................................... 105 CHAPTER 7 CONCLUSIONS AND FUTURE WORK ................................................................................. 106 7.1 Conclusions on the Study of the UAV Path Planning Problem ........................................ 106 7.2 Conclusions on the Study of the Multiple UAV Task Assignment Problem ................... 107 7.3 Conclusions on the Study of the UAV Surveillance Problem .......................................... 108 7.4 Recommendation for Future Work ................................................................................... 108 REFERENCES ........................................................................................................................... 110 vi ABSTRACT The research work reported in this thesis addresses the fixed-wing UAV mission planning problem. More specifically, the study focuses on 3 sub-problems: single UAV path planning, multiple UAV cooperative task assignment, and UAV surveillance in an urban environment. The study for each specific problem includes the development of problem modeling method and optimization solution method, followed by simulation experiments for verification. The first part considers the problem of finding an optimal path for a UAV in a realistic environment. The realistic model used in this project includes terrain, radar zone, flight prohibited zones, and UAV capability limits. The flight path is represented by piecewise cubic Bezier curves. A genetic algorithm (GA) is developed to find the optimal solutions. The unique design of the GA, including solution encoding, GA operators, and fitness function, allows the search to escape from those local minimums and manage to find the optimal or near-optimal solution in a robust manner. The second part considers the problem of multiple UAV coordination to accomplish assigned tasks on different targets. To accommodate the fact that UAVs can be heterogeneous, a multiple graph representation scheme is proposed to model the solution map for each UAV. A GA with unique design of encoding scheme and GA operators is developed to find optimal solutions. The testing results from various simulation experiments show that the GA is able to solve this coordination problem in an effective and efficient manner. The third part considers the problem of UAV surveillance in an urban environment. To overcome the limitation of previously reported models, an occlusion-aware model is developed to simulate the environment under surveillance (terrain and buildings). An efficient ray-casting based collision checking algorithm is also developed to determine the visibility of any given vii observation position of the UAV. Again, heterogeneous UAVs can be modeled. Due to the limitation of time, however, the solution method to find the optimal set of observation points for UAVs is yet to be developed. In summary, the study on the 3 different UAV mission planning problems represent a major step towards developing advanced modeling and solution tools to deal with close-to-reality problems. Although there are plenty of challenges to be tackled before the final goal can be achieved, this effort has built solid foundations for future extension. viii LIST OF FIGURES Figure 1.1 Dot-on-coordinate mission planning model and Voronoi Diagram solution Figure 1.2 Interface of Polygon-obstacles model Figure 3.1 The evolution process of giraffes Figure 3.2 Illustrations of gene and chromosome Figure 3.3 Example of solution encoding for path planning problems Figure 4.1 Grayscale height map representation of Oahu Island Figure 4.2 Rendered 3D representation of Oahu Island Figure 4.3 Illustration of UAV flight constraints Figure 4.4 Cubic Bezier interpolation Figure 4.5 Illustrative example of mission planner over Oahu Island, Hawaii Figure 4.6 A chromosome with two intermediate waypoints Figure 4.7 The cross-over operator Figure 4.8 The local mutation operator Figure 4.9 The strong mutation operator Figure 4.10 The deletion operator Figure 4.11 Test 1: simple obstacle evasion test for static fitness Figure 4.12 Test 2: concave obstacle evasion test for static fitness Figure 4.13 Different solutions with same static fitness value Figure 4.14 Test 3: concave obstacle evasion test for dynamic fitness Figure 4.15 Test 4: fitness plot for random generation Figure 4.16 Test 5: fitness plot for cross-over operator Figure 4.17 Test 6: comparative study of local and strong mutation operators Figure 4.18 Test 7: simple test without deletion/smoothening ix Figure 4.19 Test 8: simple test with deletion/smoothening operator Figure 4.20 Test 9: aggressive UAV maneuverability Figure 4.21 Test 10: conservative UAV maneuverability Figure 4.22 Test 11: fitness evolution for different values of d Figure 5.1 The minimum path problem Figure 5.2 Illustration of a feasible path Figure 5.3 Dubins path is one of the four possible choices when Figure 5.4 Two additional choices of Dubins path at close starting and ending distance Figure 5.5 Illustrative example of a mission involving two UAVs and two targets Figure 5.6 Discrete headings of UAV Figure 5.7 Two examples of paths with different heading angle on target Figure 5.8 The graph representations with 3 part discretization Figure 5.9 One solution of the example Figure 5.10 Algorithm for the random solution generation Figure 5.11 Cross-over operator on same solution across different UAVs Figure 5.12 Mutation operator changes heading angle Figure 5.13 Illustration of swap operator Figure 5.14 Fitness evolution for different GA operators Figure 5.15 Best fitness evolution for different GA operator probabilities Figure 5.16 Average fitness evolution of the different GA probabilities Figure 5.17 Result comparison for different fitness functions Figure 5.18 Fitness value evolutions for different fitness functions Figure 5.19 Simulation output for turning radius 1 Figure 5.20 Fitness evolution of the solution of experiment with turning radius 1 Figure 5.21 Simulation output for turning radius 3 x Figure 5.22 Fitness evolution of the solution of experiment with turning radius 3 Figure 5.23 Simulation output for turning radius 7 Figure 5.24 Fitness evolution of the solution of experiment with turning radius 7 Figure 5.25 Simulation output for turning radius 10 Figure 5.26 Fitness evolution of the solution of experiment with turning radius 10 Figure 5.27 Simulation output for heterogeneous turning radius Figure 5.28 Fitness evolution of the solution of experiment with heterogeneous turning radius Figure 5.29 The best fitness of 10 same-parameter simulations Figure 5.30 The average fitness of 10 same-parameter simulations Figure 6.1 Visibility test with analytical method Figure 6.2 Illustration of ray detection Figure 6.3 Occlusion aware model xi LIST OF TABLES Table 1.1 3D models by components Table 1.2 Models and solution algorithms in previous works Table 4.1 Parameters defining the piecewise cube Bezier curve Table 4.2 Static fitness value assignment Table 4.3 Dynamic fitness value assignment Table 4.4 Symbols for probability of each EA operator Table 4.5 Controlled constants for experiments on fitness function Table 4.6 Homogeneous assignment of fitness multipliers Table 4.7 Final assignment of fitness multipliers Table 4.8 Controlled variables for experiments on EA operators Table 4.9 EA operator probabilities for immigration testing Table 4.10 EA operator probabilities for cross-over testing Table 4.11 EA operator probabilities for local mutation testing Table 4.12 EA operator probabilities for local and strong mutation testing Table 4.13 EA operator probabilities for deletion/smoothening testing Table 4.14 Fitness values after 50 iterations with different EA probabilities Table 4.15 Final adopted EA operator probabilities Table 4.16 Aggressive UAV maneuverability Table 4.17 Conservative UAV maneuverability Table 4.18 Different distances between two consecutive waypoints Table 5.1 Constant parameters for genetic operators‟ test Table 5.2 Solution quality of the different GA operators Table 5.3 Constant parameters for different fitness function test xii Table 5.4 Constant parameters for UAV flight dynamics tests Table 5.5 Result statistics Table 5.6 Parameters used for repeated simulations testing stability xiii CHAPTER 1 INTRODUCTION 1.1 The History of UAVs This section firstly introduces the definitions UAVs and differentiates them from conventional aircrafts. It then outlines the essential technologies required for them. A revision of the historical development of UAVs is also given in detail. There are two ways to categorize aerial vehicles. The first way is according to their method of guidance and the second is according to whether they are expendable. In terms of guidance technology, an aerial vehicle can be either manned, remote piloted or auto piloted. Expendable aerial vehicles are not reusable and are categorized as missiles. The UAVs discussed in this thesis are auto or remote piloted and recoverable. James [1] summarizes three important technologies in order to make UAVs successful for operational use: (1) an aerial platform capable to maneuver to an appropriate objective; (2) a guidance system that would permit overthe-horizon unmanned aerial vehicle operations; (3) a payload that can perform a useful mission once the platform gets to its objective. 1.1.1 World War I Aerial balloon was a primitive form of UAV, used in the siege of Venice by the Austrian troops in August 1849. In early 20th century, the invention of airplane by Wright brothers was a major breakthrough in technology that brings UAVs into practical use. The directional mobility of an airplane is a capability that balloons do not have. Thereafter, the first essential technology is met. Aerial vehicles are able to reach their objectives. However, the guidance of such vehicles cannot be achieved without onboard pilots. In 1916, the first pilotless aircraft - the “aerial 1 torpedoes” appeared. Designed by the U.S. and Britain, these are aircrafts filled with explosives designed to fly for a set distance and crash. However, due to the inaccuracy of the guidance system, none of the attempts during WWI resulted in operational application of the UAVs. 1.1.2 World War II Each side had significant development in UAV technology during the war. The most significant and notorious was the German V-1 [1]. The V-1 was a self-guided monoplane filled with explosives that would fly a pre-set heading and time, at which time the engines would cut off and the aircraft would go into a dive, exploding on impact. Hitler‟s V-1 campaign marked the first large-scale operational employment of unmanned aircraft. Studies have shown that the cost of the Germen was only one forth of that of Britain, thanks to the use of UAVs. On the other hand, the allies achieved radio control of the unmanned aircrafts. However, without a computer, automatic control is not feasible. 1.1.3 Post-World War II, through pre-Vietnam war During this period, there were some significant improvements in guidance technologies and flight endurance, which is a direct consequence of the escalating cold war between the US and the Soviet Union. Therefore, cruise missiles and photo-reconnaissance aircrafts were developed. However, the low reliability of such aircrafts makes them not practical for operational use. 1.1.4 From Vietnam War to Gulf war In terms of capabilities, the UAVs in this period improved in both flight capacities and accuracies. Unmanned target drones increased in performance with speeds all the way up to mach 4 and service ceilings of nearly 30000 meters. Their payload also increased, allowing the 2 carriage of bombs. In terms of guidance technology, they began as programmed drones. Later, they were capable to receive order when in flight. This period was an important one during which the technology gap between UAVs and the manned aircrafts had been removed. 1.1.5 From Gulf war to present The increasing computer processing capabilities, data transmission rates and miniaturization technology were the driving forces behind the development of UAV capabilities during this period. Computer processing power allowed for smaller chips to be installed onto the aircraft, increasing the payload. It also enabled auto-piloting by assimilating all the decision variables in an efficient way. Fast data transmission rate ensures minimum delay in the information obtained from the controller on the ground. This ensured the real-time decision making by commanders. Miniaturization decreased the probability that a UAV will be detected in its mission. This greatly ensured the secrecy of the mission. Moreover, miniaturization widens the scope of missions that UAVs are capable to carry out. 1.2 Modern Research Areas Concerning UAVs Due to the widening scope and increasing complexity of UAV missions, the researches on UAVs span a very broad range. Some of the important research areas include sensor and control technology, mission management system and UAV mission planning. 1.2.1 Sensor technologies The development in sensor technologies aims at enhancing the UAV capabilities and broadening the mission types a UAV is capable of. For the reconnaissance UAVs, the complexities of the sensors decide the UAV‟s operational altitude, which is pertinent to its survivability issues [2]. The operational efficiency of small scale UAVs heavily depend on the 3 balance between weight and complexity of its sensors [3]. The equipment of chemical sensor, for example, enables the Raven UAV – a hand launched close range reconnaissance platform - to detect and track a chemical plume in a variety of atmospheric conditions [4]. 1.2.2 Control technologies Researches in control technologies look at how to better take charge of the UAVs while they are a certain distance away. There are two aspects associated with the control technology challenges. The first is how to send orders to the aircrafts while they are distant from the control center. The second is how to achieve real-time positioning of the aircraft while they are over the horizon. Cooperative control investigates the distributed control of multiple UAVs over the horizon [5]. Integration of onboard and ground camera is a proposed method for real-time positioning of UAVs [6]. Another study proposes UAV control by means of a single axis rate gyro, an absolute pressure sensor and a GPS receiver [7]. 1.2.3 Mission management system Researches on mission management system target at facilitating the battlefield commanders in high-level control and decision making. Cassandras and Wei proposed a simulated battle space and a dynamic target assignment scheme which achieves the commander‟s order with optimal performances [8]. Another example is the airspace integration through network among the UAV, the mission management station and the air traffic control [9]. 1.2.4 Mission planning UAV mission planning, which is also the focus of this study, considers optimization of flight paths in a dynamic environment in which numerous mission objectives are defined. There are two kinds of problems in this area of research. The first considers the optimization of the 4 UAV flight paths, which is called UAV Path Planning. The second considers the cooperation of different UAVs in achieving a collection of mission objectives which is called UAV Cooperative Task Assignment. More details on these two problems are discussed in the following section. 1.3 UAV Path Planning UAV mission planning is in itself a very broad research area in which the mission coordination among multiple UAVs and the path planning of UAVs are two fundamental areas. These two areas are chosen in this study as the results can be used for future research on mission planning, e.g., multiple UAV mission planning, real-time, and variable velocity UAV path planning. For example, multiple UAV mission planning requires a target allocation step before applying the results of this study. Real-time planner runs parallel threads simultaneously, of which the primary threads are to be developed in this study. Variable velocity path planner makes the flight constraint a new variable – which is assumed to be constant throughout a single UAV take-off. The previously reported work on UAV path planning is studied from the following two different perspectives: (1) The robustness of the model. This examines the UAV path planning model in terms of its accuracy, complexity and authenticity. (2) The soundness of solution algorithms. This concerns whether the solution algorithm is fast enough in terms of finding the optimum solutions. At the same time, these two perspectives are highly correlated. A very complicated model can describe the real-life situation well, yet it is more demanding on the solution algorithms. On the other hand, a simple model gives only a heuristic of the real-life yet the optimal solution 5 could be found much easier. It is such trade off that makes the researches in UAV path planning challenging, yet interesting. In general, finding the optimal solution to the route-planning problem is nondeterministic-polynomial-time complete (NP-complete). Moreover, the UAV path planning problems are characterized by its huge solution space. Early researchers focused on modeling the problem in two dimensional (2D) spaces. More solution-demanding three dimensional (3D) models appeared only after the 2D case studies were mature enough and powerful solution algorithms were developed. Even today, 3D models still have potential for further developments. Some important modeling methods reported in the literatures are briefly reviewed in the following sections. 1.3.1 Multiple travel salesman problems with time window constraints (mTSPTW) model The first modeling approach on the UAV mission planning problem derives from the famous mTSPTW [10]. In this model, the UAV has predefined starting and ending positions and a number of targets to be visited. The route connecting any pair of targets is pre-defined. Each target is to be visited within a specified time window. Tabu search was proposed to be the solution method for such model [11, 12]. Tabu search enhances the performance of a local search method by using memory structures: once a potential solution has been determined, it is marked as tabu so that the algorithm does not visit that possibility repeatedly. Such local search procedure is used iteratively to move to better solutions until the stopping criterion is reached. This model links the UAV mission planning problem to the mature problem of mTSPTW, which allows for the application of various previous results. However, mTSPTW is more of a mathematical problem than a practical model. It hardly describes any real world UAV mission 6 situations. Therefore, the path planning result is of little practical use. On the algorithm side, tabu search is not capable of solving more complex models proposed subsequently. 1.3.2 Dots-on-coordinate model This proposed model (see Figure 1.1a) describes the mission setting on a 2D coordinate. It is called “dots-on-coordinate” in this study. One or several UAVs (black triangles) with their fixed initial positions are to visit single or multiple targets (squares). The constraints are threats in the form of dots. The closer the UAV is to a dot, the greater the probability of destruction. The objective is to find a path or multiple paths that visits the targets while minimizing the probability of destruction of the UAV. (a) (b) Figure 1.1. Dot-on-coordinate mission planning model and Voronoi diagram solution [13] Studies had targeted both single [14] and multiple [13, 15, 16] UAV mission planning in such setting. One proposed solution algorithm is the two-stage Voronoi diagram (see Figure 1.2b) and virtual forces [13]. Voronoi diagram is a way of space repartition by straight lines such that a point on any given line has equal distance to the two dot-threats closest to it. Hence, finding the optimal path is transformed to a discrete problem of finding a combination of edges on the 7 Voronoi diagram. In the first stage, graph search is applied to find the best combination of Voronoi edges as a rough solution. Virtual forces are imaginary repelling forces received by the paths from the dot-threats. The closer a path is to a threat, the greater the force. Better paths are subject to smaller repelling forces. The second stage includes the trimming of the rough solution with virtual forces. A gradient decent method is applied here to arrive at the path that has the smallest repelling force. The solution algorithm proposed for target assignment is the satisfying and social welfare paradigms [17]. Multiple UAVs are first decomposed into teams with assigned targets to each team. The welfare of each team is then optimized in parallel, allowing for re-allocation of targets in the solution process. Another solution method proposed for solving the Voronoi diagram is the A-star algorithm. Genetic algorithms are also proposed for the second step of virtual forces [18]. 1.3.3 Polygon-obstacles model Another popular problem is similar to the Dots-on-coordinate model. But instead of having dots as the threats, polygons are used to enclose areas as restricted zones (see Figure 1.2). The objectives are waypoints on the 2D map to be visited and the constraints are the shaded areas that should not be visited. Two improvements in the model were made as compared to the Dots-on-coordinate model. First, the dimension of the search space is greatly increased from the Voronoi edges to the entire 2D space (Except for the prohibited zones). Second, the coupling of the UAV missions are taken into account [19]. This means that, for instance, the elimination of a hostile surface to air missile site will have a positive effect on all the UAVs concerned. 8 Figure 1.2. Interface of the Polygon-obstacles model [20] One proposed solution method is the Mixed-integer Linear Programming [19-23]. However, being a linear programming method, the computation becomes very costly when the complexity of the problem increases. Another proposed method is the A-star algorithm [24-26]. Being a heuristic search algorithm, its efficiency is not robust to the increase in complexity in the model. 1.3.4 Three-dimensional (3D) mission model Recently, evolutionary algorithm (EA) becomes a popular choice for solving route planning problem. Since it greatly increases the computational power, complex 3D models are widely used for the UAV mission planning problems. The major components of a 3D model are terrain, UAV path, and threats. For terrain modeling, one method is to artificially generate a terrain surface [27-30]. The terrain elevation of a point z(x, y) is given by: z ( x, y)  sin( y  a)  b  sin( x)  c  cos(d  y 2  x 2 )  e  cos( y)  f  sin( f  y 2  x 2 )  g  cos( y) (1.1) where a, b, c, d, e, f, g are parameters defining the terrain shape. Such method is not demanding on the memory and guarantees the smoothness of the terrain surface. However, manually 9 generated terrains lack authenticity as compared to real terrain. Yet, it is not certain whether memory overflow issues will arise when real terrain is applied. In other reported works, real-world elevation data is used for terrain representation [3136]. Digital Elevation Model (DEM) is commonly used terrain model in this case. It is a continuous representation of the ground surface landform through a matrix that contains the elevation data at the corresponding coordinates. The merit of such representation is that the DEM data is a common form of geographic data representation. As a result, DEM data of different parts of the world is widely available. However, the rendering of large area of 3D terrain from DEM data is memory-intensive. Moreover, DEM may lead to discontinuity of the terrain. Although highly unusual, this will result in inaccuracy of the path planning procedure. The choice of path model directly affects the solution algorithm and the complexity of the entire mission planning model. Paths can be in the form of point-wise linear segments determined by a list of 3D absolute Cartesian points (x, y, z) [32, 36]. However, such paths are not smooth and hence hard to be followed by UAVs. In other works, Bezier curve [31] or Bspline curve [37] is used to represent the path to be followed by the UAV. Other curve models, such as cubic spline and 3rd degree B-spline curves, were also used for path representation [3335]. In this study, piecewise cubic Bezier curve is used for the representation of the UAV path due to computational efficiency. In terms of the constraints imposed on path planning, some focused only on the collision avoidance with the terrain [27-31], while others introduced additional simple circular threat model (representing enemy radar zone) that the UAV path must avoid [32, 36]. However, such radar model differs with any real-world radar. For real radar, its probability of object detection decreases with range. Some took a further step to incorporate a realistic model of both the enemy 10 radar and the flight prohibited zones (FPZs) [33-35]. Yet with increased complexity, the computational load increases significantly, thus affecting computational efficiency. As for the solution method to work with the 3D model, EA is typically the optimization method in which the coding of the points controlling the smooth path is critical. Some EAs codify the 3D points with absolute Cartesian coordinate [27, 30, 31]. Such coding scheme suits the EA operators well; however, the search space would be enormous with such setting. Others use relative polar coordinates for (x, y) [29, 34, 35], which would greatly reduce the search space. However, with such representation, changing any intermediate point will induce the change in all the subsequent points. Relative polar coordinate encoding is thus hard to handle for local modifications. Table 1.1 summarizes the merits and disadvantages of the various modeling approaches for different components of the 3D UAV path planning model. Table 1.1. Various 3D modeling approaches for UAV path planning 3D Component Terrain Model Parametric artificial terrain Digital Elevation Model Linear Segments UAV Path Threats EA Coding Advantages Continuous, not demanding on memory Abundant realistic data Straight forward and requires less calculation Disadvantages Unrealistic Memory intensive, may have discontinuity Sharp turns is hard to be followed by UAVs B-spline curve Local control More computations Single Bezier curve Easier to compute Non-local control Circular Radar Zone Combined realistic radar and FPZs Absolute 3D Cartesian coordinate Relative polar coordinate Easy to calculate Unrealistic Demanding on the solution algorithm The search space is large Local changes are hard to handle Realistic Easy to be used for the EA operators Reduced Search Space 11 Table 1.2 summarizes the models and solution algorithms adopted in some of the important previous works on UAV mission planning. Table 1.2. Models and solution algorithms in previous works Model mTSPTW Dots-oncoordinate Solution Algorithm Tabu search Voronoi Diagram Graph search, A-star Virtual Forces Gradient Descent, genetic Algorithms Polygon-obstacles & coupled UAV missions Three-dimensional mission Related Literatures [11, 12] [13-18] Mixed-integer linear programming, [19-23] A-star [24-26] Evolutionary Algorithms [27-36] 1.4 Integrated Task Assignment and Path Optimization In a generic UAV task assignment problem, the UAVs are required to perform three tasks (classify, attack, and verify) on each of the targets in the battlefield. The requirement of flyable trajectories dictates a lower bound on the turn radius and speed of the aerial vehicles. Thus, the generic problem is denoted bounded speed task assignment problem (BSTAP) [38]. This generic problem was further defined more specifically in [39] and is adopted in this research. It is assumed that the terrain has already been searched by other means and several targets have been found. Let T = {1, 2, …, Nt } be the set of targets found, V = {1, 2, …, Nv } be the UAVs performing tasks on these targets. The set of tasks that need to be performed by the UAV team on each target is M = {Classify, Attack, Verify} and we denote Nm as the number of such tasks. Target classification, consisting of maximizing the correct target recognition under given observation ability, can be performed only if the vehicle follows a trajectory that places its sensor 12 footprint on the target. After a target has been successfully classified, one or more UAVs attack it by releasing appropriate weapons. Following target attack, cooperative damage verification is performed. In general, the objective is to minimize the total length of the paths of the UAVs combined. In the previous works, some assumptions had been used in the description of the basic problem. With these assumptions, the problem is presented in its simplest form. Relaxing assumptions will lead to more complicated problem formulation. The implications for relaxing the assumptions will be discussed further. The assumptions for the basic type of generic problems are as follows. (1) Each UAV flies at its own, fixed altitude so that there are no collisions among the UAVs. (2) All UAVs are homogenous and fly at a constant speed. They hence have the same minimum turning radius. (3) Each UAV is capable of carrying out all three tasks on each target. (4) Each UAV has only three motions: (i) flying straight ahead; (ii) turning left; or (iii) turning right at the minimum turning radius. (5) There is no constraint of fuel and weapon system. Therefore, each UAV can travel for as long as it is assigned and visit as many targets as it can. (6) It takes one UAV only one time to put its sensor footprint above (to fly over) a target in order to classify the target. (7) It takes one UAV only one time to fly over a target in order to destroy it. (8) It takes one UAV only one time to fly over a target in order to verify that the target is destroyed. (9) All the UAVs are deployed simultaneously at known initial positions. 13 (10) The targets are homogenous. There is no priority in destroying any of them. (11) Each target can be destroyed at any time. There is no time window within which a target is to be destroyed. (12) There is no communication delay or interruptions among the UAVs. In the case that one UAV detects a target; the target information is shared immediately with all other UAVs on the same battlefield. While other assumptions are easy to understand, assumption (4) is further discussed in detail here. It is the so called “Dubins vehicle” problem [40], which explicitly defines the minimum-length path given the starting and ending positions and tangents of a UAV path. 1.4.1 Dubins’ vehicle In 1957, Dubins proved the existence and uniqueness of a minimal-length curve with a constraint on average curvature, and with prescribed initial and terminal positions and tangents [40]. Let a particle pursue a continuously differentiable path from an initial point u to a terminal point v. Its speed is unity and its velocity vectors at u and v are U and V, respectively. Furthermore, if X is a curve in real n-dimensional Euclidean space, parameterized by arc length (s), for which X’’(s) exists everywhere, then the curvature, defined by X ' ' ( s) , is less than or equal to R 1 everywhere, if and only if, X ' (s1 )  X ' (s2 )  R 1 s1  s2 (1.2) for all s1 and s 2 in the interval of definition of X. A curve X in real Euclidean n-space parameterized by arc length has average curvature always less than or equal to R 1 provided that its first derivative X ' exists everywhere and satisfies the Lipschitz condition in Eq. (1.2). Such path of minimum length between points u and 14 v with starting and ending conditions of U and V is called an R-geodesic. The following theorem is the main theorem of Dubins‟ research [40]: Theorem 1 – Dubin’s Vehicle Every planar R-geodesic is necessarily a continuously differentiable curve which is either: (1) an arc of a circle of radius R, followed by a line segment, followed by an arc of a circle of radius R; or (2) a sequence of three arcs of circles of radius R; or (3) a sub-path of a path of type (1) or (2). If applied to UAV paths, the theorem basically says that given the starting and ending positions and directions, there exists only one minimum-length path, which is among the six options. Among the six possible paths, one is the minimum length path according to the theorem. In the following sections, three variations of the generic problem, reported in the previous works, are discussed. One or many assumptions mentioned above are relaxed in each variation. As a result, different problem formulations and solution algorithms are proposed. 1.4.2 The dynamic network flow model In [41, 42], the air vehicle resource allocation problem is solved using dynamic network flow optimization models. The assumptions (5), (6) and (10) are relaxed. The UAVs have only sufficient fuel to fly for 30 minutes. Differences in onboard weapons and the attack angle also restrict the attack of a target. The classification of target is a probabilistic event that depends on the direction of the sensor on top of the target. Only when the classification probability of a target is greater than 90% can the target be attacked. Otherwise it has to be classified again. Moreover, targets are not homogenous and each target is assigned a value associated with its importance. As for the solution method, Nygard et al. [41] proposed a network optimization 15 model for solving the problem and Schumacher et al. [42] employed a linear program based on the capacitated trans-shipment problem to solve the task allocation problem. In [38, 39], the problem of cooperative multiple task assignments is discussed. Further to the relaxing of assumptions (5) as in [41, 42], assumption (7) is also relaxed. A second attack must be performed on a target if its destruction is not verified or if multiple attacks are required on the target. However, assumptions (6) and (10) are still used. As for solution method, Shima et al. [39] employed genetic algorithm and Rasmussen [38] used a state-space search algorithm. 1.4.3 The tree representation model Rasmussen and Shima [43] proposed a tree search algorithm for finding the solution for this classic problem. There are two drawbacks of the tree representation. First, the exhaustive expansion of all the branches of the tree is very memory-intensive. Usually, a heuristic search is applied for tree models to avoid exhaustive search. This, however, could easily lead to local minimums. Second, the tree structure is very hard to use for the representation of target visiting sequence, which is an important factor that will affect the solution qualities. 1.4.4 The graph representation model The graph representation has certain advantages as compared to the tree representation. In such representation scheme, each solution is composed of a combination of edges of the graph. The construction of the graph is not difficult and requires limited memory space. The implementation of the graph representation will be discussed in detail in Chapter 5. 1.5 Surveillance in Complex Urban Environment The integrated task assignment problem discussed above considers the coordination among multiple UAVs to complete pre-defined tasks on spot targets. Another class of problems 16 is to find an optimal way to cover an area with complex surface conditions. For example, in the urban areas, skyscrapers usually block the line of sight of the sensors in the sky. The scenario is best described by the work of [44]. A team of UAVs with onboard sensor is the given resource, the sensors have limited covering angles. In other words, the sensor covering area is a cone-like space below the UAV. The team of UAVs is to survey a given urban area with tall skyscrapers, with the objective to minimize the aggregate non-exposure time on the ground. The problem formulation in [44] will be described in Chapter 7 in detail. The drawback of [44] is two folded. First, it proposed a two-stage solution which decouples the problem into two separate optimization problems. This greatly impedes the ability of the algorithm to find global optimum. Second, its occlusion aware model allows only for flat terrain and fixed shapes of buildings, making the problem unrealistic. This study focuses on developing an occlusion aware model that is both efficient and allows for more generalized problem definition. 1.6 Research Objectives This thesis covers three different aspects of the researches on UAV mission planning. The first is the trajectory and path planning of single UAVs. This study has gone very deep and realistic into this area as compared to previous researches. The objective is to design a realistic and complex optimization model and a robust solution algorithm efficient in solving the optimization problem. The second objective is the coordination among multiple UAVs, in which reasonably simple assumptions on the UAV trajectories are made so as to focus the computational resources on the task allocation to different UAVs. Finding the optimal paths of multiple heterogeneous UAVs is the problem to be solved. The third objective is to model an 17 urban environment in which the line of sight of UAV sensors can be blocked by skyscrapers. This lays a foundation for the future application of path planning and cooperative task assignment. 1.7 Limitations and Assumptions of Research This research project considers the mission planning of the aircraft on a macro scale. The problem studied does not include the design of the aircraft itself. All experiments are done through computer simulations, with no actual fly-testing using the physical aircrafts. In terms of solution algorithm, this study focuses only on the genetic algorithm for optimization. Although three different topics are studied in this thesis, the general limitation and assumptions of the research are listed below. (1) The UAV is assumed to be a point, rather than a mass that has volume. Therefore, the collision detection is on a single point. (2) The UAV is assumed to fly at constant velocity. As a result, the turning radius constraint is constant in each simulation. The assumptions specific to each topic are described in detail in each section. 1.8 Organization of the Thesis The thesis is organized into 7 chapters. The current chapter is an overview of the thesis and the literature reviews. Chapter 2 explains the motivations of conducting the research. It points to the weakness and room for further development in the current research. It also provides reason for the use of genetic algorithm as a solution tool. Chapter 3 explains the principles and mechanisms of Genetic Algorithm. Chapter 4 provides a detailed description of the implementation of the path planning model. Moreover, the merit of the model and the Genetic 18 Algorithm as a solution is discussed in correlation with the results from the simulation experiments. Chapter 5 starts by introducing the model for the problem of UAV cooperative task assignment. It then furthers the discussion by justifying the model and solution through simulation experiment. Chapter 6 introduces the implementation of an occlusion-aware model in the urban surveillance problem. The generation of a vantage point set will later be used as the target waypoints for multiple UAVs in the coordinate task assignment problem. Chapter 7 summarizes the major findings from the thesis and points to the possible future directions of research. 19 CHAPTER 2 RESEARCH MOTIVATIONS This chapter starts by describing the motivation of conducting research in the three areas studied. The motivations concern more on the modeling and formulation of the problem. Following this, the genetic algorithm (GA) as a solution tool is discussed. 2.1 UAV Path Planning in Complex and Realistic Environment Despite much effort that has been put on the 3D UAV mission planning and some success has been achieved, there are plenty of rooms for improvement. The following observations from the previous reported works serve as the motivations for the current study: (1) Among the reported optimization methods for path planning, there is no constraint on the solution search space initially. Instead, random solutions are generated and evolved from a very large space. The constraints are added during the evolution process as penalties for unqualified individuals. Although genetic algorithms are powerful, the large initial search space still adversely affects the path planning efficiency. This study seeks to reduce the initial search space by forcing each individual in the initial generation to obey the flight constraints. (2) The previously reported methods have limitations in solving path planning problems with concave obstacles. This study attributes it to the problem in evaluating each individual solution. A new method is proposed to escape concave cases. (3) Most of the previously reported methods are “result-driven”. They are eager to achieve good results. This study, however, aims also at fully understanding the path planning 20 problem. Different parameters of the path planner are studied in detail so as to understand how they affect the solution and algorithm convergence. 2.2 UAV Cooperative Task Assignment For multiple UAV cooperative task assignment, the following issues are not fully addressed in previous literatures. (1) The efficiency of graph representation as compared to tree representation of the problem is not demonstrated in the previous literatures. This study will demonstrate through simulation experiments the merits of such. (2) The UAVs considered in previous literatures all have homogenous flight dynamics. However, this thesis considers UAVs with different flight dynamics through the application of multiple graph structures. Apart from the above two aspects, the development of a 3D display system for the solution is also a motive for the current research. This is because all the previous literatures could only represent the solution in 2D coordinates, which prevents the further development of the problem in the future. 2.3 UAV Surveillance in Complex Urban Environment The occlusion aware model is an important part of this class of problem. This is a model that checks the visible areas on the ground from a sensor positioned in the sky. In the literatures, the check for each sensor would require iteratively going through all the buildings concerned. Moreover, the ground must be flat for such model. This gives raise to motivations to develop a model that utilizes the “ray detection” function of the JME3 package. The efficiency is greatly 21 increased with the application of the “ray detection”. Moreover, the direct use of the 3D package makes the result-rendering easier. This is an advantage for result-visualization. 2.4 Genetic Algorithms as a Solution Tool Genetic algorithms (GA) are a type of stochastic and non-gradient search optimization method. It enables searching efficiently a decision state space of possible solutions, as long as the search space is not extremely rugged. As an optimization tool, GA has the following characteristics. (1) It does not require exhaustive search of the whole solution space in order to arrive at an optimal or near optimal solution. (2) It does not require the calculation of gradient for each iteration. (3) It is exceptionally suitable for solving multi-objective optimization problems. This study is motivated to use GA as a solution tool due to the nature of mission planning problems. Firstly, the solution space of this type of problems is usually infinite and discontinuous. Therefore, the exhaustive search algorithms are not suitable in this case. Secondly, it is hard to measure the improvement on solution quality through gradient, rendering the gradient descent methods unfeasible. Thirdly, there are usually multiple objective functions to be optimized, which favor the use of GA. 2.5 Java Monkey Engine (JME) as System Implementation Java is the main programming language for implementing the methods developed in this study. As an object-orientated programming language, it is simple to use. All the unnecessary complexities of the languages such as C++ are removed. Java enables the development of robust applications on multiple platforms in heterogeneous, distributed networks. Moreover, due to the 22 3D nature of such project, the package JME is used in rendering the results. The JME package has the following merits. (1) Robust terrain package makes it easy to render the terrain from data. It also has functions for height measurements and scaling. These are used in terrain collision detection. (2) Compatibility with other Java packages. (3) Plug-in of Java-swing components that allows for the programming of Graphical User Interface (GUI) with JME. 23 CHAPTER 3 INTRODUCTION TO GENETIC ALGORITHM The idea of GA comes from the natural evolution process. The famous reasoning behind the long neck length of giraffes can be used here as an illustration. Giraffes with longer necks are capable to reach higher branches of trees. They therefore have access to more abundant food supplies as compared their shorter-necked peers. Fig. 3.1 illustrates the evolution process of two generations of giraffes. The parent generation is constituted of giraffes of great diversity – long and short neck ones. The offspring of that generation are therefore also of diversity. However, due to the scarcity of food, some offspring may die out. According to the assumption – shorterneck ones have less food, the survival rate for longer-necked giraffes is much higher than that of shorter-neck ones. This step in the evolution process is called “selection”. The environment selects the most suitable individuals and therefore the most suitable genes. After generations of evolution, shorter-neck giraffes died out and only the longer-neck giraffes survived and hence their genes passed down. The GA, which is also called evolutionary algorithm, is designed according to such process. It involves five essential steps. (1) The encoding of the solution – the mathematical representation of the solution. This resembles the construction of the genes of the giraffes. (2) Initialization – randomly generates a first generation with diversity. This compares to the first generation of giraffes as shown in Fig. 3.1. The first generation should consist of solutions with diversity. 24 (3) The fitness function – the mathematical way of quantifying how suitable is the solution to the problem. In the giraffe example, giraffes with longer necks are “more fit” to the environment. They hence have a larger fitness value as compared to shorter-necked ones. (4) The genetic operators – this resemble the process of producing offspring. The common operators include crossover, mutation, elitism, and immigration. (5) The selection – this is the analogy of the natural selection process that retains the superior individuals and kills the inferior ones. Figure 3.1. The evolution process of giraffes Each iteration of the algorithm produces a new generation. The algorithm stops when certain convergence criteria are met. Thereafter, the individual with the best fitness will be retained as the final solution. 25 Although a particular GA must be designed for each specific problem. The principle mechanisms of this class of algorithms are the same. This section introduces the important components of GAs. 3.1 Encoding of the GA Chromosome The encoding is the representation of solutions by a list of data resembling that of the genes on a chromosome. As shown in Fig. 3.2, a chromosome is an organized structure of DNA and protein found in cells. It is a single piece of coiled DNA containing many genes. Each gene can be in control of a particular biological feature of the cell, and hence some features of the entire biological entity. If the chromosome and its interpretation mechanism of a biological entity are known, this entity can hence be created. Figure 3.2. Illustrations of Gene and Chromosome [45] Derived from these biological facts, in GAs, both the chromosome interpretation process and its inverse process are important. The encoding process involves designing the genes and chromosome to represent the solutions. The decoding process involves interpreting the chromosome into a solution. In some of the previously reported works, GA is widely used to solve path planning problems. Using the waypoints‟ coordinates as genes for the chromosome is a common way of solution encoding for the path planning problems. For the example path shown in Fig. 3.3a, the encoding by its waypoints is shown in Fig. 3.3b. 26 (a) Path example (b) Path encoding Figure 3.3. Example of solution encoding for path planning problems 3.2 The Initialization The initialization is a process that randomly generates a set of solutions, which is called the “first generation”. Usually, each individual solution of the first generation satisfies a number of constraints. However, the solutions of the first generation do not have to satisfy all the constraints. This is because some constraints can later be met through evolutionary process. For a well designed GA, the initialization process must be able to produce a first generation of great diversity. Yet, the production of subsequent generations from the first one must also be efficient. 3.3 The Fitness Function The fitness function is a mapping that maps the solution space into positive real numbers. It basically describes how “optimal” the solution is, given the problem setting. The fitness function design must accurately reflect the solution quality. In the giraffe problem, it is obvious that bigger fitness values should be assigned to longer-neck giraffes. However, higher dimensional and multiple objective problems do not have obvious fitness function assignment. The fitness function assignment is usually the result of trial and error. In general, a good fitness function must distinguish better solutions. 27 3.4 The Genetic Operators Although the genetic operator is problem-specific, the principles behind some important operators are the same across different problems. Four operators are discussed here: crossover, mutation, immigration, and elitism. The details of these operators will be discussed in later chapters according to the specific problems. Crossover Just as children retain the characters of both their parents, the crossover offspring chromosomes contain the characteristics of both their parents. In other words, the crossover operator swaps certain parts of two parent chromosomes to produce two offspring chromosomes. Mutation As the name suggests, the mutation operator drastically changes a single gene or a part of the chromosome, according to certain pre-defined rules. It is designed to give big changes to the solutions. Therefore, the principle application of mutation is to escape local minimums. Immigration Immigration operator completely replaces an existing chromosome with a new randomly generated one. Without immigration, the whole generation will eventually become very similar individuals which could lead to generating local minimums. With the presence of immigration operator in each generation, the diversity of the solution can be maintained. There is always some “fresh blood” in the new generation. Elitism 28 Elitism is just the operator that retains the best solution of each generation. Since the selection of offspring is probabilistic, without elitism, the best solution is not necessarily selected. Therefore, elitism makes sure that the best solution is non-decreasing. 3.5 The Selection Process The selection process is analogue to the natural selection process in the giraffe example that kills the shorter-neck ones and retains the longer-neck ones. Before selection, the fitness value for each individual in the current generation is calculated. Thereafter, there are mainly two ways to select the individuals, the roulette-wheel selection and the tournament selection. Roulette-wheel selection In this study, the fitness proportionate selection, also known as the roulette-wheel selection, is applied. In a generation of population N, let c i be the fitness of individual i in the population. The probability that such individual will be selected for the next generation is pi  ci (3.1) N c j 1 j Tournament selection Tournament selection involves running several "tournaments" among a few individuals chosen at random from the population. The tournament selection pseudo code can be written as: choose k (the tournament size) individuals from the population at random choose the best individual from pool/tournament with probability p choose the second best individual with probability p*(1-p) choose the third best individual with probability p*((1-p)^2) and so on... 29 This study applies the roulette-wheel selection for all the three problems. This is because it involves only one sorting algorithm at the beginning of each iteration, whereas tournament selection involves multiple sorting processes. Sorting algorithm is known to be an expensive type of algorithm. Therefore, choosing roulette-wheel algorithm greatly reduces the computational burden of the solution algorithm. 30 CHAPTER 4 A GA-BASED UAV PATH PLANNER IN COMPLEX REALISTIC ENVIRONMENT As a major part of this study, a UAV path planning system will be developed for complex realistic environment. The focus is mainly on the following two aspects. (1) the modeling of UAV mission planning problem in a 3D environment (2) the design of a suitable solution algorithm for the model The modeling method must satisfy the following requirements. (1) To model the terrain on which the UAV missions are to be carried out. First, the terrain model must be able to describe the real-world terrain. Second, the terrain model must be convenient to use for common data structures available online. (2) To model the UAV and its path. First, the flight dynamics particular to the aircrafts (as opposed to ground vehicles) must be taken into account. Second, the UAV paths must be modeled in a way that is simple to be encoded into the GA planner. (3) To model the threats. This takes into account the threat from enemies and the flight restrictions from the friendly commander. There are some common challenges to the three modeling objectives. They must be realistic, suitable for encoding into the GA and must not put too much burden on the computer‟s computational resources. Each optimization problem has its unique demand on solution algorithms. The following points describe the requirements particular to this study. 31 (1) The solution algorithm must be specifically designed to suit the model of the problem, as mentioned above. (2) The solution algorithm must be robust to different situations. In the experiments presented in this thesis, a same algorithm is tested with different terrain and obstacle settings to check the robustness of the algorithm. (3) The solution algorithm must be efficient even when the search space is greatly increased. The above three optimization requirements cannot be attained in one shot. Hence, repeated experiment and testing is required in order to arrive at a satisfactory model and solution algorithm. From this perspective, this study is also a learning process, in which the characteristics of the model and solution algorithm is fully explored and understood in great depth through the fine-tuning of different parameters. Such in-depth understanding of the problem will lay a solid foundation for future research when multiple UAVs and more complicated situations are involved. Therefore, an important objective is to fully understand the mechanism, the sensitivity and the characters of the model and solution algorithm through testing and experiments. 4.1 The Proposed Model for UAV Path Planner The model incorporates various components. The choice of each component in the modeling process is the result of comparison among a number of alternatives from literatures. 4.1.1 The description of terrain The environment in which the path planner works is an area of convex shape with surface of varying height. Such surface is called terrain. Every point on the terrain can be described by its position – the x, y coordinates, and the height – the z coordinate. The terrain is hence a 2D 32 function T ( x, y) . Mathematically, the area of concern A, the height map T and the terrain M can be defined as: Area: A  ℝ2 Height map: T : A  ℝ+ Terrain: M  { x, y, z  ℝ3 | x, y  A, z  T ( x, y)} Further, the space S in which UAVs can operate without collision with terrain is defined as: UAV operating space: S  { x, y, z  ℝ3 | x, y  A, z  T ( x, y)} In practice, the input terrain data is discrete, but the rendered terrain is continuous. Among those previous effort for UAV path planning, the mTSPTW [11, 12] and the Dots-on-coordinate [13-17, 26] are 2D models. Therefore, they could not encompass elevation data. The 2D Polygon-obstacles [19-26] could model the mountains as obstacles. Yet, real-world terrains are not blocks of vertically standing rocks. The parameterized artificial terrain [27-30] is not realistic. The digital elevation model (DEM) [31-36] has been chosen in this study. Firstly, geographical data is widely available in this form, allowing for easy use of real-world terrain. Secondly, it is easy to obtain the height information at any given point of the terrain. Thirdly, when a test on the parametric artificial terrain is desired, a DEM matrix can be easily generated from the parametric terrain equation and used. The raw data can be provided in two forms. The first form is a square matrix, of which each entry represents the height information at the corresponding map coordinate. The data is 33 stored in Comma-Separated Values (CSV) file format, which can be easily edited by excel. This way of data representation is straightforward, but it is difficult to visualize the terrain pattern just by observing the data structure with eyes. Also, each terrain must be normalized in order to be used effectively and comparatively by the mission planner. Another way is the 8-bit Greyscale height map (see Fig. 4.1). On the 8-bit greyscale, the different colors between black and white are divided into 256 levels, with 0 representing black and 255, white. On such height map, each data point is represented by a pixel of color on the greyscale, with black representing the minimum altitude. In Fig. 4.1a, the Hawaiian island „Oahu‟ is represented by the Greyscale height map with the actual terrain pattern in Fig. 4.1b. In this way, given a 2D coordinate (x, y) within the terrain boundary, the elevation at such point can be found easily. Fig. 4.2 shows the rendered 2D representation of the Greyscale image shown in Fig. 4.1a. The camera is placed at sea-level and heads in a horizontal direction with the highest point of the island in sight. (a) (b) Figure 4.1. Grayscale height map representation of Oahu Island Fig. 4.2 illustrates three important parameters for every terrain. (1) The base altitude – the altitude in reality represented by the black color in the height map. In the case of Oahu island, the black area in Fig. 4.1a – which represents an actual sealevel altitude – is the lowest altitude area. The base altitude in this case is hence 0. 34 (2) The vertical scale – actual increase in elevation in unit of meters represented by 1 unit increase in the grayscale. In the case of Oahu island, an increase by 1 on the grayscale represents 4.8 meters‟ increase in elevation in reality. (3) The horizontal scale – actual distance in meters represented by each resolution on the height map. In the case of Oahu island, each pixel in the height map represents a square area of 60m×60m = 3600m2. Figure 4.2. Rendered 2D representation of Oahu Island All 3 parameters for terrain are encoded in the name of the data file. For example, the file name of Fig. 4.1a is “OAHU(0_4.8_60).png”. It means that it is a Portable Network Graphics (PNG) format image which has a base altitude of 0, vertical scale of 4.8, and horizontal scale of 60. 4.1.2 The description of the UAV path The terrains used in this study are massive enough as compared to the size of the aircraft. The UAV is hence modeled as a point. The magnitude of the UAV velocity is assumed to be constant throughout each mission. Besides, the following flight dynamic constraints are imposed on the UAV. (1) Maximum Climbing Angle (MCA) – the biggest angle at which the UAV is able to climb (see Fig. 4.3a). In reality, if the UAV climbs at an excess angle, it will soon lose its speed and enter into stall. 35 (2) Minimum Gliding Angle (MGA) – the minimum angle at which the UAV is able to glide (see Fig. 4.3b). In reality, gliding lower than such angle will result in loss of control of the aircraft. (3) Minimum Turning Radius (MTR) – The minimum radius at which the UAV is able to make a turn (see Fig. 4.3c). In reality, an aircraft will lose its speed if it enters into a sharp turn greater than the maximum turning angle. (a) (b) (c) Figure 4.3. Illustration of UAV flight constraints In summary, the following factors need to be considered in modeling the UAV path. (1) The path must satisfy the constraints illustrated by Fig. 4.3. (2) It must be easy to check the path‟s condition throughout. The section of the path within the terrain boundary, inside radar zones or inside flight prohibited zones can be identified efficiently. (3) The path is a continuous curve. But the evaluation of the path has to be done on discrete points or segments. Therefore, the path must be suitable for proper numerical methods to achieve point (2). (4) The path representation and encoding must be suitable for the future use by evolutionary algorithms. 36 In some of the previous research effort [27, 29, 30], the path is represented by B-spline curves. It is then divided into equal segments to check for their collision with terrain and flight constraint violations. This approach requires intensive computations, due to the large search space without flight constraint and the calculation for each segment. Another way is to represent the path by a single Bezier curve [31]. As discussed in the literature review, this is not a good choice since local change in the curve will result in change in all the control points. In this study, the UAV path is modeled as a piecewise C2 continuous cubic Bezier interpolation of a series of consecutive waypoints. The terminal points are the starting and ending position of the UAV. The curve between any two waypoints is a single cubic Bezier curve. For a path with N+1 waypoints (including the starting and ending points), there are N segments of cubic Bezier curves. Hence, the problem of finding a path has changed to finding the intermediate waypoints and then interpolates them. The merits of such formulation are. (1) A Bezier curve automatically has its first and last control points the same as the two waypoints that it connects. In the case of cubic Bezier, only 2 control points remains to be found. This greatly reduces the computational burden as compared to using B-spline. (2) When the intermediate points are randomly generated, it is ensured that the path does not violate the flight constraint by checking the properties at the two consecutive points. This greatly reduces search space and increases accuracy of the path for the solution algorithm. Fig. 4.4 shows the mechanism of piecewise cubic Bezier interpolation through a simple example where only the x and y coordinates are shown. Fig. 4.4a shows a path found by the mission planner from sPt to ePt, through finding two intermediate waypoints. The actual rendered path in the mission planner is shown in Fig. 4.4b. The four points are interpolated by three piecewise cubic Bezier curves, C2 continuous at the two waypoints. Fig.4.2a shows the 37 terminal points, waypoints, Bezier curves and control points in a Cartesian frame. Section k (1, 2 k k or 3) of the entire path is described by its four control points: C 0k , C1 , C 2 and C 3k by: 3  3 Bk (t )    (1  t ) 3i t i Cik i 0  i  (4.1) The data for the points describing the curve in Fig. 4.4 is shown in Table 4.1. (a) (b) Figure 4.4. Cubic Bezier interpolation Table 4.1 Parameters defining the piecewise cube Bezier curve Control Points C 01 (sPt) x 50 y 50 C11 50 48.7 52.1 44.5 54.7 42.2 C 57.3 39.9 C 60.2 39.2 63.8 38.2 C 67.4 37.1 C 71.6 35.7 72 36 C 1 3 1 2 2 0 C , C , (waypoint 1) 2 1 2 2 2 3 3 0 C , C , (waypoint 2) 3 3 3 1 3 21 C , (ePt) 38 4.1.3 The description of threats Section 1.3.1 describes past work of modeling radar zones as simple circular regions [32, 36], and flight prohibited zones (FPZ) as shaded area on 2D plan [19-23]. These two representation schemes do not well reflect the realistic situation. The two types of model in this study are described as follows. (1) FPZ – They are 3D rectangular volumes defined by the user, these regions are not to be visited by the UAV. On the other hand, flight ceiling, commercial aircraft flight zones and foreign country‟s territories are examples of FPZ. Each FPZ is defined by its center C x , C y , C z and x, y, z extents E x , E y and E z as: Center: C x , C y , C z  ℝ3 Extents: E x  ℝ+, E y  ℝ+, E z  ℝ+ FPZ: FPZ  { x, y, z  ℝ3 | x [C x  E x , C x  E x ], y [C y  E y , C y  E y ], z [Cz  Ez , Cz  Ez ]} While a single FPZ thus defined is of simple rectangular shape, the combination of FPZs can form complex concave regions. (2) Enemy Radar – They are spherical volumes whose centers are on the terrain. Enemy radars pose a threat to the UAV because they disturb the mission secrecy and for those radar site equipped with missiles, possible destruction of UAV might be induced. However, instead of modeling the radar zones as areas to be avoided, the probability of detection is measured which depends on the distance from the radar centre, such model is 39 hence closer to the real radars. Since radars are always positioned on terrain surface, the radar center CR , influence radius R and the radar zone Ra are defined as: Radar center: CR  S Radar radius: r  ℝ+   Radar zone: Ra  {x  ℝ3 | CR  x  r} The norm denotes the Euclidean distance. Fig. 4.5 shows an example of FPZ, radar and UAV path in a mission planner setting. The Oahu island height map in Fig. 4.1a is used. The height equal or below zero is marked with blue color. The color changes from green to yellow and eventually red as elevation increases. A simple mission is designed in which a UAV is approaching the Pearl Harbor from the north. Its path is actually characterized by many waypoints. Between the UAV and the Pearl Harbor are enemy radar zone on the left (black transparent), and a FPZ on the right (red transparent). Figure 4.5. An illustrative example of mission planner over Oahu Island, Hawaii 40 4.2 The Design of Genetic Algorithm As described in Chapter 2, the genetic algorithm must be designed to suit the particular problem considered. This section illustrates the design of the genetic algorithm for the problem of path planning in realistic environment. 4.2.1 The encoding of solution The solution encoding is done by representing the solutions with gene-like structures. As explained in Chapter 3, a gene is a sequence of chromosomes which describes the features of the solutions. For this problem, each gene of a given chromosome is the 3D coordinate of a waypoint on the path. A chromosome is an array of such 3D points, with its first gene corresponding to the starting point (sPt) and the last gene the destination point (ePt) of the UAV. For example, Fig. 4.6 shows the chromosome representing the path in Fig. 4.4. For illustration purpose, 2D coordinate is used. In the mission planner, instead, 3D coordinate is applied. Gene 0 (sPt) Gene 1 Gene 2 Gene 3 (ePt) (50.0, 50.0) (54.7, 42.2) (63.8, 38.2) (72.0, 36.0) Figure 4.6. A Chromosome with two intermediate waypoints The problem of finding an optimal path is thus changed to finding the intermediate waypoints to connect (sPt) and (ePt) and interpolate them. 4.2.2 Initialization For the first generation of solutions, every individual is randomly generated with the naissance operator. For later generations, there is a fixed portion of newly generated immigrants which help retain the diversity of the population. The mechanism for randomly generating a single individual is explained in the following section. 41 The UAV flight constraints, (sPt) and (ePt) are given before the iterative method. If the starting direction is not given, a random starting direction (sDir) within the flight constraint will be used. The distance between two consecutive waypoints (d) is defined by the user. All positional and directional variables in this method are in 3D. The iterative method is described as follows. (1) The variable „current waypoint‟ (currtPt) is set to be the point in the direction of (sDir) and a distance (d) away from (sPt). The variable „previous waypoint‟ (prevPt) is set to be the starting point. (2) The variable „previous direction‟ (prevDir) is defined as the direction from (prevPt) to (currtPt). The directional limits of the next direction (nextDir) can then be worked out according to flight constraints MTR and (prevDir) (3) A (nextDir) is randomly generated direction within the limit worked out in Step (2). (4) The (prevPt) is updated to the old (currtPt). Subsequently, the new (currtPt) is updated as a point in the direction of (nextDir) from the old (currtPt) and a distance (d) from the old (currtPt). (5) Steps (2) – (4) are repeated until the (ePt) can be visited in the next iteration. The direction generation random process in Step (3) must follow a probability distribution that is more likely to produce directions in line with (ePt) and less likely to produce directions away from the direction of (ePt). If this is not the case, the random individual will be extremely hard to find. Because in that case, no force is pushing the UAV towards (ePt) and the path will keep wandering about without getting close to (ePt) purposely. 42 4.2.3 The fitness function In all the references that apply GAs, the path is divided into small sections using numerical methods and the total fitness is an aggregation of each section. In this study, the fitness of a chromosome is an aggregate description of the fitness of each of its genes. For a chromosome of N genes, let f i be the fitness value of gene-i. The fitness value for the whole chromosome, denoted as F, is calculated as: F 10000 N ( f i )  d  ( N  1) (4.2) i 1 Further, a large value of F is defined to indicate better solution. Hence, large value of f i penalizes gene-i and small value is desirable. The term d  N penalizes path length. For a single gene, there are two ways to calculate f i , static and dynamic. For either method, f i is an aggregation of the fitness values in Tables 4.2 or 4.3. That is, for a point i that is both under terrain and Inside FPZ, f i is the sum of the fitness value for being under terrain and for being inside FPZ. In most literatures on genetic algorithms, violation of constraints is treated in a constant way [28, 29, 31, 35]. This is called “static fitness value assignment” in this study. Static fitness value assignment Static fitness values are constant fitness value assignments that define a single fitness value for a particular region in the space. There are in total 5 penalty fitness multipliers defined for this study. 43 f OM - Penalty value for sections outside the map boundary fUT - Penalty value for sections under the terrain f FPZ - Penalty value for sections inside the FPZ f radar - Penalty value for sections inside radar f AT - Penalty value for sections not violating any constraints, which is very small. Table 4.2 shows the static fitness value assignment particularly for this study. Table 4.2. Static fitness value assignment Situation Outside Map Under Terrain Inside FPZ Inside Radar Above Terrain Fitness Value d  f OM d  fUT d  f FPZ d  f radar d  f AT Take the path shown in Fig. 4.4a as an example. Suppose that. (1) waypoint 1 is inside a FPZ but does not violate any other constraint, and (2) all other waypoints, sPt and ePt do not violate any constraint, and (3) f FPZ  5, d  10, f AT  1 Then, the path has a total fitness value of: F 1000 N ( f i )  d  ( N  1)  (10  f AT  10  f FPZ 1000  9.10  10  f AT  10  f AT )  10  (4  1) i 1 (4.3) The path has a total fitness value of 9.10. Further, suppose that the path in Fig. 4.4 does not violate any constraint, its fitness value is 14.29 according to Eq. (4.2). 44 F 1000  14.29 (10  1)  4  10  (4  1) (4.4) It is clear that the path is penalized if it has one waypoint inside the FPZ. Dynamic fitness value assignment While most literatures adopt the static fitness assignment, this study proposes a dynamic way of fitness value assignment. Before introducing the dynamic way of assigning fitness values, the following variables are defined for a given waypoint. (1) Dvt – Vertical distance between the waypoint and terrain: positive value when the point is above the terrain, negative otherwise. (2) Dht – Horizontal distance between the waypoint and closest terrain boundary, it equals 0 if the waypoint is within the terrain boundary, non-zero otherwise. (3) Df(i) – Distance between the waypoint and centre of the ith FPZ, zero if the waypoint is not in the ith FPZ. (4) Ef(i) – Extent of the ith FPZ. It is the largest value of x, y and z extents of the ith FPZ. (5) Dr(i) - The distance from the centre of the ith radar. It equals zero if the waypoint is not within the influence area of radar i (6) N FPZ - The total number of FPZs (7) N radar - The total number of radars Table 4.3 shows the dynamic fitness value assignment of the solution individuals. As compared to the static fitness assignment, the dynamic method takes into account the relative distance of 45 violation between a waypoint and constraints. When there is a violation of constraint, not only is the particular gene penalized for having violation, but also penalized according to how “serious” such violation is. In other words, a waypoint deeply inside a FPZ is going to be penalized more seriously than a waypoint inside the same FPZ but is closer to the boundary. Table 4.3. Dynamic fitness value assignment Situation Outside the Map Under Terrain Fitness Value d  f OM  Dht d  fUT  abs(Dvt) Inside Radar Above Terrain without collision Situation N radar Fitness Value  (d  f i 1 radar  Dr(i )) Inside FPZ N FPZ  (d  f i 1 FPZ  ( Ef (i)  Df (i))) d  f AT  abs(Dvt) For illustration, a similar example to that in the static fitness value assignment is given here. Take the path in Fig. 4.4 as an example, suppose that. (1) waypoint 1 is inside a FPZ. As an example, the maximum extent of the FPZ is set to be 20 ( Ef (1)  20 ), waypoint 1 is x unit of distance from the FPZ centre ( Df (1)  x ). Waypoint 1 does not violate any other constraint, and (2) all other waypoints, sPt and ePt do not violate any constraint, and (3) all the waypoints are 10 units above the terrain ( Dvt  10 ), and (4) f FPZ  5, d  10, f AT  1 For the case that x  5 : 46 F (10  f AT  Dvt  10  f FPZ 1000  0.93 (4.5)  ( Ef (1)  Df (1))  10  f AT  Dvt  10  f AT  Dvt)  10  (4  1) For the case that x  10 (closer to the FPZ boundary): F (10  f AT  Dvt  10  f FPZ 1000  1.20 (4.6)  ( Ef (1)  Df (1))  10  f AT  Dvt  10  f AT  Dvt)  10  (4  1) A simple comparison between equations (4.5) and (4.6) shows that, for a path that has a waypoint inside the FPZ, the further away it is from the centre of the FPZ (i.e. closer to the boundary), the bigger the fitness value of the whole path. Similar conclusions can be drawn for points under terrain and inside radar zone. From equations (4.5) and (4.6), it can be noted that the fitness value is inversely affected by Dvt , the distance from terrain of each point. This models the fact that lower flying altitude of the UAV is more desirable. This is because the terrain masking effect facilitates radar evasion, hence maintaining the mission secrecy. 4.2.4 The selection mechanism The selection mechanism follows that of roulette-wheel selection described in Section 3.5. The preference of roulette-wheel selection over tournament is purely based on computational efficiency concerns. 4.2.5 The genetic operators There are in total 5 operators designed particularly to suit the UAV path planning problem. Their usefulness will be fully studied by experimental study. However, their mechanisms are explained in detail here. Crossover 47 A child inherits the characteristics of both his parents. The two offspring produced by crossing-over two individuals have common parts from both sides. Fig. 4.7 illustrates the crossover operator. C1 and C2 are the parent chromosomes, with X, Y and a, b the respective crossover points randomly chosen. ci and cj are the offspring after cross-over. The cross-over retains and swaps the head and tail genes of C1 and C2. The middle gap in blue is generated using the same mechanism as naissance for a single gene. The two offspring contain characteristics of both their parents. Figure 4.7. The cross-over operator Local mutation Fig. 4.8 shows the mechanism of a local mutation operator. A mutation point X is randomly chosen on a given chromosome. The mutation operator will retain its structure for genes before X. For genes after X, a new series of random genes (blue) are generated. This operator is expected to gradually lead the solutions out of local minimums. Figure 4.8. The local mutation operator Strong mutation Unlike local mutations, strong mutation has the capability to escape a local minimum in just one generation. As illustrated in Fig. 4.9, two random mutation points X and Y are chosen. A third point Z within the terrain boundary is randomly generated and the section between X and Z 48 is changed to random genes such that the middle section (blue) pass the intermediate point Z. This operator is extremely useful in getting around difficult obstacles. Figure 4.9. The strong mutation operator Deletion/smoothing The deletion operator has the purpose of smoothing a path that has undesirable turnings of irregularities. As illustrated by Fig. 4.10, it is a simple deletion of a gene Z. Figure 4.10. The deletion operator For each chromosome in a given generation, there is a certain probability that such chromosome is going to undergo one of the above 5 operations in the evolution process. The five probabilities are defined respectively in Table 4.4. Table 4.4. Symbols for probability of each GA operator Symbol Probability represented PIM immigration PXO cross-over PLM local mutation PSM strong mutation PDel deletion/smoothing 49 4.3 Simulation Experiments A large number of simulation experiments have been conducted with different variables. They can be classified into the following three categories. (1) Experiments on different fitness functions (2) Experiments on different GA operators (3) Experiments on different UAV and UAV path characters For each category, tests are undertaken under several different terrain scenarios. 4.3.1 Experiments on different fitness functions This section examines the pros and cons of the static and dynamic fitness value assignment through simulation experiments. For these experiments, the variables shown in Table 4.5 are controlled to be constants, while variables concerning the fitness functions are varied. Table 4.5. Controlled constants for experiments on fitness function sPt ePt sdir edir MaxTurnRad MaxClimbAngle MinGlidingAngle (50, 50) (400,400) (0,0,-1) (1,0,1) π/4 π/4 - π/4 mindistance d PIM PXO PLM PSM PDel 3 10 0.05 0.05 0.05 0.05 0.05 Two maps are used for the testing: a flat map for the radars/FPZ – evading testing and a rough terrain map for terrain evading testing. In both cases, the (sPt) and (ePt) are both 10 units above terrain. Tests with Static fitness functions Tests 1&2 have the same magnitude of penalty fitness multiplier throughout. The fitness value for above terrain is 1. The values in Table 4.6 are adopted, in which GenSize defines the 50 size of each generation. No terrain is used in Figure 4.11a, b and c. An easy, relatively flat but rough terrain is used in Figure 4.11d. Table 4.6. Homogeneous assignment of fitness multipliers f OM fUT f FPZ f radar f AT GenSize 10 10 10 10 1 1000 (a) (b) (c) (d) Figure 4.11. Test 1: simple obstacle evasion test for static fitness 51 The GA with the static fitness value assignment managed to find a valid path in all the four cases shown in Fig. 4.11. In Fig. 4.11a, a simple valid path is found in a case without obstacles. Figs. 4.11b and 4.11c show the path avoiding a FPZ and enemy radar, respectively. Fig. 4.11d shows the path not colliding with a rough terrain. However, observing that both the fitness value for the best and the generation average did not evolve much in 100 iterations (the flat profile in the charts), it is reasonable to assume that the success of Test 1 is due to the lack of difficulty in the problem. Moreover, the flat profile of fitness values makes it hard to define a convergence criterion to make the algorithm to a halt, when manual stopping is not desirable. Test 2 involves difficult terrain patterns and concave FPZ formation. Fig. 4.12 illustrates that the static fitness assignment is not robust for concavity in obstacles. In Fig. 4.12a, the bottom-right corner is a V-shape concave FPZ formed by two connected rectangular FPZs, behind which the destination is hidden. A flat terrain is used so that the only obstacles come from the FPZs. Mission planner finds a path through the concave FPZs rather than around them after more than 100 iterations. In Fig. 4.12b, the destination is at the bottom, which can only be reached after flying over the mountain. The planner finds a path through the mountain, rather than over it. Hence, the planner has failed in both cases. It means that the static fitness is not robust for either vertical (such as FPZs) or horizontal (high mountains) concavity. (a) (b) Figure 4.12. Test 2: concave obstacle evasion test for static fitness 52 Such testing results can be interpreted by the fact of static fitness value assignment. Suppose the area encircled by the red lines in Fig. 4.13 is the FPZ and a valid solution goes around it. Intuitively, the solution in Fig. 4.13b is better than that of Fig. 4.13a due to its vicinity to the valid solution. However, both solutions have the same static fitness value because they are both inside the FPZ. Hence, the solution on the right was not preferred during the selection process of the GA. Therefore, in order to escape concavities, the two solutions in Fig. 4.13 must be differentiated by means of the dynamic fitness functions. (a) (b) Figure 4.13. Different solutions with same static fitness value Tests with Dynamic fitness functions The dynamic fitness function assignment is defined by Table 4.3. Test 3 is conducted with all the fitness multiplier having the same magnitude as in Table 4.6. Test 3 is conducted in the same terrain and FPZ setting as in Test 2. The result of Test 3 is shown in Fig. 4.14. As expected, the dynamic fitness values have led the UAV path around the concave objects. Moreover, the fitness value plot for both the best and average has an increasing tendency as a larger number of iterations occurred and has a tendency to converge after 100 and 200 iterations, respectively. 53 (a) (b) Figure 4.14. Test 3: concave obstacle evasion test for dynamic fitness A remaining shortcoming of the dynamic fitness function assignment as in Table 4.3 is that the penalty value for any constraint violation is only 10 times greater than the normal value. This does not give enough positive incentives for the paths that really escaped the obstacles. According to the relative importance of the constraints, Table 4.7 shows the fitness multiplier that is adopted for all the experiments in the following section. Table 4.7. Final assignment of fitness multipliers f OM fUT f FPZ f radar f AT GenSize 100000 1000 100 100 1 1000 The reason for the adoption of Table 4.7 is quite straight forward. It is most serious for a path to go out of the map boundary, because there is unpredictability out of the scope of path planner. It is also serious for a path to go under terrain which results in collision. It is less serious for a path to go into a FPZ or radar detection zone, which puts the aircraft in potential danger but not necessarily leads to destruction. In some cases, the UAV may have to go through a radar area so as to avoid terrain collision. 54 4.3.2 Experiments on functions of different GA operators In order to study the power of each GA operator, the experiments started with the random generator process of naissance and added more GA operators one by one. Table 4.8 shows the control variables. Table 4.8. Controlled variables for experiments on GA operators (i) sPt ePt sdir edir MaxTurnRad MaxClimbAngle MinGlidingAngle (50, 50) (400,400) (0,0,-1) (1,0,1) π/4 π/4 - π/4 mindistance d f OM fUT f FPZ f radar f AT 3 10 100000 1000 100 100 1 Random generator – immigration/naissance Immigration is a random process that links the starting point and then end point of the path planner through finding the intermediate waypoints and interpolating them with piecewise cubic Bezier curves. In this case, the probabilities are set as in Table 4.9. The test result is given in Fig. 4.15 in which Fig. 4.15b shows that the situation is a simple FPZ block (red) that goes between the starting (black) and ending position (grey). Table 4.9. GA operator probabilities for immigration testing PIM PXO PLM PSM PDel 1 0 0 0 0 55 (a) (b) Figure 4.15. Test 4: fitness plot for random generation According to Fig. 4.15a, the average fitness value for all 175 random generations remains almost the same. This shows that the random generator is stable for each generation by having consistent average fitness values. For the best fitness, from time to time, it has some lucky shot, but such lucky ones are not retained overtime. Hence, the following conclusions can be drawn for the immigration/naissance operator. (1) From a collective perspective, the immigration operator is consistent in that each random generation has the same average fitness function. (2) Yet, the high volatility in the fitness of the best solution shows that the random process generates a population with diversity. (3) The non-persistence of the peaks in Fig. 4.15a shows that without the selection mechanism, there is no way to keep the good solutions. (ii) Cross-over The probabilities in Table 4.10 are adopted for the testing of the cross-over operator. The first generation relies on the random process studied previously. Each individual in all the following generations have 10% probability to go through a cross-over with another individual in 56 the same population. The cross over points on either individual is a uniformly distributed random variable. The situations to be tested are shown in Figs. 4.16b and 4.16d, respectively. Table 4.10. GA operator probabilities for cross-over testing PIM PXO PLM PSM PDel 0 0.1 0 0 0 (a) (b) (c) (d) Figure 4.16. Test 5: fitness plot for cross-over operator Looking as the testing results shown in Figs. 4.16a and 4.16c, respectively, the following observations can be concluded for the power of cross-over operator in its own right. (1) It inherits the merits of ancestors and leads the population towards a greater aggregate fitness when the situation is not concave. In fact, in the situation shown in Fig. 4.16b, the planner did indeed find a way around the FPZ. 57 (2) However, for the concave situations shown in Fig. 4.16d, neither the generation fitness nor the best fitness has any improvement over time. (iii) Cross-over and local/strong mutation Figs. 4.16c and 4.16d do not show any satisfactory solutions for concave cases. This is why the mutation operator is introduced. The local and strong mutation operators are studied in a comparative way. For the tests on local mutation, the probabilities in Table 4.11 are used. Similarly, for the local and strong mutation, the probability values shown in Table 4.12 are used. Table 4.11. GA operator probabilities for local mutation testing PIM PXO PLM PSM PDel 0 0.05 0.05 0 0 Table 4.12. GA operator probabilities for local and strong mutation testing PIM PXO PLM PSM PDel 0 0.05 0.05 0.05 0 The comparative test result is shown in Fig. 4.17 and the following characteristics can be observed. (1) Both the local and strong mutation had been useful in making the path planner escaping the local minimum created by the concave FPZ. Hence, the test shows the usefulness of the mutation operator as a good tool in escaping the local minimum of the optimization objective. This is in line with the characteristics of such operator described in literature. (2) The strong mutation is much more effective in making “big jumps” as compared to the local mutation. As shown in Fig. 4.17a for the local mutation and 4.17c for both local and strong mutation, the fitness values reach the regime of magnitude of 0.01 between 30 and 58 40 iterations for local mutation operator. Whereas, the fitness values reach the same regime in just 5 to 10 iterations. The test with strong mutation converges after 60 iterations. However, the test with local mutation does not exhibit an evident sign of convergence even after 110 iterations. (3) On the other hand, a steady increase in fitness values can be observed for the test with only local mutation. For the test with both local and strong mutation, there are several stages of different fitness increment velocities. In summary, the mutation operator is useful for the GA due to its effectiveness in evading local minimums. (a) (b) (c) (d) Figure 4.17. Test 6: comparative study of local and strong mutation operators 59 (iv) Deletion/smoothening operator Fig. 4.18 shows the optimization result of a simple planner on a flat terrain without obstacles with cross-over, local and strong mutation operators. The same result is shown from 3 different perspectives. An obvious drawback of such results is the irregularities and fluctuations in the path. Such fluctuations are direct results of the diversity created by the random path generator. However, in practical applications, such fluctuations are to be avoided for stability issues in a real flight. Hence, the smoothing operator is applied. Test 8 is conducted using the probabilities shown in Table 4.13 and the result is shown in Fig. 4.19. Figure 4.18 – Test 7: simple test without deletion/smoothening Table 4.13. GA operator probabilities for deletion/smoothening testing PIM PXO PLM PSM PDel 0 0.05 0.05 0.05 0.05 Figure 4.19. Test 8: simple test with deletion/smoothing operator 60 A direct qualitative comparison between Figs. 4.18 and 4.19 shows the power of the smoothening operator. The irregularities and fluctuations in the path are greatly reduced after the application of the smoothening operator. (v) Argument for immigration in each generation While it is hard to demonstrate the power of immigration operator through experiment, the argument is relatively obvious. Without the immigration operator, only the first generation is a random one. All the following generations are constituted of either the inherited individuals from the previous generation or the offspring of crossover and mutation. Of course, when the GA design is correct, the generations are heading towards better solutions. However, without immigration into each generation that maintains the diversity, it is very hard to pull back the generations heading towards a local minimum. For example, in the situation of Test 4 (see Fig. 4.15b), if the generations are heading towards the solution that goes around the FPZ from right, the immigration operator will keep the door open to the solutions around the left of the FPZ. 4.3.3 Experiments on magnitude of probabilities for the GA operators So far, the usefulness of each GA operator is demonstrated. It remains to determine the magnitude of the GA probabilities. Experiments have therefore been conducted with the mountainous setting of Test 3 (see Fig. 4.14b). Several combinations of GA probabilities were tested and their best and average fitness were recorded at 50 iterations (see Table 4.14). By comparing the average fitness across 5 cases, table 4.14 shows that choosing 0.05 or 0.1 as the probability for each GA operator results in the best optimization performance after 50 iterations. Greater value of GA probabilities only resulted in delaying the optimization progress. With the choice of probability of 0.2, in average, 100% of the individuals of each generation go 61 through at least one of the five GA operators, hence the good individuals from the past generation are no way to be retained. The final probabilities for the GA operators are going to adopt the values in Table 4.15. This is because the algorithm performs equally well for uniform probabilities of both 0.05 and 0.1. Yet, the choice of probability of 0.05 will leave in average 75% of the individuals of each generation unchanged. This allows for a gradual convergence of the optimization algorithm and avoids big jumps in the average fitness value of the generation. Table 4.14 – Fitness values after 50 iterations with different GA probabilities PIM PXO PLM PSM PDel Average Fitness Best Fitness 0.05 0.05 0.05 0.05 0.05 0.023 0.043 0.1 0.1 0.1 0.1 0.1 0.025 0.052 0.2 0.2 0.2 0.2 0.2 0.015 0.062 0.3 0.3 0.3 0.3 0.3 0.005 0.039 0.05 0.1 0.15 0.2 0.25 0.01 0.0325 Table 4.15 – Final adopted GA operator probabilities PIM PXO PLM PSM PDel 0.05 0.05 0.05 0.05 0.05 4.3.4 Experiments with different UAV and UAV path characters The variables MaxTurnRad, MaxClimbAngle, MinGlidingAngle, mindistance and d are to be studied in this section. Although the determination of the first 4 should depend on the actual aircraft modeled, it is worth studying how the changes in these variables affect the result. Different UAV characters Two tests with different UAV maneuverability have been conducted. The parameters for Test 9 (see Table 4.16) are considered aggressive for an aircraft. And, the parameters for Test 10 (see Table 4.17) are relatively conservative for an aircraft. The quantitative and qualitative 62 results are shown in Figs. 4.20 and 4.21, respectively. In Figs b, c, d and e of Figs. 4.20 and 4.21, the same terrain is viewed from the perspectives of top, left, bottom and right. Table 4.16. Aggressive UAV maneuverability MaxTurnRad MaxClimbAngle MinGlidingAngle mindistance π/4 π/4 π/4 3 d 10 Table 4.17. Conservative UAV maneuverability MaxTurnRad MaxClimbAngle MinGlidingAngle mindistance d π/6 π/6 π/6 3 10 63 (a) (b) (c) (d) (e) Figure 4.20. Test 9: aggressive UAV maneuverability 64 (a) (b) (c) (d) (e) Figure 4.21. Test 10: conservative UAV maneuverability Firstly, the aggressive test converged before 150 iterations, whereas the conservative test converged after 200 iterations. Secondly, the fitness values for the aggressive test reach material levels much earlier than in the conservative test. The reason for such quantitative results is that the more stringent the flight constraints, the smaller the valid search space without constraint 65 violation (in this case, the constraints are terrain only). Since the GAs work in a way that searches both the valid and non-valid spaces, the convergence is hence delayed for stricter flight constraints. Qualitatively, the conservative path had no choice but to go around the high-raising mountain in the middle, yet the aggressive path is able to somehow climb over the mountain – an evidence of larger search space for the aggressive case. In both cases, the path has a tendency to stick close to the terrain, which is in accordance with the setting of the fitness function. In the following section, the aggressive UAV maneuverability is applied and different values of the distance between any two consecutive points are tested. Different UAV path characters The tests in Test 11 were conducted with the configurations shown in Table 4.18, and the testing results in terms of fitness values are shown in Fig. 4.22. The following observations are made. (1) For all values of d, the convergence consistently occurs at 200 – 225 iterations. Hence, the magnitude of d does not affect the convergence of the GA. (2) With larger values of d, it took less time for the computation of each generation. This is because the total number of waypoints required is smaller for larger d, hence taking up less memory. (3) Too large a d is not desirable. Since the sections between any two waypoints are not checked against collision, the unlikely collision between two waypoints is therefore not penalized. Large value of d will increase the chance of happening of such misjudgment. This is especially the case when turning at sharp corners. 66 Table 4.18. Different distances between two consecutive waypoints MaxTurnRad MaxClimbAngle MinGlidingAngle mindistance d π/4 π/4 π/4 3 15, 20, 25, 30 (a) (b) (c) (d) Figure 4.22. Test 11: fitness evolution for different values of d 4.4 Discussions The aim of the study on UAV path planning is not merely to find the best solution or the most efficient path planning algorithm. Moreover, it is a thorough study of the single UAV path planning problem that seeks to understand how different fitness functions, different GA operators and different UAV flight constraints affect the path planning process. In the end, according to this study, a high-quality combination of parameters and GA operators is proposed. 67 According to the tests conducted in this study, constant assignment of fitness values for each gene is able to solve simple problems without any concavity. However, constant values differentiate valid solutions from non-valid ones but do not differentiate among the invalid solutions. As a result, it is hard to get around concave terrain and obstacle structures. On the other hand, dynamic assignment of fitness values differentiates among the invalid solutions and could tell those solutions that are “close” to the valid solutions, hence moving out of the concavity gradually by means of the selection mechanism of the developed GA. The random generator for each generation is a process that provides diversity. Cross-over is useful in combining the good characteristics of different individuals in previous generation. Local and strong mutations are important operators that help the solution evade local minimum. Deletion is a smoothing operator on the path that reduces unnecessary fluctuations. All of the five GA operators are indispensible for the path planner. The computation speed of the path planner is reduced with more conservative UAV maneuverability. Also, more generations are required before the convergence for conservative UAV characteristics. The change in the distance between any two consecutive waypoints does not affect the GA planner; however, a long distance may result in judgment errors especially at sharp obstacle corners. In summary, Chapter 4 starts by modeling the single UAV path planning problem and designing a GA to solve it. Experiments have been conducted diligently for different fitness functions, GA operators, and UAV maneuverability. As a result, a thorough understanding of the problem is achieved. Chapter 4 will also provide a solid foundation for numerous future extensions, which are to be presented in the following sections. 68 CHAPTER 5 A GENETIC ALGORITHM FOR UAV COOPERATIVE TASK ASSIGNMENT In addition to the single UAV path planning problem, another important aspect of the UAV mission planning problem involves the coordination among multiple UAVs to achieve certain objectives. In this chapter, this particular problem is clearly defined and modeled. Effective solution algorithms have also been proposed and implemented. Simulation experiments have been conducted to show the effectiveness of the developed methods. 5.1 The Problem Description The UAV mission planning is going to take place in a 3D space represented by Cartesian coordinates. Mathematically, a horizontal plane H h (c) is the set defined by H h (c)  { x, y, z  ℝ3 | z  c} , where c is a constant. The timeframe for the mission T is the subset of positive real number. T  ℝ+  {0} . Therefore, a specific time t during the mission is t  T . A UAV characteristics is a set {l , Rmin } , where l  ℝ+ denotes the UAV cruising altitude and Rmin  ℝ+ denotes the UAV minimum turning radius. The UAV velocity is not defined here, since all the UAVs are assumed to have the same velocity. With such definitions, a UAV Path Pl for a UAV flying at altitude l is a subset of H h (l ) : Pl  H h (l ) . The following properties exist for a valid UAV Path: Property 1: There exists a mapping Pl (t ) : T  Pl such that: a. Pl (t ) is two times differentiable with respect to t in both x and y directions. This is the smoothness condition for the UAV paths. 69 b. 1 '' Pl (t )  Rmin . This is the minimum turning radius constraint on the UAV path. The norm applied here is the Euclidian distance. Property 2 (Dubin‟s Vehicle): Pl is constituted of segments of straight lines and arcs of Rmin . The targets are dots on the plane H h (0) . Predefined tasks are assigned for each target. The UAVs are to fly over these targets to complete the tasks. 5.2 Dubins Vehicle As discussed in Chapter 1, the building brick of this class of cooperative mission planning problem is Dubins Vehicle. It assigns a minimum-length path for given starting and terminal positions u, v and starting and terminal directions U, V (see Fig. 5.1a). Although there have been well-known algorithms for constructing Dubins path, such as MultiUAV or MultiUAV2, it is still not convenient and efficient to use them in the Java environment [46]. In this study, the algorithm for the calculation of Dubins‟ path is developed in the Java environment. This problem can be formulated as proposed by Dubins [40]. Fig. 5.1b starts by constructing two tangent circles for u and v respectively. The radius R of the circles equals to the minimum turning radius of the UAV. The circles are directional. Each circle‟s direction is defined in such a way that its direction at the starting (ending) point is collinear with the starting (ending) direction. The purpose of the circle construction can be illustrated by Fig. 5.2, in which a feasible minimum-length path can be found [40]. It is highlighted by a thick line. 70 (a) Specified end conditions (b) The directional circles tangent to the end conditions Figure 5.1. The minimum-length path problem Figure 5.2. Illustration of a feasible path According to [40], the minimum-length path is one of the 6 cases, shown in Figs. 5.3 and 5.4, respectively. Each case is constituted of 3 piecewise continuous arcs. Denoting “R” as a right turn, “L” as a left turn and “S” as going straight, a path can be represented by 3 letters. For example, RSL means that the UAV starts by turning right, followed by a straight line segment and finishes by turning left. With such notation, the set of 6 feasible paths can be defined as {RSL, RSR, LSL, LSR, RLR, LRL} . For example, Fig. 5.2 shows the case of RLR as a solution. It can be seen that the starting and ending arcs must be a turn, whereas the arc in the middle can be a straight line or a turn. Further, a feasible path has the following variables: (i)  1 : the angle turned for the first arc, (ii)  2 : the angle turned for the second arc (in the case that the second arc 71 is not a straight line), (iii) t 2 : the length of the second part (in the case that it is a straight line), and (iv)  3 : the angle turned for the third arc. The ranges of these 4 variables are given as: 1  [0,2 )   [ ,2 )  2  t 2  [0,)  3  [0,2 ) (5.1) Fig. 5.2 illustrates a case of RLR and the 4 variables. In the case that the Euclidian distance l between the starting and ending points satisfies l  4R , only the 4 cases {RSL, RSR, LSL, LSR} , as shown in Fig. 5.3, are possible. RSL RSR LSL LSR Figure 5.3. Dubins‟ path is one of the four possible choices when l  4R When l  4R , two additional cases, i.e., RLR and LRL, are possible. LRL RLR Figure 5.4. Two additional choices of Dubins path when l  4R 72 After all the possible paths are found analytically for given starting and ending conditions, the minimum-length path between the starting and ending positions can be found by enumerating all the possibilities. 5.3 Graph Representation The weaknesses of the network flow model and the tree representation was discussed in Chapter 1. This section first explains the computation complexity of the graph representation. Following that, the graph implementation is discussed in detail. The example task shown in Fig. 5.5 will be used throughout this section for illustration purpose. In this example, there are 2 targets, each to be visited 3 times and 2 UAVs with specified starting positions and directions. Figure 5.5. An illustrative example of a mission involving two UAVs and two targets 5.3.1 Discrete UAV heading angle on target In order to implement the graph representation of the problem, the UAV headings on each target are discretized. Fig. 5.6 illustrates the discrete headings of UAV. In this case, 360 degrees are divided into 8 portions, with each accounting for 45 degrees. The direction towards the positive y axis is defined as 0 degree. The heading angle increases clockwise. 73 Figure 5.6. Discrete headings of UAV The example shown in Fig. 5.5 is used to illustrate the 8 part discretization. UAV1 has 8 different ways to fly from its initial position with its initial heading to Target1, according to the required heading angle when UAV1 is on Target1. Fig. 5.7 shows 2 out of the 8 cases with minimum-length path. Figs. 5.7a and b show the 2 cases of 0°and 180°heading on Target1, respectively. The UAV has a minimum turning radius of 0.5 units. It can be seen that different heading angles on the target will lead to different UAV paths. (a) 0°heading on target (b) 180°heading on target Figure 5.7. Two examples of minimum-length paths with different heading angles 5.3.2 The graph representation One way to represent the entire solution space is through the use of graph. Define N t be the number of targets, N d be the number of discrete heading angles of the UAVs and N u the 74 number of UAVs. Fig. 5.8a shows the scenario in which N t  2, N d  3, N u  2 . There are in total 6 target nodes and 2 UAV nodes, representing the combination of possible UAV heading angles over targets and the initial position of UAVs. Edges connecting the nodes are directional. Each edge represents the cost of going from one node to another for a UAV. There are edges connecting each pair of the target nodes in two directions. Finding the optimal solution is in fact finding an optimal combination of edges that minimizes the sum of cost. However, representing all the possible combinations by a single graph does not allow for the consideration of heterogeneous UAVs. In a single graph as in Fig. 5.8a, the cost of going from one target node to another is assumed to be the same for both UAVs. This is because the cost calculation is based on a same edge. This is not realistic because the cost for UAVs with different turning radius is usually different, even for identical trajectories. Therefore, in this study, a particular graph is constructed for each UAV, allowing the consideration of heterogeneous cost functions. The number of edges for the graph of UAV i, N Ei is, N Ei  N t N d  ( N t N d ) 2 (5.2) where the term N t N d represents the number of edges pointing from the UAVi’s starting position towards each of the nodes, and the term ( N t N d ) 2 represents the bi-directional edges pointing from each target node to every target node in the graph, including itself. In the particular example of Fig. 5.8b, where N t  2, N d  3 , N Ei  42 . Since the solution is a combination of edges, each edge is quantified to have a cost. This is defined later when the fitness function of the genetic algorithm is discussed. The use of edges as a solution representation is discussed in the next section. 75 (a) (b) Figure 5.8. The graph representations with 3 part discretization 5.4 The Solution Encoding The new graph representation provides a foundation for the solution encoding. A solution can be found by finding a combination of the edges connecting the nodes for each UAV. An example is given in Fig. 5.9. Fig. 5.9a illustrates the data structure of a solution. Figs. 5.9b and 5.9c highlights the chosen edges of the graphs for UAV1 and UAV2. It represents the solution in which UAV1 completes the first task on Target1 at 0°heading, followed by the second task on Target2 at 120°heading and finishes the job by visiting Target1 again at 240°heading. On the other hand, UAV2 starts by visiting Target2 at 0°heading, followed by Target1 at 120°, and finishes by visiting Target2 at 240°heading. This solution maintains that each target is visited exactly 3 times (Classify, Attack, and Verification). 76 (a) (b) (c) Figure 5.9. A solution from the graph representation 5.5 The Initialization The initialization process generates the first generation of solutions. Based on the first generation, future generations can be produced by applying genetic operators. The first generation must be as diversified as possible and meets all the constraints. This section describes the mechanism of generating the first generation. Without loss of generality, the process of randomly generating a single solution is discussed. The process for the whole generation is just a repetition of the single solution generation process. 77 Again, let N t be the number of targets on which at least one task is to be performed. N d be the number of discrete heading angles in which a UAV can approach a target, as described by Fig. 5.6. Further, let N mi be the number of tasks to be carried out on target i . This models the fact that some targets are classified and requires only to be destroyed (1 task). There are other targets which need to be verified as hostile before an attack can be performed (2 tasks). Finally, let N u the number of UAVs that carries out the mission. Therefore, the total number of visits N visit on all targets by UAVs can be expressed as the sum of the required tasks on each target: Nt N visit   N mi (5.3) i 1 Figure 5.10 shows the algorithm used to generate a random solution. At the beginning, two variables V1 and V2 are initialized. V1 is the solution variable which will contain the chosen nodes of the graph in sequence. V1 is empty at the beginning. V2 is the objective variable which contains the unassigned tasks on each target. Initially, V2 contains all the required tasks on targets. At each iteration, the algorithm randomly selects a remaining target from V2 and assigns it randomly to a UAV at a random heading angle, hence updating V1. Thereafter such task is removed from V2 for the next iteration. The iterative process continues to assign tasks in V2 to UAVs and record the assignment in V1. The iteration stops until V2 is empty. Then, V1 is a random solution. It contains the sequential target assignment for each UAV and their approaching angle on each of their assigned target. Essentially, each UAV is to visit the nodes in its graph sequentially. The suitability of the solution can be calculated by the sum of the edges connecting those nodes. 78 Figure 5.10. Algorithm for the random solution generation 5.6 The Fitness Function Section 5.2 describes how to generate the minimum-length path for arbitrary starting and ending positions and directions (Dubins vehicle). Based on that, the minimum-length path between any two nodes in the graph can be obtained easily. Therefore, every edge connecting a pair of nodes is assigned a cost value that equals to the length of the Dubins‟ path connecting the two nodes. Since a solution is a combination of edges, the fitness of a solution is a function F (x) that maps the space of edge combination to a positive real value: F ( x) : E  R  (5.4) Where E is the space of all possible combination of edges, and R  is the set of positive real numbers. 79 In this study, two kinds of fitness function assignments are used. The first is the inverse of the simple sum of the edge weights. Let S be a solution and ei an edge belonging to S , the first fitness function is defined by: F1 ( S )  10000 e ei S (5.5) i Therefore, solutions with shorter total distance have greater fitness value as compared to those with longer total distance. The second kind of fitness function assignment essentially derives the fitness from the mission completion time. Let S be a solution, the total distance traveled by UAV i, l i , can be calculated. Further, let v i be the velocity of UAV i . The second kind of fitness function can be expressed as: F2 ( S )  10000 l max( i ) vi (5.6) The mission completion time is defined by the maximum of the completion time of all UAVs. In this way, the solution which has a smaller mission completion time has a bigger fitness value as compared to those with longer completion time. It will be demonstrated through experiment that F2 is a more suitable fitness function. The application of F1 usually leads to a solution which assigns all the tasks to the UAV with smallest turning radius. This is because it travels less distance turning around, hence saving the total path length as compared to bigger turning radius UAVs. However, making only one UAV work and leaving others idle is not optimal. Moreover, the mission completion time will be 80 enormous if all the tasks are carried out by only one UAV. In contrast, F2 results in more reasonable solutions in that the targets are evenly distributed among UAVs. By minimizing the maximum mission completion time, UAVs are forced to visit targets in a way that minimizes the mission completion time. 5.7 The Genetic Algorithm Operators The design of GA operators for this problem is different from that for the single UAV path planner. The most important principle is that the quantity of tasks to be completed on each target must be kept constant after the operators. The design of these operators is based on the important factors that affect a solution. These factors are the assignment of targets to each UAV, the visiting sequences of targets for each UAV, and the heading angle of UAV on each target. (i) Crossover Unlike the crossover operator in the traditional sense which combines two parent solutions, the crossover applied here is done between tasks of different UAVs in the same solution. This arrangement maintains the total number of tasks completed by UAVs a constant. In essence, the crossover operator changes partially the target assignment for two different UAVs in the same solution. Fig. 5.11 shows the crossover operator on UAVs i and j in a same solution. The crossover points are gene r for UAV i and gene s for UAV j. 81 Figure 5.11. Crossover operator on same solution across different UAVs (ii) Mutation The mutation operator changes only the heading angle of a UAV on a target. The example in Fig. 5.12 shows the mechanism in which gene r of UAV i, which represents heading angle of  on target k, is mutated. Figure 5.12. Mutation operator changes heading angles (iii) Swap The swap operator is similar to crossover. However, instead of changing all the genes after the crossover point, the swap operator only swaps a single gene with that from a different UAV. Fig. 5.13 shows the swap operator on UAVs i and j in a same solution. The genes swapped are gene r for UAV i and gene s for UAV j. The swap is a more delicate operator as compared to the crossover. While crossover mainly deals with the assignment of targets to UAVs, the swap deals with both the assignment and visiting sequence of the targets. 82 Figure 5.13. An illustration of the swap operator (iv) Immigration The immigration operator in this problem is similar to the usual immigration operator. It simply replaces a solution with a new random solution. (v) Elitism Elitism is the operator that retains the best solution of each generation. With elitism, the fitness value for the best solution is always going upwards. Besides, the algorithm is deemed to have converged if the best solution remains unchanged for 50 iterations. 5.8 The Simulation Experiments The parameters that may affect the result and algorithm convergence are listed below. Targets – an array of 2D coordinates representing the targets to be visited. Tasks – an array of integer whose length equals that of the variable Targets. It defines the number of tasks to be carried out on each target. UAV Initial Positions – an array of 2D coordinates that define the starting positions of all UAVs. 83 UAV Initial Directions – an array of two dimensional coordinates which define the starting direction of all UAVs. Discrete Heading – an integer that defines the number of discrete headings for the UAV. Turning Radius – an array of float point numbers that defines the turning radius of each UAV. Generation Size – The size of generation for each iteration. XOProba, MutProba, SwapProba, ImmigProba – The four variables represent the probability for crossover, mutation, swap and immigration respectively. Convergence Iteration – The minimum number of iterations for which the best solution remains unchanged such that the algorithm is deemed to have converged. 5.8.1 Diversity of the random solution generator The success of GA depends partly on the capability to generate random solution of great diversity. For every experiment, 1000 random solutions are generated for a given problem. 5.8.2 Experiments on different probabilities of genetic operators The first set of simulation experiments were conducted to examine the usefulness of the genetic operators. The parameters in Table 5.1 are kept constant throughout the experiments. Table 5.1. Constant parameters for genetic operators‟ test Discrete Heading Generation Size Convergence Iteration Minimum Turning Radius Fitness Function 36 1000 50 10 meters Minimum Time The first set of 4 experiments were used to test the capability of each of the EA operators independently. The testing results are shown in Fig. 5.14, respectively, in which vector P is defined by: P = (XOProba, MutProba, SwapProba, ImmigProba). A summary for performance comparison is shown in Table 5.2. Except for immigration, all the other 3 operators have positive 84 contributions towards the improvement of the best solution. The crossover is able to produce better solutions by combining the desirable characteristics of parent solutions. Yet, the convergence rate of the experiment with crossover is almost 2 times of that with mutation and swap. This is because the crossover operator is not able to produce enough diversity for the population. This also explains why the final solution is worse than the ones with mutation and swap. What is remarkable about mutation is the sharp increase of average fitness value. This shows that the visiting angle is essential in determining the solution quality and that mutation is capable of arriving at good solutions by targeting at the right heading angle on targets. For the swap operator, both the convergence plot and the result are better than that of mutation and crossover. This shows that in this problem, the visiting sequence is another important factor in 19 20 18 19 17 18 16 15 Best 14 Average Fitness Value Fitness Value determining good solutions. 17 16 15 Best 14 Average 13 13 12 12 11 11 1 51 101 1 151 51 101 151 201 251 301 Generation Generation (a) P = (0.1, 0, 0, 0) (b) P = (0, 0.1, 0, 0) 18 23 17 21 17 Best Average 15 Fitness Value Fitness Value 16 19 15 Best 14 Average 13 13 12 11 11 1 51 101 151 201 251 Generation (c) P = (0, 0, 0.1, 0) 1 51 Generation (d) P = (0, 0, 0, 0.1) Figure 5.14. Fitness evolution for different GA operators 85 Table 5.2. Solution quality of the different GA operators P = (0.1, 0, 0, 0) P = (0, 0.1, 0, 0) P = (0, 0, 0.1, 0) P = (0, 0, 0, 0.1) Mission Completion Time 545.85 sec 531.90 sec 462.26 sec 590.47 sec Total UAV Path Length 1626.77 meters 1561.86 meters 1380.22 meters 1674.03 meters While the first 4 experiments proved the necessity in having all 4 GA operators, another 6 experiments were conducted to find a good combination of GA operators‟ probabilities P. The 6 cases are described by: P = (a, a, a, a); a {0.05,0.1,0.15,0.2,0.25,0.3} . For the convenience of comparison, the fitness plots (the best fitness and average fitness) for the 6 cases are put on the same graph in Figs. 5.15 and 5.16, respectively. It is obvious that the case of a = 0.15 out performs others in terms of the best fitness. In terms of average fitness, lower values of a tends to converge too fast without reaching high values. This is because the effect of GA operators is too small to contribute to the total improvement of the whole generation. Higher values of a tend to have small improvements. This is because in each generation, almost every individual goes through some sort of GA operators. Therefore, the good solutions are hardly retained. 0.05 0.1 101 151 0.15 0.2 0.25 0.3 23 22 Fitness 21 20 19 18 17 16 1 51 201 251 301 351 401 451 Generation Figure 5.15. The best fitness evolution for different GA operator probabilities 86 0.05 0.1 101 151 0.15 0.2 0.25 0.3 17 16 Fitness 15 14 13 12 11 1 51 201 251 301 351 401 451 Generation Figure 5.16. The average fitness evolution of the different GA probabilities 5.8.3 Experiments with different fitness functions Simulation experiments were also conducted with the two fitness functions. The variables in Table 5.3 were kept constant throughout these experiments. Fig. 5.17 shows the solution to the problem with 9 targets and 3 tasks on each target. The UAVs in this problem are homogenous. Their turning radius is in the same magnitude as compared to the distance among the targets (represented by the black squares). There are 3 UAVs to complete the missions and their trajectories are marked with cyan, green and blue colors. The mission completion times for the 2 cases are 444.84 sec and 827.00 sec , respectively. The total distances traveled are 1327.13 meters and 1079.14 meters, respectively. Fig. 5.17a shows that the tasks are well distributed to the three UAVs. Fig. 5.17b on the other hand, shows that the UAV in blue colors has taken the majority of the tasks whereas the other two UAVs completed much fewer tasks. It produces smaller total path length but requires a lot more time to complete. Figure 5.17 demonstrates that the minimum mission time is a much stronger and more reasonable fitness function as compared to the minimum distance traveled. 87 Table 5.3. Constant parameters for different fitness function test XOProba MutProba SwapProba ImmigProba 0.15 0.15 0.15 0.15 Discrete Heading Generation Size Convergence Iteration 36 1000 50 Minimum Turning Radius 10 meters (a) Minimum Mission Time (b) Minimum Total Length Figure 5.17. Result comparison for different fitness functions 88 On the other hand, Fig. 5.18 shows the fitness value evolution for the two different cases of fitness functions. However, the minimum length fitness produces a result that heavily assigns the tasks to a single UAV rather than allocating them to different UAVs. 25 10 Minimum Mission Time Fitness 23 9 21 8.5 19 Best 17 Average 15 Fitness Value Fitness Value Minimum Total Length Fitness 9.5 8 7.5 Best 7 Average 6.5 6 13 5.5 11 5 1 51 101 151 201 251 301 351 401 451 1 51 101 151 201 251 301 351 401 451 501 551 Generation Generation (a) Minimum Mission Time (b) Minimum Total Length Figure 5.18. Fitness value evolutions for different fitness functions 5.8.4 Experiments with different UAV flight dynamics The most prominent results come from the different UAV flight dynamics cases capable to be solved by the multi-graph representation. This section first examines the homogenous turning radius cases, following which the heterogeneous cases are studied. The variables in Table 5.4 are kept constant throughout this section. Table 5.4. Constant parameters for UAV flight dynamics tests XOProba MutProba SwapProba ImmigProba 0.15 0.15 0.15 0.15 Discrete Heading Generation Size Convergence Iteration Fitness Function 36 1000 50 Minimum Time The first experiment considers a problem for which the UAVs‟ turning radius is small r  1 meter as compared to the distance among different clusters of targets. The objective to be 89 minimized is the mission completion time. Similar to the case in Fig. 5.17, there are 9 targets and 3 tasks on each target. Fig. 5.19 shows the result, which is intuitive in this case. Since the turning radius is too small as compared to the distance among targets, each UAV will visit the three targets closest to its starting position. Moreover, each UAV revisits a target twice and completes all the three tasks in a row, before reaching for the next target. This is also because the distance between two targets is longer than the circle turn at a radius of 1 meter. Fig. 5.20 further shows the evolution of the best and average solution fitness values. Figure 5.19. Simulation output for Turning Radius of 1 Turning Radius = 1 110 100 Fitness Value 90 80 70 Best 60 Average 50 40 30 20 1 51 101 151 201 251 301 351 401 Generation Figure 5.20. Fitness evolution of the experiment with Turning Radius of 1 meter The mission completion time is 89.4 sec and the total distance traveled by three UAVs is 262.06 meters. As a first experiment, it may not have much practical significance. However, this experiment result does show that the algorithm is capable of generating reasonable solutions. First, the turning radius of 1 is small as compared to the distance between the targets. Therefore, 90 it is reasonable to have optimal results in which each UAV completes tasks on the targets which are at close vicinity of its starting positions. This is because traveling among different clusters will be too time-consuming requires unnecessary extension of the path length. Second, taking a close look at Fig. 5.19, it is not hard to observe that each UAV tends to re-visit a target until all three missions on the same target are completed, before it flies towards the next target. This can be seen by observing the two circles of radius 1 near each target. The circles show that the target is revisited twice. This is also in line with intuition. Since the distance between any two targets within the same cluster is larger than 2 at turning radius 1. It is better for a UAV to finish all three tasks on a target before flying to the next, than to fly back and forth between targets. Actually, when the turning radius tends to 0, the problem is changed to a multiple travelling salesmen problem. In the next experiment, the turning radius of the UAVs is increased to 3. All other parameters remain the same. Figs. 5.21 and 5.22 show the result output and fitness evolution of this experiment. Figure 5.21. Simulation output for turning radius of 3 91 65 Turning Radius = 3 60 55 Fitness Value 50 45 40 Best 35 Average 30 25 20 15 1 51 101 151 201 251 301 351 401 451 Generation Figure 5.22. Fitness evolution of the experiment with turning radius of 3 meters The completion time in this case is 160.20 sec and the total distance travelled by UAVs is 478.26 meters. What is interesting is the similarity and distinction between the experiments for radius of 1 and 3. It can be observed that in both cases, each UAV stays with completing missions on the three targets at its close vicinity. However, in the case of radius of 3, both revisit and coming back and forth between multiple targets can be observed. This is because the turning radius of 3 makes visiting among clusters costly, yet within each cluster of targets, each UAV has comparable cost between revisiting and flying back and forth among targets. The next experiment is on the turning radius of 7. The result, shown in Figs. 5.23 and 5.24, in this case is less intuitive. The mission completion time is 327.80 sec and the total distance travelled by three UAVs is 981.19 meters. The UAVs are cooperating in completing the tasks on different clusters of targets. UAVs are not constrained to only visit targets close to their starting positions. 92 Figure 5.23. Simulation output for turning radius of 7 meters Turning Radius = 7 32 Fitness Value 27 Best 22 Average 17 12 1 51 101 151 201 251 301 351 Generation Figure 5.24. Fitness evolution of the experiment with turning radius of 7 meters The turning radius is then increased to 10 and the result is shown in Figs. 5.25 and 5.26, respectively. The completion time is 444.84 sec and the total distance traveled is 1327.13 meters. A number of features can be observed from the result. First, the cases of re-visit are greatly reduced. This is because making a turn and coming back to a target requires longer travel distance as compared to visiting a target nearby. Second, as a result of reduced re-visits, all 3 UAVs visit targets from all 3 clusters. The UAVs are not constrained to their own clusters but fly in a cooperative manner. 93 Figure 5.25. Simulation output for turning radius of 10 meters Turning Radius = 10 24 22 Fitness Value 20 18 Best 16 Average 14 12 10 1 51 101 151 201 251 301 351 401 451 Generation Figure 5.26. Fitness evolution of the experiment with turning radius of 10 meters The next test deals with the case of heterogeneous UAV turning radius. However, their speeds are still assumed to be 1 meter. The turning radii for the 3 UAVs are 1 meter, 5 meters and 10 meters, respectively. The result is shown in Figs. 5.27 and 5.28, respectively. The observations are summarized in Table 5.5. 94 Figure 5.27. Simulation output for heterogeneous turning radius 54 Heterogeneous Turning Radius 49 Fitness Value 44 39 34 Best 29 Average 24 19 14 1 51 101 151 201 251 301 351 401 451 501 551 Generation Figure 5.28. Fitness evolution of the experiment with heterogeneous turning radius Table 5.5. The observation summary UAV No. of re-visits Total Tasks Completed Left (Cyan, radius = 10 meters) 1 5 Middle (Green, radius = 5 meters) 1 8 Right (Blue, radius = 1 meters) 7 14 It can be seen that the UAV re-visit rate and number of total tasks completed have increased with decreasing turning radius. This result demonstrates that in a realistic situation 95 with time constraints, agile UAVs are responsible for heavier work load as compared to their less agile teammates. This is because less agile UAVs relatively waste more time and distance on making turns. 5.8.5 Stability of the solution algorithm The final set of experiments is done with the parameters shown in Table 5.6. There are 9 targets and 3 tasks to be completed on each target. There are 3 UAVs to complete the tasks. The optimization objective is to minimize the mission completion time. The same experiment is repeated for 10 times to see if similar result is obtained each time. The diversity of the starting generation was demonstrated before. Therefore, through these experiments, it is to demonstrate that the algorithm is capable of arriving at similar optimums each time it is run, despite starting with different and diversified first generations. The parameters in Table 5.6 are adopted for all the 10 experiments. The results from the 10 simulation runs, in terms of the best fitness and average fitness evolution, are shown in Figs. 5.29 and 5.30, respectively. It can be concluded that the algorithm is quite consistent every time it is run. This also shows that the algorithm is able to find similar optimal solutions with good stability, which is a very desirable characteristic of the solution algorithm. Table 5.6. Parameters used for repeated simulations testing stability XOProba MutProba SwapProba ImmigProba Discrete Heading Generation Size 0.15 0.15 0.15 0.15 36 1000 Convergence Iteration Fitness Function Turning Radius 50 Minimum Time 10 meters 96 1 2 3 4 5 6 7 8 301 351 401 9 10 25 24 23 22 Fitness 21 20 19 18 17 16 15 1 51 101 151 201 251 451 501 551 Generations Figure 5.29. The evolution of the best fitness of 10 simulations 1 2 3 4 5 6 7 8 301 351 401 9 10 17 16 Fitness 15 14 13 12 11 1 51 101 151 201 251 451 501 551 Generations Figure 5.30. The evolution of average fitness of 10 simulations 5.9 Summary In this chapter, a multiple graph representation has been proposed to describe the multiple UAV mission assignment problem that allows for the consideration of heterogeneous UAV flight dynamics. A GA based solution algorithm has also been developed to find the optimal solution. 97 The robustness of the solution algorithm is demonstrated through simulations with different turning radius. The stability test justifies the performance consistency of the solution algorithm. 98 CHAPTER 6 AN OCCLUSION-AWARE MODEL FOR UAV SURVEILLIANCE IN COMPLEX URBAN ENVIRONMENT This chapter addresses the problem of multi-UAV surveillance in complex urban environments with occlusions. The objective is to allocate the UAVs such that the coverage and the information from last visit about a designated area are maximized. A generic approach to the problem is a two-stage optimization [44]. The first stage finds the set of vintage observation points in the sky. Each point on the ground can be seen by at least one observation point. The second stage is a multiple travelling salesmen problem. It allocates the UAVs to visit these observation points. In this chapter, the problem is firstly defined, followed by the development of a more generalized occlusion-aware model compared with the model in [44]. Besides, the flaws of the two-stage method are also discussed. This model forms the basis to develop a truly integrated optimization method to find the optimal solution to the UAV surveillance problem in a realistically complex urban environment. 6.1 Problem Description This study generalizes the problem description in [44]. In order to fully describe the problem, the environment, UAV and the objective function must be defined. 6.1.1 The environment The environment in which the UAV task is to be accomplished consists of a surface of varying height and all the space above the surface. The surface includes the terrain surface and rooftops. It is assumed that every point on the surface can be described by its x, y coordinates 99 and a function H ( x, y) giving its z coordinate. More formally, this set of information is described as follows. Area: A  ℝ2 Height map: H : A  ℝ+ Terrain: M  { x, y, z  ℝ3 | x, y  A, z  H ( x, y)} Compared with the model in [44], the definition of terrain here does not restrain the buildings to be on a same horizontal surface. Instead, the baseline can be of varying elevations, thanks to the occlusion aware model developed in this study. Then, the space in which the UAVs can operate (the air) is defined as everything above the surface as: Air: A3D  { x, y, z  ℝ3 | x, y  A, z  H ( x, y)} 6.1.2 The sensor capability of UAVs Particular to this problem is the model for the sensors on UAVs. The sensors are modeled to look vertically downwards (see Fig. 6.1), and have a cone-shape coverage space below the UAV. The surface that lies within the cone-shape and not blocked by any buildings can be seen. The occlusion aware model to be discussed in section 6.2 is essential for the analysis of such blocked situations. 6.1.3 The objective function The mission to be completed by a set of UAVs is to maintain surveillance over an area. Therefore, it is desirable to have as long an exposure time for each point on the ground as possible. Let T  (0,] be the total mission time, for a point ( x, y) on the ground, define 100  Expo ( x, y, t )  1 if point ( x, y) is exposed in the sensor of at least one UAV at time t . Otherwise,  Expo ( x, y, t )  0 . In light of this, the objective function to be maximized can be written as [44]: O  T M  Expo ( x, y, t )dxdydt (6.1) 6.2 The Two-Stage Optimization In [44], the maximization of Eq. (6.1) is achieved over two stages. The objective of the first stage is to find a set of observation points in the sky such that each point on the entire surface is visible by at least one observer in the sky. The minimum set of such observation points is called the vantage set, and are to be visited by a group of UAVs. In the second stage, the objective is to maximize the aggregate exposure of the UAVs and ensure that they visit all the points. This study develops an occlusion-aware model, which is the basis for finding the vantage set of observation points. 6.3 The Occlusion Aware Model In [44], the surface consists of only a horizontal base plane defined by z  0 and quadrangular prisms lying of the base plane. Therefore, it does not consider the variation in terrain elevation and the different shapes of the buildings. This leads to easy computation of visibility using analytical methods (see Fig. 6.1). However, not every city is built upon a flat surface. Moreover, real world architectures can be of all kinds of shapes. To overcome this oversimplification limit, an occlusion-aware model has been developed in this study that can be potentially applied to a situation in which complex terrain environment and building shapes are taken into consideration. 101 Figure 6.1. Visibility test with analytical method [44] Thanks to JME as a robust 3D rendering and computing package, visibility detection can be achieved through collision checking between a ray (connecting the observation point and the point to be observed) and a 3D object (the building). Each building is defined as an enclosed 3D space. As an illustrative example shown in Fig. 6.2, A is a point on the ground whose visibility is to be checked. A ray shoots from the observation point in the sky to A. Fig. 6.2a shows that the ray is intercepted by a 3D object half-way. In this case, A cannot be observed. Fig. 6.2b shows A on a flat terrain, which can be observed from the observation point in sky. However, in a rugged terrain situation, the line of sight from the sky may be blocked by the terrain. Therefore, the occlusion detection model to be developed will check the ray collision with both the terrain and the buildings. 6.4 System Implementation of the Occlusion Aware Model 102 (a) (b) Figure 6.2. Illustration of ray detection In the occlusion-aware model, the surface is reasonably treated as a discrete set of points. In such setting, given an observation point in the sky and the sensor angle, the observable area on the surface can be easily marked. Fig. 6.3 illustrates the implementation of the occlusion awareness model using ray detection. Whenever the ray detection finds an obstacle between the observation point and a point of the surface, such point on the surface is marked as being obstructed. Otherwise, the point of surface is visible. Figure 6.3. The occlusion-aware model 103 Fig. 6.3 is a screenshot from a 3D occlusion aware model simulation. The ground and rooftops are represented by discrete blue points. The buildings are represented by grey 3D objects. The observation point in the sky is represented by the green square. Its altitude is 225 meters, whereas the average height of the buildings in this case is 100 meters. The apex angle of the sensor is 45 degrees, looking vertically downwards. The camera in this case is looking downward with a 10 degree angle from the vertical axis. The actual observable points after applying the occlusion aware model by means of the ray casting checking algorithm are colored red. All other unobservable points remain blue in color. It can be seen that without the two buildings whose contours are outlined, all points inside the orange circle can be observed. However, due to the buildings, only a small part of the ground and rooftops can be observed. Fig. 6.3 also indicates the validity of the occlusion-aware model of this study. Rather than using an analytical method, the function describing the surface does not need to be known beforehand. Only the discrete points describing the surface need to be known. This greatly generalizes the occlusion detection method. It is also capable for occlusion detection on a rough terrain, whose surface is difficult to be expressed in an analytical way. However, there are two major drawback of this method. The discretized surface leads to inaccuracy of the occlusion detection, in particular for corners or edges. On the other hand, rendering the points requires a large amount of computational resources. This might make the method impractical for largescale applications. 6.5 Limitations of the Two-Stage Method The two stages are in fact inter-related. Therefore, the shortcoming of such two-stage optimization is that two different objective functions are used in sequence, instead of a single, 104 integrated one. This may easily lead to local optimum. Nevertheless, the two-stage method greatly reduces the solution space and hence enhances the efficiency of the algorithm. 6.6 Discussions In this chapter, an occlusion-aware model has been developed to generalize the problem definition in [44] for realistic visibility detection. This is done through applying the built-in functions of JME. Regardless of the optimization method applied, the occlusion-aware model is an essential part to solve the UAV surveillance problem. Therefore, this study provides a solid foundation for possible future development in the surveillance by UAVs in urban environment. 105 CHAPTER 7 CONCLUSIONS AND FUTURE WORK 7.1 Conclusions on the Study of the UAV Path Planning Problem In this study, the single UAV path planning problem is modeled in a realistic mode in terms of terrain and UAV capability. A new GA based solution algorithm has been developed and tested. The model and the solution algorithm form a new path planner. The aim of this part of the study is not merely to find the best solution or the most efficient path planning algorithm. Moreover, it is a thorough study of the single UAV mission planning problem that seeks to understand how different fitness functions, different GA operators and different UAV flight constraints affect the path planning process. In the end, according to this study, a sound combination of parameters and GA operators has been proposed. According to the tests conducted in this study, constant assignment of fitness values for each gene is able to solve simple problems without any concavity. However, constant values differentiate valid solutions from non-valid ones but do not differentiate among the invalid solutions. As a result, it is hard to negotiate around concave terrain and obstacle structures. On the other hand, dynamic assignment of fitness values differentiates among the invalid solutions and could tell those solutions that are “close” to the valid solutions, hence moving out of the concavity gradually by means of the selection mechanism of the GA. The random generator for each generation is a process that provides diversity. Cross-over is useful in combining the good characteristics of different individuals in the previous generation. Local and strong mutations are important operators that help the solution evade local minimums. 106 Deletion is a smoothing operator on the path that reduces unnecessary fluctuations. All of the five GA operators are indispensible for the path planner. The calculation speed of the path planner is reduced with more conservative UAV maneuverability. Also, more generations are required before the convergence for conservative UAV characteristics. The change in the distance between any two consecutive waypoints does not affect the GA planner; however, the long distance may result in judgment errors especially at sharp obstacle corners. In summary, this study started by modeling the single UAV path planning problem and designing a GA based algorithm to solve it. Experiments have been conducted diligently for different fitness functions, GA operators and UAV maneuverability. As a result, a thorough understanding of the problem is achieved. 7.2 Conclusions on the Study of the Multiple UAV Task Assignment Problem A thorough research on the heterogeneous UAV mission coordination problem has been conducted. The design of the multiple graph representation allows for the consideration of heterogeneous UAV flight dynamics. A GA based algorithm has been developed as the solution method. Simulation experiments demonstrate that the solution algorithm is able to solve both intuitive problem cases as well as complicated problem cases. The small turning radius cases provide solution output similar to that of the multiple traveling salesmen problem. This demonstrates the robustness of the model and solution. When the turning radius increases, the result output is less intuitive and cannot be imagined by human mind. This is when the usefulness of the solution algorithm gets illustrated. 107 The stability test in the end demonstrates the consistency in both the convergence speed and the final solution reached. Hence, it further justifies the choice of model and solution algorithm in this problem. 7.3 Conclusions on the Study of the UAV Surveillance Problem For the UAV surveillance problem, a generalized occlusion-aware model has been proposed that finds the observable area from a given sensor in the sky. Previous literatures only achieved this in an analytical way on flat terrain with buildings in the shape of quadrangular prisms. However, the model in this study could potentially be applied to any terrain condition and all sorts of building shapes, so long as the buildings are described as closed 3D objects. In this way, this study provides a solid foundation for possible future development in the surveillance by UAVs in urban environment in terms of solution efficiency and problem generalization. 7.4 Recommendation for Future Work In this study, a thread is dedicated to the single UAV path planner described in this thesis. There are other threads that take charge of the user-interface and the display of the mission planner. Such multi-thread design environment provides a good foundation for various future development. Some possible implementable extensions are described in the following section. Multiple UAV Mission Planning The following steps can be taken to extend the planner from single UAV to multiple UAV mission planning. (1) Each UAV has its pre-defined targets. Several threads run in parallel to calculate the paths for UAVs. There is no interference among the threads. 108 (2) Based on step (1), there are communications among the threads for coordination and collision avoidance. (3) Based on step (2), the targets are not assigned to the UAVs initially. The UAVs will have to coordinate among themselves for target assignment and path planning. Another important aspect is to combine the methods for UAV task assignment with that of single UAV path planning. In this way, the path planning and the coordination among UAVs are both done in an integrated manner. Variable UAV Velocity The UAV velocity is assumed to be constant throughout this study. When variable UAV velocity is considered, greater flexibility is given to the model. For example, UAVs will have to turn at greater turning radius when velocity is larger. Variable UAV velocity also gives better collision avoidance solutions for multiple UAV case. Online Mission Planner So far, the mission planner works in a way that the path is planned before the UAVs take off. However, the multi-thread structure of the planner will allow for the modification of the path even after the UAV takes off. This mode is called “online” planning since the planner must deal with unforeseen situations such as “pop-up” radar or addition of targets. 109 REFERENCES 1. James, G.K., Unmanned Aerial Vehicles and Special Operations: Future Directions. 2000: PhD Thesis, Naval Postgraduate School, Monterey, California, United States. p. 130p. 2. Canavan, G.H., UAV sensor and survivability issues. 1996: United States. p. 22p. 3. Larroque, C., K.S. Thompson, and D. Hickman. UAV sensor systems for close-range operations. in Infrared and Passive Millimeter-wave Imaging Systems: Design, Analysis, Modeling, and Testing, 3-5 April 2002. 2002. USA: SPIE-Int. Soc. Opt. Eng. 4. Altenbaugh, R., et al. Application of the Raven UAV for Chemical and Biological Detection. in Chemical, Biological, Radiological, Nuclear, and Explosives (CBRNE) Sensing XI, 6-8 April 2010. 2010. USA: SPIE - The International Society for Optical Engineering. 5. Ryan, A., et al. An overview of emerging results in cooperative UAV control. in 2004 43rd IEEE Conference on Decision and Control (CDC), 14-17 Dec. 2004. 2004. Piscataway, NJ, USA: IEEE. 6. Martinez, C., et al. On-board and ground visual pose estimation techniques for UAV control. 2011. Van Godewijckstraat 30, Dordrecht, 3311 GZ, Netherlands: Springer Netherlands. 7. Pisano, W.J., D.A. Lawrence, and P.C. Gray. Autonomous UAV control using a 3-sensor autopilot. in 2007 AIAA InfoTech at Aerospace Conference, May 7, 2007 - May 10, 2007. 2007. Rohnert Park, CA, United states: American Institute of Aeronautics and Astronautics Inc. 8. Cassandras, C.G. and L. Wei. A receding horizon approach for dynamic UAV mission management. in Enabling Technologies for Simulation Science VII, 22-25 April 2003. 2003. USA: SPIE-Int. Soc. Opt. Eng. 9. Koeners, G.J.M., et al. Exploring network enabled airspace integration functions for a UAV mission management station. in 2006 IEEE/AIAA 25th Digital Avionics Systems Conference, 1519 Oct. 2006. 2006. Piscataway, NJ, USA: IEEE. 10. Reinelt, G., TSPLIB-a traveling salesman problem library. ORSA Journal on Computing, 1991. 3(Copyright 1992, IEE): p. 376-84. 110 11. Sisson, M.R., Applying Tabu Heuristic to Wind Influenced, Minimum Risk and Maximum Expected Coverage Routes. 1997: United States. p. 83p. 12. Ryan, J.L., et al. Reactive Tabu Search in unmanned aerial reconnaissance simulations. in Proceedings of IEEE Winter Simulation Conference, 13-16 Dec. 1998. 1998. Piscataway, NJ, USA: IEEE. 13. Beard, R.W., T.W. McLain, and M. Goodrich. Coordinated target assignment and intercept for unmanned air vehicles. in 2002 IEEE International Conference on Robotics adn Automation, May 11, 2002 - May 15, 2002. 2002. Washington, DC, United states: Institute of Electrical and Electronics Engineers Inc. 14. Bortoff, S.A. Path planning for UAVs. in 2000 American Control Conference, June 28, 2000 June 30, 2000. 2000. Chicago, IL, USA: IEEE. 15. McLain, T.W., et al. Cooperative control of UAV rendezvous. in 2001 American Control Conference, June 25, 2001 - June 27, 2001. 2001. Arlington, VA, United states: Institute of Electrical and Electronics Engineers Inc. 16. Chandler, P.R., M. Pachter, and S. Rasmussen. UAV cooperative control. in Proceedings of American Control Conference, 25-27 June 2001. 2001. Piscataway, NJ, USA: IEEE. 17. Stirling, W.C. and M.A. Goodrich. Conditional preferences for social systems. in Proceedings of IEEE International Conference on Systems, Man & Cybernetics, 7-10 Oct. 2001. 2001. Piscataway, NJ, USA: IEEE. 18. Yao-hong, Q., P. Quan, and Y. Jian-guo. Flight path planning of UAV based on heuristically search and genetic algorithms. in IECON 2005. Thirty-First Annual Conference of the IEEE Industrial Electronics Society, 6-10 Nov. 2005. 2005. Piscataway, NJ, USA: IEEE. 19. Alighanbari, M., Y. Kuwata, and J.P. How. Coordination and Control of Multiple UAVs with Timing Constraints and Loitering. in 2003 American Control Conference, June 4, 2003 - June 6, 2003. 2003. Denver, CO, United states: Institute of Electrical and Electronics Engineers Inc. 111 20. Bellingham, J.S., et al. Cooperative path planning for multiple UAVs in dynamic and uncertain environments. in 41st IEEE Conference on Decision and Control, December 10, 2002 December 13, 2002. 2002. Las Vegas, NV, United states: Institute of Electrical and Electronics Engineers Inc. 21. Kuwata, Y. and J. How. Three dimensional receding horizon control for UAVs. in Collection of Technical Papers - AIAA Guidance, Navigation, and Control Conference, August 16, 2004 August 19, 2004. 2004. Providence, RI, United states: American Institute of Aeronautics and Astronautics Inc. 22. Richards, A. and J.P. How. Aircraft trajectory planning with collision avoidance using mixed integer linear programming. in Proceedings of 2002 American Control Conference, 8-10 May 2002. 2002. Danvers, MA, USA: American Automatic Control Council. 23. Ruz, J.J., et al. Using MILP for UAVs trajectory optimization under radar detection risk. in 2006 IEEE Conference on Emerging Technologies and Factory Automation, ETFA, September 20, 2006 - September 22, 2006. 2006. Hamburg, Germany: Institute of Electrical and Electronics Engineers Inc. 24. Melchior, P., et al., Consideration of obstacle danger level in path planning using A* and fastmarching optimisation: comparative study. Signal Processing, 2003. 83(Copyright 2004, IEE): p. 2387-96. 25. Szczerba, R.J., et al., Robust algorithm for real-time route planning. Ieee Transactions on Aerospace and Electronic Systems, 2000. 36(Copyright 2000, IEE): p. 869-78. 26. Qu, Y.H., Q. Pan, and H.G. Yan, Flight path planning of UAV based on heuristically search and genetic algorithms. Iecon 2005: Thirty-First Annual Conference of the Ieee Industrial Electronics Society, Vols 1-3, 2005: p. 45-49 27. Mittal, S. and K. Deb. Three-dimensional offline path planning for UAVs using multiobjective evolutionary algorithms. in 2007 IEEE Congress on Evolutionary Computation, 25-28 Sept. 2007. 2007. Piscataway, NJ, USA: IEEE. 112 28. Hasircioglu, I., H.R. Topcuoglu, and M. Ermis. 3-D path planning for the navigation of unmanned aerial vehicles by using evolutionary algorithms. in 10th Annual Genetic and Evolutionary Computation Conference, GECCO 2008, July 12, 2008 - July 16, 2008. 2008. Atlanta, GA, United states: Association for Computing Machinery. 29. Nikolos, I.K. and N.C. Tsourveloudis. Path planning for cooperating unmanned vehicles over 3-d terrain. in International Conference on Informatics in Control Automation and Robotics, ICINCO 2007, May 9, 2007 - May 12, 2007. 2009. Angers, France: Springer Verlag. 30. Nikolos, I.K., et al., Evolutionary algorithm based offline/online path planner for UAV navigation. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 2003. 33(Copyright 2003, IEE): p. 898-912. 31. Pehlivanoglu, Y.V. and A. Hacioglu. Vibrational genetic algorithm based path planner for autonomous UAV in spatial data based environments. in 3rd International Conference on Recent Advances in Space Technologies, RAST 2007, June 14, 2007 - June 16, 2007. 2007. Istanbul, Turkey: Inst. of Elec. and Elec. Eng. Computer Society. 32. Changwen, Z., et al., Evolutionary route planner for unmanned air vehicles. Ieee Transactions on Robotics, 2005. 21(Copyright 2005, IEE): p. 609-20. 33. Besada-Portas, E., et al., Evolutionary Trajectory Planner For Multiple UAVs In Realistic Scenarios. Ieee Transactions on Robotics, 2010. 26(Copyright 2010, The Institution of Engineering and Technology): p. 619-34. 34. De La Cruz-Garcia, J.M., et al. Multiobjective path planner for UAVs based on genetic algorithms. in Computational Intelligence in Decision and Control - 8th International FLINS Conference, September 21, 2008 - September 24, 2008. 2008. Madrid, Spain: World Scientific Publishing Co. Pte. Ltd. 35. De La Cruz, J.M., et al. Evolutionary path planner for UAVs realistic environments. in 10th Annual Genetic and Evolutionary Computation Conference, GECCO 2008, July 12, 2008 - July 16, 2008. 2008. Atlanta, GA, United states: Association for Computing Machinery. 113 36. Ruoding, Z., Z. Changwen, and Y. Ping. Route planning for unmanned air vehicles with multiple missions using an evolutionary algorithm. in 2007 3rd International Conference on Natural Computation, 24-27 Aug. 2007. 2007. Piscataway, NJ, USA: IEEE. 37. Farin, G.E., Curves and surfaces for computer aided geometric design : a practical guide. Computer science and scientific computing. 1988, Boston: : Academic Press. xv, 334 p., [4] p. of plates. 38. Rasmussen, S.J., et al. State-space search for improved autonomous UAVs assignment algorithm. in Decision and Control, 2004. CDC. 43rd IEEE Conference on. 2004. 39. Shima, T., S.J. Rasmussen, and A.G. Sparks. UAV cooperative multiple task assignments using genetic algorithms. in Proceedings of the 2005 American Control Conference, 8-10 June 2005. 2005. Piscataway, NJ, USA: IEEE. 40. Dubins, L.E., ON CURVES OF MINIMAL LENGTH WITH A CONSTRAINT ON AVERAGE CURVATURE, AND WITH PRESCRIBED INITIAL AND TERMINAL POSITIONS AND TANGENTS. American Journal of Mathematics, 1957. 79(3): p. 497-516. 41. Nygard, K.E., P.R. Chandler, and M. Pachter. Dynamic network flow optimization models for air vehicle resource allocation. in Proceedings of American Control Conference, 25-27 June 2001. 2001. Piscataway, NJ, USA: IEEE. 42. Schumacher, C., P.R. Chandler, and S.R. Rasmussen. Task allocation for wide area search munitions. in American Control Conference, 2002. Proceedings of the 2002. 2002. 43. Rasmussen, S.J. and T. Shima, Tree search algorithm for assigning cooperating UAVs to multiple tasks. International Journal of Robust and Nonlinear Control, 2008. 18(Compendex): p. 135-153. 44. Semsch, E., et al. Autonomous UAV Surveillance in Complex Urban Environments. in Web Intelligence and Intelligent Agent Technologies, 2009. WI-IAT '09. IEEE/WIC/ACM International Joint Conferences on. 2009. 45. Bioinformatics and Interaction Design. 2010 23 September 2010 [cited 2011 19 September]; Available from: http://biointeraction.blogspot.com/2010/09/dna-and-chromosome.html. 114 46. Rasmussen, S.J., et al. Introduction to the MultiUAV2 simulation and its application to cooperative control research. in American Control Conference, 2005. Proceedings of the 2005. 2005. 115 [...]... section 1.3 UAV Path Planning UAV mission planning is in itself a very broad research area in which the mission coordination among multiple UAVs and the path planning of UAVs are two fundamental areas These two areas are chosen in this study as the results can be used for future research on mission planning, e.g., multiple UAV mission planning, real-time, and variable velocity UAV path planning For... Concerning UAVs Due to the widening scope and increasing complexity of UAV missions, the researches on UAVs span a very broad range Some of the important research areas include sensor and control technology, mission management system and UAV mission planning 1.2.1 Sensor technologies The development in sensor technologies aims at enhancing the UAV capabilities and broadening the mission types a UAV is... targets while minimizing the probability of destruction of the UAV (a) (b) Figure 1.1 Dot-on-coordinate mission planning model and Voronoi diagram solution [13] Studies had targeted both single [14] and multiple [13, 15, 16] UAV mission planning in such setting One proposed solution algorithm is the two-stage Voronoi diagram (see Figure 1.2b) and virtual forces [13] Voronoi diagram is a way of space repartition... terrain and fixed shapes of buildings, making the problem unrealistic This study focuses on developing an occlusion aware model that is both efficient and allows for more generalized problem definition 1.6 Research Objectives This thesis covers three different aspects of the researches on UAV mission planning The first is the trajectory and path planning of single UAVs This study has gone very deep and. .. conducting research in the three areas studied The motivations concern more on the modeling and formulation of the problem Following this, the genetic algorithm (GA) as a solution tool is discussed 2.1 UAV Path Planning in Complex and Realistic Environment Despite much effort that has been put on the 3D UAV mission planning and some success has been achieved, there are plenty of rooms for improvement The... decision making Cassandras and Wei proposed a simulated battle space and a dynamic target assignment scheme which achieves the commander‟s order with optimal performances [8] Another example is the airspace integration through network among the UAV, the mission management station and the air traffic control [9] 1.2.4 Mission planning UAV mission planning, which is also the focus of this study, considers optimization... in the model 1.3.4 Three-dimensional (3D) mission model Recently, evolutionary algorithm (EA) becomes a popular choice for solving route planning problem Since it greatly increases the computational power, complex 3D models are widely used for the UAV mission planning problems The major components of a 3D model are terrain, UAV path, and threats For terrain modeling, one method is to artificially generate... hard to handle for local modifications Table 1.1 summarizes the merits and disadvantages of the various modeling approaches for different components of the 3D UAV path planning model Table 1.1 Various 3D modeling approaches for UAV path planning 3D Component Terrain Model Parametric artificial terrain Digital Elevation Model Linear Segments UAV Path Threats EA Coding Advantages Continuous, not demanding... Demanding on the solution algorithm The search space is large Local changes are hard to handle Realistic Easy to be used for the EA operators Reduced Search Space 11 Table 1.2 summarizes the models and solution algorithms adopted in some of the important previous works on UAV mission planning Table 1.2 Models and solution algorithms in previous works Model mTSPTW Dots-oncoordinate Solution Algorithm... Integration of onboard and ground camera is a proposed method for real-time positioning of UAVs [6] Another study proposes UAV control by means of a single axis rate gyro, an absolute pressure sensor and a GPS receiver [7] 1.2.3 Mission management system Researches on mission management system target at facilitating the battlefield commanders in high-level control and decision making Cassandras and Wei proposed ... used for future research on mission planning, e.g., multiple UAV mission planning, real-time, and variable velocity UAV path planning For example, multiple UAV mission planning requires a target... addresses the fixed-wing UAV mission planning problem More specifically, the study focuses on sub-problems: single UAV path planning, multiple UAV cooperative task assignment, and UAV surveillance in... mission management system and UAV mission planning 1.2.1 Sensor technologies The development in sensor technologies aims at enhancing the UAV capabilities and broadening the mission types a UAV