Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies

Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies

Advanced Engineering Informatics 35 (2018) 30–55 Contents lists available at ScienceDirect Advanced Engineering Informatics journal homepage: www.elsevier.com/locate/aei Full length article Agent-based evacuation modeling with multiple exits using NeuroEvolution of Augmenting Topologies T Mehmet Erkan Yuksel Department of Computer Engineering, Mehmet Akif Ersoy University, Burdur, Turkey A R T I C L E I N F O A B S T R A C T Keywords: Evolutionary computation Neuroevolution Genetic algorithms Agent-based modeling and simulation Evacuation modeling Emergency evacuation Evacuation modeling offers challenging research topics to solve problems related to the development of emergency planning strategies In this paper, we built an agent-based evacuation simulation model to study the pedestrian dynamics and learning process by applying the NeuroEvolution of Augmenting Topologies (NEAT) which is a powerful method to evolve artificial neural networks (ANNs) through genetic algorithms (GAs) The NEAT method strengthens the analogy between GAs and biological evolution by both optimizing and complexifying the solutions simultaneously We set our main goal to develop a model by identifying the most appropriate fitness function for the agents that can learn how to change and improve their behaviors in a simulation environment such as moving towards the visible targets, producing efficient locomotion, communicating with each other, and avoiding obstacles while reaching targets The fitness function we chose captured the learning process effectively and our NEAT-based implementation evolved suitable structures for the ANNs autonomously According to our experiments and observations in the simulated environment, the agents accomplished their tasks successfully and found their ways to the exits Introduction Evacuation modeling is an important part of pedestrian modeling expertise It is the process of guiding evacuees to the exits when an emergency situation happens It is developed to ensure the safest and most efficient evacuation time of all expected residents of a building or an area It offers design capabilities that allow the simulation of complex scenarios in different types of environments [8] Using evacuation models to optimize the flow of people allows much more ergonomic, effective and safe design at earlier stages of an evacuation plan in crowded places Some of the benefits of evacuation modeling are: designing an effective layout with respect to areas, reducing congestion and queuing at key points, optimizing the position of objects and advertising signage in relation to population profile, facilitating successful operational planning of access, safety and security with regards to normal and emergency circumstances [18] The simulation of a realistic human behavior is important for the virtual world and it has practical values such as emergency planning for evacuating the areas at risk In real world, the safe evacuation of thousands of people is a crucial operation In such cases, obtaining information on the nearest and fasted evacuation routes, the time to evacuate people, the time between the happening and the arrival of the emergency personnel, the procedures that the authorities should use to avoid confusion, and loss of life are all very important By observing and understanding the possible scenarios in a simulated environment beforehand, the authorities (i.e., decision-makers, evacuation managers, safety planners, researchers) can train emergency personnel so that the response to the actual happening is successful [54,55] Thus, we aimed to provide an advantage/benefit for the authorities by developing an agent-based simulation model to have an ideal understanding of the human behaviors and environmental characteristics in a specific zone, be able to estimate possible outcomes of different response and evacuation strategies under different conditions, and generate a series of evacuation plans accordingly In our study, we propose an agent-based evacuation modeling and simulation using the NEAT method [12,41,77,78,76,88] for emergency evacuation situation to help decision makers to determine the evacuation time for buildings or areas at risk In our experiments, we observed successful evacuation behaviors and obstacle avoidance using autonomously-evolved neural networks We also showed that the learning behavior is a crucial step in the direction of emergency planning and how we achieved sophisticated behaviors through our NEAT-based model Related work Evacuation modeling contains iterative processes to determine the best suitable egress routes/paths and calculate the estimated time E-mail address: erkanyuksel@mehmetakif.edu.tr https://doi.org/10.1016/j.aei.2017.11.003 Received 18 November 2016; Received in revised form 26 November 2017; Accepted 29 November 2017 1474-0346/ © 2017 Elsevier Ltd All rights reserved Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel movements of evacuees (i.e., physical abilities, local directional changes, walking speeds, various physiological, psychological and social factors), the interactions and decisions among evacuees They are defined as a bottom-up approach in which evacuees are modeled as individual entities that have unique characteristics such as age, gender, disability, body size and walking speed They describe the time-space behaviors of the individual evacuees [38,58] Formulas, expressions or rules involving spatial transition probabilities are repeatedly applied to temporal changes in situation or behavior Microscopic models are computationally intensive Performing simulation of large-scale crowds is difficult on traditional single-processor systems However, parallel computing techniques can be used to achieve this problem successfully The level of complexity in microscopic models can be overcome by using analytical methods (i.e., to define the path choice using mathematical equations) Analytical methods are cheap to use However, they are limited to the complexity of evacuation problems Because, as analytical methods become more sophisticated and more detailed in an evacuation model, they can become more complex to analyze the evacuation process In addition, peak or transient conditions are not modeled, since analytical models assume that the system has reached a steady state or equilibrium Finally, it is hard to determine whether the existing data characteristics can define the system under study without measuring various design parameters [92] Although much study has been done on microscopic models to improve the behavioral realism, evacuation operations, natural locomotion and decision-making process, none of the existing models can realistically analyze the high-density crowds In order to develop an efficient model, many choices and parameters (based on conditions, cases, events, environments, and so on) have to be considered Some classifications can be helpful to facilitate these choices/parameters and get an overview of the available options Zheng et al [93] indicate seven classes for microscopic approaches which are: social force models, cellular automata, agent-based models, lattice gas models, fluid-dynamic models, approaches depending on the experiments with animals, and gametheoretic models In the light of this information, we can present most common microscopic models which can be used to get better results for pedestrian simulation as follows: social forces models [29], rule-based models [63], cellular automata [17], velocity-based models [83,84], and the optimal steps model [71,70] The difference between them is in the discretization of space and time In addition, we can specify the conceptual similarities and differences of these models according to the perspective of scalar fields (also referred to as the superposition principle) for the simulation of pedestrian dynamics [70] The perspective of scalar fields is a powerful method and provides a common mathematical basis for many models, and the different use of it produces distinct emergent effects However, it has limitations such as the flexibility, superposition of binary interactions [50], calibration of model parameters [28,34,49] Furthermore, models based on scalar fields concept that is used as potential, cost, utility, benefit, or probability can be efficient for practical applications but lack a plausible representation of the decision-making process and natural locomotion Consequently, identifying similarities and differences is important to choose an appropriate model or develop a new method if no other model meets the requirements [69] Social force models describe pedestrian behavior (i.e., route choice behavior, certain actions/motions) microscopically through social fields triggered by the social behavior of the individuals In these models; social forces such as comfort zone, tension, stress, emotion, panic, anxiety, pushing and fighting for space, herding, flocking, arching and clogging are modeled, the interactions and decisions between individuals are tested and validated Social force models enable us to create simulations that look more like particle animation than human movement [59] They help researchers to find new relations and produce mathematical rules to understand crowd behaviors in normal and emergency conditions However, they are more complex decision-making processes and require much more effort to achieve realistic results required to evacuate the areas at risk Different models can be developed for evacuation applications depending on the scope, solution methodology, and input parameters In general, existing studies concentrated on modeling evacuation problems that emphasize the estimation of evacuation time are closely related to different research topics such as path planning, navigation in large virtual environments, traffic assignment, operations research, process control, simulation, optimization, network flow and many others [14,43,44,66,73,74,80] Based on the scope, the-state-of-the-art approaches in evacuation modeling research can be mainly classified as macroscopic and microscopic models, both of which can analyze the movements of evacuees over time 2.1 Macroscopic models Macroscopic evacuation models take into account the movement of evacuation as a homogeneous flow They not consider the individual behaviors and movements of evacuees, the interactions and decisions between evacuees for selecting the egress routes They focus on the system as a whole They are defined as a top-down approach in which collective evacuee movements are characterized by model parameters through closed-form expressions Crowds are represented in an aggregate manner using some distinguishing key features such as spatial density, average velocity, and flow rate in relation to the location and time Since the time is a decisive factor for evacuation process, macroscopic models are mostly used to generate good lower bounds for the evacuation time These bounds can be used to analyze existing buildings or help planning new buildings [24,65] Most of macroscopic models use mathematical or analytical methods which depends on static or dynamic network optimization formulations and some integer programming models to solve evacuation problems However, they suffer from loss of accuracy as individual behaviors of evacuees can affect evacuation time [5,26,54] Common macroscopic models are: regression models [42,48], routechoice models [32], queuing models [45], and gas-kinetic models [30] Regression models use statistically determined relations between flow variables to predict pedestrian flow under specific conditions [60] Pedestrian flow characteristics are related to infrastructures such as corridors, lobbies, stairs, ramps, walkways, and so on Route-choice models define pedestrian wayfinding depending on the utility concepts Pedestrians choose the best appropriate destinations to maximize the utility of their trips such as comfort, travel time, convenience, safety and cost Queuing models benefit from Markov chains to describe how pedestrians move from one network node to another In general, nodes represent rooms and links represent doors Markov chains are defined by a set of states together using transition probabilities [60] Only transitions causing state change are taken into account Gas-kinetics models use an analogy with fluid or gas dynamics to determine how crowd density and velocity parameters change over time using partial differential equations [58,79] Macroscopic evacuation models are mainly based on optimization approaches They are computationally efficient and suitable for largescale crowd simulations While these models are good at producing the general density-flow profiles observed in crowd evacuation, they are unable to describe emergent crowd phenomena [26,56] This restriction is understandable and acceptable given that macroscopic models are expressions of deductive reasoning In so doing, many simplifying assumptions have to be made in order to keep such theorems tractable On the contrary, emergent phenomena arises spontaneously from complex and dynamic interactions at lower levels that occur naturally without influence from external signals or conventions [58] 2.2 Microscopic models Microscopic evacuation models analyze the evacuation situation in a detailed manner They consider the individual behaviors and 31 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel The Optimal Steps model [71] is inspired by the rule based approach of cellular automata It overcomes the limitations of cellular automata and provides an improvement towards a locomotion process In contrast to cellular automata, agents not move from cell to cell The movement is not restricted to a spatial grid of any kind but rather occurs in continuous space The spatial discretization represents the human step The optimal steps model defines decision making through utility optimization and locomotion as a discrete stepping process It captures the stepwise motion of a pedestrian and uses it as discretization scheme in the simulation It finds the next position by maximizing the utility or, equivalently, by minimizing the potential It uses local optimization on a circle around a pedestrian to determine the next position In optimal steps model, the target function is a navigational field Each individual’s stride length depends on his/her speed This introduces a delay in adaptation, because all speed measurements involve the past [71] The optimal steps model has to investigate in a series of controlled experiments where pedestrians had to walk around obstacles that made them turn around to various degrees in real life Because, pedestrians are more likely to react instantaneously, and they cannot change their direction of motion arbitrarily They either have to slow down or change the direction slowly Therefore, new modeling concepts can be developed to allow to simultaneously adapt the speed and stride length [89,90] In optimal steps model, agents use a greedy algorithm to reach a target This is based on the superposition of scalar fields and a local optimization scheme The scalar field is interpreted as a utility function mapping each point in the plane to a utility value The local optimization can then be interpreted as utility optimization Although this interpretation is questionable as a representation of human decision-making processes [23], it is accessible to many disciplines (i.e., social sciences, physics) The area searched for the optimal next position coincides with the reach of the step length Thus, one step of an agent in the simulation represents the step of a pedestrian This is intended to bring closer together the physical process of human walking and pedestrian dynamic simulations [89,90] Rule-based models are used to specify more believable pedestrian behaviors for low and medium density crowds They not need to calculate collision detection and response at all They usually accept conservative approaches through applying “wait rules” (i.e., to enforce ordered crowd behavior for medium density crowds in a flocking style) which work well for low densities in daily life crowd simulation, but prevent realism for high-density conditions or panic situations [59] The rule-based models not consider the contacts (interactions) among individuals and therefore they fail to simulate the pushing behaviors of individuals These models can be combined with cognitive models, hence different behavioral rules can be applied to crowds or individuals to obtain more realistic pedestrian behaviors for crowd simulation methodologies [53,72] Cellular Automata are discrete, abstract computational systems that have demonstrated advantageous and useful both as models of complexity and as more specific representations of non-linear dynamics in various research areas They are “mathematical idealizations of physical systems in which space and time are discrete, and physical quantities take on a finite set of discrete values” [91] They are characterized by the discretization of space into cells They evolve in discrete time steps (generation, iteration or cycle) and usually provide a framework for discrete models with locally homogenous interactions Cellular automata are computationally efficient because of the inherent discrete structure of space and time in the simulation, and the implicit spatial data structure of cells The main advantage of cellular automata is their simplicity in use, which allows both for fast implementation and computation There are certain limitations to the cellular automata They limit the study of microscopic behaviors (i.e., stepping behavior, continuous motion, positions in continuous space, inertia, contact forces) in human crowds due to the fixed/coarse discretization of space They not allow for interactions and contacts among individuals They provide realistic results for low-densities, however they offer unrealistic results when individuals in high-density crowd situations are forced into discrete cells The size of cell usually specifies the physical extension of agents and hence overly limits the maximum density Smaller cells can provide more detail and show improvements in some phenomena On the other hand, such cells may cause a loss in computational efficiency More realistic behaviors can be achieved by precomputing paths towards higher-level goals and storing them within the grid [37,69] Finally, cellular automata can be classified according to some characteristic features for the simulation of pedestrian dynamics: they can be deterministic, probabilistic, rule-based approach, floor field concept, and they can use different grid structures such as rectangular, hexagonal and triangular cells Velocity-based models are usually formulated in continuous time and space However, they can also be evaluated at coarse time steps that make them discrete in time and space In order to simulate a model in a computer, discretization is required for numerical computation Contrary to cellular automata, the next position is not chosen depending on the rules or transition probabilities Instead of this, the velocity (the speed and direction) is defined by a first-order ordinary differential equation and then numerically integrated to obtain the agents' positions at discrete simulation time steps Velocity-based models can be classified into three categories: the optimal-velocity models which are also referred to as car-following models or car traffic simulations [84], obstacle-velocity models which were mainly studied in robotics, animation and computational science [87,9,11], the gradient navigation model which is an ordinary differential equation based model to simulate pedestrian dynamics [16,15] In velocity-based models, the decision-making process of an agent is performed by using the velocity function This causes some limitations to model advanced behavioral features It can also be questioned whether the decision-making of the human movement can usually be represented with such equations For the simulation of the physical environment, the velocity function is also limited because of the physical interactions that are commonly modeled with forces which lead to a second-order differential equation 2.3 Machine learning techniques Emergency management and evacuation planning can be improved significantly with the help of information technology solutions and computer-based information systems such as decision support systems Previous studies were actually driven by defence applications [20] including enhanced reality simulators [21] and evacuation models that incorporated models of human mobility and behavior [25] Recent studies focused on agent-based modeling techniques that offer some level of realism by representing each individual evacuee as an agent that follows specific goals [6,22,19,27,35,39] In real world problem of learning a task such as game playing [68] and controlling a robot [86], the behavior must be learned by taking different actions and assigning high values for good decisions and low values for bad decisions depending on the reinforcement feedback For instance, obstacle avoidance is one of the most important aspects of controlling a robot [13] The robot movement would be very restrictive and meaningless without it There are many methods for avoiding obstacles The simplest forms of them are based on image processing techniques These techniques exploit the color differences between the objects They apply edge detection algorithms such as Roberts, Sobel, Prewitt, Canny, Marr-Hildreth to produce an image that contains only edges However, they are very limited and not include any learning mechanisms Understanding the impact of Artificial Intelligence requires autonomous systems that learn to make better decisions Reinforcement learning is one powerful method for doing so, and it can be applied to a wide range of sectors, including industrial control, robotics, manufacturing, logistics, finance, consumer modeling, automotive, telecommunication, healthcare, and game playing It is one of the easy paradigms to apply machine-learning approaches if the problem is simple and cannot be solved by image processing techniques [36] 32 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel the most relevant software systems and tools for crowd simulation, pedestrian dynamics and evacuation modeling that have been developed both from academia and industry are: Acumen [2,4], AnyLogic (AnyLogic Company, 2000), Artificial Fishes [81], Autonomous Pedestrians [67], Crosses [85], Egress (AEA Technology, 2002), Exodus [7], Legion (Legion International, 2003), MACES + HiDAC [57], Massive (Massive Software Inc., 2005), Menge [10], OpenSteer [64], Reactive Navigation [40], Simulex [82], Space Syntax [61], Steps (MottMacDonald, 2003), Vadere [71,70], ViCrowd [52,51] Some of these systems (i.e., Egress, Exodus, Steps) are based on cellular automata approach, hence it is important to understand their movement simulations artifact Using a cellular automata model in these systems creates several problems about grid size, alignment of grid to environment, fatigue factor, speed, and route selection For instance, fixed densities and unrealistic flow rates at doors In such a scenario, the grid size is a critical parameter to calibrate for achieving the desired behavior [58] In addition, some of the software systems mentioned (i.e., Acumen) use particle-based approaches They lack realism when they are applied to three-dimensional virtual objects for animations Finally, most of these referred software systems (i.e., Artificial Fishes, Massive, OpenSteer, Reactive Navigation, Crosses, ViCrowd) are rule-based crowd simulation models and commonly used in industry applications They are suitable for low and medium density crowds However, they yield unnatural emergent behavior in high density crowds situations (i.e., pedestrians stop and wait for space to clear up) They also have several limitations in terms of simulating realistic human behaviors depending on the physiology, psychology, and social interactions [58] Without a supervisor, a robot can evaluate its performance only in terms of final outcomes Whenever the robot collides with an obstacle, a negative reinforcement signal can be generated In the reinforcement learning, the problem of assigning credit is solved by forming associations between sensory inputs and predictions of future actions Several authors proposed the use of reinforcement learning for robot control [31,62] However, for more complicated tasks such as avoiding an obstacle, navigating in a simulated environment and moving the agents towards an exit point using reinforcement learning is less straightforward The main difficulty in these complicated systems is representing the states and choosing the right approximation function Currently, these problems are handled by trial-and-error In addition to this, there are many other challenges in current reinforcement learning research [3,46] For example, storing the values of each state in the memory is expensive since the problems can be very complex As a solution, value approximation approaches such as decision trees or neural networks can be applied [1] Introducing these value estimations causes problems and it affects the quality of the solution Furthermore, similar behaviors usually reappear, and to avoid learning everything all over again, modularity is introduced Finally, it is generally not possible to completely determine the current state Therefore, it will be a hard task to apply traditional reinforcement learning methods to this domain Another common technique used to control a robot or avoid an obstacle is supervised learning This method requires training to control the robot correctly One way for controlling the robot is to randomly move it and record the movements One example of using the combination of reinforcement and supervised learning to avoid obstacles is to start small and allow the evolution to be increasingly complex to help to find the solutions faster, and find more complex solutions [47] In this study, the researchers did driving experiments to control cars remotely at high speeds in outdoor environments that are unstructured They first calculated the estimate depths from single monocular images by applying supervised learning After estimating the depths, the reinforcement learning was applied and a simulator rendered the synthetic scenes According to the output function of the vision system, the system learned the control policy to select the steering direction The limitation of this approach is it is hard to generate training data for complex tasks such as obstacle avoidance [47] If the environment or configuration of the robot changes, the robot must be trained again to solve the problem The obstacle avoidance problem can also be solved by using path planning algorithms Spong et al [75] described an algorithm that uses attractive and repulsive fields to create a path to avoid obstacles James and Tucker [33] used A∗ search algorithm with a path planner As a solution to accomplish complex tasks effectively, such as robot control and obstacle avoidance, researchers also developed neural network based models Neural networks have several advantages They are fast, resistant to noise in comparison to traditional approaches By using neural network based controllers, a robot can successfully navigate in an environment with obstacles by considering the obstacle and target information received from its sensors If there are moving obstacles in the environment and the environment is more complex, avoiding obstacles and moving around will be challenging Therefore, using neural networks will not be sufficient to solve the problems and more effective methods need to be developed Our Multi-Agent based evacuation simulation model Our model offers an open source cross-platform solution based on NEAT, in which physical and analytical models are combined It simulates and analyzes pedestrian dynamics and evacuation process in a public indoor environment through combining the rules based on numerical calculations that consider agent behaviors and decisions affected by environmental conditions The main idea is to create autonomous agents that use learned behaviors, develop their skills, communicate with each other, avoid obstacles and reach targets strategically Thus, our proposed model serves as a valuable software tool for estimating evacuation time, observing pedestrian flow phenomena and designing guidelines for emergency evacuation situations, in particular when the communications and interactions between individuals play a significant role 3.1 System architecture The system architecture of our model is schematically shown in Fig Each autonomous agent represents an individual pedestrian and can act in a collaborative manner An agent senses and evaluates its surroundings, interacts with the environment and other agents, makes optimal decisions depending on the agent behavior model (ABM) and several rules on agent dynamics Rules are derived at the levels of interactions between the agents Consequently, our aim is to study on the potential chaotic situations among the agents as they develop their behaviors and achieve goals in the simulation environment The agents are trained and they develop highly complex ANNs through NEAT NEAT serves as a brain for an agent as illustrated in Fig It controls the agent to select right behaviors based on the agent’s sensory inputs It gets information about the agent’s surroundings and returns the appropriate action that the agent should take at that time step Therefore, the learning process of the agent is observed, the inputs and fitness function are modified accordingly to help the agent avoid obstacles and move to the targets In our model, there are N autonomous agents {A1 , , AN } , each with a unique local position on a plane Each agent develops its behavior and reaches a certain goal position depending on the 2.4 Software systems for pedestrian dynamics and crowd simulation There are many commercially available computational tools and software systems for the design, simulation and analysis of the evacuation process Most of them focus on the modeling of spaces and occupancies, and rarely consider human and social behaviors These systems fall into macroscopic (focus on groups of pedestrians rather than individual characteristics, and analyze high-density, large-scale system) and microscopic (study the characteristics of individual pedestrians and interactions with others) classes [58] Some examples of 33 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig System architecture each agent uses this capability to share its information (i.e., unique ID, current position, desired velocity to neighbors, its fitness) Since we demand advanced behaviors from the agents, we provided them with three detailed sensor arrays for identifying the exits, obstacles, and other agents Each sensor array divides an agent’s environment into eight pie slices and also three concentric zones that act as radars The inner zone detects the presence of objects that can cause a collision with the agent, the middle zone detects the objects close to the agent, and the outer zone detects any objects farther away (Fig 3) The three sensor arrays determine an agent’s situation in the environment Each sensor array consists of three subarrays with 17-elements as illustrated in Fig The subarray with single-element defines the objects colocated in a certain area that the agent occupies The environmental conditions The agents move on the plane without colliding with each other and any static object and their motions are planned in a decentralized manner We modeled each agent Ai as a disc with a radius r (Ai ) , an initial position P (Ai ) , a goal position GP (Ai ) and a preferred velocity PV (Ai ) that is limited to the range with uniformly distributed on [Vmin,Vmax ] At time step t , the agent Ai has a new position NP (Ai ) and moves with a new velocity NV (Ai ) in the environment In the absence of any other agents and static objects on the path to the agent Ai ’s goal position, the following equation is valid; NV (Ai ) = PV (Ai ) ⩽ Vmax In addition, an agent can sense the radius, positions and velocities of its neighboring agents (a subset of agents within a limited fixed sensing range) as shown in Fig Finally, the agents are also capable of the limited one-way communication Thus, Fig NEAT-based agent behavior model 34 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig Agents with egocentric sensors movement is taken in that direction If the movement is outside the plane or onto static objects or another agent then the agent remains stationary instead We take advantage of the homogeneous groups approach in which all individuals within a group use the same controller ANN The homogeneous groups approach is suitable for crowd simulations as it provides better search space scalability From this point of view, our basic method (Fig 6) is to control the agents through neuroevolution, coevolve them in separate subpopulations, and ensure that they communicate with each other in a common task In addition, the fitness function (shared knowledge) of the simulation is represented as a set of goals (i.e., reaching a target, obstacle avoidance, collaboration), encoding the desired behaviors for a group of agents In our method, each autonomous agent performs specific tasks in every training cycle (learning process) in order to develop its behavior An agent perceives its surrounding area through its sensors It uses a database structure to create, update, store, and share knowledge (perceptual data) Its controller ANN processes the knowledge as input values and then generates output values to create a motivation pattern Finally, the agent interprets this motivation pattern to determine what optimal actions should be performed in its current location subarray with 8-elements defines the nearby objects, and the other subarray with 8-elements defines any objects farther away At the time step of the agent’s action, all elements of the three sensor arrays are loaded with the values directly related to the objects that are inside the agent’s eight radial fields of view Consequently, the three sensor arrays are combined to provide a perceptual data (a 51-elements input data) for our NEAT-based method that makes optimal decisions for an agent Each autonomous agent is controlled by an ANN as shown in Fig In order to conform to the definition of a communication or cooperation, each agent uses an identical controller ANN Hence, we basically reutilize the same ANN for each agent This approach indicates that each agent has an identical control policy and any individual differences in the agents’ behaviors arise purely from the differences in their sensor data The controller ANN has ten motion units (wait, move and the eight cardinal directions) in its output layer Thus, the agent has various choices that specify the agent’s desired direction and preferred velocity at a given time step The values obtained from the agent’s sensor arrays are calculated and matched with the ANN’s inputs The inputs are then processed by the ANN to create a motivation/reward pattern at the ANN’s output The motivation/reward pattern determines the agent’s motivation status and helps the agent to make better decisions on how to move and coordinate with the other agents It is interpreted as an optimal choice of one of the actions available to the agent For instance, if the motivation level of the wait is higher than the motivation level of the move then the agent remains stationary for the current turn Otherwise the eight outputs related to the directions are analyzed to find which has the highest motivation level and a 3.2 The NEAT method NEAT is a neuroevolution method (the evolution of ANNs using GAs) that has a great potential to solve complex control problems (with high-dimensional continuous state and action spaces) and perform Fig Agent sensor array 35 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig Structure of the autonomous agent’s controller ANN sequential decision tasks It finds solutions more efficiently than other neuroevolution techniques It consists of putting together the three key features (tracking genes through historical markings, protecting innovation through speciation, minimizing dimensionality through incremental growth from minimal structure) into one system These features cooperate to create a system that is capable of evolving solutions of minimal complexity and maintain a balance between performance and diversity of solutions [12,41,77,88] NEAT’s genetic encoding allows related genes to be easily lined up when two genomes cross over during mating As illustrated in Fig 7, a genome is the genetic material of the ANN and contains information about network connectivity It comprises two chromosomes: node Fig Training the agents through NEAT 36 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig A genotype to phenotype mapping in NEAT [77] Fig Structural mutation in NEAT [77] 37 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig Our NEAT-based pathfinding algorithm diversity of complexifying network structures/topologies simultaneously [77,76] Structural mutation occurs in two forms, as shown in Fig Each mutation enlarges the size of the genome by adding connection gene or node gene In add connection mutation, a single new connection gene is created by connecting two previously unconnected node genes, added to the end of the genome and given the next available historical marker In add node mutation, an existing connection gene is split and a new node gene is inserted between the two new connection genes that are placed where the split connection gene located The split connection gene is disabled and then the two new connection genes are added to the end of the genome and given the next available historical markers This method of adding nodes is used in order to minimize the initial effect of the mutation and incorporate new nodes immediately into the ANN [77,76] chromosome and connection chromosome Node chromosome contains a set of node genes that provides all available nodes in the network Each node gene has a unique identifier and a node type which represents input node, hidden node or output node Connection chromosome contains a group of connection genes, each of them denotes two node genes being connected Each connection gene specifies the input node, output node, connection weight, gene activation that tells whether or not the connection gene is created, and an innovation number which is assigned to each gene corresponding to its order of appearance throughout evolution Innovation number is a historical marker which states the original historical ancestor of each gene in the ANN system It can also be referred to as a global counter that indicates which mutation in the entire evolutionary history caused the creation of that gene Structural mutation in NEAT alters both connection weights and network topologies NEAT starts evolution with a uniform population of small, simple structures without hidden nodes and complexifies the network topology into diverse species over the course of generations, resulting in increasingly complex/advanced behaviors In this way, the network topology does not have to be known a priori and NEAT finds an appropriate level of complexity for the task In complexification, the main characteristic that distinguishes NEAT from other machine learning techniques is its unique approach to maintaining a healthy 3.3 System configuration and parameters We built our model by using Java Programming Language, Eclipse IDE, MongoDB, OpenGL API and the NEAT method ANJI (Another NEAT Java Implementation) [33] is used to implement our NEAT-based model for the purpose of training the agents and observing their behaviors The movement of each agent is controlled by our pathfinding algorithm with NEAT, which is presented in Fig Algorithm and 38 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Table System configuration and parameters in our NEAT-based simulation model Startup parameters Settings Definition random.seed(int) run.reset(boolean) True (1) It is the initial point to create random numbers through a random number generator that allows the exact replication of runs with random elements It specifies whether or not a run is being continued or whether all persistent data is removed and the run is started from scratch Evolution Parameters Settings Definition numberof.generations(int) population.size(int) 20 200 topology.mutation.classic(boolean) False (0) add.connection.mutation.rate(double) 0.02 remove.connection.mutation.rate(double) 0.01 remove.connection.max.weight(int) 100 remove.connection.strategy(int) Skewed It is the number of generations to perform for a run It determines the number of individuals in the initial population, and each generation after reproduction is carried out It defines how topological mutations are handled In our NEAT-based method, the mutation rates of ANJI are the rates at which new topological mutations arise between all possible locations where a valid, effective mutation occurs It is the probability of new connections being added It depends on how the “topology.mutation.classic” parameter is set The initial weights of new connections have a random value coming from the normal distribution It is the rate at which existing connections within the range of the “remove.connection.max.weight” parameter are removed All weights taking place in that range are sorted in ascending order of the weight size These weights are then removed probabilistically in proportion to their size It is the size of weights to be removed by the “remove.connection” operator These weights are sorted in ascending order by size and removed depending on the “remove.connection.mutation.rate” parameter It is a strategy for the “remove.connection” mutation Possible values are the following: Skewed: The probability of a connection being removed is inversely proportional to the weight size Small: Similar to skewed, however, it is not possible for the connections to be removed with preference for those with smallest weight size All: All connections, regardless of weight, have an equal chance of being removed It is the probability of new nodes being added A node can only mutate at an existing connection (It depends on how the “topology.mutation.classic” parameter is set) As connections are removed, some nodes can be stranded Therefore, this parameter defines the rate at which stranded nodes are removed from the chromosome It is the probability of existing connection weights being mutated by adding a random value from the range of the “weight.mutation.std.dev” parameter It is the standard deviation for weight mutation values The weights of connection are mutated by adding a random double value generated from a random normal distribution with currently defined standard deviation The maximum limit for the values of the connection weight The minimum limit for the values of the connection weight The population percentage which survives and reproduces each generation It is defined by the fitness values sorted in descending order It indicates that the fittest individuals from species with the size defined by the “selector.elitism.min.specie.size” parameter are copied unchanged into the next generation It indicates whether or not roulette selection is used The minimum number of individuals that a specie has to contain for its fittest member to be copied unchanged into the next generation • • • add.neuron.mutation.rate(double) 0.001 prune.mutation.rate(double) 1.0 weight.mutation.rate(double) 0.75 weight.mutation.std.dev(double) 1.5 weight.max(double) weight.min(double) survival.rate(double) 500.0 -500.0 0.2 selector.elitism(boolean) True (1) selector.roulette(boolean) selector.elitism.min.specie.size(int) False (0) Speciation Parameters Settings Definition chromosome.compat.excess.coefficient(double) 1.0 chromosome.compat.disjoint.coefficient(double) 1.0 chromosome.compat.common.coefficient(double) 0.4 speciation.threshold(double) 0.2 The coefficient used to specify the genetic difference between two chromosomes The compatibility value is adjusted depending on the number of excess genes The coefficient used to specify the genetic difference between two chromosomes The compatibility value is adjusted depending on the number of disjoint genes The coefficient used to specify the genetic difference between two chromosomes The compatibility value is adjusted according to the differences in the values of common connection weights The compatibility threshold that indicates whether two individuals belong to the same species Fitness Function Parameters Settings Definition stimulus.size(int) response.size(int) fitness_function.class(Java class) 51 10 com.anji.pathfind.PathFindFitnessFunction fitness.function.adjust.for.network.size.factor(double) 0.0 The number of input nodes for the initial neural network topology The number of output nodes for the initial neural network topology It is used to specify the fitness of the evolving population This class depends on the domain This parameter allows for an apparent fitness penalty to be applied to a chromosome depending on its size (continued on next page) 39 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel force approaches but in general, they are very expensive We researched into algorithms which are fast and inexpensive, easy to understand and flexible In our study, the agents have several sensors that sense collisions with other agents or obstacles Each of these sensors detect the number of collisions it is having with its surroundings The number of collisions from each sensor, along with a bias and an angle between the agent’s target and the agent, these parameters act as inputs to the ANN The ANN then generates two outputs which are the speed correction and the degree (or angle) correction These outputs are used to update the speed and the direction of each agent As soon as the agent comes near the target by a particular distance, then, it is considered to have reached the target For the collision detection, we were inspired by ray tracing algorithm which is commonly used A ray is represented using a vector which has a start point and a vector which represents the direction in which the ray travels The ray starts from the start point and travels in the direction of the direction vector The equation for the ray is: Table Computer system specifications used for our simulation model Hardware Specification Display Processor Memory Storage Graphics 27 inch (diagonal) Retina K 4.2 GHz quad-core Intel Core i7, Turbo Boost up to 4.5 GHz 32 GB 2400 MHz DDR4 TB SSD AMD Radeon Pro 580 with GB VRAM Software Specification Operating System macOS High Sierra (Version 10.13) In our model, we applied an initial configuration with a set of evolutionary parameters that are presented in Table to evolve a good ANN The agents choose right behaviors, communicate with each other and make their movements depending on the corrections given by the ANN that was evolved through our fitness function class In addition, elitism was selected as we wanted the genetic algorithm to converge more quickly With elitism, the fittest chromosomes in each generation are guaranteed to be copied unchanged into new population This means that the fittest genome will never be lost to random chance PoR = RSP + (t ∗RD) (1) In Eq (1); PoR is a point on the ray, RSP is the starting point of the ray, and RD is the direction of the ray t is a float which takes values from to infinity If t = , we have the start point and substituting other values we have the corresponding points along the ray For the agent, obstacle collision, we used Ray-Plane Intersection Detection technique The plane is represented using its vector representation as follows: 3.4 Obstacle detection and collision avoidance Xn · X = d Collision detection and response are difficult tasks and there are not easy solutions for these problems For every application, there is a different way of finding and testing for collisions There are also brute (2) In Eq (2); Xn and X are vectors Xn is its normal and X is a point on its surface d is a float representing the distance of the plane along the Fig 10 Status of the agents at the beginning of the simulation 41 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig 11 Status of the agents while the simulation is running (after 60 ss) 20 m × 80 m × m) The agents are randomly given a start position Each individual agent is trained to find a best suitable route and reach the nearest target to itself Each individual moves from a location to another by a speed uniformly distributed on [0.2 m/s, 1.6 m/s] In addition, the diameter of each individual is uniformly distributed on [0.3 m, 0.5 m] The simulation is initiated when an emergency evacuation situation caused by an incident occurs None of the agents are aware of the emergency status at the beginning of the simulation When the incident takes place around the crowd in the environment, our trained agents become aware of the incident, develop their behaviors (i.e., choosing and applying right action, communicating, collaborative learning, switching to emergency state, escaping), try to get away from the region at a certain speed in reaction to the incident, and reach the targets First, we carried out an experimental study of a scenario of 200 pedestrians for our model We then simulated the scenario by increasing the number of pedestrians Finally, we compared our simulation model with common pedestrian simulation software tools for the scenarios of 250 and 500 pedestrians, respectively Fig 10 illustrates our simulation for a scenario of 200 pedestrians After starting the simulation and finding the right number of generations and the population size, the agents developed their skills and headed towards the exists (Figs 11, 12) Towards the end of the simulation as shown in Fig 13, the agents successfully reached their targets Table indicates the number of evacuees from the exits at specific time intervals Fig 14 illustrates the density at targets depending on the time Consequently, our simulation model is capable of evacuating 200 individuals in 208 s for the first scenario, as shown in Fig 15 normal, from the center of the coordinate system If the ray intersects the plane at a point, therefore there will be some points on the ray which satisfies the plane equation as follows: (Xn ·RSP ) + t ∗ (Xn ·RD) = d (3) In Eq (3); t represents the distance from the start until the intersection point along the direction of the ray Substituting t into the ray equation, we find our collision point After we found where the collision takes place, we find the intersection in the current time step which is the time we move our agent from its current point according to its velocity After determining the time step, we move our agent, calculate its new position and find the distance between the start and end point Additionally, we find the distance from the start point to its collision point If this distance is less than the distance between start and end points, we conclude that there is a collision and avoid it Results In our simulation, the test platform is a personal computer which its system specifications are summarized in Table We simulated our model for a public indoor environment with multiple exits We can change environmental features depending on our needs and add new items to the test area It is also possible to work with real floor plans and 3D models The environment consists of agents that represent pedestrians, an indoor area of 100 m × 100 m, one arrival line where crowd is generated, three targets (w/h: 1.5 m/2.1 m), obstacles which may be walls, agents or any solid object in between the agents and exits (there are three obstacles and their dimensions are: O1_w/l/h: 20 m x 30 m x m, O2_w/l/h: 30 m × 20 m × m, O3_w/l/h: 42 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig 12 Status of the agents while the simulation is running (after 120 ss) Our methodology and findings demonstrate that neuroevolution techniques are powerful and play significant roles in areas where supervised learning is limited or not possible During our tests, we saw that when we ran the evolution for 20 generations, the fittest chromosomes were the champions of 16 previous generations It would be better if there was a way of stopping the evolution process once the fitness reaches a plateau This would also decrease the evolution time Figs 16 and 17 show the fitness values for the generations of two different trials Minimum and maximum fitness for each generation are denoted with blue⁎ line The average fitness for the population is indicated with black dot We found out that as the environment becomes more complex, the evolution took more time This is understandable as the fitness function became more complex In our simulation, we first started without any obstacles and with less number of agents; the evolution was fast in reaching the desired number of generations We then added multiple exits into the environment, and thus, the time spent for each generation increased Furthermore, we added obstacles and targets near the obstacles; at this stage, the evolution took around seven minutes per generation (population size being 200) We observed that choosing the right set of input values as the first step is important than exploring the appropriate fitness function As a result, the parameters and threshold values can be adjusted Additionally, we experimented with sigmoid activation function and function, and compared them with each other We saw that the sigmoid function performed better than the function For 15 trials, the difference was not significant However, the results agree with the common belief that the sigmoid function performs better We also experimented with the population size and observed how it affected the performance The simulation was not very successful with the population size 25 Furthermore, the agents were not able to reach the targets and they did circular movements When we increased the population size to 50 without changing the number of generations, the agents began to perform better and moved faster However, they were not able to reach the targets We also tried 100 and 200 for the population size and saw that the performance was better than the previous ones The agents moved faster and reached their targets successfully Increasing the number of generations also affected the performance of our solution In our experiments, we increased the number of generations from 25 to 50 and at the same time we increased the population size from 50 to 100 We observed that the agents’ performances improved, their skills developed, and they found their ways to the targets more effectively 4.1 Performance comparison In this section, we presented performance evaluations of our model and some of the known software tools (AnyLogic, MassMotion, Pedestrians Dynamics, PTV Viswalk, STEPS, and VADERE) These evaluations emphasize the applicability and efficiency in the evacuation domain to compare the contribution of our approach with others We implemented our method to analyze the learning process and verify the quality of generated pedestrian dynamics We first configured the elements (computational components that serve specific purposes) of our ⁎ For interpretation of color in Figs 16 and 17, the reader is referred to the web version of this article 43 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig 13 Status of the agents towards the end of the simulation (after 180 ss) comparison, we used similar configurations and settings for the mentioned software tools considering the specifications of our evacuation domain In all test cases, we observed that our model produced similar results with others, however, agents trained with NEAT tend to adapt their behaviors rather fast in order to achieve their goals Furthermore, our model provided smooth flows and realistic motions thanks to NEAT In low-density crowd, all simulation models worked very well As soon as the environment became more complex, all software tools produced some unrealistic motions at first To increase the realism of simulations, we changed the characteristics of agents, varied the speeds among agents as well as the waiting-time of each agent and the size of its personal space In both our model and other mentioned software tools, we noted that the agents speed up, slow down or change their orientations to reach their goals Although no significant differences were found for average travel times and the speeds of individuals, our model led to considerably short evacuation time Besides the quality of generated behaviors, we were also greatly interested in the performance of our proposed model Since, on the average, CPU and memory usages were not too busy for each time step of the simulation, we experimentally confirmed that our NEAT-based simulation model can be applied to pathfinding and evacuation modeling efficiently Figs 18 and 19 show performance evaluations of AnyLogic multimethod simulation modeling software In our tests, AnyLogic was capable of evacuating 250 individuals in 238 s for the scenario of 250 pedestrians, and 500 individuals in 293 s for the scenario of 500 pedestrians Figs 20 and 21 present performance evaluations of MassMotion crowd simulation and pedestrian analysis software In our tests, Table Number of evacuees at targets Time (s) Target-1- Target-2- Target-3- Total number of evacuees Average memory usage (MB) Startup 30 60 90 120 150 180 208 0 25 22 12 0 0 14 13 12 12 0 31 30 18 0 0 62 66 43 13 12 95 165 124 266 175 296 149 238 software framework, determined optimal settings, and calibrated NEAT parameters of our model to carry out simulation scenarios During tests, we observed crucial situations in the scenarios and analyzed the behaviors of agents for each time step of the simulation We devised a number of quantitative quality metrics and test-cases In particular, we examined the density at targets, distribution of evacuation, average evacuation times, crowd flow, cost-efficient (time-optimal) paths, and average speeds Total acceleration of an agent and degrees it turned were also examined to get an idea of the amount of movement and turning effort spent by the agent To demonstrate the usability of our model in evacuation modeling, we performed some experiments through creating scenarios consisting of 250 and 500 pedestrians, respectively Each agent enters the environment through an arrival gate, communicates if necessary, avoids obstacles, and moves towards randomly selected exits For a fair 44 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel Fig 14 Density at targets Fig 15 General evacuation situation Fig 16 Trial Fig 17 Trial MassMotion was capable of evacuating 250 individuals in 229 s for the scenario of 250 pedestrians, and 500 individuals in 257 s for the scenario of 500 pedestrians Figs 22 and 23 illustrate performance evaluations of Pedestrian Dynamics crowd simulation and modeling software In our tests, Pedestrian Dynamics was capable of evacuating 250 individuals in 256 s for the scenario of 250 pedestrians, and 500 individuals in 276 s for the scenario of 500 pedestrians Figs 24 and 25 show performance evaluations of PTV Viswalk pedestrian traffic simulation software In our tests, PTV Viswalk was capable of evacuating 250 individuals in 253 s for the scenario of 250 pedestrians, and 500 individuals in 281 s for the scenario of 500 pedestrians Fig 26 and 27 present performance evaluations of STEPS pedestrian movement software tool In our tests, STEPS was capable of evacuating 45 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 18 Evacuation scenario of 250 pedestrians in AnyLogic Professional 8.2 a) Number of evacuees at targets c) b) Density at targets Distribution of evacuation d) General evacuation situation Fig 19 Evacuation scenario of 500 pedestrians in AnyLogic Professional 8.2 46 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 20 Evacuation scenario of 250 pedestrians in MassMotion 9.0 a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 21 Evacuation scenario of 500 pedestrians in MassMotion 9.0 47 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 22 Evacuation scenario of 250 pedestrians in Pedestrian Dynamics 3.1 a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 23 Evacuation scenario of 500 pedestrians in Pedestrian Dynamics 3.1 48 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 24 Evacuation scenario of 250 pedestrians in PTV Viswalk 10 a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 25 Evacuation scenario of 500 pedestrians in PTV Viswalk 10 49 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 26 Evacuation scenario of 250 pedestrians in STEPS 5.4 a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 27 Evacuation scenario of 500 pedestrians in STEPS 5.4 50 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 28 Evacuation scenario of 250 pedestrians in VADERE a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 29 Evacuation scenario of 500 pedestrians in VADERE 51 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 30 Evacuation scenario of 250 pedestrians in our NEAT-based simulation model a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation d) General evacuation situation Fig 31 Evacuation scenario of 500 pedestrians in our NEAT-based simulation model 52 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel b) For 500 pedestrians a) For 250 pedestrians Fig 32 Comparison of evacuation time and general evacuation situation Table Performance comparison of our model with other simulation software tools Our Model AnyLogic MassMotion Pedestrian Dynamics PTV Viswalk STEPS VADERE Evacuation Time (s) For 250 pedestrians For 500 pedestrians Average Memory Usage For 250 pedestrians For 500 pedestrians Average CPU Usage (%) For 250 pedestrians For 500 pedestrians 210 238 229 256 253 247 220 254 293 257 276 281 265 298 238 MB 390 MB 395 MB 410 MB 454 MB 374 MB 1.25 GB 297 MB 475 MB 418 MB 456 MB 487 MB 430 MB 1.32 GB 57 85 45 53 61 92 58 62 89 48 59 64 96 65 Table Comparison of different software systems for evacuation modeling and simulation Evacuation model Behavior method Collision response Communication Learning Roles (Individuals) Real time Spatial structure Acumen Particle based Cont AnyLogic Rule based Cont Artificial Fishes Rule based Cont Autonomous Pedestrians Artificial life approach Cont Cellular Automata Cellular automata 2D Grid Crosses Rule based + FSM Egress Cellular automata some Hexagonal Grid Exodus Cellular automata some 2D Grid Legion Least-effort some Cont Maces + Hidac Social forces Massive Rule based + Fuzzy Logic OpenSteer Rule based Reactive Navigation Rule based Rule-based Rule based Simulex Distance maps some Cont Social Forces Socail forces some Cont Space Syntax Visibility graphs Steps Cellular automata Vadere Optimal Steps Model 2D Grid ViCrowd Rule based + FSM Cont Our Model NEAT Cont Cont some Cont Cont some Cont Cont Cont Cont some 2D Grid 250 individuals in 220 s for the scenario of 250 pedestrians, and 500 individuals in 298 s for the scenario of 500 pedestrians Figs 30 and 31 show performance evaluations of our NEAT-based simulation model Our model was capable of evacuating 250 individuals in 210 s for the scenario of 250 pedestrians, and 500 250 individuals in 247 s for the scenario of 250 pedestrians, and 500 individuals in 265 s for the scenario of 500 pedestrians Figs 28 and 29 illustrate performance evaluations of VADERE (open source microscopic pedestrian dynamics simulation framework) with Optimal Steps Model In our tests, VADERE was capable of evacuating 53 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel processes, and demonstrated that this neuroevolution method is feasible to apply In our simulation scenarios, agents were able to accomplish their tasks and found their ways to exists, that means topology evolution by using NEAT is efficient, provides advantage and can be applied in the area of evacuation modeling Consequently, our approach provides a good solution that helps decision-makers to meet the evacuation modeling and tactics development goals It is less complex than analytical models It provides a NEAT-based software tool to study the behavior of evacuees without incurring the cost of performing and analyzing real-time evacuation operations, and to develop alternative methods to evacuation simulation models before a commitment is made to one Decision-makers, evacuation managers, safety planners, researchers can develop their systems through our open source software platform to design and simulate various what-if scenarios, and generate different evacuation plans under different conditions by examining their results individuals in 254 s for the scenario of 500 pedestrians Figs 32 represents the comparison of evacuation time and evacuation situation between our model and other mentioned simulation software tools Furthermore, Table presents overall performance evaluation of our model and these tools Consequently, our experimental studies and results demonstrate significant improvement in evacuation rates when using NEAT This may shed light to the studies of pedestrian dynamics and crowd simulation Conclusion There are various approaches to analyze evacuation process and minimize the evacuation time based on the level of abstraction and the level of complexity In addition, several models involve simulation techniques to determine the flow conditions and traffic patterns depending on a set of rules These models can estimate the evacuation time as a function of flow They act as a software framework in order to evaluate private scenarios and generate solution recommendations Table presents a comparison of some of the most significant software tools in pedestrian dynamics and crowd simulation at a functional level This emphasizes the basic features in evacuation modeling to compare the contribution of our model with others The main problem in evacuation simulation models is that they not have the ability to analyze pedestrian and crowd behaviors, estimate the evacuation time, and choose the best suitable routes that provide safe access and egress This observation, actually, sparked the following state-of-the-art approach in our study; if we are able to build an agent based model that integrates an optimization method and a simulation routine for a specific scenario, then significant improvements to evacuation modeling can be achieved Therefore, we developed a multi-agent based simulation model with NEAT to study the pedestrian dynamics, analyze the crowd flow and group behaviors, minimize the evacuation time in an emergency evacuation situation In our study, we use NEAT to train autonomous agents in the evacuation domain, discover highly sophisticated and successful strategies, and solve challenging tasks such as navigation, communication, obstacle detection, and collision avoidance NEAT supplies GAs to evolve increasingly complex ANN architectures effectively Evolution starts with small and simple ANNs, and as evolution progresses, ANNs turn into more complex structures over generations (characteristic features of the ANN’s topology increase, either by adding a new neuron into a connection path or by adding a new connection between neurons) NEAT starts without any hidden layer and then adjusts weights while adding hidden neurons and further connections This process of starting with small networks and allowing evolution to perform complexification helps to find complex solutions (behaviors) and solve the problems faster Hence, our approach is used to model the increased complexity and detail effectively It overcomes the limitations of analytical techniques by handling the varying levels of complexity through NEAT (i.e., peak loads and transient behaviors can be examined) Since it represents system more faithfully than macroscopic and microscopic models, our model is well-defined, well-designed, well-established, and can be more common in practice It provides numerous options to design a system using NEAT The realism degree incorporated into the model directly reflects the level of detail it incorporates Sources of performance fluctuations can be identified by varying the design parameters Unfortunately, it is usually not possible to evacuate people by carrying out collectively live practices and safety procedures Because conducting live exercises and safety procedures is a time-consuming process and results in tremendous lost revenue Therefore, developing a feasible strategy for the evacuation of people is very crucial In this paper, we presented an agent-based simulation model based upon NEAT that can be used to help the authorities or emergency management personnel to develop evacuation strategies By employing the NEAT, we analyzed the agents’ behaviors, studied the learning Appendix A Supplementary material Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.aei.2017.11.003 References [1] B Abdulhai, L Kattan, Reinforcement learning: introduction to theory and potential for transport applications, Can J Civil Eng 30 (6) (2003) 981–991 [2] J Allbeck, K Kipper, C Adams, W Schuler, E Zoubanova, N.I Badler, M Palmer, A.K Joshi, ACUMEN: Amplifying Control and Understanding of Multiple ENtities, in: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems, Bologna, Italy, 2002, pp 191–198 [3] A.G Barto, S Mahadevan, Recent advances in hierarchical reinforcement learning, Disc Event Dynamic Syst.: Theory Applications 13 (4) (2003) 341–379 [4] R Bindiganavale, W Schuler, J.M Allbeck, N.I Badler, A.K Joshi, M Palmer, Dynamically altering agent behaviors using natural language instructions, in: Proceedings of the Fourth International Conference on Autonomous Agents, Barcelona, Spain, 2000, pp 293–300 [5] S Bretschneider, Mathematical Models for Evacuation Planning in Urban Areas, Springer-Verlag, Berlin Heidelberg, Germany, 2013 [6] M Busogi, N Kim, D Shin, H.B Ryu, A Yoo, D Kim, Bayesian affordance-based agent model for wayfinding behaviors in evacuation problems, in: Duffy V.G (Ed.), Digital Human Modeling and Applications in Health, Safety, Ergonomics, and Risk Management Healthcare and Safety of the Environment and Transport, SpringerVerlag, Berlin Heidelberg, Germany, 2013, pp 297–306 [7] N Chooramum, P.J Lawrence, E Galea, Urban scale evacuation simulation using buildingEXODUS, the 14th International Conference and Exhibition on Fire Science and Engineering (Interflam 2016), London, UK, 2016, pp 1645–1656 [8] A Cuesta, O Abreu, D Alvear (Eds.), Evacuation Modeling Trends, Springer, Cham, Switzerland, 2016 [9] S Curtis, Pedestrian velocity obstacles: Pedestrian simulation through reasoning in velocity space, PhD Thesis University of North Carolina at Chapel Hill, NC, USA, 2013 [10] S Curtis, A Best, D Manocha, Menge: a modular framework for simulating crowd movement, Collect Dynamics (2016) 1–40 [11] S Curtis, D Manocha, Pedestrian simulation using geometric reasoning in velocity space, in: U Weidmann, U Kirsch, M Schreckenberg (Eds.), Pedestrian and Evacuation Dynamics 2012, Springer, London, UK, 2014, pp 875–890 [12] D.B D'Ambrosio, K.O Stanley, Scalable multiagent learning through indirect encoding of policy geometry, Evol Intel (1) (2013) 1–26 [13] T D’Silva, R Miikkulainen, Learning dynamic obstacle avoidance for a robot arm using neuroevolution, Neural Process Lett 30 (1) (2009) 59–69 [14] M.J Dibley, H Li, J.C Miles, Y Rezgui, Towards intelligent agent based software for building related decision support, Adv Eng Inf 25 (2) (2011) 311–329 [15] F Dietrich, S Disselnkötter, G Köster, How to get a model in pedestrian dynamics to produce stop and go waves, in: V Knoop, W Daamen (Eds.), Traffic and Granular Flow'15, Springer, Cham, 2016, pp 161–168 [16] F Dietrich, G Köster, Gradient navigation model for pedestrian dynamics, Phys Rev E 89 (6) (2014) 062801 [17] J Dijkstra, H.J.P Timmermans, A.J Jessurun, A multi-agent cellular automata system for visualizing simulated pedestrian activity, in: S Bandini, T Worsch (Eds.), Theoretical and Practical Issues on Cellular Automata, Springer, London, UK, 2000, pp 29–36 [18] H.-S Gana, K.-F Richter, M Shic, S Winterd, Integration of simulation and optimization for evacuation planning, Simul Model Pract Theory 67 (2016) 59–73 [19] E Gelenbe, H Bi, Emergency navigation without an infrastructure, Sensors 14 (8) (2014) 15142–15162 [20] E Gelenbe, Y Cao, Autonomous search for mines, Eur J Oper Res 108 (2) (1998) 319–333 [21] E Gelenbe, K Hussain, V Kaptan, Simulating autonomous agents in augmented reality, J Syst Softw 74 (3) (2005) 255–268 [22] E Gelenbe, F Wu, Large scale simulation for human evacuation and rescue, Comput Math Appl 64 (12) (2012) 3869–3880 54 Advanced Engineering Informatics 35 (2018) 30–55 M.E Yuksel [23] G Gigerenzer, P.M Todd, A.B.C Research Group, Simple Heuristics That Make Us Smart, Oxford University Press, Oxford, UK 1999 [24] I Ginzburg, D d’Humières, Multi-reflection Boundary Conditions for Lattice Boltzmann Models, Phys Rev E 68 (6) (2003) 066614 [25] S Gwynne, E Galea, M Owen, P.J Lawrence, L Filippidis, A review of the methodologies used in evacuation modelling, Fire Mater 23 (6) (1999) 383–388 [26] H.W Hamacher, S.A Tjandra, Mathematical modelling of evacuation problems: a state of art, in: M Schreckenberg, S.D Sharma (Eds.), Pedestrian and Evacuation Dynamics, Springer, Berlin Heidelberg, Germany, 2002, pp 227–266 [27] Q Han, Managing emergencies optimally using a random neural network-based algorithm, Future Internet (2013) 515–534 [28] T Hastie, R Tibshirani, J Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Series in Statistics, Springer, NY, USA, 2009 [29] D Helbing, I Farkas, T Vicsek, Simulating dynamical features of escape panic, Nature 407 (2000) 487–490 [30] L.F Henderson, The statistics of crowd fluids, Nature 229 (5284) (1971) 381–383 [31] T Hester, M Quinlan, P Stone, RTMBA: A real-time model-based reinforcement learning architecture for robot control, in: IEEE International Conference on Robotics and Automation (ICRA 2012), St Paul, MN, USA, 2012 pp 85–90 [32] S.P Hoogendoorn, P.H.L Bovy, Pedestrian travel behavior modeling, Networks Spatial Econ (2) (2005) 193–216 [33] D James, P Tucker, A comparative analysis of simplification and complexification in the evolution of neural network topologies, in: Genetic and Evolutionary Computation Conference, Seattle, Washington, USA, 2004, pp 1–11 [34] A Johansson, D Helbing, P Shukla, Specification of the social force pedestrian model by evolutionary adjustment to video tracking data, Adv Complex Syst 10 (2007) 271–288 [35] J Joo, N Kim, R.A Wysk, L Rothrock, Y.-J Son, Y.-G Oh, S Lee, Agent-based simulation of affordance-based human behaviors in emergency evacuation, Simul Model Pract Theory 32 (2013) 99–115 [36] L.P Kaelbling, M.L Littman, A.W Moore, Reinforcement learning: a survey, J Arti Intell Res (1) (1996) 237–285 [37] A Kirchner, A Namazi, K Nishinari, A Schadschneider, Role of conflicts in the floor field cellular automaton model for pedestrian dynamics, the 2nd International Conference in Pedestrians and Evacuation Dynamics, London, UK, 2003, pp 51–62 [38] A Kneidl, A Borrmann, D Hartmann, Generation and use of sparse navigation graphs for microscopic pedestrian simulation models, Adv Eng Inf 26 (4) (2012) 669–680 [39] E.D Kuligowski, R.D Peacock, B Hoskins, A review of building evacuation models [online] Technical note 1680, Fire Research Division Engineering Laboratory, National Institute of Standards and Technology, U.S Department of Commerce, 2010 (accessed 25 November 2017) [40] F Lamarche, S Donikian, Crowd of virtual humans: a new approach for real time navigation in complex and structured environments, Comput Graph Forum 23 (3) (2004) 509–518 [41] J Lehman, S Risi, D.B D'Ambrosio, K.O Stanley, Encouraging reactivity to create robust machines, Adapt Behav 21 (6) (2013) 484–500 [42] D Levinson, W Chen, Traffic management system performance using regression analysis [online] California PATH Working Paper UCB-ITS-PWP-2006-3, Institute of Transportation Studies, University of California, Berkeley, 2006 (accessed 25 November 2017) [43] Y.-H Lin, Y.-S Liu, G Gao, X.-G Han, M Gu, The IFC-based path planning for 3D indoor spaces, Adv Eng Inf 27 (2) (2013) 189–205 [44] M Liu, F Zhang, Y Ma, H.R Pota, W Shen, Evacuation path optimization based on quantum ant colony algorithm, Adv Eng Inf 30 (3) (2016) 259–267 [45] G.C Lovas, Modeling and simulation of pedestrian traffic flow, Transport Res Board 28 (6) (1994) 429–443 [46] T.V Maia, Reinforcement learning, conditioning, and the brain: Successes and challenges, Cognitive, Affective, Behav Neurosci (4) (2009) 343–364 [47] J Michels, A Saxena, A.Y Ng, High speed obstacle avoidance using monocular vision and reinforcement learning, in: the 22nd International Conference on Machine Learning (ICML), Bonn, Germany, 2005, pp 593–600 [48] J.S Milazzo, N.M Rouphail, J.E Hummer, D.P Allen, The effect of pedestrians on the capacity of signalized intersections, Transport Res Record: J Transport Res Board 1646 (1998) 37–46 [49] M Moussaid, D Helbing, S Garnier, A Johansson, M Combe, G Theraulaz, Experimental study of the behavioural mechanisms underlying self-organization in human crowds, Procee Royal Soc B: Biolo Sci 276 (1668) (2009) 2755–2762 [50] M Moussaid, D Helbing, G Theraulaz, How simple rules determine pedestrian behavior and crowd disasters, Proc Natl Acad Sci 108 (17) (2011) 6884–6888 [51] S.R Musse, C Babski, T Capin, D Thalmann, Crowd modelling in collaborative virtual environments, in: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, Taipei, Taiwan, 1998, pp 115–123 [52] S.R Musse, D Thalmann, A model of human crowd behavior: group inter-relationship and collision detection analysis, in: D Thalmann, M van de Panne (Eds.), Computer Animation and Simulation’97, Eurographics, Springer, Vienna, 1997, pp 39–51 [53] S.R Musse, D Thalmann, Hierarchical model for real time simulation of virtual human crowds, IEEE Trans Visual Comput Graph (2) (2001) 152–164 [54] A Naser, A.K Kamrani, Intelligent Transportation and Evacuation Planning: A Modeling-Based Approach, Springer, Norwell, MA, USA, 2012 [55] X Pan, Computational Modeling of Human and Social Behaviors for Emergency Egress Analysis, PhD Thesis Stanford University, California, USA, 2006 [56] X Pan, C.S Han, K Dauber, K.H Law, A multi-agent based framework for the simulation of human and social behaviors during emergency evacuations, AI&SOCIETY 22 (2) (2007) 113–132 [57] N Pelechano, J.M Allbeck, N.I Badler, Controlling individual agents in high- [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] 55 density crowd simulation, ACM SIGGRAPH/Eurographics Symposium on Computer Animation, San Diego, CA, USA, 2007, pp 99–108 N Pelechano, J.M Allbeck, N.I Badler, Virtual Crowds: Methods, Simulation, and Control, Morgan & Claypool, San Rafael, CA, USA, 2008 N Pelechano, A Malkawi, Comparison of crowd simulation for building evacuation and an alternative approach, the 10th International Building Simulation Conference and Exhibition, Beijing, China, 2007, pp 1514–1521 N Pelechano, A Malkawi, Evacuation simulation models: challenges in modeling high rise building evacuation with cellular automata approaches, Autom Constr 17 (4) (2008) 377–385 A Penn, A Turner, Space syntax based agent simulation, in: M Schreckenberg, S Sharma (Eds.), Pedestrian and Evacuation Dynamics, Springer, Germany, 2002, pp 99–114 J Provost, B.J Kuipers, R Miikkulainen, Self-organizing perceptual and temporal abstraction for robot reinforcement learning, AAAI-04 Workshop on Learning and Planning in Markov Processes, San Jose, CA, USA, 2004, pp 79–84 C.W Reynolds, Flocks, herds, and schools: a distributed behavior model, ACM SIGGRAPH Comput Graph 21 (4) (1987) 25–34 C.W Reynolds, Steering behaviors for autonomous characters, Game Developers Conference, San Jose, CA, USA, 1999, pp 763–782 E Ronchi, D Nilsson, Modelling total evacuation strategies for high-rise buildings, Build Simul (1) (2014) 73–87 U Rüppel, K Schatz, Designing a BIM-based serious game for fire safety evacuation simulations, Adv Eng Inf 25 (4) (2011) 600–611 W Shao, D Terzopoulos, Autonomous pedestrians, Graph Models 69 (5–6) (2007) 246–274 J Schrum, Discovering multimodal behavior in MS Pac-Man through evolution of modular neural networks, IEEE Trans Comput Intell AI Games (1) (2015) 67–81 M.J Seitz, Simulating pedestrian dynamics: Towards natural locomotion and psychological decision making, PhD Thesis The Technical University of Munich, Munich, Germany, 2016 M.J Seitz, F Dietrich, G Köster, H.-J Bungartz, The superposition principle: A conceptual perspective on pedestrian stream simulations, Collect Dyn (A2) (2016) 1–19 M.J Seitz, G Köster, Natural discretization of pedestrian movement in continuous space, Phys Rev E 86 (4) (2012) 046108 W Shao, D Terzopoulos, Animating autonomous pedestrians in urban environments, ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Los Angeles, CA, USA, 2005, pp 19–28 J.A Sokolowski, C.M Banks (Eds.), Handbook of Real-World Applications in Modeling and Simulation, John Wiley & Sons, Hoboken, New Jersey, 2012 A.R Soltani, H Tawfik, J.Y Goulermas, T Fernando, Path planning in construction sites: performance evaluation of the Dijkstra, A∗, and GA search algorithms, Adv Eng Inf 16 (4) (2002) 291–303 M.W Spong, S Hutchinson, M Vidyasagar, Robot Modelling and Control, NY, USA, 2006 K.O Stanley, D.B D'Ambrosio, J Gauci, A hypercube-based encoding for evolving large-scale neural networks, Artif Life 15 (2) (2009) 185–212 K.O Stanley, R Miikkulainen, Evolving neural networks through augmenting topologies, Evol Comput 10 (2) (2002) 99–127 K.O Stanley, R Miikkulainen, Competitive coevolution through evolutionary complexification, J Arti Intell Res 21 (1) (2004) 63–100 Q Sun, A Generic Approach to modelling individual behaviors in crowd simulation, PhD Thesis University of Salford, Manchester, United Kingdom, 2013 T.-A Teo, K.-H Cho, BIM-oriented indoor network model for indoor and outdoor combined route planning, Adv Eng Inf 30 (3) (2016) 268–282 D Terzopoulos, X Tu, R Grzeszczuk, Artificial fishes: autonomous locomotion, perception, behavior, and learning in a simulated physical world, Artif Life (4) (1994) 327–351 P.A Thompson, E.W Marchant, A computer model for the evacuation of large building populations, Fire Saf J 24 (2) (1995) 131–148 A Tordeux, M Chraibi, A Seyfried, Collision-free first order model for pedestrian dynamics, in: the 11th International Conference on Traffic and Granular Flow, Nootdorp, the Netherlands, 2015, pp 71 A Tordeux, A Seyfried, Collision-free nonuniform dynamics within continuous optimal velocity models, Phys Rev E 90 (4) (2014) 042812 B Ulicny, D Thalmann, Crowd simulation for interactive virtual environments and VR training systems, in: N Magnenat-Thalmann, D Thalmann (Eds.), Computer Animation and Simulation 2001, Eurographics 2001, Springer, Vienna, 2001, pp 163–170 V.K Valsalam, J Hiller, R MacCurdy, H Lipson, R Miikkulainen, Constructing controllers for physical multilegged robots using the ENSO neuroevolution approach, Evol Intel (1) (2012) 45–56 J van den Berg, M.C Lin, D Manocha, Reciprocal velocity obstacles for real-time multi-agent navigation, IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 2008, pp 1928–1935 P Verbancsics, K.O Stanley, Evolving static representations for task transfer, J Mach Learn Res 11 (2010) 1737–1769 I von Sivers, G Köster, Dynamic stride length adaptation according to utility and personal space, Transport Res Part B: Methodol 74 (2015) 104–117 von Sivers, G Köster, Realistic Stride Length Adaptation in the Optimal Steps Model, in: M Chraibi, M Boltes, A Schadschneider, A Seyfried (Eds.), Traffic and Granular Flow'13, Springer, Cham, 2015, pp 171–178 S Wolfram, Statistical mechanics of cellular automata, Rev Mod Phys 55 (1983) 601–644 X Wu, X.-H Sun, Performance modeling for interconnection networks, in: the 4th International Conference/Exhibition on High Performance Computing in the AsiaPacific Region, Beijing, China, 2000 pp 380–385 X Zheng, T Zhong, M Liu, Modeling crowd evacuation of a building based on seven methodological approaches, Build Environ 44 (3) (2009) 437–445 ... an agent can sense the radius, positions and velocities of its neighboring agents (a subset of agents within a limited fixed sensing range) as shown in Fig Finally, the agents are also capable of. .. Distribution of evacuation d) General evacuation situation Fig 20 Evacuation scenario of 250 pedestrians in MassMotion 9.0 a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation. .. Distribution of evacuation d) General evacuation situation Fig 24 Evacuation scenario of 250 pedestrians in PTV Viswalk 10 a) Number of evacuees at targets b) Density at targets c) Distribution of evacuation