An Investigation Into the Use of Synthetic Vision for NPC’sAgents in Computer Games

Licenciatura Thesis An Investigation Into the Use of Synthetic Vision for NPC’s/Agents in Computer Games Author Enrique, Sebastian senrique@dc.uba.ar Director Co-Director Mejail, Marta Watt, Alan University of Sheffield United Kingdom Universidad de Buenos Aires Argentina a.watt@dcs.shef.ac.uk marta@dc.uba.ar Departamento de Computación Facultad de Ciencias Exactas y Naturales Universidad de Buenos Aires Argentina September 2002 Abstract The role and utility of synthetic vision in computer games is discussed An implementation of a synthetic vision module based on two viewports rendered in realtime, one representing static information and the other dynamic, with false colouring being used for object identification, depth information and movement representation is presented The utility of this synthetic vision module is demonstrated by using it as input to a simple rule-based AI module that controls agent behavior in a first-person shooter game Son discutidos en esta tesis la utilidad y el rol de la visión sintética en juegos de computadora Se presenta una implementación de un módulo de visión sintética basado en dos viewports renderizados en tiempo real, uno representando información estática y el otro dinámica, utilizando colores falsos para identificación de objetos, información de profundidad y representación del movimiento La utilidad de este módulo de visión sintética es demostrada utilizándolo como entrada a un módulo simple de IA basado en reglas que controla el comportamiento de un agente en un juego de disparos en primera persona A mis padres, quienes sacrificaron todo por mi educación An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Table of Contents TABLE OF CONTENTS ACKNOWLEDGMENTS INTRODUCTION GENERAL OVERVIEW IN DEPTH PRE-ANALYSIS 8 PREVIOUS WORK 11 PROBLEM STATEMENT 13 SYNTHETIC VISION MODEL 14 STATIC VIEWPORT DEFINITIONS LEVEL GEOMETRY DEPTH DYNAMIC VIEWPORT BUFFERS REMARKS 14 14 15 16 17 18 18 BRAIN MODULE: AI 20 AI MODULE FPS DEFINITION MAIN NPC POWER-UPS BRONTO BEHAVIOUR BEHAVIOUR STATES SOLUTION DESTINATION CALCULATION BEZIER CURVE GENERATION WALK AROUND LOOKING FOR A SPECIFIC POWER-UP 20 20 21 23 25 26 27 27 28 37 Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 LOOKING FOR ANY POWER-UP LOOKING QUICKLY FOR A SPECIFIC POWER-UP KNOWN PROBLEMS EXTENDED BEHAVIOUR WITH DYNAMIC REACTIONS DON’T WORRY AVOID INTERCEPT DEMONSTRATION LAST WORD ABOUT DYNAMIC AI 38 38 39 44 45 45 46 46 46 GAME APPLICATIONS ANALYSIS 47 ADVENTURES FIRST PERSON SHOOTERS THIRD PERSON ACTION GAMES ROLE PLAY GAMES REAL TIME STRATEGY FLIGHT SIMULATIONS OTHER SCENERIES 47 47 47 47 48 48 48 CONCLUSIONS 49 FUTURE WORK 50 REFERENCES 52 APPENDIX A – CD CONTENTS .54 APPENDIX B – IMPLEMENTATION 55 FLY3D_ENGINE CLASSES FLYBEZIERPATCH CLASS FLYBSPOBJECT CLASS FLYENGINE CLASS FLYFACE CLASS LIGHTS CLASSES SPRITE_LIGHT CLASS SVISION CLASSES AI CLASS 55 55 55 55 55 56 56 56 56 Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 SVOBJECT CLASS VISION CLASS VIEWPORT CLASSES VIEWPORT CLASS WALK CLASSES CAMERA CLASS CAMERA2 CLASS OBJECT CLASS PERSON CLASS POWERUP CLASS 57 57 57 57 57 58 58 58 58 59 APPENDIX C – SOFTWARE USERS’ GUIDE .60 SYSTEM REQUIREMENTS INSTALLING DIRECTX CONFIGURING FLY3D RUNNING FLY3D RUNNING SYNTHETIC VISION LEVELS MODIFYING SYNTHETIC VISION LEVELS PROPERTIES 60 60 60 60 61 63 APPENDIX D – GLOSSARY 65 APPENDIX E - LIST OF FIGURES 66 APPENDIX F - LIST OF TABLES .70 Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Acknowledgments I will be forever grateful with Alan Watt, who accepted to guide me through the whole project, from the very beginning I won’t ever forget all the help he offered me during my little trip to Sheffield, despite the hard time he was having Special thanks to his wife, Dionéa, who is an extremely kind person From Sheffield as well, I want to thank Manuel Sánchez and James Edge, for being very friendly and easy-going Special thanks to Steve Maddock for all the help and support he gave to me, and for the “focusing” talk that we had Very special thanks to Fabio Policarpo, for many reasons: he gave me access to the early full source code and following versions of the engine; he helped me on each piece of code when I was stuck or when I just didn’t know what to do; he even gave me a place in his offices at Niterói Furthermore, all the people from Paralelo –Gilliard Lopes, Marcos, etc.- and Fabio’s friends were very kind with me Passion for games can be smelled inside Paralelo’s offices I shall be forever in debt with Alan Cyment, Germán Batista and Javier Granada for making possible the presentation of this thesis in English Thanks to all the professors from the Computer Sciences Department who make a difference, like Gabriel Wainer, who not only teaches computer sciences, but a work philosophy as well I must mention all of my university partners and friends, with whom I shared many suffered funny study days and nights In special to Ezequiel Glinsky, an incredible teammate who was next to me on every academic step Thank you Cecilia for standing by my side all of these years I don’t want to forget to give thanks to Irene Loiseau, who initiated contact with Alan; and to all the people from the ECI 2001 committee, who accepted my suggestion to invite Alan to come to Argentina to give a very nice, interesting, and successful course And, finally, very special thanks to Claudio Delrieux, who strengthened my passion for computer graphics He was always ready to help me unconditionally before and during every stage of this thesis Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Introduction General Overview Today, 3D computer games usually uses artificial intelligence (AI) for the non-player characters (NPC’s) taking information directly from the internal database They control the NPC’s movements and actions having knowledge of the whole world information, probably cheating if the developer does not put any constraint over this Since a large quantity of computer controlled opponents or friends are human-like, it seems to be interesting and logic to give senses to those characters This means that the characters could have complete or partial systems, such as vision, aural, tactile, smell, and taste They could then process the information sensed from those systems in a brain module, learn about the world, and act depending on the character’s personality, feelings and needs The character becomes a synthetic character that lives in a virtual world The research field of synthetic characters or autonomous agents investigates the use of senses combined with personality in order to make characters’ behaviour more realistic, using cognitive memories and rule based systems, producing agents that seem to be alive and interacting in their own world, and maybe with some human interaction However, not too much effort has been given to the investigation of the use of synthetic characters in real time 3D computer games In this thesis, we propose a vision system for NPC’s, i.e a synthetic vision, and analyze how useful and feasible its usage might prove in the computer games’ industry We think that the use of synthetic vision together with complex brain modules could improve gameplay and make for better and more realistic NPC’s We have to note that our efforts were focused on the investigation of the use of the synthetic vision, and not to the AI that uses it For that reason, we developed only a simple AI module in a 3D engine, where we implemented our vision approach In Depth Pre-Analysis We can regard synthetic vision as a process that supplies an autonomous agent with a 2D view of his environment The term synthetic vision is used because we bypass the classic computer vision problems As Thalmann et al [Thal96] point out, we skip the problems of distance detection, pattern recognition and noisy images that would be appertain for vision computations for real robots Instead computer vision issues are addressed in the following ways: 1) Depth perception – we can supply pixel depth as part of the synthetic vision of an autonomous agent’s vision The actual position of objects in the agent’s field of view is then available by inverting the modeling and projection transform 2) Object recognition – we can supply object function or identity as part of the synthetic vision system Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 3) Motion determination – we can code the motion of object pixels into the synthetic vision viewport Thus the agent AI is supplied with a high-level vision system rather than an unprocessed view of the environment As an example, instead of just rendering the agent’s view into a viewport then having an AI interpret the view, we instead render objects in a colour that reflects their function or identity (although there is nothing to prevent an implementation where the agent AI has to interpret depth – from binocular vision, say, and also recognize objects) With object identity, depth and velocity presented, the synthetic vision becomes a plan of the world as seen from the agent’s viewpoint We can also consider synthetic vision in relation to a program that controls an autonomous agent by accessing the game database and the current state of play Often a game database will be tagged with extra pre-calculated information so that an autonomous agent can give a game player an effective opponent For instance, areas of the database may be tagged as good hiding places (they may be shadow areas), or pre-calculated journey paths from one database node to another may be stored In using synthetic vision, we change the way the AI works, from a prepared, programmeroriented behaviour to the possibility of novel, unpredictable behaviour A number of advantages accrue from allowing an autonomous agent to perceive his environment via a synthetic vision module First, it may enable an AI architecture for an autonomous agent that is more ‘realistic’ and easier to build Here we refer to an ‘on board’ AI for each autonomous agent Such an AI can interpret what is seen by the character, and only what is seen Isla and Blumberg [Isla02] refer to this as sensory honesty and point out that it “…forces a separation between the actual state of the world and the character’s view of the state of the world” Thus the synthetic vision may render an object but not what is behind it Second, a number of common games operations can be controlled by synthetic vision A synthetic vision can be used to implement local navigation tasks such as obstacle avoidance Here the agent’s global path through a game level may be controlled by a high level module (such as A* path planning or game logic) The local navigation task may be to attempt to follow this path by taking local deviations where appropriate Also, synthetic vision can be used to reduce collision checking In a games engine this is normally carried out every frame by checking the player's bounding box against the polygons of the level or any other dynamic object Clearly if there is free space ahead you not need every-frame collision checking Third, easy agent-directed control of the synthetic vision module may be possible, for example, look around to resolve a query, or follow the path of a moving object In the case of the former this is routinely handled as a rendering operation A synthetic vision can also function as part of a method for implementing inter-agent behaviour Thus, the provision of synthetic vision reduces to a specialized rendering which means that the same technology developed for fast real-time rendering of complex scenes is exploited in the synthetic vision module This means that real-time implementation is straightforward However, despite the ease of producing a synthetic vision, it seems to be only an occasionally employed model in computer games and virtual reality Tu and Terzopoulous [TuTe94] made an early attempt at synthetic vision for artificial fishes The emphasis of this work is a physics-based model and reactive behaviour such as obstacle avoidance, escaping and schooling Fishes are equipped with a “cyclopean” Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 vision system with a 300 degree field of view In their system an object is “seen” if any part of it enters the view volume and is not fully occluded by another object Terzopoulos et al [Terz96] followed this with a vision system that is less synthetic in that the fishes’ vision system is initially presented with retinal images which are binocular photorealistic renderings Computer vision algorithms are then used to accomplish, for example, predator recognition This work thus attempts to model, to some extent, the animal visual processes rather than bypassing these by rendering semantic information into the viewport In contrast, a simpler approach is to use false colouring in the rendering to represent semantic information Blumberg [Blum97] use this approach in a synthetic vision based on image motion energy that is used for obstacle avoidance and low-level navigation A formula derived from image frames is used to steer the agent, which, in this case, is a virtual dog Noser et al [Nose95] use false colouring to represent object identity, and in addition, introduce a dynamic octree to represent the visual memory of the agent Kuffner and Latombe [Kuff99] also discuss the role of memory in perception-based navigation, with an agent planning a path based on its learned model of the world One of the main aspects that must be addressed for computer games is to make the synthetic vision fast enough We achieve this, as discussed above, by making use of existing real-time rendering speed-ups as used to provide the game player with a view of the world We propose that two viewports can be effectively used to provide for two different kinds of semantic information Both use false colouring to present a rendered view of the autonomous agent’s field of view The first viewport represents static information and the second viewport represents dynamic information Together the two viewports can be used to control agent behaviour We discuss only the implementation of simple memoryless reactive behaviour; although far more complex behaviour is implementable by exploiting synthetic vision together with the consideration of memory and learning Our synthetic vision module is used to demonstrate low-level navigation, fast object recognition, fast dynamic object recognition and obstacle avoidance Game Applications Analysis - 10 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 lights classes sprite_light class The sprite_light is draw only if synthetic vision mode is not set This was modified in ‘draw’ method svision classes This is a new plug-ing that implements the process of saving viewports buffers to memory as well as all code to control NPC’s behaviour depending on what was defined in this thesis Global properties are: • ‘depthbuffer’, it contains last frame depth buffer • ‘dynamicbuffer’, it contains last frame dynamic viewport buffer • ‘staticbuffer’, it contains last frame static viewport buffer NPC’s present state is shown on screen through global method ‘draw_game_status’ class NPC’s behaviour code is contained in this class The most important properties are: • ‘bwalpd’, bottom margin in pixels • ‘bwaupd’, lateral and top margin in pixels • ‘cutfactor’, cut factor (percent) of current path • ‘hid’, health colour id • ‘hlt’, health lower threshold • ‘hut’, health upper threshold • ‘player’, reference to the person object controlled in this class • ‘radio’, Bbbr • ‘state’, NPC’s current state • ‘wid’, weapon colour id • ‘wlt’, weapon lower threshold • ‘wut’, weapon upper threshold Most important methods are: • ‘b_addobject’, it is used to add objects to a list when it is necessary to obtain objects present in the static viewport • ‘b_getanynextobject’, get the person’s closest object from an object list It is used when both power-ups (weapon and health) are looked for • ‘b_getnextobject’, get the person’s closest object that have a specific colour, from an object list • ‘b_lookboth’, heuristic used in looking for any power-up state • ‘b_lookfor’, heuristic used in looking for a specific power-up state • ‘b_turnleft’, heuristic used to turn left in walk around state • ‘b_turnright’, heuristic used to turn right in walk around state Game Applications Analysis - 56 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 • • ‘b_walkaround’, heuristic used for walk around state ‘step’, set current state on each frame and call the corresponding heuristic Dynamic behaviour is embedded into static heuristics’ methods svobject class It is the structure used for objects found in Looking-for strategies vision class It is used to save viewports data to memory It contains the following properties: • ‘player’, reference to the person object controlled by means of the synthetic vision • ‘vpstatic’, reference to the static viewport • ‘vpdynamic’, reference to the dynamic viewport • ‘width’, viewports width in pixels Current implementation only supports 160 pixels width • ‘height’, viewports height in pixels Current implementation only supports 120 pixels height ‘step’ is the implemented method that in a per frame basis gets from the colour buffer the static and dynamic viewports pixels information to save it in main memory It does the same to save depth information but taking data from the depth buffer (Z-buffer) viewport classes viewport class The following properties were added: • ‘coordmode’, if percentual coordinates are used as in the old viewport (xf, yf, wxf, wyf); if pixel coordinates y-inverted are used (xi, yi, sxi, syi); if demonstration’s special coordinates are used: normal viewport centered, at xi pixels from the left for static viewport, and at xi pixels from the right for dynamic one • ‘mode’, normal; static, dynamic • ‘camangle’, viewport rendering angle • ‘xi’, viewport is placed at xi pixels from the left • ‘yi’, viewport is placed at yi pixels from the window’s top • ‘sxi’, viewport pixels width • ‘syi’, viewport pixels height ‘draw’ method was modified in order to support the different viewport modes Its task is to set the corresponding debug mode: 128 (dynamic) or 256 (static) walk classes Walk plug-in suffered several changes Demonstration’s dynamic object (‘flying ball’) as well as new implemented cameras were added to it The modifications were made basically in ‘powerup’ and ‘person’ classes in order to support health and power-up power-ups as they were defined in this thesis Other methods and properties were added to ‘person’ class also in order to allow the interpretation of static viewport coordinates selected destination point and unprojection in order to obtain its world coordinates and make then a path between current and desired final positions Game Applications Analysis - 57 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 NPC’s health and weapon properties ‘draw_game_status’ global method values are showed in screen through camera class This is a new class for managing a camera that follows a target at a given distance without colliding with any other objects ‘source’ property must refer to the corresponding camera target flyBspObject ‘displacement’ property has the distance that the camera must hold from his target camera2 class This is a new class for managing a camera that follows a target at a given distance The camera collides It has the following properties: • ‘source’, a reference to the camera target flyBspObject • ‘dist’, minimum distance that the camera must hold from his target • ‘radius’, bounding box radius • ‘maxvel’, maximum velocity allowed for the camera when it need to get close to his target, until it reaches a ‘dist’ distance from it object class This is a new class It draws a moving object that changes direction when collisions are detected It uses a constant velocity indicated by inherited ‘vel’ property person class The NPC is contained in this class Once the destination point was selected, the Bezier curve is calculated and draw here One of its tasks is to make the agent to follow curve path Added properties are: • ‘mode’, if normal walk mode is desired; if interactive destinations point selection interface is desired (similar to the one found at Fly3D provided shop.dll); if a synthetic vision behavior is desired • ‘svhealth’, current health value • ‘svinithealth’, initial health value • ‘svweapon’, current weapon value • ‘svinitweapon’, initial weapon value • ‘maxhealth’, maximum allowed health value • ‘maxweapon’, maximum allowed weapon value • ‘healthtime’, time interval for health decreasing • ‘weapontime’, time interval for weapon decreasing • ‘healthfactor’, health quantity that must be decreased once the specified time interval was fulfilled • ‘weaponfactor’, weapon quantity that must be decreased once the specified time interval was fulfilled • ‘healthpowerup’, health quantity that must be increased each time a health powerup is taken • ‘weaponpowerup’, weapon quantity that must be increased each time a weapon power-up is taken • ‘healthcurtime’, time since last health power-up was took or since its value was decreased • ‘weaponcurtime’, time since last weapon power-up was took or since its value was decreased Game Applications Analysis - 58 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 • • • • • • • • • • • • • • • • • • ‘movesprite’, spritelight object reference used to show mouse cursor in modes and ‘targetsprite’, spritelight object reference used to show destination point in modes and ‘spriteflag’, indicates if movesprite must be draw ‘targetflag’, indicates if targetsprite must be draw ‘characterflag’, indicates if character is idle or in movement ‘walkpath’, Bezier curve for character path ‘path_dist’, estimated distance between Bezier curve control points ‘path_len’, Bezier curve length ‘path_factor’, Bezier curve path walked percent ‘svx’, static viewport’s x coordinate of selected destination point It is –1 if no destination was selected ‘svy’, static viewport’s y coordinate of selected destination point It is –1 if no destination was selected ‘vpstatic’, static viewport reference ‘lastpos’, character’s last position It is used to know if the character is stuck ‘stucktime’, time used to determine if character is stuck at any position ‘signo’, used to determine where to rotate once the system knows that the character is stuck ‘drawpath’, it indicates if Bezier curve must be draw ‘svwalkvel’, character’s walk velocity when the system is in mode or ‘svrunvel’, character’s run velocity when the system is in mode or Added and modified methods are: • ‘die’, simply showed message when the character dies was modified • ‘draw’, it was modified in order to draw 3D cursor and destination sprites as well as to draw Bezier curve • ‘init’, necessary initializations for synthetic vision where added • ‘move2’, if mode is 1, destination points selected by the user with the mouse are processed in order to create character Bezier curve path It implements an interface similar to Fly3D shop.dll plug-in • ‘move3’, if mode is 2, destination’s points selected by AI module are processed in order to create Bezier curve path When it does not need to generate a new curve, makes the character to continue walking over current path If it detects that the character is stuck, makes him rotate 180° • ‘step’, the corresponding move method to current mode is called in a per frame basis Furthermore, it manages health and weapon’s properties values in order to decrease their values when corresponding time was reached powerup class ‘get_mesh’ method was overrided in order to return power-up mesh only when ir is visible from the current point of view (a power-up is invisible when it was taked and respawning time was not completed) ‘powerup_get’ method was modified in order to increment health and weapon’s properties when synthetic vision mode is used Values defined in ‘person’ object are used Game Applications Analysis - 59 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Appendix C – Software Users’ Guide System Requirements The Fly3D Engine version 2.0 requires Windows 9x/Me/2k/NT4 in order to run A good 3D card supporting OpenGL is also required (NVIDIA recommended) Make sure to be using the latest drivers available for your video card and also to have the latest Windows Service Packs and latest DirectX installed In order to run the engine under NT4 you must have Service Pack or later installed For more information refer to [Fly01] Installing DirectX If you don’t have DirectX installed in your system, look at Appendix A to find the software (DirectX 8.1) at the companion CD or visit [Micr01] Run the executable file for your operating system and follow the on-line instructions Configuring Fly3D The first thing that you have to do, after having installed everything, in order to run Fly3D properly is to configure it Follow this steps: Run flyConfig.exe, located at Fly3D path A window will be opened (Figure C-1) If it is checked, uncheck the ‘Customized data and plugin folders’ option Select the desired full screen resolution at ‘Fullscreen Video Mode’ combo Select the desired rendering mode at ‘Rendering Modes’ combo The first time that you run Fly3D on your machine the best rendering mode will be automatically selected A hardware accelerated mode is always preferable to a software one Press ‘Test’ button, then ‘Save’, and then ‘Exit’ Later, if Fly3D is running too dark or bright, you will have to increment or decrement the brightness value accordingly, and repeat step The same applies if you want to change the full screen resolution or rendering mode Running Fly3D Before running Fly3D for the first time, be sure to meet the system requirements and to configure it properly For running Fly3D, execute the flyFrontEnd.exe file located at Fly3D path Game Applications Analysis - 60 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Figure C-1 Fly3D Configuration window Running Synthetic Vision Levels After running Fly3D, a menu will appear (figure C-2) Choose ‘Single Player’ pressing the ‘Enter’ key The levels selection menu will appear (figure C-3) Push ‘down arrow’ key and then push ‘right arrow’ or ‘left arrow’ several times until you find the desired level Then press ‘Enter’ to load it We provide four levels on the CD: • • • • Synthetic Vision 1: A custom level with a camera located behind Bronto, without collisions detection (it est, the camera will get into walls) Synthetic Vision 2: The same custom level but this time using a camera with collision detections Synthetic Vision 3: A Quake III Arena [Quak01] level converted to Fly3D, with custom sky and power-ups The camera collides Synthetic Vision 4: The PadCenter [Ente01] level with custom power-ups The camera collides Game Applications Analysis - 61 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Figure C-2 Fly3D FrontEnd main menu Figure C-3 Fly3D FrontEnd level selection menu Game Applications Analysis - 62 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Once the selected level is loaded, first press the ‘B’ key to activate the Dynamic Viewport, and then ‘C’ to activate the Static Viewport After both viewports have been activated (in that same order) the AI module will start to control Bronto’s behaviour At any time during the simulation you can press the ‘V’ key to activate or deactivate the viewport that shows the scene normally rendered from Bronto viewpoint To exit from the level, press ‘ESC’ After that, you could repeat the steps from the start of this section in order to run other levels If you want to quit Fly3D, press ‘down arrow’ key until ‘Quit’ option appears selected in Fly3D FrontEnt main menu, and then press ‘Enter’ Modifying Synthetic Vision Levels Properties To edit some of the properties of the synthetic vision levels, execute flyEditor.exe from the Fly3D path and load any of the available synthetic vision levels: • • • • Synthetic Synthetic Synthetic Synthetic Vision Vision Vision Vision 1: 2: 3: 4: sv08.fly sv09.fly sv10.fly sv12.fly Refer to Fly3D literature [Watt01; Watt02] or website [Fly01] to know how to use flyEditor Some of the interesting parameters that are possible to change and play with are: Global Parameters • • • • Nearplane (NP) FarPlane (FP) Camangle (Camera Angle) Camera: o camera2: Third person camera with collisions detection o 3rd_Person: Third person camera without collisions detection o player: First person camera SVision plug-in; “ai” class object properties The colour id parameters in this class are used to identify weapon and health • • • • • • Health Upper Threshold Health Lower Threshold Weapon Upper Threshold Weapon Lower Threshold Health Color Id Weapon Color Id (Hut) (Hlt) (Wut) (Wlt) Game Applications Analysis - 63 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 • • • • WA Upper Pixel Dist WA Lower Pixel Dist Cutfactor Radio (bwaupd) (bwalpd) (Cf) (Bbbr) Walk plug-in; “person” class object properties • • • • • • • • • svhealth svweapon healthtime weapontime healthfactor weaponfactor healthpowerup weaponpowerup drawpath (Hini) (Wini) (tdh) (tdw) (Dh) (Dw) (Ah) (Aw) (1 or 0: Do/Do not draw bezier curve) Walk plug-in; “powerup” class object properties Each object must have the type and corresponding colour id set • • • ColorId Type Spawntime (1: Ammo; -1: Health) (Time for power-up regeneration after taken) You can add as many power-ups as you want Walk plug-in; “object” class object properties This is the flying ball The unique colour id supported for time being is (1.0, 1.0, 1.0) • ColorId You can add as many flying balls as you want Notes • • • All time variables are in milliseconds All percent variables range between and Colour Ids are floating point vectors Actually, not all possible colours work well, since some parts of the code use unsigned int colours in place of float, and there are conversion precision errors This will be fixed in future releases Game Applications Analysis - 64 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Appendix D – Glossary 2D : Two Dimensions 3D : Three Dimensions Three Dimensional AI : Artificial Intelligence BSP : Binary Space Partition ECI : Escuela de Ciencias Informáticas FPS : First Person Shooter NPC : Non Player Character RGB : Red-Green-Blue RPG : Role Player Game Game Applications Analysis - 65 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Appendix E - List of Figures FIGURE APPROACHES: A GRAPHICAL VIEW .11 FIGURE AIMED MODEL PLACE .13 FIGURE STATIC VIEWPORT (LEFT) OBTAINED FROM THE NORMAL RENDERED VIEWPORT (RIGHT) 14 FIGURE COORDINATE SYSTEM USED 16 FIGURE FROM LEFT TO RIGHT, STATIC VIEWPORT, NORMAL RENDERED VIEWPORT, AND DYNAMIC VIEWPORT 19 FIGURE AN EXAMPLE OF WEAPON AND ENERGY DIMINISHING SYSTEM: GAME STARTS, THEN BOTH PROPERTIES DECREASE GRADUALLY WEAPON REACHES VALUE AT 100 SECONDS, WHEREAS HEALTH DOES THE SAME AT 170 SECONDS AT THAT MOMENT BRONTO DIES 22 FIGURE EVENTS THAT AFFECT AND ARE AFFECTED BY BRONTO’S HEALTH AND WEAPON VALUES DURING THE GAME 25 FIGURE BRONTO’S BEHAVIOUR REDUCED STATES DIAGRAM ONLY EXPECTED TRANSITIONS BETWEEN STATES ARE REPRESENTED WITH ARROWS, EVEN THOUGH IT IS POSSIBLE TO GO FROM ONE STATE TO ANY OTHER 26 FIGURE 10 BEZIER CURVE CONTROL POINTS (PI), INITIAL POSITION (B), DESTINATION POINT (T), AND BRONTO’S VISUAL/DIRECTION VECTOR (D) ARE REPRESENTED 28 Game Applications Analysis - 66 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 FIGURE 11 STATIC VIEWPORT AND USED WALK AROUND MARGINS BOLDER RECTANGLE OUTER PIXELS CANNOT BE CHOSEN AS DESTINATION POINTS .29 FIGURE 12 A FREE WAY INSIDE THE STATIC VIEWPORT WHITE ARE IS FLOOR, GREY AREA IS WALL BOLDED RECTANGLE SHOWS THE CHOSEN FREE WAY WITHIN THAT RECTANGLE ARE THE UPPER AND LOWER MARGIN LINES, AND THE POINT THAT THE HEURISTIC WILL SELECT AS THE NEW DESTINATION 30 FIGURE 13 STRATEGY IN ACTION IN A) PRECONDITION IS FULFILLED IN ORDER TO EMPLOY THIS STRATEGY: A CENTRAL FREE WAY DOES NOT EXISTS AND VIEWPORT’S BOTTOM CLOSEST ROW CONTAINING A PIXEL DIFFERENT THAN FLOOR INSIDE SEARCHING RECTANGLE, CONTAINS A PIXEL DIFFERENT THAN FLOOR MORE CLOSE TO THE LEFT SIDE THAN TO THE RIGHT ONE, SIGNALED WITH A DOUBLE CIRCLE DOTTED LINE SEPARATES SEARCHING RECTANGLE IN HALVES IN B) THE FREE WAY THAT THE STRATEGY WILL FINALLY CHOOSE AS WELL AS NEW DESTINATION POINT IS SHOWED 33 FIGURE 14 STRATEGY WHEN FAILS PRECONDITION IS FULFILLED IN ORDER TO EMPLOY THIS STRATEGY HOWEVER, NO FREE WAY IS FOUND WHEN THE STRATEGY LOOK FOR ONE TO THE LEFT OF THE POINT SIGNALED WITH DOUBLE CIRCLE 33 FIGURE 15 STRATEGY EXAMPLES IN ALL CASES, BEYOND A FREE WAY MAY EXISTS, SEARCH FAILS IF CENTRAL RECTANGLE FULFILLS STRATEGY PRECONDITIONS IN A) TWO PIXELS DIFFERENT THAN FLOOR AT THE SAME DISTANCE TO THE LEFT AND RIGHT SIDES OF THE RECTANGLE ARE FOUND IN B) THE SAME CASE IS PRODUCED WHEN A THIN CORRIDOR IS IN FRONT OF THE CHARACTER IN C) ONLY ONE PIXEL DIFFERENT FROM FLOOR IS FOUND AT RECTANGLE CENTER IN D) THERE IS DIRECTLY A WALL IN FRONT 34 FIGURE 16 TURN STRATEGY WALK AROUND HAS FAILED AS SHOWED IN FIGURE 15.D IN A) A LEFT TURN WAS CHOSEN WHERE THE DESTINATION POINT IS SIGNALED AS A CROSSED CIRCLE AND THE FIRST POINT FROM THE LEFT DIFFERENT THAN FLOOR IS SIGNALED AS A DOUBLE CIRCLE IN B) A RIGHT TURN WAS CHOSEN, EVERY PIXEL TO THE RIGHT Game Applications Analysis - 67 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 CORRESPONDS TO FLOOR, SO THE RIGHT MARGIN WILL BE TAKEN FROM THE RIGHT SIDE OF THE VIEWPORT THE CROSSED CIRCLE SIGNS NEW DESTINATION POINT .36 FIGURE 17 THE HIGHER FLOOR PROBLEM BRONTO IN FRONT OF A BOX OBSTACLE THAT IS PART OF THE LEVEL GEOMETRY LEFT: BLUE COLOUR OF BOX SIDES AS WELL AS GREEN COLOUR OF UPPER BOX FACE IS SEEN IN THE STATIC VIEWPORT THE UPPER FACE AND THE ‘REAL’ FLOOR CANNOT BE DIFFERENTIATED RIGHT: THE NORMAL RENDER 39 FIGURE 18 THE HIGHER FLOOR PROBLEM BRONTO CLOSER TO THE BOX OF FIGURE 17 WALK AROUND DOES NOT NOTE HEIGHT DIFFERENCES BETWEEN POLYGONS REPRESENTED AS FLOOR IN THE STATIC VIEWPORT IN THIS CASE, BRONTO WILL GET STUCK TRYING TO ‘GO THROUGH’ THE FLOOR 40 FIGURE 19 THE PERSPECTIVE PROBLEM THE STATIC VIEWPORT WHEN BRONTO IS GOING THROUGH A CORRIDOR CORRIDOR’S WIDTH IS CONSTANT IN THE WHOLE LENGTH; HOWEVER, DUE TO PERSPECTIVE, THE AMOUNT OF FLOOR PIXELS IN THE LOWER PART IS HIGHER THAN IN THE FARER ONE 41 FIGURE 20 THE PERSPECTIVE PROBLEM BRONTO IN FRONT OF A CORRIDOR IN A POSSIBLE SITUATION CURRENT FREE WAY IS SHOWN WITH CONTINUOUS LINES TOGETHER WITH DESTINATION POINT THAT WALK AROUND WILL CHOOSE FREE WAY WITH PERSPECTIVE CORRECTION IS SHOWN WITH DOTTED LINES, TOGETHER WITH DESTINATION POINT THAT SHOULD BE CHOSEN NOTE THAT DISTANCES BETWEEN BOTH DESTINATION POINTS COULD BE ENORMOUS 41 FIGURE 21 THE PERSPECTIVE PROBLEM BRONTO STUCK TRYING TO GO FORWARD BETWEEN TWO COLUMNS VERY CLOSE TO EACH OTHER UPPER-LEFT: STATIC VIEWPORT UPPER-RIGHT: NORMAL RENDERED SCENE LOWER: RENDERING FROM A LATERAL CAMERA .42 FIGURE 22 THE LOOKING-FOR PROBLEM BRONTO STUCK TRYING TO GO THROUGH THE STAIRS TO REACH A POWER-UP UPPER-LEFT: STATIC Game Applications Analysis - 68 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 VIEWPORT UPPER-RIGHT: NORMAL RENDERED SCENE LOWER: SCENE RENDERED FROM A LATERAL CAMERA 43 FIGURE 23 THE LOOKING-FOR PROBLEM IN A) THE CHOSEN WAY SELECTED BY BRONTO TO POWER-UP (PU) GO THROUGH AN OBSTACLE IN B) A POSSIBLE SOLUTION IS SHOWN: CHOOSING INTERMEDIATE POINTS THAT DO NOT CROSS THE OBSTACLE, CONSTRUCTING A COMPOSED BEZIER CURVE TO THE FINAL DESTINATION 44 FIGURE 24 THE LOOKING-FOR PROBLEM IN A) THE CHOSEN WAY SELECTED BY BRONTO TO POWER-UP (PU) DOES NOT GO THROUGH ANY OBSTACLES, BUT BRONTO WILL NOT PASS BETWEEN THE TWO OBJECTS DUE TO HIS WIDTH IN B) A SIMILAR CASE IS GIVEN WHEN BEZIER CURVE PASSES VERY CLOSE TO WALLS 44 FIGURE 25 DYNAMIC STATES DIAGRAM INITIALLY BRONTO IS IN DON’T WORRY STATE, HE CAN THEN MAKE TRANSITIONS BETWEEN ANY STATE DEPENDING ON THE VALUE OF HIS PROPERTIES AND IF AN ENEMY APPEARS OR NOT IN THE STATIC VIEWPORT, AND IF IT IS FACING OR NOT BRONTO 45 FIGURE C-1 FLY3D CONFIGURATION WINDOW 61 FIGURE C-2 FLY3D FRONTEND MAIN MENU 62 FIGURE C-3 FLY3D FRONTEND LEVEL SELECTION MENU .62 Game Applications Analysis - 69 An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis – DC – FCEyN – UBA - September 2002 Appendix F - List of Tables TABLE CLASSES OF OBJECTS AND COLOUR IDS MAPPING TABLE 15 TABLE EXTENDED CLASSES OF OBJECTS AND COLOUR IDS MAPPING TABLE WITH LEVEL GEOMETRY .16 Game Applications Analysis - 70 ... to the investigation of the use of synthetic characters in real time 3D computer games In this thesis, we propose a vision system for NPC’s, i.e a synthetic vision, and analyze how useful and... as part of the synthetic vision system Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis... before and during every stage of this thesis Game Applications Analysis - An Investigation Into the use of Synthetic Vision for NPC’s/Agents in Computer Games Enrique, Sebastian – Licenciatura Thesis