ARTIFICIAL INTELLIGENCE FOR GAMES Second Edition IAN MILLINGTON and JOHN FUNGE AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO Morgan Kaufmann Publishers is an imprint of Elsevier Morgan Kaufmann Publishers is an imprint of Elsevier 30 Corporate Drive, Suite 400, Burlington, MA 01803, USA This book is printed on acid-free paper Copyright © 2009 by Elsevier Inc All rights reserved Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters All trademarks that appear or are otherwise referred to in this work belong to their respective owners Neither Morgan Kaufmann Publishers nor the authors and other contributors of this work have any relationship or affiliation with such trademark owners nor such trademark owners confirm, endorse or approve the contents of this work Readers, however, should contact the appropriate companies for more information regarding trademarks and any related registrations No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopying, scanning, or otherwise—without prior written permission of the publisher Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, E-mail: permissions@elsevier.com You may also complete your request online via the Elsevier homepage (http://elsevier.com), by selecting “Support & Contact” then “Copyright and Permission” and then “Obtaining Permissions.” Library of Congress Cataloging-in-Publication Data Millington, Ian Artificial intelligence for games / Ian Millington, John Funge – 2nd ed p cm Includes index ISBN 978-0-12-374731-0 (hardcover : alk paper) Computer games–Programming Computer animation Artificial intelligence I Funge, John David, 1968- II Title QA76.76.C672M549 2009 006.3–dc22 2009016733 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library ISBN: 978-0-12-374731-0 For information on all Morgan Kaufmann publications visit our Website at www.mkp.com or www.elsevierdirect.com Typeset by: diacriTech, India Printed in the United States of America 09 10 11 12 13 For Conor – I.M For Xiaoyuan – J.F About the Authors Ian Millington is a partner of Icosagon Ltd (www.icosagon.com), a consulting company developing next-generation AI technologies for entertainment, modeling, and simulation Previously he founded Mindlathe Ltd., the largest specialist AI middleware company in computer games, working on a huge range of game genres and technologies He has a long background in AI, including PhD research in complexity theory and natural computing He has published academic and professional papers and articles on topics ranging from paleontology to hypertext John Funge (www.jfunge.com) recently joined Netflix to start and lead the new Game Platforms group Previously, John co-founded AiLive and spent nearly ten years helping to create a successful company that is now well known for its pioneering machine learning technology for games AiLive co-created the Wii MotionPlus hardware and has established its LiveMove products as the industry standard for automatic motion recognition At AiLive John also worked extensively on LiveAI, a real-time behavior capture product that is being used by the former lead game designer of Guitar Hero and Rock Band to create a new genre of game John is also an Assistant Adjunct Professor at the University of California, Santa Cruz (UCSC) where he teaches a Game AI course that he proposed, designed and developed John has a PhD from the University of Toronto and an MSc from the University of Oxford He holds several patents, is the author of numerous technical papers, and wrote two previous books on Game AI iv Contents About the Authors iv Acknowledgments xix Preface xxi About the Website xxiii Part I AI and Games Chapter Introduction 1.1 What Is AI? 1.1.1 Academic AI 1.1.2 Game AI 1.2 Model of Game AI 1.2.1 1.2.2 1.2.3 1.2.4 1.2.5 1.2.6 1.3 Movement Decision Making Strategy Infrastructure Agent-Based AI In the Book Algorithms, Data Structures, and Representations 1.3.1 Algorithms 1.3.2 Representations 10 10 11 11 12 12 12 15 v vi Contents 1.4 1.5 1.4.1 Programs 1.4.2 Libraries 16 16 17 Layout of the Book 18 On the Website Chapter Game AI 19 2.1 19 19 20 21 21 The Complexity Fallacy 2.1.1 2.1.2 2.1.3 2.1.4 2.2 When Simple Things Look Good When Complex Things Look Bad The Perception Window Changes of Behavior The Kind of AI in Games 2.2.1 Hacks 2.2.2 Heuristics 2.2.3 Algorithms 2.3 Speed and Memory 2.3.1 2.3.2 2.3.3 2.3.4 2.4 Processor Issues Memory Concerns PC Constraints Console Constraints The AI Engine 2.4.1 Structure of an AI Engine 2.4.2 Toolchain Concerns 2.4.3 Putting It All Together 22 22 23 24 25 25 28 29 29 31 32 33 34 Part II Techniques 37 Chapter Movement 39 3.1 40 41 42 45 The Basics of Movement Algorithms 3.1.1 Two-Dimensional Movement 3.1.2 Statics 3.1.3 Kinematics 3.2 Kinematic Movement Algorithms 3.2.1 Seek 49 49 Contents 3.3 3.2.2 Wandering 3.2.3 On the Website 53 55 Steering Behaviors 55 55 56 56 59 62 66 67 68 71 72 73 76 82 84 90 95 3.3.1 3.3.2 3.3.3 3.3.4 3.3.5 3.3.6 3.3.7 3.3.8 3.3.9 3.3.10 3.3.11 3.3.12 3.3.13 3.3.14 3.3.15 3.3.16 3.4 Blending and Arbitration Weighted Blending Priorities Cooperative Arbitration Steering Pipeline Predicting Physics 3.5.1 3.5.2 3.5.3 3.5.4 3.5.5 3.6 Steering Basics Variable Matching Seek and Flee Arrive Align Velocity Matching Delegated Behaviors Pursue and Evade Face Looking Where You’re Going Wander Path Following Separation Collision Avoidance Obstacle and Wall Avoidance Summary Combining Steering Behaviors 3.4.1 3.4.2 3.4.3 3.4.4 3.4.5 3.5 Aiming and Shooting Projectile Trajectory The Firing Solution Projectiles with Drag Iterative Targeting Jumping 3.6.1 Jump Points 3.6.2 Landing Pads 3.6.3 Hole Fillers 3.7 vii Coordinated Movement 3.7.1 3.7.2 3.7.3 3.7.4 3.7.5 Fixed Formations Scalable Formations Emergent Formations Two-Level Formation Steering Implementation 95 96 96 103 107 108 120 121 121 123 126 128 134 135 138 143 144 144 146 146 147 151 viii Contents 3.7.6 3.7.7 3.7.8 3.7.9 3.7.10 3.8 Extending to More than Two Levels Slot Roles and Better Assignment Slot Assignment Dynamic Slots and Plays Tactical Movement Motor Control 3.8.1 Output Filtering 3.8.2 Capability-Sensitive Steering 3.8.3 Common Actuation Properties 3.9 Movement in the Third Dimension 3.9.1 3.9.2 3.9.3 3.9.4 3.9.5 3.9.6 3.9.7 3.9.8 Rotation in Three Dimensions Converting Steering Behaviors to Three Dimensions Align Align to Vector Face Look Where You’re Going Wander Faking Rotation Axes 157 159 162 166 168 171 172 174 175 178 178 180 180 181 183 186 186 188 Exercises 192 Pathfinding 197 4.1 198 198 199 202 203 203 Chapter The Pathfinding Graph 4.1.1 4.1.2 4.1.3 4.1.4 4.1.5 4.2 The Problem The Algorithm Pseudo-Code Data Structures and Interfaces Performance of Dijkstra Weaknesses 204 205 206 210 212 214 214 The Problem The Algorithm Pseudo-Code Data Structures and Interfaces 215 216 216 220 223 Dijkstra 4.2.1 4.2.2 4.2.3 4.2.4 4.2.5 4.2.6 4.3 Graphs Weighted Graphs Directed Weighted Graphs Terminology Representation A* 4.3.1 4.3.2 4.3.3 4.3.4 Contents 4.3.5 4.3.6 4.3.7 4.3.8 4.4 Implementation Notes Algorithm Performance Node Array A* Choosing a Heuristic World Representations 4.4.1 4.4.2 4.4.3 4.4.4 4.4.5 4.4.6 4.4.7 Tile Graphs Dirichlet Domains Points of Visibility Navigation Meshes Non-Translational Problems Cost Functions Path Smoothing 4.5 Improving on A* 4.6 Hierarchical Pathfinding 4.6.1 4.6.2 4.6.3 4.6.4 4.6.5 4.7 Other Ideas in Pathfinding 4.7.1 4.7.2 4.7.3 4.7.4 4.7.5 4.7.6 4.8 Open Goal Pathfinding Dynamic Pathfinding Other Kinds of Information Reuse Low Memory Algorithms Interruptible Pathfinding Pooling Planners Continuous Time Pathfinding 4.8.1 4.8.2 4.8.3 4.8.4 4.8.5 4.9 The Hierarchical Pathfinding Graph Pathfinding on the Hierarchical Graph Hierarchical Pathfinding on Exclusions Strange Effects of Hierarchies on Pathfinding Instanced Geometry The Problem The Algorithm Implementation Notes Performance Weaknesses Movement Planning 4.9.1 4.9.2 4.9.3 4.9.4 Animations Movement Planning Example Footfalls Exercises ix 228 228 229 231 237 239 241 244 246 251 251 251 255 255 256 259 262 263 265 271 272 272 273 273 274 275 276 276 277 281 281 282 282 282 283 286 287 288 856 Index Goal-oriented behavior (continued) performance, 408 pseudo-code, 407–408 goals, 402–403 overview, 401–402 simple selection data structures and interfaces, 405 performance, 405 principles, 404 pseudo-code, 404–405 weaknesses, 406 smelly goal-oriented behavior compound actions action-based signals, 426–427 character-specific signals, 427 overview, 425–426 timing components, 409 data structures and interfaces, 410–411 goal change over time calculation, 411–412 performance, 411 planning needs, 412–413 pseudo-code, 410 utility involving time, 409–410 GOAP, see Goal-oriented action planning GOB, see Goal-oriented behavior Golden Axe, Goldeneye 007, 8, 817 Gran Turismo, 822 Grand Theft Auto 3, 11, 743, 821, 822 Graphics level of detail, overview, 732 Group action, see Coordinated action Group level of detail overview, 740 probability distributions, 740–741, 742 H Hacking, game development, 22–23 Half-life, 10, 818 Half-life 2, 10 Halo, 309, 816 Halo 2, 334, 817 Hashing, see Transposition table Hearing, see Sense management Hebbian learning, artificial neural network, 661–662 Herding, see Flocking and herding games Herdy Gerdy, 20, 836 Heuristics A* algorithm cluster heuristic, 233–235 Euclidean distance, 232–233 fill patterns, 235–236 function, 226–227 overestimation, 232 quality, 236–237 underestimation, 231–232 common heuristics, 24 overview, 23–24 turn-based strategy games, 708 Hierarchical N -Gram, see N -Gram Hierarchical pathfinding graph connection costs average minimum distance, 259 heuristics, 259 maximum distance, 258 minimum distance, 258 overview, 257–258 connections, 257 nodes, 256–257 pathfinding on graph algorithm, 259–260 data structures and interfaces, 261–262 performance, 262 pseudo-code, 260–261 hierarchy effects on pathfinding, 263–265 instanced geometry algorithm instance graph, 266 world graph, 266–267 data structures and interfaces, 270 implementation nodes, 270 overview, 265 performance, 270–271 pseudo-code, 267–270 setting node offsets, 271 weaknesses, 271 pathfinding on exclusions, 262–263 principles, 255–256 Hierarchical scheduling behaviors overview, 726–727 selection, 728 data structures and interfaces, 727 Index Hierarchical state machine, see State machine Hill climbing data structures and interfaces, 587 extensions adaptive resolution, 589–590 global optimum finding, 590–591 momentum, 589 multiple trials, 590 overview, 588–589 performance, 588 principles, 585 pseudo-code, 585–587 Hole fillers, jumpable gaps, 143–144 Homeworld, 146, 825 Horizon effect, board games, 705–706 Hyper-threading, processor speed, 26–27 Hysteresis, behavioral level of detail, 736–737 I ID3, decision tree learning algorithm, 614 continuous attributes data structures and interfaces, 625 multiple categories, 625–626 performance, 625 pseudo-code, 623–625 single splits, 622–623 data structures and interfaces, 621 entropy and information gain, 614–616 functions entropy, 619–620 entropy of sets, 620–621 split by attribute, 618–619 initiation, 621 more than two actions, 616 non-binary discrete attributes, 616–617 pseudo-code, 617–621 ID4, incremental learning algorithm, 627–628 limitations, 629–630 walk through, 628–629 IDA*, see Iterative deepening A* Influence mapping, see Tactical analyses Instanced geometry, hierarchical pathfinding algorithm instance graph, 266 world graph, 266–267 data structures and interfaces, 270 implementation nodes, 270 overview, 265 performance, 270–271 pseudo-code, 267–270 setting node offsets, 271 weaknesses, 271 Interruptible pathfinding, principles, 274–275 Iterative deepening A* goal-oriented action planning advantages, 419 algorithm, 419–421 data structures and interfaces, 423–425 heuristic, 419 implementation notes, 425 performance, 425 pseudo-code, 421–423 overview, 273–274 Iterative targeting algorithm, 129–130 data structures and interfaces, 132 performance, 132 problem, 128 pseudo-code, 130–132 targeting without motion equations, 133 J Jak and Daxter, 9, 249 Jak and Daxter: The Precursor Legacy, 763, 819 Java, scripting of decision making, 474 Javascript, scripting of decision making, 474 Jumping hole fillers and jumpable gaps, 143–144 jump points achieving of jump, 137 overview, 135–136 landing pads data structures and interfaces, 141–142 implementation, 142 jump links, 142–143 overview, 138 performance, 142 pseudo-code, 140–141 trajectory calculation, 139 steering algorithm limitations, 134, 137–138 857 858 Index K Kinematic movement algorithms orientation of character, 49 seek arriving, 51–53 data structures and interfaces, 50–51 flee, 51 orientation algorithm, 49–50 performance, 51 wandering data structures, 54 implementation notes, 54 overview, 53 pseudo-code, 53–54 Knowledge finding, see Decision making; Movement; Pathfinding; Waypoint tactics Kohan: Ahriman’s Gift, 825 L Landing pad, see Jumping Learning artificial intelligence action prediction, see Action prediction approaches inter-behavior learning, 581 intra-behavior learning, 580 online versus offline, 579–580 balance of effort, 582–583 decision learning, see also Artificial neural network; Decision tree learning; Reinforcement learning detail of learning, 607 potential, 606 structure, 606–607 weak versus strong supervision, 606–607 over-learning, 582 parameter modification annealing, see Annealing hill climbing, see Hill climbing landscape energy and fitness values, 584 visualization, 583–584 potential, 579 reproducibility and quality control limitations, 581–582 Leave, see Arrive and leave Level of detail artificial intelligence level of detail, 732–733 behavioral level of detail behavior compression, 735–736 data structures and interfaces, 738–739 entry and exit processing, 735 hysteresis, 736–737 implementation notes, 739 performance, 740 pseudo-code, 737–738 graphics level of detail, 732 group level of detail overview, 740 probability distributions, 740–741, 742 importance values, 733 overview, 732 scheduling combined scheduling, 734 frequency schedulers, 734 priority schedulers, 734 Lex, tokenizer building, 479 Load-balancing scheduler data structures, 726 performance, 726 principles, 724 pseudo-code, 724–726 LOD, see Level of detail Look steering behavior, 72–73 three-dimensional movement, 186 Lua, scripting of decision making, 470–471 M Mace Griffin: Bounty Hunter, 816 Madden NFL 2005, 827 Manic Karts, 822 Map flooding algorithm, 534–535 data structures and interfaces, 537 influence calculations, 522–523 overview, 533–534 performance, 537–538 pseudo-code, 535–537 Markov systems nomenclature, 396 processes applications, 397 Index conservative processes, 396 iterated processes, 397–398 overview, 396 state machine algorithm actions, 399 default transitions, 399 data structures and interfaces, 401 pseudo-code, 399–401 Massively multi-player online games, design, 819–820 Master of Orion 3, 830 Medal of Honor: Airborne, 816 Memory cache, 28–29 console constraints, 29–30 handheld consoles, 31 limitations in gaming, 25, 28 personal computer constraints, 29 rendering hardware, 30–31 Memory-enhanced test algorithms algorithm, 699–700 iterative deepening, 704–705 memory size, 700 performance, 701 pseudo-code, 700–701 test function implementation overview, 697 pseudo-code, 697–699 transposition table, 699 variations, 700 Metal Gear Solid, 8, 759, 765, 818 Military tactics, see Coordinated action; Tactical and strategic artificial intelligence Mind-reading, teaching characters, 834 Minimaxing algorithm data structures and interfaces, 677 multiple players, 678 performance, 678 pseudo-code, 676–677 combining scoring functions, 673 range of function, 673 score maximization/minimization, 674–675 simple move choice, 673–674 static evaluation function, 672–674 MMOGs, see Massively multi-player online games Momentum, hill climbing, 589 859 Motor control actuation restrictions cars and motorcycles, 176–177 human characters, 176 tanks, 178 capability-sensitive steering, 174–175 output filtering, 172–174 overview, 171–172 Movement algorithms kinematics, see also Kinematic movement algorithms angular velocity, 45–46 forces and actuation, 48–49 independent facing, 46 linear velocity, 45–46 updating position and orientation, 47–48 variable frame rates, 48 statics and mathematics, 42–45 steering behavior, see Steering behavior structure, 40–41 two-dimensional movement, 41–42 coordinated movement, see Formation movement driving game design, 820–823 evaluation in game design, 809–811 game artificial intelligence model, 9–10, 39 jumping, see Jumping knowledge finding high-level staging, 797 obstacles obstacle representation, 796–797 walls, 796 motor control actuation restrictions cars and motorcycles, 176–177 human characters, 176 tanks, 178 capability-sensitive steering, 174–175 output filtering, 172–174 overview, 171–172 planning, see Pathfinding projectiles, see Firing solution; Projectile trajectory real-time strategy game design, 825 shooter game design, 814–816 three-dimensional movement align, 180–181 860 Index Movement (continued) align to vector, 181–183 face, 183–185 looking, 186 overview, 178 rotation, 178–180 rotation axes faking aircraft example, 188–189 algorithm, 190 data structures and interfaces, 191–192 implementation, 192 performance, 192 pseudo-code, 190–191 steering behavior conversion angular steering behaviors, 180 linear steering behaviors, 180 wander, 186–188 MT algorithms, see Memory-enhanced test algorithms Multi-core processing, processor speed, 26–27 N Naive Bayes classifier, 608–613 implementation notes, 612–613 pseudo-code, 611–612 Narrowcasting, event casting, 754–755 Navigation meshes edges as nodes, 249–250 knowledge finding for pathfinding, 789 overview, 246–247 quantization and localization, 248–249 usefulness, 249 validity, 249 Negamaxing AB negamax, 682–683 data structures and interfaces, 680 implementation notes, 680 performance, 680 principles, 678 pseudo-code, 679–680 static evolution function, 678–679 Neural network, see Artificial neural network Neverwinter Nights, 816 N -Gram predictor combat applications, 605–606 computer science applications, 597–598 data structures and interfaces, 599–600 hierarchical N -Grams confidence, 605 data structures and implementation, 604 overview, 602–603 performance, 605 pseudo-code, 603–604 implementation notes, 600 performance, 600 principles, 597 pseudo-code, 598–599 window size memory concerns, 602 performance enhancement, 601 sequence length, 602 No One Lives Forever 2, 815, 817, 818 O Observational learning, teaching characters, 834 Obstacle and wall avoidance collision detection problems, 92–95 data structures and interfaces, 91–92 overview, 90 performance, 92 pseudo-code, 90–91 Oddworld: Munch’s Oddysee, 819, 836 Open goal pathfinding, principles, 272 Opening book implementation, 702 learning, 702–703 overview, 701–702 set play books, 703 Open source software, scripting languages for decision making, 469–470 Othello, artificial intelligence applications, 667 Otostaz, 553 P Pac-Man, 7–8, 19, 39 Parameter modification, see Learning artificial intelligence Parsing language processing, 476–477 Yacc, 479 Pathfinding algorithms, see A* algorithm; Dijkstra algorithm artificial intelligence model, 197–198 Index continuous time pathfinding, see Continuous time pathfinding driving game design, 823 dynamic pathfinding, 272–273 graph data structure principles, 198–199 representation, 203–204 terminology, 203 weighted graphs costs, 199–200 direct weighted graphs, 202–203 non-negative constraint, 201–202 representative points in a region, 200–201 groups, see Coordinated action hierarchical pathfinding, see Hierarchical pathfinding interruptible pathfinding, 274–275 jump incorporation, 142 knowledge finding automatic graph creation, 789 data mining character movement, 794 connection calculation, 794 limitations, 794–795 node calculation, 793–794 geometric analysis arbitrary bounding regions, 791–792 connection calculation, 790–792 cost calculation, 790 mesh representations, 792 node calculation, 792–793 point-based representations, 790–791 visibility approach limitations, 792 manual region data creation bounded regions, 789 Dirichlet domains, 788 navigation meshes, 789 tile graphs, 787 movement planning animations, 282–283 example, 286–287 footfalls, 287–288 implementation, 285 infinite graph, 285 planning graph, 284 open goal pathfinding, 272 pooling planners, 275 real-time strategy game design, 824 861 shooter game design, 818–819 tactical information, see Tactical pathfinding; Waypoint tactics world representations cost functions, 251 Dirichlet domains, 241–244 generation, 238 navigation meshes, 246–250 non-translational problems, 251 path smoothing, 251–253 points of visibility, 244–246 quantization and localization, 238 tile graphs, 239–241 validity, 238–239 Path following behavior data structures and interfaces, 80 overview, 76–77 parameter tracking, 81 path types, 80 performance, 81 pseudo-code, 77–79 Path smoothing, world representations algorithm, 253 data structures and interfaces, 254–255 performance, 255 pseudo-code, 254 Perception, see Sense management Perception window, overview, 21 Physics simulation aiming and shooting, 121 firing solution equations, 123–124 firing vector implementation, 125 iterative targeting algorithm, 129–130 data structures and interfaces, 132 performance, 132 problem, 128 pseudo-code, 130–132 targeting without motion equations, 133 long-time versus short-time trajectory, 125 projectile trajectory drag modeling, 126–128 equations, 121 landing spot prediction, 122–123 prediction applications, 133–134 rotation and lift, 128 sports game design, 827 862 Index Pikmin, 836 Pikmin 2, 836 Playbook, sports game design, 828 Ply, game theory, 668 Points of visibility, world representation division scheme, 244–245 quantization and localization, 245–246 usefulness, 246 Polling station abstract polling, 758–759 implementation notes, 757–758 overview, 746, 756 performance, 757 pseudo-code, 756–757 sense management, 761 world interfacing, 746 Polymorphism, definition, 27 Pong, 7, 39 Principal variation search, negascout, 689 Priority scheduling performance, 730 policies, 730 priority problems, 731 pseudo-code, 728–730 Probability conditional, 609, 610 conditional independence, 609 log-likelihood, 613 prior, 610 Processor speed flexibility versus indirect function calls, 27 hyper-threading, 26–27 limitations in gaming, 25 multi-core processing, 26 single instruction multiple data processing, 25–26 Projectile trajectory, see also Firing solution drag modeling, 126–128 equations, 121 landing spot prediction, 122–123 prediction applications, 133–134 rotation and lift, 128 Pseudo-code conventions, 14 definition, 13 Pursue and evade evade, 71 implementation notes, 70 overshooting, 71 paths, 68 performance, 70 pseudo-code, 69–70 PVS, see Principal variation search Python pseudo-code similarity, 15 scripting of decision making, 472–473 Q Q-learning applications, 641–644 convergence and ending, 634–635 data structures and interfaces, 636–637 doing learning, 633 exploration strategy, 634 implementation notes, 637 performance, 637 principles, 633–634 pseudo-code, 635–636 tactical defense location selection case study, 643 tailoring parameters discount rate gamma, 638 learning rate alpha, 638 length of walk nu, 639–640 randomness for exploration rho, 639 reward selection, 640–641 weaknesses, 641–642 world representation, 632–633 Quake, 468 Quake II, 22 Quiescence pruning, variable depth approaches, 706 R Radial basis function, artificial neural network, 658–660 Reinforcement learning actor–critic algorithm, 465–646 neural networks for storage, 645 on-policy algorithms, 644 overview, 631 Index problem for solution, 631–632 Q-learning applications, 641–644 convergence and ending, 634–635 data structures and interfaces, 636–637 doing learning, 633 exploration strategy, 634 implementation notes, 637 performance, 637 principles, 633–634 pseudo-code, 635–636 tactical defense location selection case study, 643 tailoring parameters discount rate gamma, 638 learning rate alpha, 638 length of walk nu, 639–640 randomness for exploration rho, 639 reward selection, 640–641 weaknesses, 641–642 world representation, 632–633 temporal difference algorithm, 644 Representations, overview, 15–16 Rete algorithm applications, 445–446 database matching, 447–451 example, 446 fact addition, 452 fact removal, 452 nodes, 446–447 performance, 454–455 update management, 453 Reversi, see Othello Reynolds, Craig, 40–41 Rotation axes, faking aircraft example, 188–189 algorithm, 190 data structures and interfaces, 191–192 implementation, 192 performance, 192 pseudo-code, 190–191 Ruby, scripting of decision making, 474 Rule-based decision making algorithm, 433 blackboard architecture, 465 components, 427–428 data structures and interfaces database, 434–435 IF clauses, 435–436 item matching data group matching, 438–439 datum matching, 437–438 matches function, 437 rules, 435 implementation, 441 justification in expert systems, 457–458 large rule set management, 456–457 problem chaining, 431 condition–action rules, 430 database matching, 429 database rewriting rules, 430–431 data format in database, 431–433 pseudo-code, 433–434 Rete algorithm applications, 445–446 database matching, 447–451 example, 446 fact addition, 452 fact removal, 452 nodes, 446–447 performance, 454–455 update management, 453 rule arbitration dynamic priority arbitration, 442–443 first applicable rule, 441–442 least recently used rule, 442 most specific conditions, 442 random rule, 442 unification overview, 443–444 performance, 444–445 S Scheduling hierarchical scheduling behaviors overview, 726–727 selection, 728 data structures and interfaces, 727–728 interruptible processes hyper-threads, 723 micro-threads, 723 863 864 Index Scheduling (continued) software threads, 722 threads, 722 quality of service, 723–724 level of detail, 733–734 load-balancing scheduler data structures, 726 performance, 726 principles, 724 pseudo-code, 724–726 overview, 714 priority scheduling performance, 730 policies, 730 priority problems, 731 pseudo-code, 728–730 schedulers, 694 artificial intelligence slicing, 714–715 automatic phasing analytic method, 721 Wright’s method, 720 direct access algorithm applications, 718 data structures and interfaces, 719 performance, 719–720 pseudo-code, 718–719 frequencies, 715–716 implementation notes, 717–718 performance, 718 phase, 716 phase quality, 720 pseudo-code, 716–717 single task spikes, 721 Scheme, scripting of decision making, 471–472 Scripting decision making building your own language, 474–479 embedding, 468 language facilities compilation and interpretation, 467 extensibility and integration, 467–468 re-entrancy, 467 speed, 467 language processing compiling, 477–478 interpreting, 478 just-in-time compiling, 478 parsing, 476–477 tokenizing, 475 language selection commercial language advantages and disadvantages, 469 Java, 474 Javascript, 474 Lua, 470–471 open source languages, 469–470 Python, 472–473 Ruby, 474 Scheme, 471–472 Tcl, 473–474 overview, 466–467 group actions creating scripts, 573 data structures and interfaces, 572 implementation notes, 572 overview, 568 performance, 572 primitives action sequence, 570 animation action, 570 artificial intelligence action, 570 compound action, 570 signal, 568–569 state change action, 570 wait, 568–569 pseudo-code, 570–572 Seek and flee kinematic movement algorithm arriving, 51–53 data structures and interfaces, 50–51 flee, 51 orientation algorithm, 49–50 performance, 51 steering behavior data structures and interfaces, 59 flee, 59 maximum speed, 56–57 performance, 59 pseudo-code, 57–59 Sense management faking it, 760 fantasy modalities, 766–767 finite element model sense manager algorithm, 779–782 Index dispatching, 782 finite element model overview, 775–776 implementation notes, 783–784 intensity calculation from node to node, 781 iterations, 782 sense graph, 776–777 sight, 777–779 smell, 781 sound, 780–781 weaknesses, 783 hearing, 765 knowledge sources, 760–761 polling and notification, 761 regional sense manager algorithm, 767–769 camouflage and shadows, 772–773 data structures and interfaces, 771–772 performance, 772 pseudo-code, 769–771 weaknesses, 773–774 shooter game design, 818–819 sight brightness, 764 cones, 762–763 differentiation, 764–765 distance, 763–764 line of sight, 763 speed of light, 762 smell, 766 touch, 766 trends in games, 759 Separation behavior attraction, 84 implementation notes, 83–84 independence, 84 overview, 82 performance, 84 pseudo-code, 82–83 Shadow point, generation, 511 Sid Meier’s Civilization III, 10, 829 Sight, see Sense management Sim City, 553 SIMD, see Single instruction multiple data Simplified memory-bounded A*, overview, 274 The Sims, 8, 10, 22, 401 Single instruction multiple data, speed, 25–26 Slots, see Formation movement SMA*, see Simplified memory-bounded A* Smell, see Sense management Sniper Elite, 816, 818 Soldier of Fortune 2: Double Helix, 818 Space Invaders, Splinter Cell, 9, 759, 817 SSX, 823 Star Wars: Episode Racer, 22 State machine algorithm, 311 data structures and interfaces, 312–313 decision machine combination overview, 331–332 pseudo-code, 332–333 finite state machines, 310, 465–466 fuzzy state machines algorithm, 391–392 data structures and interfaces, 393–394 implementation notes, 394 multiple degrees of transition, 394–395 performance, 394 problem for solution, 391 pseudo-code, 392–393 hard-coded finite state machine performance, 318 pseudo-code, 316–317 weaknesses, 318 hierarchical state machine alarm behavior expression, 318–320 algorithm, 321–323 examples, 323–325 implementation, 330 performance, 330 problem for solution, 321 pseudo-code, 325–330 implementation, 316 Markov state machine algorithm, 391–392 data structures and interfaces, 393–394 implementation notes, 394 multiple degrees of transition, 394–395 performance, 394 problem for solution, 391 pseudo-code, 392–393 overview, 309 performance, 316 problem for solution, 311 pseudo-code, 311–312 865 866 Index State machine (continued) toolchain designers, 800–801 transition implementation, 313–315 transition states, 482 weaknesses, 315 Steering behavior align implementation notes, 65 overview, 62–63 performance, 66 pseudo-code, 64–65 arrive and leave implementation notes, 62 leave, 62 paths, 60 performance, 62 pseudo-code, 61–62 capability-sensitive steering, 174–174 collision avoidance cones, 84–86 performance, 89 pseudo-code, 87–89 time of closest approach calculation, 87 combining, see Arbitration; Blending delegated behaviors, 67 face behavior, 71 flocking and herding games interactivity tuning, 837–838 stability, 838 looking where you are going, 72–73 obstacle and wall avoidance collision detection problems, 92–95 data structures and interfaces, 91–92 overview, 90 performance, 92 pseudo-code, 90–91 overview, 55 path following behavior data structures and interfaces, 80 overview, 76–77 parameter tracking, 81 path types, 80 performance, 81 pseudo-code, 77–79 pursue and evade evade, 71 implementation notes, 70 overshooting, 71 paths, 68 performance, 70 pseudo-code, 69–70 seek and flee data structures and interfaces, 59 flee, 59 maximum speed, 56–57 performance, 59 pseudo-code, 57–59 separation behavior attraction, 84 implementation notes, 83–84 independence, 84 overview, 82 performance, 84 pseudo-code, 82–83 three-dimensional conversion angular steering behaviors, 180 linear steering behaviors, 180 two-level formation steering, see Formation movement variable matching, 56 velocity matching performance, 67 pseudo-code, 66–67 wandering data structures and interfaces, 76 overview, 73–74 performance, 76 pseudo-code, 74–75 Steering pipeline algorithm actuator, 111–112 constraints, 110–111 decomposers, 110 structure, 108–109 targeters, 109–110 data structures and interfaces actuator, 115 constraint, 114–115 deadlock, 115 decomposer, 114 goal, 115 paths, 116 targeter, 114 Index examples constraint, 118–120 decomposer, 117–118 targeter, 117 performance, 116 pseudo-code, 112–114 Strategic artificial intelligence, see Tactical and strategic artificial intelligence String matching, action prediction, 597 Super Mario Sunshine, Swarming, see Flocking and swarming T Tablebase, ending databases for board games, 703 Tactical analyses cellular automata applications, 552–553 complexity of behavior, 551–552 overview, 549–550 rules, 550 running, 550–551 convolution filters algorithm, 538–539 applications, 538 boundaries, 540 data structures and interfaces, 542 Gaussian blur, 543–544 implementation notes, 542 performance, 542 pseudo-code, 541–542 separable filters, 544–547 game level representation, 518 learning in tactical artificial intelligence with frag-maps, 527–528 map flooding algorithm, 534–535 data structures and interfaces, 537 overview, 533–534 performance, 537 pseudo-code, 535–537 sharpening filters, 547–548 simple influence maps applications, 523–524 influence calculations convolution filters, 522 equations, 519–520 867 limited radius of effect, 520–522 map flooding, 522–523 lack of knowledge handling, 525 overview, 519 structure combining analyses, 532–533 complexity levels, 528–530 multi-layer analysis, 531–532 server building, 533 terrain analysis difficulty of terrain, 526 visibility map, 526–527 waypoint tactics similarity, 512 Tactical and strategic artificial intelligence coordinated action, see Coordinated action evaluation in game design, 810–811 game artificial intelligence model, 10, 493 influence mapping, see Tactical analyses real-time strategy game design, 825–826 waypoint tactics, see Waypoint tactics Tactical movement anchor point moderation, 170–171 bounding overwatch, 169, 171 cover points, 169–171 Tactical pathfinding cost function, 553–554 heuristic modification, 557 tactical graphs, 557–558 tactical weights and concern blending, 555–557 waypoints, 558–559 Targeter, steering pipeline, 109–110, 114, 117 Tcl, scripting of decision making, 473–474 TD algorithm, see Temporal difference algorithm Teaching characters action representation, 832 brain death, 836 instincts, 836 learning mechanism feedback learning, 834–835 mind-reading, 834 neural network architecture, 833–834 observational learning, 834 player expectations, 835–836 world representation, 832–833 Temporal difference algorithm board game applications, 645 reinforcement learning, 644–645 868 Index Terrain analysis, tactical analyses difficulty of terrain, 526 visibility map, 526–527 Thief: The Dark Project, 8, 759 Threads hyper-threads, 723 interruptible process implementation, 722 micro-threads, 723 software threads, 722–723 Tic-Tac-Toe artificial intelligence applications, 667 game tree, 669–670 3D Tic-Tac-Toe, 828 Tile graph division scheme, 239 generation, 240 knowledge finding for pathfinding, 787 quantization and localization, 240 usefulness, 240–241 validity, 240 Tokenizing language processing, 475 Lex, 479 Tom Clancy’s Ghost Recon, 10, 763, 764 Tomb Raider III, 819 Toolchain data-driven editors, 799–800 design tools for artificial intelligence scripting tools, 800 state machine designers, 800–801 game development importance, 785 limitation on artificial intelligence, 786 plug-ins, 802 remote debugging, 801–802 Total Annihilation, 825 Touch, see Sense management Transposition table functions, 689 hashing game class, 691–692 hash table implementation, 693–694 implementation, 690–691 incremental Zobrist hashing, 691 overview, 689 values stored in hash table, 692–693 Zobrist key, 690 implementation notes, 695 instability, 696 memory-enhanced test algorithms, 697–699 opponent thinking time utilization, 696–697 path dependency, 696 performance, 695 pseudo-code, 695 replacement strategies, 694–695 Tribes II, 814 U Unification, see Rule-based decision making Unreal, 468, 816 V Variable matching, steering behavior, 56 Velocity matching performance, 67 pseudo-code, 66–67 Visibility point, generation, 510 W Wall avoidance, see Obstacle and wall avoidance Wandering kinematic movement algorithms data structures, 54 implementation notes, 54 overview, 53 pseudo-code, 53–54 steering behavior data structures and interfaces, 76 overview, 73–74 performance, 76 pseudo-code, 74–75 three-dimensional movement, 186–188 Warcraft, 8, 23 Warcraft: Orcs and Humans, 824 Warcraft 3: Reign of Chaos, 826 Warhammer: Dark Omen, 8, 825 Waypoint tactics automatic generation comparison of approaches, 517 watching human players, 512–513 waypoint grid condensation Index algorithm, 513–515 data structures and interfaces, 516–517 overview, 513 pseudo-code, 515–516 knowledge finding automatic graph creation, 789 data mining applications, 795 character movement, 794 connection calculation, 794 limitations, 794–795 node calculation, 793–794 geometric analysis arbitrary bounding regions, 791–792 connection calculation, 790–792 cost calculation, 790 mesh representations, 792 node calculation, 792–793 point-based representations, 790–791 visibility approach limitations, 792 manual region data creation bounded regions, 789 Dirichlet domains, 788 navigation meshes, 789 tile graphs, 787 node pathfinding, 494 tactical information utilization decision trees, 504 fuzzy logic decision making, 505–506 generating nearby waypoints, 506–507 pathfinding, 507 simple tactical movement, 503–504 tactical locations complexity levels, 502 compound tactics, 496–497 context sensitivity, 500–502 continuous tactics, 499–500 graphs and topological analysis, 497–499 overview, 494–495 primitive tactics, 496–497 sets, 495–496 tactical pathfinding, 558–559 tactical property generation compound tactics, 512 cover points, 508–510 overview, 507–508 shadow points, 511 tactical analysis similarity, 512 visibility points, 510–511 Website contents action manager program, 491 artificial neural network, 658 combining steering program, 105 decision tree learning, 625 decision tree program, 302–303 finite element model sense manager, 782 flocking algorithm, 100 fuzzy state machine program, 395 hierarchical state machine program, 331 kinematic movement algorithm, 55 libraries optimizations, 17 rendering and mathematics, 17 Web site updates, 17 Markov state machine program, 401 programs, 16 Q-learning, 635, 644 random decision tree program, 309 Rete algorithm, 454 state machine program, 315 steering behavior, 59 steering pipeline program, 116 Weighted blending, see Blending Wipeout, 823 Wolfenstein 3D, 814 World Championship Pool 2004, 827 World interfacing communication, 745–746 knowledge acquisition event manager, see Event manager events, 747 polling, 746 polling station, see Polling station selection of technique, 748 sense management, see Sense management World Rally Championship, 785 World representations cost functions, 251 Dirichlet domains, 241–244 generation, 238 navigation meshes, 246–250 non-translational problems, 251 path smoothing, 251–253 869 870 Index World representations (continued) points of visibility, 244–246 quantization and localization, 238 teaching characters, 832–833 tile graphs, 239–241 validity, 238–239 Worms 3D, 829 Y Yacc, parser building, 479 Z Zelda, 10 Zobrist key, transition table hashing, 690–691 .. .ARTIFICIAL INTELLIGENCE FOR GAMES Second Edition IAN MILLINGTON and JOHN FUNGE AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN... Artificial intelligence for games / Ian Millington, John Funge – 2nd ed p cm Includes index ISBN 978-0-12-374731-0 (hardcover : alk paper) Computer games Programming Computer animation Artificial intelligence. .. Platforms group Previously, John co-founded AiLive and spent nearly ten years helping to create a successful company that is now well known for its pioneering machine learning technology for games