A Web-based Infrastructure for Simulation and Training

A Web-based Infrastructure for Simulation and Training David Macannuco, Kenneth B Donovan, Ph.D., Mark Falash, Leo Salemann Lockheed Martin Simulation, Training and Support 164 Middlesex Turnpike, Burlington, MA 01890 Ph: 781-505-9536 david.j.macannuco@lmco.com , ken.b.donovan@lmco.com, mark.falash@lmco.com, leo.salemann@lmco.com Keywords: Distributed Simulation, XMSF, XML, HLA, Homeland Security ABSTRACT: Incorporating commercial standards, practices and technologies into government simulation and training systems has the potential to reduce development, production and maintenance costs dramatically Previous distributed computing infrastructure standards for simulation and training were originated by the defense market and address defense needs, but have only limited support as standards in commercial products Recent advances in commercial web-based technologies provide an opportunity to leverage these commercial standards and technologies into simulation and training systems As developers incorporate these web-based technologies, there are a number of architectural tradeoffs and design choices During the past year, the authors developed a distributed simulation and training system for homeland security using web-based commercial standards and technologies Our system analysis addressed several of the design issues that are faced when using web-based technologies This paper presents the results of this work, including empirical performance data from a distributed team training exercise conducted with the system These results will be a useful application reference case for the simulation community as it incorporates web-based standards into simulation and training Introduction During the past year, we developed a learning enterprise to be used for training in the homeland security domain The objective of the system is to provide an efficient learning environment to support a geographically distributed, professional community in its mission of enhancing emergency preparedness for both natural and terrorist events The primary audience is the Emergency Operation Center (EOC) staff that coordinates activities in support of the on-scene incident command A key requirement for this application is the use of simulation-based exercises that allow the EOC community to practice emergency procedures that are extraordinary; such as a large-scale biological or radiological contamination Simulation is used to provide interactive, geo-specific scenarios that engage the training audience for effective learning In addition to the simulation-based system capabilities, the audience must have access to a range of other learning services, such as instruction and training courses; a learning management function to guide the participant through the learning experience and record results; and evaluation and assessment tools to track progress and lessons learned To meet these requirements, we developed a system referred to as a learning enterprise In this paper, we discuss the system concepts and architecture of the learning enterprise, with a primary focus on the web-based integration infrastructure As part of system design, we assessed several infrastructure options for integrating the components of the system; including use of the High Level Architecture (HLA) and/or web-based computing standards (Java Messaging Service (JMS), Extensible Markup Language (XML), Hyper Text Markup Language (HTML), etc.) We selected web-based technologies to implement our learning enterprise architecture After completing the system, we conducted a largescale verification exercise to demonstrate the performance of the system architecture and components across the learning enterprise This design and verification experience provides one application reference case as the simulation community considers the use of web-based standards for simulation and training System Overview The learning enterprise includes functionality for computer-based simulation, evaluation, instruction and training courses, and learning management We use a client-server architecture to support the geographically distributed users with minimal client-side software for the user interface A top-level functional architecture of the system is shown in Figure and is briefly described The user interface, shown as the client side of Figure 1, provides all users (participants, planners, developers, observers, and evaluators) access to the rest of the system functionality as remote services A design goal is to provide a consistent user experience for accessing any of the functionality of the system via a web browser The Learning Management System (LMS) function serves as the common entry point for user interaction with the rest of the system and provides user account management The LMS provides registration and tracking for all participants, including those receiving training, as well as other participants (observers, role players, developers, etc.) The user can access different toolsets such as: Distance Learning for live or pre- Client Side Server Side Training Training Audience Audience Exercise Exercise Controllers Controllers Distance Distance Learning Learning&& LMS LMS EOC EOC Operational Operational Environment Environment recorded courseware and Exercise Management to initiate and control distributed exercise The EOC Operational Environment function provides to the EOC staff the communication tools needed for internal and external information sharing The EOC Operational Environment uses standard office tools such as email, voice communication (VOIP), and crisis information management system (CIMS) The Evaluation and Assessment function is used initially by the evaluation team to define, capture and brief assessment results Following the exercise, this function is available to the training audience to review exercise performance The Simulation and Geospatial Models function provide the discrete simulation of the emergency scene, representing the affected population area and emergency response equipment and personnel Evaluators Evaluators Role Players Role Players Exercise Exercise Management Management Evaluation Evaluation and and Assessment Assessment Observers Observers Simulation Simulation && Geospatial Geospatial Models Models Figure HS Learning Enterprise System Architecture Infrastructure Alternatives An overarching approach to this system architecture was to develop an open, standards based architecture that fully embraced commercial-off-the-shelfsoftware (COTS) products and industry standards HLA and Distributed Interactive Simulation (DIS) are the two major networking infrastructure technology standards used by the defense modeling and simulation (M&S) community to create distributed simulations While these technologies have been successful within the defense M&S community, they have not achieved general acceptance outside of the defense industry, and are not directly supported by the COTS product vendors that were preferred for our system functionality In recent years there have been significant advances in the commercial market in terms of standardization efforts, capabilities, cost, performance and availability of both new products and freeware supporting the standards Many of these standards and advances have been driven by the gaming and distributed internetbased application market COTS vendors gear their product roadmaps toward current and emerging standards for web based computing such as HTML, XML, the Simple Object Access Protocol (SOAP), Workflow and WSDL Since web based computing standards have been developed to handle integration of software components across many commercial enterprises, we considered their use to integrate components across our learning enterprise Our system design considered three basic options for leveraging the web-based infrastructure with the traditional modeling and simulation infrastructure These options are illustrated in Figure Client Side HLA HLA(or (orDIS) DIS)Network Network FOM Server Side Client Side Web-based Web-basedNetwork NetworkStandards Standards Option 1) HLA Infrastructure Server Side Client Side HLA HLAor orDIS DISNetwork Network Web-based Web-basedNetwork NetworkStandards Standards FOM Option 2) Mixed HLA / Web Infrastructure Server Side Option 3) Web-based Infrastructure Figure Primary Options for Network Infrastructure Option 1) Use HLA infrastructure (or alternatively DIS.) This approach has the advantage of leveraging our prior HLA experience in many defense training systems Some disadvantages with this approach include: a) the client-side is relatively heavy in order to support the HLA Runtime Infrastructure (RTI); and b) most COTS tools would require custom interface work, or wrappers, to interoperate with the HLA Option 2) Mixed HLA/Web Infrastructure This approach leverages the web-based technology to provide the remote communication with the user clients, while using HLA primarily for the simulation traffic during an exercise This allows a very thin client such as a web-browser, and a consistent interface to a variety of commercial tools, while providing an easy interface to an existing HLA-based simulation, such as JointSAF This approach also permits the use of the web-based infrastructure as the bridge between multiple HLA-based simulations that may be running on different RTI The disadvantage of the approach is the additional system complexity of maintaining two infrastructures Option 3) Web-based Infrastructure This approach, that we selected, uses the web-based technology for all communication This approach allowed us the maximum flexibility of leveraging the web-based technologies This approach did require reimplementing some simulation services that are provided “out of the box” by RTI implementations This was fairly quickly accomplished, and was a favorable trade since it provided easy access to all of the communication data for monitoring, recording, and recovery In addition, this approach supported a rich and robust development and integration environment for a team of geographically distributed developers We note that a key factor in our selection of Option (the web-based infrastructure) is that our homeland security application required primarily new simulation models, as opposed to re-use of existing military models We initially expected to re-use models from an HLA system (such as JointSAF), but we determined that those existing models did not provide the behaviors needed for our application If there had been a significant opportunity for re-using existing models running on the HLA infrastructure, we would likely have implemented Option Our implementation of the web-based architecture is described further in the next section 4 Logical Architecture The logical architecture for the learning enterprise is shown in Figure It follows the standard, layered architecture design pattern consisting of the following: Presentation, Application Logic, Business Logic, and Data Tiers Figure Homeland Security Learning Enterprise Logical Architecture 4.1 Presentation Tier The Presentation Tier is formed primarly from webbased standards and protocols HTTP and HTML (which can include Flash, etc.) The presentation tier provides the training audience, planners, evaluators, observers and role players access to the training system The presentation tier was designed to allow any participant’s access to the system using a web browser or standard desktop configuration to the maximum extent possible The browser-based clients represent the minimum requirement needed for the training audience, evaluators, observers and exercise control participants to participate in an exercise However, given the current state of the industry, some capabilities require installation of client software Geo-spatial services and Voice-Over-IP (VOIP) are examples where this occurred To utilize the desktop tele-communications tool which is based on VOIP, a client application must be installed on each client 4.2 Application Logic Tier The Application Logic tier provides the glue that binds selected components together in a common operating environment or user interface The Presentation Tier is coupled to the server side business logic through the application logic tier which consists of presentation logic and a workflow engine The presentation layer is used to map logical data layouts to physical presentations This approach provides a method for separating the presentation from the physical data, making for a flexible and adaptable system The developer of the presentation is free to develop the look and behavior of the presentation without immediate concern with where the data lives and is formatted The workflow layer is used to determine how events are processed (e.g., what process should handle a particular event.) The events can be originated by a human participant through the presentation layer or by an application through the workflow engine using the JMS and XML documents The workflow engine’s events and action rules are data driven as well and contained in XML documents 4.3 Business Logic (and Component Clients) Tier The Business Logic Tier contains the intelligence of the learning enterprise and is made up of several loosely coupled components that provide services to other components Simulation models, geographic information services, exercise management, evaluation/assessment and data recording/playback services are part of this tier In addition this layer is used to leverage COTS products that provide system capabilities, such as the CIMS software (WebEOC) and Resound desktop delivery tools The business logic includes the GIS Services that process geospatial information requests from other server components, and from the GeoViewer for human interaction The GeoViewer provides situational awareness to the various participants and is used to perform various geospatial operations such as deploying responders and relief supplies The GeoViewer is a map display that is based on ESRI MapObjects/Java In holding with a Service Oriented Architecture (SOA) approach, the services or system interfaces were designed and implemented using XML schema This provides a straight forward mechanism to maintain compliance with the Web Services standards as they evolve and become accepted, such as Web Services Definition Language (WSDL) However, some COTS products we selected did not fully support these web standards In order to preserve the desired system object model and interfaces, we developed “component clients” that bridged the gap between the desired system interface and the COTS product interface These component clients are shown as a separate tier, although they can also be considered part of the Business Logic Tier Through the use of component clients, the component’s implementation details are hidden from the broader system, allowing other components to be designed and run independent of one another or other component implementation choices The component client packages were quite simple for some COTS products that supported web based standards and open interfaces Other COTS products required a more complex client to address vendor-proprietary interfaces One example of a simple component client is the GIS Adapter that provides the interface to the GIS Services The GIS Services were implemented as a Microsoft ActiveX COM object, so an interface was needed to our web-based infrastructure The GIS Adapter example is described briefly below to illustrate operation of the learning enterprise infrastructure (reference Figure 3.) The GIS Adapter component client is implemented as a Java Servlet that runs within the Apache Tomcat Servlet Engine It communicates to the GIS Services in the Business Logic Tier, via SOAP The GIS Adapter communicates with the JMS workflow engine, listening on a Queue dedicated to GIS Services When a message is detected, the GIS Adapter extracts the textual content (formatted as an XML document) and forwards it to SOAP, which in turn makes a COM object call to the GIS Services The GIS Services parse the XML, performing the requested operation against a GeoSpatial Database located in the Data Tier The GIS Services return the result formatted as another XML document SOAP forwards this result document to the GIS Adapter, which posts the result to the general JMS workflow engine queue With this approach, a new implementation of the GIS Services can be inserted provided that the XML interface is maintained 4.4 Data Tier The Data Tier of Figure contains the databases used by the various components of the learning enterprise Data created and used by the system is managed via an XML enabled database This approach allowed the system to use XML schema to define both the services provided and the data repository structure These databases include course material and student records for the Learning Management System, geospatial data for the GIS Services, and models/behaviors data for the Modeling and Simulation services Most of these databases are currently implemented in Oracle, with XDB, RDBMS, and XML interfaces Of particular note is the GeoSpatial Data repository This database is also implemented in Oracle, but the spatial component is managed by the ESRI ArcInfo Spatial Data Engine (ArcSDE) ArcSDE provides a unified spatial representation to a suite of ESRI Geographic Information Systems (GIS) tools, allowing the data to be stored in several databases formats in addition to Oracle, such as Microsoft SQL Server, IBM DB2, and IBM Informix The GIS Services communicates with the Oracle database via the ESRI ArcSDE interface, but the GIS Services communicate with other learning enterprise components via XML We have designed an XML schema composed of geospatial transaction messages The GIS Services receives these XML messages via JMS, performs the appropriate ArcSDE transaction, and transmits the results to JMS as another XML message Performance and Scalability The Homeland Security Learning Enterprise represents one of our first significant efforts to develop and demonstrate a web-based simulation and training system A key issue was the type of performance the system would exhibit In this section, we look at the performance from both the operational perspective and the development cycle perspective 5.1 Operational Performance The key performance goal was to support a live exercise with a representative number of distributed training sites, participants, interaction frequency, and duration We conducted a 3-day verification exercise involving over two dozen participants - including trainees, role players, controllers, evaluators and observers The exercise participants were distributed at our Orlando, FL facility across two exercise rooms, one control room, one observation room and a server room; with remote monitoring from our sites in Bellevue, WA and Burlington, MA All communication during the exercise was via the company intranet External connection to the internet was made to access the Resound servers in Baltimore, MD for distance learning and web-delivered after action review During the 3-day exercise, the audience participated in an 8-hour simulation-based exercise The simulation provided an emergency spanning two states, and included interactions with over a dozen local, state and federal agencies or positions The audience generated over 300 communications (email or phone calls) The simulation system successfully advanced the simulated emergency scene in response to audience actions and the simulation model behaviors The learning system successfully supported the entire verification exercise We also obtained two quantitative measures of the system performance as an indication of robustness The first was a measure of the utilization of the servers during the course of an exercise and the second was a measure of the responsiveness to the users as the number of users increases To obtain these measures under the most representative operating conditions, we modified the system set-up The 3-day verification tests described so far were conducted behind the corporate firewall, providing us some control over the network environment However, this verification environment did not include the web gateway and firewall that normally would exist between the servers and the user community To test robustness, we attempted to replicate the World Wide Web (WWW) without actually deploying on the WWW First, the servers were isolated to a standalone network that permitted direct communications only between the server configurations Second, a gateway and firewall were added per corporate standards and configured to mimic the bandwidth of existing installations Last but not least, another network was set up outside the firewall to host the user environment, which consisted of four (4) laptops running Microsoft Windows XP and Internet Explorer 6.0 Utilization during an exercise was measured by replaying the message traffic recorded during the 8-hour verification exercise On replay, the system reintroduces the traffic using the same mechanism as used by the application that generated the traffic We collected data at each server for CPU and Memory, using vmstat for Linux servers and Task Manager for Windows 2000 Servers Figures and show the server CPU and Memory utilization for the two most utilized Windows 2000 and Linux servers The database server chart (Figure 4) shows several spikes in the disk queue length during the early portion of the run During this period the simulation models component is accessing the geo-spatial database to initialize its models A second, much smaller spike occurs as the exercise starts Once the models have been initialized only changes or updates are communicated to and from the database server Once the exercise is running, the servers have fairly light utilization (under 20%), with little variation in utilization This utilization result was promising in showing significant spare capacity with the potential to support a larger scale simulation in a training session Figure Data Base Server Utilization Figure Arc Objects Server Utilization Our second quantitative measure was of the impact on the system as the number of users increase Test software was installed on the client laptops that simulate user logons The tool was configured to access pages that represent the users and is based on our best engineering estimate of expected user accesses Test runs were made simulating 100, 300, 400 and 1000 users with results shown in Figures through Figure 10 Figure shows the test profile where we increased the number of users over time, beginning with 100 users, followed by 300, 400 and finally 1000 users simulated Figures 7-10 show the corresponding system performance Figure Active Users Increases with Time Figure Transaction Profile as Users Increase Figure Throughput Profile as Users Increase Figure Http Hits Profile as Users Increase Figure 10 Trans (busy) ok[s] Profile as Users Increase As expected, the activity (Transactions, Throughput, and Hits) increases as the number of users increases A favorable result of this particular performance test run is that response time stays pretty constant even as number of logons increase, as shown in Figure 10 This was not entirely a surprise, since the bulk of the traffic occurs between the servers of the internal network and only passes to the users in summary form and at relatively low update rates 5.2 Development Performance The learning environment clearly benefited from the use of open standards, commercial products and open source or freeware Numerous COTS products were leveraged to provide the learning environment functional capabilities In addition, the web-enabled, COTS approach paid significant dividends in the development and integration process by facilitating a truly distributed, collaborative development effort The primary benefit came from the availability of capable and robust products used to satisfy requirements in a short period of time The infrastructure is formed from open source products such as JBOSS, e-mail server and Java Messaging Services Products such as ESRI allowed for very rapid development of GIS services and models WebEOC provide an off the shelf crisis information management system used in many emergency operations centers VOIP software phones provided voice transmission, recording and playback with no real development required Of course, the use of COTS provides a new set of integration challenges that constrain the system design or can require establishing a relationship with the vendors A disadvantage of this approach is that you may have to treat many of these products as black boxes since most were not designed with the thought of being integrated with training systems For example, the goal of providing training capabilities entirely via a web browser could not be completely achieved since many products are designed to be installed on the desktop and are implemented using proprietary protocols A second benefit was seen in the development environment The development team was distributed across the country at sites in Orlando, FL; Burlington, MA; Moorestown, NJ; Bellevue, WA; Albuquerque, NM; Marietta, GA, and Baltimore, MD The development environment leveraged COTS tools such as Netbeans for Java development; CVS for version control; Netmeeting for collaboration; XMLSPY for XML development; ESRI GIS for geographical services; and Resound for desktop delivery All development, integration and test occurred while developers collaborated from their home offices The team’s first face to face meeting occurred the week prior to the first validation exercise All follow-on exercises were performed with supporting staff participating from remote locations while the trainees participated from a simulated emergency operations center Although there clearly are some integration challenges that come with the use of COTS, these disadvantages will be minimized over time due to the rapid development and adoption of open standards The advantages will continue to increase with the significant investment in the commercial world in relevant technologies that cannot be matched by the modeling and simulation based training market Summary and Conclusions We have described an approach for leveraging webbased technologies for a system that provides a learning environment including instructional modules and simulation-based exercises We identified some of the early system design trades we made, including the choice to leverage the web-based technologies as the general infrastructure This required replacing the HLA RTI with comparable services, but yielded a more consistent and efficient logical architecture for our application Finally, we described results of our verification exercise that showed significant benefits of the web-based architecture As the modeling and simulation community looks at leveraging new technologies, there are many issues that enter into the tradeoffs These include interoperability with established modeling and simulation systems, transitioning systems currently under development to be “web-enabled”, and training the development staff in the new technologies While there are technology insertion risks, these are more than offset by the significant benefits of additional functionality and ease-of-use of the commercial technologies Our experience recommends a rapid and pervasive adoption of these technologies into modeling and simulation systems References [1] Extensible Modeling and Simulation Framework (XMSF) Challenges for Web-Based Modeling and Simulation, TECHNICAL CHALLENGES WORKSHOP, STRATEGIC OPPORTUNITIES SYMPOSIUM 22 OCTOBER 2002, Don Brutzman and Michael Zyda [2] US Department of Defense, High Level Architecture Interface Specification, Version 1.3, April 1998 [3] World Wide Web Consortium, SOAP Version 1.2, June, 2003 [4] World Wide Web Consortium, XML 1.0 Third Edition, February, 2004 [5] Sun Microsystems, Java Messaging Service Specification 1, March 2002 [6] World Wide Web Consortium, HTML 4.01, December, 1997 [7] World Wide Web Consortium, WSDL 2.0 Draft, August, 2004 Author’s Biographies DAVID MACANNUCO is a Senior Staff Software Engineer at Lockheed Martin STS Advanced Simulation Center in Burlington Massachusetts Dave led the development of the simulation software for the Homeland Security learning enterprise system Dave has over eight years experience in distributed simulation, focusing on HLA and RTI middleware solutions Dave is currently the M&S technical lead for the Lockheed Martin Global Vision Center Dave has a BSEE from the University of Rochester and an MSEE from Boston University KENNETH B DONOVAN is a Principal Engineer in Advanced Programs at Lockheed Martin Simulation, Training and Support Ken led the recent development of the homeland security learning enterprise He has 25 years experience in the simulation industry, with a focus on synthetic environment architectures and product development Ken has numerous publications and patents in the field He received a MS in Computer Engineering from Clarkson University and a PhD in Computer Science from the University of Central Florida MARK FALASH is a Senior Staff Software Engineer and Principle Investigator for Lockheed Martin Simulation, Training and Support (LM STS) Internal Research and Development organization His current responsibilities involve investigation of technologies and products applicable to the training solutions enterprise and architecture development In addition, he is the technical lead for LM STS development of training preparedness and readiness tools targeting homeland security opportunities Prior responsibilities included development of web enabling technologies for training systems, LM STS core architecture and infrastructure for virtual training simulators, reconfigurable simulator and HLA compliant integration He received an M.S in Computer Science from California State University, Chico LEO SALEMANN is a Senior Staff Software Engineer with Lockheed Martin Simulation, Training and Support Advanced Simulation Center in Bellevue, WA He received his Bachelor's of Science in Computer Science & Engineering from the University of Washington in 1993 and has been working for the LM STS Bellevue office ever since Leo is an expert in object-oriented development in Visual Basic as well as UNIX/C environments Leo contributed to the WARSIM/TDFS program from 1998 to 2003, and has participated in the design, implementation, documentation, and testing phases His focus has been in the User Interface and Application layers, in which he was primary author of numerous mission critical software components Leo's previous work includes training, documentation, release generation and customer support for the Vistaworks real-time visualization software, and the GT200 image generator Note: All trademarks property of their respective owners  ... population area and emergency response equipment and personnel Evaluators Evaluators Role Players Role Players Exercise Exercise Management Management Evaluation Evaluation and and Assessment Assessment... the standards Many of these standards and advances have been driven by the gaming and distributed internetbased application market COTS vendors gear their product roadmaps toward current and emerging... Central Florida MARK FALASH is a Senior Staff Software Engineer and Principle Investigator for Lockheed Martin Simulation, Training and Support (LM STS) Internal Research and Development organization