1. Trang chủ
  2. » Luận Văn - Báo Cáo

Research "MANAGING SIMULATION BASED TRAINING A FRAMEWORK FOR OPTIMIZING LEARNING, COST, AND TIME " pot

139 333 0
Tài liệu được quét OCR, nội dung có thể không chính xác

Đang tải... (xem toàn văn)

Tài liệu hạn chế xem trước, để xem đầy đủ mời bạn chọn Tải xuống

THÔNG TIN TÀI LIỆU

Thông tin cơ bản

Định dạng
Số trang 139
Dung lượng 5,13 MB

Nội dung

Trang 1

INFORMATION TO USERS

This manuscript has been reproduced from the microfilm master UMI films the text directly from the original or copy submitted Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer

The quality of this reproduction is dependent upon the quality of the copy submitted Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction

in the unlikely event that the author did not send UMI a complete manuscript and tnere are missing pages, these will be noted Also, if unauthorized copyright material had to be removed, a note will indicate the deletion

Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps

ProQuest Information and Learning

300 North Zeeb Road, Ann Arbor, MI 48106-1346 USA 800-521-0600

Trang 3

MANAGING SIMULATION BASED TRAINING

A FRAMEWORK FOR OPTIMIZING LEARNING, COST, AND TIME

A DISSERTATION

SUBMITTED TO THE DEPARTMENT OF MANAGEMENT SCIENCE AND ENGINEERING AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULLFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF

DOCTOR OF PHILSOPHY

Trang 4

UMI Number: 3085223

Copyright 2003 by Richmond, Noah Joseph

All rights reserved

® UMI

UMI Microform 3085223

Copyright 2003 by ProQuest Information and Learning Company All rights reserved This microform edition is protected against

unauthorized copying under Title 17, United States Code

ProQuest Information and Learning Company 300 North Zeeb Road

P.O Box 1346

Trang 5

© Copyright Noah J Richmond 2003 All Rights Reserved

Trang 6

I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy

William Perry

I certify that I have read this dissertation and that, in my opinion, it is fully adequate in

scope and quality as a dissertation for the degree of Doctor of Philosophy

Trang 7

MANAGING SIMULATION BASED TRAINING

A FRAMEWORK FOR OPTIMIZING LEARNING, COST, AND TIME

-Abstract-

This study provides a management framework for optimizing training programs for learning, cost, and time when using simulation based training (SBT) and reality based training (RBT) as resources Simulation is shown to be an effective means for

implementing activity substitution as a way to reduce risk The risk profile of 22 US Air

Force vehicles are calculated, and the potential risk reduction is calculated under the

assumption of perfect substitutability of RBT and SBT Methods are subsequently developed to relax the assumption of perfect substitutability The transfer effectiveness tatio (TER) concept is defined and modeled as a function of the quality of the simulator used, and the requirements of the activity trained The Navy F/A-18 is then analyzed in a case study illustrating how learning can be maximized subject to constraints in cost and time, and also subject to the decision maker’s preferences for the proportional and absolute use of simulation Solution methods for optimizing multiple activities across shared resources are next provided Finally, a simulation strategy including an operations planning program (OPP), an implementation program (IP), an acquisition program (AP), and a pedagogical research program (PRP) is detailed The study provides the theoretical tools to understand how to leverage SBT, a case study demonstrating these tools’

efficacy, and a set of policy recommendations to enable the US military to better utilize SBT in the future

Trang 8

In Memory of Gian-Carlo Rota

Trang 9

Managing Simulation Based Training

A Framework for Optimizing Learning, Cost, and Time

Noah Joseph Richmond

Executive Summary: Simulation Strategy 1

Organization and Principle Findings 1

Chapter I: Risk Exposure and Reduction via Simulation 1 Chapter IA: Military Personnel Risk Exposure and Risk Reduction Strategies 2 Chapter IB: Risk Assessment and Impact of 1:1 SBT:RBT exchange 3 Chapter II: Transfer Effectiveness Ratio and Total Training Resources 4 Chapter III: Optimization Framework and Computational Form 5

Chapter IIIA: Optimization Framework 5

Chapter IIIB: Computational Form 6

Chapter IV: Case Study and Algorithmic Techniques 6

Chapter IVA: F/A-18 Training Program and Formulation 6 Chapter [VB and Chapter [VC: Formulation and Solution Methods of the LATT

Activity in the (L,C,T) Framework 7

Chapter IVD: Formulation of the Multi-Activity Strategic F/A-18 Optimization 8

Closing Remarks 9

Chapter [ 10

Risk Reduction of Military Aviation through Simulation 10

Flight Training: Key Element of Aviation Accidents II

Strategies for Risk Reduction 12

Reduced Risk Training 13

Estimating the Value of Simulation Based Training 13

If SBT is a perfect substitute for RBT, how should SBT be used? 14

Chapter II 33

The Link between simulation and reality based training: TER 33

What is TER? 33

Transfer Effectiveness Ratio: Qualitative Definition 33

TER: Mathematical Definition 34

What Factors Determine TER? 36

Physiology of TER 37

The Technology of TER 40

The Interaction of Physiology and Technology 44

Chapter IITA 50

Structure of the Tactical Model 50

The Total Training Resources (TTR) Function 51

The Optimization Framework 58

Chapter IIIB 62

Computational Form 62

Trang 10

Boundary Solutions

Optimization with Respect to Cost

Optimization with Respect to Time Optimization with Respect to Cost and Time Chapter IVA Introduction: The Navy F/A-18 fighter Overview Strategic Significance Risk Profile The Inter-deployment Training Cycle Readiness: Peaks and Troughs Stages of Training Logistical Constraints

Skill Sets and Hierarchies

The Primary Mission Area

PMA Types and Associated Skills

The PMA Points System

PMA Missions

Activities

Modeling Readiness

Chapter [VB

Formulation: Low-Altitude Tactical Training (LATT)

Construction of the Learning functions

Estimating T

Estimating L

Navy Flight Simulators: Assessing TER

Simulator Types and Measures of Quality

Simulator Types: PTT, OFT, TOFT, and WTT

Capital Assets: Type, Quantity, and Location

Measures of Quality: Evaluating Vision and Acceleration Systems TER Estimation Logistic TER estimation for the F/A-18 Cost of Training Optimization Summary: Low-Altitude Tactical Training (LATT) Chapter [VC

Solution Methods and Solution Properties

Present Solution for LATT

Boundary Solutions for LATT

Parametric Solutions: Simultaneous Cost and Time Constraints

Optimal Training Programs

Constraints in a and B

Optimal Training for different values of alpha

Optimal training for different values of B

Trang 11

Chapter IV D

The Total Readiness Function

PMA Readiness Functions

Combining PMA Readiness Functions to form Total Readiness

Variations in Readiness Measures

Shared Resource Constraints: Cost and Time

Shared Resource Constraints: Simulator Capacity

Total and proportional use of SBT and RBT

A Two Activity Example

Solution Techniques and [nterpretations: Overview

Strategic-Tactical Optimality Condition (STOC)

Interpreting Results: The Implied Allocation Model (AM)

What If Analyses: Leveraging STOC and IAM

Chapter V: Simulation Strategy

Summary of Results

Developing and Implementing a Simulation Strategy

Components of a Simulation Strategy

Trang 12

Executive Summary: Simulation Strategy

This study provides a management framework for understanding the impact of simulation based training Chapters [ through IV provide the analytic machinery necessary to

analyze, formulate, and optimize training programs This chapter previews and

summarizes the principle findings of the study Chapter V closes with recommendations for present policy planning, and future research

Organization and Principle Findings

The chart below illustrates the organization of the study along single activity (“micro”) and complete training program (“macro”) lines, and between theory and application RES x x X x X x xX X xX X XxX X xX X

Methods for understanding single activity analysis, as well as complete training programs are developed Real data is supplied for the analysis of the total risk exposure of the US Air Force, and activity level analysis of the Navy’s F/A-18 A short overview of the primary findings of each chapter is next provided

Chapter |: Risk Exposure and Reduction via Simulation

Trang 13

question, “If SBT and RBT are perfect substitutes, how should SBT be used?” The chapter thus provides a descriptive analysis of current risk exposure, strategies for its reduction, and a first-cut analysis for the cost savings that can be achieved using simulation

Chapter IA: Military Personnel Risk Exposure and Risk Reduction Strategies

The military has seen a dramatic reduction is risk exposure in the last twenty years In

1980, 2400 servicemen died The Air Force alone lost 350 due to unintentional injury By

1993, 1300 lives were lost The Air Force cut losses due to accidents to 100 persons While total accidental deaths fell by 71%, the accidental death rate fell a less dramatic 58% because the military downsized

The loss reductions were driven by improved training doctrine and equipment reliability

Though accident rates have been halved throughout the force, risk exposure remains high, and more recent gains have leveled off Today, five military personnel per month die in aviation accidents Over ten years, these deaths exceed combined losses from the Gulf War and military operations in Kosovo Training is now riskier than war; accident rates are sufficiently high to threaten operational tempo While the rate averages five per month, there are periods when three or four times the average may occur nearly

simultaneously When multiple accidents occur, operations must be addressed politically in order to maintain confidence in the military

It is therefore imperative to find new ways to reduce risk exposure Chapter JA reviews

Trang 14

Chapter IB: Risk Assessment and Impact of 1:1 SBT:RBT exchange If SBT and RBT are perfect substitutes, then one training hour of SBT can be used to

replace one training hour of RBT without impacting the quality of learning that takes place The risk associated with the live flight hour is eliminated, replaced by the (nearly) risk free simulation training hour—when a simulator is “crashed” there is no loss of life or equipment! While the assumption of perfect substitutability is naive, it is a good point of departure Chapter IB provides an analysis of risk reduction through the use of

simulation under this assumption The remainder of this study provides methods for analyzing the more complex problem where training methods have different pedagogical

values, costs, and exercise times

Chapter IB begins by showing that time, i.e a training hour, is the proper measure of the independent variable Loss rates, the dependent variables, are then defined as the average number of deaths per training hour The loss rates are calculated for each of 22 vehicles

in the US Air force by analyzing their complete historical risk profiles Because loss rates

change over time, it is necessary to model the trend in these rates The cumulative average loss rates are calculated, first for loss of life, and then for cost of capital These

rates are combined to forecast total losses for each vehicle type for a five year horizon

This analysis is referred to as the Single Vehicle Loss Model (SVLM), and measures how risky each vehicle platform is using a single summary statistic

These SVLM statistics are then combined in the Multi-Vehicle Loss Model (MVLM) to

show how losses can be reduced through the use of simulation The MVLM shows that total forecast losses for the five year period 2002-2006 is $3.3 Billion Savings can be made by substituting simulation based training for live flight Because simulation is assumed to be a perfect substitute, the optimal solution is degenerate—all live flight should be replaced with simulation because of its cost advantage As a result, two

constraints are added: (1) Proportional and (2) Total use of simulation Respectively

denoted by a and , the first limits the percentage of hours that may be substituted away for any single vehicle, while the latter limits the total number of hours substituted across

Trang 15

savings as a function of a and B As expected, the larger the total number of hours permitted, the more savings that accrue However, marginal savings decline with increased volume, because the optimal approach is to substitute away the riskiest (most costly) hours first

Chapter Il: Transfer Effectiveness Ratio and Total Training Resources

Much of the debate surrounding the use of simulation today is based upon a lack of agreement on the impact of substituting one training activity for another Chapter I is performed under the restrictive analysis of perfect substitutability, despite the fact that different activities have different pedagogical value, and a method to account for these

differences is needed

Chapter II provides the theoretical foundation necessary to lift the assumption of perfect substitutability The impact on pedagogical value is separated into two parts: (1)

Calculating aggregate training, and (2) Modeling learning as a function of aggregate training The keystone is the concept of the Transfer Effectiveness Ratio (TER) The transfer effectiveness ratio is a method of converting the contribution of a single training method into a measurable, common currency of ‘training resources.’ The total of these

contributions is referred to as Total Training Resources (TTR), and learning is modeled

as a function of the single variable TTR

Chapter [I provides methods for modeling and assessing TER The analysis proceeds in four steps The first is the definition of TER, which is expressed as a simple

Trang 16

of training method thereby leads to predictable outcomes—the amount of learning accomplished

The third and final step discusses current methods for assessing TER Advantages and disadvantages of three current methods are discussed: (1) Pilot and expert opinion, (2) In-

simulator learning, and (3) Direct transfer In addition, a new method, termed ‘logistic

analysis,’ is developed It models TER as a function of two variables, Frequency and Quality Frequency is the rate at which practice cycles can be completed, and Quality is a

composite measure of the capabilities of the technology to deliver realistic synthetic stimuli This method for TER estimation thus builds on the previous analysis of

physiology and technology, and is the only method that separately accounts for the quality and quantity of training delivered

Chapter Ill: Optimization Framework and Computational Form Chapter III describes the management framework in which complete training programs are modeled and optimized The chapter has two parts Part A formulates the objective

function and constraints in their most intuitive form Part B provides an equivalent

program more suitable for computation

Chapter IIIA: Optimization Framework

The optimization framework provides a method for the efficient trading off of learning, cost, and time for the training of a single activity Mathematical definitions of learning,

cost, and time are provided Each is described as a function of the types and quantities of

training resources consumed In addition, the constraints a and B on proportional and total

use of simulation are formalized The equations provide a framework in which one of

Trang 17

Chapter I1IB: Computational Form

The optimization of principal interest to the military is maximizing learning subject to constraints in cost and time Learning, L(x), is designated as the objective function By construction, L(x) is a composite function consisting of a concave function of a linear function of x As a result, the embedded linear function can be maximized instead of L(x) Because the constraints in cost, C(x), and time, T(x), are linear, minor algebra results in a linear program of the form:

Maximize f (x)

sử

Ax<b x20

Referred to as the “computational form,” this program can be solved rapidly using simplex or any other basic solution methodology

Chapter IV: Case Study and Algorithmic Techniques

Chapter IV applies the theory of Chapters II and II in a case study of the Navy’s F/A-18 The principle strike fighter deployed on aircraft carriers, the F/A-18 forms the backbone of sea-based, power projection capabilities Because of it’s sheer numbers, and

comparatively high risk profile, the F/A-18 is an ideal subject for a case study The analysis proceeds in four steps First, a thorough review of the Navy’s current training program for the F/A-18 is provided The Low Altitude Tactical Training (LATT) activity

is then formulated in the optimization framework A numerical analysis for LATT

follows Chapter IV closes by showing how multiple activities can be simultaneously optimized over a set of common resources so as to maximize total readiness

Chapter IVA: F/A-18 Training Program and Formulation

Trang 18

“activities,” of the F/A-18 By modeling activity level readiness as a learning function L(x) in the optimization framework, the tools of Chapters II and III can be applied

Chapter IVA also describes the Navy’s simulator assets, and shows how they are modeled as training resources in the optimization framework The TER for each

simulator when applied to Low Altitude Tactical Training (LATT) is determined using the logistic analysis method Provision of the cost of training for each method completes the necessary data sets Chapter IVA concludes by providing the general form for the LATT tactical optimization program

Chapter !VB and Chapter IVC: Formulation and Solution Methods of the LATT Activity in the (L,C,T) Framework

Chapter IVB takes up where Chapter [VA ends, translating the general form of LATT

into its computational form Learning is designated as the objective function, subject to constraints in cost and time Chapter [VC then demonstrates solution methods The numerical analysis begins by parametrically maximizing learning subject to simultaneous constraints in cost and time The program is first optimized with a and B unrestricted, and

subsequently repeated with a gradually restricted and B free, and then B restricted while a

is free The step by step analysis shows how the constraints interact to select the optimal training program Two principle concepts arise: (1) The efficient frontier, and (2) a

solution heuristic

The efficient frontier is found by plotting the training methods on two dimensions: TER and Cost The efficient frontier is the boundary formed by taking convex combinations of those points with the best combinations of TER and Cost When a and B are unrestricted, the training methods selected must lie on the efficient frontier The efficient frontier is a useful concept for understanding which technologies offer advantageous TER-Cost combinations in the framework of a system of training platforms

The solution heuristic provides a way for understanding what solutions on the efficient

Trang 19

deviate from the efficient frontier When a and B are free, problems in which time is scarce will favor methods on the efficient frontier with a high TER ratio When budgets are tight, methods with a high TER/$ ratio will be preferred The heuristic states that as a

is constrained, the most efficient (i.e., closes to the efficient frontier) RBT methods will

enter the solution mix When B is constrained, RBT may or may not be used Together, these principles provide a way of looking at the training methods on a simple graph, and then understanding how the program’s constraints shape the optimal solution

Chapter IVD: Formulation of the Multi-Activity Strategic F/A-18 Optimization

The final piece of the methodology puzzle is to show how several different activities can

be optimized over a shared set of resources The total readiness (TR) function shows how

the objective functions L(x) of the individual activities can be joined into a single

strategic objective function It is simply a linear combination where the weights assigned

reflect the relative importance of the activities trained: TR(Z) =Ze L(x)

The constraints of the multi-activity (strategic) level problem are analogous to the single activity (tactical) problem Total cost and time constraints are enforced as limits on the sum of the individual costs and time limits Likewise, a and B limit the proportional and total use of simulation across all activities Capacity constraints for total available hours on each training method are added

Two properties follow from this structure The first is the Strategic-Tactical Optimality

Condition (STOC) This condition states that the solution to the strategic program consists of solutions to the tactical programs that are optimal when viewed alone The property holds when the constraints of the tactical problems are set to the values implied by the strategic solution For example, if the strategic solution requires that $4,000 and five hours be spent for Low Altitude Tactical Training, then the tactical solution is

Trang 20

The STOC property is useful because it leads to the Implied Allocation Model (LAM) This model describes strategic optimization as occurring in two steps First, the strategic constraints are “allocated” across the individual sub-problems For example, the total cost budget is divided across the individual tactical models, each of which may use up to its

allocated portion of the total available budget Second, after allocation, each model is

optimized given its now fixed resource constraints Optimal solutions for the strategic model can be found informally by trying different allocations, or through a standard search algorithm

The STOC property parallels how actual budgeting is accomplished It provides a method for conducting the what if analyses and re-planning current training regimes For

example, the strategic program can be optimized given current budget and time

allocations It can then be re-optimized with these tactical allocations left free The two model runs indicate where budgets should be tightened or increased, where time should be reduced or increased, and in what training activities simulation should be emphasized or reduced Guidance on how to change the training program is thereby provided

Closing Remarks

Trang 21

Chapter |

Risk Reduction of Military Aviation through Simulation Aviation accidents are a major source of risk for the US military Flight training in particular is a dangerous business In World War II, 120,000 American aviators were killed Of these, 15,000—one in eight—died in training.' Today, Accidents are the number one cause of death amongst US servicemen Accidents have superceded enemy action as the military’s number one source of fatalities, claiming half of all fatalities from the period spanning 1980 to 1993 The charts below show that for both men and women,

accidents amount to a clear majority of deaths, claiming respectively 58.4% and 50.7% of

persons This rate more than doubles that of death due to disease and illness.”

'đ8 Title

Fuzure Air Force ta Oismi Sut on co Maes = artes by Cause ofCeam 1930 1993 No5 292 Figure Ar Force - to Crsintabon cf Female F atanves oy Dause of Death, 1980-1999 N=337 Os ease ang liness Cusease and uress 8962 720% TS 216% } Homicide "42x i?e “39 1% tara Sot ce ald 140% Se nteraonal inate : 2048 98 KV ` ‘rentenbonal Injury 176 %0 7% “ a a ” Creer BE an 41 !0% Other ° 103% * hchudes deaths O.¢ 20 hustite action (n= ZH} and hose deaths where

Couse ss ther unclessfed_ cr uns mown [ns 18) * Includes ore death where Cause 1s csher

` ⁄/

Accidents during flight compose one of the greatest risks undertaken today: It is one of

Trang 22

(disease, homicide, and suicide) are not associated with major capital losses, accidents are the primary peace time driver for the loss of capital equipment Peacetime losses from the last 10 years exceed the combined losses of the 1991 Persian Gulf and Kosovo air campaigns

The degree to which the problem of accidental death can interfere with military

operations cannot be overstated Beyond even the loss of life and capital equipment— accepted risks in military service—is the strategic issue of maintaining operational tempo Twice in the past five years the military has temporarily ceased non-essential operations precisely because these costs became too high The strategic military mission of maintaining readiness was compromised This limitation has the potential to

significantly undermine the capability of the US military; the most recent stand down

came during the war in Afghanistan, illustrating the seriousness of this problem

Flight Training: Key Element of Aviation Accidents

Training accidents are a key driver of aviation accidents In the period spanning 1984 to 1994, 1523 aircraft crashed and 1600 lives were lost in training exercises Pro-rated, lives lost during flight training amount to over half of those lost through unintentional injury, hence over 25% of all fatalities There are approximately 30,000 pilots in the armed

forces today From an operations perspective, there are typically 1.5 major accidents for

every 100,000 flight hours As a result of the force size, operational tempo, and

associated risk of flight training, accidents resulting in death during flight accrue at a rate of about five aircraft per month.* The costs of these accidents are enormous Consider the losses due to flight training alone: At an estimated loss rate of 60 planes and 100 lives per year, with an aircraft cost of $30 million, life value of $2 million, and loss of skills of $5 million per person, the total cost of these accidents amounts to $2.5 Billion per annum The strategic value of op tempo is perhaps an order of magnitude greater

3 http://www.cnn.com/2002/US/02/08/air.force.safety/index.html See also http://www.cnn.com/US/9709/17/military.crash/index.html

* This statistic does not take into account type of aircraft or missions flown, but conveys the general level of risk faced Note that a Class A mishap, in which the aircraft is essentially lost or destroyed, typically results in more than one death Hence the loss rate of five aircraft per month results in 60 aircraft per year, and 100 lives per year

Trang 23

Strategies for Risk Reduction

The US military faces a clear problem that can be summarized in three key points: (1) Unintentional accidents are responsible for the majority of deaths of US

servicemen

(2) Unintentional accidents are responsible for a vast majority of peace time capital

equipment losses

(3) Unintentional accidents can raise risk levels to the point that it is preferable to reduce operational tempo and sacrifice readiness, the core metric of (latent) military efficacy

There are three basic strategies that the military can pursue to offset these losses: (1) Reduce the frequency of activities that lead to risk

(2) Make the activities that lead to risk safer

(3) Find alternatives to the activities that lead to risk

Option (1) is essentially the concession of op tempo: The reduction of lost life and equipment at the expense of reduced readiness.” Options (2) and (3) are preferable because they attempt to maintain readiness while simultaneously reducing risk The rigorous pursuit of option (2) explains the past reduction in the rate of unintentional accidents By all accounts, this strategy has been successful, accounting for a 25% reduction in the total rate of death amongst servicemen from the early eighties Previous gains have come through the improvement in the reliability of military equipment, and improved safety precautions in training doctrine However, the low-hanging fruit has been picked—there are fundamental risks that cannot be engineered away, either through improved reliability or operations management Unless the activities undertaken during mission rehearsal are significantly changed, it will be difficult to further improve safety Hence we turn to option (3): Reduced Risk Training

> This option can include the removal of certain training activities, and brief periods during which the military “stands down.” For example, in September of 1997, Secretary of Defense William Cohen ordered a 24 hours suspension of routine flights This period coincided with the patrol of no-fly zones in Bosnia and Iraq See http://www.cnn.com/US/9709/17/military.crash/index.html

Trang 24

Reduced Risk Training

According to the military credo, “We train as we fight,” pilots learn by doing The consequence is that high quality training equates to risky, high cost training—a cost appraised in billions of dollars per year Pursing option (3) requires that the military find new ways to train its personnel that have the same pedagogical value as the current high- risk training methods but with a lower risk profile Simulation is one approach, but has met with resistance

Juxtaposed to the military credo, ““We train as we fight,” the concept of substituting low

risk activities for high risk activities—including the use of simulators—is anathema

Two assumptions typically bolster this viewpoint: (1) Reality based training is “more real” than Simulation based training, and (2) More realism is better However, both assumptions are patently false Empirical evidence shows that (1) Reality based training need not be “more real” than Simulation based training, and (2) More realism need not have any pedagogical value whatsoever—it can, in fact, be counter-producti ve." The Gordian knot that links high quality training to high risk activities can be cut Activity substitution, specifically simulation, offers a means to this end However, it must be used appropriately, or the premonition of inadequate training may be fulfilled This chapter

sets forth to answer the following critical question: How can simulation be used to reduce

the risk of training, and what is its potential value?

Estimating the Value of Simulation Based Training

Simulation Based Training reduces risk exposure by substituting an essentially risk free activity for a high risk activity By reducing the amount of real flight training, the military can reduce the number of accidents that occur, and the subsequent loss of life, expertise, and capital This study next calculates the exact savings gained by using SBT as a substitute for RBT Real data is used The number of lives and dollars saved are calculated as functions of both the proportional and absolute quantity of SBT used Perfect substitutability of RBT and SBT is assumed for this first cut analysis

® See Taylor, Lintern, Kunde, Tschopp, and Talleur, “Effects of Scene Content, Field of View and Amount of Simulator Training in First Officer Training.” Appearing in Flight Simulation Technology, Capabilities, and Benefits, [7-18 May 1995, pp 7.1-7.11

Trang 25

If SBT is a perfect substitute for RBT, how should SBT be used? If SBT is a perfect substitute for RBT, then the calculation of how much SBT to use is direct If we replace one unit of RBT with one unit of SBT, then no change in readiness occurs However, the risk associated with the unit of RBT is removed, replaced by the risk associated with the unit of SBT added Because SBT leads to neither loss of life nor capital equipment loss, the risk associated with SBT is zero Thus every unit of SBT substituted in effectively removes the risk of one unit of RBT From this fundamental relationship we can build a function that shows the dollars and lives saved as a function of the amount of SBT used (in place of RBT) This analysis is carried out in 6 basic steps:

(1) Define the independent unit of measure

(2) Define a loss rate associated with the independent unit of measure (3) Calculate the single vehicle loss rate for human life

(4) Expand the single vehicle loss rate to include the cost of lost capital

(5) Combine data from the single vehicle loss models to generate summary statistics of losses across aircraft types

(6) Combine data from the single vehicle loss models and multi-vehicle loss models

to calculate risk reduction as a function of proportional and absolute use of SBT

This calculations tells the DM exactly what the impact of different SBT usage policies will be on cost through loss of life and equipment The policy maker can then make explicit tradeoffs between the use of SBT and RBT, and anticipate the results

The Independent Unit of Measure: Flight Hours

In order to determine how to reduce risk through activity substitution, it is necessary to have a, ‘common currency,’ for the exchange of units of one activity for those of another The essential criterion is that each activity unit should have associated with it a standard quantity of risk Then substituting units of one activity for units of the other will have a predictable effect on the amount of risk incurred Without having a standard unit of activity, and an associated standard quanta of risk, there can be no rational trade off of risks through changes in activity levels The selection of the unit of measure determines

Trang 26

how this exchange will be made The independent unit of measure used in this study is flight hours

Estimable Loss Rates

Risk is a Statistical phenomena Showing that a flight hour has associated with it a ‘standard’ quanta of risk is complex because it requires the estimation of a probability density: We must calculate the probability of an accident per flight hour We must calculate a mean and confidence interval for the losses per flight hour In mathematical terms, the loss rate is considered ‘estimable’ if the mean can be calculated with low or ne

bias (accuracy), and the confidence interval is small (precision) In the real world,

‘enough’ accidents have to occur for us to be able to develop the first order mean and

standard deviation statistics

Unfortunately (from an analytic perspective), the loss rate changes over time As a result,

the mean—a function of time—cannot be computed by taking the simple average of

accidents over flight hours Likewise, the standard formulae for the standard deviation for stationary means cannot be applied Fortunately, the rate of loss for most aircraft

approaches an asymptotic limit As a result, the mean at any time T can be calculated as

the cumulative mean—the total number of accidents over the cumulative flight hours The benefit of this method over a stationary mean is that it illustrates the change in

accident rates over time, and that the emerging trends can be forecast The ability to forecast future loss rates is critical, because the policy decisions that this study informs are designed around the amount of simulation that should be used now and in the future While this method will slightly overestimate the accident rate at time T (because the function is approaching an asymptote from above), the use of an unbiased moving average is precluded by the relative rarity of accidents Such a measure would be too

volatile to be useful, or to forecast

Trang 27

The Single Vehicle Loss Model We next describe how to estimate and forecast the loss rates for a single vehicle We first show through graphical inspection that risk is an estimable function of flight hours We then validate this explanation with computational results

Our objective is to forecast the total number of deaths that will be incurred by a particular aircraft type for the next five years The calculation is computed in two parts First, we forecast the number of flight hours for each year, and the rate of death for each year Second, these forecasts are used to compute the total number of forecast deaths as the vector product of the flight hours and death rates:

flight hours ¢ death rates

Forecast Deaths = -

discount factor

The discount factor is included in this model to reflect the change in real value of nominal losses over time

Flight Hours: Confirming the Risk Measure

Trang 28

There is a small non-linearity in the relationship between these variables, which is significant Note that the line depicting cumulative fatalities ‘bows’ out and away from cumulative flight hours This curvature occurs because, with each passing year, fewer accidents accrue per flight hour: The rate of accidents is undergoing a very progressive decline

Estimating the Death Rates

This trend is confirmed with

A-10 Fatality Rate and Cumulative Average Fatality Rate :

a second plot, illustrating the Ị

death rate for cach year, and the cumulative death rate In the chart at right, the bars Deaths per 100,000 Hours depict the single period death rate, equal to the number of persons killed per 100,000 eer FFX Ke KF PELL ESS Year flight hours ÍmBFATAL RATE ===FATAL AVE RATE

There is a clear estimable mean that slowly declines over time This mean is depicted by the blue line, which shows the cumulative average fatality rate The downward trend is initially exponential in character, then progressively more linear An asymptote appears to exist at approximately I-1.5 deaths per 100,000 flight hours The equations for these statistics are given below: deaths, Fatality Rate d, = ————_ Oey BBE =F 700,000 > deaths, Cum Ave Fatality Rate d* = —* Yh, /;00,000 i=]

death rate, period t = d, flight hours, period t = h,

Trang 29

The standard deviation of the cumulative average fatality rate is estimated as a generalized auto-regressive conditional heteroscedastic (GARCH) process The basic equation is 0° =a@eo,,+fe", where £ =d, —d,™” The variance at time t is calculated as a weighted average of the previous periods’ estimated variance, and the

current one-period observed variance This one period variance is ý”, the square of the

difference of the single period death rate from the cumulative average death rate This difference is interpreted as the one period deviation of the observed value from the estimated mean

The chart at right illustrates the time series d™*, ¢,, and a All three statistics are well

behaved As noted, d,”* follows a gradual exponential decline, then asymptotically

approaches a limit at 1-1.5 deaths per 100,000 flight

hours The series ¢, is

conditionally normal, and the resulting series o7 is stable, approaching a mean value of one (hence

G, approaches one also)

Trang 30

In conclusion, the estimate of d®* is numerically r yp precise (i.e., confidence is high) The & chart below illustrates confidence intervals centered around a mean death rate of | person per 100,000 flight hours The results indicate that the confidence intervals are valid (i.e., contain only non-negative values) up to a confidence level of 90% These values are sufficient for a first cut analysis Though the estimated loss of life will vary by a factor of 2, the risk levels associated with different aircraft vary by as much as a factor of 100 As a result, these values allow a definitive prioritization between aircraft The policy maker can identify the riskier aircraft with considerable confidence, a useful tool when

determining how best to leverage SBT Confidence Intervals for Mu=1, Sigma=0.6" Mu 1.00 1.00 1.00 1.00 1.00 1.00 Sigma 0.60 0.60 0.60 0.60 0.60 0.60 Conf Level 50% 70% 90% 95% 98% 99% High Est 1.40 1.62 1.99 2.18 2.40 2.85 Low Est 0.60 0.38 0.01 -0.18 -0.40 -0.85

Forecasting Flight Hours and Death Rates

The final step is to model both the future flight hours and death rates, and then to combine them as a vector product:

flight hours ¢ death rates Forecast Deaths = -

discount factor

For these estimates, the ten year period spanning 1992 to 2001 is used for historical data, and the five year forecast spans the years 2002 to 2006.® Data is fit both using the linear model y =ax+b, and also the exponential model y=e**?

” Setting o = 0.6 when p = | is equivalent to setting o equal to 60% of pt, reflecting the observed relationship between o, and d,""*

Trang 31

The maximum likelihood estimator can be solved for explicitly In the linear case, we have a= no 0Q) and b=Y —aX , where X and ¥ are respectively the

nà x” - (> xƑ

mean of the independent variable x (year), and dependent variable y (death rate, or flight

hours) For the exponential model the calculation ¡s the same, except that we replace the

observed dependent values y with the natural logarithm of these values, effectively estimating In y=ax+b The total error of both models is computed as the sum of squared errors over the estimated and observed values, and the model with the smaller total error is selected for forecasting

Future values are estimated with the selected model for the forecast years The chart at right depicts the ten year death rates and flight hours for the A-10 The shaded box

indicates forecast values

Forecast Cumulative Average Fatality Rate and Hours

Trang 32

In the case of the A-10, the trend is clearly linear, and the results are intuitive (the standard deviation is similarly forecast, but not shown) The final calculation is:

flight hours ¢ death rates discount factor Forecast Deaths = 4 => (1+ hd, Ind, Ind, hại, (I+r} (L+r} (L+ r} (L+r} FD = hyd, +

death rate, period t = d, flight hours, period t =h, risk free rate of return=r

For the ALO, the estimated flight hours, death rates, and ‘real deaths’ for periods zero through four are shown in the following chart Forecast Flight Hours, Loss Rates, and Deaths by Period for A-10 Thunderbolt Period ” T0——T 2 3 4 Loss Rate 1.405 1377 - 1349 1.322 1296 ‘Total Nominal Deaths | 1.416 1.322 1.233 1.150 1.073 6.194 Real Deaths’ _| 1.416 1.282 1.161 1.050 0.950 5.861

The estimated 5.86 deaths is the mean forecast deaths By forecasting the standard

deviation of the loss rates, it is possible to estimate a low and high forecast The method

used for this forecast is the same as for the mean loss rate and the number of flight hours: Linear and exponential models are fit, and the series is forecast using the method with the smaller sum of square errors The high (low) forecasts for the loss rates is then calculated as the mean loss rates plus (minus) the forecast standard deviation.'® The results for the A-10 are illustrated below

A discount rate of 3% was used See Office of Management and Budget Mid Session Review: http://www whitehouse z0v/omb/budget/fy2002/msr04 html

' It is possible to have a negative loss rate for the low estimate using this method In this case, we replace the negative loss rate with a loss rate of zero

Trang 33

High LossRate ` | 2.356 2325 - 2.296 2266 2.238 Low, Mean, and High Real Death Estimates for A-10 Thunderbolt _ Total Mean Real Deaths [1416 1282 L161 1.050 0.950 5861 ad

The Single Vehicle Loss Model: Summary of Results

The principal value of the single vehicle loss model is that it allows us to quantify, forecast, and compare the risk attributed to aviation accidents by aircraft type For the A- 10 Thunderbolt, the expected number of deaths over five years is 5.9 persons, with a low of 1.8 persons, and a high of 10.0 persons This range reflects the mean estimate plus or

minus one standard deviation, hence a confidence interval of 70% The values fall within

one order of magnitude

Because of the enormity of value attributed to a human life, this information is sufficient to enable effective decision making The questions of what percentage of reality based training should be replaced with simulation based training, and how much simulation should be used in total can be answered The multi-vehicle loss model, subsequently

described, is a method for leveraging this information to inform these decisions

However, we first summarize the findings of the single vehicle loss model across all

airplanes, and then extend the model to allows us to account for the loss of capital

equipment and skills in addition to the loss of life

Trang 34

Losses by Aircraft Type

The Single Vehicle Loss Model (SVLM) is used to determine the amount of risk incurred

by each aircraft, and

Mean, Low, and High Death Estimates by Aircraft

the comparative risk 5 Year Projection

across air craft 100

types The chart at % 80 a 1

right illustrates the J 60 — — | : High |

ø : + Low |

low, mean, and high 2 40 ‹ Mean!

death estimates for 20

0 et etetetetete+ee+ hf tt ttt i

each of twenty two PRSSERSTSLYRERaREe Seas

aircraft types ; Aircraft

There are three critical points to observe First, the aircraft fall loosely into three

categories: Low, medium, and high risk The low risk group consists of the T41, B2, C10, C17, TI, T39, C12, F4, C9, F1 L7, and B52 All of these aircraft are predicted to incur losses of fewer than two persons in the next five years The medium risk group consists of the T43, C141, T37, B1, T38, Al0 and F15 aircraft Each of these aircraft are

predicted to have more than two, but fewer than 10 lives lost The high risk group

contains the C5, F16, C135, and C130 aircraft, all of which are predicted to lose 10 or more lives

The second key observation is that the low, medium, and high risk rankings are consistent

with respect to the vehicles’ low, mean, and high loss rate estimates With the exception of the high estimate of the T43, and aircraft with highly similar loss rates, ranking the aircraft by their low risk estimates produces nearly the same results as ranking the aircraft

by their mean risk rankings.'' Ranking by the high risk rankings produces similar results

As a result, the analysis results are robust from the standpoint of relative risk

Trang 35

The third point is that the amount of risk exposure between aircraft can vary by an order

of magnitude At the low end, some aircraft have negligible death rates because either

there are few flight hours flown, or because the risk per flight hour is very low (or both)

These vehicles are expected to have no or few losses—one a year would be a surprise In contrast, there would be no surprise if the three riskiest airplanes resulted in losses of 50 or more persons each As a result, the return to investment on simulation will vary dramatically based on which airplanes the military chooses to target for simulation

Targeting Simulation Hours

When applying simulation to reduce risk, the economical solution is to replace reality based training hours with simulation based training hours for those aircraft that pose the

greatest risk per hour In order to have the maximum possible effect, the military ought to

pursue the ‘low-hanging fruit’ first—that is, the deaths that are cheapest to prevent The strategy of replacing flight hours with the highest loss rates is the most economical because simulator hours are approximately fixed in price, irrespective of airplane type rẻ The value of substituting a simulator hour for a reality based training hour is thus

determined by the amount of risk removed We thus prioritize those aircraft with the

highest loss rates The chart illustrates the low, medium, and high estimates together with

the total flight hours for the five year forecast period

Mean, Low, and High Death Estimates, & Flight Hours, by Airplane, Projected Five Years @ Hours a High « Low su Mean Flight Hours =a On TF An THORN MDM XS — ow w oO tTtomoeenrr-eMre ornre Ow 3 =e OF a 8 e oO - oO ca - eee ö K < uw k= = oO Aircraft

2 The price driver of a simulator hour is the type of system used, and the extent of amortization of the fixed costs Simulators vary in price from tens of thousands of dollars for low fidelity, fixed based systems to tens of millions of dollars for realistic cockpit simulators with synergistic motion platforms The price of each type of system is, however, relatively independent of the actual aircraft simulated

Trang 36

The loss rates govern the lives saved per training hour substituted, while the total forecast deaths for a given airplane places a ceiling on the maximum possible savings The chart

below illustrates the relationship between total flight hours and deaths accrued Aircraft with similar total deaths appear in the same ‘row.’ Those with similar flight hours

appears in the same column For two aircraft with the same number of total deaths, the aircraft to the left on the chart has a higher loss rate because it has fewer associated flight hours Estimated Deaths, Flight hours by Aircraft Type | 100.0 C145 C130 | Fié | C5 ị 10.0 - Fis | 0 1a BÍ Giái 1 137 | ề BS2 § 1.0 +—Ft17 | a ị c9 F4 | T39 | 0.0 0 200000 400000 600000 800000 1000000 1200000 1400000 1600000 Flight Hours Ị

Recall that the loss models’ confidence intervals precision is typically one order of magnitude, hence the logarithmic risk scale is an intuitive way to compare the risk of

Trang 37

The Value of Human Losses

As described, the SVLM describes how human losses accrue with aircraft accidents The cost associated contains two major components: The value associated with the loss of life, and the cost of replacing the skill set of the aviator The first figure is typically found as the implied expected value of a human life by the manifest risk preferences of decision makers In other words, if a decision maker is willing to spend $20,000 to reduce the risk of loss of life by 1%, then the associated value of a human life is

100 e $20,000 = $2 million Estimating the intrinsic value of a life at $2.0M, and the value of skills lost at $5.0M, the cost attributed to loss of a skilled personnel is approximately

$7.0M Totaling the loss of life across all aircraft produces low, mean, and high estimates

of 21, 158, and 360 lives for respective dollar equivalents of $147M, $1.1B, and $2.5B

respectively

The Value of Capital Losses

The second major cost is the loss of the aircraft The value of lost aircraft is taken to be

the dollar cost of the vehicle Note that this definition also has complications Older aircraft, when lost, are not replaced by equivalent aircraft Some analysts would argue that the value of such losses should be lower, because some useful proportion of the vehicle’s life has already been consumed These analysts value aircraft in a way similar to how insurance companies treat automobiles—they depreciate with time Others point out that older vehicles will be replaced with more expensive, newer vehicles Because the unit cost of vehicles has steadily increased, the dollar cost of replacement is actually higher than the purchase price of the lost airplane A more thorough approach might

consider the value derived from the aircraft through actual use Aircraft central to the

military’s planning might be assessed at a higher value than those considered ancillary The options are endless, for which reason this study begins at the beginning—the dollar cost of the vehicle Methods for more accurately assessing a vehicles ‘worth’ presents one area of future research for subsequent studies

Trang 38

The chart below illustrates the loss of aircraft (in units) Low, medium, and high

estimates are provided with the total flight hours for the five year forecast period Losses

are expressed in numbers of aircraft, not the value of the aircraft lost Estimated Aircraft Losses, Flight hours by Aircraft Type 100.0 F1B " F15 10.0 AAD T38 Tô C130 C135 = FIy 8 10 ~ < B52 G541 F4 0.1 TT4ãCS - > | | C12 | 0.0 0 2000 400000 600000 800000 1000000 1200000 1400000 1600000 | Flight Hours ị

There are large differences in risk exposure per flight hour, and both the risk per hour and total flight hours are needed to accurately gage the losses that will be incurred by any given aircraft type Plotting losses against flight hours on a logarithmic plot illustrates this point Note that the vertical axis traverses 4 orders of magnitude

Mean, Low, and High Aircraft Losses, |

Trang 39

The Value of Combined Losses

Ultimately, the loss of lives and aircraft must be combined into a single measure

The chart below illustrates the combined losses of life, skills, and capital equipment by aircraft type Three critical features emerge on inspection First, the scope of the problem

can now be deduced Over the five year forecast period, the total of mean losses is $3.3

Billion, while the high and low estimates are respectively $6.0 Billion and $1.1 Billion

Mean, Low, and High Value Losses, Flight Hours, by Airplane, Projected Five Years 1600000 2.00E+09 | 1400000 F 1.80E+09 | 120000 [ 180E+09 „— | £ L 1.406409 2 fo} S 5000000 | 1.40E+09 2 ia Hours | 3 1206+09 Tao = 900000 [ 100E+09 @ | g @ 60000 [ 800E+08 ® : ® LOW x sưng j{ 600E:0s we Ls Mean | ; RỊT 4.00E+08 > 200000 4 ; R Ề Bt 2.00E+08 04 Š HAS “LLỆ,,BLL o.ooE+oo Ị WÑ' >> = “= Ơ Q@ *# Ơ Œ >> Œ@ «w> €@ h CC 0 0 = © ư © i PSST RFOT OFF R SBE eOM Eee | Aircraft ị

Second, the cost by aircraft type varies Aircraft Value Lost: % of Total

considerably The F16 contributes fully 37.9% of - Other, 2.8% lost value The top five sources of loss, the F16, | - C141, 1.1%

| | 7 C135, C130, BI, and F15 together contribute approximately 80% of the cost In contrast, the | | |} B52 1.5% || (3521 F117 2.8%

bottom ten contribute less than 3% of the losses | | [ A10, 4.0% C5, 5.3% Finally, the hours flown—and hence the amount of F16, 37.8% F15, 8.6% simulation needed to replace flight hours—varies

widely by aircraft type Substituting 10% of the

flight hours of the F-16 equates to 140,000 B1, 10.7%

simulator hours, while the B1 yields 1,000

C135, 12.9% C130, 12.4%

Trang 40

The Multi-Vehicle Loss Model

The Single Vehicle Loss Model (SVLM) calculates the risk associated with each aircraft

It is a normative model, describing the mean and standard deviation of loss rates—our

principle measure of risk In contrast, the Multi Vehicle Loss Model (MVLM) is a prescriptive model, and describes how to apportion simulator hours across multiple

vehicles based upon the data produced by the SVLM Given a fixed number of simulator

hours to substitute for reality based training, the MVLM specifies which RBT hours to replace, and computes the savings resulting from the removal of the risk associated with the eliminated flight hours The method combines a single criterion for optimality, and

two constraints:

(1) Optimality: The hours with the highest associated risk, measured through dollar

equivalent losses, are eliminated first

(2) Constraint 1: A maximum percentage of training hours may be replaced for each

aircraft type This percentage, a, is held fixed across all aircraft types

(3) Constraint 2: A maximum total number of simulation hours, f, is specified This parameter reflects the maximum capacity to deliver simulation hours

For example, suppose that there are only two types of aircraft, A and B Suppose further that A has 100 flight hours, and B has 200 flight hours The losses associated with A are

$1.0 per hour, and with B are $0.5 per hour Suppose that we wish to allocate B = 10

flight hours We should first substitute flight hours for A, because each hour eliminated results in a cost avoidance of $1.0 If a = 5%, then we can substitute away up to 5 type A flight hours, and 10 type B flight hours In this case, we use up the maximum 5 type A flight hours, and an additional 5 type B flight hours for a total cost avoidance of $7.5 If a is increased to 10%, then we substitute away up to 10 type A flight hours, and 20 type B flight hours In this case we will exclusively use type A flight hours for a cost

avoidance of $10 because B = 10 If we increase the maximum simulation hours by

raising B to 20, we will first use 10 type A flight hours, and then 10 type B hours for a

savings of $15

Ngày đăng: 23/03/2014, 05:23

TÀI LIỆU CÙNG NGƯỜI DÙNG

TÀI LIỆU LIÊN QUAN