COMPONENT ROADMAP INSTRUCTIONAL DESIGN IN TECHNOLOGY-ENABLED LEARNING SYSTEMS USING SIMULATIONS AND GAMES IN LEARNING

COMPONENT ROADMAP: INSTRUCTIONAL DESIGN IN TECHNOLOGY-ENABLED LEARNING SYSTEMS: USING SIMULATIONS AND GAMES IN LEARNING Learning Science and Technology Roadmap R&D The Learning Federation Project Federation of American Scientists 1717 K St NW Suite 209 Washington, DC 20036 www.fas.org/learningfederation thelearningfederation@fas.org October 2003 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning The Learning Science and Technology R&D Roadmap Executive Summary incorporates a series of technology research roadmaps, or plans, developed over a three year period by the Federation of American Scientists and the Learning Federation, a partnership among industry, academia, and private foundations to stimulate research and development in learning science and technology The full series of research roadmaps is available at www.FAS.org/learningfederation We thank Dr Jan Cannon-Bowers for her major contribution in writing this roadmap And, we thank Dr Marianne Bakia, who left FAS just prior to completion of the roadmap, for her contributions to the Learning Federation Project and development of this roadmap We gratefully acknowledge the funding support of the 2003 Congressional appropriation to the Federation of American Scientists for the Digital Opportunity Investment Trust (DO IT) A major part of that funding supported the Learning Federation's Learning Sciences and Technology Research and Development Roadmap, which appears in the DO IT Report to Congress We also gratefully acknowledge the additional funding support of the organizations that sponsored this work and helped make possible the Roadmap: Microsoft Research National Science Foundation Hewlett Packard Department of Defense, DDR&E Carnegie Corporation of New York Hewlett Foundation ii Learning Science and Technology R&D Roadmap Component Roadmap: Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning The Learning Federation iii Copyright © 2003 by The Learning Federation All rights reserved Printed in the United States of America Washington, D.C Federation of American Scientists www.fas.org iv The Learning Federation: Learning Science and Technology Research & Development Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning i Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning i Learning Federation Steering Committee 1 About the Learning Federation .3 About this Roadmap Introduction .6 A Conceptual Framework for Roadmap Components Research Topics and Tasks 13 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Learning Federation Steering Committee Executive Leadership Henry Kelly, President, Federation of American Scientists Randy Hinrichs, Group Research Manager, Learning Science and Technology, Microsoft Research Andries van Dam, Vice President for Research, Brown University Steering Committee Members Ruzena Bajcsy, Director, Center for Information Technology, Research in the Interest of Society, University of California, Berkeley John D Bransford, College of Education, University of Washington Gene Broderson, Director of Education, Corporation of Public Broadcasting Edward Lazowska, Bill & Melinda Gates Chair in Computer Science, University of Washington Elliot Masie, President, MASIE Center Richard Newton, Dean of the College of Engineering and Roy W Carlson, Professor of Engineering, University of California, Berkeley Donald Norman, Co-founder, Nielsen Norman Group, Professor Emeritus, Cognitive Science and Psychology, University of California, San Diego Raj Reddy, Herbert A Simon University Professor of Computer Science and Robotics, Carnegie Mellon University The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Shankar Sastry, Chair of Electrical Engineering and Computer Sciences, University of California, Berkeley William Spencer, Chairman Emeritus, International SEMATECH Board Janos Sztipanovits, E Bronson Ingram Distinguished Professor of Engineering, Vanderbilt University Ann Wittbrodt, Research and Development Manager Education Services, Hewlett-Packard Company Project Management Kay Howell, Director, Information Technologies, Learning Federation Project Director, Federation of American Scientists The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning The Learning Federation Learning Science and Technology R&D Roadmap About the Learning Federation The Learning Federation was formed in 2001 as a partnership among industry, academia, and private foundations to stimulate research and development in learning science and technology The Learning Federation developed the Learning Science and Technology R&D Roadmap with the goal of providing a vision of where we can go with adequate investment in learning science and technology R&D and a detailed research plan to achieve that vision Our goal is to catalyze a partnership joining companies, universities, government agencies and private foundations to execute the research plan and make possible radically improved approaches to teaching and learning enabled by information technology The Learning Federation is led by a Steering Committee of national leaders in learning science and information technology to provide advice and guidance, review and endorse the research plan described in the Roadmap, and act as advocates on its behalf In addition, more than 70 leading researchers, from industry, academia, and government donated time and labor to help us develop the Roadmap through their participation in focused workshops, interviews, and preparation of technical plans The Learning Science and Technology R&D Roadmap is comprised of a series of five component roadmaps, focusing on the following topics: • • • • • Instructional Design: Using Simulations and Games in Learning Question Generation and Answering Systems User Modeling and Assessment Building Simulations and Exploration Environments Integration Tools for Building and Maintaining Advanced Learning Systems The roadmaps provide an assessment of the R&D needs, identify key research questions and technical requirements, and detail the chronology of the R&D activities over the next The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning five to ten years Each roadmap also articulates long-term goals and shorter-term benchmarks Collectively, by articulating a vision of next-generation learning systems, these roadmaps provide a comprehensive strategic view of the field, which can guide researchers, industry, and funding agencies as they enable continued innovation in educational technology The R&D roadmaps are constructed to support both basic research and highly applied efforts to build tools, design software, and develop courses using the products of this research The research plan is crafted to ensure that supported research will generate a continuous flow of carefully evaluated instructional components, instructional strategies, and tools adaptable to multiple contexts, including university and corporate learning The tools developed will enable increases in scale that will make these capabilities readily affordable to all In turn, affordability will permit routine use of new tools in schools, colleges, workplaces, and homes The reader is encouraged to read the Roadmap Executive Summary, which summarizes the component roadmaps and describes a research plan to manage the R&D described in the roadmap The Executive Summary and the component roadmaps are available at www.FAS.org/learningfederation About this Roadmap It has been argued that the United States’ position as a global economic leader depends largely on the degree to which a workforce of educated, adaptive, and motivated individuals can be created and maintained (US Chamber of Commerce, 2001) This challenge is intensified by the availability and sophistication of technology that is now commonplace in many jobs Fortunately, the same technology that has increased requirements for better skilled and prepared workers is also providing unprecedented opportunities to improve the education and training process Applied prudently and intelligently, technology holds great promise as a means to improve education and The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning training at all levels However, attempts to apply technology may or may not be successful if they are not based on the science of learning, a situation that is all too common in learning system design Moreover, poorly implemented systems will cause educators and trainers to abandon technologies that would be very effective if applied correctly This document presents a research and development plan, or “roadmap,” designed to improve the scientific understanding of how technology-enabled learning systems (TELS) should be designed, with particular emphasis on instructional design.1 Two aspects of technology-enabled learning system design are of particular interest here: the use of simulation(s) in learning and the application of gaming techniques for learning These two topics hold particular promise as viable alternatives to more traditional forms of instruction However, since neither represents an instructional strategy per se (both can really be considered “backdrops” for learning), the treatment of them in the roadmap is not consolidated in one place Rather, the vast majority of the questions represented here will have implications for the design of simulation(s) and educational games (for example, the level of fidelity required; the nature of examples provided; the use of challenges) This approach is necessary to understand the various aspects of simulation and game design in learning, but runs the risk of leading to a set of fragmented conclusions about design To avoid this, a final set of tasks at the conclusion of the roadmap is designed to ensure that guidelines and tools for the design of simulation(s) and games are explicit and integrated For the purposes of this document, we define TELS broadly to include any system that employs technology as a means to impart instruction or enhance learning The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning On-line learning Wisher & Champagne’s (2000) review found that the vast majority of studies of distance learning effectiveness are methodologically flawed The few that are sound enough to draw conclusions found positive results, but the possibility that these were due to factors other than the medium still exists According to Clark & Mayer (2003), one of the problems with this research has been the desire to demonstrate that web-based learning is superior to other learning approaches Clark concludes that this strategy is flawed because, based on literally thousands of studies (see Clark, 2001), “there is no evidence of learning benefits from any medium that cannot be explained by factors other than the medium” Therefore, Clark recommends that education researchers should seek to demonstrate that web-based learning is at least as effective as other types of instruction, and shift their attention toward applying powerful instructional design models to on-line learning Moreover, Clark (and others) suggest that there may be other benefits of on-line learning such as increased access to instructional resources and reduction in cost (due to increased speed of learning and per-student cost) These factors require further research Finally, the motivational benefits (or lack thereof) of web-based learning need further investigation Communities of Practice In general, on-line or web-based learning allows the notion of collaboration to be expanded well beyond the individual classroom or training center For example, evidence exists that fostering meaningful discussion among students via a computer network has a beneficial impact on learning (Scardamalia et al., 1989; Scardamalia & Bereiter, 1991; 1993) The power of this type of approach stems from the fact that learners are able to share information, integrate multiple sources of data, and engage in meaningful dialog Moreover, students are an excellent source of knowledge about each other, and are in a good position to provide mutual feedback (Bransford, Brown & Cocking, 1999) The overriding question then is: how can technology support the development of learning communities that provide social support for lifelong learning? This suggests a number of important issues, including how to help students learn to interact with others online; how to search for information without getting overwhelmed; and how to select the right set of collaborative tools that fit their current goals 56 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Blended Solutions—As noted, an issue that needs further work involves the design of blended learning solutions Relatively little systematic research is available on this topic Some of the issues include: when and how to provide synchronous versus asynchronous instruction; when classroom-based learning is appropriate; and how to balance the combination of the two 5.6 Practical Considerations & Context Even the most effective strategy for teaching will not be adopted if it is too costly or otherwise impossible to implement A reality of instructional research is that practical considerations will often limit or constrain the design of the learning environment Therefore, explicit attempts to understand the practical limitations imposed by a learning situation need to be understood as a part of the design process Moreover, even when an approach is deemed appropriate, there still remain a number of issues surrounding its successful deployment These include the redesign of curriculum (and perhaps associated standards) to accommodate the change; educating the community and various constituents who have a stake in learning, and training teachers to embrace the new technology and methods 57 Table 6: Practical considerations and context issues associated with TELS Milestones Research Tasks Utility Analysis Tool years years Empirical results indicating relationships among variables Empirical results for individual variables (e.g., development costs, delivery costs, etc.) Develop an automated utility analysis tool for TELS implementation Teaching Teachers Empirically test prototype teacher training approaches Data regarding tradeoffs among key variables Empirical results that document best practices for teacher training Develop effective teacher training Curriculum Redesign Develop mechanisms to enhance community involvement in TELS implementation Data to drive a utility analysis model for TELS Empirically validated automated tool for assessing potential utility in TELS design Validated teacher training approaches Automated design tools for constructing teacher training Demonstration of preliminary (domainspecific) approaches for curricular design Develop successful curricular redesign approaches Community-Centered Instruction 10 years Demonstration of preliminary strategies for involving the community Empirical results demonstrating successful curriculum redesign efforts across tasks Empirical data and lessons learned from implementation efforts Validated strategies for curricular redesign Automated tool to accomplish curricular redesign in 50% less time than manual processes Validated strategies and approaches for enhancing community involvement The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Utility Analysis tool To date, attempts to summarize or synthesize cost-benefit information to help select instructional approaches have not yielded a set of useful guidelines for designers Part of the problem is that fundamental cost-benefit and utility analysis data for various approaches have not been formalized Generalized models that account for the following variables are needed: development costs, delivery costs, time to develop, time to deliver, cost of content maintenance, complexity of content maintenance, infrastructure (existing and required), number of learners, location of learners, availability of content for reuse, availability of design templates Moreover, an understanding of how these factors interact is also important so that appropriate trade-offs in design can be made For example, a computer-based system may be more costly to develop, but much cheaper to maintain and deliver than more traditional approaches, so that the return on investment for such a strategy is beneficial Ultimately, an automated tool to conduct utility analyses is needed Teaching teachers Finally, the goal of this roadmap—to institute new ways of teaching by exploiting technology cannot be reached unless teacher training is addressed Obviously, teachers are the mechanism by which new forms of instruction will be realized Moreover, teachers have been consistently shown to mediate the effects of new curricula and techniques (Darling-Hammond, 1997) For this reason, methods to ensure continued professional development for teachers, as well as ways to involve them meaningfully in research are needed (Bransford, Brown & Cocking, 1999) Fortunately, technology can help here as well For example, the internet provides opportunities for teachers with a common interest to form web-based communities that can provide support and reduce isolation In addition, teacher preparation can be supported by technology in the same ways that student learning can To date, tools designed to model or demonstrate effective teaching, provide reflection opportunities, and allow for meaningful dialog and exchange among teachers have been demonstrated (see Bransford, Brown & Cocking, 1999) More work along these lines is needed Curriculum redesign A number of specific issues arise when considering the introduction of new technology in learning For example, to be useful, technologyenabled learning must be integrated into existing (or redesigned) curricula In this 60 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning regard, the CTGV argues that learning environments must be knowledge-centered This means that the curriculum must be designed to enable students to organize knowledge around key concepts or “big ideas” in a domain, and to learn to solve problems using this knowledge Students must also learn the processes of inquiry to construct new knowledge, which vary across disciplines Hence, efforts to infuse technology into a curriculum must follow these prescriptions Clearly, technology that is not well integrated—no matter how effective—is likely to be ignored Community-centered instruction The CTGV (2000) also advocates that learning environments must be community-centered (also see Bransford, Brown & Cocking, 1999) That is, successful learning environments attempt to build communities of people who share a commitment to learning This is important because it helps to ensure that all constituents—teachers, parents, community leaders and researchers—are involved in and supportive of successful learning In order to accomplish sustained change, these multiple communities must be taken into account This is true in the case of K-12 education, where perceptions by the local community (including parents and administrators) can affect the ability of research teams to their work In organizational settings, literature regarding transfer of training consistently concludes that the context in which training occurs and the environment into which new learning must be transferred has a profound effect on outcomes (e.g., Baldwin & Ford, 1988) The specific findings of this work is too extensive to review in detail here (e.g see Holton, Baldwin & Naquin, 2000) Suffice it to say that factors such as supervisor support, peer support, relapse prevention, opportunities to practice, etc., all affect the willingness of learners to apply newly acquired skills on the job 61 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning 5.7 Measures and metrics As noted, a separate workshop and roadmap devoted to the issues of assessment and measurement is being prepared However, it is necessary to introduce the topic here, since the ability to accurately measure learner performance (and other outcomes) is crucial in determining which instructional strategies are most effective In fact, the tables in this document include measures and metrics that will be used to determine whether the task has been accomplished The following measures/metrics comprise this list: Increased understanding/comprehension Better mental model development Increased transfer to new domains Increased transfer to the operational environment Reduced time to reach criterion Reduced cost to develop Reduced cost to maintain Reduced cost to implement Increased motivation to learn 10 Increased self-efficacy 11 Increased persistence (time on task) 12 Increased access to learning resources 13 Increased depth of processing 14 Increased metacognition 15 Increased self-assessment 16 Increased self-explanations 17 Increased reflection 18 Optimal cognitive load 19 Improved team performance 20 Enhanced adaptability/flexibility 21 Increased engagement 22 Accuracy of assessment 23 Accuracy of Diagnosis 24 Reduced time to develop 25 Expert consensus 26 Evidence of validity/reliability 27 Increased generalizability of approach 62 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning References America’s Soldier, found at: http://www.americasarmy.com/ Anderson, J R (1983) The architecture of cognition Cambridge, MA: Harvard University Press Anderson, J R., Boyle, C F., Corbett, A., & Lewis, M W (1990) Cognitive modeling and intelligent tutoring Artificial Intelligence, 42, 7-49 Anderson, J R., Corbett, A T., Koedinger, K R., & Pelletier, R (1995) Cognitive tutors: Lessons learned Journal of the Learning Sciences, 4, 167-207 Anderson, J R & Schunn, C D (2000) Implications of the ACT-R learning theory: No magic bullets In R Glaser (Ed), Advances in instructional psychology: Educational design and cognitive science, Vol (pp 1-33) Mahwah, NJ: Lawrence Erlbaum Associates Andrews, D H., & Bell, H H (2000) Simulation-based training In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 357-384) New York, NY: Macmillan Atkinson, R K., (2002) Optimizing learning from examples using animated pedagogical agents Journal of Educational Psychology, 94, 416-427 Ausubel, D P (1968) The psychology of meaningful verbal learning New York, NY: Grune & Stratton Baker, E L & O'Neil, H F., Jr (2003) Technological fluency: Needed skills for the future In H F O’Neil, Jr & R S Perez (Eds.), Technology applications in education: A learning view (pp 245-265) Mahwah, NJ: Lawrence Erlbaum Associates Baldwin, T T & Ford, J K (1988) Transfer of training: A review and directions for future research Personnel Psychology, 41, 63-105 Bandura, A (1977) Self-efficacy: Toward a unifying theory of behavioral change Psychological Review, 84, 191-215 Bandura, A (1986) Social foundations of thought and action: A social cognitive theory Englewood Cliffs, NJ: Prentice-Hall Barron, B., Vye, N., Zech, L., Schwartz, D., Bransford, J., Goldman, S., Pellegrino, J., Morris, J., & Garrison, S (1995) Creating contexts for community-based problem solving: "The Jasper Challenge Series." In C N Hedley, P Antonacci, & M Rabinowitz (Eds.), Thinking and literacy: The mind at work (pp 47-71) Hillsdale, NJ: Lawrence Erlbaum Associates Bell, B S & Kozlowski, S W J (2002) Adaptive guidance: Enhancing self-regulation, knowledge, and performance in technology-based training Personnel Psychology, 55, 267-306 63 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Bielaczyc, K & Collins, A (1999) Learning communities in classrooms: A reconceptualization of educational practice In C M Reigeluth (Ed.), Instructionaldesign theories and models: A new paradigm of instructional theory, Vol II (pp 269292) Mahwah, NJ: Lawrence Erlbaum Associates Biswas, G., Schwartz, D Bransford, J., & Teachable Agents Group at Vanderbilt (2001) Technology support for complex problem solving: From SAD environments to AI In K D Forbus & P J Feltovich (Eds.), Smart machines in education: The coming revolution in educational technology (pp 71-97) Cambridge, MA: MIT Press Blickensderfer, E., Cannon-Bowers, J A., Salas, E., & Baker, D P (2000) Analyzing knowledge requirements in team tasks In J M Schraagen, S F Chipman, & V L Shalin (Eds.), Cognitive task analysis (pp 431-447) Mahwah, NJ: Lawrence Erlbaum Associates Bloom, B S (Ed.) (1956) Taxonomy of educational objectives Handbook 1: Cognitive domain New York, NY: David McKay Bloom, B S (1984) The Sigma problem: the search for methods of group instruction as effective as one-to-one tutoring Educational Researcher, 13(6), 4-16 Bransford, J D., Brown, A L., & Cocking, R R (Eds.) (1999) How people learn: Brain, mind, experience, and school Washington, DC: National Academy Press Brown, A L & Campione, J C (1994) Guided discovery in a community of learners In K McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom instruction (pp 229-272) Cambridge, MA: MIT Press Brown, A L & Campione, J C (1996) Psychological theory and the design of innovative learning environments: On procedures, principles, and systems In L Schauble & R Glaser (Eds.), Innovations in learning: New environments for education (pp 289-325) Mahwah, NJ: Lawrence Erlbaum Associates Bruer, J.T (2003) Learning and technology: A view from cognitive science In H F O’Neil, Jr & R S Perez (Eds.), Technology applications in education: A learning view (pp 159-172) Mahwah, NJ: Lawrence Erlbaum Associates Cannon-Bowers, J A & Salas, E (Eds.) (1998) Making decisions under stress: Implications for individual and team training Washington, DC: APA Cannon-Bowers, J A & Salas, E (1998) Team performance and training in complex environments: Recent findings from applied research Current Directions in Psychological Science, 7, 83-87 Cannon-Bowers, J A., Tannenbaum, S I., Salas, E., & Volpe (1995) Defining competencies and establishing team training requirements In R A Guzzo & E Salas (Eds.), Team effectiveness and decision making in organizations (pp 333-380) San Francisco, CA: Jossey-Bass Chi, M T H (2000) Self-explaining: The dual processes of generating inference and repairing mental models In R Glaser (Ed), Advances in instructional psychology: Educational design and cognitive science, Vol (pp 161-238) Mahwah, NJ: Lawrence Erlbaum Associates 64 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Clark, R C (2000) Four architectures of instruction Performance Improvement, 39, 31-38 Clark, R C (2002) The new ISD: Applying cognitive strategies to instructional design Performance Improvement, 41, 8-15 Clark, R., and Mayer, R.E., (2003), e-Learning and the Science of Instruction San Francisco, CA Pfeiffer Clark, R & Wittrock, M C (2000) Psychological principles in training In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 51-84) New York, NY: Macmillan Cronbach, L J., & Snow, R E (1977) Aptitudes and instructional methods: A handbook for research on interactions Oxford, England: Irvington Cognition and Technology Group at Vanderbilt (CTGV) (1997) The Jasper Project: Lessons in curriculum, instruction, assessment, and professional development Mahwah, NJ: Lawrence Erlbaum Associates Cognition & Technology Group at Vanderbilt (CTGV) (2000) Adventures in anchored instruction: Lessons from beyond the ivory tower In R Glaser (Ed), Advances in instructional psychology: Educational design and cognitive science, Vol (pp 35-99) Mahwah, NJ: Lawrence Erlbaum Associates Cognition & Technology Group at Vanderbilt (CTGV) (2000) Connecting learning theory and instructional practice: Leveraging some powerful affordances of technology In H F O’Neil, Jr & R S Perez (Eds.), Technology applications in education: A learning view (pp 173-209) Mahwah, NJ: Lawrence Erlbaum Associates Corbett, A T., Anderson, J R., & O’Brien, A T (1995) Student modeling in the ACT Programming Tutor In P D Nichols, S F Chipman, & R L Brennan (Eds.), Cognitively diagnostic assessment (pp 19-41) Hillsdale, NJ: Lawrence Erlbaum Associates Covington, M V (2000) Goal theory, motivation, and school achievement: An interactive review Annual Review of Psychology, 171-198 Craig, S D., Gholson, B., & Driscoll, D M (2002) Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features and redundancy Journal of Educational Psychology, 94, 428-434 Darling-Hammond, L (1997) School reform at the crossroads: Confronting central issues of teaching Educational Policy, 11, 151-166 Dede, C & Ketelhut, D Designing for Motivation and Usability in a Museum-based Multi-User Virtual Environment http://muve.gse.harvard.edu/muvees2003/documents/DedeKetelMUVEaera03final.pdf DeGroot, A (1965) Thought and choice in chess The Hague: Mouton Dweck, C S (1986) Motivation processes affecting learning American Psychologist, 41, 1040-1048 65 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Dweck, C S., & Leggett, E L (1988) A social-cognitive approach to motivation and personality Psychological Review, 95, 256-273 Dwyer, D J., Oser, R L., Salas, E., & Fowlkes, J E (1999) Performance measurement in distributed environments: Initial results and implications for training Military Psychology, 11, 189-215 Egan, D E & Schwartz, B J (1979) Chunking in recall of symbolic drawings Memory & Cognition, 7, 149-158 ExploreScience.com http://www.explorescience.com/activities/activity_list.cfm? categoryID=10 ExploreScience.com http://www.explorescience.com/activities/Activity_page.cfm? ActivityID=27 Eccles, J & Wigfield, A (1995) In the mind of the actor: The structure of adolescents’ achievement task values and expectancy-related beliefs Personality and Social Psychology Bulletin, 21, 215-225 Fletcher, J D (2003) Evidence for learning from technology-assisted instruction In H F O’Neil, Jr & R S Perez (Eds.), Technology applications in education: A learning view (pp 79-99) Mahwah, NJ: Lawrence Erlbaum Associates Fowlkes, J., Dwyer, D J., Oser, R L., & Salas, E (1998) Event-based approach to training (EBAT) International Journal of Aviation Psychology, 8, 209-221 Fowlkes, J., Lane, N E., Salas, E., & Franz, T (1994) Improving the measurement of team performance: The TARGETs methodology Military Psychology, 6, 47-61 Gagne, R M (1985) The conditions of learning (4th ed.) New York, NY: Holt, Rinehart, & Wilson Gentner, D (1983) Structure-mapping: A theoretical framework for analogy Cognitive Science, 7, 155-170 Gist, M E & Mitchell, T R (1992) Self-efficacy: A theoretical analysis of its determinants and malleability Academy of Management Review, 17, 183-211 Gist, M E., Schwoerer, C., & Rosen, B (1989) Effects of alternative training methods on self-efficacy and performance in computer software training Journal of Applied Psychology, 74, 884-891 Gist, M E., Stevens, C K., & Bavetta, A G (1991) Effects of self-efficacy and posttraining intervention on the acquisition and maintenance of complex interpersonal skills Personnel Psychology, 44, 837-861 Gott, S P & Lesgold, A M (2000) Competence in the workplace: How cognitive performance models and situated instruction can accelerate skill acquisition In R Glaser (Ed), Advances in instructional psychology: Educational design and cognitive science, Vol (pp 239-327) Mahwah, NJ: Lawrence Erlbaum Associates Gott, S P & Morgan, S (2000) Front end analysis: From unimpressive beginnings to recent theory-based advances In S Tobias & J D Fletcher (Eds.), Training and 66 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning retraining: A handbook for business, industry, government, and the military (pp 148170) New York, NY: Macmillan Hannafin, M J., Hall, C., Land, S., & Hill, J (1994) Learning in open-ended learning environments: Assumptions, methods, and implications Educational Technology, 34, 4855 Hannafin, M J., Land, S., & Oliver, K (1999) Open learning environments: Foundation, methods, and models In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 115-140) Mahwah, NJ: Lawrence Erlbaum Associates Hannafin, R D., & Sullivan, H J (1995) Learner control in full and lean CAI programs Educational Technology Research & Development, 43, 19-30 Hannafin, R D & Sullivan, H J (1996) Preferences and learner control over amount of instruction Journal of Educational Psychology, 88, 162-173 Hays, R T., & Singer, M J (1989) Simulation fidelity in training system design New York, NY: Springer-Verlag Holton, E (2000) Making transfer happen Managing and changing learning transfer systems In E Holton, T Baldwin and S Naquin (Eds.) Advances in developing human resources (pp 23-35) Baton Rouge, LA: The Academy for Human Resource Development Honebein, P.C., Duffy, T.M., & Fishman, B.J (1993) Constructivism and the design of learning environments: Context and authentic activities for learning In T.M Duffy, J Lowyck, & D.H Jonassen (Eds.), Designing environments for constructive learning (pp 87-108) New York, NY: Springer-Verlag Johnston, J H., Cannon-Bowers, J A., & Smith-Jentsch, K A (1995) Event-based performance measurement system for shipboard command teams Proceedings of the First International Symposium on Command and Control Research and Technology (pp 274-276) Washington, DC: The Center for Advanced Command and Technology Jonassen, D H (1999) Designing constructivist learning environments In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 215-239) Mahwah, NJ: Lawrence Erlbaum Associates Jonassen, D H (2000) Revisiting activity theory as a framework for designing studentcentered learning environments In D H Jonassen & S M Land (Eds.) Theoretical foundations of learning environments (pp 89-121) Mahwah, NJ: Lawrence Erlbaum Associates Jonassen, D H (2000) Toward a design theory of problem solving Educational Technology Research & Development, 48, 63-85 Kaber, D B., Draper, J V., & Usher, J M (2002) Influence of individual differences on application design for individual and collaborative immersive virtual environments In K Stanney (Ed.), Handbook of virtual environments: Design, implementation, and applications (pp 379-402) Mahwah, NJ: Lawrence Erlbaum Associates 67 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Kanfer, R & McCombs, B L (2000) Motivation: Applying current theory to critical issues in training In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 85-108) New York, NY: Macmillan Kirschner, P A (2002) Cognitive load theory: Implications of cognitive load theory on the design of learning Learning & Instruction, 12, 1-10 Knerr, B W., Breaux, R., Goldberg, S L., & Thurman, R A (2002) National defense In K Stanney (Ed.), Handbook of virtual environments: Design, implementation, and applications (pp 857-872) Mahwah, NJ: Lawrence Erlbaum Associates Kozlowski, S W J., Gully, S M., Brown, K G., Salas, E., Smith, E M., & Nason, E R (2001) Effects of training goals and goal orientation traits on multidimensional training outcomes and performance adaptability Organizational Behavior & Human Decision Processes, 85, 1-31 Kulik, James A (2002) School mathematics and science programs benefit from instructional technology Infobrief Arlington, VA: NSF Kyllonen, P C (2000) Training assessment In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 525-549) New York, NY: Macmillan Lampert, M (1990) When The Problem Is Not the Question and the Answer Is Not the Answer: Mathematical Knowing and Teaching American Educational Research Journal, 27(1), 29-63 Leelawong, K., Davis, J., Vye, N., Biswas, G., Schwartz, D., Belynne, T., Katzlberger, T., & Bransford, J (2002) The effects of feedback in supporting learning by teaching in a teachable agent environment In P Bell, R Stevens, & T Satwicz (Eds.), Keeping Learning Complex: The Proceedings of the Fifth International Conference of the Learning Sciences (ICLS) (pp 245-252) Mahwah, NJ: Erlbaum Lehrer, R & Schauble, L (2000) Modeling in mathematics and science In R Glaser (Ed), Advances in instructional psychology: Educational design and cognitive science, Vol (pp 101-159) Mahwah, NJ: Lawrence Erlbaum Associates Lesgold, A (1988) Toward a theory of curriculum for use in designing intelligent instructional systems In H Mandl & A Lesgold (Eds.), Learning issues for intelligent tutoring systems (pp 114-137) New York, NY: Springer-Verlag Lock, E A., & Latham, G P (1990) A theory of goal setting & task performance Engelwood Cliffs, NJ: Prentice Hall Mayer, R E (1999) Designing instruction for constructivist learning In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 141-159) Mahwah, NJ: Lawrence Erlbaum Associates Mayer, R E (2001) Multimedia learning Cambridge, England: Cambridge University Press 68 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Merrill, M D (1983) Component display theory In C M Reigeluth (Ed.), Instructional-design theories and models: An overview of their current status (pp 279333) Hillsdale, NJ: Lawrence Erlbaum Associates Merrill, M D (1999) Instructional transaction theory (ITT): Instructional design based on knowledge objects In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 397-424) Mahwah, NJ: Lawrence Erlbaum Associates Merrill, M D (2002) A pebble-in-the pond: Model for instructional design Performance Improvement, 41, 39-44 Nelson, L M (1999) Collaborative problem solving In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 241-267) Mahwah, NJ: Lawrence Erlbaum Associates Orlansky, J., Dahlman, C.J., Hammon, C.P., Metzo, J., Taylor, H.L and Youngblut, C (1994) The Value of Simulation for Training (IDA P-2982) Alexandria, VA: Institute for Defense Analyses Perkins, D N & Unger, C (1999) Teaching and learning for understanding In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 91-114) Mahwah, NJ: Lawrence Erlbaum Associates Petraglia, J (1998) Reality by design: The rhetoric and technology of authenticity in education Mahwah, NJ: Lawrence Erlbaum Associates Pintrich, P R and Schunk, D H (1996) Motivation in Education: Theory, research and applications Englewood Cliffs: Prentice Hall Reigeluth, C M & Moore, J (1999) Cognitive education and the cognitive domain In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 51-68) Mahwah, NJ: Lawrence Erlbaum Associates Reiser, B., Tabak, I, Sandoval, W A., Smith, B K., Steinmuller, F., & Leone, A J (2001) BGuILE: Strategic and conceptual scaffolds for scientific inquiry in biology classrooms In S M Carver & D Klahr (Eds.), Cognition and instruction: Twenty-five years of progress (pp 263-305) Mahwah, NJ: Lawrence Erlbaum Associates Salas, E & Cannon-Bowers, J A (1997) Methods, tools, and strategies for team training In M A Quinones & A Ehrenstein (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp 249-279) Washington, D C.: American Psychological Association Salas, E & Cannon-Bowers, J A (2000) The anatomy of team training In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 312-335) New York, NY: Macmillan Savery, J R., & Duffy, T M (1995) Problem based learning: An instructional model and its constructivist framework Educational Technology, 35, 31-38 Savery, J R & Duffy, T M (1996) Problem based learning: An instructional model and its constructivist framework In B G Wilson (Ed.), Constructivist learning 69 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning environments: Case studies in instructional design (pp 135-148) Hillsdale, NJ: Educational Technology Publications Scardamalia, M & Bereiter, C (1991) Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media Journal of the Learning Sciences, 1, 37-68 Scardamalia, M & Bereiter, C (1993) Computer support for knowledge-building communities Journal of the Learning Sciences, 3, 265-283 Scardamalia, M., Bereiter, C., McLean, R S., & Swallow, J (1989) Computersupported intentional learning environments Journal of Educational Computing Research, 5, 51-68 Schank, R C., Berman, T R., & Macpherson, K A (1999) Learning by doing In C M Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp 161-181) Mahwah, NJ: Lawrence Erlbaum Associates Schraggen, J M., Chipman, S F., & Shalin V L (Eds.) (2000) Cognitive task analysis Mahwah, NJ: Lawrence Erlbaum Associates Sharp, D L M., Bransford, J D., Goldman, S R., & Risko, V (1995) Dynamic visual support for story comprehension and mental model building by young, at-risk children Educational Technology Research & Development, 43, 25-42 Shute, V J., Lajoie, S P & Gluck, K A (2000) Individualized and group approaches to training In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 171-207) New York, NY: Macmillan SimCity: http://simcity.ea.com Schmidt, R A., & Bjork, R A (1992) New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training Psychological Science, 3(4), 207-217 Soloway, E & Ehrlich, K (1984) Empirical studies of programming knowledge IEEE Transactions on Software Engineering, SE-10,5, 595-609 Stanney K (Ed.) (2002) Handbook of virtual environments: Design, implementation, and applications Mahwah, NJ: Lawrence Erlbaum Associates Sternberg, R J (1997) Styles of thinking and learning Canadian Journal of School Psychology, 13, 15-40 Sternberg, R J & Grigorenko, E L (2001) A capsule history of theory and research on styles In R J., Sternberg & L Zhang (Eds.), Perspectives on thinking, learning, and cognitive styles (pp 1-21) Mahwah, NJ: Lawrence Erlbaum Associates Sternberg, R J & Zhang, L (Eds.) (2001) Perspectives on thinking, learning, and cognitive styles Mahwah, NJ: Lawrence Erlbaum Associates 70 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Sugrue, B & Clark, R E (2000) Media selection for training In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 208-234) New York, NY: Macmillan Suthers, D., Weiner, A., Connelly, J., & Paolucci, M (1995) Belvedere: Engaging students in critical discussion of science and public policy issues In Proceedings of AIED-95 Sweller, J., van Merrienboer, J J G., & Paas, F G W C (1998) Cognitive architecture and instructional design Educational Psychology Review, 10, 251-296 Tannenbaum, S I., Beard, R L., Salas, E (1992) Team building and its influence on team effectiveness: An examination of conceptual and empirical developments In K Kelley (Ed.), Issues, theory, and research in industrial/organizational psychology (pp 117-153) Amsterdam: Elsevier Tannenbaum, S I., Mathieu, J E., Salas, E., Cannon-Bowers, J A (1991) Meeting trainees' expectations: The influence of training fulfillment on the development of commitment, self- efficacy, and motivation Journal of Applied Psychology, 76, 759-769 Tannenbaum, S I & Yukl, G (1992) Training and development in work organizations Annual Review of Psychology, 43, 399-441 US Chamber of Commerce (2001) Keeping competitive Vroom, V H (1964) Work and motivation New York, NY: Wiley Wetzel-Smith, S.K., Ellis, J.A., Reynolds, A., & Wulfeck, W.H (1995) The Interactive Multisensor Analysis Training (IMAT) System: An evaluation in the Sonar Technician Surface (STG) Class "A" School and the Fleet Aviation Specialized Operations Training Group Pacific Tactical Training Course (TTC) San Diego, CA: Navy Personnel Research and Development Center Wisher, R A & Champagne, M V (2000) Distance learning and training: An evaluation perspective In S Tobias & J D Fletcher (Eds.), Training and retraining: A handbook for business, industry, government, and the military (pp 385-409) New York, NY: Macmillan Zimmer, B.J., and Schunk, D.H (2001) Selfregulated and Academic Achievement Mahwah; Lawrence 71 ... Development Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning i Instructional Design in Technology-Enabled Learning Systems: Using. .. Foundation ii Learning Science and Technology R&D Roadmap Component Roadmap: Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning The Learning Federation... summarized in Table 18 The Learning Federation LS&T R&D Roadmap Instructional Design in Technology-Enabled Learning Systems: Using Simulations and Games in Learning Table 2: Determining and Assessing