Information systems slide IT security metrics

IT Security Metrics A Practical Approach to Measuring Information Security: Measuring Security at the System Level Introduction IT Security Metrics Training Audience: Federal IT security personnel with GISRA reporting responsibilities Goal: To train Federal IT security personnel how to develop metrics that they can use immediately to assist with GISRA reporting Duration: hours Objectives After completing this workshop, you will be able to: • Identify why metrics are important for IT security • Identify the relationship among GISRA, NIST SP 800-26, and IT Security Metrics • Describe IT Security Metrics • Describe metrics development process • Apply metrics development process by completing a Metrics Form for one of the OMB GISRA reporting requirements for FY02 • Identify metrics-related Roles and Responsibilities • Describe how to implement a Metrics Program Metrics Development In this section, you will: • Learn the definition and characteristics of IT Security Metrics • Identify the difference between Performance Goals and IT Security Metrics • Learn the seven-step IT Security Metrics Development Process • Discover the types of information and insights that can be gained from IT Security Metrics • Complete three examples of IT Security Metrics What are IT Security Metrics? IT Security Metrics are tools that facilitate decision making and accountability through collection, analysis, and reporting of relevant performance data • Based on IT security performance goals and objectives • Quantifiable • Obtainable/feasible to measure • Repeatable • Provide relevant performance trends over time • Useful in tracking performance and directing resources Why Measure IT Security? Categories Requirements • Government Information Security Reform Act (GISRA) Regulatory Benefits • Satisfy regulatory requirements • Clinger-Cohen Act • Government Paperwork Reduction Act (GPRA) Financial Organizational • Measure successes and failures of past and current security investments • Justify future investments • Improve accountability to stakeholders • Ensure appropriate level of mission support • Determine IT security program effectiveness • Improve customer confidence • Enable investment targeting to identified areas in need • Ensure best value from security • Build confidence in leadership • Demonstrate improvement to stakeholders • Play key role in initiating improvement actions based on performance trends • Enable relevant, realistic, appropriate security procedure modification IT Security Metrics should support IT security goals and objectives IT Security Performance Goals identify desired results of system security program implementation IT Security Performance Objectives enable accomplishment of goals by: • Identifying strategic practices, defined by security policies, procedures, and controls • Directing consistent implementation of policies and procedures across the organization IT Security Metrics monitor accomplishment of goals and objectives by: • Quantifying the level of implementation of security control objectives and techniques for a system and the effectiveness and efficiency of the controls within the organization • Using analysis of collected IT Security Metrics to determine adequacy of security activities and make appropriate business decisions Exercise: Performance Goal or IT Security Metric? Statement Performance Goal IT Security Metric Program Officials understand the risk to systems under their control and determine the acceptable level of risk Percentage of system security plans that are updated annually Duties are separated to ensure least privilege and individuals accountability Percentage of systems with automated virus updating Data integrity and validation controls are used to provide assurance that the information has not been altered and the system functions as intended 10 Metrics Development Criteria: What is a Good Metric?  Based on IT security performance goals and objectives: NIST SP 800-26 Critical Elements and Subordinate Questions are used to derive performance goals and objectives  Quantifiable: Metrics should yield quantitative rather than qualitative information to increase the objectivity and validity of data  Obtainable/Feasible to measure: Metrics data should be available or easily collected through interviewing or by accessing data repositories If a metric requires significant modification of agency processes or implementing a new tool, data collection may not be feasible at this time  Repeatable: Measurements should be able to be repeated in a standard way at predetermined intervals to identify trends or identify if positive changes have occurred as a result of corrective actions  Provide relevant performance trends over time: Repeated measurements reveal change in a timely manner  Useful in tracking performance and directing resources: Metrics should be useful to stakeholders and should yield information that is important in financial decision making 24 Breakout Session 25 Breakout Session Goal: To complete a Metric Form for one of the metrics that is required for GISRA reporting for FY 2002 This includes identifying the NIST SP 800-26 Critical Element and Subordinate Question that map to the specific GISRA question from OMB guidance Duration: 30 minutes Method: • Read the metric your Breakout Group is assigned • Select the NIST SP 800-26 Critical Element that includes your metric • Select the Subordinate Question within the Critical Element that maps to your metric Remember, a single metric can use more than one Subordinate Question • Complete the Metric Form’s sections, giving particular attention to what implementation evidence may exist that corresponds to your Subordinate Question Follow up: Each Group will have five minutes to brief their Form to the other groups This brief should include: • The metric your Breakout Group was assigned • The Critical Element and Subordinate Question that maps to your metric • The completed Metric Form, including implementation evidence and indicators A list of possible sources of the data you need to uncover for your metric 26 Critical Element: Subordinate Question: Metric Purpose Implementation Evidence Frequency Formula Data Source Indicators 27 Metrics Program Implementation 28 In this section, you will: • Receive an introduction to the IT Security Metrics-related roles and responsibilities • Learn the steps involved in IT Security Metrics program implementation by learning the process and following an example through the process 29 Multiple success factors can influence quality and sophistication of IT Security Metrics (slide of 2) Ensure that IT Security Metrics Program is manageable: • Use no more than 10-20 metrics at a time, based on current priorities • Phase old metrics out and phase new metrics in when performance targets are reached or when requirements change Ensure acceptable quality of data: • Data collection methods and data repositories should be standardized • Events must be reported in a standard manner throughout the organization and the results of such reports need to be stored in the data repository 30 Multiple success factors can influence quality and sophistication of IT Security Metrics (slide of 2) Obtain organizational acceptance: • Metrics need to be validated with organization’s stakeholders within headquarters and in the field • Metrics should be vetted through appropriate approval channels Ensure that metrics are useful and relevant: • Useful data should be collected • Not all data are useful 31 Metrics-related roles and responsibilities are dispersed throughout an organization Responsibility for Organizational Acceptance of Metrics Program Responsibility for Metrics Data Collection and Data Accuracy 32 Each organization will implement a metrics program specific to its needs • Tailor to organization and business processes • Identify IT Security Metrics-related stakeholder roles and responsibilities • Lay out required infrastructure changes, such as creation of web-based data collection tools and of new data repositories • Identify required modifications of the current data sources • Define data reporting formats 33 Output from standard security activities can be used to quantify IT security performance • Incident Handling • Testing • Network Management • Audit Logs • Network and System Billing • Configuration Management • Contingency Planning • Training • Certification and Accreditation IT Security Metrics data collection must be as transparent and non-intrusive as possible 34 IT Security Metrics Program Implementation Process • Identify stakeholders • Determine goals / objectives • Review existing metrics • Develop new metrics • Identify data collection methods and tools • Collect metrics • Analyze collected data • Conduct gap analysis - Identify gaps between actual and desired performance • Identify reasons for undesired results • Identify areas requiring improvement • Track progress • Report as required • Determine range of corrective actions • Select most appropriate corrective actions • Prioritize corrective actions based on overall risk mitigation goals • Develop cost model - Project cost for each corrective action • Perform sensitivity analysis • Develop business case • Prepare budget submission • Management • Operational • Technical • Budget allocated • Available resources prioritized • Resources assigned 35 Process Implementation Example Lack of IT security refresher training may be causing weak passwords, identified by a password cracker that is run regularly Employees should be required to take annual IT security refresher training as part of their annual review process Since annual refresher training has ceased, the number of weak passwords has increased by 50% Since the training was re-instituted, the percentage of weak passwords has decreased by 40% Annual refresher training, an operational control, is instituted A budget submission detailing metrics findings related to annual IT security refresher training was submitted, and funding received Only 5% of employees receive annual IT security refresher training 36 Summary • Discussed why Metrics are important for IT security • Obtained understanding of the relationship between GISRA, NIST SP 800-26, and IT Security Metrics • Described IT Security Metrics • Described the Metrics Development Process • Created metrics to be implemented at a system level through applying metrics development process • Discussed metrics-related Roles and Responsibilities • Described how to implement a Metrics Program 37 Next Steps • You can immediately use what you have learned today to propose some metrics within your agency • Notes of the workshop will be published in two weeks • You can use the three metrics presented during the workshop and those that we developed together for your GISRA submission • Metrics Guidance first draft will be published by September 30, 2002 • Please contact Marianne Swanson if you have any questions at marianne swanson@nist.gov, 301-975-3293 38 ... individuals accountability Percentage of systems with automated virus updating Data integrity and validation controls are used to provide assurance that the information has not been altered and the... Directives 63 • Government Information Security Reform Act (GISRA) • OMB Circular A-130, Appendix III • Critical Elements within NIST Special Publication 800-26 • Federal Information Security Compliance... administering, maintaining, or using the systems System Upgrades Security patches that have been removed during the operating system upgrades New or upgraded systems that are not configured with