Metric reference manual

Oracle® Enterprise Manager Metric Reference Manual 10g Release (10.1) Part No B12015-01 June 2004 Oracle Enterprise Manager Metric Reference Manual, 10g Release (10.1) Part No B12015-01 Copyright © 2004, Oracle All rights reserved The Programs (which include both the software and documentation) contain proprietary information; they are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright, patent, and other intellectual and industrial property laws Reverse engineering, disassembly, or decompilation of the Programs, except to the extent required to obtain interoperability with other independently created software or as specified by law, is prohibited The information contained in this document is subject to change without notice If you find any problems in the documentation, please report them to us in writing This document is not warranted to be error-free Except as may be expressly permitted in your license agreement for these Programs, no part of these Programs may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose If the Programs are delivered to the United States Government or anyone licensing or using the Programs on behalf of the United States Government, the following notice is applicable: U.S GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations As such, use, duplication, disclosure, modification, and adaptation of the Programs, including documentation and technical data, shall be subject to the licensing restrictions set forth in the applicable Oracle license agreement, and, to the extent applicable, the additional rights set forth in FAR 52.227-19, Commercial Computer Software Restricted Rights (June 1987) Oracle Corporation, 500 Oracle Parkway, Redwood City, CA 94065 The Programs are not intended for use in any nuclear, aviation, mass transit, medical, or other inherently dangerous applications It shall be the licensee's responsibility to take all appropriate fail-safe, backup, redundancy and other measures to ensure the safe use of such applications if the Programs are used for such purposes, and we disclaim liability for any damages caused by such use of the Programs Oracle is a registered trademark of Oracle Corporation and/or its affiliates Other names may be trademarks of their respective owners The Programs may provide links to Web sites and access to content, products, and services from third parties Oracle is not responsible for the availability of, or any content provided on, third-party Web sites You bear all risks associated with the use of such content If you choose to purchase any products or services from a third party, the relationship is directly between you and the third party Oracle is not responsible for: (a) the quality of third-party products or services; or (b) fulfilling any of the terms of the agreement with the third party, including delivery of products or services and warranty obligations related to purchased products or services Oracle is not responsible for any loss or damage of any sort that you may incur from dealing with any third party Contents Send Us Your Comments xxxix Preface xli Intended Audience xli Purpose of This Manual xli Documentation Accessibility xli Structure xlii Related Documents xlv Conventions xlvi How To Use This Manual Structure of the Metric Reference Manual Metric Information Examples of Metric Summary Tables Definitions of Columns in Metric Summary Tables Abbreviations and Acronyms Background Information on Metrics, Thresholds, and Alerts Editing Thresholds Specifying Multiple Thresholds Accessing Metrics Using the Grid Control Console Part I 1-1 1-1 1-2 1-2 1-3 1-4 1-4 1-4 1-5 Oracle Database Metrics Database Alert Log Alert Log Error Trace File Alert Log Name Archiver Hung Alert Log Error Data Block Corruption Alert Log Error Generic Alert Log Error Session Terminated Alert Log Error Alert Log Error Status Archiver Hung Alert Log Error Status Data Block Corruption Alert Log Error Status Generic Alert Log Error Status 2-1 2-1 2-1 2-2 2-3 2-3 2-4 2-5 2-5 2-6 2-6 iii Session Terminated Alert Log Error Status 2-7 Archive Area 2-7 Archive Area Used (%) 2-8 Archive Area Used (KB) 2-9 Free Archive Area (KB) 2-10 Total Archive Area (KB) 2-11 Data Guard 2-11 Data Guard Status 2-12 Data Not Applied (logs) 2-12 Data Not Applied (MB) 2-13 Data Not Received (logs) 2-14 Data Not Received (MB) 2-15 Database Files 2-16 Average File Read Time (centi-seconds) 2-16 Average File Write Time (centi-seconds) 2-17 Database Job Status 2-18 Broken Job Count 2-18 Failed Job Count 2-18 Database Limits 2-19 Current Logons Count 2-19 Current Open Cursors Count 2-20 Lock Limit Usage (%) 2-20 Process Limit Usage (%) 2-21 Session Limit Usage (%) 2-22 User Limit Usage (%) 2-23 Database Services 2-24 Service CPU Time (per user call) (microseconds) 2-24 Service Response Time (per user call) (microseconds) 2-24 Deferred Transactions 2-25 Deferred Transaction Count 2-25 Deferred Transaction Error Count 2-26 Dump Area 2-27 Dump Area Directory 2-27 Dump Area Used (%) 2-27 Dump Area Used (KB) 2-28 Free Dump Area (KB) 2-29 Total Dump Area (KB) 2-30 Efficiency 2-30 Buffer Cache Hit (%) 2-30 CPU Usage (per second) 2-32 CPU Usage (per transaction) 2-32 Cursor Cache Hit (%) 2-33 Data Dictionary Hit (%) 2-33 Database CPU Time (%) 2-34 Library Cache Hit (%) 2-35 Library Cache Miss (%) 2-36 Parallel Execution Downgraded (per second) 2-37 iv Parallel Execution Downgraded (per transaction) Parallel Execution Downgraded 25% or more (per second) Parallel Execution Downgraded 25% or more (per transaction) Parallel Execution Downgraded 50% or more (per second) Parallel Execution Downgraded 50% or more (per transaction) Parallel Execution Downgraded 75% or more (per second) Parallel Execution Downgraded 75% or more (per transaction) Parallel Execution Downgraded to Serial (per second) Parallel Execution Downgraded to Serial (per transaction) PGA Cache Hit (%) Redo Log Allocation Hit (%) Response Time (per transaction) Row Cache Miss Ratio (%) Sorts in Memory (%) Global Cache Statistics Global Cache Average Convert Time (centi-seconds) Global Cache Average CR Block Request Time (centi-seconds) Global Cache Average Current Block Request Time (centi-seconds) Global Cache Average Get Time (centi-seconds) Global Cache Blocks Corrupt Global Cache Blocks Lost Idle Events Invalid Objects Total Invalid Object Count Invalid Objects by Schema Owner's Invalid Object Count Recovery Area Recovery Area Free Space (%) Response State Status User Logon Time (msec) Session Suspended Session Suspended by Data Object Limitation Session Suspended by Quota Limitation Session Suspended by Rollback Segment Limitation Session Suspended by Tablespace Limitation SGA Pool Wastage Java Pool Free (%) Large Pool Free (%) Shared Pool Free (%) Snapshot Too Old Snapshot Too Old due to Rollback Segment Limit Snapshot Too Old due to Tablespace Limit SQL Response Time SQL Response Time (%) Suspended Session 2-38 2-38 2-39 2-39 2-40 2-41 2-41 2-42 2-42 2-43 2-43 2-44 2-45 2-45 2-47 2-47 2-47 2-48 2-49 2-49 2-50 2-51 2-53 2-53 2-53 2-54 2-54 2-54 2-55 2-55 2-55 2-56 2-56 2-56 2-57 2-57 2-57 2-57 2-57 2-58 2-58 2-59 2-59 2-60 2-60 2-60 2-61 v Suspended Session Count System Response Time Per Call System Response Time (centi-seconds) System Sessions Waiting Waiting Session Count Tablespaces Full Tablespace Space Used (%) Tablespaces Full (dictionary managed) Tablespace Space Used (%) (dictionary managed) Tablespaces With Problem Segments Segments Approaching Maximum Extents Segments Approaching Maximum Extents Count Segments Not Able to Extend Segments Not Able to Extend Count Throughput All Sessions BG Checkpoints (per second) Branch Node Splits (per second) Branch Node Splits (per transaction) Consistent Read Blocks Created (per second) Consistent Read Blocks Created (per transaction) Consistent Read Changes (per second) Consistent Read Changes (per transaction) Consistent Read Gets (per second) Consistent Read Gets (per transaction) Consistent Read Undo Records Applied (per second) Consistent Read Undo Records Applied (per transaction) Cumulative Logons (per second) Cumulative Logons (per transaction) Database Block Changes (per second) Database Block Changes (per transaction) Database Block Gets (per second) Database Block Gets (per transaction) DBWR Checkpoints (per second) Enqueue Deadlocks (per second) Enqueue Deadlocks (per transaction) Enqueue Requests (per second) Enqueue Requests (per transaction) Enqueue Timeout (per second) Enqueue Timeout (per transaction) Enqueue Waits (per second) Enqueue Waits (per transaction) Executes (per second) Executes Performed without Parses (%) Full Index Scans (per second) Full Index Scans (per transaction) Hard Parses (per second) vi 2-61 2-62 2-62 2-62 2-63 2-63 2-63 2-65 2-65 2-65 2-66 2-66 2-67 2-67 2-68 2-68 2-69 2-69 2-70 2-70 2-71 2-71 2-72 2-72 2-73 2-74 2-74 2-75 2-75 2-76 2-77 2-77 2-78 2-79 2-80 2-81 2-81 2-82 2-82 2-83 2-83 2-84 2-84 2-85 2-86 2-86 2-87 Hard Parses (per transaction) Leaf Node Splits (per second) Leaf Node Splits (per transaction) Network Bytes (per second) Number of Transactions (per second) Open Cursors (per second) Open Cursors (per transaction) Parse Failure Count (per second) Parse Failure Count (per transaction) Physical Reads (per second) Physical Reads (per transaction) Physical Reads Direct (per second) Physical Reads Direct (per transaction) Physical Reads Direct Lobs (per second) Physical Reads Direct Lobs (per transaction) Physical Writes (per second) Physical Writes (per transaction) Physical Writes Direct (per second) Physical Writes Direct (per transaction) Physical Writes Direct Lobs (per second) Physical Writes Direct Lobs (per transaction) Recursive Calls (per second) Recursive Calls (per transaction) Redo Generated (per second) Redo Generated (per transaction) Redo Writes (per second) Redo Writes (per transaction) Rows Processed (per sort) Scans on Long Tables (per second) Scans on Long Tables (per transaction) Session Logical Reads (per second) Session Logical Reads (per transaction) Soft Parse (%) Sorts to Disk (per second) Sorts to Disk (per transaction) Total Index Scans (per second) Total Index Scans (per transaction) Total Parses (per second) Total Parses (per transaction) Total Table Scan (per second) Total Table Scan (per transaction) Total Table Scans (per second) Total Table Scans (per transaction) User Calls (%) User Calls (per second) User Calls (per transaction) User Commits (per second) 2-88 2-90 2-90 2-91 2-92 2-93 2-94 2-94 2-95 2-95 2-97 2-98 2-99 2-99 2-100 2-100 2-101 2-102 2-103 2-103 2-104 2-105 2-106 2-107 2-108 2-109 2-110 2-112 2-113 2-114 2-115 2-116 2-117 2-118 2-119 2-121 2-121 2-122 2-123 2-125 2-125 2-126 2-126 2-127 2-129 2-129 2-130 vii User Commits (per transaction) User Rollback Undo Records Applied (per second) User Rollback Undo Records Applied (per transaction) User Rollbacks (per second) User Rollbacks (per transaction) User Audit Audited User Audited User Host Audited User Session Count User Block Blocking Session Count User Defined Metrics User Defined Numeric Metric User Defined String Metric Wait Bottlenecks Active Sessions Using CPU Active Sessions Waiting: I/O Active Sessions Waiting: Other Average Instance CPU (%) buffer busy waits (%) CPU Time Delta (sec) db file scattered read (%) db file sequential read (%) db file single write (%) direct path read (%) direct path read (lob) (%) direct path write (%) direct path write (lob) (%) enqueue - other (%) enqueue: DML - contention (%) enqueue: HW, Segment High Water Mark - contention (%) enqueue: ST, Space Transaction - contention (%) enqueue: TM, TX, Transaction - row lock contention (%) enqueue: TX mode 4, Transaction - allocate ITL entry (%) enqueue: UL: User-defined - contention (%) free buffer waits (%) Host CPU Utilization (%) latch free - other (%) latch: cache buffer chains (%) latch: library cache (%) latch: redo copy (%) latch: shared pool (%) library cache load lock (%) library cache lock (%) library cache pin (%) local write wait (%) log buffer space (%) viii 2-131 2-132 2-133 2-133 2-134 2-135 2-135 2-136 2-137 2-137 2-137 2-138 2-138 2-139 2-139 2-139 2-139 2-139 2-140 2-140 2-141 2-142 2-143 2-144 2-145 2-146 2-147 2-149 2-150 2-151 2-152 2-153 2-154 2-154 2-155 2-156 2-157 2-158 2-159 2-160 2-161 2-162 2-163 2-163 2-165 2-166 2-167 log file switch (archiving needed) (%) log file switch (checkpoint complete) (%) log file switch completion (%) log file sync (%) log switch/archive (%) pipe put (%) row cache lock (%) SQL*Net break/reset to client (%) SQL*Net break/reset to dblink (%) SQL*Net message to client (%) SQL*Net message to dblink (%) SQL*Net more data from client (%) SQL*Net more data from dblink (%) SQL*Net more data to client (%) SQL*Net more data to dblink (%) Wait Time (%) write complete waits (%) Wait by Session Count Session Waiting for Event Count Waits by Wait Class Average Users Waiting Count Database Time Spent Waiting (%) 2-168 2-169 2-169 2-170 2-171 2-172 2-173 2-173 2-175 2-176 2-176 2-177 2-178 2-179 2-180 2-180 2-181 2-182 2-182 2-183 2-183 2-184 Listener General Status Alias Security SID List SNMP Status Start Date TNS Address Trace Level Version Load Connections Established (per min) Connections Refused (per min) Response Response Time (msec) Status Part II 3-1 3-1 3-1 3-2 3-2 3-3 3-3 3-3 3-4 3-4 3-4 3-5 3-5 3-5 3-6 Host Metrics Host Aggregate Resource Usage Statistics (By Project) 4-1 Cumulative CPU Wait Time(Seconds) 4-1 Cumulative Data Page Fault Sleep Time(Seconds) 4-1 ix Cumulative Major Page Faults 4-2 Cumulative Minor Page Faults 4-2 Cumulative Number Character IO (bytes) Read and Written 4-3 Cumulative Number Of Blocks Read 4-3 Cumulative Number Of Blocks Written 4-3 Cumulative Number of Involuntary Context Switches 4-4 Cumulative Number Of Messages Received 4-4 Cumulative Number Of Messages Sent 4-4 Cumulative Number of Signals Received 4-5 Cumulative Number of System Calls Made 4-5 Cumulative Number of Voluntary Context Switches 4-6 Cumulative Project Lock-Wait Sleep Time(Seconds) 4-6 Cumulative Project Other Sleep Time(Seconds) 4-6 Cumulative Stop Time(Seconds) 4-7 Cumulative Swap Operations 4-7 Cumulative System Mode Time (seconds) 4-7 Cumulative System Page Fault Sleep Time(Seconds) 4-8 Cumulative System Trap Time (seconds) 4-8 Cumulative Text Page Fault Sleep Time(Seconds) 4-9 Cumulative User Mode Time (seconds) 4-9 Number of Processes Owned by Project 4-9 Project CPU Time (%) 4-10 Project Process Memory Size (%) 4-10 Project's Total Process Heap Size(KiloBytes) 4-10 Project's Total Process Resident Set Size(KiloBytes) 4-11 Project's Total Process Virtual Memory Size(KiloBytes) 4-11 Status 4-12 Total Number of Threads in Project's Processes 4-12 Aggregate Resource Usage Statistics(By User) 4-12 Cumulative CPU Wait Time(Seconds) 4-12 Cumulative Data Page Fault Sleep Time(Seconds) 4-13 Cumulative Major Page Faults 4-13 Cumulative Minor Page Faults 4-13 Cumulative Number Character IO (bytes) Read and Written 4-14 Cumulative Number Of Blocks Read 4-14 Cumulative Number Of Blocks Written 4-14 Cumulative Number of Involuntary Context Switches 4-15 Cumulative Number Of Messages Received 4-15 Cumulative Number Of Messages Sent 4-16 Cumulative Number of Signals Received 4-16 Cumulative Number of System Calls Made 4-16 Cumulative Number of Voluntary Context Switches 4-17 Cumulative Stop Time(Seconds) 4-17 Cumulative Swap Operations 4-17 Cumulative System Mode Time (seconds) 4-18 Cumulative System Page Fault Sleep Time(Seconds) 4-18 Cumulative System Trap Time (seconds) 4-19 x HTTP Response Intermediate Host IP This metric represents the IP address of all the nodes traversed in order to reach a destination Data Source The UNIX traceroute command User Action Specific to your site Intermediate Host Name This metric represents the name of all nodes traversed to reach its destination Data Source The UNIX traceroute command User Action Specific to your site Packet Number This metric represents the number that identifies a packet in a traceroute Data Source The UNIX traceroute command User Action Specific to your site Round Trip Time This metric represents the time taken to traverse from the originating node to the destionation node and back Data Source The UNIX traceroute command User Action Specific to your site Time To Live This metric represents the maximum number of hops to traverse before declaring an unsuccessful attempt to reach the destination node Data Source The UNIX traceroute command 43-4 Oracle Enterprise Manager Metric Reference Manual HTTP Response User Action Specific to your site Beacon 43-5 HTTP Response 43-6 Oracle Enterprise Manager Metric Reference Manual 44 OMS and Repository The OMS and Repository target exposes metrics that are useful for monitoring the Oracle Enterprise Manager Management Service (OMS) and Management Repository Notification Status This is a Management Agent metric intended to send out of band notifications when the Notification system is determined to be in a critical state DBMS Job Bad Schedule This metric flags a DBMS job whose schedule is invalid A schedule is marked 'Invalid' if it is scheduled for more than one hour in the past, or more than one year in the future An invalid schedule means that the job is in serious trouble Data Source The user_jobs.next_time table in the Management Repository User Action If the job schedule is invalid, the DBMS job should be restarted To this: Copy down the DBMS Job Name that is down from the row in the table This DBMS Job Name is 'yourDBMSjobname' in the following example Log onto the database as the repository owner Issue the following SQL statement: select dbms_jobname from mgmt_performance_names where display_ name='yourDBMSjobname'; If the dbms_jobname is 'myjob', then issue the following SQL statement: select job from all_jobs where what='myjob'; Copy down the jobid Force the job into the broken state so that it can be restarted by specifying the following DBMS job command and parameters: dbms_job.broken(jobid,true) Verify that the job has been marked as broken by using this SQL statement: select what, broken from all_jobs where broken='Y'; You should see the job in the results OMS and Repository 44-1 Notification Status Once you've verified that the DBMS job is marked broken, restart the job with the following DBMS job command and parameters: dbms_job.run(jobid) DBMS Job Processing Time, % of Last Hour The percentage of the past hour the job has been running Data Source The mgmt_system_performance_log table in the Management Repository User Action If the value of this metric is greater than 50%, then there may be a problem with the job Check the System Errors page for errors reported by the job Check the Alerts log for any alerts related to the job DBMS Job UpDown The down condition equates to the dbms_job "broken" state The Up Arrow means not broken Data Source The broken column is from the all_users table in the Management Repository User Action Determine the reason for the dbms job failure Once the reason for the failure has been determined and corrected, the job can be restarted through the dbms_job.run command To determine the reason the dbms job failed, take the following steps (replacing myjob with the displayed name of the down job): Copy down the DBMS Job Name that is down from the row in the table This DBMS Job Name is 'yourDBMSjobname' in the following example Log onto the database as the repository owner Issue the following SQL statement: select dbms_jobname from mgmt_performance_names where display_ name='yourDBMSjobname'; If the dbms_jobname is 'myjob', then issue the following SQL statement: select job from all_jobs where what='myjob'; Using the job id returned, look for ORA-12012 messages for this jobid in the alerts log and trace files and try to determine and correct the problem The job can be manually restarted through the following database command: execute dbms_job.run (jobid); Files Pending Load The number of files waiting for the loader to process, sampled every 10 minutes 44-2 Oracle Enterprise Manager Metric Reference Manual Notification Status Data Source This metric is obtained using the following query of the mgmt_oms_parameters table in the Management Repository SELECT value FROM mgmt_oms_parameters where name='loaderFileCount' User Action If the Files Pending Load number is increasing steadily over a period of time, you may consider one of these options: ■ ■ Increasing the number of background threads Adding another Management Service and pointing some of the Management Agents to the new Management Service Job Dispatcher Job Step Average Backlog The number of job steps that were ready to be scheduled but could not be because all the dispatchers were busy When this number grows steadily, it means the job scheduler is not able to keep up with the workload Data Source The value of the mgmt_oms_parameters table in the Management Repository where the host_url is the host_url of the Management Service and the parameter_name column is jobStepCount User Action This value is updated by the job dispatcher before its periodic wait If the graph of this number increases steadily over time, the user should take one of the following actions: ■ ■ Increase the em.jobs.shortPoolSize, em.jobs.longPoolSize, and em.jobs.systemPoolSize properties in the web.xml file The web.xml file specifies the number of threads allocated to process different types of job steps The short pool size should be larger than the long pool size Property Defaule value Recommended value em.jobs.shortPoolSize 10 10 - 50 Steps taking less than 15 minutes em.jobs.longPoolSize 8 - 30 Steps taking more than 15 minutes em.jobs.systemPoolSize 8 - 20 Internal jobs (e.g agent ping) Description Add another Management Service on a different host Check the job step contents to see if they can be made more efficient OMS and Repository 44-3 Notification Status Job Dispatcher Processing Time, % of Last Hour The job dispatcher is responsible for scheduling jobs as required It starts up periodically and checks if jobs need to be run If job dispatcher is running more than the threshold levels, then it is having problems handling the job load Data Source This is the sum of the amount of time the job has run over the last hour from the mgmt_system_performance_log table in the Management Repository divided by one hour, multiplied by 100 to arrive at the percent User Action Specific to your site Last Error Timestamp of the latest error for the job Data Source The mgmt_system_error_log table in the Management Repository User Action Specific to your site Last Load Error Timestamp from the last error If the loader is not reading file correctly, the system may not be receiving data correctly from the Management Agent Data Source The mgmt_system_error_log table in the Management Repository User Action Specific to your site Loader Directory The directory from which the loader is getting files Data Source This metric is obtained using the following query of the mgmt_oms_parameters table in the Management Repository SELECT value FROM mgmt_oms_parameters where name=’loaderDirectory’ User Action If the loader directory is out of space, you may want to look for the error files to investigate the problem 44-4 Oracle Enterprise Manager Metric Reference Manual Notification Status Loader Name The unique name of the loader, consisting of the Management Service name separated by a comma from the loader name on that Management Service Data Source The mgmt_system_performance_log table in the Management Repository User Action None This is the key field for the metric Loader Throughput (rows per hour) This is the number of lines of XML text processed by the loader thread over the past hour Data Source The mgmt_system_performance_log table in the Management Repository User Action If this number continues to rise over time, then the user may want to consider adding another Management Service or increasing the number of loader threads for this Management Service To increase the number of loader threads, add or change the em.loader.threadPoolSize entry in the emoms.properties file The default number of threads is Values between and 10 are common Loader Throughput (rows per second) This is the number of lines of XML text processed by the loader thread per second averaged over the past hour Data Source The mgmt_system_performance_log table in the Management Repository User Action This metric is informational only Management Service Status Shows whether the Management Service is up or down Data Source The mgmt_oms_parameters and mgmt_failover_table tables in the Management Repository User Action If the Management Service is down, start it OMS and Repository 44-5 Notification Status Message This metric lists targets for which the Management Agent has not uploaded data in the past two hours (excluding Management Agent, Beacon and Repository targets) The alert is generated each time the Message content changes The Message content changes each time the list of targets not uploading data changes Data Source The mgmt_current_availability table in the Management Repository User Action Perform the following steps: Determine the Management Agent for the target having problems Verify that the target collection schedule is under hour interval Check the agent logs for errors uploading data Check the Management System Errors page for Loader errors processing information from the Management Agent concerned Next Scheduled Runtime The next scheduled runtime for the job Data Source The user_jobs.next_date table in the Management Repository User Action Specific to your site Notification Delivery Time The time it took to deliver a notification, averaged over the past hour Data Source The mgmt_system_performance_log table in the Management Repository User Action If the average delivery time is steadily increasing, verify that the notification methods specified are valid Remove any unnecessary or out of date notification rules and schedules Notification Processing Time, % of Last Hour The percentage of the past hour that Notification delivery has been running Data Source The mgmt_system_performance_log table in the Management Repository 44-6 Oracle Enterprise Manager Metric Reference Manual Notification Status User Action If the average delivery time is steadily increasing, verify that the notification methods specified are valid Remove any unnecessary or out of date notification rules and schedules Notification UpDown Displays whether the notification DBMS job (which processes severities to determine if notifications are required) is up or down Data Source The user_jobs table in the Management Repository User Action Determine the reason for the DBMS job failure Once the reason for the failure has been determined and corrected, the job can be restarted through the dbms_job.run command To determine why the DBMS job failed, take the following steps: Log onto the database as the Management Repository owner Issue the following SQL statement: select job from all_jobs where what like '%CHECK_FOR_SEVERITIES%'; Using the job id returned, look for ORA-12012 messages for this jobid in the alerts log and trace files and try to determine and correct the problem Issue the following DBMS job command and parameters: execute dbms_job.run (jobid); Notifications Processed The total number of notifications delivered by the Management Service over the previous 10 minutes Data Source The mgmt_system_performance_log table in the Management Repository User Action If the number of notifications processed is continually increasing over several days, then you may want to consider adding another Management Service Notifications Waiting When this metric becomes critical, an out of band notification will be sent to the address specified during the installation Data Source This is the sum of the amount of time the job has run over the last hour from the mgmt_system_performance_log table in the Management Repository divided by one hour, multiplied by 100 to arrive at the percent OMS and Repository 44-7 Notification Status User Action Perform the following user actions: Check the Errors page for errors logged by the Notification Delivery dbms job Check the number of notification rules defined and verify that they are all necessary, removing those that are not Verify that the addresses being used for the notifications are correct Number of Duplicate Targets The count of duplicate targets in the Management Repository Data Source The mgmt_duplicate_targets table in the Management Repository User Action Go to the Duplicate Targets page by clicking the Duplicate targets link on the Management System Overview page The Duplicate targets link only appears on the Management System Overview page if there are problems involving duplicate targets Resolve the conflict by removing the duplicate target from the conflicting Management Agent Number of Groups The number of groups defined for Enterprise Manager Data Source The mgmt_targets table in the Management Repository User Action If you have a problem viewing the All Targets page, you may want to check the number of roles and groups Number of Roles The number of roles defined for Enterprise Manager Data Source The mgmt_roles table in the Management Repository User Action If you have a problem viewing the All Targets page, you may want to check the number of roles and groups Number of Targets The number of targets defined for Enterprise Manager Data Source The mgmt_targets table in the Management Repository 44-8 Oracle Enterprise Manager Metric Reference Manual Notification Status User Action Specific to your site Number of Users The number of users defined for Enterprise Manager Data Source The sys.dba_role_privs table in the Management Repository User Action Specific to your site Oldest Loader File This metric shows how long the loader file has been waiting to be processed by the loader This is an indicator of the delay from when the Management Agent sends out information to when the user receives the information Data Source This metric is obtained using the following query of the mgmt_oms_parameters table in the Management Repository SELECT value FROM mgmt_oms_parameters where name=’loaderOldestFile’ User Action If the oldest loader file is extremely old, you have a loader problem You may want to add another Management Service and point some of the Management Agents to the new Management Service Repository Tablespace Used This is the total number of MB that the Management Repository tablespaces are currently using Data Source The dba_data_files table in the Management Repository User Action This metric is informational only Session Count A count of the number of sessions between the Management Service and Management Repository database Data Source The v$session system view OMS and Repository 44-9 Notification Status User Action This metric is informational only Status since Timestamp of when the Management Service was marked up or down Data Source The mgmt_oms_parameters table in the Management Repository User Action Specific to your site Steps Per Second The number of job steps processed per second by the job dispatcher, averaged over the past hour and sampled every 10 minutes Data Source The mgmt_system_performance_log table in the Management Repository User Action Specific to your site Targets not providing data This metric provides a count of the targets that are not uploading data Data Source The mgmt_targets, mgmt_current_availability tables in the Management Repository User Action This metric is informational only Throughput Per Second The number of notifications delivered per second, averaged over the past hour Data Source The mgmt_system_performance_log table in the Management Repository User Action This metric is informational only Total Loader Runtime in the Last Hour This is the amount of time in milliseconds that the loader thread has been running in the past hour 44-10 Oracle Enterprise Manager Metric Reference Manual Response Data Source The mgmt_system_performance_log table in the Management Repository User Action If this number is steadily increasing along with the Loader Throughput (rows per hour) metric, then perform the actions described in the User Action section of the help topic for the Loader Throughput (rows per hour) metric If this number increases but the loader throughput does not, check for resource constraints, such as high CPU utilization by some process, deadlocks in the Management Repository database, or processor memory problems Total Repository Tablespace The total MB allocated to the Management Repository tablespaces This will always be greater than or equal to the space used Data Source The dba_free_space table in the Management Repository User Action This metric is informational only Response This page indicates whether Enterprise Manager is up or down It contains historical information for periods in which it was down Status This metric indicates whether the Management Service is up or down If you cannot access the Management Repository, you will get an out of band error Metric Summary The following table shows how often the metric's value is collected and compared against the default thresholds The 'Consecutive Number of Occurrences Preceding Notification' column indicates the consecutive number of times the comparison against thresholds should hold TRUE before an alert is generated Table 44–1 Metric Summary Table Target Version Evaluation and Collection Frequency Upload Frequency All Versions Every Minutes Not Uploaded Operator Default Warning Threshold Default Critical Threshold Consecutive Number of Occurrences Preceding Notification = Not Defined Alert Text %Message% Data Source sysman/admin/scripts/emrepresp.pl User Action This metric checks for the following: OMS and Repository 44-11 Response ■ Is the Management Repository database up and accessible? If the Management Repository database is down, start it ■ Is at least one Management Service running? If a Management Service is not running, start one 44-12 Oracle Enterprise Manager Metric Reference Manual [...]... Portal Homepage Metric Homepage Download (msec) Status Portal Metadata Repository Version Metric Portal Metadata Repository Version Response Metric Status Syndication Server Status Metric Syndication Server Status Top Level Monitoring Status Metric Top Level... 8-11 xvi OHS General Metrics Start Time (ms since Epoch) OHS Module Metrics Active Requests for a Module Request Handling Throughput, for a Module (requests per second) Request Handling Time, for a Module (seconds) OHS Process Metrics Heap Usage (MB) OHS Server Metrics Active... TTY Raw Input (chars/s) User Defined Metrics User Defined Numeric Metric User Defined String Metric Users Number of Logons Part III 5 4-87 4-87 4-88 4-88 4-88 4-89 4-89 4-89 4-90 4-90 4-91 4-91 4-92 4-92 4-92 4-92 Oracle Application Server Metrics Application Server Application Response ... Percentage of Unsuccessful Logins Most Failed Login Users Metrics Frequency Subscriber ID Subscriber Name SSO Database Instance Metric Instance Name Start Time Version SSO Status Metric Status 16 15-1 15-2 15-2 15-2 15-3... OHS Virtual Host Metrics Active Requests for a Virtual Host Request Processing Time for a Virtual Host (seconds) Request Throughput for a Virtual Host (requests per second) Response Data Processed for a Virtual Host (KB per response) Response Data Throughput for a Virtual Host (KB per second) Virtual Host Type Process Metrics ... Time (seconds) Application - Total Servlet, JSP and EJB Time (seconds) OC4J All Instances Metrics OC4J Instances - Active Sessions OC4J Instances - Request Processing Time (seconds) OC4J Instances - Requests Per Second OC4J Application Metrics Application - Active EJB Methods Application - Active Requests ... Request Processing Time (seconds) Application - Requests Per Second OC4J EJB Metrics EJB - Active Methods EJB - Method Execution Rate (per second) EJB - Method Execution Time (seconds) Type of EJB OC4J EJB Module Metrics EJB Module - Active Methods EJB Module - Method Execution Rate... Instance - Start Time (ms since Epoch) OC4J JSP Metrics Is multi-threaded? JSP - Active Instances (STM only) JSP - Active Requests JSP - Available Instances (STM only) JSP - Client Processing Time (seconds) JSP - Requests Per Second OC4J Servlet Metrics Servlet - Active Requests ... 8-7 Total HTTP responses 8-8 modplsql_RequestGroupingSQLErrorList Metrics 8-8 errorCount.count 8-8 lastErrorDate.value 8-8 lastErrorRequest.value 8-9 lastErrorText.value 8-9 Name 8-9 modplsql_RequestLast10SQLErrorList Metrics 8-10 errorDate.value 8-10 errorRequest.value ... Metric Syndication Server Status Top Level Monitoring Status Metric Top Level Monitoring Status Ultra Search Status Metric Ultra Search Status Web Portlet Metrics Cache Hits Count of HTTP 200 Response codes Count of HTTP 400 Response codes Count of HTTP 500 Response ... This Manual The Oracle Enterprise Manager Metric Reference Manual (hereafter referred to as the Metric Reference Manual) lists all the target metrics that Enterprise Manager monitors This manual. .. How To Use This Manual Structure of the Metric Reference Manual Metric Information Examples of Metric Summary Tables Definitions of Columns in Metric Summary... Summary table 1-2 Oracle Enterprise Manager Metric Reference Manual Structure of the Metric Reference Manual Table 1–1 Definitions of Columns in the Metric Summary Table Column Header Column Definition