Performance Metrics

1 Performance Metrics The Common Metrics selected /used during the performance testing is as below • Response time • Turnaround time = the time between the submission of a batch job and the completion of its output. • Stretch Factor: The ratio of the response time with single user to that of concurrent users. • Throughput: Rate (requests per unit of time) Examples: • Jobs per second • Requests per second • Millions of Instructions Per Second (MIPS) • Millions of Floating Point Operations Per Second (MFLOPS) • Packets Per Second (PPS) • Bits per second (bps) • Transactions Per Second (TPS) • Capacity: Nominal Capacity: Maximum achievable throughput under ideal workload conditions. E.g., bandwidth in bits per second. The response time at maximum throughput is too high. • Usable capacity: Maximum throughput achievable without exceeding a pre-specified response-time limit • Efficiency: Ratio usable capacity to nominal capacity. Or, the ratio of the performance of an n- processor system to that of a one-processor system is its efficiency. • Utilization: The fraction of time the resource is busy servicing requests. • Average Fraction used for memory. As tests are executed, metrics such as response times for transactions, HTTP requests per second, throughput etc., should be collected. It is also important to monitor and collect the statistics such as CPU utilization, memory, disk space and network usage on individual web, application and database servers and make sure those numbers recede as load decreases. Cognizant has built custom monitoring tools to collect the statistics. Third party monitoring tools are also used based on the requirement. 1.1 Client Side Statistics • Running Vusers • Hits per Second • Throughput • HTTP Status Code • HTTP responses per Second • Pages downloaded per Second • Transaction response time • Page Component breakdown time • Page Download time • Component size Analysis • Error Statistics • Errors per Second • Total Successful/Failed Transactions 1.2 Server Side Statistics • System Resources - Processor Utilization, Memory and Disk Space • Web Server Resources–Threads, Cache Hit Ratio • Application Server Resources–Heap size, JDBC Connection Pool • Database Server Resources–Wait Events, SQL Queries • Transaction Profiling • Code Block Analysis 1.3 Network Statistics • Bandwidth Utilization • Network delay time • Network Segment delay time 1.4 Conclusion Performance testing is an independent discipline and involves all the phases as the mainstream testing lifecycle i.e strategy, plan, design, execution, analysis and reporting. Without the rigor described in this paper, executing performance testing does not yield anything more than finding more defects in the system. However, if executed systematically with appropriate planning, performance testing can unearth issues that otherwise cannot be done through mainstream testing. It is very typical of the project manager to be overtaken by time and resource pressures leading not enough budget being allocated for performance testing, the consequences of which could be disastrous to the final system. There is another flip side of the coin. However there is an important point to be noted here. Before testing the system for performance requirements, the system should have been architected and designed for meeting the required performance goals. If not, it may be too late in the software development cycle to correct serious performance issues. Web-enabled applications and infrastructures must be able to execute evolving business processes with speed and precision while sustaining high volumes of changing and unpredictable user audiences. Load testing gives the greatest line of defense against poor performance and accommodates complementary strategies for performance management and monitoring of a production environment. The discipline helps businesses succeed in leveraging Web technologies to their best advantage, enabling new business opportunity lowering transaction costs and strengthening profitability. Fortunately, robust and viable solutions exist to help fend off disasters that result from poor performance. Automated load testing tools and services are available to meet the critical need of measuring and optimizing complex and dynamic application and infrastructure performance. Once these solutions are properly adopted and utilized, leveraging an ongoing, lifecycle-focused approach, businesses can begin to take charge and leverage information technology assets to their competitive advantage. By continuously testing and monitoring the performance of critical software applications, business can confidently and proactively execute strategic corporate initiatives for the benefit of shareholders and customers alike. . 1 Performance Metrics The Common Metrics selected /used during the performance testing is as below • Response. testing the system for performance requirements, the system should have been architected and designed for meeting the required performance goals. If not,