Regression testing

1 Regression Testing 1.1 What is regression Testing − Regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes. − Regression testing is a normal part of the program development process. Test department coders develop code test scenarios and exercises that will test new units of code after they have been written. − Before a new version of a software product is released, the old test cases are run against the new version to make sure that all the old capabilities still work. The reason they might not work because changing or adding new code to a program can easily introduce errors into code that is not intended to be changed. − The selective retesting of a software system that has been modified to ensure that any bugs have been fixed and that no other previously working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the software. Also referred to as verification testing − Regression testing is initiated after a programmer has attempted to fix a recognized problem or has added source code to a program that may have inadvertently introduced errors. − It is a quality control measure to ensure that the newly modified code still complies with its specified requirements and that unmodified code has not been affected by the maintenance activity. 1.2 Test Execution Test Execution is the heart of the testing process. Each time your application changes, you will want to execute the relevant parts of your test plan in order to locate defects and assess quality. 1.2.1 Create Test Cycles During this stage you decide the subset of tests from your test database you want to execute. Usually you do not run all the tests at once. At different stages of the quality assurance process, you need to execute different tests in order to address specific goals. A related group of tests is called a test cycle, and can include both manual and automated tests Example: You can create a cycle containing basic tests that run on each build of the application throughout development. You can run the cycle each time a new build is ready, to determine the application's stability before beginning more rigorous testing. Example: You can create another set of tests for a particular module in your application. This test cycle includes tests that check that module in depth. To decide which test cycles to build, refer to the testing goals you defined at the beginning of the process. Also consider issues such as the current state of the application and whether new functions have been added or modified. Following are examples of some general categories of test cycles to consider: 1* sanity cycle checks the entire system at a basic level (breadth, rather than depth) to see that it is functional and stable. This cycle should include basic-level tests containing mostly positive checks. 2* normal cycle tests the system a little more in depth than the sanity cycle. This cycle can group medium-level tests, containing both positive and negative checks. 3* advanced cycle tests both breadth and depth. This cycle can be run when more time is available for testing. The tests in the cycle cover the entire application (breadth), and also test advanced options in the application (depth). 4* regression cycle tests maintenance builds. The goal of this type of cycle is to verify that a change to one part of the software did not break the rest of the application. A regression cycle includes sanity-level tests for testing the entire software, as well as in-depth tests for the specific area of the application that was modified. 1.2.2 Run Test Cycles (Automated & Manual Tests) Once you have created cycles that cover your testing objectives, you begin executing the tests in the cycle. You perform manual tests using the test steps. Testing Tools executes automated tests for you. A test cycle is complete only when all tests-automatic and manual-have been run. − With Manual Test Execution you follow the instructions in the test steps of each test. You use the application, enter input, compare the application output with the expected output, and log the results. For each test step you assign either pass or fail status. − During Automated Test Execution you create a batch of tests and launch the entire batch at once. Testing Tools runs the tests one at a time. It then imports results, providing outcome summaries for each test. 1.2.3 Analyze Test Results After every test run one analyze and validate the test results. And have to identify all the failed steps in the tests and to determine whether a bug has been detected, or if the expected result needs to be updated. 1.3 Change Request 1.3.1 Initiating a Change Request A user or developer wants to suggest a modification that would improve an existing application, notices a problem with an application, or wants to recommend an enhancement. Any major or minor request is considered a problem with an application and will be entered as a change request. 1.3.2 Type of Change Request Bug the application works incorrectly or provides incorrect information. (for example, a letter is allowed to be entered in a number field) Change a modification of the existing application. (for example, sorting the files alphabetically by the second field rather than numerically by the first field makes them easier to find) Enhancement new functionality or item added to the application. (for example, a new report, a new field, or a new button) 1.3.3 Priority for the request Low the application works but this would make the function easier or more user friendly. High the application works, but this is necessary to perform a job. Critical the application does not work, job functions are impaired and there is no work around. This also applies to any Section 508 infraction. 1.4 Bug Tracking − Locating and repairing software bugs is an essential part of software development. − Bugs can be detected and reported by engineers, testers, and end-users in all phases of the testing process. − Information about bugs must be detailed and organized in order to schedule bug fixes and determine software release dates. Bug Tracking involves two main stages: reporting and tracking. 1.4.1 Report Bugs Once you execute the manual and automated tests in a cycle, you report the bugs (or defects) that you detected. The bugs are stored in a database so that you can manage them and analyze the status of your application. When you report a bug, you record all the information necessary to reproduce and fix it. You also make sure that the QA and development personnel involved in fixing the bug are notified. 1.4.2 Track and Analyze Bugs The lifecycle of a bug begins when it is reported and ends when it is fixed, verified, and closed. − First you report New bugs to the database, and provide all necessary information to reproduce, fix, and follow up the bug. − The Quality Assurance manager or Project manager periodically reviews all New bugs and decides which should be fixed. These bugs are given the status Open and are assigned to a member of the development team. − Software developers fix the Open bugs and assign them the status Fixed. − QA personnel test a new build of the application. If a bug does not reoccur, it is Closed. If a bug is detected again, it is reopened. Communication is an essential part of bug tracking; all members of the development and quality assurance team must be well informed in order to insure that bugs information is up to date and that the most important problems are addressed. The number of open or fixed bugs is a good indicator of the quality status of your application. You can use data analysis tools such as re-ports and graphs in interpret bug data. 1.5 Traceability Matrix A traceability matrix is created by associating requirements with the products that satisfy them. Tests are associated with the requirements on which they are based and the product tested to meet the requirement. Below is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown below. Traceability requires unique identifiers for each requirement and product. Numbers for products are established in a configuration management (CM) plan. Traceability ensures completeness, that all lower level requirements derive from higher level requirements, and that all higher level requirements are allocated to lower level requirements. Traceability is also used in managing change and provides the basis for test planning. SAMPLE TRACEABILITY MATRIX A traceability matrix is a report from the requirements database or repository. The examples below show traceability between user and system requirements. User requirement identifiers begin with "U" and system requirements with "S." Tracing S12 to its source makes it clear this requirement is erroneous: it must be eliminated, rewritten, or the traceability corrected. In addition to traceability matrices, other reports are necessary to manage requirements. What goes into each report depends on the information needs of those receiving the report(s). Determine their information needs and document the information that will be associated with the requirements when you set up your requirements database or repository 2 Phases of Testing 2.1 Introduction The Primary objective of testing effort is to determine the conformance to requirements specified in the contracted documents. The integration of this code with the internal code is the important objective. Goal is to evaluate the system as a whole, not its parts Techniques can be structural or functional. Techniques can be used in any stage that tests the system as a whole (System testing ,Acceptance Testing, Unit testing, Installation, etc.) 2.2 Types and Phases of Testing SDLC Document QA Document Software Requirement Specification Requirement Checklist Design Document Design Checklist Functional Specification Functional Checklist Design Document & Functional Specs Unit Test Case Documents Design Document & Functional Specs Integration Test Case Documents Design Document & Functional Specs System Test Case Documents Unit / System / Integration Test Case Documents Regression Test Case Documents Functional Specs, Performance Criteria Performance Test Case Documents Software Requirement Specification, Unit / System / Integration / Regression / Performance Test Case Documents User Acceptance Test Case Documents. 2.3 The “V”Model Requirements Specification Architecture Detailed Design Unit Testing Integration Testing System Testing Acceptance Testing Coding Requirement Study Software Requirement Specification Requirement Checklist Software Requirement Specification Functional Specification Checklist Functional Specification Document Functional Specification Document Architecture Design Architecture Design Detailed Design Document Coding Functional Specification Document Unit Test Case Documents Design Document Functional Specification Document Unit Test Case Document System Test Case Document Integration Test Case Document Unit/Integration/System Test Case Documents Regression Test Case Document Functional Specification Document Performance Criteria Performance Test Cases and Scenarios Software Requirement Specification Regression Test Case Document Performance Test Cases and Scenarios User Acceptance Test Case Documents/Scenarios Regression Round 3 Performance Testing Regression Round 2 Regression Round 1 Design Review Architecture Review Specification Review Requirements Review Requirements Specification Architecture Detailed Design Unit Testing Integration Testing System Testing Code Code Walkthrough [...]... rigorous System Testing and, where appropriate Regression Testing 4.3 Conclusion Hence the goal of acceptance testing should verify the overall quality, correct operation, scalability, completeness, usability, portability, and robustness of the functional components supplied by the Software system 5 SYSTEM TESTING 5.1 Introduction to SYSTEM TESTING For most organizations, software and system testing represents... can be applied flexibly, whether testing a financial system, e-commerce, an online casino or games testing System Testing is more than just functional testing, however, and can, when appropriate, also encompass many other types of testing, such as: o security o load/stress o performance o browser compatibility o localisation 5.2 Need for System Testing Effective software testing, as a part of software... adherence to standards security testing - test security requirements Functional techniques requirements testing - fundamental form of testing - makes sure the system does what it’s required to do regression testing - make sure unchanged functionality remains unchanged error-handling testing - test required error-handling functions (usually user error) manual-support testing - test that the system can... complexity of module C is 2 4 Acceptance Testing 4.1 Introduction – Acceptance Testing In software engineering, acceptance testing is formal testing conducted to determine whether a system satisfies its acceptance criteria and thus whether the customer should accept the system The main types of software testing are: Component Interface System Acceptance Release Acceptance Testing checks the system against... integration testing can be combined, verifying the details of each module's implementation in an integration context Many projects compromise, combining module testing with the lowest level of subsystem integration testing, and then performing pure integration testing at higher levels Each of these views of integration testing may be appropriate for any given project, so an integration testing method... stress testing - test larger-than-normal capacity in terms of transactions, data, users, speed, etc execution testing- test performance in terms of speed, precision, etc recovery testing - test how the system recovers from a disaster, how it handles corrupted data, etc operations testing - test how the system fits in with existing operations and procedures in the user organization compliance testing. .. be flexible enough to accommodate them all Combining module testing with bottom-up integration 3.1 Generalization of module testing criteria Module testing criteria can often be generalized in several possible ways to support integration testing As discussed in the previous subsection, the most obvious generalization is to satisfy the module testing criterion in an integration context, in effect using... situations output domain testing - pick test cases that will produce output at the extremes of the output domain Structural techniques statement testing - ensure the set of test cases exercises every statement at least once branch testing - each branch of an if/then statement is exercised conditional testing - each truth statement is exercised both true and false expression testing - every part of every... Acceptance Testing. The main Objective to Unit Testing are as follows : •To execute a program with the intent of finding an error.; • To uncover an as-yet undiscovered error ; and • Prepare a test case with a high probability of finding an as-yet undiscovered error Levels of Unit Testing •UNIT •100% code coverage • • • • • INTEGRATION SYSTEM ACCEPTANCE MAINTENANCE AND REGRESSION Concepts in Unit Testing: ... leverage the overall integration structure to allow rigorous testing at each phase while minimizing duplication of effort It is important to understand the relationship between module testing and integration testing In one view, modules are rigorously tested in isolation using stubs and drivers before any integration is attempted Then, integration testing concentrates entirely on module interactions, assuming . 1 Regression Testing 1.1 What is regression Testing − Regression testing is the process of testing changes to computer programs. tests the system as a whole (System testing ,Acceptance Testing, Unit testing, Installation, etc.) 2.2 Types and Phases of Testing SDLC Document QA Document