Softwaretesting 08 eng

ソフトウェアテスト 　　 [8] テストの評価と信頼性評価と信頼性と信頼性信頼性 Software Testing [8] Evaluation of testing and reliability あまん ひろひさ ひろひさ 阿萬 裕久 裕久（ AMAN Hirohisa ） aman@ehime-u.ac.jp (C) 2007-2022 Hirohisa AMAN Evaluation of the test  Testing is the activity of quality assur ance of software Perfect Testing ⇒ There is no defect (fault) and no failure occurs There are no so-called "bugs"  Evaluation of “Testing Activities” is ne cessary  Evaluation based on Coverage rate  Evaluation based on Bug resolution rate  Evaluation based on Operability (C) 2007-2022 Hirohisa AMAN Test evaluation (1) Evaluation based on coverage rate  Evaluation from the perspective of "h ow much you covered”  Focuse on state change  Focuse on Operation and input/output according to specifications *black box testing  Focus on Flow of program execution of program execution *w of program executionhite box testing (C) 2007-2022 Hirohisa AMAN Test evaluation (2) Evaluation based on bug resolution rate  Testing finds bugs and fixes them  If the total number of bugs is known, it i s possible to evaluate the testing activit y from the number of resolutions in it Bug's number of bugs resolved = resolution rate total number of bugs *Note: The term bug is not accurate; A phenomenon that does not work properly ("failure, defect") is found in the test, and "resolution" is to solve the "fault" that is the source of problem or the "error" that created it (C) 2007-2022 Hirohisa AMAN How to know "bug total"  In conclusion, the true number is not known  Comparing humans to a system is like trying to figure out how many times in a lifetime th ey get sick  If we consider that “the occurrence of a prob lem = the symptoms of a disease," A fault is a bacterial or virus that caused it, & an error is an act that leads to the infection  Estimate total number of bugs  Statistically (C) 2007-2022 Hirohisa AMAN Bug number estimation method ① c c apture/recapture method (1/4) [Example] I want to find out how many black bass are in Lake Biwa *It would be nice if we could catch all t he fish in Lake Biwa, but that is practi cally impossible (C) 2007-2022 Hirohisa AMAN Bug number estimation method ① c c apture/recapture method (2/4) Once you catch a suitable number of black bass Then, Mark the black bass and let it e scape After some time, the black bass are c aptured again, and the total number is estimated by how many marked on es are included in it (C) 2007-2022 Hirohisa AMAN Bug number estimation method ① c c apture/recapture method (3/4)  The image will like this Marking 10 fishes for the first time 2nd time, out of 10 fishes confirmed markings Recaptured After that mation of percentage of total =2:8 : This image diagram 10 : ? = : So there are 40 left (C) 2007-2022 Hirohisa AMAN Bug number estimation method ① c c apture/recapture method (4/4)  Applying this to software Prepare artificial bugs (marking) Suppose that the test finds artificial bugs and non-real bugs The total number of real bugs is estimate d by the following equation the follow of program executioning equation Than (C) 2007-2022 Hirohisa AMAN 𝑛𝑦 𝑥 Disadvantages of the capture / recapture method  It's hard to make realistic artificial bugs  Inevitably, it becomes a bug that seems to b e intentional, and it is found immediately  As an engineer, there is resistance to "creati ng bugs" (voice in the field)  They release "catchable" fish, and the es timated number is smaller than it actuall y is 𝑛𝑦 * is smaller and is 𝑥 larger than it should be (C) 2007-2022 Hirohisa AMAN 10