A parameterized unit test framework based on java pathfinder

ж /К - г / ỵm A Parameterized Unit Test Framework Based on Java PathFinder VU THANH NHAN Faculty of Information Technology University of Engineering and Technology Vietnam National University Jirtnoi Advisor: TRỪỐNG ANH HOANG, Ph.D A t h e s is s u b m i t t e d in f u lf illm e n t o f t h e r e q u ir e m e n ts M a s te r o f In fo r m a tio n T e c h n o lo g y D e c e m b e r , 2Ö Ü9 C A I H O C Q U O C G iA H À N Ó I T D IJ N S ไ Ấ M T H Õ N G T IN T H Ư V IẼ N A 그 ìù ị M fo r th e d e g re e o f Table of Contents In tr o d u c tio n 1 丁 l i e m o t i v a t i o n /2 O u r s o l i l i i o n 3 C o n t r i b u t i o n T h e s is s t i uc t u i e A u to m a tic 2 g e n e r a tio n T e s t a ( l ( | t i a c y c r i t e r i a 1 D a ta -H o w o r ie n te d a d e q u a c y c r it e r ia 2 C o n t r o l - f l o w o r i e n t e d a d e q u a c y c r i t e r i u 1Ü U n i t l e s t a n d P a r a m e te r iz e d U n i t T e s t 11 11 I V u c iM ie t e r iz e d U n i t T e s t 13 T e s t i n p u t g e n e r a t i o n b y s y m b o l i c e x e c u t i o n 15 S y m b o l i c e x e c u t i o n 1ช C J o iiť ia liz e d s y m b o lic e x e c u tio n w i t h 3 C o d e i l i s t r u i n e n t a t io n f o r s y m b o l i c e x e c u t i o n 2 2 2 2 te s t d a ta U n it T e s t T e s t ca s e s g e n e r a tio n w ith la z y i n i t i a l i z a t i o n 20 JP F 24 ] T e s t D a t a G e n e r a t o i A i c l i i t ( * t i u e o f S y m b o l i c J a v a P a t h F i i i d i T ドm T h e I n s t r u c t io n tm y 2 A t t r i b u t e s f o r s t o r i n g s v n i b o l i c i n f o r m a t i o n 31 3 lia ỉ ì d lin g 32 Sc、 w r a l e x a m p le s o f s y m b o l i c e x t '( u t u )H i n S y m b o l i c 115F JP K Ы а ш і і і п ц c o n d i t i o i j s w i t 11 C l i u i c c C ! ( sn e r a l ( > r s L is te n e r ^ 35 .3 T A B L E O F C O N T E N T S p U T F m e w o rk 42 1.1 1>1*1 F r a m e w o r k o v e r v i e w 12 M a in i l c o m p o n e n t s o f І Ч Г Г Р г а и к л ѵ о г к ,2 ] P U T R u i m e i 4 :2 P U T L is te n e r P U T D r iv e i L3 C o n f i g u r e a n d e x e c u t e P U T F r a m e w o r k J S e v e r a l e x a m p l e s 51 E x p e r im e n ta tio n and d is c u s s io n 55 I i x | > t vi i m e n t a l r e s u l t 5 R e l a t e d w o r k s F u t u r e r e s e a r c h C o n c lu s io n s C o re c o m p o n e n ts 61 o f J P F 63 с C h ü ic e C e n e r a t OI' f o r A l I m p le m e n ta tio n o f p A S y m b o l i c i m p l e m e u t a t i o i i e x a m p l e s A M e t h o d s o f V M L i s t e n e r i n t e r f a c e A I M e t h o d s o f S e a r c h L i s t e n e r i n t e r f a c e Im p o rta n t P U T F rn e w o rk i i i t c g e i n u m b e r c o m p o n e n ts B I M a in B I i i i p l e m e i i t a t i o n o f P U T D r i v e t 69 m e t h o d o f P U T R u n n e r Ỉ * บ fr L i s t e n e r a n d i m p o r t a n t m e t h o d s 71 List of Figures 1.1 F r a m e w o r k o v e r v i e w E x a m p le o f U T 22 R e l a t i o n s h i p b e t w e e n U T a m i P U T Ы E x a m p l e o f s y m b o l i c e x e c u t i o n 17 ЗЛ JP F A r c h i t e c t l i r e o f a t e s t ( l a t a g e n e r a t o r *27 3 B y t e c o d e F a c t o r i e s f o r c o n c r e t e a n d s y m b o l i c e x e c u t i o n 3 A t t r i b u t e s t o r i n g i l l S y m b o l i c J F F 3 С 1ic )i c e C e n e r a t o r s m o t i v a t i o n J a v a P a th F in d e i D e p t h f i r s t s e a r c h I i u t i t i c a t i u n a u t o m a t i o n Ỉ.1 P U i T V a m e w o r k c o m p o n e n t s ' c o - o p e r a t i o n Л T w o im p lc m e n ta tk m s o f IA D U i n s t r u c t i o n A T w o im p le m e n ta tio n s o f IFG bj s t a t e m e n t .6 a n d P U T h i g h - l e v e l s t r u c t u r e L i s t e n e r p a t t e r n 12 Chapter Introduction T lie r o a r e m a n y е х ш п р іе у o f h u g e H n a n c ia l lo s s e s ( e g A r ia n e flig h t 5Ü1 c s h , M a r s E x p lo r a tio n R o v t '1 S p i r i t t n a l i i m c t i o n ( V i s s e i ( 't ฟ , 0 a ) ) o r ОѴЧМІ t l i e c o s t o f lu iim in l liv e s ( e g h e c -2 r a d ia tio n t h e r a p y ln a d iim * fa ilu r e ( L e v \s o n к T u rn e r, 9 ) ) t h a t w e r e c a u s e d b y s o f t w a r e e r r o r s T h i s c o s t is a s m u c h a m o r e u b i q u i t o u s s o f t w a r e is i n th e m o d e rn lif e A c c o r d in g to a r e p o r t b y th e u s N a tio n a l I n s t it u t e o f S t a n d a r d s a n d T e c h n o lo g y * s o f t w a r e f a i l l i r e z c o s t t i l e u s e c o n o m y $ b i l l i o n a n n u iíL but w h ile c : o u ld it th e iin p r o v e m e n ts saw o n e - th ild in s o ftw a re o f t h is cost te s tin g in fia 、 s tru c tu re (N IS T , 0 ) a re s t ill S o ftw a re per lim it e d , te s tin g is a il i m p u i t a n t p a r t o f s o f t w a r e d e v e lo p r iio u t p r o c e s s e s a n d is a b o u t h a l f o f t h e t o t a l c o s t OÏ s o f t w a r e d e v d o p u u m t ; a n d m a i i i t c i i a t i c e ( B e id e r , 9 ) T e s tin g has m any p ro c e s s o f in c r e a s in g циаі iìììive d e fin itio n s , tru s t Л t ile m a x im u m le p K 's e n t a t iv e 'ĩìp iit o f in p u ts A f t e r te s t ( la t a ()1 m akes a test suite is e x e c u t i n g u:sf (.me A l l t)f id e a b e h in d tiie s e fo r iịựị a s i n g le test om eli' ты it on re fe rs T h e o n ly a l l p o s s ib le in p u t s , to a w a v to b u t t h is u ť i u p u t « s u t l m t t h e y le a d t o T h e r e f o r e , t h e f i r s t s t e p o f t e s t i n g is s e l e c t i n g In p u t s e le c tio n , I lu * b e h a v i o u r o f t h e p i o g r a n i o flc c o r n in o li F o r f e a s i b i l i t y , w e b u ilc l Í Ị n u m b e r o f e rro rs set th e th o c o r r e c t n e s s o f a p r o g r a r iL c o r r e c t 1HJSS o f a p r o g r a m IS u s u a l l y i m p o s s i b l e t in d about h ilt is c a l l e d is น•ร(ฟ ่ t o a v a l l i a t e 、c o i n b i i i a t i o n lest m p tii to s t of teat data test (lata or test t h r c o rre c tn e s s o f a n d th e ( o r r e s p o n d ijig o f a lo g ic a l g r o u p o f te s ts m a k e a test Sid ( llia n to la 0 ) IV s t iu g c a n b e c m iit n l o u t i l l s e v e l w a y s : M dììỉtal ivsfnty c n ii b e e x t n 'i i i r l v la b o u r Hit )1 !(>>• in t e n s i v e and t a k e s a lo t o f t im e c in d C h a p te r In tro d u c tio n Randům t t s iifiij f tiM ie r a t e s r a i i d o m it s e n d s to th e p ro g r a m its s im p lic ity , w it h th is ( liliic u lt as in p u t p a n im e te r s I h e m a in a tIv a n ta g e o f th is m e th o d i t a ls o i c q i i i i e s a l i t t l e 1Ш Ч 1Ю Ѵ a n d o t h e r m a d i i n e r e s o u r c e s a p p ro a c h to d a t a , u s u a lly u s in g a s t r e a m o f h it s , w h ic h one p a th g e t e x e c u te d m ay h e e x e c u te d m any a (k 、 q u a < :y c r i t e r i a o r th e t im e s , (Z h u et and a l som e 1997) is H o w e v e r, p a th s a re is ( l i f f i ť i i l r to a c h ie v e Symbolic txccution th e c a n r e s o lv e t h e d r a w b a c k s o f t a n d o i n v a r i a b le s t h a t n o r m a l l V c o n t a i n t e s tin g I ll t h is m e th o d c o n c r e t e v a lu e s a r e r e p l a c e d b y t h e i r s y m b o l i c c o i m h n p a r t s , w h i c h e x p r e s s a r a n g e o f p a s s i b l e v a lu e s u s i n g s y m b o l i c e x p r e s s io n s lJ a ， s e (l o n t h is s y m b o lic d a ta , th e m o d e l c h e c k e r g e n e te s a n c o v e r s a l l o f p o s s ib le e x e c u t i o n s o f t h e p r o g r a i i b in p u t d a ta se t th a t I t u s u a l l y I n q u i r e s e x l e i n a l s o lv e r s ( K n ) e n i u g &: S t i i c l i n n u i , 0 ) t o f i n d t h e s o i u t i o n s f o r s y m b o l i c e x p r e s s io n s A u to - n ia t( 、 ( l t e s t i n g h e lp s l e d I ICO t h e ť o s t o ť p i o d i i d n g s o f t w a r e a n d in c r e a s e s t l u 、r e l i a b i l i t y o f th e s o ftw a re I n 0 ，D ir e c t t H Ỉ A u t o m a t e d R a n d o m T e s tin g s - D A R T ( G o d e fr o id e t ฟ า 0 ) m i n k s I l i e b o r n o f a n e w t e s t i n g a p p r o a c h w h i c h is t i l e c o m b i n a t i o n o f l a n d o i n t e s t i n g a n d s y n i b o l i c e x e c 'u t io n T h e i d e a o f t h i s a p p r o a c h is s t a r t i n g t h e u n d e r t e s t Ị M O g r a m w iih n d o m v a lu e s o f p a r a m e t e r s , ( h i r i i i g p a ỉh c o n s t r a i n s a n 、c o l l e c t e d A c o n s t r a i n t รบ！ v e r is u s e d t o g e n e r a t e v a lu e s o f [ ) a i a i u e t e r s t h a t ( • ( ) m \ s | > o ii d i a g p a t h T h is t i l e e x e c u t i o n u f u n d e r t e s t |> K > ftr a in , t h e a p p ro a c h to s o lv e fo rc e th e u n d e r te s t p ro g r a m c a lle d coricali с testing a ls o p a th c o n s tr a in s e x e c u te f o llo w A n o th e r n a m e o f t h i s a p p r o a c h is d y n a m i c s y m b o l i c e x e c u t i o n w h i c h i n t r o d u c e i n S e c t i o n I l l t h i s t h e s is , \\v a re c o ijc e r iio d [і； ('И {ьг м ( іо п a n d e x e c u t i o n w ith ( u m b in in g t w o fLiěaxS O Í t e s t i n g : t(,\st c a s e s H e n c e a [Ѵ а ш е ѵ ѵ о г к ( w e п а ш е it p บ ï F r a m e w o r k ( T i lio n g Ắ.* V u Ü ) ) is p r o p o s e d t o g e n e r a t e t ( 、 s t c a 's e s f u r J a v a | ； K ) ^ r a m s a n d c x e r u te th e s e te s t ca se s a u to m a tic a lly P a lil i* in d ( u tio n (P a s a rra n u F o r t e s t c a s e s g e n e r a t i o n , t h i s f r a m e w o r k is b a « e d O lì J a v a et a l., 0 )- a in o d o l d ie r k e i th a t u se s s y n ih u lic execu­ F o r te s t c a s e s e x w u t io n t h e it r ( 、 lic s o n J 1/1 l i t , a n ( ) | ) ( 'n - s o u r c e t e s t I r a i n e w o r k Ilu u c ío n í, b e fo re in t r o d u c in g t h e ( l i 't a i l o f th e (Т Н І ( li s c t is s io n s ( la ta g r iu T ü t io u a b o n I s y m b o lic e x e c u ỵ i( ) ỵ L its f ir im e w o i k , t h is th e s is p re s e n ts s e v I is a g o in J a v a P a t liF in d i'i fo r te s t In a d d itio n , th e th ( 、 s is i n t r o d u c e s d e t a i l o f J U n i t a n d t h e w a y t f i ІП ІИ Л Ѵ О Ік u s e s t i l l s t o o l ť o i t ( vs t CHSPS ^ e t i r r a t i o n he 1 T h e 1.1 m o tiv a tio n The motivation A lth o u g h s o ftw a re » t e s t i n g s t i l l f a c e d i H i c u l ỵ ic s in te c h n iq u e s t h e ir w o r k have such gained as: th e lit ig e i m p r o v e r iif u t s , d e v e lo p e r s in s u ttic ie n c y o f m o d e l c h e c k e rs fo r ( l i t f e r e n t p r o g r a m m i n g la n g u a g e s , l a c k o f e f f i c i e n t t e c h n i q u e s f o r t e s t ( rusi's g e n e r a t i o n a n d e x e c u tio n S P IN g m s ( H o l / i n a n n , Ü Ü ) is t h e m o s t p o p u l a r m o d e l c h e c k e r f u r с and P R Ü M E LA (a P R O re s s M E ta LAuguage) s p e c ific a tio n s 0 ) A n o t h e r p o w e r f u l m o d e l c h e c k e r is P e x ( P r o g r a m w h i c h is d e v e l o p e d f o r c h e c k i n g N E T u la r p r o g ia iiin iin g p ro g m s w t it le n ib i la n g u a g e , b u t M o r e o v e r, a lt h o u g h th e re J a v a is o n e o f t h e m o s t p o p ­ is Ỵ 1Ü c o m p l e t e m o d e l c h e c k e r f o r Java Path F in d e r no c o m p le to lĩio d e l c h e c k e r is a m o d e l c h e c k e r f o r J a v a p r i i n i t i v e d a t a t y p e s , s u c h a s n u m e r ic a n d b o o le a n Pararnetť.ň zn l U n it Test ( P U T ) ( T i i l m a n i i fc S c h u lt e ' 0 : & S c l i u l t e , 0 ) พ ล ร p r o p o s e d f o r r e d u c i n g t h e m u i i b e r o f a-ssess t e s t u n i t s t h a t te s te r s m u s t w r it e , a b ig p r o b le m is h o w t o in s t a n t ia t e g iv e n s e t o f t e s t c a s o s w h i c h s a t is f ie s s t iv e r a i c o d e c o v e r a g e c r i t e r i a n u m e r o u s a p p r o a c h e s t o r e s o lv e t h i s p r o b l e m P U T s to a g o o d S in c e MOW, t h e r e a r e b u t n o o n e h a s a c o m p le te s o lu tio n A n d e v e n w h e n w e h a v e a g o o d s e t o f te s t ca se s, th e e x e c u tin g th e m \\พ b u g s ( lu r iĩầ g І Ш Ш ІШ the» e x e c u t i o n is a b i g p r o b le m These p m v ( T a n d m o n e y , e s |) ( ： Ì H lly w h e n w o r k i n g w i t h 1.2 ( H o lz m a n n , E x p lo r a t io n ) o f M ic r o s o ft, W e s a y t h a t t h e r e is b e c a u s e a lt h o u g h p ro g r a m s , it c a n w o r k o n ly w it h T illm a im u n t il n o w in t h is U m g u a g e J a v a p r o g r a m s p r o g r a m s o n ly la n g u a g e p r o w o rk s ta k n a n d t r a c k in g a lo t o f tim e a h u g e set ฟ t e s t c; s t s a i i l o m a t к г н ііѵ i h is c lia p t t M is C p te r Intro ductio n c o n c e ! ІК Ч І w ir l i i n t i c x l u n g t h r e e m a in с о т р о ш 'п ! ร o f t h e ііа п и ч ѵ ч я к a n d t h ( 、 ir c o o p r r a tio n s W e a ls o c le s ( ： r il) e h o w t o c o n f i s u r e a n d e x e c u it e t h e f i a n ie w u i k a n d p re s e n t a s im p le e x a m p le o f f r a m e w o r k D |) tn a tio r i C h a p te r r v a lu a te s th e r e s u lt o f t h is w o r k , a n a ly s e t h e i n i t i a l r e s u l t s a n d d r a w ­ PU TFram cicork b a c k s o f t he fra m e w o rk W e a ls o m a k e c o m p a i i s o n s b e t w e e n a n d s e v e l r e la te d o n e s , h i t h ť l a t t e r p a r t o f t h i s c h a p t e r W(? p o i n t o u t s o m e a re a 's f o r f u r t h e r s t u d y C h a p te r c o n c lu d a s our w o rk and s u m n ia r ix e s th e s itu a tio n t e d i n i( | iu 's , t h e a d v a n ta g e s a n d l i n i i t a t i o i i s o f t h e m o ř c u rre n t, t e s t iì iịị Appendix A Core components of JP F A l Implementation of PCChoiceGenerator for in­ teger number p u b lic c la s s I ' C - C ' l i o i c t ' i ì i ' n e i al t.»r e x t e n d s P a t b C o n d i t i o n ；| p u b lic PCChoiceGciHMUtor ( i i i t super (0 , -si ze PC- = n e w I » 11 n t G f V i l I u ầẻ ti (* r a U i f K ’; size ) { 】): h C ' n i n l i iur» I s i vsir Ị ; } / / ร- i：t ร p u b lic Ih c v o i ci l * ( ' c o ท ร t r ị i ì ท I ร f о г th e c u r r e n t , c /| o Ĩ i ：il -set ( 'ill r e n t l JC ( P a t h C o n d i t i o n PC* i g « t N с X t ( ' Il и i j f * c í เ< - f jM i't hi» r| t e s lM e l h o d — t e i s t i ' Ỉ il s s г c t e i v e r о b jtM I h u rj ( Iť b tM v t.h tíd N iL U íe Mi/.1 ІН니11ŨI ge t i y аг а Irỉ ^•11ฯ 'I ' y p try } «41 g s = { S t I i І»й s p t' « Ỉ / / í ịl t i о ท 1/ I - TI i t •• s Co n s t J v t l > s t ( " o i l ! t 1.111 H • 1' ( ( M ils '* < ? t li e с 4.1> { c la s s } rcm p { re t lirn c a tc h r e t ІІГ ІІ B.3 t h e C l s s i g e l C o n s t I- U ( ： I o r ( a l'g ร ) ( N o S u ch M e th o d K x c y p tio n e ) { to r (n ew t lì c C 1a s я g e t c u nì * C la s s Ị) ; PUTListener and im portant methods p u b lic c la s s 'Г Listener Seal с h L i St flit* r / / ru u.K Ị p u b Ỉic H с ỊỊ i b't ť r p u b lic extCỵfid» V M Listenci ( i t' n I*r i с p |Je r I > irnpleiTUMits I ' u t»I i >li ť I к Xt e n ы о ÏI { L la t e n t r void s e a I-c h St a r t eti ( s ti a I