Tài liệu A parameterized unit test framework based on Java PathFinder

ж /К - г../ îm A Parameterized Unit Test Framework Based on Java PathFinder VU THANH NHAN Faculty of Information Technology University of Engineering and Technology Vietnam National University. Jirtnoi Advisor: TRỪỐNG ANH HOANG, Ph.D. A t h e s is s u b m i t t e d in f u lf illm e n t o f t h e r e q u ir e m e n ts M a s te r o f In fo r m a tio n T e c h n o lo g y D e c e m b e r , 2Ö Ü9 C A I H O C Q U O C G iA H À N Ó I T D IJ N S ไ Ấ M T H Õ N G T IN T H Ư V IẼ N A 그 ìù ị M fo r th e d e g re e o f Table of Contents 1 2 In tr o d u c tio n 1 1 .1 丁 l i e m o t i v a t i o n .................................................................................................................................... 3 1 /2 O u r s o l i l i i o n ............................................................................................................................................. 3 1 .3 C o n t r i b u t i o n ............................................................................................................................................. 5 1 .4 T h e s is s t i uc t u i e .................................................................................................................................... 5 A u to m a tic 2 .1 2 .2 g e n e r a tio n 7 T e s t. a ( l ( 4 | t i a c y c r i t e r i a .................................................................................................................. 8 2 . 1 .1 D a ta -H o w o r ie n te d a d e q u a c y c r it e r ia ............................................................. 9 2 .1 .2 C o n t r o l - f l o w o r i e n t e d a d e q u a c y c r i t e r i u ........................................................ 1Ü U n i t l e s t a n d P a r a m e te r iz e d U n i t T e s t ........................................................................... 11 .................................................................................................................................. 11 I V u c iM ie t e r iz e d U n i t T e s t . ............................................................................................ 13 T e s t i n p u t g e n e r a t i o n b y s y m b o l i c e x e c u t i o n ............................................................. 15 2 .3 .1 S y m b o l i c e x e c u t i o n ........................................................................................................... 1ช 2 .3 .2 C J o iiť ia liz e d s y m b o lic e x e c u tio n w i t h 2 .3 .3 C o d e i l i s t r u i n e n t a t io n f o r s y m b o l i c e x e c u t i o n ...........................................2 2 2 .2 .1 2 .2 .2 2 .3 te s t d a ta U n it T e s t T e s t ca s e s g e n e r a tio n w ith la z y i n i t i a l i z a t i o n . . . . 20 JP F 24 3. ] T e s t D a t a G e n e r a t o i ...................................................................................................................... 2 6 3 .2 A i c l i i t ( 4 * t i u e o f S y m b o l i c J a v a P a t h F i i i d i T ............................................................... 2 8 ドm 3 .2 .1 T h e I n s t r u c t io n tm y ............................................................................................. 2 9 3 .2 .2 A t t r i b u t e s f o r s t o r i n g s v n i b o l i c i n f o r m a t i o n ................................................ 31 3 .2 .3 lia ỉ ì d lin g 32 3 .2 .1 Sc、 w r a l e x a m p le s o f s y m b o l i c e x t '( u t u )H i n S y m b o l i c 115F 3 .2 .5 JP K Ы а ш і і і п ц c o n d i t i o i j s w i t 11 C l i u i c c C ! ( sn e r a l. ( > r s .................... L is te n e r ^ . . . 35 .....................................................................................................................3 0 T A B L E O F C O N T E N T S p U T F ra m e w o rk 42 1.1 1>1*1 F r a m e w o r k o v e r v i e w ............................................................................................................. 12 1 .2 M a in i l c o m p o n e n t s o f І Ч Г Г Р г а и к л ѵ о г к ............................................................................. 1 ,2 .] P U T R u i m e i ............................................................................................................................. 4 4 1 .2 :2 P U T L is te n e r 1 .2 .3 P U T D r iv e i .........................................................................................................................4 8 ..............................................................................................................................5 0 L3 C o n f i g u r e a n d e x e c u t e P U T F r a m e w o r k ........................................................................ 5 0 4. J S e v e r a l e x a m p l e s ................................................................................................................................51 E x p e r im e n ta tio n and d is c u s s io n 55 5 .1 I i x | > t vi i m e n t a l r e s u l t ........................................................................................................................... 5 5 5 .2 R e l a t e d w o r k s .........................................................................................................................................5 8 5 .3 F u t u r e r e s e a r c h .................................................................................................................................... 5 9 C o n c lu s io n s C o re c o m p o n e n ts 61 o f J P F 63 с C h ü ic e C e n e r a t OI' f o r A .l I m p le m e n ta tio n o f p A .2 S y m b o l i c i m p l e m e u t a t i o i i e x a m p l e s ................................................................................. 6 4 A .3 M e t h o d s o f V M L i s t . e n e r i n t e r f a c e ...................................................................................... 6 7 A. I M e t h o d s o f S e a r c h L i s t e n e r i n t e r f a c e ................................................................................. 6 8 Im p o rta n t P U T F ra rn e w o rk i i i t c g e i n u m b e r ...................... 6 3 c o m p o n e n ts B. I M a in B .2 I i i i p l e m e i i t a t i o n o f P U T D r i v e t ............................................................................................... 7 0 69 m e t h o d o f P U T R u n n e r ....................................................................................................6 9 Ỉ * บ fr L i s t e n e r a n d i m p o r t a n t m e t h o d s .............................................................................. 71 List of Figures 1.1 F r a m e w o r k o v e r v i e w ............................................................................................................................ 2 .1 E x a m p le o f U T 22 R e l a t i o n s h i p b e t w e e n U T a m i P U T ...................................................................................... Ы 2 .3 E x a m p l e o f s y m b o l i c e x e c u t i o n ................................................................................................. 17 ЗЛ JP F 3 .2 A r c h i t e c t l i r e o f a t e s t ( l a t a g e n e r a t o r .................................................................................. *27 3 .3 B y t e c o d e F a c t o r i e s f o r c o n c r e t e a n d s y m b o l i c e x e c u t i o n ...................................3 0 3 .1 A t t r i b u t e s t o r i n g i l l S y m b o l i c J F F ........................................................................................3 2 3 .5 С 1ic )i c e C e n e r a t o r s m o t i v a t i o n ......................................................................................................3 4 3 .6 J a v a P a th F in d e i 3 .7 D e p t h f i r s t s e a r c h I i u t i t i c a t i u n a u t o m a t i o n ..................................................................... 3 8 Ỉ.1 P U i T V a m e w o r k c o m p o n e n t s ' c o - o p e r a t i o n ................................................................... 4 5 Л .1 T w o im p lc m e n ta tk m s o f IA D U i n s t r u c t i o n ..................................................................6 4 A .2 T w o im p le m e n ta tio n s o f IFG bj . s t a t e m e n t .....................................................................6 6 a n d P U T ............................................................................................................... h i g h - l e v e l s t r u c t u r e .................................................................................................................. 2 6 L i s t e n e r p a t t e r n ...........................................................................................3 7 4 12 Chapter 1 Introduction T lie r o a r e m a n y е х ш п р іе у o f h u g e H n a n c ia l lo s s e s ( e .g . A r ia n e 5 flig h t 5Ü1 c ra s h , M a r s E x p lo r a tio n R o v t '1 S p i r i t t n a l i i m c t i o n ( V i s s e i ( 't ฟ . , 2 0 0 1 a ) ) o r ОѴЧМІ t l i e c o s t o f lu iim in l liv e s ( e . g . h e ra c -2 5 r a d ia tio n t h e r a p y ln a d iim * fa ilu r e ( L e v 4 \s o n к T u rn e r, 1 9 9 3 ) ) t h a t w e r e c a u s e d b y s o f t w a r e e r r o r s . T h i s c o s t is a s m u c h a 8 m o r e u b i q u i t o u s s o f t w a r e is i n th e m o d e rn lif e . A c c o r d in g to a r e p o r t b y th e u s N a tio n a l I n s t it u t e o f S t a n d a r d s a n d T e c h n o lo g y * s o f t w a r e f a i l l i r e z c o s t t i l e u s e c o n o m y $ 6 0 b i l l i o n a n n u iíL but w h ile c : o u ld it th e iin p r o v e m e n ts saw o n e - th ild in s o ftw a re o f t h is cost te s tin g in fia 、 s tru c tu re (N IS T , 2 0 0 2 ). a re s t ill S o ftw a re per lim it e d , te s tin g is a il i m p u i t a n t p a r t o f s o f t w a r e d e v e lo p r iio u t . p r o c e s s e s a n d is a b o u t h a l f o f t h e t o t a l c o s t OÏ s o f t w a r e d e v d o p u u m t ; a n d m a i i i t c i i a t i c e ( B e id e r , 1 9 9 0 ) . T e s tin g has m any p ro c e s s o f in c r e a s in g циаі iìììive d e fin itio n s , tru s t Л t ile m a x im u m le p K 's e n t a t iv e 'ĩìp iit. o f in p u ts . A f t e r te s t ( la t a ()1 m akes a test suite is e x e c u t i n g u:sf (.me. A l l t)f id e a b e h in d tiie s e fo r iịựị a s i n g le test om eli' ты it on re fe rs T h e o n ly a l l p o s s ib le in p u t s , to a w a v to b u t t h is u ť i u p u t « s u t l m t t h e y le a d t o T h e r e f o r e , t h e f i r s t s t e p o f t e s t i n g is s e l e c t i n g In p u t s e le c tio n , I lu * b e h a v i o u r o f t h e p i o g r a n i . o ra flc c o r n in o li F o r f e a s i b i l i t y , w e b u ilc l Í Ị n u m b e r o f e rro rs . set th e th o c o r r e c t n e s s o f a p r o g r a r iL c o r r e c t 1HJSS o f a p r o g r a m IS u s u a l l y i m p o s s i b l e . t in d about h ilt is c a l l e d is น•ร(ฟ ่ t o a v a l l i a t e 、c o i n b i i i a t i o n lest m p tii to s t of teat data test (lata or test t h r c o rre c tn e s s o f a n d th e ( o r r e s p o n d ijig o f a lo g ic a l g r o u p o f te s ts m a k e a test Sid ( llia n to la . 2 0 0 6 ). IV s t iu g c a n b e c m iit n l o u t i l l s e v e ra l w a y s : M dììỉtal ivsfnty c n ii b e e x t n 'i i i r l v la b o u r Hit )1 !(>>•. 1 in t e n s i v e and t a k e s a lo t o f t im e c in d C h a p te r 1. In tro d u c tio n 2 Randům t t s iifiij f tiM ie r a t e s r a i i d o m it s e n d s to th e p ro g r a m its s im p lic ity , w it h th is ( liliic u lt as in p u t p a n im e te r s . I h e m a in a tIv a n ta g e o f th is m e th o d i t a ls o i . c q i i i i e s a l i t t l e 1Ш Ч 1Ю 1 Ѵ a n d o t h e r m a d i i n e r e s o u r c e s . a p p ro a c h to d a t a , u s u a lly u s in g a s t r e a m o f h it s , w h ic h one p a th g e t e x e c u te d m ay h e e x e c u te d m any a (k 、 q u a < :y c r i t e r i a o r th e t im e s , (Z h u et and a l.. som e 1997) is H o w e v e r, p a th s a re is ( l i f f i ť i i l r to a c h ie v e . Symbolic txccution th e c a n r e s o lv e t h e d r a w b a c k s o f t a n d o i n v a r i a b le s t h a t n o r m a l l V c o n t a i n t e s tin g . I ll t h is m e th o d c o n c r e t e v a lu e s a r e r e p l a c e d b y t h e i r s y m b o l i c c o i m h n . p a r t s , w h i c h e x p r e s s a r a n g e o f p a s s i b l e v a lu e s u s i n g s y m b o l i c e x p r e s s io n s . lJ a ， s e (l o n t h is s y m b o lic d a ta , th e m o d e l c h e c k e r g e n e ra te s a n c o v e r s a l l o f p o s s ib le e x e c u t i o n s o f t h e p r o g r a i i b in p u t d a ta se t th a t I t u s u a l l y I n q u i r e s e x l e i n a l s o lv e r s ( K n ) e n i u g &: S t i i c l i n n u i , 2 0 0 8 ) t o f i n d t h e s o i u t i o n s f o r s y m b o l i c e x p r e s s io n s . A u to - n ia t( 、 ( l t e s t i n g h e lp s l e d I ICO t h e ť o s t o ť p i o d i i d n g s o f t w a r e a n d in c r e a s e s t l u 、r e l i a b i l i t y o f th e s o ftw a re . I n 2 0 0 5 ，D ir e c t t H Ỉ A u t o m a t e d R a n d o m T e s tin g s - D A R T ( G o d e fr o id e t ฟ . า 2 0 0 5 ) m i n k s I l i e b o r n o f a n e w t e s t i n g a p p r o a c h w h i c h is t i l e c o m b i n a t i o n o f l a n d o i n t e s t i n g a n d s y n i b o l i c e x e c 'u t io n . T h e i d e a o f t h i s a p p r o a c h is s t a r t i n g t h e u n d e r t e s t Ị M O g r a m w iih ra n d o m v a lu e s o f p a r a m e t e r s , ( h i r i i i g p a ỉh c o n s t r a i n s a n 、c o l l e c t e d . A c o n s t r a i n t .รบ！ v e r is u s e d t o g e n e r a t e v a lu e s o f [ ) a i a i u e t e r s t h a t ( • ( ) m \ s | > o ii d i a g p a t h . T h is t i l e e x e c u t i o n u f u n d e r t e s t |> K > ftr a in , t h e a p p ro a c h to s o lv e fo rc e th e u n d e r te s t p ro g r a m c a lle d coricali с testing a ls o . p a th c o n s tr a in s e x e c u te f o llo w A n o th e r n a m e o f t h i s a p p r o a c h is d y n a m i c s y m b o l i c e x e c u t i o n w h i c h i n t r o d u c e i n S e c t i o n 2 . 3 . I l l t h i s t h e s is , \\v a re c o ijc e r iio d [і； ('И {ьг м ( .іо п a n d e x e c u t i o n . w ith ( u m b in in g t w o fLiěaxS O Í t e s t i n g : t(,\st c a s e s H e n c e a [Ѵ а ш е ѵ ѵ о г к ( w e п а ш е it. p บ ï F r a m e w o r k ( T i lio n g Ắ.* V u . 2 0 Ü 9 ) ) is p r o p o s e d t o g e n e r a t e t ( 、 s t c a 's e s f u r J a v a | ； K ) ^ r a m s a n d c x e r u te th e s e te s t ca se s a u to m a tic a lly . P a lil i* in d ( u tio n . (P a .s a rra n u F o r t e s t c a s e s g e n e r a t i o n , t h i s f r a m e w o r k is b a « e d O lì J a v a et a l., 2 0 0 8 )- a in o d o l d ie r k e i th a t u se s s y n ih u lic execu­ F o r te s t c a s e s e x w u t io n . t h e it r ( 、 lic s o n J 1/1 l i t , a n ( ) | ) ( 'n - s o u r c e t e s t I r a i n e w o r k . Ilu u c ío n í, b e fo re in t r o d u c in g t h e ( l i 't a i l o f th e (Т Н І ( li s c t is s io n s ( la ta g r iu T ü t io u . a b o n I s y m b o lic e x e c u î i( . ) î L its f ir im e w o i k , t h is th e s is p re s e n ts s e v I is a g o in .J a v a P a t liF in d i'i fo r te s t In a d d itio n , th e th ( 、 s is i n t r o d u c e s d e t a i l o f . J U n i t a n d t h e w a y t f i ІП ІИ Л Ѵ О Ік u s e s t i l l s t o o l ť o i t ( vs t CHSPS ^ e t i r r a t i o n . he 1 .1 . T h e 1.1 m o tiv a tio n 3 The motivation A lth o u g h s o ftw a re » t e s t i n g s t i l l f a c e d i H i c u l î ic s in te c h n iq u e s t h e ir w o r k have such gained as: th e lit ig e i m p r o v e r iif u t s , d e v e lo p e r s in s u ttic ie n c y o f m o d e l c h e c k e rs fo r ( l i t f e r e n t p r o g r a m m i n g la n g u a g e s , l a c k o f e f f i c i e n t t e c h n i q u e s f o r t e s t ( rusi's g e n e r a t i o n a n d e x e c u tio n . S P IN g ra m s ( H o l / i n a n n , 2 Ü Ü 3 ) is t h e m o s t p o p u l a r m o d e l c h e c k e r f u r с and P R Ü M E LA (a P R O re s s M E ta LAuguage) s p e c ific a tio n s 2 0 0 3 ) . A n o t h e r p o w e r f u l m o d e l c h e c k e r is P e x 1 ( P r o g r a m w h i c h is d e v e l o p e d f o r c h e c k i n g . N E T u la r p r o g ia iiin iin g p ro g ra m s w t it le n ib i la n g u a g e , b u t M o r e o v e r, a lt h o u g h th e re J a v a is o n e o f t h e m o s t p o p ­ is Î 1Ü c o m p l e t e m o d e l c h e c k e r f o r Java Path F in d e r no c o m p le to lĩio d e l c h e c k e r is a m o d e l c h e c k e r f o r J a v a p r i i n i t i v e d a t a t y p e s , s u c h a s n u m e r ic a n d b o o le a n . Pararnetť.ň zn l U n it Test ( P U T ) ( T i i l m a n i i fc S c h u lt e ' 2 0 0 5 : & S c l i u l t e , 2 0 0 6 ) พ ล .ร p r o p o s e d f o r r e d u c i n g t h e m u i i b e r o f a-ssess t e s t u n i t s t h a t te s te r s m u s t w r it e , a b ig p r o b le m is h o w t o in s t a n t ia t e g iv e n s e t o f t e s t c a s o s w h i c h s a t is f ie s . s t iv e r a i c o d e c o v e r a g e c r i t e r i a . n u m e r o u s a p p r o a c h e s t o r e s o lv e t h i s p r o b l e m P U T s to a g o o d S in c e MOW, t h e r e a r e b u t n o o n e h a s a c o m p le te s o lu tio n . A n d e v e n w h e n w e h a v e a g o o d s e t o f te s t ca se s, th e e x e c u tin g th e m \\พ b u g s ( lu r iĩầ g І Ш Ш ІШ the» e x e c u t i o n is a. b i g p r o b le m . These p m v ( T a n d m o n e y , e s |) ( 4 ： Ì H lly w h e n w o r k i n g w i t h 1.2 ( H o lz m a n n , E x p lo r a t io n ) o f M ic r o s o ft, W e s a y t h a t t h e r e is b e c a u s e a lt h o u g h p ro g r a m s , it c a n w o r k o n ly w it h T illm a im u n t il n o w in t h is U m g u a g e . .J a v a p r o g r a m s p r o g r a m s o n ly . la n g u a g e p r o w o rk s ta k n a n d t r a c k in g a lo t o f tim e . a. h u g e set. ฟ . t e s t c;S('(1 framewenk: P U T F r a m e w o r k . which accepts USCT P l l s . ^ (ฯ К ฯ a t e s a l l i l o x r c i i t e s L ilit I > s t s a i i l o m a t к г н ііѵ . i h is c lia p t t M is C ha p te r 1. Intro ductio n 6 c o n c e ! ІК Ч І w ir l i i n t i c x l u à n g t h r e e m a in с о т р о ш . 'п ! ร o f t h e ііа п и ч ѵ ч я к a n d t h ( 、 ir c o o p r r a tio n s . W e a ls o c le s ( ： r il) e h o w t o c o n f i s u r e a n d e x e c u it e t h e f i. a n ie w u i k a n d p re s e n t a s im p le e x a m p le o f f r a m e w o r k D |) tn a tio r i. C h a p te r 5 r v a lu a te s th e r e s u lt o f t h is w o r k , a n a ly s e t h e i n i t i a l r e s u l t s a n d d r a w ­ PU TFram cicork b a c k s o f t he fra m e w o rk . W e a ls o m a k e c o m p a i i s o n s b e t w e e n a n d s e v e ra l r e la te d o n e s , h i t h ť l a t t e r p a r t o f t h i s c h a p t e r . W(? p o i n t o u t s o m e a re a 's f o r f u r t h e r s t u d y . C h a p te r 6 c o n c lu d a s our w o rk and s u m n ia r ix e s th e s itu a tio n t e d i n i( | iu 's , t h e a d v a n ta g e s a n d l i n i i t a t i o i i s o f t h e m . o ř c u rre n t, t e s t iì iịị Chapter 2 Automatic test data generation 111 t h i s c h a p t (나 w e i n t r o d u c e m a in in g p a r t s o f t h i s th e s is . s e v e r a l b a s ic ( . Q iic e p t s , w h i c h F ir s t, w ill be use d w e w a n t t o in t r o d u c e a n o v e r v ie w in th e re ­ o f te s t d a ta A s w e m e n t i이le d i l l C h a p t e r 1, t h e u n iq u e w a y t o g u a r a n te e c o r r e c tn e s s g e n e r a tio n . o f a p ro g ra m is t o e x e c u t e i t o n a s m a n y i n p u t c a s t 's a s p o s s ib le * rh o r e fo re , th e te s t d a t a g e n e r a t i o n is o n e o f t h e m o s t i m p o r t a n t ; p a r t s o f a t e s t i n g p r o c e s s . A g o o d te s t d a t a g e n e r a t o r is o n e o f t h e f a c t o r s t h a t m a k e cl g o o d t e s t i n g m o d e l. T e s t d a t a g e n e r a t io n c a n b e a p p r o a c h e d fr o m d i f f e r e n t a s p e c ts : 1. H o w S(、 li4 :u 、 (] t e s t a fle q u a c y l(*vel c a n b e К Ч К ІК Ч І (i.e . a lg o r it h m s f o r t h a t ) , 2. H o w to 3. H o w t o ( T C r tte m i n i m a l t e s t s e t s f o r t h e s o le c tc ic l c r i t e r i a To m a k e t e s t d a t a g e n e r a t io n œ m p u t a t i o î i a l l y e f fic ie n t. a n s w e r th e r a i l b e d iv id e d H e u r is tic a b o v e q u e s tio n s , s e v e ra l te c h n iq u e s h a v e b e e n p ro p o s e d . They b a s (、 d o n s o m e t y p i c a l c a t e g o ii e s : - e x a c t: 111 g e n e r a l , l u m i i s t i c a p p r o a c h e s m i g h t i n c l u d e r a n d o m n e s s a n d a re m o r e e ffic ie n t in t h e m o s t ca se s. n o t o p t im a l o r in a d é q u a t e r e s u it . H o w e v e r , t h o s e a p p r o a c h e s c a n le a d t o a O n t h e o p p o s i t e s id e , e x a c t a p p r o a c h e s a r e m o i e l i k e l y t o f i n d a l l o p t i m a l r e s u l t b u ( a r e t y p i c a l l y le s s ( 、 出 (‘i r i i t - B la c k - b o x ( • t ille d - W h ite - b o x : If th e t e s t in g w i l i t (*-!)< >x t e s t i n g . ( ) t h ( M \ v is e i t t i l l * s p r 'c i f i c a t io n s o f s y s t ( * n i t e n t u r e s . te c h n iq u e is b a s e d is Ы ж . к - І ю х o il s o u rc e c o d e , i t te s t in g , w h ir h is is b a s e d o i l 8 C h a p te r 2. A u to m a tic test d a ta generation w ІІІІС-ІЮХ น:รtin y vocìi) ІІ)(Л s o ilin ' is t h (、a p p r o a c h t h a t a n a ly s e s ( a ls o k n o w a、 s s t r u c t u r a l te s tiiig ) : ( if th e p r o g r a m t o d e s ig n t e s t s S(.) t h a t th e y r e a c h t o o n e o f so v e t a I a d e q i m r y c r i t e r i a ( s u d i ( ^ . e v c 'i y с ч к іе l i n e is e x e c u t e d a t le a s t o n c e , o r e v ť i V f u n c t i o n is i n d i v i d i m l l y BI(K k'-box ttíỳ tn ự Ị e rro rs in a p ro g ra m a s y s te m W h i t ( 니 )o x t e s t i n g w i l l b e t l u ' f o e แ ร i n t h i s t h e s is . te s te d , e tc .). (s o m e tim e s ( a lli'd b e h a v io u ra l te s tin g ) is a il a p p ro a c h to fin d by v n lid a tin g its fu n c tio n a litie s w it h o u t th e d e ta ils o f th e c o d e . ba、 s e (l o n T h e g o a l o f t h is r i f t (ฯ П 1І 1К.* w h e t h e r t h e s y s t e m s a t i s f i e s i t s s p e c i f i c a t i o n . s im p le r th a n w h ite - b o x te s tin g b e ca u se it d e n t. (V o m t h e d e t a i l o f s t r u c t lu e s . im p lr m e n ta tio n s le a d s t o In a p ro g ra m , w o rk , w e p ro p o s e a ( l i i \ •(ฯ th e s e P U T w h ic h te s tin g a p p ro a c h of is t o T h i s t y p e o f t e s t i n g is m u c h is d o n e a t a h i g h e r l e v e l a n d is in c ie p e n - I l l a d d it io n , b la c k - b o x t e s t in g c a n f in d in c o r r e c t b e c a u s e o f s p e c itic a tio n a d e c ỊU it c ỵ and w h iit-b o x tts tim j c o n t a i n s o n e ()1 m is im d e r s ta n d in g , th a t m any m o d e l. I ll t h is m o d e l, u s e rs w r it e P U T ( S ) , o u r ir a iiu n v o r k th e n p ro c e s s e s b y s y m b o lic e x e c u tio n f o r g e n e r a tin g U T s s u c h t h a t t h e y re a c h t o p a th ( o r น ฯ !g t І М 1 p a t h ) c o v e ra g e . и г s p e c ific a tio n in c o r r e c t b e h a v io u r . O UI te s t o f th e t ile m e t l ic s P U T (w a y 넜 to ไ ,b o r e f o r e , i n e v a lu a te th e c o n c e p ts a re in t r o d u c e d t h is c h a p te r , w e in t r o d u c e d iff e ie n t w o r th in o f a te s t in p u t s e t) in S e c t i o n 2 .2 . 2 .1 ， S e c t io n 2 . 3 l o o k s a t t h e a p ­ symbolic (ixccution p ro a c h t o g r iie ia t e te s t ca se s a u to iu a tic ： a lly b y S e c tio n te s t. a n d it s s u p p o r t in g te r lm iq iie s . 2.1 Test adequacy criteria 111 1972，D jis k tia claimed th a t лProỊỊram testing can be uacd to show the presť.nrt o f buys, bat ïicva- th e ir ab^itîiœ. a p p r o a c h is n o t a c c e p t a b l e . in b o t h llib d a im can be t h a t ： te s tin g u iu le is to o d H o w e v e r , l i o w a d a v s s o f t w a 14* t e s t i n o Ііа ъ a r a p i d g r o w t h р іш Ч іс е a n d e x ỊH T Ìiiie iit.s a n d h a s b e c a m e a n in d is ịx u is a b le p a r t o f s o ftw a r e d (、 \ v l ( ) | ) m ( * i i t . A n d a s i g n i f i c a n t m i l e s t o n e o f s o f t w a r e t e s t i n g g r o w t h is t h e r e s e a i c i i o f G o o ( l( * n ( H ig li a n d G e r h a r t o f s o f t w a r e 1 t e s t i n g is a d e q u a c y o f a te s t. M Ц К 'ill w h a t 18 te s t crite rio n г trs t ( Z lm a ( h * q iu u v . , th a t m eans h o w t o m e a is u r e t h e S in c e t h e n , t e s t c r i t e r i a b e c a m e a n a t t r a c t i v e r e s e a r c h f o c u s a n d n i m i b r r o f t o s t C i i t e i ia h a w Z h u e t.a l. u n it ( 1 9 / ช - ] ! ) / / ), w h i c h p o i n t e d o u t t h a t t i l l ' c e n t r a l q u e s t i o n e t a l.. Ill I )(.*en |> io p o s e c l. 1 9 ÍJ 7 ) is OIK* o f m o s t c u m p i d i r i i s i v e tliis М 11Л ѴѴ. a te s t ( le tta a d (* < |u a < y ะรШ VC v s O li s u f t w a i v (1 ì t e i i o i i is seen as H 2 Л . Test adequacy criteria s to p p in g vw\v a ll p ro g ra m . ร o f th e g e n e r a tim i p ro c e s s . a set o f s p e (ฯ íic a tio r is . [T ะ= 2l) ) . t h e s e t o f a ll te s t s e ts С. С 9 D F o llo w in g a set o f in p u ts fo r p io g r a m p X ร X T —> 0 , 1 and a te s t-s e t to [ ( Z i n i et. a l. . 1 9 9 7 ) , |> a g e 3 6 8 Ị. r e q u i i 4 x l , o t lie r v v i s e . t i l t * U î tit s in to ill th e o f a p r o g m iiL of < td ， f b o o le a n ('d > 5 w h e re v a lu e . T ilt ? E. f € v a lu e o f / is a. f u n c t i o n is u s e d to fro m t e ll w h ic h b r a n c h s l i o u l d b e f o l l o w e d w i i e n l ) i a i K l i i n g is p o s s i b l e , v a i l l e o f v a r i a b l e s is a v e c t o r Г. \ \ ( 、c a n s e e t h a t • ร ť p because no p re d e c e s s o rs , F o r each t\ e m u s t h a v e n o s u c c e s s o r. € N รุ and cd =： < ท. m n node e x a c tly o n e e d g e II each > in s ta n c e w ith t(L f > la b e l < V, v a lu e su ch th a t th e rö f( v ) = tra t. lu u ร o n l y o n e siK *( ： e s s o i\ t h e la t ) o l o f r e la t e c i a r c is a lw a y s tru t Ị), That im * a n s . t h e c o n d i t i o n b e p u t i n t o H « in g le n o d e , в .4 , t h e s e c o n d n o d e is В .4 л в CFG ( w h o r e .4 a n d 1)0 e x ( 4 4 i t k m ， o p p o n it e ' t o ( I i t r i ia . tí a re a to m ic c o n d it io n s ) r a n A a n in p u t f fo r p. vve c a l l is A — tru e . feasible path if th e re itijcasiblr. palli. A c c o r d in g to ( Z im Ị x it ì i( l\ t) We is a t use l( 、 ast f e a s ib le an —< ร. ( Ị\A I ) . .. .. in p u t p a th Node to th a t ( le f i n e p [t) . e > And cause th a t a c ie q u a c .v e t a l.. 1 9 9 7 ) a n d ( E t ỉ v a r d s s o n , 1 9 9 9 ). t h e r e a io f o llo w in g ін іг ф іа х .у c r i t e r i a : S ta te m e n t c o v e ra g e : not t n i( \ p a t h 4 n o d e s a re i l l th e s a m e o r d e r a s th e y a re e x e c u te d i l l p a th s c a n can be m u s t n o t h a v e b r a n c h i i i f t in s i d e . a n d t h e a r c c o n n e c t s t w o n o d e s h a y la b e l ( liv e n a p io g r a in " a n d e x e c u tio n p M o re - r i l i s c o n d i t i o n u iiK s t b e p u t i n k ) 2 n o d e s : t h e f i r s t n o d e is is n o t a l w a y s e x e c u t e d , i t is e v a l u a t e d o n l y i f is e x e c u t i o n tru e . a t a tim e . i f a l l n o d (썼 e x c e p t t h e la ,S t o n e h a v e o n l y o n e s u c c e s s o r , ( • () lla j> s o (l i n t o a s i u g l r n o d e . A s in g l e m x l e o f is e x is ts 4 ’ h c s e K i q u i r e m e n t s ๓ ) แรш ,е t h e ( I c t . e n i i i n i s t i c o f b r a n r h i n g i n a. p r o g r a m . ill a p a th p T h e e x e c u tio n o f o f v a r i a b le s i f t h e r e w e r e m o r e t h a n o n e s u c c e s s o r , o n l y o n e a r c is OVVI , 、 is u u a n i b i g u o i i s . i t r e q u i r e s .ร- h a s o n e a n d o n l y o n e s u c c e s s o r . th u s T h a t r iie a u s , i f p .ร is t h e f i r s t s t a t e m e n t o f h a s n o s u c c e s s o rs a n d c a n h a v e m a n y p re d e c e s s o rs . fin is h e d i l l • s a t is f ie s f o l l o w i n g r e q u i r e m e n t s : h a s e x a c t l y erne s iK 'c e s s o r a n d and • CFG a ll s ta te m e n ts o f C F G a r c (*xe c ： u t e ( l. 11 2.2. U n it Test and P aram eterized U n it Test B ra n c h ff c o v e ra g e : K u c o u i i t e r a l l b r a n d i e s i l l th e * i > i o g r a m , s ta te m e n t m u s t b e e v a lu a to il to b o th P a th cove ra g e ะ H o w e v e r, a l l p o s s ib le e x e c u t i o n th is c o v e ra g e is not t r u e a n d f a ls e . p a th s ШЧ、i n c l u d e d 】e a ,(*h e d . a lw a y s t h e Ị )1 e d i c a t (л o f a l l A i l l th e s e t o f te s t p ro g ra m h a v in g p a th s . lo o p s or r e c u r s i o n s c a n h a v e a n i n f i n i t e n u m b e r o f e x e c u t i o n p a t h s , t h is is A ll i n f e a s i b l e r e q u ir e m e n t b e c a u s e t h e te s t s e t s h o u ld b e f in it e . L e n g th -n p a th c o v e ra g e : a b o v e p r o b le m , in a n o th e r adequacy t h is c r it e r io n , a ll t h e c r ite r io n is d e f in e d p a th s o f le n g th ท to r e s o lv e th e a re c o v e re d i l l th e t e s t s e t. \ Y ( 、( a i l s e e t h a t t h e s t a U u n e i i t (..o v e r a g e is t h e w e a k e s t , i f b r a n d i c o v e r a g e is r e a d i e d , a l l s t i i t e m e n t s o f |>i o g i a i ท m u s t 1)0 e x e c u t e d ( i. e . s t a t e m e n t s is I ( in c h e d ) . And p a t h c o v e r a g e is t h e s t t o l l t e s t , i f t h is a d e q u a c y is r e a c h e d , a l l b r a n c h o f C F G h e e v a lu a te d t o b o t h th e m ust t r u e a n d f a ls e . P a t h c o v e r a g e is n o t a n i m p l i c i t s t a t e m e n t c o v e r a g e , b u t i t is m e a n i n g f u l i n t h e p r a c tic e . T o a v o id s e a r c h in g a n d t r y th e s ta te to ivdch e x p lo s io n , Java P ath F in de r lim its th e d e p th o f p a th to t lic น -p a th c o v e ra g e . 2.2U nit Test and Parameterized U nit Test 2.2.1 U n it Test I n c o m p u t e r p r o g r a m m i n g , u n i t t e s t i n g is a t e s t i n g m e t h o d , i l l w h i c h a t e s t e r t e s t s i f i n d i v i d u a l u n i t s o f a s o u r c e c o d e a r e f i t f o r u s e * Л u n i t is t h e s m a l l e s t t e s t a b l e p a r t o f a n a p p lic a tio n . fu n c tio n , dass. In | ) K ) c e ( ỉ iỉ ic t l p r o g r a r i i m i n g , a u n i t p r o c e d i m i f 't c . I - n it t e s tin g c a n e x e c u tio n o f e a ih t e s t i n g c la s s e s a r e w r i t t e n , e a c h t e s t i n g o t a c la s s - u ii( le r - t ( 's t w ith t e s t iu g o f il p ro p e r c la s s o f t e n has m an y I b s t i u g m e t. h o c Is a r e s u b - f V m c ^ io iis t h a t h a v e n o i n p u t p a t a n u i t e r . I t re p r e s e n ts A i.s a m e t h o d n o t e n s u re th e c o r r e c tn e s s o f a ll p r o g r a m s b u t th e r o t I II 11 a v o i d v a lu e . r e s u lt, o r i i i u l i n ^ p r u g r a n m iin g . H u n it t u i i t is t h e b a s is o f. c o i r e c t n e s s o f p r o g r a m . F o r u n it te s tin g , t( \s M n g m e t h o d s . I l l o l) j( 4 l- o r ie n te d m a y b e a n in d iv id u a l p ro g ra m 1 ii s in g le te s t c a s e a n d t y p i c a l l y e x e c u te s a m e t h o d f i x e d a i ^ u m e T i t s f o r V ( M if v iiig t h a t it K 't u r u s t h e (、 X |H 、 ( * te (l I h e r e is a n y u n - l i a i i d l e d e x ( 4 、 |> ti( iU i l l t h e ( 'X e c u l ІО ІІ. !U ('t h i) (,l c a l l e d t e s t c a s e is І Ш І 이 ) r i i d r n t fro m a U nit Ih ý t th e o th r is . ( V l') O l fl teĩit ease F o r e x a m p le , wv a ls o , have a id e a lly ; each и ĩtderTt>it dass 12 C h a p te r 2. A u to m a tic test d a ta generation w it h Add a s im p le fu n c tio n w h ic h r e tu r n s th e s u m o f a b s o l u t e v a lu e s o f t w o i n p u t p a m in e t e is i l l i u t (방 (ฯ t y p e а я fo llo w s : pIlk)lie class p u b lic บіккм Гі/уІ { i lit A t id ( I f i t X. i lit V) i if (x < 0 ) X if V = X . (y< 0 ) = ... у 1 re tu r n x-fv 1 E x a m p le p u b iน U T s and P U T f l . t s ๙ ÏJT te s t pubi i«. Viriti i X - in t v = 1; A dd () 1; X i г»t у = 1; i\iv lis t e d in F i» t u e 2 .1 : у) { ưnde rTest ut = I»“ w Under Tes t 0 ; in t ш = Mat h.abs ix.) + Math. abs (y) ; v<*i«1 te s t ilit m e th o d piใы i с Í：1 s s PUT test_add () ( |Mih»Ị.i«. V«>id PlJT]_add (in t :ч, ііИ add (M te s t_ .a d d _ l_ l 0 í аз s e r t Equal s (Add } publ ic fu i t e s tin g a b o v e y) / 2) ; a s s e rtT ru e ( u t . Add(X, add minus 1 1 0 { y) == m); } = ะ - 1; y) 1 2) a s s e rt Equals (Add ) pub i i i Lil Í лII \ V« »id te s t a dd mi nus 1 min นร 1 0 { к = -1 ; y = - li asseť tE q u a ls (Add () iWiเโ x< líỉ ẲCẢ: у >0 »VẢ; v < I о ) ; /Л rranýť. i lit y. — о, Г nde-rl'est ut new u miéťTesi ( ), //.,ใ./ z — lit Add( X 1 у ) ; /A s s e ri assert Kq ua!» (z . Math . abă ( x)-Ỷ- Math . abb ( v ) ) : III t e s t in g , w e c a ll g e n e r a liz e e n t ir e ly iie w I ^ a ia m e te r iz e d u n i t te s ts th a t, d e s c r ib e the b r l m v i ü u r o f t h e in v o lv e d m e t h o d s f o r a ll t e s t a r g u m e n ts , o r g e n e r a liz e g iv e n U T s t o P U T s o r a g a i n s t , w e i n s t a n t i a t e g iv e n P U T s t o U T s b y p a s s i n g t h e m c o n c r e t e v a lu e s o f in p u t p a ra m e te rs . The r e la tio n s h ip c a n b e r e p r e s e n te d b v f ig u r e 2 .2 ( r e fe re n c e d to (E d v a rd s s o n , 1 9 9 9 )) Traditional Unit Test Generalization : ) Instantiation \ / Parameterized Unit 了est Generation Implementation F i g l il o 2 .2 : N o w , w e CrHi ร(น* I h i l t a ls o ) , t h e I te s t d a ta I s t h a t I.e a cli t o a R e la tio n s h ip b e tw e e n U T i f t h e r e is a P U ^ (.ฯ น ฯ ,a t i n g г g iv e n and PU T. b v p io v r a , m in e r s ( c a lle d te s t d r i v e r Ь е счя и гк a p ro c e s 유 th a l in s t a n tia t e s g iv ( 'ii P U T to c o v e ra g e Cl ilo i LOU. u l t h a b o v e p r o p e r tie s . \v r ( a n t h i n k ІІІНЩІ u s in g syiiỉl> ()lic r x c c u tio u гШ(І c o ib S fia iu s o lv in g to a u t o m a t i r a l k a n a ly s e a pi I a n d im it a n t ia t r it fo U T s s iir li th a t th e set of a rc !liv e d U T s is in in im a l a m i 2.3. Test input, generation by sym bolic execution re a c h t o h ig h c o d e c o v e ra g e . .st/rnljolic tjx c a tio n w h i c h c illo w 2.3 เท í ì d d i t i o n . p и F s a m 15 symbolic summaries 1)(' u s ẹ d l o s c a le f o r a i ' b i t i 'c i i y a b s t r a c t i o n l r v ( 、 ls . Test input generation by symbolic execution T h e re a n ' s e v e ra l d iffe r e n t t e c lin i( iu e s (i(、is i n s t i u m r n t e d . o n s v in b o liť l i l t h is by e x e c u tio n . in s t r u m e n t , th e v a r i a b l e s i n o r i g i n a l d a r a t y p e s ( I n t e g e r f o r e x H in p U * ) a r e ( ‘h a n g e d t o c o r r e s p o t K Ì i a g C h a p te r 2. A u to m a tic test d ata generation 16 lib ia r y c la .s s ( I n t e g o i K x p n .^ s io n fo r e x a m p le th a t s u p p o rts m a n ip u la tio n o f s y ia - b o i i c I n t e g e r e x p r e s s io n s ) a n d o p e r a t i o n s i a v o l v i u g t h e s e v a r i a b l e s a r e r e p la c e d w i t h t i|) |) io |) iia ie m e th o d c a lls t h a t m a n i p u l a t e o b j e c t s o f c o n e s p Q T id in g t y p e ( i. e . In te - g c i E x p r e s s io n ) . I ll t lio s v m h o lic e x e c u tio n Path C o n diti on, p a th o f th e p ro g ra m . P C s ta n d s fo r t h a t is rt b o o l e a n f o n n u l a o v e r i n p u t v a r i a b l e s a n d d e s c r ib e s w h i c h c o n d i t i o n s m u s t he tru e in th e s ta to . A PC ill o rd e r fo r a n e x e c u tio n if th e b la n c h in g n e g a tio n . Then, f u h f 、s y m b o lic * , to fo llo w c o n d itio n c a n b e g e n e ra te d : a c c i u i m l a t e s ( X ) n s t r a iu t s t h a t th e p a r t ic u la r p a tli. depends o n e is t h e f o r m u l a on in p u t m u s t b a c k tra c k a n d c o n tin u e to a d d я c o n d itio n t o c h e c k t h a t t h e in d e x s ta y s w i t h i n I h i'e e p a r t s : W h e n w o r k in g w it h th e a re tw o PCs th a t p ro g ra m . A s ta te in symbolic values o f the variables, a Ш d e t e r n iin e d a lw a y s w it h a n o th e r b ra n c h v a r ia b le s o f a il a r r a y ty p e , P C r e c o g n iz e t h a t s y m b o l i c e x e c u t i o n a s s ig n m e n t s t a t e m e n t s in m u c h s a tis fy o f t h e b r a n c h in g c o n d it io n , a n d o t h e r f o r it s tru e . is e a s y t o th e re a c o n s t r a i n t s o lv e r e v a l u a t e s P C s , i f a P C e x c 'c u t io n in p u ts I l l e a c h b r a n c h s t a t . e m e iit . s y m b o l(ร ), h a s th e P C m a y b e It th e th a t needs t h e s iz e O Í t i l e a r m y . depends on th e b r a n c h in g a n d s y m b o lic e x e c u tio n path condition and a c o n s is t s o f yroqmm counter w h ic h in d ic a te s t h e n e x t s ta te m e n t to b e e x e c u te d . I n s y m b o l i c (、 x ( 4 . u t i o n ，a a ll e x e c u tio n p a th s o f th e Symbolic Exœ ution Tret p io g r a m . ( S E l ' ) is u s e d t o ť l i m a c t e r i z e Т І Н 、S E T o f a p i . o g i a m is a ( p o s s i b l y in fin ite ) t r e e w h e r e n o d e s a r e p r o g r a m s t a t e s d u r i n g s y m b o l i c e x e c u t i o n a n d a r c s a r e [p o s s ib le t r a n s i t i o n s b e t w e e n s t a t e s . A l l t h e l e a f n o d e s o f t h e S E T w h e r e t h e P C is s a t i s f i a b l e n ’in v s n it d iffe r e n t e x e c u tio n p a th s . M o r e o v e r , a l l f e a s ib le e x e c u t i o n |> ř it lis o f t h e p r o t r a ili a re p re s e n t(、 ( l i l l S E T . T i l e m a i n p r o b l e m is s o l v i n g t h e p a t i i c o n d i t i o n s , a l l s n tis fia b le v a lu a tio n s fo r a P C ( in Л le a f n o d e ) w i l l g iv e u s a r e a l i n p u t a n d o x e c u t i o n p a t h s w i t h a l l t h o s e i n p u t ШХ1 e q u a l . Ш К І t h e 1Ц1Ш ІШ 1 o f (’( m c r e t e e x t 'c n t i o i i s m a y b e in f in it e . F o r e x a m p le , w e h a v e a s im p le p r o g r a m e x (4 ‘iit io n a n d s y m b o lic e x e c u tio n t h a t s w a p s t w o in te g e r s a n d it s c o n c r e te p a t li ( fig u r e 2 .3 ):