Symbolic search-based testing

Arthur Baars; Mark Harman; Youssef Hassoun; Kiran Lakhotia; Phil McMinn; Paolo Tonella; Tanja Vos

doi:10.1109/ASE.2011.6100119

Symbolic search-based testing

Arthur Baars^*, Mark Harman, Youssef Hassoun, Kiran Lakhotia, Phil McMinn, Paolo Tonella, Tanja Vos

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

Abstract

We present an algorithm for constructing fitness functions that improve the efficiency of search-based testing when trying to generate branch adequate test data. The algorithm combines symbolic information with dynamic analysis and has two key advantages: It does not require any change in the underlying test data generation technique and it avoids many problems traditionally associated with symbolic execution, in particular the presence of loops. We have evaluated the algorithm on industrial closed source and open source systems using both local and global search-based testing techniques, demonstrating that both are statistically significantly more efficient using our approach. The test for significance was done using a one-sided, paired Wilcoxon signed rank test. On average, the local search requires 23.41% and the global search 7.78% fewer fitness evaluations when using a symbolic execution based fitness function generated by the algorithm.

Original language	English
Title of host publication	2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, Proceedings
Pages	53-62
Number of pages	10
DOIs	https://doi.org/10.1109/ASE.2011.6100119
Publication status	Published - 2011
Externally published	Yes
Event	2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011 - Lawrence, KS, United States Duration: 6 Nov 2011 → 10 Nov 2011

Publication series

Series	Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE)

Conference

Conference	2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011
Abbreviated title	ASE 2011
Country/Territory	United States
City	Lawrence, KS
Period	6/11/11 → 10/11/11

Keywords

Fitness Functions
Search-Based Testing
Symbolic Execution

Access to Document

10.1109/ASE.2011.6100119

Cite this

@inproceedings{8d83005567ce4038909026d839fde780,

title = "Symbolic search-based testing",

abstract = "We present an algorithm for constructing fitness functions that improve the efficiency of search-based testing when trying to generate branch adequate test data. The algorithm combines symbolic information with dynamic analysis and has two key advantages: It does not require any change in the underlying test data generation technique and it avoids many problems traditionally associated with symbolic execution, in particular the presence of loops. We have evaluated the algorithm on industrial closed source and open source systems using both local and global search-based testing techniques, demonstrating that both are statistically significantly more efficient using our approach. The test for significance was done using a one-sided, paired Wilcoxon signed rank test. On average, the local search requires 23.41\% and the global search 7.78\% fewer fitness evaluations when using a symbolic execution based fitness function generated by the algorithm.",

keywords = "Fitness Functions, Search-Based Testing, Symbolic Execution",

author = "Arthur Baars and Mark Harman and Youssef Hassoun and Kiran Lakhotia and Phil McMinn and Paolo Tonella and Tanja Vos",

year = "2011",

doi = "10.1109/ASE.2011.6100119",

language = "English",

isbn = "9781457716393",

series = "Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE)",

publisher = "IEEE",

pages = "53--62",

booktitle = "2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, Proceedings",

note = "2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, ASE 2011 ; Conference date: 06-11-2011 Through 10-11-2011",

}

Baars, A, Harman, M, Hassoun, Y, Lakhotia, K, McMinn, P, Tonella, P & Vos, T 2011, Symbolic search-based testing. in 2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, Proceedings., 6100119, Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 53-62, 2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, Lawrence, KS, Kansas, United States, 6/11/11. https://doi.org/10.1109/ASE.2011.6100119

Symbolic search-based testing. / Baars, Arthur; Harman, Mark; Hassoun, Youssef et al.
2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, Proceedings. 2011. p. 53-62 6100119 (Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE)).

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

TY - GEN

T1 - Symbolic search-based testing

AU - Baars, Arthur

AU - Harman, Mark

AU - Hassoun, Youssef

AU - Lakhotia, Kiran

AU - McMinn, Phil

AU - Tonella, Paolo

AU - Vos, Tanja

PY - 2011

Y1 - 2011

N2 - We present an algorithm for constructing fitness functions that improve the efficiency of search-based testing when trying to generate branch adequate test data. The algorithm combines symbolic information with dynamic analysis and has two key advantages: It does not require any change in the underlying test data generation technique and it avoids many problems traditionally associated with symbolic execution, in particular the presence of loops. We have evaluated the algorithm on industrial closed source and open source systems using both local and global search-based testing techniques, demonstrating that both are statistically significantly more efficient using our approach. The test for significance was done using a one-sided, paired Wilcoxon signed rank test. On average, the local search requires 23.41% and the global search 7.78% fewer fitness evaluations when using a symbolic execution based fitness function generated by the algorithm.

AB - We present an algorithm for constructing fitness functions that improve the efficiency of search-based testing when trying to generate branch adequate test data. The algorithm combines symbolic information with dynamic analysis and has two key advantages: It does not require any change in the underlying test data generation technique and it avoids many problems traditionally associated with symbolic execution, in particular the presence of loops. We have evaluated the algorithm on industrial closed source and open source systems using both local and global search-based testing techniques, demonstrating that both are statistically significantly more efficient using our approach. The test for significance was done using a one-sided, paired Wilcoxon signed rank test. On average, the local search requires 23.41% and the global search 7.78% fewer fitness evaluations when using a symbolic execution based fitness function generated by the algorithm.

KW - Fitness Functions

KW - Search-Based Testing

KW - Symbolic Execution

UR - https://www.scopus.com/pages/publications/84855417349

U2 - 10.1109/ASE.2011.6100119

DO - 10.1109/ASE.2011.6100119

M3 - Conference Article in proceeding

AN - SCOPUS:84855417349

SN - 9781457716393

T3 - Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE)

SP - 53

EP - 62

BT - 2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, Proceedings

T2 - 2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011

Y2 - 6 November 2011 through 10 November 2011

ER -

Symbolic search-based testing

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this