Grammar-Based Action Selection Rules for Scriptless Testing

Scriptless testing at the GUI level involves generating test sequences on the fly. These test sequences mimic user interactions on the GUI. The creation of these sequences works through action selection rules, which is most commonly based on stochastic methods. Script-less tests are reliable because they work with the actual state of the System Under Test (SUT). However, the tests are less specific, harder to interpret, and it is difficult to test concrete use cases or workflows. We want to tackle this drawback of scriptless tests by introducing action selection rules that are easier to guide than pure stochastic methods. In this paper, a new approach based on a grammar for the action selection rules is proposed, enabling scriptless testing tools to mimic user behaviour when interacting with web applications. While grammars have been used in software testing to generate input data for test cases, the proposed approach uses grammars to specify action selection rules to generate test sequences that mimic testing strategies employed by human testers. An empirical study has been performed to evaluate the effectiveness and the efficiency of the grammar-based action selection rules to filing web forms in comparison with random action selection rules. In the study, two SUTs were used: WebformSUT and Parabank. The average success rate for the grammar-based approach was 95.9% against random's 57.0% for WebformSUT and 99.8% against 55.7% for Parabank. For the widget interaction grammar-based had an average deviation from the ideal ratio of 0.06165 (WebformSUT) and 0.0180 (Parabank), compared random's 0.4318 (WebformSUT) and 0.7774 (Parabank). The results demonstrate the effectiveness of the grammar-based approach and the improvement in the use of resources.