Reinforcement Learning for Scriptless Testing: An Empirical Investigation of Reward Functions

Testing web applications through the GUI can be complex and time-consuming, as it involves checking the functionality of the system under test (SUT) from the user’s perspective. Random testing can improve test efficiency by automating the process, but achieving good exploration is difficult because it requires uniform distribution over a large search space while also taking into account the dynamic content commonly found in web applications. Reinforcement learning can improve the efficiency of random testing by guiding the generation of test sequences. This is achieved by assigning rewards to specific actions and using them to determine which actions are most likely to lead to a desired outcome. While rewards based on the difference between consecutive states are commonly used in modern tools, they can lead to the Jumping Between States (JBS) problem, where large rewards are generated without significantly increasing exploration. We propose a solution to the JBS problem by combining rewards based on the change of state and a metric to estimate the level of exploration reached in the next state based on the frequency of actions executed. Our results show that this multi-faceted approach increases the exploration efficiency.