LLM-Empowered Scriptless Functional Testing

Scriptless testing generates test sequences dynamically by automatically exploring the Graphical User Interface (GUI). Instead of relying on predefined scripts-which have proven expensive to maintain-scriptless tools detect available widgets, derive possible actions, and select actions on the fly using exploratory techniques such as random selection, model-based inference, or reinforcement learning. While scriptless testing is a valuable complement to scripted approaches, current techniques lack the intelligence needed to strategically select and execute GUI actions that fulfill specific functional testing goals-such as those derived from requirements, use cases, or user stories. Unsurprisingly, this leads companies to question the viability of scriptless testing tools and continue relying on scripts for test automation. This paper reports on the integration of Large Language Models (LLMs) into a scriptless GUI testing tool for action selection, aiming to determine whether it can generate effective action sequences to test specific functional requirements. Our results demonstrate that a multi-objective test goal structure, combined with historical and feedback context, enables LLM-empowered scriptless testing to automate functional testing. Although further research is needed to tackle challenges in complex test scenarios, our findings offer promising results that LLM-empowered scriptless testing can reduce reliance on the maintenance-heavy aspects of traditional scripted testing.