Learn how I used mutation testing to introduce hundreds of bugs into the Robot Framework’s own code and how I applied an AI-based testing approach to the Robot Framework’s test suite to find ⅔ of these bugs in ¼ of the time.
Running tests as often and as early as possible is the dream of many agile testers. Ideally, after every commit and on all branches, so that we immediately notice when we break something.
But what if my tests take hours or even days? For many, the dream of an accelerated testing process seems unattainable or at least impractical.
However, research shows a possible solution: One approach to providing quick feedback even with slow tests is to run a small subset that is fast enough. This is worthwhile if this subset finds a majority of the defects in a fraction of the time. For example, 80% of defects in 10% of the time it takes to execute all tests. We need innovative methods to accomplish this, but they also need to be practically feasible.
In this presentation, I introduce an approach that can be implemented with little effort in existing projects to uncover most defects with minimal testing effort and without changing anything about your tests!
The method uses large language models (AI) and clustering to create an effective smoke test suite. This can be used for arbitrary changes, to identify defects across the entire code base with minimal testing effort. Thus, providing quick feedback on new bugs.
I’ll present the fundamentals, explain how it works and show research results about the effectiveness of the technique.