As our software grows, we face the challenge of testing an increasing amount of functionality. This leads to more and more automated tests. Many teams I collaborate with manage thousands, or even hundreds of thousands, of automated tests. To identify bugs early, these tests are executed frequently, often many times each day. Typically, all of them are executed in every single test run.
While this approach reduces feedback times and helps in early bug detection, it comes with a cost: the more tests we execute, the greater the energy consumption, compute cost and waiting times.

We can do better: The more frequently we run our tests, the less beneficial each individual test execution becomes, as the likelihood of uncovering new bugs diminishes. For instance, if no change was made to the code a test executes since a test’s last execution, it cannot find a new bug. Therefore, it is unnecessary to execute all the tests all the time.

Test selection approaches automatically identify those tests that have the highest probability of discovering new bugs at any given time. By running these tests more frequently than the other tests, we can save a lot of effort and still find the bugs in our software.

For the past decade, I’ve devoted my work to test selection in both research and practical applications. Our team has developed and tested various test selection strategies within our own development processes and those of our clients. In this talk, I will share our successes and setbacks, and discuss the true impact of these strategies on making test execution more sustainable.