Many teams have to test more and more functionality in less and less time. Historically grown test suites often don't cope well with this, since they test both too much and too little. Too much, since they contain tests that are expensive to execute (and maintain) but add little value over similar tests in the same test suite. Too little, because important functionality remains untested.

In the talk, I will present analysis techniques that reveal such problems in your own test suites:

• Test gap analysis shows, which code changes are still untested.
• Test impact analysis and pareto-optimization of test suites identify tests that can find new bugs faster than other tests in the suite.

All analyses work on data from the system under test, such as change information from the version control system and test-wise coverage from the automated tests.

For each analysis, I outline the underlying research and present examples from practice. For some analyses, I use the Robot Framework itself as example, for others I use customer system or data from our own software development process.

Slides