Most bugs occur in recently modified code. Therefore, as testers or test automators, we generally try to focus our efforts on these areas. However, our research shows that, in practice, a significant proportion of code changes are released untested, leading to the majority of problems down the line.
This even occurs, if advanced test planning procedures are in place or if large automated test suites are executed periodically. The primary issue is that we lack visibility into what exactly has been changed, and whether our tests managed to cover these changes thoroughly enough.
In this presentation, I introduce Test Gap Analysis, an innovative solution that combines static and dynamic analyses to uncover these untested changes. Test Gap Analysis is a highly flexible tool usable in various situations. For example, it allows us to compare different test suites (e.g. unit tests and integration tests) of a system in terms of coverage. It thus helps us to decide, how we can best close remaining test gaps. I will share my hands-on experiences of using Test Gap Analysis in practice across a broad range of teams and domains. Moreover, I will discuss best practices for improving hotfix-tests, continuous integration, and strategic test planning.
Key takeaways: Most bugs in production are caused by code changes that slipped through testing without being executed. Test gap analysis reveals untested code changes before release, for any types of tests. In more detail: Software development & testing is a complex process with many moving parts. Even very mature organizations ship untested code. Test Gap Analysis can show us where those untested changes are, before we release our software. How to employ Test Gap Analysis for manual and automated tests and what its limitations are. On the level of single tickets, test gaps are often a catalyzer for discussions between testers and developers. We can have much more transparency about our work (both development and testing) than we currently do have.
Session recommended for: Testers, Test Managers, Developer, Architects