Summary

When a system has "grown historically," this often applies to the tests as well: they run for too long but find too few bugs. The reason is that such test suites often simultaneously test too much and too little. They test too much because they include tests that incur costs but provide little added value compared to similar tests. They test too little because important functionality remains untested. In the talk, I will present analyses that uncover this issue: Pareto optimization of test suites and test impact analysis identify the tests that currently have the best cost-benefit ratio. Test gap analysis reveals which code changes are still untested, indicating missing tests. By doing so, these analyses provide us as architects with effective tools to understand and improve our test architecture.