Evaluate The Quality of Your Test Cases With Mutation Testing

How good are your test cases? Maybe they are good, or maybe you need to add some new ones.

How to tell? You can measure things like code coverage, e.g. to check if some parts of your code were not executed.

Still, this does not tell you anything about the quality of your assertions.

In the extreme case, a test suite with no assertions might still achieve 100% code coverage, although being of questionable value.

Mutation testing is a technique that is used to automatically inject faults in your code, and then check if your test cases can “kill” those “mutants”.

In this way, the quality and effectiveness of your assertions can be evaluated.

In this talk, I will discuss the benefits and challenges of mutation testing, and also show open source tools like PIT.