In addition to UI, where you can visually analyze results of mutation analysis,
there is a possibility to export them to external files. Most comprehensive format
is
Raw XML, which contains all analysis results for
future investigation, but probably will require additional post-processing.
Two other formats (
html and
Excel-compatible)
represents test-case dependency table. It is squared table, which represent
how many mutations was killed by given combination of test cases. On the main diagonal
you can see the number of mutations that were killed by the given test case only.
The last column and the last row in the table are total values.
Unit-test effectiveness checklist
- Check all items, that has the same value in total and in one of the columns. For instance,
you can see in this sample that BagNegative() <2> killed 21
mutations total, and the same mutations were also killed by MixedSimpleAdd() <7>. It
does not necessary means that we should eliminate BagNegative(), because MixedSimpleAdd()
supersede it, but it definitely looks like a good candidates for refactoring.
- Check all items, that have exceptionally low score of killed mutations. You should clearly
understand what was the reason to write test case with such a small coverage. Obviously the problem
with having too small tests would be increased size of unit testing code and thus more
expensive maintenance.
- Check all items, that have exceptionally high total score. It might be that this case really testing
more than one unit. Analyzing failures of such tests can be difficult.
Overall, remember that unit tests are not just regression tools - they are also used as a live
documentation of your main code. If I would have to use one of your classes, first thing I would
do is to look for the test case that uses it.