Mutation testing

Mutation testing evaluates test quality by deliberately introducing small code changes ("mutations") and seeing if your tests catch them. 

It's essentially the process of systematically and purposely planting bugs to check if your warning systems work. When tests fail after a mutation, it confirms they're actually verifying the code's behavior, not just going through motions.

A developer might change if (balance > 0) to if (balance >= 0) or flip a plus to minus (total = price + tax becomes total = price - tax). These tiny changes should trigger test failures. Tools like PIT for Java or Stryker for JavaScript automate this process by generating hundreds of these mutations and tracking which ones your tests catch.

Mutation testing reveals blind spots in your test suite that test coverage metrics miss. You might have 100% line coverage but still miss critical bugs if your assertions aren't thorough. It helps teams write meaningful tests rather than superficial ones. 

For testers specifically, it provides concrete evidence of test effectiveness and helps prioritize which areas need better testing.

Mutation testing is computationally expensive—running your entire test suite against hundreds of code variations takes time. Some mutations create equivalent code that behaves identically (making them impossible to detect), while others introduce unrealistic bugs. Most teams run mutation testing periodically rather than in every build. Modern tools like Pitest help by parallelizing tests and using smart sampling to focus on the most valuable mutations.

I see mutation testing as a process for thoroughly testing our tests to ensure that we can really rely on them. My recommendation is to do mutation testing every time you create or edit a test. You should also review your automated tests making sure it is still relevant. You should do this regularly. You'll need to make sure that tests keeps up to date
with changes to the software. 

My process for mutation testing involves going through all the steps of a test and listing all the ways each individual step could fail. I then attempt to simulate these failures by either changing the state of the software before the test is run, running the test in debug mode, and then pausing the test using breakpoints to try and simulate a failure, changing the software at that point, or adjusting the test code itself to check the accuracy. Another reason I find mutation testing useful is because it allows me to design the tests to give useful messages if certain failures have occurred.

Mutation testing is a method used in the context of automated checks / tests.  Rather than testing the code, mutation testing tests the tests.

The purpose of automated checks is to detect change.  With mutation testing, one introduces small changes to the code to see whether they are successfully detected.  If they are, the mutation is said to be "killed", but if they are not, then one may have a gap in the coverage of automated checks*.

One can also use what I call "reverse mutation testing" - changing the test so that it should fail, even though the system under test has not.  This reveals whether the test can actually fail, and whether you are really testing what you think you're testing.

*Side note: A gap in test coverage may be intentional.  It's important to think carefully about one's test strategy, whether 100% code coverage is desired, and on which level(s).