Mutation Testing

Q: What is Mutation Testing?

Measure real test quality with Stryker mutation testing to find gaps that code coverage misses and strengthen your test suite

What is Mutation Testing?

Mutation testing evaluates the quality of your tests by introducing small changes (mutations) to your source code and checking if your tests catch them. If a test fails when the code is mutated, the mutant is "killed" (good). If all tests still pass, the mutant "survived" (your tests have a gap). It answers the question: "Do my tests actually detect bugs?"

Coverage vs Mutation Testing:

100% code coverage means every line runs during tests. But running code is not the same as verifying it. Mutation testing checks whether your assertions would catch actual bugs.

Mutation Operators

Arithmetic Mutations

a + b becomes a - b
a * b becomes a / b
a++ becomes a--

Conditional Mutations

a > b becomes a >= b
a === b becomes a !== b
if (cond) becomes if (true)

Logical Mutations

a && b becomes a || b
!a becomes a
true becomes false

Other Mutations

Remove function calls
Return empty values
Change string literals

Setting Up Stryker

# Install Stryker for JavaScript/TypeScript
npm install --save-dev @stryker-mutator/core \
  @stryker-mutator/jest-runner \
  @stryker-mutator/typescript-checker

# Initialize configuration
npx stryker init

# stryker.config.mjs
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
export default {
  packageManager: 'npm',
  reporters: ['html', 'clear-text', 'progress'],
  testRunner: 'jest',
  checkers: ['typescript'],
  tpiCheckerPlugin: '@stryker-mutator/typescript-checker',
  coverageAnalysis: 'perTest',

  // Target specific files for mutation
  mutate: [
    'src/**/*.ts',
    '!src/**/*.test.ts',
    '!src/**/*.spec.ts',
    '!src/**/*.d.ts',
  ],

  // Thresholds
  thresholds: {
    high: 80,    // Green: 80%+ mutation score
    low: 60,     // Yellow: 60-80%
    break: 50,   // Red/fail: below 50%
  },

  // Performance: limit mutations for faster runs
  concurrency: 4,
  timeoutMS: 10000,
};

# Run mutation testing
npx stryker run

# Run on specific files
npx stryker run --mutate "src/utils/**/*.ts"

Interpreting Results

// Example: This function has 100% code coverage...
export function calculateDiscount(price: number, tier: string): number {
  if (tier === 'gold') return price * 0.8;
  if (tier === 'silver') return price * 0.9;
  return price;
}

// ...but this test has WEAK assertions
test('calculates discount', () => {
  const result = calculateDiscount(100, 'gold');
  expect(result).toBeDefined(); // Passes even if logic is wrong!
});

// Stryker would find surviving mutants:
// Mutant 1: price * 0.8 -> price * 0.2 (SURVIVED - test still passes!)
// Mutant 2: price * 0.9 -> price * 0.1 (SURVIVED!)
// Mutant 3: 'gold' -> '' (SURVIVED!)

// FIXED: Strong assertions kill all mutants
test('gold tier gets 20% discount', () => {
  expect(calculateDiscount(100, 'gold')).toBe(80);
});

test('silver tier gets 10% discount', () => {
  expect(calculateDiscount(100, 'silver')).toBe(90);
});

test('no tier gets no discount', () => {
  expect(calculateDiscount(100, 'basic')).toBe(100);
});

test('handles boundary: zero price', () => {
  expect(calculateDiscount(0, 'gold')).toBe(0);
});

Mutation Score Guide:

80%+: Excellent test quality. Your tests catch most real bugs.
60-80%: Good, but some gaps. Review surviving mutants for weak assertions.
Below 60%: Significant test quality issues. Tests are running code but not verifying it.

Practical Tips

Making Mutation Testing Practical:

Start small: Run on critical business logic, not the entire codebase.
Focus on surviving mutants: Each one reveals a missing or weak assertion.
Not all survivors are bugs: Some mutations are equivalent (produce same result).
Run incrementally: Mutate only changed files in CI to keep build times reasonable.
Pair with code reviews: Use the mutation report to guide test review discussions.

Key Takeaways

Mutation testing measures test effectiveness, not just code coverage
Surviving mutants reveal weak assertions and missing test cases
Use Stryker for JavaScript/TypeScript projects with Jest or Vitest
Aim for 80%+ mutation score on critical business logic
Run incrementally on changed files to keep CI fast

What is Mutation Testing?

Mutation Operators

Arithmetic Mutations

Conditional Mutations

Logical Mutations

Other Mutations

Setting Up Stryker

Interpreting Results

Mutation Score Guide:

Practical Tips

Making Mutation Testing Practical:

Key Takeaways

Continue Learning

React

Node.js

DevTools & Productivity

Engineering Leadership

Python