Testing Fundamentals

What Is Flaky Test?

A flaky test is an automated test that produces inconsistent pass/fail results across multiple runs without any changes to the source code or test itself.

Start a Free Mock Interview

Free to start · 7-day trial on paid plans

In Depth

Flaky tests are one of the most insidious problems in test automation. They erode team confidence in the test suite because engineers stop trusting failures as genuine signals. When a test passes on one run and fails on the next with identical code, the usual culprits are race conditions, shared mutable state between tests, dependency on external services, non-deterministic data ordering, or timing-sensitive assertions that do not account for asynchronous behavior.

The real cost of flakiness is not the debugging time for a single failure but the cultural damage. Teams start reflexively re-running pipelines instead of investigating failures, which masks real regressions. In large organizations, flaky tests can waste hundreds of compute hours per week and significantly slow release velocity.

Effective strategies include quarantining known flaky tests into a separate suite, adding retry annotations with investigation tickets, using deterministic waits instead of arbitrary sleeps, isolating test data per run, and tracking flakiness rates over time with tools like a test analytics dashboard.

Why Interviewers Ask About This

Interviewers ask about flaky tests to gauge whether you have real-world automation experience. Anyone can write a happy-path test; handling flakiness demonstrates you understand the operational side of test infrastructure.

Example Scenario

A Selenium test checks that a toast notification appears after form submission. It passes locally but fails in CI 30% of the time because the CI environment is slower and the toast animation has not completed before the assertion fires. The fix is replacing a hardcoded sleep with an explicit wait that polls for the toast element to be visible.

Interview Tip

When discussing flaky tests, describe a specific flaky test you triaged. Walk through the symptoms, your debugging process, and the root cause. Interviewers value the investigation story more than the fix itself.