Test Automation
Flaky Test
A test that sometimes passes and sometimes fails without any code changes — unreliable and harmful to trust in the test suite.
Full definition
A flaky test is an automated test that produces inconsistent results — passing sometimes and failing other times with no code changes. Flaky tests erode confidence in the test suite and slow down development.
Common causes:
- Timing issues: Test doesn't wait long enough for async operations
- Test order dependency: Test relies on another test running first
- Shared state: Tests modify shared data without cleanup
- Environment differences: Works locally, fails in CI
- Race conditions: Parallel execution causes conflicts
- External dependencies: Network calls, third-party APIs
- Animation/transition timing: UI tests clicking before elements are ready
How to fix flaky tests:
- 1.Identify: Track tests that intermittently fail
- 2.Quarantine: Move flaky tests out of the main suite
- 3.Investigate: Reproduce and find the root cause
- 4.Fix: Usually involves better waits, test isolation, or mocking
- 5.Verify: Run the test 100 times to confirm the fix
Prevention:
- Use explicit waits (not sleep)
- Make tests independent (own data, own cleanup)
- Mock external dependencies
- Avoid time-dependent assertions