How to Eliminate Flaky Tests: A Practical Guide for QA Engineers

Flaky tests are the silent killers of confidence. One minute they pass, the next they explode for no clear reason, and you’re left wondering if the product is broken or just your test suite. In today’s fast‑paced release cycles, that uncertainty can cost time, money, and morale. Let’s cut through the noise and get those tests behaving.

What Makes a Test Flaky?

Before we can fix anything, we need to know why a test is flaky. In my early days at a startup, I spent more time chasing ghost failures than writing new tests. Here’s what I learned – flaky tests usually fall into three buckets:

1. Timing Issues

Tests that depend on exact timing are the most common culprits. Think of a UI test that clicks a button right after a page loads. If the page takes a fraction of a second longer, the click lands on the wrong element and the test fails.

2. External Dependencies

Anything that reaches out to a service you don’t control – a third‑party API, a database, a file system – can introduce randomness. If the service is slow or returns a different response, the test outcome changes.

3. Shared State

When two tests run in parallel and both touch the same data, they can step on each other’s toes. A test that deletes a record might cause another test that expects that record to exist to fail.

Step‑by‑Step: Taming Flaky Tests

Below is a practical checklist I use on every project. Pick the items that fit your context and run with them.

Step 1: Identify the Flaky Tests

Run your suite multiple times in a row. A simple script that executes the tests 10–20 times and logs failures is enough. If a test fails more than 20 % of the runs, flag it as flaky.

Pro tip: In our team at Testing Insights we keep a “flaky‑watch” folder. Any test that shows up there gets a “do not merge” label until it’s fixed.

Step 2: Add Logging and Screenshots

When a test fails, you need context. Add logs that print out the state of the application, the values of key variables, and timestamps. For UI tests, capture a screenshot at the moment of failure. This extra data often points straight to the root cause.

Step 3: Stabilize Timing

Explicit Waits: Replace generic sleep statements with waits that look for a specific condition – e.g., “element is visible” or “API response received”. Most test frameworks have built‑in wait utilities.
Retry Logic: For actions that can legitimately take a bit longer, wrap them in a small retry loop (max 3 attempts). This is a safety net, not a cure – if you need retries everywhere, you probably have a deeper timing problem.

Step 4: Mock or Stub External Services

If your test talks to a payment gateway, replace that call with a mock that returns a predictable response. Tools like WireMock or simple in‑code stubs work well. The goal is to make the test independent of the outside world.

Step 5: Isolate Test Data

Unique Test Data: Generate unique identifiers (GUIDs, timestamps) for each test run. This prevents two tests from trying to create the same record.
Database Transactions: Wrap each test in a transaction and roll it back at the end. That way the database returns to a clean state no matter what.
Parallel Execution Guardrails: If you must run tests in parallel, assign each worker its own sandbox – a separate database schema or a separate folder for file writes.

Step 6: Refactor the Test Itself

Sometimes the test is trying to do too much. A test that logs in, creates a user, uploads a file, and then checks a report is a recipe for flakiness. Break it into smaller, focused tests. Each test should verify one behavior.

Step 7: Review the Test Environment

Consistent Configurations: Ensure every developer’s machine, CI runner, and staging server use the same versions of browsers, drivers, and libraries.
Resource Limits: Low memory or CPU can cause timeouts that look like flaky failures. Monitor resource usage during test runs and adjust the environment if needed.

Step 8: Automate Flake Detection

Add a nightly job that runs the flaky‑watch suite. If a test that was previously stable starts failing intermittently, the job should raise an alert. Early detection prevents the flake from spreading into the main build.

Real‑World Example: Fixing a Flaky Login Test

At Testing Insights we once had a login test that failed randomly on Chrome CI runners. The failure log showed a “ElementNotInteractable” error. Here’s what we did:

Added an explicit wait for the username field to become visible.
Removed a hard‑coded sleep of 2 seconds that was meant to give the page time to load.
Mocked the authentication API in the test environment, returning a static token.
Generated a unique email address for each run, avoiding conflicts with previous runs.

After these changes, the test passed 100 % of the time across 30 consecutive runs. The lesson? A flaky test is rarely a single problem; it’s a chain of small issues that add up.

When to Accept Flakiness (and When Not To)

I get asked if it’s ever okay to leave a flaky test in the suite. My answer is simple: only if the test adds no real value. If the test is covering a critical path, you must fix it. If it’s a “nice‑to‑have” sanity check that rarely catches bugs, consider removing it or marking it as “manual only”.

Quick Checklist for Your Next Sprint

[ ] Run the suite 10× and log failures.
[ ] Add logs and screenshots to failing tests.
[ ] Replace sleeps with explicit waits.
[ ] Mock all external calls.
[ ] Use unique data per test.
[ ] Wrap DB changes in transactions.
[ ] Keep environment versions in sync.
[ ] Set up a nightly flake detector.

Flaky tests don’t have to be a permanent headache. With a systematic approach, you can turn a shaky suite into a reliable safety net that lets you ship faster and with confidence. Happy testing!