Design Principles for Stable E2E Tests

You’ve heard it before: E2E tests are unreliable. They’re flaky. They can’t be trusted.

Is that true? Unfortunately, yes. But the problem isn’t inherent to E2E testing itself. The problem is usually how the tests are written. Generating an automated test is easy. Making it fail only when there’s a real issue, and not randomly, is an art. It’s knowledge you acquire through long, sometimes painful experience fighting the system.

Here are the principles from my arsenal that have proven most valuable.

1. Prefer stable locators

Tests that rely on CSS classes, element positions, or auto-generated IDs break the moment a designer tweaks a stylesheet or a developer refactors a component. Instead, lean on locators that reflect what the element is rather than how it looks.

Playwright’s built-in locators make this straightforward:

// Fragile: breaks when class names change
await page.locator('.submit-btn.primary').click();

// Stable: targets by semantic role and visible text
await page.getByRole('button', { name: 'Submit' }).click();

// Stable: explicit test ID, unaffected by UI changes
await page.getByTestId('submit-button').click();

The getByRole, getByLabel, and getByTestId family of locators survive refactors because they’re tied to meaning, not structure. If your app doesn’t use data-testid attributes yet, adding them in the components you test is a worthwhile investment.

2. Mock external dependencies you don’t control

Most real-world applications have external dependencies: third-party webhooks, payment providers, notification services. In a test environment, these are unreliable by definition. A webhook might be delayed, arrive out of order, or never come at all.

The solution is to mock what you don’t own, so you’re testing your application, not the reliability of a third-party service.

In Playwright, you can intercept and fulfill network requests:

test('shows payment confirmation after webhook', async ({ page }) => {
  // Intercept the webhook relay endpoint and respond immediately
  await page.route('**/api/webhooks/payment', route => {
    route.fulfill({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify({ event: 'payment.completed', orderId: 'ord_123' })
    });
  });

  await page.goto('/checkout');
  await page.getByRole('button', { name: 'Pay Now' }).click();

  await expect(page.getByText('Payment confirmed')).toBeVisible();
});

One important caveat: mocking a dependency means you’re no longer testing the actual integration. Don’t treat the mock as a substitute. Pair it with a non-UI integration test that exercises the real webhook path against your backend. The E2E test verifies the UI reacts correctly; the integration test verifies the plumbing is wired up.

3. Keep the test environment controlled

If your tests run in a shared or deployed environment, any deployment, background job, or performance issue that happens during a test run is a potential source of failure that has nothing to do with your test.

The ideal setup is a dedicated, stable environment for automated tests, one that doesn’t receive deployments mid-run and isn’t competing for resources with other workloads. This is easier said than done, and what’s achievable depends heavily on your infrastructure. Treat this as a guiding direction rather than a hard rule, and adjust based on your actual constraints.

4. Wait for network responses, not arbitrary time

page.waitForTimeout(2000) is a red flag. Fixed delays are either too short (causing intermittent failures on slow runs) or too long (needlessly slowing down your suite). Neither is good.

Instead, wait for the specific condition that signals the UI is ready to interact with:

test('displays order summary after submission', async ({ page }) => {
  await page.goto('/orders/new');
  await page.getByLabel('Item').fill('Widget');

  // Start waiting before the action that triggers the request
  const orderResponse = page.waitForResponse('**/api/orders');
  await page.getByRole('button', { name: 'Place Order' }).click();

  // Wait for the actual response, then proceed
  const response = await orderResponse;
  expect(response.status()).toBe(201);

  await expect(page.getByRole('heading', { name: 'Order Summary' })).toBeVisible();
});

Playwright also provides waitForLoadState, waitForURL, and waitForSelector for other common wait conditions. Use the one that matches what your UI actually does.

5. Create and control your test data

Tests that depend on pre-existing data in a shared database are asking for trouble. Another test, a manual operation, or a data migration can silently change the state your test depends on.

Where possible, create the exact data your test needs before it runs, and clean it up afterward. Playwright’s request fixture lets you call your API directly in beforeEach:

test.describe('Account transfer', () => {
  let sourceAccountId: string;
  let targetAccountId: string;

  test.beforeEach(async ({ request }) => {
    const source = await request.post('/api/accounts', {
      data: { name: 'Source', balance: 1000 }
    });
    const target = await request.post('/api/accounts', {
      data: { name: 'Target', balance: 0 }
    });
    sourceAccountId = (await source.json()).id;
    targetAccountId = (await target.json()).id;
  });

  test.afterEach(async ({ request }) => {
    await request.delete(`/api/accounts/${sourceAccountId}`);
    await request.delete(`/api/accounts/${targetAccountId}`);
  });

  test('transfers funds between accounts', async ({ page }) => {
    await page.goto(`/accounts/${sourceAccountId}/transfer`);
    await page.getByLabel('To Account').fill(targetAccountId);
    await page.getByLabel('Amount').fill('250');
    await page.getByRole('button', { name: 'Transfer' }).click();

    await expect(page.getByText('Transfer complete')).toBeVisible();
  });
});

If direct data creation isn’t possible, use pre-seeded deterministic data that other tests are explicitly prohibited from modifying.

6. Write short, focused tests

A test that reads like a novel has many reasons to fail at any intermediate step, making it hard to tell whether the failure is a genuine regression or just noise from an unrelated setup step.

This point might seem to conflict with point 5. If you’re testing an account transfer, doesn’t setting up two accounts make the test long? It does, and that tension is real. The resolution is to push setup out of band (via API calls or pre-seeded data, as above) so the test body focuses only on the behaviour under test.

If the setup itself is complex and brittle, consider mocking the API responses to simulate existing accounts. Yes, this makes the test less “end-to-end” in the purest sense. That’s a deliberate tradeoff: you split the verification responsibility across layers. The E2E test verifies the transfer UI flow; a backend integration test verifies the transfer actually processes. This is just the testing pyramid in practice. Keeping it in mind is the single most useful mental model for deciding how to structure your automation.

7. Flakiness can be a symptom, not just a problem

Sometimes a test flakes not because it’s poorly written, but because it’s exposing a real defect in the application.

Consider a button that’s momentarily enabled while a validation check is running, then immediately disabled again. A real user won’t notice that half-second window. But an automated test moving at machine speed might click the button in that window and then find itself in an unexpected state.

The instinct is to add a retry or a wait to work around it. Resist that. If your investigation points to an application bug, fix the application. Hiding a real defect behind a test workaround makes it invisible while it continues affecting users.

8. Annotate known flakiness, sparingly

Despite all of the above, some flakiness is irreducible in the short term. Maybe the underlying cause is a known architectural issue that’s on the roadmap to fix, but not yet.

In those cases, surface the known issue in the assertion message itself so that when the test fails, the output tells you exactly why it might be flaky:

test('syncs inventory after supplier update', async ({ page }) => {
  await page.goto('/inventory');
  await page.getByRole('button', { name: 'Sync Now' }).click();

  await expect(
    page.getByText('Inventory up to date'),
    'Known flaky in staging when supplier webhook is delayed. See issue #847.'
  ).toBeVisible();
});

When this fails, Playwright prints the message alongside the error, making it immediately clear that the failure may be a known environmental issue rather than a regression. Far more useful than a comment that nobody reads during a red CI run.

Use this technique sparingly. If it becomes a habit, annotations lose their meaning and become a way to ignore problems rather than track them. Your goal is always to eliminate flakiness, not to document its existence.

On AI-powered test self-healing

There’s a category of tooling emerging that uses LLMs to automatically repair failing tests when locators change. The idea is appealing, but I’m skeptical of it as a primary strategy.

You’re introducing a non-deterministic layer into a system whose entire value proposition is determinism. Every run now has an additional failure mode: the LLM heals the test in a way that masks a real regression rather than surfacing it. On top of that, you’re adding latency and token costs to every CI run, and E2E tests are already among the most expensive parts of a pipeline.

A better use of AI here is in the debugging loop, not the execution loop. When a test fails, feed the failure logs, the relevant component code, the backend architecture, and any application logs captured during the test run to a model and ask it to suggest what’s wrong. That’s a one-time investigation cost with high return, not a recurring per-run overhead.

Closing thoughts

There is no single formula for a stable test automation framework. The right approach depends on your application’s architecture, your team’s capacity, and your risk tolerance. Sometimes flakiness points to a bad test. Sometimes it points to an inherent weakness in the application itself.

What doesn’t change: it is the responsibility of the test automation engineer to keep flakiness under control. When the team can’t trust the pipeline, the pipeline stops being an asset and becomes noise. All that CI/CD infrastructure, all those engineering hours, wasted.

What are your stories? How are you fighting your flaky test monsters?