# High-Quality, Low-Flake Playwright Testing Guide

This guide defines how we write and evolve Playwright tests in bitvid so they stay reliable while increasing coverage.

## Goals

1. Catch user-visible regressions early.
2. Keep CI signal trustworthy (low flake rate).
3. Improve runtime coverage with deterministic tests, not brittle ones.

## Core Principles

1. Test behavior, not implementation details.
2. Prefer deterministic setup over live-network dependence.
3. Wait for explicit app state, not arbitrary sleep.
4. Use resilient selectors and scoped locators.
5. Keep assertions strict on outcomes, flexible on timing and non-critical internals.

## Determinism First

Use the existing harness and fixture stack:

- `tests/e2e/helpers/bitvidTestFixture.ts`
- `window.__bitvidTest__` APIs in `js/testHarness.js`

Prefer these patterns:

1. Use seeded relay data instead of remote relays.
2. Use programmatic login (`loginAs`) instead of manual UI auth for most tests.
3. Use harness APIs to force edge conditions (`setSignerDecryptBehavior`, relay overrides).
4. Open complex modals via deterministic controller/harness methods when available.

Why: nondeterministic setup is the top source of flaky tests.

## Selector Strategy

1. Prefer `data-testid` selectors.
2. Scope locators to visible/active containers when duplicates can exist.
3. Avoid broad selectors like `button` unless nested under a stable root.

Example:

- Better: `#playerModal #closeModal:visible`
- Worse: `#closeModal` (can resolve hidden duplicate elements)

## Waiting Strategy (Do This, Not That)

Do:

1. Wait for explicit state changes (`data-open="true"`, absence of `.hidden`, expected text/attribute).
2. Use `waitForFunction` only with concrete conditions and clear timeouts.
3. Add fallback action paths for known intermittent UI event misses when justified.

Avoid:

1. `waitForTimeout(...)` as primary synchronization.
2. Assuming a click always opens a modal immediately.
3. Chained interactions without state confirmation between steps.

## Modal Testing Pattern

Recommended flow:

1. Trigger modal open via deterministic helper (harness/controller), fallback to UI click.
2. Wait for modal-open condition.
3. Assert visible + semantic state (`role`, `aria-modal`, `data-open`).
4. Interact with modal.
5. Close modal and verify closed state.

If your tests repeatedly fail opening the same modal, add a harness helper for that modal instead of adding sleeps.

## Accessibility Test Stability

Accessibility tests are especially prone to timing flake. For keyboard and modal tests:

1. Ensure focus target exists and is visible before key presses.
2. Prefer element-scoped key actions (`locator.press("Enter")`) over global keyboard when possible.
3. Verify modal opened before asserting internals.
4. For Escape-close tests, first confirm modal open state deterministically.

## Coverage Without Brittle UI Tests

Not all coverage gains require full UI choreography.

Use runtime module tests in browser context for pure/mostly-pure modules:

1. `page.evaluate(async () => import("/js/module.js"))`
2. Execute exported APIs with deterministic inputs.
3. Assert normalized outputs and edge cases.

This is effective for utilities, state transforms, and view-logic modules that are hard to reach through full flows.

## Anti-Patterns to Avoid

1. Duplicate clicks on close/open controls “just in case.”
2. Assertions that only check element attachment but not visible state when visibility matters.
3. Overly permissive assertions (`expect(x || true).toBe(true)`) unless documented as intentional.
4. Tests that rely on console noise rather than asserting intended state transitions.

## Flake Triage Workflow

When a test flakes:

1. Reproduce with repeats:
- `npx playwright test <spec> --project=e2e --repeat-each=5`
2. Identify failure type:
- setup nondeterminism
- selector ambiguity
- missing state wait
- race in app init
3. Fix in this order:
- add deterministic harness/controller hook
- tighten selectors
- add explicit state waits
- only then consider timeout tuning
4. Re-run repeats and then run coverage lane.

## Quality Gates for Sustainable Improvement

1. Keep `npm run test:e2e:coverage` green before raising coverage thresholds.
2. Track coverage trend, but do not trade reliability for raw percentage.
3. Ratchet coverage targets gradually (example: 40% -> 45% -> 50%).
4. Require flaky-test fixes to include deterministic synchronization changes, not just longer timeouts.

## PR Checklist for E2E Changes

1. Does this test use deterministic fixture/harness setup?
2. Are selectors stable (`data-testid` or scoped stable IDs)?
3. Are waits condition-based (not sleep-based)?
4. Does the test fail for real regressions and pass reliably under repeats?
5. If adding coverage tests, do they increase useful coverage in high-risk paths?

## Useful Commands

1. Single spec:
- `npx playwright test tests/e2e/<spec>.spec.ts --project=e2e`
2. Repeat to detect flake:
- `npx playwright test tests/e2e/<spec>.spec.ts --project=e2e --repeat-each=5`
3. Full deterministic E2E coverage run:
- `npm run test:e2e:coverage`
4. Regenerate report from existing raw artifacts:
- `npm run test:e2e:coverage:report`

## Related Docs

- `docs/testing/playwright-function-coverage.md`
- `docs/testing/playwright-integration-recommendations-2026-02-22.md`
- `AGENTS.md` (testing infrastructure and harness references)