update

dist/docs/testing/high-quality-playwright-tests.md

@@ -0,0 +1,148 @@
+# High-Quality, Low-Flake Playwright Testing Guide
+
+This guide defines how we write and evolve Playwright tests in bitvid so they stay reliable while increasing coverage.
+
+## Goals
+
+1. Catch user-visible regressions early.
+2. Keep CI signal trustworthy (low flake rate).
+3. Improve runtime coverage with deterministic tests, not brittle ones.
+
+## Core Principles
+
+1. Test behavior, not implementation details.
+2. Prefer deterministic setup over live-network dependence.
+3. Wait for explicit app state, not arbitrary sleep.
+4. Use resilient selectors and scoped locators.
+5. Keep assertions strict on outcomes, flexible on timing and non-critical internals.
+
+## Determinism First
+
+Use the existing harness and fixture stack:
+
+- `tests/e2e/helpers/bitvidTestFixture.ts`
+- `window.__bitvidTest__` APIs in `js/testHarness.js`
+
+Prefer these patterns:
+
+1. Use seeded relay data instead of remote relays.
+2. Use programmatic login (`loginAs`) instead of manual UI auth for most tests.
+3. Use harness APIs to force edge conditions (`setSignerDecryptBehavior`, relay overrides).
+4. Open complex modals via deterministic controller/harness methods when available.
+
+Why: nondeterministic setup is the top source of flaky tests.
+
+## Selector Strategy
+
+1. Prefer `data-testid` selectors.
+2. Scope locators to visible/active containers when duplicates can exist.
+3. Avoid broad selectors like `button` unless nested under a stable root.
+
+Example:
+
+- Better: `#playerModal #closeModal:visible`
+- Worse: `#closeModal` (can resolve hidden duplicate elements)
+
+## Waiting Strategy (Do This, Not That)
+
+Do:
+
+1. Wait for explicit state changes (`data-open="true"`, absence of `.hidden`, expected text/attribute).
+2. Use `waitForFunction` only with concrete conditions and clear timeouts.
+3. Add fallback action paths for known intermittent UI event misses when justified.
+
+Avoid:
+
+1. `waitForTimeout(...)` as primary synchronization.
+2. Assuming a click always opens a modal immediately.
+3. Chained interactions without state confirmation between steps.
+
+## Modal Testing Pattern
+
+Recommended flow:
+
+1. Trigger modal open via deterministic helper (harness/controller), fallback to UI click.
+2. Wait for modal-open condition.
+3. Assert visible + semantic state (`role`, `aria-modal`, `data-open`).
+4. Interact with modal.
+5. Close modal and verify closed state.
+
+If your tests repeatedly fail opening the same modal, add a harness helper for that modal instead of adding sleeps.
+
+## Accessibility Test Stability
+
+Accessibility tests are especially prone to timing flake. For keyboard and modal tests:
+
+1. Ensure focus target exists and is visible before key presses.
+2. Prefer element-scoped key actions (`locator.press("Enter")`) over global keyboard when possible.
+3. Verify modal opened before asserting internals.
+4. For Escape-close tests, first confirm modal open state deterministically.
+
+## Coverage Without Brittle UI Tests
+
+Not all coverage gains require full UI choreography.
+
+Use runtime module tests in browser context for pure/mostly-pure modules:
+
+1. `page.evaluate(async () => import("/js/module.js"))`
+2. Execute exported APIs with deterministic inputs.
+3. Assert normalized outputs and edge cases.
+
+This is effective for utilities, state transforms, and view-logic modules that are hard to reach through full flows.
+
+## Anti-Patterns to Avoid
+
+1. Duplicate clicks on close/open controls “just in case.”
+2. Assertions that only check element attachment but not visible state when visibility matters.
+3. Overly permissive assertions (`expect(x || true).toBe(true)`) unless documented as intentional.
+4. Tests that rely on console noise rather than asserting intended state transitions.
+
+## Flake Triage Workflow
+
+When a test flakes:
+
+1. Reproduce with repeats:
+- `npx playwright test <spec> --project=e2e --repeat-each=5`
+2. Identify failure type:
+- setup nondeterminism
+- selector ambiguity
+- missing state wait
+- race in app init
+3. Fix in this order:
+- add deterministic harness/controller hook
+- tighten selectors
+- add explicit state waits
+- only then consider timeout tuning
+4. Re-run repeats and then run coverage lane.
+
+## Quality Gates for Sustainable Improvement
+
+1. Keep `npm run test:e2e:coverage` green before raising coverage thresholds.
+2. Track coverage trend, but do not trade reliability for raw percentage.
+3. Ratchet coverage targets gradually (example: 40% -> 45% -> 50%).
+4. Require flaky-test fixes to include deterministic synchronization changes, not just longer timeouts.
+
+## PR Checklist for E2E Changes
+
+1. Does this test use deterministic fixture/harness setup?
+2. Are selectors stable (`data-testid` or scoped stable IDs)?
+3. Are waits condition-based (not sleep-based)?
+4. Does the test fail for real regressions and pass reliably under repeats?
+5. If adding coverage tests, do they increase useful coverage in high-risk paths?
+
+## Useful Commands
+
+1. Single spec:
+- `npx playwright test tests/e2e/<spec>.spec.ts --project=e2e`
+2. Repeat to detect flake:
+- `npx playwright test tests/e2e/<spec>.spec.ts --project=e2e --repeat-each=5`
+3. Full deterministic E2E coverage run:
+- `npm run test:e2e:coverage`
+4. Regenerate report from existing raw artifacts:
+- `npm run test:e2e:coverage:report`
+
+## Related Docs
+
+- `docs/testing/playwright-function-coverage.md`
+- `docs/testing/playwright-integration-recommendations-2026-02-22.md`
+- `AGENTS.md` (testing infrastructure and harness references)

dist/docs/testing/playwright-function-coverage.md

@@ -0,0 +1,90 @@
+# Playwright Function Coverage + Console Diagnostics
+
+Use this workflow to measure how much of bitvid's runtime code is exercised by E2E Playwright tests and to export browser console/page errors for triage.
+
+## Run
+
+```bash
+npm run test:e2e:coverage
+```
+
+This command:
+
+1. Builds `dist/`
+2. Runs `tests/e2e` on Chromium with browser JS coverage enabled
+3. Captures browser `console` and `pageerror` output for every test
+4. Generates aggregate reports
+
+## Artifacts
+
+- `artifacts/playwright-coverage/raw/*.json`
+  - Per-test raw coverage + console/page-error logs
+- `artifacts/playwright-coverage/function-coverage-summary.json`
+  - Machine-readable function coverage report
+- `artifacts/playwright-coverage/function-coverage-summary.md`
+  - Human-readable summary with coverage percentage and low-coverage files
+- `artifacts/playwright-coverage/console-log-summary.json`
+  - Full exported console/page-error logs
+- `artifacts/playwright-coverage/console-issues.log`
+  - Grouped issue signals for fast triage
+
+## Notes
+
+- Coverage is captured from Chromium runtime execution (`page.coverage`), then combined with static function estimates for source files in `js/` and `torrent/`.
+- Firefox coverage is not included in this report because Playwright JS coverage is Chromium-only.
+- For reliability patterns while increasing coverage, see `docs/testing/high-quality-playwright-tests.md`.
+
+## Current Baseline (2026-02-22)
+
+From a full real run (`npm run test:e2e:coverage`) on 2026-02-22:
+
+- E2E result: `40 passed`
+- Function coverage: `29.9%` (`2863/9576`)
+- Loaded source files: `297/313`
+- Console logs: `72`
+- Console error-like entries: `4`
+- Page errors: `0`
+
+Primary issue groups captured:
+
+1. `404 Not Found` resource errors (2 occurrences)
+2. Forced decrypt-timeout warning in diagnostics test (expected by test, 1 occurrence)
+3. CORS-blocked `.torrent` fetch in webseed playback test (1 occurrence)
+4. `net::ERR_FAILED` in webseed playback test (1 occurrence)
+5. WebSocket tracker handshake failure for `wss://tracker.btorrent.xyz/` (1 occurrence)
+
+Recent high-impact coverage improvements:
+
+- `js/nostr/commentEvents.js`: `90.91%` (`20/20`)
+- `js/nostr/reactionEvents.js`: `78.57%` (`11/14`)
+- `js/nostr/publishHelpers.js`: `79.41%` (`27/34`)
+- `js/nostr/sessionActor.js`: `79.41%` (`27/34`)
+
+## Integration Rules For Agents
+
+1. For `tests/e2e/*.spec.ts`, import from `tests/e2e/helpers/instrumentedTest.ts` so coverage/log capture stays enabled.
+2. For harness-based tests, use `tests/e2e/helpers/bitvidTestFixture.ts` (it already includes auto-capture).
+3. Keep coverage runs Chromium-only when collecting percentages (`page.coverage` is Chromium-specific).
+4. Do not remove `playwright.config.ts` global setup (`tests/e2e/helpers/playwrightCoverageGlobalSetup.ts`), which resets raw coverage artifacts.
+5. Use `npm run test:e2e:coverage:report` only after a prior coverage-enabled run; it reads existing raw artifacts.
+
+## How To Improve Coverage
+
+Priority order for fastest gains:
+
+1. Add E2E for high-function files currently at `0%` but loaded:
+   - Start with `js/nostr/commentEvents.js`, `js/nostr/reactionEvents.js`, `js/nostr/publishHelpers.js`, `js/nostr/sessionActor.js`.
+2. Add deterministic DM coverage lane in E2E harness:
+   - Exercise NIP-04 and NIP-44 decrypt/encrypt flows through UI and harness hooks.
+3. Add upload + publish deep-path tests:
+   - Validate URL-only, magnet-only, and dual-source publish paths with assertions on fallback behavior.
+4. Add negative-path assertions that currently only surface in console:
+   - Explicitly assert/handle CORS/torrent fetch failures and tracker handshake failures.
+5. Keep tests routed through test-harness APIs:
+   - Prefer `setTestRelays`, `setSignerDecryptBehavior`, and seeded relay events over live-network dependence.
+
+## Suggested Quality Gates
+
+1. Track `functionCoveragePct` trend in CI artifacts per PR.
+2. Add a soft warning threshold first (for example `>=30%`), then ratchet upward as flaky paths are stabilized.
+3. Fail PRs only after baseline is stable across multiple runs.

dist/docs/testing/playwright-integration-recommendations-2026-02-22.md

@@ -0,0 +1,63 @@
+# Playwright Integration Recommendations (2026-02-22)
+
+This document captures practical follow-ups for improving bitvid's end-to-end and extension integration testing.
+
+## What is now in place
+
+1. Deterministic E2E matrix projects:
+- Chromium (`e2e`)
+- Firefox (`e2e-firefox`)
+
+2. Optional extension project:
+- `e2e-nip07-extension` (opt-in via `PLAYWRIGHT_ENABLE_EXTENSION_E2E=1` and `PLAYWRIGHT_EXTENSION_PATH`)
+
+3. Persistent-context extension launcher:
+- Script: `scripts/playwright/run-extension-persistent.mjs`
+- Command: `npm run test:e2e:extension`
+- Purpose: run extension tests in Chromium with a real persistent profile context.
+
+4. Test harness upgrades:
+- `setTestRelays(urls)` aligns `relayManager` and `nostrClient`.
+- `setSignerDecryptBehavior(mode, opts)` forces decrypt behavior for deterministic list-sync tests.
+- `getListSyncEvents()`, `clearListSyncEvents()`, `waitForListSyncEvent()` capture and assert sync lifecycle.
+
+5. Stable sync/test selectors:
+- Permission prompt, hashtag/subscription statuses, blocked-list status/list/empty state.
+
+## Recommended next steps
+
+1. Add CI extension smoke lane (manual workflow dispatch only)
+- Keep extension smoke out of default PR path.
+- Add a separate workflow that runs only when maintainers provide:
+  - `PLAYWRIGHT_EXTENSION_PATH` artifact or unpacked extension directory
+  - any extension secrets/profile setup needed by the signer.
+
+2. Add extension artifact capture on failure
+- Always upload:
+  - `playwright-report`
+  - video/trace/screenshot for `tests/e2e-extension/**`
+  - JSON summary of `window.nostr` capabilities.
+
+3. Add capability contract test
+- Add a single test that asserts minimum NIP-07 surface before running deeper flows:
+  - `getPublicKey`
+  - `signEvent`
+  - at least one decrypt path (`nip04` or `nip44`).
+
+4. Add "auth ready" test harness API
+- Add a harness method that waits for authenticated app-ready state (`header UI + signer + relay ready`) so tests stop duplicating custom waits.
+
+5. Add deterministic clock/random seed helper in test mode
+- Expose optional fixed clock/random behavior to reduce cross-run variance in event ordering and timeout-sensitive tests.
+
+6. Add a small extension compatibility matrix note
+- Track known-good extension versions for:
+  - Alby/Nostr Connect style extensions
+  - Chromium version pinned in CI.
+
+## Operational guardrails
+
+1. Keep deterministic tests (mock signer + mock relay) as the default required PR gate.
+2. Keep extension tests opt-in until flake rate is low and extension lifecycle setup is automated.
+3. Do not treat extension `content.js` runtime noise as app regression unless bitvid telemetry/events also fail in the same window.
+