Merge pull request #2297 from PR0M3TH3AN/claude/improve-playwright-testability-8mNO7

Add Playwright agent testing infrastructure with test harness and mock relay
This commit is contained in:
thePR0M3TH3AN authored and GitHub committed 2026-02-10 00:29:35 -05:00
commit d58cff2eac
15 files changed
+972 -29

No files matched your search

+107
View File
@@ -301,6 +301,113 @@ Three automated lint checks protect the codebase from common growth problems. Th
---
## 14. Playwright Agent Testing Infrastructure
bitvid includes a test harness and fixtures that let Playwright CLI agents programmatically log in, seed relay data, and inspect app state — no browser extensions or real relays required.
### Activating Test Mode
Add `?__test__=1` to the URL, or set `localStorage.__bitvidTestMode__ = "1"` via `addInitScript`. This installs `window.__bitvidTest__` on the page.
### Overriding Relays
Point the app at a local mock relay instead of production:
```
?__test__=1&__testRelays__=ws://127.0.0.1:8877
```
Or set `localStorage.__bitvidTestRelays__` to a JSON array of relay URLs.
### Test Harness API (`window.__bitvidTest__`)
| Method | Returns | Purpose |
|--------|---------|---------|
| `loginWithNsec(hexKey)` | `Promise<string>` (pubkey) | Programmatic login, bypasses the modal |
| `logout()` | `void` | Clear active signer |
| `getAppState()` | `{ isLoggedIn, activePubkey, relays, ... }` | Inspect current state |
| `getFeedItems()` | `Array<{ title, pubkey, dTag, hasUrl, hasMagnet }>` | Scrape video cards from DOM |
| `waitForFeedItems(n, ms)` | `Promise<Array>` | Wait for N cards to appear |
| `waitForSelector(sel, ms)` | `Promise<true>` | Wait for a DOM element |
| `getRelayHealth()` | `{ relays, unreachable, backoff }` | Relay connection status |
| `applyRelayOverrides(urls)` | `boolean` | Redirect relay connections |
| `nostrClient` | `NostrClient` | Direct access for advanced use |
### Mock Relay (`scripts/agent/simple-relay.mjs`)
Start with `startRelay(port, { httpPort })`. Alongside the Nostr WebSocket protocol, exposes an HTTP API:
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/seed` | `POST` | Inject signed events (single object or array) |
| `/events` | `GET` | List all stored events |
| `/events` | `DELETE` | Clear all events |
| `/health` | `GET` | Event count + connection count |
Programmatic API: `relay.seedEvent(event)`, `relay.clearEvents()`, `relay.getEvents()`.
### Playwright Fixture (`tests/e2e/helpers/bitvidTestFixture.ts`)
Import `test` and `expect` from the fixture for tests that need the full stack:
```typescript
import { test, expect } from "./helpers/bitvidTestFixture";
test("agent can seed and view videos", async ({ page, gotoApp, loginAs, seedEvent }) => {
await seedEvent({ title: "Test Video", url: "https://example.com/v.mp4" });
await gotoApp();
await loginAs(page);
// ... assertions
});
```
**Available fixture values:**
| Fixture | Type | Purpose |
|---------|------|---------|
| `relay` | Relay instance | Auto-started/stopped per test |
| `seedEvent(video)` | `(TestVideoEvent) => Promise` | Create a signed kind 30078 event and inject it |
| `seedRawEvent(event)` | `(any) => Promise` | Inject a pre-built event |
| `clearRelay()` | `() => Promise` | Wipe all relay events |
| `gotoApp(path?)` | `(string?) => Promise` | Navigate with test mode + relay overrides |
| `loginAs(page)` | `(Page) => Promise<string>` | Login with deterministic test key |
| `testPubkey` | `string` | Hex pubkey of the test key |
| `relayUrl` | `string` | WebSocket URL of the mock relay |
### `data-testid` Selectors
Use these for stable element targeting:
| Selector | Element |
|----------|---------|
| `[data-testid="login-button"]` | Header login button |
| `[data-testid="upload-button"]` | Header upload button (hidden until logged in) |
| `[data-testid="profile-button"]` | Header profile button (hidden until logged in) |
| `[data-testid="search-input"]` | Header search field |
| `[data-testid="login-modal"]` | Login modal container |
| `[data-testid="login-provider-button"]` | Login provider option buttons |
| `[data-testid="nsec-secret-input"]` | Private key textarea in nsec login |
| `[data-testid="nsec-submit"]` | Nsec login submit button |
| `[data-testid="upload-modal"]` | Upload modal container |
| `[data-testid="upload-title"]` | Video title input |
| `[data-testid="upload-url"]` | Video URL input |
| `[data-testid="upload-magnet"]` | Magnet link input |
| `[data-testid="upload-submit"]` | Publish button |
| `[data-testid="video-modal"]` | Video player modal |
| `[data-testid="video-card"]` | Individual video card in the feed |
| `[data-testid="video-list"]` | Video feed grid container |
### Key Files
| File | Purpose |
|------|---------|
| `js/testHarness.js` | Test harness module (installs `window.__bitvidTest__`) |
| `scripts/agent/simple-relay.mjs` | Mock relay with HTTP seeding API |
| `tests/e2e/helpers/bitvidTestFixture.ts` | Reusable Playwright fixture |
| `tests/e2e/agent-testability.spec.ts` | Infrastructure validation tests |
---
## Next
Please read these documents next.
+26
View File
@@ -80,6 +80,9 @@ bitvid/
| `js/nostrEventSchemas.js` | **Source of truth** for all Nostr event schemas |
| `js/constants.js` | Feature flags and runtime configuration |
| `config/instance-config.js` | Deployment-specific settings |
| `js/testHarness.js` | Playwright test harness (`window.__bitvidTest__`) |
| `tests/e2e/helpers/bitvidTestFixture.ts` | Reusable Playwright fixture for agent testing |
| `scripts/agent/simple-relay.mjs` | Mock Nostr relay with HTTP seeding API |
| `docs/nostr-event-schemas.md` | Event schema documentation |
---
@@ -330,6 +333,29 @@ Failed visual tests store artifacts in `artifacts/test-results/`:
View with: `./scripts/show-artifacts.sh`
### Agent E2E Testing (Playwright)
bitvid includes a test harness for programmatic Playwright testing without browser extensions or real relays. See `AGENTS.md` Section 14 for the full reference. Quick start:
```typescript
import { test, expect } from "./helpers/bitvidTestFixture";
test("my test", async ({ page, gotoApp, loginAs, seedEvent }) => {
await seedEvent({ title: "Test Video", url: "https://example.com/v.mp4" });
await gotoApp();
await loginAs(page);
// interact with the app using data-testid selectors
await expect(page.locator('[data-testid="upload-button"]')).toBeVisible();
});
```
Key capabilities:
- **Programmatic login**: `loginAs(page)` uses `window.__bitvidTest__.loginWithNsec()` — no modal interaction needed
- **Mock relay**: Each test gets an isolated in-memory relay with HTTP seeding
- **Relay override**: `?__test__=1&__testRelays__=ws://localhost:8877` redirects all connections
- **State inspection**: `window.__bitvidTest__.getAppState()` returns login status, relays, etc.
- **Stable selectors**: All key elements have `data-testid` attributes (see AGENTS.md Section 14)
---
## Common Tasks
+4 -1
View File
@@ -6,6 +6,7 @@
aria-labelledby="loginModalTitle"
aria-describedby="loginModalDescription"
data-ds="new"
data-testid="login-modal"
>
<div class="bv-modal-backdrop" data-dismiss></div>
@@ -91,6 +92,7 @@
class="login-option"
type="button"
data-provider-button
data-testid="login-provider-button"
>
<span class="login-option__header">
<span class="login-option__copy">
@@ -204,6 +206,7 @@
spellcheck="false"
autocomplete="off"
data-nsec-secret
data-testid="nsec-secret-input"
></textarea>
<label class="inline-flex items-center gap-2 text-sm text-text">
<input type="checkbox" class="checkbox" data-nsec-remember />
@@ -235,7 +238,7 @@
</fieldset>
<div class="flex justify-end gap-2">
<button type="button" class="btn-ghost" data-nsec-cancel>Cancel</button>
<button type="submit" class="btn" data-nsec-submit>Continue</button>
<button type="submit" class="btn" data-nsec-submit data-testid="nsec-submit">Continue</button>
</div>
</form>
</template>
+5 -4
View File
@@ -7,6 +7,7 @@
aria-labelledby="uploadModalTitle"
aria-describedby="uploadModalDescription"
data-ds="new"
data-testid="upload-modal"
>
<div class="bv-modal-backdrop" data-dismiss></div>
@@ -192,12 +193,12 @@
<div id="section-source-external" class="space-y-4 hidden">
<div class="space-y-2">
<label for="input-url" class="block text-sm font-medium text-text">Video URL</label>
<input id="input-url" type="url" class="input" placeholder="https://example.com/video.mp4" />
<input id="input-url" type="url" class="input" placeholder="https://example.com/video.mp4" data-testid="upload-url" />
<p class="text-xs text-muted">Primary hosted video source (URL-first playback).</p>
</div>
<div class="space-y-2">
<label for="input-magnet" class="block text-sm font-medium text-text">Magnet Link</label>
<input id="input-magnet" type="text" class="input" placeholder="magnet:?xt=urn:btih:..." />
<input id="input-magnet" type="text" class="input" placeholder="magnet:?xt=urn:btih:..." data-testid="upload-magnet" />
<p class="text-xs text-muted">Optional WebTorrent fallback when the URL is unavailable.</p>
</div>
</div>
@@ -208,7 +209,7 @@
<div class="space-y-4">
<div class="space-y-2">
<label for="input-title" class="block text-sm font-medium text-text">Title</label>
<input id="input-title" type="text" required class="input" placeholder="Give your video a title" />
<input id="input-title" type="text" required class="input" placeholder="Give your video a title" data-testid="upload-title" />
</div>
<div class="space-y-2">
@@ -388,7 +389,7 @@
<!-- Submit -->
<div class="pt-4">
<button type="submit" id="btn-submit" class="btn w-full py-3 text-base font-semibold shadow-md">
<button type="submit" id="btn-submit" class="btn w-full py-3 text-base font-semibold shadow-md" data-testid="upload-submit">
Publish Video
</button>
<p id="submit-status" class="text-center text-xs text-muted mt-2"></p>
+1
View File
@@ -5,6 +5,7 @@
role="dialog"
aria-modal="true"
data-ds="new"
data-testid="video-modal"
hidden
>
<div class="bv-modal-backdrop" data-dismiss></div>
+4
View File
@@ -207,6 +207,7 @@
name="query"
placeholder="Search videos"
autocomplete="off"
data-testid="search-input"
/>
<button type="submit" class="header-search__submit">
<span class="sr-only">Submit search</span>
@@ -264,6 +265,7 @@
id="loginButton"
class="btn focus-ring h-12 w-12 rounded-full p-0"
aria-label="Open login modal"
data-testid="login-button"
>
<svg
class="h-9 w-9"
@@ -284,6 +286,7 @@
<button
id="uploadButton"
class="btn focus-ring hidden h-12 w-12 rounded-full p-0 text-xl font-bold leading-none"
data-testid="upload-button"
>
+
</button>
@@ -292,6 +295,7 @@
<button
id="profileButton"
class="btn-ghost focus-ring relative hidden h-12 w-12 rounded-full p-0"
data-testid="profile-button"
>
<div class="h-10 w-10 overflow-hidden rounded-full">
<img
+5
View File
@@ -54,9 +54,14 @@ import { userBlocks } from "./userBlocks.js";
import { relayManager } from "./relayManager.js";
import createApplication from "./bootstrap.js";
import SearchFilterModal from "./ui/components/SearchFilterModal.js";
import { installTestHarness } from "./testHarness.js";
validateInstanceConfig();
// Install test harness early so relay overrides apply before connections.
// Only activates when ?__test__=1 or localStorage __bitvidTestMode__ is set.
installTestHarness();
applyDesignSystemAttributes();
initThemeController();
+281
View File
@@ -0,0 +1,281 @@
// js/testHarness.js
//
// Playwright test harness — exposes window.__bitvidTest__ for E2E agent testing.
// This module is loaded conditionally and provides programmatic access to:
// - Login with a private key (bypassing the UI)
// - Override relay URLs to point at a local mock relay
// - Inspect app state (logged-in pubkey, feed items, relay health)
// - Wait for async operations (feed load, relay connection)
//
// Usage from Playwright:
// await page.evaluate(() => window.__bitvidTest__.loginWithNsec(hexPrivateKey));
// await page.evaluate(() => window.__bitvidTest__.getAppState());
import { nostrClient } from "./nostrClientFacade.js";
import {
registerSigner,
setActiveSigner,
getActiveSigner,
listRegisteredSigners,
} from "./nostrClientRegistry.js";
import { devLogger } from "./utils/logger.js";
const HARNESS_VERSION = 1;
/**
* Check if test mode is enabled via URL param or localStorage.
*/
export function isTestMode() {
if (typeof window === "undefined") return false;
// Check URL parameter
const params = new URLSearchParams(window.location.search);
if (params.get("__test__") === "1") return true;
// Check localStorage flag
try {
if (localStorage.getItem("__bitvidTestMode__") === "1") return true;
} catch {
// Ignore storage errors
}
return false;
}
/**
* Read test relay URLs from query params or localStorage.
* Format: ?__testRelays__=ws://localhost:8899,ws://localhost:8900
*/
export function getTestRelayOverrides() {
if (typeof window === "undefined") return null;
const params = new URLSearchParams(window.location.search);
const paramRelays = params.get("__testRelays__");
if (paramRelays) {
return paramRelays
.split(",")
.map((r) => r.trim())
.filter(Boolean);
}
try {
const stored = localStorage.getItem("__bitvidTestRelays__");
if (stored) {
const parsed = JSON.parse(stored);
if (Array.isArray(parsed)) return parsed;
}
} catch {
// Ignore
}
return null;
}
/**
* Apply relay overrides to the nostrClient so it connects to test relays
* instead of production relays.
*/
function applyRelayOverrides(relayUrls) {
if (!relayUrls || !relayUrls.length) return false;
try {
if (typeof nostrClient.applyRelayPreferences === "function") {
nostrClient.applyRelayPreferences({
all: relayUrls,
read: relayUrls,
write: relayUrls,
});
return true;
}
// Fallback: directly set relay arrays if available
if (nostrClient.connectionManager) {
const cm = nostrClient.connectionManager;
cm.relays = [...relayUrls];
if (Array.isArray(cm.readRelays)) cm.readRelays = [...relayUrls];
if (Array.isArray(cm.writeRelays)) cm.writeRelays = [...relayUrls];
return true;
}
} catch (err) {
devLogger.error("[testHarness] Failed to apply relay overrides:", err);
}
return false;
}
/**
* Login programmatically with a hex private key. Bypasses the login modal.
* Returns the hex pubkey on success.
*/
async function loginWithNsec(hexPrivateKey) {
if (!hexPrivateKey || typeof hexPrivateKey !== "string") {
throw new Error("[testHarness] hexPrivateKey is required");
}
// Use the nostrClient's built-in method to derive and register
if (typeof nostrClient.derivePrivateKeyFromSecret !== "function") {
throw new Error("[testHarness] nostrClient.derivePrivateKeyFromSecret not available");
}
const { privateKey, pubkey } = await nostrClient.derivePrivateKeyFromSecret(hexPrivateKey);
if (typeof nostrClient.registerPrivateKeySigner !== "function") {
throw new Error("[testHarness] nostrClient.registerPrivateKeySigner not available");
}
await nostrClient.registerPrivateKeySigner({
privateKey,
pubkey,
persist: false,
});
return pubkey;
}
/**
* Logout the current signer.
*/
function logout() {
const signer = getActiveSigner();
if (signer) {
setActiveSigner(null);
}
}
/**
* Get current app state for test assertions.
*/
function getAppState() {
const signer = getActiveSigner();
const signers = listRegisteredSigners();
return {
isLoggedIn: Boolean(signer),
activePubkey: signer?.pubkey || null,
registeredSignerCount: signers.length,
relays: nostrClient.connectionManager
? {
all: [...(nostrClient.connectionManager.relays || [])],
read: [...(nostrClient.connectionManager.readRelays || [])],
write: [...(nostrClient.connectionManager.writeRelays || [])],
}
: null,
};
}
/**
* Get the current feed video cards from the DOM.
*/
function getFeedItems() {
const cards = document.querySelectorAll("[data-video-card]");
return Array.from(cards).map((card) => ({
title: card.querySelector("[data-video-title]")?.textContent?.trim() || "",
pubkey: card.getAttribute("data-video-pubkey") || "",
dTag: card.getAttribute("data-video-dtag") || "",
hasUrl: Boolean(card.getAttribute("data-play-url")),
hasMagnet: Boolean(card.getAttribute("data-play-magnet")),
}));
}
/**
* Wait for feed items to appear in the DOM.
*/
function waitForFeedItems(minCount = 1, timeoutMs = 15000) {
return new Promise((resolve, reject) => {
const start = Date.now();
function check() {
const items = getFeedItems();
if (items.length >= minCount) {
resolve(items);
return;
}
if (Date.now() - start > timeoutMs) {
reject(
new Error(
`[testHarness] Timed out waiting for ${minCount} feed items (found ${items.length})`,
),
);
return;
}
requestAnimationFrame(check);
}
check();
});
}
/**
* Wait for a specific DOM element matching a selector.
*/
function waitForSelector(selector, timeoutMs = 10000) {
return new Promise((resolve, reject) => {
const start = Date.now();
function check() {
const el = document.querySelector(selector);
if (el) {
resolve(true);
return;
}
if (Date.now() - start > timeoutMs) {
reject(
new Error(`[testHarness] Timed out waiting for selector: ${selector}`),
);
return;
}
requestAnimationFrame(check);
}
check();
});
}
/**
* Get relay connection health info.
*/
function getRelayHealth() {
if (!nostrClient.connectionManager) return null;
const cm = nostrClient.connectionManager;
return {
relays: [...(cm.relays || [])],
unreachable: [...(cm.unreachableRelays || [])],
backoff: cm.relayBackoff ? Object.fromEntries(cm.relayBackoff) : {},
};
}
/**
* Install the test harness on window.__bitvidTest__.
* Only installs when test mode is active.
*/
export function installTestHarness() {
if (typeof window === "undefined") return false;
if (!isTestMode()) return false;
const testRelays = getTestRelayOverrides();
if (testRelays) {
applyRelayOverrides(testRelays);
}
window.__bitvidTest__ = Object.freeze({
version: HARNESS_VERSION,
loginWithNsec,
logout,
getAppState,
getFeedItems,
waitForFeedItems,
waitForSelector,
getRelayHealth,
applyRelayOverrides,
// Expose nostrClient for advanced test scenarios
get nostrClient() {
return nostrClient;
},
});
devLogger.log("[testHarness] Test harness installed (v" + HARNESS_VERSION + ")");
return true;
}
+1
View File
@@ -393,6 +393,7 @@ export class VideoCard {
VideoCard.observeViewport(this.root);
this.root.dataset.component = "video-card";
this.root.dataset.testid = "video-card";
this.syncCardState();
this.syncMotionState();
if (this.video?.id) {
+138 -21
View File
@@ -1,14 +1,33 @@
import { WebSocketServer } from "ws";
import { createServer } from "http";
import { matchFilters } from "nostr-tools/filter";
export function startRelay(port = 8888) {
/**
* Start a simple in-memory Nostr relay with optional HTTP seeding API.
*
* @param {number} port - WebSocket port (default 8888)
* @param {Object} [options]
* @param {number} [options.httpPort] - HTTP API port for seeding events (default: port + 1)
* @returns {{ close: () => Promise<void>, port: number, httpPort: number, seedEvent: (event) => void, getEvents: () => Map, clearEvents: () => void }}
*/
export function startRelay(port = 8888, options = {}) {
const wss = new WebSocketServer({ port });
const events = new Map(); // id -> event
const maxEvents = 10000;
const subs = new Map(); // ws -> Map(subId -> filters)
console.log(`Simple relay starting on port ${port}`);
function broadcastEvent(event) {
for (const [client, clientSubs] of subs.entries()) {
if (client.readyState !== 1) continue; // OPEN
for (const [subId, filters] of clientSubs.entries()) {
if (matchFilters(filters, event)) {
client.send(JSON.stringify(["EVENT", subId, event]));
}
}
}
}
wss.on("connection", (ws) => {
subs.set(ws, new Map());
@@ -26,23 +45,9 @@ export function startRelay(port = 8888) {
return;
}
// Basic validation (optional, but good for sanity)
// We won't verify signature here to save CPU for the load test runner,
// assuming the client does it. But a real relay would.
// For load testing "relay writes", we primarily care about the I/O and handling overhead.
if (!events.has(event.id)) {
events.set(event.id, event);
// Broadcast to matching subs
for (const [client, clientSubs] of subs.entries()) {
if (client.readyState !== 1) continue; // OPEN
for (const [subId, filters] of clientSubs.entries()) {
if (matchFilters(filters, event)) {
client.send(JSON.stringify(["EVENT", subId, event]));
}
}
}
broadcastEvent(event);
}
ws.send(JSON.stringify(["OK", event.id, true, "saved"]));
@@ -54,7 +59,7 @@ export function startRelay(port = 8888) {
subs.get(ws).set(subId, filters);
// Send stored events
// Send stored events matching the filter
for (const event of events.values()) {
if (matchFilters(filters, event)) {
ws.send(JSON.stringify(["EVENT", subId, event]));
@@ -64,8 +69,18 @@ export function startRelay(port = 8888) {
} else if (type === "CLOSE") {
const subId = payload[0];
if (typeof subId === "string") {
subs.get(ws).delete(subId);
subs.get(ws)?.delete(subId);
}
} else if (type === "COUNT") {
const subId = payload[0];
const filters = payload.slice(1);
let count = 0;
for (const event of events.values()) {
if (matchFilters(filters, event)) {
count++;
}
}
ws.send(JSON.stringify(["COUNT", subId, { count }]));
}
} catch (err) {
console.error("Relay error processing message:", err);
@@ -81,13 +96,115 @@ export function startRelay(port = 8888) {
});
});
// --- Programmatic API for test seeding ---
function seedEvent(event) {
if (!event || !event.id) return false;
if (!events.has(event.id)) {
events.set(event.id, event);
broadcastEvent(event);
}
return true;
}
function clearEvents() {
events.clear();
}
function getEvents() {
return events;
}
// --- Optional HTTP API for seeding events from Playwright tests ---
let httpServer = null;
const httpPort = options.httpPort ?? port + 1;
if (options.httpPort !== false) {
httpServer = createServer((req, res) => {
// CORS headers for test environments
res.setHeader("Access-Control-Allow-Origin", "*");
res.setHeader("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS");
res.setHeader("Access-Control-Allow-Headers", "Content-Type");
if (req.method === "OPTIONS") {
res.writeHead(204);
res.end();
return;
}
if (req.method === "POST" && req.url === "/seed") {
let body = "";
req.on("data", (chunk) => (body += chunk));
req.on("end", () => {
try {
const payload = JSON.parse(body);
const eventsToSeed = Array.isArray(payload) ? payload : [payload];
const seeded = [];
for (const event of eventsToSeed) {
if (seedEvent(event)) {
seeded.push(event.id);
}
}
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ ok: true, seeded }));
} catch (err) {
res.writeHead(400, { "Content-Type": "application/json" });
res.end(JSON.stringify({ ok: false, error: err.message }));
}
});
return;
}
if (req.method === "GET" && req.url === "/events") {
const all = Array.from(events.values());
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify(all));
return;
}
if (req.method === "DELETE" && req.url === "/events") {
clearEvents();
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ ok: true }));
return;
}
if (req.method === "GET" && req.url === "/health") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(
JSON.stringify({
ok: true,
eventCount: events.size,
connectionCount: wss.clients.size,
}),
);
return;
}
res.writeHead(404, { "Content-Type": "application/json" });
res.end(JSON.stringify({ error: "not found" }));
});
httpServer.listen(httpPort, () => {
console.log(`Relay HTTP API listening on port ${httpPort}`);
});
}
return {
close: () => {
return new Promise((resolve) => {
wss.close(resolve);
const closeHttp = httpServer
? new Promise((r) => httpServer.close(r))
: Promise.resolve();
closeHttp.then(() => wss.close(resolve));
});
},
port
port,
httpPort: httpServer ? httpPort : null,
seedEvent,
getEvents,
clearEvents,
};
}
+187
View File
@@ -0,0 +1,187 @@
/**
* Agent testability E2E tests.
*
* These tests demonstrate that bitvid is fully testable by Playwright CLI agents.
* They exercise:
* 1. Test harness installation and relay override
* 2. Programmatic login (bypassing the UI)
* 3. Seeding test data into a local mock relay
* 4. Inspecting app state through the harness API
* 5. Key data-testid selectors for UI interaction
*/
import { test, expect } from "./helpers/bitvidTestFixture";
test.describe("Agent testability infrastructure", () => {
test("test harness installs when __test__=1 is set", async ({
page,
relay,
gotoApp,
}) => {
await gotoApp();
const harness = await page.evaluate(() => {
const h = (window as any).__bitvidTest__;
return h ? { version: h.version } : null;
});
expect(harness).not.toBeNull();
expect(harness!.version).toBe(1);
});
test("relay overrides are applied via query params", async ({
page,
relay,
gotoApp,
relayUrl,
}) => {
await gotoApp();
const state = await page.evaluate(() => {
return (window as any).__bitvidTest__.getAppState();
});
// Relay overrides should include our test relay
expect(state.relays).not.toBeNull();
if (state.relays) {
const allRelays = state.relays.all || [];
expect(allRelays).toContain(relayUrl);
}
});
test("programmatic login works via test harness", async ({
page,
relay,
gotoApp,
loginAs,
testPubkey,
}) => {
await gotoApp();
const pubkey = await loginAs(page);
expect(pubkey).toBe(testPubkey);
const state = await page.evaluate(() => {
return (window as any).__bitvidTest__.getAppState();
});
expect(state.isLoggedIn).toBe(true);
expect(state.activePubkey).toBe(testPubkey);
});
test("can seed events into mock relay and retrieve them", async ({
relay,
seedEvent,
clearRelay,
}) => {
await clearRelay();
await seedEvent({
title: "Test Video Alpha",
url: "https://example.com/alpha.mp4",
dTag: "alpha-001",
});
await seedEvent({
title: "Test Video Beta",
url: "https://example.com/beta.mp4",
magnet: "magnet:?xt=urn:btih:0000000000000000000000000000000000000000",
dTag: "beta-001",
});
// Verify via HTTP API
const resp = await fetch("http://127.0.0.1:8878/events");
const events = await resp.json();
expect(events.length).toBe(2);
});
test("relay health endpoint works", async ({ relay }) => {
const resp = await fetch("http://127.0.0.1:8878/health");
const health = await resp.json();
expect(health.ok).toBe(true);
expect(typeof health.eventCount).toBe("number");
expect(typeof health.connectionCount).toBe("number");
});
});
test.describe("Data-testid selectors are present", () => {
test("critical UI elements have data-testid attributes", async ({
page,
relay,
gotoApp,
}) => {
await gotoApp();
// Header controls
await expect(page.locator('[data-testid="login-button"]')).toBeVisible();
await expect(page.locator('[data-testid="search-input"]')).toBeVisible();
// Upload button is hidden until logged in — verify it exists in DOM
const uploadBtn = page.locator('[data-testid="upload-button"]');
await expect(uploadBtn).toBeAttached();
});
test("login modal has testid selectors", async ({
page,
relay,
gotoApp,
}) => {
await gotoApp();
// Open the login modal
await page.locator('[data-testid="login-button"]').click();
// Wait for modal to be visible
await expect(page.locator('[data-testid="login-modal"]')).toBeVisible({
timeout: 5000,
});
// Check provider buttons exist
const providerButtons = page.locator(
'[data-testid="login-provider-button"]',
);
// At least one provider button should be present
await expect(providerButtons.first()).toBeVisible({ timeout: 5000 });
});
});
test.describe("App state inspection", () => {
test("getAppState reports logged-out state by default", async ({
page,
relay,
gotoApp,
}) => {
await gotoApp();
const state = await page.evaluate(() => {
return (window as any).__bitvidTest__.getAppState();
});
expect(state.isLoggedIn).toBe(false);
expect(state.activePubkey).toBeNull();
});
test("logout clears active signer", async ({
page,
relay,
gotoApp,
loginAs,
}) => {
await gotoApp();
await loginAs(page);
let state = await page.evaluate(() => {
return (window as any).__bitvidTest__.getAppState();
});
expect(state.isLoggedIn).toBe(true);
await page.evaluate(() => {
(window as any).__bitvidTest__.logout();
});
state = await page.evaluate(() => {
return (window as any).__bitvidTest__.getAppState();
});
expect(state.isLoggedIn).toBe(false);
});
});
+210
View File
@@ -0,0 +1,210 @@
/**
* Playwright test fixture for bitvid agent testing.
*
* Provides:
* - Local mock relay with HTTP seeding API
* - Programmatic login via the test harness
* - Test data seeding helpers
* - App state inspection utilities
*
* Usage:
* import { test, expect } from './helpers/bitvidTestFixture';
*
* test('can publish a video', async ({ page, relay, seedEvent, loginAs }) => {
* await loginAs(page);
* // ... interact with the app
* });
*/
import { test as base, expect, type Page } from "@playwright/test";
import { startRelay } from "../../../scripts/agent/simple-relay.mjs";
import { finalizeEvent, getPublicKey } from "nostr-tools";
import { WebSocket } from "ws";
// Polyfill WebSocket for node environment
if (!global.WebSocket) {
(global as any).WebSocket = WebSocket;
}
// Fixed test key for deterministic testing.
// This is a throwaway key used only in test environments.
const TEST_PRIVATE_KEY_HEX =
"7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b7b";
const TEST_SK = Uint8Array.from(Buffer.from(TEST_PRIVATE_KEY_HEX, "hex"));
const TEST_PK = getPublicKey(TEST_SK);
const RELAY_PORT = 8877;
const RELAY_HTTP_PORT = 8878;
const RELAY_WS_URL = `ws://127.0.0.1:${RELAY_PORT}`;
const RELAY_HTTP_URL = `http://127.0.0.1:${RELAY_HTTP_PORT}`;
export interface TestVideoEvent {
title: string;
url?: string;
magnet?: string;
description?: string;
dTag?: string;
}
/**
* Create a signed Nostr video event (kind 30078) for test seeding.
*/
function createVideoEvent(video: TestVideoEvent) {
const dTag = video.dTag || `test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const contentObj: Record<string, unknown> = {
version: 3,
title: video.title,
videoRootId: dTag,
mode: "dev",
isPrivate: false,
deleted: false,
};
if (video.url) contentObj.url = video.url;
if (video.magnet) contentObj.magnet = video.magnet;
if (video.description) contentObj.description = video.description;
const tags: string[][] = [
["d", dTag],
["title", video.title],
];
if (video.url) tags.push(["url", video.url]);
return finalizeEvent(
{
kind: 30078,
created_at: Math.floor(Date.now() / 1000),
tags,
content: JSON.stringify(contentObj),
pubkey: TEST_PK,
},
TEST_SK,
);
}
/**
* Seed an event into the relay via HTTP API.
*/
async function seedEventViaHttp(event: any) {
const resp = await fetch(`${RELAY_HTTP_URL}/seed`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(event),
});
if (!resp.ok) {
throw new Error(`Failed to seed event: ${resp.status} ${await resp.text()}`);
}
return resp.json();
}
/**
* Clear all events from the relay.
*/
async function clearRelayEvents() {
const resp = await fetch(`${RELAY_HTTP_URL}/events`, { method: "DELETE" });
if (!resp.ok) {
throw new Error(`Failed to clear events: ${resp.status}`);
}
}
/**
* Navigate to bitvid with test mode and relay overrides enabled.
*/
async function gotoWithTestMode(page: Page, path = "/") {
const separator = path.includes("?") ? "&" : "?";
const testUrl = `${path}${separator}__test__=1&__testRelays__=${encodeURIComponent(RELAY_WS_URL)}`;
// Set up localStorage before navigation
await page.addInitScript(() => {
localStorage.setItem("hasSeenDisclaimer", "true");
localStorage.setItem("__bitvidTestMode__", "1");
});
await page.goto(testUrl);
// Wait for the test harness to be available
await page.waitForFunction(
() => typeof (window as any).__bitvidTest__ === "object",
{ timeout: 15000 },
);
}
/**
* Login programmatically via the test harness.
*/
async function loginWithTestKey(page: Page) {
const pubkey = await page.evaluate(async (hexKey) => {
const harness = (window as any).__bitvidTest__;
if (!harness) throw new Error("Test harness not installed");
return harness.loginWithNsec(hexKey);
}, TEST_PRIVATE_KEY_HEX);
return pubkey;
}
// ------------------------------------------------------------------
// Fixture type definitions
// ------------------------------------------------------------------
type BitvidFixtures = {
relay: Awaited<ReturnType<typeof startRelay>>;
seedEvent: (video: TestVideoEvent) => Promise<any>;
seedRawEvent: (event: any) => Promise<any>;
clearRelay: () => Promise<void>;
gotoApp: (path?: string) => Promise<void>;
loginAs: (page: Page) => Promise<string>;
testPubkey: string;
testPrivateKey: string;
relayUrl: string;
};
// ------------------------------------------------------------------
// Shared relay fixture (one relay per worker)
// ------------------------------------------------------------------
export const test = base.extend<BitvidFixtures>({
relay: [
async ({}, use) => {
const relayInstance = startRelay(RELAY_PORT, { httpPort: RELAY_HTTP_PORT });
await use(relayInstance);
await relayInstance.close();
},
{ scope: "test" },
],
seedEvent: async ({ relay }, use) => {
await use(async (video: TestVideoEvent) => {
const event = createVideoEvent(video);
return seedEventViaHttp(event);
});
},
seedRawEvent: async ({ relay }, use) => {
await use(async (event: any) => {
return seedEventViaHttp(event);
});
},
clearRelay: async ({ relay }, use) => {
await use(async () => {
await clearRelayEvents();
});
},
gotoApp: async ({ page }, use) => {
await use(async (path = "/") => {
await gotoWithTestMode(page, path);
});
},
loginAs: async ({}, use) => {
await use(async (page: Page) => {
return loginWithTestKey(page);
});
},
testPubkey: TEST_PK,
testPrivateKey: TEST_PRIVATE_KEY_HEX,
relayUrl: RELAY_WS_URL,
});
export { expect };
+1 -1
View File
@@ -26,7 +26,7 @@
</button>
</div>
<div id="videoListTags" class="mb-4" hidden></div>
<div id="videoList" class="bv-grid-video">
<div id="videoList" class="bv-grid-video" data-testid="video-list">
<div class="sidebar-loading-wrapper" role="status" aria-live="polite">
<div class="sidebar-loading-indicator">
<span
+1 -1
View File
@@ -26,7 +26,7 @@
</button>
</div>
<div id="videoList" class="bv-grid-video">
<div id="videoList" class="bv-grid-video" data-testid="video-list">
<div class="sidebar-loading-wrapper" role="status" aria-live="polite">
<div class="sidebar-loading-indicator">
<span
+1 -1
View File
@@ -29,7 +29,7 @@
<!-- Video List -->
<div id="videoListTags" class="mb-4" hidden></div>
<div id="videoList" class="bv-grid-video">
<div id="videoList" class="bv-grid-video" data-testid="video-list">
<div class="sidebar-loading-wrapper" role="status" aria-live="polite">
<div class="sidebar-loading-indicator">
<span