From 0d6af82b30495dd1b6e7498918dca779573648ed Mon Sep 17 00:00:00 2001 From: thePR0M3TH3AN <53631862+PR0M3TH3AN@users.noreply.github.com> Date: Tue, 16 Jun 2026 16:36:35 -0400 Subject: [PATCH] docs(agents): capture NIP-07 reliability + upload-path lessons MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - AGENTS.md §4a: Cloudflare/S3 upload-path gotchas from the 2026-06-16 audit (r2Service/s3UploadService duplication, buildR2Key collision = data loss, whole-file hash before upload, optimistic publish success, silent partial failures / empty-file guard). - AGENTS.md §17 (new, appended so existing section numbers/cross-refs stay valid): NIP-07 Signer Reliability & Encrypted-List Loading — raw-probe diagnostic + the resilience invariants (relay cap, circuit breaker, no transient-error swallowing, decrypt budget, lazy modal population) and the deterministic fake-signer + mock-relay test harness. - CLAUDE.md: matching condensed "NIP-07 Signer Reliability (hard-won)" subsection under Nostr Integration, cross-linking AGENTS.md §17 and KNOWN_BUGS #0. Co-Authored-By: Claude Opus 4.8 --- AGENTS.md | 24 ++++++++++++++++++++++++ CLAUDE.md | 12 ++++++++++++ 2 files changed, 36 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 2252920b..2f274687 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -63,6 +63,14 @@ For any nostr related work, please review the nip documentation located in /docs 5. **Telemetry hooks:** Keep existing analytics/logging untouched unless instructed. If you add new modal states, annotate them clearly in code comments. 6. **Modal regressions:** If validation or helper wiring breaks in Main, disable new feature flags and ship a revert PR immediately. Note the rollback steps in AGENTS.md for posterity. +### 4a. Cloudflare/S3 upload-path gotchas (audited 2026-06-16) + +* **Two near-identical upload services exist** — `js/services/r2Service.js` (Cloudflare R2) and `js/services/s3UploadService.js` (generic S3). They share `buildR2Key`, `resolveUploadIdentifier`, and the magnet build but are *copies*. **Any fix to one must be mirrored in the other** or they silently drift. +* **Storage-key collisions = data loss.** `buildR2Key` (`js/r2.js`) builds `u///.` where `namespace` falls back to the literal `"uploads"` and `slug` to `"video"` (e.g. a non-ASCII filename, or no info-hash). WebTorrent uploads are content-addressed (info-hash namespaces the key) and safe, but a **URL-first upload with no info-hash and a duplicate filename overwrites the previous object** while the old note still points at that URL. Include a unique component (e.g. `videoRootId` or a random suffix) when no info-hash is available. +* **Whole-file hash before upload.** When the modal doesn't pre-seed an info-hash, `resolveUploadIdentifier` runs `calculateTorrentInfoHash(file)`, which reads the *entire* file before the upload begins and before any progress is emitted — on multi-GB files this looks like a hang. Emit a "preparing/hashing" status or parallelize. +* **Publish can report success unconfirmed.** `publishEventToRelay` (`js/nostrPublish.js`) has an optimistic-success fallthrough when the relay handle exposes no `on()`/`then()` — a note may show "Published" with no relay ACK. Don't tighten blindly (legacy `seen`-only relays rely on it), but log it. +* **Silent partial failures.** Thumbnail and `.torrent` upload failures are caught and `warn`-only; the video note still publishes with no user signal. Empty (0-byte) files hit `CompleteMultipartUpload` with `Parts:[]` and surface a cryptic error — guard `file.size === 0`. + --- ## 5. Manual QA Checklist @@ -682,6 +690,22 @@ This agent’s goal is NOT “make CI green.” Its goal is “make the suite re --- +## 17. NIP-07 Signer Reliability & Encrypted-List Loading (hard-won, 2026-06-16) + +The single biggest source of "DMs / hashtags / watch-history / block & subscription lists won't load after login" is **not** bitvid — it's an **unresponsive NIP-07 signer**. The extension's MV3 background service-worker can die or its content-script↔worker channel can orphan, after which raw `window.nostr` calls hang forever. No client change can force a dead signer to answer. + +* **Diagnose first, don't guess.** Run a single raw `window.nostr` probe in the page console (`getPublicKey → nip04.encrypt → nip04.decrypt`) that bypasses bitvid entirely. If *that* hangs, it's the extension/environment. Recommend a well-maintained signer (nos2x, Alby); KeysBand's dead worker was the root cause and switching to nos2x fixed everything. +* **Resilience invariants — do not regress these** (see `docs/KNOWN_BUGS.md` #0 for the full history and the files): + 1. **Cap relay fan-out** (`js/nostr/toolkit.js` `capReadRelays`, ≤8, user-relays-first + 2 reserved default slots). An uncapped cold-login REQ storm to ~20 dead NIP-65 relays starves the single-threaded signer's postMessage round-trips. + 2. **Circuit breaker** on the NIP-07 channel (`js/nostr/nip07Permissions.js`): after N consecutive *timeouts* fast-fail instead of hanging ~15s each. Channel-death errors must count toward opening (not reset it); interactive permission prompts bypass it; one periodic probe detects recovery. + 3. **Never swallow a transient decrypt error as an empty result.** Re-throw channel-death/timeout sub-errors so the retry path runs — returning `[]` turns "signer is slow" into "user has no blocks/lists" and kills retries. + 4. **Generous decrypt budget** (~25–30s/call, ~60s backoff cap); a 6s timeout kills slow-but-responsive signers mid-decrypt. Handshake variant timeout is ~20s for the same reason. + 5. **One actionable user notice** (`js/utils/signerHealthNotice.js`) after a few timeouts with `signerStatus: "present"` — not a silent forever-retry. +* **Don't render into closed modals at login.** Once a responsive signer makes decryption instant, eagerly populating every profile panel at login (friends avatars, subscriptions, blocks, DM summaries) freezes the main thread ~10–15s. Populate each pane lazily on open (`selectPane`) and gate data-change re-render listeners on "is the view open". +* **Deterministic testing.** Reproduce signer-dependent bugs headlessly with a fake `window.nostr` (Playwright `exposeBinding` + `addInitScript`) + a mock relay + configurable channel models (healthy/slow/overload/dead, latency override). See `scripts/perf/nip07-channel-sim.mjs`. + +--- + ## Next Please read these documents next. diff --git a/CLAUDE.md b/CLAUDE.md index f20400a5..5dbc4a8d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -310,6 +310,18 @@ const dTag = event.tags.find(t => t[0] === 'd')?.[1]; // WRONG: Don't use logical IDs like videoRootId for relay lookups ``` +### NIP-07 Signer Reliability (hard-won) + +The #1 cause of "DMs / hashtags / watch-history / lists won't load after login" is an **unresponsive NIP-07 signer**, not bitvid. The extension's MV3 background worker can die and `window.nostr` calls then hang forever — no client change fixes a dead signer. **Diagnose with a raw `window.nostr` probe** (`getPublicKey → nip04.encrypt → nip04.decrypt`) in the console; if that hangs, it's the extension (recommend nos2x/Alby). See `AGENTS.md` §17 and `docs/KNOWN_BUGS.md` #0. + +Resilience invariants — **do not regress**: + +1. **Cap relay fan-out** (`js/nostr/toolkit.js` `capReadRelays`, ≤8, user-relays-first + 2 reserved defaults) — an uncapped cold-login REQ storm starves the single-threaded signer. +2. **Circuit breaker** on consecutive NIP-07 timeouts (`js/nostr/nip07Permissions.js`) — fast-fail instead of hanging; channel-death counts toward opening; permission prompts bypass. +3. **Never swallow a transient decrypt error as `[]`** — re-throw channel-death/timeout so retries run (returning empty = "user has no lists" + no retry). +4. **Generous decrypt budget** (~25–30s/call) — a 6s timeout kills slow signers mid-decrypt. +5. **Lazy modal population** — don't render every profile panel into a closed modal at login; populate per-pane on open (`selectPane`). + --- ## Testing & Validation: Scenario-First, Cheat-Resistant (Dark Factory Rules)