KNOWN_BUGS #1: logged out, the video grid stayed empty after dismissing the
disclaimer until a manual page refresh.
Reproduced deterministically with scripts/perf/disclaimer-grid.mjs (fresh
context, real relays) once it actually waits for and dismisses the
disclaimer: post-dismiss card timeline was 0,0,0,… and only a reload
yielded 132 cards.
Root cause: bootstrapInterface (js/index.js) opens the disclaimer — which
makes the background inert via staticModalAccessibility — and THEN awaits
handleHashChange(), which loads the feed view and renders into that
inert/hidden background, so the initial grid mount/render is dropped.
Dismissing the disclaimer never re-triggered a render.
Fix: disclaimer.js hide() now dispatches a `bitvid:disclaimer-dismissed`
event; index.js listens once and calls application.loadVideos(true) so the
feed re-renders into the now-interactive background. Only fires when a shown
disclaimer is dismissed, so returning visitors (who never see it) get no
redundant reload.
Harness after fix: post-dismiss timeline 132,132,… immediately. Full build
+ unit suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console: signerStatus "present" but every block/hashtag decrypt
timed out at the 15s service budget, forever, retrying every ~15s for
15+ minutes — lists never loaded.
Reproduced deterministically with the channel-sim harness (latency
override): with per-call latency BELOW the service budget all lists load;
the instant it crosses the budget every list fails (NO) and retries
forever. A nip-07 decrypt is a postMessage round-trip to the extension's
single-threaded, possibly-sleeping service worker — under cold-login load
real extensions take 15-25s to answer even though the crypto is instant.
The earlier 6s per-call timeout was killing those in-progress decrypts.
Raise the budget across blocks/hashtags/subscriptions so a slow-but-
responsive signer completes instead of timing out:
- service-level DECRYPT_TIMEOUT_MS 15s → 30s
- per-call nip-07 decrypt timeout 6s → 25s interactive / ~10-12s background
- backoff cap 30s → 60s so a persistently dead signer settles into an
occasional heartbeat instead of pinning the extension every ~15s
Harness: at 16s/call (previously NO across the board) blocks +
subscriptions + hashtags now all decrypt (0 timeouts); healthy mode
unchanged (signer-ready ~1.2s). Full unit suite green. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Two redundancies were inflating the login REQ burst (kind 30000=19 storm in
the real-env console — the cause of the slow hashtags, KNOWN_BUGS #0),
isolated with the channel-sim harness's REQ-signature + timing capture:
1. Duplicate trigger: ProfileModalController.handleAuthLogin called
subscriptions.loadSubscriptions() AND userBlocks.loadBlocks() DIRECTLY at
login, on top of the auth coordinator's gated list loads — a second,
ungated relay round-trip for each. Route them through ensureLoaded()
instead so they coalesce with the coordinator's in-flight load. Harness:
subscriptions updateFromRelays now fires ONCE at login instead of 3×
(≈3× fewer kind-30000 fetches in prod, where each is multiplied by the
relay set).
2. Fetch/decrypt coupling: a decrypt retry re-ran the full relay fetch even
though only signer availability had changed. Cache the fetched ciphertext
(_pendingDecryptEvent) and have scheduleDecryptRetry re-decrypt it via a
cachedDecryptEvent fast-path in updateFromRelays — no redundant relay
query per retry.
Validated: MODE=overload (channel drops) still decrypts blocks +
subscriptions + hashtags (DMs OK); full unit suite green; subscription loop
+ aggregation tests still pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The live subscription-list subscription (kind 30000) delivers our OWN list
event back through handleSubscriptionListEvent, which called updateFromRelays
on EVERY event. updateFromRelays re-fetches the list, the relay echoes the
same event through the live sub, and the handler fires again — a
self-sustaining loop that floods the cold-login burst with redundant
kind-30000 REQs (the kind 30000=19 storm in the real-env console,
KNOWN_BUGS #0). Same shape as the contact-list emit→fetch→emit loop fixed
earlier.
Add an emit-on-change guard: skip the refresh when the event is one we've
already acted on (by id — matters during the pre-decrypt login window where
subsEventCreatedAt isn't set yet) or is not newer than what we've applied.
A genuinely newer list event still refreshes exactly once.
Also extend the channel-sim harness to capture per-filter REQ signatures and
kind-30000 REQ timing, which is how this loop was isolated.
Test: tests/subscriptions-manager.test.mjs — echo of the same event id does
not re-refresh; an older event is ignored; a newer event refreshes once.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Grow the deterministic channel-simulation harness from blocks-only to a
two-session, multi-list rig so every cold-login decrypt path can be
validated locally:
- Session 1 logs in programmatically (loginWithNsec, same key the fake
extension uses) on a HEALTHY channel and publishes the user's real
encrypted lists via the services (block list + subscription list +
hashtag interest) through new testHarness.seedTestLists().
- Session 2 logs in via the NIP-07 path under the target channel MODE and
reports, per list, whether it decrypted — plus signer-ready time, the
login REQ burst (per kind), decrypt timeouts, breaker activity, and a DM
load probe.
testHarness: expose subscriptions + hashtag load state on getAppState().lists
and add seedTestLists() (resolves the actor from nostrClient.pubkey, which —
unlike the registry getActiveSigner() — is populated for a NIP-07 login).
Verdict is mirrored to a report file and uses process.exitCode so results
survive stdout buffering when piped/backgrounded.
Validation:
MODE=healthy → blocks/subscriptions/hashtags all decrypt, DMs OK.
MODE=overload → 26 injected channel drops, ALL THREE lists still recover
via the transient-retry fix (db0fc079), DMs OK.
The harness exits non-zero if a list that should load doesn't, so it can
gate CI. Confirms the burst is still ~60 REQ frames at login (kind 30000
dominant) — the next target. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Built a deterministic NIP-07 channel-simulation harness
(scripts/perf/nip07-channel-sim.mjs) that reproduces the real cold-login
failure locally with a fake extension whose message channel drops under
concurrent load (MODE=overload), recovers (MODE=slow), or stays dead. It
seeds a real encrypted block list and reports whether it decrypts.
The harness pinned the actual root cause that the relay/breaker/timeout
work hadn't fixed: when the channel briefly drops during the login burst,
the per-scheme decrypt probe (Promise.any) rejects with
"message channel closed" — and every list service SWALLOWED that and
treated it as "the user has no blocks / no subs / no hashtags", or routed
it to a give-up branch. Either way: no retry, so the list stayed empty
until a full page refresh, even though the channel recovered seconds later.
Fix: classify transient failures (channel-death messages, breaker fast-fail
code, our own decrypt-timeout) and route them to the existing keep-stale +
scheduleDecryptRetry path instead of swallowing/abandoning:
- userBlocks: re-throw a transient sub-error out of the Promise.any probe
(was silently returning [] = "no blocks"); route it (and channel/breaker
errors) through the retry branch.
- hashtagPreferencesService + subscriptions: broaden the retry condition
from the timeout code only to any transient error, recursing into the
nested Promise.any AggregateError causes.
Harness verdict (MODE=overload, 16 channel drops): block list
BLOCK LIST DECRYPTED NO -> YES after the fix; signer-ready ~1.9s.
Adds getAppState().lists observability to the test harness. Full unit
suite green. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
userBlocks grew (P4 handle teardown) — bump baseline. flaky-relays grid-render
timeout 15s->30s so dead-relay connect timeouts don't cause a false-negative
(ux-harness with a healthy relay renders in ~1.2s; the harness now reliably
passes 4/4).
Stands up several mock relays — some alive (seeded), some dead (unreachable
ports) — points the app at all of them, logs in via the fake nip-07 extension,
and asserts the refactor invariants under reconnect churn:
1. grid renders from alive relays
2. subscribes on <= the capped core (not all configured relays)
3. no steady-state REQ storm despite dead relays (was 1000+/s)
4. report subs are batched (one kind-1984 filter with many #e, no per-id)
All four PASS on the current build. This is the realism the single perfect mock
relay lacked — the reason earlier 'fixed' verdicts didn't hold in production.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Headless Playwright probes that reproduce the real login path via a fake
window.nostr extension (which loginWithNsec bypasses) and measure
steady-state extension-call and relay-REQ rates:
- nip07-heat.mjs, req-capture.mjs, ws-lifecycle.mjs, loopc-capture.mjs, etc.
Plus js/devReqMonitor.js: a localhost-only WebSocket REQ-rate monitor that
warns in-console when relay traffic spikes, naming the storming kinds — used
to capture the real-environment storm the mock-relay harness could not see.
Removable once the refactor lands.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
wh-repro.mjs uses a SEPARATE browser context per session (seed vs cold), so the
persisted decrypt cache doesn't warm between runs, and captures the page-side
stack of every window.nostr.nip44.decrypt call.
It root-caused the watch-history cold-load cost: history is bucketed by month
(one encrypted kind-30079 snapshot per month) and the load decrypts EVERY month
via the extension — 600 items / 18 months => 18 serialized decrypt calls. So the
cost is ~1 nip-07 decrypt per month of history; a multi-year history is the ~20s
cold load. (The earlier "~180" figure was a shared-context harness artifact.)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Consolidates the persisted-plaintext-decrypt-cache pattern (already used by watch
history) into a shared factory, persistedPlaintextCache.js, and adds a DM cache
on top of it. watchHistoryDecryptCache now delegates to the factory (same public
API; tests unchanged).
decryptDirectMessageEvent now also consults/writes a persisted cache keyed by the
immutable DM event id (JSON of the decrypt result; flushed once per
listDirectMessages batch; cleared on logout alongside the in-memory cache). DM
events never change, so a reload re-decrypts only messages it has not seen and
skips the (serialized, slow) nip-07 extension for the rest.
Harness (dm-repro, now cold/warm): 20 DMs -> COLD nip04.decrypt 50, WARM 0 —
reloads cost zero extension decrypt calls. Also excludes the actor's own key from
NIP-04 remote candidates (decrypting against self is always wasted). DM unit +
integration and watch-history cache tests green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The DM-repro harness (fake nip-07 extension) showed legacy DM history made ~12
extension decrypt round-trips PER message (20 DMs -> 241 calls: nip44.decrypt
121 + nip04.decrypt 120). At a real ~100ms extension latency that's ~24s, which
is why DM history "looked broken" — it was just catastrophically slow.
Two causes in decryptLegacyDm:
- It raced EVERY (decryptor x remote-pubkey) combination in parallel. That looked
like a latency win, but the nip-07 extension serializes calls, so racing N
combinations just queued N slow round-trips per message.
- orderDecryptors ranked nip44 first (by candidate priority), so every kind-4
(always NIP-04) message burned a failing nip44 extension call first.
Fix: order nip04 first for legacy DMs (preferLegacy) and try candidates
SEQUENTIALLY, stopping at the first success. With the right scheme and remote
party tried first, it's ~1-3 calls/message instead of ~12. Harness after: 20 DMs
-> 60 calls, zero nip44 attempts (was 241). DM unit + integration tests green.
Adds scripts/perf/dm-repro.mjs (seeds kind-4 DMs, logs in via the fake extension,
calls listDirectMessages, reports decrypted count + extension call counts).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Extends the fake-nip-07-extension harness to a multi-session flow that publishes
a watch history via the app (real chunking + encryption through the fake
extension), then reloads to measure the decrypt-cache effect by counting
nip44.decrypt extension round-trips:
seed : publishes the snapshot (nip44.encrypt per chunk)
COLD : first reload — decrypts chunks via the extension (~180 calls @ 300 items)
WARM : second reload — decrypt cache hits, ~2 calls
Validates the watch-history decrypt cache (watchHistoryDecryptCache.js): the
warm reload skips ~180 serialized extension round-trips (~9s @ 50ms, more at
real latency). Uses context-level bindings so localStorage (and the cache)
persists across reloads; bypasses feed signature verification of seeded data via
the test-only videoEventVerifier hook so the feed renders deterministically.
Also surfaced: cold load still pays the full decrypt cost, and the decrypt path
makes far more extension calls than there are chunks — an inefficiency to chase.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The UX harness (fake nip-07 extension) revealed that the off-thread verification
could blank the feed: if nostr-tools is slow/unable to load inside the verify
worker, the worker hangs, each batch waits the full 10s timeout, and the
main-thread fallback could return an empty set (fail-closed) — dropping every
event so no videos ever render, silently (devLogger is suppressed in prod).
Harden it so verification can never stall or blank the read feed:
- Worker timeout 10s -> 2.5s, and disable the worker for the session after one
timeout/error so we never pay the timeout on every batch.
- Bound the main-thread nostr-tools wait (Promise.race ~2s) so a slow/failed
load can't hang the fallback.
- Fail OPEN when the verifier is genuinely unavailable: pass events through
unverified rather than dropping them (the pool already does not verify, so this
does not regress the prior posture). Real verification still runs whenever
nostr-tools is available.
Adds a regression test (hung worker -> resolves via fallback, worker disabled).
Also updates the harness to seed mode:"live" videos + a NIP-65 relay list for
mock-relay isolation.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Lets the agent observe the real app the way a user does: runs bitvid in headless
Chromium against an in-process mock relay, injects a fake window.nostr extension
backed by Node-side nostr-tools crypto (via page.exposeFunction) with a
serialized ~100ms latency queue to model a real extension. Drives the real
nip-07 login flow, isolates to the mock relay via a seeded NIP-65 list, and
captures console logs, per-method extension call counts, feed-render timing, and
screenshots.
Phase 1 (this commit) verifies fake-extension login + relay isolation + metrics.
Already surfaced a stuck "Fetching recent videos…" state to investigate. Next
phases: seed encrypted watch-history chunks and DMs to reproduce/verify the
decrypt-cache wins and the DM-history bug, with before/after extension-call
counts.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Standalone Playwright/CDP profilers used to find the main-thread bottlenecks:
- profile-live.mjs: cold-load CPU profile + long-task report against a URL.
- diagnose-feed.mjs: dumps feed DOM/app-state/console to explain empty feeds.
- profile-seeded.mjs: seeds an in-process mock relay, drives login/tab-switch/
modal-open in test mode, and attributes secp256k1 self-time to its nearest
app-level caller.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
