Foundation for bounded watch-history loading (slice B part 2). A deep
history references many monthly chunks; under a NIP-46 remote signer each
uncached chunk decrypt is a serialized published relay RPC, so decrypting
them all up front blocks the first render and risks the relay rate limit.
manager.fetch now:
- sorts chunk events newest-first by created_at, so the decrypt budget is
spent on the most recent history;
- accepts a chunkDecryptLimit option that caps how many *uncached*,
genuinely-encrypted chunks are decrypted per fetch. Cached/plaintext
chunks are always decrypted (they cost no RPC) and never count against
the limit. Default is Infinity -> no behavior change for existing
callers;
- returns deferredChunkCount so a caller can tell more history is
available on demand (re-fetching with a larger limit reuses the chunk
cache, so only the newly requested older chunks cost RPCs).
Test: two uncached nip44 chunks (newest + older) with chunkDecryptLimit:1
asserts only the newest is decrypted (1 RPC), deferredChunkCount is 1, and
only the newest chunk's items appear. Mutation-verified against both
removing the cap and removing the newest-first sort.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch-history decryption previously ordered the matching cipher family
first but kept the other family as a *fallback*. The opposite family can
never decrypt a given ciphertext (NIP-04 always carries a literal "?iv="
marker; NIP-44 is plain base64), so that fallback was a guaranteed-wasted
decrypt attempt. Under a NIP-46 remote signer every decrypt is a
*published* relay RPC routed through the serial 250ms request queue, so a
transient failure on the correct family triggered a second wasted RPC
that burned the queue and pushed cold loads toward the relay rate limit
("noting too much") — the symptom users hit.
determineWatchHistoryDecryptionOrder now filters the attempt set to the
single family implied by the ciphertext shape (trusting the ciphertext
over a possibly-misleading "encrypted" tag), with a safety fallback to
the full ordered list when nothing matches so the only available
decryptor is never stranded. This mirrors the list-decrypt routing
already shipped for userBlocks / subscriptions / hashtags (slice A).
Test: a NIP-44 ciphertext mislabeled with an ["encrypted","nip04"] tag,
against a signer exposing both decryptors where nip04 returns decoy data.
Asserts nip04 is never attempted and the nip44 plaintext is used.
Mutation-verified: removing the filter fails the test.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Under a NIP-46 remote signer, every decrypt is a PUBLISHED relay RPC. The list
services (blocks/subscriptions/hashtags) probed all decryptor families
concurrently via Promise.any — so each list event published nip04 AND nip44 RPCs
at once. On login that flooded the relay ("rate-limited: you are noting too
much"), and the racing/queued RPCs blew past the 30s decrypt budget → the
"Decryption timed out" failures.
A NIP-04 payload carries the "?iv=" marker and can ONLY be decrypted by nip04;
otherwise it's nip44. The other family can never succeed on a given ciphertext,
so racing it is pure waste (and a wasted published RPC under NIP-46). Restrict
the decrypt probe to the family matching the ciphertext format (fall back to all
if the format is somehow unknown). Halves cold-load RPCs and stops the racing
publishes that tripped the rate limit; benefits all signer types.
Spec correction: the hashtag "retries decryptors when hinted scheme fails" test
stubbed an impossible scenario (the same ciphertext decrypting under nip04 after
nip44 failed). Rewritten to assert the real contract — the ciphertext format
routes decryption and overrides a misleading `encrypted` tag, invoking only the
matching decryptor.
test_integrity_note:
change_type: ["spec_correction"]
scenarios:
- id: SCN-decrypt-format-routing
given: "a nip04-format ciphertext (?iv=) tagged misleadingly as nip44"
when: "a list service decrypts it"
then: "only the nip04 decryptor is invoked; nip44 is never attempted"
anti_cheat_rationale:
prevents: ["over-mocking internal logic", "asserting impossible cross-family decryption"]
relaxation:
did_relax_any_assertion: false
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The NIP-46 login dialog is purely the QR / connect-link flow ("scan with Amber/
NsecBunker"), but it failed with "A remote signer pubkey is required." A
nostrconnect:// link carries the CLIENT pubkey; the remote signer's pubkey is
only learned after it scans the QR and acknowledges — and the active connect
path (SignerManager) only supported bunker:// (pubkey already in the URI). The
"wait for the signer to acknowledge" logic existed only on the unused
Nip46Connector.
Port it into SignerManager as _waitForRemoteSignerHandshake: subscribe to
kind-24133 events addressed to the client pubkey, decrypt the signer's response,
match the handshake secret (or a generic ack), surface auth_url challenges, and
resolve with the signer's pubkey (with a timeout + a cancel hook wired into
disconnectRemoteSigner). connectRemoteSigner now performs this wait for
client-initiated (nostrconnect) connections before constructing the RPC client;
bunker:// is unchanged.
Decryption uses the existing attemptDecryptNip46HandshakePayload (injectable via
_decryptHandshakePayload for testing). Test: tests/nostr/nip46-wait-handshake.test.mjs
(secret match resolves with signer pubkey; wrong secret ignored -> timeout;
auth_url surfaced without resolving). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
connectRemoteSigner and useStoredRemoteSigner accepted but never used the
`validator` the auth provider passes (accessControl.canAccess) -- so NIP-46
logins bypassed the block/allow gate that nsec/NIP-07 logins enforce. (The
validation lived only on the unused Nip46Connector.)
Both paths now run the validator against the remote signer's resolved user
pubkey BEFORE activating the signer (via _enforceRemoteSignerValidator): the
validator throws on a blocked/non-permitted pubkey, or returning false rejects
with code remote-signer-access-denied. A stored session rejected by the
validator is forgotten so a blocked pubkey can't keep retrying.
Residual (noted in TODO): the silent startup auto-restore
(scheduleStoredRemoteSignerRestore) still reconnects without a validator.
Test: tests/nostr/nip46-prepare-handshake.test.mjs (validator throws / false /
true / absent). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
NIP-46 login via a generated connect link failed with
"nostrClient.prepareRemoteSignerHandshake is not a function". The auth provider
(authProviders/nip46.js) calls nostrClient.prepareRemoteSignerHandshake() to mint
an ephemeral client keypair + nostrconnect:// URI before connecting, but that
method existed only on the unused Nip46Connector class — neither SignerManager
nor NostrClient exposed it.
Add SignerManager.prepareRemoteSignerHandshake (generates the keypair via
nostr-tools, resolves relays, builds the nostrconnect:// URI with relay/secret/
perms/metadata params) and proxy it on NostrClient next to connectRemoteSigner.
Test: tests/nostr/nip46-prepare-handshake.test.mjs. Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Two gaps where generic S3 trailed Cloudflare R2:
1. Object cleanup on delete/edit didn't work for generic S3. deleteVideoStorage
built its client via makeR2Client, which hard-forces path-style and ignores
the connection's forcePathStyle -- so vhost-style S3 deletes were misaddressed
and cleanup silently failed (S3 users' deleted/edited files orphaned). It is
now provider-aware (mirrors uploadFile): R2 -> makeR2Client (path-style),
generic S3 -> makeS3Client honoring forcePathStyle. makeS3Client is now
injectable for testing.
2. CORS guidance on upload failure was R2-only. Extracted isLikelyCorsError +
a generic-S3 guidance builder into js/utils/uploadErrorHints.js (re-exported
from r2Service for back-compat); s3UploadService now appends actionable CORS
guidance on an opaque "Failed to fetch" too.
Generic S3 is now at functional parity with R2 for upload -> publish -> cleanup.
Tests: r2-delete-video-storage (adds generic-S3 forcePathStyle routing),
s3-upload-cors-guidance (new). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A new user's top first-upload failure is a missing R2 CORS policy (browser
PUT/POST blocked) -- and object-scoped R2 tokens often cannot auto-set CORS.
The connection test already surfaces buildCorsGuidance on the opaque
"Failed to fetch", but a user who skips the test and uploads directly got a
bare "Upload failed: Failed to fetch". The upload error handler now detects the
browser CORS/network rejection (isLikelyCorsError -- Chrome "Failed to fetch",
Firefox "NetworkError", Safari "Load failed") and appends the same actionable
guidance. Updated the R2 follow-up TODO (credential audit done; CORS reviewed).
Test: tests/services/r2-cors-error-classifier.test.mjs. Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Editing a video with a newly-uploaded file produced a new content-addressed
object and orphaned the old one. editModalController.handleSubmit now removes the
old object (best-effort) when the hosted URL was genuinely replaced — guarded so
it only fires when both old and new URLs are present and different (so clearing
the URL, which may still be referenced by a magnet web-seed, never deletes it),
and it deliberately leaves thumbnails alone (the new note may still point at
them). Reuses R2Service.deleteVideoStorage (owned-objects-only, never throws).
Documented the cleanup contract in AGENTS.md §4a.
Test: tests/ui/edit-modal-storage-cleanup.test.mjs (3). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Deleting a video removed the Nostr note but left the actual file (plus its
thumbnail and .torrent) publicly downloadable at its R2/S3 URL forever — a
privacy gap and ongoing storage cost. Now the delete flow also removes the
backing objects, best-effort.
- collectVideoStorageKeys() (storagePointer.js): pure helper that derives the
object keys (video + sibling .torrent + thumbnail) from a note, ONLY for URLs
under the owner's bucket base — external/3rd-party URLs are never returned.
- R2Service.deleteVideoStorage(): resolves the owner's connection, and deletes
those keys. Skips cleanly when storage is locked / no creds / no owned
objects, and never throws (failures are recorded, not propagated) so storage
cleanup can't block note deletion. R2Service class now exported for testing.
- app.handleDeleteModalConfirm: calls it after handleFullDeleteVideo succeeds.
Tests: tests/storage-key-collection.test.mjs (4),
tests/services/r2-delete-video-storage.test.mjs (4). Full suite green.
Note: superseded versions orphaned by past edits aren't retroactively cleaned;
edit-time cleanup (next) prevents new orphans going forward.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Follow-up to c79120ba (which only captured the file deletion): removes the
sidebar nav link and the unused DEV_DASHBOARD view constant. No dangling refs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
publishEventToRelay optimistically reports success when a relay handle exposes
no ok/failed event and isn't thenable (legacy 'seen'-only relays). That made an
UNCONFIRMED publish indistinguishable from a real ack, so a note could show
'Published' with nothing actually stored. The result now carries optimistic:true
for that path (callers can distinguish it from a confirmed ack) and logs a
dev.warn. Behavior is otherwise unchanged (still treated as success for legacy
relays). Publish audit #3.
Test: tests/nostr-publish-optimistic.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
These optional-asset uploads are non-fatal, but failures were warn-only — the
user got a silently-missing thumbnail or torrent with no signal. Both upload
services now emit a 'warning' status at the point of failure and append a note
to the final 'Published' status (downgraded to warning variant) listing what
failed. Cloudflare-upload audit #5.
Test: tests/services/s3-upload-optional-asset-failures.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A 0-byte file produced a multipart upload with zero parts, which S3/R2 reject
with a cryptic CompleteMultipartUpload error. Guard file.size === 0 up front
with a clear message, before any S3 request is issued. Cloudflare-upload #4.
Test: tests/multipart-empty-file-guard.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Computing the torrent info-hash (and the content-hash fallback) reads the entire
file before the upload starts and before any progress is emitted, so on
multi-GB videos there was a long silent pause that looked like a freeze. Both
upload services now emit an 'Analyzing video…' status when a full-file hash is
required (i.e. no valid info-hash was pre-supplied). Cloudflare-upload audit #2.
Test: tests/services/s3-upload-analyzing-status.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
buildR2Key fell back to a static "uploads" namespace (and "video" slug) when
no torrent info-hash was available, so two distinct URL-first uploads sharing a
filename produced the SAME R2/S3 key — the second silently overwrote the first
object while the first video note still pointed at that URL (data loss).
WebTorrent uploads were already safe because the info-hash namespaces the key.
Add computeStorageContentHash(file) in js/r2.js: when no info-hash is present it
derives a deterministic content-based namespace — full SHA-256 of the bytes for
files ≤512 MB (idempotent: re-uploading the same file reuses the key), and a
SHA-256 fingerprint over name/size/mtime + sampled head/tail bytes above that,
so we never buffer a multi-GB video in memory. Wired into BOTH upload paths
(r2Service + s3UploadService) via a new storageIdentifier kept separate from the
info-hash, so magnet generation is unchanged.
Test: tests/storage-key-collision.test.mjs (distinct same-named uploads get
distinct keys; same content is idempotent; large-file fingerprint path samples
edges without buffering and stays deterministic). Full unit suite green.
test_integrity_note:
change_type: ["new_tests"]
scenarios:
- id: SCN-upload-key-no-collision
given: "a URL-first upload with no torrent info-hash"
when: "the storage key is derived (computeStorageContentHash -> buildR2Key)"
then: "distinct same-named files get distinct keys, same content is idempotent, and no key uses the colliding 'uploads' namespace"
observable_outcomes:
- "returned key strings for real Blob/File content and a fake >512MB file"
determinism_controls:
- "fixed byte content; fake large file with stubbed slice() and fixed lastModified"
anti_cheat_rationale:
prevents:
- "hard-coded return value"
- "over-mocking internal logic"
relaxation:
did_relax_any_assertion: false
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Post-login the site froze for ~10-15s and the profile panel wouldn't open.
Cause: profileModalController.handleAuthLogin eagerly rendered EVERY profile
panel at login (twice) — friends list + per-contact avatar fetches,
subscriptions, blocks, relays, wallet, hashtags, storage — into a CLOSED
modal. With a responsive signer (nos2x) making decryption instant, that all
fired in one main-thread burst right after login. The block- and
subscription-"change" listeners and the moderation "contacts" listener also
re-rendered their panels unconditionally as that data loaded, compounding it.
selectPane() already lazily populates each pane when it's shown, so the eager
pass was redundant. Changes:
- handleAuthLogin: only run the full panel population when the modal is
already open (e.g. account switch with it visible); otherwise rely on
open()/selectPane. Kept the cheap DM-identity setup eager.
- Gate the userBlocks "change", subscriptions "change", and moderation
"contacts" re-renders on isProfileModalOpen() so they don't paint into a
closed modal during the post-login data burst.
- Add a "friends" case to selectPane() (the only pane without a lazy-populate
trigger) so it still populates on open now that login doesn't.
- Add isProfileModalOpen() + populateOpenProfilePanels() helpers.
All panes remain covered (account is static; admin refreshes independently).
Full build + unit suite green. KNOWN_BUGS #0 (post-login freeze).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
KNOWN_BUGS #1: logged out, the video grid stayed empty after dismissing the
disclaimer until a manual page refresh.
Reproduced deterministically with scripts/perf/disclaimer-grid.mjs (fresh
context, real relays) once it actually waits for and dismisses the
disclaimer: post-dismiss card timeline was 0,0,0,… and only a reload
yielded 132 cards.
Root cause: bootstrapInterface (js/index.js) opens the disclaimer — which
makes the background inert via staticModalAccessibility — and THEN awaits
handleHashChange(), which loads the feed view and renders into that
inert/hidden background, so the initial grid mount/render is dropped.
Dismissing the disclaimer never re-triggered a render.
Fix: disclaimer.js hide() now dispatches a `bitvid:disclaimer-dismissed`
event; index.js listens once and calls application.loadVideos(true) so the
feed re-renders into the now-interactive background. Only fires when a shown
disclaimer is dismissed, so returning visitors (who never see it) get no
redundant reload.
Harness after fix: post-dismiss timeline 132,132,… immediately. Full build
+ unit suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The controller→Actions split left a batch of method calls in
ProfileDirectMessageActions pointing at `this` (the Actions instance)
instead of the object that actually defines them. Opening a conversation
threw "this.setFocusedDmConversation is not a function"; the same latent
bug also broke DM send-error messaging, read receipts, settings publish,
and the whole attachment flow (generate/render/reset/upload) — the user
just hadn't exercised them yet.
Route each to its real receiver:
- DM-controller methods → this.controller.*:
handleDmSettingsPublish, describeDirectMessageSendError,
maybePublishReadReceipt, setFocusedDmConversation
- main-controller methods → this.mainController.*:
generateAttachmentId, renderAttachmentQueue, resetAttachmentQueue,
uploadAttachmentQueue, clearProfileMessages
(Builds on the earlier handleDmConversationSelect delegation fix.) A static
sweep confirms every remaining `this.*` call in Actions resolves to an
Actions method or an explicit controller/mainController/helper receiver.
Test: tests/dm-conversation-select.test.mjs (cheat-resistant — drives the
real handleDmConversationSelect with stub receivers; would throw on the old
wrong-receiver code). DM unit + integration suites green; full build OK.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
After switching to a responsive signer (nos2x), two DM-specific bugs surfaced.
1. Clicking a conversation in the list threw
"this.controller.handleDmConversationSelect is not a function"
(ProfileDirectMessageRenderer onSelectConversation). The method exists on
ProfileDirectMessageActions and the controller delegates ~15 sibling Actions
methods — but the handleDmConversationSelect delegation was missing. Add it.
2. Loading a conversation's messages threw "DM decryption helpers are
unavailable" even though the signer decrypts blocks/subscriptions/hashtags
fine. buildDmDecryptContext gated decryptor registration on (a) the dynamic
`capabilities` getter (which probes window.nostr and false-negatives on lazy
injection / nos2x's module shape) and (b) the permission-probe result. The
list services already learned to register by METHOD EXISTENCE and validate at
call time; align the DM path to do the same so a present, working signer
never yields zero decryptors. The permission probe still runs.
Tests: tests/dm-decrypt-context.test.mjs (cheat-resistant — a signer whose
capabilities getter reports false and whose permission probe returns ok:false
must still register nip04+nip44 decryptors). Full unit + dm:unit + dm:integration
suites green. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A raw window.nostr probe in the user's browser (getPublicKey → nip04.encrypt
→ nip04.decrypt, bypassing ALL of bitvid) HUNG forever, with the extension's
own console showing `ObjectMultiplex - orphaned data for "background-liveness"`
+ `malformed chunk` + listener-leak warnings. That is the NIP-07 extension's
content-script ↔ background-service-worker channel being dead (classic MV3
worker death) — an extension/environment failure, not a bitvid bug. No client
change can force an unresponsive signer to answer; the fix is to reload/unlock
the extension.
What bitvid CAN do is stop failing silently. Add js/utils/signerHealthNotice.js:
after a few consecutive list-decrypt timeouts while the signer reports itself
present, emit ONE rate-limited, actionable user notice ("your signer isn't
responding to decryption — unlock/reload the extension and refresh") instead of
the silent 15-minute retry loop. Reset on any decrypt success so a transient
blip doesn't warn. Wired into blocks, hashtags, and subscriptions.
Also documents the definitive root-cause + all resilience increments in
KNOWN_BUGS #0. Test: tests/signer-health-notice.test.mjs (threshold, rate-limit,
success-reset). Full unit suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
On a cold login / permission cache-miss, requestEnablePermissions probes
several permission-payload variants before the no-arg prompt. The specific
(silent) variants were capped at 3s "to fail fast" — but real extensions
REJECT an unsupported payload shape almost immediately (and a fast
rejection already falls through to the next variant), so the 3s cap added
no real safety. What it DID do: kill a slow-but-responsive signer
mid-grant, forcing all 3 specific variants to time out (3s each) before the
prompt variant finally succeeded — ~25s of dead time before signer-ready.
Raise the specific-variant timeout to 20s (SPECIFIC_VARIANT_TIMEOUT_MS).
The bound now only matters for a variant that genuinely HANGS; unsupported
shapes still reject fast, and a slow signer can actually answer the first
supported variant.
Channel-sim at 16s/call: enable extension calls 4 → 1, signer-ready
~42s → ~33s, lists-sync ~58s → ~50s; healthy login unchanged (~1.6s),
all lists still decrypt. Full unit suite green (pre-existing Node
localStorage failures unrelated). KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console: signerStatus "present" but every block/hashtag decrypt
timed out at the 15s service budget, forever, retrying every ~15s for
15+ minutes — lists never loaded.
Reproduced deterministically with the channel-sim harness (latency
override): with per-call latency BELOW the service budget all lists load;
the instant it crosses the budget every list fails (NO) and retries
forever. A nip-07 decrypt is a postMessage round-trip to the extension's
single-threaded, possibly-sleeping service worker — under cold-login load
real extensions take 15-25s to answer even though the crypto is instant.
The earlier 6s per-call timeout was killing those in-progress decrypts.
Raise the budget across blocks/hashtags/subscriptions so a slow-but-
responsive signer completes instead of timing out:
- service-level DECRYPT_TIMEOUT_MS 15s → 30s
- per-call nip-07 decrypt timeout 6s → 25s interactive / ~10-12s background
- backoff cap 30s → 60s so a persistently dead signer settles into an
occasional heartbeat instead of pinning the extension every ~15s
Harness: at 16s/call (previously NO across the board) blocks +
subscriptions + hashtags now all decrypt (0 timeouts); healthy mode
unchanged (signer-ready ~1.2s). Full unit suite green. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Two redundancies were inflating the login REQ burst (kind 30000=19 storm in
the real-env console — the cause of the slow hashtags, KNOWN_BUGS #0),
isolated with the channel-sim harness's REQ-signature + timing capture:
1. Duplicate trigger: ProfileModalController.handleAuthLogin called
subscriptions.loadSubscriptions() AND userBlocks.loadBlocks() DIRECTLY at
login, on top of the auth coordinator's gated list loads — a second,
ungated relay round-trip for each. Route them through ensureLoaded()
instead so they coalesce with the coordinator's in-flight load. Harness:
subscriptions updateFromRelays now fires ONCE at login instead of 3×
(≈3× fewer kind-30000 fetches in prod, where each is multiplied by the
relay set).
2. Fetch/decrypt coupling: a decrypt retry re-ran the full relay fetch even
though only signer availability had changed. Cache the fetched ciphertext
(_pendingDecryptEvent) and have scheduleDecryptRetry re-decrypt it via a
cachedDecryptEvent fast-path in updateFromRelays — no redundant relay
query per retry.
Validated: MODE=overload (channel drops) still decrypts blocks +
subscriptions + hashtags (DMs OK); full unit suite green; subscription loop
+ aggregation tests still pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The live subscription-list subscription (kind 30000) delivers our OWN list
event back through handleSubscriptionListEvent, which called updateFromRelays
on EVERY event. updateFromRelays re-fetches the list, the relay echoes the
same event through the live sub, and the handler fires again — a
self-sustaining loop that floods the cold-login burst with redundant
kind-30000 REQs (the kind 30000=19 storm in the real-env console,
KNOWN_BUGS #0). Same shape as the contact-list emit→fetch→emit loop fixed
earlier.
Add an emit-on-change guard: skip the refresh when the event is one we've
already acted on (by id — matters during the pre-decrypt login window where
subsEventCreatedAt isn't set yet) or is not newer than what we've applied.
A genuinely newer list event still refreshes exactly once.
Also extend the channel-sim harness to capture per-filter REQ signatures and
kind-30000 REQ timing, which is how this loop was isolated.
Test: tests/subscriptions-manager.test.mjs — echo of the same event id does
not re-refresh; an older event is ignored; a newer event refreshes once.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Grow the deterministic channel-simulation harness from blocks-only to a
two-session, multi-list rig so every cold-login decrypt path can be
validated locally:
- Session 1 logs in programmatically (loginWithNsec, same key the fake
extension uses) on a HEALTHY channel and publishes the user's real
encrypted lists via the services (block list + subscription list +
hashtag interest) through new testHarness.seedTestLists().
- Session 2 logs in via the NIP-07 path under the target channel MODE and
reports, per list, whether it decrypted — plus signer-ready time, the
login REQ burst (per kind), decrypt timeouts, breaker activity, and a DM
load probe.
testHarness: expose subscriptions + hashtag load state on getAppState().lists
and add seedTestLists() (resolves the actor from nostrClient.pubkey, which —
unlike the registry getActiveSigner() — is populated for a NIP-07 login).
Verdict is mirrored to a report file and uses process.exitCode so results
survive stdout buffering when piped/backgrounded.
Validation:
MODE=healthy → blocks/subscriptions/hashtags all decrypt, DMs OK.
MODE=overload → 26 injected channel drops, ALL THREE lists still recover
via the transient-retry fix (db0fc079), DMs OK.
The harness exits non-zero if a list that should load doesn't, so it can
gate CI. Confirms the burst is still ~60 REQ frames at login (kind 30000
dominant) — the next target. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Built a deterministic NIP-07 channel-simulation harness
(scripts/perf/nip07-channel-sim.mjs) that reproduces the real cold-login
failure locally with a fake extension whose message channel drops under
concurrent load (MODE=overload), recovers (MODE=slow), or stays dead. It
seeds a real encrypted block list and reports whether it decrypts.
The harness pinned the actual root cause that the relay/breaker/timeout
work hadn't fixed: when the channel briefly drops during the login burst,
the per-scheme decrypt probe (Promise.any) rejects with
"message channel closed" — and every list service SWALLOWED that and
treated it as "the user has no blocks / no subs / no hashtags", or routed
it to a give-up branch. Either way: no retry, so the list stayed empty
until a full page refresh, even though the channel recovered seconds later.
Fix: classify transient failures (channel-death messages, breaker fast-fail
code, our own decrypt-timeout) and route them to the existing keep-stale +
scheduleDecryptRetry path instead of swallowing/abandoning:
- userBlocks: re-throw a transient sub-error out of the Promise.any probe
(was silently returning [] = "no blocks"); route it (and channel/breaker
errors) through the retry branch.
- hashtagPreferencesService + subscriptions: broaden the retry condition
from the timeout code only to any transient error, recursing into the
nested Promise.any AggregateError causes.
Harness verdict (MODE=overload, 16 channel drops): block list
BLOCK LIST DECRYPTED NO -> YES after the fix; signer-ready ~1.9s.
Adds getAppState().lists observability to the test harness. Full unit
suite green. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console (2026-06-15) showed [signer-ready] taking ~48s and every
list decrypt timing out before AND after it. Root mechanism: a priority
inversion on the 2-slot NIP-07 request queue.
- List decrypts run at NIP07_PRIORITY.HIGH; the readiness gate's
permission/enable handshake ran at NORMAL. So a flood of HIGH-priority
decrypts jumped ahead of the gate and starved it — the channel never got
warmed, so the decrypts kept failing. Fix: add NIP07_PRIORITY.CRITICAL
(above HIGH) and run the permission/enable handshake at CRITICAL so it
always wins queue ordering.
- Login-time decrypt timeouts were 12-15s even though a healthy
nip04/nip44 decrypt is local crypto (~1s); the long window was only to
tolerate a first-grant prompt, which the gate already pre-grants. A 15s
timeout just monopolizes one of the two queue slots on a doomed call,
blocking the gate for 15s at a time. Cut blocks/hashtags/subscriptions
login decrypt timeouts to 6s so slots cycle ~2.5x faster (and the circuit
breaker trips in ~18s instead of ~45s). Service-level 15s wrappers stay
as a backstop (now strictly longer than the nip07 layer).
Part of KNOWN_BUGS #0 ("cut the burst, then add a safety net"). Full unit
suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console (2026-06-15) showed the breaker never opening even as
decrypts failed for minutes. Cause: the breaker treated every non-timeout
error as "channel alive" and RESET the timeout streak — but
"message channel closed before a response was received" is the exact
dead-channel signal, and it rejects as a (fast) non-timeout error. So each
such rejection reset the counter and the circuit could never open.
Classify channel-death messages (message channel closed / could not
establish connection / receiving end does not exist / extension context
invalidated / connection lost) as channel failures that count TOWARD
opening the circuit, instead of as responsive failures that reset it.
Genuine extension responses (user rejection, permission denied, etc.)
still reset the streak.
Test: extends nip07-channel-breaker with a synchronous channel-closed op
asserting consecutive channel-death errors open the circuit and then
fast-fail (no waits).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console (2026-06-15) showed that once the extension's message
port drops under the post-login burst ("message channel closed"), EVERY
subsequent decrypt call hangs to its full ~15s timeout. Each list service
(blocks, hashtags, subscriptions, watch history) independently retries on
that timeout, so the app sits in a permanent loop of 15s-blocking
extension calls — pinning the CPU ("fans at max") and never recovering
until a manual page refresh.
Add a shared circuit breaker at the runNip07WithRetry chokepoint:
- After CIRCUIT_TIMEOUT_THRESHOLD (3) consecutive timeouts the circuit
OPENS; further calls fail fast with code "nip07-channel-unresponsive"
(no multi-second hang), so the retry loops become cheap and the CPU
settles.
- While open, a single probe per CIRCUIT_PROBE_INTERVAL_MS is allowed
through; one success CLOSES the circuit and normal decryption resumes —
no page refresh required.
- Responsive failures (extension answered with an error) reset the
timeout streak — only genuine channel hangs open the circuit.
- Interactive permission prompts bypass the breaker entirely: a user
taking their time at the extension UI can neither trip it nor be
blocked by it, and such a prompt can itself heal the channel.
Increment 2 of the "cut the login burst, then add a safety net" work for
KNOWN_BUGS #0. Test: tests/nostr/nip07-channel-breaker.test.mjs
(cheat-resistant — drives the real queue with hanging/responsive/success
ops; asserts open-after-threshold, fast-fail, probe-recovery, responsive
errors don't open, and bypass is never fast-failed). Full unit suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console (cold nip-07 login) showed list decryption failing en
masse: blocks/hashtags/subscriptions fetched fine (relay cap working —
every sub now fans out to exactly 8 relays) but decryption timed out for
minutes with "message channel closed", only recovering on a page refresh.
Root cause: the single-threaded nip-07 extension channel is overwhelmed
during the post-login handshake. One contributor is the eager DM load —
loadDirectMessages({ limit: 50, initialLoad: true }) ran at login and
fired a burst of decrypts that competed with the feed-driving lists
(blocks/subscriptions/hashtags) for the same channel. DMs are lower
priority (the user isn't viewing Messages at login) and already load
lazily when the Messages tab opens.
Defer the eager DM load; report dms state as "deferred". The lightweight
unread-indicator refresh still runs so the badge stays accurate. First
increment of the "cut the login burst" work (KNOWN_BUGS #0).
Test: tests/app/login-defers-dms.test.mjs (cheat-resistant — asserts
loadDirectMessages is never called at login, dms state is "deferred", and
the unread indicator still runs).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
On cold login the app opened WebSockets to a user's full NIP-65 read set
(in prod often ~20 relays, most dead). The flood of failing connections
churned the main thread and starved the nip-07 extension's postMessage
round-trips, so encrypted lists (blocks, subscriptions, hashtags) were
fetched but timed out during decryption ("message channel closed",
"Decryption timed out").
Bound every read/subscribe path to MAX_SUBSCRIBE_RELAYS=8 via a shared,
test-aware capReadRelays helper, applied at the chokepoints:
- client.js applyRelayPreferences (this.relays / this.readRelays)
- relayBatchFetcher.js fetchListIncrementally (blocks, subs, hashtags)
Writes stay uncapped (publish reach preserved).
capReadRelays is user-relays-first (a user's own relays are where their
data authoritatively lives, and small fast-path limits would otherwise
crowd them out — the regression that reverted the first attempt) while
reserving RESERVED_DEFAULT_RELAY_SLOTS=2 for reliable default aggregators
so the "~20 personal relays, all dead" case still reaches a live relay.
Under test the reserved core resolves to the test-relay override, keeping
harnesses isolated.
Tests: new relay-subscribe-cap coverage (bounded + reserved-defaults +
writes-uncapped); spec-corrected user-blocks, subscriptions-manager and
nostr/client fetch tests to bounded (not exact) relay fan-out while
keeping all behavioral/security assertions (per-relay coverage, correct
decrypt, block hydration, dedup) intact. See TEST_INTEGRITY.md and
KNOWN_BUGS #0/#4. Full unit + dm:unit suites green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console shows the encrypted lists are FETCHED but DECRYPTION fails on
cold login: hashtags time out at 6s, blocks report signerStatus 'missing', DMs
throw 'helpers unavailable'. Root cause: a relay-connection storm to the user's
~20 dead NIP-65 relays (subsystems use uncapped relayManager read+write) starves
the nip-07 extension. Bumping the decrypt timeout to 15s (matching blocks) is a
partial mitigation for hashtags/subscriptions. The proper fix (bound reads to a
liveness-ranked healthy core incl. defaults + DM decrypt retry) is documented in
KNOWN_BUGS #0 — a first capReadRelays attempt was reverted because capping block
reads broke the security-critical user-blocks integration test.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The For You feed was chronological, hashtag-interest-EXCLUSIVE, and ignored
follows entirely (docs/feed-algo-audit.md). Now it is inclusive + ranked:
- createForYouScorerStage scores each video: follows-boost (author you follow)
+ interest/watch-topic affinity (cosine vs your interests+watch-history vector)
+ freshness - disinterest penalty. Reuses Explore's vector/similarity helpers.
- createForYouScoreSorter ranks by that score (muted last, recency tiebreak).
- For You pipeline: interests now BOOST not exclude (enforceInterests:false);
scorer added; chronological sorter replaced. Never empty (falls back to
recency when you have no follows/interests/history).
- buildForYouFeedRuntime now feeds subscriptionAuthors + watchHistoryTagCounts.
Watched/blocked/muted are still removed (suppression/blacklist/moderation
stages unchanged). Cheat-resistant tests (older boosted items only win via the
boost). Grid renders end-to-end (~1.3s). Weight tuning is a follow-up.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
handleBlocksLoaded reloaded the feed on every 'blocks-loaded' signal — but a
forced reload re-fetches the lists, which re-emit 'blocks-loaded', a
self-sustaining loop. Now it computes a signature of the block set and reloads
only on actual change (emit-on-change — the P5 principle). handleRelaysLoaded
only touches relay UI; hashtag prefs already dedup by signature.
Cheat-resistant regression test added. Bumped authSessionCoordinator file-size
baseline. Logged 2 pre-existing app test failures (fail at pre-session baseline)
in KNOWN_BUGS. Marks KNOWN_BUGS #2 resolved.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
performProfileFetch now issues ONE deduped, health-gated list() across all
relays via the L1 SubscriptionManager (the pool merges per-relay results),
replacing the manual per-relay fan-out. Newest-per-author selection unchanged;
legacy per-relay path retained as the manager-less fallback.
Spec correction: tests/app-batch-fetch-profiles.test.mjs asserted one query
PER relay (implementation detail); updated to assert one BATCHED query across
all relays. All behavioral assertions (cached hydration, newest profile
rendered, failing relay tolerated) unchanged; added a relays-targeting
assertion. Documented in TEST_INTEGRITY.md (SCN-profiles-batched).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
kind-10000/30002 block-list live subscription now goes through L1 (dedup +
reconnect re-issue), handle closed on teardown; relaySubscriptionService kept as
manager-less fallback. user-blocks + dm-block-filter suites pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The two feed-driving list subscriptions (kind-30000 follows, kind-30015/30005
hashtag prefs) now go through the L1 SubscriptionManager instead of the global
relaySubscriptionService singleton: centralized dedup + reconnect re-issue (the
lists self-heal after a relay drop). Handles are stored and closed on teardown;
relaySubscriptionService is retained as the manager-less fallback.
Verified: subscriptions-manager/-feed + hashtag-preferences suites pass (32);
flaky multi-relay harness still all-PASS; pool-access lint clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
watchHistory.fetch now reads via the L1 SubscriptionManager (deduped +
health-gated) instead of pool.list directly: a new getSubscriptionManager dep
is threaded from the client, and both fetches (index + chunks) go through a
listEvents() helper. Raw pool.list is retained only as a fallback for
environments without a manager (tests/mocks), so the file stays grandfathered.
Verified: watch-history unit test passes; wh-repro cold resolve returns 10
items with 2 decrypts (routing intact end-to-end).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Replaces the per-card view-count transport (each card: list + NIP-45 COUNT +
live sub, × relays — the ~880 REQ/s storm) with ONE shared batched kind-30079
subscription + ONE backfill list() over all visible cards, bucketed back to
per-pointer state by #a. Re-enables the VideoListView circuit-breaker.
Grid counts are convergent/approximate (cheap, persists via the existing view
cache); the watch page opts into an exact NIP-45 COUNT for its single pointer
(modalCoordinator passes { exact: true }) — accurate where it matters, one
request, no storm. Legacy per-pointer path retained for SubscriptionManager-less
environments (tests/mocks).
Verified: nip07-heat steady-idle with views on; flaky multi-relay harness still
all-PASS; new cheat-resistant batching+bucketing test. (Pre-existing
view-counter publish-session-tag failure is unrelated — fails at baseline.)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Replaces the per-video live report subscription (one pool.sub per visible
video × ~20 relays, re-fired on every reconnect — the ~1000 REQ/s storm) with
ONE batched SubscriptionManager subscription over the whole active id set,
updated in place as the set changes. Backfill fetches history only for
newly-added ids (one batched list()). Unchanged sets are a no-op (dedup guard).
MODERATION_REPORT_SUBS_ENABLED flipped back on. Trusted-mute subs were already
batched-by-author, so the relay cap + contact-loop fix already tamed them.
Legacy per-id path retained only for SubscriptionManager-less environments
(unit tests / mock clients). All 39 moderation tests pass; new cheat-resistant
batching test added; nip07-heat shows reports no longer storm (steady idle).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
subscribeVideos now goes through the L1 SubscriptionManager instead of calling
pool.sub directly: health-gated relay resolution (healthy-or-default fallback
preserved), centralized dedup, and registration for reconnect re-issue. The
buffer + .unsub() contract are unchanged. The client's manager uses its own
RelaySubscriptionService instance (isolated from the global singleton).
Verified: subscribeVideos relay-fallback + client + nostr-service suites pass;
ux-harness renders the grid (first card ~1.1s); steady-state REQ/ext idle.
(Pre-existing nostr-delete-flow failure confirmed unrelated — fails at the
pre-session baseline too.)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
P1 foundation of the data-flow refactor (docs/architecture-refactor.md):
- js/nostr/subscriptionManager.js: the single chokepoint for relay reads.
Builds on relaySubscriptionService (dedup by key) and adds: health-gated
default relays, deduped one-shot list() with timeout, and a reconnect
registry that re-issues active subs centrally (instead of every subsystem
re-subscribing on reconnect — the storm amplifier). Callers pass batched
filters (one REQ for many ids), never per-item.
- Export RelaySubscriptionService class for test isolation.
- scripts/check-direct-pool-access.mjs + lint:pool-access: fail on NEW direct
pool.sub/list usage; 12 grandfathered files listed as migration targets
(allowlist may only shrink).
Nothing rewired yet — additive. Pilot (video feed) and per-subsystem
migration follow. Cheat-resistant tests included.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A user's NIP-65 list can carry ~20 relays, most dead. Subscribing on all of
them is the dominant REQ-storm amplifier: every subsystem fans every query
across all relays and re-fires on each flaky-relay reconnect. Cap the
read/subscribe set to a bounded core (6), prioritizing the bundled known-good
defaults; WRITES still fan out to the full set so publish reach is unaffected.
Interim until RelayHealth ranks by liveness (docs/architecture-refactor.md P1).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Headless Playwright probes that reproduce the real login path via a fake
window.nostr extension (which loginWithNsec bypasses) and measure
steady-state extension-call and relay-REQ rates:
- nip07-heat.mjs, req-capture.mjs, ws-lifecycle.mjs, loopc-capture.mjs, etc.
Plus js/devReqMonitor.js: a localhost-only WebSocket REQ-rate monitor that
warns in-console when relay traffic spikes, naming the storming kinds — used
to capture the real-environment storm the mock-relay harness could not see.
Removable once the refactor lands.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-environment console logs revealed nip-07 login pegging the CPU and
starving encrypted-list decryption via relay REQ storms (~1000 REQ/s):
- moderationService.rebuildTrustedContacts emitted "contacts"
unconditionally, forming an emit->fetch->emit loop (~1600 kind-3 REQ/s).
Now emits only on actual change. Regression test added.
- Per-video live report (kind 1984) and view-count (kind 30079)
subscriptions fan out O(videos x relays) and re-fire on every flaky-relay
reconnect. Circuit-broken behind flags until batched (see refactor plan).
- onVideosShouldRefresh now coalesces forced reloads (2s leading-edge
throttle) so list-change signals can't drive a reload loop.
- nip-07 queue reserves one foreground slot; DM decryption demoted to LOW
so background DM backfill can't starve feed-driving lists. Test added.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The legacy-DM perf reorder regressed tests/nostr/dmDecryptorPreference:
1. preferLegacy short-circuited BEFORE the explicit-hint ranking, so a kind-4
event tagged ["encrypted","nip44"] would still try nip04 first. Explicit
hints now win; preferLegacy only flips the *no-hint* default.
2. The "skip the actor's own key" optimization dropped self even when it was the
ONLY candidate (note-to-self DMs), leaving zero candidates -> ok:false. Self
is now deferred to last and attempted as a fallback instead of dropped.
Spec correction (per Dark Factory rules): for a kind-4 DM with no hints, nip04
is now the preferred default (protocol-correct + avoids a wasted serialized
nip-07 round-trip). Rewrote the preference test to assert: nip04-first default,
explicit nip44 hint wins, nip04->nip44 fallback, and note-to-self decrypts.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
updateFingerprintCache (and the fingerprint getter) emitted "fingerprint"
unconditionally on every load. The For You feed subscribes to that event and
re-runs refreshFeed -> loadLatest -> updateFingerprintCache -> emit, so once the
For You feed was active the app entered an infinite loop: a full feed rebuild
plus a watch-history reload (with extension decrypts) every ~400ms. That pegged
the CPU (fans maxing) and starved every other load — videos, hashtags, subs,
DMs and watch history all failed to appear.
Emit "fingerprint" only when the fingerprint actually changes. The first real
resolution emits once (triggering a single refresh); the follow-up load produces
the same fingerprint and emits nothing, so the loop settles immediately.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Root cause of the ~20s watch-history load: history is bucketed by month (one
encrypted snapshot event per month) and the fetch decrypted EVERY month via the
serialized nip-07 extension — ~1 slow call per month, so accounts with years of
history paid dozens of back-to-back extension decrypts on every load.
Cap the load to the most recent RECENT_MONTHS_DECRYPT_LIMIT (6) months by default
— enough for the feed's watched-video suppression and the initial view. Callers
that need the full history (the explicit History view) pass recentMonthsLimit: 0,
threaded loadLatest -> scheduleWatchHistoryRefresh -> resolveWatchHistory ->
resolve -> fetch.
Harness (fresh-context wh-repro): 600 items / 18 months goes from 18 nip-07
decrypt calls to 6. Watch-history tests green. Bumps the watchHistory.js
file-size baseline for the added logic.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
