Foundation for bounded watch-history loading (slice B part 2). A deep
history references many monthly chunks; under a NIP-46 remote signer each
uncached chunk decrypt is a serialized published relay RPC, so decrypting
them all up front blocks the first render and risks the relay rate limit.
manager.fetch now:
- sorts chunk events newest-first by created_at, so the decrypt budget is
spent on the most recent history;
- accepts a chunkDecryptLimit option that caps how many *uncached*,
genuinely-encrypted chunks are decrypted per fetch. Cached/plaintext
chunks are always decrypted (they cost no RPC) and never count against
the limit. Default is Infinity -> no behavior change for existing
callers;
- returns deferredChunkCount so a caller can tell more history is
available on demand (re-fetching with a larger limit reuses the chunk
cache, so only the newly requested older chunks cost RPCs).
Test: two uncached nip44 chunks (newest + older) with chunkDecryptLimit:1
asserts only the newest is decrypted (1 RPC), deferredChunkCount is 1, and
only the newest chunk's items appear. Mutation-verified against both
removing the cap and removing the newest-first sort.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch-history decryption previously ordered the matching cipher family
first but kept the other family as a *fallback*. The opposite family can
never decrypt a given ciphertext (NIP-04 always carries a literal "?iv="
marker; NIP-44 is plain base64), so that fallback was a guaranteed-wasted
decrypt attempt. Under a NIP-46 remote signer every decrypt is a
*published* relay RPC routed through the serial 250ms request queue, so a
transient failure on the correct family triggered a second wasted RPC
that burned the queue and pushed cold loads toward the relay rate limit
("noting too much") — the symptom users hit.
determineWatchHistoryDecryptionOrder now filters the attempt set to the
single family implied by the ciphertext shape (trusting the ciphertext
over a possibly-misleading "encrypted" tag), with a safety fallback to
the full ordered list when nothing matches so the only available
decryptor is never stranded. This mirrors the list-decrypt routing
already shipped for userBlocks / subscriptions / hashtags (slice A).
Test: a NIP-44 ciphertext mislabeled with an ["encrypted","nip04"] tag,
against a signer exposing both decryptors where nip04 returns decoy data.
Asserts nip04 is never attempted and the nip44 plaintext is used.
Mutation-verified: removing the filter fails the test.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Under a NIP-46 remote signer, every decrypt is a PUBLISHED relay RPC. The list
services (blocks/subscriptions/hashtags) probed all decryptor families
concurrently via Promise.any — so each list event published nip04 AND nip44 RPCs
at once. On login that flooded the relay ("rate-limited: you are noting too
much"), and the racing/queued RPCs blew past the 30s decrypt budget → the
"Decryption timed out" failures.
A NIP-04 payload carries the "?iv=" marker and can ONLY be decrypted by nip04;
otherwise it's nip44. The other family can never succeed on a given ciphertext,
so racing it is pure waste (and a wasted published RPC under NIP-46). Restrict
the decrypt probe to the family matching the ciphertext format (fall back to all
if the format is somehow unknown). Halves cold-load RPCs and stops the racing
publishes that tripped the rate limit; benefits all signer types.
Spec correction: the hashtag "retries decryptors when hinted scheme fails" test
stubbed an impossible scenario (the same ciphertext decrypting under nip04 after
nip44 failed). Rewritten to assert the real contract — the ciphertext format
routes decryption and overrides a misleading `encrypted` tag, invoking only the
matching decryptor.
test_integrity_note:
change_type: ["spec_correction"]
scenarios:
- id: SCN-decrypt-format-routing
given: "a nip04-format ciphertext (?iv=) tagged misleadingly as nip44"
when: "a list service decrypts it"
then: "only the nip04 decryptor is invoked; nip44 is never attempted"
anti_cheat_rationale:
prevents: ["over-mocking internal logic", "asserting impossible cross-family decryption"]
relaxation:
did_relax_any_assertion: false
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The NIP-46 login dialog is purely the QR / connect-link flow ("scan with Amber/
NsecBunker"), but it failed with "A remote signer pubkey is required." A
nostrconnect:// link carries the CLIENT pubkey; the remote signer's pubkey is
only learned after it scans the QR and acknowledges — and the active connect
path (SignerManager) only supported bunker:// (pubkey already in the URI). The
"wait for the signer to acknowledge" logic existed only on the unused
Nip46Connector.
Port it into SignerManager as _waitForRemoteSignerHandshake: subscribe to
kind-24133 events addressed to the client pubkey, decrypt the signer's response,
match the handshake secret (or a generic ack), surface auth_url challenges, and
resolve with the signer's pubkey (with a timeout + a cancel hook wired into
disconnectRemoteSigner). connectRemoteSigner now performs this wait for
client-initiated (nostrconnect) connections before constructing the RPC client;
bunker:// is unchanged.
Decryption uses the existing attemptDecryptNip46HandshakePayload (injectable via
_decryptHandshakePayload for testing). Test: tests/nostr/nip46-wait-handshake.test.mjs
(secret match resolves with signer pubkey; wrong secret ignored -> timeout;
auth_url surfaced without resolving). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
connectRemoteSigner and useStoredRemoteSigner accepted but never used the
`validator` the auth provider passes (accessControl.canAccess) -- so NIP-46
logins bypassed the block/allow gate that nsec/NIP-07 logins enforce. (The
validation lived only on the unused Nip46Connector.)
Both paths now run the validator against the remote signer's resolved user
pubkey BEFORE activating the signer (via _enforceRemoteSignerValidator): the
validator throws on a blocked/non-permitted pubkey, or returning false rejects
with code remote-signer-access-denied. A stored session rejected by the
validator is forgotten so a blocked pubkey can't keep retrying.
Residual (noted in TODO): the silent startup auto-restore
(scheduleStoredRemoteSignerRestore) still reconnects without a validator.
Test: tests/nostr/nip46-prepare-handshake.test.mjs (validator throws / false /
true / absent). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
NIP-46 login via a generated connect link failed with
"nostrClient.prepareRemoteSignerHandshake is not a function". The auth provider
(authProviders/nip46.js) calls nostrClient.prepareRemoteSignerHandshake() to mint
an ephemeral client keypair + nostrconnect:// URI before connecting, but that
method existed only on the unused Nip46Connector class — neither SignerManager
nor NostrClient exposed it.
Add SignerManager.prepareRemoteSignerHandshake (generates the keypair via
nostr-tools, resolves relays, builds the nostrconnect:// URI with relay/secret/
perms/metadata params) and proxy it on NostrClient next to connectRemoteSigner.
Test: tests/nostr/nip46-prepare-handshake.test.mjs. Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Drives the real s3UploadService.uploadVideo for a generic (vhost-style) S3
connection with only the S3 I/O mocked, then feeds the published note's URLs
into r2Service.deleteVideoStorage. Asserts the upload writes video + .thumb +
.torrent, the note is valid (URL + magnet with info-hash), and cleanup deletes
exactly those objects using a forcePathStyle-aware S3 client — i.e. the upload
key and the cleanup-derived key agree (the parity property).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Two gaps where generic S3 trailed Cloudflare R2:
1. Object cleanup on delete/edit didn't work for generic S3. deleteVideoStorage
built its client via makeR2Client, which hard-forces path-style and ignores
the connection's forcePathStyle -- so vhost-style S3 deletes were misaddressed
and cleanup silently failed (S3 users' deleted/edited files orphaned). It is
now provider-aware (mirrors uploadFile): R2 -> makeR2Client (path-style),
generic S3 -> makeS3Client honoring forcePathStyle. makeS3Client is now
injectable for testing.
2. CORS guidance on upload failure was R2-only. Extracted isLikelyCorsError +
a generic-S3 guidance builder into js/utils/uploadErrorHints.js (re-exported
from r2Service for back-compat); s3UploadService now appends actionable CORS
guidance on an opaque "Failed to fetch" too.
Generic S3 is now at functional parity with R2 for upload -> publish -> cleanup.
Tests: r2-delete-video-storage (adds generic-S3 forcePathStyle routing),
s3-upload-cors-guidance (new). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Existing upload/feed specs seed events straight into the relay, bypassing the
app's own publish path. This spec drives the actual UI: open the upload modal,
switch to External Link mode, fill title + URL, submit -> the app signs and
publishes to the mock relay -> the video appears in the feed; then the owner
deletes it via the gear menu and it leaves the feed.
Closes the launch-readiness coverage gap on the publish lifecycle (the S3/R2
file upload + cleanup are covered by the unit suites). Verified passing on
chromium (2/2).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A new user's top first-upload failure is a missing R2 CORS policy (browser
PUT/POST blocked) -- and object-scoped R2 tokens often cannot auto-set CORS.
The connection test already surfaces buildCorsGuidance on the opaque
"Failed to fetch", but a user who skips the test and uploads directly got a
bare "Upload failed: Failed to fetch". The upload error handler now detects the
browser CORS/network rejection (isLikelyCorsError -- Chrome "Failed to fetch",
Firefox "NetworkError", Safari "Load failed") and appends the same actionable
guidance. Updated the R2 follow-up TODO (credential audit done; CORS reviewed).
Test: tests/services/r2-cors-error-classifier.test.mjs. Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Pre-launch credential-security regression: scans the ENTIRE serialized account
record (meta + encrypted blob) for the secret/access-key strings and asserts
they never appear in plaintext, that listConnections() doesn't expose them, that
they round-trip while unlocked, and that they're unreachable once locked. The
prior check only compared the cipher field to JSON.stringify(payload), which
wouldn't catch a secret leaking into plaintext meta.
Audit conclusion: credential storage is sound — envelope encryption (AES-GCM-256
payload, fresh per-encrypt IV; master key encrypted to-self via the Nostr signer,
NIP-44 preferred / NIP-04 fallback), in-memory master key cleared on logout, no
secret logging, legacy plaintext migrated then cleared.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Editing a video with a newly-uploaded file produced a new content-addressed
object and orphaned the old one. editModalController.handleSubmit now removes the
old object (best-effort) when the hosted URL was genuinely replaced — guarded so
it only fires when both old and new URLs are present and different (so clearing
the URL, which may still be referenced by a magnet web-seed, never deletes it),
and it deliberately leaves thumbnails alone (the new note may still point at
them). Reuses R2Service.deleteVideoStorage (owned-objects-only, never throws).
Documented the cleanup contract in AGENTS.md §4a.
Test: tests/ui/edit-modal-storage-cleanup.test.mjs (3). Full suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Deleting a video removed the Nostr note but left the actual file (plus its
thumbnail and .torrent) publicly downloadable at its R2/S3 URL forever — a
privacy gap and ongoing storage cost. Now the delete flow also removes the
backing objects, best-effort.
- collectVideoStorageKeys() (storagePointer.js): pure helper that derives the
object keys (video + sibling .torrent + thumbnail) from a note, ONLY for URLs
under the owner's bucket base — external/3rd-party URLs are never returned.
- R2Service.deleteVideoStorage(): resolves the owner's connection, and deletes
those keys. Skips cleanly when storage is locked / no creds / no owned
objects, and never throws (failures are recorded, not propagated) so storage
cleanup can't block note deletion. R2Service class now exported for testing.
- app.handleDeleteModalConfirm: calls it after handleFullDeleteVideo succeeds.
Tests: tests/storage-key-collection.test.mjs (4),
tests/services/r2-delete-video-storage.test.mjs (4). Full suite green.
Note: superseded versions orphaned by past edits aren't retroactively cleaned;
edit-time cleanup (next) prevents new orphans going forward.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
publishEventToRelay optimistically reports success when a relay handle exposes
no ok/failed event and isn't thenable (legacy 'seen'-only relays). That made an
UNCONFIRMED publish indistinguishable from a real ack, so a note could show
'Published' with nothing actually stored. The result now carries optimistic:true
for that path (callers can distinguish it from a confirmed ack) and logs a
dev.warn. Behavior is otherwise unchanged (still treated as success for legacy
relays). Publish audit #3.
Test: tests/nostr-publish-optimistic.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
These optional-asset uploads are non-fatal, but failures were warn-only — the
user got a silently-missing thumbnail or torrent with no signal. Both upload
services now emit a 'warning' status at the point of failure and append a note
to the final 'Published' status (downgraded to warning variant) listing what
failed. Cloudflare-upload audit #5.
Test: tests/services/s3-upload-optional-asset-failures.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A 0-byte file produced a multipart upload with zero parts, which S3/R2 reject
with a cryptic CompleteMultipartUpload error. Guard file.size === 0 up front
with a clear message, before any S3 request is issued. Cloudflare-upload #4.
Test: tests/multipart-empty-file-guard.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Computing the torrent info-hash (and the content-hash fallback) reads the entire
file before the upload starts and before any progress is emitted, so on
multi-GB videos there was a long silent pause that looked like a freeze. Both
upload services now emit an 'Analyzing video…' status when a full-file hash is
required (i.e. no valid info-hash was pre-supplied). Cloudflare-upload audit #2.
Test: tests/services/s3-upload-analyzing-status.test.mjs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
buildR2Key fell back to a static "uploads" namespace (and "video" slug) when
no torrent info-hash was available, so two distinct URL-first uploads sharing a
filename produced the SAME R2/S3 key — the second silently overwrote the first
object while the first video note still pointed at that URL (data loss).
WebTorrent uploads were already safe because the info-hash namespaces the key.
Add computeStorageContentHash(file) in js/r2.js: when no info-hash is present it
derives a deterministic content-based namespace — full SHA-256 of the bytes for
files ≤512 MB (idempotent: re-uploading the same file reuses the key), and a
SHA-256 fingerprint over name/size/mtime + sampled head/tail bytes above that,
so we never buffer a multi-GB video in memory. Wired into BOTH upload paths
(r2Service + s3UploadService) via a new storageIdentifier kept separate from the
info-hash, so magnet generation is unchanged.
Test: tests/storage-key-collision.test.mjs (distinct same-named uploads get
distinct keys; same content is idempotent; large-file fingerprint path samples
edges without buffering and stays deterministic). Full unit suite green.
test_integrity_note:
change_type: ["new_tests"]
scenarios:
- id: SCN-upload-key-no-collision
given: "a URL-first upload with no torrent info-hash"
when: "the storage key is derived (computeStorageContentHash -> buildR2Key)"
then: "distinct same-named files get distinct keys, same content is idempotent, and no key uses the colliding 'uploads' namespace"
observable_outcomes:
- "returned key strings for real Blob/File content and a fake >512MB file"
determinism_controls:
- "fixed byte content; fake large file with stubbed slice() and fixed lastModified"
anti_cheat_rationale:
prevents:
- "hard-coded return value"
- "over-mocking internal logic"
relaxation:
did_relax_any_assertion: false
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The controller→Actions split left a batch of method calls in
ProfileDirectMessageActions pointing at `this` (the Actions instance)
instead of the object that actually defines them. Opening a conversation
threw "this.setFocusedDmConversation is not a function"; the same latent
bug also broke DM send-error messaging, read receipts, settings publish,
and the whole attachment flow (generate/render/reset/upload) — the user
just hadn't exercised them yet.
Route each to its real receiver:
- DM-controller methods → this.controller.*:
handleDmSettingsPublish, describeDirectMessageSendError,
maybePublishReadReceipt, setFocusedDmConversation
- main-controller methods → this.mainController.*:
generateAttachmentId, renderAttachmentQueue, resetAttachmentQueue,
uploadAttachmentQueue, clearProfileMessages
(Builds on the earlier handleDmConversationSelect delegation fix.) A static
sweep confirms every remaining `this.*` call in Actions resolves to an
Actions method or an explicit controller/mainController/helper receiver.
Test: tests/dm-conversation-select.test.mjs (cheat-resistant — drives the
real handleDmConversationSelect with stub receivers; would throw on the old
wrong-receiver code). DM unit + integration suites green; full build OK.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
After switching to a responsive signer (nos2x), two DM-specific bugs surfaced.
1. Clicking a conversation in the list threw
"this.controller.handleDmConversationSelect is not a function"
(ProfileDirectMessageRenderer onSelectConversation). The method exists on
ProfileDirectMessageActions and the controller delegates ~15 sibling Actions
methods — but the handleDmConversationSelect delegation was missing. Add it.
2. Loading a conversation's messages threw "DM decryption helpers are
unavailable" even though the signer decrypts blocks/subscriptions/hashtags
fine. buildDmDecryptContext gated decryptor registration on (a) the dynamic
`capabilities` getter (which probes window.nostr and false-negatives on lazy
injection / nos2x's module shape) and (b) the permission-probe result. The
list services already learned to register by METHOD EXISTENCE and validate at
call time; align the DM path to do the same so a present, working signer
never yields zero decryptors. The permission probe still runs.
Tests: tests/dm-decrypt-context.test.mjs (cheat-resistant — a signer whose
capabilities getter reports false and whose permission probe returns ok:false
must still register nip04+nip44 decryptors). Full unit + dm:unit + dm:integration
suites green. KNOWN_BUGS #0.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A raw window.nostr probe in the user's browser (getPublicKey → nip04.encrypt
→ nip04.decrypt, bypassing ALL of bitvid) HUNG forever, with the extension's
own console showing `ObjectMultiplex - orphaned data for "background-liveness"`
+ `malformed chunk` + listener-leak warnings. That is the NIP-07 extension's
content-script ↔ background-service-worker channel being dead (classic MV3
worker death) — an extension/environment failure, not a bitvid bug. No client
change can force an unresponsive signer to answer; the fix is to reload/unlock
the extension.
What bitvid CAN do is stop failing silently. Add js/utils/signerHealthNotice.js:
after a few consecutive list-decrypt timeouts while the signer reports itself
present, emit ONE rate-limited, actionable user notice ("your signer isn't
responding to decryption — unlock/reload the extension and refresh") instead of
the silent 15-minute retry loop. Reset on any decrypt success so a transient
blip doesn't warn. Wired into blocks, hashtags, and subscriptions.
Also documents the definitive root-cause + all resilience increments in
KNOWN_BUGS #0. Test: tests/signer-health-notice.test.mjs (threshold, rate-limit,
success-reset). Full unit suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The live subscription-list subscription (kind 30000) delivers our OWN list
event back through handleSubscriptionListEvent, which called updateFromRelays
on EVERY event. updateFromRelays re-fetches the list, the relay echoes the
same event through the live sub, and the handler fires again — a
self-sustaining loop that floods the cold-login burst with redundant
kind-30000 REQs (the kind 30000=19 storm in the real-env console,
KNOWN_BUGS #0). Same shape as the contact-list emit→fetch→emit loop fixed
earlier.
Add an emit-on-change guard: skip the refresh when the event is one we've
already acted on (by id — matters during the pre-decrypt login window where
subsEventCreatedAt isn't set yet) or is not newer than what we've applied.
A genuinely newer list event still refreshes exactly once.
Also extend the channel-sim harness to capture per-filter REQ signatures and
kind-30000 REQ timing, which is how this loop was isolated.
Test: tests/subscriptions-manager.test.mjs — echo of the same event id does
not re-refresh; an older event is ignored; a newer event refreshes once.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console (2026-06-15) showed the breaker never opening even as
decrypts failed for minutes. Cause: the breaker treated every non-timeout
error as "channel alive" and RESET the timeout streak — but
"message channel closed before a response was received" is the exact
dead-channel signal, and it rejects as a (fast) non-timeout error. So each
such rejection reset the counter and the circuit could never open.
Classify channel-death messages (message channel closed / could not
establish connection / receiving end does not exist / extension context
invalidated / connection lost) as channel failures that count TOWARD
opening the circuit, instead of as responsive failures that reset it.
Genuine extension responses (user rejection, permission denied, etc.)
still reset the streak.
Test: extends nip07-channel-breaker with a synchronous channel-closed op
asserting consecutive channel-death errors open the circuit and then
fast-fail (no waits).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console (2026-06-15) showed that once the extension's message
port drops under the post-login burst ("message channel closed"), EVERY
subsequent decrypt call hangs to its full ~15s timeout. Each list service
(blocks, hashtags, subscriptions, watch history) independently retries on
that timeout, so the app sits in a permanent loop of 15s-blocking
extension calls — pinning the CPU ("fans at max") and never recovering
until a manual page refresh.
Add a shared circuit breaker at the runNip07WithRetry chokepoint:
- After CIRCUIT_TIMEOUT_THRESHOLD (3) consecutive timeouts the circuit
OPENS; further calls fail fast with code "nip07-channel-unresponsive"
(no multi-second hang), so the retry loops become cheap and the CPU
settles.
- While open, a single probe per CIRCUIT_PROBE_INTERVAL_MS is allowed
through; one success CLOSES the circuit and normal decryption resumes —
no page refresh required.
- Responsive failures (extension answered with an error) reset the
timeout streak — only genuine channel hangs open the circuit.
- Interactive permission prompts bypass the breaker entirely: a user
taking their time at the extension UI can neither trip it nor be
blocked by it, and such a prompt can itself heal the channel.
Increment 2 of the "cut the login burst, then add a safety net" work for
KNOWN_BUGS #0. Test: tests/nostr/nip07-channel-breaker.test.mjs
(cheat-resistant — drives the real queue with hanging/responsive/success
ops; asserts open-after-threshold, fast-fail, probe-recovery, responsive
errors don't open, and bypass is never fast-failed). Full unit suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-env console (cold nip-07 login) showed list decryption failing en
masse: blocks/hashtags/subscriptions fetched fine (relay cap working —
every sub now fans out to exactly 8 relays) but decryption timed out for
minutes with "message channel closed", only recovering on a page refresh.
Root cause: the single-threaded nip-07 extension channel is overwhelmed
during the post-login handshake. One contributor is the eager DM load —
loadDirectMessages({ limit: 50, initialLoad: true }) ran at login and
fired a burst of decrypts that competed with the feed-driving lists
(blocks/subscriptions/hashtags) for the same channel. DMs are lower
priority (the user isn't viewing Messages at login) and already load
lazily when the Messages tab opens.
Defer the eager DM load; report dms state as "deferred". The lightweight
unread-indicator refresh still runs so the badge stays accurate. First
increment of the "cut the login burst" work (KNOWN_BUGS #0).
Test: tests/app/login-defers-dms.test.mjs (cheat-resistant — asserts
loadDirectMessages is never called at login, dms state is "deferred", and
the unread indicator still runs).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
On cold login the app opened WebSockets to a user's full NIP-65 read set
(in prod often ~20 relays, most dead). The flood of failing connections
churned the main thread and starved the nip-07 extension's postMessage
round-trips, so encrypted lists (blocks, subscriptions, hashtags) were
fetched but timed out during decryption ("message channel closed",
"Decryption timed out").
Bound every read/subscribe path to MAX_SUBSCRIBE_RELAYS=8 via a shared,
test-aware capReadRelays helper, applied at the chokepoints:
- client.js applyRelayPreferences (this.relays / this.readRelays)
- relayBatchFetcher.js fetchListIncrementally (blocks, subs, hashtags)
Writes stay uncapped (publish reach preserved).
capReadRelays is user-relays-first (a user's own relays are where their
data authoritatively lives, and small fast-path limits would otherwise
crowd them out — the regression that reverted the first attempt) while
reserving RESERVED_DEFAULT_RELAY_SLOTS=2 for reliable default aggregators
so the "~20 personal relays, all dead" case still reaches a live relay.
Under test the reserved core resolves to the test-relay override, keeping
harnesses isolated.
Tests: new relay-subscribe-cap coverage (bounded + reserved-defaults +
writes-uncapped); spec-corrected user-blocks, subscriptions-manager and
nostr/client fetch tests to bounded (not exact) relay fan-out while
keeping all behavioral/security assertions (per-relay coverage, correct
decrypt, block hydration, dedup) intact. See TEST_INTEGRITY.md and
KNOWN_BUGS #0/#4. Full unit + dm:unit suites green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The For You feed was chronological, hashtag-interest-EXCLUSIVE, and ignored
follows entirely (docs/feed-algo-audit.md). Now it is inclusive + ranked:
- createForYouScorerStage scores each video: follows-boost (author you follow)
+ interest/watch-topic affinity (cosine vs your interests+watch-history vector)
+ freshness - disinterest penalty. Reuses Explore's vector/similarity helpers.
- createForYouScoreSorter ranks by that score (muted last, recency tiebreak).
- For You pipeline: interests now BOOST not exclude (enforceInterests:false);
scorer added; chronological sorter replaced. Never empty (falls back to
recency when you have no follows/interests/history).
- buildForYouFeedRuntime now feeds subscriptionAuthors + watchHistoryTagCounts.
Watched/blocked/muted are still removed (suppression/blacklist/moderation
stages unchanged). Cheat-resistant tests (older boosted items only win via the
boost). Grid renders end-to-end (~1.3s). Weight tuning is a follow-up.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
handleBlocksLoaded reloaded the feed on every 'blocks-loaded' signal — but a
forced reload re-fetches the lists, which re-emit 'blocks-loaded', a
self-sustaining loop. Now it computes a signature of the block set and reloads
only on actual change (emit-on-change — the P5 principle). handleRelaysLoaded
only touches relay UI; hashtag prefs already dedup by signature.
Cheat-resistant regression test added. Bumped authSessionCoordinator file-size
baseline. Logged 2 pre-existing app test failures (fail at pre-session baseline)
in KNOWN_BUGS. Marks KNOWN_BUGS #2 resolved.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
performProfileFetch now issues ONE deduped, health-gated list() across all
relays via the L1 SubscriptionManager (the pool merges per-relay results),
replacing the manual per-relay fan-out. Newest-per-author selection unchanged;
legacy per-relay path retained as the manager-less fallback.
Spec correction: tests/app-batch-fetch-profiles.test.mjs asserted one query
PER relay (implementation detail); updated to assert one BATCHED query across
all relays. All behavioral assertions (cached hydration, newest profile
rendered, failing relay tolerated) unchanged; added a relays-targeting
assertion. Documented in TEST_INTEGRITY.md (SCN-profiles-batched).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Replaces the per-card view-count transport (each card: list + NIP-45 COUNT +
live sub, × relays — the ~880 REQ/s storm) with ONE shared batched kind-30079
subscription + ONE backfill list() over all visible cards, bucketed back to
per-pointer state by #a. Re-enables the VideoListView circuit-breaker.
Grid counts are convergent/approximate (cheap, persists via the existing view
cache); the watch page opts into an exact NIP-45 COUNT for its single pointer
(modalCoordinator passes { exact: true }) — accurate where it matters, one
request, no storm. Legacy per-pointer path retained for SubscriptionManager-less
environments (tests/mocks).
Verified: nip07-heat steady-idle with views on; flaky multi-relay harness still
all-PASS; new cheat-resistant batching+bucketing test. (Pre-existing
view-counter publish-session-tag failure is unrelated — fails at baseline.)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Replaces the per-video live report subscription (one pool.sub per visible
video × ~20 relays, re-fired on every reconnect — the ~1000 REQ/s storm) with
ONE batched SubscriptionManager subscription over the whole active id set,
updated in place as the set changes. Backfill fetches history only for
newly-added ids (one batched list()). Unchanged sets are a no-op (dedup guard).
MODERATION_REPORT_SUBS_ENABLED flipped back on. Trusted-mute subs were already
batched-by-author, so the relay cap + contact-loop fix already tamed them.
Legacy per-id path retained only for SubscriptionManager-less environments
(unit tests / mock clients). All 39 moderation tests pass; new cheat-resistant
batching test added; nip07-heat shows reports no longer storm (steady idle).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
P1 foundation of the data-flow refactor (docs/architecture-refactor.md):
- js/nostr/subscriptionManager.js: the single chokepoint for relay reads.
Builds on relaySubscriptionService (dedup by key) and adds: health-gated
default relays, deduped one-shot list() with timeout, and a reconnect
registry that re-issues active subs centrally (instead of every subsystem
re-subscribing on reconnect — the storm amplifier). Callers pass batched
filters (one REQ for many ids), never per-item.
- Export RelaySubscriptionService class for test isolation.
- scripts/check-direct-pool-access.mjs + lint:pool-access: fail on NEW direct
pool.sub/list usage; 12 grandfathered files listed as migration targets
(allowlist may only shrink).
Nothing rewired yet — additive. Pilot (video feed) and per-subsystem
migration follow. Cheat-resistant tests included.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A user's NIP-65 list can carry ~20 relays, most dead. Subscribing on all of
them is the dominant REQ-storm amplifier: every subsystem fans every query
across all relays and re-fires on each flaky-relay reconnect. Cap the
read/subscribe set to a bounded core (6), prioritizing the bundled known-good
defaults; WRITES still fan out to the full set so publish reach is unaffected.
Interim until RelayHealth ranks by liveness (docs/architecture-refactor.md P1).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-environment console logs revealed nip-07 login pegging the CPU and
starving encrypted-list decryption via relay REQ storms (~1000 REQ/s):
- moderationService.rebuildTrustedContacts emitted "contacts"
unconditionally, forming an emit->fetch->emit loop (~1600 kind-3 REQ/s).
Now emits only on actual change. Regression test added.
- Per-video live report (kind 1984) and view-count (kind 30079)
subscriptions fan out O(videos x relays) and re-fire on every flaky-relay
reconnect. Circuit-broken behind flags until batched (see refactor plan).
- onVideosShouldRefresh now coalesces forced reloads (2s leading-edge
throttle) so list-change signals can't drive a reload loop.
- nip-07 queue reserves one foreground slot; DM decryption demoted to LOW
so background DM backfill can't starve feed-driving lists. Test added.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The legacy-DM perf reorder regressed tests/nostr/dmDecryptorPreference:
1. preferLegacy short-circuited BEFORE the explicit-hint ranking, so a kind-4
event tagged ["encrypted","nip44"] would still try nip04 first. Explicit
hints now win; preferLegacy only flips the *no-hint* default.
2. The "skip the actor's own key" optimization dropped self even when it was the
ONLY candidate (note-to-self DMs), leaving zero candidates -> ok:false. Self
is now deferred to last and attempted as a fallback instead of dropped.
Spec correction (per Dark Factory rules): for a kind-4 DM with no hints, nip04
is now the preferred default (protocol-correct + avoids a wasted serialized
nip-07 round-trip). Rewrote the preference test to assert: nip04-first default,
explicit nip44 hint wins, nip04->nip44 fallback, and note-to-self decrypts.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Locks in the fix for the infinite feed-refresh loop: asserts "fingerprint" is
emitted only when the value actually changes (unchanged re-evaluation must not
emit, or the For You feed would refresh -> reload -> emit forever). Verified to
fail when the change-guard is removed.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The UX harness (fake nip-07 extension) revealed that the off-thread verification
could blank the feed: if nostr-tools is slow/unable to load inside the verify
worker, the worker hangs, each batch waits the full 10s timeout, and the
main-thread fallback could return an empty set (fail-closed) — dropping every
event so no videos ever render, silently (devLogger is suppressed in prod).
Harden it so verification can never stall or blank the read feed:
- Worker timeout 10s -> 2.5s, and disable the worker for the session after one
timeout/error so we never pay the timeout on every batch.
- Bound the main-thread nostr-tools wait (Promise.race ~2s) so a slow/failed
load can't hang the fallback.
- Fail OPEN when the verifier is genuinely unavailable: pass events through
unverified rather than dropping them (the pool already does not verify, so this
does not regress the prior posture). Real verification still runs whenever
nostr-tools is available.
Adds a regression test (hung worker -> resolves via fallback, worker disabled).
Also updates the harness to seed mode:"live" videos + a NIP-65 relay list for
mock-relay isolation.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch history is ~150 separately encrypted chunk events; decrypting each one is
a serialized round-trip to the nip-07 extension (the key lives in the extension
and cannot be cached locally), so a full re-decrypt cost many seconds on EVERY
load. Chunk events are addressable/replaceable: an unchanged chunk keeps its
event id while a modified chunk republishes under a new id.
Add a persisted cache (watchHistoryDecryptCache.js) keyed by chunk event id. The
fetch path checks it before calling the extension and stores successful
decryptions, so reloads decrypt only the chunk(s) that actually changed
(typically one) and skip the extension for the rest — across sessions. Bounded
to 400 entries; cleared on logout alongside the conversation-key cache. The
cached plaintext is no more sensitive than the watch-history items already
persisted by the manager.
Adds watchHistoryDecryptCache.test.mjs (round-trip, persistence/re-hydrate,
bounds, clear).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
loadVideos awaited ensureAccessControlReady -> waitForReady -> _performRefresh
-> loadAdminState, which awaits a remote admin-list fetch. On a cold load that
waits for relays to connect (~5s) plus the fetch, stalling the feed ~10s before
any video renders. The cached admin state IS hydrated synchronously at
construction, but with markLoaded:false, so waitForReady blocked on a full
remote refresh even when a usable cache existed ("refresh fixes it").
ensureAccessControlReady now blocks only on a first-ever cold load WITH whitelist
mode (where showing non-whitelisted content before the whitelist arrives would be
wrong). In every other case — admin state hydrated from the localStorage cache,
already loaded, or blacklist-only mode — it kicks the refresh off in the
background (ensureReady) and renders immediately; the reactive
onWhitelist/onBlacklistChange handlers re-filter the grid once fresh lists land.
Adds accessControl.isHydrated() and a branching test
(nostr-service-access-control-ready.test.mjs).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The For You feed awaited watchHistoryService.loadLatest() before rendering, and
loadLatest blocked on the full ~1500-item watch-history fetch on a cold load even
with allowStale:true (the `!allowStale || !hasCachedItems` guard awaited whenever
there was no cache). So the main feed froze behind the entire watch-history
resolution, and nothing re-ran the feed once history finished loading — watched
videos were never suppressed and watch-history tags never applied until a manual
refresh.
- watchHistoryService.loadLatest: allowStale callers now NEVER block. On a cold
load the refresh is kicked off in the background and the call returns
immediately with whatever is available (cached items, else the local queue).
- feedCoordinator.subscribeWatchHistoryFeedRefresh: subscribes to the
"fingerprint" event and, when the For You feed is active, debounced-re-runs it
so newly-resolved watched videos are suppressed and tags applied. Wired in at
bootstrap alongside feed registration.
The re-run reads the now-fresh cache (no further refresh scheduled), so there is
no loop. Adds feedCoordinatorWatchHistoryRefresh.test.mjs (debounced single
refresh while active; no refresh when inactive; idempotent subscription).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Profiling the live app showed the single biggest CPU cost was the relay pool
verifying every incoming event's schnorr signature synchronously on the main
thread: ~1.2s of secp256k1 work ([email protected] 855ms) during feed load and tab
switches, dropping to ~16ms when skipped. The pool was constructed as
`new SimplePool()` with no options, so nostr-tools used its default verifying
verifyEvent; the `verifyEvent: () => true` that looked like it disabled this
only existed on a mock tools object and was never wired to the real pool.
Move verification off the main thread instead of dropping it:
- Construct the pool with `verifyEvent: () => true` so it never blocks.
- Add signatureVerifyWorker (+ client) that verifies event batches in a Web
Worker and returns the valid ids; falls back to main-thread verification only
when Workers are unavailable (never silently trusts).
- VideoEventBuffer verifies each batch via the worker before committing, and
drops events that fail (fails closed). The verifier is injectable so tests can
virtualize the worker; subscribeVideos passes it through and returns the flush
promise from EOSE so commits are awaitable.
Tests: new signatureVerifyWorkerClient.test.mjs (worker protocol + drop-invalid
+ empty/no-id handling); videoEventBuffer.test.mjs and client.test.mjs adapted
to the async, verified commit path with a passthrough verifier.
Note: tests/nostr-delete-flow.test.mjs has a PRE-EXISTING failure on
`should not trigger subscription callbacks` (fails on clean unstable, and is a
standalone script not run by the unit-test runner); its edits here only adapt it
to the now-async flush.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
fetchLatestListEvent fetched admin lists via fetchListIncrementally without an
explicit `since`, so it relied on the persisted per-relay lastSeen. Admin lists
are replaceable events with a single current version whose created_at does not
advance, so once lastSeen == that created_at, every later load queries
since=lastSeen+1 and the relay returns nothing — perpetually hiding the current
list and dropping admin lists whenever the local cache is absent.
Pass since: 0 to force a full fetch of the (tiny) current event each time. Other
list callers anchor an explicit since to their own cached state; admin lists
have no comparable per-list timestamp, so always re-fetching current is both
correct and cheap.
Adds tests/nostr/relayBatchFetcherReplaceable.test.mjs documenting the
replaceable-event since-gate at the fetcher boundary: relying on persisted
lastSeen hides the current event; since: 0 returns it.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
subscribeVideos called pool.sub(getHealthyRelays(this.relays)) with no
empty-relay fallback, unlike the fetch paths in the same file. On a cold first
load the 5s relay connect probe can time out, marking every relay unreachable;
init() only warns and schedules a background reconnect (it does not throw), so
boot proceeds and the feed subscribes to ZERO relays. The background reconnect
later succeeds but nothing re-issues the subscription, so the feed stays empty
until the user refreshes (when the SWR cache renders) — the reported
"notes don't load on first visit / need a refresh" symptom.
Fall back to DEFAULT_RELAY_URLS when no relays are healthy, mirroring the
established pattern. Because the fallback bypasses the unreachable cooldown, the
subscription retries those relays directly and self-heals instead of staying
dead until refresh.
Adds tests/nostr/subscribeVideosRelayFallback.test.mjs: empty healthy set =>
default relays (never zero), populated healthy set => uses those relays.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch history is stored as up to ~150 separately NIP-44-encrypted chunk
events. Each chunk built a fresh cipher suite whose conversation-key cache
lived only in its own closure, so the ECDH conversation key (~12ms each in
the bundle) was re-derived once per chunk — multiplied across actor-candidate
and scheme-fallback retries this synchronously blocked the main thread for up
to ~20s after login/profile open (the reported freeze + fan spin-up).
Memoize the derived key by (scheme, privateKey, target) at module scope so it
is computed once per load and reused. Cleared on logout/identity change next
to clearDecryptionSchemeCache so a derived shared secret never outlives its
session. On the 150-chunk path this drops decrypt time ~1955ms -> ~182ms.
Adds tests/nostr/watchHistoryConversationKeyCache.test.js covering: 150 chunks
=> one derivation, distinct identities derive independently (no cross-identity
reuse), and cache-clear forces re-derivation.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Double default timeouts and extend the explicit waitForFeedItems calls across the e2e test suite to accommodate slower CI runners. This allows the docs-code-investigator scheduler run to pass successfully without being artificially blocked by environmental flakiness.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Increased global timeout to 120,000 in playwright.config.ts
- Increased waitForFeedItems timeout to 60,000 in various E2E tests
- Increased toHaveClass and other specific assertion timeouts to 60,000 in E2E tests
Co-authored-by: PR0M3TH3AN <[email protected]>
