watchHistory.fetch now reads via the L1 SubscriptionManager (deduped +
health-gated) instead of pool.list directly: a new getSubscriptionManager dep
is threaded from the client, and both fetches (index + chunks) go through a
listEvents() helper. Raw pool.list is retained only as a fallback for
environments without a manager (tests/mocks), so the file stays grandfathered.
Verified: watch-history unit test passes; wh-repro cold resolve returns 10
items with 2 decrypts (routing intact end-to-end).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
subscribeVideos now goes through the L1 SubscriptionManager instead of calling
pool.sub directly: health-gated relay resolution (healthy-or-default fallback
preserved), centralized dedup, and registration for reconnect re-issue. The
buffer + .unsub() contract are unchanged. The client's manager uses its own
RelaySubscriptionService instance (isolated from the global singleton).
Verified: subscribeVideos relay-fallback + client + nostr-service suites pass;
ux-harness renders the grid (first card ~1.1s); steady-state REQ/ext idle.
(Pre-existing nostr-delete-flow failure confirmed unrelated — fails at the
pre-session baseline too.)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
P1 foundation of the data-flow refactor (docs/architecture-refactor.md):
- js/nostr/subscriptionManager.js: the single chokepoint for relay reads.
Builds on relaySubscriptionService (dedup by key) and adds: health-gated
default relays, deduped one-shot list() with timeout, and a reconnect
registry that re-issues active subs centrally (instead of every subsystem
re-subscribing on reconnect — the storm amplifier). Callers pass batched
filters (one REQ for many ids), never per-item.
- Export RelaySubscriptionService class for test isolation.
- scripts/check-direct-pool-access.mjs + lint:pool-access: fail on NEW direct
pool.sub/list usage; 12 grandfathered files listed as migration targets
(allowlist may only shrink).
Nothing rewired yet — additive. Pilot (video feed) and per-subsystem
migration follow. Cheat-resistant tests included.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A user's NIP-65 list can carry ~20 relays, most dead. Subscribing on all of
them is the dominant REQ-storm amplifier: every subsystem fans every query
across all relays and re-fires on each flaky-relay reconnect. Cap the
read/subscribe set to a bounded core (6), prioritizing the bundled known-good
defaults; WRITES still fan out to the full set so publish reach is unaffected.
Interim until RelayHealth ranks by liveness (docs/architecture-refactor.md P1).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real-environment console logs revealed nip-07 login pegging the CPU and
starving encrypted-list decryption via relay REQ storms (~1000 REQ/s):
- moderationService.rebuildTrustedContacts emitted "contacts"
unconditionally, forming an emit->fetch->emit loop (~1600 kind-3 REQ/s).
Now emits only on actual change. Regression test added.
- Per-video live report (kind 1984) and view-count (kind 30079)
subscriptions fan out O(videos x relays) and re-fire on every flaky-relay
reconnect. Circuit-broken behind flags until batched (see refactor plan).
- onVideosShouldRefresh now coalesces forced reloads (2s leading-edge
throttle) so list-change signals can't drive a reload loop.
- nip-07 queue reserves one foreground slot; DM decryption demoted to LOW
so background DM backfill can't starve feed-driving lists. Test added.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Root cause of the ~20s watch-history load: history is bucketed by month (one
encrypted snapshot event per month) and the fetch decrypted EVERY month via the
serialized nip-07 extension — ~1 slow call per month, so accounts with years of
history paid dozens of back-to-back extension decrypts on every load.
Cap the load to the most recent RECENT_MONTHS_DECRYPT_LIMIT (6) months by default
— enough for the feed's watched-video suppression and the initial view. Callers
that need the full history (the explicit History view) pass recentMonthsLimit: 0,
threaded loadLatest -> scheduleWatchHistoryRefresh -> resolveWatchHistory ->
resolve -> fetch.
Harness (fresh-context wh-repro): 600 items / 18 months goes from 18 nip-07
decrypt calls to 6. Watch-history tests green. Bumps the watchHistory.js
file-size baseline for the added logic.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Per decision: the off-thread signature-verification worker added feed latency
(the worker can hang loading nostr-tools, costing up to its timeout on the first
batch) and fragility (it could blank the feed) for little benefit on a PUBLIC
video feed. Default the VideoEventBuffer verifier to trust-all (the relay pool
already does not verify), so feed events commit immediately. Verification is now
opt-in via the verifyEvents option (the worker client remains for that).
Security note: a malicious relay could serve events with forged authorship for
the read feed; verify on sensitive actions instead. Reversible.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Multiple subscribers (history list + notifications) decrypt the same DM event
at once; before the in-memory cache fills, each hit the serialized nip-07
extension, multiplying round-trips per message. Cache the in-flight decrypt
promise by event id so concurrent callers share one decrypt.
Harness: cold-load extension decrypt calls drop from ~50-60 to exactly 20 for 20
DMs (1 per message); warm reload stays 0 via the persisted cache. Combined with
the earlier ordering/sequential and persist-cache fixes, legacy DM decryption
went from ~241 extension calls (20 DMs) to 20 cold / 0 warm.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Consolidates the persisted-plaintext-decrypt-cache pattern (already used by watch
history) into a shared factory, persistedPlaintextCache.js, and adds a DM cache
on top of it. watchHistoryDecryptCache now delegates to the factory (same public
API; tests unchanged).
decryptDirectMessageEvent now also consults/writes a persisted cache keyed by the
immutable DM event id (JSON of the decrypt result; flushed once per
listDirectMessages batch; cleared on logout alongside the in-memory cache). DM
events never change, so a reload re-decrypts only messages it has not seen and
skips the (serialized, slow) nip-07 extension for the rest.
Harness (dm-repro, now cold/warm): 20 DMs -> COLD nip04.decrypt 50, WARM 0 —
reloads cost zero extension decrypt calls. Also excludes the actor's own key from
NIP-04 remote candidates (decrypting against self is always wasted). DM unit +
integration and watch-history cache tests green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The UX harness (fake nip-07 extension) revealed that the off-thread verification
could blank the feed: if nostr-tools is slow/unable to load inside the verify
worker, the worker hangs, each batch waits the full 10s timeout, and the
main-thread fallback could return an empty set (fail-closed) — dropping every
event so no videos ever render, silently (devLogger is suppressed in prod).
Harden it so verification can never stall or blank the read feed:
- Worker timeout 10s -> 2.5s, and disable the worker for the session after one
timeout/error so we never pay the timeout on every batch.
- Bound the main-thread nostr-tools wait (Promise.race ~2s) so a slow/failed
load can't hang the fallback.
- Fail OPEN when the verifier is genuinely unavailable: pass events through
unverified rather than dropping them (the pool already does not verify, so this
does not regress the prior posture). Real verification still runs whenever
nostr-tools is available.
Adds a regression test (hung worker -> resolves via fallback, worker disabled).
Also updates the harness to seed mode:"live" videos + a NIP-65 relay list for
mock-relay isolation.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch history is ~150 separately encrypted chunk events; decrypting each one is
a serialized round-trip to the nip-07 extension (the key lives in the extension
and cannot be cached locally), so a full re-decrypt cost many seconds on EVERY
load. Chunk events are addressable/replaceable: an unchanged chunk keeps its
event id while a modified chunk republishes under a new id.
Add a persisted cache (watchHistoryDecryptCache.js) keyed by chunk event id. The
fetch path checks it before calling the extension and stores successful
decryptions, so reloads decrypt only the chunk(s) that actually changed
(typically one) and skip the extension for the rest — across sessions. Bounded
to 400 entries; cleared on logout alongside the conversation-key cache. The
cached plaintext is no more sensitive than the watch-history items already
persisted by the manager.
Adds watchHistoryDecryptCache.test.mjs (round-trip, persistence/re-hydrate,
bounds, clear).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Profiling the live app showed the single biggest CPU cost was the relay pool
verifying every incoming event's schnorr signature synchronously on the main
thread: ~1.2s of secp256k1 work ([email protected] 855ms) during feed load and tab
switches, dropping to ~16ms when skipped. The pool was constructed as
`new SimplePool()` with no options, so nostr-tools used its default verifying
verifyEvent; the `verifyEvent: () => true` that looked like it disabled this
only existed on a mock tools object and was never wired to the real pool.
Move verification off the main thread instead of dropping it:
- Construct the pool with `verifyEvent: () => true` so it never blocks.
- Add signatureVerifyWorker (+ client) that verifies event batches in a Web
Worker and returns the valid ids; falls back to main-thread verification only
when Workers are unavailable (never silently trusts).
- VideoEventBuffer verifies each batch via the worker before committing, and
drops events that fail (fails closed). The verifier is injectable so tests can
virtualize the worker; subscribeVideos passes it through and returns the flush
promise from EOSE so commits are awaitable.
Tests: new signatureVerifyWorkerClient.test.mjs (worker protocol + drop-invalid
+ empty/no-id handling); videoEventBuffer.test.mjs and client.test.mjs adapted
to the async, verified commit path with a passthrough verifier.
Note: tests/nostr-delete-flow.test.mjs has a PRE-EXISTING failure on
`should not trigger subscription callbacks` (fails on clean unstable, and is a
standalone script not run by the unit-test runner); its edits here only adapt it
to the now-async flush.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
subscribeVideos called pool.sub(getHealthyRelays(this.relays)) with no
empty-relay fallback, unlike the fetch paths in the same file. On a cold first
load the 5s relay connect probe can time out, marking every relay unreachable;
init() only warns and schedules a background reconnect (it does not throw), so
boot proceeds and the feed subscribes to ZERO relays. The background reconnect
later succeeds but nothing re-issues the subscription, so the feed stays empty
until the user refreshes (when the SWR cache renders) — the reported
"notes don't load on first visit / need a refresh" symptom.
Fall back to DEFAULT_RELAY_URLS when no relays are healthy, mirroring the
established pattern. Because the fallback bypasses the unreachable cooldown, the
subscription retries those relays directly and self-heals instead of staying
dead until refresh.
Adds tests/nostr/subscribeVideosRelayFallback.test.mjs: empty healthy set =>
default relays (never zero), populated healthy set => uses those relays.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch history is stored as up to ~150 separately NIP-44-encrypted chunk
events. Each chunk built a fresh cipher suite whose conversation-key cache
lived only in its own closure, so the ECDH conversation key (~12ms each in
the bundle) was re-derived once per chunk — multiplied across actor-candidate
and scheme-fallback retries this synchronously blocked the main thread for up
to ~20s after login/profile open (the reported freeze + fan spin-up).
Memoize the derived key by (scheme, privateKey, target) at module scope so it
is computed once per load and reused. Cleared on logout/identity change next
to clearDecryptionSchemeCache so a derived shared secret never outlives its
session. On the 150-chunk path this drops decrypt time ~1955ms -> ~182ms.
Adds tests/nostr/watchHistoryConversationKeyCache.test.js covering: 150 chunks
=> one derivation, distinct identities derive independently (no cross-identity
reuse), and cache-clear forces re-derivation.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
- Updated `js/nostr/adapters/nsecAdapter.js` to ensure private keys are `Uint8Array` bytes before passing to `getPublicKey`, fixing compatibility with `nostr-tools` v2.
- Updated `tests/nostr/nip46Client.test.js` to use byte arrays for keys in crypto operations.
- Implemented missing `setupModalMorePopover`, `setupModalSharePopover`, and `refreshModalMoreMenuPanel` in `VideoModal.js`.
- Fixed a bug in `VideoModal.js` where `setCommentComposerState` was overriding `CommentsController` send button logic; it now delegates to the controller if available.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Fix duplicate imports/declarations in `js/ui/components/RevertModal.js`
- Update `js/nostr/nip46Client.js` to pass `Uint8Array` keys to `getConversationKey` for compatibility with `nostr-tools` v2
- Fix NIP-46 compliance tests by using correct key types and improved mocks
- Implement missing methods in `js/ui/components/VideoModal.js` (`handleCopyRequest`, `setCommentsVisibility`, etc.) to fix runtime crashes in tests
- Ensure `VideoModal` stores comment composer state locally to support external controller updates in tests
- Update DM integration tests to use `Uint8Array` keys for `nostr-tools` helpers
- Fix linting issue in `tests/ui/components/DeleteModal.test.mjs` using `Object.defineProperty`
Co-authored-by: PR0M3TH3AN <[email protected]>
- Fixed duplicate import and variable declaration in `js/ui/components/RevertModal.js` which caused syntax errors.
- Fixed missing method implementations in `js/ui/components/VideoModal.js` which caused `bind` errors during initialization.
- Fixed NIP-46 compliance tests by updating mocks to match `nostr-tools` v2 structure.
- Updated `js/nostr/nip46Client.js` to ensure keys are passed as `Uint8Array` to `nostr-tools` helpers, maintaining compatibility with legacy hex strings.
- Updated `tests/nostr/dm-direct-message-flow.test.mjs` to use `Uint8Array` keys, fixing "expected Uint8Array" errors.
- Maintained the previous lint fix for `tests/ui/components/DeleteModal.test.mjs`.
Co-authored-by: PR0M3TH3AN <[email protected]>
Optimizes the `subscribeVideos` callback in `js/services/nostrService.js` to handle video updates incrementally, avoiding O(N*M) complexity during high-volume updates (e.g., initial load).
Changes:
- Modified `js/nostr/videoEventBuffer.js` to include deleted videos in the flushed batch, allowing consumers to handle deletions incrementally.
- Updated `js/services/nostrService.js` to process `updatedVideos` incrementally:
- Separates updates into active and deleted.
- Updates `this.videosMap` directly (removing deleted, adding/updating active).
- Filters and caches only the active updates (O(M)).
- Emits `videos:updated` with the full sorted list (derived from the updated map) to maintain API contract with consumers.
- Persists the full sorted list to localStorage.
This reduces the per-batch overhead significantly by avoiding O(N) filtering and map reconstruction for every batch of updates.
Co-authored-by: PR0M3TH3AN <[email protected]>
Replaced `pMap` with `Promise.all` in `publishRecords` to remove concurrency limit for faster syncing of multiple month records.
Measured ~50% improvement in mock benchmark (200ms -> 100ms for 10 items with concurrency 5 vs unbounded).
Note: The task description described the existing code as a sequential `for` loop, but the actual file used `pMap` with concurrency 5. This change optimizes it further by removing the concurrency limit as requested.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Replaced unbounded `Promise.all` with `pMap` and `RELAY_BACKGROUND_CONCURRENCY || 3` in `js/nostr/dmSignalEvents.js` and `js/services/dmNostrService.js` to prevent network saturation.
- Increased `maxBuffer` in `scripts/check-hex.js` to 10MB to prevent `ENOBUFS` when git text search matches large minified files in CI.
- Verified documentation in `content/docs/guides/upload-content.md` correctly specifies port 3000.
- Passed all unit and integration tests.
Co-authored-by: PR0M3TH3AN <[email protected]>
Replaces unbounded Promise.all with pMap and RELAY_BACKGROUND_CONCURRENCY (3) in js/nostr/dmSignalEvents.js and js/services/dmNostrService.js to prevent network saturation. Includes context logs and daily performance report.
Co-authored-by: PR0M3TH3AN <[email protected]>
Refactors `_softDeleteVersions` to execute `revertVideo` calls in parallel using `Promise.all`. This allows network publishing operations (which are the bottleneck) to overlap, while `SignRequestQueue` ensures signing remains sequential and safe.
Measured improvement for 20 versions: ~2200ms -> ~300ms.
Verification:
- `tests/nostr-delete-flow.test.mjs` passed.
- `npm run lint` checks passed.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Refactored duplicate `5000` ms timeouts in `js/nostr/nip07Permissions.js`, `js/nostr/managers/SignerManager.js`, and `js/nostr/nip46Client.js`.
- Introduced `NIP07_EXTENSION_WAIT_TIMEOUT_MS` and `NIP46_PING_TIMEOUT_MS`.
- Created execution artifacts and scheduler completion log.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Add "Dev Tools" button to the sidebar footer menu.
- Configure `js/constants.js` with `DEV_DASHBOARD` view type.
- Refactor `views/dev/agent-dashboard.html` to be compatible with `loadView` (moved styles to body, updated links).
- Add cleanup logic to dashboard script to prevent memory leaks when navigating away.
- Fix duplicate import in `js/nostr/reactionEvents.js`.
Co-authored-by: PR0M3TH3AN <[email protected]>
Removed `read_relays` and `write_relays` from `DEFAULT_NIP07_CORE_METHODS` in `js/nostr/nip07Permissions.js` as they are not standard NIP-07 or NIP-46 permission strings and were causing login failures with some extensions. Also ensured `jsdom` is installed for unit tests.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Extracted `CommentTargetNormalizer` and related helpers to `js/nostr/commentTargetNormalizer.js`.
- Refactored `js/nostr/commentEvents.js` to import from the new module.
- Compacted helper functions in `js/nostr/commentTargetNormalizer.js` to ensure it stays under the 1000-line limit for new files.
- Verified with `npm run lint:file-size` and `tests/nostr/comment-events.test.mjs`.
Co-authored-by: PR0M3TH3AN <[email protected]>
Replaces unbounded `Promise.all(relays.map(...))` calls with `pMap` using a concurrency limit of 3 for background operations like history hydration, metadata fetching, and DM sending.
This prevents network saturation and potential main-thread jank when many relays are configured.
Includes benchmark verification script `benchmarks/relay_concurrency_repro.mjs`.
Audited upload docs and confirmed accuracy.
Co-authored-by: PR0M3TH3AN <[email protected]>
