Legacy/migrated items with no watchedAt were siloed into a literal
"1970-01" month event that self-perpetuated (read back as 0 -> re-bucketed
-> re-published with 0). Derive a stable watch time from the pointer
d-tag's embedded ms-timestamp and backfill it so they land in a real month
and the loop stops. Bucketing extracted to watchHistoryBucketing.js to keep
watchHistory.js under its size cap.
Also fixes three stale tests in watch-history.test.mjs (partial-acceptance,
actor alignment, session flag) so the file is un-quarantined.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
This is why logged-in removal "worked but threw a flag error", and likely why
new watches stopped appearing in history. publishMonthRecord reported success
only when EVERY write relay accepted (acceptedCount === relays.length). With a
20+ relay list many always reject (auth-required/rate-limited/not-in-write-set),
so every snapshot — removals AND new-watch tracking — was marked
"partial-relay-acceptance" -> ok:false -> "watch-history-snapshot-failed". The
data WAS published (major relays accepted), but the UI showed a failure and the
queue treated it as not-yet-synced.
A watch-history event is durable if any relay accepted it (reads take the newest
copy per d-tag), so success = acceptedCount > 0; only a zero-accept publish is a
retryable failure. Also dropped the redundant partialAcceptance alias to stay
within the file-size budget.
Tests + mutation-verified: 1-of-3 accept -> ok/non-retryable; 0-of-3 ->
failure/retryable. Reverting to the all-relays criterion fails the partial case.
Build + full lint green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Third and final watch-history-delete gap: removing the LAST item of a month (or
clearing all) left the file's items live on relays. updateWatchHistoryList
already sets records[month]=[] to clear an emptied month, but publishMonthRecord
early-returned on an empty month (`if (!items.length) return {ok:true}`), so the
clearing event was never published and relays kept serving the old, full copy.
Now an empty month still publishes a newest, EMPTY replaceable event for that
d-tag, overwriting the old copy so the clear actually takes effect (and reads —
which take the newest per d-tag after fff2bef2 — see it as empty).
Tests + mutation-verified: an emptied month publishes a clearing event with the
right d-tag and no pointer tags, while a month that still has items publishes its
remaining pointers. Restoring the early-return fails the clearing scenario.
Build + full lint green (watchHistory.js under file-size budget).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Second root cause of "watch history delete doesn't work / list isn't getting
overwritten": reads UNION items across every event the relays return, and a
publish only reports success when ALL relays accept. With a flaky relay list the
reduced (post-removal) event reaches some relays while others keep serving the
old, larger copy — and the reader deduped only by event id, so both the stale
and fresh versions of the same month (d-tag) were unioned, resurrecting removed
items.
Add dedupeNewestPerReplaceableAddress and apply it at both read union sites (the
chunk-events path and the pointer-events path): keep only the newest event per
d-tag (month) before unioning, so a lagging relay can't reintroduce removed
items. Distinct months are all preserved; events without a d-tag are kept by id.
Tests + mutation-verified: newest-per-month wins (incl. the fewer-items removal
case), distinct months preserved, newest-first ordering, no-d-tag kept by id.
Reverting to id-dedup fails the resurrection scenarios.
Build + full lint green (watchHistory.js back under file-size budget).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
File-size decomposition (no behavior change). Move the dependency-free encode/
decode helpers (sanitizeWatchHistoryMetadata, serializeWatchHistoryItems,
bytesToHex, looksLikeJsonStructure, hexToBytesCompat) into a new
js/nostr/watchHistoryCodec.js and import them back.
watchHistory.js: 2512 -> 2435 lines (back under its 2438 limit).
Build + full unit suite green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Service-layer plumbing for bounded watch-history loading (slice B part 2).
manager.resolve() now forwards chunkDecryptLimit to fetch() and records the
resulting deferred-chunk count in a per-actor side-channel, so resolve() keeps
returning a bare items array (its sole caller depends on that) while the
"more older chunks available" signal is still retrievable. Exposed via
getWatchHistoryDeferredChunkCount() on the manager and the NostrClient facade.
clear() resets the side-channel.
Still inert by default (no caller passes a finite limit yet); this is the
substrate for activating fast-first-render. Tested: resolve under
chunkDecryptLimit:1 returns only the newest chunk's items and reports
deferredChunkCount=1, reset to 0 after clear().
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Foundation for bounded watch-history loading (slice B part 2). A deep
history references many monthly chunks; under a NIP-46 remote signer each
uncached chunk decrypt is a serialized published relay RPC, so decrypting
them all up front blocks the first render and risks the relay rate limit.
manager.fetch now:
- sorts chunk events newest-first by created_at, so the decrypt budget is
spent on the most recent history;
- accepts a chunkDecryptLimit option that caps how many *uncached*,
genuinely-encrypted chunks are decrypted per fetch. Cached/plaintext
chunks are always decrypted (they cost no RPC) and never count against
the limit. Default is Infinity -> no behavior change for existing
callers;
- returns deferredChunkCount so a caller can tell more history is
available on demand (re-fetching with a larger limit reuses the chunk
cache, so only the newly requested older chunks cost RPCs).
Test: two uncached nip44 chunks (newest + older) with chunkDecryptLimit:1
asserts only the newest is decrypted (1 RPC), deferredChunkCount is 1, and
only the newest chunk's items appear. Mutation-verified against both
removing the cap and removing the newest-first sort.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch-history decryption previously ordered the matching cipher family
first but kept the other family as a *fallback*. The opposite family can
never decrypt a given ciphertext (NIP-04 always carries a literal "?iv="
marker; NIP-44 is plain base64), so that fallback was a guaranteed-wasted
decrypt attempt. Under a NIP-46 remote signer every decrypt is a
*published* relay RPC routed through the serial 250ms request queue, so a
transient failure on the correct family triggered a second wasted RPC
that burned the queue and pushed cold loads toward the relay rate limit
("noting too much") — the symptom users hit.
determineWatchHistoryDecryptionOrder now filters the attempt set to the
single family implied by the ciphertext shape (trusting the ciphertext
over a possibly-misleading "encrypted" tag), with a safety fallback to
the full ordered list when nothing matches so the only available
decryptor is never stranded. This mirrors the list-decrypt routing
already shipped for userBlocks / subscriptions / hashtags (slice A).
Test: a NIP-44 ciphertext mislabeled with an ["encrypted","nip04"] tag,
against a signer exposing both decryptors where nip04 returns decoy data.
Asserts nip04 is never attempted and the nip44 plaintext is used.
Mutation-verified: removing the filter fails the test.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
watchHistory.fetch now reads via the L1 SubscriptionManager (deduped +
health-gated) instead of pool.list directly: a new getSubscriptionManager dep
is threaded from the client, and both fetches (index + chunks) go through a
listEvents() helper. Raw pool.list is retained only as a fallback for
environments without a manager (tests/mocks), so the file stays grandfathered.
Verified: watch-history unit test passes; wh-repro cold resolve returns 10
items with 2 decrypts (routing intact end-to-end).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Root cause of the ~20s watch-history load: history is bucketed by month (one
encrypted snapshot event per month) and the fetch decrypted EVERY month via the
serialized nip-07 extension — ~1 slow call per month, so accounts with years of
history paid dozens of back-to-back extension decrypts on every load.
Cap the load to the most recent RECENT_MONTHS_DECRYPT_LIMIT (6) months by default
— enough for the feed's watched-video suppression and the initial view. Callers
that need the full history (the explicit History view) pass recentMonthsLimit: 0,
threaded loadLatest -> scheduleWatchHistoryRefresh -> resolveWatchHistory ->
resolve -> fetch.
Harness (fresh-context wh-repro): 600 items / 18 months goes from 18 nip-07
decrypt calls to 6. Watch-history tests green. Bumps the watchHistory.js
file-size baseline for the added logic.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch history is ~150 separately encrypted chunk events; decrypting each one is
a serialized round-trip to the nip-07 extension (the key lives in the extension
and cannot be cached locally), so a full re-decrypt cost many seconds on EVERY
load. Chunk events are addressable/replaceable: an unchanged chunk keeps its
event id while a modified chunk republishes under a new id.
Add a persisted cache (watchHistoryDecryptCache.js) keyed by chunk event id. The
fetch path checks it before calling the extension and stores successful
decryptions, so reloads decrypt only the chunk(s) that actually changed
(typically one) and skip the extension for the rest — across sessions. Bounded
to 400 entries; cleared on logout alongside the conversation-key cache. The
cached plaintext is no more sensitive than the watch-history items already
persisted by the manager.
Adds watchHistoryDecryptCache.test.mjs (round-trip, persistence/re-hydrate,
bounds, clear).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Watch history is stored as up to ~150 separately NIP-44-encrypted chunk
events. Each chunk built a fresh cipher suite whose conversation-key cache
lived only in its own closure, so the ECDH conversation key (~12ms each in
the bundle) was re-derived once per chunk — multiplied across actor-candidate
and scheme-fallback retries this synchronously blocked the main thread for up
to ~20s after login/profile open (the reported freeze + fan spin-up).
Memoize the derived key by (scheme, privateKey, target) at module scope so it
is computed once per load and reused. Cleared on logout/identity change next
to clearDecryptionSchemeCache so a derived shared secret never outlives its
session. On the 150-chunk path this drops decrypt time ~1955ms -> ~182ms.
Adds tests/nostr/watchHistoryConversationKeyCache.test.js covering: 150 chunks
=> one derivation, distinct identities derive independently (no cross-identity
reuse), and cache-clear forces re-derivation.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Replaced `pMap` with `Promise.all` in `publishRecords` to remove concurrency limit for faster syncing of multiple month records.
Measured ~50% improvement in mock benchmark (200ms -> 100ms for 10 items with concurrency 5 vs unbounded).
Note: The task description described the existing code as a sequential `for` loop, but the actual file used `pMap` with concurrency 5. This change optimizes it further by removing the concurrency limit as requested.
Co-authored-by: PR0M3TH3AN <[email protected]>
Implement "race-to-success" in publishEventToRelays to reduce perceived latency.
Default behavior now resolves as soon as one relay accepts the event.
Added `waitForAll: true` option for strict consistency requirements (e.g. watchHistory).
Co-authored-by: PR0M3TH3AN <[email protected]>
Verified performance improvement via benchmark:
- Serial (Baseline): 805ms
- Parallel (Concurrency 3): 295ms
- Parallel (Concurrency 5): 216ms
The repository was found to be already using pMap with concurrency 3, contrary to the issue description. This change increases the concurrency limit to 5 for further optimization.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Use pMap for parallel publishing of history months
- Increase concurrency limit to 4 for better throughput
- Add benchmark script for measuring publish performance
Co-authored-by: PR0M3TH3AN <[email protected]>
Replaces `Promise.all` with `pMap` (concurrency: 5) in `WatchHistoryManager.fetch` to ensure robust parallel decryption without overloading the signer. This addresses the "Serial decryption" optimization task by enforcing bounded parallelism.
Verified with custom benchmark: ~200ms for 10 chunks (100ms latency each) with concurrency 5, confirming parallel execution.
Co-authored-by: PR0M3TH3AN <[email protected]>
Replaces serial decryption loop with Promise.all to improve fetch performance for users with large history.
Measured ~50x speedup for 50 chunks (5000ms -> 100ms) with local signer.
Extension signers are safely queued by Nip07RequestQueue.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Moved heavy tag counting and IDF calculation to `js/workers/exploreData.worker.js`.
- Extracted `collectVideoTags` to `js/utils/videoTags.js` (preserved original behavior).
- Extracted pointer normalization logic to `js/utils/pointerNormalization.js`.
- Refactored `js/services/exploreDataService.js` to use the worker.
- Added benchmark test `tests/benchmark-explore-data.spec.mjs`.
Measured Improvement:
- `buildTagIdf` execution time on main thread reduced from ~130ms blocking to ~0ms blocking (141ms total async time).
- Unblocks UI during periodic data refreshes.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Reduced view threshold from 12s to 5s in watchHistoryTelemetry.js
- Enabled immediate watch history republishing (0ms delay) for the first attempt in watchHistory.js
- Updated watchHistoryService.js to recursively schedule queue flushes if items accumulate during an inflight sync
- Update `cachePolicies.js` to include `defaultDTag` for watch history.
- Update `profileCache.js` to resolve policies dynamically via `SECTION_TO_NOTE_TYPE`.
- Implement `resolveAddressKey` in `profileCache.js` for canonical key generation.
- Implement TTL expiration logic in `profileCache.js`.
- Update `watchHistory.js` to use `NOTE_TYPES.WATCH_HISTORY` for storage operations.
- Updated `profileCache.js` to support clearing runtime memory on signer change.
- Updated `NostrClient` to trigger `clearSignerRuntime` on `setActiveSigner`.
- Updated list services (`subscriptions`, `userBlocks`, `hashtagPreferences`, `watchHistory`) to listen for `runtimeCleared` and reload data.
- Updated `NostrClient` sync metadata methods to default to the active profile pubkey.
- Added `partition-updated` event emission to `profileCache`.
- Updated CSS for `.watch-history-card` to use `flex-col` on mobile for stacked layout, switching to `sm:flex-row` on larger screens.
- Added `w-full` to `.watch-history-card__thumbnail` on mobile to constrain image width.
- Fixed a bug in `js/nostr/watchHistory.js` where `computeWatchHistoryFingerprintForItems` was sorting the live history array in-place by ID, causing the UI to display an incorrect (non-chronological) order. The fix involves creating a shallow copy before sorting.
- Updated `tests/nostr/watchHistory.test.js` to verify behavior against the current storage version (`v5`).
- Enforce descending month order in `js/nostr/watchHistory.js` by explicitly sorting keys before flattening buckets.
- Update `js/feedEngine/watchHistoryFeed.js` and `js/historyView.js` to use `video.created_at` as a fallback sort key when `watchedAt` is missing or equal.
- Verified logic with `tests/verify_sort.mjs`.
- Removed `WATCH_HISTORY_LEGACY_KIND` (30078) support from watch history fetch and config.
- Removed `WATCH_HISTORY_LEGACY_LIST_IDENTIFIERS` from config.
- Removed `LEGACY_EVENTS_STORAGE_KEY` (bitvidEvents) migration logic from client.
- Removed `loadLegacyAdminState` and `LEGACY_STORAGE_KEYS` from admin list store.
- Updated tests to remove references to legacy kinds.
Migrates watch history persistence from Kind 30078 (Application Data) to Kind 30003 (Bookmark List) to resolve blocking issues on some relays. Updates fetch logic to query both kinds for backward compatibility.
Removes the metadata argument from `WatchHistoryService.publishView` and cleans up the associated caching and resolution logic in `watchHistory.js` and `watchHistoryFeed.js`. Deletes `js/watchHistoryMetadata.js` as it is no longer used.
This simplification ensures watch history strictly manages pointers and timestamps, leaving metadata resolution to higher-level consumers. It also reduces the payload size and complexity of persisted history records.
Tests have been updated to reflect the API changes.
Removes legacy snapshot/chunk retry scheduling in favor of a monthly upsert approach.
Updates state persistence to remove `pendingSnapshotId`.
Fixes race condition in `scheduleRepublishForQueue` by using selective queue pruning.
Ensures data is not lost on publish failure in `snapshot`.
Updates tests to align with new architecture.
- Update `js/nostrEventSchemas.js` to remove encryption tagging and update schema for `WATCH_HISTORY`.
- Update `js/nostr/watchHistory.js` to:
- Bump storage version to 4.
- Bucket items by `YYYY-MM`.
- Publish plaintext JSON payloads with only event IDs and timestamps.
- Remove encryption logic from publishing.
- Update fetching to handle plaintext and fallback to legacy parsing.
- Update `js/watchHistoryService.js` to strip metadata from queued items and handle new snapshot response.
- Update `tests/watch-history.test.mjs` to reflect new plaintext behavior and disabled obsolete tests.
Refactors the watch-history system to prioritize `YYYY-MM` month identifiers over legacy snapshot IDs in event tags, storage keys, and cross-module references. Updates event schemas, storage keying (v3), and client interfaces to align with the new month-based structure. Renames `snapshotId` parameters to `taskId` for republishing operations.
Refactors `js/nostr/watchHistory.js` to store and publish watch history items in monthly buckets (`YYYY-MM`) instead of using chunk splitting. Updates storage version to 3, migrates existing data, and simplifies fetch/merge logic to handle monthly records. Removes legacy chunk rebalancing code. Updates tests to verify monthly bucketing behavior.
- Updated `buildWatchHistoryPayload` to remove legacy `snapshotId` logic.
- Updated `publishSnapshot` to use `YYYY-MM` as the `d` tag identifier.
- Updated `fetch` logic to retrieve all history events (removing restrictive `#d` filter) to ensure complete history across months is loaded.
- Fixed a runtime error in `fetch` where `now` was undefined.
- Updated `js/nostrEventSchemas.js` to prioritize `monthIdentifier` for the `d` tag.
- Removed legacy `snapshot` field from payload serialization.
- Rename `WATCH_HISTORY_CHUNK` schema to "Watch history monthly payload" in `js/nostrEventSchemas.js`.
- Update `js/nostr/watchHistory.js` to replace `snapshotId` with `recordId` and `monthIdentifier` in logging and error tracking context.
- Ensure `fetch` logic uses `recordId` and `month` in chunk decryption context for clarity.
- Fix indentation for added fields in logging objects.
This change updates `normalizePointersFromPayload` to process both legacy `items` and new `events` arrays from the watch history payload. It also adds logic to merge `watchedAt` timestamps from a map, supporting the new monthly format. Additionally, `mergeWatchHistoryItemsWithFallback` is updated to rely on the unified `items` list and hydrate metadata from fallback items for all entries, ensuring robust handling of sparse event pointers.
