diff --git a/docs/nostr-client-overview.md b/docs/nostr-client-overview.md new file mode 100644 index 00000000..f5e19a0d --- /dev/null +++ b/docs/nostr-client-overview.md @@ -0,0 +1,91 @@ +# NostrClient Overview + +`js/nostr/client.js` exports the `NostrClient` class, which serves as the central controller for all Nostr network interactions and state management within the application. It acts as a facade over `nostr-tools`, managing connections, subscriptions, caching, and signing. + +## Key Responsibilities + +1. **Connection Management**: Maintains a pool of connections to multiple relays (`SimplePool`). +2. **State Management**: Tracks video events, deduplicates them, and manages deletions via tombstones. +3. **Caching**: Implements a "stale-while-revalidate" strategy using IndexedDB (with localStorage fallback) to restore UI state immediately upon load. +4. **Event Publishing**: Orchestrates the signing and broadcasting of complex event chains (Video Note -> NIP-94 Mirror -> NIP-71 Metadata). +5. **Signer Abstraction**: Unifies NIP-07 (Extension), NIP-46 (Remote/Bunker), and Local (nsec) signing under a single interface. +6. **Video History**: Reconstructs the edit history of videos using `videoRootId` and `d` tags. + +## Architecture & State + +The client maintains several internal maps to manage state: + +* `allEvents`: A raw map of all fetched video events (key: event ID). +* `activeMap`: A derived map containing only the *latest* version of each video, keyed by a stable identifier (`videoRootId` or addressable pointer). +* `rawEvents`: A cache of the exact raw JSON events from relays, used for signature verification and re-publishing. +* `tombstones`: Tracks timestamps of deletion events to prevent "zombie" events (deleted events reappearing from slow relays) from resurfacing. + +### Caching Strategy (`EventsCacheStore`) + +To ensure instant load times, the client persists its state to IndexedDB (`bitvid-events-cache`). +* **Snapshot**: On a debounced timer (or idle callback), the current state (`allEvents`, `tombstones`) is serialized. +* **Fingerprinting**: Events are fingerprinted to avoid writing unchanged data to the DB. +* **Restore**: On `init()`, the client loads the latest snapshot into memory, allowing the UI to render before any network requests complete. + +## Key Flows + +### 1. Initialization (`init`) + +```mermaid +graph TD + A[init()] --> B[restoreLocalData] + B --> C{IndexedDB Available?} + C -- Yes --> D[Load Snapshot] + C -- No --> E[Load localStorage] + D & E --> F[Render UI with Cached Data] + F --> G[ensurePool] + G --> H[connectToRelays] + H --> I[Ready for Subscriptions] +``` + +### 2. Video Subscription (`subscribeVideos`) + +To prevent UI thrashing during high-volume event ingress (e.g., initial flood from relays), the client uses a **buffering and debouncing** strategy. + +1. **Buffer**: Incoming events are pushed to `eventBuffer` instead of being processed immediately. +2. **Debounce**: A flush timer is scheduled (75ms). +3. **Flush**: When the timer fires, the entire buffer is processed: + * Events are validated and converted. + * `activeMap` is updated (latest-wins logic). + * `onVideo` callback is fired once with the batch updates. + * Cache persistence is scheduled. + +### 3. Publishing a Video (`publishVideo`) + +Publishing is a multi-step process ensuring compatibility and discoverability. + +1. **Video Note (Kind 30078)**: The core content (magnet, infohash, metadata) is signed and published. A unique `videoRootId` is generated for new videos. +2. **NIP-94 Mirror (Kind 1063)**: If a direct HTTP URL is provided, a file header event is published to point to the hosted file. +3. **NIP-71 Metadata (Kind 22/21)**: Addressable metadata tags are published to support categorization/grouping without modifying the immutable video note. + +### 4. Remote Signer Handshake (`connectRemoteSigner`) + +For NIP-46 (Nostr Connect), the client implements a full handshake flow: + +1. **Ephemeral Key**: Generates a local key pair for the session. +2. **Handshake**: Publishes a Kind 24133 (`NIP46_RPC_KIND`) event to the relay. +3. **Wait**: Listens for an acknowledgment or `auth_url` challenge from the signer. +4. **Session**: Once connected, stores the session metadata to allow auto-reconnection. + +## Why it works this way + +* **Buffering**: essential for performance. React rendering cycles are expensive; updating state for every single event in a 5000-event relay dump would freeze the browser. +* **Tombstones**: essential for eventual consistency. Relays are distributed and can be out of sync. If a user deletes a video, we must remember that deletion timestamp so a slow relay doesn't "undelete" it by serving the old event later. +* **Dual-Layer Cache**: IndexedDB is asynchronous and large-capacity; localStorage is synchronous but small. We use IndexedDB for the bulk data but keep a fallback for environments where IDB might be flaky or slow to initialize. + +## When to change + +* **Refactor `EventsCacheStore`**: If moving to a more robust state management library (like RxDB or dedicated worker), this logic should be extracted. +* **New Event Kinds**: If the video data model changes (e.g., V4 schema), `convertEventToVideo` and `publishVideo` will need updates. +* **Signer upgrades**: As NIP-46 evolves (e.g., Bunker standards), the handshake and connection logic in `nip46Client.js` (and its orchestration here) will need adjustment. + +## Related Files + +* `js/nostrEventSchemas.js`: Defines the structure of the events being built/parsed. +* `js/nostr/nip46Client.js`: Handles the low-level RPC for remote signers. +* `js/state/profileCache.js`: Handles user profile metadata (Kind 0), which is separate from video data. diff --git a/js/nostr/client.js b/js/nostr/client.js index f53b6fac..9a263105 100644 --- a/js/nostr/client.js +++ b/js/nostr/client.js @@ -397,6 +397,19 @@ function waitForTransaction(tx) { }); } +/** + * Persists video events and tombstones to IndexedDB. + * + * Schema: + * - `events`: Store for video objects. Key: `id`. + * - `tombstones`: Store for deletion timestamps. Key: `key`. + * - `meta`: Store for versioning and timestamps. Key: `key`. + * + * Strategy: + * - Snapshots are saved periodically (debounced). + * - "Fingerprints" (hashes/stringified) are tracked to minimize writes. + * - If the TTL expires, the cache is cleared on restore. + */ class EventsCacheStore { constructor() { this.dbPromise = null; @@ -503,6 +516,7 @@ class EventsCacheStore { return null; } + // Check TTL: If cache is too old, wipe it and return null to force a fresh fetch. const now = Date.now(); if (now - meta.savedAt > EVENTS_CACHE_TTL_MS) { const tx = db.transaction(["events", "tombstones", "meta"], "readwrite"); @@ -4635,12 +4649,15 @@ export class NostrClient { devLogger.log("Event content:", event.content); try { + // 1. Publish the primary Video Note (Kind 30078) const { signedEvent } = await this.signAndPublishEvent(event, { context: "video note", logName: "Video note", devLogLabel: "video note", }); + // 2. Publish NIP-94 Mirror (Kind 1063) if a hosted URL is present. + // This allows clients that only understand NIP-94 to find the file. if (finalUrl) { const inferredMimeType = inferMimeTypeFromUrl(finalUrl); const mimeTypeSource = @@ -4751,6 +4768,9 @@ export class NostrClient { ); } } else devLogger.log("Skipping NIP-94 mirror: no hosted URL provided."); + + // 3. Publish NIP-71 Metadata (Kind 22) if categories/tags were added. + // This is a separate event linked to the video via `a` tag or `e` tag. const hasMetadataObject = nip71Metadata && typeof nip71Metadata === "object"; const metadataWasEdited = @@ -5816,6 +5836,13 @@ export class NostrClient { const EVENT_FLUSH_DEBOUNCE_MS = 75; let flushTimerId = null; + /** + * Flushes the pending event buffer. + * + * This batch processing is critical for performance during the initial + * relay dump (thousands of events). It prevents React from re-rendering + * for every single event. + */ const flushEventBuffer = () => { if (!eventBuffer.length) { return; @@ -5836,19 +5863,24 @@ export class NostrClient { continue; } + // Merge any NIP-71 metadata we might already have cached for this video this.mergeNip71MetadataIntoVideo(video); + // Determine the "true" creation time of the root video this.applyRootCreatedAt(video); const activeKey = getActiveKey(video); const wasDeletedEvent = video.deleted === true; + // If this is a deletion event (Kind 5 or deletion marker), record a tombstone + // to prevent older versions from resurrecting. if (wasDeletedEvent) { this.recordTombstone(activeKey, video.created_at); } else { + // Otherwise, check if this video is already known to be deleted this.applyTombstoneGuard(video); } - // Store in allEvents + // Store in allEvents (history preservation) this.allEvents.set(evt.id, video); // If it's a "deleted" note, remove from activeMap @@ -5866,11 +5898,13 @@ export class NostrClient { continue; } - // Otherwise, if it's newer than what we have, update activeMap + // Latest-wins logic: only update activeMap if this version is newer const prevActive = this.activeMap.get(activeKey); if (!prevActive || video.created_at > prevActive.created_at) { this.activeMap.set(activeKey, video); onVideo(video); // Trigger the callback that re-renders + + // Fetch NIP-71 metadata (categorization tags) in the background this.populateNip71MetadataForVideos([video]) .then(() => { this.applyRootCreatedAt(video); @@ -6734,6 +6768,9 @@ export class NostrClient { typeof candidate.videoRootId === "string" ? candidate.videoRootId : ""; const candidateDTag = this.resolveEventDTag(candidate, video); + // A video is "the same" if: + // 1. It shares the same V3 `videoRootId`. + // 2. It shares the same NIP-33 `d` tag (for addressable updates). const sameRoot = targetRoot && candidateRoot === targetRoot; const sameD = targetDTag && candidateDTag === targetDTag;