diff --git a/docs/nostr-client-overview.md b/docs/nostr-client-overview.md index 5ddcbaea..bdb392ab 100644 --- a/docs/nostr-client-overview.md +++ b/docs/nostr-client-overview.md @@ -1,92 +1,76 @@ -# NostrClient Overview +# NostrClient Architecture Overview -`NostrClient` (`js/nostr/client.js`) is the central controller for all Nostr network interactions, state management, and persistence in the application. It acts as the bridge between the raw Nostr protocol (relays, events) and the application's UI domain model (Videos, Profiles). +The `NostrClient` (`js/nostr/client.js`) is the central controller for all Nostr network interactions in the application. It acts as a singleton facade over the `nostr-tools` library, managing relay connections, event subscriptions, state synchronization, and cryptographic signing. ## Core Responsibilities -1. **Connection Management**: Maintains a pool of WebSocket connections to relays via `nostr-tools/SimplePool`. -2. **Event Processing**: Subscribes to event streams, buffers incoming bursts, validates schemas, and converts raw events into domain objects. -3. **State Management**: - * **`allEvents`**: A complete history of every video version fetched. - * **`activeMap`**: A materialized view of the *latest* valid version of each video series (deduplicated by Root ID). - * **`tombstones`**: A set of deletion timestamps to prevent "zombie" events (deleted items reappearing from old caches) from being displayed. -4. **Persistence**: Implements a "Stale-While-Revalidate" strategy using IndexedDB (with LocalStorage fallback) to restore the UI instantly on load. -5. **Signer Orchestration**: Manages NIP-07 (Browser Extension), NIP-46 (Remote/Bunker), and Local (nsec) signers. +1. **Connection Management**: Maintains a persistent connection pool (`SimplePool`) to multiple relays. +2. **State Management**: Tracks the global state of video events, including history, active versions, and deletions. +3. **Event Ingestion**: Handles high-volume event streams with buffering and deduplication. +4. **Caching**: Implements a persistent IndexedDB cache to support "Stale-While-Revalidate" loading (instant UI render). +5. **Signer Orchestration**: Manages NIP-07 (Browser Extension), NIP-46 (Remote Signer), and Local (nsec) signers seamlessly. -## State Management Architecture +## State Management -The client maintains three primary maps to handle the "Eventual Consistency" nature of Nostr: +The client maintains three primary state maps to ensure data consistency: -* **`allEvents` (`Map`)**: - * Stores every valid video event received. - * Used for history hydration (viewing edit logs) and debugging. -* **`activeMap` (`Map`)**: - * Stores only the *most recent* version of a video. - * Keyed by `videoRootId` (or `pubkey:dTag` for legacy). - * **Invariant**: `activeMap.get(id).created_at` is always >= any other version of that ID. -* **`tombstones` (`Map`)**: - * Stores the timestamp of the latest deletion event for a video. - * **Invariant**: Any event with `created_at <= tombstone` is implicitly treated as deleted, even if the "deleted" flag isn't set on the event itself. This handles out-of-order delivery where a "delete" arrives before the "video" it deletes. +* **`allEvents`** (`Map`): + * Stores *every* valid video event fetched from relays, indexed by Event ID. + * Used for history preservation and resolving direct links to specific versions. -## Caching Strategy +* **`activeMap`** (`Map`): + * The "materialized view" used by the UI. + * Keys are `videoRootId` (for versioned edits) or `pubkey:dTag` (for addressable events). + * Values are the *latest* valid version of the video. + * **Invariant**: Only one entry exists per "video entity", ensuring the UI displays the most recent edit. -The client uses a multi-layered approach to ensure fast boot times: +* **`tombstones`** (`Map`): + * Tracks deletions (Kind 5). + * Keys are the `activeKey` of the deleted item. + * Values are the timestamp of the deletion. + * **Invariant**: Events older than the tombstone are strictly ignored, preventing "zombie" events from reappearing. -1. **IndexedDB (`events` store)**: - * Primary cache for video events. - * Stores fingerprints of objects to avoid redundant writes. -2. **LocalStorage (`bitvid:eventsCache:v1`)**: - * Fallback if IndexedDB is unavailable. - * Size-limited (typically < 5MB). -3. **Buffering**: - * Incoming events are buffered in memory and flushed to IDB periodically (debounced) to avoid I/O thrashing during heavy relay traffic. +## Data Ingestion Flow -## Event Processing (Buffering & Debouncing) +The `subscribeVideos` method implements a buffered ingestion pipeline to handle high-throughput streams without freezing the UI: -When `subscribeVideos` is called, the client expects a flood of events (e.g., 500+ items from relay history). Processing these one-by-one would freeze the UI. - -### Flow +1. **Subscription**: Opens a subscription to all connected relays. +2. **Buffering**: Incoming events are pushed to an `eventBuffer` array immediately. +3. **Debouncing**: A timer flushes the buffer every ~250ms (or when full). +4. **Processing (Flush)**: + * Events are parsed and validated. + * **Tombstone Check**: If an event predates a known deletion, it is discarded. + * **Latest-Wins Resolution**: The new event's `created_at` is compared against the existing entry in `activeMap`. + * **Update**: If the new event is newer, `activeMap` is updated, and the UI callback is triggered. +5. **Persistence**: The updated state is asynchronously persisted to IndexedDB. ```mermaid -sequenceDiagram - participant Relay - participant ClientBuffer - participant Processor - participant UI - - Relay->>ClientBuffer: Event 1 - Relay->>ClientBuffer: Event 2 - Relay->>ClientBuffer: ... Event 50 - ClientBuffer->>ClientBuffer: Buffer (Debounce 75ms) - - loop Flush Timer - ClientBuffer->>Processor: Flush Batch (50 events) - Processor->>Processor: Validate & Convert - Processor->>Processor: Apply Tombstones - Processor->>Processor: Update activeMap (Last-Write-Wins) - Processor->>UI: Callback(onVideo) (Single Render) - Processor->>IndexedDB: Persist Snapshot - end +graph TD + Relay[Relays] -->|Event Stream| Buffer[Event Buffer] + Buffer -->|Debounce Timer| Processor[Processor] + Processor -->|Check| Tombstones{Is Deleted?} + Tombstones -->|Yes| Discard[Discard] + Tombstones -->|No| Compare{Is Newer?} + Compare -->|No| Discard + Compare -->|Yes| ActiveMap[Update ActiveMap] + ActiveMap --> UI[Update UI] + ActiveMap --> IDB[(IndexedDB Cache)] ``` -## History Reconstruction +## Caching Strategy (`EventsCacheStore`) -Videos in Nostr are immutable events. "Editing" means publishing a new event with the same identifier. `NostrClient` reconstructs the history chain using: +To provide an "app-like" experience, the client persists its state to IndexedDB (`bitvid-events-cache`). -1. **`videoRootId` (V3)**: The canonical identifier for a video series. -2. **`d` Tag (NIP-33)**: The addressable identifier. -3. **Legacy IDs**: Fallback to Event ID for V1/V2 posts. +* **Fingerprinting**: To minimize I/O, the store tracks JSON fingerprints (hashes) of all persisted events. +* **Incremental Persistence**: Only changed events (where fingerprint differs) are written to IDB during a snapshot. +* **Stale-While-Revalidate**: On startup (`init()`), the client immediately loads data from IDB to render the UI, then connects to relays to fetch updates. -`hydrateVideoHistory` fetches all events matching these identifiers, sorts them by `created_at` descending, and returns the full edit log. +## Signer Integration -## Why it works this way +The client abstracts the signing mechanism, allowing the user to switch between methods without changing application logic: -* **Eventual Consistency**: The `tombstones` map is essential because relays guarantee neither order nor completeness. A client might receive a "delete" event for a video it hasn't seen yet, or receive an old version of a video after receiving the deletion. Tombstones ensure that once something is known to be deleted, it stays deleted (until a newer re-creation event appears). -* **Performance**: The `eventBuffer` mechanism allows the client to ingest thousands of events without blocking the main thread, as React rendering is expensive. Batching updates reduces render cycles from O(N) to O(1) per batch. -* **Offline-First**: Restoring from IndexedDB immediately allows the app to be interactive before any network connection is established. +1. **NIP-07 (Extension)**: Delegates signing to a browser extension (e.g., Alby, nos2x). +2. **NIP-46 (Remote/Bunker)**: Uses a partial `NostrClient` instance to communicate with a remote signer via a relay. +3. **Local (NIP-01)**: Uses a local private key (nsec) stored in `sessionStorage` (encrypted). -## When to change - -* **Storage Backend**: If the cache schema becomes too complex for IndexedDB, or if we move to a native wrapper (React Native/Tauri) with a SQL backend, the `EventsCacheStore` class should be swapped out. -* **Event Loop**: If we switch to a worker-based architecture (off-main-thread processing), the `subscribeVideos` logic would move to a Web Worker, and `NostrClient` would become a message passer. -* **Protocol Upgrades**: If NIP-33 (Parameterized Replaceable Events) is superseded or the video event kind (30078) changes, the filters in `subscribeVideos` and `fetchVideos` must be updated. +The `activeSigner` object is normalized to expose a standard interface (`signEvent`, `nip04Encrypt`, `nip44Encrypt`), regardless of the underlying implementation. diff --git a/js/nostr/client.js b/js/nostr/client.js index 039d6e17..2916cdcc 100644 --- a/js/nostr/client.js +++ b/js/nostr/client.js @@ -462,6 +462,12 @@ class EventsCacheStore { return typeof indexedDB !== "undefined"; } + /** + * Opens (or returns existing) connection to the IndexedDB database. + * Handles schema upgrades (creating object stores). + * + * @returns {Promise} The database instance or null if not supported. + */ async getDb() { if (!this.isSupported()) { return null; @@ -507,6 +513,14 @@ class EventsCacheStore { return `ts:${timestamp}`; } + /** + * Pre-loads fingerprints of all persisted items into memory. + * + * This is critical for the "Incremental Persistence" strategy. By knowing + * what is already on disk (via fingerprints), we can avoid writing unchanged records. + * + * @param {IDBDatabase} db - The database instance. + */ async ensureFingerprintsLoaded(db) { if (this.hasLoadedFingerprints || !db) { return; @@ -1086,6 +1100,8 @@ export class NostrClient { * - State tracking (allEvents, activeMap). * - Signer negotiation (NIP-07, NIP-46, local). * - Background caching with IndexedDB. + * + * For a detailed architectural overview, see `docs/nostr-client-overview.md`. */ constructor() { /** @@ -3685,6 +3701,18 @@ export class NostrClient { return this.allEvents.size > 0; } + /** + * Restores application state from the best available local cache. + * + * 1. Attempts to load from `IndexedDB` (preferred). + * 2. Falls back to `localStorage` if IDB fails or is empty. + * 3. Rehydrates `allEvents`, `activeMap`, and `tombstones`. + * + * This implements the "Stale-While-Revalidate" pattern: the UI renders + * immediately with this data while network requests proceed in the background. + * + * @returns {Promise} True if data was successfully restored. + */ async restoreLocalData() { if (this.hasRestoredLocalData) { return this.allEvents.size > 0; @@ -4196,6 +4224,16 @@ export class NostrClient { * * @returns {Promise} Resolves when the relay pool is initialized and connections are attempted. */ + /** + * Main entry point for the client. + * + * 1. Restores local state from IndexedDB (Stale-While-Revalidate). + * 2. Initializes the relay connection pool. + * 3. Connects to configured relays. + * 4. Restores any persistent remote signer sessions. + * + * @returns {Promise} Resolves when at least one relay is connected. + */ async init() { if (this.isInitialized) { return; @@ -4304,6 +4342,15 @@ export class NostrClient { // while the 5s timer guards against relays that never respond. We immediately // `unsub` to avoid leaking subscriptions. Note: any future change must still // provide a lightweight readiness check with similar timeout semantics. + /** + * Connects to all configured relays in parallel. + * + * - Skips relays that are already connected. + * - Updates internal connectivity state. + * - Does NOT throw if individual relays fail, but returns a summary. + * + * @returns {Promise>} Connection results. + */ async connectToRelays() { const relayTargets = this.getHealthyRelays(this.relays); const results = await Promise.all( @@ -6899,12 +6946,16 @@ export class NostrClient { } /** - * Schedules a persistence task to save the current state (events, tombstones) to local storage. + * Schedules a persistence operation to save current state to cache. + * + * - Uses a debounce strategy (75ms) to prevent write thrashing during bursts. + * - Can be forced to run immediately via `options.immediate`. + * - Coordinates with `requestIdleCallback` to avoid blocking the main thread. * * @param {string} [reason="unspecified"] - Debug label for why persistence was triggered. - * @param {object} [options] - * @param {boolean} [options.immediate=false] - If true, bypasses the debounce timer and saves immediately. - * @returns {Promise} A promise resolving to whether persistence succeeded. + * @param {object} [options] - Configuration. + * @param {boolean} [options.immediate=false] - If true, bypasses debounce and persists immediately. + * @returns {Promise|null} The persistence promise or null if debounced. */ saveLocalData(reason = "unspecified", options = {}) { const { immediate = false } = options;