thePR0M3TH3AN/bitvid
1
0 Fork 0
mirror of https://github.com/PR0M3TH3AN/bitvid.git synced 2026-10-27 07:52:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(ai): doc js/nostr/client.js (agent)

Browse Source 
Co-authored-by: PR0M3TH3AN <[email protected]>
This commit is contained in:
google-labs-jules[bot]andPR0M3TH3AN committed 2026-01-27 15:23:09 +00:00
1 parent 36ba0db0b5
commit 6165473219
2 files changed
+108 -163
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/nostr-client-overview.md
+71 -163
View File
@@ -1,184 +1,92 @@
# NostrClient Overview
`js/nostr/client.js` exports the `NostrClient` class, which serves as the central controller for all Nostr protocol interactions in the application. It acts as a bridge between the UI components, the local state (React/Redux/Listeners), and the decentralized Nostr network.
`NostrClient` (`js/nostr/client.js`) is the central controller for all Nostr network interactions, state management, and persistence in the application. It acts as the bridge between the raw Nostr protocol (relays, events) and the application's UI domain model (Videos, Profiles).
## Responsibilities
## Core Responsibilities
1. **Connection Management**:
* Maintains a persistent connection pool (`SimplePool` from `nostr-tools`) to multiple relays.
* Handles relay discovery, connection retries, and error reporting.
* Supports "Read" vs. "Write" relay splitting.
1. **Connection Management**: Maintains a pool of WebSocket connections to relays via `nostr-tools/SimplePool`.
2. **Event Processing**: Subscribes to event streams, buffers incoming bursts, validates schemas, and converts raw events into domain objects.
3. **State Management**:
* **`allEvents`**: A complete history of every video version fetched.
* **`activeMap`**: A materialized view of the *latest* valid version of each video series (deduplicated by Root ID).
* **`tombstones`**: A set of deletion timestamps to prevent "zombie" events (deleted items reappearing from old caches) from being displayed.
4. **Persistence**: Implements a "Stale-While-Revalidate" strategy using IndexedDB (with LocalStorage fallback) to restore the UI instantly on load.
5. **Signer Orchestration**: Manages NIP-07 (Browser Extension), NIP-46 (Remote/Bunker), and Local (nsec) signers.
2. **Event Publishing (Write Path)**:
* Constructs spec-compliant Nostr events (Kind 1, 30078, etc.).
* Handles **signing**:
* NIP-07 (Browser Extensions like Alby/nos2x).
* NIP-46 (Remote Signers / Bunker).
* Local Private Keys (nsec).
* Broadcasts events to write relays and verifies acceptance.
* Implements **NIP-94** (File Metadata) and **NIP-71** (Video Categorization) mirroring.
## State Management Architecture
3. **Data Fetching & Subscription (Read Path)**:
* **Live Feeds**: Uses persistent subscriptions (`sub`) to stream new events.
* **Buffering**: Implements a debounced buffer strategy (`subscribeVideos`) to handle relay floods without freezing the UI.
* **Caching**: Stores events in `allEvents` (memory) and persists to IndexedDB/LocalStorage for instant load ("Stale-While-Revalidate").
The client maintains three primary maps to handle the "Eventual Consistency" nature of Nostr:
4. **State Management**:
* **`allEvents`**: Map of all raw video events fetched, keyed by Event ID.
* **`activeMap`**: A "materialized view" of the latest video state. It deduplicates multiple versions of the same video (edits) based on `videoRootId` or `d` tag, ensuring the UI only shows the most recent valid version.
* **`tombstones`**: Tracks deleted events to prevent "zombie" events from reappearing during partial syncs.
* **`allEvents` (`Map<EventID, Video>`)**:
* Stores every valid video event received.
* Used for history hydration (viewing edit logs) and debugging.
* **`activeMap` (`Map<RootID, Video>`)**:
* Stores only the *most recent* version of a video.
* Keyed by `videoRootId` (or `pubkey:dTag` for legacy).
* **Invariant**: `activeMap.get(id).created_at` is always >= any other version of that ID.
* **`tombstones` (`Map<RootID, Timestamp>`)**:
* Stores the timestamp of the latest deletion event for a video.
* **Invariant**: Any event with `created_at <= tombstone` is implicitly treated as deleted, even if the "deleted" flag isn't set on the event itself. This handles out-of-order delivery where a "delete" arrives before the "video" it deletes.
## Key Architectures
## Caching Strategy
### 1. The "Active Map" & Deduplication
Nostr is immutable; "editing" a video means publishing a NEW event with the same identifier (`d` tag).
The client listens to *all* versions but only keeps the latest one in `activeMap`.
The client uses a multi-layered approach to ensure fast boot times:
```javascript
// Simplified Logic
if (incomingEvent.createdAt > currentActiveEvent.createdAt) {
activeMap.set(videoRootId, incomingEvent);
notifyUI();
}
1. **IndexedDB (`events` store)**:
* Primary cache for video events.
* Stores fingerprints of objects to avoid redundant writes.
2. **LocalStorage (`bitvid:eventsCache:v1`)**:
* Fallback if IndexedDB is unavailable.
* Size-limited (typically < 5MB).
3. **Buffering**:
* Incoming events are buffered in memory and flushed to IDB periodically (debounced) to avoid I/O thrashing during heavy relay traffic.
## Event Processing (Buffering & Debouncing)
When `subscribeVideos` is called, the client expects a flood of events (e.g., 500+ items from relay history). Processing these one-by-one would freeze the UI.
### Flow
```mermaid
sequenceDiagram
participant Relay
participant ClientBuffer
participant Processor
participant UI
Relay->>ClientBuffer: Event 1
Relay->>ClientBuffer: Event 2
Relay->>ClientBuffer: ... Event 50
ClientBuffer->>ClientBuffer: Buffer (Debounce 75ms)
loop Flush Timer
ClientBuffer->>Processor: Flush Batch (50 events)
Processor->>Processor: Validate & Convert
Processor->>Processor: Apply Tombstones
Processor->>Processor: Update activeMap (Last-Write-Wins)
Processor->>UI: Callback(onVideo) (Single Render)
Processor->>IndexedDB: Persist Snapshot
end
```
### 2. Buffering Strategy
When the app loads, relays might send 1000+ events in a second. Rendering React components 1000 times is too slow.
`subscribeVideos` pushes events to a buffer and flushes them every 75ms.
## History Reconstruction
```javascript
sub.on("event", (evt) => {
buffer.push(evt);
debounce(flushBuffer, 75);
});
```
Videos in Nostr are immutable events. "Editing" means publishing a new event with the same identifier. `NostrClient` reconstructs the history chain using:
### 3. Tombstoning (Deletes)
Since relays are eventually consistent, a "delete" event (Kind 5) might arrive *before* the event it deletes, or *after*.
The client tracks `tombstones` (map of ID -> deletedAt timestamp). Any incoming event older than its tombstone is immediately discarded.
1. **`videoRootId` (V3)**: The canonical identifier for a video series.
2. **`d` Tag (NIP-33)**: The addressable identifier.
3. **Legacy IDs**: Fallback to Event ID for V1/V2 posts.
## Main Entry Points
* `init()`: Bootstraps the client, restores local cache, connects to relays.
* `subscribeVideos(callback)`: Main feed subscription.
* `publishVideo(payload)`: Complex flow to upload -> sign -> publish -> mirror.
* `connectRemoteSigner(uri)`: Handshake protocol for NIP-46.
`hydrateVideoHistory` fetches all events matching these identifiers, sorts them by `created_at` descending, and returns the full edit log.
## Why it works this way
### 1. Immutability & "Active Map"
Nostr events are immutable signed messages. To support "edits", we must allow multiple events to represent the same "entity" (Video).
- **The Trade-off**: We accept higher memory usage (storing multiple versions in `allEvents`) to gain strict cryptographic verifiability.
- **The Solution**: `activeMap` acts as a "Latest-State Projection" (Materialized View) that the UI consumes. This decouples the network reality (append-only log) from the user expectation (mutable state).
### 2. Buffering vs. Streaming
Directly piping relay events to React State (`setState`) triggers a Re-Render Cycle for every message.
- **The Bottleneck**: Initial loads often fetch 500-2000 events in <2 seconds.
- **The Solution**: A 75ms debounce buffer collects bursts and processes them in one microtask. This turns 1000 re-renders into ~10-15, keeping the UI thread responsive (60fps).
### 3. Dual-Layer Caching (IDB + Memory)
Relays are slow and sometimes fail.
- **Offline-First**: We restore from IndexedDB immediately on boot (`init`).
- **Fingerprinting**: We hash events before writing to IDB. If `JSON.stringify(video)` hasn't changed, we skip the write syscall. This reduces I/O freezing on mobile devices.
## Advanced Topics
### Version History Reconstruction (`hydrateVideoHistory`)
Since edits create new events, a video might have 10+ versions scattered across relays.
`hydrateVideoHistory` stitches them together by:
1. **Scanning Local Cache**: Finds all events matching the `videoRootId`.
2. **Relay Fallback**: If local history is sparse, queries relays for `kind: 30078` + `#d` tag.
3. **Root Recovery**: Explicitly fetches the original "Root" event if missing, to establish the timeline start.
### Optimized List Fetching (`fetchListIncrementally`)
Used for things like Mute Lists (Kind 10000) or Pin Lists.
- **Last-Seen Check**: Tracks the `created_at` of the last item seen from *each* relay.
- **Delta Sync**: Asks for `since: lastSeen + 1`.
- **Fallback**: If the delta sync fails, performs a full fetch to ensure consistency.
- **Concurrency**: Batches requests to 4 relays at a time to prevent browser network saturation.
### Video Deletion Flow (`deleteAllVersions`)
Nostr's immutability makes "deleting" a video series complex, as multiple edit versions might exist across relays.
`deleteAllVersions` handles this via a multi-stage process:
1. **Hydration**: Calls `hydrateVideoHistory` to find *every* event related to the video (scanning by `videoRootId` and `d` tag).
2. **Soft Delete (Revert)**: For each unique version identifier (Root + D-Tag combination), it publishes a new event with `deleted: true` and cleared content. This ensures clients resolving the "latest" version see the deletion immediately.
3. **Hard Delete (NIP-09)**: Publishes a Kind 5 Deletion Request referencing the Event IDs of *all* versions found. Compliant relays will physically remove the data.
4. **Tombstoning**: Updates the local `tombstones` map with the deletion timestamp to preventing caching mechanisms from resurrecting old versions.
```mermaid
sequenceDiagram
participant UI as VideoEditModal
participant Client as NostrClient
participant Relay
UI->>Client: deleteAllVersions(rootId)
activate Client
Note right of Client: 1. Find all versions
Client->>Client: hydrateVideoHistory(video)
Client->>Relay: Query (Kind 30078 + #d)
Relay-->>Client: Events [v1, v2, v3]
Note right of Client: 2. Soft Delete (Revert)
loop For each unique branch
Client->>Relay: Publish Kind 30078 (deleted=true)
end
Note right of Client: 3. Hard Delete (NIP-09)
Client->>Relay: Publish Kind 5 (referencing v1, v2, v3 ids)
Note right of Client: 4. Local Cleanup
Client->>Client: recordTombstone(rootId)
Client->>Client: saveLocalData()
Client-->>UI: Success
deactivate Client
```
* **Eventual Consistency**: The `tombstones` map is essential because relays guarantee neither order nor completeness. A client might receive a "delete" event for a video it hasn't seen yet, or receive an old version of a video after receiving the deletion. Tombstones ensure that once something is known to be deleted, it stays deleted (until a newer re-creation event appears).
* **Performance**: The `eventBuffer` mechanism allows the client to ingest thousands of events without blocking the main thread, as React rendering is expensive. Batching updates reduces render cycles from O(N) to O(1) per batch.
* **Offline-First**: Restoring from IndexedDB immediately allows the app to be interactive before any network connection is established.
## When to change
- **Update `EVENTS_CACHE_VERSION`**: If you change the internal `Video` object structure (e.g., rename fields in `convertEventToVideo`), bump the DB version in `EventsCacheStore` to force a wipe/rebuild on client devices.
- **Modify `subscribeVideos`**: If adding new filters (e.g., "Kids Only Mode"), update the subscription filter here. Ensure the buffer logic remains to protect performance.
- **Refactor Signers**: If NIP-46 changes or a new signer standard (NIP-XX) emerges, modify `signerHelpers.js` and `client.js` methods (`signAndPublishEvent`).
## Sequence Example: Publish Video
Publishing is not atomic; it's a multi-step orchestration to ensure compatibility across different Nostr clients (WebSeed vs. File Metadata vs. Categorization).
```mermaid
sequenceDiagram
participant UI as VideoUploadModal
participant Client as NostrClient
participant Signer as NIP-07/46
participant Relay
UI->>Client: publishVideo(payload)
activate Client
Note right of Client: 1. Construct Main Video Note
Client->>Client: buildVideoPostEvent (Kind 30078)
Client->>Signer: signEvent(Kind 30078)
Signer-->>Client: signedEvent
Client->>Relay: Publish Kind 30078
Note right of Client: 2. NIP-94 Mirror (Optional)
opt has hosted URL
Client->>Client: buildMirrorEvent (Kind 1063)
Client->>Signer: signEvent(Kind 1063)
Signer-->>Client: signedMirror
Client->>Relay: Publish Kind 1063
end
Note right of Client: 3. NIP-71 Metadata (Optional)
opt has tags/categories
Client->>Client: buildNip71Event (Kind 22)
Client->>Signer: signEvent(Kind 22)
Signer-->>Client: signedWrapper
Client->>Relay: Publish Kind 22
end
Client-->>UI: Success (returns Main Event)
deactivate Client
```
* **Storage Backend**: If the cache schema becomes too complex for IndexedDB, or if we move to a native wrapper (React Native/Tauri) with a SQL backend, the `EventsCacheStore` class should be swapped out.
* **Event Loop**: If we switch to a worker-based architecture (off-main-thread processing), the `subscribeVideos` logic would move to a Web Worker, and `NostrClient` would become a message passer.
* **Protocol Upgrades**: If NIP-33 (Parameterized Replaceable Events) is superseded or the video event kind (30078) changes, the filters in `subscribeVideos` and `fetchVideos` must be updated.
js/nostr/client.js
+37
View File
@@ -3559,6 +3559,12 @@ export class NostrClient {
fetchFn,
since,
} = {}) {
// Explanation:
// Optimizes network usage by only fetching "new" events from each relay.
// It uses `SyncMetadataStore` to track the `created_at` of the last seen event per relay.
// - If we have a `lastSeen` timestamp, we request `since: lastSeen + 1`.
// - If that returns nothing (or fails), we might fall back to a full fetch depending on context.
// - Updates the `lastSeen` timestamp upon success.
if (!kind || !pubkey) {
throw new Error("fetchListIncrementally requires kind and pubkey");
}
@@ -6589,6 +6595,12 @@ export class NostrClient {
* @private
*/
async persistLocalData(reason = "unspecified") {
// Strategy:
// 1. Try to persist to IndexedDB (preferred, large capacity).
// 2. If IDB fails or returns false, fall back to LocalStorage (limited capacity).
// The `EventsCacheStore` handles differential updates (only saving changed fingerprints)
// to keep IDB writes fast.
const payload = this.buildCachePayload();
const startedAt = Date.now();
let summary = null;
@@ -6705,6 +6717,12 @@ export class NostrClient {
* @returns {import("nostr-tools").Sub} The subscription object. Call `unsub()` to stop.
*/
subscribeVideos(onVideo, options = {}) {
// Explanation:
// This method handles the primary video feed. It uses a buffering strategy to
// prevent UI thrashing when thousands of events arrive at once (e.g. initial load).
// Incoming events are pushed to `eventBuffer` and processed in batches
// via `flushEventBuffer` which is debounced.
const { since, until, limit } = options;
const latestCachedCreatedAt = this.getLatestCachedCreatedAt();
@@ -6738,6 +6756,7 @@ export class NostrClient {
// BUFFERING STATE
// We collect events here instead of processing them instantly to avoid
// 1000s of React re-renders during the initial relay dump.
// The buffer acts as a pressure valve between the network and the UI.
let eventBuffer = [];
const EVENT_FLUSH_DEBOUNCE_MS = 75;
let flushTimerId = null;
@@ -6748,6 +6767,13 @@ export class NostrClient {
* This batch processing is critical for performance during the initial
* relay dump (thousands of events). It prevents React from re-rendering
* for every single event.
*
* Algorithm:
* 1. Drain the buffer.
* 2. Validate & Convert events to Video objects.
* 3. Apply tombstones (filter out known deleted items).
* 4. Update `activeMap` (Last-Write-Wins based on created_at).
* 5. Persist the new state to cache.
*/
const flushEventBuffer = () => {
if (!eventBuffer.length) {
@@ -7651,6 +7677,17 @@ export class NostrClient {
* @returns {Promise<object[]>} A promise resolving to an array of Video objects (Newest -> Oldest).
*/
async hydrateVideoHistory(video) {
// Explanation:
// This method reconstructs the edit history of a video series.
// Since Nostr events are immutable, "editing" creates a new event.
// We link these events together using:
// 1. `videoRootId` (V3 canonical ID)
// 2. `d` tag (NIP-33 addressability)
// 3. Fallback: Matching ID for legacy V1/V2 posts
//
// The method first checks local cache (`allEvents`), and if data is sparse,
// queries relays for the specific `d` tag to find missing links.
if (!video || typeof video !== "object") {
return [];
}