docs(nip71): interop migration plan + refresh stale vendored NIP-71 spec

Adds docs/nip71-migration-plan.md — the agreed spec to build from: - Dual-event, opt-in (off by default): keep canonical kind 30078; add addressable 34235/34236 mirror (d=videoRootId, edits in lockstep). - Research correction: 34235/36 are NOT deprecated — current NIP-71 designates the addressable pair for content that needs post-publication updates (exactly bitvid's edit-in-place model). - WebTorrent rides STANDARD NIP-94 imeta fields (magnet, i) rather than a hack; private videos never mirrored; HTTPS url required to mirror. - Phases: 0 retarget builders → 1 outbound publish/edit/delete → 1.5 NIP-89 "Open in bitvid" handler + NIP-51 kind 30005 portable playlists → 2 inbound ingest. Also refreshes docs/nips/71.md (was stale: only 21/22; now the current 233-line spec with 34235/36 addressable kinds, waveform, origin, multi-audio). Updates TODO #17 to point at the plan. Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-27 07:52:24 +00:00 · 2026-06-21 22:40:57 -04:00
1 parent 2a979c2eca
commit bd056b2e3b
@@ -0,0 +1,154 @@
+# NIP-71 Interop Plan for bitvid
+
+Status: **planning / spec** (no code yet). Last updated 2026-06-22.
+
+Goal: let a publisher **opt in** to making a bitvid video fully NIP-71 compliant so
+it (a) shows up on other Nostr video clients (Nostube, Amethyst, Flare, zap.stream
+recordings, …) and (b) still carries bitvid's extra data (WebTorrent, flags) for a
+richer experience inside bitvid. Eventually bitvid also **ingests** other clients'
+NIP-71 videos. Decisions below were made with the maintainer; see "Decisions".
+
+---
+
+## Decisions (locked)
+
+1. **Kind: addressable `34235` (normal) / `34236` (short).**
+2. **Opt-in, off by default** — per-video toggle; private videos never mirrored.
+3. **Scope: outbound first, inbound later** (Phase 1 vs Phase 2).
+
+### Why 34235/34236 (research-validated)
+
+The earlier assumption that 34235/36 were "deprecated legacy" was **wrong**. The
+current NIP-71 (master) defines **all four** kinds and explicitly designates the
+addressable pair for editable content:
+
+> "For content that may need updates after publication (such as correcting
+> metadata, descriptions, or handling URL migrations), addressable versions are
+> available: Kind `34235` … Kind `34236` … include a `d` tag as a unique
+> identifier and can be updated while maintaining the same addressable reference."
+
+That is **exactly** bitvid's model (replaceable kind 30078, edit-in-place via
+`d` = `videoRootId`). So 34235/36 is not a compromise — it's the spec-recommended
+kind for what bitvid does, and it's what Nostube/Goblinbox consume today.
+
+> Hedge: keep the builder **kind-parameterized** so we can also emit / switch to
+> `21`/`22` (regular) later without a rewrite if the ecosystem leans that way.
+
+---
+
+## Architecture: dual-event, additive
+
+- **Canonical stays kind 30078** (NIP-78 app-data), unchanged. Source of truth for
+  WebTorrent, private videos, moderation, view counts — none of which NIP-71 models.
+- **Mirror = kind 34235/34236**, addressable with **`d` = `videoRootId`** (same
+  d-tag as the 30078), so the mirror edits/deletes in **lockstep** with the canonical.
+- Correlation is automatic via `(pubkey, videoRootId)`. We also set:
+  - `["origin", "bitvid", "<videoRootId>", "https://bitvid.network/?v=<nevent|naddr>"]`
+    so other clients can attribute + deep-link back ("watch on bitvid").
+  - Optionally `["a", "30078:<pubkey>:<videoRootId>"]` back-pointer (harmless elsewhere).
+
+---
+
+## Field mapping: 30078 JSON → 34235/36 tags
+
+Sourced against NIP-71 + NIP-92 (`imeta`) + NIP-94 (file metadata fields).
+
+| bitvid (30078 content) | NIP-71 / imeta | notes |
+|---|---|---|
+| — | `["d", videoRootId]` | required for addressable; lockstep with 30078 |
+| `title` | `["title", …]` | required |
+| `description` | `.content` and `["alt", …]` | |
+| first publish time | `["published_at", …]` | **stable across edits** (no reordering elsewhere) |
+| `url` (HTTPS) | `imeta url <…>` | primary playable source for foreign clients |
+| derived MIME | `imeta m <…>` | not stored today → derive from extension, default `video/mp4` |
+| `thumbnail` | `imeta image <…>` | |
+| `fileSha256` | `imeta x <…>` | |
+| `originalFileSha256` | `imeta ox <…>` | |
+| `magnet` | `imeta magnet <magnet:?xt=…&ws=…&xs=…>` | **standard NIP-94 field** — torrent-aware clients can use it |
+| `infoHash` | `imeta i <infohash>` | **standard NIP-94 field** |
+| `ws` / `xs` | encoded into the `magnet` URI (`&ws=`/`&xs=`) | also kept on 30078 canonical |
+| `isNsfw` | `["content-warning", "<reason>"]` | standard |
+| hashtags (if/when added) | `["t", …]` (repeated) | bitvid has no hashtags field yet — future |
+| `isForKids`, `mode`, etc. | bitvid-namespaced extra tags | ignored by other clients |
+| — | `["client", "bitvid"]` | |
+
+**Two hard rules that fall out of this:**
+
+1. **Private videos are never mirrored.** NIP-71 has no encryption; mirroring would
+   leak them. The toggle is disabled (with explanation) for private videos.
+2. **A hosted HTTPS `url` is required to mirror.** Foreign clients can't play a
+   magnet-only video; bitvid's magnet still rides along (`magnet`/`i`) for
+   torrent-aware clients, but `url` must be present or the toggle is disabled.
+
+**Timestamp hygiene:** `created_at` = actual signing time; `published_at` = display
+time. (Avoids the future-`created_at` weirdness seen in some published events.)
+
+---
+
+## Phases
+
+### Phase 0 — Retarget builders (no UX; flag stays off)
+- Point existing `buildNip71VideoEvent` at **34235/36 + `d`=`videoRootId`** (it
+  currently targets 21/22). Implement the field mapping incl. `magnet`/`i`/`ox`.
+- Normal-vs-short selection: default 34235; choose 34236 when known-portrait
+  (dimensions when available; otherwise a manual "short" toggle).
+- Cheat-resistant tests: bitvid video → spec-valid 34235 event → round-trips back
+  via `convertEventToVideo`; magnet/infohash present; private refused; url-less refused.
+
+### Phase 1 — Outbound publish/edit/delete (the headline feature)
+- Edit form toggle: **"Also publish to other Nostr video apps (NIP-71)"** — off by
+  default, gated to public + HTTPS-url videos.
+- Publish: 30078 (as today) **+** the 34235/36 mirror to the write-relay set
+  (reuse `getDeletePublishRelays` write set + the publish-outcome toast).
+- Lifecycle parity: **edit** updates both (same d-tag); **delete** NIP-09s both
+  addresses; **toggle-off** NIP-09s the mirror (removes it from other clients).
+- Mutation-verified lifecycle tests, then flip `FEATURE_PUBLISH_NIP71` on.
+
+### Phase 1.5 — Discoverability glue (cheap, high interop value)
+- **NIP-89 handler registration** (kind `31990`): publish once that bitvid handles
+  video kinds `21/22/34235/34236` with a `web` URL template. Result: other clients
+  (Amethyst, etc.) can show **"Open in bitvid"** when they encounter a video event.
+- **NIP-51 video curation sets** (kind `30005`): publish/read bitvid playlists as
+  portable video sets so playlists interop across clients. (Sets reference videos
+  by `e`/`a` tags.)
+
+### Phase 2 — Inbound ingest (separate, bigger)
+- Feed source querying `34235/36` (and `21/22`) from relays; map via
+  `convertEventToVideo`.
+- Dedup vs bitvid-native (by `a`-address / `x` hash / `url`), moderation + trust
+  gating (reuse blacklist / NIP-51 mute), and a **distinct discovery surface** so
+  foreign videos read as "from the wider Nostrverse," not bitvid-native.
+- Honor `origin` for attribution; respect the no-URL-allowlist rule but apply
+  existing moderation.
+- (Cross-client comments/zaps via NIP-22 `kind:1111` addressed to the 34235 event
+  is a further follow-on — bitvid comments currently target 30078.)
+
+---
+
+## Enrichment backlog (nice-to-have, not v1)
+`dim`, `duration`, `bitrate`, `waveform`, multiple `imeta` variants + multi-audio
+(`ov`/language) tracks, `text-track` (WebVTT captions/subtitles/chapters),
+`segment` (chapters) — all need more captured at upload time. v1 ships the minimal
+spec-valid imeta (`url`/`m`/`image` + `magnet`/`i`) and grows from there.
+
+---
+
+## Risks / watch-items
+- **Editing addressable events across foreign clients**: some cache aggressively;
+  edits may lag elsewhere. Acceptable; document it.
+- **Magnet-only catalogs**: those videos simply won't appear on non-torrent clients
+  (by design — they still work in bitvid).
+- **Kind drift**: if the ecosystem consolidates on 21/22, add/dual-emit later
+  (builder stays kind-parameterized).
+- **Ingest safety (Phase 2)**: arbitrary foreign media + metadata → keep moderation,
+  no autoplay of untrusted sources, sanitize before render.
+
+---
+
+## Sources
+- NIP-71 Video Events (current): https://github.com/nostr-protocol/nips/blob/master/71.md (vendored at `docs/nips/71.md`)
+- NIP-92 inline metadata (`imeta`): https://github.com/nostr-protocol/nips/blob/master/92.md
+- NIP-94 file metadata (`magnet`, `i`, `x`, `ox`, `fallback`, …): https://github.com/nostr-protocol/nips/blob/master/94.md
+- NIP-89 recommended application handlers (`31989`/`31990`): https://github.com/nostr-protocol/nips/blob/master/89.md
+- NIP-51 lists/sets incl. kind `30005` video curation: https://github.com/nostr-protocol/nips/blob/master/51.md
+- Existing bitvid code: `js/nostr/nip71.js` (builders/parser), `js/nostrEventSchemas.js` (`NIP71_VIDEO`/`NIP71_SHORT_VIDEO`), `js/constants.js` (`FEATURE_PUBLISH_NIP71`).
@@ -20,16 +20,34 @@ Nothing except cavaliership and common sense prevents a _short_ video from being

 The format uses a _regular event_ kind `21` for _normal_ videos and `22` for _short_ videos.

+## Addressable Video Events
+
+For content that may need updates after publication (such as correcting metadata, descriptions, or handling URL migrations), addressable versions are available:
+
+- Kind `34235` for _addressable normal videos_
+- Kind `34236` for _addressable short videos_
+
+These addressable events follow the same format as their regular counterparts but include a `d` tag as a unique identifier and can be updated while maintaining the same addressable reference. This is particularly useful for:
+
+- Metadata corrections (descriptions, titles, tags) without republishing
+- Preservation of imported content IDs from legacy platforms
+- URL migration when hosting changes
+- Platform migration tracking
+
 The `.content` of these events is a summary or description on the video content.

 The primary source of video information is the `imeta` tags which is defined in [NIP-92](92.md)

-Each `imeta` tag can be used to specify a variant of the video by the `dim` & `m` properties.
+Each `imeta` tag can be used to specify a variant of the video by the `dim` & `m` properties, as well as multiple audio tracks (languages).

-This NIP defines the following additional `imeta` properties aside form those listen in [NIP-92](92.md) & [NIP-94](94.md):
+Clients should check if audio tracks are available and prefer them over in-video audio 
+to allow smooth resolution switching without interrupting audio. The `ov` (original version) flags the original language track.
+
+This NIP defines the following additional `imeta` properties aside from those listed in [NIP-92](92.md) & [NIP-94](94.md):

 * `duration` (recommended) the duration of the video/audio in seconds (floating point number)
 * `bitrate` (recommended) the average bitrate of the video/audio in bits/sec
+* `waveform` (optional, audio only) amplitude values over time, space separated full integers, less than 100 values should be enough to render a nice visual

 Example:
 ```json
@@ -72,6 +90,18 @@ Example:
    "service nip96",
    "duration 29.21"
  ],
+  ["imeta",
+    "url https://myaudio.com/audio/en/12345.mp3",
+    "x b2e0a7a82ac9f3f3a71f1d9a78c381d5be9d1cf19dce258765c17c8a76287c93",
+    "m audio/mp3",
+    "waveform 0 7 35 8 100 100 49 8 4 16 8 10 7 2 20 10 100 100 100 100 100 100 15 100 100 100 25 60 5 4 3 1 0 100 100 15 100 29 88 0 33 11 39 100 100 19 4 100 42 35 5 0 1 5 0 0 11 38 100 94 17 11 44 58 5 100 100 100 55 14 72 100 100 57 6 1 14 2 16 100 100 40 16 100 100 6 32 14 13 41 36 16 14 6 3 0 1 2 1 6 0",
+    "l en ISO-639-1 ov",
+    "fallback https://myotherserver.com/audio/en/12345.mp3",
+    "fallback https://andanotherserver.com/audio/en/12345.mp3",
+    "service nip96",
+    "bitrate  320000",
+    "duration 29.24"
+  ],
 ]
 ```

@@ -81,6 +111,9 @@ The `image` tag contains a preview image (at the same resolution). Multiple `ima

 Additionally `service nip96` may be included to allow clients to search the authors NIP-96 server list to find the file using the hash.

+### Required tags for addressable events:
+* `d` - Unique identifier for this video (user-chosen string, required for kinds 34235, 34236)
+
 ### Other tags:
 * `title` (required) title of the video
 * `published_at`, for the timestamp in unix seconds (stringified) of the first time the video was published
@@ -92,7 +125,10 @@ Additionally `service nip96` may be included to allow clients to search the auth
 * `p` (optional, repeated) 32-bytes hex pubkey of a participant in the video, optional recommended relay URL
 * `r` (optional, repeated) references / links to web pages

-```jsonc
+### Optional tags for imported content:
+* `origin` - Track original platform and ID: `["origin", "<platform>", "<external-id>", "<original-url>", "<optional-metadata>"]`
+
+```yaml
 {
  "id": "<32-bytes lowercase hex-encoded SHA-256 of the the serialized event data>",
  "pubkey": "<32-bytes lowercase hex-encoded public key of the event creator>",
@@ -135,3 +171,63 @@ Additionally `service nip96` may be included to allow clients to search the auth
  ]
 }
 ```
+
+## Addressable Event Example
+
+```yaml
+{
+  "id": <32-bytes lowercase hex-encoded SHA-256 of the the serialized event data>,
+  "pubkey": <32-bytes lowercase hex-encoded public key of the event creator>,
+  "created_at": <Unix timestamp in seconds>,
+  "kind": 34235 | 34236,
+  "content": "<summary / description of video>",
+  "tags": [
+    ["d", "<unique-identifier>"],
+    ["title", "<title of video>"],
+    ["published_at", "<unix timestamp>"],
+    ["alt", "<description for accessibility>"],
+
+    // video data
+    ["imeta",
+      "url https://example.com/media.mp4",
+      "m video/mp4",
+      "dim 480x480",
+      "blurhash eVF$^OI:${M{%LRjWBoLoLaeR*",
+      "image https://example.com/thumb.jpg",
+      "x 3093509d1e0bc604ff60cb9286f4cd7c781553bc8991937befaacfdc28ec5cdc"
+    ],
+    // audio data
+    ["imeta",
+      "url https://example.com/audio.mp3",
+      "x b2e0a7a82ac9f3f3a71f1d9a78c381d5be9d1cf19dce258765c17c8a76287c93",
+      "m audio/mp3",
+      "l en ISO-639-1 ov"
+    ],
+
+    ["duration", <duration in seconds>],
+    ["content-warning", "<reason>"],
+
+    // origin tracking for imported content
+    ["origin", "<platform>", "<external-id>", "<original-url>", "<optional-metadata>"],
+
+    // participants
+    ["p", "<32-bytes hex of a pubkey>", "<optional recommended relay URL>"],
+
+    // hashtags
+    ["t", "<tag>"],
+    ["t", "<tag>"],
+
+    // reference links
+    ["r", "<url>"]
+  ]
+}
+```
+
+## Referencing Addressable Events
+
+To reference an addressable video:
+
+```
+["a", "34235:<pubkey>:<d-tag-value>", "<relay-url>"]  // for normal videos
+["a", "34236:<pubkey>:<d-tag-value>", "<relay-url>"]  // for short videos
+```
@@ -248,12 +248,18 @@ Threat model / cautions (write these into the feature + docs):
 - [ ] Research: confirm NIP-53 kinds + tag shapes, how zap.stream vs shosho.live
      populate `streaming`/`recording`, and HLS playback support in the player.

-### 17. Short-form / external NIP-71 video note support
- [ ] Likely **NIP-71 Video Events** — the format Amethyst & co. use for "short
-      form" vertical video: kind **22** (short video) and kind **21** (normal
-      video). (Confirm current kinds; NIP-71 churned from the older 34235/34236.)
- [ ] bitvid currently publishes its OWN kind 30078 + attaches NIP-71 metadata.
-      This item is about CONSUMING external NIP-71 notes (render them in the feed /
-      a short-form view) so bitvid can show video posted by other Nostr apps.
- [ ] Research: confirm whether short-form is purely NIP-71 kind 22, how it maps
-      onto bitvid's video model (url/magnet/thumbnail), and the vertical/feed UX.
+### 17. NIP-71 interop (full plan in `docs/nip71-migration-plan.md`)
+Research done; decisions locked. **See the plan doc** — it supersedes the rough
+notes below. Summary:
+- Opt-in, off by default. Dual-event: keep canonical kind 30078; add an addressable
+  **34235/34236** mirror (`d`=`videoRootId`, edits in lockstep). 34235/36 are NOT
+  deprecated — current NIP-71 designates them for editable content.
+- WebTorrent rides standard NIP-94 `imeta` fields (`magnet`, `i`); private videos
+  never mirrored; HTTPS `url` required to mirror.
+- [ ] Phase 0: retarget `buildNip71VideoEvent` to 34235/36 + field mapping + tests.
+- [ ] Phase 1: opt-in toggle, publish/edit/delete lifecycle parity, flip
+      `FEATURE_PUBLISH_NIP71` on.
+- [ ] Phase 1.5: NIP-89 handler reg (kind 31990 → "Open in bitvid" elsewhere);
+      NIP-51 kind 30005 portable playlists.
+- [ ] Phase 2: inbound ingest of external NIP-71 videos (dedup, moderation, trust,
+      `origin` attribution, distinct discovery surface).