mirror of
https://github.com/PR0M3TH3AN/bitvid.git
synced 2026-11-06 04:42:57 +00:00
docs: NIP-71 on-boarding + cross-ecosystem dedup dev plan
Adds docs/nip71-onboarding-plan.md and pre-launch TODO 17c for the agreed two-way NIP-71 interop: - Phase 1 (first): cross-ecosystem dedup — prefer the bitvid (30078) version, show exactly one in every grid; identity = author + import-link/hash/infohash. - Phase 2: creator-initiated self-import (model A) — re-host to the creator's storage (CDN) + WebTorrent, publish a signed 30078 with an imported-from provenance tag. No forging others' notes; no instance auto-rehost. Planning only; no code changes. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
1 parent
7236496a16
commit
0112903a6d
2 files changed
+144
No files matched your search
@@ -0,0 +1,126 @@
|
||||
# NIP-71 On-Boarding + Cross-Ecosystem Dedup — Dev Plan
|
||||
|
||||
Status: **planned** (not started). Owner: TBD. Branch: `unstable`.
|
||||
Companion to `docs/nip71-migration-plan.md` (off-boarding / mirror / ingest, all shipped).
|
||||
|
||||
## Goal
|
||||
|
||||
Make NIP-71 interop a **two-way street**:
|
||||
|
||||
1. **Off-boarding (DONE):** a bitvid video (kind 30078) is mirrored *out* to NIP-71
|
||||
(addressable 34235/36) so other apps can discover it.
|
||||
2. **On-boarding (THIS PLAN):** a creator's NIP-71 video published via another app
|
||||
can be pulled *into* bitvid so it gains bitvid's advantages — CDN-hosted URL,
|
||||
a WebTorrent source, and (optionally) bitvid extras.
|
||||
|
||||
And the invariant that makes both safe to coexist:
|
||||
|
||||
3. **One video, never two.** When the same video exists in both ecosystems, always
|
||||
prefer the **bitvid** (kind-30078) version, show **exactly one**, in **every**
|
||||
grid/list/scenario. Never both.
|
||||
|
||||
## The hard constraint (why on-boarding is creator-initiated)
|
||||
|
||||
Off-boarding works because the bitvid creator owns the video and signs BOTH events
|
||||
with their own key. On-boarding is **not symmetric**: a foreign NIP-71 video is
|
||||
signed by the foreign creator's key, and bitvid cannot forge a kind-30078 note
|
||||
authored by someone else (no private key). Therefore:
|
||||
|
||||
- **On-boarding = creator-initiated self-import only.** A creator who published
|
||||
NIP-71 videos elsewhere logs into bitvid **with the same npub**, and bitvid
|
||||
offers to import them: re-host the file to **their own** connected storage (CDN),
|
||||
optionally seed WebTorrent, and publish a bitvid kind-30078 version **signed by
|
||||
their key**. No auto-rehosting of other people's files (consent / cost /
|
||||
liability). This mirrors off-boarding exactly: both directions author-owned and
|
||||
consensual.
|
||||
|
||||
## Locked decisions (from design discussion)
|
||||
|
||||
- On-boarding model: **A — creator-initiated self-import.**
|
||||
- Dedup identity: **explicit import-link + content hash** (`fileSha256`/`ox`) +
|
||||
infohash (`i`/magnet btih). NO fuzzy url/title matching (avoids false merges).
|
||||
Note: hash-less foreign videos (e.g. url+image only) won't dedup unless they
|
||||
were on-boarded (which adds an explicit link) — acceptable.
|
||||
- Advantages on import: **CDN url + WebTorrent source** (baseline). bitvid extras
|
||||
(comments, view counts, edit-in-place, private toggle) come free once it's a
|
||||
real 30078 note; no extra work.
|
||||
- Cadence: **one-shot import with a re-run button** (not continuous lockstep).
|
||||
Re-running re-imports anything new.
|
||||
|
||||
## Phasing — dedup FIRST (it's needed regardless of on-boarding)
|
||||
|
||||
### Phase 1 — Cross-ecosystem dedup ("prefer bitvid, show one, everywhere")
|
||||
|
||||
Why first: even today, a creator who independently posts to bitvid AND Nostube can
|
||||
produce two cards. And the moment on-boarding exists, the original + imported
|
||||
versions both exist on relays. Dedup must land before/with on-boarding.
|
||||
|
||||
- **Identity key:** `author + (importLink ‖ fileSha256 ‖ ox ‖ infoHash)`.
|
||||
- `importLink`: pointer an on-boarded 30078 carries to the original NIP-71
|
||||
event/address (added in Phase 2). Most reliable.
|
||||
- Hashes/infohash already extracted by `convertEventToVideo` (30078) and
|
||||
`buildVideoFromNip71Event` (foreign) — confirm both populate `fileSha256` /
|
||||
`originalFileSha256` / `infoHash`.
|
||||
- **Preference:** within an identity group, keep the bitvid version
|
||||
(`source !== "nip71-ingest"` / kind 30078); else keep the NIP-71 one.
|
||||
- **Placement (one chokepoint → all grids):** the feed AND channel profile already
|
||||
call `app.dedupeVideosByRoot(...)`. Add a pure `collapseCrossEcosystem(videos)`
|
||||
pass and either chain it after root-dedup inside that shared helper, or wrap
|
||||
both into one `dedupeVideos(videos)` used everywhere (feed engine
|
||||
`createDedupeByRootStage`, `buildRenderableChannelVideos`, My Videos, search,
|
||||
playlists, watch history). Audit every list builder for the shared call.
|
||||
- **Bitvid's own outbound mirrors** are already skipped at ingest
|
||||
(`client=bitvid` + `a` back-pointer) — keep that; this pass handles the rest.
|
||||
- **Tests (cheat-resistant, pure):** same-hash bitvid+foreign → only bitvid kept;
|
||||
foreign-only (no bitvid) → foreign kept; distinct videos not merged; hash-less
|
||||
pair NOT merged (documented limitation); import-link match → original hidden.
|
||||
Mutation-verify the prefer-bitvid selection and the identity grouping.
|
||||
|
||||
### Phase 2 — Creator-initiated on-board import
|
||||
|
||||
- **Discovery:** on login, find the logged-in creator's own NIP-71 videos (reuse
|
||||
the ingest subscription/adapter, scoped to `authors:[viewerPubkey]`) that have
|
||||
**no** corresponding bitvid 30078 yet (via the Phase-1 identity key).
|
||||
- **UI:** a "Import your videos from other apps" surface (My Videos pane or a
|
||||
one-time prompt): list importable items, let the creator pick, show what import
|
||||
does. Requires their **storage connected** (re-host target) and signer.
|
||||
- **Import action (per video), reusing existing plumbing:**
|
||||
1. Fetch the source file from the foreign `url` (or resolve via magnet).
|
||||
2. Re-host to the creator's connected storage (reuse `storageService` /
|
||||
`r2Service` upload path used by normal uploads) → bitvid CDN url.
|
||||
3. Optionally create a `.torrent` + WebTorrent magnet (reuse upload's torrent
|
||||
path) so the video gets a P2P source.
|
||||
4. Build + publish a kind-30078 note **signed by the creator** (reuse
|
||||
`videoPublisher` / `buildVideoPostEvent`), carrying: title, description,
|
||||
hashtags, dims/duration (from the foreign imeta via the adapter), the new
|
||||
CDN url + magnet, AND an explicit `["imported-from", "<orig event id / a-addr>"]`
|
||||
provenance tag for dedup + attribution.
|
||||
5. (Optional, symmetric) since it's now a normal bitvid video, the existing
|
||||
outbound mirror/auto-share can keep the NIP-71 side in lockstep going forward.
|
||||
- **Result:** the bitvid 30078 is now canonical; Phase-1 dedup hides the original
|
||||
foreign event everywhere; the creator gets CDN + WebTorrent + bitvid features.
|
||||
- **Guards:**
|
||||
- Only the **same-pubkey** creator can import their own videos (signer == author).
|
||||
- Never import NSFW outward against `ALLOW_NSFW_CONTENT` (mirror the existing
|
||||
gate); carry `content-warning → isNsfw`.
|
||||
- Re-host uses the creator's storage only; surface cost/ownership in the copy.
|
||||
- One-shot; a "re-scan / import new" button re-runs discovery.
|
||||
|
||||
## Non-goals / future
|
||||
|
||||
- No instance-level auto-rehosting of arbitrary creators' files (model B) — consent/
|
||||
cost/liability. Revisit only with explicit per-instance policy + creator opt-in.
|
||||
- No fuzzy (url/title) dedup — revisit only if hash-less dual-posts become a real
|
||||
problem.
|
||||
- Continuous lockstep on the inbound side (re-import on the creator editing the
|
||||
original elsewhere) — deferred; one-shot + re-run is enough for launch.
|
||||
|
||||
## Reuse map (existing code to lean on)
|
||||
|
||||
- `js/nostr/nip71IngestAdapter.js` — foreign event → video (hashes, dims, imeta).
|
||||
- `js/nostr/nip71Mirror.js` — outbound mirror (keeps NIP-71 side in lockstep after import).
|
||||
- `js/services/nip71IngestService.js` — author-scoped subscription pattern for discovery.
|
||||
- `js/channelProfileVideos.js`, feed `createDedupeByRootStage`, `app.dedupeVideosByRoot`
|
||||
— the dedup chokepoints to extend.
|
||||
- `storageService` / `r2Service` / upload + torrent paths — re-host + seed.
|
||||
- `videoPublisher` / `buildVideoPostEvent` — publish the signed 30078.
|
||||
@@ -316,6 +316,24 @@ notes below. Summary:
|
||||
nip71.publishedAt so the stage short-circuits.
|
||||
- Verified live: Goblinbox (Nostube) content renders; liveness check works.
|
||||
|
||||
### 17c. NIP-71 on-boarding + cross-ecosystem dedup (IMPORTANT — full plan in docs/nip71-onboarding-plan.md)
|
||||
Make NIP-71 interop two-way and guarantee a video is never shown twice.
|
||||
**See `docs/nip71-onboarding-plan.md` for the full dev plan.** Locked decisions:
|
||||
creator-initiated self-import (model A — can't forge others' notes), dedup on
|
||||
explicit import-link + content hash/infohash (no fuzzy url/title), import grants
|
||||
CDN url + WebTorrent (bitvid extras come free), one-shot import + re-run button.
|
||||
- [ ] **Phase 1 (do first): cross-ecosystem dedup** — `collapseCrossEcosystem`
|
||||
keyed on `author + (importLink ‖ fileSha256 ‖ ox ‖ infoHash)`, prefer the
|
||||
bitvid (kind-30078) version, applied at the shared dedupe chokepoint so
|
||||
EVERY grid (feed, channel profile, My Videos, search, playlists, watch
|
||||
history) collapses dupes. Pure + mutation-tested. Needed regardless of
|
||||
on-boarding (independent dual-posts already double up).
|
||||
- [ ] **Phase 2: creator-initiated import** — discover the logged-in creator's own
|
||||
NIP-71 videos lacking a bitvid version; re-host the file to THEIR storage
|
||||
(CDN) + optional WebTorrent seed; publish a kind-30078 signed by them with an
|
||||
`["imported-from", <orig>]` provenance tag. Same-pubkey-only guard;
|
||||
ALLOW_NSFW gate; reuses ingest adapter + storage/upload + videoPublisher.
|
||||
|
||||
### 17b. Channel Profile wall shows only bitvid videos, not NIP-71 (LAUNCH-BLOCKER)
|
||||
- [x] **Fixed** (`js/channelProfileVideos.js`): the channel grid now fetches the
|
||||
author's NIP-71 videos (kinds 21/22/34235/34236) alongside kind-30078 via
|
||||
|
||||
Reference in new issue
Block a user