bitvid/docs/lessons/cloudflare-s3-upload-gotchas.md

Cloudflare/S3 Upload-Path Gotchas (audited 2026-06-16)

Extracted verbatim from AGENTS.md §4a, which retains a short summary. Dated incident writeup from the 2026-06-16 upload-path audit.

• Two near-identical upload services exist — js/services/r2Service.js (Cloudflare R2) and js/services/s3UploadService.js (generic S3). They share buildR2Key, resolveUploadIdentifier, and the magnet build but are copies. Any fix to one must be mirrored in the other or they silently drift.
• Storage-key collisions = data loss — FIXED 2026-06-16. buildR2Key (js/r2.js) builds u/<npub>/<namespace>/<slug>.<ext> where namespace used to fall back to the literal "uploads" and slug to "video" (e.g. a non-ASCII filename, or no info-hash) — so a URL-first upload with no info-hash and a duplicate filename overwrote the previous object while the old note still pointed at that URL. WebTorrent uploads were always safe (info-hash namespaces the key = content-addressed). Fix: computeStorageContentHash(file) (js/r2.js) derives a content-based namespace when no info-hash is available (full SHA-256 for ≤512 MB, sampled metadata+edge fingerprint above that to avoid buffering huge files), wired into both r2Service and s3UploadService. Test: tests/storage-key-collision.test.mjs. If you add a third upload path, reuse computeStorageContentHash.
• Whole-file hash before upload. When the modal doesn't pre-seed an info-hash, resolveUploadIdentifier runs calculateTorrentInfoHash(file), which reads the entire file before the upload begins and before any progress is emitted — on multi-GB files this looks like a hang. Emit a "preparing/hashing" status or parallelize.
• Publish can report success unconfirmed. publishEventToRelay (js/nostrPublish.js) has an optimistic-success fallthrough when the relay handle exposes no on()/then() — a note may show "Published" with no relay ACK. Don't tighten blindly (legacy seen-only relays rely on it), but log it.
• Silent partial failures. Thumbnail and .torrent upload failures are caught and warn-only; the video note still publishes with no user signal. Empty (0-byte) files hit CompleteMultipartUpload with Parts:[] and surface a cryptic error — guard file.size === 0.
• Object cleanup (deletes/edits). R2Service.deleteVideoStorage() removes the backing objects so "deleted" videos aren't left publicly downloadable. Contract to preserve: it's best-effort and never throws (must not block note deletion), it derives keys via collectVideoStorageKeys() and only ever touches URLs under the owner's bucket base (never external/3rd-party URLs), and it skips cleanly when storage is locked/credential-less. Wired into delete (app.handleDeleteModalConfirm) and edit-with-replaced-URL (editModalController.handleSubmit). Edit cleanup intentionally does not delete thumbnails (the new note may still reference them) and only fires when both old and new URLs are present and different.