mirror of
https://github.com/PR0M3TH3AN/bitvid.git
synced 2026-10-27 07:52:24 +00:00
Update docs to describe the extensible model
- Create `docs/storage-tab-design.md` covering StorageService architecture - Update `docs/storage-r2.md` to reflect the new provider pattern - Expand Kind 22242 details in `docs/nostr-event-schemas.md`
This commit is contained in:
google-labs-jules[bot] committed
1 parent
95b86a3a13
commit
d5da4a42d1
3 files changed
+147
-59
No files matched your search
@@ -152,7 +152,7 @@ import { updateWatchHistoryListWithDefaultClient } from "./nostrWatchHistoryFaca
|
||||
| Admin moderation list (`NOTE_TYPES.ADMIN_MODERATION_LIST`) | `30000` | `['d', 'bitvid:admin:editors']`, repeated `['p', <pubkey>]` entries | Empty content |
|
||||
| Admin blacklist (`NOTE_TYPES.ADMIN_BLACKLIST`) | `30000` | `['d', 'bitvid:admin:blacklist']`, repeated `['p', <pubkey>]` entries | Empty content |
|
||||
| Admin whitelist (`NOTE_TYPES.ADMIN_WHITELIST`) | `30000` | `['d', 'bitvid:admin:whitelist']`, repeated `['p', <pubkey>]` entries | Empty content |
|
||||
| Storage challenge (`NOTE_TYPES.STORAGE_CHALLENGE`) | `22242` | `['challenge', <hex>]`, `['purpose', 'bitvid-storage-key']` | Empty content; ephemeral event used for key derivation only. |
|
||||
| Storage challenge (`NOTE_TYPES.STORAGE_CHALLENGE`) | `22242` | `['challenge', <hex>]`, `['purpose', 'bitvid-storage-key']` | Empty content. This event is **ephemeral** and never published to relays. It serves as a challenge for the signer; the deterministic signature produced is used to derive the local encryption Master Key for the Storage Service. |
|
||||
|
||||
Subscription lists therefore match the
|
||||
[NIP-51 follow-set specification](./nips/51.md#sets) by emitting kind `30000`
|
||||
|
||||
+32
-58
@@ -1,15 +1,14 @@
|
||||
# Cloudflare R2 integration (r2Service)
|
||||
|
||||
## Purpose and scope
|
||||
`js/services/r2Service.js` is the client-side coordinator for Cloudflare R2 uploads. It owns:
|
||||
|
||||
- Local persistence of Cloudflare credentials and bucket metadata used by the Upload modal.
|
||||
- Uploading the selected video/HLS asset via multipart S3 uploads and publishing the resulting
|
||||
public URL into the video note payload.
|
||||
`js/services/r2Service.js` is the specialized provider handler for Cloudflare R2 uploads. While the new **Storage Tab** (powered by `StorageService`) manages the secure storage of credentials, `r2Service` is responsible for:
|
||||
|
||||
- Uploading the selected video/HLS asset via multipart S3 uploads.
|
||||
- Verifying public access via the configured Public Bucket URL.
|
||||
- Publishing the resulting public URL into the video note payload.
|
||||
|
||||
The service is used from the Upload modal UI (via `js/ui/components/UploadModal.js`) and is not a
|
||||
backend integration.
|
||||
> **Architecture Note:** For the high-level design of credential storage, encryption, and the provider adapter pattern, please refer to [`docs/storage-tab-design.md`](./storage-tab-design.md).
|
||||
|
||||
## Assets stored in R2
|
||||
R2 is used as the URL-first playback source. The upload path supports a single file per upload, but
|
||||
@@ -44,74 +43,49 @@ Manually add a CORS policy in the bucket settings to allow uploads from your app
|
||||
4. Copy the **Account ID**, **Access Key ID**, and **Secret Access Key**.
|
||||
|
||||
### Step 4: Configure the App
|
||||
1. Open the Upload Modal in the app.
|
||||
2. Expand **Storage Settings**.
|
||||
1. Open the **Storage Tab** (in Profile or Upload modal).
|
||||
2. Add a new connection and select **Cloudflare R2**.
|
||||
3. Enter the **Account ID**, **Access Key ID**, **Secret Access Key**, and **Public Bucket URL**.
|
||||
4. Click **Verify & Save**.
|
||||
|
||||
The app will attempt to upload a small test file to verify the credentials and public access configuration.
|
||||
|
||||
## Upload lifecycle
|
||||
1. **Settings saved locally**
|
||||
- The Upload modal calls `r2Service.saveSettings()` to persist Cloudflare credentials. Settings are stored in IndexedDB (`bitvidSettings` → `kv`)
|
||||
with a localStorage fallback (`bitvid:r2Settings`).
|
||||
|
||||
2. **Bucket preparation**
|
||||
- The service relies on the user ensuring the bucket exists and is configured for CORS.
|
||||
- It attempts a "best-effort" auto-configuration using the S3 keys (requires broader permissions), but strictly supports the manual "Object Read & Write" flow.
|
||||
1. **Credential Retrieval:**
|
||||
- When an upload starts, `UploadModal` retrieves the decrypted credentials from `StorageService`.
|
||||
- `StorageService` handles the NIP-44/04 decryption of the Master Key and subsequent decryption of the R2 connection payload.
|
||||
|
||||
3. **Upload and publish**
|
||||
- The service builds the R2 object key and public URL.
|
||||
- `multipartUpload()` uploads the file using S3 multipart uploads.
|
||||
- On success, `publishVideoNote()` is called with the hosted URL, so playback can be URL-first
|
||||
with magnet fallback.
|
||||
2. **Upload Execution:**
|
||||
- `r2Service.multipartUpload()` uses the standard AWS SDK commands (`CreateMultipartUpload`, `UploadPart`, etc.) via the generic `makeR2Client` adapter.
|
||||
- The adapter automatically constructs the R2 endpoint (`https://<accountId>.r2.cloudflarestorage.com`) using the stored `accountId`.
|
||||
|
||||
4. **UI feedback**
|
||||
- The Upload modal receives progress, status, and verification errors via `r2Service` events.
|
||||
3. **Publish:**
|
||||
- On success, `publishVideoNote()` is called with the hosted URL, enabling URL-first playback.
|
||||
|
||||
## Download / playback flow
|
||||
There is no direct “download” API for R2 in `r2Service`. Instead, R2 provides the public URL that
|
||||
gets saved into the video note payload. Playback uses that URL as the primary source and only falls
|
||||
back to magnet/WebTorrent when needed, preserving the URL-first playback promise.
|
||||
## Configuration keys
|
||||
|
||||
## Configuration keys and where they live
|
||||
**Stored locally (IndexedDB / localStorage):**
|
||||
R2 credentials are no longer stored in plain `bitvidSettings`. They are encapsulated within the `StorageService` data model.
|
||||
|
||||
**R2 Connection Profile (Encrypted):**
|
||||
- `provider`: `cloudflare_r2`
|
||||
- `accountId`: Cloudflare account ID.
|
||||
- `accessKeyId`: R2 S3 access key ID.
|
||||
- `secretAccessKey`: R2 S3 secret access key.
|
||||
- `baseDomain`: The Public Bucket URL (e.g., `https://pub-xxxx.r2.dev`) used to generate file URLs.
|
||||
- `buckets`: Cached map of `npub` → `{ bucket, publicBaseUrl, domainType, lastUpdated }`.
|
||||
- `accessKeyId`: S3 access key ID.
|
||||
- `secretAccessKey`: S3 secret access key.
|
||||
- `bucket`: The target bucket name.
|
||||
|
||||
**Environment / global flags:**
|
||||
**Metadata (Plaintext):**
|
||||
- `baseDomain`: The Public Bucket URL (e.g., `https://pub-xxxx.r2.dev`).
|
||||
- `defaultForUploads`: Boolean flag indicating if this is the active upload target.
|
||||
|
||||
## Environment / global flags
|
||||
- `globalThis.__BITVID_DISABLE_NETWORK_IMPORTS__`: disables the AWS SDK network import used by the
|
||||
R2 S3 client loader (`js/storage/r2-s3.js`). When true, uploads will fail with a load error until
|
||||
the SDK is bundled or otherwise provided.
|
||||
|
||||
**Config files:**
|
||||
- There are no R2-specific keys in `config/` today; all R2 settings are entered through the UI and
|
||||
stored locally in the browser.
|
||||
|
||||
## Retry and fallback behavior
|
||||
- **Bucket/domain setup:**
|
||||
- The system now relies on explicit user configuration for the Public Bucket URL (`baseDomain`). If this is missing or incorrect, verification will fail.
|
||||
- **CORS:**
|
||||
- The app attempts to set CORS if permissions allow, but largely assumes the user has configured it manually if using minimal tokens.
|
||||
- **Multipart uploads:**
|
||||
- There is no custom retry loop inside `multipartUpload()`. Failures propagate to the UI; the
|
||||
caller can retry by re-initiating the upload.
|
||||
|
||||
## Security considerations
|
||||
- **Credential storage:** R2 credentials are stored in browser storage. Treat them as
|
||||
sensitive and avoid sharing browser profiles. Clear settings via the Upload modal when done.
|
||||
- **Public exposure:** The generated URLs are public; uploaded assets should be safe for public
|
||||
access. Use signed URLs only if you add a separate access layer.
|
||||
- **CORS and origins:** Ideally, scope CORS rules to the current origin to avoid overly broad exposure, though `*` is often used for ease of setup.
|
||||
- **Minimal Permissions:** We recommend "Object Read & Write" tokens to limit the scope of the keys to specific buckets and prevent admin actions.
|
||||
|
||||
## Impact on uploads and playback
|
||||
- **Uploads:** Successful R2 uploads produce a hosted URL stored in the video note payload, enabling
|
||||
URL-first playback. Metadata such as MIME type and the public URL are also stitched into the NIP-71
|
||||
metadata block during publish.
|
||||
- **Playback:** The hosted URL is the primary stream source. If it fails, playback can still fall
|
||||
back to the magnet/WebTorrent path, so the R2 integration improves reliability without removing
|
||||
P2P fallback.
|
||||
- **Encrypted Storage:** All sensitive keys (`accessKeyId`, `secretAccessKey`) are encrypted at rest using the user's Nostr signer.
|
||||
- **Session Security:** Decrypted keys exist in memory only during the active session.
|
||||
- **Public Exposure:** The generated playback URLs are public. Ensure your bucket permissions match your intent.
|
||||
- **Least Privilege:** Always use "Object Read & Write" tokens rather than Admin tokens.
|
||||
@@ -0,0 +1,114 @@
|
||||
# Storage Tab & Connection Architecture
|
||||
|
||||
This document describes the design and architecture of the **Storage Tab** (manage via the Profile or Upload modals) and the underlying `StorageService`.
|
||||
|
||||
The system allows users to securely store and manage multiple cloud storage connections (Providers) for video hosting. The architecture prioritizes security (encryption at rest), extensibility (adapter pattern), and provider flexibility (Generic S3 vs. specific implementations like Cloudflare R2).
|
||||
|
||||
## 1. Core Architecture: `StorageService`
|
||||
|
||||
The `StorageService` (`js/services/storageService.js`) acts as the secure vault for connection secrets. It sits between the UI and the actual storage providers.
|
||||
|
||||
### Data Model
|
||||
|
||||
All data is stored in **IndexedDB** under the database `bitvid-storage` and object store `accounts`.
|
||||
|
||||
**Schema Structure:**
|
||||
|
||||
```javascript
|
||||
{
|
||||
pubkey: "hex-pubkey", // Primary Key
|
||||
encryptedMasterKey: { // The "Key to the Kingdom"
|
||||
method: "nip44" | "nip04", // Encryption method used (NIP-44 preferred)
|
||||
ciphertext: "..." // The AES-GCM Master Key encrypted by the User's Signer
|
||||
},
|
||||
connections: { // Map of connection configurations
|
||||
"conn-uuid-1": {
|
||||
id: "conn-uuid-1",
|
||||
provider: "cloudflare_r2", // Provider ID
|
||||
meta: { // Plaintext Metadata (safe to read without unlocking)
|
||||
label: "My R2 Bucket",
|
||||
defaultForUploads: true,
|
||||
lastSaved: 1234567890
|
||||
},
|
||||
encrypted: { // Encrypted Secrets (AES-GCM)
|
||||
cipher: "hex-cipher-string",
|
||||
iv: "hex-iv-string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Encryption Strategy (Dual-Layer)
|
||||
|
||||
To balance security with usability (minimizing NIP-07/NIP-44 prompts), the system uses a dual-layer encryption approach:
|
||||
|
||||
1. **Layer 1: The Master Key (Session Key)**
|
||||
* A random 256-bit **AES-GCM** key is generated when the user first sets up storage.
|
||||
* This key is encrypted **once** using the user's Nostr Signer (NIP-44 or NIP-04) and stored as `encryptedMasterKey`.
|
||||
* **Unlocking:** When the user "unlocks" storage, the app requests the signer to decrypt this key. The decrypted Master Key is held in memory for the session.
|
||||
|
||||
2. **Layer 2: Payload Encryption**
|
||||
* Actual connection secrets (Access Keys, Secret Keys) are encrypted using the **Master Key** (AES-GCM).
|
||||
* This allows the app to save/load multiple connections during a session without re-prompting the user's signer for every single field.
|
||||
|
||||
## 2. Provider Adapter Pattern
|
||||
|
||||
The system uses an adapter pattern to support multiple providers.
|
||||
|
||||
* **Provider IDs:** Defined in `PROVIDERS` (e.g., `cloudflare_r2`, `aws_s3`, `generic_s3`).
|
||||
* **Test Handlers:** A `PROVIDER_TESTS` map routes connection tests to the appropriate handler.
|
||||
|
||||
```javascript
|
||||
// js/services/storageService.js
|
||||
const PROVIDER_TESTS = {
|
||||
[PROVIDERS.R2]: testS3Connection,
|
||||
[PROVIDERS.S3]: testS3Connection, // Reuses S3 logic
|
||||
[PROVIDERS.GENERIC]: testS3Connection // Reuses S3 logic
|
||||
};
|
||||
```
|
||||
|
||||
This design allows adding new providers (e.g., FTP, IPFS, etc.) by registering a new ID and a corresponding test function.
|
||||
|
||||
## 3. Connection Payloads & Profiles
|
||||
|
||||
Different providers require different configuration schemas.
|
||||
|
||||
### Generic S3 Provider (`generic_s3`)
|
||||
The baseline schema for any S3-compatible service.
|
||||
|
||||
* **Payload (Encrypted):**
|
||||
* `endpoint`: Full URL to the S3 API (e.g., `https://s3.us-east-1.amazonaws.com`).
|
||||
* `region`: AWS Region (e.g., `us-east-1`).
|
||||
* `accessKeyId`: API Key ID.
|
||||
* `secretAccessKey`: API Secret.
|
||||
* `bucket`: Target bucket name.
|
||||
|
||||
### Cloudflare R2 Provider (`cloudflare_r2`)
|
||||
A specialized profile of S3 with specific constraints to improve UX.
|
||||
|
||||
* **Payload (Encrypted):**
|
||||
* `accountId`: Cloudflare Account ID (used to construct the endpoint).
|
||||
* `accessKeyId` & `secretAccessKey`.
|
||||
* `bucket`: (Optional) Target bucket.
|
||||
* **Adapter Logic:**
|
||||
The `makeR2Client` adapter (`js/storage/r2-s3.js`) detects the `accountId` and automatically constructs the standard Cloudflare endpoint:
|
||||
`https://<accountId>.r2.cloudflarestorage.com`
|
||||
|
||||
## 4. Security & Privacy Warnings
|
||||
|
||||
### Local Storage & XSS
|
||||
* **Risk:** The encrypted data resides in the browser's **IndexedDB**. While encrypted at rest, a malicious script (XSS attack) running in the context of the application could potentially request the signer to decrypt the Master Key or read the decrypted key from memory if the session is active.
|
||||
* **Mitigation:** The application must maintain strict Content Security Policy (CSP) and dependency hygiene to prevent XSS.
|
||||
|
||||
### Ephemeral Key Derivation (Kind 22242)
|
||||
To consistently derive or verify keys without publishing sensitive data, the system uses a **Storage Challenge Event**.
|
||||
|
||||
* **Kind:** `22242` (`NOTE_TYPES.STORAGE_CHALLENGE`)
|
||||
* **Purpose:** This event is created ephemerally to challenge the signer. The signature of this event is used as a seed to derive the encryption keys or to verify ownership.
|
||||
* **Privacy:** This event is **NEVER** published to relays. It exists solely in the local client memory during the unlocking process.
|
||||
|
||||
### Secret Handling
|
||||
* Secrets (Access Keys) are **never** stored in plain text in IndexedDB or LocalStorage.
|
||||
* Secrets are only decrypted in memory when needed (e.g., to perform an upload or list buckets).
|
||||
* Logging of secret values is strictly prohibited.
|
||||
Reference in new issue
Block a user