[storage] #38: add Backblaze B2 as a first-class storage provider

B2 doesn't map onto the generic-S3 "paste an endpoint" model: its S3-compatible
endpoint is region-scoped (s3.<region>.backblazeb2.com) and a public bucket is
addressed virtual-hosted-style, so the public download URL is
https://<bucket>.s3.<region>.backblazeb2.com/<key> — not derivable from a pasted
generic endpoint. So B2 silently failed under "Custom S3".

- s3-url: deriveB2Endpoint(region) -> https://s3.<region>.backblazeb2.com (empty for
  auto/blank/pasted-host so callers can require a real region); re-exported from
  s3Service.
- storageService: PROVIDERS.B2 + resolveB2Endpoint (explicit endpoint wins, else
  derive) + PROVIDER_TESTS[B2] (derived endpoint, forcePathStyle false).
- Public URL = S3-style virtual-hosted, auto-derived; an explicit Public Access URL
  (custom domain/CDN) overrides it. The save path now forwards the user's public URL
  to prepareS3Connection (also fixes generic-S3 ignoring a custom CDN URL when blank
  -> still derives, so no regression).
- UI: "Backblaze B2" provider option + helper; raw Endpoint hidden for B2 (region
  drives it), region shows a us-west-004 example, force-path-style toggle hidden;
  save/test derive endpoint from region with a clear "enter your region" error.
  UploadModal shows B2-specific labels.
- Tests: tests/storage-b2-provider.test.mjs (8). All s3/storage suites green (50);
  build + lint clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
thePR0M3TH3ANandClaude Opus 4.8 committed 2026-06-30 10:05:53 -04:00
1 parent 35bb1bc576
commit e79595b902
8 files changed
+259 -19

No files matched your search

+6
View File
@@ -898,6 +898,7 @@
<span class="mb-2 block font-medium">Provider</span>
<select id="storageProvider" class="input">
<option value="cloudflare_r2" selected>Cloudflare R2</option>
<option value="backblaze_b2">Backblaze B2</option>
<option value="generic_s3">Custom S3</option>
</select>
</label>
@@ -907,6 +908,11 @@
Enter your Cloudflare R2 credentials. Your Account ID (from dashboard) is used to connect. For Public Access URL, use your custom domain or R2.dev subdomain. Ensure the token has "Object Read & Write" permissions.
</div>
<!-- B2 Helper -->
<div id="storageB2Helper" class="card border border-info/30 bg-info/10 p-3 text-xs text-info hidden">
Enter your Backblaze B2 Application Key. Set Region to your bucket's region (e.g. us-west-004) — bitvid derives the endpoint and the public URL (https://&lt;bucket&gt;.s3.&lt;region&gt;.backblazeb2.com). Your bucket must be Public for playback; override the Public Access URL if you serve files from a custom domain/CDN.
</div>
<!-- S3 Helper -->
<div id="storageS3Helper" class="card border border-info/30 bg-info/10 p-3 text-xs text-info hidden">
Enter your S3-compatible storage details. The Endpoint URL is your provider's API address (e.g., https://s3.amazonaws.com or your MinIO URL).
+3
View File
@@ -3,8 +3,11 @@ import { ensureBucketCors, ensureBucketExists } from "../storage/s3-multipart.js
import {
buildS3PublicUrl,
buildS3UrlFromBase,
deriveB2Endpoint,
normalizeS3PublicBaseUrl,
} from "../storage/s3-url.js";
export { deriveB2Endpoint };
import { userLogger } from "../utils/logger.js";
function normalizeEndpoint(endpoint) {
+21
View File
@@ -2,6 +2,7 @@ import { userLogger } from "../utils/logger.js";
import { clearLegacyR2Settings, loadLegacyR2Settings } from "../r2.js";
import { bytesToHex } from "../../vendor/crypto-helpers.bundle.min.js";
import { testS3Connection } from "../storage/r2-s3.js";
import { deriveB2Endpoint } from "../storage/s3-url.js";
const DB_NAME = "bitvid-storage";
const DB_VERSION = 1;
@@ -33,6 +34,7 @@ export const PROVIDERS = Object.freeze({
R2: "cloudflare_r2",
S3: "aws_s3",
GENERIC: "generic_s3",
B2: "backblaze_b2",
});
function resolveConnectionMeta(config) {
@@ -86,6 +88,16 @@ function resolveR2Endpoint(config) {
return `https://${accountId}.r2.cloudflarestorage.com`;
}
// Backblaze B2: prefer an explicit endpoint (lets a user override), otherwise derive
// the region-scoped S3 endpoint (https://s3.<region>.backblazeb2.com) from the region.
function resolveB2Endpoint(config) {
const explicit = resolveEndpoint(config);
if (explicit) {
return explicit;
}
return deriveB2Endpoint(resolveRegion(config));
}
function resolveForcePathStyle(config, fallback = false) {
const meta = resolveConnectionMeta(config);
if (typeof config.forcePathStyle === "boolean") {
@@ -142,6 +154,15 @@ const PROVIDER_TESTS = {
forcePathStyle: resolveForcePathStyle(config, true),
})
),
// B2 is addressed virtual-hosted-style; the public URL then derives to
// https://<bucket>.s3.<region>.backblazeb2.com (forcePathStyle false).
[PROVIDERS.B2]: (config) =>
testS3Connection(
buildS3TestConfig(config, {
endpoint: resolveB2Endpoint(config),
forcePathStyle: resolveForcePathStyle(config, false),
})
),
};
function hexToBytes(hex) {
+19
View File
@@ -17,6 +17,25 @@ function normalizeKey(key) {
return String(key || "").replace(/^\/+/, "").trim();
}
// Backblaze B2's S3-compatible endpoint is region-scoped:
// region "us-west-004" -> https://s3.us-west-004.backblazeb2.com
// Unlike generic S3, the user supplies a region (not a pasted endpoint) and bitvid
// derives the endpoint. Returns "" for a missing/placeholder region so callers can
// surface a "region required" error. B2 buckets are addressed virtual-hosted-style,
// so the public URL then derives to https://<bucket>.s3.<region>.backblazeb2.com via
// buildS3PublicUrl({ forcePathStyle: false }).
export function deriveB2Endpoint(region) {
const normalizedRegion = String(region || "").trim().toLowerCase();
if (!normalizedRegion || normalizedRegion === "auto") {
return "";
}
// Defend against a user pasting a full endpoint/host into the region box.
if (normalizedRegion.includes("/") || normalizedRegion.includes(".")) {
return "";
}
return `https://s3.${normalizedRegion}.backblazeb2.com`;
}
export function normalizeS3PublicBaseUrl(publicBaseUrl) {
const trimmed = String(publicBaseUrl || "").trim();
if (!trimmed) {
+6
View File
@@ -867,6 +867,9 @@ export class UploadModal {
if (provider === PROVIDERS.S3) {
return "Amazon S3";
}
if (provider === PROVIDERS.B2) {
return "Backblaze B2";
}
return "S3 Compatible";
}
@@ -881,6 +884,9 @@ export class UploadModal {
if (provider === PROVIDERS.R2 || provider === "cloudflare_r2") {
return "Uploads will target your Cloudflare R2 bucket.";
}
if (provider === PROVIDERS.B2) {
return "Uploads will target your Backblaze B2 bucket.";
}
return "Uploads will target your S3-compatible bucket.";
}
+67 -12
View File
@@ -3,6 +3,7 @@ import { PROVIDERS } from "../../services/storageService.js";
import {
prepareS3Connection,
getCorsOrigins,
deriveB2Endpoint,
} from "../../services/s3Service.js";
import { getActiveSigner } from "../../nostr/client.js";
import { DEFAULT_NIP07_ENCRYPTION_METHODS } from "../../nostr/nip07Permissions.js";
@@ -31,6 +32,7 @@ export class ProfileStorageController {
this.storagePrefixWarning = null;
this.storageDefaultInput = null;
this.storageR2Helper = null;
this.storageB2Helper = null;
this.storageS3Helper = null;
this.storageForcePathStyleInput = null;
this.storageForcePathStyleLabel = null;
@@ -63,6 +65,7 @@ export class ProfileStorageController {
this.storagePrefixWarning = document.getElementById("storagePrefixWarning") || null;
this.storageDefaultInput = document.getElementById("storageDefault") || null;
this.storageR2Helper = document.getElementById("storageR2Helper") || null;
this.storageB2Helper = document.getElementById("storageB2Helper") || null;
this.storageS3Helper = document.getElementById("storageS3Helper") || null;
this.storageForcePathStyleInput = document.getElementById("storageForcePathStyle") || null;
this.storageForcePathStyleLabel = document.getElementById("storageForcePathStyleLabel") || null;
@@ -403,6 +406,9 @@ export class ProfileStorageController {
updateStorageFormVisibility() {
const provider = this.storageProviderInput?.value || "cloudflare_r2";
const isR2 = provider === "cloudflare_r2";
// B2 derives its endpoint from the region (s3.<region>.backblazeb2.com), so the
// raw Endpoint field is hidden and the region drives the connection.
const isB2 = provider === PROVIDERS.B2;
if (this.storageEndpointInput) {
const label =
@@ -416,6 +422,13 @@ export class ProfileStorageController {
if (this.storagePrefixInput) {
this.storagePrefixInput.placeholder = "https://pub-xxx.r2.dev";
}
} else if (isB2) {
// Endpoint is derived from the region; hide the raw endpoint input.
this.storageEndpointInput.parentElement.classList.add("hidden");
if (this.storagePrefixInput) {
this.storagePrefixInput.placeholder =
"Optional — derived from region/bucket (or your CDN domain)";
}
} else {
if (label) label.textContent = "Endpoint URL";
this.storageEndpointInput.placeholder = "https://s3.example.com";
@@ -427,24 +440,38 @@ export class ProfileStorageController {
}
}
// Region matters for B2 (drives the endpoint) and AWS-style S3; nudge the example.
if (this.storageRegionInput && isB2) {
this.storageRegionInput.placeholder = "us-west-004";
} else if (this.storageRegionInput) {
this.storageRegionInput.placeholder = "auto";
}
if (this.storageR2Helper) {
if (isR2) this.storageR2Helper.classList.remove("hidden");
else this.storageR2Helper.classList.add("hidden");
this.storageR2Helper.classList.toggle("hidden", !isR2);
}
if (this.storageB2Helper) {
this.storageB2Helper.classList.toggle("hidden", !isB2);
}
if (this.storageS3Helper) {
if (!isR2) this.storageS3Helper.classList.remove("hidden");
else this.storageS3Helper.classList.add("hidden");
// Generic/custom S3 helper only — not R2, not B2 (each has its own helper).
this.storageS3Helper.classList.toggle("hidden", isR2 || isB2);
}
if (this.storageForcePathStyleLabel) {
if (!isR2)
this.storageForcePathStyleLabel.classList.remove("hidden", "flex");
else this.storageForcePathStyleLabel.classList.add("hidden");
if (!isR2) {
this.storageForcePathStyleLabel.classList.add("flex");
}
// Force-path-style is a generic-S3 knob; R2 (account-derived) and B2
// (virtual-hosted) fix it internally, so hide the toggle for both.
const showForcePathStyle = !isR2 && !isB2;
this.storageForcePathStyleLabel.classList.toggle(
"hidden",
!showForcePathStyle,
);
this.storageForcePathStyleLabel.classList.toggle(
"flex",
showForcePathStyle,
);
}
}
@@ -731,6 +758,18 @@ export class ProfileStorageController {
const prefix = this.storagePrefixInput?.value?.trim() || "";
const isDefault = this.storageDefaultInput?.checked || false;
// B2: the endpoint isn't entered — derive it from the region.
if (provider === PROVIDERS.B2) {
endpointOrAccount = deriveB2Endpoint(region);
if (!endpointOrAccount) {
this.setStorageFormStatus(
"Enter your Backblaze B2 region (e.g. us-west-004).",
"error",
);
return;
}
}
if (!accessKeyId || !secretAccessKey || !bucket || !endpointOrAccount) {
this.setStorageFormStatus("Please fill in all required fields.", "error");
return;
@@ -772,6 +811,10 @@ export class ProfileStorageController {
secretAccessKey,
bucket,
forcePathStyle,
// Honor an explicit Public Access URL (e.g. a B2 custom domain / CDN in front
// of the bucket); when blank, validateS3Connection derives it from the
// endpoint + bucket (for B2: https://<bucket>.s3.<region>.backblazeb2.com).
publicBaseUrl: prefix || undefined,
origins: getCorsOrigins(),
});
endpointOrAccount = normalized.endpoint;
@@ -854,7 +897,7 @@ export class ProfileStorageController {
}
const provider = this.storageProviderInput?.value || "cloudflare_r2";
const endpointOrAccount = this.storageEndpointInput?.value?.trim() || "";
let endpointOrAccount = this.storageEndpointInput?.value?.trim() || "";
const region = this.storageRegionInput?.value?.trim() || "auto";
const accessKeyId = this.storageAccessKeyInput?.value?.trim() || "";
const secretAccessKey = this.storageSecretKeyInput?.value?.trim() || "";
@@ -867,6 +910,18 @@ export class ProfileStorageController {
const publicBaseUrl = this.storagePrefixInput?.value?.trim() || "";
// B2: the endpoint isn't entered — derive it from the region.
if (provider === PROVIDERS.B2) {
endpointOrAccount = deriveB2Endpoint(region);
if (!endpointOrAccount) {
this.setStorageFormStatus(
"Enter your Backblaze B2 region (e.g. us-west-004) to test.",
"error",
);
return;
}
}
if (!accessKeyId || !secretAccessKey || !endpointOrAccount) {
this.setStorageFormStatus("Missing credentials for test.", "error");
return;
+111
View File
@@ -0,0 +1,111 @@
// #38: Backblaze B2 as a first-class storage provider.
//
// B2 doesn't map onto the generic-S3 "paste an endpoint" model: its S3-compatible
// endpoint is region-scoped (s3.<region>.backblazeb2.com) and a public bucket is
// addressed virtual-hosted-style, so the public download URL is
// https://<bucket>.s3.<region>.backblazeb2.com/<key> — NOT endpoint/bucket/key.
//
// These tests pin the derivation that makes B2 work: region -> endpoint, and the
// (forcePathStyle:false) public-URL derivation, plus the explicit-override path.
import test from "node:test";
import { strict as assert } from "node:assert";
import {
deriveB2Endpoint,
derivePublicBaseUrl,
validateS3Connection,
buildS3ObjectUrl,
} from "../js/services/s3Service.js";
import { PROVIDERS } from "../js/services/storageService.js";
test("PROVIDERS exposes a dedicated Backblaze B2 id", () => {
assert.equal(PROVIDERS.B2, "backblaze_b2");
});
test("deriveB2Endpoint builds the region-scoped S3 endpoint", () => {
assert.equal(
deriveB2Endpoint("us-west-004"),
"https://s3.us-west-004.backblazeb2.com",
);
assert.equal(
deriveB2Endpoint(" EU-CENTRAL-003 "),
"https://s3.eu-central-003.backblazeb2.com",
"trims + lowercases",
);
});
test("deriveB2Endpoint returns empty for a missing/placeholder region (so callers can require it)", () => {
assert.equal(deriveB2Endpoint(""), "");
assert.equal(deriveB2Endpoint("auto"), "");
assert.equal(deriveB2Endpoint(undefined), "");
});
test("deriveB2Endpoint refuses a pasted host/endpoint in the region box", () => {
assert.equal(deriveB2Endpoint("s3.us-west-004.backblazeb2.com"), "");
assert.equal(deriveB2Endpoint("https://s3.us-west-004.backblazeb2.com"), "");
});
test("a B2 bucket's public URL derives virtual-hosted (the S3-style download host)", () => {
// This is the crux: with the derived endpoint + forcePathStyle false, the bucket
// becomes a subdomain — the URL a public B2 bucket actually serves files from.
const endpoint = deriveB2Endpoint("us-west-004");
const publicBaseUrl = derivePublicBaseUrl({
endpoint,
bucket: "my-videos",
forcePathStyle: false,
});
assert.equal(
publicBaseUrl,
"https://my-videos.s3.us-west-004.backblazeb2.com",
);
});
test("validateS3Connection derives the B2 public URL end-to-end (no manual public URL)", () => {
const normalized = validateS3Connection({
endpoint: deriveB2Endpoint("us-west-004"),
region: "us-west-004",
accessKeyId: "k",
secretAccessKey: "s",
bucket: "my-videos",
forcePathStyle: false,
// publicBaseUrl intentionally omitted -> must derive
});
assert.equal(normalized.endpoint, "https://s3.us-west-004.backblazeb2.com");
assert.equal(
normalized.publicBaseUrl,
"https://my-videos.s3.us-west-004.backblazeb2.com",
);
assert.equal(normalized.forcePathStyle, false);
});
test("an explicit Public Access URL (custom domain / CDN) overrides the derived B2 URL", () => {
const normalized = validateS3Connection({
endpoint: deriveB2Endpoint("us-west-004"),
region: "us-west-004",
accessKeyId: "k",
secretAccessKey: "s",
bucket: "my-videos",
forcePathStyle: false,
publicBaseUrl: "https://cdn.example.com",
});
assert.equal(normalized.publicBaseUrl, "https://cdn.example.com");
});
test("object URLs for a B2 connection resolve under the virtual-hosted bucket host", () => {
const endpoint = deriveB2Endpoint("us-west-004");
const url = buildS3ObjectUrl({
publicBaseUrl: derivePublicBaseUrl({
endpoint,
bucket: "my-videos",
forcePathStyle: false,
}),
endpoint,
bucket: "my-videos",
key: "u/npub1/abc/video.mp4",
forcePathStyle: false,
});
assert.equal(
url,
"https://my-videos.s3.us-west-004.backblazeb2.com/u/npub1/abc/video.mp4",
);
});
+26 -7
View File
@@ -837,13 +837,32 @@ Reported 2026-06-25. Relates to #17 (NIP-71 interop) / the bitvid→NIP-71 mirro
where it slots into channel profile + the feed engine.
### 38. Generic S3 doesn't work with Backblaze B2 — add B2 as a first-class provider
- [ ] Backblaze **B2** fails under the generic-S3 path: B2's model doesn't map cleanly
onto the assumed endpoint / public-access-URL fields (one of them is derived
differently), so credentials fail to fetch/validate. Investigate B2's S3-compatible
endpoint + friendly-URL/`f000`-style download host, and likely add **B2 as its own
provider** (`PROVIDERS`) with B2-specific endpoint + public-URL derivation rather
than forcing it through generic S3. See `js/services/storageService.js` /
`s3Service.js` (provider abstraction from #6).
- [x] **DONE 2026-06-30.** Added **Backblaze B2** as its own provider
(`PROVIDERS.B2 = "backblaze_b2"`). Root issue: B2's S3 endpoint is *region-scoped*
(`s3.<region>.backblazeb2.com`) and a public bucket is addressed virtual-hosted, so
the public download URL is `https://<bucket>.s3.<region>.backblazeb2.com/<key>` —
not derivable from a pasted generic-S3 endpoint. Implementation:
- `deriveB2Endpoint(region)` (`js/storage/s3-url.js`, re-exported from `s3Service`):
region → `https://s3.<region>.backblazeb2.com`; returns "" for `auto`/blank/pasted
host so callers can require a real region.
- `storageService`: `PROVIDERS.B2` + `resolveB2Endpoint` (explicit endpoint wins,
else derive from region) + a `PROVIDER_TESTS[B2]` handler (derived endpoint,
`forcePathStyle:false`).
- **Public URL strategy = S3-style virtual-hosted** (user-chosen): auto-derives to
`https://<bucket>.s3.<region>.backblazeb2.com`; an explicit Public Access URL
(custom domain/CDN) overrides it — the save path now passes the user's URL to
`prepareS3Connection` (also fixes generic-S3 silently ignoring a custom CDN URL).
- UI (`profile-modal.html` + `ProfileStorageController`): "Backblaze B2" option +
helper; for B2 the raw Endpoint field is hidden (region drives it), Region shows a
`us-west-004` example, force-path-style toggle hidden (B2 is virtual-hosted), and
save/test derive the endpoint from region (clear "enter your region" error if
missing). `UploadModal` shows B2-specific labels.
- Tests: `tests/storage-b2-provider.test.mjs` (8 — endpoint derivation, virtual-
hosted public URL, end-to-end validate, explicit override, object URLs). All
s3/storage suites green (50); build + lint clean.
- [ ] **VERIFY live** with a real B2 Application Key + public bucket on unstable: pick
Backblaze B2, enter region (e.g. `us-west-004`) + bucket + key/secret, Test → passes;
upload a video → plays from `https://<bucket>.s3.<region>.backblazeb2.com/...`.
### 39. Image uploads should prefer uploading to configured storage (UX)
- [ ] Anywhere the user adds an image, prefer letting them **upload an image file to