land: area/storage-cors 984c219cd2 (storage-cors)

This commit is contained in:
thePR0M3TH3AN committed 2026-10-22 10:31:28 -04:00
commit 63e2b51961
17 files changed
+433 -25

No files matched your search

@@ -0,0 +1,4 @@
- storage-cors-3 resumed by dispatch/campaign; baseline c5b678f7. Read-only product/retained tests, offline synthetic data only.
- Goal: real deletion ingest followed by stale refresh/late older/view switch; preserve unrelated/newer content under existing semantics.
- Seven scratch scenarios: three controls pass, four strict failures. Evidence docs/testing/storage-cors-3-investigation-2026-10-22.md; script /tmp/bitvid-storage-cors-3-audit.mjs.
- Await independent reproduction/precise ownership routing. No browser/full/build/normal submission or live services authorized for this investigation.
@@ -0,0 +1,4 @@
- Treat kind30078 deleted markers using current timestamp semantics; do not infer new NIP09 or root/author ownership policy.
- Use actual subscribe/buffer/client/service with fake transport and synthetic verifier; retain real filtered localStorage writes. Stub unrelated network hydration and IndexedDB scheduling only.
- Capture actual coordinator render boundary with deferred engine decorator; explicitly leave full UI/DOM qualification open.
- Route discovered shared service/buffer/coordinator changes before edits. No upload/cloud/CORS code changes.
@@ -0,0 +1,4 @@
- Dispatch independent audit confirmed7=3pass/4fail and routed buffer/service only; merged routing076ec802 after evidence67140d89, merged HEAD89c5494a.
- Buffer defers deletion of active versions to existing client.recordTombstone; preserve guarded-event same-ID cleanup. No new deletion timestamp/key policy.
- Service rejects deleted/invalid/client-tombstone-covered objects before author/access bypass and direct caching; prune affected cached copies using authoritative guard plus existing explicit deletion IDs. Persist cleaned optimistic snapshots and empty active results.
- Separate buffer commit ed5fb649; service commit follows. No app/feed coordinator/client.js writes. Coordinator race remains open pending faithful registered-pipeline/view/disposal evidence and new routing.
+9 -9
View File
@@ -1,11 +1,11 @@
{
"id": "storage-cors-3",
"area": "storage-cors",
"status": "open",
"title": "Audit feed/view tombstone stale-refresh regression gap with offline integration",
"blockedOn": null,
"addedOn": "2026-10-20",
"filedAt": "7f99c0456931c917082d6a6680c79655221dcc82",
"resumedOn": "2026-10-22",
"brief": "docs/agents/CAMPAIGN-2026-10-22.md"
"id": "storage-cors-3",
"area": "storage-cors",
"status": "in-progress",
"title": "Audit feed/view tombstone stale-refresh regression gap with offline integration",
"blockedOn": null,
"addedOn": "2026-10-20",
"filedAt": "7f99c0456931c917082d6a6680c79655221dcc82",
"resumedOn": "2026-10-22",
"brief": "docs/agents/CAMPAIGN-2026-10-22.md"
}
+81
View File
@@ -0,0 +1,81 @@
# storage-cors-3 test integrity
## Buffer boundary
```yaml
test_integrity_note:
change_type: ["new_tests"]
scenarios:
- id: SCN-storage3-buffer-newer-orders
given: "strictly newer same-root video and older deletion, in either order or one batch"
when: "actual subscribeVideos buffer ingests and replays the deletion"
then: "newer live version and unrelated content remain active; tombstone timestamp persists"
- id: SCN-storage3-buffer-zombies
given: "two versions and unrelated content"
when: "equal/newer tombstone then late older content then strictly newer content arrive"
then: "older versions stay deleted, unrelated stays active, strictly newer content may return"
- id: SCN-storage3-buffer-author
given: "legacy synthesized roots with equal d-tags from distinct authors"
when: "one author's deletion and late older event arrive"
then: "authoritative client tombstone keys suppress only the intended author's legacy series"
observable_outcomes:
- "real client active/history maps and tombstones after actual subscription transport callbacks"
determinism_controls:
- "synthetic events; explicit awaited EOSE; no sleeps, retries or real transport"
- "fake signature acceptance, metadata hydration and IndexedDB IO only; filtered localStorage remains real"
- "every fixture asserts zero network attempts and closed subscriptions"
anti_cheat_rationale:
prevents: ["unconditional deletion", "no-op deletion", "root-only legacy matching", "over-mocking ingest"]
relaxation:
did_relax_any_assertion: false
if_true_explain_spec_basis: ""
```
Retained `tests/storage-tombstone-buffer.test.mjs`: old buffer baseline 2pass/3fail. The preliminary legacy fixture used empty roots, which real conversion replaces with distinct event IDs; it was corrected to actual synthesized `LEGACY:author:d` roots before recording baseline, preserving strict author-scoping expectations. Fixed buffer 5/5. Existing buffer6/6 and delete-flow4/4 pass naturally.
Restored mutants: unconditional deletion fails all3 ordering scenarios; omitted authoritative tombstone fails all5 scenarios. Commands/scripts/logs: `/tmp/storage3-buffer-mutants.py`, `/tmp/storage3-unconditional-marker-delete.log`, `/tmp/storage3-omit-authoritative-tombstone.log`, `/tmp/storage3-buffer-baseline.log`, `/tmp/storage3-buffer-focused.log`, `/tmp/storage3-buffer-existing.log`. No client policy changes: `recordTombstone` remains the sole timestamp-safe active deletion authority; the buffer keeps the guarded-event identity cleanup.
An initial focused command named nonexistent `tests/video-event-buffer.test.mjs` and failed before tests. Corrected to existing `tests/nostr/videoEventBuffer.test.mjs`; no registration/assertion changes.
No browser/full/build/live signer/cloud/relay qualification. Coordinator stale-render remains open, read-only and outside this change.
## Service snapshot/persistence boundary
```yaml
test_integrity_note:
change_type: ["new_tests"]
scenarios:
- id: SCN-storage3-service-prune
given: "multiple cached versions and unrelated content"
when: "different-ID deletion marker and late older version arrive through actual subscription"
then: "service map, callback, emitter and persisted snapshot contain only unrelated content"
- id: SCN-storage3-service-empty
given: "single live version or an all-ineligible persisted cache"
when: "deletion empties feed or empty active cache loads"
then: "empty persistence replaces stale cache and replacement consumer sees no resurrected entry"
- id: SCN-storage3-service-persistent
given: "deleted and invalid persisted objects for public or author viewer"
when: "optimistic load filters the cache"
then: "only live content is notified, cached and persisted before author bypass"
- id: SCN-storage3-service-known-tombstone
given: "stale nondeleted persisted copy covered by authoritative client tombstone"
when: "reload or direct cache API encounters stale copy"
then: "stale copy stays absent while newer valid and unrelated entries survive"
- id: SCN-storage3-service-author
given: "legacy synthesized roots sharing d-tag across distinct authors"
when: "one author deletion then newer live content and replayed older deletion arrive"
then: "unrelated author and strictly newer content remain in every service snapshot"
observable_outcomes:
- "actual service map, callback, videos:updated and filtered localStorage snapshots"
determinism_controls:
- "same actual subscription/client fixture as buffer cases, explicit EOSE and zero real network attempts"
anti_cheat_rationale:
prevents: ["ID-only removal", "flag-only filtering", "author bypass of deleted state", "empty-persistence skip", "overbroad author cleanup", "over-mocking ingest"]
relaxation:
did_relax_any_assertion: false
if_true_explain_spec_basis: ""
```
New service tests: old service baseline 0pass/8fail; fixed8/8. Eight restored mutants all fail: old-service8; allow-deleted4; allow-invalid4; ignore-known-tombstone2; skip-map-prune5; skip-empty-persist1; skip-direct-cache-guard1; skip-optimistic-persist2. Every retained scenario rejects the old service and at least one selective mutant. Mutator `/tmp/storage3-service-mutants.py` restores product source in finally; logs `/tmp/storage3-service-mutant-results.log` and uniquely named `/tmp/storage3-<mutant>.log`.
Existing live-feed regression exposed that an update may carry the original event ID and a separate deleted object rather than a shared client object. Preserve explicit deletion-ID removal alongside authoritative client tombstone cleanup; the regression was not modified. No new root/address matching implementation: service calls client `isOlderThanTombstone`, including its existing legacy/author key derivation. Existing access/privacy/NSFW/consumer/subscription behavior is covered by unchanged focused suites.
@@ -0,0 +1,42 @@
# storage-cors-3: offline tombstone investigation
- Baseline SHA: `c5b678f7c9d0b92fbf786b16ae831f1be1d0b899` (own clean branch fast-forwarded from `6609269e`). Item filed at `7f99c0456931c917082d6a6680c79655221dcc82`; current paths still exhibit the gap.
- Start: 2026-10-22T14:08:34Z. First reproduction completed by 14:11:03Z. No browser/gate wait; one Git metadata sandbox denial resolved by approved ff-only escalation.
- Routing: dispatch message 20261022T140749; current campaign/rules permit investigation and unique evidence only. No product or retained test changes.
- Reproducer: `/tmp/bitvid-storage-cors-3-audit.mjs`.
- Command: `env -u NODE_TEST_CONTEXT node /tmp/bitvid-storage-cors-3-audit.mjs`.
- Result: natural exit 1; 7 scenarios, 3 pass, 4 fail, 0 cancelled/skipped. Every fixture asserts zero attempted fetch/WebSocket calls.
## Real execution and virtualized boundaries
Real `NostrClient.subscribeVideos` creates the real `VideoEventBuffer`; a fake subscription-manager transport supplies synthetic kind-30078 events and explicit EOSE. Real conversion, root timestamps, tombstones, active/history maps, `NostrService.loadVideos`, filtering, notifications, service map, and filtered localStorage persistence execute. Synthetic verification accepts fixture IDs; unrelated NIP-71 metadata hydration and client IndexedDB scheduling are stubbed IO boundaries. No signer, relay, cloud, storage delete, browser, build or full suite runs.
The stale-refresh scenario also executes actual feed engine, active source and coordinator `refreshFeed`. A controlled deferred decorator holds one completed source snapshot while a second refresh finishes. `renderVideoList` is captured at the presentation boundary; actual DOM/card rendering and the entire registered production stage pipeline are not exercised.
## Passing controls
1. Live original at t100 and unrelated root at t110 populate active state and service notifications.
2. Tombstone t200 followed by late nondeleted t90 leaves target absent from active map and actual active feed source; older stored event is marked deleted; unrelated remains.
3. Tombstone t200 followed by live same-root t300 allows the newer content under existing timestamp semantics.
## Strict failures
1. **Service snapshot/persistence:** original t100 followed by tombstone t200 correctly removes original from client active map. Service removes only the tombstone event ID, leaving original ID in service map. Callback and persisted filtered list contain `original` with `deleted:true` plus unrelated. Expected only unrelated.
2. **Reload/view consumer replacement:** another `loadVideos` emits `persistent-cache` containing original(deleted:true), then emits correct active cache containing unrelated only. Expected no deleted target in either notification. `shouldIncludeVideo` does not reject deleted entries.
3. **Newer live content loss:** live t300 then late tombstone t200 leaves newer history object `deleted:false`, but removes it from active map. `recordTombstone` correctly preserves newer state; `VideoEventBuffer` subsequently unconditionally deletes the active key for a deletion marker. Expected newer and unrelated remain; actual unrelated only.
4. **Stale refresh:** older run captures original/unrelated; delete arrives; newer refresh renders unrelated; release older run and it renders original(deleted:true)/unrelated last. Expected unrelated only. Coordinator applies async run results without checking current refresh/view ownership or deletion state.
`VideoListView.render` source inspection at lines353–410 shows filtering for NSFW/private, with no deleted guard. This is source evidence, not browser/DOM qualification.
## Proposed routing, not implementation
- `js/nostr/videoEventBuffer.js`: preserve live active entries newer than a deletion marker, keeping existing timestamp guard semantics.
- `js/services/nostrService.js`: prevent deleted/tombstone-covered entries in public feed snapshots and optimistic persistence notifications, prune affected stale service entries rather than relying on matching tombstone event IDs. Preserve unrelated and strictly newer live content.
- `js/app/feedCoordinator.js`: investigate smallest refresh/view ownership and final eligibility guard before rendering stale results; do not assume a same-feed request counter alone covers delayed cross-view work.
- New unique retained integration regression after routing, with actual ingest/service and coordinator controls; old implementation must fail. No existing assertions changed.
Dispatch received reproduction/script/counts and proposed scopes at14:11:18Z. Await independent reproduction and precise ownership routing before product/test edits. Full registered pipeline, DOM/browser view switching, equal-timestamp edge cases, legacy d-tag fallback, real signature verification, NIP-09 kind5 ingest, restart persistence/IndexedDB, public relay/cloud behavior remain unverified. These findings concern existing kind30078 soft-delete markers; they do not establish additional deletion policy.
## Invocation limitation retained
An initial `env -u NODE_TEST_CONTEXT node --test /tmp/bitvid-storage-cors-3-audit.mjs` returned only a child-file failure (exit1, no scenario output). The direct Node invocation above exposes all seven scenarios naturally. No runner/gate configuration changed and the opaque wrapper result is not counted as product evidence.
@@ -0,0 +1,38 @@
# storage-cors-3 coordinator follow-on investigation
- Preserved candidate SHA: `89f8923c6e83b79a2b8874f811cdff2af6a809d0` (buffer/service commits already applied).
- Authorized by dispatch14:25:40Z: offline registered coordinator/feed/view/disposal investigation only. Coordinator, app, client and retained tests unchanged. No browser/full/build/queue slot used.
- Reproducer `/tmp/bitvid-storage-cors-3-registered-view-audit.mjs`; raw output `/tmp/storage3-registered-view-audit.log`.
- Command `env -u NODE_TEST_CONTEXT node /tmp/bitvid-storage-cors-3-registered-view-audit.mjs`: natural exit1;4 scenarios,2 pass,2 fail,0 cancelled/skipped;182.585705ms test duration. Zero attempted global fetch/WebSocket calls asserted per fixture. This is JSDOM/offline proof, not native browser or complete application startup proof.
- Completed and reported to dispatch by2026-10-22T14:29:48Z.
## Execution fidelity
Uses retained actual-subscription fixture from the buffer/service regressions: real client conversion/buffer/tombstone state and real service load/notifications/persistence. Synthetic signatures and fake subscription transport; metadata/IndexedDB IO virtualized as previously documented.
Creates real feed engine and invokes actual coordinator `registerRecentFeed` and `registerExploreFeed`. Source, blacklist, dedupe, moderation, posted-at and Explore scoring stages, and actual sorters execute. Uses real playback coordinator timestamp helpers and real timestamp utilities. Calls actual `loadFeedVideos`, `refreshFeed`, `renderVideoList`, `VideoListView.render` and `VideoCard` against a JSDOM document; asserts actual `[data-component="video-card"]` DOM IDs.
The controlled asynchronous IO boundary is the moderation adapter's `setActiveEventIds` method, which the registered moderation stage already awaits. Its first call containing `original` waits on an explicit deferred promise; subsequent calls resolve immediately. Moderator policy is no-reports/default, with other moderator methods absent; view-count subscriptions are disabled at the view adapter. App runtime builders provide empty/default runtimes; creator/moderation decorators are identity adapters, impression/similar-content/health-warning effects disabled. No custom engine stage/decorator or fake feed engine is inserted. This establishes registered pipeline and real list/card behavior under a controlled adapter latency, not real moderation relay timing or all navigation code.
The view replacement scenario calls real `VideoListView.destroy`, assigns a new real view, then calls actual `loadFeedVideos(EXPLORE)` which mounts it and changes actual feed telemetry ownership. It does not invoke the full application navigation/router or auth teardown entrypoint.
## Results
1. PASS: valid delayed Recent pipeline resolves and renders original+unrelated cards. Pending valid work is not categorically discarded.
2. FAIL: Recent pipeline holds before deletion; actual deletion ingestion triggers current subscription refresh and DOM becomes unrelated-only. Releasing older Recent work restores original(deleted:true)+unrelated DOM cards. Expected unrelated-only throughout after current deletion refresh.
3. FAIL: during the same overlap, destroy old view and mount replacement through actual Explore loader. Explore is active and its DOM contains unrelated-only. Releasing previous Recent work renders original(deleted:true)+unrelated into the replacement view; Explore metadata is overwritten by old Recent metadata. Expected unrelated-only and Explore ownership/metadata preserved.
4. PASS: destroy the view and set `videoListView=null`, then release pending pipeline. Actual coordinator does not remount a fully absent view. No claim of a full-app destroy lifecycle test.
The earlier buffer/service fixes do not prevent a source snapshot already captured by an asynchronous registered pipeline from passing deleted objects to the presentation boundary after later deletion. Service current active/cache snapshots remain corrected; coordinator lifecycle/late-result application is a distinct boundary.
## Fixture development failures retained
- Initial JSDOM run lacked the global `HTMLElement` constructor required by actual `VideoCard`:4 errors, no product conclusion. Added scoped constructor from this JSDOM instance, restored in teardown.
- Next attempt deferred timestamp-history IO, but all four scenarios cancelled waiting for an unreachable signal. Measured real `getKnownVideoPostedAt` returns converted `nip71Source.created_at`; no history request occurs. Debug script `/tmp/storage3-registered-view-debug.mjs` records this. Replaced the unreachable wait with the actual awaited moderation subscription adapter boundary; no timestamp fields were deleted or corrupted to force a path.
- Final four scenarios above exit naturally with strict failures and passing controls. No sleeps, retries, force exits, timeouts, assertion relaxation or product changes.
## Proposed next scope (not authorized implementation)
Coordinator should validate freshness/view ownership before applying async results, and validate latest deletion eligibility before passing results or fallback snapshots to a view. New retained regression should exercise the actual registered path, include live/current/error fallback/disposal and cross-view controls, and reject the original coordinator. A request counter alone must not authorize older work to populate a replacement view or discard unrelated current feed work. Exact ownership and semantics require dispatch routing before any coordinator/app changes.
Native browser/full-app router/manual view switching, real moderation/signature/relay timing and complete auth disposal remain unverified. Existing buffer/service candidate remains the first READY gate; its full/build/normal submission slot is still pending.
+5 -8
View File
@@ -168,16 +168,13 @@ export class VideoEventBuffer {
this.client.allEvents.set(evt.id, video);
this.client.dirtyEventIds.add(evt.id);
// If it's a "deleted" note, remove from activeMap
// recordTombstone already removes active versions at or before the
// deletion timestamp. A late/replayed marker must preserve newer ones.
if (video.deleted) {
if (activeKey) {
if (wasDeletedEvent) {
if (activeKey && !wasDeletedEvent) {
const currentActive = this.client.activeMap.get(activeKey);
if (currentActive?.id === video.id) {
this.client.activeMap.delete(activeKey);
} else {
const currentActive = this.client.activeMap.get(activeKey);
if (currentActive?.id === video.id) {
this.client.activeMap.delete(activeKey);
}
}
}
updatedVideos.push(video);
+16 -8
View File
@@ -1449,7 +1449,7 @@ export class NostrService {
const map = this.ensureVideosMap();
for (const video of videos) {
if (video && typeof video.id === "string" && video.id) {
if (video && typeof video.id === "string" && video.id && !this.isVideoUnavailable(video)) {
map.set(video.id, video);
}
}
@@ -1516,11 +1516,17 @@ export class NostrService {
return this.videosByAuthorIndex;
}
isVideoUnavailable(video) {
return video?.deleted === true || video?.invalid === true ||
(typeof this.nostrClient?.isOlderThanTombstone === "function" &&
this.nostrClient.isOlderThanTombstone(video));
}
shouldIncludeVideo(video, {
blacklistedEventIds = new Set(),
isAuthorBlocked = () => false,
} = {}) {
if (!video || typeof video !== "object") {
if (!video || typeof video !== "object" || this.isVideoUnavailable(video)) {
return false;
}
@@ -1737,9 +1743,7 @@ export class NostrService {
this.emit("videos:updated", { videos: filtered, reason });
// Persist the fresh filtered result for next time
if (filtered.length > 0) {
persistFilteredVideos(filtered);
}
persistFilteredVideos(filtered);
return filtered;
};
@@ -1758,6 +1762,7 @@ export class NostrService {
if (optimisticFiltered.length > 0) {
this.cacheVideos(optimisticFiltered);
persistFilteredVideos(optimisticFiltered);
hasOptimisticData = true;
optimisticResult = optimisticFiltered;
@@ -1821,9 +1826,12 @@ export class NostrService {
if (deletedUpdates.length > 0) {
const map = this.ensureVideosMap();
for (const v of deletedUpdates) {
if (v.id) {
map.delete(v.id);
const deletedIds = new Set(deletedUpdates.map((video) => video.id));
// Deletion markers have their own event IDs. Consult the
// client's authoritative timestamp/key guard for cached copies.
for (const [id, video] of map) {
if (deletedIds.has(id) || this.isVideoUnavailable(video)) {
map.delete(id);
}
}
this.markAuthorIndexDirty();
@@ -0,0 +1,6 @@
- SHA c5b678f7c9d0b92fbf786b16ae831f1be1d0b899.
- `env -u NODE_TEST_CONTEXT node --test /tmp/bitvid-storage-cors-3-audit.mjs`: exit1, opaque child-file failure, no scenario detail; not product evidence.
- `env -u NODE_TEST_CONTEXT node /tmp/bitvid-storage-cors-3-audit.mjs`: natural exit1; seven scenarios, three pass, four fail, zero cancelled/skipped. Zero attempted network per fixture.
- Failures: deleted original remains in service callback/persisted list; optimistic reload emits deleted original; late older tombstone removes newer active entry; stale engine run renders deleted original after fresh refresh.
- Passing controls: initial live/unrelated ingestion, tombstone+late older absent from active source, newer post-tombstone content allowed.
- No product/retained tests modified, lint/build/full/browser not run. Limits and transport details in docs/testing/storage-cors-3-investigation-2026-10-22.md.
@@ -0,0 +1,7 @@
- Baseline merged HEAD89c5494aaa23ae2ce2c1c044d0146c922959cdcf.
- `env -u NODE_TEST_CONTEXT node tests/storage-tombstone-buffer.test.mjs`: old buffer exit1,2pass/3fail; fixed exit0,5pass/0fail.
- `python3 /tmp/storage3-buffer-mutants.py`: two mutants each exit1, failures3 and5; original file restored in finally. Every new scenario rejects at least one mutant.
- `env -u NODE_TEST_CONTEXT node scripts/run-targeted-tests.mjs tests/storage-tombstone-buffer.test.mjs tests/nostr-delete-flow.test.mjs`: exit0,5+4 pass.
- `env -u NODE_TEST_CONTEXT node scripts/run-targeted-tests.mjs tests/nostr/videoEventBuffer.test.mjs`: exit0,6pass.
- Mistyped nonexistent existing test path failed before tests and was corrected; retained in integrity note.
- Lint pending after second scoped service fix. No full/build/browser/product submission while QA owns resource slot. Coordinator stale render unresolved.
@@ -0,0 +1,10 @@
- Product buffer commit ed5fb64991ae4aaf5df4ac76f4bd13ca9633aea0; service edits not yet committed when checked.
- `env -u NODE_TEST_CONTEXT node tests/storage-tombstone-service.test.mjs`: old service natural exit1,0pass/8fail; fixed exit0,8pass.
- `python3 /tmp/storage3-service-mutants.py`: exit0,8 deliberately broken variants each exited1, product restored; failing scenario counts8/4/4/2/5/1/1/2.
- Focused command: `env -u NODE_TEST_CONTEXT node scripts/run-targeted-tests.mjs tests/storage-tombstone-buffer.test.mjs tests/storage-tombstone-service.test.mjs tests/nostr/videoEventBuffer.test.mjs tests/nostr-delete-flow.test.mjs tests/nostr-service-live-feed-notification.test.mjs tests/nostr-service-access-control.test.mjs tests/nostr-service-access-control-ready.test.mjs tests/services/nostr-service.test.mjs tests/author-moderation-override.test.mjs tests/moderation-stage.test.mjs tests/app/feedCoordinator.test.mjs tests/app/feedCoordinatorWatchHistoryRefresh.test.mjs`.
- Final focused exit0 across12 files:11 files summarize59 passing tests; existing access-control suite outputs5 named passing checks and exit0 but no final TAP summary. That output limit is disclosed; not called a complete footer. New tests13/13 across two files.
- Initial focused run stopped at existing live-feed test4 (6pass/1fail); explicit same-event-ID removal restored alongside new cleanup; no test changes.
- `env -u NODE_TEST_CONTEXT npm run lint`: initial sandbox exit2 at git-grep EPERM; approved unchanged command outside sandbox exits0,all11 stages.
- `git diff --check`: exit0. Service net growth8 lines; grandfather limit passes.
- Raw logs `/tmp/storage3-focused.log`, `/tmp/storage3-lint.log`, `/tmp/storage3-service-baseline.log`, `/tmp/storage3-service-mutant-results.log`.
- No browser/full/build/queue. Coordinator stale-render remains unresolved/read-only. Live signer/relay/cloud/IndexedDB restart and DOM/full registered pipeline unverified.
@@ -0,0 +1,7 @@
- Committed fixes:buffer ed5fb64991ae4aaf5df4ac76f4bd13ca9633aea0;service fb1997de42dd1d62f2b5be30150bcab350010152.
- Merged latest local unstable e04549f8 into own89f8923c6e83b79a2b8874f811cdff2af6a809d0. Merge had only coordination docs; runtime/test/script diff0 from checked product tip.
- Merged rerun of12-file focused command from14:21log:exit0,59 summarized passing tests plus5 named access-control PASS lines/exit0 without final TAP footer. New tests13/13; all10 restored mutants previously killed.
- Merged `env -u NODE_TEST_CONTEXT npm run lint` outside sandbox for approved git-grep exception:exit0,11stages. Logs /tmp/storage3-merged-focused.log and /tmp/storage3-merged-lint.log.
- READY/full slot request sent14:22:48Z; dispatch14:25:40Z confirms first READY queue, QA still holds browser/full slot. No full/build/normal queue started; no rerunning passing checks merely waiting.
- `env -u NODE_TEST_CONTEXT node /tmp/bitvid-storage-cors-3-registered-view-audit.mjs`:naturalexit1,2pass/2fail/0cancelled/0skipped,182.585705ms. Actual registered Recent/Explore pipelines and JSDOM view/card DOM. Preliminary fixture failures and exact controlled boundaries retained in unique registered-view evidence note.
- Coordinator remains read-only/open. No native browser/full-app/router/live cloud/signer/relay qualification.
+46
View File
@@ -0,0 +1,46 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { tombstoneFixture, videoEvent, ids } from './test-helpers/storage-tombstone-fixture.mjs';
// Scenarios and deterministic IO boundaries: docs/testing/storage-cors-3-integrity.md.
for (const order of ['newer-first', 'delete-first', 'same-batch']) {
test(`buffer preserves strictly newer active content: ${order}`, async t => {
const f = tombstoneFixture(t); await f.load();
const newer = videoEvent('newer', 300);
const deletion = videoEvent('deletion', 200, { deleted: true });
if (order === 'same-batch') await f.ingest(newer, deletion);
else if (order === 'newer-first') { await f.ingest(newer); await f.ingest(deletion); }
else { await f.ingest(deletion); await f.ingest(newer); }
await f.ingest(videoEvent('unrelated', 100, { root: 'other', pubkey: 'b'.repeat(64) }));
assert.deepEqual(ids(f.client.getActiveVideos()), ['newer', 'unrelated']);
assert.equal(f.client.allEvents.get('newer').deleted, false);
assert.equal(f.client.tombstones.get('ROOT:target'), 200);
// Replayed markers must not erase a newer version either.
await f.ingest(deletion);
assert.deepEqual(ids(f.client.getActiveVideos()), ['newer', 'unrelated']);
});
}
test('newer/equal tombstone removes older versions, late zombies stay absent, newer content may return', async t => {
const f = tombstoneFixture(t); await f.load();
await f.ingest(videoEvent('first', 100), videoEvent('edit', 150), videoEvent('other', 100, { root: 'other' }));
await f.ingest(videoEvent('delete', 150, { deleted: true }));
assert.deepEqual(ids(f.client.getActiveVideos()), ['other']);
assert.equal(f.client.allEvents.get('first').deleted, true);
assert.equal(f.client.allEvents.get('edit').deleted, true);
await f.ingest(videoEvent('late', 120));
assert.equal(f.client.allEvents.get('late').deleted, true);
assert.deepEqual(ids(f.client.getActiveVideos()), ['other']);
await f.ingest(videoEvent('fresh', 151));
assert.deepEqual(ids(f.client.getActiveVideos()), ['fresh', 'other']);
});
test('legacy d-tag tombstones retain authoritative author scoping', async t => {
const f = tombstoneFixture(t); await f.load();
await f.ingest(videoEvent('target-legacy', 100, { root: `LEGACY:${'a'.repeat(64)}:same`, dTag: 'same' }), videoEvent('other-author', 100, { root: `LEGACY:${'b'.repeat(64)}:same`, dTag: 'same', pubkey: 'b'.repeat(64) }));
await f.ingest(videoEvent('delete-legacy', 200, { root: `LEGACY:${'a'.repeat(64)}:same`, dTag: 'same', deleted: true }));
assert.deepEqual(ids(f.client.getActiveVideos()), ['other-author']);
await f.ingest(videoEvent('target-late', 90, { root: `LEGACY:${'a'.repeat(64)}:same`, dTag: 'same' }));
assert.deepEqual(ids(f.client.getActiveVideos()), ['other-author']);
assert.equal(f.client.allEvents.get('target-late').deleted, true);
});
+85
View File
@@ -0,0 +1,85 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { tombstoneFixture, videoEvent, ids, author, filteredStorageKey } from './test-helpers/storage-tombstone-fixture.mjs';
const cachedVideo = (id, extra = {}) => ({ id, pubkey: author, videoRootId: id, created_at: 100, title: id, url: 'https://example.invalid/video.mp4', ...extra });
const assertSnapshot = (f, expected) => {
assert.deepEqual(ids([...f.service.getVideosMap().values()]), expected);
assert.deepEqual(ids(f.notifications.at(-1).videos), expected);
assert.deepEqual(ids(f.updates.at(-1).videos), expected);
assert.deepEqual(ids(f.persisted()), expected);
};
test('different-ID tombstone removes every obsolete cached version and preserves unrelated live content', async t => {
const f = tombstoneFixture(t); await f.load();
await f.ingest(videoEvent('first', 100));
await f.ingest(videoEvent('edit', 150), videoEvent('other', 100, { root: 'other', pubkey: 'b'.repeat(64) }));
await f.ingest(videoEvent('delete', 200, { deleted: true }));
assertSnapshot(f, ['other']);
assert.deepEqual(ids(f.updates.at(-1).deleted), ['delete']);
await f.ingest(videoEvent('late', 120));
assertSnapshot(f, ['other']);
});
test('deletion-only empty result persists and second consumer receives no optimistic resurrection', async t => {
const f = tombstoneFixture(t); await f.load(); await f.ingest(videoEvent('first', 100));
await f.ingest(videoEvent('delete', 200, { deleted: true }));
assertSnapshot(f, []);
const count = f.notifications.length;
await f.load();
assert.deepEqual(f.notifications.slice(count).map(entry => ids(entry.videos)), [[]]);
assert.deepEqual(f.persisted(), []);
});
for (const viewer of ['public', 'author']) {
test(`persistent deleted and invalid entries are rejected before ${viewer} access bypass`, async t => {
const f = tombstoneFixture(t, { persisted: [cachedVideo('deleted', { deleted: true }), cachedVideo('invalid', { invalid: true }), cachedVideo('live')] });
if (viewer === 'author') f.client.pubkey = author;
const initial = await f.load();
assert.deepEqual(ids(initial), ['live']);
assert.equal(f.notifications.some(entry => entry.videos.some(video => video.deleted || video.invalid)), false);
assert.deepEqual(ids([...f.service.getVideosMap().values()]), ['live']);
assert.deepEqual(ids(f.persisted()), ['live']);
});
}
test('all-ineligible persistent cache is cleared even when active snapshot is empty', async t => {
const f = tombstoneFixture(t, { persisted: [cachedVideo('deleted', { deleted: true }), cachedVideo('invalid', { invalid: true })] });
assert.deepEqual(await f.load(), []);
assert.deepEqual(f.persisted(), []);
assert.deepEqual(f.notifications.map(entry => ids(entry.videos)), [[]]);
});
test('stale nondeleted persistent copies honor known tombstone while strictly newer and unrelated entries survive', async t => {
const f = tombstoneFixture(t); await f.load();
await f.ingest(videoEvent('old', 100));
const stale = f.persisted()[0];
await f.ingest(videoEvent('delete', 200, { deleted: true }));
await f.ingest(videoEvent('newer', 300), videoEvent('other', 150, { root: 'other' }));
// A stale storage snapshot contains an independent, still-nondeleted copy.
localStorage.setItem(filteredStorageKey, JSON.stringify([stale, ...f.persisted()]));
const start = f.notifications.length;
await f.load();
assert.equal(f.notifications.slice(start).some(entry => entry.videos.some(video => video.id === 'old')), false);
assertSnapshot(f, ['newer', 'other']);
});
test('direct cache API never reintroduces deleted invalid or tombstone-covered copies', async t => {
const f = tombstoneFixture(t); await f.load(); await f.ingest(videoEvent('old', 100));
const stale = f.persisted()[0];
await f.ingest(videoEvent('delete', 200, { deleted: true }));
f.service.cacheVideos([stale, cachedVideo('deleted', { deleted: true }), cachedVideo('invalid', { invalid: true }), cachedVideo('live')]);
assert.deepEqual(ids([...f.service.getVideosMap().values()]), ['live']);
});
test('legacy tombstone cleanup retains other author with same d-tag and strictly newer content', async t => {
const f = tombstoneFixture(t); await f.load();
const legacy = { root: `LEGACY:${author}:same`, dTag: 'same' };
const other = { root: `LEGACY:${'b'.repeat(64)}:same`, dTag: 'same', pubkey: 'b'.repeat(64) };
await f.ingest(videoEvent('old', 100, legacy), videoEvent('other-author', 100, other));
await f.ingest(videoEvent('delete', 200, { ...legacy, deleted: true }));
assertSnapshot(f, ['other-author']);
await f.ingest(videoEvent('newer', 300, legacy));
await f.ingest(videoEvent('delete-replayed', 200, { ...legacy, deleted: true }));
assertSnapshot(f, ['newer', 'other-author']);
});
@@ -0,0 +1,64 @@
import './setup-localstorage.mjs';
import assert from 'node:assert/strict';
import { NostrClient } from '../../js/nostr/client.js';
import { NostrService } from '../../js/services/nostrService.js';
import { getVideosMap, setVideosMap, getVideoSubscription, setVideoSubscription } from '../../js/state/appState.js';
export const filteredStorageKey = 'bitvid:filtered-videos:v1';
export const author = 'a'.repeat(64);
export function videoEvent(id, created_at, { root = 'target', deleted = false, pubkey = author, dTag = root, ...payload } = {}) {
return {
id, kind: 30078, pubkey, created_at,
tags: [['t', 'video'], ['d', dTag]],
content: JSON.stringify({ version: 3, title: id, url: 'https://example.invalid/video.mp4', mode: 'live', videoRootId: root, deleted, ...payload }),
};
}
export const ids = videos => videos.map(video => video.id).sort();
export function tombstoneFixture(t, { persisted = [] } = {}) {
const previous = { map: getVideosMap(), sub: getVideoSubscription(), storage: localStorage.getItem(filteredStorageKey) };
setVideosMap(new Map()); setVideoSubscription(null);
localStorage.setItem(filteredStorageKey, JSON.stringify(persisted));
let networkAttempts = 0;
const oldFetch = globalThis.fetch;
const oldSocket = globalThis.WebSocket;
globalThis.fetch = () => { networkAttempts++; throw new Error('Unexpected network'); };
globalThis.WebSocket = class { constructor() { networkAttempts++; throw new Error('Unexpected socket'); } };
const client = new NostrClient();
// Virtualize unrelated metadata and IndexedDB IO, keeping all ingest,
// tombstone, active/history, service and filtered-localStorage logic real.
client.saveLocalData = () => {};
client.populateNip71MetadataForVideos = async () => {};
client.videoEventVerifier = async events => new Set(events.map(event => event.id));
const channels = [];
client._subscriptionManager = { subscribe(config) {
const channel = { ...config, closed: false };
channels.push(channel);
return { close() { channel.closed = true; } };
} };
const service = new NostrService();
service.nostrClient = client;
service.moderationService = null;
service.accessControl = { isHydrated: () => true, ensureReady: async () => {}, canAccess: video => video.title !== 'denied' };
const notifications = [];
const updates = [];
const off = service.on('videos:updated', detail => updates.push(structuredClone(detail)));
const load = (options = {}) => service.loadVideos({ onVideos: (videos, detail) => notifications.push({ videos: structuredClone(videos), ...detail }), ...options });
t.after(() => {
service.clearVideoSubscription(); off();
setVideosMap(previous.map); setVideoSubscription(previous.sub);
if (previous.storage === null) localStorage.removeItem(filteredStorageKey);
else localStorage.setItem(filteredStorageKey, previous.storage);
globalThis.fetch = oldFetch; globalThis.WebSocket = oldSocket;
assert.equal(networkAttempts, 0, 'no real network attempted');
assert.ok(channels.every(channel => channel.closed), 'subscriptions naturally closed');
});
return { client, service, channels, notifications, updates, load,
async ingest(...events) {
const channel = channels.at(-1);
for (const event of events) channel.onEvent(event);
await channel.onEose();
},
persisted: () => JSON.parse(localStorage.getItem(filteredStorageKey) || '[]'),
};
}
@@ -0,0 +1,5 @@
- [x] Read mail/current campaign/rules; safely ff own clean branch; claim storage-cors-3 and acknowledge.
- [x] Trace real client/buffer/service/source/coordinator; reproduce offline with strict controls.
- [x] Report seven scenarios/three pass/four fail and proposed scopes to dispatch.
- [ ] Independent confirmation and precise implementation routing; product/retained tests remain read-only meanwhile.
- [ ] Full registered pipeline/view-switch regression, failing mutants, required gates and durable landing only after routing/resource grant.