views: deterministic d-tag so relays dedupe a viewer's repeat views (#4)

View counts inflated. viewCounter.js dedupes locally by (viewer, time-window)
correctly, but the published events couldn't be deduped by relays:
generateViewEventDedupeTag built the kind-30079 `d` tag with random entropy +
exact timestamp, so every view was a UNIQUE event. And exactCountForPointer runs
a NIP-45 COUNT (raw event count, no dedupe) and bumped the displayed total up to
it (max(deduped, raw)). Net: reloads / multi-device / re-watches each added a
view.

Make the `d` tag deterministic and window-bucketed (scope:viewer:bucket, no
entropy) so kind-30079 view events are genuinely parameterized-replaceable:
relays keep one event per (viewer, video, window), so the cheap NIP-45 COUNT
becomes accurate AND scalable and the raw count stops inflating. buildViewEvent
no longer invents a random `d` tag (the only production caller, publishViewEvent,
always supplies the deterministic one). Pre-existing entropy events age out over
the backfill horizon.

Tests: tests/view-event-dedupe.test.mjs.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
thePR0M3TH3ANandClaude Opus 4.8 committed 2026-06-24 12:01:25 -04:00
1 parent 926652156d
commit a9cb8a94b4
4 files changed
+110 -16

No files matched your search

+8 -5
View File
@@ -145,11 +145,14 @@ function generateViewEventDedupeTag(actorPubkey, pointer, createdAtSeconds) {
typeof actorPubkey === "string" && actorPubkey.trim()
? actorPubkey.trim().toLowerCase()
: "anon";
const timestamp = Number.isFinite(createdAtSeconds)
? Math.max(0, Math.floor(createdAtSeconds))
: Math.floor(Date.now() / 1000);
const entropy = generateViewEventEntropy();
return `${scope}:${normalizedActor}:${timestamp}:${entropy}`;
// DETERMINISTIC + window-bucketed. Kind-30079 view events are parameterized-
// replaceable (deduped by kind+pubkey+d-tag), so a stable d-tag per
// (viewer, video, dedupe-window) makes the relay collapse a viewer's repeat
// views in the same window into ONE event. That keeps both the NIP-45 COUNT and
// a list() accurate AND scalable. The old entropy+exact-timestamp tag made every
// view a unique event, defeating dedupe and inflating counts (item #4 audit).
const bucketIndex = deriveViewEventBucketIndex(createdAtSeconds);
return `${scope}:${normalizedActor}:${bucketIndex}`;
}
export function hasRecentViewPublish(scope, bucketIndex, actorPubkey) {
+5 -5
View File
@@ -2199,11 +2199,11 @@ export function buildViewEvent(params) {
tags.push(...sanitizedAdditionalTags.map((tag) => tag.slice()));
}
const resolvedDedupeTag =
dedupeTag ||
(schema?.identifierTag?.name
? Math.random().toString(36).slice(2) + Math.random().toString(36).slice(2)
: "");
// No random fallback: view events must be parameterized-replaceable on the
// exact d-tag the caller supplies (publishViewEvent computes a deterministic,
// window-bucketed one). A random d-tag here would make every view a unique
// event and defeat relay-side dedupe, inflating view counts (item #4 audit).
const resolvedDedupeTag = typeof dedupeTag === "string" ? dedupeTag : "";
if (resolvedDedupeTag && schema?.identifierTag?.name) {
const identifierName = schema.identifierTag.name;
+74
View File
@@ -0,0 +1,74 @@
// Item #4 (view-count accuracy): a view event's `d` tag must be DETERMINISTIC and
// window-bucketed so kind-30079 view events are parameterized-replaceable —
// relays then collapse a viewer's repeat views in the same window into one event,
// making both the NIP-45 COUNT and a list() accurate and scalable. The old
// entropy+timestamp tag made every view unique and inflated counts.
import test from "node:test";
import assert from "node:assert/strict";
import { buildViewEvent } from "../js/nostrEventSchemas.js";
import { __testExports } from "../js/nostr/viewEvents.js";
import { VIEW_COUNT_DEDUPE_WINDOW_SECONDS } from "../js/config.js";
const { generateViewEventDedupeTag } = __testExports;
const VIEWER = "f".repeat(64);
const OTHER = "0".repeat(64);
const VIDEO = { type: "e", value: "video-abc" };
const VIDEO2 = { type: "e", value: "video-xyz" };
const W = VIEW_COUNT_DEDUPE_WINDOW_SECONDS;
// A timestamp comfortably inside one window bucket, and another in the same one.
const T0 = 100 * W + 10;
const T0_SAME = 100 * W + (W - 1); // same bucket as T0
const T_NEXT = 101 * W + 5; // next bucket
test("same viewer + video within one window → identical d-tag (replaceable)", () => {
const a = generateViewEventDedupeTag(VIEWER, VIDEO, T0);
const b = generateViewEventDedupeTag(VIEWER, VIDEO, T0_SAME);
assert.equal(a, b, "repeat views in the same window must dedupe to one event");
});
test("d-tag has no entropy: stable across calls", () => {
const a = generateViewEventDedupeTag(VIEWER, VIDEO, T0);
const b = generateViewEventDedupeTag(VIEWER, VIDEO, T0);
assert.equal(a, b);
});
test("next window → different d-tag (counts as a new view)", () => {
const a = generateViewEventDedupeTag(VIEWER, VIDEO, T0);
const c = generateViewEventDedupeTag(VIEWER, VIDEO, T_NEXT);
assert.notEqual(a, c);
});
test("different viewer or different video → different d-tag", () => {
const base = generateViewEventDedupeTag(VIEWER, VIDEO, T0);
assert.notEqual(base, generateViewEventDedupeTag(OTHER, VIDEO, T0));
assert.notEqual(base, generateViewEventDedupeTag(VIEWER, VIDEO2, T0));
});
test("buildViewEvent does NOT invent a random d-tag when none is supplied", () => {
const event = buildViewEvent({
pubkey: VIEWER,
created_at: T0,
pointerValue: VIDEO.value,
pointerTag: ["e", VIDEO.value],
});
const dTags = event.tags.filter((t) => Array.isArray(t) && t[0] === "d");
assert.equal(dTags.length, 0, "no d tag should be auto-generated (no entropy)");
});
test("buildViewEvent uses the exact deterministic d-tag it is given", () => {
const dedupeTag = generateViewEventDedupeTag(VIEWER, VIDEO, T0);
const event = buildViewEvent({
pubkey: VIEWER,
created_at: T0,
pointerValue: VIDEO.value,
pointerTag: ["e", VIDEO.value],
dedupeTag,
});
assert.ok(
event.tags.some((t) => Array.isArray(t) && t[0] === "d" && t[1] === dedupeTag),
"the supplied deterministic d-tag must be on the event",
);
});
+23 -6
View File
@@ -198,12 +198,29 @@ Audit started 2026-06-23. See the doc for architecture map + findings. Summary:
mis-position making it hard to use — see 3a).
### 4. View counter accuracy & reliability — DEEP AUDIT
- [ ] **Deep audit of the whole view-counting system** (`viewEvents.js`,
`reactionCounter.js`, the quarantined `tests/view-counter.test.mjs`): accuracy
and reliability — dedup per viewer/session, relay aggregation, double-counting,
and resilience to dead relays (now that feed defaults are reserved, ensure view
reads/writes use a sane relay set too). This audit underpins the popularity
chart (#26) and the Trending tab (#27), so do it first / alongside them.
Audit finding (2026-06-24): counts INFLATE. `viewCounter.js` dedupes locally by
(viewer, time-window) correctly, but two things defeated it: (a)
`generateViewEventDedupeTag` built the kind-30079 `d` tag with random entropy +
exact timestamp, so every view was a UNIQUE event and relays could not dedupe the
parameterized-replaceable event; (b) `exactCountForPointer` runs a NIP-45 COUNT
(raw event count, no per-viewer dedupe) and bumped the displayed total UP to it
(`max(deduped, raw)`). Net: reloads / multi-device / re-watches each added a view.
- [x] **Root fix — deterministic, window-bucketed `d` tag** (`viewEvents.js`):
`scope:viewer:bucket`, no entropy. Kind-30079 view events are now genuinely
replaceable, so relays collapse a viewer's repeat views in a window into ONE
event → the NIP-45 COUNT is accurate AND scalable, and `max(deduped, raw)`
stops inflating. `buildViewEvent` no longer invents a random `d` tag
(`nostrEventSchemas.js`). Transition: pre-existing entropy events age out over
`VIEW_COUNT_BACKFILL_MAX_DAYS`. Tests: `tests/view-event-dedupe.test.mjs`
(same-window dedupe, next-window separate, per-viewer/per-video, no auto-d).
- [ ] **Remaining triage of the quarantined `tests/view-counter.test.mjs`** (still
quarantined): the `d`-tag/session assertions now pass; it next fails on
`viewCounter.js` cache-TTL/hydration behavior (stale cache should re-fetch
fresh from relays). Separate from the accuracy root cause — triage + un-
quarantine as its own pass.
- [ ] **`reactionCounter.js`** (likes/kind-7) not yet audited; same dedupe-by-
replaceable question may apply. Underpins the popularity chart (#26) /
Trending tab (#27).
### 5. Card-hide / video-liveness check — full plan in docs/video-liveness-plan.md
Audited (2026-06-23). Finding: the hide/show **policy is already correct**