mirror of
https://github.com/PR0M3TH3AN/SeedPass.git
synced 2026-11-04 03:42:04 +00:00
thePR0M3TH3ANandClaude Opus 5
8414deff20
Restoring the wrong manifest is not an ordering detail -- the manifest that
wins the comparison IS the vault the user gets back. Two defects compounded:
TypeScript ordered manifests by created_at with the event id as tie-break.
created_at is whole seconds, so two snapshots published in the same second
tied, and the id tie-break -- stable and relay-independent, but arbitrary with
respect to time -- returned the OLDER vault about half the time. Entries
created between the two syncs disappeared silently; skippedNewerManifests
stayed empty because nothing was skipped, the wrong manifest simply sorted
first. It surfaced only as an intermittent test failure.
Python never ordered at all: it asked the relay for limit(1) and restored
whatever came back, handing the relay the choice of which snapshot wins.
Both now stamp published_ms -- publication time in milliseconds -- inside the
manifest JSON, so it is covered by the event signature and no relay can steer
it, and both order by it. The value is monotonic per process (max(now,
last+1)), because two back-to-back publishes can share a millisecond and would
otherwise fall back to the same arbitrary id tie-break in a narrower window.
Across machines the clock remains the only shared reference; that case is
unchanged and documented.
Backwards compatible: the field is optional, and a manifest without it orders
at the start of its created_at second. That biases a new-format manifest to
win a same-second tie against an old-format one, which is the right direction
-- old-format manifests are by definition from before the upgrade. Ordering
the unknown one last would let a pre-upgrade snapshot outrank every
post-upgrade one for that second: the same bug from the other side.
Two things found while doing it:
- Python's publish_delta republishes the manifest with an updated
delta_since. It needed the new field too: without it the manifest carrying
the newest delta_since sorted BELOW the snapshot it supersedes, so readers
fetched deltas from a stale watermark and the update never arrived. Caught
by the existing roundtrip tests, which is why they are worth having.
- Raising Python's fetch limit turned its vestigial fallback loop into a
live downgrade path: a relay withholding one chunk of the newest snapshot
could walk a client silently backwards through its history. The loop now
skips unparseable events (relay noise must not deny service) but refuses
to reach past the newest manifest that genuinely parses, and says why.
A restore that quietly returns last week's vault is indistinguishable from
one that worked.
Tests: js relay.test.ts gains a 12-round same-second case -- one pair
reproduces the bug only half the time, which is how it stayed a "flake" --
plus a legacy-manifest case built by hand in the true old format. Python gains
test_manifest_ordering.py (6 cases: order key, both fallbacks, arrival order
irrelevant, id tie-break, unparseable content). Mutation-verified on both
sides: restoring the old comparison turns the matching tests red.
The nostr CLI test now passes 6/6 consecutive runs, having failed roughly 1
in 4 before.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN
113 lines
4.4 KiB
Python
113 lines
4.4 KiB
Python
"""Manifest ordering: the newest snapshot wins, and the relay does not choose.
|
|
|
|
Restoring the wrong manifest is not a cosmetic ordering bug -- the manifest
|
|
that wins the comparison IS the vault the user gets back. Two failures fed
|
|
into each other here:
|
|
|
|
1. ``_fetch_manifest_with_keys`` asked the relay for ``limit(1)`` and used
|
|
whatever came back, which handed the relay the choice of which snapshot to
|
|
restore.
|
|
2. ``created_at`` is whole seconds, so two snapshots published in the same
|
|
second tie on time regardless. The TypeScript port broke that tie by event
|
|
id -- stable, but arbitrary with respect to time, so it returned the older
|
|
vault roughly half the time.
|
|
|
|
``published_ms`` lives inside the signed manifest: finer-grained than
|
|
``created_at`` and not something a relay can steer. These tests pin the
|
|
ordering in Python; the equivalent TypeScript tests live in
|
|
js/packages/core/test/relay.test.ts, and the two must agree or the
|
|
implementations will disagree about which snapshot is latest.
|
|
"""
|
|
|
|
import json
|
|
|
|
import nostr.client # noqa: F401 (resolves the package's circular import)
|
|
from nostr.backup_models import Manifest, manifest_order_ms
|
|
from nostr.snapshot import _sort_manifest_events_newest_first
|
|
|
|
SAME_SECOND = 1700001234
|
|
|
|
|
|
class FakeEvent:
|
|
"""Enough of the SDK event surface for the ordering code."""
|
|
|
|
def __init__(self, event_id: str, created_at: int, published_ms: int | None):
|
|
self._id = event_id
|
|
self._created_at = created_at
|
|
body = {
|
|
"ver": 1,
|
|
"algo": "gzip",
|
|
"chunks": [],
|
|
"delta_since": None,
|
|
"nonce": None,
|
|
"index0": None,
|
|
}
|
|
if published_ms is not None:
|
|
body["published_ms"] = published_ms
|
|
self._content = json.dumps(body)
|
|
|
|
def content(self) -> str:
|
|
return self._content
|
|
|
|
@property
|
|
def id(self) -> str:
|
|
return self._id
|
|
|
|
@property
|
|
def created_at(self) -> int:
|
|
return self._created_at
|
|
|
|
|
|
def test_order_key_prefers_published_ms():
|
|
with_ms = Manifest(ver=1, algo="gzip", chunks=[], published_ms=SAME_SECOND * 1000 + 900)
|
|
assert manifest_order_ms(with_ms, SAME_SECOND) == SAME_SECOND * 1000 + 900
|
|
|
|
|
|
def test_order_key_falls_back_to_the_created_at_second():
|
|
# A manifest written before published_ms existed sorts at the start of its
|
|
# second, so a new-format manifest from the same second outranks it.
|
|
legacy = Manifest(ver=1, algo="gzip", chunks=[])
|
|
assert manifest_order_ms(legacy, SAME_SECOND) == SAME_SECOND * 1000
|
|
|
|
|
|
def test_newer_same_second_snapshot_wins_despite_a_smaller_event_id():
|
|
# The exact shape of the bug: ordering by event id, the older snapshot
|
|
# wins whenever its id happens to sort higher.
|
|
older = FakeEvent("ffff", SAME_SECOND, SAME_SECOND * 1000 + 100)
|
|
newer = FakeEvent("0000", SAME_SECOND, SAME_SECOND * 1000 + 900)
|
|
|
|
ordered = _sort_manifest_events_newest_first([older, newer])
|
|
assert [e.id for e in ordered] == ["0000", "ffff"]
|
|
|
|
# Order of arrival must not matter either -- that is the relay's choice.
|
|
ordered_reversed = _sort_manifest_events_newest_first([newer, older])
|
|
assert [e.id for e in ordered_reversed] == ["0000", "ffff"]
|
|
|
|
|
|
def test_a_legacy_manifest_does_not_outrank_a_newer_one():
|
|
legacy = FakeEvent("aaaa", SAME_SECOND - 60, None)
|
|
current = FakeEvent("bbbb", SAME_SECOND, SAME_SECOND * 1000 + 5)
|
|
ordered = _sort_manifest_events_newest_first([legacy, current])
|
|
assert ordered[0].id == "bbbb"
|
|
|
|
|
|
def test_event_id_still_breaks_an_exact_millisecond_tie():
|
|
# Deterministic and relay-independent: the id is a hash of the event's own
|
|
# contents, so no party chooses the winner.
|
|
a = FakeEvent("1111", SAME_SECOND, SAME_SECOND * 1000 + 500)
|
|
b = FakeEvent("2222", SAME_SECOND, SAME_SECOND * 1000 + 500)
|
|
assert [e.id for e in _sort_manifest_events_newest_first([a, b])] == ["2222", "1111"]
|
|
assert [e.id for e in _sort_manifest_events_newest_first([b, a])] == ["2222", "1111"]
|
|
|
|
|
|
def test_an_unparseable_manifest_orders_by_created_at_without_raising():
|
|
class Broken(FakeEvent):
|
|
def content(self) -> str:
|
|
return "{not json"
|
|
|
|
broken = Broken("cccc", SAME_SECOND - 120, None)
|
|
good = FakeEvent("dddd", SAME_SECOND, SAME_SECOND * 1000)
|
|
ordered = _sort_manifest_events_newest_first([broken, good])
|
|
# It is rejected later, during parsing; ordering must not blow up first.
|
|
assert ordered[0].id == "dddd"
|