The Tests workflow was red on every OS and Python version for one reason:
`black --check` found 28 unformatted files. Part predates this branch (the
tui_v3 screens, core/api.py, core/index0.py); part is mine from this session.
Formatting only — verified rather than assumed, because a formatter touching
363 files is exactly where an unnoticed behaviour change would hide:
- 1223 Python tests pass, unchanged
- regenerating the fixtures after reformatting the generator produces
byte-identical output, so no hashed value moved
- differential fuzz: no divergences
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN
index0 is the per-vault activity ledger under `_system.index0`: an append-only
event stream per writer, daily checkpoints rolling those events up, and
derived canonical views (children_of, counts_by_kind, recent_activity).
It was the last piece carried through verbatim rather than computed — which
was safe for round-tripping but meant a TypeScript client never UPDATED it. A
vault written by this port went stale the moment it was touched, and a Python
client reading it afterwards saw an activity log missing everything the TS
client had done. Worse, `nostr restore` kept the local block verbatim, so a
restore silently discarded the remote's history.
Hash parity is the entire contract here: event ids, integrity hashes,
checkpoint summaries and view hashes are SHA-256 over Python's canonical JSON,
and the merge picks winners by comparing those hashes — one byte of encoding
disagreement means the two implementations permanently disagree about which
record wins. Everything routes through the existing canonicalJson (Python's
ensure_ascii escaping, code-point key ordering, CPython number formatting)
rather than JSON.stringify, and the tests compare whole structures via
canonical JSON rather than toEqual, so they pass only if the bytes that get
hashed are identical. Verified against Python fixtures covering the append
chain, compaction, manifest metadata, views and a two-replica merge —
including non-ASCII labels and tags, a non-numeric entry id, two writers
across two days, and a managed-account scope path.
Two Python quirks are replicated deliberately rather than "fixed", because
diverging would change which events exist rather than merely how they hash:
`str(None)` is "None" (so a JSON null in a required field is a non-empty
string there), and a stored integrity_hash is preserved even when it
disagrees — this normalizes, it does not verify, and silently rewriting a
mismatched hash would hide the tampering the hash exists to reveal.
Events are emitted by DIFFING the index at the single mutation funnel each
surface goes through, rather than from each call site as Python does. That is
deliberate: a call-site emitter is something a new code path can forget, and a
silently incomplete ledger is worse than an absent one because it reads as
"nothing happened". The event vocabulary matches Python's, so a stream from
either implementation reads the same. Emission is best-effort — the atlas is
derived state, and it must never fail a committed vault write.
mergeIndexPayloads now defaults to merging index0 with the same deterministic
rules; "preserve-current" and "reject" remain for callers that want them, and
`nostr restore` no longer discards the remote's history.
Also made gen_high_risk deterministic. Fernet embeds a random IV and the
current time, so that fixture regenerated differently every run and would have
failed the fixture-drift CI job each time — it now pins both, with a note that
production must never do this.
Verified end to end: the TS binary performed four mutations, and Python read
the resulting atlas, normalized it without dropping a record, and compacted it
into checkpoints and views.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN
High-risk partitions move ssh/pgp/seed/nostr/managed_account records out of
the vault index into a separately encrypted file, so those kinds need a SECOND
factor beyond the master password. The index keeps a stub carrying kind, label
and archived — enough to list, nothing more.
Both file formats are byte-compatible with Python and verified bidirectionally:
TS unwraps a Python-written key envelope and reads a Python-written partition,
and Python unwraps a TS-written envelope and reads a TS-written partition. That
matters more than usual here — the entries are SSH keys, PGP keys and seeds, so
a partition only one implementation can open is lost secrets, not an
inconvenience. This required a Fernet encryptor in core, which had been
decrypt-only by design; the header now says why that exception exists and that
no other Fernet payload should ever be written.
**One deliberate divergence, and it is the reason this is not a straight port.**
Python records an unlock session in agent_high_risk_unlock.json, and that record
contains the partition_key_tag — the value the partition file's encryption key
is derived from. So for the life of any Python unlock session the partition is
decryptable from disk alone, with no factor, by anything that can read the app
directory; both files live in the same directory, so "can read one" implies "can
read the other". That voids the guarantee the partition exists to provide.
Verified empirically before writing any of this.
Here the tag lives only in the session agent's memory, beside the parent seeds
it already holds, with the same TTL discipline — and in the API process's memory
for the API. Nothing derived from the factor reaches disk, asserted by a test
that scans every file in the app directory for the tag, and confirmed on a live
install. No file format or interop changes; only where the live session lives.
Other properties the tests pin:
- a locked partition refuses reveal rather than deriving from the stub, which
would return a plausible wrong secret
- locking the vault drops the high-risk grant, which is scoped to it
- the grant expires on its own clock, typically much shorter than the seed's
- unlock is owner-gated: a scoped token cannot reach it
- the tag is absent from the audit log, which records only that an unlock
happened
- the factor comes from the environment, never argv
- factor-set refuses to replace an existing factor, since a new partition key
would make an existing partition permanently unreadable
- migrate writes the partition BEFORE the index, so a crash between them
cannot leave stubs pointing at records that were never stored
Approval gates port straight across: one-shot (or N-use) authorizations for
export / reveal_parent_seed / private_key_retrieval, consumed under the store
lock so two callers cannot both spend the last use, with every failure mode
getting its own reason string matching Python's. A corrupt store reads as empty,
which fails closed.
API: /api/v1/high-risk/{status,unlock,lock} are real rather than 501. Unlock
needs the bearer token, the master password header AND the factor header; a
wrong factor gets one indistinguishable reason so probing cannot tell failures
apart.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN
The "semantic (vector search)" entry in not_yet_ported was a misnomer. It is
not vector search and needs no model: MODEL_ID is "seedpass-token-overlap-v1",
and it scores by Jaccard overlap over tokenized metadata. That is why it could
be ported at all — there is nothing to embed.
Porting it surfaced two defects in the Python implementation.
**Stored secrets were written to a plaintext, 0664 file.** `_extract_text`
appended a key_value entry's `value` — the stored secret itself — and
`build()` wrote it to semantic_index/records.json in the clear, along with a
tokenized copy that leaked it just as thoroughly. The file was created at the
process umask, so on a default install it was group- and world-readable, next
to a vault that is encrypted and 0600. Anything that could read the directory
— a backup, a sync client, another user on a shared machine — got the secret
without the master password.
Fixed in Python (index the key NAME, never the value; chmod 0600 on both
files) and never done in TypeScript. The rule is worth keeping simple: nobody
searches for a secret they do not already know, so indexing one is pure
downside. MODEL_ID is bumped to v2 so an index built by the old code is
distinguishable — a v1 index on disk still contains the secrets and should be
rebuilt.
**Entry 0 was silently unsearchable.** The check was
`int(entry.get("id", 0) or 0) <= 0`, conflating "no id" with "id 0" — and
entry ids start at 0, so the first entry any user ever created was absent from
every search result with nothing to indicate why. Found by writing a test that
queried for it. Fixed in both implementations: skip an id that is absent,
unparseable or negative, which is what the check was reaching for.
The TypeScript port is wired into the CLI (`semantic build/status/search`),
the API (the /api/v1/semantic routes are now real rather than 501), and
writes through atomicWrite for a genuine 0600 inode. Fixtures come from
scripts/generate_ts_port_fixtures.py, which asserts the leaked value is absent
from its own output, so the fixture-drift job proves provenance the way it
does for every other fixture.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN
The TUI had a QR action that answered "QR rendering is not available in this
build". The stated reason was that a QR library is not worth a dependency in a
process holding unlocked seeds — which is true, and is why this implements the
encoder instead: byte mode, versions 1-40, all four error-correction levels,
zero dependencies.
A wrong QR code is worse than no QR code. It scans cleanly, yields the wrong
secret, and the user finds out when their 2FA stops working — so this is
verified rather than eyeballed:
- The block-structure table was extracted programmatically from qrcode's
RS_BLOCK_TABLE rather than transcribed. 160 rows of three-number triples
are exactly the kind of data a human copies wrong once and never notices.
- qr.json holds reference matrices from that library, and the tests compare
EVERY module — 10 cases spanning all four EC levels and versions 1 to 17,
including past version 7 where version-information blocks start being
written.
- Generation moved into scripts/generate_ts_port_fixtures.py, so the
fixture-drift CI job proves the committed fixture came from the committed
generator, like every other fixture. Regenerating produced byte-identical
output for all pre-existing fixtures.
Two things are pinned so the comparison is between implementations rather than
between configurations. Byte mode is forced, because the reference
auto-selects alphanumeric for an all-uppercase string — a different encoding
of the same text, not a different answer. And the mask is pinned, because the
two disagree about mask SELECTION: the spec scores the finished symbol, while
the reference scores one with its format modules blanked and its dark module
light (makeImpl(test=True, ...)), which is not a legal symbol. All eight masks
produce valid scannable codes, so the selection rule is checked as a property
instead of reproducing a known quirk to make an equality assertion pass. At a
pinned mask the matrices are identical, which is what establishes that the
data encoding, Reed-Solomon, interleaving, placement, format and version
information are all correct.
In the TUI, TOTP entries encode their otpauth:// URI so an authenticator can
consume them; everything else encodes its own value. Secret Mode suppresses
the code entirely — a QR code is the secret in a form a camera reads from
across a room, so the setting that exists to keep secrets off the screen
applies here more than anywhere — and an entry too large to encode says so
rather than looking unavailable.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN
The last unported derivation, and the one the plan flagged as highest risk.
It turned out to be fully reproducible: the key material is deterministic,
EdDSA signatures are deterministic (RFC 8032), and the packet layout is
fixed -- so the armored blocks match exactly, with no OpenPGP dependency.
core/derive/pgp.ts emits the packets directly: new-format headers, the
secret-key packet (unencrypted MPI plus 16-bit checksum), the user-id
packet, and the self-certification signature with PGPy's exact subpacket
set and order, wrapped in ASCII armor with a CRC-24 footer. Verified
against PGPy across two seeds, three indices and three user-id shapes --
armored private key, armored public key, and fingerprint.
RSA is deliberately unsupported: PyCryptodome's seeded prime search is not
reproducible, so the TS port refuses the key type rather than silently
deriving a different key.
Two things worth recording from the build: the v4 fingerprint reproduces
from first principles (SHA-1 over the public-key packet), which is what
made byte-parity look feasible before committing to it; and the only
mismatch in the first run was packet framing -- PGPy uses new-format
headers where I had emitted old-format.
CLI: `entry add pgp`, reveal (armored private key), `entry pgp-public`.
Cross-impl phases A/B now cover a pgp entry in both directions, completing
coverage of every creatable entry kind.
193 core + 47 CLI tests green; 30/30 cross-implementation checks green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- core/derive/ssh.ts: BIP-85 app 32 entropy used directly as the Ed25519
private key, serialized as PKCS#8 and SubjectPublicKeyInfo PEM. Both DER
shapes are fixed for Ed25519, so they are emitted from literal prefixes
rather than adding an ASN.1 dependency to the crypto path.
- Output matches Python's `cryptography` byte-for-byte across two seeds and
three indices, including PEM line wrapping.
- CLI: `entry add ssh`, reveal (private key PEM), and `entry ssh-public`
which prints the public key without touching secret material. The
OpenSSH authorized_keys line is a TS-only convenience -- Python emits
only PEM, which ssh(1) cannot consume directly -- and is marked as such
in the compatibility matrix.
- Cross-impl phases A and B now include an ssh entry in both directions.
One harness fix: PEM values carry a trailing newline that the CLI capture
strips, so the cross-impl comparison normalizes trailing whitespace. The
unit tests still compare exactly.
170 core + 46 CLI tests green; 16/16 cross-implementation checks green.
Co-Authored-By: Claude Fable 5 <[email protected]>
This was the one remaining gap that could strand a real user: TS refused
any index below schema_version 4, which is every profile written before
tags were added.
- core/schema/migrations.ts ports seedpass.core.migrations exactly,
including the quirks: v1->v2 copies `website` to `label` and drops it
only for password entries; `kind` is NOT backfilled (Python leaves
pre-v2 entries `type`-only and falls back on read). Migration works on a
copy, so a failure can never leave a half-converted vault.
- parseVaultIndex now migrates on open and still refuses future versions;
pass {migrate:false} where silent upgrades are unwanted.
- Two bugs this surfaced: the version probe could not read a v0 index (no
schema_version field at all), and migrated legacy entries failed schema
validation because the discriminator `kind` was missing. The parser now
fills `kind` from `type` at validation time -- the same shape Python
writes for new entries, documented in the matrix.
Evidence: fixtures generated by running Python's own apply_migrations over
v0/v1/v2/v3 payloads, plus cross-impl phase E, which writes a real legacy
v2 index into a Python profile, opens it with the TS CLI, reveals a
migrated secret, and confirms Python still reads the profile afterwards.
162 core + 46 CLI tests green; 16/16 cross-implementation checks green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- core entryMod: modify_entry port with the per-kind allowed-field matrix,
policy merging, blacklisted-field cleanup and timestamp touching;
archive/restore; typed links (add/remove/get with Python's normalization,
dedupe on target/relation/note, self-link and missing-target refusal).
- Parity fixture entry_mods.json: the real Python EntryManager applies a
10-op modification sequence at a later pinned clock; TS replays the same
ops from the creation state and matches byte-for-byte.
- CLI: entry modify/archive/unarchive/links/link-add/link-remove,
entry totp-codes (explicit egress, honors --at and archived state),
util generate-password (Python parity: index 0 + gen v1 defaults, full
policy flags — v1/v2 outputs verified against password fixtures).
- capabilities: egress command list updated.
- 148 core parity tests (Node + jsdom) + 22 CLI tests green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- NIP-01 event serialization/id computation, BIP-340 signing and
verification via @noble/curves, SeedPass event builders (manifest 30070,
chunk 30071, delta 30072 with d/e tags), and REQ/EVENT/EOSE/OK/NOTICE/
CLOSED client-relay framing. Transport-agnostic; the WebSocket adapter
lands with the interface milestones.
- Cross-implementation fixtures generated through the Python side's
nostr_sdk (rust-nostr) with pinned created_at: TS ids match theirs
byte-for-byte, their signatures verify in TS, and TS-signed events
reproduce identical ids. The generator caches signatures across runs
(BIP-340 aux randomness) so fixture output stays byte-deterministic and
the CI drift job holds.
- 143 core parity tests (Node + jsdom) + 16 CLI tests green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- core entryOps: add* functions for all creatable kinds rebuild the Python
EntryManager fixture index byte-for-byte under a pinned clock — field
shapes, id allocation, independent TOTP derivation-index allocation, and
Python's ISO timestamp format all verified.
- CLI 'entry add password/totp/key-value/document/seed/managed-account/
nostr': provision-blind — returns the reference and metadata, never the
secret the entry represents; tests assert the derived password / imported
secret / child mnemonic appear nowhere in provisioning output.
- Parity bug found and fixed by the new entry_secrets fixtures: nostr ENTRY
keys derive via the default BIP-85 app 39 path (EntryManager), not app
1237 (sync client identity); the CLI had used 1237. Reveal values for
password/TOTP/nostr/seed/managed-account now match Python retrieval.
- 129 core parity tests (Node + jsdom) + 16 CLI tests green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- Portable backup (format_version 1): TS imports Python exports, re-exports
byte-identically under pinned nonce/timestamp, verifies the canonical
checksum, rejects tampering and unknown versions.
- .github/workflows/ts-parity.yml: typecheck + Node/jsdom parity suite on
js/ changes, plus a fixture-drift job that regenerates fixtures from
Python and fails if the committed ones don't match (manifest commit field
excluded by construction).
- packages/cli — seedpass-js MVP implementing plan section 9.3:
* reference-first output: entry list/get/search emit sp://entry/<id> refs
and metadata; secret-bearing fields become has_* flags
* 'entry reveal' is the only plaintext egress command
* 'use --clipboard|--exec|--stdin-to' delivers secrets to sinks; --exec
injects SEEDPASS_SECRET into the child env, never argv or stdout
* vault export/import over portable backups; capabilities --format json
* mnemonic comes from SEEDPASS_MNEMONIC env, never the command line
- CLI tests prove the blindness properties: derived password matches the v2
fixture, TOTP at pinned time, secrets absent from all default output and
from sink-delivery confirmations.
- 125 core parity tests (Node + jsdom) + 11 CLI tests green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- Snapshot model parity (kinds 30070/30071/30072): gzip chunking via
platform CompressionStream, chunk-hash verification, reassembly of
Python-produced chunks, manifest JSON schema, manifest id
HMAC-SHA256(key_index, 'manifest|'+nonce) with the key_index HKDF chain.
- Full merge_index_payloads port (modified_ts_hash_tombstone_v2):
Python-compatible canonical JSON (sort_keys + ensure_ascii) for hash
parity, equal-ts field-level union, tombstone create/prefer/apply/cap,
_sync_meta bookkeeping. Populated _system.index0 content refuses to merge
loudly (atlas-milestone work) instead of silently dropping data.
- Delta replay: encrypted V3 deltas decrypt and merge in order with
source_tag = sha256(payload)[:16]; final state matches Python.
- New fixtures: nostr_snapshot.json, sync_merge.json (8 scenarios),
delta_replay.json; generator remains byte-deterministic.
- 117 parity tests green in Node and jsdom.
Co-Authored-By: Claude Fable 5 <[email protected]>
- scripts/generate_ts_port_fixtures.py: deterministic fixtures from public
test mnemonics (passwords v1+v2, BIP-39/85, TOTP, Nostr keys, managed
seeds, fingerprints, index keys, entry schemas, V3 vault payload).
Byte-identical across runs; v1 values match the frozen entropy-integrity
vectors.
- js/: pnpm workspace with @seedpass/core and @seedpass/test-vectors.
Core implements BIP-85, password gen v1 (frozen) + v2, TOTP, Nostr keys,
fingerprints, vault index key, and V3 AES-GCM decrypt using only
noble/scure crypto + WebCrypto. Strict TS, no Node/DOM deps in core.
- 82 parity tests pass in Node and a jsdom browser-like environment.
- docs/typescript_port_compatibility_matrix.md tracks per-area status.
Co-Authored-By: Claude Fable 5 <[email protected]>
