diff --git a/TODO.md b/TODO.md index e92bf26e..39c7fe01 100644 --- a/TODO.md +++ b/TODO.md @@ -127,7 +127,32 @@ everything still open, in the order it should be tackled. ### Blockers before real secrets -- [ ] **Independent security review of the TypeScript branch.** Review +- [~] **Independent security review — restated, since there is no third party.** + The point of the gate was catching what the author cannot see, which + needs an oracle that is not the author's judgement rather than another + person. Progress and what remains: + - [x] **Differential fuzzing vs Python** (`scripts/differential_fuzz.py`), + ~10,000 cases over canonical JSON, password derivation, the sync + CRDT, index0, recovery shares, the semantic index and entry + hashing. Found a real defect: integral floats hash differently in + the two implementations, silently, which could make two clients + converge to different vaults. Also found — via mutation testing + turned on the fuzzer itself — that it could not reach the + tombstone or subject caps; fixed and re-verified. + - [ ] **Wire the fuzzer and `cross_impl_check.py` into CI.** Neither + runs automatically, and this branch has a documented history of + CI going unwatched (ts-parity red for 33 runs). A check nobody + runs is not a check. + - [ ] **Systematic mutation testing** of the security-critical modules + that have NO differential oracle because Python has no + equivalent: the session agent, the API HTTP layer, high-risk + session handling, sinks. Ad-hoc mutation was applied to every fix + in this branch, but never measured as coverage. + - [ ] **Adversarial pass over TS-only surfaces**, framed as "what does + this trust?" rather than "is this correct?" — the framing that + finds trust-boundary bugs rather than logic bugs. + Original note follows. +- [ ] **(superseded by the above) Independent security review of the TypeScript branch.** Review strategy (decided 2026-08-18): two AI families plus vectors, because they fail differently. (1) `/code-review ultra` — **blocked**: PR #989 spans main→port including beta's 150 commits (6795 files/720k lines), diff --git a/docs/typescript_cutover_plan.md b/docs/typescript_cutover_plan.md index ad18c061..b6aea815 100644 --- a/docs/typescript_cutover_plan.md +++ b/docs/typescript_cutover_plan.md @@ -21,12 +21,43 @@ P0 gate is green, Python remains the normative reference | Interactive mode (TUI) | green — legacy (v1) menu tree ported | | Packaging + release | green | | Migration + rollback story | green | -| Independent security review | **not done — the one open blocker** | +| Independent security review | **partly substituted — see below; (3) and (4) outstanding** | The gates below are green. The remaining blocker is not a gate in this list: -no third party has reviewed the code. Two rounds of subagent review found 9 -criticals/highs between them that self-review missed entirely, which is the -evidence for treating that as load-bearing rather than optional. +the code has not been reviewed by anyone who did not write it. Two rounds of +subagent review found 9 criticals/highs between them that self-review missed +entirely, which is the evidence for treating that as load-bearing rather than +optional. + +**There is no third party to hand this to.** So the blocker is restated in +terms of what it was actually for: catching what the author's mental model +misses. That does not require another person. It requires an ORACLE THAT IS +NOT THE AUTHOR'S JUDGEMENT, and the substitutes available are ranked by how +little they depend on someone already knowing what to look for: + +1. **Differential testing against a second implementation.** `scripts/ + differential_fuzz.py` feeds randomly generated inputs to both + implementations and compares canonical output. Any disagreement is a bug + in one of them, found without anyone knowing in advance which answer is + right. **Done**, ~10,000 cases across seven surfaces; it found a real + silent-divergence defect in canonical number encoding on its first + scaled run. +2. **Differential testing against a third-party reference.** Used where one + exists: the official BIP-85 spec vectors, Python's `qrcode` library for + the QR encoder, Python's `Fernet` for the partition format. +3. **Mutation testing.** Measures whether the suite would actually catch a + regression, rather than whether it passes. Applied ad hoc to every fix in + this branch; not yet systematic. +4. **Adversarial tests written from the attacker's position** — the only + technique available for surfaces the Python side does not have (the + session agent, the API's HTTP layer, high-risk session handling), where + there is no second implementation to disagree with. + +**The known blind spot of (1):** it can only find bugs where the two +implementations differ. A flaw both share by design is invisible to it — the +recovery-split secrecy defect was found by reading, not fuzzing — as is +anything TypeScript implements alone. Those need (3) and (4), which is what +remains. ## P0 gates (must be green to cut over)