From 864229bad185a26afc3a394d808ce14c69e486ce Mon Sep 17 00:00:00 2001 From: thePR0M3TH3AN Date: Thu, 20 Aug 2026 13:21:38 -0400 Subject: [PATCH] Add local CI: a preflight that refuses to lie, and a gate under a minute MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Today cost roughly sixteen push-and-wait cycles discovering serially what one local run could have surfaced at once. This is the local half of that, built around one rule: a local gate is worth having only if it reaches the same verdict CI would. THE PREFLIGHT IS THE POINT scripts/ci_preflight.py checks that this machine agrees with the lockfiles, and refuses everything downstream when it does not. That is not theoretical hygiene. Run right now it says: black 26.1.0 != locked 24.10.0 pytest 9.0.2 != locked 8.4.1 coverage 7.10.1 != locked 7.10.2 which is exactly today's failure: `poetry run black --check .` passes here and fails on GitHub, and an earlier commit titled "satisfy the black gate" had already reformatted five files with the WRONG black, making the gate worse while looking like a fix. A green local gate that disagrees with CI is not a weak check, it is a harmful one, because people stop reading the real result. It is stdlib-only and run by the system interpreter, so it works when the environment it is checking is broken -- which is when it is needed. And it only ever CHECKS. scripts/dev_sync.sh repairs. A gate that silently fixes your environment hides the drift it exists to surface. TOOLS COME FROM THE REPOSITORY, NEVER FROM PATH Every invocation is `poetry run ...` or `pnpm -C js ...`. A bare tool name is precisely how a local gate ends up disagreeing with CI, so the preflight inspects those environments rather than PATH. THE BUDGET IS THE DESIGN Measured here: preflight 2.6s, lock check 0.6s, black 2.5s, typecheck 7.3s, core vitest 24.0s, determinism subset 5.4s — about 44s. What that excludes was measured too: the cli suite alone is 176.3s, three times the whole budget, and full pytest is ~350s. Those go in ci_parity.sh, which is what you run before calling a branch ready. A hook that takes minutes becomes `git push --no-verify` inside a week, at which point it is a control everyone believes in and nobody runs. The cli suite is slow for a good reason — real daemons, real sockets, real subprocesses instead of mocks — so the answer is to run it less often, not to make it lie faster. THE SMOKE TEST IS NOW NODE, NOT BASH js/scripts/smoke-cli.mjs replaces an inline shell test that used mktemp, export and `test -n`, and therefore could only run on Unix — leaving the one artifact users actually execute never smoke-tested on Windows. Every Windows failure today was a Unix assumption leaking. It also checks more than the original: that the bundle runs with no node_modules in sight, that the secret honours the requested length, and that revealing twice yields the same value, because determinism is the product rather than an incidental property. The lint step is written but commented out: `pnpm -C js lint` still exits non-zero on 35 known assertions, and shipping a red check into the push gate is how gates get routed around. It goes in when it goes green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN --- .githooks/pre-push | 13 +++ js/package.json | 9 +- js/scripts/smoke-cli.mjs | 120 ++++++++++++++++++++++++++ scripts/ci_parity.sh | 51 +++++++++++ scripts/ci_preflight.py | 181 +++++++++++++++++++++++++++++++++++++++ scripts/ci_prepush.sh | 78 +++++++++++++++++ scripts/dev_sync.sh | 24 ++++++ 7 files changed, 474 insertions(+), 2 deletions(-) create mode 100755 .githooks/pre-push create mode 100644 js/scripts/smoke-cli.mjs create mode 100755 scripts/ci_parity.sh create mode 100755 scripts/ci_preflight.py create mode 100755 scripts/ci_prepush.sh create mode 100755 scripts/dev_sync.sh diff --git a/.githooks/pre-push b/.githooks/pre-push new file mode 100755 index 00000000..3154f7c1 --- /dev/null +++ b/.githooks/pre-push @@ -0,0 +1,13 @@ +#!/bin/sh +# Version-controlled so every clone gets the same gate. Enable once with: +# +# git config core.hooksPath .githooks +# +# Budget is the design: this must stay well under a minute, because a hook +# that takes minutes becomes `git push --no-verify` and then it is a control +# everyone believes in and nobody runs. Anything slower belongs in +# scripts/ci_parity.sh, which you run before calling a branch ready. +# +# To skip deliberately (a docs-only push, say): git push --no-verify +set -e +exec "$(dirname "$0")/../scripts/ci_prepush.sh" diff --git a/js/package.json b/js/package.json index cd352bf9..b5e38dfd 100644 --- a/js/package.json +++ b/js/package.json @@ -6,12 +6,17 @@ "scripts": { "test": "pnpm -r test", "typecheck": "pnpm -r typecheck", - "lint": "oxlint" + "lint": "oxlint", + "smoke": "node scripts/smoke-cli.mjs", + "build": "pnpm --filter @seedpass/cli build" }, "packageManager": "pnpm@11.21.0", "devDependencies": { "@oxlint/plugins": "1.78.0", "oxlint": "1.78.0" }, - "type": "module" + "type": "module", + "engines": { + "node": ">=22" + } } diff --git a/js/scripts/smoke-cli.mjs b/js/scripts/smoke-cli.mjs new file mode 100644 index 00000000..433b0e74 --- /dev/null +++ b/js/scripts/smoke-cli.mjs @@ -0,0 +1,120 @@ +/** + * Smoke-test the shipped CLI bundle, on any operating system. + * + * WHY THIS IS NOT A SHELL SCRIPT + * + * The version this replaces lived inline in ts-parity.yml and used mktemp, + * export and `test -n`, so it could only ever run on Unix -- which meant the + * one artifact users actually execute was never smoke-tested on Windows at + * all. That mattered: on 2026-08-20 every Windows CI failure was a Unix + * assumption leaking, from MSYS2 shadowing `python` to pwsh swallowing exit + * codes to shlex eating the backslashes out of Windows paths. Written in + * Node, the same file runs on Linux, macOS, Windows, GitHub and a laptop. + * + * WHAT IT PROVES + * + * 1. The bundle runs with NO node_modules anywhere near it. It executes from + * a fresh temp directory, so a dependency that failed to bundle shows up + * here rather than in a user's install. + * 2. A vault can be created and an entry added and revealed end to end. + * 3. The revealed secret is DETERMINISTIC -- revealing twice gives the same + * value. For a deterministic password manager that is the product, not an + * incidental property, and "some output appeared" would not have caught a + * derivation that silently changed. + * 4. The secret honours the requested length. + */ + +import { spawnSync } from "node:child_process"; +import { mkdtempSync, existsSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; +import process from "node:process"; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const BUNDLE = resolve(HERE, "..", "packages", "cli", "dist", "seedpass-js.mjs"); + +// The 12-word all-`abandon` vector, used throughout the fixtures. +const MNEMONIC = + "abandon abandon abandon abandon abandon abandon abandon abandon " + + "abandon abandon abandon about"; +const PASSWORD = "ci-smoke-password"; +const LENGTH = 16; + +function fail(message) { + console.error(`\nsmoke-cli: ${message}`); + process.exit(1); +} + +/** Run the bundle from `cwd`, which deliberately has no node_modules. */ +function cli(args, { cwd, env, expectOk = true }) { + const result = spawnSync(process.execPath, [BUNDLE, ...args], { + cwd, + env, + encoding: "utf8", + // No shell: argument quoting differs between cmd.exe and sh, and this + // test exists precisely to stop shell differences from mattering. + shell: false, + }); + if (result.error) fail(`could not run the bundle: ${result.error.message}`); + if (expectOk && result.status !== 0) { + fail( + `\`${args.join(" ")}\` exited ${result.status}\n` + + `stdout: ${result.stdout}\nstderr: ${result.stderr}`, + ); + } + return result; +} + +if (!existsSync(BUNDLE)) { + fail(`no bundle at ${BUNDLE} — run \`pnpm -C js build\` first`); +} + +const appDir = mkdtempSync(join(tmpdir(), "seedpass-smoke-app-")); +// Somewhere with no package.json and no node_modules above it, so a missing +// bundled dependency cannot be papered over by the workspace's own tree. +const runDir = mkdtempSync(join(tmpdir(), "seedpass-smoke-run-")); + +try { + const bare = { ...process.env, SEEDPASS_APP_DIR: appDir }; + delete bare.SEEDPASS_MNEMONIC; + delete bare.SEEDPASS_PASSWORD; + + const version = cli(["--version"], { cwd: runDir, env: bare }); + if (!/\d+\.\d+/.test(version.stdout)) { + fail(`--version printed something unversion-like: ${version.stdout.trim()}`); + } + cli(["capabilities"], { cwd: runDir, env: bare }); + + const env = { + ...bare, + SEEDPASS_MNEMONIC: MNEMONIC, + SEEDPASS_PASSWORD: PASSWORD, + }; + + cli(["fingerprint", "add", "--name", "ci"], { cwd: runDir, env }); + cli(["entry", "add", "password", "ci-site", "--length", String(LENGTH)], { + cwd: runDir, + env, + }); + + const first = cli(["entry", "reveal", "ci-site"], { cwd: runDir, env }).stdout.trim(); + if (!first) fail("revealing the entry produced no output"); + if (first.length !== LENGTH) { + fail(`asked for a ${LENGTH}-character password and got ${first.length}`); + } + + // The product is determinism. Same seed, same entry, same secret. + const second = cli(["entry", "reveal", "ci-site"], { cwd: runDir, env }).stdout.trim(); + if (first !== second) { + fail("revealing the same entry twice produced two different secrets"); + } + + console.log( + `smoke-cli: ok — bundle runs standalone, ${LENGTH}-character secret ` + + `derived deterministically`, + ); +} finally { + rmSync(appDir, { recursive: true, force: true }); + rmSync(runDir, { recursive: true, force: true }); +} diff --git a/scripts/ci_parity.sh b/scripts/ci_parity.sh new file mode 100755 index 00000000..5ca0d861 --- /dev/null +++ b/scripts/ci_parity.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# The gate that runs before a branch is called ready. Minutes, not seconds. +# +# ci_prepush.sh answers "is this push going to embarrass me". This answers +# "do the two implementations still agree, and does the thing we ship still +# work" -- which is the question SeedPass actually cares about, because the +# TypeScript port only has value if it derives byte-identical secrets to the +# Python it replaces. +# +# Everything expensive lives here on purpose. The cli suite spawns real +# daemons and real subprocesses; the fuzzer runs both implementations over +# generated cases; the packaging check installs the tarball the way a user +# would. None of that belongs on the path of every `git push`. +# +# Fixed seeds, deliberately. GitHub additionally runs an exploratory seed +# derived from the run id so the explored space grows over time; locally you +# want the same cases every run, so a failure is reproducible rather than a +# rumour. +set -euo pipefail +cd "$(dirname "$0")/.." + +step() { printf "\n\033[1m==> %s\033[0m\n" "$1"; } + +step "everything in ci:prepush" +./scripts/ci_prepush.sh + +step "javascript: full test suite (cli included, node + jsdom)" +pnpm -C js test + +step "javascript: build and smoke the shipped bundle" +pnpm -C js build +node js/scripts/smoke-cli.mjs + +step "python: full test suite" +poetry run pytest -q + +step "parity: python <-> typescript cross-implementation" +poetry run python scripts/cross_impl_check.py + +step "parity: fixtures still come from the committed generator" +poetry run python scripts/generate_ts_port_fixtures.py +git diff --exit-code -- \ + js/packages/test-vectors/fixtures \ + ':(exclude)js/packages/test-vectors/fixtures/manifest.json' + +step "parity: differential fuzz, fixed seeds" +for seed in 1 2 3; do + poetry run python scripts/differential_fuzz.py --cases 150 --seed "$seed" +done + +printf "\n\033[32mci:parity passed.\033[0m This branch is ready for review.\n" diff --git a/scripts/ci_preflight.py b/scripts/ci_preflight.py new file mode 100755 index 00000000..a42bba86 --- /dev/null +++ b/scripts/ci_preflight.py @@ -0,0 +1,181 @@ +#!/usr/bin/env python3 +"""Prove the local toolchain matches the lockfiles, before anything trusts it. + +WHY THIS EXISTS + +A local CI gate is only worth running if it reaches the same verdict CI would. +On 2026-08-20 this repository's local environment disagreed with its own +lockfile on every pinned tool: + + black local 26.1.0 locked 24.10.0 + pytest local 9.0.2 locked 8.4.1 + coverage local 7.10.1 locked 7.10.2 + +`poetry run black --check .` passed locally and failed on GitHub, and an +earlier commit titled "satisfy the black gate" had already reformatted five +files with the WRONG black -- making the gate worse while looking like a fix. +A green local gate that disagrees with CI is not a weak check, it is a +harmful one: people stop reading the real result. + +So this runs first, and everything downstream is meaningless without it. + +TWO RULES IT ENFORCES + +1. Tools are executed through the repository-controlled environment + (`poetry run ...`, `pnpm exec ...`), never by bare name off PATH. This + script therefore inspects those environments, not PATH. +2. It CHECKS and never REPAIRS. `scripts/dev_sync.sh` repairs. A gate that + silently mutates your environment hides the drift it exists to surface. + +It is deliberately stdlib-only and run by the system interpreter, because it +has to work when the environment it is checking is broken -- which is exactly +when it is needed. +""" + +from __future__ import annotations + +import json +import re +import subprocess +import sys +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +JS = REPO / "js" + +# Tools whose version changes what CI decides. A mismatch here means a local +# run and a CI run can disagree about the same source. +VERDICT_TOOLS = ("black", "pytest", "coverage") + +GREEN, RED, DIM, RESET = "\033[32m", "\033[31m", "\033[2m", "\033[0m" +if not sys.stdout.isatty(): + GREEN = RED = DIM = RESET = "" + +failures: list[str] = [] + + +def ok(label: str, detail: str = "") -> None: + print(f" {GREEN}ok{RESET} {label}" + (f" {DIM}{detail}{RESET}" if detail else "")) + + +def bad(label: str, detail: str, fix: str) -> None: + print(f" {RED}FAIL{RESET} {label} {DIM}{detail}{RESET}") + failures.append(f"{label}: {detail}\n fix: {fix}") + + +def run(cmd: list[str], cwd: Path = REPO) -> tuple[int, str]: + try: + p = subprocess.run(cmd, cwd=cwd, capture_output=True, text=True, timeout=180) + return p.returncode, (p.stdout + p.stderr).strip() + except FileNotFoundError: + return 127, f"{cmd[0]} not found" + except subprocess.TimeoutExpired: + return 124, "timed out" + + +def parse_lock_versions(lock: Path) -> dict[str, str]: + """name -> version from poetry.lock, without needing a toml parser.""" + out: dict[str, str] = {} + name = None + for line in lock.read_text().splitlines(): + if m := re.match(r'^name = "([^"]+)"', line): + name = m.group(1) + elif name and (m := re.match(r'^version = "([^"]+)"', line)): + out[name] = m.group(1) + name = None + return out + + +def check_python_env() -> None: + print("\nPython (poetry-controlled environment)") + + code, out = run(["poetry", "check", "--lock"]) + if code == 0: + ok("poetry.lock agrees with pyproject.toml") + else: + bad( + "poetry.lock disagrees with pyproject.toml", + out.splitlines()[-1] if out else "", + "poetry lock", + ) + + locked = parse_lock_versions(REPO / "poetry.lock") + code, out = run( + ["poetry", "run", "python", "-c", + "import json;from importlib.metadata import distributions;" + "print(json.dumps({d.metadata['Name'].lower(): d.version " + "for d in distributions() if d.metadata['Name']}))"] + ) + if code != 0: + bad("cannot inspect the poetry environment", out.splitlines()[-1] if out else "", + "./scripts/dev_sync.sh") + return + installed = json.loads(out.splitlines()[-1]) + + drift = [] + for tool in VERDICT_TOOLS: + want, have = locked.get(tool), installed.get(tool) + if want is None: + continue + if have is None: + drift.append(f"{tool} missing (locked {want})") + elif have != want: + drift.append(f"{tool} {have} != locked {want}") + if drift: + bad("installed tools disagree with poetry.lock", "; ".join(drift), + "./scripts/dev_sync.sh") + else: + ok("black, pytest and coverage match poetry.lock", + " ".join(f"{t}={locked.get(t,'?')}" for t in VERDICT_TOOLS)) + + +def check_js_env() -> None: + print("\nJavaScript (pnpm-controlled environment)") + pkg = json.loads((JS / "package.json").read_text()) + + want_node = str(pkg.get("engines", {}).get("node", "")).lstrip("^>=~ ") + have_node = sys.version and run(["node", "--version"])[1].lstrip("v") + if not want_node: + bad("js/package.json declares no engines.node", "nothing to check against", + 'add "engines": {"node": ">=22"} to js/package.json') + elif have_node.split(".")[0] != want_node.split(".")[0]: + bad("node major differs from engines.node", f"have {have_node}, want {want_node}", + f"install Node {want_node.split('.')[0]}") + else: + ok("node matches engines.node", f"v{have_node}") + + want_pnpm = str(pkg.get("packageManager", "")).removeprefix("pnpm@") + have_pnpm = run(["pnpm", "--version"])[1] + if want_pnpm and have_pnpm != want_pnpm: + bad("pnpm differs from packageManager", f"have {have_pnpm}, want {want_pnpm}", + "corepack enable && corepack prepare --activate") + else: + ok("pnpm matches packageManager", have_pnpm) + + # --dry-run so the check cannot become the repair. + code, out = run(["pnpm", "install", "--frozen-lockfile", "--dry-run"], cwd=JS) + if code == 0: + ok("node_modules satisfies pnpm-lock.yaml") + else: + bad("pnpm-lock.yaml not satisfied", out.splitlines()[-1] if out else "", + "./scripts/dev_sync.sh") + + +def main() -> int: + print("SeedPass CI preflight — does this machine agree with the lockfiles?") + check_python_env() + check_js_env() + + print() + if failures: + print(f"{RED}preflight failed{RESET} — a local gate run now could disagree with CI.\n") + for f in failures: + print(f" - {f}") + print("\nRepair with: ./scripts/dev_sync.sh") + return 1 + print(f"{GREEN}preflight passed{RESET} — local checks can be trusted to match CI.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/ci_prepush.sh b/scripts/ci_prepush.sh new file mode 100755 index 00000000..fc84db00 --- /dev/null +++ b/scripts/ci_prepush.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# The gate that runs before every push. Target: well under a minute. +# +# WHAT IS IN HERE AND WHY THAT LIST +# +# The budget is the design constraint, not an afterthought. A hook that takes +# minutes becomes `git push --no-verify` within a week, at which point it is a +# ceremonial control that everyone believes in and nobody runs. So this holds +# only checks that are cheap AND catch things CI would otherwise catch late. +# +# Measured on this machine, 2026-08-20: +# +# preflight 2.6s +# poetry check --lock 0.6s +# black --check 2.5s +# oxlint 0.9s (not enabled yet, see below) +# tsc --noEmit (workspace) 7.3s +# pytest determinism subset 5.4s +# vitest core (node) 24.0s +# ----- +# ~44s +# +# What is deliberately NOT here: +# +# vitest cli 176.3s <- alone, three times the budget +# pytest (full) ~350s +# cross-impl, fuzz, packaging smoke +# +# Those live in ci_parity.sh, which is what you run before calling a branch +# ready rather than before every push. The cli suite is slow for a good +# reason -- it spawns real daemons, real sockets and real subprocesses rather +# than mocking them -- and that is worth keeping, not worth waiting on twenty +# times a day. +# +# EVERY TOOL IS INVOKED THROUGH THE REPOSITORY ENVIRONMENT. `poetry run ...` +# and `pnpm -C js ...`, never a bare `black` or `vitest` off PATH. That rule is +# the whole reason preflight runs first: a bare tool name is how a local gate +# ends up passing while CI fails. +set -euo pipefail +cd "$(dirname "$0")/.." + +step() { printf "\n\033[1m==> %s\033[0m\n" "$1"; } + +step "preflight: does this machine agree with the lockfiles?" +python3 scripts/ci_preflight.py + +step "python: lockfile consistency" +poetry check --lock + +step "python: formatting (the pinned black, not whatever is on PATH)" +poetry run black --check . + +# NOT YET. `pnpm -C js lint` currently exits non-zero on 35 known chained type +# assertions -- one design problem (VaultIndex does not model what a vault +# index contains) counted 35 times. Putting a red check in the push gate is +# how a gate becomes something people route around, so it goes in when it goes +# green, not before. Track it with: pnpm -C js lint +# +# step "javascript: lint" +# pnpm -C js lint + +step "javascript: typecheck" +pnpm -C js typecheck + +step "javascript: core tests" +pnpm -C js --filter @seedpass/core test:node + +step "python: deterministic artifact regression" +poetry run pytest -q src/tests/test_deterministic_artifact_regression.py + +if command -v actionlint >/dev/null 2>&1; then + step "workflows: actionlint" + actionlint +else + printf "\n\033[2m(actionlint not installed — skipping workflow lint)\033[0m\n" +fi + +printf "\n\033[32mci:prepush passed.\033[0m Run ./scripts/ci_parity.sh before calling a branch ready.\n" diff --git a/scripts/dev_sync.sh b/scripts/dev_sync.sh new file mode 100755 index 00000000..e36b811a --- /dev/null +++ b/scripts/dev_sync.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +# Bring this machine's toolchains into agreement with the lockfiles. +# +# The counterpart to scripts/ci_preflight.py, which only ever CHECKS. The +# split is deliberate: a gate that silently repairs your environment hides the +# drift it exists to surface, and you find out on GitHub instead. +# +# This one mutates. `poetry sync` will DOWNGRADE packages that are ahead of +# poetry.lock -- that is the point, since the lockfile is what CI installs. +set -euo pipefail +cd "$(dirname "$0")/.." + +echo "==> poetry sync (installs exactly poetry.lock, removing anything else)" +poetry sync + +echo "==> corepack enable (pins pnpm to package.json's packageManager)" +corepack enable + +echo "==> pnpm install --frozen-lockfile" +pnpm -C js install --frozen-lockfile + +echo +echo "Re-checking:" +exec python3 scripts/ci_preflight.py