diff --git a/docs/ai_agent_tui_testing.md b/docs/ai_agent_tui_testing.md index b23ff294..6e53ce14 100644 --- a/docs/ai_agent_tui_testing.md +++ b/docs/ai_agent_tui_testing.md @@ -132,6 +132,7 @@ Useful options: Coverage gate helper: - `python scripts/check_critical_coverage.py artifacts/coverage/coverage.json` +- `./scripts/tui2_coverage_gate.sh` (focused TUI v2 + service-path thresholds) - Use `--threshold PATH=PERCENT` to override defaults. - Use `--no-default-thresholds` for focused local smoke checks. diff --git a/docs/dev_control_center.md b/docs/dev_control_center.md index d923392c..41ecdee1 100644 --- a/docs/dev_control_center.md +++ b/docs/dev_control_center.md @@ -64,10 +64,9 @@ Use these docs as inputs, but treat this file as the decision layer: ## 4) Recommended Next Steps (Priority Order) -1. Strengthen testing gates by adding critical coverage thresholds for TUI v2 and key service modules used by v2. -2. Advance Nostr resilience validation with deterministic failure-mode suites (then optional real-relay soak lane). -3. Update cutover memo to reflect current reality and remaining concrete blockers. -4. Finish supply-chain readiness evidence and release-protection policy. +1. Advance Nostr resilience validation with deterministic failure-mode suites (then optional real-relay soak lane). +2. Update cutover memo to reflect current reality and remaining concrete blockers. +3. Finish supply-chain readiness evidence and release-protection policy. ## 5) Next Slice Definition (Recommended Immediate Work) @@ -100,6 +99,12 @@ Progress update (2026-03-02): - Full legacy workflow coverage closed with bug-bash evidence: - `python scripts/ai_tui2_agent_test.py --scenario extended --verbose` => PASS - `python scripts/ai_tui_agent_test.py --scenario extended --verbose` => PASS +- Coverage gates strengthened for TUI v2 critical path: + - `scripts/run_ci_tests.sh` now enables critical coverage gate by default. + - `scripts/check_critical_coverage.py` default thresholds now include `src/seedpass/core/api.py=85`. + - new focused gate `scripts/tui2_coverage_gate.sh` enforces: + - `src/seedpass/tui_v2/app.py >= 78%` + - `src/seedpass/core/api.py >= 85%` ## 6) Working Rule diff --git a/docs/security_readiness_checklist.md b/docs/security_readiness_checklist.md index cd73ac16..138429c0 100644 --- a/docs/security_readiness_checklist.md +++ b/docs/security_readiness_checklist.md @@ -10,7 +10,7 @@ Status legend: `Not Started`, `In Progress`, `Blocked`, `Done` | 4 | Backup and restore integrity validation | In Progress | Core / QA | `src/tests/test_account_roundtrip_restore.py`, `src/tests/test_portable_backup.py` | | 5 | Nostr sync security and resilience validation | In Progress | Core / QA | `src/tests/test_nostr_real.py`, `src/tests/test_nostr_index_size.py` | | 6 | Auth, lock/unlock, and access-control hardening | Done | Core | `docs/auth_lock_unlock_access_review.md`, `src/tests/test_api_new_endpoints.py`, `src/tests/test_vault_lock_flag.py`, `src/tests/test_inactivity_lock.py`, `src/tests/test_unlock_sync.py` | -| 7 | Testing gates and quality thresholds | In Progress | QA | `scripts/check_critical_coverage.py`, CI workflows | +| 7 | Testing gates and quality thresholds | In Progress | QA | `scripts/check_critical_coverage.py`, `scripts/tui2_coverage_gate.sh`, CI workflows | | 8 | Supply chain and release integrity | In Progress | DevOps | `docs/supply_chain_release_integrity.md`, `.github/workflows/release-integrity.yml`, `scripts/release_integrity.py`, `src/tests/test_release_integrity_utils.py` | | 9 | Operational readiness and incident runbooks | Not Started | Security / Ops | | | 10 | External audit and staged production rollout | Not Started | Maintainers | | diff --git a/memory-update.md b/memory-update.md index dbf9a53c..5df2316c 100644 --- a/memory-update.md +++ b/memory-update.md @@ -690,3 +690,34 @@ - `docs/dev_control_center.md` - removed parity-closure pending state - shifted priority stack to post-parity cutover readiness (testing gates, Nostr resilience, cutover memo, supply-chain readiness). + +## 2026-03-02 Testing-gate hardening: TUI v2 + service coverage thresholds +- Added dedicated focused gate script: `scripts/tui2_coverage_gate.sh`. +- Gate behavior: + - uses `.venv/bin/python` when available (fallback `python`/`python3`) + - checks Textual runtime availability before running focused TUI v2 coverage suite + - runs focused tests with coverage output: `artifacts/coverage/coverage.tui2_gate.json` + - enforces thresholds via `scripts/check_critical_coverage.py --no-default-thresholds`: + - `src/seedpass/tui_v2/app.py >= 78%` + - `src/seedpass/core/api.py >= 85%` + - writes machine-readable result to `artifacts/coverage/critical_gate.tui2.json`. +- Strengthened default critical-coverage policy: + - `scripts/check_critical_coverage.py` default thresholds now include: + - `src/seedpass/core/api.py = 85%`. +- CI runner hardening: + - `scripts/run_ci_tests.sh` now enables `CRITICAL_COVERAGE_GATE` by default (`1`). + - adds JSON output for full-suite gate: `artifacts/coverage/critical_gate.full.json`. + - adds focused TUI v2 gate call controlled by `TUI2_COVERAGE_GATE` (default `1`). + - coverage-gate Python invocation now uses `python`/`python3` fallback resolution. +- Test/doc updates: + - `src/tests/test_check_critical_coverage.py` includes assertion for new default API threshold. + - docs updated in: + - `docs/ai_agent_tui_testing.md` + - `docs/security_readiness_checklist.md` + - `docs/dev_control_center.md` +- Validation run (this slice): + - `pytest -q src/tests/test_check_critical_coverage.py` -> `9 passed` + - `./scripts/tui2_coverage_gate.sh` -> PASS + - `src/seedpass/core/api.py: 86.64%` + - `src/seedpass/tui_v2/app.py: 79.87%` + - `python3 scripts/check_critical_coverage.py artifacts/coverage/coverage.json --json-output artifacts/coverage/critical_gate.full.json` -> PASS. diff --git a/scripts/check_critical_coverage.py b/scripts/check_critical_coverage.py index 29afe3f7..abd09978 100644 --- a/scripts/check_critical_coverage.py +++ b/scripts/check_critical_coverage.py @@ -19,6 +19,7 @@ from pathlib import Path DEFAULT_THRESHOLDS: dict[str, float] = { "src/main.py": 50.0, "src/seedpass/core/manager.py": 60.0, + "src/seedpass/core/api.py": 85.0, "src/seedpass/core/encryption.py": 80.0, "src/seedpass/core/migrations.py": 90.0, } diff --git a/scripts/run_ci_tests.sh b/scripts/run_ci_tests.sh index 0682861a..6b69b029 100755 --- a/scripts/run_ci_tests.sh +++ b/scripts/run_ci_tests.sh @@ -5,6 +5,11 @@ mkdir -p artifacts/coverage ./scripts/tui2_check_smoke.sh +py_bin="python" +if ! command -v "$py_bin" >/dev/null 2>&1; then + py_bin="python3" +fi + if [[ "${DETERMINISM_GATE:-1}" == "1" ]]; then ./scripts/run_determinism_tests.sh fi @@ -48,7 +53,12 @@ if [[ $status -eq 124 ]]; then tail -n 20 pytest.log exit 1 fi -if [[ $status -eq 0 && "${CRITICAL_COVERAGE_GATE:-0}" == "1" ]]; then - python scripts/check_critical_coverage.py artifacts/coverage/coverage.json +if [[ $status -eq 0 && "${CRITICAL_COVERAGE_GATE:-1}" == "1" ]]; then + "$py_bin" scripts/check_critical_coverage.py \ + artifacts/coverage/coverage.json \ + --json-output artifacts/coverage/critical_gate.full.json +fi +if [[ $status -eq 0 && "${TUI2_COVERAGE_GATE:-1}" == "1" ]]; then + ./scripts/tui2_coverage_gate.sh fi exit $status diff --git a/scripts/tui2_coverage_gate.sh b/scripts/tui2_coverage_gate.sh new file mode 100755 index 00000000..dba5f988 --- /dev/null +++ b/scripts/tui2_coverage_gate.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +set -euo pipefail + +mkdir -p artifacts/coverage + +if [[ -x ".venv/bin/python" ]]; then + py_bin=".venv/bin/python" +elif command -v python >/dev/null 2>&1; then + py_bin="python" +else + py_bin="python3" +fi + +if ! "$py_bin" - <<'PY' >/dev/null 2>&1 +import importlib.util +raise SystemExit(0 if importlib.util.find_spec("textual") else 1) +PY +then + if [[ "${REQUIRE_TEXTUAL:-0}" == "1" ]]; then + echo "Error: textual runtime is unavailable; cannot run TUI v2 coverage gate." >&2 + exit 1 + fi + echo "tui2 coverage gate: skipped (textual runtime unavailable)" + exit 0 +fi + +"$py_bin" -m pytest -q \ + --cov=src \ + --cov-report=json:artifacts/coverage/coverage.tui2_gate.json \ + src/tests/test_tui_v2_helpers.py \ + src/tests/test_tui_v2_textual_interactions.py \ + src/tests/test_tui_v2_action_matrix.py \ + src/tests/test_tui_v2_parity_scenarios.py \ + src/tests/test_core_api_services.py + +"$py_bin" scripts/check_critical_coverage.py \ + artifacts/coverage/coverage.tui2_gate.json \ + --no-default-thresholds \ + --threshold src/seedpass/tui_v2/app.py=78 \ + --threshold src/seedpass/core/api.py=85 \ + --json-output artifacts/coverage/critical_gate.tui2.json + +echo "tui2 coverage gate: ok" diff --git a/src/tests/test_check_critical_coverage.py b/src/tests/test_check_critical_coverage.py index 32428e8e..d0f15760 100644 --- a/src/tests/test_check_critical_coverage.py +++ b/src/tests/test_check_critical_coverage.py @@ -23,6 +23,10 @@ def _load_module(): coverage_gate = _load_module() +def test_default_thresholds_include_tui2_service_module(): + assert coverage_gate.DEFAULT_THRESHOLDS["src/seedpass/core/api.py"] == 85.0 + + def test_parse_threshold_overrides_valid(): parsed = coverage_gate._parse_threshold_overrides( ["src/main.py=40", "src/seedpass/core/manager.py=55.5"]