Add robust AI TUI harness, docs roadmap, and lock/unlock fix

This commit is contained in:
thePR0M3TH3AN committed 2026-02-23 14:35:40 -05:00
1 parent 60d9df5489
commit dd7290e83b
12 files changed
+1159 -71

No files matched your search

+5
View File
@@ -668,6 +668,11 @@ pip install --require-hashes -r requirements.lock
pytest -vv
```
For AI-agent driven interactive testing and current test command tiers, see:
- `docs/ai_agent_tui_testing.md`
- `docs/agent_testing_roadmap.md`
### Exploring Nostr Index Size Limits
`test_nostr_index_size.py` demonstrates how SeedPass rotates snapshots after too many delta events. Each chunk is limited to 50 KB, so the test gradually grows the vault to observe when a new snapshot is triggered. Use the `NOSTR_TEST_DELAY` environment variable to control the delay between publishes when experimenting with large vaults.
+73
View File
@@ -0,0 +1,73 @@
# Agent Testing Roadmap
This roadmap tracks how AI-agent driven testing should evolve for SeedPass.
## Current Baseline (Implemented)
The current baseline is available via:
- `python scripts/ai_tui_agent_test.py`
- `docs/ai_agent_tui_testing.md`
Current strengths:
- deterministic end-to-end TUI flow in isolated `HOME`
- structured artifacts (`report.json`, clean/raw transcripts)
- coverage of onboarding, invalid inputs, add/retrieve/search/list/modify/archive, 2FA display, settings toggles, lock/unlock, stats, graceful exit
Known limitations:
- single large scenario can be harder to triage than smaller suites
- prompt matching is text-driven and sensitive to wording changes
- no dedicated “stress mode” for repeated random-but-seeded menu fuzzing yet
## Near-Term Roadmap (Next 1-2 cycles)
1. Split harness into scenario profiles:
- `core`: fast smoke path
- `extended`: all entry types + settings paths
- `stress`: repeated deterministic negative-input cycles
2. Add stricter artifact schema checks:
- fail when unknown coverage keys appear
- include app version, git SHA, and runtime metadata in `report.json`
3. Add targeted assertions for critical post-conditions:
- archive state consistency after restore
- lock/unlock recovers all managers and key hierarchy
4. Add a compact summary command for agents:
- parse latest report and print one-line status + failed checkpoints
## Mid-Term Roadmap
1. Differential transcript checks:
- compare current clean transcript against approved baselines
- allow configurable “soft differences” (timestamps, progress bars)
2. Seeded input-fuzz campaigns:
- deterministic generation of invalid menu/index/text inputs
- reproducible runs via fixed RNG seed in report metadata
3. Failure clustering:
- classify failures into prompt drift, logic errors, state corruption, crash
4. CI lane for agent harness:
- run `core` on every PR
- run `extended` on merge queue/nightly
## Long-Term Roadmap
1. Coverage-driven TUI planning:
- map harness steps to code regions and prioritize unexercised paths
2. Stateful model checking for menu transitions:
- verify reachable/valid transitions and back-navigation invariants
3. Multi-profile and migration scenario pack:
- legacy index import/migration and cross-profile operations in one harness family
4. Security-oriented interactive tests:
- secret-mode leakage checks in output streams
- clipboard behavior assertions in controlled environments
## Operating Rules for Agents
1. Prefer deterministic inputs and stable prompt matching.
2. Keep each new checkpoint tied to a concrete user-visible behavior.
3. Add regression tests in `src/tests/` for every discovered logic bug.
4. Treat harness failures as either:
- product defect to fix, or
- harness synchronization defect to harden.
5. Record the final decision in PR notes with report paths.
+119
View File
@@ -0,0 +1,119 @@
# AI Agent TUI Testing
This document defines a repeatable, methodical TUI test workflow for AI agents.
## Goal
Run SeedPass as a real interactive user and produce deterministic artifacts:
1. Step-level pass/fail.
2. Coverage checkpoints for major TUI features.
3. Full transcript for debugging and regression comparison.
## Runner
Use:
```bash
source .venv/bin/activate
python scripts/ai_tui_agent_test.py
```
The runner:
- creates an isolated temporary `HOME`
- launches `src/main.py --no-clipboard`
- executes an end-to-end scenario through onboarding and major menu flows
- writes artifacts to `artifacts/agent_tui_test/<timestamp>/`
## Agent Commands Today
Use these command tiers when agents are validating changes:
1. Fast targeted checks (recommended during iteration):
```bash
source .venv/bin/activate
pytest -q src/tests/test_manager_add_password.py src/tests/test_archive_restore.py src/tests/test_stats_screen.py src/tests/test_bip85_init.py
python scripts/ai_tui_agent_test.py
```
2. Full regression check (before merge):
```bash
source .venv/bin/activate
pytest -q
python scripts/ai_tui_agent_test.py
```
3. Stability sweep for TUI prompt timing:
```bash
source .venv/bin/activate
python scripts/ai_tui_agent_test.py && python scripts/ai_tui_agent_test.py && python scripts/ai_tui_agent_test.py
```
## Artifacts
Each run writes:
- `report.json`: status, step results, coverage points, failure info.
- `transcript.clean.txt`: ANSI-stripped terminal output.
- `transcript.raw.txt`: raw terminal output with ANSI.
## Current Coverage Points
The harness marks these checkpoints as `true` only when observed:
- `startup_onboarding`
- `main_menu`
- `invalid_input_resilience`
- `add_password_quick`
- `invalid_length_validation`
- `add_totp`
- `add_all_entry_types`
- `retrieve_entry`
- `search_entries`
- `list_entries`
- `modify_entry`
- `archive_restore`
- `totp_codes_view`
- `settings_toggles_lock_unlock`
- `settings_stats`
- `graceful_exit`
## CI/Agent Use Pattern
Recommended agent sequence:
1. `python scripts/ai_tui_agent_test.py`
2. Parse `report.json`
3. Fail task if `status != "passed"` or any checkpoint is `false`
4. If failed, inspect `transcript.clean.txt` for prompt-flow drift or regressions
## Optional Flags
```bash
python scripts/ai_tui_agent_test.py --help
```
Useful options:
- `--timeout`: per-prompt timeout
- `--password`: fixed password used during test
- `--output-dir`: custom artifact root
- `--keep-home`: preserve generated profile state for local debugging
- `--verbose`: print step progress
## Extending Coverage
When adding new TUI flows:
1. Add deterministic steps to `scripts/ai_tui_agent_test.py`.
2. Add a new coverage checkpoint key.
3. Assert on concrete prompt/output text, not timing.
4. Keep inputs deterministic and avoid clipboard-dependent behavior.
## Roadmap
Future testing milestones are tracked in `docs/agent_testing_roadmap.md`.
+713
View File
@@ -0,0 +1,713 @@
#!/usr/bin/env python3
"""
Deterministic AI-agent harness for SeedPass TUI testing.
This script drives the interactive CLI like a user and records:
1. step-level pass/fail status
2. coverage checkpoints for key menu flows
3. full terminal transcript
Usage example:
source .venv/bin/activate
python scripts/ai_tui_agent_test.py
"""
from __future__ import annotations
import argparse
import json
import os
import pty
import re
import select
import shutil
import subprocess
import sys
import tempfile
import time
from dataclasses import dataclass, asdict
from datetime import datetime, UTC
from pathlib import Path
from typing import Pattern
ANSI_RE = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")
@dataclass
class StepResult:
step_id: str
description: str
status: str
duration_sec: float
detail: str | None = None
class TUIHarnessError(RuntimeError):
"""Raised when the interactive harness cannot advance as expected."""
class TUIRunner:
def __init__(
self, proc: subprocess.Popen[bytes], master_fd: int, read_chunk: int = 4096
):
self.proc = proc
self.master_fd = master_fd
self.read_chunk = read_chunk
self.raw_output = ""
self.clean_output = ""
self.search_pos = 0
self._fd_closed = False
def _refresh_clean_output(self) -> None:
self.clean_output = ANSI_RE.sub("", self.raw_output)
def _read_available(self, timeout: float) -> bool:
if self._fd_closed:
return False
ready, _, _ = select.select([self.master_fd], [], [], timeout)
if not ready:
return False
try:
chunk = os.read(self.master_fd, self.read_chunk)
except OSError:
return False
if not chunk:
return False
self.raw_output += chunk.decode("utf-8", errors="ignore")
self._refresh_clean_output()
return True
def wait_for(self, pattern: str | Pattern[str], timeout: float) -> re.Match[str]:
end = time.monotonic() + timeout
compiled = re.compile(pattern) if isinstance(pattern, str) else pattern
while time.monotonic() < end:
match = compiled.search(self.clean_output, self.search_pos)
if match:
self.search_pos = match.end()
return match
if self.proc.poll() is not None:
raise TUIHarnessError(
f"Process exited before pattern appeared: {compiled.pattern!r}"
)
self._read_available(0.2)
preview = self.clean_output[max(0, len(self.clean_output) - 1200) :]
raise TUIHarnessError(
f"Timed out waiting for pattern: {compiled.pattern!r}\n"
f"Recent output:\n{preview}"
)
def try_wait_for(self, pattern: str | Pattern[str], timeout: float) -> bool:
try:
self.wait_for(pattern, timeout)
return True
except TUIHarnessError:
return False
def sendline(self, value: str = "") -> None:
os.write(self.master_fd, (value + "\n").encode("utf-8"))
def expect_and_send(
self, pattern: str | Pattern[str], value: str = "", timeout: float = 20.0
) -> None:
self.wait_for(pattern, timeout)
self.sendline(value)
def pump(self, duration: float = 0.2) -> None:
self._read_available(duration)
def close(self, kill_after: float = 1.5) -> int:
end = time.monotonic() + kill_after
while time.monotonic() < end:
code = self.proc.poll()
if code is not None:
if not self._fd_closed:
try:
os.close(self.master_fd)
except Exception:
pass
self._fd_closed = True
return code
self.pump(0.1)
if not self._fd_closed:
try:
os.close(self.master_fd)
except Exception:
pass
self._fd_closed = True
self.proc.kill()
return self.proc.wait(timeout=2.0)
def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Run deterministic, repeatable TUI coverage checks for SeedPass."
)
parser.add_argument(
"--repo-root",
type=Path,
default=Path(__file__).resolve().parents[1],
help="SeedPass repository root (default: script parent repo).",
)
parser.add_argument(
"--python-bin",
default=sys.executable,
help="Python executable used to launch SeedPass (default: current interpreter).",
)
parser.add_argument(
"--timeout",
type=float,
default=25.0,
help="Per-prompt timeout in seconds.",
)
parser.add_argument(
"--password",
default="AgentPass123!",
help="Master password used during the run.",
)
parser.add_argument(
"--output-dir",
type=Path,
default=Path("artifacts/agent_tui_test"),
help="Directory for transcript/report artifacts.",
)
parser.add_argument(
"--keep-home",
action="store_true",
help="Keep the temporary HOME directory after the test run.",
)
parser.add_argument(
"--verbose",
action="store_true",
help="Print step progress while running.",
)
return parser.parse_args()
def _run_step(
step_id: str,
description: str,
fn,
results: list[StepResult],
*,
verbose: bool = False,
) -> None:
start = time.monotonic()
try:
fn()
except Exception as exc:
duration = time.monotonic() - start
results.append(
StepResult(
step_id=step_id,
description=description,
status="failed",
duration_sec=duration,
detail=str(exc),
)
)
raise
duration = time.monotonic() - start
results.append(
StepResult(
step_id=step_id,
description=description,
status="passed",
duration_sec=duration,
)
)
if verbose:
print(f"[PASS] {step_id} ({duration:.2f}s) - {description}")
def _drain_to_main_menu(runner: TUIRunner, timeout: float) -> None:
main_menu_prompt = r"Enter your choice \(1-8\) or press Enter to exit:"
helper_prompts: list[tuple[str, str]] = [
(r"Press Enter to continue\.?", ""),
(r"Select an action or press Enter to return:", ""),
(r"Enter index to view details or press Enter to go back:", ""),
(
r"Enter 'v' to view details, 'r' to restore, or press Enter to go back:",
"",
),
(r"Select an option or press Enter to go back:", ""),
(r"Enter index to manage or press Enter to go back:", ""),
]
end = time.monotonic() + timeout
while time.monotonic() < end:
try:
runner.wait_for(main_menu_prompt, timeout=1.0)
return
except TUIHarnessError:
progressed = False
for pattern, value in helper_prompts:
try:
runner.expect_and_send(pattern, value, timeout=0.25)
progressed = True
break
except TUIHarnessError:
continue
if progressed:
continue
runner.pump(0.2)
raise TUIHarnessError("Unable to return to main menu within timeout.")
def _complete_onboarding_to_main_menu(
runner: TUIRunner, timeout: float, password: str
) -> None:
end = time.monotonic() + timeout
while time.monotonic() < end:
if runner.try_wait_for(
r"Enter your choice \(1-8\) or press Enter to exit:", timeout=0.4
):
return
if runner.try_wait_for(r"Enter a new password:", timeout=0.2):
runner.sendline(password)
continue
if runner.try_wait_for(r"Confirm your password:", timeout=0.2):
runner.sendline(password)
continue
if runner.try_wait_for(r"Passwords do not match\.", timeout=0.2):
continue
if runner.try_wait_for(r"Press Enter to continue\.?", timeout=0.2):
runner.sendline("")
continue
runner.pump(0.1)
raise TUIHarnessError("Onboarding did not reach main menu within timeout.")
def _add_entry_menu_open(
runner: TUIRunner, timeout: float, choice: str | None = None
) -> None:
runner.sendline("1")
if choice is None:
runner.wait_for(r"Select entry type or press Enter to go back:", timeout)
else:
runner.expect_and_send(
r"Select entry type or press Enter to go back:", choice, timeout
)
def _scenario(
runner: TUIRunner, timeout: float, password: str, coverage: dict[str, bool]
) -> None:
# First-run onboarding.
runner.expect_and_send(r"Select an option:\s*$", "1", timeout)
runner.expect_and_send(r"Enter choice \(1/2/3/4/5\):", "3", timeout)
runner.expect_and_send(
r"Do you want to use this generated seed\? \(Y/N\):", "Y", timeout
)
_complete_onboarding_to_main_menu(runner, timeout, password)
coverage["startup_onboarding"] = True
coverage["main_menu"] = True
# Invalid input resilience checks.
runner.sendline("99")
runner.wait_for(r"Invalid choice\. Please select a valid option\.", timeout)
_add_entry_menu_open(runner, timeout, "9")
runner.wait_for(r"Invalid choice\.", timeout)
runner.expect_and_send(r"Select entry type or press Enter to go back:", "", timeout)
_drain_to_main_menu(runner, timeout)
runner.sendline("2")
runner.expect_and_send(
r"Enter the index number of the entry to retrieve:", "abc", timeout
)
runner.wait_for(r"Error: Index must be a number\.", timeout)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
coverage["invalid_input_resilience"] = True
# Add password in quick mode, including validation checks.
_add_entry_menu_open(runner, timeout, "1")
runner.expect_and_send(r"Choose mode: \[Q\]uick or \[A\]dvanced\?", "q", timeout)
runner.expect_and_send(
r"Enter the label or website name:", "agent.example", timeout
)
runner.expect_and_send(r"Enter the username \(optional\):", "agent", timeout)
runner.expect_and_send(
r"Enter the URL \(optional\):", "https://agent.example", timeout
)
runner.expect_and_send(r"Enter desired password length .*:", "3", timeout)
runner.wait_for(r"Password length must be between 8 and 128", timeout)
coverage["invalid_length_validation"] = True
runner.expect_and_send(r"Enter desired password length .*:", "12", timeout)
runner.expect_and_send(r"Include special characters\? \(Y/n\):", "", timeout)
runner.wait_for(r"Password generated and indexed with ID 0", timeout)
coverage["add_password_quick"] = True
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
# Retrieve and action menu.
runner.sendline("2")
runner.expect_and_send(
r"Enter the index number of the entry to retrieve:", "0", timeout
)
runner.wait_for(r"Retrieved Password for agent\.example", timeout)
if runner.try_wait_for(r"Reveal hidden fields\? \(y/N\):", timeout=0.4):
runner.sendline("")
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
runner.wait_for(r"Entry Actions:", timeout)
coverage["retrieve_entry"] = True
runner.expect_and_send(r"Select an action or press Enter to return:", "", timeout)
_drain_to_main_menu(runner, timeout)
# Search.
runner.sendline("3")
runner.expect_and_send(r"Enter search string:", "agent", timeout)
runner.wait_for(r"Search Results", timeout)
runner.wait_for(r"0\. Password - agent\.example", timeout)
coverage["search_entries"] = True
runner.expect_and_send(
r"Enter index to view details or press Enter to go back:", "", timeout
)
_drain_to_main_menu(runner, timeout)
# List entries.
runner.sendline("4")
runner.expect_and_send(
r"Select entry type or press Enter to go back:", "1", timeout
)
runner.wait_for(r"\[\+\] Entries:", timeout)
runner.wait_for(r"0\. Password - agent\.example", timeout)
coverage["list_entries"] = True
runner.expect_and_send(
r"Enter index to view details or press Enter to go back:", "", timeout
)
runner.expect_and_send(r"Select entry type or press Enter to go back:", "", timeout)
_drain_to_main_menu(runner, timeout)
# Add generated TOTP and verify 2FA screen shows non-empty codes.
_add_entry_menu_open(runner, timeout, "2")
runner.expect_and_send(r"Select option or press Enter to go back:", "1", timeout)
runner.expect_and_send(r"Label:", "agent-totp", timeout)
runner.expect_and_send(r"Period \(default 30\):", "", timeout)
runner.expect_and_send(r"Digits \(default 6\):", "", timeout)
runner.expect_and_send(r"Notes \(optional\):", "", timeout)
runner.expect_and_send(r"Enter tags \(comma-separated, optional\):", "", timeout)
runner.wait_for(r"TOTP entry added with ID", timeout)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
coverage["add_totp"] = True
runner.sendline("6")
runner.wait_for(r"Press Enter to return to the menu\.", timeout)
runner.wait_for(r"(Generated|Imported) 2FA Codes:", timeout)
runner.wait_for(r"agent-totp", timeout)
runner.sendline("")
_drain_to_main_menu(runner, timeout)
# Add all remaining entry types and verify they appear in listing.
_add_entry_menu_open(runner, timeout, "3")
runner.expect_and_send(r"Label \(key\):", "agent-ssh", timeout)
runner.expect_and_send(r"Notes \(optional\):", "", timeout)
runner.expect_and_send(r"Enter tags \(comma-separated, optional\):", "", timeout)
runner.expect_and_send(
r"WARNING: Displaying SSH keys reveals sensitive information\. Continue\? \(Y/N\):",
"y",
timeout,
)
runner.wait_for(r"SSH key entry added with ID", timeout)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
_add_entry_menu_open(runner, timeout, "4")
runner.expect_and_send(r"Label:", "agent-seed", timeout)
runner.expect_and_send(r"Word count \(12 or 24, default 24\):", "12", timeout)
runner.expect_and_send(r"Notes \(optional\):", "", timeout)
runner.expect_and_send(r"Enter tags \(comma-separated, optional\):", "", timeout)
runner.expect_and_send(
r"WARNING: Displaying the seed phrase reveals sensitive information\. Continue\? \(Y/N\):",
"y",
timeout,
)
runner.wait_for(r"Seed entry 'agent-seed' added with ID", timeout)
runner.expect_and_send(r"Show Compact Seed QR\? \(Y/N\):", "n", timeout)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
_add_entry_menu_open(runner, timeout, "5")
runner.expect_and_send(r"Label:", "agent-nostr", timeout)
runner.expect_and_send(r"Notes \(optional\):", "", timeout)
runner.expect_and_send(r"Enter tags \(comma-separated, optional\):", "", timeout)
runner.wait_for(r"Nostr key entry added with ID", timeout)
runner.expect_and_send(r"Show QR code for npub\? \(Y/N\):", "n", timeout)
runner.expect_and_send(
r"WARNING: Displaying the nsec QR reveals your private key\. Continue\? \(Y/N\):",
"n",
timeout,
)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
_add_entry_menu_open(runner, timeout, "6")
runner.expect_and_send(r"Label:", "agent-pgp", timeout)
runner.expect_and_send(
r"Key type \(ed25519 or rsa, default ed25519\):", "", timeout
)
runner.expect_and_send(r"User ID \(optional\):", "", timeout)
runner.expect_and_send(r"Notes \(optional\):", "", timeout)
runner.expect_and_send(r"Enter tags \(comma-separated, optional\):", "", timeout)
runner.expect_and_send(
r"WARNING: Displaying the PGP key reveals sensitive information\. Continue\? \(Y/N\):",
"y",
timeout,
)
runner.wait_for(r"PGP key entry added with ID", timeout)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
_add_entry_menu_open(runner, timeout, "7")
runner.expect_and_send(r"Label:", "agent-kv", timeout)
runner.expect_and_send(r"Key:", "token", timeout)
runner.expect_and_send(r"Value:", "abc123", timeout)
runner.expect_and_send(r"Notes \(optional\):", "", timeout)
runner.expect_and_send(r"Enter tags \(comma-separated, optional\):", "", timeout)
runner.expect_and_send(r"Add custom field\? \(y/N\):", "n", timeout)
runner.wait_for(r"Key/Value entry added with ID", timeout)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
_add_entry_menu_open(runner, timeout, "8")
runner.expect_and_send(r"Label:", "agent-managed", timeout)
runner.expect_and_send(r"Notes \(optional\):", "", timeout)
runner.expect_and_send(r"Enter tags \(comma-separated, optional\):", "", timeout)
runner.wait_for(r"Managed account 'agent-managed' added with ID", timeout)
runner.expect_and_send(r"Reveal seed now\? \(y/N\):", "n", timeout)
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
runner.sendline("4")
runner.expect_and_send(
r"Select entry type or press Enter to go back:", "1", timeout
)
runner.wait_for(r"agent\.example", timeout)
runner.wait_for(r"agent-totp", timeout)
runner.wait_for(r"agent-ssh", timeout)
runner.wait_for(r"agent-seed", timeout)
runner.wait_for(r"agent-nostr", timeout)
runner.wait_for(r"agent-pgp", timeout)
runner.wait_for(r"agent-kv", timeout)
runner.wait_for(r"agent-managed", timeout)
runner.expect_and_send(
r"Enter index to view details or press Enter to go back:", "", timeout
)
runner.expect_and_send(r"Select entry type or press Enter to go back:", "", timeout)
_drain_to_main_menu(runner, timeout)
coverage["add_all_entry_types"] = True
# Modify entry.
runner.sendline("5")
runner.expect_and_send(
r"Enter the index number of the entry to modify:", "0", timeout
)
runner.expect_and_send(r"Enter new label .*:", "", timeout)
runner.expect_and_send(r"Enter new username .*:", "agent2", timeout)
runner.expect_and_send(r"Enter new URL .*:", "", timeout)
runner.expect_and_send(r"Archive this password\? .*:", "n", timeout)
runner.expect_and_send(r"Enter new notes .*:", "", timeout)
runner.expect_and_send(r"Edit custom fields\? \(y/N\):", "n", timeout)
runner.expect_and_send(
r"Enter tags \(comma-separated, leave blank to keep current\):", "", timeout
)
runner.wait_for(r"Entry updated successfully for index 0", timeout)
coverage["modify_entry"] = True
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
_drain_to_main_menu(runner, timeout)
# Archive + restore via archived menu.
runner.sendline("2")
runner.expect_and_send(
r"Enter the index number of the entry to retrieve:", "0", timeout
)
if runner.try_wait_for(r"Reveal hidden fields\? \(y/N\):", timeout=0.4):
runner.sendline("")
runner.expect_and_send(r"Press Enter to continue\.\.\.", "", timeout)
runner.expect_and_send(r"Select an action or press Enter to return:", "A", timeout)
runner.wait_for(r"Entry at index 0 modified successfully", timeout)
runner.expect_and_send(r"Select an action or press Enter to return:", "", timeout)
_drain_to_main_menu(runner, timeout)
runner.sendline("8")
runner.wait_for(r"Archived Entries", timeout)
runner.wait_for(r"0\. agent\.example", timeout)
runner.expect_and_send(
r"Enter index to manage or press Enter to go back:", "0", timeout
)
runner.expect_and_send(
r"Enter 'v' to view details, 'r' to restore, or press Enter to go back:",
"r",
timeout,
)
runner.wait_for(r"Entry at index 0 modified successfully", timeout)
coverage["archive_restore"] = True
_drain_to_main_menu(runner, timeout)
# 2FA codes view remains accessible after mixed operations.
runner.sendline("6")
runner.wait_for(r"Press Enter to return to the menu\.", timeout)
runner.wait_for(r"(Generated|Imported) 2FA Codes:", timeout)
runner.wait_for(r"agent-totp", timeout)
runner.sendline("")
coverage["totp_codes_view"] = True
_drain_to_main_menu(runner, timeout)
# Settings toggles and lock/unlock flow.
runner.sendline("7")
runner.expect_and_send(
r"Select an option or press Enter to go back:", "15", timeout
)
runner.expect_and_send(
r"Enable secret mode\? \(y/n, blank to keep\):", "y", timeout
)
runner.expect_and_send(r"Clipboard clear delay in seconds \[[0-9]+\]:", "", timeout)
runner.expect_and_send(r"Press Enter to continue\.?", "", timeout)
runner.expect_and_send(
r"Select an option or press Enter to go back:", "16", timeout
)
runner.expect_and_send(
r"Enable offline mode\? \(y/n, blank to keep\):", "n", timeout
)
runner.expect_and_send(r"Press Enter to continue\.?", "", timeout)
runner.expect_and_send(
r"Select an option or press Enter to go back:", "17", timeout
)
runner.expect_and_send(
r"Enable Quick Unlock\? \(y/n, blank to keep\):", "y", timeout
)
runner.expect_and_send(r"Press Enter to continue\.?", "", timeout)
runner.expect_and_send(
r"Select an option or press Enter to go back:", "13", timeout
)
# Unlock prompt text can vary; accept either and submit password.
if runner.try_wait_for(r"Enter your master password to continue:", timeout=timeout):
runner.sendline(password)
elif runner.try_wait_for(r"Enter your master password:", timeout=timeout):
runner.sendline(password)
elif runner.try_wait_for(r"Enter your password:", timeout=timeout):
runner.sendline(password)
runner.expect_and_send(r"Press Enter to continue\.?", "", timeout)
runner.expect_and_send(r"Select an option or press Enter to go back:", "", timeout)
_drain_to_main_menu(runner, timeout)
coverage["settings_toggles_lock_unlock"] = True
# Settings -> Stats and back.
runner.sendline("7")
runner.expect_and_send(
r"Select an option or press Enter to go back:", "14", timeout
)
runner.wait_for(r"=== Seed Profile Stats ===", timeout)
runner.expect_and_send(r"Press Enter to continue\.?", "", timeout)
runner.expect_and_send(r"Select an option or press Enter to go back:", "", timeout)
coverage["settings_stats"] = True
_drain_to_main_menu(runner, timeout)
# Graceful exit.
runner.sendline("")
runner.wait_for(r"Exiting the program\.", timeout)
coverage["graceful_exit"] = True
def main() -> int:
args = _parse_args()
repo_root = args.repo_root.resolve()
src_main = repo_root / "src" / "main.py"
if not src_main.exists():
print(f"Error: cannot find {src_main}", file=sys.stderr)
return 2
timestamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
artifact_dir = args.output_dir.resolve() / timestamp
artifact_dir.mkdir(parents=True, exist_ok=True)
temp_home_dir = Path(tempfile.mkdtemp(prefix="seedpass-agent-home-"))
env = os.environ.copy()
env["HOME"] = str(temp_home_dir)
env["PYTHONUNBUFFERED"] = "1"
env.setdefault("TERM", "xterm")
cmd = [args.python_bin, str(src_main), "--no-clipboard"]
master_fd, slave_fd = pty.openpty()
proc = subprocess.Popen(
cmd,
cwd=str(repo_root),
env=env,
stdin=slave_fd,
stdout=slave_fd,
stderr=slave_fd,
close_fds=True,
)
os.close(slave_fd)
runner = TUIRunner(proc, master_fd)
results: list[StepResult] = []
coverage = {
"startup_onboarding": False,
"main_menu": False,
"invalid_input_resilience": False,
"add_password_quick": False,
"invalid_length_validation": False,
"add_totp": False,
"add_all_entry_types": False,
"retrieve_entry": False,
"search_entries": False,
"list_entries": False,
"modify_entry": False,
"archive_restore": False,
"totp_codes_view": False,
"settings_toggles_lock_unlock": False,
"settings_stats": False,
"graceful_exit": False,
}
failure: str | None = None
start_time = time.monotonic()
try:
_run_step(
"full_scenario",
"Drive end-to-end deterministic TUI coverage workflow.",
lambda: _scenario(runner, args.timeout, args.password, coverage),
results,
verbose=args.verbose,
)
except Exception as exc:
failure = str(exc)
finally:
exit_code = runner.close()
total_duration = time.monotonic() - start_time
transcript_clean = artifact_dir / "transcript.clean.txt"
transcript_raw = artifact_dir / "transcript.raw.txt"
transcript_clean.write_text(runner.clean_output, encoding="utf-8")
transcript_raw.write_text(runner.raw_output, encoding="utf-8")
report = {
"timestamp_utc": timestamp,
"command": cmd,
"repo_root": str(repo_root),
"home_dir": str(temp_home_dir),
"duration_sec": round(total_duration, 3),
"seedpass_exit_code": exit_code,
"status": "passed" if failure is None and all(coverage.values()) else "failed",
"failure": failure,
"steps": [asdict(item) for item in results],
"coverage_points": coverage,
"transcript_clean": str(transcript_clean),
"transcript_raw": str(transcript_raw),
}
report_file = artifact_dir / "report.json"
report_file.write_text(json.dumps(report, indent=2), encoding="utf-8")
if not args.keep_home:
shutil.rmtree(temp_home_dir, ignore_errors=True)
print(f"AI agent TUI report: {report_file}")
print(f"Status: {report['status']}")
if failure:
print(f"Failure: {failure}")
if report["status"] != "passed":
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
+3 -52
View File
@@ -316,10 +316,9 @@ def handle_display_npub(password_manager: PasswordManager):
def _display_live_stats(
password_manager: PasswordManager, interval: float = 1.0
) -> None:
"""Continuously refresh stats until the user presses Enter.
"""Display stats and wait for the user to continue.
Each refresh also triggers a background sync so the latest stats are
displayed if newer data exists on Nostr.
A background sync runs before rendering so the latest stats are shown.
"""
stats_mgr = getattr(password_manager, "stats_manager", None)
@@ -346,38 +345,6 @@ def _display_live_stats(
stats_mgr.reset()
return
# Flush any pending input so an accidental newline doesn't exit immediately
try: # pragma: no cover - depends on platform
import termios
termios.tcflush(sys.stdin, termios.TCIFLUSH)
except Exception:
try: # pragma: no cover - Windows fallback
import msvcrt
while msvcrt.kbhit():
msvcrt.getwch()
except Exception:
pass
while True:
# Break out immediately if the user has already pressed Enter
try: # pragma: no cover - non-interactive environments
import select
ready, _, _ = select.select([sys.stdin], [], [], 0)
if ready:
line = sys.stdin.readline().strip()
if line == "" or line.lower() == "b":
break
except Exception:
pass
if callable(sync_fn):
try:
sync_fn()
except Exception: # pragma: no cover - sync best effort
logging.debug("Background sync failed during stats display")
clear_screen()
display_fn()
note = get_notification_text(password_manager)
@@ -386,14 +353,9 @@ def _display_live_stats(
print(colored("Press Enter to continue.", "cyan"))
sys.stdout.flush()
try:
user_input = timed_input("", interval)
if user_input.strip() == "" or user_input.strip().lower() == "b":
break
except TimeoutError:
pass
timed_input("", None)
except KeyboardInterrupt:
print()
break
if stats_mgr is not None:
stats_mgr.reset()
@@ -1250,17 +1212,6 @@ def display_menu(
elif choice == "2":
password_manager.update_activity()
password_manager.handle_retrieve_entry()
fp, parent_fp, child_fp = getattr(
password_manager,
"header_fingerprint_args",
(getattr(password_manager, "current_fingerprint", None), None, None),
)
clear_header_with_notification(
fp,
"Main Menu",
parent_fingerprint=parent_fp,
child_fingerprint=child_fp,
)
elif choice == "3":
password_manager.update_activity()
password_manager.handle_search_entries()
+17 -8
View File
@@ -36,7 +36,8 @@ class EntryService:
child_fingerprint=child_fp,
)
def prompt_length() -> int | None:
def prompt_length() -> int:
while True:
length_input = input(
f"Enter desired password length (default {DEFAULT_PASSWORD_LENGTH}): "
).strip()
@@ -44,9 +45,11 @@ class EntryService:
if length_input:
if not length_input.isdigit():
print(
colored("Error: Password length must be a number.", "red")
colored(
"Error: Password length must be a number.", "red"
)
return None
)
continue
length = int(length_input)
if not (MIN_PASSWORD_LENGTH <= length <= MAX_PASSWORD_LENGTH):
print(
@@ -55,7 +58,7 @@ class EntryService:
"red",
)
)
return None
continue
return length
def finalize_entry(index: int, label: str, length: int) -> None:
@@ -96,7 +99,17 @@ class EntryService:
)
pause()
while True:
mode = input("Choose mode: [Q]uick or [A]dvanced? ").strip().lower()
if not mode:
return
if mode.startswith("q") or mode.startswith("a"):
break
print(
colored(
"Invalid mode. Enter 'q' for quick or 'a' for advanced.", "red"
)
)
website_name = input("Enter the label or website name: ").strip()
if not website_name:
@@ -108,8 +121,6 @@ class EntryService:
if mode.startswith("q"):
length = prompt_length()
if length is None:
return
include_special_input = (
input("Include special characters? (Y/n): ").strip().lower()
)
@@ -149,8 +160,6 @@ class EntryService:
)
length = prompt_length()
if length is None:
return
include_special_input = (
input("Include special characters? (Y/n): ").strip().lower()
+19 -11
View File
@@ -1385,7 +1385,8 @@ class PasswordManager:
self.bip85.derive_entropy = cached_derive
logging.debug("BIP-85 initialized successfully.")
self.clear_bip85_cache()
# Reset cache directly so initialization is valid even while locked.
self._bip85_cache.clear()
except Exception as e:
logging.error(f"Failed to initialize BIP-85: {e}", exc_info=True)
print(colored(f"Error: Failed to initialize BIP-85: {e}", "red"))
@@ -3628,9 +3629,6 @@ class PasswordManager:
exc_info=True,
)
updated_entry = self.entry_manager.retrieve_entry(index)
if updated_entry:
self._prompt_toggle_archive(updated_entry, index)
pause()
except Exception as e:
@@ -3911,15 +3909,26 @@ class PasswordManager:
def handle_view_archived_entries(self) -> None:
"""Display archived entries and optionally view or restore them."""
try:
archived = self.entry_manager.list_entries(
def _load_archived_entries() -> (
list[tuple[int, str, str | None, str | None, bool]]
):
entries = self.entry_manager.list_entries(
include_archived=True, verbose=False
)
archived = [e for e in archived if e[4]]
return [e for e in entries if e[4]]
archived = _load_archived_entries()
if not archived:
self.notify("No archived entries found.", level="WARNING")
pause()
return
while True:
archived = _load_archived_entries()
if not archived:
self.notify("No archived entries found.", level="WARNING")
pause()
return
fp, parent_fp, child_fp = self.header_fingerprint_args
clear_header_with_notification(
self,
@@ -3952,16 +3961,15 @@ class PasswordManager:
)
if action == "v":
self.show_entry_details_by_index(entry_index)
pause()
archived = _load_archived_entries()
if entry_index not in [e[0] for e in archived]:
break
elif action == "r":
self.entry_manager.restore_entry(entry_index)
self.is_dirty = True
self.last_update = time.time()
pause()
archived = self.entry_manager.list_entries(
include_archived=True, verbose=False
)
archived = [e for e in archived if e[4]]
archived = _load_archived_entries()
if not archived:
print(colored("All entries restored.", "green"))
pause()
+10
View File
@@ -60,6 +60,16 @@ class MenuHandler:
filter_kinds, include_archived=False
)
if not summaries:
if filter_kinds is None:
print(colored("No active entries found.", "yellow"))
else:
entry_label = filter_kinds[0].replace("_", " ").title()
print(
colored(
f"No active {entry_label} entries found.",
"yellow",
)
)
break
fp, parent_fp, child_fp = pm.header_fingerprint_args
clear_header_with_notification(
+41
View File
@@ -193,3 +193,44 @@ def test_archived_entries_menu_hides_active(monkeypatch, capsys):
out = capsys.readouterr().out
assert "archived.com" in out
assert "active.com" not in out
def test_view_archived_entries_refreshes_after_view_action(monkeypatch):
with TemporaryDirectory() as tmpdir:
tmp_path = Path(tmpdir)
vault, enc_mgr = create_vault(tmp_path, TEST_SEED, TEST_PASSWORD)
cfg_mgr = ConfigManager(vault, tmp_path)
backup_mgr = BackupManager(tmp_path, cfg_mgr)
entry_mgr = EntryManager(vault, backup_mgr)
pm = PasswordManager.__new__(PasswordManager)
pm.encryption_mode = EncryptionMode.SEED_ONLY
pm.encryption_manager = enc_mgr
pm.vault = vault
pm.entry_manager = entry_mgr
pm.backup_manager = backup_mgr
pm.parent_seed = TEST_SEED
pm.nostr_client = SimpleNamespace()
pm.fingerprint_dir = tmp_path
pm.is_dirty = False
pm.notifications = queue.Queue()
idx = entry_mgr.add_entry("example.com", 8)
entry_mgr.archive_entry(idx)
assert entry_mgr.retrieve_entry(idx)["archived"] is True
monkeypatch.setattr(
pm,
"show_entry_details_by_index",
lambda index: entry_mgr.restore_entry(index),
)
monkeypatch.setattr("seedpass.core.manager.pause", lambda: None)
inputs = iter([str(idx), "v"])
monkeypatch.setattr("builtins.input", lambda *_: next(inputs))
pm.handle_view_archived_entries()
assert entry_mgr.retrieve_entry(idx)["archived"] is False
note = pm.notifications.get_nowait()
assert note.level == "WARNING"
assert note.message == "No archived entries found."
+18
View File
@@ -6,6 +6,7 @@ sys.path.append(str(Path(__file__).resolve().parents[1]))
from bip_utils import Bip39SeedGenerator
from local_bip85.bip85 import BIP85
from helpers import TEST_SEED
from seedpass.core.manager import PasswordManager
MASTER_XPRV = "xprv9s21ZrQH143K2LBWUUQRFXhucrQqBpKdRRxNVq2zBqsx8HVqFk2uYo8kmbaLLHRdqtQpUm98uKfu3vca1LqdGhUtyoFnCNkfmXRyPXLjbKb"
@@ -19,3 +20,20 @@ def test_init_with_seed_bytes():
def test_init_with_xprv():
bip85 = BIP85(MASTER_XPRV)
assert isinstance(bip85, BIP85)
def test_initialize_bip85_works_while_locked():
pm = PasswordManager.__new__(PasswordManager)
pm.parent_seed = TEST_SEED
pm.is_locked = True
pm._bip85_cache = {(39, 0): b"stale"}
pm.KEY_STORAGE = None
pm.KEY_INDEX = None
pm.KEY_PW_DERIVE = None
pm.KEY_TOTP_DET = None
pm.master_key = None
pm.initialize_bip85()
assert isinstance(pm.bip85, BIP85)
assert pm._bip85_cache == {}
+128
View File
@@ -208,3 +208,131 @@ def test_handle_add_password_quick_mode(monkeypatch, dummy_nostr_client, capsys)
}
assert f"pw-0-{DEFAULT_PASSWORD_LENGTH}" in out
def test_handle_add_password_quick_mode_reprompts_invalid_length(
monkeypatch, dummy_nostr_client, capsys
):
client, _relay = dummy_nostr_client
with TemporaryDirectory() as tmpdir:
tmp_path = Path(tmpdir)
vault, enc_mgr = create_vault(tmp_path, TEST_SEED, TEST_PASSWORD)
cfg_mgr = ConfigManager(vault, tmp_path)
backup_mgr = BackupManager(tmp_path, cfg_mgr)
entry_mgr = EntryManager(vault, backup_mgr)
pm = PasswordManager.__new__(PasswordManager)
pm.encryption_mode = EncryptionMode.SEED_ONLY
pm.encryption_manager = enc_mgr
pm.vault = vault
pm.entry_manager = entry_mgr
pm.backup_manager = backup_mgr
pm.password_generator = FakePasswordGenerator()
pm.parent_seed = TEST_SEED
pm.nostr_client = client
pm.fingerprint_dir = tmp_path
pm.secret_mode_enabled = False
pm.is_dirty = False
inputs = iter(
[
"q",
"Example",
"",
"",
"3",
"abc",
"12",
"",
]
)
monkeypatch.setattr("builtins.input", lambda *a, **k: next(inputs))
monkeypatch.setattr("seedpass.core.manager.pause", lambda *a, **k: None)
monkeypatch.setattr(pm, "start_background_vault_sync", lambda *a, **k: None)
pm.handle_add_password()
out = capsys.readouterr().out
assert "Error: Password length must be between" in out
assert "Error: Password length must be a number." in out
entries = entry_mgr.list_entries(verbose=False)
assert entries == [(0, "Example", "", "", False)]
entry = entry_mgr.retrieve_entry(0)
assert entry["length"] == 12
def test_handle_add_password_reprompts_invalid_mode(
monkeypatch, dummy_nostr_client, capsys
):
client, _relay = dummy_nostr_client
with TemporaryDirectory() as tmpdir:
tmp_path = Path(tmpdir)
vault, enc_mgr = create_vault(tmp_path, TEST_SEED, TEST_PASSWORD)
cfg_mgr = ConfigManager(vault, tmp_path)
backup_mgr = BackupManager(tmp_path, cfg_mgr)
entry_mgr = EntryManager(vault, backup_mgr)
pm = PasswordManager.__new__(PasswordManager)
pm.encryption_mode = EncryptionMode.SEED_ONLY
pm.encryption_manager = enc_mgr
pm.vault = vault
pm.entry_manager = entry_mgr
pm.backup_manager = backup_mgr
pm.password_generator = FakePasswordGenerator()
pm.parent_seed = TEST_SEED
pm.nostr_client = client
pm.fingerprint_dir = tmp_path
pm.secret_mode_enabled = False
pm.is_dirty = False
inputs = iter(
[
"z", # invalid mode
"q", # then quick mode
"Example",
"",
"",
"",
"",
]
)
monkeypatch.setattr("builtins.input", lambda *a, **k: next(inputs))
monkeypatch.setattr("seedpass.core.manager.pause", lambda *a, **k: None)
monkeypatch.setattr(pm, "start_background_vault_sync", lambda *a, **k: None)
pm.handle_add_password()
out = capsys.readouterr().out
assert "Invalid mode. Enter 'q' for quick or 'a' for advanced." in out
entries = entry_mgr.list_entries(verbose=False)
assert entries == [(0, "Example", "", "", False)]
def test_handle_add_password_empty_mode_cancels(monkeypatch, dummy_nostr_client):
client, _relay = dummy_nostr_client
with TemporaryDirectory() as tmpdir:
tmp_path = Path(tmpdir)
vault, enc_mgr = create_vault(tmp_path, TEST_SEED, TEST_PASSWORD)
cfg_mgr = ConfigManager(vault, tmp_path)
backup_mgr = BackupManager(tmp_path, cfg_mgr)
entry_mgr = EntryManager(vault, backup_mgr)
pm = PasswordManager.__new__(PasswordManager)
pm.encryption_mode = EncryptionMode.SEED_ONLY
pm.encryption_manager = enc_mgr
pm.vault = vault
pm.entry_manager = entry_mgr
pm.backup_manager = backup_mgr
pm.password_generator = FakePasswordGenerator()
pm.parent_seed = TEST_SEED
pm.nostr_client = client
pm.fingerprint_dir = tmp_path
pm.secret_mode_enabled = False
pm.is_dirty = False
inputs = iter([""])
monkeypatch.setattr("builtins.input", lambda *a, **k: next(inputs))
monkeypatch.setattr(pm, "start_background_vault_sync", lambda *a, **k: None)
pm.handle_add_password()
assert entry_mgr.list_entries(verbose=False) == []
+13
View File
@@ -93,6 +93,19 @@ def test_menu_handler_list_entries(monkeypatch, capsys):
assert "acct" in out
def test_menu_handler_list_entries_shows_empty_message(monkeypatch, capsys):
with TemporaryDirectory() as tmpdir:
pm = _setup_pm(Path(tmpdir), SimpleNamespace())
# Keep only an archived password so active listing is empty.
idx = pm.entry_manager.add_entry("example.com", 12)
pm.entry_manager.archive_entry(idx)
inputs = iter(["1", ""]) # list all, then exit
monkeypatch.setattr("builtins.input", lambda *_: next(inputs))
pm.menu_handler.handle_list_entries()
out = capsys.readouterr().out
assert "No active entries found." in out
def test_profile_service_switch(monkeypatch):
class DummyFingerprintManager:
def __init__(self):