This commit is contained in:
thePR0M3TH3AN committed 2026-02-20 12:51:17 -05:00
1 parent 03b6725cfc
commit 2b8a0c3d0d
25 files changed
+1126 -5

No files matched your search

+3 -3
View File
@@ -7,7 +7,7 @@ This document identifies areas of the TORCH repository that lack specific agent
| Gap | Evidence | Proposed Agent |
| :--- | :--- | :--- |
| **Database Migrations** | `migrations/` directory and SQL files present. | `migration-agent` |
| **E2E Testing** | `playwright` dependency and `test/` integration files. | `e2e-agent` |
| **E2E Testing** | `test/` integration/e2e files (Node.js `--test`). | `e2e-agent` |
| **Memory Subsystem** | `scripts/memory/`, `.scheduler-memory/` exist. | `memory-health-agent` |
| **Relay Stability** | `src/relay-health.mjs` and known relay flakiness. | `relay-health-agent` |
| **Skills Management** | `skills/` directory and `torch-basics.md`. | `skills-agent` |
@@ -24,8 +24,8 @@ This document identifies areas of the TORCH repository that lack specific agent
- **Priority**: Medium
### 2. `e2e-agent`
- **Gap**: `playwright` is used for dashboard and integration testing, but no agent is dedicated to scenario maintenance.
- **Mission**: Maintain Playwright E2E scenarios, fix flaky tests, and ensure the dashboard remains functional across protocol changes.
- **Gap**: E2E/integration tests exist (e.g., `test/scheduler-preflight-lock.e2e.test.mjs` via Node.js `--test`), but no agent is dedicated to scenario maintenance.
- **Mission**: Maintain E2E scenarios, fix flaky tests, and ensure the dashboard and scheduler flow remain functional across protocol changes.
- **Priority**: High (given the "Scenario-First" mandate in `AGENTS.md`)
### 3. `memory-health-agent`
@@ -0,0 +1,11 @@
# Docs Agent: README/TORCH.md Synchronization Pattern
- **Context:** docs-agent run 2026-02-20. Audited README.md, CONTRIBUTING.md, AGENTS.md against package.json, src/, and TORCH.md.
- **Observation:** `TORCH.md` is the authoritative configuration and env-var reference. `README.md` had drifted and was missing 4 env vars and 8 `torch-config.json` keys present in TORCH.md and verified in `src/torch-config.mjs`.
- **Action taken:** Added missing items to README.md's "Environment variables" and "Configuration" sections. No changes to CONTRIBUTING.md or AGENTS.md — both were accurate.
- **Validation performed:** Grep confirmed all 4 missing env vars are used in `src/torch-config.mjs`. All referenced file paths verified to exist via Glob.
- **Recommendation for next agents:**
1. When updating env vars or torch-config keys in TORCH.md or `src/torch-config.mjs`, also update README.md's matching sections.
2. CONTRIBUTING.md is minimal and accurate — only update if new scripts are added to package.json.
3. AGENTS.md is a policy-only file — do not edit it for accuracy; open an issue if policy appears wrong.
4. The 4 env vars most often missed in README: `NOSTR_LOCK_PUBLISH_TIMEOUT_MS`, `NOSTR_LOCK_MIN_SUCCESSFUL_PUBLISHES`, `NOSTR_LOCK_RELAY_FALLBACKS`, `NOSTR_LOCK_MIN_ACTIVE_RELAY_POOL`.
@@ -0,0 +1,15 @@
# Docs Alignment Agent: Claims Tracing Pattern
- **Context:** docs-alignment-agent run 2026-02-20. Daily pass auditing docs/lib-overview.md, docs/lock-ops-overview.md, docs/memory-index-overview.md, docs/removal.md, docs/architecture/system-overview.md, docs/AGENT_COVERAGE_GAPS.md, src/constants.mjs USAGE_TEXT.
- **Observation:** Four confirmed doc/code divergences found and fixed:
1. `docs/lib-overview.md` claimed RACE_CHECK_DELAY_MS = 2000ms; actual is 1500ms (src/constants.mjs:2).
2. `src/constants.mjs` USAGE_TEXT said `NOSTR_LOCK_QUERY_TIMEOUT_MS` default is 15000; actual default is 30000 (`DEFAULT_QUERY_TIMEOUT_MS`). The 15000 value matches `DEFAULT_PUBLISH_TIMEOUT_MS` — likely copy-paste confusion between the two timeouts.
3. `docs/removal.md` env var cleanup list was missing 4 vars: `NOSTR_LOCK_PUBLISH_TIMEOUT_MS`, `NOSTR_LOCK_MIN_SUCCESSFUL_PUBLISHES`, `NOSTR_LOCK_RELAY_FALLBACKS`, `NOSTR_LOCK_MIN_ACTIVE_RELAY_POOL`.
4. `docs/AGENT_COVERAGE_GAPS.md` claimed "playwright dependency" — Playwright is not in package.json; E2E tests use Node.js built-in `--test` runner.
- **Action taken:** Fixed all four issues with minimal, targeted edits.
- **Validation performed:** Grep confirmed exact corrected values in all edited files post-edit.
- **Recommendation for next agents:**
1. When auditing docs, always trace numeric constants (timeouts, delays, counts) to `src/constants.mjs` — these drift most often.
2. The USAGE_TEXT in `src/constants.mjs` is the CLI `--help` string — verify its defaults match the exported constants above it.
3. `docs/removal.md` env var list must stay in sync with `src/torch-config.mjs` — the four new relay-reliability vars are the most commonly missed.
4. When AGENT_COVERAGE_GAPS.md mentions technology dependencies, verify against `package.json` before trusting the claim.
@@ -0,0 +1,11 @@
# Learning: Scheduler Cycle File Documentation Patterns
- **Context:** docs-code-investigator run 2026-02-20. Documented `scripts/agent/run-scheduler-cycle.mjs` (926 lines) with JSDoc, inline comments, and `docs/scheduler-cycle-overview.md`.
- **Observation:** The file's logic precisely mirrors the 19 numbered steps in `src/prompts/scheduler-flow.md`. The top-of-file comment already referenced scheduler-flow.md but didn't enumerate the steps — adding them makes the code immediately navigable.
- **Action taken:** Added module-level JSDoc with 19-step flow, JSDoc to 15 internal functions, targeted inline comments, and a full overview doc.
- **Validation performed:** Static read of full 926-line file. Call graph traced to scheduler-utils.mjs, scheduler-lock.mjs, src/utils.mjs, roster.json, and test files.
- **Recommendation for next agents:**
1. When documenting scheduler-related files, always cross-reference `src/prompts/scheduler-flow.md` — it is the spec; the code implements it.
2. The most non-obvious invariant: lock:complete is called ONLY after all validation gates pass. Document this explicitly when touching the scheduler.
3. Three test files cover scheduler behavior: run-scheduler-cycle-memory-policy.test.mjs, scheduler-time-window.test.mjs, scheduler-preflight-lock.e2e.test.mjs. Run all three when changing scheduler logic.
4. `task-logs/<cadence>/.scheduler-run-state.json` is day-scoped (resets at midnight UTC). Do not treat it as persistent cross-day storage.
+1 -1
View File
@@ -40,7 +40,7 @@ Displays a CLI-friendly table of active locks.
- Content: JSON payload with metadata.
4. **Publish**: Send to all configured relays.
5. **Race Check**:
- Wait `RACE_CHECK_DELAY_MS` (default 2000ms).
- Wait `RACE_CHECK_DELAY_MS` (default 1500ms).
- Query relays again.
- If another valid event exists with an *earlier* `created_at`: **Abort** (Race lost).
- Otherwise: **Success**.
+4
View File
@@ -106,6 +106,10 @@ If you configured TORCH environment variables in your shell profile, CI, or `.en
- `NOSTR_LOCK_RELAYS`
- `NOSTR_LOCK_TTL`
- `NOSTR_LOCK_QUERY_TIMEOUT_MS`
- `NOSTR_LOCK_PUBLISH_TIMEOUT_MS`
- `NOSTR_LOCK_MIN_SUCCESSFUL_PUBLISHES`
- `NOSTR_LOCK_RELAY_FALLBACKS`
- `NOSTR_LOCK_MIN_ACTIVE_RELAY_POOL`
- `NOSTR_LOCK_DAILY_ROSTER`
- `NOSTR_LOCK_WEEKLY_ROSTER`
- `TORCH_CONFIG_PATH`
+262
View File
@@ -0,0 +1,262 @@
# Scheduler Cycle Overview (`scripts/agent/run-scheduler-cycle.mjs`)
## What this module does
`run-scheduler-cycle.mjs` is the main scheduler entry point. Given a cadence
(`daily` or `weekly`), it selects the next eligible agent from the roster,
acquires a distributed Nostr lock to prevent duplicate execution across agents,
runs the selected agent's prompt via a configured handoff command, verifies that
required run artifacts and memory evidence were produced, then publishes task
completion.
This script owns the full agent lifecycle for one slot per invocation:
lock acquisition → prompt execution → artifact/memory verification → lock:complete.
Spawned agents must **not** call `lock:complete` themselves — the scheduler does it
after all validation passes.
## Where it fits
```
npm run scheduler:daily
└─ scripts/agent/run-scheduler-cycle.mjs daily
├─ src/prompts/roster.json (agent list)
├─ torch-config.json (handoff command, memory policy, etc.)
├─ scripts/agent/scheduler-utils.mjs
├─ scripts/agent/scheduler-lock.mjs
├─ npm run lock:check:daily (relay query)
├─ npm run lock:lock (relay publish)
├─ <handoffCommand> (spawns the agent)
├─ scripts/agent/verify-run-artifacts.mjs
└─ npm run lock:complete (relay publish, permanent)
```
## Typical call flow
```
1. Parse args → cadence='daily', platform='codex', model=null
2. Load roster from src/prompts/roster.json → ['audit-agent', 'docs-agent', ...]
3. Read AGENTS.md → print to stdout (best-effort; non-fatal if missing)
4. [LOOP] Read run-state → { run_date: '2026-02-20', lock_deferral: null }
5. [LOOP] Lock health preflight (if enabled):
npm run lock:health -- --cadence daily
→ non-zero: write _deferred.md or _failed.md, exit
6. [LOOP] npm run lock:check:daily -- --json --quiet
→ exclusion set: { locked: ['ci-health-agent'], paused: [], completed: [] }
7. [LOOP] Time-window guard: exclude agents that ran within 24h window
8. [LOOP] Round-robin select:
roster = ['audit-agent', 'ci-health-agent', 'docs-agent', ...]
previous = 'ci-health-agent' (from latest log)
excluded = { 'ci-health-agent' }
→ selected = 'docs-agent'
9. [LOOP] npm run lock:lock -- --agent docs-agent --cadence daily
→ exit 0: lock acquired (continue)
→ exit 3: race lost (restart loop at step 4)
→ exit 2: backend error → defer (non-strict) or fail (strict)
10. [LOOP] Validate src/prompts/daily/docs-agent.md
→ not readable → _failed.md (prompt_parse_error), exit 1
→ bad first line → _failed.md (prompt_schema_error), exit 1
11. [LOOP] Clear deferral state. Run memory retrieve command (if configured).
12. [LOOP] Run handoff command (e.g. `claude --prompt "$SCHEDULER_PROMPT_PATH"`)
→ non-zero → _failed.md (execution_error), exit non-zero
→ missing → _failed.md, exit 1
13. [LOOP] Run memory store command (if configured).
14. [LOOP] Verify memory evidence (markers/artifacts):
→ mode='required' + missing → _failed.md, exit 1
→ mode='optional' + missing → console.warn, continue
15. [LOOP] node scripts/agent/verify-run-artifacts.mjs --since <runStart>
→ non-zero → _failed.md (prompt_schema_error), exit non-zero
16. [LOOP] npm run lint (or configured validationCommands)
→ non-zero → _failed.md (execution_error), exit non-zero
17. [LOOP] npm run lock:complete -- --agent docs-agent --cadence daily
→ non-zero → _failed.md, exit non-zero
18. [LOOP] Write _completed.md task log. Print run summary. Exit 0.
```
## Public API summary
This is a CLI script — it has no exported functions. The entry point is:
```bash
node scripts/agent/run-scheduler-cycle.mjs <daily|weekly> [--platform <p>] [--model <m>]
# or via npm:
npm run scheduler:daily
npm run scheduler:weekly
```
**Environment variables:**
| Variable | Source | Purpose |
|----------|--------|---------|
| `AGENT_PLATFORM` | env | Platform identifier (codex, claude, linux) |
| `AGENT_MODEL` | env | Model identifier forwarded to lock events |
| `SCHEDULER_LOCK_MAX_RETRIES` | env | Override lock:lock retry count (default: 2) |
| `SCHEDULER_LOCK_BACKOFF_MS` | env | Override base backoff ms (default: 250) |
| `SCHEDULER_LOCK_JITTER_MS` | env | Override jitter ms (default: 75) |
| `SCHEDULER_LOCK_HEALTH_PREFLIGHT` | env | Enable/disable relay health preflight |
| `SCHEDULER_SKIP_LOCK_HEALTH_PREFLIGHT` | env | Skip preflight even if enabled |
| `SCHEDULER_STRICT_LOCK` | env | Override strict_lock policy |
| `SCHEDULER_DEGRADED_LOCK_RETRY_WINDOW_MS` | env | Override deferral window ms |
| `SCHEDULER_MAX_DEFERRALS` | env | Override max deferral count |
**Variables injected into child processes:**
| Variable | Value | Purpose |
|----------|-------|---------|
| `SCHEDULER_AGENT` | selected agent name | Used by handoff/memory commands |
| `SCHEDULER_CADENCE` | `daily`\|`weekly` | Used by handoff/memory commands |
| `SCHEDULER_PROMPT_PATH` | absolute path | Path to prompt file |
| `SCHEDULER_MEMORY_FILE` | absolute path | Agent writes learnings here |
## Key internal functions
| Function | Purpose |
|----------|---------|
| `parseArgs(argv)` | Parses cadence, platform, model from CLI args |
| `getSchedulerConfig(cadence, opts)` | Merges torch-config.json + env overrides |
| `readRunState(cadence)` / `writeRunState(path, state)` | Deferral state persistence (day-scoped) |
| `getLatestFile(logDir)` | Finds most recent valid task log for previous-agent detection |
| `selectNextAgent({roster, excludedSet, previousAgent, firstPrompt})` | Round-robin with exclusion |
| `validatePromptFile(promptPath)` | Reads and validates prompt format |
| `verifyMemoryStep({name, markers, artifacts, outputText, sinceMs})` | Evidence check |
| `writeLog({cadence, agent, status, ...})` | Writes YAML-frontmatter task log file |
| `exitWithSummary(code, data)` | Prints summary + calls process.exit |
## Key invariants
1. **lock:complete gate**: `npm run lock:complete` is called only after memory
verification (step 14), artifact verification (step 15), and all validation
commands (step 16) pass. If any gate fails, the run ends without publishing
completion.
2. **Completed log gate**: `_completed.md` task log is written only after
lock:complete exits 0 (step 17). Earlier failures always write `_failed.md`
or `_deferred.md`.
3. **Race resolution**: Lock exit code 3 (race lost) causes the scheduler to
restart the loop at step 4, re-checking what agents are available. This is
the normal race resolution path — not an error.
4. **Time-window guard**: Agents that appear in a recent local log (within 24h
for daily, 7d for weekly) are added to the exclusion set regardless of relay
state. This handles the cross-midnight edge case where the relay date-scope
has rolled over.
5. **Cycle saturation**: When all roster agents are in the time-window exclusion
set, the scheduler exits 0 with a "cycle complete" reason — not treated as
an error by cron/CI.
6. **Deferral state is day-scoped**: `task-logs/<cadence>/.scheduler-run-state.json`
is reset whenever `run_date` changes, so deferral budgets do not carry across days.
## Edge cases & error paths
| Scenario | Behavior |
|----------|----------|
| No roster entries | `process.exit(1)` before loop |
| Roster fully excluded (not cycle-saturated) | `_failed.md` + exit 1 |
| All relays unhealthy (preflight) | `_deferred.md` + exit 0 |
| Lock backend error (strict mode) | `_failed.md` + exit 2 |
| Lock backend error (non-strict, within window/budget) | `_deferred.md` + exit 0; deferral state persisted |
| Lock backend error (non-strict, budget exhausted) | `_failed.md` + exit 2 |
| Prompt file missing or unreadable | `_failed.md (prompt_parse_error)` + exit 1 |
| Prompt first line not `#` or `>` | `_failed.md (prompt_schema_error)` + exit 1 |
| handoffCommand missing (non-interactive mode) | `_failed.md (execution_error)` + exit 1 |
| handoffCommand exits non-zero | `_failed.md (execution_error)` + exit non-zero |
| Memory evidence missing (required mode) | `_failed.md (prompt_schema_error)` + exit 1 |
| Memory evidence missing (optional mode) | console.warn, continue |
| Artifact verification fails | `_failed.md (prompt_schema_error)` + exit non-zero |
| Validation command exits non-zero | `_failed.md (execution_error)` + exit non-zero |
| lock:complete exits non-zero | `_failed.md` + exit non-zero |
## Security considerations
- ⚠️ Environment variables injected into child processes (`SCHEDULER_AGENT`,
`SCHEDULER_CADENCE`, `SCHEDULER_PROMPT_PATH`, `SCHEDULER_MEMORY_FILE`) are derived
from the roster and config — not from user input. No sanitization issue here, but
ensure `torch-config.json` is not writable by untrusted parties.
- The handoff command is executed via `bash -lc` with a login shell. The command
string comes from `torch-config.json`. Do not allow untrusted config sources.
- No cryptographic or signing logic in this file.
## Performance & concurrency
- The loop is single-threaded and sequential; no parallel lock attempts.
- Each iteration makes several network calls (lock:check, lock:lock, lock:complete)
that may take seconds each. Total wall-clock time per cycle depends heavily on
relay latency and handoff command duration.
- `getLatestFile()` reads all log filenames and file contents synchronously in
memory — acceptable for typical log counts (< 1000 files) but worth monitoring
if logs accumulate without rotation.
## Related files and call graph
| File | Relationship |
|------|-------------|
| `scripts/agent/scheduler-utils.mjs` | Utility functions (runCommand, readJson, parsers, etc.) |
| `scripts/agent/scheduler-lock.mjs` | Lock acquisition, error classification, preflight |
| `scripts/agent/verify-run-artifacts.mjs` | Run artifact validation (called as subprocess) |
| `scripts/agent/scheduler-utils.mjs:buildRecentlyRunExclusionSet` | Time-window guard |
| `src/utils.mjs:detectPlatform` | Auto-detects agent platform |
| `src/prompts/roster.json` | Agent roster (read at startup) |
| `src/prompts/scheduler-flow.md` | Authoritative numbered steps (implemented by this script) |
| `torch-config.json` | Handoff command, memory policy, lock policy |
| `task-logs/<cadence>/` | Task logs written and read here |
| `task-logs/<cadence>/.scheduler-run-state.json` | Deferral state (day-scoped) |
| `memory-updates/` | Memory update files (written path injected as env var) |
## Why it works this way
- **Infinite loop with restart on race loss**: Distributed lock systems inherently
have races; the loop lets the scheduler immediately try the next available agent
without requiring the caller to re-invoke the script.
- **Day-scoped run-state**: Deferral tracking needs to persist across multiple rapid
scheduler invocations on the same day (e.g., cron every 5 minutes) but should
reset daily so a failed relay doesn't block the next day's run.
- **lock:complete gated behind all validation**: Ensures a completion event on the
relay (which prevents future agents from re-running the same slot) is only
published after the run is genuinely finished and verified. This prevents
"green-but-worthless" completions.
- **Subprocess model for handoff**: Agents run in their own process with
environment variables injected by the scheduler. This isolates agent crashes
and allows the scheduler to check exit codes cleanly.
## When to change
- **New validation gate**: Add a step between artifact check and lock:complete.
Maintain the gate ordering: memory evidence → artifact check → validation → lock:complete.
- **New failure category**: Add to the `FAILURE_CATEGORY` constant and document
in `src/prompts/scheduler-flow.md`.
- **New config option**: Add parsing in `getSchedulerConfig()` with an env var
override following the existing pattern.
- **Log format change**: Update `writeLog()` — also update `verify-run-artifacts.mjs`
and `scheduler-utils.mjs:parseFrontmatterAgent` to match.
## Tests that validate behavior
- `test/run-scheduler-cycle-memory-policy.test.mjs` — memory policy enforcement
- `test/scheduler-time-window.test.mjs` — time-window guard behavior
- `test/scheduler-preflight-lock.e2e.test.mjs` — preflight + lock flow
```bash
# Run all scheduler tests:
node --test test/run-scheduler-cycle-memory-policy.test.mjs test/scheduler-time-window.test.mjs
# Or full suite (may be slow in sandbox):
npm test
```