TORCH/docs/scheduler-cycle-overview.md

Scheduler Cycle Overview (scripts/agent/run-scheduler-cycle.mjs)

What this module does

run-scheduler-cycle.mjs is the main scheduler entry point. Given a cadence (daily or weekly), it selects the next eligible agent from the roster, acquires a distributed Nostr lock to prevent duplicate execution across agents, runs the selected agent's prompt via a configured handoff command, verifies that required run artifacts and memory evidence were produced, then publishes task completion.

This script owns the full agent lifecycle for one slot per invocation: lock acquisition → prompt execution → artifact/memory verification → lock:complete. Spawned agents must not call lock:complete themselves — the scheduler does it after all validation passes.

Where it fits

npm run scheduler:daily
   └─ scripts/agent/run-scheduler-cycle.mjs daily
         ├─ src/prompts/roster.json          (agent list)
         ├─ torch-config.json                (handoff command, memory policy, etc.)
         ├─ scripts/agent/scheduler-utils.mjs
         ├─ scripts/agent/scheduler-lock.mjs
         ├─ npm run lock:check:daily         (relay query)
         ├─ npm run lock:lock                (relay publish)
         ├─ <handoffCommand>                 (spawns the agent)
         ├─ scripts/agent/verify-run-artifacts.mjs
         └─ npm run lock:complete            (relay publish, permanent)

Typical call flow

1. Parse args → cadence='daily', platform='codex', model=null

2. Load roster from src/prompts/roster.json → ['audit-agent', 'docs-agent', ...]

3. Read AGENTS.md → print to stdout (best-effort; non-fatal if missing)

4. [LOOP] Read run-state → { run_date: '2026-02-20', lock_deferral: null }

5. [LOOP] Lock health preflight (if enabled):
     npm run lock:health -- --cadence daily
     → non-zero: write _deferred.md or _failed.md, exit

6. [LOOP] npm run lock:check:daily -- --json --quiet
     → exclusion set: { locked: ['ci-health-agent'], paused: [], completed: [] }

7. [LOOP] Time-window guard: exclude agents that ran within 24h window

8. [LOOP] Round-robin select:
     roster = ['audit-agent', 'ci-health-agent', 'docs-agent', ...]
     previous = 'ci-health-agent' (from latest log)
     excluded = { 'ci-health-agent' }
     → selected = 'docs-agent'

9. [LOOP] npm run lock:lock -- --agent docs-agent --cadence daily
     → exit 0: lock acquired (continue)
     → exit 3: race lost (restart loop at step 4)
     → exit 2: backend error → defer (non-strict) or fail (strict)

10. [LOOP] Validate src/prompts/daily/docs-agent.md
      → not readable → _failed.md (prompt_parse_error), exit 1
      → bad first line → _failed.md (prompt_schema_error), exit 1

11. [LOOP] Clear deferral state. Run memory retrieve command (if configured).

12. [LOOP] Run handoff command (e.g. `claude --prompt "$SCHEDULER_PROMPT_PATH"`)
      → non-zero → _failed.md (execution_error), exit non-zero
      → missing → _failed.md, exit 1

13. [LOOP] Run memory store command (if configured).

14. [LOOP] Verify memory evidence (markers/artifacts):
      → mode='required' + missing → _failed.md, exit 1
      → mode='optional' + missing → console.warn, continue

15. [LOOP] node scripts/agent/verify-run-artifacts.mjs --since <runStart>
      → non-zero → _failed.md (prompt_schema_error), exit non-zero

16. [LOOP] npm run lint (or configured validationCommands)
      → non-zero → _failed.md (execution_error), exit non-zero

17. [LOOP] npm run lock:complete -- --agent docs-agent --cadence daily
      → non-zero → _failed.md, exit non-zero

18. [LOOP] Write _completed.md task log. Print run summary. Exit 0.

Public API summary

This is a CLI script — it has no exported functions. The entry point is:

node scripts/agent/run-scheduler-cycle.mjs <daily|weekly> [--platform <p>] [--model <m>]
# or via npm:
npm run scheduler:daily
npm run scheduler:weekly

Environment variables:

Variable	Source	Purpose
AGENT_PLATFORM	env	Platform identifier (codex, claude, linux)
AGENT_MODEL	env	Model identifier forwarded to lock events
SCHEDULER_LOCK_MAX_RETRIES	env	Override lock:lock retry count (default: 2)
SCHEDULER_LOCK_BACKOFF_MS	env	Override base backoff ms (default: 250)
SCHEDULER_LOCK_JITTER_MS	env	Override jitter ms (default: 75)
SCHEDULER_LOCK_HEALTH_PREFLIGHT	env	Enable/disable relay health preflight
SCHEDULER_SKIP_LOCK_HEALTH_PREFLIGHT	env	Skip preflight even if enabled
SCHEDULER_STRICT_LOCK	env	Override strict_lock policy
SCHEDULER_DEGRADED_LOCK_RETRY_WINDOW_MS	env	Override deferral window ms
SCHEDULER_MAX_DEFERRALS	env	Override max deferral count

Variables injected into child processes:

Variable	Value	Purpose
SCHEDULER_AGENT	selected agent name	Used by handoff/memory commands
SCHEDULER_CADENCE	daily|weekly	Used by handoff/memory commands
SCHEDULER_PROMPT_PATH	absolute path	Path to prompt file
SCHEDULER_MEMORY_FILE	absolute path	Agent writes learnings here

Key internal functions

Function	Purpose
parseArgs(argv)	Parses cadence, platform, model from CLI args
getSchedulerConfig(cadence, opts)	Merges torch-config.json + env overrides
readRunState(cadence) / writeRunState(path, state)	Deferral state persistence (day-scoped)
getLatestFile(logDir)	Finds most recent valid task log for previous-agent detection
selectNextAgent({roster, excludedSet, previousAgent, firstPrompt})	Round-robin with exclusion
validatePromptFile(promptPath)	Reads and validates prompt format
verifyMemoryStep({name, markers, artifacts, outputText, sinceMs})	Evidence check
writeLog({cadence, agent, status, ...})	Writes YAML-frontmatter task log file
exitWithSummary(code, data)	Prints summary + calls process.exit

Key invariants

1. lock:complete gate: npm run lock:complete is called only after memory verification (step 14), artifact verification (step 15), and all validation commands (step 16) pass. If any gate fails, the run ends without publishing completion.

2. Completed log gate: _completed.md task log is written only after lock:complete exits 0 (step 17). Earlier failures always write _failed.md or _deferred.md.

3. Race resolution: Lock exit code 3 (race lost) causes the scheduler to restart the loop at step 4, re-checking what agents are available. This is the normal race resolution path — not an error.

4. Time-window guard: Agents that appear in a recent local log (within 24h for daily, 7d for weekly) are added to the exclusion set regardless of relay state. This handles the cross-midnight edge case where the relay date-scope has rolled over.

5. Cycle saturation: When all roster agents are in the time-window exclusion set, the scheduler exits 0 with a "cycle complete" reason — not treated as an error by cron/CI.

6. Deferral state is day-scoped: task-logs/<cadence>/.scheduler-run-state.json is reset whenever run_date changes, so deferral budgets do not carry across days.

Edge cases & error paths

Scenario	Behavior
No roster entries	process.exit(1) before loop
Roster fully excluded (not cycle-saturated)	_failed.md + exit 1
All relays unhealthy (preflight)	_deferred.md + exit 0
Lock backend error (strict mode)	_failed.md + exit 2
Lock backend error (non-strict, within window/budget)	_deferred.md + exit 0; deferral state persisted
Lock backend error (non-strict, budget exhausted)	_failed.md + exit 2
Prompt file missing or unreadable	_failed.md (prompt_parse_error) + exit 1
Prompt first line not # or >	_failed.md (prompt_schema_error) + exit 1
handoffCommand missing (non-interactive mode)	_failed.md (execution_error) + exit 1
handoffCommand exits non-zero	_failed.md (execution_error) + exit non-zero
Memory evidence missing (required mode)	_failed.md (prompt_schema_error) + exit 1
Memory evidence missing (optional mode)	console.warn, continue
Artifact verification fails	_failed.md (prompt_schema_error) + exit non-zero
Validation command exits non-zero	_failed.md (execution_error) + exit non-zero
lock:complete exits non-zero	_failed.md + exit non-zero

Security considerations

• ⚠️ Environment variables injected into child processes (SCHEDULER_AGENT, SCHEDULER_CADENCE, SCHEDULER_PROMPT_PATH, SCHEDULER_MEMORY_FILE) are derived from the roster and config — not from user input. No sanitization issue here, but ensure torch-config.json is not writable by untrusted parties.
• The handoff command is executed via bash -lc with a login shell. The command string comes from torch-config.json. Do not allow untrusted config sources.
• No cryptographic or signing logic in this file.

Performance & concurrency

• The loop is single-threaded and sequential; no parallel lock attempts.
• Each iteration makes several network calls (lock:check, lock:lock, lock:complete) that may take seconds each. Total wall-clock time per cycle depends heavily on relay latency and handoff command duration.
• getLatestFile() reads all log filenames and file contents synchronously in memory — acceptable for typical log counts (< 1000 files) but worth monitoring if logs accumulate without rotation.

Related files and call graph

File	Relationship
scripts/agent/scheduler-utils.mjs	Utility functions (runCommand, readJson, parsers, etc.)
scripts/agent/scheduler-lock.mjs	Lock acquisition, error classification, preflight
scripts/agent/verify-run-artifacts.mjs	Run artifact validation (called as subprocess)
scripts/agent/scheduler-utils.mjs:buildRecentlyRunExclusionSet	Time-window guard
src/utils.mjs:detectPlatform	Auto-detects agent platform
src/prompts/roster.json	Agent roster (read at startup)
src/prompts/scheduler-flow.md	Authoritative numbered steps (implemented by this script)
torch-config.json	Handoff command, memory policy, lock policy
task-logs/<cadence>/	Task logs written and read here
task-logs/<cadence>/.scheduler-run-state.json	Deferral state (day-scoped)
memory-updates/	Memory update files (written path injected as env var)

Why it works this way

• Infinite loop with restart on race loss: Distributed lock systems inherently have races; the loop lets the scheduler immediately try the next available agent without requiring the caller to re-invoke the script.
• Day-scoped run-state: Deferral tracking needs to persist across multiple rapid scheduler invocations on the same day (e.g., cron every 5 minutes) but should reset daily so a failed relay doesn't block the next day's run.
• lock:complete gated behind all validation: Ensures a completion event on the relay (which prevents future agents from re-running the same slot) is only published after the run is genuinely finished and verified. This prevents "green-but-worthless" completions.
• Subprocess model for handoff: Agents run in their own process with environment variables injected by the scheduler. This isolates agent crashes and allows the scheduler to check exit codes cleanly.

When to change

• New validation gate: Add a step between artifact check and lock:complete. Maintain the gate ordering: memory evidence → artifact check → validation → lock:complete.
• New failure category: Add to the FAILURE_CATEGORY constant and document in src/prompts/scheduler-flow.md.
• New config option: Add parsing in getSchedulerConfig() with an env var override following the existing pattern.
• Log format change: Update writeLog() — also update verify-run-artifacts.mjs and scheduler-utils.mjs:parseFrontmatterAgent to match.

Tests that validate behavior

• test/run-scheduler-cycle-memory-policy.test.mjs — memory policy enforcement
• test/scheduler-time-window.test.mjs — time-window guard behavior
• test/scheduler-preflight-lock.e2e.test.mjs — preflight + lock flow

# Run all scheduler tests:
node --test test/run-scheduler-cycle-memory-policy.test.mjs test/scheduler-time-window.test.mjs
# Or full suite (may be slow in sandbox):
npm test