From 9e5797c5e099d338852992a571c5a51e78c4fb33 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Feb 2026 20:29:38 +0000 Subject: [PATCH 1/4] fix(agents): add GitHub API fallback for task locking in restricted environments Jules agents cannot git push due to sandbox restrictions, which breaks the draft PR-based distributed lock mechanism. Multiple agents can pick up the same task because the claim is never visible remotely. This adds scripts/agent/claim-task-api.sh which creates the remote branch, started.md log file, and draft PR entirely via curl and the GitHub REST API, bypassing git push. The scheduler-flow.md, META_PROMPTS.md, and AGENTS.md are updated to document the fallback: try git push first, fall back to the API script if push is blocked, fail-closed if neither method works. https://claude.ai/code/session_01BpDsVSAkVWtJLnfKxsB9ib --- AGENTS.md | 18 +- docs/agents/prompts/META_PROMPTS.md | 18 +- docs/agents/prompts/scheduler-flow.md | 66 +++++++- scripts/agent/claim-task-api.sh | 228 ++++++++++++++++++++++++++ 4 files changed, 320 insertions(+), 10 deletions(-) create mode 100755 scripts/agent/claim-task-api.sh diff --git a/AGENTS.md b/AGENTS.md index fe9949e1..ae8a1ba6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -256,14 +256,28 @@ To prevent multiple agents from working on the same task simultaneously, every a ``` If a matching PR exists (open or draft) for the same agent or subsystem, **skip the task** — another agent is already working on it. -2. **Claim the task.** If no matching PR exists: +2. **Claim the task.** If no matching PR exists, use one of these methods: + + **Method A — git push (preferred when available):** a. Create your working branch. b. Make a minimal initial commit (e.g., create a `context/CONTEXT_.md` with the task scope). c. Create a `started` log file in the appropriate task-log directory. d. Push the branch immediately to make the claim visible. e. Only after the branch is pushed and the claim is visible, begin the actual work. -3. **Handle race conditions.** After pushing your claim, re-check for duplicates: + **Method B — GitHub API fallback (when git push is blocked):** + If `git push` is unavailable (e.g., Jules sandbox), use the API claim script: + ```bash + bash scripts/agent/claim-task-api.sh \ + --agent \ + --cadence \ + --base unstable + ``` + This creates the remote branch, `_started.md` log file, and draft PR entirely via `curl` and the GitHub REST API. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. The script outputs `CLAIM_BRANCH`, `CLAIM_PR_NUMBER`, and `CLAIM_PR_URL` for use in subsequent steps. + + If **neither method** works, do not proceed — write a `_failed.md` log and stop. + +3. **Handle race conditions.** After your claim is visible remotely, re-check for duplicates: ``` curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, titles: [.[].title]}' ``` diff --git a/docs/agents/prompts/META_PROMPTS.md b/docs/agents/prompts/META_PROMPTS.md index 07f7c875..ccd36885 100644 --- a/docs/agents/prompts/META_PROMPTS.md +++ b/docs/agents/prompts/META_PROMPTS.md @@ -24,11 +24,16 @@ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_pag MUST 3: Run the exact command list in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 -3) [claim] create branch, create+commit `_started.md`, push +3) [claim] Try: create branch, create+commit `_started.md`, push. + IF git push fails or is blocked, use the API fallback instead: + bash scripts/agent/claim-task-api.sh --agent --cadence daily --base unstable + This creates the remote branch, started.md, and draft PR via curl. + Requires GITHUB_TOKEN or GH_TOKEN in environment. + IF both methods fail, write `_failed.md` and stop. 4) rerun preflight command from MUST 2 5) execute selected prompt from docs/agents/prompts/daily/ 6) npm run lint -7) create `_completed.md` or `_failed.md`, commit, push +7) create `_completed.md` or `_failed.md`, commit, push (or API update) MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #)`. @@ -55,11 +60,16 @@ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_pag MUST 3: Run the exact command list in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/weekly/ | sort | tail -n 1 -3) [claim] create branch, create+commit `_started.md`, push +3) [claim] Try: create branch, create+commit `_started.md`, push. + IF git push fails or is blocked, use the API fallback instead: + bash scripts/agent/claim-task-api.sh --agent --cadence weekly --base unstable + This creates the remote branch, started.md, and draft PR via curl. + Requires GITHUB_TOKEN or GH_TOKEN in environment. + IF both methods fail, write `_failed.md` and stop. 4) rerun preflight command from MUST 2 5) execute selected prompt from docs/agents/prompts/weekly/ 6) npm run lint -7) create `_completed.md` or `_failed.md`, commit, push +7) create `_completed.md` or `_failed.md`, commit, push (or API update) MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #)`. diff --git a/docs/agents/prompts/scheduler-flow.md b/docs/agents/prompts/scheduler-flow.md index 23ae58f5..703827e4 100644 --- a/docs/agents/prompts/scheduler-flow.md +++ b/docs/agents/prompts/scheduler-flow.md @@ -50,19 +50,35 @@ During scheduler execution, the scheduler-specific instructions in this document 6. **MUST** stop and write a `_failed.md` log with `All roster tasks currently claimed by other agents` when every roster agent is excluded. -7. **MUST** claim the selected agent in this exact order: +7. **MUST** claim the selected agent using **one** of these methods (try in order): + + **Method A — git push (preferred):** 1) create branch, 2) create and commit `__started.md` in ``, 3) push branch and commit. -8. **MUST** re-run the same PR preflight command from Step 2 immediately after push. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #)`, abandon the run, and return to Step 5. Otherwise print `RACE CHECK: won`. + **Method B — GitHub API fallback (when git push is blocked):** + If `git push` fails or is unavailable (e.g., Jules sandbox), use the API claim script: + ```bash + bash scripts/agent/claim-task-api.sh \ + --agent \ + --cadence \ + --base unstable + ``` + This creates the remote branch, `_started.md` log file, and draft PR entirely via the GitHub REST API using `curl`. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. + + If **both** methods fail (no push access AND no API token), write a `_failed.md` log with reason `Cannot create remote claim: git push blocked and no GITHUB_TOKEN available` and stop. + +8. **MUST** re-run the same PR preflight command from Step 2 immediately after the claim is visible remotely. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #)`, abandon the run, and return to Step 5. Otherwise print `RACE CHECK: won`. 9. **MUST** execute `/` end-to-end. -10. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push. +10. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push (or use the API to update the remote branch if push is blocked). ## Canonical Example Run Output +### Method A (git push) + ```text $ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' { "count": 12, "prs": [ ... ] } @@ -78,7 +94,49 @@ Selected agent: nip-research-agent Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md Pushed branch: agents/daily/nip-research-agent/2026-02-14-run -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' +{ "count": 13, "prs": [ ... ] } +RACE CHECK: won + +$ npm run lint +> lint passed + +Wrote: docs/agents/task-logs/daily/2026-02-14_00-12-00_nip-research-agent_completed.md +``` + +### Method B (API fallback — for Jules or other restricted environments) + +```text +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' +{ "count": 12, "prs": [ ... ] } + +$ cat AGENTS.md CLAUDE.md +... + +$ ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 +2026-02-13_18-40-00_load-test-agent_completed.md + +Selected agent: nip-research-agent + +# git push is blocked — use API fallback: +$ bash scripts/agent/claim-task-api.sh --agent nip-research-agent --cadence daily --base unstable +Claiming task: nip-research-agent (daily) +Branch: agents/daily/nip-research-agent/2026-02-14-run-1707868800 +Step 1: Fetching base branch SHA (unstable)... + Base SHA: abc123... +Step 2: Creating remote branch... + Created: refs/heads/agents/daily/nip-research-agent/2026-02-14-run-1707868800 +Step 3: Creating started.md log file... + Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md +Step 4: Creating draft PR... + PR : +CLAIM_BRANCH=agents/daily/nip-research-agent/2026-02-14-run-1707868800 +CLAIM_PR_NUMBER= +CLAIM_PR_URL= +CLAIM_LOG_FILE=docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md + +# Re-run preflight to check for race: +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' { "count": 13, "prs": [ ... ] } RACE CHECK: won diff --git a/scripts/agent/claim-task-api.sh b/scripts/agent/claim-task-api.sh new file mode 100755 index 00000000..2ed9c3ce --- /dev/null +++ b/scripts/agent/claim-task-api.sh @@ -0,0 +1,228 @@ +#!/usr/bin/env bash +set -euo pipefail + +# claim-task-api.sh — Create a remote branch + started.md + draft PR +# entirely via GitHub REST API, bypassing 'git push'. +# +# This solves the Jules sandbox restriction where git push is blocked +# but curl/HTTP requests are allowed. +# +# Usage: +# ./scripts/agent/claim-task-api.sh \ +# --agent \ +# --cadence \ +# [--base ] \ +# [--run-id ] +# +# Environment: +# GITHUB_TOKEN — Required. A token with repo write access. +# Falls back to GH_TOKEN if GITHUB_TOKEN is unset. +# +# Outputs (stdout): +# CLAIM_BRANCH= +# CLAIM_PR_NUMBER= +# CLAIM_PR_URL= +# +# Exit codes: +# 0 — Claim created successfully +# 1 — Usage error or missing token +# 2 — API error (branch/file/PR creation failed) + +OWNER="PR0M3TH3AN" +REPO="bitvid" +API_BASE="https://api.github.com/repos/${OWNER}/${REPO}" + +# --------------------------------------------------------------------------- +# Parse arguments +# --------------------------------------------------------------------------- +AGENT="" +CADENCE="" +BASE_BRANCH="unstable" +RUN_ID="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --agent) AGENT="$2"; shift 2 ;; + --cadence) CADENCE="$2"; shift 2 ;; + --base) BASE_BRANCH="$2"; shift 2 ;; + --run-id) RUN_ID="$2"; shift 2 ;; + *) + echo "ERROR: Unknown argument: $1" >&2 + echo "Usage: $0 --agent --cadence [--base ] [--run-id ]" >&2 + exit 1 + ;; + esac +done + +if [[ -z "$AGENT" || -z "$CADENCE" ]]; then + echo "ERROR: --agent and --cadence are required." >&2 + exit 1 +fi + +if [[ "$CADENCE" != "daily" && "$CADENCE" != "weekly" ]]; then + echo "ERROR: --cadence must be 'daily' or 'weekly'." >&2 + exit 1 +fi + +# --------------------------------------------------------------------------- +# Resolve token +# --------------------------------------------------------------------------- +TOKEN="${GITHUB_TOKEN:-${GH_TOKEN:-}}" +if [[ -z "$TOKEN" ]]; then + echo "ERROR: GITHUB_TOKEN (or GH_TOKEN) must be set for API access." >&2 + echo "Without a token, remote task locking is impossible." >&2 + exit 1 +fi + +# --------------------------------------------------------------------------- +# Helper: authenticated GitHub API call +# --------------------------------------------------------------------------- +gh_api() { + local method="$1" + local endpoint="$2" + local data="${3:-}" + + local args=( + -s -f + -X "$method" + -H "Authorization: Bearer ${TOKEN}" + -H "Accept: application/vnd.github+json" + -H "X-GitHub-Api-Version: 2022-11-28" + ) + + if [[ -n "$data" ]]; then + args+=(-H "Content-Type: application/json" -d "$data") + fi + + curl "${args[@]}" "${API_BASE}${endpoint}" +} + +# --------------------------------------------------------------------------- +# Generate identifiers +# --------------------------------------------------------------------------- +TIMESTAMP=$(date -u +"%Y-%m-%d_%H-%M-%S") +DATE_ONLY=$(date -u +"%Y-%m-%d") +RUN_ID="${RUN_ID:-$(date -u +"%s")}" +BRANCH_NAME="agents/${CADENCE}/${AGENT}/${DATE_ONLY}-run-${RUN_ID}" +LOG_DIR="docs/agents/task-logs/${CADENCE}" +LOG_FILE="${TIMESTAMP}_${AGENT}_started.md" +LOG_PATH="${LOG_DIR}/${LOG_FILE}" + +echo "Claiming task: ${AGENT} (${CADENCE})" >&2 +echo "Branch: ${BRANCH_NAME}" >&2 +echo "Log file: ${LOG_PATH}" >&2 + +# --------------------------------------------------------------------------- +# Step 1: Get the latest commit SHA of the base branch +# --------------------------------------------------------------------------- +echo "Step 1: Fetching base branch SHA (${BASE_BRANCH})..." >&2 +BASE_REF_JSON=$(gh_api GET "/git/refs/heads/${BASE_BRANCH}") +BASE_SHA=$(echo "$BASE_REF_JSON" | jq -r '.object.sha // empty') + +if [[ -z "$BASE_SHA" ]]; then + echo "ERROR: Could not resolve base branch '${BASE_BRANCH}'." >&2 + echo "API response: ${BASE_REF_JSON}" >&2 + exit 2 +fi +echo " Base SHA: ${BASE_SHA}" >&2 + +# --------------------------------------------------------------------------- +# Step 2: Create the remote branch +# --------------------------------------------------------------------------- +echo "Step 2: Creating remote branch..." >&2 +CREATE_REF_JSON=$(gh_api POST "/git/refs" "$(jq -n \ + --arg ref "refs/heads/${BRANCH_NAME}" \ + --arg sha "$BASE_SHA" \ + '{ref: $ref, sha: $sha}')") + +CREATED_REF=$(echo "$CREATE_REF_JSON" | jq -r '.ref // empty') +if [[ -z "$CREATED_REF" ]]; then + # Check if it's a 422 (already exists) — another agent may have claimed it + ERROR_MSG=$(echo "$CREATE_REF_JSON" | jq -r '.message // empty') + if [[ "$ERROR_MSG" == *"Reference already exists"* ]]; then + echo "RACE DETECTED: Branch '${BRANCH_NAME}' already exists. Another agent claimed first." >&2 + exit 2 + fi + echo "ERROR: Failed to create branch." >&2 + echo "API response: ${CREATE_REF_JSON}" >&2 + exit 2 +fi +echo " Created: ${CREATED_REF}" >&2 + +# --------------------------------------------------------------------------- +# Step 3: Create the started.md log file on the branch +# --------------------------------------------------------------------------- +echo "Step 3: Creating started.md log file..." >&2 + +LOG_CONTENT=$(cat </dev/null || echo "$LOG_CONTENT" | base64) + +CREATE_FILE_JSON=$(gh_api PUT "/contents/${LOG_PATH}" "$(jq -n \ + --arg message "chore(agents): claim ${CADENCE} task for ${AGENT}" \ + --arg content "$ENCODED_CONTENT" \ + --arg branch "$BRANCH_NAME" \ + '{message: $message, content: $content, branch: $branch}')") + +FILE_PATH_CREATED=$(echo "$CREATE_FILE_JSON" | jq -r '.content.path // empty') +if [[ -z "$FILE_PATH_CREATED" ]]; then + echo "ERROR: Failed to create log file." >&2 + echo "API response: ${CREATE_FILE_JSON}" >&2 + # Clean up: delete the branch we created + gh_api DELETE "/git/refs/heads/${BRANCH_NAME}" >/dev/null 2>&1 || true + exit 2 +fi +echo " Created: ${FILE_PATH_CREATED}" >&2 + +# --------------------------------------------------------------------------- +# Step 4: Create a draft PR +# --------------------------------------------------------------------------- +echo "Step 4: Creating draft PR..." >&2 +PR_TITLE="[${CADENCE}] ${AGENT}: ${DATE_ONLY} run" +PR_BODY="Automated task claim for **${AGENT}** (${CADENCE} cadence). + +Branch: \`${BRANCH_NAME}\` +Claimed at: ${TIMESTAMP} UTC +Method: GitHub API (claim-task-api.sh) + +--- +*This draft PR serves as a distributed lock. Do not merge until the agent completes its work.*" + +CREATE_PR_JSON=$(gh_api POST "/pulls" "$(jq -n \ + --arg title "$PR_TITLE" \ + --arg body "$PR_BODY" \ + --arg head "$BRANCH_NAME" \ + --arg base "$BASE_BRANCH" \ + '{title: $title, body: $body, head: $head, base: $base, draft: true}')") + +PR_NUMBER=$(echo "$CREATE_PR_JSON" | jq -r '.number // empty') +PR_URL=$(echo "$CREATE_PR_JSON" | jq -r '.html_url // empty') + +if [[ -z "$PR_NUMBER" ]]; then + echo "WARNING: Draft PR creation failed. Branch and log file exist as partial lock." >&2 + echo "API response: ${CREATE_PR_JSON}" >&2 + # Don't exit — the branch itself is still a visible claim + PR_NUMBER="FAILED" + PR_URL="FAILED" +else + echo " PR #${PR_NUMBER}: ${PR_URL}" >&2 +fi + +# --------------------------------------------------------------------------- +# Output machine-readable results +# --------------------------------------------------------------------------- +echo "CLAIM_BRANCH=${BRANCH_NAME}" +echo "CLAIM_PR_NUMBER=${PR_NUMBER}" +echo "CLAIM_PR_URL=${PR_URL}" +echo "CLAIM_LOG_FILE=${LOG_PATH}" +echo "" +echo "Task claimed successfully." >&2 From 86f322550a3df0a36fc652daecef67a8b39535cd Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Feb 2026 20:53:14 +0000 Subject: [PATCH 2/4] feat(agents): add Nostr-based distributed task locking via NIP-78 Replace GitHub draft PR as the primary locking mechanism with Nostr relay-based locks. Each agent generates an ephemeral keypair, publishes a NIP-78 (Kind 30078) lock event with NIP-40 auto-expiration, and discards the key. No tokens, no secrets, no git push required. New script: scripts/agent/nostr-lock.mjs - check: query relays for active locks, return JSON exclusion set - lock: claim a task with built-in race detection (exit 3 = lost) - list: human-readable table of all active locks The locking fallback chain is now: 1. Nostr lock (no credentials needed, works in any sandbox) 2. git push + draft PR (traditional method) 3. GitHub API via curl (needs GITHUB_TOKEN) Updated scheduler-flow.md, META_PROMPTS.md, and AGENTS.md to document all three methods with Nostr as the preferred primary. https://claude.ai/code/session_01BpDsVSAkVWtJLnfKxsB9ib --- AGENTS.md | 46 ++- docs/agents/prompts/META_PROMPTS.md | 56 +-- docs/agents/prompts/scheduler-flow.md | 132 ++++--- scripts/agent/nostr-lock.mjs | 517 ++++++++++++++++++++++++++ 4 files changed, 662 insertions(+), 89 deletions(-) create mode 100755 scripts/agent/nostr-lock.mjs diff --git a/AGENTS.md b/AGENTS.md index ae8a1ba6..ee495cb1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -244,46 +244,60 @@ When you create a PR or start work, leave a clear signal: - **PR title prefix**: Use a descriptive prefix like `[nostr-core]`, `[playback]`, `[ui]`, `[ci]` so the scope is visible at a glance. - **PR description**: Include a "Files Modified" section listing the key files touched, so other agents can quickly detect conflicts. -### Task Claiming Protocol (Draft PR Lock) +### Task Claiming Protocol (Distributed Lock) -To prevent multiple agents from working on the same task simultaneously, every agent must **check open PRs and create a claim** before starting work. The claim (pushed branch + `started` log file) acts as a distributed lock visible to all agents across all platforms (Claude Code, Codex, Jules). +To prevent multiple agents from working on the same task simultaneously, every agent must **check for existing locks and create a claim** before starting work. The lock is broadcast via Nostr relays (primary) or GitHub PRs (fallback), making it visible to all agents across all platforms (Claude Code, Codex, Jules). **Before starting any task:** -1. **Check for existing claims.** Search for open PRs matching the task: +1. **Check for existing claims.** Run both checks and merge the exclusion sets: + + **Nostr lock check (primary):** + ```bash + node scripts/agent/nostr-lock.mjs check --cadence + ``` + Returns JSON with `locked` (claimed agents) and `available` (free agents) arrays. + + **GitHub PR check (supplemental):** ``` curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, titles: [.[].title]}' ``` - If a matching PR exists (open or draft) for the same agent or subsystem, **skip the task** — another agent is already working on it. + If an agent appears in either the Nostr lock list or has an open PR, **skip that agent**. -2. **Claim the task.** If no matching PR exists, use one of these methods: +2. **Claim the task.** Use one of these methods (try in order): - **Method A — git push (preferred when available):** + **Method A — Nostr lock (preferred, works everywhere Node.js is available):** + ```bash + AGENT_PLATFORM= \ + node scripts/agent/nostr-lock.mjs lock \ + --agent \ + --cadence + ``` + Generates an ephemeral keypair, publishes a NIP-78 lock event with NIP-40 auto-expiration (2 hours) to public Nostr relays. No tokens or secrets needed — the key is used once and discarded. The script includes a built-in race check: exit code 0 = lock acquired, exit code 3 = lost race (another agent claimed first). + + **Method B — git push (fallback when relays are unreachable):** a. Create your working branch. b. Make a minimal initial commit (e.g., create a `context/CONTEXT_.md` with the task scope). c. Create a `started` log file in the appropriate task-log directory. d. Push the branch immediately to make the claim visible. e. Only after the branch is pushed and the claim is visible, begin the actual work. - **Method B — GitHub API fallback (when git push is blocked):** - If `git push` is unavailable (e.g., Jules sandbox), use the API claim script: + **Method C — GitHub API (fallback when git push is also blocked):** ```bash bash scripts/agent/claim-task-api.sh \ --agent \ --cadence \ --base unstable ``` - This creates the remote branch, `_started.md` log file, and draft PR entirely via `curl` and the GitHub REST API. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. The script outputs `CLAIM_BRANCH`, `CLAIM_PR_NUMBER`, and `CLAIM_PR_URL` for use in subsequent steps. + Creates the remote branch, `_started.md` log file, and draft PR via `curl` and the GitHub REST API. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. - If **neither method** works, do not proceed — write a `_failed.md` log and stop. + If **no method** works, do not proceed — write a `_failed.md` log and stop. -3. **Handle race conditions.** After your claim is visible remotely, re-check for duplicates: - ``` - curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, titles: [.[].title]}' - ``` - If you find another agent's PR for the same task that was created **before** yours, abandon your branch and skip the task. +3. **Handle race conditions.** + - If Method A (Nostr) was used, the race check is automatic (built into the script). + - If Method B or C was used, re-check for duplicates via the GitHub PR preflight. If you find another agent's claim that was created **before** yours, abandon your branch and skip the task. -**Scheduler agents:** Daily and weekly scheduler agents must follow this protocol in addition to their directory-based rotation logic (`docs/agents/task-logs/daily/` and `docs/agents/task-logs/weekly/`). The draft PR check happens _after_ determining the next task but _before_ executing it. See the scheduler prompts for the specific implementation steps. +**Scheduler agents:** Daily and weekly scheduler agents must follow this protocol in addition to their directory-based rotation logic (`docs/agents/task-logs/daily/` and `docs/agents/task-logs/weekly/`). The lock check happens _after_ determining the next task but _before_ executing it. See the scheduler prompts for the specific implementation steps. ### Currently In-Flight Work diff --git a/docs/agents/prompts/META_PROMPTS.md b/docs/agents/prompts/META_PROMPTS.md index ccd36885..7ac4d2fe 100644 --- a/docs/agents/prompts/META_PROMPTS.md +++ b/docs/agents/prompts/META_PROMPTS.md @@ -17,25 +17,31 @@ MUST 1: Set cadence config to: - branch_prefix = agents/daily/ - prompt_dir = docs/agents/prompts/daily/ -MUST 2: Run this exact preflight command first: +MUST 2: Run BOTH preflight commands to build the exclusion set: +a) Nostr lock check (primary): +node scripts/agent/nostr-lock.mjs check --cadence daily + +b) GitHub PR check (supplemental): curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' +Exclusion set = union of Nostr locked agents + agents with open PRs. + MUST 3: Run the exact command list in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 -3) [claim] Try: create branch, create+commit `_started.md`, push. - IF git push fails or is blocked, use the API fallback instead: +3) [claim] Use the Nostr lock (preferred, no tokens needed): + AGENT_PLATFORM=jules node scripts/agent/nostr-lock.mjs lock --agent --cadence daily + Exit 0 = lock acquired. Exit 3 = race lost, go back to step 2. + IF Nostr relays are unreachable, try git push fallback. + IF git push also fails, try: bash scripts/agent/claim-task-api.sh --agent --cadence daily --base unstable - This creates the remote branch, started.md, and draft PR via curl. - Requires GITHUB_TOKEN or GH_TOKEN in environment. - IF both methods fail, write `_failed.md` and stop. -4) rerun preflight command from MUST 2 -5) execute selected prompt from docs/agents/prompts/daily/ -6) npm run lint -7) create `_completed.md` or `_failed.md`, commit, push (or API update) + IF all methods fail, write `_failed.md` and stop. +4) execute selected prompt from docs/agents/prompts/daily/ +5) npm run lint +6) create `_completed.md` or `_failed.md`, commit, push (or API update) -MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #)`. +MUST 4: Race rule: the Nostr lock script handles race detection automatically (exit code 3 = lost). For git push/API fallbacks: rerun GitHub PR preflight after claiming. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost`. MUST 5: If all daily agents are excluded, stop and write `_failed.md` with this exact reason: `All roster tasks currently claimed by other agents`. ``` @@ -53,25 +59,31 @@ MUST 1: Set cadence config to: - branch_prefix = agents/weekly/ - prompt_dir = docs/agents/prompts/weekly/ -MUST 2: Run this exact preflight command first: +MUST 2: Run BOTH preflight commands to build the exclusion set: +a) Nostr lock check (primary): +node scripts/agent/nostr-lock.mjs check --cadence weekly + +b) GitHub PR check (supplemental): curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/weekly/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' +Exclusion set = union of Nostr locked agents + agents with open PRs. + MUST 3: Run the exact command list in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/weekly/ | sort | tail -n 1 -3) [claim] Try: create branch, create+commit `_started.md`, push. - IF git push fails or is blocked, use the API fallback instead: +3) [claim] Use the Nostr lock (preferred, no tokens needed): + AGENT_PLATFORM=jules node scripts/agent/nostr-lock.mjs lock --agent --cadence weekly + Exit 0 = lock acquired. Exit 3 = race lost, go back to step 2. + IF Nostr relays are unreachable, try git push fallback. + IF git push also fails, try: bash scripts/agent/claim-task-api.sh --agent --cadence weekly --base unstable - This creates the remote branch, started.md, and draft PR via curl. - Requires GITHUB_TOKEN or GH_TOKEN in environment. - IF both methods fail, write `_failed.md` and stop. -4) rerun preflight command from MUST 2 -5) execute selected prompt from docs/agents/prompts/weekly/ -6) npm run lint -7) create `_completed.md` or `_failed.md`, commit, push (or API update) + IF all methods fail, write `_failed.md` and stop. +4) execute selected prompt from docs/agents/prompts/weekly/ +5) npm run lint +6) create `_completed.md` or `_failed.md`, commit, push (or API update) -MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #)`. +MUST 4: Race rule: the Nostr lock script handles race detection automatically (exit code 3 = lost). For git push/API fallbacks: rerun GitHub PR preflight after claiming. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost`. MUST 5: If all weekly agents are excluded, stop and write `_failed.md` with this exact reason: `All roster tasks currently claimed by other agents`. ``` diff --git a/docs/agents/prompts/scheduler-flow.md b/docs/agents/prompts/scheduler-flow.md index 703827e4..67678261 100644 --- a/docs/agents/prompts/scheduler-flow.md +++ b/docs/agents/prompts/scheduler-flow.md @@ -27,14 +27,24 @@ During scheduler execution, the scheduler-specific instructions in this document - `branch_prefix` = `agents//` - `prompt_dir` = `docs/agents/prompts//` -2. **MUST** run this command first and save the JSON output as the preflight exclusion set: +2. **MUST** run the preflight check to build the exclusion set. Use the Nostr lock check (preferred) with GitHub PR check as supplemental: + **Primary — Nostr lock check:** + ```bash + node scripts/agent/nostr-lock.mjs check --cadence + ``` + This queries Nostr relays for active lock events and returns JSON with `locked` and `available` agent lists. + + **Supplemental — GitHub PR check** (catches agents with open PRs that predate the Nostr system): ```bash curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents//(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' ``` -3. **MUST** derive excluded agents from command output using this order only: - 1) parse `head.ref` as `agents///...`; 2) fallback to title regex; 3) if still unknown, stop with a global lock message. + The exclusion set is the **union** of agents locked via Nostr AND agents with open PRs. + +3. **MUST** derive excluded agents from both outputs: + - From Nostr check: use the `locked` array directly. + - From GitHub PR check: 1) parse `head.ref` as `agents///...`; 2) fallback to title regex; 3) if still unknown, stop with a global lock message. 4. **MUST** run this command and read both policy files before selecting an agent: @@ -42,7 +52,7 @@ During scheduler execution, the scheduler-specific instructions in this document cat AGENTS.md CLAUDE.md ``` -5. **MUST** run this command to identify the latest cadence log file, then choose the next roster agent not excluded by Step 2: +5. **MUST** run this command to identify the latest cadence log file, then choose the next roster agent not in the exclusion set: ```bash ls -1 | sort | tail -n 1 @@ -52,24 +62,34 @@ During scheduler execution, the scheduler-specific instructions in this document 7. **MUST** claim the selected agent using **one** of these methods (try in order): - **Method A — git push (preferred):** + **Method A — Nostr lock (preferred, works everywhere Node.js is available):** + ```bash + AGENT_PLATFORM= \ + node scripts/agent/nostr-lock.mjs lock \ + --agent \ + --cadence + ``` + This generates an ephemeral keypair, publishes a NIP-78 lock event to Nostr relays with NIP-40 auto-expiration (2 hours default), and performs a built-in race check. No tokens or secrets needed. Exit code 0 = won, exit code 3 = lost race. + + **Method B — git push (when Nostr relays are unreachable):** 1) create branch, 2) create and commit `__started.md` in ``, 3) push branch and commit. - **Method B — GitHub API fallback (when git push is blocked):** - If `git push` fails or is unavailable (e.g., Jules sandbox), use the API claim script: + **Method C — GitHub API fallback (when git push is also blocked):** ```bash bash scripts/agent/claim-task-api.sh \ --agent \ --cadence \ --base unstable ``` - This creates the remote branch, `_started.md` log file, and draft PR entirely via the GitHub REST API using `curl`. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. + Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. - If **both** methods fail (no push access AND no API token), write a `_failed.md` log with reason `Cannot create remote claim: git push blocked and no GITHUB_TOKEN available` and stop. + If **all three methods** fail, write a `_failed.md` log with reason `Cannot create remote claim: all locking methods failed` and stop. -8. **MUST** re-run the same PR preflight command from Step 2 immediately after the claim is visible remotely. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #)`, abandon the run, and return to Step 5. Otherwise print `RACE CHECK: won`. +8. **MUST** verify the lock was acquired: + - If Method A was used: the script's built-in race check already ran. If exit code was 0, print `RACE CHECK: won`. If exit code 3, print `RACE CHECK: lost`, abandon, and return to Step 5. + - If Method B or C was used: re-run the GitHub PR preflight from Step 2. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #)`, abandon, and return to Step 5. 9. **MUST** execute `/` end-to-end. @@ -77,71 +97,81 @@ During scheduler execution, the scheduler-specific instructions in this document ## Canonical Example Run Output -### Method A (git push) +### Method A — Nostr lock (preferred) ```text -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' -{ "count": 12, "prs": [ ... ] } +# Step 2: Preflight — check Nostr locks + GitHub PRs +$ node scripts/agent/nostr-lock.mjs check --cadence daily +{ "cadence": "daily", "date": "2026-02-14", "locked": ["audit-agent"], "available": [...], ... } +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' +{ "count": 2, "prs": [ ... ] } + +# Steps 3-5: Read policies, check logs, select agent $ cat AGENTS.md CLAUDE.md -# bitvid — AI Agent Guide ... $ ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 2026-02-13_18-40-00_load-test-agent_completed.md +Selected agent: nip-research-agent + +# Step 7: Claim via Nostr lock +$ AGENT_PLATFORM=jules node scripts/agent/nostr-lock.mjs lock --agent nip-research-agent --cadence daily +Locking: agent=nip-research-agent, cadence=daily, date=2026-02-14 +Step 1: Checking for existing locks... +Step 2: Generating ephemeral keypair... +Step 3: Building lock event... +Step 4: Publishing to relays... + Published to 3/3 relays +Step 5: Race check... +RACE CHECK: won +LOCK_STATUS=ok +LOCK_EVENT_ID=ea2724c76d2bd707... +LOCK_AGENT=nip-research-agent +LOCK_CADENCE=daily +LOCK_EXPIRES_ISO=2026-02-14T02:00:00.000Z + +# Step 9: Execute agent prompt +(agent runs nip-research-agent prompt...) + +# Step 10: Finalize +$ npm run lint +> lint passed + +Wrote: docs/agents/task-logs/daily/2026-02-14_00-12-00_nip-research-agent_completed.md +``` + +### Method B — git push (fallback if relays are unreachable) + +```text +$ node scripts/agent/nostr-lock.mjs check --cadence daily +nostr-lock failed: Relay query timed out +# Nostr relays unreachable — fall back to GitHub PR check only + +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' +{ "count": 12, "prs": [ ... ] } + +$ cat AGENTS.md CLAUDE.md +... + Selected agent: nip-research-agent Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md Pushed branch: agents/daily/nip-research-agent/2026-02-14-run $ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' -{ "count": 13, "prs": [ ... ] } RACE CHECK: won - -$ npm run lint -> lint passed - -Wrote: docs/agents/task-logs/daily/2026-02-14_00-12-00_nip-research-agent_completed.md ``` -### Method B (API fallback — for Jules or other restricted environments) +### Method C — GitHub API (fallback if git push is also blocked) ```text -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' -{ "count": 12, "prs": [ ... ] } - -$ cat AGENTS.md CLAUDE.md -... - -$ ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 -2026-02-13_18-40-00_load-test-agent_completed.md - -Selected agent: nip-research-agent - -# git push is blocked — use API fallback: +# Both Nostr relays and git push failed $ bash scripts/agent/claim-task-api.sh --agent nip-research-agent --cadence daily --base unstable -Claiming task: nip-research-agent (daily) -Branch: agents/daily/nip-research-agent/2026-02-14-run-1707868800 -Step 1: Fetching base branch SHA (unstable)... - Base SHA: abc123... -Step 2: Creating remote branch... - Created: refs/heads/agents/daily/nip-research-agent/2026-02-14-run-1707868800 -Step 3: Creating started.md log file... - Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md -Step 4: Creating draft PR... - PR : CLAIM_BRANCH=agents/daily/nip-research-agent/2026-02-14-run-1707868800 CLAIM_PR_NUMBER= CLAIM_PR_URL= -CLAIM_LOG_FILE=docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md -# Re-run preflight to check for race: $ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' -{ "count": 13, "prs": [ ... ] } RACE CHECK: won - -$ npm run lint -> lint passed - -Wrote: docs/agents/task-logs/daily/2026-02-14_00-12-00_nip-research-agent_completed.md ``` diff --git a/scripts/agent/nostr-lock.mjs b/scripts/agent/nostr-lock.mjs new file mode 100755 index 00000000..d3b02942 --- /dev/null +++ b/scripts/agent/nostr-lock.mjs @@ -0,0 +1,517 @@ +#!/usr/bin/env node + +// scripts/agent/nostr-lock.mjs +// +// Nostr-based distributed task locking for multi-agent coordination. +// +// Each lock generates a fresh ephemeral keypair and discards it. +// Locks auto-expire via NIP-40 — no cleanup needed. +// Other agents discover locks by querying relays for bitvid lock events. +// +// Commands: +// check --cadence JSON list of locked agents +// lock --agent --cadence Claim a lock (exit 0=won, 3=lost) +// list [--cadence ] Human-readable lock table +// +// Environment: +// NOSTR_LOCK_RELAYS Comma-separated relay WSS URLs (overrides defaults) +// NOSTR_LOCK_TTL Lock TTL in seconds (default: 7200 = 2 hours) + +import { generateSecretKey, getPublicKey, finalizeEvent } from 'nostr-tools/pure'; +import { SimplePool } from 'nostr-tools/pool'; +import { useWebSocketImplementation } from 'nostr-tools/relay'; +import WebSocket from 'ws'; + +useWebSocketImplementation(WebSocket); + +// --------------------------------------------------------------------------- +// Configuration +// --------------------------------------------------------------------------- + +const DEFAULT_RELAYS = [ + 'wss://relay.damus.io', + 'wss://nos.lol', + 'wss://relay.primal.net', +]; + +const DEFAULT_TTL = 7200; // 2 hours in seconds +const QUERY_TIMEOUT_MS = 15_000; +const VALID_CADENCES = new Set(['daily', 'weekly']); + +const DAILY_ROSTER = [ + 'audit-agent', + 'ci-health-agent', + 'const-refactor-agent', + 'content-audit-agent', + 'decompose-agent', + 'deps-security-agent', + 'design-system-audit-agent', + 'docs-agent', + 'docs-alignment-agent', + 'docs-code-investigator', + 'innerhtml-migration-agent', + 'known-issues-agent', + 'load-test-agent', + 'nip-research-agent', + 'onboarding-audit-agent', + 'perf-agent', + 'prompt-curator-agent', + 'scheduler-update-agent', + 'style-agent', + 'test-audit-agent', + 'todo-triage-agent', +]; + +const WEEKLY_ROSTER = [ + 'bug-reproducer-agent', + 'changelog-agent', + 'dead-code-agent', + 'event-schema-agent', + 'frontend-console-debug-agent', + 'fuzz-agent', + 'interop-agent', + 'perf-deepdive-agent', + 'perf-optimization-agent', + 'pr-review-agent', + 'race-condition-agent', + 'refactor-agent', + 'smoke-agent', + 'telemetry-agent', + 'test-coverage-agent', + 'weekly-synthesis-agent', +]; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +function getRelays() { + const envRelays = process.env.NOSTR_LOCK_RELAYS; + if (envRelays) { + return envRelays + .split(',') + .map((r) => r.trim()) + .filter(Boolean); + } + return DEFAULT_RELAYS; +} + +function getTtl() { + const envTtl = process.env.NOSTR_LOCK_TTL; + if (envTtl) { + const parsed = parseInt(envTtl, 10); + if (!Number.isNaN(parsed) && parsed > 0) return parsed; + } + return DEFAULT_TTL; +} + +function todayDateStr() { + return new Date().toISOString().slice(0, 10); // YYYY-MM-DD +} + +function todayStartUnix() { + const d = new Date(); + d.setUTCHours(0, 0, 0, 0); + return Math.floor(d.getTime() / 1000); +} + +function nowUnix() { + return Math.floor(Date.now() / 1000); +} + +function getRoster(cadence) { + return cadence === 'daily' ? DAILY_ROSTER : WEEKLY_ROSTER; +} + +/** Parse a lock event into a structured object. */ +function parseLockEvent(event) { + const dTag = event.tags.find((t) => t[0] === 'd')?.[1] ?? ''; + const expTag = event.tags.find((t) => t[0] === 'expiration')?.[1]; + const expiresAt = expTag ? parseInt(expTag, 10) : null; + + let content = {}; + try { + content = JSON.parse(event.content); + } catch { + // content might not be valid JSON + } + + return { + eventId: event.id, + pubkey: event.pubkey, + createdAt: event.created_at, + createdAtIso: new Date(event.created_at * 1000).toISOString(), + expiresAt, + expiresAtIso: expiresAt ? new Date(expiresAt * 1000).toISOString() : null, + dTag, + agent: content.agent ?? null, + cadence: content.cadence ?? null, + status: content.status ?? null, + date: content.date ?? null, + platform: content.platform ?? null, + }; +} + +/** Filter out expired locks. */ +function filterActiveLocks(locks) { + const now = nowUnix(); + return locks.filter((lock) => { + if (lock.expiresAt && lock.expiresAt <= now) return false; + return true; + }); +} + +// --------------------------------------------------------------------------- +// Relay operations +// --------------------------------------------------------------------------- + +/** + * Query relays for lock events matching a cadence and date. + * Returns parsed lock objects. + */ +async function queryLocks(relays, cadence, dateStr) { + const pool = new SimplePool(); + const tagFilter = `bitvid-lock-${cadence}-${dateStr}`; + + try { + const events = await Promise.race([ + pool.querySync(relays, { + kinds: [30078], + '#t': [tagFilter], + }), + new Promise((_, reject) => + setTimeout(() => reject(new Error('Relay query timed out')), QUERY_TIMEOUT_MS), + ), + ]); + + return filterActiveLocks(events.map(parseLockEvent)); + } finally { + pool.close(relays); + } +} + +/** + * Publish a lock event to relays. + * Returns the signed event. + */ +async function publishLock(relays, event) { + const pool = new SimplePool(); + + try { + const publishPromises = pool.publish(relays, event); + const results = await Promise.allSettled(publishPromises); + + const successes = results.filter((r) => r.status === 'fulfilled'); + if (successes.length === 0) { + const errors = results.map((r, i) => { + if (r.status === 'rejected') { + const reason = r.reason; + const msg = reason instanceof Error ? reason.message : String(reason ?? 'unknown'); + return `${relays[i]}: ${msg}`; + } + return `${relays[i]}: unknown`; + }); + throw new Error(`Failed to publish to any relay:\n ${errors.join('\n ')}`); + } + + console.error(` Published to ${successes.length}/${relays.length} relays`); + return event; + } finally { + pool.close(relays); + } +} + +// --------------------------------------------------------------------------- +// Commands +// --------------------------------------------------------------------------- + +async function cmdCheck(cadence) { + const relays = getRelays(); + const dateStr = todayDateStr(); + + console.error(`Checking locks: cadence=${cadence}, date=${dateStr}`); + console.error(`Relays: ${relays.join(', ')}`); + + const locks = await queryLocks(relays, cadence, dateStr); + const lockedAgents = [...new Set(locks.map((l) => l.agent).filter(Boolean))]; + const roster = getRoster(cadence); + const available = roster.filter((a) => !lockedAgents.includes(a)); + + const result = { + cadence, + date: dateStr, + locked: lockedAgents.sort(), + available: available.sort(), + lockCount: locks.length, + locks: locks.map((l) => ({ + agent: l.agent, + eventId: l.eventId, + createdAt: l.createdAtIso, + expiresAt: l.expiresAtIso, + platform: l.platform, + })), + }; + + // Machine-readable JSON to stdout + console.log(JSON.stringify(result, null, 2)); +} + +async function cmdLock(agent, cadence, dryRun = false) { + const relays = getRelays(); + const dateStr = todayDateStr(); + const ttl = getTtl(); + const now = nowUnix(); + const expiresAt = now + ttl; + + console.error(`Locking: agent=${agent}, cadence=${cadence}, date=${dateStr}`); + console.error(`TTL: ${ttl}s, expires: ${new Date(expiresAt * 1000).toISOString()}`); + console.error(`Relays: ${relays.join(', ')}`); + + // Step 1: Check for existing locks on this agent + console.error('Step 1: Checking for existing locks...'); + const existingLocks = await queryLocks(relays, cadence, dateStr); + const conflicting = existingLocks.filter((l) => l.agent === agent); + + if (conflicting.length > 0) { + const earliest = conflicting.sort((a, b) => a.createdAt - b.createdAt)[0]; + console.error( + `LOCK DENIED: ${agent} already locked by event ${earliest.eventId} ` + + `(created ${earliest.createdAtIso}, platform: ${earliest.platform})`, + ); + console.log(`LOCK_STATUS=denied`); + console.log(`LOCK_REASON=already_locked`); + console.log(`LOCK_EXISTING_EVENT=${earliest.eventId}`); + process.exit(3); + } + + // Step 2: Generate ephemeral keypair + console.error('Step 2: Generating ephemeral keypair...'); + const sk = generateSecretKey(); + const pk = getPublicKey(sk); + console.error(` Ephemeral pubkey: ${pk.slice(0, 16)}...`); + + // Step 3: Build and sign the lock event + console.error('Step 3: Building lock event...'); + const event = finalizeEvent( + { + kind: 30078, + created_at: now, + tags: [ + ['d', `bitvid-lock/${cadence}/${agent}/${dateStr}`], + ['t', 'bitvid-agent-lock'], + ['t', `bitvid-lock-${cadence}`], + ['t', `bitvid-lock-${cadence}-${dateStr}`], + ['expiration', String(expiresAt)], + ], + content: JSON.stringify({ + agent, + cadence, + status: 'started', + date: dateStr, + platform: process.env.AGENT_PLATFORM || 'unknown', + lockedAt: new Date(now * 1000).toISOString(), + expiresAt: new Date(expiresAt * 1000).toISOString(), + }), + }, + sk, + ); + + console.error(` Event ID: ${event.id}`); + + // Step 4: Publish to relays + if (dryRun) { + console.error('Step 4: [DRY RUN] Skipping publish — event built but not sent'); + console.error('RACE CHECK: won (dry run — no real contention possible)'); + } else { + console.error('Step 4: Publishing to relays...'); + await publishLock(relays, event); + + // Step 5: Race check — re-query and verify we're the earliest + console.error('Step 5: Race check...'); + // Brief pause to let relays propagate + await new Promise((resolve) => setTimeout(resolve, 1500)); + + const postLocks = await queryLocks(relays, cadence, dateStr); + const racingLocks = postLocks + .filter((l) => l.agent === agent) + .sort((a, b) => a.createdAt - b.createdAt); + + if (racingLocks.length > 1 && racingLocks[0].eventId !== event.id) { + const winner = racingLocks[0]; + console.error( + `RACE CHECK: lost (earlier lock by event ${winner.eventId}, ` + + `created ${winner.createdAtIso})`, + ); + console.log(`LOCK_STATUS=race_lost`); + console.log(`LOCK_REASON=earlier_claim_exists`); + console.log(`LOCK_WINNER_EVENT=${winner.eventId}`); + process.exit(3); + } + + console.error('RACE CHECK: won'); + } + + // Output machine-readable results to stdout + console.log(`LOCK_STATUS=ok`); + console.log(`LOCK_EVENT_ID=${event.id}`); + console.log(`LOCK_PUBKEY=${pk}`); + console.log(`LOCK_AGENT=${agent}`); + console.log(`LOCK_CADENCE=${cadence}`); + console.log(`LOCK_DATE=${dateStr}`); + console.log(`LOCK_EXPIRES=${expiresAt}`); + console.log(`LOCK_EXPIRES_ISO=${new Date(expiresAt * 1000).toISOString()}`); +} + +async function cmdList(cadence) { + const relays = getRelays(); + const dateStr = todayDateStr(); + + const cadences = cadence ? [cadence] : ['daily', 'weekly']; + console.error(`Listing active locks for: ${cadences.join(', ')}`); + + for (const c of cadences) { + const locks = await queryLocks(relays, c, dateStr); + + console.log(`\n${'='.repeat(72)}`); + console.log(`Active ${c} locks (${dateStr})`); + console.log('='.repeat(72)); + + if (locks.length === 0) { + console.log(' (no active locks)'); + continue; + } + + const sorted = locks.sort((a, b) => a.createdAt - b.createdAt); + for (const lock of sorted) { + const age = nowUnix() - lock.createdAt; + const ageMin = Math.round(age / 60); + const remaining = lock.expiresAt ? lock.expiresAt - nowUnix() : null; + const remainMin = remaining ? Math.round(remaining / 60) : '?'; + + console.log( + ` ${(lock.agent ?? 'unknown').padEnd(30)} ` + + `age: ${String(ageMin).padStart(4)}m ` + + `ttl: ${String(remainMin).padStart(4)}m ` + + `platform: ${lock.platform ?? '?'} ` + + `event: ${lock.eventId?.slice(0, 12)}...`, + ); + } + + const roster = getRoster(c); + const lockedAgents = new Set(locks.map((l) => l.agent).filter(Boolean)); + const available = roster.filter((a) => !lockedAgents.has(a)); + console.log(`\n Locked: ${lockedAgents.size}/${roster.length}`); + console.log(` Available: ${available.join(', ') || '(none)'}`); + } +} + +// --------------------------------------------------------------------------- +// CLI argument parsing +// --------------------------------------------------------------------------- + +function parseArgs(argv) { + const args = { command: null, agent: null, cadence: null, dryRun: false }; + let i = 0; + + // First positional arg is the command + if (argv.length > 0 && !argv[0].startsWith('-')) { + args.command = argv[0]; + i = 1; + } + + for (; i < argv.length; i++) { + const arg = argv[i]; + if (arg === '--agent' || arg === '-a') { + args.agent = argv[++i]; + } else if (arg === '--cadence' || arg === '-c') { + args.cadence = argv[++i]; + } else if (arg.startsWith('--agent=')) { + args.agent = arg.split('=')[1]; + } else if (arg.startsWith('--cadence=')) { + args.cadence = arg.split('=')[1]; + } else if (arg === '--dry-run') { + args.dryRun = true; + } + } + + return args; +} + +function usage() { + console.error(`Usage: node scripts/agent/nostr-lock.mjs [options] + +Commands: + check --cadence Check which agents are locked (JSON) + lock --agent --cadence Claim a lock for an agent + list [--cadence ] Human-readable lock table + +Options: + --dry-run Build and sign the event but don't publish (for testing) + +Environment: + NOSTR_LOCK_RELAYS Comma-separated relay WSS URLs + NOSTR_LOCK_TTL Lock TTL in seconds (default: 7200) + AGENT_PLATFORM Platform identifier (e.g., "jules", "claude-code") + +Exit codes: + 0 Success (lock acquired or check complete) + 1 Usage error + 2 Relay/network error + 3 Lock denied (already locked or race lost)`); +} + +// --------------------------------------------------------------------------- +// Main +// --------------------------------------------------------------------------- + +async function main() { + const args = parseArgs(process.argv.slice(2)); + + if (!args.command) { + usage(); + process.exit(1); + } + + switch (args.command) { + case 'check': { + if (!args.cadence || !VALID_CADENCES.has(args.cadence)) { + console.error('ERROR: --cadence is required for check'); + process.exit(1); + } + await cmdCheck(args.cadence); + break; + } + + case 'lock': { + if (!args.agent) { + console.error('ERROR: --agent is required for lock'); + process.exit(1); + } + if (!args.cadence || !VALID_CADENCES.has(args.cadence)) { + console.error('ERROR: --cadence is required for lock'); + process.exit(1); + } + await cmdLock(args.agent, args.cadence, args.dryRun); + break; + } + + case 'list': { + if (args.cadence && !VALID_CADENCES.has(args.cadence)) { + console.error('ERROR: --cadence must be daily or weekly'); + process.exit(1); + } + await cmdList(args.cadence || null); + break; + } + + default: + console.error(`ERROR: Unknown command: ${args.command}`); + usage(); + process.exit(1); + } +} + +main().catch((err) => { + console.error(`nostr-lock failed: ${err.message}`); + process.exit(2); +}); From 445262110b61c9945dd315464c321c656a2a4537 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Feb 2026 20:59:44 +0000 Subject: [PATCH 3/4] refactor(agents): simplify scheduler to Nostr-only locking MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove the git push and GitHub API fallback methods from the scheduler flow, leaving only the Nostr lock as the single locking mechanism. This significantly reduces the agent reasoning chain: - scheduler-flow.md: 10 steps → 8 steps, single code path - META_PROMPTS.md: 5 MUST blocks → 4, no jq pipeline or branching - AGENTS.md: 3-method protocol → 2 commands (check + lock) The agent no longer needs to parse jq output, decide between methods, handle cascading fallbacks, or perform manual race detection. It runs two scripts and checks exit codes. https://claude.ai/code/session_01BpDsVSAkVWtJLnfKxsB9ib --- AGENTS.md | 45 +++------- docs/agents/prompts/META_PROMPTS.md | 58 +++++-------- docs/agents/prompts/scheduler-flow.md | 115 +++++++------------------- 3 files changed, 61 insertions(+), 157 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ee495cb1..ce7d3cd2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -244,58 +244,35 @@ When you create a PR or start work, leave a clear signal: - **PR title prefix**: Use a descriptive prefix like `[nostr-core]`, `[playback]`, `[ui]`, `[ci]` so the scope is visible at a glance. - **PR description**: Include a "Files Modified" section listing the key files touched, so other agents can quickly detect conflicts. -### Task Claiming Protocol (Distributed Lock) +### Task Claiming Protocol (Nostr Distributed Lock) -To prevent multiple agents from working on the same task simultaneously, every agent must **check for existing locks and create a claim** before starting work. The lock is broadcast via Nostr relays (primary) or GitHub PRs (fallback), making it visible to all agents across all platforms (Claude Code, Codex, Jules). +To prevent multiple agents from working on the same task simultaneously, every agent must **check for existing locks and create a claim** before starting work. Locks are broadcast via Nostr relays using NIP-78 events with NIP-40 auto-expiration, making them visible to all agents across all platforms (Claude Code, Codex, Jules). No tokens or secrets are needed. **Before starting any task:** -1. **Check for existing claims.** Run both checks and merge the exclusion sets: - - **Nostr lock check (primary):** +1. **Check for existing claims:** ```bash node scripts/agent/nostr-lock.mjs check --cadence ``` - Returns JSON with `locked` (claimed agents) and `available` (free agents) arrays. + Returns JSON with `locked` (claimed agents) and `available` (free agents) arrays. If an agent appears in the `locked` list, **skip it**. - **GitHub PR check (supplemental):** - ``` - curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, titles: [.[].title]}' - ``` - If an agent appears in either the Nostr lock list or has an open PR, **skip that agent**. - -2. **Claim the task.** Use one of these methods (try in order): - - **Method A — Nostr lock (preferred, works everywhere Node.js is available):** +2. **Claim the task:** ```bash AGENT_PLATFORM= \ node scripts/agent/nostr-lock.mjs lock \ --agent \ --cadence ``` - Generates an ephemeral keypair, publishes a NIP-78 lock event with NIP-40 auto-expiration (2 hours) to public Nostr relays. No tokens or secrets needed — the key is used once and discarded. The script includes a built-in race check: exit code 0 = lock acquired, exit code 3 = lost race (another agent claimed first). + The script generates an ephemeral keypair, publishes a lock event to public Nostr relays, and performs a built-in race check. The key is used once and discarded. Locks auto-expire after 2 hours (configurable via `NOSTR_LOCK_TTL`). - **Method B — git push (fallback when relays are unreachable):** - a. Create your working branch. - b. Make a minimal initial commit (e.g., create a `context/CONTEXT_.md` with the task scope). - c. Create a `started` log file in the appropriate task-log directory. - d. Push the branch immediately to make the claim visible. - e. Only after the branch is pushed and the claim is visible, begin the actual work. + - **Exit 0** = lock acquired, begin work. + - **Exit 3** = race lost, another agent claimed first. Go back to step 1. + - **Exit 2** = relay error. Write a `_failed.md` log and stop. - **Method C — GitHub API (fallback when git push is also blocked):** +3. **View all active locks** (optional, for debugging): ```bash - bash scripts/agent/claim-task-api.sh \ - --agent \ - --cadence \ - --base unstable + node scripts/agent/nostr-lock.mjs list ``` - Creates the remote branch, `_started.md` log file, and draft PR via `curl` and the GitHub REST API. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. - - If **no method** works, do not proceed — write a `_failed.md` log and stop. - -3. **Handle race conditions.** - - If Method A (Nostr) was used, the race check is automatic (built into the script). - - If Method B or C was used, re-check for duplicates via the GitHub PR preflight. If you find another agent's claim that was created **before** yours, abandon your branch and skip the task. **Scheduler agents:** Daily and weekly scheduler agents must follow this protocol in addition to their directory-based rotation logic (`docs/agents/task-logs/daily/` and `docs/agents/task-logs/weekly/`). The lock check happens _after_ determining the next task but _before_ executing it. See the scheduler prompts for the specific implementation steps. diff --git a/docs/agents/prompts/META_PROMPTS.md b/docs/agents/prompts/META_PROMPTS.md index 7ac4d2fe..2e2f85ea 100644 --- a/docs/agents/prompts/META_PROMPTS.md +++ b/docs/agents/prompts/META_PROMPTS.md @@ -17,33 +17,24 @@ MUST 1: Set cadence config to: - branch_prefix = agents/daily/ - prompt_dir = docs/agents/prompts/daily/ -MUST 2: Run BOTH preflight commands to build the exclusion set: +MUST 2: Run preflight to get the exclusion set: -a) Nostr lock check (primary): node scripts/agent/nostr-lock.mjs check --cadence daily -b) GitHub PR check (supplemental): -curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' +Use the `locked` array from the JSON output as the exclusion set. -Exclusion set = union of Nostr locked agents + agents with open PRs. - -MUST 3: Run the exact command list in this order: +MUST 3: Run these commands in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 -3) [claim] Use the Nostr lock (preferred, no tokens needed): +3) Select next roster agent not in exclusion set +4) Claim via Nostr lock: AGENT_PLATFORM=jules node scripts/agent/nostr-lock.mjs lock --agent --cadence daily - Exit 0 = lock acquired. Exit 3 = race lost, go back to step 2. - IF Nostr relays are unreachable, try git push fallback. - IF git push also fails, try: - bash scripts/agent/claim-task-api.sh --agent --cadence daily --base unstable - IF all methods fail, write `_failed.md` and stop. -4) execute selected prompt from docs/agents/prompts/daily/ -5) npm run lint -6) create `_completed.md` or `_failed.md`, commit, push (or API update) + Exit 0 = lock acquired, proceed. Exit 3 = race lost, go back to step 2. +5) Execute selected prompt from docs/agents/prompts/daily/ +6) npm run lint +7) Create `_completed.md` or `_failed.md`, commit, push -MUST 4: Race rule: the Nostr lock script handles race detection automatically (exit code 3 = lost). For git push/API fallbacks: rerun GitHub PR preflight after claiming. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost`. - -MUST 5: If all daily agents are excluded, stop and write `_failed.md` with this exact reason: `All roster tasks currently claimed by other agents`. +MUST 4: If all daily agents are excluded, stop and write `_failed.md` with this exact reason: `All roster tasks currently claimed by other agents`. ``` ## Weekly Scheduler Meta Prompt @@ -59,31 +50,22 @@ MUST 1: Set cadence config to: - branch_prefix = agents/weekly/ - prompt_dir = docs/agents/prompts/weekly/ -MUST 2: Run BOTH preflight commands to build the exclusion set: +MUST 2: Run preflight to get the exclusion set: -a) Nostr lock check (primary): node scripts/agent/nostr-lock.mjs check --cadence weekly -b) GitHub PR check (supplemental): -curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/weekly/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' +Use the `locked` array from the JSON output as the exclusion set. -Exclusion set = union of Nostr locked agents + agents with open PRs. - -MUST 3: Run the exact command list in this order: +MUST 3: Run these commands in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/weekly/ | sort | tail -n 1 -3) [claim] Use the Nostr lock (preferred, no tokens needed): +3) Select next roster agent not in exclusion set +4) Claim via Nostr lock: AGENT_PLATFORM=jules node scripts/agent/nostr-lock.mjs lock --agent --cadence weekly - Exit 0 = lock acquired. Exit 3 = race lost, go back to step 2. - IF Nostr relays are unreachable, try git push fallback. - IF git push also fails, try: - bash scripts/agent/claim-task-api.sh --agent --cadence weekly --base unstable - IF all methods fail, write `_failed.md` and stop. -4) execute selected prompt from docs/agents/prompts/weekly/ -5) npm run lint -6) create `_completed.md` or `_failed.md`, commit, push (or API update) + Exit 0 = lock acquired, proceed. Exit 3 = race lost, go back to step 2. +5) Execute selected prompt from docs/agents/prompts/weekly/ +6) npm run lint +7) Create `_completed.md` or `_failed.md`, commit, push -MUST 4: Race rule: the Nostr lock script handles race detection automatically (exit code 3 = lost). For git push/API fallbacks: rerun GitHub PR preflight after claiming. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost`. - -MUST 5: If all weekly agents are excluded, stop and write `_failed.md` with this exact reason: `All roster tasks currently claimed by other agents`. +MUST 4: If all weekly agents are excluded, stop and write `_failed.md` with this exact reason: `All roster tasks currently claimed by other agents`. ``` diff --git a/docs/agents/prompts/scheduler-flow.md b/docs/agents/prompts/scheduler-flow.md index 67678261..96b7f4c0 100644 --- a/docs/agents/prompts/scheduler-flow.md +++ b/docs/agents/prompts/scheduler-flow.md @@ -27,96 +27,75 @@ During scheduler execution, the scheduler-specific instructions in this document - `branch_prefix` = `agents//` - `prompt_dir` = `docs/agents/prompts//` -2. **MUST** run the preflight check to build the exclusion set. Use the Nostr lock check (preferred) with GitHub PR check as supplemental: +2. **MUST** run the preflight check to build the exclusion set: - **Primary — Nostr lock check:** ```bash node scripts/agent/nostr-lock.mjs check --cadence ``` - This queries Nostr relays for active lock events and returns JSON with `locked` and `available` agent lists. - **Supplemental — GitHub PR check** (catches agents with open PRs that predate the Nostr system): - ```bash - curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents//(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' - ``` + This queries Nostr relays for active lock events and returns JSON with `locked` and `available` agent lists. Use the `locked` array as the exclusion set. - The exclusion set is the **union** of agents locked via Nostr AND agents with open PRs. - -3. **MUST** derive excluded agents from both outputs: - - From Nostr check: use the `locked` array directly. - - From GitHub PR check: 1) parse `head.ref` as `agents///...`; 2) fallback to title regex; 3) if still unknown, stop with a global lock message. - -4. **MUST** run this command and read both policy files before selecting an agent: +3. **MUST** read both policy files before selecting an agent: ```bash cat AGENTS.md CLAUDE.md ``` -5. **MUST** run this command to identify the latest cadence log file, then choose the next roster agent not in the exclusion set: +4. **MUST** run this command to identify the latest cadence log file, then choose the next roster agent not in the exclusion set: ```bash ls -1 | sort | tail -n 1 ``` -6. **MUST** stop and write a `_failed.md` log with `All roster tasks currently claimed by other agents` when every roster agent is excluded. +5. **MUST** stop and write a `_failed.md` log with `All roster tasks currently claimed by other agents` when every roster agent is excluded. -7. **MUST** claim the selected agent using **one** of these methods (try in order): +6. **MUST** claim the selected agent: - **Method A — Nostr lock (preferred, works everywhere Node.js is available):** ```bash AGENT_PLATFORM= \ node scripts/agent/nostr-lock.mjs lock \ --agent \ --cadence ``` - This generates an ephemeral keypair, publishes a NIP-78 lock event to Nostr relays with NIP-40 auto-expiration (2 hours default), and performs a built-in race check. No tokens or secrets needed. Exit code 0 = won, exit code 3 = lost race. - **Method B — git push (when Nostr relays are unreachable):** - 1) create branch, - 2) create and commit `__started.md` in ``, - 3) push branch and commit. + The script generates an ephemeral keypair, publishes a NIP-78 lock event to Nostr relays with NIP-40 auto-expiration (2 hours), and performs a built-in race check. No tokens or secrets needed. - **Method C — GitHub API fallback (when git push is also blocked):** - ```bash - bash scripts/agent/claim-task-api.sh \ - --agent \ - --cadence \ - --base unstable - ``` - Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. + - **Exit code 0** = lock acquired, proceed to Step 7. + - **Exit code 3** = race lost (another agent claimed first). Return to Step 2. + - **Exit code 2** = relay error. Write a `_failed.md` log with reason `Nostr relay error` and stop. - If **all three methods** fail, write a `_failed.md` log with reason `Cannot create remote claim: all locking methods failed` and stop. +7. **MUST** execute `/` end-to-end. -8. **MUST** verify the lock was acquired: - - If Method A was used: the script's built-in race check already ran. If exit code was 0, print `RACE CHECK: won`. If exit code 3, print `RACE CHECK: lost`, abandon, and return to Step 5. - - If Method B or C was used: re-run the GitHub PR preflight from Step 2. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #)`, abandon, and return to Step 5. +8. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push. -9. **MUST** execute `/` end-to-end. - -10. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push (or use the API to update the remote branch if push is blocked). - -## Canonical Example Run Output - -### Method A — Nostr lock (preferred) +## Canonical Example Run ```text -# Step 2: Preflight — check Nostr locks + GitHub PRs +# Step 1: Set cadence +cadence=daily + +# Step 2: Preflight — check Nostr locks $ node scripts/agent/nostr-lock.mjs check --cadence daily -{ "cadence": "daily", "date": "2026-02-14", "locked": ["audit-agent"], "available": [...], ... } +{ + "cadence": "daily", + "date": "2026-02-14", + "locked": ["audit-agent"], + "available": ["ci-health-agent", "const-refactor-agent", ...], + "lockCount": 1, + "locks": [...] +} -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' -{ "count": 2, "prs": [ ... ] } - -# Steps 3-5: Read policies, check logs, select agent +# Step 3: Read policies $ cat AGENTS.md CLAUDE.md ... +# Step 4: Check logs, select next agent $ ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 2026-02-13_18-40-00_load-test-agent_completed.md Selected agent: nip-research-agent -# Step 7: Claim via Nostr lock +# Step 6: Claim via Nostr lock $ AGENT_PLATFORM=jules node scripts/agent/nostr-lock.mjs lock --agent nip-research-agent --cadence daily Locking: agent=nip-research-agent, cadence=daily, date=2026-02-14 Step 1: Checking for existing locks... @@ -132,46 +111,12 @@ LOCK_AGENT=nip-research-agent LOCK_CADENCE=daily LOCK_EXPIRES_ISO=2026-02-14T02:00:00.000Z -# Step 9: Execute agent prompt +# Step 7: Execute agent prompt (agent runs nip-research-agent prompt...) -# Step 10: Finalize +# Step 8: Finalize $ npm run lint > lint passed Wrote: docs/agents/task-logs/daily/2026-02-14_00-12-00_nip-research-agent_completed.md ``` - -### Method B — git push (fallback if relays are unreachable) - -```text -$ node scripts/agent/nostr-lock.mjs check --cadence daily -nostr-lock failed: Relay query timed out -# Nostr relays unreachable — fall back to GitHub PR check only - -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' -{ "count": 12, "prs": [ ... ] } - -$ cat AGENTS.md CLAUDE.md -... - -Selected agent: nip-research-agent -Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md -Pushed branch: agents/daily/nip-research-agent/2026-02-14-run - -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' -RACE CHECK: won -``` - -### Method C — GitHub API (fallback if git push is also blocked) - -```text -# Both Nostr relays and git push failed -$ bash scripts/agent/claim-task-api.sh --agent nip-research-agent --cadence daily --base unstable -CLAIM_BRANCH=agents/daily/nip-research-agent/2026-02-14-run-1707868800 -CLAIM_PR_NUMBER= -CLAIM_PR_URL= - -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' -RACE CHECK: won -``` From 9d4c98a8726935fb56b1dbe0263a07258013922e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Feb 2026 21:09:30 +0000 Subject: [PATCH 4/4] docs(agents): add TORCH protocol documentation TORCH (Task Orchestration via Relay-Coordinated Handoff) is the name for the Nostr-based distributed task locking system used to coordinate multi-agent development. Adds docs/agents/TORCH.md covering: - Protocol overview with sequence diagram - NIP-78 event schema with tag breakdown - Ephemeral key lifecycle and why no secrets are needed - Race detection mechanism - Auto-expiration via NIP-40 - Configuration reference - Architecture diagram - Path toward a generalizable drop-in package for any repo Also references TORCH by name in AGENTS.md, scheduler-flow.md, nostr-lock.mjs header, and CLAUDE.md key documentation table. https://claude.ai/code/session_01BpDsVSAkVWtJLnfKxsB9ib --- AGENTS.md | 4 +- CLAUDE.md | 1 + docs/agents/TORCH.md | 309 ++++++++++++++++++++++++++ docs/agents/prompts/scheduler-flow.md | 2 +- scripts/agent/nostr-lock.mjs | 8 +- 5 files changed, 318 insertions(+), 6 deletions(-) create mode 100644 docs/agents/TORCH.md diff --git a/AGENTS.md b/AGENTS.md index ce7d3cd2..a6145a01 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -244,9 +244,9 @@ When you create a PR or start work, leave a clear signal: - **PR title prefix**: Use a descriptive prefix like `[nostr-core]`, `[playback]`, `[ui]`, `[ci]` so the scope is visible at a glance. - **PR description**: Include a "Files Modified" section listing the key files touched, so other agents can quickly detect conflicts. -### Task Claiming Protocol (Nostr Distributed Lock) +### Task Claiming Protocol (TORCH) -To prevent multiple agents from working on the same task simultaneously, every agent must **check for existing locks and create a claim** before starting work. Locks are broadcast via Nostr relays using NIP-78 events with NIP-40 auto-expiration, making them visible to all agents across all platforms (Claude Code, Codex, Jules). No tokens or secrets are needed. +Task coordination uses **TORCH** (Task Orchestration via Relay-Coordinated Handoff) — a decentralized locking protocol built on Nostr. Agents publish ephemeral lock events to public relays; locks auto-expire via NIP-40. No tokens, no secrets, no git push required. Works across all platforms (Claude Code, Codex, Jules). See `docs/agents/TORCH.md` for the full protocol documentation. **Before starting any task:** diff --git a/CLAUDE.md b/CLAUDE.md index 4318b565..be15c0fe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -488,6 +488,7 @@ When assigning work to agents: | `docs/playback-fallback.md` | URL-first strategy | | `docs/moderation/README.md` | Moderation system | | `docs/logging.md` | Logger usage | +| `docs/agents/TORCH.md` | TORCH distributed task locking protocol | | `context/` / `todo/` / `decisions/` / `test_logs/` | Agent persistent state files (see AGENTS.md §15) | --- diff --git a/docs/agents/TORCH.md b/docs/agents/TORCH.md new file mode 100644 index 00000000..83e7e5c4 --- /dev/null +++ b/docs/agents/TORCH.md @@ -0,0 +1,309 @@ +# TORCH: Task Orchestration via Relay-Coordinated Handoff + +TORCH is a decentralized task-locking protocol for multi-agent development. It uses the Nostr protocol to coordinate AI coding agents (Claude Code, Google Jules, OpenAI Codex, etc.) working in parallel on the same repository, preventing them from colliding on the same task. + +## The Problem + +When multiple AI agents operate on a shared codebase, they need a way to signal "I'm working on this" so other agents don't duplicate the effort. The traditional approach uses draft pull requests as locks, but this breaks down in practice: + +- **Sandboxed environments** (e.g., Google Jules) block `git push` and don't provide GitHub CLI tools. +- **Token management** adds operational overhead — GitHub PATs must be created, rotated, and injected into each agent environment. +- **Platform coupling** ties the coordination mechanism to GitHub specifically, making it non-portable. +- **Complex reasoning chains** force agents to parse jq pipelines, handle multi-method fallbacks, and perform manual race detection, consuming tokens and increasing error rates. + +## The Solution + +TORCH replaces all of that with a single protocol: + +1. An agent **publishes a Nostr event** to claim a task. +2. Other agents **query the relay** to see what's claimed. +3. Locks **auto-expire** via NIP-40 — no cleanup required. +4. Each lock uses an **ephemeral keypair** — no secrets to manage. + +The entire locking mechanism is handled by a single script (`scripts/agent/nostr-lock.mjs`) that any agent can call with two commands. The agent's reasoning chain reduces to: "run check, run lock, read exit code." + +## How It Works + +### Protocol Overview + +``` +Agent A Nostr Relays Agent B + | | | + |-- check (query locks) ---->| | + |<--- JSON: locked=[] -------| | + | | | + |-- lock (publish event) --->| | + |<--- OK (published) --------| | + | | | + |-- race check (re-query) -->| | + |<--- only my event ---------| | + | | | + | RACE CHECK: won | | + | (begin work) | | + | | | + | |<-- check (query locks) ----| + | |--- JSON: locked=[A] ------>| + | | | + | | (Agent B picks | + | | a different task) | + | | | + | (2 hours pass) | | + | |--- lock auto-expires ----->| + | | | +``` + +### Event Schema (NIP-78 / Kind 30078) + +Each lock is a NIP-33 parameterized replaceable event: + +```json +{ + "kind": 30078, + "pubkey": "", + "created_at": 1739484362, + "tags": [ + ["d", "bitvid-lock/daily/nip-research-agent/2026-02-13"], + ["t", "bitvid-agent-lock"], + ["t", "bitvid-lock-daily"], + ["t", "bitvid-lock-daily-2026-02-13"], + ["expiration", "1739491562"] + ], + "content": "{\"agent\":\"nip-research-agent\",\"cadence\":\"daily\",\"status\":\"started\",\"date\":\"2026-02-13\",\"platform\":\"jules\",\"lockedAt\":\"2026-02-13T20:06:02.000Z\",\"expiresAt\":\"2026-02-13T22:06:02.000Z\"}" +} +``` + +**Tag breakdown:** + +| Tag | Purpose | +|-----|---------| +| `d` | NIP-33 identifier: `///`. Makes the event replaceable per agent per day. | +| `t` (bitvid-agent-lock) | Broad category filter for all TORCH lock events. | +| `t` (bitvid-lock-daily) | Cadence-specific filter. | +| `t` (bitvid-lock-daily-2026-02-13) | Date-specific filter for precise relay-side queries. | +| `expiration` | NIP-40 auto-expiration timestamp. Relays may garbage-collect after this time. | + +### Ephemeral Keys + +Each `lock` invocation generates a fresh secp256k1 keypair: + +``` +generateSecretKey() → 32 random bytes → sign event → discard key +``` + +The private key exists only in process memory for the duration of the script. It is never written to disk, never stored, never reused. The public key is included in the event for Nostr protocol compliance but has no ongoing identity or authority. + +This means: +- **No secret management.** No tokens, no `.env` files, no rotation schedules. +- **No key reuse.** Every lock is cryptographically independent. +- **No cleanup authority.** The lock can't be manually deleted (the key is gone), but it doesn't need to be — NIP-40 expiration handles it. + +### Race Detection + +When two agents try to claim the same task simultaneously: + +1. Both agents run `check` — neither sees a lock. +2. Both agents run `lock` — both publish events to relays. +3. Both agents re-query after a 1.5-second propagation delay. +4. Both agents see two lock events for the same task. +5. The event with the **earlier `created_at`** wins. +6. The loser exits with code 3. The winner proceeds. + +The losing agent's event remains on relays but is harmless — it auto-expires, and it doesn't prevent the winner from working. Other agents checking later see both events but this just means the task appears (correctly) as locked. + +### Auto-Expiration + +Locks expire via NIP-40 after a configurable TTL (default: 2 hours). This provides: + +- **Self-healing.** If an agent crashes mid-task, its lock doesn't persist forever. +- **No stale state.** No equivalent of orphaned draft PRs blocking future runs. +- **Zero maintenance.** No cron jobs to clean up expired locks, no manual intervention. + +The client-side `check` command also filters by expiration, so even if a relay hasn't garbage-collected an expired event, it won't appear in the exclusion set. + +## Usage + +### Commands + +**Check what's locked:** +```bash +node scripts/agent/nostr-lock.mjs check --cadence daily +``` +Returns JSON to stdout: +```json +{ + "cadence": "daily", + "date": "2026-02-13", + "locked": ["audit-agent", "ci-health-agent"], + "available": ["const-refactor-agent", "content-audit-agent", ...], + "lockCount": 2, + "locks": [...] +} +``` + +**Claim a task:** +```bash +AGENT_PLATFORM=jules \ +node scripts/agent/nostr-lock.mjs lock \ + --agent nip-research-agent \ + --cadence daily +``` +Returns key=value pairs to stdout: +``` +LOCK_STATUS=ok +LOCK_EVENT_ID=ea2724c76d2bd707... +LOCK_PUBKEY=7ee606dd0619e339... +LOCK_AGENT=nip-research-agent +LOCK_CADENCE=daily +LOCK_DATE=2026-02-13 +LOCK_EXPIRES=1739491562 +LOCK_EXPIRES_ISO=2026-02-13T22:06:02.000Z +``` + +Exit codes: +- `0` — Lock acquired. +- `3` — Race lost or task already locked. +- `2` — Relay error. +- `1` — Usage error. + +**List all active locks (human-readable):** +```bash +node scripts/agent/nostr-lock.mjs list +``` + +**Test without publishing:** +```bash +node scripts/agent/nostr-lock.mjs lock --agent test --cadence daily --dry-run +``` + +### Configuration + +| Environment Variable | Default | Description | +|---------------------|---------|-------------| +| `NOSTR_LOCK_RELAYS` | `wss://relay.damus.io,wss://nos.lol,wss://relay.primal.net` | Comma-separated relay URLs | +| `NOSTR_LOCK_TTL` | `7200` (2 hours) | Lock expiration in seconds | +| `AGENT_PLATFORM` | `unknown` | Platform identifier included in lock metadata | + +## Integration with the Scheduler + +The bitvid scheduler uses TORCH as its sole coordination mechanism. The full flow is documented in `docs/agents/prompts/scheduler-flow.md`, but the TORCH-specific part is just two steps: + +```bash +# 1. Build exclusion set +node scripts/agent/nostr-lock.mjs check --cadence daily +# → Use the "locked" array to skip already-claimed agents + +# 2. Claim the selected agent +AGENT_PLATFORM=jules \ +node scripts/agent/nostr-lock.mjs lock --agent --cadence daily +# → Exit 0: proceed with work +# → Exit 3: pick a different agent +``` + +The scheduler meta prompts (`docs/agents/prompts/META_PROMPTS.md`) embed these commands directly, so agents execute them without needing to understand the underlying protocol. + +## Architecture + +``` +┌──────────────────────────────────────────────────────────┐ +│ Agent Platforms │ +│ │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ Claude Code │ │ Google Jules│ │ OpenAI Codex│ │ +│ │ │ │ │ │ │ │ +│ │ nostr-lock │ │ nostr-lock │ │ nostr-lock │ │ +│ │ .mjs │ │ .mjs │ │ .mjs │ │ +│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ +│ │ │ │ │ +└─────────┼────────────────┼────────────────┼──────────────┘ + │ │ │ + │ WebSocket (NIP-01) │ + │ │ │ +┌─────────▼────────────────▼────────────────▼──────────────┐ +│ Nostr Relay Network │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │relay.damus.io│ │ nos.lol │ │relay.primal │ │ +│ │ │ │ │ │ .net │ │ +│ │ Kind 30078 │ │ Kind 30078 │ │ Kind 30078 │ │ +│ │ lock events │ │ lock events │ │ lock events │ │ +│ │ (NIP-40 │ │ (NIP-40 │ │ (NIP-40 │ │ +│ │ auto-exp) │ │ auto-exp) │ │ auto-exp) │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +│ │ +│ Events replicate across relays. Queries go to all. │ +│ Publishing succeeds if at least 1 relay accepts. │ +└──────────────────────────────────────────────────────────┘ +``` + +### Dependencies + +TORCH uses only packages already in the bitvid project: + +- **nostr-tools** (v2.19.4) — Key generation, event signing, relay pool +- **ws** (v8.19.0) — WebSocket for Node.js relay connections + +No additional dependencies are needed. + +## Why Nostr? + +The bitvid project is built on Nostr. Using Nostr for agent coordination means: + +1. **No new infrastructure.** Public relays already exist and are free to use. +2. **No platform coupling.** Works with any agent platform that has Node.js and WebSocket access, regardless of their relationship with GitHub. +3. **No credentials.** Ephemeral keys eliminate the entire category of secret management. +4. **Protocol-native race resolution.** NIP-33 replaceable events and relay-side ordering provide deterministic conflict resolution without custom logic. +5. **Self-healing by default.** NIP-40 expiration means locks clean themselves up. +6. **Dogfooding.** A Nostr project using Nostr for its own development tooling. + +## Toward a Generalizable Drop-In + +TORCH is designed to be extracted from bitvid and used in any repository. The components that would need to be generalized: + +| Component | Current State | To Generalize | +|-----------|--------------|---------------| +| `nostr-lock.mjs` | Hardcoded bitvid agent rosters | Accept roster via config file or CLI flag | +| Tag namespace | `bitvid-agent-lock` | Configurable project prefix | +| `d` tag pattern | `bitvid-lock///` | `-lock///` | +| Relay defaults | 3 hardcoded relays | Config file with sensible defaults | +| Scheduler flow | bitvid-specific docs | Template scheduler docs with project placeholders | +| Agent rosters | Inline arrays in the script | External `torch.config.json` or similar | + +A future `torch` npm package could look like: + +```bash +# Install +npm install --save-dev @bitvid/torch + +# Initialize in a new repo +npx torch init --project myproject --relays wss://relay.damus.io + +# Creates: +# torch.config.json — project name, relays, TTL, roster +# scripts/torch-lock.mjs — pre-configured lock script +# docs/torch-scheduler.md — template scheduler flow + +# Use in agent prompts +npx torch check --cadence daily +npx torch lock --agent my-agent --cadence daily +``` + +The protocol itself (NIP-78 events with NIP-40 expiration) is already generic. The bitvid-specific parts are just the roster definitions and tag namespaces. + +## Related Files + +| File | Purpose | +|------|---------| +| `scripts/agent/nostr-lock.mjs` | TORCH lock script (check, lock, list) | +| `docs/agents/prompts/scheduler-flow.md` | Scheduler procedure using TORCH | +| `docs/agents/prompts/META_PROMPTS.md` | Agent meta prompts with TORCH commands | +| `AGENTS.md` | Task Claiming Protocol referencing TORCH | +| `scripts/agents/claim-audit.mjs` | Legacy claim audit (predates TORCH) | + +## NIPs Referenced + +| NIP | Usage in TORCH | +|-----|---------------| +| [NIP-01](https://github.com/nostr-protocol/nips/blob/master/01.md) | Basic event structure and relay communication | +| [NIP-33](https://github.com/nostr-protocol/nips/blob/master/33.md) | Parameterized replaceable events (d-tag uniqueness) | +| [NIP-40](https://github.com/nostr-protocol/nips/blob/master/40.md) | Expiration tag for auto-expiring locks | +| [NIP-78](https://github.com/nostr-protocol/nips/blob/master/78.md) | Application-specific data (Kind 30078) | diff --git a/docs/agents/prompts/scheduler-flow.md b/docs/agents/prompts/scheduler-flow.md index 96b7f4c0..7a85b6c7 100644 --- a/docs/agents/prompts/scheduler-flow.md +++ b/docs/agents/prompts/scheduler-flow.md @@ -1,6 +1,6 @@ # Scheduler Flow (Single Source of Truth) -Use this document for **all scheduler runs**. +Use this document for **all scheduler runs**. Task locking uses the **TORCH** protocol (see `docs/agents/TORCH.md`). ## Scheduler Override (Top-Line Rule) diff --git a/scripts/agent/nostr-lock.mjs b/scripts/agent/nostr-lock.mjs index d3b02942..5d907ac0 100755 --- a/scripts/agent/nostr-lock.mjs +++ b/scripts/agent/nostr-lock.mjs @@ -2,11 +2,13 @@ // scripts/agent/nostr-lock.mjs // -// Nostr-based distributed task locking for multi-agent coordination. +// TORCH — Task Orchestration via Relay-Coordinated Handoff // +// Decentralized task locking for multi-agent development using Nostr. // Each lock generates a fresh ephemeral keypair and discards it. -// Locks auto-expire via NIP-40 — no cleanup needed. -// Other agents discover locks by querying relays for bitvid lock events. +// Locks auto-expire via NIP-40 — no cleanup, no tokens, no secrets. +// +// See docs/agents/TORCH.md for the full protocol documentation. // // Commands: // check --cadence JSON list of locked agents