From 9e5797c5e099d338852992a571c5a51e78c4fb33 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Feb 2026 20:29:38 +0000 Subject: [PATCH] fix(agents): add GitHub API fallback for task locking in restricted environments Jules agents cannot git push due to sandbox restrictions, which breaks the draft PR-based distributed lock mechanism. Multiple agents can pick up the same task because the claim is never visible remotely. This adds scripts/agent/claim-task-api.sh which creates the remote branch, started.md log file, and draft PR entirely via curl and the GitHub REST API, bypassing git push. The scheduler-flow.md, META_PROMPTS.md, and AGENTS.md are updated to document the fallback: try git push first, fall back to the API script if push is blocked, fail-closed if neither method works. https://claude.ai/code/session_01BpDsVSAkVWtJLnfKxsB9ib --- AGENTS.md | 18 +- docs/agents/prompts/META_PROMPTS.md | 18 +- docs/agents/prompts/scheduler-flow.md | 66 +++++++- scripts/agent/claim-task-api.sh | 228 ++++++++++++++++++++++++++ 4 files changed, 320 insertions(+), 10 deletions(-) create mode 100755 scripts/agent/claim-task-api.sh diff --git a/AGENTS.md b/AGENTS.md index fe9949e1..ae8a1ba6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -256,14 +256,28 @@ To prevent multiple agents from working on the same task simultaneously, every a ``` If a matching PR exists (open or draft) for the same agent or subsystem, **skip the task** — another agent is already working on it. -2. **Claim the task.** If no matching PR exists: +2. **Claim the task.** If no matching PR exists, use one of these methods: + + **Method A — git push (preferred when available):** a. Create your working branch. b. Make a minimal initial commit (e.g., create a `context/CONTEXT_.md` with the task scope). c. Create a `started` log file in the appropriate task-log directory. d. Push the branch immediately to make the claim visible. e. Only after the branch is pushed and the claim is visible, begin the actual work. -3. **Handle race conditions.** After pushing your claim, re-check for duplicates: + **Method B — GitHub API fallback (when git push is blocked):** + If `git push` is unavailable (e.g., Jules sandbox), use the API claim script: + ```bash + bash scripts/agent/claim-task-api.sh \ + --agent \ + --cadence \ + --base unstable + ``` + This creates the remote branch, `_started.md` log file, and draft PR entirely via `curl` and the GitHub REST API. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. The script outputs `CLAIM_BRANCH`, `CLAIM_PR_NUMBER`, and `CLAIM_PR_URL` for use in subsequent steps. + + If **neither method** works, do not proceed — write a `_failed.md` log and stop. + +3. **Handle race conditions.** After your claim is visible remotely, re-check for duplicates: ``` curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, titles: [.[].title]}' ``` diff --git a/docs/agents/prompts/META_PROMPTS.md b/docs/agents/prompts/META_PROMPTS.md index 07f7c875..ccd36885 100644 --- a/docs/agents/prompts/META_PROMPTS.md +++ b/docs/agents/prompts/META_PROMPTS.md @@ -24,11 +24,16 @@ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_pag MUST 3: Run the exact command list in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 -3) [claim] create branch, create+commit `_started.md`, push +3) [claim] Try: create branch, create+commit `_started.md`, push. + IF git push fails or is blocked, use the API fallback instead: + bash scripts/agent/claim-task-api.sh --agent --cadence daily --base unstable + This creates the remote branch, started.md, and draft PR via curl. + Requires GITHUB_TOKEN or GH_TOKEN in environment. + IF both methods fail, write `_failed.md` and stop. 4) rerun preflight command from MUST 2 5) execute selected prompt from docs/agents/prompts/daily/ 6) npm run lint -7) create `_completed.md` or `_failed.md`, commit, push +7) create `_completed.md` or `_failed.md`, commit, push (or API update) MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #)`. @@ -55,11 +60,16 @@ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_pag MUST 3: Run the exact command list in this order: 1) cat AGENTS.md CLAUDE.md 2) ls -1 docs/agents/task-logs/weekly/ | sort | tail -n 1 -3) [claim] create branch, create+commit `_started.md`, push +3) [claim] Try: create branch, create+commit `_started.md`, push. + IF git push fails or is blocked, use the API fallback instead: + bash scripts/agent/claim-task-api.sh --agent --cadence weekly --base unstable + This creates the remote branch, started.md, and draft PR via curl. + Requires GITHUB_TOKEN or GH_TOKEN in environment. + IF both methods fail, write `_failed.md` and stop. 4) rerun preflight command from MUST 2 5) execute selected prompt from docs/agents/prompts/weekly/ 6) npm run lint -7) create `_completed.md` or `_failed.md`, commit, push +7) create `_completed.md` or `_failed.md`, commit, push (or API update) MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #)`. diff --git a/docs/agents/prompts/scheduler-flow.md b/docs/agents/prompts/scheduler-flow.md index 23ae58f5..703827e4 100644 --- a/docs/agents/prompts/scheduler-flow.md +++ b/docs/agents/prompts/scheduler-flow.md @@ -50,19 +50,35 @@ During scheduler execution, the scheduler-specific instructions in this document 6. **MUST** stop and write a `_failed.md` log with `All roster tasks currently claimed by other agents` when every roster agent is excluded. -7. **MUST** claim the selected agent in this exact order: +7. **MUST** claim the selected agent using **one** of these methods (try in order): + + **Method A — git push (preferred):** 1) create branch, 2) create and commit `__started.md` in ``, 3) push branch and commit. -8. **MUST** re-run the same PR preflight command from Step 2 immediately after push. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #)`, abandon the run, and return to Step 5. Otherwise print `RACE CHECK: won`. + **Method B — GitHub API fallback (when git push is blocked):** + If `git push` fails or is unavailable (e.g., Jules sandbox), use the API claim script: + ```bash + bash scripts/agent/claim-task-api.sh \ + --agent \ + --cadence \ + --base unstable + ``` + This creates the remote branch, `_started.md` log file, and draft PR entirely via the GitHub REST API using `curl`. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. + + If **both** methods fail (no push access AND no API token), write a `_failed.md` log with reason `Cannot create remote claim: git push blocked and no GITHUB_TOKEN available` and stop. + +8. **MUST** re-run the same PR preflight command from Step 2 immediately after the claim is visible remotely. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #)`, abandon the run, and return to Step 5. Otherwise print `RACE CHECK: won`. 9. **MUST** execute `/` end-to-end. -10. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push. +10. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push (or use the API to update the remote branch if push is blocked). ## Canonical Example Run Output +### Method A (git push) + ```text $ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' { "count": 12, "prs": [ ... ] } @@ -78,7 +94,49 @@ Selected agent: nip-research-agent Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md Pushed branch: agents/daily/nip-research-agent/2026-02-14-run -$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?[^/]+)/")?.agent) // (.title | capture("(?[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}' +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' +{ "count": 13, "prs": [ ... ] } +RACE CHECK: won + +$ npm run lint +> lint passed + +Wrote: docs/agents/task-logs/daily/2026-02-14_00-12-00_nip-research-agent_completed.md +``` + +### Method B (API fallback — for Jules or other restricted environments) + +```text +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' +{ "count": 12, "prs": [ ... ] } + +$ cat AGENTS.md CLAUDE.md +... + +$ ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1 +2026-02-13_18-40-00_load-test-agent_completed.md + +Selected agent: nip-research-agent + +# git push is blocked — use API fallback: +$ bash scripts/agent/claim-task-api.sh --agent nip-research-agent --cadence daily --base unstable +Claiming task: nip-research-agent (daily) +Branch: agents/daily/nip-research-agent/2026-02-14-run-1707868800 +Step 1: Fetching base branch SHA (unstable)... + Base SHA: abc123... +Step 2: Creating remote branch... + Created: refs/heads/agents/daily/nip-research-agent/2026-02-14-run-1707868800 +Step 3: Creating started.md log file... + Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md +Step 4: Creating draft PR... + PR : +CLAIM_BRANCH=agents/daily/nip-research-agent/2026-02-14-run-1707868800 +CLAIM_PR_NUMBER= +CLAIM_PR_URL= +CLAIM_LOG_FILE=docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md + +# Re-run preflight to check for race: +$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...' { "count": 13, "prs": [ ... ] } RACE CHECK: won diff --git a/scripts/agent/claim-task-api.sh b/scripts/agent/claim-task-api.sh new file mode 100755 index 00000000..2ed9c3ce --- /dev/null +++ b/scripts/agent/claim-task-api.sh @@ -0,0 +1,228 @@ +#!/usr/bin/env bash +set -euo pipefail + +# claim-task-api.sh — Create a remote branch + started.md + draft PR +# entirely via GitHub REST API, bypassing 'git push'. +# +# This solves the Jules sandbox restriction where git push is blocked +# but curl/HTTP requests are allowed. +# +# Usage: +# ./scripts/agent/claim-task-api.sh \ +# --agent \ +# --cadence \ +# [--base ] \ +# [--run-id ] +# +# Environment: +# GITHUB_TOKEN — Required. A token with repo write access. +# Falls back to GH_TOKEN if GITHUB_TOKEN is unset. +# +# Outputs (stdout): +# CLAIM_BRANCH= +# CLAIM_PR_NUMBER= +# CLAIM_PR_URL= +# +# Exit codes: +# 0 — Claim created successfully +# 1 — Usage error or missing token +# 2 — API error (branch/file/PR creation failed) + +OWNER="PR0M3TH3AN" +REPO="bitvid" +API_BASE="https://api.github.com/repos/${OWNER}/${REPO}" + +# --------------------------------------------------------------------------- +# Parse arguments +# --------------------------------------------------------------------------- +AGENT="" +CADENCE="" +BASE_BRANCH="unstable" +RUN_ID="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --agent) AGENT="$2"; shift 2 ;; + --cadence) CADENCE="$2"; shift 2 ;; + --base) BASE_BRANCH="$2"; shift 2 ;; + --run-id) RUN_ID="$2"; shift 2 ;; + *) + echo "ERROR: Unknown argument: $1" >&2 + echo "Usage: $0 --agent --cadence [--base ] [--run-id ]" >&2 + exit 1 + ;; + esac +done + +if [[ -z "$AGENT" || -z "$CADENCE" ]]; then + echo "ERROR: --agent and --cadence are required." >&2 + exit 1 +fi + +if [[ "$CADENCE" != "daily" && "$CADENCE" != "weekly" ]]; then + echo "ERROR: --cadence must be 'daily' or 'weekly'." >&2 + exit 1 +fi + +# --------------------------------------------------------------------------- +# Resolve token +# --------------------------------------------------------------------------- +TOKEN="${GITHUB_TOKEN:-${GH_TOKEN:-}}" +if [[ -z "$TOKEN" ]]; then + echo "ERROR: GITHUB_TOKEN (or GH_TOKEN) must be set for API access." >&2 + echo "Without a token, remote task locking is impossible." >&2 + exit 1 +fi + +# --------------------------------------------------------------------------- +# Helper: authenticated GitHub API call +# --------------------------------------------------------------------------- +gh_api() { + local method="$1" + local endpoint="$2" + local data="${3:-}" + + local args=( + -s -f + -X "$method" + -H "Authorization: Bearer ${TOKEN}" + -H "Accept: application/vnd.github+json" + -H "X-GitHub-Api-Version: 2022-11-28" + ) + + if [[ -n "$data" ]]; then + args+=(-H "Content-Type: application/json" -d "$data") + fi + + curl "${args[@]}" "${API_BASE}${endpoint}" +} + +# --------------------------------------------------------------------------- +# Generate identifiers +# --------------------------------------------------------------------------- +TIMESTAMP=$(date -u +"%Y-%m-%d_%H-%M-%S") +DATE_ONLY=$(date -u +"%Y-%m-%d") +RUN_ID="${RUN_ID:-$(date -u +"%s")}" +BRANCH_NAME="agents/${CADENCE}/${AGENT}/${DATE_ONLY}-run-${RUN_ID}" +LOG_DIR="docs/agents/task-logs/${CADENCE}" +LOG_FILE="${TIMESTAMP}_${AGENT}_started.md" +LOG_PATH="${LOG_DIR}/${LOG_FILE}" + +echo "Claiming task: ${AGENT} (${CADENCE})" >&2 +echo "Branch: ${BRANCH_NAME}" >&2 +echo "Log file: ${LOG_PATH}" >&2 + +# --------------------------------------------------------------------------- +# Step 1: Get the latest commit SHA of the base branch +# --------------------------------------------------------------------------- +echo "Step 1: Fetching base branch SHA (${BASE_BRANCH})..." >&2 +BASE_REF_JSON=$(gh_api GET "/git/refs/heads/${BASE_BRANCH}") +BASE_SHA=$(echo "$BASE_REF_JSON" | jq -r '.object.sha // empty') + +if [[ -z "$BASE_SHA" ]]; then + echo "ERROR: Could not resolve base branch '${BASE_BRANCH}'." >&2 + echo "API response: ${BASE_REF_JSON}" >&2 + exit 2 +fi +echo " Base SHA: ${BASE_SHA}" >&2 + +# --------------------------------------------------------------------------- +# Step 2: Create the remote branch +# --------------------------------------------------------------------------- +echo "Step 2: Creating remote branch..." >&2 +CREATE_REF_JSON=$(gh_api POST "/git/refs" "$(jq -n \ + --arg ref "refs/heads/${BRANCH_NAME}" \ + --arg sha "$BASE_SHA" \ + '{ref: $ref, sha: $sha}')") + +CREATED_REF=$(echo "$CREATE_REF_JSON" | jq -r '.ref // empty') +if [[ -z "$CREATED_REF" ]]; then + # Check if it's a 422 (already exists) — another agent may have claimed it + ERROR_MSG=$(echo "$CREATE_REF_JSON" | jq -r '.message // empty') + if [[ "$ERROR_MSG" == *"Reference already exists"* ]]; then + echo "RACE DETECTED: Branch '${BRANCH_NAME}' already exists. Another agent claimed first." >&2 + exit 2 + fi + echo "ERROR: Failed to create branch." >&2 + echo "API response: ${CREATE_REF_JSON}" >&2 + exit 2 +fi +echo " Created: ${CREATED_REF}" >&2 + +# --------------------------------------------------------------------------- +# Step 3: Create the started.md log file on the branch +# --------------------------------------------------------------------------- +echo "Step 3: Creating started.md log file..." >&2 + +LOG_CONTENT=$(cat </dev/null || echo "$LOG_CONTENT" | base64) + +CREATE_FILE_JSON=$(gh_api PUT "/contents/${LOG_PATH}" "$(jq -n \ + --arg message "chore(agents): claim ${CADENCE} task for ${AGENT}" \ + --arg content "$ENCODED_CONTENT" \ + --arg branch "$BRANCH_NAME" \ + '{message: $message, content: $content, branch: $branch}')") + +FILE_PATH_CREATED=$(echo "$CREATE_FILE_JSON" | jq -r '.content.path // empty') +if [[ -z "$FILE_PATH_CREATED" ]]; then + echo "ERROR: Failed to create log file." >&2 + echo "API response: ${CREATE_FILE_JSON}" >&2 + # Clean up: delete the branch we created + gh_api DELETE "/git/refs/heads/${BRANCH_NAME}" >/dev/null 2>&1 || true + exit 2 +fi +echo " Created: ${FILE_PATH_CREATED}" >&2 + +# --------------------------------------------------------------------------- +# Step 4: Create a draft PR +# --------------------------------------------------------------------------- +echo "Step 4: Creating draft PR..." >&2 +PR_TITLE="[${CADENCE}] ${AGENT}: ${DATE_ONLY} run" +PR_BODY="Automated task claim for **${AGENT}** (${CADENCE} cadence). + +Branch: \`${BRANCH_NAME}\` +Claimed at: ${TIMESTAMP} UTC +Method: GitHub API (claim-task-api.sh) + +--- +*This draft PR serves as a distributed lock. Do not merge until the agent completes its work.*" + +CREATE_PR_JSON=$(gh_api POST "/pulls" "$(jq -n \ + --arg title "$PR_TITLE" \ + --arg body "$PR_BODY" \ + --arg head "$BRANCH_NAME" \ + --arg base "$BASE_BRANCH" \ + '{title: $title, body: $body, head: $head, base: $base, draft: true}')") + +PR_NUMBER=$(echo "$CREATE_PR_JSON" | jq -r '.number // empty') +PR_URL=$(echo "$CREATE_PR_JSON" | jq -r '.html_url // empty') + +if [[ -z "$PR_NUMBER" ]]; then + echo "WARNING: Draft PR creation failed. Branch and log file exist as partial lock." >&2 + echo "API response: ${CREATE_PR_JSON}" >&2 + # Don't exit — the branch itself is still a visible claim + PR_NUMBER="FAILED" + PR_URL="FAILED" +else + echo " PR #${PR_NUMBER}: ${PR_URL}" >&2 +fi + +# --------------------------------------------------------------------------- +# Output machine-readable results +# --------------------------------------------------------------------------- +echo "CLAIM_BRANCH=${BRANCH_NAME}" +echo "CLAIM_PR_NUMBER=${PR_NUMBER}" +echo "CLAIM_PR_URL=${PR_URL}" +echo "CLAIM_LOG_FILE=${LOG_PATH}" +echo "" +echo "Task claimed successfully." >&2