thePR0M3TH3AN/bitvid
1
0 Fork 0
mirror of https://github.com/PR0M3TH3AN/bitvid.git synced 2026-10-27 07:52:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(agents): add GitHub API fallback for task locking in restricted environments

Browse Source 
Jules agents cannot git push due to sandbox restrictions, which breaks the
draft PR-based distributed lock mechanism. Multiple agents can pick up the
same task because the claim is never visible remotely.

This adds scripts/agent/claim-task-api.sh which creates the remote branch,
started.md log file, and draft PR entirely via curl and the GitHub REST API,
bypassing git push. The scheduler-flow.md, META_PROMPTS.md, and AGENTS.md
are updated to document the fallback: try git push first, fall back to the
API script if push is blocked, fail-closed if neither method works.

https://claude.ai/code/session_01BpDsVSAkVWtJLnfKxsB9ib
This commit is contained in:
Claude committed 2026-02-13 20:29:38 +00:00
1 parent 510efeafaf
commit 9e5797c5e0
4 files changed
+320 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+16 -2
View File
@@ -256,14 +256,28 @@ To prevent multiple agents from working on the same task simultaneously, every a
```
If a matching PR exists (open or draft) for the same agent or subsystem, **skip the task** — another agent is already working on it.
2. **Claim the task.** If no matching PR exists:
2. **Claim the task.** If no matching PR exists, use one of these methods:
**Method A — git push (preferred when available):**
a. Create your working branch.
b. Make a minimal initial commit (e.g., create a `context/CONTEXT_<timestamp>.md` with the task scope).
c. Create a `started` log file in the appropriate task-log directory.
d. Push the branch immediately to make the claim visible.
e. Only after the branch is pushed and the claim is visible, begin the actual work.
3. **Handle race conditions.** After pushing your claim, re-check for duplicates:
**Method B — GitHub API fallback (when git push is blocked):**
If `git push` is unavailable (e.g., Jules sandbox), use the API claim script:
```bash
bash scripts/agent/claim-task-api.sh \
--agent <agent-name> \
--cadence <daily|weekly> \
--base unstable
```
This creates the remote branch, `_started.md` log file, and draft PR entirely via `curl` and the GitHub REST API. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment. The script outputs `CLAIM_BRANCH`, `CLAIM_PR_NUMBER`, and `CLAIM_PR_URL` for use in subsequent steps.
If **neither method** works, do not proceed — write a `_failed.md` log and stop.
3. **Handle race conditions.** After your claim is visible remotely, re-check for duplicates:
```
curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, titles: [.[].title]}'
```
docs/agents/prompts/META_PROMPTS.md
+14 -4
View File
@@ -24,11 +24,16 @@ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_pag
MUST 3: Run the exact command list in this order:
1) cat AGENTS.md CLAUDE.md
2) ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1
3) [claim] create branch, create+commit `_started.md`, push
3) [claim] Try: create branch, create+commit `_started.md`, push.
IF git push fails or is blocked, use the API fallback instead:
bash scripts/agent/claim-task-api.sh --agent <agent-name> --cadence daily --base unstable
This creates the remote branch, started.md, and draft PR via curl.
Requires GITHUB_TOKEN or GH_TOKEN in environment.
IF both methods fail, write `_failed.md` and stop.
4) rerun preflight command from MUST 2
5) execute selected prompt from docs/agents/prompts/daily/
6) npm run lint
7) create `_completed.md` or `_failed.md`, commit, push
7) create `_completed.md` or `_failed.md`, commit, push (or API update)
MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #<number>)`.
@@ -55,11 +60,16 @@ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_pag
MUST 3: Run the exact command list in this order:
1) cat AGENTS.md CLAUDE.md
2) ls -1 docs/agents/task-logs/weekly/ | sort | tail -n 1
3) [claim] create branch, create+commit `_started.md`, push
3) [claim] Try: create branch, create+commit `_started.md`, push.
IF git push fails or is blocked, use the API fallback instead:
bash scripts/agent/claim-task-api.sh --agent <agent-name> --cadence weekly --base unstable
This creates the remote branch, started.md, and draft PR via curl.
Requires GITHUB_TOKEN or GH_TOKEN in environment.
IF both methods fail, write `_failed.md` and stop.
4) rerun preflight command from MUST 2
5) execute selected prompt from docs/agents/prompts/weekly/
6) npm run lint
7) create `_completed.md` or `_failed.md`, commit, push
7) create `_completed.md` or `_failed.md`, commit, push (or API update)
MUST 4: Race rule: after step 3, compare only matching derived `agent` claims from the rerun. Earlier `created_at` wins; if tied, lower PR number wins. Print exactly one line: `RACE CHECK: won` or `RACE CHECK: lost (agent already claimed by PR #<number>)`.
docs/agents/prompts/scheduler-flow.md
+62 -4
View File
@@ -50,19 +50,35 @@ During scheduler execution, the scheduler-specific instructions in this document
6. **MUST** stop and write a `_failed.md` log with `All roster tasks currently claimed by other agents` when every roster agent is excluded.
7. **MUST** claim the selected agent in this exact order:
7. **MUST** claim the selected agent using **one** of these methods (try in order):
**Method A — git push (preferred):**
1) create branch,
2) create and commit `<timestamp>_<agent>_started.md` in `<log_dir>`,
3) push branch and commit.
8. **MUST** re-run the same PR preflight command from Step 2 immediately after push. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #<number>)`, abandon the run, and return to Step 5. Otherwise print `RACE CHECK: won`.
**Method B — GitHub API fallback (when git push is blocked):**
If `git push` fails or is unavailable (e.g., Jules sandbox), use the API claim script:
```bash
bash scripts/agent/claim-task-api.sh \
--agent <agent-name> \
--cadence <cadence> \
--base unstable
```
This creates the remote branch, `_started.md` log file, and draft PR entirely via the GitHub REST API using `curl`. Requires `GITHUB_TOKEN` or `GH_TOKEN` in the environment.
If **both** methods fail (no push access AND no API token), write a `_failed.md` log with reason `Cannot create remote claim: git push blocked and no GITHUB_TOKEN available` and stop.
8. **MUST** re-run the same PR preflight command from Step 2 immediately after the claim is visible remotely. If an earlier claim exists for the same derived agent, print `RACE CHECK: lost (agent already claimed by PR #<number>)`, abandon the run, and return to Step 5. Otherwise print `RACE CHECK: won`.
9. **MUST** execute `<prompt_dir>/<prompt-file>` end-to-end.
10. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push.
10. **MUST** create exactly one final status file (`_completed.md` or `_failed.md`), run `npm run lint`, then commit and push (or use the API to update the remote branch if push is blocked).
## Canonical Example Run Output
### Method A (git push)
```text
$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?<agent>[^/]+)/")?.agent) // (.title | capture("(?<agent>[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}'
{ "count": 12, "prs": [ ... ] }
@@ -78,7 +94,49 @@ Selected agent: nip-research-agent
Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md
Pushed branch: agents/daily/nip-research-agent/2026-02-14-run
$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '{count: length, prs: [.[] | {number, created_at, draft, head: {ref: .head.ref}, agent: ((.head.ref | capture("^agents/daily/(?<agent>[^/]+)/")?.agent) // (.title | capture("(?<agent>[A-Za-z0-9-]+-agent)")?.agent // null))}] | sort_by(.created_at, .number)}'
$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...'
{ "count": 13, "prs": [ ... ] }
RACE CHECK: won
$ npm run lint
> lint passed
Wrote: docs/agents/task-logs/daily/2026-02-14_00-12-00_nip-research-agent_completed.md
```
### Method B (API fallback — for Jules or other restricted environments)
```text
$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...'
{ "count": 12, "prs": [ ... ] }
$ cat AGENTS.md CLAUDE.md
...
$ ls -1 docs/agents/task-logs/daily/ | sort | tail -n 1
2026-02-13_18-40-00_load-test-agent_completed.md
Selected agent: nip-research-agent
# git push is blocked — use API fallback:
$ bash scripts/agent/claim-task-api.sh --agent nip-research-agent --cadence daily --base unstable
Claiming task: nip-research-agent (daily)
Branch: agents/daily/nip-research-agent/2026-02-14-run-1707868800
Step 1: Fetching base branch SHA (unstable)...
Base SHA: abc123...
Step 2: Creating remote branch...
Created: refs/heads/agents/daily/nip-research-agent/2026-02-14-run-1707868800
Step 3: Creating started.md log file...
Created: docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md
Step 4: Creating draft PR...
PR <number>: <url>
CLAIM_BRANCH=agents/daily/nip-research-agent/2026-02-14-run-1707868800
CLAIM_PR_NUMBER=<number>
CLAIM_PR_URL=<url>
CLAIM_LOG_FILE=docs/agents/task-logs/daily/2026-02-14_00-00-00_nip-research-agent_started.md
# Re-run preflight to check for race:
$ curl -s "https://api.github.com/repos/PR0M3TH3AN/bitvid/pulls?state=open&per_page=100" | jq '...'
{ "count": 13, "prs": [ ... ] }
RACE CHECK: won
scripts/agent/claim-task-api.sh
Executable
+228
View File
@@ -0,0 +1,228 @@
#!/usr/bin/env bash
set -euo pipefail
# claim-task-api.sh — Create a remote branch + started.md + draft PR
# entirely via GitHub REST API, bypassing 'git push'.
#
# This solves the Jules sandbox restriction where git push is blocked
# but curl/HTTP requests are allowed.
#
# Usage:
# ./scripts/agent/claim-task-api.sh \
# --agent <agent-name> \
# --cadence <daily|weekly> \
# [--base <branch>] \
# [--run-id <id>]
#
# Environment:
# GITHUB_TOKEN — Required. A token with repo write access.
# Falls back to GH_TOKEN if GITHUB_TOKEN is unset.
#
# Outputs (stdout):
# CLAIM_BRANCH=<branch-name>
# CLAIM_PR_NUMBER=<number>
# CLAIM_PR_URL=<url>
#
# Exit codes:
# 0 — Claim created successfully
# 1 — Usage error or missing token
# 2 — API error (branch/file/PR creation failed)
OWNER="PR0M3TH3AN"
REPO="bitvid"
API_BASE="https://api.github.com/repos/${OWNER}/${REPO}"
# ---------------------------------------------------------------------------
# Parse arguments
# ---------------------------------------------------------------------------
AGENT=""
CADENCE=""
BASE_BRANCH="unstable"
RUN_ID=""
while [[ $# -gt 0 ]]; do
case "$1" in
--agent) AGENT="$2"; shift 2 ;;
--cadence) CADENCE="$2"; shift 2 ;;
--base) BASE_BRANCH="$2"; shift 2 ;;
--run-id) RUN_ID="$2"; shift 2 ;;
*)
echo "ERROR: Unknown argument: $1" >&2
echo "Usage: $0 --agent <name> --cadence <daily|weekly> [--base <branch>] [--run-id <id>]" >&2
exit 1
;;
esac
done
if [[ -z "$AGENT" || -z "$CADENCE" ]]; then
echo "ERROR: --agent and --cadence are required." >&2
exit 1
fi
if [[ "$CADENCE" != "daily" && "$CADENCE" != "weekly" ]]; then
echo "ERROR: --cadence must be 'daily' or 'weekly'." >&2
exit 1
fi
# ---------------------------------------------------------------------------
# Resolve token
# ---------------------------------------------------------------------------
TOKEN="${GITHUB_TOKEN:-${GH_TOKEN:-}}"
if [[ -z "$TOKEN" ]]; then
echo "ERROR: GITHUB_TOKEN (or GH_TOKEN) must be set for API access." >&2
echo "Without a token, remote task locking is impossible." >&2
exit 1
fi
# ---------------------------------------------------------------------------
# Helper: authenticated GitHub API call
# ---------------------------------------------------------------------------
gh_api() {
local method="$1"
local endpoint="$2"
local data="${3:-}"
local args=(
-s -f
-X "$method"
-H "Authorization: Bearer ${TOKEN}"
-H "Accept: application/vnd.github+json"
-H "X-GitHub-Api-Version: 2022-11-28"
)
if [[ -n "$data" ]]; then
args+=(-H "Content-Type: application/json" -d "$data")
fi
curl "${args[@]}" "${API_BASE}${endpoint}"
}
# ---------------------------------------------------------------------------
# Generate identifiers
# ---------------------------------------------------------------------------
TIMESTAMP=$(date -u +"%Y-%m-%d_%H-%M-%S")
DATE_ONLY=$(date -u +"%Y-%m-%d")
RUN_ID="${RUN_ID:-$(date -u +"%s")}"
BRANCH_NAME="agents/${CADENCE}/${AGENT}/${DATE_ONLY}-run-${RUN_ID}"
LOG_DIR="docs/agents/task-logs/${CADENCE}"
LOG_FILE="${TIMESTAMP}_${AGENT}_started.md"
LOG_PATH="${LOG_DIR}/${LOG_FILE}"
echo "Claiming task: ${AGENT} (${CADENCE})" >&2
echo "Branch: ${BRANCH_NAME}" >&2
echo "Log file: ${LOG_PATH}" >&2
# ---------------------------------------------------------------------------
# Step 1: Get the latest commit SHA of the base branch
# ---------------------------------------------------------------------------
echo "Step 1: Fetching base branch SHA (${BASE_BRANCH})..." >&2
BASE_REF_JSON=$(gh_api GET "/git/refs/heads/${BASE_BRANCH}")
BASE_SHA=$(echo "$BASE_REF_JSON" | jq -r '.object.sha // empty')
if [[ -z "$BASE_SHA" ]]; then
echo "ERROR: Could not resolve base branch '${BASE_BRANCH}'." >&2
echo "API response: ${BASE_REF_JSON}" >&2
exit 2
fi
echo " Base SHA: ${BASE_SHA}" >&2
# ---------------------------------------------------------------------------
# Step 2: Create the remote branch
# ---------------------------------------------------------------------------
echo "Step 2: Creating remote branch..." >&2
CREATE_REF_JSON=$(gh_api POST "/git/refs" "$(jq -n \
--arg ref "refs/heads/${BRANCH_NAME}" \
--arg sha "$BASE_SHA" \
'{ref: $ref, sha: $sha}')")
CREATED_REF=$(echo "$CREATE_REF_JSON" | jq -r '.ref // empty')
if [[ -z "$CREATED_REF" ]]; then
# Check if it's a 422 (already exists) — another agent may have claimed it
ERROR_MSG=$(echo "$CREATE_REF_JSON" | jq -r '.message // empty')
if [[ "$ERROR_MSG" == *"Reference already exists"* ]]; then
echo "RACE DETECTED: Branch '${BRANCH_NAME}' already exists. Another agent claimed first." >&2
exit 2
fi
echo "ERROR: Failed to create branch." >&2
echo "API response: ${CREATE_REF_JSON}" >&2
exit 2
fi
echo " Created: ${CREATED_REF}" >&2
# ---------------------------------------------------------------------------
# Step 3: Create the started.md log file on the branch
# ---------------------------------------------------------------------------
echo "Step 3: Creating started.md log file..." >&2
LOG_CONTENT=$(cat <<LOGEOF
# Started ${AGENT} run
Agent: ${AGENT}
Date: ${DATE_ONLY}
Branch: ${BRANCH_NAME}
Claimed-via: GitHub API (claim-task-api.sh)
LOGEOF
)
# Base64 encode the content (compatible with both GNU and BSD base64)
ENCODED_CONTENT=$(echo "$LOG_CONTENT" | base64 -w 0 2>/dev/null || echo "$LOG_CONTENT" | base64)
CREATE_FILE_JSON=$(gh_api PUT "/contents/${LOG_PATH}" "$(jq -n \
--arg message "chore(agents): claim ${CADENCE} task for ${AGENT}" \
--arg content "$ENCODED_CONTENT" \
--arg branch "$BRANCH_NAME" \
'{message: $message, content: $content, branch: $branch}')")
FILE_PATH_CREATED=$(echo "$CREATE_FILE_JSON" | jq -r '.content.path // empty')
if [[ -z "$FILE_PATH_CREATED" ]]; then
echo "ERROR: Failed to create log file." >&2
echo "API response: ${CREATE_FILE_JSON}" >&2
# Clean up: delete the branch we created
gh_api DELETE "/git/refs/heads/${BRANCH_NAME}" >/dev/null 2>&1 || true
exit 2
fi
echo " Created: ${FILE_PATH_CREATED}" >&2
# ---------------------------------------------------------------------------
# Step 4: Create a draft PR
# ---------------------------------------------------------------------------
echo "Step 4: Creating draft PR..." >&2
PR_TITLE="[${CADENCE}] ${AGENT}: ${DATE_ONLY} run"
PR_BODY="Automated task claim for **${AGENT}** (${CADENCE} cadence).
Branch: \`${BRANCH_NAME}\`
Claimed at: ${TIMESTAMP} UTC
Method: GitHub API (claim-task-api.sh)
---
*This draft PR serves as a distributed lock. Do not merge until the agent completes its work.*"
CREATE_PR_JSON=$(gh_api POST "/pulls" "$(jq -n \
--arg title "$PR_TITLE" \
--arg body "$PR_BODY" \
--arg head "$BRANCH_NAME" \
--arg base "$BASE_BRANCH" \
'{title: $title, body: $body, head: $head, base: $base, draft: true}')")
PR_NUMBER=$(echo "$CREATE_PR_JSON" | jq -r '.number // empty')
PR_URL=$(echo "$CREATE_PR_JSON" | jq -r '.html_url // empty')
if [[ -z "$PR_NUMBER" ]]; then
echo "WARNING: Draft PR creation failed. Branch and log file exist as partial lock." >&2
echo "API response: ${CREATE_PR_JSON}" >&2
# Don't exit — the branch itself is still a visible claim
PR_NUMBER="FAILED"
PR_URL="FAILED"
else
echo " PR #${PR_NUMBER}: ${PR_URL}" >&2
fi
# ---------------------------------------------------------------------------
# Output machine-readable results
# ---------------------------------------------------------------------------
echo "CLAIM_BRANCH=${BRANCH_NAME}"
echo "CLAIM_PR_NUMBER=${PR_NUMBER}"
echo "CLAIM_PR_URL=${PR_URL}"
echo "CLAIM_LOG_FILE=${LOG_PATH}"
echo ""
echo "Task claimed successfully." >&2