chore: standardize agent report locations and fix script paths

- Created `reports/` directory with subdirectories for each category (audit, performance, security, etc.).
- Moved existing reports to the new `reports/` structure.
- Updated `AGENTS.md` to define the "Report Location Standard".
- Updated all agent prompts in `src/prompts/` to:
    - Output reports to the new locations.
    - Fix incorrect script paths (replaced `torch/scripts/` with `scripts/`).
- Updated `load-test-agent.md` to produce a Markdown summary report.

Co-authored-by: PR0M3TH3AN <[email protected]>
This commit is contained in:
google-labs-jules[bot]andPR0M3TH3AN committed 2026-02-18 19:05:42 +00:00
1 parent 15375022bc
commit 31b18f5d5a
27 files changed
+167 -141

No files matched your search

+25
View File
@@ -112,3 +112,28 @@ Its goal is “make the suite reflect reality and reject fake passes.”
* Proposed test diffs
* Test Integrity Note
* “Cheat vectors blocked” summary (what trivial implementations it prevents)
---
## Report Location Standard
All agents that generate reports must adhere to the following directory structure. Reports must be placed in `reports/<category>/`.
### Categories
* **Audit**: `reports/audit/` (e.g., `audit-report-YYYY-MM-DD.md`)
* **Design System**: `reports/design-system/`
* **Known Issues**: `reports/known-issues/`
* **Load Test**: `reports/load-test/`
* **Performance**: `reports/performance/` (e.g., `daily-perf-report-YYYY-MM-DD.md`)
* **Protocol**: `reports/protocol/` (e.g., `protocol-report-YYYY-MM-DD.md`)
* **Security**: `reports/security/`
* **Telemetry**: `reports/telemetry/`
* **Test Audit**: `reports/test-audit/`
* **Weekly Synthesis**: `reports/weekly-synthesis/`
### Format
* Use Markdown (`.md`) for human-readable reports.
* Use JSON (`.json`) or Log (`.log`) for raw data/artifacts, but prefer placing a summary Markdown report in the corresponding folder.
* Naming convention: `<type>-report-YYYY-MM-DD.md` (e.g., `daily-perf-report-2024-01-01.md`).
File renamed without changes.
File renamed without changes.
File renamed without changes.
+25 -25
View File
@@ -29,11 +29,11 @@ REPO PREP (create/update these artifacts)
- `src/todo/TODO_<timestamp>.md` — checklist for the audit run and follow-ups.
- `src/decisions/DECISIONS_<timestamp>.md` — any decisions or tradeoffs (e.g., what qualifies as grandfathered).
- `src/test_logs/TEST_LOG_<timestamp>.md` — exact commands run + raw outputs (timestamped).
- `artifacts/audit/YYYY-MM-DD/` — store raw reports and parsed summaries:
- `reports/audit/` — store raw reports and parsed summaries:
- `file-size-report.json` (script raw output + parsed)
- `innerhtml-report.json`
- `lint-report.json`
- `summary.md` (human readable)
- `audit-report-YYYY-MM-DD.md` (human readable summary)
- `audit/` helper scripts (optional): parsing helpers, one-off scripts.
Commit these artifacts only if your process requires saving them to the repo; otherwise store them in a workspace and attach to GitHub items.
@@ -46,15 +46,15 @@ PRIMARY WORKFLOW (end-to-end)
- Read `AGENTS.md` and `CLAUDE.md`.
- Record environment:
- `node -v`, `npm -v` (or `pnpm`/`yarn`), OS, current `git rev-parse HEAD`
- Create the artifact directory: `artifacts/audit/$(date +%F)/`
- Create the artifact directory: `reports/audit/`
2. **Run audit scripts (report mode)**
- Run the three audit commands as stated:
- `node torch/scripts/check-file-size.mjs --report 2>&1 | tee artifacts/audit/$(date +%F)/raw-check-file-size.log`
- `node torch/scripts/check-innerhtml.mjs --report 2>&1 | tee artifacts/audit/$(date +%F)/raw-check-innerhtml.log`
- `npm run lint 2>&1 | tee artifacts/audit/$(date +%F)/raw-lint.log`
- `node scripts/check-file-size.mjs --report 2>&1 | tee reports/audit/raw-check-file-size-$(date +%F).log`
- `node scripts/check-innerhtml.mjs --report 2>&1 | tee reports/audit/raw-check-innerhtml-$(date +%F).log`
- `npm run lint 2>&1 | tee reports/audit/raw-lint-$(date +%F).log`
- If the repository uses `pnpm`/`yarn`, run lint with the repo’s recommended command from `package.json`.
- Save raw outputs in `artifacts/audit/YYYY-MM-DD/`.
- Save raw outputs in `reports/audit/`.
3. **Parse and compute metrics**
- **File-size script parsing**:
@@ -78,8 +78,8 @@ PRIMARY WORKFLOW (end-to-end)
- `files_with_errors`
- Examples / first N errors per file
- If the lint runner produces machine-readable JSON, prefer that for reliable parsing.
- Save parsed JSONs in `artifacts/audit/YYYY-MM-DD/`:
- `file-size-report.json`, `innerhtml-report.json`, `lint-report.json`.
- Save parsed JSONs in `reports/audit/`:
- `file-size-report-YYYY-MM-DD.json`, `innerhtml-report-YYYY-MM-DD.json`, `lint-report-YYYY-MM-DD.json`.
4. **Compare with previous week**
- Try to find last week’s audit report:
@@ -92,7 +92,7 @@ PRIMARY WORKFLOW (end-to-end)
- If not found, note this is the first report.
5. **Synthesize summary**
- Create `artifacts/audit/YYYY-MM-DD/summary.md` with:
- Create `reports/audit/audit-report-YYYY-MM-DD.md` with:
- Header: Date, branch, commit SHA, environment.
- Metrics:
- `Grandfathered oversized files`: N files, sum of excess lines M.
@@ -134,9 +134,9 @@ ADDITIONAL GUIDANCE & BEST PRACTICES
===============================================================================
SEARCH / PARSING HELPER SCRIPTS (examples)
- Parse file-size raw log into JSON:
- `node torch/scripts/parse-file-size-report.js artifacts/audit/YYYY-MM-DD/raw-check-file-size.log > artifacts/audit/YYYY-MM-DD/file-size-report.json`
- `node scripts/parse-file-size-report.js reports/audit/raw-check-file-size-YYYY-MM-DD.log > reports/audit/file-size-report-YYYY-MM-DD.json`
- Parse innerHTML log similarly:
- `node torch/scripts/parse-innerhtml-report.js ...`
- `node scripts/parse-innerhtml-report.js ...`
- If the repo’s scripts support `--json` or `--output`, prefer those flags.
If such parsers do not exist, implement tiny Node scripts that reliably parse the scripts’ expected output.
@@ -220,16 +220,16 @@ FIRST-RUN CHECKLIST (do this now)
1. Checkout and update:
- `git checkout <default-branch> && git pull --ff-only --ff-only`
2. Create artifacts directory:
- `mkdir -p artifacts/audit/$(date +%F)`
- `mkdir -p reports/audit/`
3. Run the 3 audit scripts in report mode:
- `node torch/scripts/check-file-size.mjs --report`
- `node torch/scripts/check-innerhtml.mjs --report`
- `node scripts/check-file-size.mjs --report`
- `node scripts/check-innerhtml.mjs --report`
- `npm run lint`
- Save raw outputs to `artifacts/audit/$(date +%F)/`
- Save raw outputs to `reports/audit/`
- Record commands & outputs in `src/test_logs/TEST_LOG_<timestamp>.md`
4. Parse outputs into JSON (use existing script or small Node parser).
5. Compute metrics and compare with the last report (search issues/PRs).
6. Create `artifacts/audit/$(date +%F)/summary.md`.
6. Create `reports/audit/audit-report-YYYY-MM-DD.md`.
7. Post summary comment on existing audit issue/PR or open a new `Audit Report — YYYY-MM-DD` issue (label `audit-report`) with artifacts attached.
8. If thresholds exceeded, open follow-up issues and tag maintainers.
@@ -241,14 +241,14 @@ FAILURE MODES
- If specific resources (files, URLs) are unavailable, log the error and skip.
OUTPUTS (what you must produce each run)
- `artifacts/audit/YYYY-MM-DD/` containing:
- `raw-check-file-size.log`
- `raw-check-innerhtml.log`
- `raw-lint.log`
- `file-size-report.json`
- `innerhtml-report.json`
- `lint-report.json`
- `summary.md`
- `reports/audit/` containing:
- `raw-check-file-size-YYYY-MM-DD.log`
- `raw-check-innerhtml-YYYY-MM-DD.log`
- `raw-lint-YYYY-MM-DD.log`
- `file-size-report-YYYY-MM-DD.json`
- `innerhtml-report-YYYY-MM-DD.json`
- `lint-report-YYYY-MM-DD.json`
- `audit-report-YYYY-MM-DD.md`
- `src/test_logs/TEST_LOG_<timestamp>.md` updated with commands, environment, and outputs.
- GitHub: comment on existing audit issue/PR or new issue `Audit Report — YYYY-MM-DD` with the summary and artifacts.
- If thresholds exceeded: new issues for regressions (label `perf`, `security`, or `lint`).
+9 -9
View File
@@ -40,7 +40,7 @@ Create / update these files in each PR branch and include them in the PR:
- `src/todo/TODO_<timestamp>.md` — List of duplicated constants to canonicalize and status for each.
- `src/decisions/DECISIONS_<timestamp>.md` — Where canonical constants were placed and why. Record alternatives considered.
- `src/test_logs/TEST_LOG_<timestamp>.md` — Exact commands run (lint/tests) and outputs.
- `perf/constants-refactor/` — optional folder to keep discovery data (raw grep output, candidate lists).
- `reports/performance/constants-refactor/` — optional folder to keep discovery data (raw grep output, candidate lists).
Commit these artifacts as part of the PR branch (except large raw logs you may attach to the PR).
@@ -67,14 +67,14 @@ DETAILED WORKFLOW (step-by-step)
- Thresholds: `0.8`, `0.9`, percent/limits like `100`, `0.5`
- Run a first-pass ripgrep to list numeric literals in `js/`:
```
rg --hidden --no-ignore -n --glob 'js/**/*.js' -e '\b(5000|10000|30000|60000|15000|3|5|0\.5|0\.8|600000|100)\b' > perf/constants-refactor/raw-numeric-hits.txt
rg --hidden --no-ignore -n --glob 'js/**/*.js' -e '\b(5000|10000|30000|60000|15000|3|5|0\.5|0\.8|600000|100)\b' > reports/performance/constants-refactor/raw-numeric-hits.txt
```
- (Adjust regex to include more candidates you want.)
- For a more robust approach, run an AST scan (Node/ES module) to find numeric literals:
- If comfortable, use `node` + `acorn` or `@babel/parser` to create a script (e.g., `torch/scripts/find-numeric-literals.js`) that produces JSON output of numeric literals with file, line, column, and nearby AST context (identifier or property name).
- If comfortable, use `node` + `acorn` or `@babel/parser` to create a script (e.g., `scripts/find-numeric-literals.js`) that produces JSON output of numeric literals with file, line, column, and nearby AST context (identifier or property name).
- Example (conceptual — script must be created first):
```
node torch/scripts/find-numeric-literals.js js > perf/constants-refactor/numeric-literals.json
node scripts/find-numeric-literals.js js > reports/performance/constants-refactor/numeric-literals.json
```
- From results, group occurrences by numeric value and inspect contexts to find semantic similarity (timeouts, retries, TTLs).
@@ -85,7 +85,7 @@ DETAILED WORKFLOW (step-by-step)
- Retry: look for `retry`, `attempt`, `republish`, `RETRY_DELAY_MS` usage.
- TTL: look for `cache`, `PROFILE_CACHE_TTL_MS`, `PERSIST_TTL`.
- Threshold: look for `threshold`, `limit`, `max`, `MIN_SCORE`.
- Produce a candidate entry in `perf/constants-refactor/candidates.json`:
- Produce a candidate entry in `reports/performance/constants-refactor/candidates.json`:
```
{
"value": 5000,
@@ -194,13 +194,13 @@ SEARCH / TOOLING SUGGESTIONS
- Quick search for specific numeric candidates:
````
rg --hidden --no-ignore -n --glob 'js/**/*.js' '\b(5000|10000|30000|60000|15000|3|5|0.5|600000)\b' > perf/constants-refactor/raw-hits.txt
rg --hidden --no-ignore -n --glob 'js/**/*.js' '\b(5000|10000|30000|60000|15000|3|5|0.5|600000)\b' > reports/performance/constants-refactor/raw-hits.txt
````
- AST-based approach (recommended for accuracy):
- Use `@babel/parser` to parse, walk nodes, and collect numeric literals with context (parent identifiers or property names). Example script:
- `torch/scripts/find-numeric-literals.js` (create if needed)
- Run: `node torch/scripts/find-numeric-literals.js js > perf/constants-refactor/numeric-literals.json`
- `scripts/find-numeric-literals.js` (create if needed)
- Run: `node scripts/find-numeric-literals.js js > reports/performance/constants-refactor/numeric-literals.json`
- For replacements, prefer editor/IDE or `jscodeshift` transforms to keep edits safe and consistent:
- Example: create a `jscodeshift` codemod that replaces numeric literal with identifier and adds import.
- If using manual edits, ensure ESLint passes and imports are correct.
@@ -254,7 +254,7 @@ FINAL NOTES & Etiquette
* Record every decision and test run to make audits traceable.
* Stop and request human review when unsure or when edits touch security-critical paths.
Begin now: checkout `<default-branch>`, run discovery searches/AST scan, produce `perf/constants-refactor/candidates.json`, and then implement the lowest-risk canonicalizations first following the guidance above.
Begin now: checkout `<default-branch>`, run discovery searches/AST scan, produce `reports/performance/constants-refactor/candidates.json`, and then implement the lowest-risk canonicalizations first following the guidance above.
FAILURE MODES
- If preconditions are not met, stop.
+11 -11
View File
@@ -12,13 +12,13 @@ You are: **decompose-agent**, a senior software engineer AI agent working inside
Your mission: **safely decompose a single large grandfathered file** (the largest one not decomposed recently) by extracting 2–3 cohesive blocks of logic into new modules. Keep the change purely structural — preserve runtime behavior exactly — and reduce the original file by **at least 200 lines** in the PR. Make one-file-per-PR changes only. Run lint & unit tests, update the file-size baseline, and submit an auditable PR.
This document is your operating manual. Follow it exactly: pick a single file, do small safe extractions, test thoroughly, update torch/scripts/check-file-size.mjs baseline, and open a PR with supporting documentation.
This document is your operating manual. Follow it exactly: pick a single file, do small safe extractions, test thoroughly, update scripts/check-file-size.mjs baseline, and open a PR with supporting documentation.
-------------------------------------------------------------------------------
HIGH-LEVEL RULES & SAFETY
- **Target branch:** `<default-branch>`. Always operate against `<default-branch>`.
- **One file per PR only.** Do not decompose multiple files in one PR or one run.
- **Pick the SINGLE largest grandfathered file** from `node torch/scripts/check-file-size.mjs --report` that **has not been decomposed recently**.
- **Pick the SINGLE largest grandfathered file** from `node scripts/check-file-size.mjs --report` that **has not been decomposed recently**.
- “Not decomposed recently” means no substantial refactor/large extraction commit touching this file in the last N weeks (choose 8 weeks by default) — check `git log`.
- **Do not change behavior.** This is a pure extraction refactor. Tests and manual QA must pass.
- **Do not alter generated or vendored files.** Only source files under `js/`.
@@ -32,14 +32,14 @@ Create or update these artifacts and include them in your PR branch:
- `src/todo/TODO_<timestamp>.md` — checklist of extraction items (2–3 blocks) and status.
- `src/decisions/DECISIONS_<timestamp>.md` — rationale for chosen extraction boundaries, naming, and module placement.
- `src/test_logs/TEST_LOG_<timestamp>.md` — exact commands run with outputs (lint/tests/manual checks).
- `perf/decompose/` — optional helpers, raw `node torch/scripts/check-file-size.mjs --report` output, and before/after line counts.
- `reports/performance/` — optional helpers, raw `node scripts/check-file-size.mjs --report` output, and before/after line counts.
-------------------------------------------------------------------------------
SELECTION PROCESS — how to pick the file
1. Run the file-size report:
```
node torch/scripts/check-file-size.mjs --report | tee perf/decompose/raw-file-size-report-$(date +%F).log
node scripts/check-file-size.mjs --report | tee reports/performance/raw-file-size-report-$(date +%F).log
````
2. From the report, find **grandfathered** oversized files and sort by largest line count.
@@ -176,19 +176,19 @@ TESTING & VERIFICATION (mandatory)
---
UPDATE BASELINE (torch/scripts/check-file-size.mjs)
UPDATE BASELINE (scripts/check-file-size.mjs)
* After the refactor, run the file-size report again:
```
node torch/scripts/check-file-size.mjs --report | tee perf/decompose/after-report-$(date +%F).log
node scripts/check-file-size.mjs --report | tee reports/performance/after-report-$(date +%F).log
```
* Run the update mode:
```
node torch/scripts/check-file-size.mjs --update
node scripts/check-file-size.mjs --update
```
* The script will print the new `BASELINE` object. **Copy the new `BASELINE`** into `torch/scripts/check-file-size.mjs` (replace old baseline object).
* The script will print the new `BASELINE` object. **Copy the new `BASELINE`** into `scripts/check-file-size.mjs` (replace old baseline object).
* Add a note to `src/decisions/DECISIONS_<timestamp>.md` explaining why the baseline changed (file reduced, new counts).
**Important:** Only update the baseline for this file if the size reduction is confirmed; do not change other files’ baseline entries.
@@ -229,7 +229,7 @@ ACCEPTANCE CRITERIA (must be true before merge)
* The original file is reduced by **at least 200 lines**.
* `npm run lint` passes.
* `npm run test:unit` passes.
* The updated `BASELINE` object in `torch/scripts/check-file-size.mjs` reflects the new smaller size.
* The updated `BASELINE` object in `scripts/check-file-size.mjs` reflects the new smaller size.
* No behavioral changes observed in smoke tests.
* PR contains required artifacts (files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`) and clear QA instructions.
@@ -259,14 +259,14 @@ FIRST-RUN CHECKLIST (execute now)
git checkout <default-branch>
git pull --ff-only
node -v && npm -v
node torch/scripts/check-file-size.mjs --report | tee perf/decompose/raw-file-size-report.log
node scripts/check-file-size.mjs --report | tee reports/performance/raw-file-size-report.log
```
2. Identify the largest grandfathered file not decomposed recently (use `git log`).
3. Create branch `ai/decompose/<short-file-name>-v1`.
4. Read file, identify 2–3 cohesive blocks, document in `src/context/CONTEXT_<timestamp>.md` and `src/todo/TODO_<timestamp>.md`.
5. Extract blocks to new files, import in original file.
6. Run `npm run lint` and `npm run test:unit`. Fix issues.
7. Update `torch/scripts/check-file-size.mjs` baseline via `--update`; copy new `BASELINE`.
7. Update `scripts/check-file-size.mjs` baseline via `--update`; copy new `BASELINE`.
8. Commit, push, open PR with required artifacts and description.
---
+18 -18
View File
@@ -22,7 +22,7 @@ SUMMARY / PRIMARY GOALS
- Creates issues for major/risky/security-sensitive upgrades with clear upgrade plans, tests, and rollback instructions.
- Escalates any critical vulnerabilities that affect production or crypto code immediately to maintainers.
- Success criteria:
- Daily artifacts: `artifacts/npm-audit.json`, `artifacts/npm-outdated.json`, `artifacts/deps-report.md`.
- Daily artifacts: `reports/security/npm-audit-YYYY-MM-DD.json`, `reports/security/npm-outdated-YYYY-MM-DD.json`, `reports/security/deps-report-YYYY-MM-DD.md`.
- Low-risk bumps produce PRs with test evidence. Major/risky upgrades produce issues describing the plan.
- Critical vulnerabilities are opened as high-priority `security` issues and maintainers are notified.
- Crypto/protocol or native-binding upgrades are never merged without explicit human review.
@@ -39,12 +39,12 @@ HARD CONSTRAINTS & GUARDRAILS
--------------------------------------------------------------------------------
REPO PREP — artifacts you must create/maintain
Create/update these files/folders (commit them in the branch when PRing):
- `artifacts/` — hold all audit/outdated/json/report outputs.
- `reports/security/` — hold all audit/outdated/json/report outputs.
- `src/context/CONTEXT_<timestamp>.md` — run metadata: date, package manager, Node version, CI matrix.
- `src/todo/TODO_<timestamp>.md` — upgrade tasks and statuses.
- `src/decisions/DECISIONS_<timestamp>.md` — rationale for upgrade choices and tradeoffs.
- `src/test_logs/TEST_LOG_<timestamp>.md` — exact commands run and their outputs (timestamped).
- Optionally: `torch/scripts/deps-audit.sh` for reproducible automation.
- Optionally: `scripts/deps-audit.sh` for reproducible automation.
Also read `AGENTS.md` and `KNOWN_ISSUES.md` for project-specific caveats (e.g., the repo uses `integration-tools`, `webtorrent`, `Playwright`, `Tailwind`). Tag these libraries for special handling.
@@ -70,12 +70,12 @@ B. Clean install
- Save outputs to `src/test_logs/TEST_LOG_<timestamp>.md`.
C. Run audits & outdated scans
- `npm audit --json > artifacts/npm-audit.json` (or `pnpm audit --json` / `yarn audit --json`)
- `npm outdated --json > artifacts/npm-outdated.json` (or `pnpm outdated --json` / `yarn outdated --json`)
- Optionally: `npx npm-check-updates --jsonUpgraded > artifacts/ncu.json` for a view of latest semver.
- `npm audit --json > reports/security/npm-audit-YYYY-MM-DD.json` (or `pnpm audit --json` / `yarn audit --json`)
- `npm outdated --json > reports/security/npm-outdated-YYYY-MM-DD.json` (or `pnpm outdated --json` / `yarn outdated --json`)
- Optionally: `npx npm-check-updates --jsonUpgraded > reports/security/ncu-YYYY-MM-DD.json` for a view of latest semver.
- If the org uses Snyk or similar, fetch current advisories and note them.
D. Produce `artifacts/deps-report.md` summarizing:
D. Produce `reports/security/deps-report-YYYY-MM-DD.md` summarizing:
- Vulnerable packages grouped by severity.
- Outdated packages separated into major/minor/patch.
- Immediate CRITICAL/HIGH vulnerabilities and audit paths (direct vs transitive).
@@ -86,7 +86,7 @@ E. Triage rules — automatic categorization
- If no work is required, exit without making changes.
- **CRITICAL / UNSAFE**:
- `critical` severity or CVSS >= 9 or direct exploit applicable to repo runtime, or crypto/protocol native package.
- Action: open `security` issue (P0), attach audit artifacts, and notify maintainers. Do **not** auto-bump.
- Action: open `security` issue (P0), attach audit reports, and notify maintainers. Do **not** auto-bump.
- **HIGH**:
- Try automatic patch/minor upgrade for **direct dependencies** if non-breaking. Run tests. If upgrade fails or is major, open `upgrade-<pkg>` issue.
- **MEDIUM**:
@@ -109,7 +109,7 @@ F. Attempt safe upgrades (patch/minor) — strict flow
- Open PR:
- Branch: `ai/deps-<pkg>-vX.Y.Z`
- Title: `chore(deps): bump <pkg> to vX.Y.Z`
- Body: include `artifacts/npm-audit.json`, `artifacts/npm-outdated.json`, `src/test_logs/TEST_LOG_<timestamp>.md` snippets, `src/decisions/DECISIONS_<timestamp>.md` rationale, QA steps, and rollback plan.
- Body: include `reports/security/npm-audit-*.json`, `reports/security/npm-outdated-*.json`, `src/test_logs/TEST_LOG_<timestamp>.md` snippets, `src/decisions/DECISIONS_<timestamp>.md` rationale, QA steps, and rollback plan.
- Labels: `chore(deps)` + `requires-review` (if not trivial) or `security` (if security-related).
6. If tests **FAIL**:
- Revert local changes.
@@ -152,7 +152,7 @@ K. Automation & bots
L. Escalation & immediate mitigation
- For critical exploitable vulnerabilities:
- Open a `security` issue marked `P0`.
- Attach `artifacts/npm-audit.json` and reproduction steps.
- Attach `reports/security/npm-audit-*.json` and reproduction steps.
- Propose immediate mitigation: pin, rollback, or temporary patch.
- Notify maintainers per repo policy (mention/tag team).
@@ -163,10 +163,10 @@ M. Lockfile & CI hygiene
--------------------------------------------------------------------------------
REPORTING — artifacts you must produce
- `artifacts/npm-audit.json` — raw audit output.
- `artifacts/npm-outdated.json` — raw outdated output.
- `artifacts/ncu.json` (optional) — output from `npm-check-updates`.
- `artifacts/deps-report.md` — human-readable triage: top risks, candidate upgrades, PRs created, issues filed, and escalation notes.
- `reports/security/npm-audit-YYYY-MM-DD.json` — raw audit output.
- `reports/security/npm-outdated-YYYY-MM-DD.json` — raw outdated output.
- `reports/security/ncu-YYYY-MM-DD.json` (optional) — output from `npm-check-updates`.
- `reports/security/deps-report-YYYY-MM-DD.md` — human-readable triage: top risks, candidate upgrades, PRs created, issues filed, and escalation notes.
- `src/test_logs/TEST_LOG_<timestamp>.md` — commands executed and test outputs for each attempted upgrade.
- PRs for safe upgrades and issues for risky/major/security upgrades.
@@ -177,7 +177,7 @@ PR & ISSUE TEMPLATE GUIDANCE
- PR body:
- Why: security/bug/maintenance reason.
- Commands run + test outputs (snippets), link to `src/test_logs/TEST_LOG_<timestamp>.md`.
- Artifacts: attach `artifacts/npm-audit.json` and `artifacts/npm-outdated.json`.
- Artifacts: attach `reports/security/npm-audit-*.json` and `reports/security/npm-outdated-*.json`.
- Risk assessment, manual QA steps, rollback plan.
- Labels: `chore(deps)`, `requires-review` / `security` as appropriate.
- Issue: `upgrade-<pkg>` — include reproduction steps, failures, approvals required, and remediation options.
@@ -205,15 +205,15 @@ NOTIFICATION & MAINTAINER ESCALATION
FIRST-RUN CHECKLIST (run immediately)
1. Detect package manager and record versions.
2. Run clean install (`npm ci` / `pnpm install` / `yarn install`) and record.
3. Run `npm audit --json > artifacts/npm-audit.json` and `npm outdated --json > artifacts/npm-outdated.json`.
4. Produce `artifacts/deps-report.md` with triage and a prioritized candidate list for safe bumps.
3. Run `npm audit --json > reports/security/npm-audit-YYYY-MM-DD.json` and `npm outdated --json > reports/security/npm-outdated-YYYY-MM-DD.json`.
4. Produce `reports/security/deps-report-YYYY-MM-DD.md` with triage and a prioritized candidate list for safe bumps.
5. Attempt the first small safe bump(s) (follow the safe upgrade flow) and open PR(s) or issue(s) accordingly.
--------------------------------------------------------------------------------
FINAL NOTE
Run this daily as a scheduled job or as part of CI cadence. Be conservative around cryptography, protocol, and native binary updates — always stop and open an issue for human review. Keep all actions reproducible: include commands, environment, Node & package manager versions, timestamps, and test logs.
Begin now: detect the package manager, run a fresh clean install, produce `artifacts/npm-audit.json` and `artifacts/npm-outdated.json`, and draft `artifacts/deps-report.md` with the initial triage and any immediate `P0` items.
Begin now: detect the package manager, run a fresh clean install, produce `reports/security/npm-audit-YYYY-MM-DD.json` and `reports/security/npm-outdated-YYYY-MM-DD.json`, and draft `reports/security/deps-report-YYYY-MM-DD.md` with the initial triage and any immediate `P0` items.
FAILURE MODES
- If preconditions are not met, stop.
@@ -120,7 +120,7 @@ WORKFLOW
- line number (when available)
- snippet (1–2 lines max)
- category
- Produce a remediation report (see “Reporting format required”).
- Produce a remediation report at `reports/design-system/design-system-report-YYYY-MM-DD.md` (and PR/Issue body).
4) Auto-fix safe, trivial cases (only if policy allows)
Allowed examples (only if verified as correct in repo policy):
@@ -151,7 +151,7 @@ WORKFLOW
`chore(ui): design system audit autofixes (agent)`
- Open a PR including the remediation report and commands run.
- If no safe auto-fixes were possible, or remaining violations need manual work:
- Open a GitHub issue with the remediation report, suggested owners, and
- Open a GitHub issue with the remediation report (linked or inline), suggested owners, and
next steps.
6) Verification
@@ -232,4 +232,4 @@ Partial success:
- Issue documenting remaining violations with owners + next steps
Always:
- A concise remediation report in the PR/issue following the required format
- A concise remediation report at `reports/design-system/design-system-report-YYYY-MM-DD.md` (and in the PR/issue following the required format)
@@ -18,7 +18,7 @@ This document is your operating manual. Follow it exactly: do not attempt to mig
===============================================================================
HARD CONSTRAINTS & SAFETY
- Target branch: **<default-branch>**. Do not modify other branches.
- Migrate **only ONE** file per PR. Choose a single file from `node torch/scripts/check-innerhtml.mjs --report`.
- Migrate **only ONE** file per PR. Choose a single file from `node scripts/check-innerhtml.mjs --report`.
- Do **not** change semantics. Preserve structure, event handlers, attributes, dataset values, and ordering.
- **Never** output user-supplied data as raw HTML. All interpolated user content must be escaped with `escapeHtml()` from `js/utils/domUtils.js`.
- Do **not** introduce new third-party libraries unless approved.
@@ -31,7 +31,7 @@ REPO PREP (create/update these artifacts)
- `src/todo/TODO_<timestamp>.md` — checklist of assignments in the file and status.
- `src/decisions/DECISIONS_<timestamp>.md` — decisions & rationale for each replaced `innerHTML` (escape vs textContent vs createElement).
- `src/test_logs/TEST_LOG_<timestamp>.md` — commands executed and outputs (lint/tests/manual checks).
- `perf/innerhtml/` — (optional) logs and the `check-innerhtml` raw report output.
- `reports/audit/` — (optional) logs and the `check-innerhtml` raw report output.
Commit these artifacts in the PR branch so reviewers can reproduce.
@@ -43,7 +43,7 @@ WORKFLOW — top-level steps (one file only)
- `node -v` and `npm -v` recorded in `src/context/CONTEXT_<timestamp>.md`
- Run the innerHTML report:
```
node torch/scripts/check-innerhtml.mjs --report | tee perf/innerhtml/raw-report-$(date +%F).log
node scripts/check-innerhtml.mjs --report | tee reports/audit/raw-report-$(date +%F).log
```
- Inspect the report and **choose exactly one file** to migrate. Prefer:
- Files labeled `RISKY` in the audit (user data without escaping).
@@ -114,9 +114,9 @@ WORKFLOW — top-level steps (one file only)
7. **Update baseline**
- After verifying, update the baseline counts so the `check-innerhtml` script knows this file was addressed:
```
node torch/scripts/check-innerhtml.mjs --update
node scripts/check-innerhtml.mjs --update
```
- The script prints a new `BASELINE` object. **Copy** the updated `BASELINE` object into `torch/scripts/check-innerhtml.mjs` (replace existing `BASELINE`).
- The script prints a new `BASELINE` object. **Copy** the updated `BASELINE` object into `scripts/check-innerhtml.mjs` (replace existing `BASELINE`).
- Commit the updated script with a note in `src/decisions/DECISIONS_<timestamp>.md` explaining the baseline change.
NOTE: Only update the baseline after verifying replacements and ensuring the overall innerHTML count has decreased appropriately.
@@ -136,7 +136,7 @@ WORKFLOW — top-level steps (one file only)
- files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`
- A short summary of the file and changes:
- List of assignments replaced and strategy for each.
- Before/after innerHTML counts (run `node torch/scripts/check-innerhtml.mjs --report` before and after and paste results).
- Before/after innerHTML counts (run `node scripts/check-innerhtml.mjs --report` before and after and paste results).
- Lint and unit test results.
- Manual QA steps and results.
- Highlight any remaining `innerHTML` assignments intentionally left (with rationale).
@@ -240,8 +240,8 @@ ACCEPTANCE CRITERIA (before merging)
* The chosen file’s `innerHTML` assignments are replaced or documented; user data is escaped or DOM-built.
* Lint passes and unit tests pass locally and in CI.
* `torch/scripts/check-innerhtml.mjs --report` shows reduced count for the chosen file.
* Updated `BASELINE` object copied into `torch/scripts/check-innerhtml.mjs`.
* `scripts/check-innerhtml.mjs --report` shows reduced count for the chosen file.
* Updated `BASELINE` object copied into `scripts/check-innerhtml.mjs`.
* PR contains files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`, summary of before/after, and QA steps.
* Commit message prefix is `[security]`.
@@ -263,7 +263,7 @@ FINAL NOTES & Etiquette
Begin now:
1. Run `node torch/scripts/check-innerhtml.mjs --report`
1. Run `node scripts/check-innerhtml.mjs --report`
2. Pick **ONE** file (RISKY/high-count preferred).
3. Follow the workflow above and open a `[security]` PR to `<default-branch>` when ready.
+4 -4
View File
@@ -98,8 +98,8 @@ DAILY WORKFLOW (mandatory)
- release channel expectations)
1) Create run artifacts (only if repo conventions allow committing artifacts)
- Prefer a dated folder:
- `artifacts/known-issues/YYYY-MM-DD/`
- Use report folder:
- `reports/known-issues/`
- If the repo does not commit artifacts, keep the report in the PR body or in
`docs/` per repo conventions.
@@ -176,7 +176,7 @@ DAILY WORKFLOW (mandatory)
- Remove dead links, fix paths, and clarify ambiguous instructions.
8) Daily report (required)
Produce a short report (PR body or artifact) containing:
Produce a short report at `reports/known-issues/known-issues-report-YYYY-MM-DD.md` (and optional PR body) containing:
- Headline: ✓ all verified / ⚠️ active issues remain
- Table/list: each entry → status (Resolved/Active/Unknown), last checked date
- PRs opened
@@ -221,4 +221,4 @@ Daily:
- 0–1 small PR fixing a trivial, safe known issue (or none if nothing safe)
- Updated `KNOWN_ISSUES.md` with last-checked notes and links
- 0–N GitHub issues for medium/large/risky items
- A short daily remediation report (artifact or PR body)
- A short daily remediation report at `reports/known-issues/known-issues-report-YYYY-MM-DD.md`
+9 -8
View File
@@ -28,12 +28,12 @@ policy and open an issue rather than improvising.
SCOPE
In scope:
- Creating/maintaining a load test script under `torch/scripts/agent/` (only if that
- Creating/maintaining a load test script under `scripts/agent/` (only if that
directory exists in repo; verify first).
- Simulating many clients connecting to a relay and publishing configurable
mixes of events at configurable rates.
- Measuring latency, throughput, error rates, and basic resource usage.
- Producing machine-readable reports under `artifacts/` (only if repo
- Producing machine-readable reports under `reports/load-test/` (only if repo
conventions permit committing artifacts).
- Opening issues for bottlenecks or unsafe behaviors.
@@ -98,12 +98,12 @@ WORKFLOW
- existing event schema builders (e.g., integration event schema utilities)
- existing relay interaction clients/helpers
- docs describing playback fallback and relay write safety
- Confirm whether `torch/scripts/agent/` and `artifacts/` are established patterns.
- Confirm whether `scripts/agent/` and `reports/load-test/` are established patterns.
If not, do not invent directory conventions—open an issue proposing layout.
2) Implement load harness (minimal, configurable)
Target file (only if verified appropriate in repo):
- `torch/scripts/agent/load-test.mjs`
- `scripts/agent/load-test.mjs`
The harness must support configuration via env vars and/or CLI flags:
- `RELAY_URL` (required)
@@ -143,8 +143,9 @@ WORKFLOW
- resource snapshots at intervals
4) Report generation
Output file (follow repo conventions; if allowed):
- `artifacts/load-report-YYYYMMDD.json`
Output files (follow repo conventions; if allowed):
- `reports/load-test/load-report-YYYY-MM-DD.json`
- `reports/load-test/load-test-report-YYYY-MM-DD.md` (human summary)
The report should include:
- Config used (relay URL redacted to hostname if needed, N, duration, rate, mix)
@@ -195,7 +196,7 @@ Open an issue instead of proceeding when:
───────────────────────────────────────────────────────────────────────────────
OUTPUTS PER RUN
- `torch/scripts/agent/load-test.mjs` (or repo-approved equivalent)
- A load report (JSON) produced locally (and committed only if repo allows)
- `scripts/agent/load-test.mjs` (or repo-approved equivalent)
- A load report (JSON + Markdown) produced locally (and committed only if repo allows)
- A prioritized list of bottlenecks tied to measured evidence
- 0–N issues for non-trivial bottlenecks, especially security-sensitive ones
+6 -6
View File
@@ -66,7 +66,7 @@ DAILY WORKFLOW (run each day)
2. **Search & Inventory**
- Run search patterns (see below).
- For each hit record: file, function, lines, trigger (load/login/visibility/user), frequency, main-thread? network intensity, user-visible impact.
- Produce `perf/hits-YYYY-MM-DD.json`.
- Produce `reports/performance/hits-YYYY-MM-DD.json`.
3. **Prioritize**
- P0: Blocks login/auth or decrypts UX-critical lists.
@@ -83,7 +83,7 @@ DAILY WORKFLOW (run each day)
- If upload/ingest code or `/content` is touched, run the docs audit workflow (inventory → verify → update → validate → deliver). Always leave `/content` runnable/precise.
6. **Report**
- Produce `daily-perf-report-YYYY-MM-DD.md` summarizing findings, PRs/issues, metrics, and blockers.
- Produce `reports/performance/daily-perf-report-YYYY-MM-DD.md` summarizing findings, PRs/issues, metrics, and blockers.
-------------------------------------------------------------------------------
TECHNICAL HEURISTICS & PATTERNS (what to look for & how to fix)
@@ -176,10 +176,10 @@ ESCALATION & RISK POLICY
-------------------------------------------------------------------------------
FIRST-RUN CHECKLIST (execute now)
1. Commit files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`, `INITIAL_BASELINE.md`.
2. Run Search Patterns across repo and save `perf/hits-YYYY-MM-DD.json`.
2. Run Search Patterns across repo and save `reports/performance/hits-YYYY-MM-DD.json`.
3. Prioritize P0 findings, open at most 1–3 small PRs (bounded concurrency, worker queue limits, visibility gating).
4. Audit `/content` upload pages: inventory claims → map to code → verify → open docs PR or issue.
5. Produce `daily-perf-report-YYYY-MM-DD.md` summarizing results and PR links.
5. Produce `reports/performance/daily-perf-report-YYYY-MM-DD.md` summarizing results and PR links.
-------------------------------------------------------------------------------
@@ -189,7 +189,7 @@ FAILURE MODES
- If specific resources (files, URLs) are unavailable, log the error and skip.
OUTPUTS (per run)
- `daily-perf-report-YYYY-MM-DD.md` with:
- `reports/performance/daily-perf-report-YYYY-MM-DD.md` with:
- Summary line.
- P0/P1/P2 findings (file/func/lines/impact/proposed fix/PR/Issue links).
- Metrics (login-time, queue sizes).
@@ -210,4 +210,4 @@ BEHAVIORAL GUARDRAILS & QUALITY BAR
FINAL NOTE
You are a helper, not a helicopter. Make safe, measurable progress daily. Keep the trail of decisions and tests clear and auditable for maintainers.
Begin now: run the Search Patterns, collect hits, and draft the first `daily-perf-report-YYYY-MM-DD.md`.
Begin now: run the Search Patterns, collect hits, and draft the first `reports/performance/daily-perf-report-YYYY-MM-DD.md`.
+2 -2
View File
@@ -37,12 +37,12 @@ Mission: identify external protocol/spec dependencies used by this codebase, map
3. Update `PROTOCOL_INVENTORY.md` with concrete code pointers.
3. Validate behavior with targeted unit/integration tests.
4. Classify gaps and propose minimal remediations.
5. Publish a dated report: `protocol-report-YYYY-MM-DD.md`.
5. Publish a dated report: `reports/protocol/protocol-report-YYYY-MM-DD.md`.
## Deliverables
- `PROTOCOL_INVENTORY.md`
- `protocol-report-YYYY-MM-DD.md`
- `reports/protocol/protocol-report-YYYY-MM-DD.md`
- Test additions or issue links for identified gaps
Keep outputs practical, evidence-based, and easy for maintainers to act on.
+1 -1
View File
@@ -19,4 +19,4 @@ MISSION:
STOP CONDITIONS:
- If intended behavior is unclear, do not guess and do not weaken tests.
Produce a “Needs Spec Clarification” report + propose candidate scenarios.
Produce a “Needs Spec Clarification” report in `reports/test-audit/test-audit-report-YYYY-MM-DD.md` + propose candidate scenarios.
@@ -93,8 +93,8 @@ WORKFLOW
- inspect repo for the correct serve path and update README or open an issue.
3) Implement a Playwright capture script
- Create (only if `torch/scripts/agent/` exists or is repo convention; verify first):
- `torch/scripts/agent/debug_frontend.py`
- Create (only if `scripts/agent/` exists or is repo convention; verify first):
- `scripts/agent/debug_frontend.py`
- The script must:
- open `http://localhost:8000`
- collect and print:
@@ -107,12 +107,12 @@ WORKFLOW
Logging/output:
- Save output to:
- `artifacts/debug_frontend.log`
- `reports/debug/debug_frontend.log`
(only commit artifacts if repo conventions allow; otherwise attach excerpt in PR)
4) Run capture and identify the first blocker
- Run:
- `python3 torch/scripts/agent/debug_frontend.py | tee artifacts/debug_frontend.log`
- `python3 scripts/agent/debug_frontend.py | tee reports/debug/debug_frontend.log`
- Diagnose the earliest blocking error:
- first `pageerror` OR first `console:error` that prevents initialization
- Extract:
@@ -136,7 +136,7 @@ WORKFLOW
- `npm run lint`
- `npm run test:unit`
- Re-run:
- `python3 torch/scripts/agent/debug_frontend.py | tee artifacts/debug_frontend.log`
- `python3 scripts/agent/debug_frontend.py | tee reports/debug/debug_frontend.log`
- Confirm:
- the original blocking error is gone
- no new initialization-blocking error was introduced
@@ -149,7 +149,7 @@ WORKFLOW
- Commit message (adjust to policy):
- `fix(ai): frontend console error — <short> (agent)`
- PR body must include:
- the top error excerpt(s) from `artifacts/debug_frontend.log`
- the top error excerpt(s) from `reports/debug/debug_frontend.log`
- root cause summary (file/function)
- files changed
- commands run + results
@@ -172,7 +172,7 @@ Issue must include:
───────────────────────────────────────────────────────────────────────────────
OUTPUTS PER RUN
- `torch/scripts/agent/debug_frontend.py` (if repo conventions allow)
- `artifacts/debug_frontend.log` (committed only if repo norms allow; otherwise excerpt in PR)
- `scripts/agent/debug_frontend.py` (if repo conventions allow)
- `reports/debug/debug_frontend.log` (committed only if repo norms allow; otherwise excerpt in PR)
- 0–1 PR fixing the first initialization-blocking console/page error
- 0–N issues for unsafe/ambiguous follow-ups
+7 -7
View File
@@ -90,9 +90,9 @@ WORKFLOW
1) Preflight
- Read `AGENTS.md` and `CLAUDE.md` for:
- security constraints (crypto, key handling, relay policy)
- conventions for `torch/scripts/agent/`, `examples/`, and artifacts
- conventions for `scripts/agent/`, `examples/`, and artifacts
- Confirm whether these directories exist and are used in repo:
- `torch/scripts/agent/`
- `scripts/agent/`
- `examples/reproducers/`
- `artifacts/`
If not present, do not invent structure—open an issue proposing where fuzz
@@ -112,8 +112,8 @@ WORKFLOW
- unhandled exception from invalid inputs
3) Implement the fuzz harness
File naming (only if `torch/scripts/agent/` is valid in repo):
- `torch/scripts/agent/fuzz-<target>.mjs`
File naming (only if `scripts/agent/` is valid in repo):
- `scripts/agent/fuzz-<target>.mjs`
Harness requirements:
- Inputs: randomized cases covering:
@@ -154,7 +154,7 @@ WORKFLOW
6) Reporting
Create a report file (commit only if repo conventions allow):
- `artifacts/fuzz-report-<target>-YYYYMMDD.json`
- `reports/security/fuzz-report-<target>-YYYY-MM-DD.json`
Report must include:
- target name + code pointers (file/function)
@@ -215,8 +215,8 @@ Issue must include:
───────────────────────────────────────────────────────────────────────────────
OUTPUTS PER RUN
- `torch/scripts/agent/fuzz-<target>.mjs` (if repo conventions allow)
- `scripts/agent/fuzz-<target>.mjs` (if repo conventions allow)
- 0–N reproducers under `examples/reproducers/` (if allowed)
- `artifacts/fuzz-report-<target>-YYYYMMDD.json` (committed only if allowed)
- `reports/security/fuzz-report-<target>-YYYY-MM-DD.json` (committed only if allowed)
- 0–1 PR (harness + repros, optionally small safe guards/tests)
- 0–N issues for sensitive or non-trivial fixes
+16 -16
View File
@@ -27,7 +27,7 @@ If a higher-level policy or doc conflicts with this prompt, follow the higher-le
SCOPE
In scope:
- Implement a small smoke-test harness at `torch/scripts/agent/smoke-test.mjs` (or `.ts`/`.py` if repo prefers), exercising:
- Implement a small smoke-test harness at `scripts/agent/smoke-test.mjs` (or `.ts`/`.py` if repo prefers), exercising:
- app start/serve (per README)
- headless browser/client run (Playwright preferred if available)
- sharedClient connection to a local/test relay
@@ -35,8 +35,8 @@ In scope:
- publish `VIDEO_POST` or `VIEW_EVENT` and verify read-back
- send encrypted DM and verify decryption with `js/dmDecryptor.js`
- Produce artifacts:
- `artifacts/smoke-YYYYMMDD.log` (human-readable)
- `artifacts/smoke-YYYYMMDD.json` (structured summary with timestamps)
- `reports/smoke-test/smoke-YYYY-MM-DD.log` (human-readable)
- `reports/smoke-test/smoke-YYYY-MM-DD.json` (structured summary with timestamps)
- screenshots for UI flows where applicable
- Provide run instructions for local and CI use, and open a PR with the test and docs.
@@ -83,7 +83,7 @@ PREPARATION (preflight)
WORKFLOW (implementation & run steps)
1) Create harness file
- Path: `torch/scripts/agent/smoke-test.mjs`
- Path: `scripts/agent/smoke-test.mjs`
- Language: follow repo conventions (Node/Playwright preferred). If Playwright is used, the harness should run headless.
- CLI/ENV options:
- `--relays` or `RELAY_URLS` (csv) — **required**
@@ -91,7 +91,7 @@ WORKFLOW (implementation & run steps)
- `--dry-run` — do everything but network/publish
- `--timeout` — per-step timeout (default 30s)
- `--burst` — per-publish burst (default 1)
- `--out` — artifacts output dir (default `artifacts/`)
- `--out` — artifacts output dir (default `reports/smoke-test/`)
- `--confirm-public` — explicit consent for limited public-relay runs
2) Start local server
@@ -133,9 +133,9 @@ WORKFLOW (implementation & run steps)
8) Logging & artifacts
- Log each step with timestamps and outcomes (success/fail)
- Save:
- `artifacts/smoke-YYYYMMDD.log` (text)
- `artifacts/smoke-YYYYMMDD.json` (summary: events published, ids, relays, timestamps, verification results)
- `artifacts/smoke-YYYYMMDD-screenshots/*` (Playwright screenshots on failure and optionally success)
- `reports/smoke-test/smoke-YYYY-MM-DD.log` (text)
- `reports/smoke-test/smoke-YYYY-MM-DD.json` (summary: events published, ids, relays, timestamps, verification results)
- `reports/smoke-test/screenshots/*` (Playwright screenshots on failure and optionally success)
- If failures occur, capture:
- console logs (browser and client)
- stack traces
@@ -162,12 +162,12 @@ EXAMPLE RUN
> Prompt authors: follow the canonical artifact paths in [Scheduler Flow → Canonical artifact paths](../scheduler-flow.md#canonical-artifact-paths).
RELAY_URLS="ws://localhost:8080" node torch/scripts/agent/smoke-test.mjs \
--serve=npx --timeout=30 --burst=1 --out=artifacts/
RELAY_URLS="ws://localhost:8080" node scripts/agent/smoke-test.mjs \
--serve=npx --timeout=30 --burst=1 --out=reports/smoke-test/
# public relays only with explicit confirmation and extreme throttle
RELAY_URLS="wss://relay.example" node torch/scripts/agent/smoke-test.mjs \
--confirm-public --burst=1 --timeout=30 --out=artifacts/
RELAY_URLS="wss://relay.example" node scripts/agent/smoke-test.mjs \
--confirm-public --burst=1 --timeout=30 --out=reports/smoke-test/
```
───────────────────────────────────────────────────────────────────────────────
@@ -175,7 +175,7 @@ FAILURE MODES
- **Startup Failure:** If the application fails to start (e.g., port in use, build error), log the stderr output and abort.
- **Relay Connection Failure:** If the client cannot connect to the specified relay, retry with backoff. If it still fails, abort and report "Relay Unreachable".
- **Element Not Found (UI):** If Playwright cannot find a required UI element (e.g., login button, post input), capture a screenshot and dump the DOM state to `artifacts/`. Fail the test step.
- **Element Not Found (UI):** If Playwright cannot find a required UI element (e.g., login button, post input), capture a screenshot and dump the DOM state to `reports/smoke-test/`. Fail the test step.
- **Verification Failure:** If the read-back event does not match the published event (content mismatch, signature error), log the diff and fail the test.
- **Timeout:** If any step exceeds the configured timeout, abort the run and capture a screenshot/log of the current state.
@@ -197,9 +197,9 @@ PR & COMMIT CONVENTIONS
OUTPUTS PER RUN
- **Artifacts:**
- `artifacts/smoke-YYYYMMDD.log`: Detailed execution log.
- `artifacts/smoke-YYYYMMDD.json`: Structured summary of the test run (steps, duration, result).
- `artifacts/smoke-YYYYMMDD-screenshots/`: Directory containing screenshots of failures (and success states if configured).
- `reports/smoke-test/smoke-YYYY-MM-DD.log`: Detailed execution log.
- `reports/smoke-test/smoke-YYYY-MM-DD.json`: Structured summary of the test run (steps, duration, result).
- `reports/smoke-test/screenshots/`: Directory containing screenshots of failures (and success states if configured).
- **Pull Request:** A PR containing the new or updated smoke test script and any necessary documentation changes.
- **Console Output:** Real-time progress updates and a final summary (Pass/Fail) printed to stdout.
+11 -11
View File
@@ -30,7 +30,7 @@ In scope:
- Collect CI/test run failures, smoke-test artifacts, and agent job errors (only from sources that have opted-in).
- Sanitize logs to remove PII, secrets, IPs, keys, and any sensitive identifiers.
- Fingerprint/cluster errors by normalized stack trace, count occurrences, compute recency and severity.
- Produce `artifacts/error-aggregates-YYYYMMDD.json` and `ai/reports/telemetry-YYYYMMDD.md`.
- Produce `reports/telemetry/error-aggregates-YYYY-MM-DD.json` and `reports/telemetry/telemetry-report-YYYY-MM-DD.md`.
- Suggest owners (via CODEOWNERS / module owners when available) and prioritized next actions.
Out of scope:
@@ -134,7 +134,7 @@ WORKFLOW (detailed)
- Keep only sanitized sample traces (max length), counts and metadata.
5) Generate artifacts
- `artifacts/error-aggregates-YYYYMMDD.json` (structured):
- `reports/telemetry/error-aggregates-YYYY-MM-DD.json` (structured):
```json
{
"generated_at": "...",
@@ -154,7 +154,7 @@ WORKFLOW (detailed)
]
}
```
- `ai/reports/telemetry-YYYYMMDD.md` (human summary):
- `reports/telemetry/telemetry-report-YYYY-MM-DD.md` (human summary):
- Headline: total errors, total groups
- Top-10 groups table: fingerprint, severity, count, owner, sample sanitized trace
- Prioritized Top-5 recommended human actions
@@ -170,11 +170,11 @@ WORKFLOW (detailed)
7) Commit / PR (if allowed)
- If committing artifacts is permitted:
- Branch: `ai/telemetry-YYYYMMDD`
- Branch: `ai/telemetry-YYYY-MM-DD`
- Commit message: `chore(ai): telemetry error aggregates (YYYY-MM-DD)`
- PR title: `docs(ai): telemetry report — YYYY-MM-DD`
- PR body: include the telemetry report summary and privacy audit
- If not permitted, attach `artifacts/error-aggregates-*.json` and the report to the PR body.
- If not permitted, attach `reports/telemetry/error-aggregates-*.json` and the report to the PR body.
───────────────────────────────────────────────────────────────────────────────
FAILURE MODES & GUARDRAILS (when to stop)
@@ -189,7 +189,7 @@ FAILURE MODES & GUARDRAILS (when to stop)
───────────────────────────────────────────────────────────────────────────────
REPORT FORMAT (required)
`ai/reports/telemetry-YYYYMMDD.md` should include:
`reports/telemetry/telemetry-report-YYYY-MM-DD.md` should include:
1. **Headline**
- `Total sanitized errors: N — Groups: M — Top severity: S`
@@ -224,10 +224,10 @@ PR & COMMIT CONVENTIONS
─────────────────────────────────────────────────────────────────────────────
OUTPUTS PER RUN
- `artifacts/error-aggregates-YYYYMMDD.json` (sanitized, structured)
- `ai/reports/telemetry-YYYYMMDD.md` (human-ready)
- `reports/telemetry/error-aggregates-YYYY-MM-DD.json` (sanitized, structured)
- `reports/telemetry/telemetry-report-YYYY-MM-DD.md` (human-ready)
- 0–N issues suggested for top errors (only if policy allows automation)
- If permitted: a PR `ai/telemetry-YYYYMMDD` with artifacts and privacy audit
- If permitted: a PR `ai/telemetry-YYYY-MM-DD` with artifacts and privacy audit
─────────────────────────────────────────────────────────────────────────────
BEGIN
@@ -236,7 +236,7 @@ BEGIN
2. Discover opt-in artifacts (CI runs with `TELEMETRY=1`, `artifacts/telemetry-optin/`, agent-run logs).
3. Sanitize each artifact (apply PII/key/IP/email redaction and log an audit of replacements).
4. Normalize & fingerprint stack traces; aggregate counts and compute severity.
5. Produce `artifacts/error-aggregates-YYYYMMDD.json` and `ai/reports/telemetry-YYYYMMDD.md`.
6. If artifacts are permitted to be committed, open `ai/telemetry-YYYYMMDD` PR with the report. Otherwise, attach in PR body and open issue(s) for human triage.
5. Produce `reports/telemetry/error-aggregates-YYYY-MM-DD.json` and `reports/telemetry/telemetry-report-YYYY-MM-DD.md`.
6. If artifacts are permitted to be committed, open `ai/telemetry-YYYY-MM-DD` PR with the report. Otherwise, attach in PR body and open issue(s) for human triage.
**Privacy-first rule:** If you cannot guarantee all PII/keys are redacted from every artifact, stop and request maintainer review — do not commit or publish raw logs.
+3 -3
View File
@@ -71,7 +71,7 @@ WORKFLOW
- branch naming conventions (e.g., `ai/*`)
- label conventions (e.g., `requires-security-review`, `requires-protocol-review`)
- Verify whether the target folder exists:
- `ai/reports/`
- `reports/weekly-synthesis/`
If it does not exist, do not invent new structure without checking policy:
- Prefer placing the report where existing reports live, or open an issue
proposing the location.
@@ -125,7 +125,7 @@ Optional method (use curl to query the GitHub API):
6) Write the report file
File name (if allowed by repo conventions):
- `ai/reports/weekly-YYYYMMDD.md`
- `reports/weekly-synthesis/weekly-report-YYYY-MM-DD.md`
Keep the report compact and scannable.
@@ -193,5 +193,5 @@ If report directory doesn’t exist and policy is unclear:
───────────────────────────────────────────────────────────────────────────────
OUTPUTS PER RUN
- `ai/reports/weekly-YYYYMMDD.md` (or repo-approved equivalent)
- `reports/weekly-synthesis/weekly-report-YYYY-MM-DD.md` (or repo-approved equivalent)
- One commit containing only the report (if committing is allowed by policy)