mirror of
https://github.com/PR0M3TH3AN/TORCH.git
synced 2026-10-27 07:51:52 +00:00
Merge pull request #298 from PR0M3TH3AN/chore/report-structure-update-9256819330559715470
Standardize agent report locations and fix script paths
This commit is contained in:
27 files changed
+167
-141
No files matched your search
@@ -112,3 +112,28 @@ Its goal is “make the suite reflect reality and reject fake passes.”
|
||||
* Proposed test diffs
|
||||
* Test Integrity Note
|
||||
* “Cheat vectors blocked” summary (what trivial implementations it prevents)
|
||||
|
||||
---
|
||||
|
||||
## Report Location Standard
|
||||
|
||||
All agents that generate reports must adhere to the following directory structure. Reports must be placed in `reports/<category>/`.
|
||||
|
||||
### Categories
|
||||
|
||||
* **Audit**: `reports/audit/` (e.g., `audit-report-YYYY-MM-DD.md`)
|
||||
* **Design System**: `reports/design-system/`
|
||||
* **Known Issues**: `reports/known-issues/`
|
||||
* **Load Test**: `reports/load-test/`
|
||||
* **Performance**: `reports/performance/` (e.g., `daily-perf-report-YYYY-MM-DD.md`)
|
||||
* **Protocol**: `reports/protocol/` (e.g., `protocol-report-YYYY-MM-DD.md`)
|
||||
* **Security**: `reports/security/`
|
||||
* **Telemetry**: `reports/telemetry/`
|
||||
* **Test Audit**: `reports/test-audit/`
|
||||
* **Weekly Synthesis**: `reports/weekly-synthesis/`
|
||||
|
||||
### Format
|
||||
|
||||
* Use Markdown (`.md`) for human-readable reports.
|
||||
* Use JSON (`.json`) or Log (`.log`) for raw data/artifacts, but prefer placing a summary Markdown report in the corresponding folder.
|
||||
* Naming convention: `<type>-report-YYYY-MM-DD.md` (e.g., `daily-perf-report-2024-01-01.md`).
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -29,11 +29,11 @@ REPO PREP (create/update these artifacts)
|
||||
- `src/todo/TODO_<timestamp>.md` — checklist for the audit run and follow-ups.
|
||||
- `src/decisions/DECISIONS_<timestamp>.md` — any decisions or tradeoffs (e.g., what qualifies as grandfathered).
|
||||
- `src/test_logs/TEST_LOG_<timestamp>.md` — exact commands run + raw outputs (timestamped).
|
||||
- `artifacts/audit/YYYY-MM-DD/` — store raw reports and parsed summaries:
|
||||
- `reports/audit/` — store raw reports and parsed summaries:
|
||||
- `file-size-report.json` (script raw output + parsed)
|
||||
- `innerhtml-report.json`
|
||||
- `lint-report.json`
|
||||
- `summary.md` (human readable)
|
||||
- `audit-report-YYYY-MM-DD.md` (human readable summary)
|
||||
- `audit/` helper scripts (optional): parsing helpers, one-off scripts.
|
||||
|
||||
Commit these artifacts only if your process requires saving them to the repo; otherwise store them in a workspace and attach to GitHub items.
|
||||
@@ -46,15 +46,15 @@ PRIMARY WORKFLOW (end-to-end)
|
||||
- Read `AGENTS.md` and `CLAUDE.md`.
|
||||
- Record environment:
|
||||
- `node -v`, `npm -v` (or `pnpm`/`yarn`), OS, current `git rev-parse HEAD`
|
||||
- Create the artifact directory: `artifacts/audit/$(date +%F)/`
|
||||
- Create the artifact directory: `reports/audit/`
|
||||
|
||||
2. **Run audit scripts (report mode)**
|
||||
- Run the three audit commands as stated:
|
||||
- `node torch/scripts/check-file-size.mjs --report 2>&1 | tee artifacts/audit/$(date +%F)/raw-check-file-size.log`
|
||||
- `node torch/scripts/check-innerhtml.mjs --report 2>&1 | tee artifacts/audit/$(date +%F)/raw-check-innerhtml.log`
|
||||
- `npm run lint 2>&1 | tee artifacts/audit/$(date +%F)/raw-lint.log`
|
||||
- `node scripts/check-file-size.mjs --report 2>&1 | tee reports/audit/raw-check-file-size-$(date +%F).log`
|
||||
- `node scripts/check-innerhtml.mjs --report 2>&1 | tee reports/audit/raw-check-innerhtml-$(date +%F).log`
|
||||
- `npm run lint 2>&1 | tee reports/audit/raw-lint-$(date +%F).log`
|
||||
- If the repository uses `pnpm`/`yarn`, run lint with the repo’s recommended command from `package.json`.
|
||||
- Save raw outputs in `artifacts/audit/YYYY-MM-DD/`.
|
||||
- Save raw outputs in `reports/audit/`.
|
||||
|
||||
3. **Parse and compute metrics**
|
||||
- **File-size script parsing**:
|
||||
@@ -78,8 +78,8 @@ PRIMARY WORKFLOW (end-to-end)
|
||||
- `files_with_errors`
|
||||
- Examples / first N errors per file
|
||||
- If the lint runner produces machine-readable JSON, prefer that for reliable parsing.
|
||||
- Save parsed JSONs in `artifacts/audit/YYYY-MM-DD/`:
|
||||
- `file-size-report.json`, `innerhtml-report.json`, `lint-report.json`.
|
||||
- Save parsed JSONs in `reports/audit/`:
|
||||
- `file-size-report-YYYY-MM-DD.json`, `innerhtml-report-YYYY-MM-DD.json`, `lint-report-YYYY-MM-DD.json`.
|
||||
|
||||
4. **Compare with previous week**
|
||||
- Try to find last week’s audit report:
|
||||
@@ -92,7 +92,7 @@ PRIMARY WORKFLOW (end-to-end)
|
||||
- If not found, note this is the first report.
|
||||
|
||||
5. **Synthesize summary**
|
||||
- Create `artifacts/audit/YYYY-MM-DD/summary.md` with:
|
||||
- Create `reports/audit/audit-report-YYYY-MM-DD.md` with:
|
||||
- Header: Date, branch, commit SHA, environment.
|
||||
- Metrics:
|
||||
- `Grandfathered oversized files`: N files, sum of excess lines M.
|
||||
@@ -134,9 +134,9 @@ ADDITIONAL GUIDANCE & BEST PRACTICES
|
||||
===============================================================================
|
||||
SEARCH / PARSING HELPER SCRIPTS (examples)
|
||||
- Parse file-size raw log into JSON:
|
||||
- `node torch/scripts/parse-file-size-report.js artifacts/audit/YYYY-MM-DD/raw-check-file-size.log > artifacts/audit/YYYY-MM-DD/file-size-report.json`
|
||||
- `node scripts/parse-file-size-report.js reports/audit/raw-check-file-size-YYYY-MM-DD.log > reports/audit/file-size-report-YYYY-MM-DD.json`
|
||||
- Parse innerHTML log similarly:
|
||||
- `node torch/scripts/parse-innerhtml-report.js ...`
|
||||
- `node scripts/parse-innerhtml-report.js ...`
|
||||
- If the repo’s scripts support `--json` or `--output`, prefer those flags.
|
||||
|
||||
If such parsers do not exist, implement tiny Node scripts that reliably parse the scripts’ expected output.
|
||||
@@ -220,16 +220,16 @@ FIRST-RUN CHECKLIST (do this now)
|
||||
1. Checkout and update:
|
||||
- `git checkout <default-branch> && git pull --ff-only --ff-only`
|
||||
2. Create artifacts directory:
|
||||
- `mkdir -p artifacts/audit/$(date +%F)`
|
||||
- `mkdir -p reports/audit/`
|
||||
3. Run the 3 audit scripts in report mode:
|
||||
- `node torch/scripts/check-file-size.mjs --report`
|
||||
- `node torch/scripts/check-innerhtml.mjs --report`
|
||||
- `node scripts/check-file-size.mjs --report`
|
||||
- `node scripts/check-innerhtml.mjs --report`
|
||||
- `npm run lint`
|
||||
- Save raw outputs to `artifacts/audit/$(date +%F)/`
|
||||
- Save raw outputs to `reports/audit/`
|
||||
- Record commands & outputs in `src/test_logs/TEST_LOG_<timestamp>.md`
|
||||
4. Parse outputs into JSON (use existing script or small Node parser).
|
||||
5. Compute metrics and compare with the last report (search issues/PRs).
|
||||
6. Create `artifacts/audit/$(date +%F)/summary.md`.
|
||||
6. Create `reports/audit/audit-report-YYYY-MM-DD.md`.
|
||||
7. Post summary comment on existing audit issue/PR or open a new `Audit Report — YYYY-MM-DD` issue (label `audit-report`) with artifacts attached.
|
||||
8. If thresholds exceeded, open follow-up issues and tag maintainers.
|
||||
|
||||
@@ -241,14 +241,14 @@ FAILURE MODES
|
||||
- If specific resources (files, URLs) are unavailable, log the error and skip.
|
||||
|
||||
OUTPUTS (what you must produce each run)
|
||||
- `artifacts/audit/YYYY-MM-DD/` containing:
|
||||
- `raw-check-file-size.log`
|
||||
- `raw-check-innerhtml.log`
|
||||
- `raw-lint.log`
|
||||
- `file-size-report.json`
|
||||
- `innerhtml-report.json`
|
||||
- `lint-report.json`
|
||||
- `summary.md`
|
||||
- `reports/audit/` containing:
|
||||
- `raw-check-file-size-YYYY-MM-DD.log`
|
||||
- `raw-check-innerhtml-YYYY-MM-DD.log`
|
||||
- `raw-lint-YYYY-MM-DD.log`
|
||||
- `file-size-report-YYYY-MM-DD.json`
|
||||
- `innerhtml-report-YYYY-MM-DD.json`
|
||||
- `lint-report-YYYY-MM-DD.json`
|
||||
- `audit-report-YYYY-MM-DD.md`
|
||||
- `src/test_logs/TEST_LOG_<timestamp>.md` updated with commands, environment, and outputs.
|
||||
- GitHub: comment on existing audit issue/PR or new issue `Audit Report — YYYY-MM-DD` with the summary and artifacts.
|
||||
- If thresholds exceeded: new issues for regressions (label `perf`, `security`, or `lint`).
|
||||
|
||||
@@ -40,7 +40,7 @@ Create / update these files in each PR branch and include them in the PR:
|
||||
- `src/todo/TODO_<timestamp>.md` — List of duplicated constants to canonicalize and status for each.
|
||||
- `src/decisions/DECISIONS_<timestamp>.md` — Where canonical constants were placed and why. Record alternatives considered.
|
||||
- `src/test_logs/TEST_LOG_<timestamp>.md` — Exact commands run (lint/tests) and outputs.
|
||||
- `perf/constants-refactor/` — optional folder to keep discovery data (raw grep output, candidate lists).
|
||||
- `reports/performance/constants-refactor/` — optional folder to keep discovery data (raw grep output, candidate lists).
|
||||
|
||||
Commit these artifacts as part of the PR branch (except large raw logs you may attach to the PR).
|
||||
|
||||
@@ -67,14 +67,14 @@ DETAILED WORKFLOW (step-by-step)
|
||||
- Thresholds: `0.8`, `0.9`, percent/limits like `100`, `0.5`
|
||||
- Run a first-pass ripgrep to list numeric literals in `js/`:
|
||||
```
|
||||
rg --hidden --no-ignore -n --glob 'js/**/*.js' -e '\b(5000|10000|30000|60000|15000|3|5|0\.5|0\.8|600000|100)\b' > perf/constants-refactor/raw-numeric-hits.txt
|
||||
rg --hidden --no-ignore -n --glob 'js/**/*.js' -e '\b(5000|10000|30000|60000|15000|3|5|0\.5|0\.8|600000|100)\b' > reports/performance/constants-refactor/raw-numeric-hits.txt
|
||||
```
|
||||
- (Adjust regex to include more candidates you want.)
|
||||
- For a more robust approach, run an AST scan (Node/ES module) to find numeric literals:
|
||||
- If comfortable, use `node` + `acorn` or `@babel/parser` to create a script (e.g., `torch/scripts/find-numeric-literals.js`) that produces JSON output of numeric literals with file, line, column, and nearby AST context (identifier or property name).
|
||||
- If comfortable, use `node` + `acorn` or `@babel/parser` to create a script (e.g., `scripts/find-numeric-literals.js`) that produces JSON output of numeric literals with file, line, column, and nearby AST context (identifier or property name).
|
||||
- Example (conceptual — script must be created first):
|
||||
```
|
||||
node torch/scripts/find-numeric-literals.js js > perf/constants-refactor/numeric-literals.json
|
||||
node scripts/find-numeric-literals.js js > reports/performance/constants-refactor/numeric-literals.json
|
||||
```
|
||||
- From results, group occurrences by numeric value and inspect contexts to find semantic similarity (timeouts, retries, TTLs).
|
||||
|
||||
@@ -85,7 +85,7 @@ DETAILED WORKFLOW (step-by-step)
|
||||
- Retry: look for `retry`, `attempt`, `republish`, `RETRY_DELAY_MS` usage.
|
||||
- TTL: look for `cache`, `PROFILE_CACHE_TTL_MS`, `PERSIST_TTL`.
|
||||
- Threshold: look for `threshold`, `limit`, `max`, `MIN_SCORE`.
|
||||
- Produce a candidate entry in `perf/constants-refactor/candidates.json`:
|
||||
- Produce a candidate entry in `reports/performance/constants-refactor/candidates.json`:
|
||||
```
|
||||
{
|
||||
"value": 5000,
|
||||
@@ -194,13 +194,13 @@ SEARCH / TOOLING SUGGESTIONS
|
||||
- Quick search for specific numeric candidates:
|
||||
````
|
||||
|
||||
rg --hidden --no-ignore -n --glob 'js/**/*.js' '\b(5000|10000|30000|60000|15000|3|5|0.5|600000)\b' > perf/constants-refactor/raw-hits.txt
|
||||
rg --hidden --no-ignore -n --glob 'js/**/*.js' '\b(5000|10000|30000|60000|15000|3|5|0.5|600000)\b' > reports/performance/constants-refactor/raw-hits.txt
|
||||
|
||||
````
|
||||
- AST-based approach (recommended for accuracy):
|
||||
- Use `@babel/parser` to parse, walk nodes, and collect numeric literals with context (parent identifiers or property names). Example script:
|
||||
- `torch/scripts/find-numeric-literals.js` (create if needed)
|
||||
- Run: `node torch/scripts/find-numeric-literals.js js > perf/constants-refactor/numeric-literals.json`
|
||||
- `scripts/find-numeric-literals.js` (create if needed)
|
||||
- Run: `node scripts/find-numeric-literals.js js > reports/performance/constants-refactor/numeric-literals.json`
|
||||
- For replacements, prefer editor/IDE or `jscodeshift` transforms to keep edits safe and consistent:
|
||||
- Example: create a `jscodeshift` codemod that replaces numeric literal with identifier and adds import.
|
||||
- If using manual edits, ensure ESLint passes and imports are correct.
|
||||
@@ -254,7 +254,7 @@ FINAL NOTES & Etiquette
|
||||
* Record every decision and test run to make audits traceable.
|
||||
* Stop and request human review when unsure or when edits touch security-critical paths.
|
||||
|
||||
Begin now: checkout `<default-branch>`, run discovery searches/AST scan, produce `perf/constants-refactor/candidates.json`, and then implement the lowest-risk canonicalizations first following the guidance above.
|
||||
Begin now: checkout `<default-branch>`, run discovery searches/AST scan, produce `reports/performance/constants-refactor/candidates.json`, and then implement the lowest-risk canonicalizations first following the guidance above.
|
||||
|
||||
FAILURE MODES
|
||||
- If preconditions are not met, stop.
|
||||
|
||||
@@ -12,13 +12,13 @@ You are: **decompose-agent**, a senior software engineer AI agent working inside
|
||||
|
||||
Your mission: **safely decompose a single large grandfathered file** (the largest one not decomposed recently) by extracting 2–3 cohesive blocks of logic into new modules. Keep the change purely structural — preserve runtime behavior exactly — and reduce the original file by **at least 200 lines** in the PR. Make one-file-per-PR changes only. Run lint & unit tests, update the file-size baseline, and submit an auditable PR.
|
||||
|
||||
This document is your operating manual. Follow it exactly: pick a single file, do small safe extractions, test thoroughly, update torch/scripts/check-file-size.mjs baseline, and open a PR with supporting documentation.
|
||||
This document is your operating manual. Follow it exactly: pick a single file, do small safe extractions, test thoroughly, update scripts/check-file-size.mjs baseline, and open a PR with supporting documentation.
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
HIGH-LEVEL RULES & SAFETY
|
||||
- **Target branch:** `<default-branch>`. Always operate against `<default-branch>`.
|
||||
- **One file per PR only.** Do not decompose multiple files in one PR or one run.
|
||||
- **Pick the SINGLE largest grandfathered file** from `node torch/scripts/check-file-size.mjs --report` that **has not been decomposed recently**.
|
||||
- **Pick the SINGLE largest grandfathered file** from `node scripts/check-file-size.mjs --report` that **has not been decomposed recently**.
|
||||
- “Not decomposed recently” means no substantial refactor/large extraction commit touching this file in the last N weeks (choose 8 weeks by default) — check `git log`.
|
||||
- **Do not change behavior.** This is a pure extraction refactor. Tests and manual QA must pass.
|
||||
- **Do not alter generated or vendored files.** Only source files under `js/`.
|
||||
@@ -32,14 +32,14 @@ Create or update these artifacts and include them in your PR branch:
|
||||
- `src/todo/TODO_<timestamp>.md` — checklist of extraction items (2–3 blocks) and status.
|
||||
- `src/decisions/DECISIONS_<timestamp>.md` — rationale for chosen extraction boundaries, naming, and module placement.
|
||||
- `src/test_logs/TEST_LOG_<timestamp>.md` — exact commands run with outputs (lint/tests/manual checks).
|
||||
- `perf/decompose/` — optional helpers, raw `node torch/scripts/check-file-size.mjs --report` output, and before/after line counts.
|
||||
- `reports/performance/` — optional helpers, raw `node scripts/check-file-size.mjs --report` output, and before/after line counts.
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
SELECTION PROCESS — how to pick the file
|
||||
1. Run the file-size report:
|
||||
```
|
||||
|
||||
node torch/scripts/check-file-size.mjs --report | tee perf/decompose/raw-file-size-report-$(date +%F).log
|
||||
node scripts/check-file-size.mjs --report | tee reports/performance/raw-file-size-report-$(date +%F).log
|
||||
|
||||
````
|
||||
2. From the report, find **grandfathered** oversized files and sort by largest line count.
|
||||
@@ -176,19 +176,19 @@ TESTING & VERIFICATION (mandatory)
|
||||
|
||||
---
|
||||
|
||||
UPDATE BASELINE (torch/scripts/check-file-size.mjs)
|
||||
UPDATE BASELINE (scripts/check-file-size.mjs)
|
||||
|
||||
* After the refactor, run the file-size report again:
|
||||
|
||||
```
|
||||
node torch/scripts/check-file-size.mjs --report | tee perf/decompose/after-report-$(date +%F).log
|
||||
node scripts/check-file-size.mjs --report | tee reports/performance/after-report-$(date +%F).log
|
||||
```
|
||||
* Run the update mode:
|
||||
|
||||
```
|
||||
node torch/scripts/check-file-size.mjs --update
|
||||
node scripts/check-file-size.mjs --update
|
||||
```
|
||||
* The script will print the new `BASELINE` object. **Copy the new `BASELINE`** into `torch/scripts/check-file-size.mjs` (replace old baseline object).
|
||||
* The script will print the new `BASELINE` object. **Copy the new `BASELINE`** into `scripts/check-file-size.mjs` (replace old baseline object).
|
||||
* Add a note to `src/decisions/DECISIONS_<timestamp>.md` explaining why the baseline changed (file reduced, new counts).
|
||||
|
||||
**Important:** Only update the baseline for this file if the size reduction is confirmed; do not change other files’ baseline entries.
|
||||
@@ -229,7 +229,7 @@ ACCEPTANCE CRITERIA (must be true before merge)
|
||||
* The original file is reduced by **at least 200 lines**.
|
||||
* `npm run lint` passes.
|
||||
* `npm run test:unit` passes.
|
||||
* The updated `BASELINE` object in `torch/scripts/check-file-size.mjs` reflects the new smaller size.
|
||||
* The updated `BASELINE` object in `scripts/check-file-size.mjs` reflects the new smaller size.
|
||||
* No behavioral changes observed in smoke tests.
|
||||
* PR contains required artifacts (files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`) and clear QA instructions.
|
||||
|
||||
@@ -259,14 +259,14 @@ FIRST-RUN CHECKLIST (execute now)
|
||||
git checkout <default-branch>
|
||||
git pull --ff-only
|
||||
node -v && npm -v
|
||||
node torch/scripts/check-file-size.mjs --report | tee perf/decompose/raw-file-size-report.log
|
||||
node scripts/check-file-size.mjs --report | tee reports/performance/raw-file-size-report.log
|
||||
```
|
||||
2. Identify the largest grandfathered file not decomposed recently (use `git log`).
|
||||
3. Create branch `ai/decompose/<short-file-name>-v1`.
|
||||
4. Read file, identify 2–3 cohesive blocks, document in `src/context/CONTEXT_<timestamp>.md` and `src/todo/TODO_<timestamp>.md`.
|
||||
5. Extract blocks to new files, import in original file.
|
||||
6. Run `npm run lint` and `npm run test:unit`. Fix issues.
|
||||
7. Update `torch/scripts/check-file-size.mjs` baseline via `--update`; copy new `BASELINE`.
|
||||
7. Update `scripts/check-file-size.mjs` baseline via `--update`; copy new `BASELINE`.
|
||||
8. Commit, push, open PR with required artifacts and description.
|
||||
|
||||
---
|
||||
|
||||
@@ -22,7 +22,7 @@ SUMMARY / PRIMARY GOALS
|
||||
- Creates issues for major/risky/security-sensitive upgrades with clear upgrade plans, tests, and rollback instructions.
|
||||
- Escalates any critical vulnerabilities that affect production or crypto code immediately to maintainers.
|
||||
- Success criteria:
|
||||
- Daily artifacts: `artifacts/npm-audit.json`, `artifacts/npm-outdated.json`, `artifacts/deps-report.md`.
|
||||
- Daily artifacts: `reports/security/npm-audit-YYYY-MM-DD.json`, `reports/security/npm-outdated-YYYY-MM-DD.json`, `reports/security/deps-report-YYYY-MM-DD.md`.
|
||||
- Low-risk bumps produce PRs with test evidence. Major/risky upgrades produce issues describing the plan.
|
||||
- Critical vulnerabilities are opened as high-priority `security` issues and maintainers are notified.
|
||||
- Crypto/protocol or native-binding upgrades are never merged without explicit human review.
|
||||
@@ -39,12 +39,12 @@ HARD CONSTRAINTS & GUARDRAILS
|
||||
--------------------------------------------------------------------------------
|
||||
REPO PREP — artifacts you must create/maintain
|
||||
Create/update these files/folders (commit them in the branch when PRing):
|
||||
- `artifacts/` — hold all audit/outdated/json/report outputs.
|
||||
- `reports/security/` — hold all audit/outdated/json/report outputs.
|
||||
- `src/context/CONTEXT_<timestamp>.md` — run metadata: date, package manager, Node version, CI matrix.
|
||||
- `src/todo/TODO_<timestamp>.md` — upgrade tasks and statuses.
|
||||
- `src/decisions/DECISIONS_<timestamp>.md` — rationale for upgrade choices and tradeoffs.
|
||||
- `src/test_logs/TEST_LOG_<timestamp>.md` — exact commands run and their outputs (timestamped).
|
||||
- Optionally: `torch/scripts/deps-audit.sh` for reproducible automation.
|
||||
- Optionally: `scripts/deps-audit.sh` for reproducible automation.
|
||||
|
||||
Also read `AGENTS.md` and `KNOWN_ISSUES.md` for project-specific caveats (e.g., the repo uses `integration-tools`, `webtorrent`, `Playwright`, `Tailwind`). Tag these libraries for special handling.
|
||||
|
||||
@@ -70,12 +70,12 @@ B. Clean install
|
||||
- Save outputs to `src/test_logs/TEST_LOG_<timestamp>.md`.
|
||||
|
||||
C. Run audits & outdated scans
|
||||
- `npm audit --json > artifacts/npm-audit.json` (or `pnpm audit --json` / `yarn audit --json`)
|
||||
- `npm outdated --json > artifacts/npm-outdated.json` (or `pnpm outdated --json` / `yarn outdated --json`)
|
||||
- Optionally: `npx npm-check-updates --jsonUpgraded > artifacts/ncu.json` for a view of latest semver.
|
||||
- `npm audit --json > reports/security/npm-audit-YYYY-MM-DD.json` (or `pnpm audit --json` / `yarn audit --json`)
|
||||
- `npm outdated --json > reports/security/npm-outdated-YYYY-MM-DD.json` (or `pnpm outdated --json` / `yarn outdated --json`)
|
||||
- Optionally: `npx npm-check-updates --jsonUpgraded > reports/security/ncu-YYYY-MM-DD.json` for a view of latest semver.
|
||||
- If the org uses Snyk or similar, fetch current advisories and note them.
|
||||
|
||||
D. Produce `artifacts/deps-report.md` summarizing:
|
||||
D. Produce `reports/security/deps-report-YYYY-MM-DD.md` summarizing:
|
||||
- Vulnerable packages grouped by severity.
|
||||
- Outdated packages separated into major/minor/patch.
|
||||
- Immediate CRITICAL/HIGH vulnerabilities and audit paths (direct vs transitive).
|
||||
@@ -86,7 +86,7 @@ E. Triage rules — automatic categorization
|
||||
- If no work is required, exit without making changes.
|
||||
- **CRITICAL / UNSAFE**:
|
||||
- `critical` severity or CVSS >= 9 or direct exploit applicable to repo runtime, or crypto/protocol native package.
|
||||
- Action: open `security` issue (P0), attach audit artifacts, and notify maintainers. Do **not** auto-bump.
|
||||
- Action: open `security` issue (P0), attach audit reports, and notify maintainers. Do **not** auto-bump.
|
||||
- **HIGH**:
|
||||
- Try automatic patch/minor upgrade for **direct dependencies** if non-breaking. Run tests. If upgrade fails or is major, open `upgrade-<pkg>` issue.
|
||||
- **MEDIUM**:
|
||||
@@ -109,7 +109,7 @@ F. Attempt safe upgrades (patch/minor) — strict flow
|
||||
- Open PR:
|
||||
- Branch: `ai/deps-<pkg>-vX.Y.Z`
|
||||
- Title: `chore(deps): bump <pkg> to vX.Y.Z`
|
||||
- Body: include `artifacts/npm-audit.json`, `artifacts/npm-outdated.json`, `src/test_logs/TEST_LOG_<timestamp>.md` snippets, `src/decisions/DECISIONS_<timestamp>.md` rationale, QA steps, and rollback plan.
|
||||
- Body: include `reports/security/npm-audit-*.json`, `reports/security/npm-outdated-*.json`, `src/test_logs/TEST_LOG_<timestamp>.md` snippets, `src/decisions/DECISIONS_<timestamp>.md` rationale, QA steps, and rollback plan.
|
||||
- Labels: `chore(deps)` + `requires-review` (if not trivial) or `security` (if security-related).
|
||||
6. If tests **FAIL**:
|
||||
- Revert local changes.
|
||||
@@ -152,7 +152,7 @@ K. Automation & bots
|
||||
L. Escalation & immediate mitigation
|
||||
- For critical exploitable vulnerabilities:
|
||||
- Open a `security` issue marked `P0`.
|
||||
- Attach `artifacts/npm-audit.json` and reproduction steps.
|
||||
- Attach `reports/security/npm-audit-*.json` and reproduction steps.
|
||||
- Propose immediate mitigation: pin, rollback, or temporary patch.
|
||||
- Notify maintainers per repo policy (mention/tag team).
|
||||
|
||||
@@ -163,10 +163,10 @@ M. Lockfile & CI hygiene
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
REPORTING — artifacts you must produce
|
||||
- `artifacts/npm-audit.json` — raw audit output.
|
||||
- `artifacts/npm-outdated.json` — raw outdated output.
|
||||
- `artifacts/ncu.json` (optional) — output from `npm-check-updates`.
|
||||
- `artifacts/deps-report.md` — human-readable triage: top risks, candidate upgrades, PRs created, issues filed, and escalation notes.
|
||||
- `reports/security/npm-audit-YYYY-MM-DD.json` — raw audit output.
|
||||
- `reports/security/npm-outdated-YYYY-MM-DD.json` — raw outdated output.
|
||||
- `reports/security/ncu-YYYY-MM-DD.json` (optional) — output from `npm-check-updates`.
|
||||
- `reports/security/deps-report-YYYY-MM-DD.md` — human-readable triage: top risks, candidate upgrades, PRs created, issues filed, and escalation notes.
|
||||
- `src/test_logs/TEST_LOG_<timestamp>.md` — commands executed and test outputs for each attempted upgrade.
|
||||
- PRs for safe upgrades and issues for risky/major/security upgrades.
|
||||
|
||||
@@ -177,7 +177,7 @@ PR & ISSUE TEMPLATE GUIDANCE
|
||||
- PR body:
|
||||
- Why: security/bug/maintenance reason.
|
||||
- Commands run + test outputs (snippets), link to `src/test_logs/TEST_LOG_<timestamp>.md`.
|
||||
- Artifacts: attach `artifacts/npm-audit.json` and `artifacts/npm-outdated.json`.
|
||||
- Artifacts: attach `reports/security/npm-audit-*.json` and `reports/security/npm-outdated-*.json`.
|
||||
- Risk assessment, manual QA steps, rollback plan.
|
||||
- Labels: `chore(deps)`, `requires-review` / `security` as appropriate.
|
||||
- Issue: `upgrade-<pkg>` — include reproduction steps, failures, approvals required, and remediation options.
|
||||
@@ -205,15 +205,15 @@ NOTIFICATION & MAINTAINER ESCALATION
|
||||
FIRST-RUN CHECKLIST (run immediately)
|
||||
1. Detect package manager and record versions.
|
||||
2. Run clean install (`npm ci` / `pnpm install` / `yarn install`) and record.
|
||||
3. Run `npm audit --json > artifacts/npm-audit.json` and `npm outdated --json > artifacts/npm-outdated.json`.
|
||||
4. Produce `artifacts/deps-report.md` with triage and a prioritized candidate list for safe bumps.
|
||||
3. Run `npm audit --json > reports/security/npm-audit-YYYY-MM-DD.json` and `npm outdated --json > reports/security/npm-outdated-YYYY-MM-DD.json`.
|
||||
4. Produce `reports/security/deps-report-YYYY-MM-DD.md` with triage and a prioritized candidate list for safe bumps.
|
||||
5. Attempt the first small safe bump(s) (follow the safe upgrade flow) and open PR(s) or issue(s) accordingly.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
FINAL NOTE
|
||||
Run this daily as a scheduled job or as part of CI cadence. Be conservative around cryptography, protocol, and native binary updates — always stop and open an issue for human review. Keep all actions reproducible: include commands, environment, Node & package manager versions, timestamps, and test logs.
|
||||
|
||||
Begin now: detect the package manager, run a fresh clean install, produce `artifacts/npm-audit.json` and `artifacts/npm-outdated.json`, and draft `artifacts/deps-report.md` with the initial triage and any immediate `P0` items.
|
||||
Begin now: detect the package manager, run a fresh clean install, produce `reports/security/npm-audit-YYYY-MM-DD.json` and `reports/security/npm-outdated-YYYY-MM-DD.json`, and draft `reports/security/deps-report-YYYY-MM-DD.md` with the initial triage and any immediate `P0` items.
|
||||
|
||||
FAILURE MODES
|
||||
- If preconditions are not met, stop.
|
||||
|
||||
@@ -120,7 +120,7 @@ WORKFLOW
|
||||
- line number (when available)
|
||||
- snippet (1–2 lines max)
|
||||
- category
|
||||
- Produce a remediation report (see “Reporting format required”).
|
||||
- Produce a remediation report at `reports/design-system/design-system-report-YYYY-MM-DD.md` (and PR/Issue body).
|
||||
|
||||
4) Auto-fix safe, trivial cases (only if policy allows)
|
||||
Allowed examples (only if verified as correct in repo policy):
|
||||
@@ -151,7 +151,7 @@ WORKFLOW
|
||||
`chore(ui): design system audit autofixes (agent)`
|
||||
- Open a PR including the remediation report and commands run.
|
||||
- If no safe auto-fixes were possible, or remaining violations need manual work:
|
||||
- Open a GitHub issue with the remediation report, suggested owners, and
|
||||
- Open a GitHub issue with the remediation report (linked or inline), suggested owners, and
|
||||
next steps.
|
||||
|
||||
6) Verification
|
||||
@@ -232,4 +232,4 @@ Partial success:
|
||||
- Issue documenting remaining violations with owners + next steps
|
||||
|
||||
Always:
|
||||
- A concise remediation report in the PR/issue following the required format
|
||||
- A concise remediation report at `reports/design-system/design-system-report-YYYY-MM-DD.md` (and in the PR/issue following the required format)
|
||||
@@ -18,7 +18,7 @@ This document is your operating manual. Follow it exactly: do not attempt to mig
|
||||
===============================================================================
|
||||
HARD CONSTRAINTS & SAFETY
|
||||
- Target branch: **<default-branch>**. Do not modify other branches.
|
||||
- Migrate **only ONE** file per PR. Choose a single file from `node torch/scripts/check-innerhtml.mjs --report`.
|
||||
- Migrate **only ONE** file per PR. Choose a single file from `node scripts/check-innerhtml.mjs --report`.
|
||||
- Do **not** change semantics. Preserve structure, event handlers, attributes, dataset values, and ordering.
|
||||
- **Never** output user-supplied data as raw HTML. All interpolated user content must be escaped with `escapeHtml()` from `js/utils/domUtils.js`.
|
||||
- Do **not** introduce new third-party libraries unless approved.
|
||||
@@ -31,7 +31,7 @@ REPO PREP (create/update these artifacts)
|
||||
- `src/todo/TODO_<timestamp>.md` — checklist of assignments in the file and status.
|
||||
- `src/decisions/DECISIONS_<timestamp>.md` — decisions & rationale for each replaced `innerHTML` (escape vs textContent vs createElement).
|
||||
- `src/test_logs/TEST_LOG_<timestamp>.md` — commands executed and outputs (lint/tests/manual checks).
|
||||
- `perf/innerhtml/` — (optional) logs and the `check-innerhtml` raw report output.
|
||||
- `reports/audit/` — (optional) logs and the `check-innerhtml` raw report output.
|
||||
|
||||
Commit these artifacts in the PR branch so reviewers can reproduce.
|
||||
|
||||
@@ -43,7 +43,7 @@ WORKFLOW — top-level steps (one file only)
|
||||
- `node -v` and `npm -v` recorded in `src/context/CONTEXT_<timestamp>.md`
|
||||
- Run the innerHTML report:
|
||||
```
|
||||
node torch/scripts/check-innerhtml.mjs --report | tee perf/innerhtml/raw-report-$(date +%F).log
|
||||
node scripts/check-innerhtml.mjs --report | tee reports/audit/raw-report-$(date +%F).log
|
||||
```
|
||||
- Inspect the report and **choose exactly one file** to migrate. Prefer:
|
||||
- Files labeled `RISKY` in the audit (user data without escaping).
|
||||
@@ -114,9 +114,9 @@ WORKFLOW — top-level steps (one file only)
|
||||
7. **Update baseline**
|
||||
- After verifying, update the baseline counts so the `check-innerhtml` script knows this file was addressed:
|
||||
```
|
||||
node torch/scripts/check-innerhtml.mjs --update
|
||||
node scripts/check-innerhtml.mjs --update
|
||||
```
|
||||
- The script prints a new `BASELINE` object. **Copy** the updated `BASELINE` object into `torch/scripts/check-innerhtml.mjs` (replace existing `BASELINE`).
|
||||
- The script prints a new `BASELINE` object. **Copy** the updated `BASELINE` object into `scripts/check-innerhtml.mjs` (replace existing `BASELINE`).
|
||||
- Commit the updated script with a note in `src/decisions/DECISIONS_<timestamp>.md` explaining the baseline change.
|
||||
|
||||
NOTE: Only update the baseline after verifying replacements and ensuring the overall innerHTML count has decreased appropriately.
|
||||
@@ -136,7 +136,7 @@ WORKFLOW — top-level steps (one file only)
|
||||
- files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`
|
||||
- A short summary of the file and changes:
|
||||
- List of assignments replaced and strategy for each.
|
||||
- Before/after innerHTML counts (run `node torch/scripts/check-innerhtml.mjs --report` before and after and paste results).
|
||||
- Before/after innerHTML counts (run `node scripts/check-innerhtml.mjs --report` before and after and paste results).
|
||||
- Lint and unit test results.
|
||||
- Manual QA steps and results.
|
||||
- Highlight any remaining `innerHTML` assignments intentionally left (with rationale).
|
||||
@@ -240,8 +240,8 @@ ACCEPTANCE CRITERIA (before merging)
|
||||
|
||||
* The chosen file’s `innerHTML` assignments are replaced or documented; user data is escaped or DOM-built.
|
||||
* Lint passes and unit tests pass locally and in CI.
|
||||
* `torch/scripts/check-innerhtml.mjs --report` shows reduced count for the chosen file.
|
||||
* Updated `BASELINE` object copied into `torch/scripts/check-innerhtml.mjs`.
|
||||
* `scripts/check-innerhtml.mjs --report` shows reduced count for the chosen file.
|
||||
* Updated `BASELINE` object copied into `scripts/check-innerhtml.mjs`.
|
||||
* PR contains files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`, summary of before/after, and QA steps.
|
||||
* Commit message prefix is `[security]`.
|
||||
|
||||
@@ -263,7 +263,7 @@ FINAL NOTES & Etiquette
|
||||
|
||||
Begin now:
|
||||
|
||||
1. Run `node torch/scripts/check-innerhtml.mjs --report`
|
||||
1. Run `node scripts/check-innerhtml.mjs --report`
|
||||
2. Pick **ONE** file (RISKY/high-count preferred).
|
||||
3. Follow the workflow above and open a `[security]` PR to `<default-branch>` when ready.
|
||||
|
||||
|
||||
@@ -98,8 +98,8 @@ DAILY WORKFLOW (mandatory)
|
||||
- release channel expectations)
|
||||
|
||||
1) Create run artifacts (only if repo conventions allow committing artifacts)
|
||||
- Prefer a dated folder:
|
||||
- `artifacts/known-issues/YYYY-MM-DD/`
|
||||
- Use report folder:
|
||||
- `reports/known-issues/`
|
||||
- If the repo does not commit artifacts, keep the report in the PR body or in
|
||||
`docs/` per repo conventions.
|
||||
|
||||
@@ -176,7 +176,7 @@ DAILY WORKFLOW (mandatory)
|
||||
- Remove dead links, fix paths, and clarify ambiguous instructions.
|
||||
|
||||
8) Daily report (required)
|
||||
Produce a short report (PR body or artifact) containing:
|
||||
Produce a short report at `reports/known-issues/known-issues-report-YYYY-MM-DD.md` (and optional PR body) containing:
|
||||
- Headline: ✓ all verified / ⚠️ active issues remain
|
||||
- Table/list: each entry → status (Resolved/Active/Unknown), last checked date
|
||||
- PRs opened
|
||||
@@ -221,4 +221,4 @@ Daily:
|
||||
- 0–1 small PR fixing a trivial, safe known issue (or none if nothing safe)
|
||||
- Updated `KNOWN_ISSUES.md` with last-checked notes and links
|
||||
- 0–N GitHub issues for medium/large/risky items
|
||||
- A short daily remediation report (artifact or PR body)
|
||||
- A short daily remediation report at `reports/known-issues/known-issues-report-YYYY-MM-DD.md`
|
||||
@@ -28,12 +28,12 @@ policy and open an issue rather than improvising.
|
||||
SCOPE
|
||||
|
||||
In scope:
|
||||
- Creating/maintaining a load test script under `torch/scripts/agent/` (only if that
|
||||
- Creating/maintaining a load test script under `scripts/agent/` (only if that
|
||||
directory exists in repo; verify first).
|
||||
- Simulating many clients connecting to a relay and publishing configurable
|
||||
mixes of events at configurable rates.
|
||||
- Measuring latency, throughput, error rates, and basic resource usage.
|
||||
- Producing machine-readable reports under `artifacts/` (only if repo
|
||||
- Producing machine-readable reports under `reports/load-test/` (only if repo
|
||||
conventions permit committing artifacts).
|
||||
- Opening issues for bottlenecks or unsafe behaviors.
|
||||
|
||||
@@ -98,12 +98,12 @@ WORKFLOW
|
||||
- existing event schema builders (e.g., integration event schema utilities)
|
||||
- existing relay interaction clients/helpers
|
||||
- docs describing playback fallback and relay write safety
|
||||
- Confirm whether `torch/scripts/agent/` and `artifacts/` are established patterns.
|
||||
- Confirm whether `scripts/agent/` and `reports/load-test/` are established patterns.
|
||||
If not, do not invent directory conventions—open an issue proposing layout.
|
||||
|
||||
2) Implement load harness (minimal, configurable)
|
||||
Target file (only if verified appropriate in repo):
|
||||
- `torch/scripts/agent/load-test.mjs`
|
||||
- `scripts/agent/load-test.mjs`
|
||||
|
||||
The harness must support configuration via env vars and/or CLI flags:
|
||||
- `RELAY_URL` (required)
|
||||
@@ -143,8 +143,9 @@ WORKFLOW
|
||||
- resource snapshots at intervals
|
||||
|
||||
4) Report generation
|
||||
Output file (follow repo conventions; if allowed):
|
||||
- `artifacts/load-report-YYYYMMDD.json`
|
||||
Output files (follow repo conventions; if allowed):
|
||||
- `reports/load-test/load-report-YYYY-MM-DD.json`
|
||||
- `reports/load-test/load-test-report-YYYY-MM-DD.md` (human summary)
|
||||
|
||||
The report should include:
|
||||
- Config used (relay URL redacted to hostname if needed, N, duration, rate, mix)
|
||||
@@ -195,7 +196,7 @@ Open an issue instead of proceeding when:
|
||||
───────────────────────────────────────────────────────────────────────────────
|
||||
OUTPUTS PER RUN
|
||||
|
||||
- `torch/scripts/agent/load-test.mjs` (or repo-approved equivalent)
|
||||
- A load report (JSON) produced locally (and committed only if repo allows)
|
||||
- `scripts/agent/load-test.mjs` (or repo-approved equivalent)
|
||||
- A load report (JSON + Markdown) produced locally (and committed only if repo allows)
|
||||
- A prioritized list of bottlenecks tied to measured evidence
|
||||
- 0–N issues for non-trivial bottlenecks, especially security-sensitive ones
|
||||
@@ -66,7 +66,7 @@ DAILY WORKFLOW (run each day)
|
||||
2. **Search & Inventory**
|
||||
- Run search patterns (see below).
|
||||
- For each hit record: file, function, lines, trigger (load/login/visibility/user), frequency, main-thread? network intensity, user-visible impact.
|
||||
- Produce `perf/hits-YYYY-MM-DD.json`.
|
||||
- Produce `reports/performance/hits-YYYY-MM-DD.json`.
|
||||
|
||||
3. **Prioritize**
|
||||
- P0: Blocks login/auth or decrypts UX-critical lists.
|
||||
@@ -83,7 +83,7 @@ DAILY WORKFLOW (run each day)
|
||||
- If upload/ingest code or `/content` is touched, run the docs audit workflow (inventory → verify → update → validate → deliver). Always leave `/content` runnable/precise.
|
||||
|
||||
6. **Report**
|
||||
- Produce `daily-perf-report-YYYY-MM-DD.md` summarizing findings, PRs/issues, metrics, and blockers.
|
||||
- Produce `reports/performance/daily-perf-report-YYYY-MM-DD.md` summarizing findings, PRs/issues, metrics, and blockers.
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
TECHNICAL HEURISTICS & PATTERNS (what to look for & how to fix)
|
||||
@@ -176,10 +176,10 @@ ESCALATION & RISK POLICY
|
||||
-------------------------------------------------------------------------------
|
||||
FIRST-RUN CHECKLIST (execute now)
|
||||
1. Commit files in `src/context/`, `src/todo/`, `src/decisions/`, `src/test_logs/`, `INITIAL_BASELINE.md`.
|
||||
2. Run Search Patterns across repo and save `perf/hits-YYYY-MM-DD.json`.
|
||||
2. Run Search Patterns across repo and save `reports/performance/hits-YYYY-MM-DD.json`.
|
||||
3. Prioritize P0 findings, open at most 1–3 small PRs (bounded concurrency, worker queue limits, visibility gating).
|
||||
4. Audit `/content` upload pages: inventory claims → map to code → verify → open docs PR or issue.
|
||||
5. Produce `daily-perf-report-YYYY-MM-DD.md` summarizing results and PR links.
|
||||
5. Produce `reports/performance/daily-perf-report-YYYY-MM-DD.md` summarizing results and PR links.
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
@@ -189,7 +189,7 @@ FAILURE MODES
|
||||
- If specific resources (files, URLs) are unavailable, log the error and skip.
|
||||
|
||||
OUTPUTS (per run)
|
||||
- `daily-perf-report-YYYY-MM-DD.md` with:
|
||||
- `reports/performance/daily-perf-report-YYYY-MM-DD.md` with:
|
||||
- Summary line.
|
||||
- P0/P1/P2 findings (file/func/lines/impact/proposed fix/PR/Issue links).
|
||||
- Metrics (login-time, queue sizes).
|
||||
@@ -210,4 +210,4 @@ BEHAVIORAL GUARDRAILS & QUALITY BAR
|
||||
FINAL NOTE
|
||||
You are a helper, not a helicopter. Make safe, measurable progress daily. Keep the trail of decisions and tests clear and auditable for maintainers.
|
||||
|
||||
Begin now: run the Search Patterns, collect hits, and draft the first `daily-perf-report-YYYY-MM-DD.md`.
|
||||
Begin now: run the Search Patterns, collect hits, and draft the first `reports/performance/daily-perf-report-YYYY-MM-DD.md`.
|
||||
@@ -37,12 +37,12 @@ Mission: identify external protocol/spec dependencies used by this codebase, map
|
||||
3. Update `PROTOCOL_INVENTORY.md` with concrete code pointers.
|
||||
3. Validate behavior with targeted unit/integration tests.
|
||||
4. Classify gaps and propose minimal remediations.
|
||||
5. Publish a dated report: `protocol-report-YYYY-MM-DD.md`.
|
||||
5. Publish a dated report: `reports/protocol/protocol-report-YYYY-MM-DD.md`.
|
||||
|
||||
## Deliverables
|
||||
|
||||
- `PROTOCOL_INVENTORY.md`
|
||||
- `protocol-report-YYYY-MM-DD.md`
|
||||
- `reports/protocol/protocol-report-YYYY-MM-DD.md`
|
||||
- Test additions or issue links for identified gaps
|
||||
|
||||
Keep outputs practical, evidence-based, and easy for maintainers to act on.
|
||||
|
||||
@@ -19,4 +19,4 @@ MISSION:
|
||||
|
||||
STOP CONDITIONS:
|
||||
- If intended behavior is unclear, do not guess and do not weaken tests.
|
||||
Produce a “Needs Spec Clarification” report + propose candidate scenarios.
|
||||
Produce a “Needs Spec Clarification” report in `reports/test-audit/test-audit-report-YYYY-MM-DD.md` + propose candidate scenarios.
|
||||
@@ -93,8 +93,8 @@ WORKFLOW
|
||||
- inspect repo for the correct serve path and update README or open an issue.
|
||||
|
||||
3) Implement a Playwright capture script
|
||||
- Create (only if `torch/scripts/agent/` exists or is repo convention; verify first):
|
||||
- `torch/scripts/agent/debug_frontend.py`
|
||||
- Create (only if `scripts/agent/` exists or is repo convention; verify first):
|
||||
- `scripts/agent/debug_frontend.py`
|
||||
- The script must:
|
||||
- open `http://localhost:8000`
|
||||
- collect and print:
|
||||
@@ -107,12 +107,12 @@ WORKFLOW
|
||||
|
||||
Logging/output:
|
||||
- Save output to:
|
||||
- `artifacts/debug_frontend.log`
|
||||
- `reports/debug/debug_frontend.log`
|
||||
(only commit artifacts if repo conventions allow; otherwise attach excerpt in PR)
|
||||
|
||||
4) Run capture and identify the first blocker
|
||||
- Run:
|
||||
- `python3 torch/scripts/agent/debug_frontend.py | tee artifacts/debug_frontend.log`
|
||||
- `python3 scripts/agent/debug_frontend.py | tee reports/debug/debug_frontend.log`
|
||||
- Diagnose the earliest blocking error:
|
||||
- first `pageerror` OR first `console:error` that prevents initialization
|
||||
- Extract:
|
||||
@@ -136,7 +136,7 @@ WORKFLOW
|
||||
- `npm run lint`
|
||||
- `npm run test:unit`
|
||||
- Re-run:
|
||||
- `python3 torch/scripts/agent/debug_frontend.py | tee artifacts/debug_frontend.log`
|
||||
- `python3 scripts/agent/debug_frontend.py | tee reports/debug/debug_frontend.log`
|
||||
- Confirm:
|
||||
- the original blocking error is gone
|
||||
- no new initialization-blocking error was introduced
|
||||
@@ -149,7 +149,7 @@ WORKFLOW
|
||||
- Commit message (adjust to policy):
|
||||
- `fix(ai): frontend console error — <short> (agent)`
|
||||
- PR body must include:
|
||||
- the top error excerpt(s) from `artifacts/debug_frontend.log`
|
||||
- the top error excerpt(s) from `reports/debug/debug_frontend.log`
|
||||
- root cause summary (file/function)
|
||||
- files changed
|
||||
- commands run + results
|
||||
@@ -172,7 +172,7 @@ Issue must include:
|
||||
───────────────────────────────────────────────────────────────────────────────
|
||||
OUTPUTS PER RUN
|
||||
|
||||
- `torch/scripts/agent/debug_frontend.py` (if repo conventions allow)
|
||||
- `artifacts/debug_frontend.log` (committed only if repo norms allow; otherwise excerpt in PR)
|
||||
- `scripts/agent/debug_frontend.py` (if repo conventions allow)
|
||||
- `reports/debug/debug_frontend.log` (committed only if repo norms allow; otherwise excerpt in PR)
|
||||
- 0–1 PR fixing the first initialization-blocking console/page error
|
||||
- 0–N issues for unsafe/ambiguous follow-ups
|
||||
@@ -90,9 +90,9 @@ WORKFLOW
|
||||
1) Preflight
|
||||
- Read `AGENTS.md` and `CLAUDE.md` for:
|
||||
- security constraints (crypto, key handling, relay policy)
|
||||
- conventions for `torch/scripts/agent/`, `examples/`, and artifacts
|
||||
- conventions for `scripts/agent/`, `examples/`, and artifacts
|
||||
- Confirm whether these directories exist and are used in repo:
|
||||
- `torch/scripts/agent/`
|
||||
- `scripts/agent/`
|
||||
- `examples/reproducers/`
|
||||
- `artifacts/`
|
||||
If not present, do not invent structure—open an issue proposing where fuzz
|
||||
@@ -112,8 +112,8 @@ WORKFLOW
|
||||
- unhandled exception from invalid inputs
|
||||
|
||||
3) Implement the fuzz harness
|
||||
File naming (only if `torch/scripts/agent/` is valid in repo):
|
||||
- `torch/scripts/agent/fuzz-<target>.mjs`
|
||||
File naming (only if `scripts/agent/` is valid in repo):
|
||||
- `scripts/agent/fuzz-<target>.mjs`
|
||||
|
||||
Harness requirements:
|
||||
- Inputs: randomized cases covering:
|
||||
@@ -154,7 +154,7 @@ WORKFLOW
|
||||
|
||||
6) Reporting
|
||||
Create a report file (commit only if repo conventions allow):
|
||||
- `artifacts/fuzz-report-<target>-YYYYMMDD.json`
|
||||
- `reports/security/fuzz-report-<target>-YYYY-MM-DD.json`
|
||||
|
||||
Report must include:
|
||||
- target name + code pointers (file/function)
|
||||
@@ -215,8 +215,8 @@ Issue must include:
|
||||
───────────────────────────────────────────────────────────────────────────────
|
||||
OUTPUTS PER RUN
|
||||
|
||||
- `torch/scripts/agent/fuzz-<target>.mjs` (if repo conventions allow)
|
||||
- `scripts/agent/fuzz-<target>.mjs` (if repo conventions allow)
|
||||
- 0–N reproducers under `examples/reproducers/` (if allowed)
|
||||
- `artifacts/fuzz-report-<target>-YYYYMMDD.json` (committed only if allowed)
|
||||
- `reports/security/fuzz-report-<target>-YYYY-MM-DD.json` (committed only if allowed)
|
||||
- 0–1 PR (harness + repros, optionally small safe guards/tests)
|
||||
- 0–N issues for sensitive or non-trivial fixes
|
||||
@@ -27,7 +27,7 @@ If a higher-level policy or doc conflicts with this prompt, follow the higher-le
|
||||
SCOPE
|
||||
|
||||
In scope:
|
||||
- Implement a small smoke-test harness at `torch/scripts/agent/smoke-test.mjs` (or `.ts`/`.py` if repo prefers), exercising:
|
||||
- Implement a small smoke-test harness at `scripts/agent/smoke-test.mjs` (or `.ts`/`.py` if repo prefers), exercising:
|
||||
- app start/serve (per README)
|
||||
- headless browser/client run (Playwright preferred if available)
|
||||
- sharedClient connection to a local/test relay
|
||||
@@ -35,8 +35,8 @@ In scope:
|
||||
- publish `VIDEO_POST` or `VIEW_EVENT` and verify read-back
|
||||
- send encrypted DM and verify decryption with `js/dmDecryptor.js`
|
||||
- Produce artifacts:
|
||||
- `artifacts/smoke-YYYYMMDD.log` (human-readable)
|
||||
- `artifacts/smoke-YYYYMMDD.json` (structured summary with timestamps)
|
||||
- `reports/smoke-test/smoke-YYYY-MM-DD.log` (human-readable)
|
||||
- `reports/smoke-test/smoke-YYYY-MM-DD.json` (structured summary with timestamps)
|
||||
- screenshots for UI flows where applicable
|
||||
- Provide run instructions for local and CI use, and open a PR with the test and docs.
|
||||
|
||||
@@ -83,7 +83,7 @@ PREPARATION (preflight)
|
||||
WORKFLOW (implementation & run steps)
|
||||
|
||||
1) Create harness file
|
||||
- Path: `torch/scripts/agent/smoke-test.mjs`
|
||||
- Path: `scripts/agent/smoke-test.mjs`
|
||||
- Language: follow repo conventions (Node/Playwright preferred). If Playwright is used, the harness should run headless.
|
||||
- CLI/ENV options:
|
||||
- `--relays` or `RELAY_URLS` (csv) — **required**
|
||||
@@ -91,7 +91,7 @@ WORKFLOW (implementation & run steps)
|
||||
- `--dry-run` — do everything but network/publish
|
||||
- `--timeout` — per-step timeout (default 30s)
|
||||
- `--burst` — per-publish burst (default 1)
|
||||
- `--out` — artifacts output dir (default `artifacts/`)
|
||||
- `--out` — artifacts output dir (default `reports/smoke-test/`)
|
||||
- `--confirm-public` — explicit consent for limited public-relay runs
|
||||
|
||||
2) Start local server
|
||||
@@ -133,9 +133,9 @@ WORKFLOW (implementation & run steps)
|
||||
8) Logging & artifacts
|
||||
- Log each step with timestamps and outcomes (success/fail)
|
||||
- Save:
|
||||
- `artifacts/smoke-YYYYMMDD.log` (text)
|
||||
- `artifacts/smoke-YYYYMMDD.json` (summary: events published, ids, relays, timestamps, verification results)
|
||||
- `artifacts/smoke-YYYYMMDD-screenshots/*` (Playwright screenshots on failure and optionally success)
|
||||
- `reports/smoke-test/smoke-YYYY-MM-DD.log` (text)
|
||||
- `reports/smoke-test/smoke-YYYY-MM-DD.json` (summary: events published, ids, relays, timestamps, verification results)
|
||||
- `reports/smoke-test/screenshots/*` (Playwright screenshots on failure and optionally success)
|
||||
- If failures occur, capture:
|
||||
- console logs (browser and client)
|
||||
- stack traces
|
||||
@@ -162,12 +162,12 @@ EXAMPLE RUN
|
||||
|
||||
> Prompt authors: follow the canonical artifact paths in [Scheduler Flow → Canonical artifact paths](../scheduler-flow.md#canonical-artifact-paths).
|
||||
|
||||
RELAY_URLS="ws://localhost:8080" node torch/scripts/agent/smoke-test.mjs \
|
||||
--serve=npx --timeout=30 --burst=1 --out=artifacts/
|
||||
RELAY_URLS="ws://localhost:8080" node scripts/agent/smoke-test.mjs \
|
||||
--serve=npx --timeout=30 --burst=1 --out=reports/smoke-test/
|
||||
|
||||
# public relays only with explicit confirmation and extreme throttle
|
||||
RELAY_URLS="wss://relay.example" node torch/scripts/agent/smoke-test.mjs \
|
||||
--confirm-public --burst=1 --timeout=30 --out=artifacts/
|
||||
RELAY_URLS="wss://relay.example" node scripts/agent/smoke-test.mjs \
|
||||
--confirm-public --burst=1 --timeout=30 --out=reports/smoke-test/
|
||||
```
|
||||
|
||||
───────────────────────────────────────────────────────────────────────────────
|
||||
@@ -175,7 +175,7 @@ FAILURE MODES
|
||||
|
||||
- **Startup Failure:** If the application fails to start (e.g., port in use, build error), log the stderr output and abort.
|
||||
- **Relay Connection Failure:** If the client cannot connect to the specified relay, retry with backoff. If it still fails, abort and report "Relay Unreachable".
|
||||
- **Element Not Found (UI):** If Playwright cannot find a required UI element (e.g., login button, post input), capture a screenshot and dump the DOM state to `artifacts/`. Fail the test step.
|
||||
- **Element Not Found (UI):** If Playwright cannot find a required UI element (e.g., login button, post input), capture a screenshot and dump the DOM state to `reports/smoke-test/`. Fail the test step.
|
||||
- **Verification Failure:** If the read-back event does not match the published event (content mismatch, signature error), log the diff and fail the test.
|
||||
- **Timeout:** If any step exceeds the configured timeout, abort the run and capture a screenshot/log of the current state.
|
||||
|
||||
@@ -197,9 +197,9 @@ PR & COMMIT CONVENTIONS
|
||||
OUTPUTS PER RUN
|
||||
|
||||
- **Artifacts:**
|
||||
- `artifacts/smoke-YYYYMMDD.log`: Detailed execution log.
|
||||
- `artifacts/smoke-YYYYMMDD.json`: Structured summary of the test run (steps, duration, result).
|
||||
- `artifacts/smoke-YYYYMMDD-screenshots/`: Directory containing screenshots of failures (and success states if configured).
|
||||
- `reports/smoke-test/smoke-YYYY-MM-DD.log`: Detailed execution log.
|
||||
- `reports/smoke-test/smoke-YYYY-MM-DD.json`: Structured summary of the test run (steps, duration, result).
|
||||
- `reports/smoke-test/screenshots/`: Directory containing screenshots of failures (and success states if configured).
|
||||
- **Pull Request:** A PR containing the new or updated smoke test script and any necessary documentation changes.
|
||||
- **Console Output:** Real-time progress updates and a final summary (Pass/Fail) printed to stdout.
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ In scope:
|
||||
- Collect CI/test run failures, smoke-test artifacts, and agent job errors (only from sources that have opted-in).
|
||||
- Sanitize logs to remove PII, secrets, IPs, keys, and any sensitive identifiers.
|
||||
- Fingerprint/cluster errors by normalized stack trace, count occurrences, compute recency and severity.
|
||||
- Produce `artifacts/error-aggregates-YYYYMMDD.json` and `ai/reports/telemetry-YYYYMMDD.md`.
|
||||
- Produce `reports/telemetry/error-aggregates-YYYY-MM-DD.json` and `reports/telemetry/telemetry-report-YYYY-MM-DD.md`.
|
||||
- Suggest owners (via CODEOWNERS / module owners when available) and prioritized next actions.
|
||||
|
||||
Out of scope:
|
||||
@@ -134,7 +134,7 @@ WORKFLOW (detailed)
|
||||
- Keep only sanitized sample traces (max length), counts and metadata.
|
||||
|
||||
5) Generate artifacts
|
||||
- `artifacts/error-aggregates-YYYYMMDD.json` (structured):
|
||||
- `reports/telemetry/error-aggregates-YYYY-MM-DD.json` (structured):
|
||||
```json
|
||||
{
|
||||
"generated_at": "...",
|
||||
@@ -154,7 +154,7 @@ WORKFLOW (detailed)
|
||||
]
|
||||
}
|
||||
```
|
||||
- `ai/reports/telemetry-YYYYMMDD.md` (human summary):
|
||||
- `reports/telemetry/telemetry-report-YYYY-MM-DD.md` (human summary):
|
||||
- Headline: total errors, total groups
|
||||
- Top-10 groups table: fingerprint, severity, count, owner, sample sanitized trace
|
||||
- Prioritized Top-5 recommended human actions
|
||||
@@ -170,11 +170,11 @@ WORKFLOW (detailed)
|
||||
|
||||
7) Commit / PR (if allowed)
|
||||
- If committing artifacts is permitted:
|
||||
- Branch: `ai/telemetry-YYYYMMDD`
|
||||
- Branch: `ai/telemetry-YYYY-MM-DD`
|
||||
- Commit message: `chore(ai): telemetry error aggregates (YYYY-MM-DD)`
|
||||
- PR title: `docs(ai): telemetry report — YYYY-MM-DD`
|
||||
- PR body: include the telemetry report summary and privacy audit
|
||||
- If not permitted, attach `artifacts/error-aggregates-*.json` and the report to the PR body.
|
||||
- If not permitted, attach `reports/telemetry/error-aggregates-*.json` and the report to the PR body.
|
||||
|
||||
───────────────────────────────────────────────────────────────────────────────
|
||||
FAILURE MODES & GUARDRAILS (when to stop)
|
||||
@@ -189,7 +189,7 @@ FAILURE MODES & GUARDRAILS (when to stop)
|
||||
───────────────────────────────────────────────────────────────────────────────
|
||||
REPORT FORMAT (required)
|
||||
|
||||
`ai/reports/telemetry-YYYYMMDD.md` should include:
|
||||
`reports/telemetry/telemetry-report-YYYY-MM-DD.md` should include:
|
||||
|
||||
1. **Headline**
|
||||
- `Total sanitized errors: N — Groups: M — Top severity: S`
|
||||
@@ -224,10 +224,10 @@ PR & COMMIT CONVENTIONS
|
||||
─────────────────────────────────────────────────────────────────────────────
|
||||
OUTPUTS PER RUN
|
||||
|
||||
- `artifacts/error-aggregates-YYYYMMDD.json` (sanitized, structured)
|
||||
- `ai/reports/telemetry-YYYYMMDD.md` (human-ready)
|
||||
- `reports/telemetry/error-aggregates-YYYY-MM-DD.json` (sanitized, structured)
|
||||
- `reports/telemetry/telemetry-report-YYYY-MM-DD.md` (human-ready)
|
||||
- 0–N issues suggested for top errors (only if policy allows automation)
|
||||
- If permitted: a PR `ai/telemetry-YYYYMMDD` with artifacts and privacy audit
|
||||
- If permitted: a PR `ai/telemetry-YYYY-MM-DD` with artifacts and privacy audit
|
||||
|
||||
─────────────────────────────────────────────────────────────────────────────
|
||||
BEGIN
|
||||
@@ -236,7 +236,7 @@ BEGIN
|
||||
2. Discover opt-in artifacts (CI runs with `TELEMETRY=1`, `artifacts/telemetry-optin/`, agent-run logs).
|
||||
3. Sanitize each artifact (apply PII/key/IP/email redaction and log an audit of replacements).
|
||||
4. Normalize & fingerprint stack traces; aggregate counts and compute severity.
|
||||
5. Produce `artifacts/error-aggregates-YYYYMMDD.json` and `ai/reports/telemetry-YYYYMMDD.md`.
|
||||
6. If artifacts are permitted to be committed, open `ai/telemetry-YYYYMMDD` PR with the report. Otherwise, attach in PR body and open issue(s) for human triage.
|
||||
5. Produce `reports/telemetry/error-aggregates-YYYY-MM-DD.json` and `reports/telemetry/telemetry-report-YYYY-MM-DD.md`.
|
||||
6. If artifacts are permitted to be committed, open `ai/telemetry-YYYY-MM-DD` PR with the report. Otherwise, attach in PR body and open issue(s) for human triage.
|
||||
|
||||
**Privacy-first rule:** If you cannot guarantee all PII/keys are redacted from every artifact, stop and request maintainer review — do not commit or publish raw logs.
|
||||
@@ -71,7 +71,7 @@ WORKFLOW
|
||||
- branch naming conventions (e.g., `ai/*`)
|
||||
- label conventions (e.g., `requires-security-review`, `requires-protocol-review`)
|
||||
- Verify whether the target folder exists:
|
||||
- `ai/reports/`
|
||||
- `reports/weekly-synthesis/`
|
||||
If it does not exist, do not invent new structure without checking policy:
|
||||
- Prefer placing the report where existing reports live, or open an issue
|
||||
proposing the location.
|
||||
@@ -125,7 +125,7 @@ Optional method (use curl to query the GitHub API):
|
||||
|
||||
6) Write the report file
|
||||
File name (if allowed by repo conventions):
|
||||
- `ai/reports/weekly-YYYYMMDD.md`
|
||||
- `reports/weekly-synthesis/weekly-report-YYYY-MM-DD.md`
|
||||
|
||||
Keep the report compact and scannable.
|
||||
|
||||
@@ -193,5 +193,5 @@ If report directory doesn’t exist and policy is unclear:
|
||||
───────────────────────────────────────────────────────────────────────────────
|
||||
OUTPUTS PER RUN
|
||||
|
||||
- `ai/reports/weekly-YYYYMMDD.md` (or repo-approved equivalent)
|
||||
- `reports/weekly-synthesis/weekly-report-YYYY-MM-DD.md` (or repo-approved equivalent)
|
||||
- One commit containing only the report (if committing is allowed by policy)
|
||||
Reference in new issue
Block a user