mirror of
https://github.com/PR0M3TH3AN/TORCH.git
synced 2026-10-27 07:51:52 +00:00
Make CLAUDE.md a thin wrapper over canonical AGENTS.md; untrack agent memory scratch
Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
1 parent
9325f91582
commit
331cd82b3d
4 files changed
+10
-84
No files matched your search
@@ -31,3 +31,8 @@ benchmark-baseline.txt
|
||||
coverage/
|
||||
playwright-report/
|
||||
test-results/
|
||||
|
||||
# Agent runtime state (local only)
|
||||
memory-update.json
|
||||
memory-update.md
|
||||
.claude/
|
||||
@@ -20,6 +20,7 @@ We treat tests as **behavioral specifications** expressed as **scenarios**:
|
||||
2) **Never change expected behavior to match buggy output** (“rubber-stamp the snapshot”, “update golden”, “adjust assertion”) unless it is a **spec correction** (see below).
|
||||
3) Do not “fix” flaky tests with retries, sleeps, timeouts, or looser assertions. **Remove nondeterminism** instead (control time, randomness, IO, network).
|
||||
4) Prefer black-box assertions at system boundaries; avoid tests that merely mirror internal logic.
|
||||
5) Do not “hard-code to the test” (single fixtures, single input paths, brittle constants), and do not mock so heavily that tests can pass with a broken system.
|
||||
|
||||
### Definitions
|
||||
|
||||
|
||||
@@ -1,82 +1,5 @@
|
||||
## Testing & Validation: Scenario-First, Cheat-Resistant (Dark Factory Rules)
|
||||
# Claude instructions
|
||||
|
||||
You are operating inside a “Dark Factory” workflow where **validation replaces human code review**. Passing tests is only meaningful if tests are truthful.
|
||||
|
||||
### Prime directive
|
||||
|
||||
**Never alter a test merely to make it pass.**
|
||||
If you change tests to match buggy behavior, you are destroying the only feedback signal and producing code with no real-world value.
|
||||
|
||||
StrongDM describes the exact failure mode we are preventing: agents discovering shortcuts (“return true”) and narrow tests being reward-hacked, leading to a shift from “tests” to scenario-based validation. :contentReference[oaicite:3]{index=3}
|
||||
|
||||
### How to think about tests here
|
||||
|
||||
Tests are not procedural checklists. They are **behavioral specifications**:
|
||||
- Write tests as **scenarios** (Given/When/Then or equivalent).
|
||||
- Assert **externally observable outcomes** at boundaries.
|
||||
- Avoid “testing the implementation” (internal call sequences, private methods, mirrored logic).
|
||||
- Prefer deterministic/harmonic execution: fixed seeds, controlled clocks, hermetic IO, virtualized dependencies.
|
||||
|
||||
### Prohibited behaviors (reward-hacking patterns)
|
||||
|
||||
Do NOT:
|
||||
- weaken assertions, broaden tolerances, or reduce coverage to get green
|
||||
- update snapshots/goldens without proving the change is intended behavior
|
||||
- add retries/sleeps/timeouts to hide flakes
|
||||
- mock so heavily that tests can pass with a broken system
|
||||
- “hard-code to the test” (single fixtures, single input paths, brittle constants)
|
||||
|
||||
### Allowed test changes: Spec Correction Protocol
|
||||
|
||||
You may change test expectations only if the test is wrong relative to intended behavior.
|
||||
|
||||
Required steps:
|
||||
1) Identify the scenario/spec that defines correct behavior.
|
||||
2) Explain why the previous expectation was incorrect.
|
||||
3) Replace it with an equally strict (or stricter) behavioral assertion.
|
||||
4) Provide a **Test Integrity Note** (machine-readable) documenting the change.
|
||||
|
||||
If you cannot justify a change with scenario/spec truth: **do not change the test**. Instead, output a “Needs Spec Clarification” note and propose candidate scenarios.
|
||||
|
||||
### Required output when you touch tests
|
||||
|
||||
Include this YAML block in your PR description (or reference it from a `TEST_INTEGRITY.md` entry):
|
||||
|
||||
```yaml
|
||||
test_integrity_note:
|
||||
change_type: ["new_tests" | "refactor_tests" | "spec_correction" | "flake_fix"]
|
||||
scenarios:
|
||||
- id: SCN-<slug>
|
||||
given: "<preconditions>"
|
||||
when: "<stimulus>"
|
||||
then: "<observable outcomes>"
|
||||
observable_outcomes:
|
||||
- "<boundary/output asserted>"
|
||||
determinism_controls:
|
||||
- "<fixed seed / fake clock / hermetic env / service virtualization>"
|
||||
anti_cheat_rationale:
|
||||
prevents:
|
||||
- "hard-coded return value"
|
||||
- "over-mocking internal logic"
|
||||
- "snapshot rubber-stamping"
|
||||
- "retry/sleep-based flake masking"
|
||||
relaxation:
|
||||
did_relax_any_assertion: false
|
||||
if_true_explain_spec_basis: ""
|
||||
```
|
||||
|
||||
### If external services are involved
|
||||
|
||||
Prefer boundary-level service virtualization (“digital twins” / behavioral clones) so scenario tests remain:
|
||||
|
||||
* high fidelity (realistic edge cases)
|
||||
* deterministic and replayable
|
||||
* not rate-limited / not flaky
|
||||
|
||||
(StrongDM’s “Digital Twin Universe” is the reference pattern.) ([factory.strongdm.ai][2])
|
||||
|
||||
### Clean Repo Standard
|
||||
|
||||
* **No Root Clutter**: Do not create logs, artifacts, or temporary files in the repository root.
|
||||
* **Structured Output**: Use `reports/`, `test_logs/`, `task-logs/`, or `artifacts/` as appropriate.
|
||||
* **Redirect Output**: If a command writes to root, redirect it (e.g., `cmd > test_logs/output.txt`).
|
||||
Read and follow [`AGENTS.md`](AGENTS.md). It is the canonical project guidance. Then read
|
||||
`README.md`, `TORCH.md`, `CONTRIBUTING.md`, `KNOWN_ISSUES.md`, and the relevant overviews
|
||||
under `docs/` before making changes.
|
||||
@@ -1,3 +0,0 @@
|
||||
{
|
||||
"learnings": "Audit completed with no new issues."
|
||||
}
|
||||
Reference in new issue
Block a user