fix(test): isolate install_dir_validation tests

Refactor test/install_dir_validation.test.mjs to use unique temporary directories for each subtest, eliminating shared state and race conditions that caused flakiness.
Update KNOWN_ISSUES.md to mark the issue as resolved.

Co-authored-by: PR0M3TH3AN <[email protected]>
This commit is contained in:
google-labs-jules[bot]andPR0M3TH3AN committed 2026-02-21 02:12:23 +00:00
1 parent 5f79d94754
commit 2430efc256
6 files changed
+168 -78

No files matched your search

+2 -2
View File
@@ -140,10 +140,10 @@ Track only **active, reproducible, unresolved** issues.
- **Last verified:** 2026-02-21
### `cmdInit should validate install directory name` test is flaky
- **Status:** Active
- **Status:** Resolved
- **Area:** Test
- **Symptom:** Test failed 1/5 times during flakiness check.
- **Trigger/Conditions:** Running `node scripts/test-audit/run-flaky-check.mjs`.
- **Workaround:** Retry.
- **Workaround:** None needed.
- **Impact:** CI instability.
- **Last verified:** 2026-02-21
@@ -0,0 +1,12 @@
# Memory Update — ci-health-agent — 2026-02-21
## Key findings
- `test/install_dir_validation.test.mjs` flakiness was caused by shared directory state between subtests.
- `cmdInit` writes multiple files and ensures directory structures, making reuse of the same directory risky for validation tests.
## Patterns / reusable knowledge
- Always use unique temporary directories (`fs.mkdtempSync`) for file system tests, especially those testing initialization or file creation logic.
- Avoid `fs.rmSync` in `finally` blocks acting on shared resources; use isolated resources instead.
## Warnings / gotchas
- `cmdInit` behaves differently depending on whether `installDir` is `.` or a subdirectory; tests cover both but require clean environments.
@@ -0,0 +1,23 @@
**Agent**: ci-health-agent
**Cadence**: daily
**Run-Start**: 2026-02-21T02-22-00Z
**Prompt**: src/prompts/daily/ci-health-agent.md
# Context — ci-health-agent — 2026-02-21
## Goal
Improve CI reliability by identifying and fixing flaky tests.
## Scope
- `test/install_dir_validation.test.mjs` was identified as flaky in KNOWN_ISSUES.md.
- Refactor the test to use unique temporary directories for each subtest to ensure isolation and determinism.
## Constraints
- Must not modify core logic unless necessary.
- Changes must be safe and reviewable.
- Must follow `AGENTS.md` testing guidelines (improve determinism).
## Findings
- `test/install_dir_validation.test.mjs` reused a single `test_validation_env` directory for all subtests.
- `cmdInit` has side effects (writing files) that could persist between subtests if cleanup failed or race conditions occurred.
- Refactoring to use `fs.mkdtempSync` for each subtest eliminates shared state.
+15
View File
@@ -0,0 +1,15 @@
**Agent**: ci-health-agent
**Cadence**: daily
**Run-Start**: 2026-02-21T02-22-00Z
**Prompt**: src/prompts/daily/ci-health-agent.md
# Todo — ci-health-agent — 2026-02-21
## Pending
- Continue monitoring `test/install_dir_validation.test.mjs` for any recurrence (unlikely with fix).
- Investigate `scripts/agent/load-test.mjs` and `scripts/agent/smoke-test.mjs` failures (future run).
## Completed
- [x] Refactor `test/install_dir_validation.test.mjs` to use unique temporary directories.
- [x] Verify fix by running test 20 times.
- [x] Update `KNOWN_ISSUES.md`.
@@ -0,0 +1,23 @@
---
agent: ci-health-agent
platform: jules
status: completed
---
# Task Completed
- **Agent**: ci-health-agent
- **Prompt**: src/prompts/daily/ci-health-agent.md
- **Status**: Success
## Learnings
### Key findings
- `test/install_dir_validation.test.mjs` flakiness was caused by shared directory state between subtests.
- `cmdInit` writes multiple files and ensures directory structures, making reuse of the same directory risky for validation tests.
### Patterns / reusable knowledge
- Always use unique temporary directories (`fs.mkdtempSync`) for file system tests, especially those testing initialization or file creation logic.
- Avoid `fs.rmSync` in `finally` blocks acting on shared resources; use isolated resources instead.
### Warnings / gotchas
- `cmdInit` behaves differently depending on whether `installDir` is `.` or a subdirectory; tests cover both but require clean environments.
+93 -76
View File
@@ -3,107 +3,124 @@ import assert from 'node:assert';
import { cmdInit } from '../src/ops.mjs';
import fs from 'node:fs';
import path from 'node:path';
const TEST_DIR = 'test_validation_env';
import os from 'node:os';
test('cmdInit should validate install directory name', async (t) => {
if (fs.existsSync(TEST_DIR)) {
fs.rmSync(TEST_DIR, { recursive: true, force: true });
}
fs.mkdirSync(TEST_DIR);
const mockPkg = { name: "host", scripts: {} };
fs.writeFileSync(path.join(TEST_DIR, 'package.json'), JSON.stringify(mockPkg));
async function runWithTempDir(fn) {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'torch-test-'));
// Setup mock package.json
const mockPkg = { name: "host", scripts: {} };
fs.writeFileSync(path.join(tmpDir, 'package.json'), JSON.stringify(mockPkg));
try {
await fn(tmpDir);
} finally {
if (fs.existsSync(tmpDir)) {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
}
}
await t.test('should reject directory with semicolon', async () => {
await assert.rejects(async () => {
await cmdInit(false, TEST_DIR, {
installDir: 'torch; echo pwned',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
await runWithTempDir(async (cwd) => {
await assert.rejects(async () => {
await cmdInit(false, cwd, {
installDir: 'torch; echo pwned',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
});
});
await t.test('should reject directory with spaces', async () => {
await assert.rejects(async () => {
await cmdInit(false, TEST_DIR, {
installDir: 'torch dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
await runWithTempDir(async (cwd) => {
await assert.rejects(async () => {
await cmdInit(false, cwd, {
installDir: 'torch dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
});
});
await t.test('should reject directory with quotes', async () => {
await assert.rejects(async () => {
await cmdInit(false, TEST_DIR, {
installDir: 'torch"dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
await runWithTempDir(async (cwd) => {
await assert.rejects(async () => {
await cmdInit(false, cwd, {
installDir: 'torch"dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
});
});
await t.test('should reject directory with backticks', async () => {
await assert.rejects(async () => {
await cmdInit(false, TEST_DIR, {
installDir: 'torch`dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
await runWithTempDir(async (cwd) => {
await assert.rejects(async () => {
await cmdInit(false, cwd, {
installDir: 'torch`dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
});
});
await t.test('should reject directory with $', async () => {
await assert.rejects(async () => {
await cmdInit(false, TEST_DIR, {
installDir: 'torch$dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
await runWithTempDir(async (cwd) => {
await assert.rejects(async () => {
await cmdInit(false, cwd, {
installDir: 'torch$dir',
namespace: 'ns',
relays: []
});
}, /Invalid directory name/);
});
});
await t.test('should accept valid directory names', async () => {
try {
await cmdInit(false, TEST_DIR, {
installDir: 'valid-dir_123',
namespace: 'ns',
relays: []
});
} catch (e) {
// It might fail because of other initialization steps, but shouldn't fail validation
assert.notMatch(e.message, /Invalid directory name/);
}
await runWithTempDir(async (cwd) => {
try {
await cmdInit(false, cwd, {
installDir: 'valid-dir_123',
namespace: 'ns',
relays: []
});
} catch (e) {
assert.notMatch(e.message, /Invalid directory name/);
}
});
});
await t.test('should accept "."', async () => {
try {
await cmdInit(false, TEST_DIR, {
installDir: '.',
namespace: 'ns',
relays: []
});
} catch (e) {
assert.notMatch(e.message, /Invalid directory name/);
}
await runWithTempDir(async (cwd) => {
try {
await cmdInit(false, cwd, {
installDir: '.',
namespace: 'ns',
relays: []
});
} catch (e) {
assert.notMatch(e.message, /Invalid directory name/);
}
});
});
await t.test('should accept nested paths with slashes', async () => {
try {
await cmdInit(false, TEST_DIR, {
installDir: 'path/to/torch',
namespace: 'ns',
relays: []
});
} catch (e) {
assert.notMatch(e.message, /Invalid directory name/);
}
await runWithTempDir(async (cwd) => {
try {
await cmdInit(false, cwd, {
installDir: 'path/to/torch',
namespace: 'ns',
relays: []
});
} catch (e) {
assert.notMatch(e.message, /Invalid directory name/);
}
});
});
// Cleanup
if (fs.existsSync(TEST_DIR)) {
fs.rmSync(TEST_DIR, { recursive: true, force: true });
}
});