Commit Graph
48 Commits
Author SHA1 Message Date
google-labs-jules[bot]andPR0M3TH3AN bdd3ed8c8f fix(security): prevent command injection in governance service and fix build errors
- Mitigate command injection in `src/services/governance/index.js` by using explicit array arguments with `execFileSync` and adding a security comment.
- Add regression test `test/governance-security.test.mjs` verifying the fix against simulated exploit attempts.
- Fix CI build failures by resolving circular dependencies and duplicate declarations in `src/lock-utils.mjs`, `src/lock-ops.mjs`, `src/lock-publisher.mjs`, and `src/relay-health.mjs`.
- Resolve linting errors in tests and implementation files.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-21 19:35:31 +00:00
thePR0M3TH3AN 8d43fc1be8 Merge pull request #376 from PR0M3TH3AN/perf/optimize-embedder-norms-8957795496330689334
⚡ Optimize vector similarity with precomputed norms
2026-02-20 16:41:51 -05:00
thePR0M3TH3AN 7e01b951a8 Merge pull request #381 from PR0M3TH3AN/memory-save-perf-6430625137963819127
⚡ Optimize memory store persistence with throttling and atomic writes
2026-02-20 16:35:38 -05:00
google-labs-jules[bot]andPR0M3TH3AN 98d7bdb3ba perf(memory): optimize save operations with throttling and atomic writes
- Implemented throttling in `performSave` to limit disk writes to max 1 per second.
- Refactored `updateMemoryUsage` to use non-blocking (fire-and-forget) save calls, eliminating latency for memory reads.
- Added atomic write strategy using unique temporary files and `fs.rename` to prevent data corruption.
- Verified with benchmarks: `getRelevantMemories` latency reduced from ~100ms to ~30ms for 10k items.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-20 18:25:45 +00:00
google-labs-jules[bot]andPR0M3TH3AN 823d296b4a perf(memory): optimize cosineSimilarity with precomputed norms
- Precompute vector norms during `upsertVector` in `InMemoryEmbedderAdapter`.
- Compute query vector norm once per query.
- Update `cosineSimilarity` to accept precomputed norms, avoiding redundant calculations.
- Optimize `cosineSimilarity` loop to iterate only up to `Math.min(left.length, right.length)` and remove redundant casts.
- Achieved ~27% improvement in query time (57ms vs 78ms for 50 queries against 2000 vectors).

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-20 18:10:50 +00:00
google-labs-jules[bot]andPR0M3TH3AN 258da844a6 refactor(governance): optimize applyProposal to reuse proposal object in validation
Updates `validateProposal` to accept either a proposal ID or a proposal object.
Updates `applyProposal` to pass the already-fetched proposal object to `validateProposal`,
eliminating a redundant file system read. This improves efficiency and code health.
Tests were verified to ensure no regression.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-20 18:07:22 +00:00
thePR0M3TH3AN be849ceae8 Merge pull request #356 from PR0M3TH3AN/perf-lexical-search-index-1433868811411746305
⚡ Performance Optimization: Inverted Index for Lexical Search
2026-02-19 22:52:33 -05:00
google-labs-jules[bot]andPR0M3TH3AN 9e6d99eb1e ⚡ optimize lexical search with inverted index
- Implemented LexicalIndex class in src/services/memory/retriever.js.
- Added WeakMap caching for indices based on memory array references.
- Improved memory array stability in src/services/memory/index.js using a storeVersion counter.
- Achieved ~2x speedup for 10k memories and ~9x speedup for 100k memories in benchmarks.
- Maintains compatibility with existing textMatchSimilarity scoring.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-20 03:51:34 +00:00
thePR0M3TH3AN a35286f17d Merge pull request #355 from PR0M3TH3AN/refactor-governance-target-validation-5425533680072569362
🧹 refactor(governance): extract duplicated target validation logic
2026-02-19 22:51:13 -05:00
google-labs-jules[bot]andPR0M3TH3AN cd744982a1 refactor(governance): extract duplicated target validation logic
Extracted duplicated target path resolution and allowlist validation logic
into a shared `resolveAndValidateTarget` helper function in
`src/services/governance/index.js`.

Updated `createProposal`, `validateProposal`, `applyProposal`, and
`rollbackPrompt` to use the new helper, ensuring consistent validation
and path handling across the service.

Added `test/governance-health.test.mjs` to verify validation logic.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-20 03:50:36 +00:00
google-labs-jules[bot]andPR0M3TH3AN 1e4d9720e8 fix(security): prevent command injection in governance service
Replaced `execSync` with `execFileSync` in `src/services/governance/index.js` to avoid shell interpretation of variable-derived inputs (such as agent IDs and file targets). This mitigates command injection risks in `createProposal`, `applyProposal`, and `rollbackPrompt`.

- Used `execFileSync` for all git operations.
- Updated logic to correctly handle `git diff` exit code 1.
- Removed unused `execSync` import.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-20 03:46:36 +00:00
thePR0M3TH3AN 5783f995d7 update 2026-02-19 22:22:45 -05:00
Claude be5ec21d5c feat(governance): add prompt versioning, rollback --list, and state backup
- governance/index.js: archive filenames now include ISO timestamp for
  deterministic sorting (<base>_<ts>_<hash>.md); write .meta.json sidecar
  on every archive with proposalId, author, reason, and archivedAt
- governance/index.js: add listPromptVersions(target) — returns archived
  versions for a prompt sorted newest-first with sidecar metadata surfaced
- cmd-rollback.mjs: add --list flag to list available versions without
  restoring (delegates to listPromptVersions)
- cli-parser.mjs: add --list and --output flags
- lib.mjs: pass list flag to cmdRollback; add backup command dispatch
- cmd-backup.mjs: new torch-lock backup command — snapshots
  .scheduler-memory/memory-store.json and scheduler run state to
  .torch/backups/<timestamp>/; supports --output override and --list
- docs/prompt-versioning.md: comprehensive guide to the full
  proposal → apply → list-versions → rollback workflow, state backup
  and restore procedure, and storage location reference
- test/governance.test.mjs: 7 scenario-based tests (SCN-proposal-create,
  SCN-proposal-list, SCN-proposal-reject, SCN-archive-naming,
  SCN-version-list, SCN-rollback-latest, SCN-rollback-hash)

https://claude.ai/code/session_01Ndq2D72Kp77BYo4owD63uY
2026-02-20 02:27:57 +00:00
Claude 639ad6877d chore: code quality, CI/CD, and DX improvements
Phase 3 — Code Quality:
- Deduplicate nowUnix() in lock-ops.mjs (import from utils.mjs)
- Promote ESLint no-unused-vars to error, fix all 46 violations
- Remove dead code: unused imports, variables, and catch bindings
- Add /* global marked */ for CDN-loaded library in dashboard

Phase 4 — CI/CD:
- Add Node.js version matrix (22, 23) to CI workflow
- Add npm audit security scan step to CI
- Add innerHTML usage check to CI pipeline
- Enable Dependabot for npm and GitHub Actions dependencies
- Create PR template

Phase 5 — Developer Experience:
- Create .env.example documenting all environment variables
- Expand CONTRIBUTING.md with commit conventions, agent/prompt
  creation guide, architecture overview, and test integrity
- Add system architecture diagram with Mermaid (docs/architecture/)

https://claude.ai/code/session_01GfwUNbgeMgXreFDbsNQMsS
2026-02-19 21:03:30 +00:00
google-labs-jules[bot]andPR0M3TH3AN dd9e5b7921 feat: add standard CI workflow and fix lint/test failures
- Added `.github/workflows/ci.yml` to run build, lint, and test on PRs/push to main.
- Updated `.github/workflows/ci-resilience-nightly.yml` to use Node.js 22.
- Fixed lint errors in `src/services/governance/index.js` (no-empty, preserve-caught-error).
- Added `governance-agent` to daily roster and scheduler prompt to fix `npm test` scheduler validation failure.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 21:24:43 +00:00
google-labs-jules[bot]andPR0M3TH3AN 64b6f43df1 feat: add governance system for prompt evolution
Implement a comprehensive governance system to manage agent prompt changes safely.

Key features:
- **Proposal Workflow:** Prompt changes are now "proposals" stored in `src/proposals/` with metadata and diffs.
- **Governance Agent:** A new `governance-agent` reviews and applies proposals using `scripts/governance/process-proposals.mjs`.
- **Validation:** Strict validation ensures changes are limited to allowed directories (`src/prompts/daily|weekly`) and preserve required headers.
- **Lock Enhancements:** Lock events now include `promptHash`, `promptPath`, and `gitCommit` for full auditability.
- **Rollback:** New `torch-lock rollback` command restores prompts from local history (`.torch/prompt-history/`) or git.
- **CLI:** Added `proposal` (create, list, show, apply, reject) and `rollback` commands.

This establishes a "Stage -> Evaluate -> Apply" gate for prompt evolution, preventing silent drift and ensuring constitutional stability.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 21:09:39 +00:00
thePR0M3TH3AN 0dfbf3f86f Merge branch 'main' into perf/async-memory-save-18240015948353785856 2026-02-18 09:48:42 -05:00
thePR0M3TH3AN ae85044021 Merge pull request #281 from PR0M3TH3AN/perf/optimize-pruner-lifecycle-3710831960416065931
⚡ Optimize createLifecyclePlan with Promise.all
2026-02-17 22:42:24 -05:00
thePR0M3TH3AN fabe54ca50 Merge pull request #286 from PR0M3TH3AN/perf-optimize-text-match-similarity-10778846404637796189
⚡ Optimize memory text similarity check
2026-02-17 22:42:05 -05:00
thePR0M3TH3AN 11c7510b95 Merge pull request #287 from PR0M3TH3AN/perf/optimize-pruner-lifecycle-15735036455677538666
⚡ Parallelize memory lifecycle actions in pruner
2026-02-17 22:41:48 -05:00
thePR0M3TH3AN 84e1da5825 Merge pull request #279 from PR0M3TH3AN/perf-pruner-optimization-6586476437632671797
⚡ Optimize memory pruner groupNearDuplicates
2026-02-17 22:41:07 -05:00
google-labs-jules[bot]andPR0M3TH3AN c123dd9a77 perf: make saveMemoryStore async and non-blocking
Optimizes `saveMemoryStore` in `src/services/memory/index.js` to use `fs.promises.writeFile` instead of synchronous `fs.writeFileSync`.
Implements a coalescing mechanism to handle concurrent save requests efficiently, preventing race conditions and reducing I/O operations by batching pending updates into a single write.
Reduces event loop blocking from ~420ms (sync) to ~3ms (async) for large write operations in benchmarks.
Updates all `memoryRepository` methods to await the async save operation.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 03:37:12 +00:00
google-labs-jules[bot]andPR0M3TH3AN 0f29e9126e perf: parallelize applyLifecycleActions for faster pruning
- Use Promise.all to execute memory lifecycle actions concurrently.
- Maintain intra-item dependency order (merged_into before action).
- Update tests to support non-deterministic execution order of independent items.
- Benchmark shows ~65x improvement for 100 items with 10ms latency.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 03:32:03 +00:00
google-labs-jules[bot]andPR0M3TH3AN 850be9c37d perf: Optimize textMatchSimilarity in memory retriever
Avoids redundant string concatenation and lowercasing in the inner loop of `buildSimilarityIndex`.
Pre-computes query terms and regexes once per query.
Uses RegExp.test for case-insensitive matching without allocation.
Benchmark shows ~2x speedup (99ms vs 191ms for 10k memories).

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 03:31:22 +00:00
google-labs-jules[bot]andPR0M3TH3AN 7305dd964d Optimize createLifecyclePlan with Promise.all
Replaced sequential `await condenseGroup` calls with `Promise.all` to run condensation tasks concurrently.
Added `scripts/benchmark-pruner.mjs` to measure performance improvement.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 03:30:28 +00:00
google-labs-jules[bot]andPR0M3TH3AN ace069cc19 perf(pruner): optimize groupNearDuplicates to O(N log N) from O(N^2)
Sorts memories by created_at and uses an early exit condition in the inner loop to avoid checking all pairs. This reduces complexity from O(N^2) to roughly O(N * k) where k is the number of items within the duplicate window.

Measured improvement for 30k items: ~2.1s -> ~0.25s (8x speedup).

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 03:30:11 +00:00
google-labs-jules[bot]andPR0M3TH3AN ba5d86142c Refactor: Move ensureDir to src/utils.mjs
Moved the `ensureDir` utility function from `src/ops.mjs` to `src/utils.mjs` to promote code reuse and centralization.
Updated `src/ops.mjs` and `src/services/memory/index.js` to use the shared utility.
Added unit tests for `ensureDir` in `test/utils.test.mjs`.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-18 03:28:09 +00:00
google-labs-jules[bot]andPR0M3TH3AN 694afb1462 feat: make getRelevantMemories testable and add tests
- Modified `getRelevantMemories` in `src/services/memory/index.js` to accept optional `ranker` and `cache` parameters for dependency injection.
- Added `test/memory-index.test.mjs` with comprehensive unit tests for `getRelevantMemories` using `node:test` mocks.
- Verified that existing behavior is preserved via default parameters.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-17 18:53:52 +00:00
thePR0M3TH3AN c549c0e8b0 Merge pull request #227 from PR0M3TH3AN/test-mark-memory-merged-6408631238217390232
test(memory): add tests for markMemoryMerged
2026-02-17 13:10:10 -05:00
thePR0M3TH3AN e61b4e3d4a Merge pull request #226 from PR0M3TH3AN/memory-telemetry-fix-8633028829140804238
🧹 Use util.debuglog for memory telemetry
2026-02-17 13:09:23 -05:00
google-labs-jules[bot]andPR0M3TH3AN 78c31e828d test(memory): add tests for markMemoryMerged and enable dependency injection
- Update `markMemoryMerged` to accept `options.repository` for better testability.
- Add unit tests in `test/memory-admin.test.mjs` to verify:
  - Delegation to repository.
  - Cache clearing side effect.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-17 17:32:45 +00:00
google-labs-jules[bot]andPR0M3TH3AN 53aa68ac64 Refactor memory telemetry to use util.debuglog
Replaced `console.log` with `util.debuglog('torch-memory')` in `src/services/memory/index.js` to prevent stdout pollution. Telemetry and metrics are now logged to stderr only when `NODE_DEBUG=torch-memory` is set, improving library behavior and observability control.

Added regression test `test/memory-telemetry.test.mjs`.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-17 17:31:52 +00:00
google-labs-jules[bot]andPR0M3TH3AN a3b181b3d8 test: add unit tests for pinMemory and fix dependency injection in getRelevantMemories
- Add `test/memory-pinning.test.mjs` to test `pinMemory` happy path and edge cases.
- Update `getRelevantMemories` in `src/services/memory/index.js` to use `repository.listMemories` when available, enabling isolated unit testing.
- Verified all memory tests pass.

Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-17 17:31:44 +00:00
google-labs-jules[bot]andPR0M3TH3AN 825d596a59 Weekly dead-code-agent execution: removed scripts/fix-duplication.mjs and src/services/memory/db-adapter.js
Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-17 14:33:14 +00:00
google-labs-jules[bot]andPR0M3TH3AN c6428c42c8 Implement persistent JSON memory store and disable embeddings
Co-authored-by: PR0M3TH3AN <[email protected]>
2026-02-15 23:18:13 +00:00
thePR0M3TH3AN d1586228be Add memory rollout feature flags and docs 2026-02-14 15:23:51 -05:00
thePR0M3TH3AN 7cef8242a1 Add memory admin CLI hooks and safe telemetry events 2026-02-14 15:15:15 -05:00
thePR0M3TH3AN 5e609fabe8 feat(memory): inject retrieval context into final prompt rendering 2026-02-14 15:09:34 -05:00
thePR0M3TH3AN 3ddd05017a feat(memory): add distributed-safe maintenance scheduler jobs 2026-02-14 15:05:59 -05:00
thePR0M3TH3AN 106154263c Implement memory pruning lifecycle planning and actions 2026-02-14 15:02:49 -05:00
thePR0M3TH3AN 0236958f63 Implement vector-aware memory retrieval and prompt formatter 2026-02-14 14:59:05 -05:00
thePR0M3TH3AN b8d4d2ac82 Add pluggable memory embedder adapter and vector metadata 2026-02-14 14:55:07 -05:00
thePR0M3TH3AN 3665aa0f4f Add memory prompt templates and robust JSON summarization 2026-02-14 14:51:50 -05:00
thePR0M3TH3AN 8e1ff73cbb Implement batched memory ingestion pipeline with dedupe 2026-02-14 14:48:19 -05:00
thePR0M3TH3AN 3c24b29799 Add scoped runtime memory cache with LRU and metrics 2026-02-14 14:44:07 -05:00
thePR0M3TH3AN 99248b5008 Add memory persistence migration and repository adapter methods 2026-02-14 14:41:08 -05:00
thePR0M3TH3AN 5f9b1a270a Add durable memory schema validation in ingest pipeline 2026-02-14 14:37:28 -05:00
thePR0M3TH3AN 9c2524540c Add modular memory service API scaffold 2026-02-14 14:33:23 -05:00