- Implemented throttling in `performSave` to limit disk writes to max 1 per second.
- Refactored `updateMemoryUsage` to use non-blocking (fire-and-forget) save calls, eliminating latency for memory reads.
- Added atomic write strategy using unique temporary files and `fs.rename` to prevent data corruption.
- Verified with benchmarks: `getRelevantMemories` latency reduced from ~100ms to ~30ms for 10k items.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Implemented LexicalIndex class in src/services/memory/retriever.js.
- Added WeakMap caching for indices based on memory array references.
- Improved memory array stability in src/services/memory/index.js using a storeVersion counter.
- Achieved ~2x speedup for 10k memories and ~9x speedup for 100k memories in benchmarks.
- Maintains compatibility with existing textMatchSimilarity scoring.
Co-authored-by: PR0M3TH3AN <[email protected]>
Optimizes `saveMemoryStore` in `src/services/memory/index.js` to use `fs.promises.writeFile` instead of synchronous `fs.writeFileSync`.
Implements a coalescing mechanism to handle concurrent save requests efficiently, preventing race conditions and reducing I/O operations by batching pending updates into a single write.
Reduces event loop blocking from ~420ms (sync) to ~3ms (async) for large write operations in benchmarks.
Updates all `memoryRepository` methods to await the async save operation.
Co-authored-by: PR0M3TH3AN <[email protected]>
Moved the `ensureDir` utility function from `src/ops.mjs` to `src/utils.mjs` to promote code reuse and centralization.
Updated `src/ops.mjs` and `src/services/memory/index.js` to use the shared utility.
Added unit tests for `ensureDir` in `test/utils.test.mjs`.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Modified `getRelevantMemories` in `src/services/memory/index.js` to accept optional `ranker` and `cache` parameters for dependency injection.
- Added `test/memory-index.test.mjs` with comprehensive unit tests for `getRelevantMemories` using `node:test` mocks.
- Verified that existing behavior is preserved via default parameters.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Update `markMemoryMerged` to accept `options.repository` for better testability.
- Add unit tests in `test/memory-admin.test.mjs` to verify:
- Delegation to repository.
- Cache clearing side effect.
Co-authored-by: PR0M3TH3AN <[email protected]>
Replaced `console.log` with `util.debuglog('torch-memory')` in `src/services/memory/index.js` to prevent stdout pollution. Telemetry and metrics are now logged to stderr only when `NODE_DEBUG=torch-memory` is set, improving library behavior and observability control.
Added regression test `test/memory-telemetry.test.mjs`.
Co-authored-by: PR0M3TH3AN <[email protected]>
- Add `test/memory-pinning.test.mjs` to test `pinMemory` happy path and edge cases.
- Update `getRelevantMemories` in `src/services/memory/index.js` to use `repository.listMemories` when available, enabling isolated unit testing.
- Verified all memory tests pass.
Co-authored-by: PR0M3TH3AN <[email protected]>
