From 832e8d476a373d299fe73e0afc0dd7ea82c01d8e Mon Sep 17 00:00:00 2001 From: thePR0M3TH3AN <53631862+PR0M3TH3AN@users.noreply.github.com> Date: Wed, 21 May 2025 16:58:54 -0400 Subject: [PATCH] Add rename handling spec and roadmap entry --- docs/roadmap.md | 3 +- docs/spec-details/Rename+Move-Handling.md | 71 +++++++++++++++++++++++ 2 files changed, 73 insertions(+), 1 deletion(-) create mode 100644 docs/spec-details/Rename+Move-Handling.md diff --git a/docs/roadmap.md b/docs/roadmap.md index 8c324cf..5b073f9 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -25,7 +25,7 @@ | Phase / Sprint | Timeline | Focus & Rationale | ✦ Key UX Deliverables | △ Engineering artefacts / tasks | Definition of Done | | --------------------------------------------- | -------- | ---------------------------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | ~~**Epic 1 — Scale & Reliability**~~ | ~~2025-Q2~~ | ~~Stay fast @ 100 k files~~ | ~~• `scan --dirty` (re-index touched rows only)~~ | ~~• DP-002 Dirty-flag design + FTS rebuild cadence
• Hyperfine benchmark script committed~~ | ~~Dirty scan vs full ≤ 15 % runtime on 100 k corpus; benchmark job passes~~ | -| **Epic 2 — Live Mode & Self-Pruning Backups** | 2025-Q2 | “Just works” indexing, DB never explodes | • `marlin watch ` (notify/FSEvents)
• `backup --prune N` & auto-prune | • DP-003 file-watcher life-cycle & debouncing
• Integration test with inotify-sim
• Cron-style GitHub job for nightly prune | 8 h stress-watch alters 10 k files < 1 % misses; backup dir ≤ N | +| **Epic 2 — Live Mode & Self-Pruning Backups** | 2025-Q2 | “Just works” indexing, DB never explodes | • `marlin watch ` (notify/FSEvents)
• `backup --prune N` & auto-prune
• rename/move tracking keeps paths current | • DP-003 file-watcher life-cycle & debouncing
• Integration test with inotify-sim
• Rename/Move handling spec & tests
• Cron-style GitHub job for nightly prune | 8 h stress-watch alters 10 k files < 1 % misses; backup dir ≤ N | | **Phase 3 — Content FTS + Annotations** | 2025-Q3 | Search inside files, leave notes | • Grep-style snippet output (`-C3`)
• `marlin annotate add/list` | • DP-004 content-blob strategy (inline vs ext-table)
• Syntax-highlight via `syntect` PoC
• New FTS triggers unit-tested | Indexes 1 GB corpus in ≤ 30 min; snippet CLI passes golden-file tests | | **Phase 4 — Versioning & Deduplication** | 2025-Q3 | Historic diffs, detect dupes | • `scan --rehash` (SHA-256)
• `version diff ` | • DP-005 hash column + Bloom-de-dupe
• Binary diff adapter research | Diff on 10 MB file ≤ 500 ms; dupes listed via CLI | | **Phase 5 — Tag Aliases & Semantic Booster** | 2025-Q3 | Tame tag sprawl, start AI hints | • `tag alias add/ls/rm`
• `tag suggest`, `summary` | • DP-006 embeddings size & model choice
• Vector store schema + k-NN index bench | 95 % of “foo/bar~foo” alias look-ups resolve in one hop; suggest CLI returns ≤ 150 ms | @@ -46,6 +46,7 @@ | Tarpaulin coverage gate | S0 | — | – | | Watch mode (FS events) | Epic 1 | `marlin watch .` | DP‑002 | | Backup auto‑prune | Epic 1 | `backup --prune N` | – | +| Rename/move tracking | Epic 2 | automatic path update | – | | Dirty‑scan | Epic 2 | `scan --dirty` | DP‑002 | | Grep snippets | Phase 3 | `search -C3 …` | DP‑004 | | Hash / dedupe | Phase 4 | `scan --rehash` | DP‑005 | diff --git a/docs/spec-details/Rename+Move-Handling.md b/docs/spec-details/Rename+Move-Handling.md new file mode 100644 index 0000000..8daefa5 --- /dev/null +++ b/docs/spec-details/Rename+Move-Handling.md @@ -0,0 +1,71 @@ +# Marlin — Rename & Move Handling + +**Integration Specification · v0.1 (2025-05-19)** + +--- + +## 0 · Scope + +This document outlines how Marlin should respond when files or folders are renamed or moved. It extends the watcher life‑cycle design (DP‑003) so that metadata remains consistent without requiring a full re‑scan. + +## 1 · Background + +The current watcher maps any `notify::EventKind::Modify(_)` – including renames – to the generic `EventPriority::Modify` and merely logs the event: + +``` +415 let prio = match event.kind { +416 EventKind::Create(_) => EventPriority::Create, +417 EventKind::Remove(_) => EventPriority::Delete, +418 EventKind::Modify(_) => EventPriority::Modify, +419 EventKind::Access(_) => EventPriority::Access, +420 _ => EventPriority::Modify, +421 }; +... +455 for event_item in &evts_to_process { +456 info!("Processing event (DB available): {:?} for path {:?}", +457 event_item.kind, event_item.path); +458 } +``` + +No database update occurs, so renamed files keep their old `path` in the `files` table. The schema does have a trigger to propagate `path` updates to the FTS index: + +``` +72 -- When a file’s path changes +73 DROP TRIGGER IF EXISTS files_fts_au_file; +74 CREATE TRIGGER files_fts_au_file +75 AFTER UPDATE OF path ON files +76 BEGIN +77 UPDATE files_fts +78 SET path = NEW.path +79 WHERE rowid = NEW.id; +80 END; +``` + +## 2 · Requirements + +1. **Detect old and new paths** from `Rename` events provided by the `notify` crate. +2. **Update the `files` table** with the new absolute path when the target remains inside a scanned root. +3. **Mark as removed** if the new location is outside all configured roots. +4. **Batch updates** to avoid excessive writes during large folder moves. +5. **Integration tests** exercising rename and move scenarios across platforms. + +## 3 · Implementation Sketch + +* Extend `ProcessedEvent` to carry `old_path` and `new_path` for `Rename` events. +* Upon flushing events, call `db::mark_dirty` for the affected row, then update the `files.path` column. The existing trigger keeps `files_fts` in sync. +* For directory renames, update child paths with a single SQL `UPDATE ... WHERE path LIKE 'old/%'` inside a transaction. +* Emit `Create` and `Remove` events for files crossing watch boundaries so `scan --dirty` can prune or index them accordingly. + +## 4 · Edge Cases + +* **Atomic cross-filesystem moves** may surface as `Remove` + `Create`; both should be handled. +* **Concurrent modifications** while moving should result in the newer metadata winning when `scan --dirty` runs. + +## 5 · Future Work + +Large scale refactors (e.g. moving an entire project) may benefit from a high‑level command that updates tags and links en masse. That is outside the scope of this spec but enabled by accurate rename tracking. + +--- + +*End of document* +