From 84df9589783094c476dcf2bd81f69a03b13f0694 Mon Sep 17 00:00:00 2001 From: thePR0M3TH3AN <53631862+PR0M3TH3AN@users.noreply.github.com> Date: Thu, 15 May 2025 16:16:19 -0400 Subject: [PATCH] update --- roadmap.md | 57 ++++++++++++++++++++++++++++++------------------------ 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/roadmap.md b/roadmap.md index 5efe6dc..9032721 100644 --- a/roadmap.md +++ b/roadmap.md @@ -1,33 +1,40 @@ -# Roadmap +Here’s the updated roadmap with each new feature slotted in where its dependencies are best met: -| Phase | Focus | Why now? | Key deliverables | -| --------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -| **1. 2025-Q2 – “Bedrock”** | Migrations + CI baseline | We’ve landed versioned migrations and removed runtime column hacks – ensure it stays solid. | • CI job runs `cargo test` + `cargo sqlx migrate run --dry-run` | -| **2. 2025-Q2** | Leaner FTS maintenance | Per-row triggers don’t scale past ~100 k files. | • Replace triggers with “dirty” flag + periodic rebuild
• Benchmark on 100 k files | -| **2.1 2025-Q2** (adjunct) | Dirty-row FTS + CI | Prep for both scale and live-watcher—avoid full rebuilds on every change. | • `scan --dirty` reindexes only files whose SHA-256 hash has changed | -| **2.2 2025-Q2** | Live file watching | Offer a true “working-dir” mode—auto-scan on FS events, zero manual rescans. | • New `marlin watch [dir]` subcommand using `notify` crate
• Incremental scans on create/delete/modify/rename events | -| **2.3 2025-Q2** | Self-pruning backups | Instant protection and bounded storage—no manual snapshot cleanup. | • `marlin backup --prune ` flag
• Hook post-scan (when rows change) to auto-prune to latest 10
• Daily cron/automation to keep last 7 | -| **3. 2025-Q3** | FTS5 content indexing | Extend beyond path/tags/attrs to full file bodies for true on-disk search. | • Add `files.content` column
• New migration to include `content` in `files_fts`
• Update triggers to populate it | -| **4. 2025-Q3** | Content hashing & dedup | Detect duplicates, enable future integrity checks. | • SHA-256 in `files.hash`
• `scan --rehash` option | -| **5. 2025-Q3** | Tag aliases / canonicals | Control tag sprawl before users accumulate thousands. | • `canonical_id` enforcement
• `tag alias add|ls|rm` CLI | -| **6. 2025-Q4** | Search DSL v2 | Power users want grouping, boolean ops, quoted phrases. | • Replace ad-hoc parser with `nom` grammar
• Unit-tested examples | -| **7. 2025-Q4** | Attribute templates | Structured metadata unlocks real workflows. | • `templates` + `template_fields` tables
• Validation on `attr set` | -| **8. 2026-Q1** | Dolphin read-only plugin | Browse tags/attrs inside the default file manager. | • Qt sidebar showing metadata | -| **9. 2026-Q1** | Full edit UI | After read-only proves stable, add editing. | • Tag editor widget, attribute dialog | -| **10. 2026-Q2** | Multi-device sync | Final frontier: optional sync/replication layer. | • Choose between rqlite / Litestream / bespoke
• Read-only mode for network mounts | +| Phase | Focus | Why now? | Key deliverables | +| -------------------------- | -------------------------------------- | --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | +| **1. 2025-Q2 – “Bedrock”** | Migrations + CI baseline + core schema | We’ve stabilized migrations; now add foundational tables for links, groups, views | • CI: `cargo test` + `cargo sqlx migrate run --dry-run`
• New migrations: | + +* `links(src_file,dst_file,link_type)` +* `collections(name)` + `collection_files` +* `views(name,query)`
• CLI stubs for `marlin link` / `unlink` / `list-links` / `backlinks`, `marlin coll` and `marlin view` | + \| **2. 2025-Q2** | Leaner FTS maintenance | Per-row triggers don’t scale past \~100 k files | • Replace per-row triggers with a “dirty” flag + periodic rebuild
• Benchmark end-to-end on 100 k files | + \| **2.1 2025-Q2** | Dirty-row FTS + CI | Prep for both scale and live-watcher—avoid full rebuilds on every change | • `scan --dirty` reindexes only changed files
• CI coverage for dirty-scan edge cases | + \| **2.2 2025-Q2** | Live file watching | Offer true “working-dir” mode—auto-scan on FS events | • `marlin watch [dir]` via `notify` crate
• Incremental scan on create/modify/delete/rename | + \| **2.3 2025-Q2** | Self-pruning backups | Instant protection and bounded storage—no manual snapshot cleanup | • `marlin backup --prune ` flag
• Post-scan hook to prune to latest 10
• Daily prune automation (cron or CI) | + \| **3. 2025-Q3** | FTS5 content indexing & annotations | Full-text search over file bodies + per-file notes/highlights | • Add `files.content` column + migration
• Extend `files_fts` to include `content`
• New `annotations` table + FTS triggers
• CLI: `marlin annotate add|list` | + \| **4. 2025-Q3** | Content hashing, dedup & versioning | Detect duplicates, track history, enable diffs | • Populate `files.hash` with SHA-256
• `scan --rehash` option
• CLI: `marlin version diff ` | + \| **5. 2025-Q3** | Tag aliases/canonicals & semantic/AI enhancements | Control tag sprawl and lay groundwork for AI-driven suggestions | • Enforce `canonical_id` on `tags` + `tag alias add|ls|rm` CLI
• Create `embeddings` table
• `scan --embed` to generate vectors
• CLI: `marlin tag suggest`, `marlin summary `, `marlin similarity scan` | + \| **6. 2025-Q4** | Search DSL v2 & Smart Views | More powerful query grammar + reusable “virtual folders” | • Replace ad-hoc parser with a `nom`-based grammar
• CLI: `marlin view save|list|exec` | + \| **7. 2025-Q4** | Attribute templates, states, tasks & timeline | Structured metadata unlocks workflows, reminders & temporal context | • `templates` + `template_fields` tables + validation
• CLI: +* `marlin state set|transitions add|state log` +* `marlin task scan|task list` +* `marlin remind set ""` +* `marlin event add ""`, `marlin timeline` | + \| **8. 2026-Q1** | Dolphin read-only plugin | Surface metadata, links, annotations in native file manager | • Qt sidebar showing tags, attributes, links, annotations | + \| **9. 2026-Q1** | Full edit UI | After proving read-only stable, add in-place editing | • Tag editor, collection & view managers, state/task/event dialogs | + \| **10. 2026-Q2** | Multi-device sync | Final frontier: optional sync/replication layer | • Choose sync backend (rqlite / Litestream / bespoke)
• Support read-only mounts for remote indexes | --- ### Current sprint (ends **2025-06-01**) -1. **FTS rebuild prototype** – dirtied-rows approach, measure on 50 k files. -2. **`backup --prune`** to keep only N most recent snapshots. -3. Integration tests for tag/attr workflows on Windows via GitHub Actions. +1. FTS rebuild prototype (dirty-rows) – measure on 50 k files +2. `backup --prune` implementation + auto-prune hook +3. Integration tests for tag/attr workflows on Windows via GitHub Actions +4. **New:** basic `links`, `collections`, `views` migrations + CLI stubs ---- +**Development principles remain**: -### Development principles - -* **Local-first** – every feature must work offline. -* **Zero manual migrations** – shipping code *is* the migration. -* **Instrumentation first** – every new command logs trace spans and timings. +* Local-first, offline-capable +* Ship code = ship migrations +* Instrumentation first (trace spans & timings on all new commands)