From 9674611990bb9c122ffb74247b6ae0b0727a5751 Mon Sep 17 00:00:00 2001 From: thePR0M3TH3AN <53631862+PR0M3TH3AN@users.noreply.github.com> Date: Fri, 16 May 2025 22:24:55 -0400 Subject: [PATCH] update --- README.md | 144 ++++++++++++++++++++++++------------------------- marlin_demo.md | 130 ++++++++++++++++++++++---------------------- roadmap.md | 81 ++++++++++++++++++---------- 3 files changed, 186 insertions(+), 169 deletions(-) diff --git a/README.md b/README.md index a615e86..55b467c 100644 --- a/README.md +++ b/README.md @@ -2,21 +2,24 @@ # Marlin -**Marlin** is a lightweight, metadata-driven file indexer that runs 100 % on your computer. It scans folders, stores paths and file stats in SQLite, lets you attach hierarchical **tags** and **custom attributes**, takes automatic snapshots, and offers instant full-text search via FTS5. +**Marlin** is a lightweight, metadata-driven file indexer that runs **100 % on your computer**. +It scans folders, stores paths and file stats in SQLite, lets you attach hierarchical **tags** and **custom attributes**, keeps timestamped **snapshots**, and offers instant full-text search via FTS5. _No cloud, no telemetry – your data never leaves the machine._ --- ## Feature highlights -| Area | What you get | -| -------------- | --------------------------------------------------------------------------------- | -| **Safety** | Timestamped backups (`marlin backup`) and one-command restore (`marlin restore`) | -| **Resilience** | Versioned, idempotent schema migrations – zero-downtime upgrades | -| **Indexing** | Fast multi-path scanner with SQLite WAL concurrency | -| **Metadata** | Hierarchical tags (`project/alpha`) & key-value attributes (`reviewed=yes`) | -| **Search** | Prefix-aware FTS5 across paths, tags, and attributes; optional `--exec` per match | -| **DX / Logs** | Structured tracing (`RUST_LOG=debug`) for every operation | +| Area | What you get | +| ------------------- | ----------------------------------------------------------------------------------------------------- | +| **Safety** | Timestamped backups (`marlin backup`) and one-command restore (`marlin restore`) | +| **Resilience** | Versioned, idempotent schema migrations – zero-downtime upgrades | +| **Indexing** | Fast multi-path scanner with SQLite WAL concurrency | +| **Metadata** | Hierarchical tags (`project/alpha`) & key-value attributes (`reviewed=yes`) | +| **Relations** | Typed file ↔ file links (`marlin link`) with backlinks viewer | +| **Collections / Views** | Named playlists (`marlin coll`) & saved searches (`marlin view`) for instant recall | +| **Search** | Prefix-aware FTS5 across paths, tags, attrs & links; optional `--exec` per match
(grep-style context snippets coming Q3) | +| **DX / Logs** | Structured tracing (`RUST_LOG=debug`) for every operation | --- @@ -26,11 +29,11 @@ _No cloud, no telemetry – your data never leaves the machine._ ┌──────────────┐ marlin scan ┌─────────────┐ │ your files │ ─────────────────────▶│ SQLite │ │ (any folder) │ │ files/tags │ -└──────────────┘ tag / attr │ attrs / FTS │ - ▲ search / exec └──────┬──────┘ +└──────────────┘ tag / attr / link │ attrs / FTS │ + ▲ search / exec └──────┬──────┘ └────────── backup / restore ▼ timestamped snapshots -``` +```` --- @@ -38,7 +41,7 @@ _No cloud, no telemetry – your data never leaves the machine._ | Requirement | Why | | ------------------ | ----------------------------- | -| **Rust** ≥ 1.77 | Build toolchain (`rustup.rs`) | +| **Rust ≥ 1.77** | Build toolchain (`rustup.rs`) | | C build essentials | Builds bundled SQLite (Linux) | macOS & Windows users: let the Rust installer pull the matching build tools. @@ -48,78 +51,63 @@ macOS & Windows users: let the Rust installer pull the matching build tools. ## Build & install ```bash -git clone https://github.com/yourname/marlin.git -cd marlin +git clone https://github.com/PR0M3TH3AN/Marlin.git +cd Marlin cargo build --release -# (Optional) Install the binary into your PATH: +# (Optional) install into your PATH sudo install -Dm755 target/release/marlin /usr/local/bin/marlin ``` +--- + ## Quick start -For a concise walkthrough, see [Quick start & Demo](marlin_demo.md). +For a concise walkthrough—including **links, collections and views**—see +[**Quick start & Demo**](marlin_demo.md). -## Testing +--- + +## Testing Below is a **repeat-able 3-step flow** you can use **every time you pull fresh code**. ---- - -### 0 Prepare once +### 0 Prepare once ```bash -# Run once (or add to ~/.bashrc) so debug + release artefacts land -# in the same predictable place. Speeds-up future builds. +# Put build artefacts in one place (faster incremental builds) export CARGO_TARGET_DIR=target ``` ---- - -### 1 Build the new binary +### 1 Build the new binary ```bash -git pull # grab the latest commit +git pull cargo build --release sudo install -Dm755 target/release/marlin /usr/local/bin/marlin ``` -* `cargo build --release` – builds the optimised binary. -* `install …` – copies it into your `$PATH` so `marlin` on the CLI is the fresh one. - ---- - -### 2 Run the smoke-test suite +### 2 Run the smoke-test suite ```bash -# Runs the end-to-end test we added in tests/e2e.rs cargo test --test e2e -- --nocapture ``` -* `--test e2e` – compiles and runs **only** `tests/e2e.rs`; other unit-tests are skipped (add them later if you like). -* `--nocapture` – streams stdout/stderr so you can watch each CLI step in real time. -* Exit-code **0** ➜ everything passed. - Any non-zero exit or a red ✗ line means a step failed; the assert’s diff will show the command and its output. +*Streams CLI output live; exit-code 0 = all good.* ---- - -### 3 (Optionally) run all tests +### 3 (Optionally) run **all** tests ```bash cargo test --all -- --nocapture ``` -This will execute: +This now covers: * unit tests in `src/**` -* every file in `tests/` +* positive & negative integration suites (`tests/pos.rs`, `tests/neg.rs`) * doc-tests -If you wire **“cargo test --all”** into CI (GitHub Actions, GitLab, etc.), pushes that break a workflow will be rejected automatically. - ---- - -#### One-liner helper (copy/paste) +#### One-liner helper ```bash git pull && cargo build --release && @@ -127,15 +115,19 @@ sudo install -Dm755 target/release/marlin /usr/local/bin/marlin && cargo test --test e2e -- --nocapture ``` -Stick that in a shell alias (`alias marlin-ci='…'`) and you’ve got a 5-second upgrade-and-verify loop. +Alias it as `marlin-ci` for a 5-second upgrade-and-verify loop. + +--- ### Database location -* **Linux** `~/.local/share/marlin/index.db` -* **macOS** `~/Library/Application Support/marlin/index.db` -* **Windows** `%APPDATA%\marlin\index.db` +| OS | Default path | +| ----------- | ----------------------------------------------- | +| **Linux** | `~/.local/share/marlin/index.db` | +| **macOS** | `~/Library/Application Support/marlin/index.db` | +| **Windows** | `%APPDATA%\marlin\index.db` | -Override with: +Override: ```bash export MARLIN_DB_PATH=/path/to/custom.db @@ -148,55 +140,57 @@ export MARLIN_DB_PATH=/path/to/custom.db ```text marlin [ARGS] -init create / migrate database -scan ... walk directories & index files -tag "" add hierarchical tag -attr set manage custom attributes -attr ls -search [--exec CMD] FTS5 query, optionally run CMD on each hit -backup create timestamped snapshot in backups/ -restore replace DB with snapshot -completions generate shell completions +init create / migrate DB **and perform an initial scan of the cwd** +scan ... walk directories & (re)index files +tag "" add hierarchical tag +attr set set or update custom attribute +attr ls list attributes +link add|rm|list|backlinks manage typed file-to-file relations +coll create|add|list manage named collections (“playlists”) +view save|list|exec save and run smart views (saved queries) +search [--exec CMD] FTS5 query; optionally run CMD per hit +backup create timestamped snapshot in `backups/` +restore replace DB with snapshot +completions generate shell completions ``` -### Attribute subcommands +### Attribute sub-commands -| Command | Example | -| ---------- | ---------------------------------------------- | -| `attr set` | `marlin attr set ~/Docs/**/*.pdf reviewed yes` | -| `attr ls` | `marlin attr ls ~/Docs/report.pdf` | +| Command | Example | +| ----------- | ------------------------------------------------ | +| `attr set` | `marlin attr set ~/Docs/**/*.pdf reviewed yes` | +| `attr ls` | `marlin attr ls ~/Docs/report.pdf` | +| JSON output | `marlin --format=json attr ls ~/Docs/report.pdf` | --- ## Backups & restore -**Create snapshot** - ```bash marlin backup # → ~/.local/share/marlin/backups/backup_2025-05-14_22-15-30.db ``` -**Restore snapshot** - ```bash marlin restore ~/.local/share/marlin/backups/backup_2025-05-14_22-15-30.db ``` -Marlin also takes an **automatic safety backup before every non-init command**. +> Marlin also creates an **automatic safety backup before every non-`init` command.** +> *Auto-prune (`backup --prune `) lands in Q2.* --- ## Upgrading ```bash -cargo install --path . --force # rebuild & replace installed binary +cargo install --path . --force # rebuild & replace installed binary ``` -The versioned migration system preserves your data across upgrades. +Versioned migrations preserve your data across upgrades. --- ## License -MIT – see `LICENSE` +MIT – see [`LICENSE`](LICENSE). + diff --git a/marlin_demo.md b/marlin_demo.md index 6b5b3ac..28c2b14 100644 --- a/marlin_demo.md +++ b/marlin_demo.md @@ -1,6 +1,8 @@ -# Marlin Demo +# Marlin Demo 🚀 -Below is the **“hello-world” demo** that matches the current master branch (auto-scan on `marlin init`, no more forced-migration noise, and cleaner build). +Below is a **“hello-world” walk-through** that matches the current `main` +branch (auto-scan on `marlin init`, no more forced-migration chatter, cleaner +build). Everything runs offline on a throw-away directory under `~/marlin_demo`. --- @@ -8,11 +10,11 @@ Below is the **“hello-world” demo** that matches the current master branch ( ```bash # inside the repo -cargo build --release # build the new binary +export CARGO_TARGET_DIR=target # <-- speeds up future builds (once) +cargo build --release # build the new binary sudo install -Dm755 target/release/marlin /usr/local/bin/marlin -``` - -*(`cargo install --path . --locked --force` works too if you prefer.)* +# (cargo install --path . --locked --force works too) +```` --- @@ -21,96 +23,74 @@ sudo install -Dm755 target/release/marlin /usr/local/bin/marlin ```bash rm -rf ~/marlin_demo mkdir -p ~/marlin_demo/{Projects/{Alpha,Beta,Gamma},Logs,Reports,Scripts,Media/Photos} +# (zsh users: quote the pattern or enable braceexpand first) -# Projects +# ── Projects ─────────────────────────────────────────────────── cat < ~/marlin_demo/Projects/Alpha/draft1.md # Alpha draft 1 - - [ ] TODO: outline architecture - [ ] TODO: write tests EOF - cat < ~/marlin_demo/Projects/Alpha/draft2.md # Alpha draft 2 - - [x] TODO: outline architecture - [ ] TODO: implement feature X EOF - cat < ~/marlin_demo/Projects/Beta/notes.md Beta meeting notes: - decided on roadmap -- ACTION: follow up with design team +- ACTION: follow-up with design team EOF - cat < ~/marlin_demo/Projects/Beta/final.md # Beta Final - All tasks complete. Ready to ship! EOF - cat < ~/marlin_demo/Projects/Gamma/TODO.txt Gamma tasks: - TODO: refactor module Y EOF -# Logs -echo "2025-05-15 12:00:00 INFO Starting app" > ~/marlin_demo/Logs/app.log -echo "2025-05-15 12:01:00 ERROR Oops, crash" >> ~/marlin_demo/Logs/app.log -echo "2025-05-15 00:00:00 INFO System check OK" > ~/marlin_demo/Logs/system.log +# ── Logs & Reports ───────────────────────────────────────────── +echo "2025-05-15 12:00:00 INFO Starting app" > ~/marlin_demo/Logs/app.log +echo "2025-05-15 12:01:00 ERROR Oops, crash" >> ~/marlin_demo/Logs/app.log +echo "2025-05-15 00:00:00 INFO System check OK" > ~/marlin_demo/Logs/system.log +printf "Q1 financials\n" > ~/marlin_demo/Reports/Q1_report.pdf -# Reports -printf "Q1 financials -" > ~/marlin_demo/Reports/Q1_report.pdf - -# Scripts +# ── Scripts & Media ──────────────────────────────────────────── cat <<'EOF' > ~/marlin_demo/Scripts/deploy.sh #!/usr/bin/env bash -echo "Deploying version $1..." +echo "Deploying version $1…" EOF chmod +x ~/marlin_demo/Scripts/deploy.sh - -# Media echo "JPEGDATA" > ~/marlin_demo/Media/Photos/event.jpg ``` -*(copy the file-creation block from your original instructions — nothing about the files needs to change)* - --- ## 2 Initialise **and** index (one step) -`marlin init` now performs a first-time scan of whatever directory you run it in. -So just: - ```bash -cd ~/marlin_demo # <-- important: run init from the folder you want indexed -marlin init +cd ~/marlin_demo # run init from the folder you want indexed +marlin init # • creates or migrates DB + # • runs *first* full scan of this directory ``` -That will: - -1. create/upgrade the DB, -2. run all migrations exactly once, -3. walk the current directory and ingest every file it finds. - -Need to add more paths later? Use `marlin scan ` exactly as before. +Add more directories later with `marlin scan `. --- ## 3 Tagging examples ```bash -# Tag all project markdown as “project/md” -marlin tag "~/marlin_demo/Projects/**/*.md" project/md +# Tag all project markdown as ‘project/md’ +marlin tag '~/marlin_demo/Projects/**/*.md' project/md # Tag your logs -marlin tag "~/marlin_demo/Logs/**/*.log" logs/app +marlin tag '~/marlin_demo/Logs/**/*.log' logs/app -# Tag everything under Projects/Beta as “project/beta” -marlin tag "~/marlin_demo/Projects/Beta/**/*" project/beta +# Tag everything under Beta as ‘project/beta’ +marlin tag '~/marlin_demo/Projects/Beta/**/*' project/beta ``` --- @@ -118,8 +98,8 @@ marlin tag "~/marlin_demo/Projects/Beta/**/*" project/beta ## 4 Set custom attributes ```bash -marlin attr set "~/marlin_demo/Projects/Beta/final.md" status complete -marlin attr set "~/marlin_demo/Reports/*.pdf" reviewed yes +marlin attr set '~/marlin_demo/Projects/Beta/final.md' status complete +marlin attr set '~/marlin_demo/Reports/*.pdf' reviewed yes ``` --- @@ -129,19 +109,19 @@ marlin attr set "~/marlin_demo/Reports/*.pdf" reviewed yes ```bash marlin search TODO marlin search tag:project/md -marlin search "tag:logs/app AND ERROR" -marlin search "attr:status=complete" -marlin search "attr:reviewed=yes AND pdf" -marlin search "attr:reviewed=yes" --exec 'xdg-open {}' +marlin search 'tag:logs/app AND ERROR' +marlin search 'attr:status=complete' +marlin search 'attr:reviewed=yes AND pdf' +marlin search 'attr:reviewed=yes' --exec 'xdg-open {}' +marlin --format=json search 'attr:status=complete' # machine-readable output ``` --- -## 6 JSON output & verbose mode +## 6 Verbose mode ```bash -marlin --format=json attr ls ~/marlin_demo/Projects/Beta/final.md -marlin --verbose scan ~/marlin_demo # re-scan to see debug logs +marlin --verbose scan ~/marlin_demo # watch debug logs stream by ``` --- @@ -150,25 +130,43 @@ marlin --verbose scan ~/marlin_demo # re-scan to see debug logs ```bash snap=$(marlin backup | awk '{print $NF}') -rm ~/.local/share/marlin/index.db # simulate disaster +rm ~/.local/share/marlin/index.db # simulate disaster marlin restore "$snap" -marlin search TODO # should still work +marlin search TODO # still works ``` +*(Reminder: Marlin also makes an **auto-backup** before every non-`init` +command, so manual snapshots are extra insurance.)* + --- ## 8 Linking demo ```bash touch ~/marlin_demo/foo.txt ~/marlin_demo/bar.txt -marlin scan ~/marlin_demo # index the new files +marlin scan ~/marlin_demo # index the new files foo=~/marlin_demo/foo.txt bar=~/marlin_demo/bar.txt -marlin link add "$foo" "$bar" # create link -marlin link list "$foo" # outgoing links from foo -marlin link backlinks "$bar" # incoming links to bar +marlin link add "$foo" "$bar" --type references # create typed link +marlin link list "$foo" # outgoing links from foo +marlin link backlinks "$bar" # incoming links to bar +``` + +--- + +## 9 Collections & smart views + +```bash +# Collection +marlin coll create SetA +marlin coll add SetA '~/marlin_demo/Projects/**/*.md' +marlin coll list SetA + +# Saved view (smart folder) +marlin view save tasks 'attr:status=complete OR TODO' +marlin view exec tasks ``` --- @@ -176,8 +174,10 @@ marlin link backlinks "$bar" # incoming links to bar ### Recap * `cargo build --release` + `sudo install …` is still the build path. -* **`cd` to the folder you want indexed and run `marlin init`** — first scan happens automatically. -* Subsequent scans (`marlin scan …`) are only needed for *new* directories you add later. -* No more “forcing reapplication of migration 4” banner and the unused-import warnings are gone. +* **`marlin init`** scans the **current working directory** on first run. +* Scan again only when you add *new* directories (`marlin scan …`). +* Auto-backups happen before every command; manual `marlin backup` gives you extra restore points. Happy organising! + +``` \ No newline at end of file diff --git a/roadmap.md b/roadmap.md index 4535881..c45e256 100644 --- a/roadmap.md +++ b/roadmap.md @@ -1,36 +1,59 @@ -Here’s a slimmed-down, re-organized roadmap that groups related work into bigger milestones, highlights key deliverables (including a “demo” command and grep-style context snippets), and stages integrations for maximal developer velocity: +# Marlin Roadmap 2025 → 2026 📜 -| Phase / Sprint | Timeline | Focus & Rationale | Key Deliverables | -| ---------------------------------------- | ---------------------- | --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | -| **Sprint α: Bedrock & Metadata Domains** | **2025-Q2 (now–6/01)** | Stabilize core schema, migrations, CI, and introduce the first metadata domains **with discoverability**. | - **CI:** `cargo test` + SQL migration dry-run coverage
- **Migrations:** | +This document outlines the **official delivery plan** for Marlin over the next four quarters. +Every work-item below is *time-boxed, testable,* and traceable back to an end-user benefit. -* `links(src,dst,type)` -* `collections(name)+collection_files` -* `views(name,query)`
- **CLI stubs** for `marlin link|unlink|list|backlinks`, `marlin coll`, `marlin view`
- **`marlin demo`** command that walks you through a mini-drive-by tutorial of link/coll/view flows | - \| **Epic 1: Scale & Reliability** | **2025-Q2** | Prototype “dirty-row” FTS to avoid per-row triggers, benchmark at 100 k files, and shore up CI for edge cases. | - Dirty-flag + `scan --dirty` reindex only changed rows
- Replace per-row triggers with periodic FTS rebuild
- End-to-end benchmarks on \~100 k files
- CI tests for dirty-scan edge-cases | - \| **Epic 2: Live Mode & Self-Pruning Backups** | **2025-Q2** | Automate continuous indexing & backup hygiene so Marlin “just works” in a real workspace. | - `marlin watch [dir]` via `notify` crate (create/modify/delete/rename)
- `backup --prune ` flag + post-scan auto-prune to latest N
- Daily/pr-merge auto-prune automation in CI | - \| **Phase 3: Content FTS & Annotations** | **2025-Q3** | Go beyond metadata—index file bodies, provide grep-style context, and add annotation support. | - New `files.content` column + migration
- Extend `files_fts` to include `content` and emit context snippets (`-C` style)
- `annotations` table + FTS triggers
- CLI: `marlin annotate add|list` | - \| **Phase 4: Versioning & Deduplication** | **2025-Q3** | Enable history, diffs, and duplicate detection with content hashing. | - Compute & store `files.hash` (SHA256)
- `scan --rehash` to refresh hashes
- CLI: `marlin version diff ` to show changes between revisions | - \| **Phase 5: Tag Aliases & Semantic Enhancements** | **2025-Q3** | Tame tag sprawl and seed AI-powered suggestions via embeddings. | - Enforce `canonical_id` on `tags`; CLI: `marlin tag alias add|ls|rm`
- `embeddings` table + `scan --embed` to generate vectors
- CLI: `marlin tag suggest`, `marlin summary `, `marlin similarity scan` | - \| **Phase 6: Search DSL v2 & Smart Views** | **2025-Q4** | Offer a robust query grammar and “virtual folders” powered by views. | - Swap ad-hoc parser for a `nom`-based grammar with `AND`, `OR`, parentheses, wildcards…
- CLI: `marlin view save|list|exec` with aliases and pagination | - \| **Phase 7: Structured Workflows** | **2025-Q4** | Unlock full task, state, reminder & event workflows directly on files. | - `templates` + `template_fields` + validation engine
- CLI: -* `marlin state set|transitions add|state log` -* `marlin task scan|task list` -* `marlin remind set ""` -* `marlin event add ""` + `marlin timeline` | - \| **Phase 8: Lightweight Integrations** | **2026-Q1** | Surface Marlin inside your editor/terminal before diving into a full GUI. | - **VSCode & terminal UI extension**: file-tree sidebar showing tags/attrs/links/annotations | - \| **Phase 9: Dolphin Sidebar Plugin (MVP)** | **2026-Q1** | Prototype a read-only Qt sidebar for Linux file managers—peek metadata without leaving your file browser. | - Qt plugin showing tags, attributes, links, and annotations alongside files | - \| **Phase 10: Full Edit UI & Multi-Device Sync** | **2026-Q2** | Ship an in-place metadata editor and optional sync layer for distributed workflows. | - Tag & view editors, task/reminder/event dialogs in GUI
- Choose/implement sync backend (rqlite, Litestream or custom) for optional read-only remote mounts | +> **Legend** +> ✅ = item added/clarified in the latest planning round +> Δ = new sub-deliverable (wasn’t in the previous version) --- -### Why this order? +## 1 Bird’s-eye Table -1. **Lock down core schema & domains** (links, collections, views) **with a “demo” helper** so users can explore right away. -2. **Scale & CI** unlocks safe indexing at volume, then -3. **Live mode & auto-prune** keep your index fresh without manual steps. -4. **Content FTS + annotations** builds on an efficient, reliable plumbing layer—you’ll love grep-style context snippets. -5. **Versioning & semantic layers** ride atop a stable full-text index and annotation system. -6. **Advanced queries & workflows** expand power users’ toolsets before branching into GUIs and sync. +| Phase / Sprint | Timeline | Focus & Rationale | Key Deliverables (Δ = new) | | | +| ----------------------------------------------- | ------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------ | +| **Sprint α – Bedrock & Metadata Domains** | **2025-Q2 (now → 6 Jun)** | Stabilise schema & CI; land first metadata domains with discoverability. | Δ CI: `cargo test` + SQL dry-run
Δ Unit tests (`determine_scan_root`, `escape_fts`)
Δ Coverage: e2e `attr --format=json`
Δ Refactor: move `naive_substring_search` to shared util
Migrations: `links`, `collections`, `views`
CLI stubs: `link`, `coll`, `view`
`marlin demo` walkthrough | | | +| **Epic 1 – Scale & Reliability** | 2025-Q2 | Keep scans fast; bullet-proof CI at 100 k files. | Δ Dirty-flag column + `scan --dirty`
Benchmarks: full vs dirty scan (100 k)
Replace per-row triggers with periodic rebuild
CI edge-case tests | | | +| **Epic 2 – Live Mode & Self-Pruning Backups** | 2025-Q2 | Continuous indexing & hygiene—Marlin “just works”. | Δ `marlin watch [dir]` (notify/FSEvents)
Δ `backup --prune ` + auto-prune post-scan
Daily / PR-merge prune in CI | | | +| **Phase 3 – Content FTS & Annotations** | 2025-Q3 | Index file bodies, grep-style context, inline notes. | `files.content` + migration
Extend `files_fts` (context snippets `-C`)
`annotations` table + triggers
CLI \`annotate add | list\` | | +| **Phase 4 – Versioning & Deduplication** | 2025-Q3 | History, diffs & duplicate detection. | `files.hash` (SHA-256)
`scan --rehash` refresh
CLI `version diff ` | | | +| **Phase 5 – Tag Aliases & Semantic Booster** | 2025-Q3 | Tame tag sprawl; seed AI-powered suggestions. | `canonical_id` on `tags`; CLI `tag alias …`
`embeddings` table + `scan --embed`
CLI `tag suggest`, `similarity scan`, `summary ` | | | +| **Phase 6 – Search DSL v2 & Smart Views** | 2025-Q4 | Robust grammar + virtual folders. | Replace parser with **`nom`** grammar (`AND`, `OR`, `()` …)
CLI \`view save | list | exec\` with aliases & paging | +| **Phase 7 – Structured Workflows** | 2025-Q4 | First-class task / state / reminder / event life-cycles. | ✅ State engine (`files.state`, `state_changes`)
CLI \`state set | transitions add | log`
✅ Task extractor (`tasks` table) + CLI
`templates`+ validation
CLI`remind …`, `event …`, `timeline\` | +| **Phase 8 – Lightweight Integrations** | 2026-Q1 | Surface Marlin in editors / terminal. | VS Code & TUI extension (tags / attrs / links / notes) | | | +| **Phase 9 – Dolphin Sidebar Plugin (MVP)** | 2026-Q1 | Read-only Qt sidebar for Linux file managers. | Qt plug-in: tags, attrs, links, annotations | | | +| **Phase 10 – Full Edit UI & Multi-Device Sync** | 2026-Q2 | In-place metadata editor & optional sync layer. | GUI editors (tags, views, tasks, reminders, events)
Pick/implement sync backend (rqlite, Litestream, …) | | | -This grouping ensures every new layer rests on a solid, tested foundation—maximizing both developer speed and user delight. +--- + +## 2 Narrative & Dependencies + +1. **Lock down core schema & demo** *(Sprint α).* + Developers get immediate feedback via the `marlin demo` command while CI ensures migrations never regress. + +2. **Scale & Live Mode** *(Epics 1-2).* + Dirty scanning, file-watching and auto-pruned backups guarantee snappy, hands-off operation even on six-figure corpora. + +3. **Richer Search** *(Phases 3-6).* + Body-content FTS + grep-style snippets lay the groundwork; `nom` grammar then elevates power-user queries and smart views. + +4. **Workflow Layers** *(Phase 7).* + State transitions, tasks and reminders turn Marlin from a passive index into an active workflow engine. + +5. **UX Expansions** *(Phases 8-10).* + Start lightweight (VS Code / TUI), graduate to a read-only Dolphin plug-in, then ship full editing & sync for multi-device teams. + +Every outer milestone depends only on the completion of the rows above it, **so shipping discipline in early sprints de-risks the headline features down the line.** + +--- + +## 3 Next Steps + +* **Sprint α kickoff:** break deliverables into stories, estimate, assign. +* **Add roadmap as `docs/ROADMAP.md`** (this file). +* Wire a **Checklist issue** on GitHub: one task per Δ bullet for instant tracking. + +--- + +*Last updated · 2025-05-16*