diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e69de29..07bef3b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -0,0 +1,103 @@ +name: CI + +on: + push: + branches: [ main ] + pull_request: + branches: [ main ] + +jobs: + build-and-test: + name: Build & Test + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v3 + + - name: Install Rust (stable) + uses: actions-rs/toolchain@v1 + with: + toolchain: stable + override: true + + - name: Build (release) + run: cargo build --workspace --release + + - name: Run tests + run: cargo test --all -- --nocapture + + coverage: + name: Code Coverage (Tarpaulin) + needs: build-and-test + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v3 + + - name: Install Rust (nightly) + uses: actions-rs/toolchain@v1 + with: + toolchain: nightly + override: true + + - name: Install tarpaulin prerequisites + run: | + rustup component add llvm-tools-preview + + - name: Install cargo-tarpaulin + run: cargo install cargo-tarpaulin + + - name: Run coverage + run: cargo tarpaulin --workspace --out Xml --fail-under 85 + + # Optionally upload the coverage report as an artifact: + #- name: Upload coverage report + # uses: actions/upload-artifact@v3 + # with: + # name: coverage-report + # path: tarpaulin-report.xml + + benchmark: + name: Performance Benchmark (Hyperfine) + needs: build-and-test + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v3 + + - name: Install Rust (stable) + uses: actions-rs/toolchain@v1 + with: + toolchain: stable + override: true + + - name: Install benchmarking tools + run: | + sudo apt-get update + sudo apt-get install -y hyperfine jq bc + + - name: Build release binary + run: cargo build --release + + - name: Run cold-start benchmark + run: | + # measure cold start init latency + hyperfine \ + --warmup 3 \ + --export-json perf.json \ + 'target/release/marlin init' + + - name: Enforce P95 ≤ 3s + run: | + p95=$(jq '.results[0].percentiles["95.00"]' perf.json) + echo "P95 init latency: ${p95}s" + if (( $(echo "$p95 > 3.0" | bc -l) )); then + echo "::error ::Performance threshold exceeded (P95 > 3.0s)" + exit 1 + fi + # Optionally upload perf.json + #- name: Upload benchmark results + # uses: actions/upload-artifact@v3 + # with: + # name: perf-results + # path: perf.json diff --git a/docs/adr/DP-001_schema_v1.1.md b/docs/adr/DP-001_schema_v1.1.md new file mode 100644 index 0000000..6ad0e81 --- /dev/null +++ b/docs/adr/DP-001_schema_v1.1.md @@ -0,0 +1,169 @@ +# DP-001: Schema v1.1 – Core Metadata Domains + +**Status**: Proposed +**Authors**: @carol +**Date**: 2025-05-17 + +## 1. Context + +We’ve landed a basic SQLite-backed `files` table and a contentless FTS5 index. Before we build out higher-level features, we need to lock down our **v1.1** metadata schema for: + +- **Hierarchical tags** (`tags` + `file_tags`) +- **Custom attributes** (`attributes`) +- **File-to-file relationships** (`links`) +- **Named collections** (`collections` + `collection_files`) +- **Saved views** (`views`) + +Locking this schema now lets downstream CLI & GUI work against a stable model and ensures our migrations stay easy to reason about. + +## 2. Decision + +1. **Bump to schema version 1.1** in our migration table. +2. Provide four migration scripts, applied in order: + 1. `0001_initial_schema.sql` – create `files`, `tags`, `file_tags`, `attributes`, `files_fts`, core FTS triggers. + 2. `0002_update_fts_and_triggers.sql` – replace old tag/attr FTS triggers with `INSERT OR REPLACE` semantics for full-row refresh. + 3. `0003_create_links_collections_views.sql` – introduce `links`, `collections`, `collection_files`, `views` tables. + 4. `0004_fix_hierarchical_tags_fts.sql` – refine FTS triggers to index full hierarchical tag-paths via a recursive CTE. +3. Expose this schema through our library (`libmarlin::db::open`) so any client sees a v1.1 store. + +## 3. ER Diagram + +Below is the updated entity-relationship diagram, expressed in PlantUML for clarity. It shows all of the core metadata domains and their relationships: + +```plantuml +@startuml +entity files { + * id : INTEGER <> + -- + path : TEXT + size : INTEGER + mtime : INTEGER + hash : TEXT +} + +entity tags { + * id : INTEGER <> + -- + name : TEXT + parent_id : INTEGER <> + canonical_id : INTEGER <> +} + +entity file_tags { + * file_id : INTEGER <> + * tag_id : INTEGER <> +} + +entity attributes { + * id : INTEGER <> + -- + file_id : INTEGER <> + key : TEXT + value : TEXT +} + +entity links { + * id : INTEGER <> + -- + src_file_id : INTEGER <> + dst_file_id : INTEGER <> + type : TEXT +} + +entity collections { + * id : INTEGER <> + -- + name : TEXT +} + +entity collection_files { + * collection_id : INTEGER <> + * file_id : INTEGER <> +} + +entity views { + * id : INTEGER <> + -- + name : TEXT + query : TEXT +} + +files ||--o{ file_tags +tags ||--o{ file_tags + +files ||--o{ attributes + +files ||--o{ links : "src_file_id" +files ||--o{ links : "dst_file_id" + +collections ||--o{ collection_files +files ||--o{ collection_files + +views ||..|| files : "smart queries (via FTS)" +@enduml +```` + +*(If you prefer a plain‐ASCII sketch, you can replace the above PlantUML block with:)* + +```ascii +┌────────┐ ┌────────────┐ ┌───────┐ +│ files │1────*──│ file_tags │*────1─│ tags │ +└────────┘ └────────────┘ └───────┘ + │ + │1 + * +┌────────────┐ +│ attributes │ +└────────────┘ + +┌────────┐ ┌────────┐ ┌────────┐ +│ files │1──*──│ links │*───1──│ files │ +└────────┘ └────────┘ └────────┘ + +┌─────────────┐ ┌──────────────────┐ ┌────────┐ +│ collections │1──*─│ collection_files │*──1─│ files │ +└─────────────┘ └──────────────────┘ └────────┘ + +┌───────┐ +│ views │ +└───────┘ +``` + +## 4. Migration Summary + +| File | Purpose | +| ----------------------------------------------- | ------------------------------------------------------- | +| **0001\_initial\_schema.sql** | Core tables + contentless FTS + path/triggers | +| **0002\_update\_fts\_and\_triggers.sql** | Full-row FTS refresh on tag/attr changes | +| **0003\_create\_links\_collections\_views.sql** | Add `links`, `collections`, `collection_files`, `views` | +| **0004\_fix\_hierarchical\_tags\_fts.sql** | Recursive CTE for full path tag indexing | + +## 5. Example CLI Session + +```bash +$ marlin init +Database initialised at ~/.local/share/marlin/index_*.db +Initial scan complete – indexed/updated 42 files + +$ marlin link add ./todo.md ./projects/plan.md +Linked './todo.md' → './projects/plan.md' + +$ marlin coll create "MyDocs" +Created collection 'MyDocs' + +$ marlin view save tasks "tag:project AND TODO" +Saved view 'tasks' = tag:project AND TODO + +$ marlin view list +tasks: tag:project AND TODO +``` + +## 6. Consequences + +* **Backward compatibility**: older v1.0 stores will be migrated on first open. +* **Stability**: downstream features (TUI, VS Code, web UI) can depend on a stable v1.1 schema. +* **Simplicity**: by consolidating metadata domains now, future migrations remain small and focused. + +--- + +*End of DP-001* \ No newline at end of file