mirror of
https://github.com/PR0M3TH3AN/Marlin.git
synced 2025-09-08 07:08:44 +00:00
Clarify saved_views naming and add note on deferred tag aliases
* Rename `views` table references to `saved_views` for consistency. * Update migration step 3 to create `saved_views` instead of `views`. * Add a note that tag aliases (`canonical_id`) are deferred to DP-006 (v1.5). * Ensure all PRAGMA statements (`foreign_keys`, `journal_mode=WAL`) apply to every migration. * Adjust ER diagram section to reflect `saved_views` entity and its relationship.
This commit is contained in:
thePR0M3TH3AN
@@ -12,23 +12,34 @@ We’ve landed a basic SQLite-backed `files` table and a contentless FTS5 index.
|
||||
- **Custom attributes** (`attributes`)
|
||||
- **File-to-file relationships** (`links`)
|
||||
- **Named collections** (`collections` + `collection_files`)
|
||||
- **Saved views** (`views`)
|
||||
- **Saved views** (`saved_views`)
|
||||
|
||||
Locking this schema now lets downstream CLI & GUI work against a stable model and ensures our migrations stay easy to reason about.
|
||||
Locking this schema now lets downstream CLI & GUI work against a stable model and ensures our migrations stay easy to reason about.
|
||||
*Note: Tag aliases and their `canonical_id` support are deferred to DP-006 (v1.5).*
|
||||
|
||||
## 2. Decision
|
||||
|
||||
1. **Bump to schema version 1.1** in our migration table.
|
||||
Each migration will begin by enabling foreign-key enforcement and WAL journaling:
|
||||
|
||||
```sql
|
||||
PRAGMA foreign_keys = ON;
|
||||
PRAGMA journal_mode = WAL;
|
||||
````
|
||||
|
||||
All foreign keys use `ON DELETE CASCADE` so deleting a file, tag, etc. automatically cleans up dependents.
|
||||
|
||||
1. **Bump to schema version 1.1** in our `schema_version` table.
|
||||
2. Provide four migration scripts, applied in order:
|
||||
1. `0001_initial_schema.sql` – create `files`, `tags`, `file_tags`, `attributes`, `files_fts`, core FTS triggers.
|
||||
2. `0002_update_fts_and_triggers.sql` – replace old tag/attr FTS triggers with `INSERT OR REPLACE` semantics for full-row refresh.
|
||||
3. `0003_create_links_collections_views.sql` – introduce `links`, `collections`, `collection_files`, `views` tables.
|
||||
4. `0004_fix_hierarchical_tags_fts.sql` – refine FTS triggers to index full hierarchical tag-paths via a recursive CTE.
|
||||
|
||||
1. **0001\_initial\_schema.sql** – create core tables (`files`, `tags`, `file_tags`, `attributes`), a contentless FTS5 table (`files_fts`), core FTS triggers, and performance-critical indexes.
|
||||
2. **0002\_update\_fts\_and\_triggers.sql** – replace old tag/attr FTS triggers with `INSERT OR REPLACE` semantics for full-row refresh.
|
||||
3. **0003\_create\_links\_collections\_saved\_views.sql** – introduce `links`, `collections`, `collection_files`, and `saved_views` tables.
|
||||
4. **0004\_fix\_hierarchical\_tags\_fts.sql** – refine FTS triggers to index full hierarchical tag-paths via a recursive CTE.
|
||||
3. Expose this schema through our library (`libmarlin::db::open`) so any client sees a v1.1 store.
|
||||
|
||||
## 3. ER Diagram
|
||||
|
||||
Below is the updated entity-relationship diagram, expressed in PlantUML for clarity. It shows all of the core metadata domains and their relationships:
|
||||
Below is the updated entity-relationship diagram (PlantUML):
|
||||
|
||||
```plantuml
|
||||
@startuml
|
||||
@@ -42,11 +53,10 @@ entity files {
|
||||
}
|
||||
|
||||
entity tags {
|
||||
* id : INTEGER <<PK>>
|
||||
* id : INTEGER <<PK>>
|
||||
--
|
||||
name : TEXT
|
||||
parent_id : INTEGER <<FK>>
|
||||
canonical_id : INTEGER <<FK>>
|
||||
name : TEXT
|
||||
parent_id : INTEGER <<FK>>
|
||||
}
|
||||
|
||||
entity file_tags {
|
||||
@@ -63,7 +73,7 @@ entity attributes {
|
||||
}
|
||||
|
||||
entity links {
|
||||
* id : INTEGER <<PK>>
|
||||
* id : INTEGER <<PK>>
|
||||
--
|
||||
src_file_id : INTEGER <<FK>>
|
||||
dst_file_id : INTEGER <<FK>>
|
||||
@@ -81,7 +91,7 @@ entity collection_files {
|
||||
* file_id : INTEGER <<FK>>
|
||||
}
|
||||
|
||||
entity views {
|
||||
entity saved_views {
|
||||
* id : INTEGER <<PK>>
|
||||
--
|
||||
name : TEXT
|
||||
@@ -99,11 +109,11 @@ files ||--o{ links : "dst_file_id"
|
||||
collections ||--o{ collection_files
|
||||
files ||--o{ collection_files
|
||||
|
||||
views ||..|| files : "smart queries (via FTS)"
|
||||
saved_views ||..|| files : "exec via FTS"
|
||||
@enduml
|
||||
````
|
||||
```
|
||||
|
||||
*(If you prefer a plain‐ASCII sketch, you can replace the above PlantUML block with:)*
|
||||
Or in plain-ASCII:
|
||||
|
||||
```ascii
|
||||
┌────────┐ ┌────────────┐ ┌───────┐
|
||||
@@ -124,19 +134,28 @@ views ||..|| files : "smart queries (via FTS)"
|
||||
│ collections │1──*─│ collection_files │*──1─│ files │
|
||||
└─────────────┘ └──────────────────┘ └────────┘
|
||||
|
||||
┌───────┐
|
||||
│ views │
|
||||
└───────┘
|
||||
┌─────────────┐
|
||||
│ saved_views │
|
||||
│ (exec FTS) │
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
## 4. Migration Summary
|
||||
|
||||
| File | Purpose |
|
||||
| ----------------------------------------------- | ------------------------------------------------------- |
|
||||
| **0001\_initial\_schema.sql** | Core tables + contentless FTS + path/triggers |
|
||||
| **0002\_update\_fts\_and\_triggers.sql** | Full-row FTS refresh on tag/attr changes |
|
||||
| **0003\_create\_links\_collections\_views.sql** | Add `links`, `collections`, `collection_files`, `views` |
|
||||
| **0004\_fix\_hierarchical\_tags\_fts.sql** | Recursive CTE for full path tag indexing |
|
||||
| File | Purpose |
|
||||
| ------------------------------------------------------ | ------------------------------------------------------------- |
|
||||
| **0001\_initial\_schema.sql** | Core tables + contentless FTS + core triggers + indexes |
|
||||
| **0002\_update\_fts\_and\_triggers.sql** | Full-row FTS refresh on tag/attr changes |
|
||||
| **0003\_create\_links\_collections\_saved\_views.sql** | Add `links`, `collections`, `collection_files`, `saved_views` |
|
||||
| **0004\_fix\_hierarchical\_tags\_fts.sql** | Recursive CTE for full tag-path indexing in FTS triggers |
|
||||
|
||||
### Performance-Critical Indexes
|
||||
|
||||
* `idx_files_path` on `files(path)`
|
||||
* `idx_files_hash` on `files(hash)`
|
||||
* `idx_tags_name_parent` on `tags(name, parent_id)`
|
||||
* `idx_file_tags_tag_id` on `file_tags(tag_id)`
|
||||
* `idx_attr_file_key` on `attributes(file_id, key)`
|
||||
|
||||
## 5. Example CLI Session
|
||||
|
||||
@@ -156,6 +175,10 @@ Saved view 'tasks' = tag:project AND TODO
|
||||
|
||||
$ marlin view list
|
||||
tasks: tag:project AND TODO
|
||||
|
||||
$ marlin view exec tasks
|
||||
~/Projects/Alpha/draft1.md
|
||||
~/Projects/Beta/final.md
|
||||
```
|
||||
|
||||
## 6. Consequences
|
||||
@@ -163,6 +186,7 @@ tasks: tag:project AND TODO
|
||||
* **Backward compatibility**: older v1.0 stores will be migrated on first open.
|
||||
* **Stability**: downstream features (TUI, VS Code, web UI) can depend on a stable v1.1 schema.
|
||||
* **Simplicity**: by consolidating metadata domains now, future migrations remain small and focused.
|
||||
* **Performance**: v1.1 schema meets our cold-start P95 ≤ 3 s on a 100 k-file corpus (with CI-enforced benchmarks and the indexes above).
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user