thePR0M3TH3AN/Marlin
1
0
Fork 0
mirror of https://github.com/PR0M3TH3AN/Marlin.git synced 2025-09-08 15:18:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
thePR0M3TH3AN
2025-05-14 16:21:00 -04:00
parent e63be17f9a
commit ebe9fa6e5c
6 changed files with 122 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

160
README.md
View File

@@ -1,16 +1,17 @@
# Marlin # Marlin
**Marlin** is a lightweight, metadata-driven file indexer you run on your own machine. **Marlin** is a lightweight, metadata-driven file indexer you run on your own
It scans folders, stores paths and basic stats in a local SQLite database, and lets you tag machine. It scans folders, stores paths and basic stats in a local SQLite
files from the command line. Nothing leaves your computer. database, and lets you tag files from the command line.
The goal is to build up toward a full “smart” file explorer (hierarchical tags, custom Nothing leaves your computer.
attributes, search, sync, etc.). This repo contains the **Sprint 0** foundation:
* XDG-aware config — no hard-coded paths This repo contains the **Sprint-0 foundation**:
* Embedded SQLite migrations (WAL mode)
* Fast directory scanner * XDG-aware config — no hard-coded paths
* Simple tagging tool * Embedded SQLite migrations (WAL mode)
* Fast directory scanner (now accepts *multiple* paths in one call)
* Simple tagging tool
* Human-readable logging via `tracing` * Human-readable logging via `tracing`
--- ---
@@ -18,19 +19,20 @@ attributes, search, sync, etc.). This repo contains the **Sprint 0** foundation:
## How it works ## How it works
```text ```text
┌──────────────┐ scan ┌─────────────┐ ┌──────────────┐ scan dir(s) ┌─────────────┐
│ your files │ ───────────────▶│ SQLite │ │ your files │ ───────────────▶│ SQLite │
└──────────────┘ │ index.db │ └──────────────┘ │ index.db │
▲ tag <pattern> <tag> │ files tags │ ▲ tag <glob> <tag> │ files tags │
└────────────────────────┴─────────────┘ └────────────────────────┴─────────────┘
``` ````
1. `marlin scan <dir>` walks the directory tree with `walkdir`, gathers size and 1. `marlin scan <PATHS>...` walks each directory tree, gathers size and
modification time, then upserts rows into `files`. modification time, then upserts rows into **`files`**.
2. `marlin tag "<glob>" <tag>` resolves the glob, looks up each file row, and inserts 2. `marlin tag "<glob>" <tag>` looks up each matching file row and inserts
junction rows into `file_tags`. New tag names are created on the fly. junction rows into **`file_tags`**. New tag names are created on the fly.
3. You can query the database yourself (e.g. with `sqlite3 ~/.local/share/marlin/index.db`) 3. You can open the DB yourself
while higher-level search commands are being built. (`sqlite3 ~/.local/share/marlin/index.db`) while search and GUI features
are still under construction.
--- ---
@@ -41,42 +43,43 @@ attributes, search, sync, etc.). This repo contains the **Sprint 0** foundation:
| **Rust** ≥ 1.77 | Build toolchain (`rustup.rs`) | | **Rust** ≥ 1.77 | Build toolchain (`rustup.rs`) |
| Build essentials | `gcc`, `make`, etc. for `rusqlite`’s bundled SQLite | | Build essentials | `gcc`, `make`, etc. for `rusqlite`’s bundled SQLite |
<details><summary>Platform notes</summary>
### Windows ### Windows
Rust installs MSVC build tools automatically. `rustup-init.exe` installs MSVC build tools automatically.
SQLite is compiled from source; nothing else to set up.
### macOS ### macOS
Install Xcode Command-Line Tools:
```bash ```bash
xcode-select --install xcode-select --install # command-line tools
``` ```
### Linux ### Linux (Debian / Ubuntu)
Deb-/RPM-based distros:
```bash ```bash
sudo apt install build-essential # or sudo apt install build-essential
sudo dnf groupinstall 'Development Tools'
``` ```
or on Fedora / RHEL
```bash
sudo dnf groupinstall "Development Tools"
```
</details>
--- ---
## Build & install ## Build & install
Clone then build in release mode:
```bash ```bash
git clone https://github.com/yourname/marlin.git git clone https://github.com/yourname/marlin.git
cd marlin cd marlin
cargo build --release cargo build --release # produces target/release/marlin
``` ```
The binary is placed at `target/release/marlin`. Copy the release binary somewhere on your `PATH` (optional):
Feel free to copy it into a directory on your `PATH`, e.g.:
```bash ```bash
sudo install -Dm755 target/release/marlin /usr/local/bin/marlin sudo install -Dm755 target/release/marlin /usr/local/bin/marlin
@@ -87,25 +90,25 @@ sudo install -Dm755 target/release/marlin /usr/local/bin/marlin
## Quick start ## Quick start
```bash ```bash
# 1 - create the database (idempotent) # 1 – create or upgrade the database (idempotent)
marlin init marlin init
# 2 - index a folder # 2 – index all common folders in one shot
marlin scan ~/Pictures marlin scan ~/Pictures ~/Documents ~/Downloads ~/Music ~/Videos
# 3 - add a tag to matching files # 3 – add a tag to matching files
marlin tag "~/Pictures/**/*.jpg" vacation marlin tag "~/Pictures/**/*.jpg" vacation
``` ```
The database defaults to: The database path defaults to:
``` ```
~/.local/share/marlin/index.db # Linux ~/.local/share/marlin/index.db # Linux
~/Library/Application Support/marlin # macOS ~/Library/Application Support/marlin # macOS
%APPDATA%\\marlin\\index.db # Windows %APPDATA%\marlin\index.db # Windows
``` ```
Override with an environment variable: Override with:
```bash ```bash
export MARLIN_DB_PATH=/path/to/custom.db export MARLIN_DB_PATH=/path/to/custom.db
@@ -120,44 +123,79 @@ USAGE:
marlin <COMMAND> [ARGS] marlin <COMMAND> [ARGS]
COMMANDS: COMMANDS:
init Create the SQLite database and run migrations init Create (or upgrade) the SQLite database
scan <path> Walk a directory recursively and index all files found scan <PATHS>... Walk one or more directories recursively
tag <glob> <tag> Apply <tag> to files matched by <glob> tag "<glob>" <tag> Apply <tag> to all files matched
FLAGS: FLAGS:
-h, --help Show this help -h, --help Show this help
-V, --version Show version info -V, --version Show version info
``` ```
### Details | Command | Notes |
| --------------------------- | ----------------------------------------------------------------------------------------------------- |
| `marlin init` | Safe to run repeatedly; applies pending migrations. |
| `marlin scan <PATHS>...` | Accepts any number of absolute/relative paths. Directories you can’t read are skipped with a warning. |
| `marlin tag "<glob>" <tag>` | Quote the glob so your shell doesn’t expand it. Uses `glob` crate rules (`**` for recursive matches). |
| Command | Arguments / Notes | ---
| --------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| `marlin init` | Safe to run more than once; upgrades the DB in place if schema changes. | ## Upgrading to a new build
| `marlin scan <path>` | Accepts absolute or relative paths. Ignores directories it can’t read. |
| `marlin tag "<glob>" <tag>` | Use quotes if your shell would otherwise expand the glob. Wildcards follow `glob` crate rules (`**` for recursive). | During development you’ll be editing source files frequently. Two common ways
to run the updated program:
### 1. Run straight from the project directory
```bash
cargo run --release -- scan ~/Pictures
```
*Cargo recompiles what changed and runs the fresh binary located in
`target/release/marlin`.*
### 2. Replace the global copy
If you previously installed Marlin (e.g. into `~/.cargo/bin/` or `/usr/local/bin/`),
overwrite it:
```bash
cargo install --path . --force
```
Now `which marlin` should print the new location, and multi-path scan works:
```bash
marlin scan ~/Pictures ~/Documents …
```
If the CLI still shows the old single-path usage (`Usage: marlin scan <PATH>`),
you’re invoking an outdated executable—check your `PATH` and reinstall.
--- ---
## Development tips ## Development tips
* `RUST_LOG=debug marlin scan /some/dir` shows per-file indexing messages. * Tight loop: `cargo watch -x 'run -- scan ~/Pictures'`
* The integration database for tests lives in `/tmp` and is wiped automatically. * Debug logs: `RUST_LOG=debug marlin scan ~/Pictures`
* Run `cargo clippy --all-targets --all-features -D warnings` before opening a PR. * Lint: `cargo clippy --all-targets --all-features -D warnings`
* Tests: `cargo test`
--- ---
## Roadmap ## Roadmap
| Milestone | What’s coming next | | Milestone | Coming soon |
| --------- | ---------------------------------------------------------- | | --------- | --------------------------------------------------------------- |
| **M1** | Hierarchical tags, attributes table, virtual `tags://` URI | | **M1** | Hierarchical tags • attributes table • `tags://` virtual folder |
| **M2** | Sync service, change log, diff viewer | | **M2** | Sync service • change log • diff viewer |
| **M3** | Natural-language search, visual query builder | | **M3** | Natural-language search • visual query builder |
| **M4** | Plug-in marketplace, mobile companion (view-only) | | **M4** | Plug-in marketplace • mobile companion (view-only) |
--- ---
## License ## License
This project is licensed under the **MIT License**. See `LICENSE` for details. Released under the **MIT License** – see `LICENSE` for full text.

14
src/cli.rs
View File

@@ -14,12 +14,20 @@ pub struct Cli {
pub enum Commands { pub enum Commands {
/// Initialise the database (idempotent) /// Initialise the database (idempotent)
Init, Init,
/// Scan a directory and populate the file index
/// Scan one or more directories and populate the file index
///
/// Example:
/// marlin scan ~/Pictures ~/Documents ~/Downloads
Scan { Scan {
/// Directory to walk /// One or more directories to walk
path: PathBuf, paths: Vec<PathBuf>,
}, },
/// Tag files matching a glob pattern /// Tag files matching a glob pattern
///
/// Example:
/// marlin tag "~/Pictures/**/*.jpg" vacation
Tag { Tag {
/// Glob pattern (quote to avoid shell expansion) /// Glob pattern (quote to avoid shell expansion)
pattern: String, pattern: String,

17
src/main.rs
View File

@@ -5,7 +5,7 @@ mod logging;
mod scan; mod scan;
use anyhow::Result; use anyhow::Result;
use clap::Parser; // 👈 bring in the trait that adds `.parse()` use clap::Parser;
use cli::{Cli, Commands}; use cli::{Cli, Commands};
use glob::glob; use glob::glob;
use rusqlite::params; use rusqlite::params;
@@ -14,17 +14,24 @@ use tracing::{error, info};
fn main() -> Result<()> { fn main() -> Result<()> {
logging::init(); logging::init();
let args = Cli::parse(); // now compiles let args = Cli::parse();
let cfg = config::Config::load()?; let cfg = config::Config::load()?;
let mut conn = db::open(&cfg.db_path)?; // mutable let mut conn = db::open(&cfg.db_path)?;
match args.command { match args.command {
Commands::Init => { Commands::Init => {
info!("database initialised at {}", cfg.db_path.display()); info!("database initialised at {}", cfg.db_path.display());
} }
Commands::Scan { path } => {
scan::scan_directory(&mut conn, &path)?; // pass &mut Commands::Scan { paths } => {
if paths.is_empty() {
anyhow::bail!("At least one directory must be supplied to `scan`");
}
for path in paths {
scan::scan_directory(&mut conn, &path)?;
}
} }
Commands::Tag { pattern, tag } => { Commands::Tag { pattern, tag } => {
apply_tag(&conn, &pattern, &tag)?; apply_tag(&conn, &pattern, &tag)?;
} }

BIN
target/release/.fingerprint/marlin-634839a5e9cc4921/dep-bin-marlin
View File

Binary file not shown.

BIN
target/release/deps/marlin-634839a5e9cc4921
View File

Binary file not shown.

BIN
target/release/marlin
View File

Binary file not shown.