mirror of
https://github.com/PR0M3TH3AN/python_charts.git
synced 2026-10-27 08:50:39 +00:00
Update AGENTS.md and scripts for global dependency installation
- Revise instructions in AGENTS.md to clarify that Python scripts should be run via `startup.sh`, which now installs dependencies globally. - Enhance project overview and command sections for better clarity on usage. - Update coding standards to reflect Python version requirements and Bash script practices. - Improve formatting and organization of command examples for better readability.
This commit is contained in:
thePR0M3TH3AN committed
1 parent
0b491c4416
commit
a1cf3c888a
5 files changed
+118
-135
No files matched your search
@@ -1,17 +1,18 @@
|
||||
# AGENTS.md — Guidance for OpenAI Codex
|
||||
|
||||
## Execution Commands
|
||||
- **Always** invoke Python scripts via the startup wrapper to activate dependencies:
|
||||
- **Always** invoke Python scripts via the startup wrapper, which now installs dependencies globally and then runs your code under the system Python:
|
||||
|
||||
```bash
|
||||
./startup.sh --offset <months> --end <YYYY-MM-DD> --extend-years <N>
|
||||
````
|
||||
|
||||
## Project Overview
|
||||
|
||||
This repository, **python\_charts**, provides a framework for fetching economic time series from FRED, storing them locally, and producing custom matplotlib charts with configurable parameters. The core components are:
|
||||
|
||||
* **`startup.sh`**: Single-entrypoint script to bootstrap a Python venv, install dependencies, fetch data, and run specified commands under the venv interpreter.
|
||||
* **`Makefile`**: Convenient high-level targets (`install`, `data`, `plot`, `plot-custom`, `test`, `clean`) that funnel through `startup.sh` for reproducible builds.
|
||||
* **`startup.sh`**: Single‐entrypoint script to install dependencies globally, fetch data, and run specified commands under system Python.
|
||||
* **`Makefile`**: Convenient high‐level targets (`install`, `data`, `plot`, `plot‐custom`, `test`, `clean`) that funnel through `startup.sh` for reproducible builds.
|
||||
* **`scripts/refresh_data.py`**: Downloads UNRATE and DCOILWTICO series into `data/fred.db`.
|
||||
* **`scripts/lagged_oil_unrate_chart_styled.py`**: Generates the lagged oil vs. unemployment chart with fixed axes styling and log scale.
|
||||
|
||||
@@ -21,51 +22,71 @@ Future extensions will add additional custom chart scripts under `scripts/`, all
|
||||
|
||||
### Primary Commands
|
||||
|
||||
* **Bootstrap & default plot**: `./startup.sh` or `make plot` — runs the lagged-oil chart with an 18‑month offset.
|
||||
* **Custom plot**: `./startup.sh --offset 12 --end 2025-05-31 --extend-years 5` or `make plot-custom ARGS="--offset 12 --end 2025-05-31 --extend-years 5"`.
|
||||
* **Data refresh**: `./startup.sh scripts/refresh_data.py` or `make data`.
|
||||
* **Run tests**: `./startup.sh pytest -q` or `make test`.
|
||||
* **Bootstrap & default plot**:
|
||||
|
||||
```bash
|
||||
./startup.sh
|
||||
```
|
||||
|
||||
Installs/upgrades dependencies globally and runs the default lagged‐oil chart with an 18-month offset.
|
||||
|
||||
* **Custom plot**:
|
||||
|
||||
```bash
|
||||
./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
|
||||
```
|
||||
|
||||
* **Data refresh**:
|
||||
|
||||
```bash
|
||||
./startup.sh scripts/refresh_data.py
|
||||
```
|
||||
|
||||
* **Run tests**:
|
||||
|
||||
```bash
|
||||
./startup.sh pytest -q
|
||||
```
|
||||
|
||||
### Argument Rules
|
||||
|
||||
* Arguments beginning with `--` are forwarded to the default chart script.
|
||||
* If the first token is a `.py` script or executable, it is invoked under the venv’s Python.
|
||||
* Otherwise, commands are executed directly (e.g. linters, test frameworks).
|
||||
* If the first token starts with `--`, it’s forwarded to the default chart script.
|
||||
* If the first token names a `.py` script, it’s executed via `python <script>`.
|
||||
* Otherwise, commands are executed directly (e.g., `pytest`, `flake8`).
|
||||
|
||||
## Coding Standards
|
||||
|
||||
* **Python**: Target version 3.12. Use type hints for all function signatures. Follow PEP8 with **Black** formatting.
|
||||
* **Bash**: `startup.sh` must use `set -euo pipefail` and always call `$VENV_DIR/bin/python` to avoid system‑Python mismatches.
|
||||
* **Makefile**: Use tab‑indented commands. All tasks should funnel to `startup.sh`.
|
||||
* **Python**: Target version ≥ 3.8. Use type hints for all function signatures. Follow PEP8 with Black formatting.
|
||||
* **Bash**: `startup.sh` uses `set -euo pipefail`.
|
||||
* **Makefile**: Use tab‐indented commands. All targets should funnel through `startup.sh`.
|
||||
|
||||
## Testing & Validation
|
||||
|
||||
* **Automated tests**: No current tests; add pytest modules under `tests/` following the pattern `test_*.py`.
|
||||
* **Automated tests**: Add pytest modules under `tests/` following the pattern `test_*.py`.
|
||||
* **Data mocking**: For new chart scripts, mock FRED downloads via monkeypatching `pandas_datareader.data.DataReader`.
|
||||
|
||||
## Extending with New Charts
|
||||
|
||||
1. **Create a new script** under `scripts/`, for example `scripts/custom_chart.py`.
|
||||
2. **Implement** a `main()` accepting `--start`, `--end`, `--offset` or custom args using `argparse`.
|
||||
3. **Leverage** `startup.sh` invocation: add usage example in your script’s docstring.
|
||||
4. **Update** the `Makefile` if you need a dedicated target, otherwise users can call:
|
||||
1. **Create** `scripts/custom_chart.py`.
|
||||
2. **Implement** a `main()` with `argparse` for `--start`, `--end`, `--offset`, etc.
|
||||
3. **Invoke** via:
|
||||
|
||||
```bash
|
||||
./startup.sh python scripts/custom_chart.py --your-args
|
||||
```
|
||||
4. **Optionally** add a Makefile target if desired.
|
||||
|
||||
## CI / Codex Integration
|
||||
|
||||
* In Codex cloud or CLI tasks, set the **Setup Script** to `./startup.sh install data` to ensure dependencies and data are ready.
|
||||
* **Task Commands** should call `make plot` or `make test`, which delegates to `startup.sh`.
|
||||
* **AGENTS.md** is automatically loaded by Codex to understand structure and commands.
|
||||
* **Setup Script** →
|
||||
|
||||
## PR & Commit Guidelines
|
||||
```bash
|
||||
chmod +x startup.sh && ./startup.sh
|
||||
```
|
||||
* **Task Command** →
|
||||
|
||||
* Prefix branches/PRs with `feature/` or `fix/`.
|
||||
* Commit messages: `<type>(<scope>): <description>` (e.g. `feat(charts): add custom inflation plot`).
|
||||
* Include screenshots or saved chart outputs under `outputs/` for visual verification.
|
||||
|
||||
---
|
||||
```bash
|
||||
./startup.sh --offset 18
|
||||
```
|
||||
|
||||
*End of AGENTS.md — tailored for python\_charts repo*
|
||||
@@ -1,29 +1,26 @@
|
||||
# Makefile for python_charts
|
||||
# Defines standard targets for Codex/CI to use startup.sh
|
||||
.PHONY: install data plot plot-custom test clean
|
||||
|
||||
.PHONY: install data plot test clean
|
||||
|
||||
# Install dependencies into venv and prepare data
|
||||
# Install dependencies & fetch data
|
||||
install:
|
||||
./startup.sh pip install
|
||||
./startup.sh
|
||||
|
||||
# Download FRED data (if needed)
|
||||
data:
|
||||
./startup.sh scripts/refresh_data.py
|
||||
|
||||
# Generate the default chart (18-month lag)
|
||||
# Default 18-month lag plot
|
||||
plot:
|
||||
./startup.sh
|
||||
|
||||
# Generate a custom chart
|
||||
# Usage: make plot ARGS="--offset 12 --end 2025-05-31 --extend-years 5"
|
||||
# Custom plot
|
||||
# Usage: make plot-custom ARGS="--offset 12 --end 2025-05-31 --extend-years 5"
|
||||
plot-custom:
|
||||
./startup.sh $(ARGS)
|
||||
|
||||
# Run test suite
|
||||
# Run tests
|
||||
test:
|
||||
./startup.sh pytest -q
|
||||
|
||||
# Remove virtual environment and outputs
|
||||
# Clean up any outputs (no venv to delete)
|
||||
clean:
|
||||
rm -rf venv data outputs
|
||||
rm -rf outputs
|
||||
@@ -1,12 +1,10 @@
|
||||
# Lagged Oil–Unemployment Chart
|
||||
# Python Charts
|
||||
|
||||
A self-contained toolkit that:
|
||||
This initial chart is a self-contained toolkit that:
|
||||
|
||||
* Fetches U.S. unemployment rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED
|
||||
* Stores the series in a local SQLite database (`data/fred.db`)
|
||||
* Produces a dual-axis plot of unemployment vs. lagged oil price, with customizable lag, date range, and styling
|
||||
|
||||
|
||||
* Fetches U.S. unemployment rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED
|
||||
* Stores the series in a local SQLite database (`data/fred.db`)
|
||||
* Produces a dual-axis plot of unemployment vs. lagged oil price, with customizable lag, date range, and styling
|
||||
|
||||
---
|
||||
|
||||
@@ -18,17 +16,19 @@ git clone https://github.com/your-org/python_charts.git
|
||||
cd python_charts
|
||||
|
||||
# One-command: install deps, download data, and plot
|
||||
./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \
|
||||
--offset 12 --end 2025-05-31 --extend-years 5
|
||||
```
|
||||
./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
|
||||
````
|
||||
|
||||
This will:
|
||||
|
||||
1. Create (or reuse) a Python 3 virtual environment (`venv/`)
|
||||
2. Upgrade `pip` & `setuptools` (providing `distutils` support)
|
||||
3. Install required packages (offline wheelhouse first, PyPI fallback)
|
||||
4. Download UNRATE & DCOILWTICO into `data/fred.db` if missing
|
||||
5. Generate the chart with a 12-month oil lead, extending the x-axis by 5 years
|
||||
1. Upgrade `pip` & `setuptools` globally
|
||||
2. Install required packages from `requirements.txt` via
|
||||
|
||||
```bash
|
||||
pip install --upgrade -r requirements.txt
|
||||
```
|
||||
3. Download UNRATE & DCOILWTICO into `data/fred.db` if missing
|
||||
4. Generate the chart with a 12-month oil lead, extending the x-axis by 5 years
|
||||
|
||||
---
|
||||
|
||||
@@ -37,46 +37,28 @@ This will:
|
||||
```
|
||||
python_charts/
|
||||
├── data/
|
||||
│ └── fred.db # Local FRED snapshot (SQLite)
|
||||
│ └── fred.db # Local FRED snapshot (SQLite)
|
||||
├── scripts/
|
||||
│ ├── refresh_data.py # Download and store FRED series
|
||||
│ ├── refresh_data.py # Download and store FRED series
|
||||
│ ├── lagged_oil_unrate_chart_styled.py # Plotting script
|
||||
│ └── pyproject.toml # Packaging metadata
|
||||
├── startup.sh # Bootstraps venv, deps, data, and runs commands
|
||||
├── requirements.txt # Python dependencies
|
||||
└── README.md # This file
|
||||
│ └── pyproject.toml # Packaging metadata
|
||||
├── Makefile # High-level targets (install, plot, etc.)
|
||||
├── startup.sh # Installs deps globally, fetches data, runs commands
|
||||
├── requirements.txt # Python dependencies
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Installation & Setup
|
||||
|
||||
1. **Ensure** you have Python 3.8+ installed on your system.
|
||||
2. **Clone** and navigate into the project:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/your-org/python_charts.git
|
||||
cd python_charts
|
||||
```
|
||||
3. **Make** the startup script executable:
|
||||
|
||||
```bash
|
||||
chmod +x startup.sh
|
||||
```
|
||||
|
||||
> On first run, `startup.sh` will build the environment, install dependencies, and pull data.
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Usage
|
||||
|
||||
### Refresh FRED data
|
||||
|
||||
```bash
|
||||
./startup.sh python scripts/refresh_data.py
|
||||
./startup.sh scripts/refresh_data.py
|
||||
```
|
||||
|
||||
Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db`.
|
||||
Downloads UNRATE and WTI series (from 1948 to today) into `data/fred.db`.
|
||||
|
||||
### Generate the chart
|
||||
|
||||
@@ -84,17 +66,16 @@ Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db
|
||||
# Default 18-month lag
|
||||
./startup.sh
|
||||
|
||||
# Custom lag, range, and extension
|
||||
./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \
|
||||
--offset 6 --start 1980-01-01 --end 2025-05-31 --extend-years 2
|
||||
# Custom lag, date range, and extension
|
||||
./startup.sh --offset 6 --start 1980-01-01 --end 2025-05-31 --extend-years 2
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
| ---------------- | --------------------------------------------------------- |
|
||||
| `--offset` | Lag in months (positive = oil leads; negative = oil lags) |
|
||||
| `--start` | Start date (YYYY-MM-DD) |
|
||||
| `--end` | End date (YYYY-MM-DD) |
|
||||
| `--extend-years` | Years to extend the x-axis beyond the end date |
|
||||
| Option | Description |
|
||||
| ---------------- | ----------------------------------------------- |
|
||||
| `--offset` | Lag in months (pos = oil leads; neg = oil lags) |
|
||||
| `--start` | Start date (YYYY-MM-DD) |
|
||||
| `--end` | End date (YYYY-MM-DD) |
|
||||
| `--extend-years` | Years to extend the x-axis beyond the end date |
|
||||
|
||||
---
|
||||
|
||||
@@ -105,7 +86,7 @@ Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db
|
||||
|
||||
* Left y-axis: unemployment rate (%) from 3 to 15%, ticks every 2%.
|
||||
* Right y-axis: log scale oil price (USD), doubling ticks from \$3 upward.
|
||||
* X-axis: years, major tick every 5 years, minor tick every year.
|
||||
* X-axis: major tick every 5 years, minor tick every year.
|
||||
|
||||
---
|
||||
|
||||
@@ -121,18 +102,17 @@ lxml>=4.6.0
|
||||
python-dateutil>=2.8.1
|
||||
```
|
||||
|
||||
`startup.sh` also upgrades `pip` & `setuptools` to ensure `distutils` is available.
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ Troubleshooting
|
||||
|
||||
* **ModuleNotFoundError: distutils**: resolved automatically by upgrading `setuptools` in `startup.sh`.
|
||||
* **Database not found**: run `./startup.sh python scripts/refresh_data.py` to populate `data/fred.db`.
|
||||
* **Virtual-env issues**: delete `venv/` and re-run `startup.sh`.
|
||||
* **ModuleNotFoundError: distutils**: resolved by `pip install --upgrade setuptools`.
|
||||
* **Database not found**: run `./startup.sh scripts/refresh_data.py`.
|
||||
* **Permission errors**: ensure write access to the project directory.
|
||||
|
||||
---
|
||||
|
||||
## 📄 License
|
||||
|
||||
This project is released under the MIT License. See [LICENSE](LICENSE) for details.
|
||||
Released under the MIT License. See [LICENSE](LICENSE) for details.
|
||||
|
||||
@@ -25,6 +25,12 @@ Generated on: 2025-05-30
|
||||
```markdown
|
||||
# AGENTS.md — Guidance for OpenAI Codex
|
||||
|
||||
## Execution Commands
|
||||
- **Always** invoke Python scripts via the startup wrapper to activate dependencies:
|
||||
|
||||
```bash
|
||||
./startup.sh --offset <months> --end <YYYY-MM-DD> --extend-years <N>
|
||||
|
||||
## Project Overview
|
||||
|
||||
This repository, **python\_charts**, provides a framework for fetching economic time series from FRED, storing them locally, and producing custom matplotlib charts with configurable parameters. The core components are:
|
||||
|
||||
+17
-38
@@ -1,69 +1,48 @@
|
||||
#!/usr/bin/env bash
|
||||
# -------------------------------------------------------------------
|
||||
# startup.sh — prepare Python env, ensure deps & data, then run.
|
||||
# startup.sh — install deps globally, fetch data, then run command
|
||||
# Usage:
|
||||
# ./startup.sh [options] # passes options to default plot script
|
||||
# ./startup.sh [script or cmd] # runs specified script/command
|
||||
# ./startup.sh [options] # passes options to default plot script
|
||||
# ./startup.sh <script> [args] # runs specified script with system Python
|
||||
# Examples:
|
||||
# ./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
|
||||
# ./startup.sh pytest -q
|
||||
# -------------------------------------------------------------------
|
||||
set -euo pipefail
|
||||
|
||||
VENV_DIR="venv"
|
||||
REQ_FILE="requirements.txt"
|
||||
WHEELHOUSE="wheelhouse"
|
||||
DATA_DB="data/fred.db"
|
||||
PYTHON="$VENV_DIR/bin/python"
|
||||
DEFAULT_SCRIPT="scripts/lagged_oil_unrate_chart_styled.py"
|
||||
|
||||
# 1) Create or reuse venv
|
||||
if [[ ! -d "$VENV_DIR" ]]; then
|
||||
echo "🔧 Creating virtual-env…"
|
||||
python3 -m venv "$VENV_DIR"
|
||||
fi
|
||||
|
||||
# 2) Activate venv
|
||||
# shellcheck disable=SC1090
|
||||
source "$VENV_DIR/bin/activate"
|
||||
|
||||
# 3) Upgrade pip & setuptools
|
||||
# 1) Ensure pip & setuptools are up to date
|
||||
echo "🛠 Upgrading pip & setuptools…"
|
||||
$PYTHON -m pip install --upgrade pip setuptools
|
||||
pip install --upgrade pip setuptools
|
||||
|
||||
# 4) Install dependencies
|
||||
PIP_OPTS=(install --upgrade --no-input)
|
||||
if [[ -d "$WHEELHOUSE" ]]; then
|
||||
echo "📦 Installing from wheelhouse…"
|
||||
$PYTHON -m pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \
|
||||
|| { echo "⚠️ Wheelhouse failed—falling back…"; \
|
||||
$PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; }
|
||||
else
|
||||
echo "🌐 Installing from PyPI…"
|
||||
$PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"
|
||||
fi
|
||||
# 2) Install project dependencies globally
|
||||
echo "📦 Installing dependencies from $REQ_FILE…"
|
||||
pip install --upgrade -r "$REQ_FILE"
|
||||
|
||||
echo "✅ Virtual-env ready ($($PYTHON -V))"
|
||||
|
||||
# 5) Ensure data exists
|
||||
# 3) Ensure data exists
|
||||
if [[ ! -f "$DATA_DB" ]]; then
|
||||
echo "🔄 Data not found—downloading FRED series…"
|
||||
$PYTHON scripts/refresh_data.py
|
||||
python scripts/refresh_data.py
|
||||
fi
|
||||
|
||||
# 6) Build command
|
||||
# 4) Dispatch to the right command
|
||||
if [[ $# -eq 0 ]]; then
|
||||
CMD=("$PYTHON" "$DEFAULT_SCRIPT" --offset 18)
|
||||
# no args → run default plot
|
||||
CMD=(python "$DEFAULT_SCRIPT" --offset 18)
|
||||
elif [[ "$1" == --* ]]; then
|
||||
CMD=("$PYTHON" "$DEFAULT_SCRIPT" "$@")
|
||||
# args start with -- → forward to default plot
|
||||
CMD=(python "$DEFAULT_SCRIPT" "$@")
|
||||
else
|
||||
# first token is script or other command
|
||||
if [[ -f "$1" || "$1" == *.py ]]; then
|
||||
CMD=("$PYTHON" "$@")
|
||||
CMD=(python "$@")
|
||||
else
|
||||
CMD=("$@")
|
||||
fi
|
||||
fi
|
||||
|
||||
# 7) Execute
|
||||
echo "🚀 Executing: ${CMD[*]}"
|
||||
exec "${CMD[@]}"
|
||||
Reference in new issue
Block a user