Update AGENTS.md and scripts for global dependency installation

- Revise instructions in AGENTS.md to clarify that Python scripts should be run via `startup.sh`, which now installs dependencies globally.
- Enhance project overview and command sections for better clarity on usage.
- Update coding standards to reflect Python version requirements and Bash script practices.
- Improve formatting and organization of command examples for better readability.
This commit is contained in:
thePR0M3TH3AN committed 2025-05-30 17:48:38 -04:00
1 parent 0b491c4416
commit a1cf3c888a
5 files changed
+118 -135

No files matched your search

+48 -27
View File
@@ -1,17 +1,18 @@
# AGENTS.md — Guidance for OpenAI Codex # AGENTS.md — Guidance for OpenAI Codex
## Execution Commands ## Execution Commands
- **Always** invoke Python scripts via the startup wrapper to activate dependencies: - **Always** invoke Python scripts via the startup wrapper, which now installs dependencies globally and then runs your code under the system Python:
```bash ```bash
./startup.sh --offset <months> --end <YYYY-MM-DD> --extend-years <N> ./startup.sh --offset <months> --end <YYYY-MM-DD> --extend-years <N>
````
## Project Overview ## Project Overview
This repository, **python\_charts**, provides a framework for fetching economic time series from FRED, storing them locally, and producing custom matplotlib charts with configurable parameters. The core components are: This repository, **python\_charts**, provides a framework for fetching economic time series from FRED, storing them locally, and producing custom matplotlib charts with configurable parameters. The core components are:
* **`startup.sh`**: Single-entrypoint script to bootstrap a Python venv, install dependencies, fetch data, and run specified commands under the venv interpreter. * **`startup.sh`**: Single‐entrypoint script to install dependencies globally, fetch data, and run specified commands under system Python.
* **`Makefile`**: Convenient high-level targets (`install`, `data`, `plot`, `plot-custom`, `test`, `clean`) that funnel through `startup.sh` for reproducible builds. * **`Makefile`**: Convenient high‐level targets (`install`, `data`, `plot`, `plot‐custom`, `test`, `clean`) that funnel through `startup.sh` for reproducible builds.
* **`scripts/refresh_data.py`**: Downloads UNRATE and DCOILWTICO series into `data/fred.db`. * **`scripts/refresh_data.py`**: Downloads UNRATE and DCOILWTICO series into `data/fred.db`.
* **`scripts/lagged_oil_unrate_chart_styled.py`**: Generates the lagged oil vs. unemployment chart with fixed axes styling and log scale. * **`scripts/lagged_oil_unrate_chart_styled.py`**: Generates the lagged oil vs. unemployment chart with fixed axes styling and log scale.
@@ -21,51 +22,71 @@ Future extensions will add additional custom chart scripts under `scripts/`, all
### Primary Commands ### Primary Commands
* **Bootstrap & default plot**: `./startup.sh` or `make plot` — runs the lagged-oil chart with an 18‑month offset. * **Bootstrap & default plot**:
* **Custom plot**: `./startup.sh --offset 12 --end 2025-05-31 --extend-years 5` or `make plot-custom ARGS="--offset 12 --end 2025-05-31 --extend-years 5"`.
* **Data refresh**: `./startup.sh scripts/refresh_data.py` or `make data`. ```bash
* **Run tests**: `./startup.sh pytest -q` or `make test`. ./startup.sh
```
Installs/upgrades dependencies globally and runs the default lagged‐oil chart with an 18-month offset.
* **Custom plot**:
```bash
./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
```
* **Data refresh**:
```bash
./startup.sh scripts/refresh_data.py
```
* **Run tests**:
```bash
./startup.sh pytest -q
```
### Argument Rules ### Argument Rules
* Arguments beginning with `--` are forwarded to the default chart script. * If the first token starts with `--`, it’s forwarded to the default chart script.
* If the first token is a `.py` script or executable, it is invoked under the venv’s Python. * If the first token names a `.py` script, it’s executed via `python <script>`.
* Otherwise, commands are executed directly (e.g. linters, test frameworks). * Otherwise, commands are executed directly (e.g., `pytest`, `flake8`).
## Coding Standards ## Coding Standards
* **Python**: Target version 3.12. Use type hints for all function signatures. Follow PEP8 with **Black** formatting. * **Python**: Target version ≥ 3.8. Use type hints for all function signatures. Follow PEP8 with Black formatting.
* **Bash**: `startup.sh` must use `set -euo pipefail` and always call `$VENV_DIR/bin/python` to avoid system‑Python mismatches. * **Bash**: `startup.sh` uses `set -euo pipefail`.
* **Makefile**: Use tab‑indented commands. All tasks should funnel to `startup.sh`. * **Makefile**: Use tab‐indented commands. All targets should funnel through `startup.sh`.
## Testing & Validation ## Testing & Validation
* **Automated tests**: No current tests; add pytest modules under `tests/` following the pattern `test_*.py`. * **Automated tests**: Add pytest modules under `tests/` following the pattern `test_*.py`.
* **Data mocking**: For new chart scripts, mock FRED downloads via monkeypatching `pandas_datareader.data.DataReader`. * **Data mocking**: For new chart scripts, mock FRED downloads via monkeypatching `pandas_datareader.data.DataReader`.
## Extending with New Charts ## Extending with New Charts
1. **Create a new script** under `scripts/`, for example `scripts/custom_chart.py`. 1. **Create** `scripts/custom_chart.py`.
2. **Implement** a `main()` accepting `--start`, `--end`, `--offset` or custom args using `argparse`. 2. **Implement** a `main()` with `argparse` for `--start`, `--end`, `--offset`, etc.
3. **Leverage** `startup.sh` invocation: add usage example in your script’s docstring. 3. **Invoke** via:
4. **Update** the `Makefile` if you need a dedicated target, otherwise users can call:
```bash ```bash
./startup.sh python scripts/custom_chart.py --your-args ./startup.sh python scripts/custom_chart.py --your-args
``` ```
4. **Optionally** add a Makefile target if desired.
## CI / Codex Integration ## CI / Codex Integration
* In Codex cloud or CLI tasks, set the **Setup Script** to `./startup.sh install data` to ensure dependencies and data are ready. * **Setup Script** →
* **Task Commands** should call `make plot` or `make test`, which delegates to `startup.sh`.
* **AGENTS.md** is automatically loaded by Codex to understand structure and commands.
## PR & Commit Guidelines ```bash
chmod +x startup.sh && ./startup.sh
```
* **Task Command** →
* Prefix branches/PRs with `feature/` or `fix/`. ```bash
* Commit messages: `<type>(<scope>): <description>` (e.g. `feat(charts): add custom inflation plot`). ./startup.sh --offset 18
* Include screenshots or saved chart outputs under `outputs/` for visual verification. ```
---
*End of AGENTS.md — tailored for python\_charts repo* *End of AGENTS.md — tailored for python\_charts repo*
+9 -12
View File
@@ -1,29 +1,26 @@
# Makefile for python_charts .PHONY: install data plot plot-custom test clean
# Defines standard targets for Codex/CI to use startup.sh
.PHONY: install data plot test clean # Install dependencies & fetch data
# Install dependencies into venv and prepare data
install: install:
./startup.sh pip install ./startup.sh
# Download FRED data (if needed) # Download FRED data (if needed)
data: data:
./startup.sh scripts/refresh_data.py ./startup.sh scripts/refresh_data.py
# Generate the default chart (18-month lag) # Default 18-month lag plot
plot: plot:
./startup.sh ./startup.sh
# Generate a custom chart # Custom plot
# Usage: make plot ARGS="--offset 12 --end 2025-05-31 --extend-years 5" # Usage: make plot-custom ARGS="--offset 12 --end 2025-05-31 --extend-years 5"
plot-custom: plot-custom:
./startup.sh $(ARGS) ./startup.sh $(ARGS)
# Run test suite # Run tests
test: test:
./startup.sh pytest -q ./startup.sh pytest -q
# Remove virtual environment and outputs # Clean up any outputs (no venv to delete)
clean: clean:
rm -rf venv data outputs rm -rf outputs
+38 -58
View File
@@ -1,12 +1,10 @@
# Lagged Oil–Unemployment Chart # Python Charts
A self-contained toolkit that: This initial chart is a self-contained toolkit that:
* Fetches U.S. unemployment rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED * Fetches U.S. unemployment rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED
* Stores the series in a local SQLite database (`data/fred.db`) * Stores the series in a local SQLite database (`data/fred.db`)
* Produces a dual-axis plot of unemployment vs. lagged oil price, with customizable lag, date range, and styling * Produces a dual-axis plot of unemployment vs. lagged oil price, with customizable lag, date range, and styling
![Example chart](scripts/lagged_oil_unrate_chart_styled.py)
--- ---
@@ -18,17 +16,19 @@ git clone https://github.com/your-org/python_charts.git
cd python_charts cd python_charts
# One-command: install deps, download data, and plot # One-command: install deps, download data, and plot
./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \ ./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
--offset 12 --end 2025-05-31 --extend-years 5 ````
```
This will: This will:
1. Create (or reuse) a Python 3 virtual environment (`venv/`) 1. Upgrade `pip` & `setuptools` globally
2. Upgrade `pip` & `setuptools` (providing `distutils` support) 2. Install required packages from `requirements.txt` via
3. Install required packages (offline wheelhouse first, PyPI fallback)
4. Download UNRATE & DCOILWTICO into `data/fred.db` if missing ```bash
5. Generate the chart with a 12-month oil lead, extending the x-axis by 5 years pip install --upgrade -r requirements.txt
```
3. Download UNRATE & DCOILWTICO into `data/fred.db` if missing
4. Generate the chart with a 12-month oil lead, extending the x-axis by 5 years
--- ---
@@ -37,46 +37,28 @@ This will:
``` ```
python_charts/ python_charts/
├── data/ ├── data/
│ └── fred.db # Local FRED snapshot (SQLite) │ └── fred.db # Local FRED snapshot (SQLite)
├── scripts/ ├── scripts/
│ ├── refresh_data.py # Download and store FRED series │ ├── refresh_data.py # Download and store FRED series
│ ├── lagged_oil_unrate_chart_styled.py # Plotting script │ ├── lagged_oil_unrate_chart_styled.py # Plotting script
│ └── pyproject.toml # Packaging metadata │ └── pyproject.toml # Packaging metadata
├── startup.sh # Bootstraps venv, deps, data, and runs commands ├── Makefile # High-level targets (install, plot, etc.)
├── requirements.txt # Python dependencies ├── startup.sh # Installs deps globally, fetches data, runs commands
└── README.md # This file ├── requirements.txt # Python dependencies
└── README.md # This file
``` ```
--- ---
## 🛠️ Installation & Setup
1. **Ensure** you have Python 3.8+ installed on your system.
2. **Clone** and navigate into the project:
```bash
git clone https://github.com/your-org/python_charts.git
cd python_charts
```
3. **Make** the startup script executable:
```bash
chmod +x startup.sh
```
> On first run, `startup.sh` will build the environment, install dependencies, and pull data.
---
## ⚙️ Usage ## ⚙️ Usage
### Refresh FRED data ### Refresh FRED data
```bash ```bash
./startup.sh python scripts/refresh_data.py ./startup.sh scripts/refresh_data.py
``` ```
Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db`. Downloads UNRATE and WTI series (from 1948 to today) into `data/fred.db`.
### Generate the chart ### Generate the chart
@@ -84,17 +66,16 @@ Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db
# Default 18-month lag # Default 18-month lag
./startup.sh ./startup.sh
# Custom lag, range, and extension # Custom lag, date range, and extension
./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \ ./startup.sh --offset 6 --start 1980-01-01 --end 2025-05-31 --extend-years 2
--offset 6 --start 1980-01-01 --end 2025-05-31 --extend-years 2
``` ```
| Option | Description | | Option | Description |
| ---------------- | --------------------------------------------------------- | | ---------------- | ----------------------------------------------- |
| `--offset` | Lag in months (positive = oil leads; negative = oil lags) | | `--offset` | Lag in months (pos = oil leads; neg = oil lags) |
| `--start` | Start date (YYYY-MM-DD) | | `--start` | Start date (YYYY-MM-DD) |
| `--end` | End date (YYYY-MM-DD) | | `--end` | End date (YYYY-MM-DD) |
| `--extend-years` | Years to extend the x-axis beyond the end date | | `--extend-years` | Years to extend the x-axis beyond the end date |
--- ---
@@ -105,7 +86,7 @@ Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db
* Left y-axis: unemployment rate (%) from 3 to 15%, ticks every 2%. * Left y-axis: unemployment rate (%) from 3 to 15%, ticks every 2%.
* Right y-axis: log scale oil price (USD), doubling ticks from \$3 upward. * Right y-axis: log scale oil price (USD), doubling ticks from \$3 upward.
* X-axis: years, major tick every 5 years, minor tick every year. * X-axis: major tick every 5 years, minor tick every year.
--- ---
@@ -121,18 +102,17 @@ lxml>=4.6.0
python-dateutil>=2.8.1 python-dateutil>=2.8.1
``` ```
`startup.sh` also upgrades `pip` & `setuptools` to ensure `distutils` is available.
--- ---
## 🛡️ Troubleshooting ## 🛡️ Troubleshooting
* **ModuleNotFoundError: distutils**: resolved automatically by upgrading `setuptools` in `startup.sh`. * **ModuleNotFoundError: distutils**: resolved by `pip install --upgrade setuptools`.
* **Database not found**: run `./startup.sh python scripts/refresh_data.py` to populate `data/fred.db`. * **Database not found**: run `./startup.sh scripts/refresh_data.py`.
* **Virtual-env issues**: delete `venv/` and re-run `startup.sh`. * **Permission errors**: ensure write access to the project directory.
--- ---
## 📄 License ## 📄 License
This project is released under the MIT License. See [LICENSE](LICENSE) for details. Released under the MIT License. See [LICENSE](LICENSE) for details.
+6
View File
@@ -25,6 +25,12 @@ Generated on: 2025-05-30
```markdown ```markdown
# AGENTS.md — Guidance for OpenAI Codex # AGENTS.md — Guidance for OpenAI Codex
## Execution Commands
- **Always** invoke Python scripts via the startup wrapper to activate dependencies:
```bash
./startup.sh --offset <months> --end <YYYY-MM-DD> --extend-years <N>
## Project Overview ## Project Overview
This repository, **python\_charts**, provides a framework for fetching economic time series from FRED, storing them locally, and producing custom matplotlib charts with configurable parameters. The core components are: This repository, **python\_charts**, provides a framework for fetching economic time series from FRED, storing them locally, and producing custom matplotlib charts with configurable parameters. The core components are:
+17 -38
View File
@@ -1,69 +1,48 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# ------------------------------------------------------------------- # -------------------------------------------------------------------
# startup.sh — prepare Python env, ensure deps & data, then run. # startup.sh — install deps globally, fetch data, then run command
# Usage: # Usage:
# ./startup.sh [options] # passes options to default plot script # ./startup.sh [options] # passes options to default plot script
# ./startup.sh [script or cmd] # runs specified script/command # ./startup.sh <script> [args] # runs specified script with system Python
# Examples: # Examples:
# ./startup.sh --offset 12 --end 2025-05-31 --extend-years 5 # ./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
# ./startup.sh pytest -q # ./startup.sh pytest -q
# ------------------------------------------------------------------- # -------------------------------------------------------------------
set -euo pipefail set -euo pipefail
VENV_DIR="venv"
REQ_FILE="requirements.txt" REQ_FILE="requirements.txt"
WHEELHOUSE="wheelhouse"
DATA_DB="data/fred.db" DATA_DB="data/fred.db"
PYTHON="$VENV_DIR/bin/python"
DEFAULT_SCRIPT="scripts/lagged_oil_unrate_chart_styled.py" DEFAULT_SCRIPT="scripts/lagged_oil_unrate_chart_styled.py"
# 1) Create or reuse venv # 1) Ensure pip & setuptools are up to date
if [[ ! -d "$VENV_DIR" ]]; then
echo "🔧 Creating virtual-env…"
python3 -m venv "$VENV_DIR"
fi
# 2) Activate venv
# shellcheck disable=SC1090
source "$VENV_DIR/bin/activate"
# 3) Upgrade pip & setuptools
echo "🛠 Upgrading pip & setuptools…" echo "🛠 Upgrading pip & setuptools…"
$PYTHON -m pip install --upgrade pip setuptools pip install --upgrade pip setuptools
# 4) Install dependencies # 2) Install project dependencies globally
PIP_OPTS=(install --upgrade --no-input) echo "📦 Installing dependencies from $REQ_FILE…"
if [[ -d "$WHEELHOUSE" ]]; then pip install --upgrade -r "$REQ_FILE"
echo "📦 Installing from wheelhouse…"
$PYTHON -m pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \
|| { echo "⚠️ Wheelhouse failed—falling back…"; \
$PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; }
else
echo "🌐 Installing from PyPI…"
$PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"
fi
echo "✅ Virtual-env ready ($($PYTHON -V))" # 3) Ensure data exists
# 5) Ensure data exists
if [[ ! -f "$DATA_DB" ]]; then if [[ ! -f "$DATA_DB" ]]; then
echo "🔄 Data not found—downloading FRED series…" echo "🔄 Data not found—downloading FRED series…"
$PYTHON scripts/refresh_data.py python scripts/refresh_data.py
fi fi
# 6) Build command # 4) Dispatch to the right command
if [[ $# -eq 0 ]]; then if [[ $# -eq 0 ]]; then
CMD=("$PYTHON" "$DEFAULT_SCRIPT" --offset 18) # no args → run default plot
CMD=(python "$DEFAULT_SCRIPT" --offset 18)
elif [[ "$1" == --* ]]; then elif [[ "$1" == --* ]]; then
CMD=("$PYTHON" "$DEFAULT_SCRIPT" "$@") # args start with -- → forward to default plot
CMD=(python "$DEFAULT_SCRIPT" "$@")
else else
# first token is script or other command
if [[ -f "$1" || "$1" == *.py ]]; then if [[ -f "$1" || "$1" == *.py ]]; then
CMD=("$PYTHON" "$@") CMD=(python "$@")
else else
CMD=("$@") CMD=("$@")
fi fi
fi fi
# 7) Execute
echo "🚀 Executing: ${CMD[*]}" echo "🚀 Executing: ${CMD[*]}"
exec "${CMD[@]}" exec "${CMD[@]}"