diff --git a/.gitignore b/.gitignore index ad0b2e2..57dd0ef 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ wheelhouse/*.log wheelhouse/*.md # keep wheel files themselves! data/*.bak +repo-context.txt diff --git a/repo-context.txt b/repo-context.txt index 22118d1..a037143 100644 --- a/repo-context.txt +++ b/repo-context.txt @@ -21,8 +21,144 @@ Generated on: 2025-05-30 ## python_charts/README.md ```markdown +# Lagged Oil–Unemployment Chart + +A self-contained toolkit that: + +* Fetches U.S. unemployment rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED +* Stores the series in a local SQLite database (`data/fred.db`) +* Produces a dual-axis plot of unemployment vs. lagged oil price, with customizable lag, date range, and styling + +![Example chart](scripts/lagged_oil_unrate_chart_styled.py) + +--- + +## 🚀 Quickstart + +```bash +# Clone the repo +git clone https://github.com/your-org/python_charts.git +cd python_charts + +# One-command: install deps, download data, and plot ./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \ - --offset 12 --end 2025-05-31 --extend-years 5 + --offset 12 --end 2025-05-31 --extend-years 5 +``` + +This will: + +1. Create (or reuse) a Python 3 virtual environment (`venv/`) +2. Upgrade `pip` & `setuptools` (providing `distutils` support) +3. Install required packages (offline wheelhouse first, PyPI fallback) +4. Download UNRATE & DCOILWTICO into `data/fred.db` if missing +5. Generate the chart with a 12-month oil lead, extending the x-axis by 5 years + +--- + +## 📁 Repository Structure + +``` +python_charts/ +├── data/ +│ └── fred.db # Local FRED snapshot (SQLite) +├── scripts/ +│ ├── refresh_data.py # Download and store FRED series +│ ├── lagged_oil_unrate_chart_styled.py # Plotting script +│ └── pyproject.toml # Packaging metadata +├── startup.sh # Bootstraps venv, deps, data, and runs commands +├── requirements.txt # Python dependencies +└── README.md # This file +``` + +--- + +## 🛠️ Installation & Setup + +1. **Ensure** you have Python 3.8+ installed on your system. +2. **Clone** and navigate into the project: + + ```bash + git clone https://github.com/your-org/python_charts.git + cd python_charts + ``` +3. **Make** the startup script executable: + + ```bash + chmod +x startup.sh + ``` + +> On first run, `startup.sh` will build the environment, install dependencies, and pull data. + +--- + +## ⚙️ Usage + +### Refresh FRED data + +```bash +./startup.sh python scripts/refresh_data.py +``` + +Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db`. + +### Generate the chart + +```bash +# Default 18-month lag +./startup.sh + +# Custom lag, range, and extension +./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \ + --offset 6 --start 1980-01-01 --end 2025-05-31 --extend-years 2 +``` + +| Option | Description | +| ---------------- | --------------------------------------------------------- | +| `--offset` | Lag in months (positive = oil leads; negative = oil lags) | +| `--start` | Start date (YYYY-MM-DD) | +| `--end` | End date (YYYY-MM-DD) | +| `--extend-years` | Years to extend the x-axis beyond the end date | + +--- + +## 📝 Details + +* **Data storage**: `data/fred.db` (SQLite) holds two tables: `UNRATE` and `DCOILWTICO`. +* **Plot styling**: + + * Left y-axis: unemployment rate (%) from 3 to 15%, ticks every 2%. + * Right y-axis: log scale oil price (USD), doubling ticks from \$3 upward. + * X-axis: years, major tick every 5 years, minor tick every year. + +--- + +## 📦 Dependencies + +Listed in `requirements.txt`: + +```text +pandas>=1.5.0 +pandas_datareader>=0.10.0 +matplotlib>=3.5.0 +lxml>=4.6.0 +python-dateutil>=2.8.1 +``` + +`startup.sh` also upgrades `pip` & `setuptools` to ensure `distutils` is available. + +--- + +## 🛡️ Troubleshooting + +* **ModuleNotFoundError: distutils**: resolved automatically by upgrading `setuptools` in `startup.sh`. +* **Database not found**: run `./startup.sh python scripts/refresh_data.py` to populate `data/fred.db`. +* **Virtual-env issues**: delete `venv/` and re-run `startup.sh`. + +--- + +## 📄 License + +This project is released under the MIT License. See [LICENSE](LICENSE) for details. ``` diff --git a/startup.sh b/startup.sh index 05b1229..91981e2 100755 --- a/startup.sh +++ b/startup.sh @@ -1,14 +1,12 @@ #!/usr/bin/env bash # ------------------------------------------------------------------- -# startup.sh — prepare Python env, ensure dependencies, FRED data, -# and run a command inside the venv +# startup.sh — prepare Python env, ensure deps & data, then run. # Usage: -# ./startup.sh [command…] -# Automatically: -# - Upgrades pip & setuptools (ensuring distutils on Python 3.12+) -# - Creates venv if missing -# - Installs requirements -# - Downloads data/fred.db if missing +# ./startup.sh [options] # passes options to default plot script +# ./startup.sh [script or cmd] # runs specified script/command +# Examples: +# ./startup.sh --offset 12 --end 2025-05-31 --extend-years 5 +# ./startup.sh pytest -q # ------------------------------------------------------------------- set -euo pipefail @@ -16,46 +14,56 @@ VENV_DIR="venv" REQ_FILE="requirements.txt" WHEELHOUSE="wheelhouse" DATA_DB="data/fred.db" +PYTHON="$VENV_DIR/bin/python" +DEFAULT_SCRIPT="scripts/lagged_oil_unrate_chart_styled.py" # 1) Create or reuse venv if [[ ! -d "$VENV_DIR" ]]; then echo "🔧 Creating virtual-env…" python3 -m venv "$VENV_DIR" fi -# Activate + +# 2) Activate venv # shellcheck disable=SC1090 source "$VENV_DIR/bin/activate" -# 2) Upgrade pip & setuptools (to provide distutils on Py3.12+) +# 3) Upgrade pip & setuptools echo "🛠 Upgrading pip & setuptools…" -pip install --upgrade pip setuptools +$PYTHON -m pip install --upgrade pip setuptools -# 3) Install project dependencies +# 4) Install dependencies PIP_OPTS=(install --upgrade --no-input) if [[ -d "$WHEELHOUSE" ]]; then echo "📦 Installing from wheelhouse…" - pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \ - || { echo "⚠️ Wheelhouse failed—falling back to PyPI…"; \ - pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; } + $PYTHON -m pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \ + || { echo "⚠️ Wheelhouse failed—falling back…"; \ + $PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; } else echo "🌐 Installing from PyPI…" - pip "${PIP_OPTS[@]}" -r "$REQ_FILE" + $PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE" fi -echo "✅ Virtual-env ready ($(python -V))" +echo "✅ Virtual-env ready ($($PYTHON -V))" -# 4) Ensure data/fred.db exists +# 5) Ensure data exists if [[ ! -f "$DATA_DB" ]]; then - echo "🔄 $DATA_DB not found—downloading FRED series…" - python scripts/refresh_data.py + echo "🔄 Data not found—downloading FRED series…" + $PYTHON scripts/refresh_data.py fi -# 5) Decide what to run -if [[ $# -gt 0 ]]; then - CMD=("$@") +# 6) Build command +if [[ $# -eq 0 ]]; then + CMD=("$PYTHON" "$DEFAULT_SCRIPT" --offset 18) +elif [[ "$1" == --* ]]; then + CMD=("$PYTHON" "$DEFAULT_SCRIPT" "$@") else - CMD=(python scripts/lagged_oil_unrate_chart_styled.py --offset 18) + if [[ -f "$1" || "$1" == *.py ]]; then + CMD=("$PYTHON" "$@") + else + CMD=("$@") + fi fi +# 7) Execute echo "🚀 Executing: ${CMD[*]}" -exec "${CMD[@]}" \ No newline at end of file +exec "${CMD[@]}"