# Repository Context

Generated on: 2025-05-30

## Directory Tree (Whitelist Only)

```
.
    ├── scripts/
        ├── lagged_oil_unrate_chart_styled.py
        ├── pyproject.toml
        ├── refresh_data.py
    ├── README.md
    ├── requirements.txt
    ├── startup.sh
```

## Important Files

## python_charts/README.md
```markdown
# Lagged Oil vs. Unemployment Charting

This repository provides two Python scripts to fetch the latest US Unemployment Rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED, apply a configurable lag or lead to the oil series, and generate polished dual‑axis charts.

## Files

* **`lagged_oil_unrate_chart.py`**
  A lightweight script that:

  1. Downloads monthly UNRATE and averages daily WTI to monthly.
  2. Shifts the oil series by `N` months (default = 18).
  3. Plots both series on a dual‑axis Matplotlib chart.

* **`lagged_oil_unrate_chart_styled.py`**
  A more polished version that adds:

  * Title, subtitle, footnote with date range and source.
  * Clean dual‑axis styling with percentage and dollar‑format ticks.
  * Customizable date range and extended x‑axis (e.g., 1973–2028).
  * CLI flags for offset, start/end dates, and axis extension.

* **`requirements.txt`**
  Lists all Python dependencies for both scripts.

## Prerequisites

* Python 3.7 or later
* Internet connection (to fetch data from FRED)

## Setup

Clone or download this repository:

```bash
git clone https://your-repo-url.git
cd your-repo-folder
```

### Create a Virtual Environment & Install Dependencies

#### On Linux / macOS

```bash
# 1. Create a venv
python3 -m venv venv

# 2. Activate it
source venv/bin/activate

# 3. Upgrade pip and install requirements
pip install --upgrade pip
pip install -r requirements.txt
```

#### On Windows (PowerShell)

```powershell
# 1. Create a venv
python -m venv venv

# 2. Activate it
venv\Scripts\Activate.ps1

# 3. Upgrade pip and install requirements
python -m pip install --upgrade pip
pip install -r requirements.txt
```

> **Note:** If you get execution policy errors on Windows, you may need to run:
>
> ```powershell
> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
> ```

## Usage

Once the virtual environment is active and dependencies are installed, run either script from the project root.

### 1. Basic Lag Script

```bash
# Default: oil lags unemployment by 18 months (~1.5 years)
python lagged_oil_unrate_chart.py

# Other offsets (in months)
python lagged_oil_unrate_chart.py --offset 6    # 6‑month lag
python lagged_oil_unrate_chart.py --offset -12  # 12‑month lead
```

### 2. Styled & Configurable Chart

```bash
python lagged_oil_unrate_chart_styled.py \
    --offset 18 \
    --start 1973-01-01 \
    --end 2025-05-31 \
    --extend-years 3
```

| Flag               | Description                                                    |
| ------------------ | -------------------------------------------------------------- |
| `--offset N`       | Lag (positive) or lead (negative) in months (default = 18)     |
| `--start DATE`     | Series start date (`YYYY-MM-DD`, default = `1973-01-01`)       |
| `--end DATE`       | Series end date (`YYYY-MM-DD`, default = today)                |
| `--extend-years M` | How many years beyond `end` to extend the x‑axis (default = 3) |

## Customization

* **Normalization**: Insert a normalization step (e.g., percent change from a base date).
* **Export**: Modify the script to save figures with `plt.savefig('chart.png', dpi=300)`.
* **Statistics**: Add `ta.correl()` or use Pandas to compute rolling correlations.

## Troubleshooting

* **"ModuleNotFoundError"**: Ensure your venv is activated, then reinstall dependencies.
* **SSL/DataReader errors**: Upgrade `pandas_datareader` or switch to an alternate data source.

## License

This project is released under the MIT License.

```

## python_charts/requirements.txt
```
pandas>=1.5.0
pandas_datareader>=0.10.0
matplotlib>=3.5.0
lxml>=4.6.0
python-dateutil>=2.8.1

```

## python_charts/startup.sh
```
#!/usr/bin/env bash
set -euo pipefail

python3 -m venv venv
source venv/bin/activate

pip install --no-index --find-links=wheelhouse -r requirements.txt

echo "✅  Virtual-env ready (offline install)."

```

## scripts/refresh_data.py
```python
#!/usr/bin/env python3
"""
refresh_data.py
---------------
Download UNRATE and DCOILWTICO from FRED and store them in SQLite
(data/fred.db).  Run on a connected machine; commit the DB afterwards.
"""

from datetime import date
import pathlib
import sqlite3

import pandas as pd
from pandas_datareader.data import DataReader

HERE = pathlib.Path(__file__).resolve().parent
DATA_PATH = HERE.parent / "data"
DB_FILE = DATA_PATH / "fred.db"

START = "1948-01-01"          # earliest UNRATE observation
END = date.today().isoformat()
SERIES = ("UNRATE", "DCOILWTICO")


def main() -> None:
    DATA_PATH.mkdir(exist_ok=True)
    with sqlite3.connect(DB_FILE) as conn:
        for s in SERIES:
            print(f"↯ downloading {s} …")
            df = DataReader(s, "fred", START, END)
            df.to_sql(s, conn, if_exists="replace", index_label="date")
            print(f"✓ wrote {s}: {len(df):,} rows")
    print(f"\nDone.  SQLite DB at {DB_FILE} ({DB_FILE.stat().st_size/1024:.0f} KB)")


if __name__ == "__main__":
    main()

```

## scripts/pyproject.toml
```
# pyproject.toml

[project]
name = "lagged-oil-unrate"
version = "0.1.0"
description = "Plot UNRATE vs WTI oil price with adjustable lag"
authors = ["Your Name <you@example.com>"]
dependencies = [
  "pandas>=1.3.0",
  "pandas_datareader>=0.10.0",
  "matplotlib>=3.4.0",
  "python-dateutil>=2.8.0",
  "lxml>=4.6.0"
]

```

## scripts/lagged_oil_unrate_chart_styled.py
```python
#!/usr/bin/env python3
"""
lagged_oil_unrate_chart_styled.py
---------------------------------
Offline version: reads UNRATE & WTI series from data/fred.db (populated
by scripts/refresh_data.py) and plots them with a configurable lag.

Usage
-----
  python scripts/lagged_oil_unrate_chart_styled.py \
      --offset 18 \
      --start 1973-01-01 \
      --end   2025-05-31 \
      --extend-years 3
"""

from __future__ import annotations

import argparse
from datetime import datetime
from pathlib import Path

import matplotlib.dates as mdates
import matplotlib.pyplot as plt
import pandas as pd
from dateutil.relativedelta import relativedelta
import sqlite3


# ──────────────────────────────────────────────────────────────────────────
# Data helpers
# ──────────────────────────────────────────────────────────────────────────
def fetch_series(
    start: datetime,
    end: datetime,
    db_path: str | Path = Path("data/fred.db"),
) -> tuple[pd.DataFrame, pd.DataFrame]:
    """
    Load UNRATE and DCOILWTICO from a local SQLite DB, then:

    * convert UNRATE to month-end timestamps
    * average WTI daily prices to month-end
    """
    db_path = Path(db_path)
    if not db_path.exists():
        raise FileNotFoundError(
            f"{db_path} not found.  Run scripts/refresh_data.py on a "
            "machine with internet, commit the new DB, then retry."
        )

    with sqlite3.connect(db_path) as conn:
        unrate = (
            pd.read_sql(
                "SELECT date, UNRATE AS value FROM UNRATE", conn,
                parse_dates=["date"], index_col="date"
            )
            .loc[start:end]
        )
        oil = (
            pd.read_sql(
                "SELECT date, DCOILWTICO AS value FROM DCOILWTICO", conn,
                parse_dates=["date"], index_col="date"
            )
            .loc[start:end]
        )

    # Resample WTI to month-end average
    oil_monthly = oil.resample("M").mean()
    # Ensure UNRATE is at month-end, too
    unrate.index = unrate.index.to_period("M").to_timestamp("M")
    return unrate, oil_monthly


# ──────────────────────────────────────────────────────────────────────────
# Plotting
# ──────────────────────────────────────────────────────────────────────────
def plot_lagged(
    unrate: pd.DataFrame,
    oil: pd.DataFrame,
    offset_months: int,
    start_date: datetime,
    end_date: datetime,
    extend_years: int,
) -> None:
    """Render the dual-axis chart."""
    oil_shifted = oil.shift(offset_months)

    fig, ax1 = plt.subplots(figsize=(14, 7))
    plt.title("Unemployment Rate and US Oil Price", fontsize=20, weight="bold")
    plt.suptitle(
        "Civilian Unemployment Rate and WTI Crude Oil Price",
        fontsize=14,
        y=0.93,
    )

    # Left axis – UNRATE
    ax1.plot(
        unrate.index,
        unrate["value"],
        label="UNRATE (LHS)",
        linewidth=2.5,
        color="#1f77b4",
    )
    ax1.set_ylabel("Unemployment rate (%)", fontsize=12)
    ax1.set_ylim(0, unrate["value"].max() * 1.1)
    ax1.yaxis.set_major_formatter("{x:.0f}%")
    ax1.grid(axis="y", linestyle="--", alpha=0.4)

    # Right axis – shifted oil
    ax2 = ax1.twinx()
    ax2.plot(
        oil_shifted.index,
        oil_shifted["value"],
        label="USOIL (RHS)",
        linewidth=2,
        linestyle="-",
        alpha=0.8,
        color="#FF9900",
    )
    ax2.set_ylabel("Oil price (USD)", fontsize=12)
    ax2.set_ylim(0, oil_shifted["value"].max() * 1.1)
    ax2.yaxis.set_major_formatter("${x:,.0f}")

    # X-axis
    ax1.set_xlim(start_date, end_date + relativedelta(years=extend_years))
    ax1.xaxis.set_major_locator(mdates.YearLocator(base=5))
    ax1.xaxis.set_minor_locator(mdates.YearLocator(1))
    ax1.xaxis.set_major_formatter(mdates.DateFormatter("%Y"))
    plt.setp(ax1.get_xticklabels(), rotation=0, fontsize=11)

    # Legends
    lines, labels = ax1.get_legend_handles_labels()
    l2, l2l = ax2.get_legend_handles_labels()
    ax1.legend(lines + l2, labels + l2l, loc="upper left", frameon=False, fontsize=12)

    # Footnote
    foot = (
        f"Dates: {start_date.strftime('%Y')} through "
        f"{unrate.index[-1].strftime('%d %b %Y')}.\n"
        "Source: Local FRED snapshot (offline)."
    )
    plt.annotate(
        foot,
        (0, 0),
        (0, -40),
        xycoords="axes fraction",
        textcoords="offset points",
        va="top",
        fontsize=10,
        color="gray",
    )

    fig.tight_layout(rect=[0, 0.03, 1, 0.95])
    plt.show()


# ──────────────────────────────────────────────────────────────────────────
# CLI
# ──────────────────────────────────────────────────────────────────────────
def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument(
        "-o",
        "--offset",
        type=int,
        default=18,
        help="lag in months (positive=lag, negative=lead)",
    )
    p.add_argument(
        "--start",
        type=str,
        default="1973-01-01",
        help="series start date (YYYY-MM-DD)",
    )
    p.add_argument(
        "--end",
        type=str,
        default=datetime.today().strftime("%Y-%m-%d"),
        help="series end date (YYYY-MM-DD)",
    )
    p.add_argument(
        "--extend-years",
        type=int,
        default=3,
        help="how many years beyond end_date to show on the x-axis",
    )
    p.add_argument(
        "--db",
        type=str,
        default="data/fred.db",
        help="path to local FRED SQLite file",
    )
    args = p.parse_args()

    start_dt = datetime.fromisoformat(args.start)
    end_dt = datetime.fromisoformat(args.end)

    unrate, oil = fetch_series(start_dt, end_dt, args.db)
    plot_lagged(unrate, oil, args.offset, start_dt, end_dt, args.extend_years)


if __name__ == "__main__":
    main()

```