Update .gitignore, enhance repo-context documentation, and refine startup script

- Add `repo-context.txt` to .gitignore to prevent tracking of generated context files. - Expand documentation in `repo-context.txt` to include detailed usage instructions, quickstart guide, and repository structure for the lagged oil-unemployment chart project. - Refine `startup.sh` script comments for clarity on its purpose and usage.
2026-10-27 08:50:39 +00:00 · 2025-05-30 17:04:12 -04:00
1 parent 8797ef1d04
commit 3c097cc847
@@ -7,3 +7,4 @@ wheelhouse/*.log
 wheelhouse/*.md
 # keep wheel files themselves!
 data/*.bak
+repo-context.txt
@@ -21,8 +21,144 @@ Generated on: 2025-05-30

 ## python_charts/README.md
 ```markdown
+# Lagged Oil–Unemployment Chart
+
+A self-contained toolkit that:
+
+* Fetches U.S. unemployment rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED
+* Stores the series in a local SQLite database (`data/fred.db`)
+* Produces a dual-axis plot of unemployment vs. lagged oil price, with customizable lag, date range, and styling
+
+![Example chart](scripts/lagged_oil_unrate_chart_styled.py)
+
+---
+
+## 🚀 Quickstart
+
+```bash
+# Clone the repo
+git clone https://github.com/your-org/python_charts.git
+cd python_charts
+
+# One-command: install deps, download data, and plot
 ./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \
-  --offset 12 --end 2025-05-31 --extend-years 5
+    --offset 12 --end 2025-05-31 --extend-years 5
+```
+
+This will:
+
+1. Create (or reuse) a Python 3 virtual environment (`venv/`)
+2. Upgrade `pip` & `setuptools` (providing `distutils` support)
+3. Install required packages (offline wheelhouse first, PyPI fallback)
+4. Download UNRATE & DCOILWTICO into `data/fred.db` if missing
+5. Generate the chart with a 12-month oil lead, extending the x-axis by 5 years
+
+---
+
+## 📁 Repository Structure
+
+```
+python_charts/
+├── data/
+│   └── fred.db                 # Local FRED snapshot (SQLite)
+├── scripts/
+│   ├── refresh_data.py         # Download and store FRED series
+│   ├── lagged_oil_unrate_chart_styled.py  # Plotting script
+│   └── pyproject.toml          # Packaging metadata
+├── startup.sh                  # Bootstraps venv, deps, data, and runs commands
+├── requirements.txt            # Python dependencies
+└── README.md                   # This file
+```
+
+---
+
+## 🛠️ Installation & Setup
+
+1. **Ensure** you have Python 3.8+ installed on your system.
+2. **Clone** and navigate into the project:
+
+   ```bash
+   git clone https://github.com/your-org/python_charts.git
+   cd python_charts
+   ```
+3. **Make** the startup script executable:
+
+   ```bash
+   chmod +x startup.sh
+   ```
+
+> On first run, `startup.sh` will build the environment, install dependencies, and pull data.
+
+---
+
+## ⚙️ Usage
+
+### Refresh FRED data
+
+```bash
+./startup.sh python scripts/refresh_data.py
+```
+
+Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db`.
+
+### Generate the chart
+
+```bash
+# Default 18-month lag
+./startup.sh
+
+# Custom lag, range, and extension
+./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \
+    --offset 6 --start 1980-01-01 --end 2025-05-31 --extend-years 2
+```
+
+| Option           | Description                                               |
+| ---------------- | --------------------------------------------------------- |
+| `--offset`       | Lag in months (positive = oil leads; negative = oil lags) |
+| `--start`        | Start date (YYYY-MM-DD)                                   |
+| `--end`          | End date (YYYY-MM-DD)                                     |
+| `--extend-years` | Years to extend the x-axis beyond the end date            |
+
+---
+
+## 📝 Details
+
+* **Data storage**: `data/fred.db` (SQLite) holds two tables: `UNRATE` and `DCOILWTICO`.
+* **Plot styling**:
+
+  * Left y-axis: unemployment rate (%) from 3 to 15%, ticks every 2%.
+  * Right y-axis: log scale oil price (USD), doubling ticks from \$3 upward.
+  * X-axis: years, major tick every 5 years, minor tick every year.
+
+---
+
+## 📦 Dependencies
+
+Listed in `requirements.txt`:
+
+```text
+pandas>=1.5.0
+pandas_datareader>=0.10.0
+matplotlib>=3.5.0
+lxml>=4.6.0
+python-dateutil>=2.8.1
+```
+
+`startup.sh` also upgrades `pip` & `setuptools` to ensure `distutils` is available.
+
+---
+
+## 🛡️ Troubleshooting
+
+* **ModuleNotFoundError: distutils**: resolved automatically by upgrading `setuptools` in `startup.sh`.
+* **Database not found**: run `./startup.sh python scripts/refresh_data.py` to populate `data/fred.db`.
+* **Virtual-env issues**: delete `venv/` and re-run `startup.sh`.
+
+---
+
+## 📄 License
+
+This project is released under the MIT License. See [LICENSE](LICENSE) for details.

 ```

@@ -1,14 +1,12 @@
 #!/usr/bin/env bash
 # -------------------------------------------------------------------
-# startup.sh  — prepare Python env, ensure dependencies, FRED data,
-#              and run a command inside the venv
+# startup.sh  — prepare Python env, ensure deps & data, then run.
 # Usage:
-#   ./startup.sh [command…]
-# Automatically:
-#  - Upgrades pip & setuptools (ensuring distutils on Python 3.12+)
-#  - Creates venv if missing
-#  - Installs requirements
-#  - Downloads data/fred.db if missing
+#   ./startup.sh [options]      # passes options to default plot script
+#   ./startup.sh [script or cmd] # runs specified script/command
+# Examples:
+#   ./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
+#   ./startup.sh pytest -q
 # -------------------------------------------------------------------
 set -euo pipefail

@@ -16,46 +14,56 @@ VENV_DIR="venv"
 REQ_FILE="requirements.txt"
 WHEELHOUSE="wheelhouse"
 DATA_DB="data/fred.db"
+PYTHON="$VENV_DIR/bin/python"
+DEFAULT_SCRIPT="scripts/lagged_oil_unrate_chart_styled.py"

 # 1) Create or reuse venv
 if [[ ! -d "$VENV_DIR" ]]; then
  echo "🔧 Creating virtual-env…"
  python3 -m venv "$VENV_DIR"
 fi
-# Activate
+
+# 2) Activate venv
 # shellcheck disable=SC1090
 source "$VENV_DIR/bin/activate"

-# 2) Upgrade pip & setuptools (to provide distutils on Py3.12+)
+# 3) Upgrade pip & setuptools
 echo "🛠 Upgrading pip & setuptools…"
-pip install --upgrade pip setuptools
+$PYTHON -m pip install --upgrade pip setuptools

-# 3) Install project dependencies
+# 4) Install dependencies
 PIP_OPTS=(install --upgrade --no-input)
 if [[ -d "$WHEELHOUSE" ]]; then
  echo "📦 Installing from wheelhouse…"
-  pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \
-    || { echo "⚠️ Wheelhouse failed—falling back to PyPI…"; \
-         pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; }
+  $PYTHON -m pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \
+    || { echo "⚠️ Wheelhouse failed—falling back…"; \
+         $PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; }
 else
  echo "🌐 Installing from PyPI…"
-  pip "${PIP_OPTS[@]}" -r "$REQ_FILE"
+  $PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"
 fi

-echo "✅ Virtual-env ready ($(python -V))"
+echo "✅ Virtual-env ready ($($PYTHON -V))"

-# 4) Ensure data/fred.db exists
+# 5) Ensure data exists
 if [[ ! -f "$DATA_DB" ]]; then
-  echo "🔄 $DATA_DB not found—downloading FRED series…"
-  python scripts/refresh_data.py
+  echo "🔄 Data not found—downloading FRED series…"
+  $PYTHON scripts/refresh_data.py
 fi

-# 5) Decide what to run
-if [[ $# -gt 0 ]]; then
-  CMD=("$@")
+# 6) Build command
+if [[ $# -eq 0 ]]; then
+  CMD=("$PYTHON" "$DEFAULT_SCRIPT" --offset 18)
+elif [[ "$1" == --* ]]; then
+  CMD=("$PYTHON" "$DEFAULT_SCRIPT" "$@")
 else
-  CMD=(python scripts/lagged_oil_unrate_chart_styled.py --offset 18)
+  if [[ -f "$1" || "$1" == *.py ]]; then
+    CMD=("$PYTHON" "$@")
+  else
+    CMD=("$@")
+  fi
 fi

+# 7) Execute
 echo "🚀 Executing: ${CMD[*]}"
-exec "${CMD[@]}"
+exec "${CMD[@]}"