mirror of
https://github.com/PR0M3TH3AN/python_charts.git
synced 2026-10-29 09:50:40 +00:00
Update .gitignore, enhance repo-context documentation, and refine startup script
- Add `repo-context.txt` to .gitignore to prevent tracking of generated context files. - Expand documentation in `repo-context.txt` to include detailed usage instructions, quickstart guide, and repository structure for the lagged oil-unemployment chart project. - Refine `startup.sh` script comments for clarity on its purpose and usage.
This commit is contained in:
1 parent
8797ef1d04
commit
3c097cc847
3 files changed
+171
-26
No files matched your search
@@ -7,3 +7,4 @@ wheelhouse/*.log
|
|||||||
wheelhouse/*.md
|
wheelhouse/*.md
|
||||||
# keep wheel files themselves!
|
# keep wheel files themselves!
|
||||||
data/*.bak
|
data/*.bak
|
||||||
|
repo-context.txt
|
||||||
+137
-1
@@ -21,8 +21,144 @@ Generated on: 2025-05-30
|
|||||||
|
|
||||||
## python_charts/README.md
|
## python_charts/README.md
|
||||||
```markdown
|
```markdown
|
||||||
|
# Lagged Oil–Unemployment Chart
|
||||||
|
|
||||||
|
A self-contained toolkit that:
|
||||||
|
|
||||||
|
* Fetches U.S. unemployment rate (UNRATE) and WTI crude oil price (DCOILWTICO) from FRED
|
||||||
|
* Stores the series in a local SQLite database (`data/fred.db`)
|
||||||
|
* Produces a dual-axis plot of unemployment vs. lagged oil price, with customizable lag, date range, and styling
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Quickstart
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Clone the repo
|
||||||
|
git clone https://github.com/your-org/python_charts.git
|
||||||
|
cd python_charts
|
||||||
|
|
||||||
|
# One-command: install deps, download data, and plot
|
||||||
./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \
|
./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \
|
||||||
--offset 12 --end 2025-05-31 --extend-years 5
|
--offset 12 --end 2025-05-31 --extend-years 5
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
|
||||||
|
1. Create (or reuse) a Python 3 virtual environment (`venv/`)
|
||||||
|
2. Upgrade `pip` & `setuptools` (providing `distutils` support)
|
||||||
|
3. Install required packages (offline wheelhouse first, PyPI fallback)
|
||||||
|
4. Download UNRATE & DCOILWTICO into `data/fred.db` if missing
|
||||||
|
5. Generate the chart with a 12-month oil lead, extending the x-axis by 5 years
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📁 Repository Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
python_charts/
|
||||||
|
├── data/
|
||||||
|
│ └── fred.db # Local FRED snapshot (SQLite)
|
||||||
|
├── scripts/
|
||||||
|
│ ├── refresh_data.py # Download and store FRED series
|
||||||
|
│ ├── lagged_oil_unrate_chart_styled.py # Plotting script
|
||||||
|
│ └── pyproject.toml # Packaging metadata
|
||||||
|
├── startup.sh # Bootstraps venv, deps, data, and runs commands
|
||||||
|
├── requirements.txt # Python dependencies
|
||||||
|
└── README.md # This file
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🛠️ Installation & Setup
|
||||||
|
|
||||||
|
1. **Ensure** you have Python 3.8+ installed on your system.
|
||||||
|
2. **Clone** and navigate into the project:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/your-org/python_charts.git
|
||||||
|
cd python_charts
|
||||||
|
```
|
||||||
|
3. **Make** the startup script executable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
chmod +x startup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
> On first run, `startup.sh` will build the environment, install dependencies, and pull data.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ Usage
|
||||||
|
|
||||||
|
### Refresh FRED data
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./startup.sh python scripts/refresh_data.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Downloads the full UNRATE and WTI series (from 1948 to today) into `data/fred.db`.
|
||||||
|
|
||||||
|
### Generate the chart
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Default 18-month lag
|
||||||
|
./startup.sh
|
||||||
|
|
||||||
|
# Custom lag, range, and extension
|
||||||
|
./startup.sh python scripts/lagged_oil_unrate_chart_styled.py \
|
||||||
|
--offset 6 --start 1980-01-01 --end 2025-05-31 --extend-years 2
|
||||||
|
```
|
||||||
|
|
||||||
|
| Option | Description |
|
||||||
|
| ---------------- | --------------------------------------------------------- |
|
||||||
|
| `--offset` | Lag in months (positive = oil leads; negative = oil lags) |
|
||||||
|
| `--start` | Start date (YYYY-MM-DD) |
|
||||||
|
| `--end` | End date (YYYY-MM-DD) |
|
||||||
|
| `--extend-years` | Years to extend the x-axis beyond the end date |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Details
|
||||||
|
|
||||||
|
* **Data storage**: `data/fred.db` (SQLite) holds two tables: `UNRATE` and `DCOILWTICO`.
|
||||||
|
* **Plot styling**:
|
||||||
|
|
||||||
|
* Left y-axis: unemployment rate (%) from 3 to 15%, ticks every 2%.
|
||||||
|
* Right y-axis: log scale oil price (USD), doubling ticks from \$3 upward.
|
||||||
|
* X-axis: years, major tick every 5 years, minor tick every year.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 Dependencies
|
||||||
|
|
||||||
|
Listed in `requirements.txt`:
|
||||||
|
|
||||||
|
```text
|
||||||
|
pandas>=1.5.0
|
||||||
|
pandas_datareader>=0.10.0
|
||||||
|
matplotlib>=3.5.0
|
||||||
|
lxml>=4.6.0
|
||||||
|
python-dateutil>=2.8.1
|
||||||
|
```
|
||||||
|
|
||||||
|
`startup.sh` also upgrades `pip` & `setuptools` to ensure `distutils` is available.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🛡️ Troubleshooting
|
||||||
|
|
||||||
|
* **ModuleNotFoundError: distutils**: resolved automatically by upgrading `setuptools` in `startup.sh`.
|
||||||
|
* **Database not found**: run `./startup.sh python scripts/refresh_data.py` to populate `data/fred.db`.
|
||||||
|
* **Virtual-env issues**: delete `venv/` and re-run `startup.sh`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📄 License
|
||||||
|
|
||||||
|
This project is released under the MIT License. See [LICENSE](LICENSE) for details.
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
+33
-25
@@ -1,14 +1,12 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# -------------------------------------------------------------------
|
# -------------------------------------------------------------------
|
||||||
# startup.sh — prepare Python env, ensure dependencies, FRED data,
|
# startup.sh — prepare Python env, ensure deps & data, then run.
|
||||||
# and run a command inside the venv
|
|
||||||
# Usage:
|
# Usage:
|
||||||
# ./startup.sh [command…]
|
# ./startup.sh [options] # passes options to default plot script
|
||||||
# Automatically:
|
# ./startup.sh [script or cmd] # runs specified script/command
|
||||||
# - Upgrades pip & setuptools (ensuring distutils on Python 3.12+)
|
# Examples:
|
||||||
# - Creates venv if missing
|
# ./startup.sh --offset 12 --end 2025-05-31 --extend-years 5
|
||||||
# - Installs requirements
|
# ./startup.sh pytest -q
|
||||||
# - Downloads data/fred.db if missing
|
|
||||||
# -------------------------------------------------------------------
|
# -------------------------------------------------------------------
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -16,46 +14,56 @@ VENV_DIR="venv"
|
|||||||
REQ_FILE="requirements.txt"
|
REQ_FILE="requirements.txt"
|
||||||
WHEELHOUSE="wheelhouse"
|
WHEELHOUSE="wheelhouse"
|
||||||
DATA_DB="data/fred.db"
|
DATA_DB="data/fred.db"
|
||||||
|
PYTHON="$VENV_DIR/bin/python"
|
||||||
|
DEFAULT_SCRIPT="scripts/lagged_oil_unrate_chart_styled.py"
|
||||||
|
|
||||||
# 1) Create or reuse venv
|
# 1) Create or reuse venv
|
||||||
if [[ ! -d "$VENV_DIR" ]]; then
|
if [[ ! -d "$VENV_DIR" ]]; then
|
||||||
echo "🔧 Creating virtual-env…"
|
echo "🔧 Creating virtual-env…"
|
||||||
python3 -m venv "$VENV_DIR"
|
python3 -m venv "$VENV_DIR"
|
||||||
fi
|
fi
|
||||||
# Activate
|
|
||||||
|
# 2) Activate venv
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
source "$VENV_DIR/bin/activate"
|
source "$VENV_DIR/bin/activate"
|
||||||
|
|
||||||
# 2) Upgrade pip & setuptools (to provide distutils on Py3.12+)
|
# 3) Upgrade pip & setuptools
|
||||||
echo "🛠 Upgrading pip & setuptools…"
|
echo "🛠 Upgrading pip & setuptools…"
|
||||||
pip install --upgrade pip setuptools
|
$PYTHON -m pip install --upgrade pip setuptools
|
||||||
|
|
||||||
# 3) Install project dependencies
|
# 4) Install dependencies
|
||||||
PIP_OPTS=(install --upgrade --no-input)
|
PIP_OPTS=(install --upgrade --no-input)
|
||||||
if [[ -d "$WHEELHOUSE" ]]; then
|
if [[ -d "$WHEELHOUSE" ]]; then
|
||||||
echo "📦 Installing from wheelhouse…"
|
echo "📦 Installing from wheelhouse…"
|
||||||
pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \
|
$PYTHON -m pip "${PIP_OPTS[@]}" --no-index --find-links="$WHEELHOUSE" -r "$REQ_FILE" \
|
||||||
|| { echo "⚠️ Wheelhouse failed—falling back to PyPI…"; \
|
|| { echo "⚠️ Wheelhouse failed—falling back…"; \
|
||||||
pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; }
|
$PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"; }
|
||||||
else
|
else
|
||||||
echo "🌐 Installing from PyPI…"
|
echo "🌐 Installing from PyPI…"
|
||||||
pip "${PIP_OPTS[@]}" -r "$REQ_FILE"
|
$PYTHON -m pip "${PIP_OPTS[@]}" -r "$REQ_FILE"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "✅ Virtual-env ready ($(python -V))"
|
echo "✅ Virtual-env ready ($($PYTHON -V))"
|
||||||
|
|
||||||
# 4) Ensure data/fred.db exists
|
# 5) Ensure data exists
|
||||||
if [[ ! -f "$DATA_DB" ]]; then
|
if [[ ! -f "$DATA_DB" ]]; then
|
||||||
echo "🔄 $DATA_DB not found—downloading FRED series…"
|
echo "🔄 Data not found—downloading FRED series…"
|
||||||
python scripts/refresh_data.py
|
$PYTHON scripts/refresh_data.py
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# 5) Decide what to run
|
# 6) Build command
|
||||||
if [[ $# -gt 0 ]]; then
|
if [[ $# -eq 0 ]]; then
|
||||||
CMD=("$@")
|
CMD=("$PYTHON" "$DEFAULT_SCRIPT" --offset 18)
|
||||||
|
elif [[ "$1" == --* ]]; then
|
||||||
|
CMD=("$PYTHON" "$DEFAULT_SCRIPT" "$@")
|
||||||
else
|
else
|
||||||
CMD=(python scripts/lagged_oil_unrate_chart_styled.py --offset 18)
|
if [[ -f "$1" || "$1" == *.py ]]; then
|
||||||
|
CMD=("$PYTHON" "$@")
|
||||||
|
else
|
||||||
|
CMD=("$@")
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# 7) Execute
|
||||||
echo "🚀 Executing: ${CMD[*]}"
|
echo "🚀 Executing: ${CMD[*]}"
|
||||||
exec "${CMD[@]}"
|
exec "${CMD[@]}"
|
||||||
Reference in new issue
Block a user