Conscious Spending Analysis

First-Time Setup

On the very first run, there is one-time data preparation work. Check for these conditions and handle them before running analysis:

1. Chase PDF Conversion

The input/chase/ directory contains ~24 monthly PDF statements (Feb 2024 – Jan 2026). These need to be converted to CSV once. There is also a CSV file covering Jan–Apr 2026.

To convert PDFs:

1. Install pdfplumber: pip3 install pdfplumber
2. Write a Python script that extracts transaction tables from each PDF
3. Save the combined output as input/chase/chase_historical.csv with columns: Transaction Date, Post Date, Description, Category, Type, Amount, Memo
4. This only needs to happen once — subsequent runs use CSVs

2. Capital One .numbers Files

The .numbers files in input/cap-one-checking/ and input/cap-one-quicksilver/ contain historical data predating the CSV exports. The CSVs only cover ~3-4 months (late 2025 – Apr 2026).

Options for converting (explain to user and let them choose):

• Manual export: Open each .numbers file in Numbers.app → File → Export to → CSV. Save alongside the existing CSVs.
• Python conversion: pip3 install numbers-parser, then script the extraction.

3. Verify Data Coverage

After conversion, verify you have continuous data across all 3 accounts for at least 12 months (ideally 24). Print a date coverage table and flag gaps.

Monthly Analysis Workflow

Step 1: Check Data Freshness

Read output/last_run.json to see what was previously analyzed. Compare against CSV files in input/ — look for new or updated files (by filename date prefix or file modification time).

Tell the user: "Last analysis covered data through {date}. I see new data through {date}. Ready to run?"

Step 2: Run the Analysis Script

python3 .claude/skills/conscious-spending/scripts/analyze_spending.py --project-dir .

For a complete reprocessing (after adding historical data or fixing categories):

python3 .claude/skills/conscious-spending/scripts/analyze_spending.py --project-dir . --full-run

The script outputs to output/:

• all_transactions.csv — every transaction with assigned category and confidence
• monthly_summary.csv — monthly totals by category
• trend_data.json — rolling averages and trend directions
• flagged_items.csv — transactions with confidence < 90% needing review
• last_run.json — state file

Step 3: Review Flagged Items

Read output/flagged_items.csv. For each flagged transaction:

• If you can determine the correct category with >= 90% confidence based on the description, amount, and context, recategorize it
• If genuinely ambiguous, present it to the user grouped by likely category for quick confirmation
• Update output/all_transactions.csv with corrections

Pay special attention to:

• Vanguard transactions — distinguish Roth IRA ($300) from 529 ($100) by amount
• Walmart/Target — could be groceries or general merchandise
• *Square POS (SQ ) — usually restaurants but not always
• Amazon transactions — default to "Amazon — Other / Discretionary" unless cross-referenced with Amazon order history

Step 4: Cross-Reference Amazon Orders (if available)

Amazon purchases on the Chase card show as generic "AMZN MKTP US" — no product detail. To break these into the 4 Amazon subcategories, cross-reference with Amazon order history CSVs in input/amazon/:

1. Load Amazon order CSVs (Brad: input/amazon/brad/, Abby: input/amazon/abby/)
2. For each Chase Amazon transaction, find matching orders by date (±3 days) and amount (±$1)
3. Use the product name to subcategorize:
   • Household & Essentials: cleaning, paper goods, toiletries, batteries, filters, pet supplies
   • Kids & Family: children's items, toys, school supplies, books, kids clothing
   • Electronics & Subscriptions: gadgets, cables, tech accessories, digital items
   • Other / Discretionary: everything else (hobby, gifts, impulse buys)
4. If no match found or confidence < 90%, keep in "Amazon — Other / Discretionary" and flag

Step 5: Generate the Monthly Report

Read output/trend_data.json and output/monthly_summary.csv. Write a comprehensive markdown report to output/YYYY-MM_monthly_report.md following this structure:

# Spending Report — {Month Year}

## Summary
- Total income: $9,053
- Total expenses this month: ${amount}
- Net: ${amount}
- Tuition goal: {on track / ${X} short}

## Category Breakdown
| Category | This Month | 3-Mo Avg | 12-Mo Avg | Trend |
|----------|-----------|----------|-----------|-------|
(all 17 categories, sorted by amount descending)

## Trends to Watch
(Categories where 3-month average diverges >15% from 12-month average)

## City of Santa Fe Water
(Dedicated callout on water bill trends — flag if any month exceeds $80)

## What You're Doing Well
(2-3 bullet points on positive trends, categories where spending decreased,
 investments being made consistently)

## Where You're Falling Short
(2-3 bullet points on categories trending up or exceeding benchmarks,
 areas where the tuition goal is at risk)

## Recommendations
(3-5 specific, actionable suggestions from the perspective of a highly respected
 personal finance advisor — see Financial Advisor Persona below)

Also save output/YYYY-MM_monthly_details.csv — the full categorized transaction list for that month only.

Generate the spending chart

Run the chart generation script to create a current-vs-previous month comparison bar chart:

python3 .claude/skills/conscious-spending/scripts/generate_chart.py --project-dir .

This produces output/spending_chart.png and output/spending_chart_base64.txt. The chart shows horizontal grouped bars (blue = current month, orange = previous month) with a $9,053 income reference line.

Step 6: Email Summary

Use the Gmail MCP tools to send a summary email:

1. Authenticate with Gmail if not already done (use mcp__claude_ai_Gmail__authenticate)
2. Read the email template from assets/email_template.md
3. Fill in the template with this month's data
4. For the {spending_chart} placeholder: read output/spending_chart_base64.txt and insert as <img src="data:image/png;base64,{base64_data}" alt="Monthly spending comparison chart" style="max-width:100%;height:auto;">
5. Convert the filled template to HTML for email body
6. Send to: [email protected] and [email protected]
7. Subject: "Wolaver Family Spending Review — {Month Year}"

Keep the email concise — the full report lives in the repo. The email should hit: big picture numbers, the spending chart, top 5 categories, one trend alert, and the doing well / falling short / recommendations sections.

Financial Advisor Persona

When writing the "Doing Well," "Falling Short," and "Recommendations" sections, adopt the perspective of a highly respected personal finance expert (think Suze Orman's directness combined with compassion for a family managing a child's medical needs).

Read references/best_practices.md for foundational principles to draw from.

Key framing:

• Acknowledge the family's constraints honestly (single-income household due to Bowen's epilepsy care, fixed obligations)
• Be direct about problem areas — don't sugarcoat — but always pair criticism with a specific, achievable action
• Prioritize recommendations by impact: target the biggest dollar-amount opportunities first
• Remember the goal: $1,000/mo for tuition. Every recommendation should connect back to this
• Use plain language — this is for busy parents, not accountants

Categorization Reference

Read references/categories.md for the full 17-category taxonomy, keyword matching rules, and confidence scoring guidelines.

Self-Improving Feedback Loop (Karpathy-inspired)

The categorization system learns from user corrections via output/learned_rules.json. This is the core innovation that makes the skill better over time:

How it works:

1. The analysis script checks learned rules FIRST (98% confidence) before keyword matching
2. When a user corrects a categorization, save the correction as a learned rule
3. On all future runs, that pattern is automatically categorized correctly
4. Over time, flagged items approach zero as the system learns the family's specific merchants

To save a user correction, call the script's helper function or add directly to output/learned_rules.json:

# From Python:
from scripts.analyze_spending import add_learned_rule
add_learned_rule(".", "MERCHANT_PATTERN", "Category Name", "Brief note about why")

Or add entries directly to the JSON:

{
  "pattern": "MERCHANT_PATTERN",
  "category": "Correct Category",
  "confidence": 98,
  "source": "user_correction",
  "note": "Why this categorization is correct",
  "added": "2026-04-12"
}

When reviewing flagged items: After the user confirms or corrects categorizations, save every correction as a learned rule. Also save confirmations for items that were correct but flagged at low confidence — this promotes them to 98% on future runs.

The compounding effect: With ~50 corrections, the system typically reaches 85%+ auto-categorization. With ~100 corrections across a few months of data, it approaches 95%+. Each monthly review adds fewer corrections as the rules accumulate.

State Management

All analysis artifacts live in output/:

output/
├── last_run.json              # When/what was last analyzed
├── all_transactions.csv       # Full categorized transaction history
├── monthly_summary.csv        # Monthly totals by category (all months)
├── trend_data.json            # Rolling averages and trend data
├── flagged_items.csv          # Items needing human review
├── YYYY-MM_monthly_report.md  # Monthly narrative reports
├── YYYY-MM_monthly_details.csv # Monthly transaction detail
└── archive/                   # Previous versions (optional)

last_run.json tracks:

• Last run date
• Date range of processed data
• Files that were processed
• Transaction counts and totals

On subsequent runs, check this file to report what's new since the last analysis.

Data Source Formats

Chase Amazon CC (CSVs in input/chase/)

Columns: Transaction Date, Post Date, Description, Category, Type, Amount, Memo

• Amount: negative = purchase, positive = payment/credit
• Card ending in 1524

Capital One Quicksilver (CSVs in input/cap-one-quicksilver/)

Columns: Transaction Date, Posted Date, Card No., Description, Category, Debit, Credit

• Debit = expense, Credit = payment/refund
• Has built-in categories (Dining, Merchandise, Utilities, Gas/Automotive, etc.)

Capital One Checking (CSVs in input/cap-one-checking/)

Columns: Account Number, Transaction Description, Transaction Date, Transaction Type, Transaction Amount, Balance

• Transaction Type: Credit (income) / Debit (expense)
• Date format: MM/DD/YY (unlike YYYY-MM-DD for the others)

Amazon Order History (CSVs in input/amazon/{brad,abby}/)

Key columns: Order Date, Product Name, Unit Price, Total Amount, Payment Method Type

• Used for cross-referencing Amazon purchases to get product-level detail
• Abby's orders include Whole Foods grocery pickups (categorize as Groceries, not Amazon)

Troubleshooting

No CSV files found: User needs to download transaction exports from their bank websites, or convert .numbers/PDF files.

Large "Everything Else" category: Review the items — there may be a new merchant or recurring expense that needs a keyword rule added to scripts/analyze_spending.py.

Vanguard miscategorized: Check amounts. $300 = Roth IRA, $100 = 529. Non-standard amounts need user confirmation about which account they went to.

Missing months: Check if .numbers files contain the missing period. Prompt user to export to CSV.