finance-app/CLAUDE.md

# CLAUDE.md

Guidance for Claude Code when working in this repository.

## Project Overview

Personal finance tracker. Bank statements are ingested via an N8N workflow (in the smarthome repo at `docker/automation/workflows/cc-statement-processor-paperless.json`) that sends PDFs to Gemini 2.5 Flash for extraction, then inserts into PostgreSQL.

- **App**: Next.js 16 App Router, TypeScript, Tailwind CSS
- **DB**: PostgreSQL container `postgres-personal`, database `personal`, user `personal`
- **Auth**: `X-Forwarded-User` header (email) set by Traefik → `participants.email`. In dev/fallback: participant id=1 ("Me")
- **Runs at**: port 3000 inside container, exposed on host port 4100, proxied at `https://finance.bosecamp.com`

## Common Commands

**Deployment is push-to-deploy via Komodo** (since 2026-07-19): pushing to `main` on
Gitea triggers the `deploy-finance` Procedure, which runs DeployStack `--build` on the
`finance` stack (files_on_host over `docker/finance/` in the smarthome repo). Just
commit and push — no manual deploy needed.

```bash
# Manual fallback only (from smarthome repo root), e.g. if Komodo is down
docker compose --env-file docker/common.env --env-file docker/finance/.env \
  -f docker/finance/docker-compose.yml up -d --build

# IMPORTANT: docker restart does NOT pick up a new image — push to main (or use the compose command above)

# DB access
docker exec postgres-personal psql -U personal -d personal

# View logs
docker logs finance -f
```

## Architecture

### Key Files

| File | Purpose |
|------|---------|
| `src/lib/db.ts` | `queryRaw<T>()` — the only DB query function; uses `pg` directly |
| `src/lib/queries.ts` | All SQL query functions (no ORM); import `queryRaw` from `@/lib/db` |
| `src/lib/hooks.ts` | TanStack Query hooks for all API calls |
| `src/lib/auth.ts` | `getCurrentUser()` — reads `X-Forwarded-User` header |
| `src/lib/categories.ts` | Canonical category list (`CATEGORIES` array + `formatCategory()`) |
| `src/app/api/*/route.ts` | API route handlers |
| `src/components/` | Shared UI components |

### Data Flow

- All queries in `src/lib/queries.ts` use raw SQL via `queryRaw` from `src/lib/db.ts`
- API routes call query functions and return `NextResponse.json()`
- Frontend uses hooks from `src/lib/hooks.ts` (TanStack Query) — never fetches directly
- Auth is always checked first in every API route: `const user = await getCurrentUser(req)`

### Owner Scoping

All data is scoped by `owner_id`. The effective owner of a transaction is:
```sql
COALESCE(t.owner_id, s.owner_id)
```
- Statement-linked transactions: owner comes from `statements.owner_id`
- Manual transactions: `statement_id IS NULL`, owner stored directly in `transactions.owner_id`

The effective merchant and category always prefer overrides:
```sql
COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name)  -- merchant
COALESCE(o.category_override, t.category)                                 -- category
```

## Database

```bash
# Schema inspection
docker exec postgres-personal psql -U personal -d personal -c "\d transactions"

# Apply a migration SQL file
docker exec postgres-personal psql -U personal -d personal < prisma/migrations/<name>/migration.sql
```

### Key Tables

- `statements` — one row per billing period per bank account
- `transactions` — line items; `statement_id` is nullable (NULL = manual entry); `reconciled_with_id` links a manual tx to its matched statement tx; `payment_method` (migration 0016) is `card | cash | bank_transfer | other`, NULL = unknown

### Cash and reconciliation

`payment_method = 'cash'` excludes a transaction from reconciliation via the
`notCash()` fragment in `queries.ts`. Cash never appears on a statement, so
without it a cash entry sits in the pending queue forever being offered matches
within 3 days and 1% on amount — and accepting one is silently destructive:
reconciled manual rows are filtered out of every query, so the cash spend
disappears while the card transaction it matched claims to be that same spend.

Only cash is excluded. Bank transfers *do* appear on a statement now that
transaction accounts are imported, and NULL means unknown — both stay
candidates, preserving the behaviour of every pre-existing row.

ATM withdrawals stay categorised as spend rather than `transfers`. Treating them
as transfers only works if every cash purchase is logged; with partial logging
it silently deletes the unlogged remainder from spend totals.
- `transaction_overrides` — user corrections to AI-extracted data (category, merchant, notes)
- `transaction_splits` — shared expense tracking (participant, share_percent, settled)
- `split_payments` — recorded cash settlements between participants
- `transaction_tags` — many-to-many join to `tags`
- `rules` — auto-categorisation rules (JSONB conditions + actions)
- `rule_apply_runs` — audit log of bulk rule-apply runs with full snapshot for revert
- `expense_metadata` — enrichment from email receipts; `transaction_id` nullable until reconciled
- `participants` — people; `id=1` is "Me" (the primary user)
- `account_owner_mappings` — persists bank+account → owner assignments

### Shared expenses and settlement — read before touching

Rebuilt 2026-07-28. `docs/shared-expenses-design.md` describes the live model;
it is no longer a proposal. Everything the old version of this section warned
about has changed — if you are working from memory of it, re-read.

**The cutover date is the primary balance gate, not the `settled` flag.**
`ACTIVE_OBLIGATION` (`src/lib/analytics-sql.ts`) is
`ts.settled = false AND t.transaction_date >= '2026-01-09'`. Nothing dated
before the cutover can ever be owed, because carryover transaction **2348**
(dated 2026-01-09, $1,093.22) already carries the whole pre-cutover balance as
one figure. The bound is **inclusive** — 2348 is itself dated 2026-01-09, so an
exclusive bound would drop the carryover and the entire pre-cutover balance.

Consequence: **pre-2026 transactions can be split freely.** A split on a 2024
grocery shop describes how the expense was shared — which is what stops it
inflating spend — without asserting a debt. 657 pre-2026 transactions carry
1,266 such splits, imported from SplitMyExpenses and marked `settled`.

**Spend counts settled splits; owed does not.** `myShare`/`mySplitOf` must NOT
filter on `settled` — half a 2025 grocery shop was your expense whether or not
the other half was repaid. Filtering it out re-inflates exactly the figures the
historical import exists to correct.

**Any split write path that deletes-and-recreates must carry `settled` across.**
`POST /api/transactions/[id]/splits` did not, and silently converted discharged
obligations into live debt — $37,233.28 was exposed. Fixed in `6add958`.
`rule-actions.ts` is safe only by the shape of its upsert
(`ON CONFLICT DO UPDATE SET share_percent` never touches the flag).
The rules-apply revert route restores it explicitly.

**Settling up is recording a payment.** There is deliberately no "mark settled"
action. `settled` marks obligations discharged *outside* this app; doing both
would subtract the settlement twice.

**Payments carry scope, and one transfer can carry several rows.**
`split_payments.trip_id` (migration 0022) says which tab a payment settles;
NULL is the ongoing household tab. There is no unique constraint on
`linked_transaction_id`, so a grouped transfer is recorded as one row per scope
that re-add to the transfer — that is how Sonu's $3,779.33 and $4,794.06 were
allocated Europe-first with the remainder to household.

**Trip owed must be owner-scoped; trip cost must not be.** The owed query
applies `OWNER_SCOPE`; without it a debt between the *other two* participants
reads as owed to you ($1,605.49 on Europe 2026). Trip *cost* deliberately counts
every payer — a trip cost what the group put into it — which is why the stat card
says "all payers, net of refunds". Do not "fix" the missing scoping there.

**Duplicates are superseded, never deleted.** `transactions.superseded_by_id`
(migration 0023); 31 rows / $42,040.68 from overlapping ANZ statements 107/142/143.
Every child of `transactions` is `ON DELETE CASCADE`. The exclusion lives *inside*
`EXCLUDE_RECONCILED_SOURCE`, so any query applying that fragment gets it free —
and any query that does not still double-counts.

**Refunds:** a *partial* refund is netted in SQL (`NET_SPEND_ROWS`/`SPEND_SIGNED`);
a *cancelled* booking has both legs untagged from the trip by hand, because a
trip never incurred a cost it cancelled.

**Trips:** Europe 2026 (id 1, 19 Mar–12 Apr), Auckland 2026 (id 2),
Europe — Sonu + Sunny (id 3, 12–28 Apr, created 2026-07-28 from tag 5).

**Partial split coverage inside a category is usually correct, not a gap.** Only
*shared* items are split. `utilities` sits at 69% yours because Globird, OVO, GWW
and home telecoms are split while Telstra, Vodafone, Optus and JB Hi-Fi Mobile
are personal. `subscriptions` is 91% because Uber One, Amazon Prime and OnePass
are shared while Claude, OpenAI, Anthropic, OpenRouter, You.com, LinkedIn, Xero,
Billdu, Spotify and Patreon are not. `fees` and `charity` are 100% yours and
correct. Check the merchants before concluding a rule was never applied — a
category-level ratio that "looks wrong" usually is not.

**Still true, and still a caveat:** Sonu's loan contributions (`…emi` in the
offset account, 39 rows, $37,980.24) are categorised `transfers`, indistinguishable
from ordinary internal transfers. The loan model below is unbuilt.

**`order_reviews` is a table wired to nothing** — 0 rows, no API, no UI, no
writes. The "never order from here again" capability does not exist.

### The shared loan

The loan is a **separate ledger**, not a shared expense and not a settlement
context — a contribution must never be able to settle a dinner. Sonu's obligation
is a fixed 50% of the repayment; actual contributions vary, and the difference is
a tracked receivable ($4,000.00 over 2025-07 → 2026-06).

Do not derive the share from actual payments. During her leave the obligation did
not change, only the payment did — a percentage-of-actual model would silently
redefine her share as 30% and make the shortfall vanish.

Loan interest reconciles exactly: `repayments − interest − fees = balance
reduction`. It stays categorised `loan_interest` and counts as spend — over 12
months $63,500 of cash left and debt fell $44,127.36, and the $16,523.64
difference bought nothing. Excluding it would leave the balance sheet unable to
reconcile cash out against equity gained.

**The repayment is voluntarily above contracted, and the gap is the largest
flexible cost in the whole picture.** Contracted is $1,190.54/fortnight
($2,579.50/mo annualised); the actual direct debit is $2,500.00/fortnight
($5,416.67/mo). That is $2,837.17/mo of overpayment, and it is *not* sunk — it
shows up as `statements.redraw_available`, which grew $62,387.17 → $81,017.42
across the two most recent loan statements. Sonu returned to $1,250/fortnight in
July 2026 after the reduced $750 period during her leave.

Treat the repayment as two figures whenever asking "what does this cost me":
the contracted floor and the actual. `scheduled_repayment` holds the actual
($2,500), not the contracted minimum — the contracted figure is not in the DB at
all. See `docs/expense-baseline.md`.

### Import Date (`created_at`)

`transactions.created_at` is the import timestamp (DB default `now()`). In the transactions and shared views, the "Imported" column shows:
- For statement transactions: when the statement was processed by N8N
- For reconciled transactions: the `created_at` of the original manual/CSV transaction (via `LEFT JOIN transactions src ON src.reconciled_with_id = t.id`) — so the original import date is preserved post-reconciliation

Use `created_at` (not `transaction_date`) to answer "what was added since the last settlement?". Sort by `created_at` is supported server-side in `getTransactions` and client-side in the shared view.

### Rules System

Conditions are AND-evaluated. Fields: `merchant_normalized`, `description`, `category`, `bank_name`, `amount`, `transaction_type`. Operators: `contains`, `equals`, `starts_with`, `gt`, `lt`, `not_equals`. Actions: `set_category`, `set_merchant`, `add_tag_ids`, `apply_split`.

`contains` and `equals` operators are case-insensitive (both sides `.toLowerCase()`).

**A rule with zero conditions matches every transaction.** Both apply paths use
`conditions.length === 0 || conditions.every(...)`. Rule 43 "Home 50/50 Sonu" has
no conditions and a 50/50 split action — applying it blindly would split all
~3,700 transactions with another participant. That is what `manual_only` is for:
those rules are excluded from bulk runs and fire from the transactions page
against a hand-picked selection.

### Previewing a rule before applying it

`GET /api/rules/[id]/matches` is a dry run — it writes nothing and returns only
the transactions a rule would actually *change*, with already-correct rows
summarised as a count. The Preview button on the rules page uses it.

Apply then goes through `POST /api/transactions/bulk` with `action: "apply_rule"`
and **explicit transaction ids**, not the conditions. That is the safety
property: a rule whose conditions are too broad cannot reach further than what
the preview showed and the user ticked.

Prefer this over auto-applying rules on ingestion. It fails safe, works
retroactively, and tells you which rules are consistent enough to automate later.

### Rule apply history

`rule_apply_runs` snapshots the before-state so a run can be reverted, and since
migration 0017 also records `rule_id`, `rule_name` and `source`
(`all` | `rule` | `selection`). `rule_name` is denormalised deliberately and
there is no FK to `rules` — history must stay readable after a rule is renamed or
deleted, and deleting a rule must not cascade away the audit trail.

`GET /api/rules/runs/[id]` diffs that snapshot against current values. Rows
changed by something else since the run are flagged, because reverting restores
the pre-run value and discards the later edit.

### Trusting extracted statement data

**Balance assertions are the check that works.** `getStatements` computes
`opening + movement − closing`; the statements page flags any statement that does
not reconcile. Sign depends on what the balance means — on a credit card or loan
it is what you owe, so spending increases it; on a transaction or offset account
it is what you hold. 11 pre-existing statements currently fail, ~$4,177
unexplained, including two adjacent ANZ statements off by exactly ±$230.38 (a
transaction filed against the wrong one).

**Do not derive `opening_balance` from `closing − movement`.** It is an accounting
identity, so every statement would reconcile and the check would go permanently
green. A null that reads "unverified" is worth more than a number that is right
by construction. For the same reason, do not add a totals assertion comparing
`total_debits` to the summed rows — those totals are now *computed* from the rows
(see the N8N `Parse Gemini Result` node), so that check can never fail.

**Gemini invents summary fields the statement does not print.** Wise PDFs show
only a closing balance; asked for an opening balance anyway, the model produced
11,277.08 against a truth of 0.00, and on another statement read the running
balance of the oldest row. Every transaction was extracted perfectly in both
cases — verified row for row against the CSV exports. When a balance assertion
fails, suspect the summary before the transactions.

**Gemini drops rows silently on long tables.** `finishReason` was `STOP`, not
`MAX_TOKENS`, so raising `maxOutputTokens` does not help. This did *not* actually
occur on the Wise imports (that was the summary bug above), but it is why an
empty statement must not throw: a document that errors never gets tagged, so it
is re-fetched every poll forever and blocks everything behind it in the queue
(`ordering=-created`, `page_size=1`).

**FX is per transaction date**, via Frankfurter (ECB daily, free, no key), with
weekends resolving to the prior publication. A single spot rate across a 15-month
statement is wrong by up to 20%. Wise's own rates are more accurate in principle
but differ by only 0.05% and exist on 44 of 194 rows, so mixing bases is not
worth it.

**When comparing CSV exports to extracted data, order by full timestamp
including milliseconds.** Two of one statement's rows are 1ms apart; dropping the
fraction reversed them and produced a bogus opening balance.

## Development Patterns

### Adding a new API route

1. Create `src/app/api/<resource>/route.ts`
2. Always call `getCurrentUser(req)` first; return 403 if null
3. Write SQL in `src/lib/queries.ts` using `queryRaw`
4. Add a TanStack Query hook in `src/lib/hooks.ts`

### Adding a new condition field to rules

Two files only:
- `src/app/api/rules/apply/route.ts` — add to `Condition.field` union, `TxFields` interface, and `evaluateCondition()` switch
- `src/app/rules/page.tsx` — add to `FIELDS` array; add special rendering if needed (e.g. enum dropdown for `transaction_type`)

### Modifying queries

- All JOINs to `statements` must be `LEFT JOIN` (manual transactions have no statement)
- Owner filter pattern: `WHERE COALESCE(t.owner_id, s.owner_id) = $1`
- Bank name pattern: `COALESCE(s.bank_name, 'Manual') as bank_name`

Analytics queries must import the fragments from `src/lib/analytics-sql.ts`
(`STATEMENTS_JOIN`, `OWNER_SCOPE`, `EFFECTIVE_CATEGORY`, `EXCLUDE_NON_SPEND`)
rather than hand-rolling them. Two failure modes they exist to prevent:

- An `INNER JOIN statements` + `WHERE s.owner_id = $1` silently drops every
  manual/CSV transaction (`statement_id IS NULL`).
- Spend must exclude the `transfers` and `investment` categories. Once bank
  statements are imported alongside card statements, a credit-card payment
  appears twice — as a debit leaving the bank account and as the underlying
  purchases on the card statement. Excluding `transfers` is what nets it out.
  Use the `EXCLUDE_NON_SPEND` fragment: a bare `category NOT IN (...)` evaluates
  to NULL for uncategorised rows and drops them from totals.

### Statement types

`statements.statement_type` is constrained to `credit_card | transaction |
savings | loan | offset | investment | other`. Migration 0013 added a
`normalize_statement_type()` SQL function plus a BEFORE INSERT/UPDATE trigger, so
the N8N workflow can keep sending raw free text (`'ACCESS ADVANTAGE'`, `'Business
Card'`) and the DB normalises it on write. The raw extracted value is preserved in
`account_type`.

The TypeScript mirror is `src/lib/statement-types.ts` — keep the list, the SQL
function, and the CHECK constraint in sync when adding a type.

### Loans

A loan repayment is **not** an expense. It is part principal (equity, a
balance-sheet move) and part interest (the only part that is spend). Migration
0014 adds:

- `transactions.principal_amount` / `interest_amount` — populated only when the
  lender itemises the split on the repayment row itself
- `statements.interest_rate`, `scheduled_repayment`, `repayment_frequency`,
  `redraw_available`, `loan_term_months`

Two statement shapes, both handled:

1. **Separate rows** (the common Australian case) — the loan statement lists
   repayments and "Interest Charged" separately. `transaction_type` alone is
   enough: `interest` rows count as spend, `payment` rows don't. No split columns
   needed.
2. **Itemised repayment row** — some lenders print principal and interest on the
   repayment line. That row is typed `payment`, so it would be skipped entirely
   and its interest lost. The `SPEND_ROWS` / `SPEND_BASE` fragments in
   `analytics-sql.ts` handle it: a row with a non-null `interest_amount` counts
   as spend, valued at `interest_amount` rather than `amount`.

The N8N `Parse Gemini Result` node only accepts a split when both parts are
present *and* they sum to the row amount (±2c) — a half-extracted split would
silently misreport spend, so it is discarded rather than trusted.

Loan interest uses the `loan_interest` category; principal repayments use
`investment` (excluded from spend, surfaced on the investments line in monthly
analytics).

### Prisma

The schema at `prisma/schema.prisma` covers all tables. The generated client (gitignored) must be regenerated after schema changes:
```bash
cd /mnt/m2cache/appdata/finance-app && npx prisma generate
```
Docker builds run `npx prisma generate` automatically. Do not commit `src/generated/prisma/` — it is gitignored.

## Agent / MCP Access

Agents read this DB through the read-only `postgres-personal` MCP server (lives in the
`personal-agent-gateway` repo, not here): `agent_ro` role, SELECT-only, SQLGlot guardrail,
100-row cap, every call audited to `mcp_query_log`. See `docs/agent-access.md` for the tool
list, the five analysis views, and per-client setup (Claude Code, Codex, Hermes).

Two things to remember when changing the schema: the agent views are created by
`smarthome/personal-agent/migrations/006_agent_read_role_views.sql` (not Prisma) and read
`transactions`/`statements`/`expense_metadata` columns directly — rename a column and they
break or go stale. And the views are **not** owner-scoped and do **not** merge
`transaction_overrides`, so agent numbers can differ from the UI.

## Known Gaps / TODOs

See `README.md` → **Known Gaps / TODOs** for full details.

**Payment provider tracking**: `merchant_normalized` currently conflates payment provider (PayPal, Afterpay, Zip) with the actual merchant. Plan: add `payment_provider` column, update Gemini prompt to extract it separately, backfill from `merchant_name` patterns, surface in UI filters.

### Open as of 2026-07-26

- **Shared expenses redesign** — `docs/shared-expenses-design.md`. Phase 0 done;
  Phases 1–4 unbuilt. Deliberately paused to live with the current behaviour
  before committing to a model designed in one session.
- **Expense baseline / emergency reserve** — `docs/expense-baseline.md`. One-off
  analysis, nothing built. Records four data corrections the raw numbers need
  (misfiled Raiz/super/brokerage debits, `other` credits read as negative spend,
  `government` conflating ATO with rates/rego, `fees` being mostly annual) and
  why only Feb–Jun 2026 is trustworthy for per-person figures.
- **11 statements fail the balance assertion**, ~$4,177 unexplained. Predates
  this work. One ANZ statement is off by exactly $0.50, traced to a misread digit
  in fee rows ($5.00 vs $5.50).
- **28 Up Bank debits are categorised `other`** ($3,760.74). Up only categorised
  16 of 88 rows. The `Payee` field is populated throughout, so merchant rules plus
  the rule preview should clear most of it.
- **Up item sales are categorised `income`** ($4,238.04 across 19 credits — iPad,
  drone, camera). Correct in that they are excluded from spend, but it mixes
  asset disposals into the income line alongside salary.
- **`payment_method` is not shown in the transactions list** — settable on create
  and edit only. Worth a column or filter if cash becomes routine.
- **Raw statement exports live in `dump/`**, gitignored since `31a8177`. They were
  committed by accident in `030490e` and remain in that commit's history; the repo
  has no GitHub remote, so exposure is limited to the local Gitea. Purging history
  was offered and not actioned.