finance-app/CLAUDE.md

# CLAUDE.md

Guidance for Claude Code when working in this repository.

## Project Overview

Personal finance tracker. Bank statements are ingested via an N8N workflow (in the smarthome repo at `docker/automation/workflows/cc-statement-processor-paperless.json`) that sends PDFs to Gemini 2.5 Flash for extraction, then inserts into PostgreSQL.

- **App**: Next.js 16 App Router, TypeScript, Tailwind CSS
- **DB**: PostgreSQL container `postgres-personal`, database `personal`, user `personal`
- **Auth**: `X-Forwarded-User` header (email) set by Traefik → `participants.email`. In dev/fallback: participant id=1 ("Me")
- **Runs at**: port 3000 inside container, exposed on host port 4100, proxied at `https://finance.bosecamp.com`

## Work tracking — the board, not a markdown file

Outstanding work for this app lives on the Vikunja board at
`https://tasks.bosecamp.com` (project **Work**), which replaced the smarthome
repo's `ACTIONS.md` on 2026-07-30.

Find this app's work with the saved filter **finance-app**, or
`done = false && labels in <finance-app-label-id>`. Note that a ticket can carry
*several* system labels — the receipt→pantry work is labelled `finance-app`,
`pantry-app` **and** `email-ingestion` — so don't assume the finance filter shows
everything that will touch this codebase.

Epics that own most finance work: **Order & receipt ingestion (ING-7/8/10)**,
**Cashback tracking (ING-6)**, **Utility bills slice**, **Lane C — Ingestion
engine**, **Postgres estate & PG 14 EOL**.

**Update the ticket in the same change as the code.** The board is only worth
having if its status is true, and the previous system drifted precisely because
status lived somewhere nobody touched while shipping.

Token and API conventions: smarthome `CLAUDE.md` → "Work tracking". The token is
in smarthome `docker/utilities/.env`; this repo does not carry it.

**One deadline here is real:** `postgres-personal` runs **PostgreSQL 14, EOL 12
November 2026** — it holds `statements`, `transactions`, `orders` and
`expense_metadata`. Tracked in the Postgres epic, not here.

## Common Commands

**Deployment is push-to-deploy via Komodo** (since 2026-07-19): pushing to `main` on
Gitea triggers the `deploy-finance` Procedure, which runs DeployStack `--build` on the
`finance` stack (files_on_host over `docker/finance/` in the smarthome repo). Just
commit and push — no manual deploy needed.

```bash
# Manual fallback only (from smarthome repo root), e.g. if Komodo is down
docker compose --env-file docker/common.env --env-file docker/finance/.env \
  -f docker/finance/docker-compose.yml up -d --build

# IMPORTANT: docker restart does NOT pick up a new image — push to main (or use the compose command above)

# DB access
docker exec postgres-personal psql -U personal -d personal

# View logs
docker logs finance -f
```

## Architecture

### Key Files

| File | Purpose |
|------|---------|
| `src/lib/db.ts` | `queryRaw<T>()` — the only DB query function; uses `pg` directly |
| `src/lib/queries.ts` | All SQL query functions (no ORM); import `queryRaw` from `@/lib/db` |
| `src/lib/hooks.ts` | TanStack Query hooks for all API calls |
| `src/lib/auth.ts` | `getCurrentUser()` — reads `X-Forwarded-User` header |
| `src/lib/categories.ts` | Canonical category list (`CATEGORIES` array + `formatCategory()`) |
| `src/app/api/*/route.ts` | API route handlers |
| `src/components/` | Shared UI components |

### Data Flow

- All queries in `src/lib/queries.ts` use raw SQL via `queryRaw` from `src/lib/db.ts`
- API routes call query functions and return `NextResponse.json()`
- Frontend uses hooks from `src/lib/hooks.ts` (TanStack Query) — never fetches directly
- Auth is always checked first in every API route: `const user = await getCurrentUser(req)`

### Owner Scoping

All data is scoped by `owner_id`. The effective owner of a transaction is:
```sql
COALESCE(t.owner_id, s.owner_id)
```
- Statement-linked transactions: owner comes from `statements.owner_id`
- Manual transactions: `statement_id IS NULL`, owner stored directly in `transactions.owner_id`

The effective merchant and category always prefer overrides:
```sql
COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name)  -- merchant
COALESCE(o.category_override, t.category)                                 -- category
```

### Hiding categories in the transactions view

`getTransactions` takes `exclude_categories`. It is **opt-in per caller and
never defaulted in `queries.ts`** — `GET /api/rules/[id]/matches` and
`POST /api/rules/apply` both read their candidate rows through `getTransactions`,
so a default exclusion there would silently shrink what a rule can preview and
reach. Only the transactions page sets it.

The transactions view defaults it to `["transfers"]` (433 of 3,996 rows, ~11%),
with a visible "Hide transfers" checkbox. Two rules the implementation depends
on, both tested:

- **An explicit category pick beats the exclusion.** Selecting "Transfers" while
  the default is on subtracts it from the hidden list rather than returning zero
  rows — otherwise the view reads "you have no transfers".
- **`COALESCE(..., '')` before `<> ALL`.** `NULL <> ALL(...)` is NULL, not true,
  so an uncategorised row would vanish from a filter that never named its
  category. Same trap `EXCLUDE_NON_SPEND` documents.

It defaults **off** when the view is scoped to a statement (`?statement_id=`).
That is a reconciliation view — the row count has to match the statement, and a
credit-card payment is exactly the row you went there to check.

## Database

```bash
# Schema inspection
docker exec postgres-personal psql -U personal -d personal -c "\d transactions"

# Apply a migration SQL file
docker exec postgres-personal psql -U personal -d personal < prisma/migrations/<name>/migration.sql
```

### Key Tables

- `statements` — one row per billing period per bank account
- `transactions` — line items; `statement_id` is nullable (NULL = manual entry); `reconciled_with_id` links a manual tx to its matched statement tx; `payment_method` (migration 0016) is `card | cash | bank_transfer | other`, NULL = unknown; `source` / `source_ref` / `source_account` (migration 0028) identify a row that came from an account feed rather than a statement or a hand entry — `source_ref` is the provider's own id and carries a partial unique index, which is the only thing making a re-import idempotent

### Cash and reconciliation

`payment_method = 'cash'` excludes a transaction from reconciliation via the
`notCash()` fragment in `queries.ts`. Cash never appears on a statement, so
without it a cash entry sits in the pending queue forever being offered matches
within 3 days and 1% on amount — and accepting one is silently destructive:
reconciled manual rows are filtered out of every query, so the cash spend
disappears while the card transaction it matched claims to be that same spend.

Only cash is excluded on payment method. Bank transfers *do* appear on a
statement now that transaction accounts are imported, and NULL means unknown —
both stay candidates, preserving the behaviour of every pre-existing row.

**Account-feed rows are NOT excluded, and the removed exclusion is worth
knowing about.** There was an `awaitsStatementLine()` predicate here, added with
the Frollo importer on the premise that a feed row is the account's own ledger
entry with no statement line coming. That premise was asserted and never tested.
It was false for almost every account: 422 of the first 550 imported rows already
had a statement twin.

What matters is how it got in. The queue jumped from 8 to 558 the moment the feed
landed, and that jump was read as noise and filtered out. **The queue was right** —
those rows genuinely were provisional entries awaiting statement lines — and
filtering it removed the only mechanism that would ever have collapsed them, so
the duplicates became permanent instead of transient and stayed invisible until a
human saw one salary payment listed twice in two currencies.

A feed row belongs in the queue. Queue volume is solved **upstream**, by not
importing rows the ledger already holds (`LEDGER_MATCH_DAYS`), never downstream
by hiding the ones that are there. If a genuinely statement-less feed is ever
added, give it its own predicate and prove the premise with a query first.

ATM withdrawals stay categorised as spend rather than `transfers`. Treating them
as transfers only works if every cash purchase is logged; with partial logging
it silently deletes the unlogged remainder from spend totals.
- `transaction_overrides` — user corrections to AI-extracted data (category, merchant, notes)
- `transaction_splits` — shared expense tracking (participant, share_percent, settled)
- `split_payments` — recorded cash settlements between participants
- `transaction_tags` — many-to-many join to `tags`
- `rules` — auto-categorisation rules (JSONB conditions + actions)
- `rule_apply_runs` — audit log of bulk rule-apply runs with full snapshot for revert
- `expense_metadata` — enrichment from email receipts; `transaction_id` nullable until reconciled
- `participants` — people; `id=1` is "Me" (the primary user)
- `account_owner_mappings` — persists bank+account → owner assignments

### Shared expenses and settlement — read before touching

Rebuilt 2026-07-28. `docs/shared-expenses-design.md` describes the live model;
it is no longer a proposal. Everything the old version of this section warned
about has changed — if you are working from memory of it, re-read.

**The cutover date is the primary balance gate, not the `settled` flag.**
`ACTIVE_OBLIGATION` (`src/lib/analytics-sql.ts`) is
`ts.settled = false AND t.transaction_date >= '2026-01-09'`. Nothing dated
before the cutover can ever be owed, because carryover transaction **2348**
(dated 2026-01-09, $1,093.22) already carries the whole pre-cutover balance as
one figure. The bound is **inclusive** — 2348 is itself dated 2026-01-09, so an
exclusive bound would drop the carryover and the entire pre-cutover balance.

Consequence: **pre-2026 transactions can be split freely.** A split on a 2024
grocery shop describes how the expense was shared — which is what stops it
inflating spend — without asserting a debt. 657 pre-2026 transactions carry
1,266 such splits, imported from SplitMyExpenses and marked `settled`.

**Spend counts settled splits; owed does not.** `myShare`/`mySplitOf` must NOT
filter on `settled` — half a 2025 grocery shop was your expense whether or not
the other half was repaid. Filtering it out re-inflates exactly the figures the
historical import exists to correct.

**Every split totals 100%, and the payer's row is written down.** `myShare`
resolves the payer's share as `100 - SUM(everyone else)`, so a 50/50 stored as
a lone "Sonu 50%" row still computed correctly — and still read on screen as a
50% share against a blank. `completeSplit` (`src/lib/splits.ts`) is the single
place that materialises the remainder, and every write path ends in it:
`applyRuleActions`, `POST /api/transactions`, the Slack nudge's share button,
and the rule-revert restore. `POST /api/transactions/[id]/splits` needs no call
— it already rejects anything not summing to 100.

The remainder always goes to the transaction's **owner**, never to "me". The
owner's row on their own transaction is excluded from both halves of
`getParticipantBalances` (`ts.participant_id != $1` on transactions I own; the
converse on ones I do not), so writing it cannot create, enlarge or discharge a
debt. A row for *me* on someone else's transaction is a real obligation — never
synthesise one. This is what made the 7-row backfill in `22e4a1e` safe;
balances were byte-identical across it.

There is no database-level constraint on the sum. Enforcing it needs a deferred
constraint trigger, and the rule path commits its DELETE and INSERT as separate
autocommitted statements, so the trigger would reject the intermediate state.

`share_percent` has a CHECK of `> 0 AND <= 100`, so a 0% row cannot be stored —
when the others grow to cover the whole amount, the owner's row is deleted
rather than zeroed.

**Un-sharing needs DELETE, not an empty POST.** The splits route rejects an
empty array ("splits array required"), so `DELETE /api/transactions/[id]/splits`
is the only way to clear. The order panel's "Shared 50/50" toggle was inert in
both directions until `22e4a1e` because it posted a lone 50% row to share and
`[]` to un-share, and the endpoint rejected both.

**Any split write path that deletes-and-recreates must carry `settled` across.**
`POST /api/transactions/[id]/splits` did not, and silently converted discharged
obligations into live debt — $37,233.28 was exposed. Fixed in `6add958`.
`rule-actions.ts` is safe only by the shape of its upsert
(`ON CONFLICT DO UPDATE SET share_percent` never touches the flag).
The rules-apply revert route restores it explicitly.

**Settling up is recording a payment.** There is deliberately no "mark settled"
action. `settled` marks obligations discharged *outside* this app; doing both
would subtract the settlement twice.

**Payments carry scope, and one transfer can carry several rows.**
`split_payments.trip_id` (migration 0022) says which tab a payment settles;
NULL is the ongoing household tab. There is no unique constraint on
`linked_transaction_id`, so a grouped transfer is recorded as one row per scope
that re-add to the transfer — that is how Sonu's $3,779.33 and $4,794.06 were
allocated Europe-first with the remainder to household.

**Trip owed must be owner-scoped; trip cost must not be.** The owed query
applies `OWNER_SCOPE`; without it a debt between the *other two* participants
reads as owed to you ($1,605.49 on Europe 2026). Trip *cost* deliberately counts
every payer — a trip cost what the group put into it — which is why the stat card
says "all payers, net of refunds". Do not "fix" the missing scoping there.

**Duplicates are superseded, never deleted.** `transactions.superseded_by_id`
(migration 0023); 31 rows / $42,040.68 from overlapping ANZ statements 107/142/143.
Every child of `transactions` is `ON DELETE CASCADE`. The exclusion lives *inside*
`EXCLUDE_RECONCILED_SOURCE`, so any query applying that fragment gets it free —
and any query that does not still double-counts.

**Refunds:** a *partial* refund is netted in SQL (`NET_SPEND_ROWS`/`SPEND_SIGNED`);
a *cancelled* booking has both legs untagged from the trip by hand, because a
trip never incurred a cost it cancelled.

**Trips:** Europe 2026 (id 1, 19 Mar–12 Apr), Auckland 2026 (id 2),
Europe — Sonu + Sunny (id 3, 12–28 Apr, created 2026-07-28 from tag 5),
Singapore + Bangkok 2026 (id 4).

### Why `travel` looked useless on a trip page, and the fix (2026-08-02)

`travel` was ~60% of every trip and told you nothing. The tempting fix — a finer
travel taxonomy (flights / stays / getting around) — needs a hand-maintained
merchant list, which is the trap ticket #19 already describes. It is also the
wrong diagnosis.

**Measured on Europe 2026, `travel` is the only category that spans both phases
of a trip. Every other category is 100% on-the-ground — dining, transport,
entertainment, groceries and shopping are all exactly $0.00 before departure.**
So the category chart was not a bad chart; it was two different economies stacked
into one, and travel was the only thing visible in the union.

The fix is to split on `trips.start_date` and use the axis that carries
information in each phase:

- **Booked ahead** (before `start_date`, **$21,229.56 / 56%** on Europe, 30
  bookings) — everything is a flight, a stay or a rail ticket, so category is a
  constant and **merchant** is the axis: Agoda $4,491, Air India $3,454, Azwebin
  $2,696, Luxury Escapes $2,463.
- **On the ground** (on/after `start_date`, **$16,946.77**, 180 charges) — travel
  falls to $8,241 among dining $4,452, transport $2,938, entertainment $698,
  groceries $589, shopping $29. **Category is finally worth charting.**

These are the **net** figures the page shows, and they are lower than a raw
`SUM(amount)` by design: the phase queries apply `NET_SPEND_ROWS` / `SPEND_SIGNED`
and `EXCLUDE_RECONCILED_SOURCE`, the same fragments as every other analytic. Raw
sums give $22,050.51 committed and Luxury Escapes at $3,283.80 — the $820.95 gap
is a partial refund on that booking. Do not "fix" the difference; a partly
refunded booking must not read at full price.

`getTripAnalytics` returns `phases`, `committed_merchants`, `on_ground_categories`
and `on_ground_daily`. A trip with a NULL `start_date` has no knowable departure,
so the SQL folds everything into on-ground rather than reporting it as committed.

**The daily rate is the only figure comparable between trips**, because totals are
not — trips differ in length. Europe $677.88/day, Sonu + Sunny $573.57, Singapore
+ Bangkok $309.76, Auckland $83.39. The page ranks the current trip against the
others from the already-loaded `useTrips()` list.

**A trip with near-zero committed spend is a filing artefact, not a cheap trip.**
Europe — Sonu + Sunny shows $184.84 committed against Europe 2026's $22,050.51
because both legs' flights and stays were filed on the first trip. The page says
so rather than letting the ratio read as missing data.

Two chart rules this page now follows, both from the `dataviz` skill and both
previously broken here:

- **One series → one colour.** The category bars use a single copper hue with the
  category as a direct label. `CATEGORY_COLORS` was a per-bar rainbow, which
  double-encodes identity the label already carries — and the trip subset **fails**
  CVD validation on this surface (`other` ↔ `shopping` ΔE 5.0 protan, below the
  floor of 6). Do not reintroduce per-category colour on a labelled bar chart.
- **No serif and no `tabular-nums` on the hero figure.** Fraunces is for section
  headings; a display face on a large number reads as decoration, and equal-width
  digits make it look loose.

The phase split bar is two ordinal steps of one hue (`#7c4820` → `#d28a47`),
validated with `--ordinal` against the `#171410` card surface, with a 2px gap so
the boundary is an edge rather than a colour change. Both segments are
direct-labelled, so it needs no legend.

### Trip participation is derived, and a trip is shared

Rebuilt 2026-08-02. Trips were scoped to `trips.owner_id`, so Sonu saw **no
trips at all** despite paying for 104 of the tagged rows herself — her own
spending was invisible on the only page organised around it.

**A participant is anyone with a split on, who paid for, or whose payment is
scoped to, a transaction tagged to the trip.** Derived (`TRIP_PARTICIPANT` in
`queries.ts`), never stored. A membership table was designed and rejected: it
would be a second record of a fact the expenses already carry, and two records
of one fact drift — the same reason sharing is a real split rather than a flag.
The derivation also gets the exclusions right for free, which a table has to be
kept in sync to do: Singapore + Bangkok 2026 has no Sonu split and no Sonu
payment, so she is not a participant and never sees it. Live result is
Siddharth 4 trips, Sonu 3, Molina 1.

**Everything is shared except delete.** Read, edit and assign are open to any
participant. `deleteTrip` stays `owner_id`-only because both trip foreign keys
are `ON DELETE SET NULL`, so deleting Europe 2026 untags 210 transactions *and*
NULLs the trip scope on 6 payments — which is where the hand-derived
Europe-first allocation lives, and nothing recomputes it. The route returns 403
with the reason rather than a 404 that pretends the trip is missing.

**`getTransactions` gained `trip_all_rows`, and it is opt-in for a reason.** A
participant sees every row on a trip, not only their own — the trip total
already counts every payer. It must NOT be implied by `trip_id` being present:
`GET /api/transactions` is also the main transactions list, and its trip filter
has to keep owner scoping or filtering your own ledger by "Europe 2026" would
quietly fill it with someone else's rows. Participation is re-checked in SQL, so
passing the flag for a trip you are not on returns nothing rather than
everything. Only `trips/[id]/page.tsx` sets it.

**Trip owed is pairwise and returns BOTH directions, never netted.** `owed` is
unchanged — their share of rows *the viewer* paid. `i_owe` is the mirror: the
viewer's share of rows *that participant* paid. Rendering the pair from the
viewer's side is the whole fix; an obligation lives on a row someone else paid
for, so a viewer-as-payer figure can never contain it, and Sonu's Europe 2026
read "you are owed $2,408.24" while omitting the $8,004.04 she owed.

**The API returns both halves whole; the trip page nets them for display.** One
figure per person, with the breakdown beside it, because a net nobody can
decompose is how a wrong figure survives.

**A NEGATIVE trip net is not a bill — this is the trap, and it was got wrong
twice.** A payment is allocated to a scope as a lump sum, and the grouped-payment
allocation gave each trip enough to clear the payer's **gross** share. So netting
the other side off leaves a fully-paid trip negative by exactly what the payment
over-covered: Europe reads **−$802.75** because Sonu paid $8,004.04 against a net
share of $7,201.30. That surplus is already carried in the overall balance —
**she still owes $5,313.38 overall** — so labelling it "you owe them" was flatly
wrong. Scope nets sum to the overall figure; a negative simply means this scope
was over-covered and the excess sits in another.

The discriminator is `paid_to_me`:

- negative **with** a payment into the scope → over-covered, nothing to pay
- negative **with no** payment → genuinely owed, because the viewer's share of
  the other person's spending exceeds theirs

All three of today's negatives are the first kind (Europe Sonu −$802.75,
Sonu + Sunny −$936.34, Europe Molina −$816.16). Both cases are tested. The trip
table therefore carries an **Overall balance** column from the unscoped
`getParticipantBalances` — the trip figure alone cannot tell you whether to pay
anyone, and **settlement is always against the overall figure, never one trip.**

**On whether to net — the reasoning reversed once, and the second answer is the
right one.** The first objection was that the grouped-payment allocation (memory
case `allocate_grouped_payments`) cleared each trip against the *one-directional*
gross, Europe first with the remainder to household, so netting would redefine
that debt after the fact. Checking the underlying rows overturned it: Europe's
$802.75 is **56 real transactions Sonu paid** across Rome, Venice, the Dolomites,
Bellagio, Lucerne and Paris on which Siddharth holds 25% — and `paid_by_me` is
**$0.00 on every row of every trip**, because nothing has ever been recorded
going from him to her. The one-directional view was concealing a live obligation,
not protecting an allocation. Her side was paid in full and looked settled only
because the allocation derived her payment split *from* her gross, so it lands on
zero by construction.

Current nets: Auckland Sonu **+$1,077.25** (1,505.64 − 428.39), Europe Sonu
**−$802.75**, Sonu + Sunny **−$936.34**, Europe Molina **−$816.16** — the three
negatives all being over-coverage, per the rule above. The `owed` column itself
was verified byte-identical when the mirror was added — $1,505.64, −$816.16,
$0.00, $0.00.

**A payment left on the household tab makes a settled trip debt read as
outstanding.** Payment 5 (Molina → Sonu, $1,605.49) discharged the Europe debt
between those two but carries `trip_id IS NULL`, so a trip-scoped net cannot see
it and Europe still shows it owing. That is the cost of scope being optional, and
the reason the Record Payment modal now asks. Fixable per row with
`UPDATE split_payments SET trip_id = 1 WHERE id = 5` — not done, it is a data
decision.

### The Shared view shows payer and category, and is searchable (2026-08-02)

`getSharedTransactions` already returned `owner_name` and `effective_category`;
the table simply never rendered them. **Paid by** sits next to **Splits**
deliberately — together they are the two halves of the question the page exists
to answer, whose money went out and whose share it was. It shows the *effective*
owner (`COALESCE(t.owner_id, s.owner_id)`), which is the account the spend left,
and the same figure every balance on the page is computed from. Category uses the
override-first COALESCE, so a correction made anywhere shows here.

Search is **client-side**, unlike the transactions page. This endpoint returns
every split row in one request (1,267 today) with no pagination, so there is
nothing for a server round-trip to narrow, and the sort was already client-side.
It matches description, merchant, notes, category and payer — deliberately **not**
participant names, because the participant dropdown already does that and typing
"sonu" matching every row she is split on would read as broken.

### Payment scope reaches the API (2026-08-02)

`split_payments.trip_id` has existed since migration 0022, but `POST
/api/split-payments` never read it and `GET` never returned it — so **every
payment recorded through the app landed on the household tab**, and the 9
trip-scoped rows had to be written by hand in SQL. A $11k Europe settlement was
silently reducing the ongoing household balance.

Both fixed. The modal has a "Settles" selector (Household or a trip) and history
shows each payment's scope as a chip. **"Both" needs no new shape:** one
transfer becomes one row per scope sharing a `linked_transaction_id`, which is
why there is deliberately no unique constraint on it — tx 4121's $4,794.06 sits
as $1,145.52 against Europe — Sonu + Sunny and $3,648.54 against household, and
tx 4111's $3,779.33 spans two trips. All six linked transfers reconcile to the
cent.

### Three write paths that had no authorisation

All closed 2026-08-02. Each was reachable by any authenticated participant:

- **`assignTransactionsToTrip`** took no caller and checked nothing, so
  `PATCH /api/trips/[id]/transactions` and `POST /api/transactions/bulk`
  (`assign_trip`) let anyone move any transaction id into any trip id. Not being
  able to *see* a trip was no obstacle, because the write path never read one.
  Now: only rows the caller can already see move, and a non-null destination must
  be a trip they participate in — enforced in the query, not the route, so
  neither caller can bypass it. Returns the count actually moved.
- **`DELETE /api/split-payments?id=`** deleted by id with no check at all. Erasing
  a settlement silently resurrects a discharged debt — the same class of damage
  as the split rewrite that reset `settled`. Now limited to the two people the
  payment is between.
- **`POST /api/split-payments`** accepted any `from`/`to` pair. Now the payment
  must involve the caller, and a trip scope must be a trip they are on.

**Partial split coverage inside a category is usually correct, not a gap.** Only
*shared* items are split. `utilities` sits at 69% yours because Globird, OVO, GWW
and home telecoms are split while Telstra, Vodafone, Optus and JB Hi-Fi Mobile
are personal. `subscriptions` is 91% because Uber One, Amazon Prime and OnePass
are shared while Claude, OpenAI, Anthropic, OpenRouter, You.com, LinkedIn, Xero,
Billdu, Spotify and Patreon are not. `fees` and `charity` are 100% yours and
correct. Check the merchants before concluding a rule was never applied — a
category-level ratio that "looks wrong" usually is not.

**Still true, and still a caveat:** Sonu's loan contributions (`…emi` in the
offset account, 39 rows, $37,980.24) are categorised `transfers`, indistinguishable
from ordinary internal transfers. The loan model below is unbuilt.

### Order verdicts — "never order from here again"

Built 2026-07-28 (migrations 0024, 0025). `order_reviews` was previously a table
wired to nothing; it now backs `GET`/`PUT /api/transactions/[id]/review`, the UI
in `components/order-details.tsx`, and the Slack card in
`lib/slack-blocks.ts` + `app/api/slack/interactive/route.ts`. Logic in
`lib/order-reviews.ts`.

Five things about the shape, each load-bearing:

- **Per person, not per order.** `UNIQUE (transaction_id, participant_id)`. A
  shared meal produces two opinions that routinely disagree, and the
  disagreement is the useful part. `PUT` defaults to the **signed-in user**,
  not the owner — Sonu authenticates through the same Traefik OAuth as
  participant 4, so an owner default would file her verdict under his name.
- **Five levels** — `loved`, `liked`, `ok`, `bad`, `never`. Three collapsed the
  distinction that decides a re-order; `bad` was added because the jump from
  `ok` to `never again` is too big and most disappointments live in the gap.
- **Only `never` sets `warn`.** A blacklist that fires for every mediocre meal
  is one nobody reads. `bad` and `never` both set `order_again = false` — you
  would not choose either — but only `never` raises the alarm on a future
  order. "Would I order it" and "warn me about it" are different questions.
- **Item verdicts key on the item DESCRIPTION**, not its index — an index is
  meaningless across orders, and "the Pad Thai here is good" has to survive
  into the next order from the same merchant. Pooled case-folded across the
  merchant's orders. Only `loved`/`never`: a per-item "ok" answers neither
  question you ask at order time.
- **An ABSENT `item_verdicts` means "leave them alone"; `[]` clears them.**
  Without that distinction a note-only save wipes every per-item opinion — the
  same shape as the bug that reset `settled` on split rewrites, and just as
  invisible on screen. Mutation-tested.

**The merchant is the restaurant, not the courier.** `orderDescription` does not
append the platform — that was added on request and reversed on 2026-07-28,
because it fragmented the merchant and the platform already renders in the Order
details panel. 81 descriptions were backfilled. `merchantVerdict` joins
**case-folded**: the platforms capitalise differently (`TEG Kebabs & Biryani` vs
`TEG KEBABS & BIRYANI`) and an exact match kept two separate histories, so a
"never again" through one app never warned in the other.

The merchant signal is *derived* by aggregating on
`expense_metadata.merchant_normalized` — never `transactions.merchant_name`,
which is a bank descriptor.

**Sharing is a real 50/50 split, not a flag** — the split already IS the record,
and two records of one fact drift apart. The toggle **refuses** when a
participant outside {1, 4} is present or the second consumer's share is not 50:
splits are made by hand here, so a third party or an uneven share is deliberate
and one tap must not flatten it. It says which case it refused on.

`/api/orders/ingest` returns `prior_verdict` (so the nudge can warn inline) and
`slack_blocks` (so Block Kit stays in tested code rather than n8n expressions).

### Slack cards — two rules that cost real data

1. **Update via `response_url`, never the HTTP response body.** Block Kit
   interactivity ignores the response body; replacing a message that way is
   legacy attachment-style behaviour. Getting this wrong meant every press wrote
   correctly and left the card stale, so a working button looked dead, got
   pressed again, and toggled itself back — three splits were lost before
   `conversations.history` showing `edited: false` settled it. `response_url`
   needs no bot token, so the app posts it directly.
2. **Never hand-write a card.** A card built with a guessed `shared: false`
   mislabels an already-shared order and the button then deletes the split.
   Render by calling the interactive endpoint with a no-op verb (`<id>:noop`):
   it writes nothing and returns blocks built from live state.

Slack reaches this route through an n8n webhook, not directly — see the
smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory.

### The shared loan

The loan is a **separate ledger**, not a shared expense and not a settlement
context — a contribution must never be able to settle a dinner. Sonu's obligation
is a fixed 50% of the repayment; actual contributions vary, and the difference is
a tracked receivable ($4,000.00 over 2025-07 → 2026-06).

Do not derive the share from actual payments. During her leave the obligation did
not change, only the payment did — a percentage-of-actual model would silently
redefine her share as 30% and make the shortfall vanish.

Loan interest reconciles exactly: `repayments − interest − fees = balance
reduction`. It stays categorised `loan_interest` and counts as spend — over 12
months $63,500 of cash left and debt fell $44,127.36, and the $16,523.64
difference bought nothing. Excluding it would leave the balance sheet unable to
reconcile cash out against equity gained.

**The repayment is voluntarily above contracted, and the gap is the largest
flexible cost in the whole picture.** Contracted is $1,190.54/fortnight
($2,579.50/mo annualised); the actual direct debit is $2,500.00/fortnight
($5,416.67/mo). That is $2,837.17/mo of overpayment, and it is *not* sunk — it
shows up as `statements.redraw_available`, which grew $62,387.17 → $81,017.42
across the two most recent loan statements. Sonu returned to $1,250/fortnight in
July 2026 after the reduced $750 period during her leave.

Treat the repayment as two figures whenever asking "what does this cost me":
the contracted floor and the actual. `scheduled_repayment` holds the actual
($2,500), not the contracted minimum — the contracted figure is not in the DB at
all. See `docs/expense-baseline.md`.

### Import Date (`created_at`)

`transactions.created_at` is the import timestamp (DB default `now()`). In the transactions and shared views, the "Imported" column shows:
- For statement transactions: when the statement was processed by N8N
- For reconciled transactions: the `created_at` of the original manual/CSV transaction (via `LEFT JOIN transactions src ON src.reconciled_with_id = t.id`) — so the original import date is preserved post-reconciliation

Use `created_at` (not `transaction_date`) to answer "what was added since the last settlement?". Sort by `created_at` is supported server-side in `getTransactions` and client-side in the shared view.

### Rules System

Conditions are AND-evaluated. Fields: `merchant_normalized`, `description`, `category`, `bank_name`, `amount`, `transaction_type`. Operators: `contains`, `equals`, `starts_with`, `gt`, `lt`, `not_equals`. Actions: `set_category`, `set_merchant`, `add_tag_ids`, `apply_split`.

`contains` and `equals` operators are case-insensitive (both sides `.toLowerCase()`).

**A rule with zero conditions matches every transaction.** Both apply paths use
`conditions.length === 0 || conditions.every(...)`. Rule 43 "Home 50/50 Sonu" has
no conditions and a 50/50 split action — applying it blindly would split all
~3,700 transactions with another participant. That is what `manual_only` is for:
those rules are excluded from bulk runs and fire from the transactions page
against a hand-picked selection.

### Previewing a rule before applying it

`GET /api/rules/[id]/matches` is a dry run — it writes nothing and returns only
the transactions a rule would actually *change*, with already-correct rows
summarised as a count. The Preview button on the rules page uses it.

Apply then goes through `POST /api/transactions/bulk` with `action: "apply_rule"`
and **explicit transaction ids**, not the conditions. That is the safety
property: a rule whose conditions are too broad cannot reach further than what
the preview showed and the user ticked.

Prefer this over auto-applying rules on ingestion. It fails safe, works
retroactively, and tells you which rules are consistent enough to automate later.

### Rule apply history

`rule_apply_runs` snapshots the before-state so a run can be reverted, and since
migration 0017 also records `rule_id`, `rule_name` and `source`
(`all` | `rule` | `selection`). `rule_name` is denormalised deliberately and
there is no FK to `rules` — history must stay readable after a rule is renamed or
deleted, and deleting a rule must not cascade away the audit trail.

`GET /api/rules/runs/[id]` diffs that snapshot against current values. Rows
changed by something else since the run are flagged, because reverting restores
the pre-run value and discards the later edit.

### Importing an aggregator/bank CSV (`/api/import/csv`)

There is **one** import path — the CSV import modal — and Frollo goes through it
like any other file. A bespoke Frollo importer, API route, CLI and two scheduled
n8n workflows existed for a day and were deleted on 2026-08-13: net −1,152 lines.

**The rule that made the rest unnecessary is statement coverage.** Each account's
newest `billing_end_date` is a watermark; a row on or before it is already in the
ledger. On the real 2,607-row export that leaves **171 rows** — with no account
allowlist and no credit-card exclusion, because cards have current statements and
drop out on their own. Every bit of the deleted apparatus was doing by hand what
`getStatementCoverage()` does generically.

Note what this replaced. The first attempt asked whether a row's date fell inside
a statement's min–max *window*, which for accounts whose statements span 182 to
460 days swallows a year and answers nothing — it let 422 duplicates into 550
rows. Amount-matching cannot substitute either: the two sources decompose the
same event differently, bundling a Wise transfer fee into the transfer (10001.13)
where the statement itemises it (10000.00 + 1.13).

**Map these optional columns or lose something silently:**

| Column | Without it |
|---|---|
| **Account** | nothing is excluded — you review the whole file |
| **Currency** | a foreign row is stored as if AUD (USD 10,782 → A$10,782) |
| **Row ID** | a re-import duplicates instead of no-opping |

Leave **Category** unmapped for an aggregator: its spend categories are not
trusted, and you set them per row in the review step.

**Two bugs fixed the day this shipped, both of which the flow depends on:**
`batchInsertCSVTransactions` accepted a `category` and then omitted it from the
INSERT column list, so every category chosen in review was discarded and the
trigger wrote `'other'` — not cosmetic, because an uncategorised credit is
admitted by `NET_SPEND_ROWS` and negated by `SPEND_SIGNED`, so 62 imported
transfers cancelled **$74,338** of spend while counting as no income. And the
path had no idempotency at all: `row_index` is assigned MAX+1 every run, making
`uq_transaction_identity` structurally unable to fire. It now writes
`source`/`source_ref` with `ON CONFLICT DO NOTHING`.

**The in-file duplicate warning is not redundant with `source_ref`.** A CDR
re-consent re-exports an account's whole history under fresh ids while the
originals survive — 385 twins in one 2,563-row export, doubling every salary
payment. Fresh ids mean `source_ref` sees new rows, and 385 is not visible by eye
in a review table. Consents expire annually, so expect it.

Undo an import: `DELETE FROM transactions WHERE source = '<source>'`.

### Zip, and the three things a BNPL account needs (2026-08-15)

24 statements, Dec 2020 → Jun 2026, all balance-asserting at $0.00. Zip is a
credit account, so importing it changes how three existing row types must read.

**1. A repayment leaving the bank is a duplicate ONLY where a Zip statement
covers the purchases.** Zip's own `Payment` rows are already `transfers`. On the
bank side, `ZIPPAY*` debits are the same money — but only 2 of 8 fall inside a
Zip statement period. Those two are now `transfers`; **the other six must stay as
spend**, because for those periods the bank debit is the only record the purchase
ever happened. Moving all eight would have deleted $703.35 of real spend. Same
statement-coverage rule as the CSV importer, and the same trap.

**2. `Account Credit` is cashback, and Zip labels its three money-in types
differently.** `Payment` = repayment from the bank. `Refund <merchant>` = a
genuine merchant refund. `Account Credit` = Zip cashback — 34 rows, $740, round
amounts ($20 ×20, $30 ×10, $10 ×4), with no matching debit from any bank account.

Gemini filed the identical description two ways — 23 as `other` (which *subtracts*
from spend) and 11 as `transfers` (excluded entirely) — so one thing was doing two
opposite things. All 34 are now `shopping` + `credit`, **netting against spend**:
cashback lowers what a purchase actually cost, which is how `NET_SPEND_ROWS` /
`SPEND_SIGNED` already treat a refund. Filing it as `income` was rejected for the
reason the Up item sales note gives — it flatters `net = income − spent` and mixes
non-earnings into the salary line. Ties to the **Cashback tracking (ING-6)** epic;
a dedicated `cashback` category would need the CHECK constraint, the TS mirror,
and a decision in every analytics fragment, so it was not done here.

**3. A gift card stays spend.** Prezzee is $1,580 of the history. Stored value is
tempting to call `transfers`, but that only works if every downstream purchase is
logged — it is not, for 2020. Identical reasoning to the ATM-withdrawal rule.

All 137 Zip rows are categorised; net spend is $11,135.11 against $10,085.11 of
`transfers`. **Statement 200 (Sept 2021) came through incomplete** — no
`billing_end_date`, no balances, 2 rows — so it is the one statement not covered
by `uq_statement_identity` (the index is partial on `billing_end_date IS NOT
NULL`) and a re-import would duplicate it. Worth re-processing paperless doc 296.

### Trusting extracted statement data

**Balance assertions are the check that works.** `getStatements` computes
`opening + movement − closing`; the statements page flags any statement that does
not reconcile. Sign depends on what the balance means — on a credit card or loan
it is what you owe, so spending increases it; on a transaction or offset account
it is what you hold. 11 pre-existing statements currently fail, ~$4,177
unexplained, including two adjacent ANZ statements off by exactly ±$230.38 (a
transaction filed against the wrong one).

**Do not derive `opening_balance` from `closing − movement`.** It is an accounting
identity, so every statement would reconcile and the check would go permanently
green. A null that reads "unverified" is worth more than a number that is right
by construction. For the same reason, do not add a totals assertion comparing
`total_debits` to the summed rows — those totals are now *computed* from the rows
(see the N8N `Parse Gemini Result` node), so that check can never fail.

**Gemini invents summary fields the statement does not print.** Wise PDFs show
only a closing balance; asked for an opening balance anyway, the model produced
11,277.08 against a truth of 0.00, and on another statement read the running
balance of the oldest row. Every transaction was extracted perfectly in both
cases — verified row for row against the CSV exports. When a balance assertion
fails, suspect the summary before the transactions.

**Gemini drops rows silently on long tables.** `finishReason` was `STOP`, not
`MAX_TOKENS`, so raising `maxOutputTokens` does not help. This did *not* actually
occur on the Wise imports (that was the summary bug above), but it is why an
empty statement must not throw: a document that errors never gets tagged, so it
is re-fetched every poll forever and blocks everything behind it in the queue
(`ordering=-created`, `page_size=1`).

**FX is per transaction date**, via Frankfurter (ECB daily, free, no key), with
weekends resolving to the prior publication. A single spot rate across a 15-month
statement is wrong by up to 20%. Wise's own rates are more accurate in principle
but differ by only 0.05% and exist on 44 of 194 rows, so mixing bases is not
worth it.

**When comparing CSV exports to extracted data, order by full timestamp
including milliseconds.** Two of one statement's rows are 1ms apart; dropping the
fraction reversed them and produced a bogus opening balance.

## Development Patterns

### Adding a new API route

1. Create `src/app/api/<resource>/route.ts`
2. Always call `getCurrentUser(req)` first; return 403 if null
3. Write SQL in `src/lib/queries.ts` using `queryRaw`
4. Add a TanStack Query hook in `src/lib/hooks.ts`

### Adding a new condition field to rules

Two files only:
- `src/app/api/rules/apply/route.ts` — add to `Condition.field` union, `TxFields` interface, and `evaluateCondition()` switch
- `src/app/rules/page.tsx` — add to `FIELDS` array; add special rendering if needed (e.g. enum dropdown for `transaction_type`)

### Modifying queries

- All JOINs to `statements` must be `LEFT JOIN` (manual transactions have no statement)
- Owner filter pattern: `WHERE COALESCE(t.owner_id, s.owner_id) = $1`
- Bank name pattern: `COALESCE(s.bank_name, 'Manual') as bank_name`

Analytics queries must import the fragments from `src/lib/analytics-sql.ts`
(`STATEMENTS_JOIN`, `OWNER_SCOPE`, `EFFECTIVE_CATEGORY`, `EXCLUDE_NON_SPEND`)
rather than hand-rolling them. Two failure modes they exist to prevent:

- An `INNER JOIN statements` + `WHERE s.owner_id = $1` silently drops every
  manual/CSV transaction (`statement_id IS NULL`).
- Spend must exclude the `transfers` and `investment` categories. Once bank
  statements are imported alongside card statements, a credit-card payment
  appears twice — as a debit leaving the bank account and as the underlying
  purchases on the card statement. Excluding `transfers` is what nets it out.
  Use the `EXCLUDE_NON_SPEND` fragment: a bare `category NOT IN (...)` evaluates
  to NULL for uncategorised rows and drops them from totals.

### Spend is gated on MY_SPEND_SCOPE, not OWNER_SCOPE (2026-08-15)

**Ownership is not a precondition for an expense being yours.** Every spend
analytic used to gate on `OWNER_SCOPE = $1`, with `mySplitOf` scaling *within*
that gate — so your half of a shop Sonu paid for counted as **zero**, in monthly,
daily, merchants, subscriptions, fees and the budget page. 167 rows / **$3,210.91**
across Jan–Jul 2026, worst in April (+$1,354.83, the Europe trips). Meanwhile
`getParticipantBalances` booked the matching debt correctly, so the app could say
you owed her for a shop while insisting you had not spent anything on it.

Use `MY_SPEND_SCOPE()` — `owner = me OR I hold a split` — for anything measuring
what *I* spent. `OWNER_SCOPE` is still right for anything measuring an **account**:
the income and investment lines in `/analytics/monthly`, and the statement-level
fee rollup in `/analytics/fees` (which reads `statements`, where splits are
meaningless).

**`myShare` had to change with it, and widening the gate alone would have been
worse than the bug.** Its old third fallback, `100 - everyone else`, is the
*payer's* remainder — on someone else's unsplit row it returns 100 and would have
moved their entire bill onto you. It now branches on ownership:

- **My row** — unchanged: explicit split, then `my_share_percent`, then the
  remainder.
- **Their row** — an explicit split row only; absent means **0**.

`my_share_percent` is deliberately not consulted on someone else's row. It is one
unscoped column on `transaction_overrides` writable by anyone who can see the row,
so "my" can only mean the owner's. All 402 rows carrying one today are owner-side.

That 0 is what makes the wider gate safe: admitting a row can never add more than
the share actually held. Tested both ways in
`src/__tests__/integration/analytics-sql.test.ts` — the discriminating cases are
all ones where you hold **no** split on someone else's row, because that is the
only place old and new disagree.

`MY_SHARE_PCT` in `queries.ts` mirrors `myShare` in subselect form for the
transactions list (which has no viewer-scoped `ts` join). A test asserts the two
agree row for row across seven fixture shapes — keep it that way, since the list
is the drill-down for these totals.

**No historical restatement:** every non-owner split is 2026-dated. The 1,266
pre-2026 SplitMyExpenses splits are all on rows you own, so nothing before the
cutover moves.

### Changing who paid (2026-08-15)

Owner was **write-once for every ingestion path** until now — a pantry receipt
hardcodes `DEFAULT_OWNER_ID` (`receipt-ingestion.ts`), and there was not one
`UPDATE ... SET owner_id` in `src/`. A shop the other person paid for was
permanently filed as yours.

Two routes, because the owner lives in two places:

- **`PATCH /api/transactions/[id]` with `owner_id`** — manual rows only
  (`statement_id IS NULL`). A statement row returns 400 `statement_owned` and
  points at the statements page; its effective owner is
  `COALESCE(t.owner_id, s.owner_id)`, so writing it here would either no-op or
  detach one row from the account it was extracted from.
- **`PATCH /api/statements/[id]` with `owner_id`** — the statements page has had
  this dropdown since it was built, wired to a route with **no PATCH handler**.
  Every change 405'd, and `useUpdateStatement` never checked `res.ok`, so it
  failed silently and the select just snapped back.

**The statement route writes both tables.** 2,194 statement rows carry their own
`owner_id` against 1,803 that inherit, so updating `statements` alone moves less
than half and splits one account's history between two people. It updates rows
matching the *old* owner and returns `rows_moved`; all 2,194 agree today, and a
row that disagrees was set deliberately and is left alone.

**Reassignment is a one-way door, and the guard is the point.** Access is
`owner OR holds a split` (`canAccessTransactions`), so handing a row over while
holding no split removes it from your list and 404s every route that could put
it back — only the new owner can undo it. The route returns **409
`would_lose_access`** and the modal offers the two real ways forward: add your
split first, or "Give it away anyway" (`release: true`). Taking a row *onto*
your ledger is never blocked — there is no door to close behind you.

Reassignment is a correction to a row you already hold, never a way to reach one
you do not: `canAccessTransactions` runs first, so claiming a stranger's
transaction is a 404 before any owner logic runs.

**Splits are deliberately not rewritten.** They record shares, not direction —
`getParticipantBalances` derives who owes whom from ownership, so a 50/50 flips
from "they owe me" to "I owe them" untouched, `settled` included. Tested both
directions.

**Order matters when you do this by hand:** split first, then reassign. The
reverse locks you out, which is exactly what the 409 exists to stop.

`useUpdateTransaction` also gained the missing `res.ok` check. Without it every
rejection resolved as success — the modal closed, the list refetched, and the
edit silently vanished.

### Statement types

`statements.statement_type` is constrained to `credit_card | transaction |
savings | loan | offset | investment | other`. Migration 0013 added a
`normalize_statement_type()` SQL function plus a BEFORE INSERT/UPDATE trigger, so
the N8N workflow can keep sending raw free text (`'ACCESS ADVANTAGE'`, `'Business
Card'`) and the DB normalises it on write. The raw extracted value is preserved in
`account_type`.

The TypeScript mirror is `src/lib/statement-types.ts` — keep the list, the SQL
function, and the CHECK constraint in sync when adding a type.

### Bank names are normalised on write (migration 0031)

**`bank_name` is a component of the duplicate check.** `uq_statement_identity`
is `(bank_name, account_number, billing_end_date)`, so two spellings of one
institution mean the same statement can be ingested twice without the index ever
firing. Gemini reads the name off whatever the PDF prints, so the spelling varies
per *document*, not per account.

Zip exposed it: account `2705256` arrived under four names in sixteen minutes —
`ZipMoney Payments Pty Ltd`, `ZipMoney Payments Pty Limited`, `Zip Pay`,
`ZipPay` — with ten more in the queue. It was not unique. 17 `bank_name` values
represented 13 institutions, and every split pair shared an account number:
Amex (14/1), Citibank/Citi (8/4), NAB (3/1). Fragmentation also splits the
statements and transactions bank filters and `by_bank` in `/api/analytics/fees`.

`normalize_bank_name()` + `trg_statements_normalize_bank_name` (BEFORE INSERT OR
UPDATE OF bank_name) mirror the 0013 `statement_type` pattern, with **one
deliberate difference: the vocabulary is open.** An unrecognised name passes
through tidied, never collapsed to a fallback, and there is **no CHECK
constraint** — a bank this household has never used must be able to arrive
without a migration, and destroying its name on first contact is worse than
leaving it unmapped.

Matching is by **prefix, not by enumerated spelling** (`zip%`, `citi%`,
`%american express%`, `%national australia bank%`), so unseen variants like
`Zip Co Australia Pty Ltd` normalise with no code change. Branches for Westpac,
ANZ, HSBC, ING, AMP, Up and CommBank are no-ops today that map *to* the spelling
already in use — they exist to catch the legal-entity variant a future PDF might
print.

Wise is deliberately unmapped: `Wise Australia Pty Ltd.` is a single spelling, so
shortening it would be a rename nobody asked for rather than a merge.

**One institution's national entities stay apart (migration 0032).** A prefix
generalises past the evidence it was built on: every 0031 merge was provably one
account, but `citi%` would have flattened Citibank India into Citibank Australia
on arrival. `hsbc%` and `%american express%` had the same defect. The country now
comes from the name when the name states it and from the currency otherwise, with
AUD as home taking no suffix — so nothing renamed (`UPDATE 0`).

Currency alone cannot be the discriminator: **Wise holds AUD, EUR and USD
accounts under one provider**, so "non-AUD means a different bank" would shatter
it into three. That is why Wise must stay unmapped, and why mapping any
multi-currency provider through this function would be wrong.

`N.A.` is deliberately not a country marker — it means "National Association", a
US legal form printed on Citibank letterhead worldwide including India.

**The residual gap and its tell.** The function cannot know that two names for a
*new* bank are one institution. That always shows up the same way, so check the
`statement_identity_drift` view (migration 0033) rather than trusting the map.
Non-empty means an account is fragmented, by name or by number.

### Account identity is derived, not the raw number (migration 0033)

`account_number` fragments exactly like `bank_name` did, from the same cause —
Gemini copies what the PDF prints. ANZ's Access Advantage arrived as both
`4085-56264` (4 statements) and `408556264` (1). Since `uq_statement_identity`
keyed on it, re-importing one period under the other spelling evaded the
duplicate check.

`account_number_key` is a **GENERATED ALWAYS** column — `account_number` with
separators stripped — and the unique index is now
`(bank_name, account_number_key, billing_end_date)`. **Never write to it;
display `account_number`.**

Derived rather than rewritten because the raw number is the readable one and
some of it is structure: Up stores `633-123 / 176540052`, a BSB *and* an account
number, and flattening it would lose a distinction a human reads at a glance to
fix a machine problem. Case and the Amex mask (`XXXX-XXXXXX-01000`) are
preserved — `X` records which digits were redacted.

This is **not** what caused the documented 31-row / $42,040.68 ANZ duplication.
Statements 107/142/143 overlap on *different* end dates, which that index cannot
catch at any spelling. Different problem, same table.

`account_owner_mappings` also keys on `(bank_name, account_number)` with its own
UNIQUE constraint, so any future rename must update it too or strand its rows.
It is empty today; the migration handles it anyway.

**Normalising `bank_name` broke the N8N new-bank alert, and the fix lives in the
workflow.** `Check Known Bank` ran
`SELECT COUNT(id) FROM statements WHERE bank_name = '<raw Gemini name>'`,
comparing the raw extraction against the now-canonical column *before* the insert
— so the trigger had not fired yet and nothing ever matched. That branch tags the
document **pending and holds it for Slack approval**, so every Zip statement
stalled awaiting a manual click, not merely a noisy alert. It was a regression
made worse by normalisation: previously a repeat of the same spelling at least
matched.

Fixed 2026-08-15 by comparing like with like:

```sql
SELECT COUNT(id) as count FROM statements
WHERE bank_name = normalize_bank_name('{{ ...summary.bank_name }}', '{{ ...summary.currency }}')
```

Currency is passed so a genuinely new national entity (Citibank India) still
alerts. Verified live: the run after the fix took 12s against 30–53s for the
approval-branch runs before it. Workflow `FysADdFwEtwONQl4` in the smarthome
repo — **any future change to `normalize_bank_name()` must keep that node in
step**, since it is the one caller outside this codebase.

### Loans

A loan repayment is **not** an expense. It is part principal (equity, a
balance-sheet move) and part interest (the only part that is spend). Migration
0014 adds:

- `transactions.principal_amount` / `interest_amount` — populated only when the
  lender itemises the split on the repayment row itself
- `statements.interest_rate`, `scheduled_repayment`, `repayment_frequency`,
  `redraw_available`, `loan_term_months`

Two statement shapes, both handled:

1. **Separate rows** (the common Australian case) — the loan statement lists
   repayments and "Interest Charged" separately. `transaction_type` alone is
   enough: `interest` rows count as spend, `payment` rows don't. No split columns
   needed.
2. **Itemised repayment row** — some lenders print principal and interest on the
   repayment line. That row is typed `payment`, so it would be skipped entirely
   and its interest lost. The `SPEND_ROWS` / `SPEND_BASE` fragments in
   `analytics-sql.ts` handle it: a row with a non-null `interest_amount` counts
   as spend, valued at `interest_amount` rather than `amount`.

The N8N `Parse Gemini Result` node only accepts a split when both parts are
present *and* they sum to the row amount (±2c) — a half-extracted split would
silently misreport spend, so it is discarded rather than trusted.

Loan interest uses the `loan_interest` category; principal repayments use
`investment` (excluded from spend, surfaced on the investments line in monthly
analytics).

### The investments line is signed

A withdrawal from a fund is a **disinvestment**, not income. Units convert back
to cash; net worth is unchanged. `INVESTMENT_SIGNED` (`analytics-sql.ts`) makes
credits and refunds negative so they net against contributions, and
`/api/analytics/monthly` is the only consumer.

Summed unsigned, a withdrawal read as *more* money invested: March 2026 showed
$38,615.34 of investing in a month that was net **−$11,384.66**, because a
$25,000 Raiz withdrawal was added to an $8,563.80 IBKR deposit instead of
cancelling it. **Each credit costs twice** — once for being added, once for not
being subtracted — so the error is double the credit, $50,000 in that month.

Filing withdrawals as `income` is the other tempting answer and is worse: it
books an asset disposal as earnings and feeds `net = income − spent −
investments` with a flattering sign. Same reason the Up item sales in Known Gaps
do not belong on the income line.

**What this cannot resolve:** part of a withdrawal genuinely *is* income — the
capital gain. The bank descriptor is one gross figure with no cost base
(`TRANSFER FROM RAIZ WITHDRAWAL 7D5262D8A839248A12`), so it cannot be decomposed
from statement data. Netting tracks cash committed against cash returned and
leaves the gain for holdings data to surface; it does not assert the gain is zero.

Consequence for the UI: a net-disinvesting month is real data, so the budget page
gates on `!== 0`, not `> 0`, and renders negatives in amber.

### Prisma

The schema at `prisma/schema.prisma` covers all tables. The generated client (gitignored) must be regenerated after schema changes:
```bash
cd /mnt/m2cache/appdata/finance-app && npx prisma generate
```
Docker builds run `npx prisma generate` automatically. Do not commit `src/generated/prisma/` — it is gitignored.

## Agent / MCP Access

Agents read this DB through the read-only `postgres-personal` MCP server (lives in the
`personal-agent-gateway` repo, not here): `agent_ro` role, SELECT-only, SQLGlot guardrail,
100-row cap, every call audited to `mcp_query_log`. See `docs/agent-access.md` for the tool
list, the five analysis views, and per-client setup (Claude Code, Codex, Hermes).

Two things to remember when changing the schema: the agent views are created by
`smarthome/personal-agent/migrations/006_agent_read_role_views.sql` (not Prisma) and read
`transactions`/`statements`/`expense_metadata` columns directly — rename a column and they
break or go stale. And the views are **not** owner-scoped and do **not** merge
`transaction_overrides`, so agent numbers can differ from the UI.

## Known Gaps / TODOs

See `README.md` → **Known Gaps / TODOs** for full details.

**Payment provider tracking**: `merchant_normalized` currently conflates payment provider (PayPal, Afterpay, Zip) with the actual merchant. Plan: add `payment_provider` column, update Gemini prompt to extract it separately, backfill from `merchant_name` patterns, surface in UI filters.

### Open as of 2026-07-29

- **Shared expenses: loan section only** — the redesign in
  `docs/shared-expenses-design.md` is built and live as of 2026-07-28; the
  loan model at the end of that doc remains a proposal (Sonu's `…emi`
  contributions still read as ordinary transfers).
- **Expense baseline / emergency reserve** — `docs/expense-baseline.md`. One-off
  analysis, nothing built. Records four data corrections the raw numbers need
  (misfiled Raiz/super/brokerage debits, `other` credits read as negative spend,
  `government` conflating ATO with rates/rego, `fees` being mostly annual) and
  why only Feb–Jun 2026 is trustworthy for per-person figures.
- **11 statements fail the balance assertion**, ~$4,177 unexplained. Predates
  this work. One ANZ statement is off by exactly $0.50, traced to a misread digit
  in fee rows ($5.00 vs $5.50).
- **28 Up Bank debits are categorised `other`** ($3,760.74). Up only categorised
  16 of 88 rows. The `Payee` field is populated throughout, so merchant rules plus
  the rule preview should clear most of it.
- **Up item sales are categorised `income`** ($4,238.04 across 19 credits — iPad,
  drone, camera). Correct in that they are excluded from spend, but it mixes
  asset disposals into the income line alongside salary.
- **`payment_method` is not shown in the transactions list** — settable on create
  and edit only. Worth a column or filter if cash becomes routine.
- **Pantry receipts still land as yours.** `processReceiptIngestion` hardcodes
  `DEFAULT_OWNER_ID`; the ingest route takes no owner. Correctable per row now
  (see "Changing who paid"), but a "Paid by" step at capture time would stop the
  correction being needed.
- **Raw statement exports live in `dump/`**, gitignored since `31a8177`. They were
  committed by accident in `030490e` and remain in that commit's history; the repo
  has no GitHub remote, so exposure is limited to the local Gitea. Purging history
  was offered and not actioned.