CLAUDE.md: document the Frollo account feed and the new reconcile exclusion
ci / lint-test (push) Successful in 39s

Two things a future reader would otherwise have to reverse-engineer.

The reconcile queue now excludes account-feed rows for a reason unrelated to
payment method: a feed row is the account's own ledger entry, not a receipt
awaiting a statement line, because the importer only covers accounts whose
statements are deliberately not imported. Measured, the queue went from 8 to
558 without it.

And the de-duplication rule needs its history attached, because the obvious
version is wrong in a way that passes tests: keeping close-id rows as genuine
repeats survived 29 tests and a clean dry run while still doubling three
salary payments. In-scope duplicate pairs have id gaps from 58 to 260
million; real repeats sat at 1-4.
This commit is contained in:
2026-08-13 11:29:08 +10:00
parent 21e9e765a3
commit 3e826a317b
+59 -4
View File
@@ -131,7 +131,7 @@ docker exec postgres-personal psql -U personal -d personal < prisma/migrations/<
### Key Tables ### Key Tables
- `statements` — one row per billing period per bank account - `statements` — one row per billing period per bank account
- `transactions` — line items; `statement_id` is nullable (NULL = manual entry); `reconciled_with_id` links a manual tx to its matched statement tx; `payment_method` (migration 0016) is `card | cash | bank_transfer | other`, NULL = unknown - `transactions` — line items; `statement_id` is nullable (NULL = manual entry); `reconciled_with_id` links a manual tx to its matched statement tx; `payment_method` (migration 0016) is `card | cash | bank_transfer | other`, NULL = unknown; `source` / `source_ref` / `source_account` (migration 0028) identify a row that came from an account feed rather than a statement or a hand entry — `source_ref` is the provider's own id and carries a partial unique index, which is the only thing making a re-import idempotent
### Cash and reconciliation ### Cash and reconciliation
@@ -142,9 +142,19 @@ within 3 days and 1% on amount — and accepting one is silently destructive:
reconciled manual rows are filtered out of every query, so the cash spend reconciled manual rows are filtered out of every query, so the cash spend
disappears while the card transaction it matched claims to be that same spend. disappears while the card transaction it matched claims to be that same spend.
Only cash is excluded. Bank transfers *do* appear on a statement now that Only cash is excluded on payment method. Bank transfers *do* appear on a
transaction accounts are imported, and NULL means unknown — both stay statement now that transaction accounts are imported, and NULL means unknown —
candidates, preserving the behaviour of every pre-existing row. both stay candidates, preserving the behaviour of every pre-existing row.
**Account-feed rows are excluded separately, and for a different reason**
(`awaitsStatementLine()`, added with the Frollo importer). A feed row is the
account's own ledger entry, not a receipt awaiting a statement line: the importer
only covers accounts whose statements are deliberately *not* imported. Without
the exclusion those ~600 rows a year sit in the queue forever and bury the
receipts that genuinely need a decision — measured, the queue went from 8 to 558.
It is scoped to a named list (`ACCOUNT_FEED_SOURCES`) rather than to
`source IS NOT NULL`, so a future source that *does* await a statement is not
swept up by it.
ATM withdrawals stay categorised as spend rather than `transfers`. Treating them ATM withdrawals stay categorised as spend rather than `transfers`. Treating them
as transfers only works if every cash purchase is logged; with partial logging as transfers only works if every cash purchase is logged; with partial logging
@@ -617,6 +627,51 @@ deleted, and deleting a rule must not cascade away the audit trail.
changed by something else since the run are flagged, because reverting restores changed by something else since the run are flagged, because reverting restores
the pre-run value and discards the later edit. the pre-run value and discards the later edit.
### Frollo account feed (`source = 'frollo'`)
Covers the accounts whose statements arrive every 182 to 460 days — AMP including
the loan, ANZ Access, Wise including the income account, Up, ING, the small
transaction accounts. **Credit cards are excluded on purpose**: their monthly
statements already cover them and are authoritative, and a second producer for
the same rows would only create reconciliation work.
- Logic: `src/lib/frollo-csv.ts` (pure, parsing + scoping + de-duplication) and
`src/lib/frollo-ingest.ts` (report + insert). 42 unit tests.
- Two callers share that module so they cannot drift: `scripts/import-frollo.mts`
(dry run by default) and `POST /api/frollo/ingest` (token auth, called daily by
the n8n `Frollo Import` workflow). The DB is reached through an injected
executor because one runs under Prisma and the other under a bare `pg` client.
- **Export with pending EXCLUDED.** A pending row changes both its id *and* its
description when it settles, so importing one guarantees a duplicate next run.
**The one thing to understand before touching this: a CDR re-consent makes Frollo
re-ingest an account's whole history under fresh transaction ids while the
originals survive**, and consents expire annually. On one export 385 of 1,989 rows
were such twins and every salary payment appeared twice — a blind import doubles
reported income. `dedupe()` collapses each natural-key group to its **lowest** id
(lowest because old ids survive re-ingestion, which keeps `source_ref` stable and
makes a re-import insert nothing).
An earlier version kept close-id rows on a 10,000 threshold, reasoning that
genuine same-day repeats have consecutive ids. It passed 29 tests and a clean dry
run and was still wrong: in-scope duplicate pairs have id gaps from **58 to 260
million**, while real repeats sat at 14. Nothing separates them. It now collapses
unconditionally and *flags* anything within 10 for review, because the errors are
asymmetric — keeping a twin doubles income, collapsing a real repeat understates
by one row.
`category` is not imported: Frollo assigns its own and the owner reports they are
often wrong. Note a trigger rewrites NULL to `'other'` on insert, which is
equivalent for display and leaves the rules engine (which writes
`transaction_overrides.category_override`) free to decide.
Foreign rows carry only the native figure — there is no AUD equivalent anywhere in
the export — so they take order-ingestion's shape: `amount` is native,
`foreign_currency_code` names it, `amount_aud` stays NULL, and
`AMOUNT_UNCONVERTED` already reports the balance as incomplete.
Full history and the reasoning: `DECISIONS.md` ING-11 in the smarthome repo.
### Trusting extracted statement data ### Trusting extracted statement data
**Balance assertions are the check that works.** `getStatements` computes **Balance assertions are the check that works.** `getStatements` computes