statements: one institution, one bank name (migration 0031)
ci / lint-test (push) Successful in 46s

bank_name is a component of uq_statement_identity
(bank_name, account_number, billing_end_date), so two spellings of one
institution mean the same statement can be ingested twice without the
index ever firing. Gemini reads the name off whatever the PDF prints, so
the spelling varies per document, not per account.

Zip exposed it: account 2705256 arrived under four names in sixteen
minutes — ZipMoney Payments Pty Ltd, ZipMoney Payments Pty Limited,
Zip Pay, ZipPay — with ten more in the ingestion queue. Not unique:
17 bank_name values represented 13 institutions, every split pair sharing
an account number (Amex 14/1, Citibank/Citi 8/4, NAB 3/1). Fragmentation
also splits both bank filters and by_bank in /api/analytics/fees.

normalize_bank_name() + a BEFORE INSERT OR UPDATE trigger, mirroring the
0013 statement_type pattern with one deliberate difference: the
vocabulary is OPEN. An unrecognised name passes through tidied rather
than collapsing to a fallback, and there is no CHECK constraint — a bank
never used before must be able to arrive without a migration, and
destroying its name on first contact is worse than leaving it unmapped.

Matching is by prefix, not by enumerated spelling, so unseen variants
normalise with no code change. Branches for Westpac, ANZ, HSBC, ING, AMP,
Up and CommBank are no-ops that map to the spelling already in use, to
catch the legal-entity variant a future PDF might print. Wise is left
alone: one spelling is a rename, not a merge.

Backfill verified collision-free against uq_statement_identity first.
17 names -> 12. account_owner_mappings keys on (bank_name,
account_number) with its own UNIQUE constraint and is updated too; it is
empty today.
This commit is contained in:
2026-08-15 16:45:04 +10:00
parent 95d8544752
commit 0bb4329249
2 changed files with 195 additions and 0 deletions
+47
View File
@@ -865,6 +865,53 @@ Card'`) and the DB normalises it on write. The raw extracted value is preserved
The TypeScript mirror is `src/lib/statement-types.ts` — keep the list, the SQL
function, and the CHECK constraint in sync when adding a type.
### Bank names are normalised on write (migration 0031)
**`bank_name` is a component of the duplicate check.** `uq_statement_identity`
is `(bank_name, account_number, billing_end_date)`, so two spellings of one
institution mean the same statement can be ingested twice without the index ever
firing. Gemini reads the name off whatever the PDF prints, so the spelling varies
per *document*, not per account.
Zip exposed it: account `2705256` arrived under four names in sixteen minutes —
`ZipMoney Payments Pty Ltd`, `ZipMoney Payments Pty Limited`, `Zip Pay`,
`ZipPay` — with ten more in the queue. It was not unique. 17 `bank_name` values
represented 13 institutions, and every split pair shared an account number:
Amex (14/1), Citibank/Citi (8/4), NAB (3/1). Fragmentation also splits the
statements and transactions bank filters and `by_bank` in `/api/analytics/fees`.
`normalize_bank_name()` + `trg_statements_normalize_bank_name` (BEFORE INSERT OR
UPDATE OF bank_name) mirror the 0013 `statement_type` pattern, with **one
deliberate difference: the vocabulary is open.** An unrecognised name passes
through tidied, never collapsed to a fallback, and there is **no CHECK
constraint** — a bank this household has never used must be able to arrive
without a migration, and destroying its name on first contact is worse than
leaving it unmapped.
Matching is by **prefix, not by enumerated spelling** (`zip%`, `citi%`,
`%american express%`, `%national australia bank%`), so unseen variants like
`Zip Co Australia Pty Ltd` normalise with no code change. Branches for Westpac,
ANZ, HSBC, ING, AMP, Up and CommBank are no-ops today that map *to* the spelling
already in use — they exist to catch the legal-entity variant a future PDF might
print.
Wise is deliberately unmapped: `Wise Australia Pty Ltd.` is a single spelling, so
shortening it would be a rename nobody asked for rather than a merge.
**The residual gap and its tell.** The function cannot know that two names for a
*new* bank are one institution. That always shows up the same way, so check it
rather than trusting the map:
```sql
SELECT account_number, array_agg(DISTINCT bank_name)
FROM statements WHERE account_number IS NOT NULL
GROUP BY 1 HAVING count(DISTINCT bank_name) > 1;
```
`account_owner_mappings` also keys on `(bank_name, account_number)` with its own
UNIQUE constraint, so any future rename must update it too or strand its rows.
It is empty today; the migration handles it anyway.
### Loans
A loan repayment is **not** an expense. It is part principal (equity, a