docs: one CSV import path, and the statement-coverage rule that replaced the Frollo importer

CLAUDE.md

@@ -636,87 +636,54 @@ deleted, and deleting a rule must not cascade away the audit trail.
 changed by something else since the run are flagged, because reverting restores
 the pre-run value and discards the later edit.
 
-### Frollo account feed (`source = 'frollo'`)
+### Importing an aggregator/bank CSV (`/api/import/csv`)
 
-Covers the accounts whose statements arrive every 182 to 460 days — AMP including
+There is **one** import path — the CSV import modal — and Frollo goes through it
-the loan, ANZ Access, Wise including the income account, Up, ING, the small
+like any other file. A bespoke Frollo importer, API route, CLI and two scheduled
-transaction accounts. **Credit cards are excluded on purpose**: their monthly
+n8n workflows existed for a day and were deleted on 2026-08-13: net −1,152 lines.
-statements already cover them and are authoritative, and a second producer for
-the same rows would only create reconciliation work.
 
-- Logic: `src/lib/frollo-csv.ts` (pure, parsing + scoping + de-duplication) and
+**The rule that made the rest unnecessary is statement coverage.** Each account's
-  `src/lib/frollo-ingest.ts` (report + insert + the ledger-duplicate guard).
+newest `billing_end_date` is a watermark; a row on or before it is already in the
-  53 unit tests across `frollo-csv.test.ts` and `frollo-ingest.test.ts`.
+ledger. On the real 2,607-row export that leaves **171 rows** — with no account
+allowlist and no credit-card exclusion, because cards have current statements and
+drop out on their own. Every bit of the deleted apparatus was doing by hand what
+`getStatementCoverage()` does generically.
 
-**There are TWO de-duplications and they solve different problems.** `dedupe()`
+Note what this replaced. The first attempt asked whether a row's date fell inside
-compares the file against itself (CDR re-consent twins, below). The
+a statement's min–max *window*, which for accounts whose statements span 182 to
-ledger-duplicate guard compares the file against `transactions`, and its absence
+460 days swallows a year and answers nothing — it let 422 duplicates into 550
-is what made the first import write 422 duplicates out of 550 — $1,023,824.63
+rows. Amount-matching cannot substitute either: the two sources decompose the
-counted twice — because "these accounts issue no statements" had been asserted
+same event differently, bundling a Wise transfer fee into the transfer (10001.13)
-rather than queried. The check that would have caught it is one SQL statement.
+where the statement itemises it (10000.00 + 1.13).
-The check run instead compared each row's date to the statement's min–max window,
-which for accounts whose statements span 182 to 460 days swallows a year and
-cannot distinguish covered from uncovered at all.
 
-The guard matches on **amount + direction, within `LEDGER_MATCH_DAYS` (3)**, and
+**Map these optional columns or lose something silently:**
-consumes each ledger row once so a genuine repeat survives. Three details, each
-of which was wrong first and each caught only by rehearsing against the real
-database rather than fixtures:
 
-- **`pg` returns a DATE as a JS `Date`; Prisma and the CSV give strings.**
+| Column | Without it |
-  `String(date).slice(0, 10)` is `"Wed Mar 10"` → NaN → every comparison skipped,
+|---|---|
-  silently. The first dry run reported 550 to insert against a ledger holding 422
+| **Account** | nothing is excluded — you review the whole file |
-  of them. `dayMs()` accepts both, and both shapes are tested because both
+| **Currency** | a foreign row is stored as if AUD (USD 10,782 → A$10,782) |
-  drivers are in use.
+| **Row ID** | a re-import duplicates instead of no-opping |
-- **Direction must be in the key.** This ledger is full of internal transfers
-  between the owner's own accounts and the feed carries *both legs*: 2026-05-18
-  has +3076.04 into ANZ and −3076.04 out of AMP. On amount alone the credit leg
-  consumed the ledger's debit row and the real duplicate was written — 20 rows.
-- **`refund` is money in.** Ledger types are debit | credit | payment | fee |
-  interest | refund; the feed calls a reversed account fee a `credit` while the
-  statement importer types it `refund`. Classifying it as an outflow left every
-  ANZ servicing-fee reversal behind.
 
-Residual after all three: **4 rows / $15.01**, all sub-$5 account fees where
+Leave **Category** unmapped for an aggregator: its spend categories are not
-several identical amounts fall in overlapping windows and greedy consumption
+trusted, and you set them per row in the review step.
-picks the wrong one. Left alone deliberately at this scale.
 
-The count is printed by the CLI even when zero. A number you have to go looking
+**Two bugs fixed the day this shipped, both of which the flow depends on:**
-for is a number nobody looks at.
+`batchInsertCSVTransactions` accepted a `category` and then omitted it from the
-- Two callers share that module so they cannot drift: `scripts/import-frollo.mts`
+INSERT column list, so every category chosen in review was discarded and the
-  (dry run by default) and `POST /api/frollo/ingest` (token auth, called daily by
+trigger wrote `'other'` — not cosmetic, because an uncategorised credit is
-  the n8n `Frollo Import` workflow). The DB is reached through an injected
+admitted by `NET_SPEND_ROWS` and negated by `SPEND_SIGNED`, so 62 imported
-  executor because one runs under Prisma and the other under a bare `pg` client.
+transfers cancelled **$74,338** of spend while counting as no income. And the
-- **Export with pending EXCLUDED.** A pending row changes both its id *and* its
+path had no idempotency at all: `row_index` is assigned MAX+1 every run, making
-  description when it settles, so importing one guarantees a duplicate next run.
+`uq_transaction_identity` structurally unable to fire. It now writes
+`source`/`source_ref` with `ON CONFLICT DO NOTHING`.
 
-**The one thing to understand before touching this: a CDR re-consent makes Frollo
+**The in-file duplicate warning is not redundant with `source_ref`.** A CDR
-re-ingest an account's whole history under fresh transaction ids while the
+re-consent re-exports an account's whole history under fresh ids while the
-originals survive**, and consents expire annually. On one export 385 of 1,989 rows
+originals survive — 385 twins in one 2,563-row export, doubling every salary
-were such twins and every salary payment appeared twice — a blind import doubles
+payment. Fresh ids mean `source_ref` sees new rows, and 385 is not visible by eye
-reported income. `dedupe()` collapses each natural-key group to its **lowest** id
+in a review table. Consents expire annually, so expect it.
-(lowest because old ids survive re-ingestion, which keeps `source_ref` stable and
-makes a re-import insert nothing).
 
-An earlier version kept close-id rows on a 10,000 threshold, reasoning that
+Undo an import: `DELETE FROM transactions WHERE source = '<source>'`.
-genuine same-day repeats have consecutive ids. It passed 29 tests and a clean dry
-run and was still wrong: in-scope duplicate pairs have id gaps from **58 to 260
-million**, while real repeats sat at 1–4. Nothing separates them. It now collapses
-unconditionally and *flags* anything within 10 for review, because the errors are
-asymmetric — keeping a twin doubles income, collapsing a real repeat understates
-by one row.
-
-`category` is not imported: Frollo assigns its own and the owner reports they are
-often wrong. Note a trigger rewrites NULL to `'other'` on insert, which is
-equivalent for display and leaves the rules engine (which writes
-`transaction_overrides.category_override`) free to decide.
-
-Foreign rows carry only the native figure — there is no AUD equivalent anywhere in
-the export — so they take order-ingestion's shape: `amount` is native,
-`foreign_currency_code` names it, `amount_aud` stays NULL, and
-`AMOUNT_UNCONVERTED` already reports the balance as incomplete.
-
-Full history and the reasoning: `DECISIONS.md` ING-11 in the smarthome repo.
 
 ### Trusting extracted statement data