frollo: don't re-import what the ledger already has, and stop hiding the queue

The first import wrote 550 rows on 2026-08-13. 422 of them (77%) were second
copies of transactions the ledger already held from statements — $1,023,824.63
of movement counted twice. The owner found it by opening the transactions view
and seeing one HDR Global salary listed twice, once as A$15,518.53 from the
statement and once as US$10,782.00 from the feed.

The currency was never the defect. toLedgerRow already left amount_aud NULL and
named the currency in foreign_currency_code, which is the documented contract for
a row whose AUD value is unknown, and the transactions page labels it. What made
it look wrong was the duplicate sitting beside it.

Two changes.

Upstream, ingestFrolloCsv now drops rows the ledger already holds, matching on
amount + direction within LEDGER_MATCH_DAYS (3). Three things had to be right and
the first two were not, each caught only by rehearsing against real data rather
than fixtures:

  - pg returns a DATE as a JS Date while Prisma and the CSV give strings.
    String(date).slice(0,10) is "Wed Mar 10", which parses to NaN, so the first
    dry run reported 550 to insert and zero duplicates. dayMs() takes both.
  - Direction has to be in the key. This ledger is full of internal transfers
    between the owner's own accounts and the feed carries both legs: 2026-05-18
    has +3076.04 into ANZ and -3076.04 out of AMP. Matching on amount alone let
    the credit leg consume the ledger's debit row, so the real duplicate was
    written — 20 rows got in that way.
  - 'refund' is money in. The feed calls a reversed account fee a credit and the
    statement importer types it 'refund'; classifying it as an outflow left every
    ANZ servicing-fee reversal behind.

Downstream, awaitsStatementLine() is removed. Its premise — feed rows never await
a statement line — was asserted, never tested, and false for almost every
account. Worse is how it got there: the reconcile queue jumped 8 -> 558 when the
feed landed, that jump was read as noise and filtered away, and filtering it
removed the only mechanism that would ever have collapsed the duplicates. The
queue was right. A feed row IS a row awaiting its statement line.

Re-imported: 375 dropped as already-on-ledger, 175 inserted. Residual duplicates
4 rows / $15.01, all sub-$5 account fees where several identical amounts fall in
overlapping windows and greedy consumption picks the wrong one; not chased
further at this scale.

The CLI prints the already-on-ledger count even when zero — a number you have to
go looking for is a number nobody looks at.

src/__tests__/unit/frollo-ingest.test.ts

@@ -0,0 +1,189 @@
+import { describe, it, expect } from "vitest";
+import { ingestFrolloCsv, LEDGER_MATCH_DAYS, type SqlExecutor } from "@/lib/frollo-ingest";
+
+/**
+ * Tests for the ledger-duplicate guard.
+ *
+ * This is the check whose absence caused the 2026-08-13 first import to write
+ * 422 duplicate rows out of 550 — every one of them a second copy of a payment
+ * the ledger already held from a statement. It passed 29 unit tests and a clean
+ * dry run at the time, because every one of those tests compared the CSV against
+ * itself. Nothing compared it against the database.
+ */
+
+const HEADER =
+  "transaction_id,description,user_description,amount,currency,transaction_date,posted_date," +
+  "account_number,account_name,credit_debit,transaction_type,provider_name,merchant_name," +
+  "budget_category,category_name,user_tags,notes,included";
+
+function line(o: { id: string; desc?: string; amount: string; date: string }): string {
+  return [
+    o.id,
+    o.desc ?? "SOME MERCHANT",
+    "",
+    o.amount,
+    "AUD",
+    o.date,
+    "",
+    "xxx-xxx xxxx1910",
+    "Smart Access",
+    o.amount.startsWith("-") ? "debit" : "credit",
+    "payment",
+    "CommBank",
+    "Some Merchant",
+    "lifestyle",
+    "Groceries",
+    "",
+    "",
+    "true",
+  ].join(",");
+}
+
+const csv = (...lines: string[]) => [HEADER, ...lines].join("\n");
+
+/**
+ * Stands in for the database. The two queries are told apart by their text,
+ * which is crude but keeps the test free of a live connection — and the guard
+ * is pure logic over what comes back.
+ */
+function exec(prior: { transaction_date: string; amount: string; transaction_type?: string }[]): SqlExecutor {
+  return (async (sql: string) => {
+    if (sql.includes("WHERE source = $1")) return [];
+    if (sql.includes("superseded_by_id IS NULL")) return prior.map((p) => ({ transaction_type: "debit", ...p }));
+    return [];
+  }) as SqlExecutor;
+}
+
+describe("ledger-duplicate guard", () => {
+  it("drops a row the ledger already holds on the same day", async () => {
+    const r = await ingestFrolloCsv(
+      csv(line({ id: "1", amount: "-42.50", date: "2026-03-10" })),
+      exec([{ transaction_date: "2026-03-10", amount: "42.50" }])
+    );
+    expect(r.ledgerDuplicates).toBe(1);
+    expect(r.toInsert).toBe(0);
+  });
+
+  it("drops a row dated within the match window", async () => {
+    // The feed dates a transaction when the provider posted it and a statement
+    // when the bank did; a weekend puts two days between them.
+    const r = await ingestFrolloCsv(
+      csv(line({ id: "1", amount: "-42.50", date: "2026-03-10" })),
+      exec([{ transaction_date: "2026-03-13", amount: "42.50" }])
+    );
+    expect(LEDGER_MATCH_DAYS).toBe(3);
+    expect(r.ledgerDuplicates).toBe(1);
+    expect(r.toInsert).toBe(0);
+  });
+
+  it("keeps a row beyond the match window", async () => {
+    const r = await ingestFrolloCsv(
+      csv(line({ id: "1", amount: "-42.50", date: "2026-03-10" })),
+      exec([{ transaction_date: "2026-03-14", amount: "42.50" }])
+    );
+    expect(r.ledgerDuplicates).toBe(0);
+    expect(r.toInsert).toBe(1);
+  });
+
+  it("keeps a row whose amount differs", async () => {
+    const r = await ingestFrolloCsv(
+      csv(line({ id: "1", amount: "-42.50", date: "2026-03-10" })),
+      exec([{ transaction_date: "2026-03-10", amount: "42.51" }])
+    );
+    expect(r.ledgerDuplicates).toBe(0);
+    expect(r.toInsert).toBe(1);
+  });
+
+  it("consumes each ledger row once, so a genuine repeat survives", async () => {
+    // Two real charges of the same amount in the same week, one of which the
+    // ledger already has. Matching without consuming would drop both and lose a
+    // transaction that never existed anywhere else.
+    const r = await ingestFrolloCsv(
+      csv(
+        line({ id: "1", desc: "COFFEE ONE", amount: "-5.00", date: "2026-03-10" }),
+        line({ id: "2", desc: "COFFEE TWO", amount: "-5.00", date: "2026-03-11" })
+      ),
+      exec([{ transaction_date: "2026-03-10", amount: "5.00" }])
+    );
+    expect(r.ledgerDuplicates).toBe(1);
+    expect(r.toInsert).toBe(1);
+  });
+
+  it("matches when the driver returns Date objects, not strings", async () => {
+    // `pg` maps a Postgres DATE to a JS Date; Prisma and the CSV give strings.
+    // The first cut of this guard read one as the other, and because the failure
+    // is a silent NaN it reported 550 rows to insert against a ledger holding
+    // 422 of them. Both shapes are tested because both drivers are in use: the
+    // CLI runs on pg, the API route on Prisma.
+    const asDate = await ingestFrolloCsv(
+      csv(line({ id: "1", amount: "-42.50", date: "2026-03-10" })),
+      exec([{ transaction_date: new Date(2026, 2, 10) as unknown as string, amount: "42.50" }])
+    );
+    expect(asDate.ledgerDuplicates).toBe(1);
+    expect(asDate.toInsert).toBe(0);
+  });
+
+  it("never matches a credit against a debit", async () => {
+    // Internal transfers between the owner's own accounts put both legs in the
+    // feed: 2026-05-18 carries +3076.04 into ANZ and -3076.04 out of AMP. On
+    // amount alone the credit leg consumed the ledger's debit row and the real
+    // duplicate was written. 20 rows got in this way before direction was part
+    // of the key.
+    const r = await ingestFrolloCsv(
+      csv(
+        line({ id: "1", desc: "PAYMENT FROM SELF", amount: "3076.04", date: "2026-05-18" }),
+        line({ id: "2", desc: "Transfer to Self", amount: "-3076.04", date: "2026-05-18" })
+      ),
+      exec([{ transaction_date: "2026-05-18", amount: "3076.04", transaction_type: "debit" }])
+    );
+    // The debit leg is the duplicate; the credit leg is a genuinely new row.
+    expect(r.ledgerDuplicates).toBe(1);
+    expect(r.toInsert).toBe(1);
+  });
+
+  it("treats a statement 'refund' as money in", async () => {
+    // The feed calls a reversed account fee a credit; the statement importer
+    // types it 'refund'. Classifying refund as an outflow left every ANZ
+    // servicing-fee reversal in the file as a duplicate.
+    const r = await ingestFrolloCsv(
+      csv(line({ id: "1", desc: "REVERSAL OF ACCOUNT SERVICING FEE", amount: "5.00", date: "2026-02-27" })),
+      exec([{ transaction_date: "2026-02-27", amount: "5.00", transaction_type: "refund" }])
+    );
+    expect(r.ledgerDuplicates).toBe(1);
+    expect(r.toInsert).toBe(0);
+  });
+
+  it("treats a statement 'fee' as money out", async () => {
+    const r = await ingestFrolloCsv(
+      csv(line({ id: "1", desc: "ACCOUNT SERVICING FEE", amount: "-5.00", date: "2026-03-31" })),
+      exec([{ transaction_date: "2026-03-31", amount: "5.00", transaction_type: "fee" }])
+    );
+    expect(r.ledgerDuplicates).toBe(1);
+    expect(r.toInsert).toBe(0);
+  });
+
+  it("keeps everything when the ledger is empty", async () => {
+    const r = await ingestFrolloCsv(
+      csv(
+        line({ id: "1", desc: "A", amount: "-5.00", date: "2026-03-10" }),
+        line({ id: "2", desc: "B", amount: "-6.00", date: "2026-03-11" })
+      ),
+      exec([])
+    );
+    expect(r.ledgerDuplicates).toBe(0);
+    expect(r.toInsert).toBe(2);
+  });
+
+  it("reports duplicates rather than silently discarding them", async () => {
+    // The owner's standing requirement: a row that disappears must be counted,
+    // because a missing transaction can mean several things and one of them is
+    // worth a follow-up.
+    const r = await ingestFrolloCsv(
+      csv(line({ id: "1", amount: "-42.50", date: "2026-03-10" })),
+      exec([{ transaction_date: "2026-03-10", amount: "42.50" }])
+    );
+    expect(r).toHaveProperty("ledgerDuplicates");
+    expect(r.inScope).toBe(1);
+    expect(r.ledgerDuplicates + r.toInsert).toBe(r.inScope);
+  });
+});