"We should not be importing from Frollo what we already have from statements" (owner). The amount+direction guard could not deliver that, because the two sources decompose the same event differently: Frollo bundles the Wise fee into the transfer (10001.13) where the statement itemises it (10000.00 + 1.13). So 38 Wise USD rows passed the amount guard as new while being the same money, and were the reason the transactions view showed a USD figure where every neighbouring row showed AUD. A statement's billing_end_date is a hard watermark: everything on that account up to that date is already in the ledger, itemised and converted. Guard 1 now drops any feed row at or before its account's newest statement. Guard 2 (amount + direction) stays as the net for accounts that have no statement at all. Note this is the coverage test the first import needed and got wrong. That one asked whether a row's date fell inside a statement's min-max window, which for periods spanning 182 to 460 days swallows a year and answers nothing. The watermark asks a question that has an answer: up to what date is this account complete? Matching is on last4, verified against the live statement set — the eight in-scope accounts with statements each map to one bank, no cross-bank collision. The watermarks are printed by the CLI so a wrong boundary is visible rather than inferred from a row count. Re-imported from empty: 425 covered by statement, 15 amount twins, 110 inserted. Exactly one row now carries a foreign currency with no AUD figure — the 2026-08-12 HDR salary, which is the genuinely-new pre-statement row this feed exists for. Was 39.
This commit is contained in:
@@ -46,9 +46,13 @@ const csv = (...lines: string[]) => [HEADER, ...lines].join("\n");
|
||||
* which is crude but keeps the test free of a live connection — and the guard
|
||||
* is pure logic over what comes back.
|
||||
*/
|
||||
function exec(prior: { transaction_date: string; amount: string; transaction_type?: string }[]): SqlExecutor {
|
||||
function exec(
|
||||
prior: { transaction_date: string; amount: string; transaction_type?: string }[],
|
||||
coverage: { last4: string; covered_to: string }[] = []
|
||||
): SqlExecutor {
|
||||
return (async (sql: string) => {
|
||||
if (sql.includes("WHERE source = $1")) return [];
|
||||
if (sql.includes("FROM statements")) return coverage;
|
||||
if (sql.includes("superseded_by_id IS NULL")) return prior.map((p) => ({ transaction_type: "debit", ...p }));
|
||||
return [];
|
||||
}) as SqlExecutor;
|
||||
@@ -162,6 +166,42 @@ describe("ledger-duplicate guard", () => {
|
||||
expect(r.toInsert).toBe(0);
|
||||
});
|
||||
|
||||
it("drops anything a statement already covers, whatever the amount", async () => {
|
||||
// The primary guard. Amount-matching cannot see this case: the two sources
|
||||
// decompose the same event differently — Frollo bundles the Wise fee into
|
||||
// the transfer (10001.13) where the statement itemises it (10000.00 + 1.13)
|
||||
// — so 38 Wise USD rows survived the amount guard while being the same
|
||||
// money. A statement's billing_end_date is a hard watermark instead.
|
||||
const r = await ingestFrolloCsv(
|
||||
csv(
|
||||
line({ id: "1", desc: "Interactive Brokers LLC", amount: "-10001.13", date: "2026-07-25" }),
|
||||
line({ id: "2", desc: "HDR Global Services", amount: "10782.00", date: "2026-08-12" })
|
||||
),
|
||||
exec([], [{ last4: "1910", covered_to: "2026-07-26" }])
|
||||
);
|
||||
expect(r.coveredByStatement).toBe(1);
|
||||
expect(r.toInsert).toBe(1); // only the post-watermark row survives
|
||||
expect(r.statementWatermarks).toEqual([{ last4: "1910", coveredTo: "2026-07-26" }]);
|
||||
});
|
||||
|
||||
it("keeps everything for an account that has no statement at all", async () => {
|
||||
const r = await ingestFrolloCsv(
|
||||
csv(line({ id: "1", amount: "-42.50", date: "2020-01-01" })),
|
||||
exec([], [{ last4: "9999", covered_to: "2026-07-26" }])
|
||||
);
|
||||
expect(r.coveredByStatement).toBe(0);
|
||||
expect(r.toInsert).toBe(1);
|
||||
});
|
||||
|
||||
it("treats the watermark date itself as covered", async () => {
|
||||
const r = await ingestFrolloCsv(
|
||||
csv(line({ id: "1", amount: "-42.50", date: "2026-07-26" })),
|
||||
exec([], [{ last4: "1910", covered_to: "2026-07-26" }])
|
||||
);
|
||||
expect(r.coveredByStatement).toBe(1);
|
||||
expect(r.toInsert).toBe(0);
|
||||
});
|
||||
|
||||
it("keeps everything when the ledger is empty", async () => {
|
||||
const r = await ingestFrolloCsv(
|
||||
csv(
|
||||
|
||||
Reference in New Issue
Block a user