reconcile: automatic pass for unambiguous statement matches (board #252)
ci / lint-test (push) Successful in 56s
ci / lint-test (push) Successful in 56s
Prerequisite for the wallet lane writing a transaction per tap. The queue is worked by hand and already ~177 deep; wallet adds ~63 rows a month, ~767 a year. Filtering those rows back out is explicitly ruled out by the awaitsStatementLine() removal note: "an imported row that has no statement belongs in the queue; volume is solved upstream, never downstream by hiding what is there." The rule is deliberately stricter than the queue's own candidate ranking (top 5 within +/-1% and +/-3 days, for a human to choose between). A machine gets one shot and no judgement, so: exact amount, never a tolerance band; and unique BOTH ways, so two identical $13.50 bakery taps in one window are refused rather than paired arbitrarily. Foreign rows match on the native pair, because `amount` is the bank's own conversion and a wallet row cannot predict it. VALIDATED AGAINST REAL DECISIONS, and that overturned my reasoning. I first required both sides to be non-foreign, arguing a currency mismatch is ambiguous. Checked against the 48 reconciliations a human has actually made: that version matched ZERO of them. All 48 are exactly the shape it refused — a manual row carrying no currency, matched on exact AUD against a statement row that does carry one (EUR, CHF) — because overseas receipts and orders record what was charged in AUD, and only the wallet lane records natively. Reverted; the rule now reproduces all 48 and proposes nothing they did not. The coincidence it was meant to guard against is already covered by unique-both-ways. Extracted the merge into src/lib/reconcile.ts so the manual route and the automatic pass cannot drift — there must be exactly one implementation of something that moves curation off a row reconciliation is about to hide. Fixed a latent bug in that merge while extracting it: splits were recreated WITHOUT `settled`/`settled_at`, and the column defaults to false, so reconciling a settled split silently marked a paid debt owed again. No queue row carries one today, but this path is about to run unattended, where that surfaces as money rather than as an exception. PATCH ?dry=1 reports without writing. Each pair is its own transaction, so one bad pair leaves the rest untouched and in the queue. 11 new integration tests, most of them about refusal. Live dry-run proposes 0 pairs for either owner, which is correct: nothing in the queue has an exact statement twin yet. npm run test:all green (182 + 287).
This commit is contained in:
@@ -0,0 +1,117 @@
|
||||
import { queryRaw } from "@/lib/db";
|
||||
|
||||
/**
|
||||
* Unambiguous statement matches for statementless ("manual") transactions.
|
||||
*
|
||||
* WHY THIS EXISTS
|
||||
* ---------------
|
||||
* The reconcile queue is worked by hand and is already ~177 deep. The wallet
|
||||
* lane writes a transaction for every tap (DECISIONS.md ING-11, 2026-08-22),
|
||||
* which at the observed rate adds ~63 rows a month, ~767 a year. Without an
|
||||
* automatic pass for the easy cases that queue stops being workable, and the
|
||||
* tempting alternative -- filtering the new rows back out of it -- is
|
||||
* explicitly ruled out by the `awaitsStatementLine()` removal note in
|
||||
* queries.ts: "an imported row that has no statement belongs in the queue;
|
||||
* volume is solved upstream, never downstream by hiding what is there."
|
||||
*
|
||||
* WHAT COUNTS AS UNAMBIGUOUS
|
||||
* --------------------------
|
||||
* Deliberately much stricter than the queue's own candidate ranking, which
|
||||
* offers the top 5 within +/-1% and +/-3 days for a human to choose between.
|
||||
* A machine gets one shot and no judgement, so:
|
||||
*
|
||||
* - EXACT amount equality, not the queue's +/-1% band. A near-miss is exactly
|
||||
* the case a person should look at.
|
||||
* - Same owner, and the statement row must not already be reconciled against.
|
||||
* - UNIQUE BOTH WAYS: the manual row must have exactly one candidate, AND
|
||||
* that candidate must be claimed by exactly one manual row. This is the
|
||||
* order_transaction_bridge rule, and it is what stops two identical $13.50
|
||||
* bakery taps in one window from being paired arbitrarily.
|
||||
*
|
||||
* Foreign rows match on the NATIVE pair (foreign_currency_amount +
|
||||
* foreign_currency_code), never on `amount`: `amount` is the AUD the bank
|
||||
* charged after applying its own rate, so a wallet row cannot predict it, while
|
||||
* the native figure is exact on both sides. Same reasoning as
|
||||
* order_transaction_bridge's FX path.
|
||||
*
|
||||
* This function only PROPOSES. Applying a pair goes through the same merge the
|
||||
* manual route uses, so overrides/tags/splits/metadata move identically.
|
||||
*/
|
||||
|
||||
export type AutoMatch = {
|
||||
manual_id: number;
|
||||
statement_tx_id: number;
|
||||
amount: string;
|
||||
manual_date: string;
|
||||
statement_date: string;
|
||||
description: string;
|
||||
basis: "aud" | "native";
|
||||
};
|
||||
|
||||
/** Days either side of the manual row's date to consider. Authorisation and
|
||||
* settlement are rarely the same day; 3 matches the queue's own window and
|
||||
* the wallet reconciliation check's default. */
|
||||
export const AUTO_MATCH_WINDOW_DAYS = 3;
|
||||
|
||||
export async function findAutoMatches(
|
||||
ownerId: number,
|
||||
windowDays: number = AUTO_MATCH_WINDOW_DAYS
|
||||
): Promise<AutoMatch[]> {
|
||||
return queryRaw<AutoMatch>(
|
||||
`WITH pairs AS (
|
||||
SELECT m.id AS manual_id,
|
||||
t.id AS statement_tx_id,
|
||||
t.amount::text AS amount,
|
||||
m.transaction_date::text AS manual_date,
|
||||
t.transaction_date::text AS statement_date,
|
||||
t.description AS description,
|
||||
CASE WHEN m.foreign_currency_code IS NOT NULL
|
||||
AND m.foreign_currency_amount IS NOT NULL
|
||||
THEN 'native' ELSE 'aud' END AS basis
|
||||
FROM transactions m
|
||||
JOIN transactions t
|
||||
ON t.statement_id IS NOT NULL
|
||||
AND t.transaction_type = 'debit'
|
||||
AND t.transaction_date BETWEEN m.transaction_date - $2::int
|
||||
AND m.transaction_date + $2::int
|
||||
AND (
|
||||
-- Foreign: compare what both sides actually recorded natively.
|
||||
(m.foreign_currency_code IS NOT NULL
|
||||
AND m.foreign_currency_amount IS NOT NULL
|
||||
AND t.foreign_currency_code = m.foreign_currency_code
|
||||
AND t.foreign_currency_amount = m.foreign_currency_amount)
|
||||
OR -- AUD side: exact equality, never a tolerance band. The
|
||||
-- statement row MAY be foreign here, and that is not a mistake:
|
||||
-- all 48 reconciliations a human has made on this ledger are
|
||||
-- exactly this shape -- a manual row carrying no currency,
|
||||
-- matched on exact AUD against a statement row that does carry
|
||||
-- one (EUR, CHF). Overseas receipts and orders record what was
|
||||
-- charged in AUD; only the wallet lane records natively, which
|
||||
-- is why the branch above exists. Requiring both sides to be
|
||||
-- non-foreign was tried and rejected: it matched 0 of those 48.
|
||||
-- The coincidence risk (a domestic tap whose amount happens to
|
||||
-- equal a foreign charge's AUD figure) is already covered by
|
||||
-- unique-both-ways, which refuses as soon as a second candidate
|
||||
-- exists.
|
||||
(m.foreign_currency_code IS NULL
|
||||
AND t.amount = m.amount)
|
||||
)
|
||||
JOIN statements s ON s.id = t.statement_id
|
||||
WHERE m.statement_id IS NULL
|
||||
AND m.owner_id = $1
|
||||
AND m.reconciled_with_id IS NULL
|
||||
AND m.superseded_by_id IS NULL
|
||||
AND (m.payment_method IS NULL OR m.payment_method NOT IN ('cash', 'credits'))
|
||||
AND t.superseded_by_id IS NULL
|
||||
AND COALESCE(t.owner_id, s.owner_id) = $1
|
||||
-- The statement row must be unclaimed.
|
||||
AND NOT EXISTS (SELECT 1 FROM transactions mt WHERE mt.reconciled_with_id = t.id)
|
||||
)
|
||||
SELECT manual_id, statement_tx_id, amount, manual_date, statement_date, description, basis
|
||||
FROM pairs p
|
||||
WHERE (SELECT count(*) FROM pairs q WHERE q.manual_id = p.manual_id) = 1
|
||||
AND (SELECT count(*) FROM pairs q WHERE q.statement_tx_id = p.statement_tx_id) = 1
|
||||
ORDER BY manual_id`,
|
||||
[ownerId, windowDays]
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user