ci / lint-test (push) Successful in 51s
Step 2 of wallet-first ingestion. Signals for one purchase arrive in sequence, not in competition: the notification within seconds of the tap, the receipt hours or days later. Until now the receipt path always INSERTed its own transaction, so the same shop became two rows. The case this is built from is measured, not hypothetical. Woolworths $103.30 on 2026-08-18: the wallet event landed 09:00:14 on the 18th and nothing created a transaction until 20:25:06 on the 19th -- 35h25m later, by this path, from scratch -- discarding what only the notification knew (the instrument, and the suburb). findAdoptableTransaction() is deliberately as narrow as findAutoMatches(): exact amount, a few days either side, unclaimed, and UNIQUE. Two candidates means two plausible purchases, and guessing between them moves one shop's line items onto another shop's row. It also refuses a row that already carries line items, because a fully-described row is a different purchase that happens to share an amount. Returning null is always safe -- the caller falls back to creating its own row, which is the old behaviour. On adoption, both the transaction UPDATE and the expense_metadata UPDATE have a column list that IS the ownership boundary, following order_transaction_bridge's UPGRADE pass: the receipt read the register tape so it wins on what was bought, and `location` is structurally ABSENT from the update rather than conditionally skipped, so the notification's suburb cannot be erased by a bug here. expense_metadata.transaction_id is UNIQUE, so an adopted row that already has metadata is enriched in place rather than raising. 5 new integration tests including both refusals. Existing receipt tests unchanged and passing. npm run test:all green.
164 lines
7.7 KiB
TypeScript
164 lines
7.7 KiB
TypeScript
import { queryRaw } from "@/lib/db";
|
|
|
|
/**
|
|
* Unambiguous statement matches for statementless ("manual") transactions.
|
|
*
|
|
* WHY THIS EXISTS
|
|
* ---------------
|
|
* The reconcile queue is worked by hand and is already ~177 deep. The wallet
|
|
* lane writes a transaction for every tap (DECISIONS.md ING-11, 2026-08-22),
|
|
* which at the observed rate adds ~63 rows a month, ~767 a year. Without an
|
|
* automatic pass for the easy cases that queue stops being workable, and the
|
|
* tempting alternative -- filtering the new rows back out of it -- is
|
|
* explicitly ruled out by the `awaitsStatementLine()` removal note in
|
|
* queries.ts: "an imported row that has no statement belongs in the queue;
|
|
* volume is solved upstream, never downstream by hiding what is there."
|
|
*
|
|
* WHAT COUNTS AS UNAMBIGUOUS
|
|
* --------------------------
|
|
* Deliberately much stricter than the queue's own candidate ranking, which
|
|
* offers the top 5 within +/-1% and +/-3 days for a human to choose between.
|
|
* A machine gets one shot and no judgement, so:
|
|
*
|
|
* - EXACT amount equality, not the queue's +/-1% band. A near-miss is exactly
|
|
* the case a person should look at.
|
|
* - Same owner, and the statement row must not already be reconciled against.
|
|
* - UNIQUE BOTH WAYS: the manual row must have exactly one candidate, AND
|
|
* that candidate must be claimed by exactly one manual row. This is the
|
|
* order_transaction_bridge rule, and it is what stops two identical $13.50
|
|
* bakery taps in one window from being paired arbitrarily.
|
|
*
|
|
* Foreign rows match on the NATIVE pair (foreign_currency_amount +
|
|
* foreign_currency_code), never on `amount`: `amount` is the AUD the bank
|
|
* charged after applying its own rate, so a wallet row cannot predict it, while
|
|
* the native figure is exact on both sides. Same reasoning as
|
|
* order_transaction_bridge's FX path.
|
|
*
|
|
* This function only PROPOSES. Applying a pair goes through the same merge the
|
|
* manual route uses, so overrides/tags/splits/metadata move identically.
|
|
*/
|
|
|
|
export type AutoMatch = {
|
|
manual_id: number;
|
|
statement_tx_id: number;
|
|
amount: string;
|
|
manual_date: string;
|
|
statement_date: string;
|
|
description: string;
|
|
basis: "aud" | "native";
|
|
};
|
|
|
|
/** Days either side of the manual row's date to consider. Authorisation and
|
|
* settlement are rarely the same day; 3 matches the queue's own window and
|
|
* the wallet reconciliation check's default. */
|
|
export const AUTO_MATCH_WINDOW_DAYS = 3;
|
|
|
|
export async function findAutoMatches(
|
|
ownerId: number,
|
|
windowDays: number = AUTO_MATCH_WINDOW_DAYS
|
|
): Promise<AutoMatch[]> {
|
|
return queryRaw<AutoMatch>(
|
|
`WITH pairs AS (
|
|
SELECT m.id AS manual_id,
|
|
t.id AS statement_tx_id,
|
|
t.amount::text AS amount,
|
|
m.transaction_date::text AS manual_date,
|
|
t.transaction_date::text AS statement_date,
|
|
t.description AS description,
|
|
CASE WHEN m.foreign_currency_code IS NOT NULL
|
|
AND m.foreign_currency_amount IS NOT NULL
|
|
THEN 'native' ELSE 'aud' END AS basis
|
|
FROM transactions m
|
|
JOIN transactions t
|
|
ON t.statement_id IS NOT NULL
|
|
AND t.transaction_type = 'debit'
|
|
AND t.transaction_date BETWEEN m.transaction_date - $2::int
|
|
AND m.transaction_date + $2::int
|
|
AND (
|
|
-- Foreign: compare what both sides actually recorded natively.
|
|
(m.foreign_currency_code IS NOT NULL
|
|
AND m.foreign_currency_amount IS NOT NULL
|
|
AND t.foreign_currency_code = m.foreign_currency_code
|
|
AND t.foreign_currency_amount = m.foreign_currency_amount)
|
|
OR -- AUD side: exact equality, never a tolerance band. The
|
|
-- statement row MAY be foreign here, and that is not a mistake:
|
|
-- all 48 reconciliations a human has made on this ledger are
|
|
-- exactly this shape -- a manual row carrying no currency,
|
|
-- matched on exact AUD against a statement row that does carry
|
|
-- one (EUR, CHF). Overseas receipts and orders record what was
|
|
-- charged in AUD; only the wallet lane records natively, which
|
|
-- is why the branch above exists. Requiring both sides to be
|
|
-- non-foreign was tried and rejected: it matched 0 of those 48.
|
|
-- The coincidence risk (a domestic tap whose amount happens to
|
|
-- equal a foreign charge's AUD figure) is already covered by
|
|
-- unique-both-ways, which refuses as soon as a second candidate
|
|
-- exists.
|
|
(m.foreign_currency_code IS NULL
|
|
AND t.amount = m.amount)
|
|
)
|
|
JOIN statements s ON s.id = t.statement_id
|
|
WHERE m.statement_id IS NULL
|
|
AND m.owner_id = $1
|
|
AND m.reconciled_with_id IS NULL
|
|
AND m.superseded_by_id IS NULL
|
|
AND (m.payment_method IS NULL OR m.payment_method NOT IN ('cash', 'credits'))
|
|
AND t.superseded_by_id IS NULL
|
|
AND COALESCE(t.owner_id, s.owner_id) = $1
|
|
-- The statement row must be unclaimed.
|
|
AND NOT EXISTS (SELECT 1 FROM transactions mt WHERE mt.reconciled_with_id = t.id)
|
|
)
|
|
SELECT manual_id, statement_tx_id, amount, manual_date, statement_date, description, basis
|
|
FROM pairs p
|
|
WHERE (SELECT count(*) FROM pairs q WHERE q.manual_id = p.manual_id) = 1
|
|
AND (SELECT count(*) FROM pairs q WHERE q.statement_tx_id = p.statement_tx_id) = 1
|
|
ORDER BY manual_id`,
|
|
[ownerId, windowDays]
|
|
);
|
|
}
|
|
|
|
/**
|
|
* An existing statementless transaction that a newly-arriving receipt or order
|
|
* describes -- i.e. the row the wallet lane already created for this purchase.
|
|
*
|
|
* WHY: signals for one purchase arrive in sequence, not in competition. The
|
|
* wallet notification lands within seconds of the tap; the receipt is shared
|
|
* hours or days later. Measured on txn 6858: the wallet event arrived 35h25m
|
|
* before anything created a transaction, and pantry then created one from
|
|
* scratch -- so the same Woolworths shop would have had two rows. The receipt
|
|
* must adopt the row that already exists and enrich it.
|
|
*
|
|
* Deliberately narrow, for the same reason findAutoMatches() is: exact amount,
|
|
* a few days either side, unclaimed, and UNIQUE. Two candidates means two
|
|
* plausible purchases, and inventing a link between the wrong pair moves one
|
|
* shop's line items onto another shop's row. Returning null is always safe --
|
|
* the caller falls back to creating its own row, which is today's behaviour.
|
|
*/
|
|
export async function findAdoptableTransaction(
|
|
ownerId: number,
|
|
amount: number,
|
|
date: string,
|
|
windowDays: number = AUTO_MATCH_WINDOW_DAYS
|
|
): Promise<number | null> {
|
|
const rows = await queryRaw<{ id: number }>(
|
|
`SELECT t.id
|
|
FROM transactions t
|
|
WHERE t.statement_id IS NULL
|
|
AND t.owner_id = $1
|
|
AND t.amount = $2::numeric
|
|
AND t.transaction_date BETWEEN $3::date - $4::int AND $3::date + $4::int
|
|
AND t.reconciled_with_id IS NULL
|
|
AND t.superseded_by_id IS NULL
|
|
-- Never adopt a row that another source already fully described; that
|
|
-- is a different purchase that happens to share an amount.
|
|
AND NOT EXISTS (
|
|
SELECT 1 FROM expense_metadata em
|
|
WHERE (em.transaction_id = t.id OR em.matched_transaction_id = t.id)
|
|
AND jsonb_typeof(em.line_items) = 'array'
|
|
AND jsonb_array_length(em.line_items) > 0
|
|
)
|
|
LIMIT 2`,
|
|
[ownerId, amount, date, windowDays]
|
|
);
|
|
return rows.length === 1 ? rows[0].id : null;
|
|
}
|