wallet: a transaction for every tap (board #252, ING-11)
ci / lint-test (push) Successful in 52s

Step 3, and the last app-side piece. POST /api/wallet/ingest takes an
interpreted notification and writes a statementless card transaction plus
one expense_metadata row. Sibling to /api/orders/ingest and
/api/receipts/ingest, same shared-secret shape, its own token so the
wallet credential rotates without touching the order flow.

Nothing here knows about statements, and that is the point: statementless
+ payment_method 'card' is exactly the reconcile queue's predicate, so a
wallet row enters it unaided and the auto-matcher confirms it later.

THE PAIR IS MERGED, NEVER DEDUPED. One tap emits two notifications ~2-8s
apart: the issuer app has the card tail, Apple Wallet has the merchant and
the suburb, and neither alone is complete. Matched on amount and date
because the two share no identifier. My first version COALESCEd every
field, which made the result depend on arrival order — the issuer half
usually lands first, so the truncated all-caps "WYNDHAM VAL AU" beat the
subtitle's "Wyndham Vale, VIC". A subtitle-derived place now overwrites; a
descriptor-derived one only fills a gap. The test that caught this asserts
both arrival orders.

CURRENCY IS NEVER ASSUMED. Only Amex states one ("A$5,103.49"); Westpac,
CommBank and Apple Wallet write a bare "$", which is 14 of 21 captured
rows. Recorded as unknown and flagged, not assumed AUD — harmless at home,
a 23x error on a THB tap abroad. When a currency IS stated and is not AUD,
the NATIVE figure goes to foreign_currency_amount, because the AUD amount
is the bank's own conversion and a notification cannot predict it, while
the native pair is exact on both sides and is what the statement will
match on. transactions.amount is NOT NULL so it takes a provisional
conversion from the ledger-derived median rate, flagged aud_provisional,
overwritten at reconcile. No rate history means no_rate_reference rather
than an invented number.

wallet-descriptor.ts moves the positional parsing out of the model, which
handled ONE fixed-width format three ways on the same day's data: kept the
country code on "WYNDHAM VAL AU", stripped it on "Sydney" and "Perth", and
swallowed "SAN FRANCISOUS" whole — that being "SAN FRANCISO" truncated to
the 12-char city field plus "US", on the only foreign merchant in the
entire capture window. Every unit-test input is a real captured string.

14 unit + 12 integration tests. npm run test:all green (196 + 304), lint
clean.
This commit is contained in:
2026-08-22 16:24:28 +10:00
parent 8e66e527ae
commit 8d1875225f
5 changed files with 736 additions and 0 deletions
+228
View File
@@ -0,0 +1,228 @@
import { queryRow, queryRaw } from "@/lib/db";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import {
parseWestpacDescriptor,
parseWalletSubtitle,
currencyFromAmountText,
} from "@/lib/wallet-descriptor";
/**
* Wallet notifications -> transactions.
*
* A transaction is created for EVERY tap (DECISIONS.md ING-11, 2026-08-22),
* because overseas trips are where live spend matters most: the statement lag
* is the same 6-35 days abroad but knowing what you have spent matters far more
* in an unfamiliar currency.
*
* SEQUENCING IS THE WHOLE DESIGN. Wallet is the FIRST signal, not a late one --
* measured on txn 6858, the notification arrived 35h25m before anything created
* a transaction. So this creates the row, the receipt enriches it
* (receipt-ingestion.ts adopts it), and the statement confirms it via the
* reconcile queue, which a wallet row enters unaided: it is statementless with
* payment_method 'card'. Nothing here needs to know about statements.
*/
export type WalletEvent = {
/** capture.wallet_events.id -- the idempotency key. */
event_id: number;
/** 'transaction' | 'not_transaction' | 'unsure' from the interpreter. */
kind: string;
direction?: string | null;
amount: number | null;
/** The exact substring the model read the amount from, e.g. "A$5,103.49". */
amount_text?: string | null;
currency?: string | null;
merchant?: string | null;
card_tail?: string | null;
issuer?: string | null;
location?: string | null;
notif_app?: string | null;
notif_body?: string | null;
notif_subtitle?: string | null;
occurred_at?: string | null;
};
export type WalletIngestResult = {
transactionId: number | null;
metadataId: number | null;
skipped?: "not_a_transaction" | "already_ingested" | "no_amount" | "merged_pair";
flags: string[];
};
/** Days either side within which a second notification is the SAME tap. */
const PAIR_WINDOW_DAYS = 1;
/**
* A ledger-derived AUD rate for a currency: the median of what the bank
* actually charged divided by the native figure, over statement rows.
*
* Same source order_transaction_bridge uses for its plausibility guard. This is
* only ever a PROVISIONAL value -- transactions.amount is NOT NULL and always
* AUD, and a notification cannot know the AUD figure because the bank sets the
* rate at settlement. The statement overwrites it at reconcile time.
* Returns null when there is not enough history to be worth anything.
*/
async function medianRateToAud(code: string): Promise<number | null> {
const row = await queryRow<{ rate: string | null }>(
`SELECT percentile_cont(0.5) WITHIN GROUP (
ORDER BY amount / NULLIF(foreign_currency_amount, 0)
)::text AS rate
FROM transactions
WHERE foreign_currency_code = $1
AND foreign_currency_amount > 0
AND superseded_by_id IS NULL
HAVING count(*) >= 5`,
[code]
);
const rate = row?.rate ? Number(row.rate) : null;
return rate && Number.isFinite(rate) && rate > 0 ? rate : null;
}
/**
* The other half of a paired notification, if it has already been ingested.
*
* One tap emits TWO notifications ~2-8s apart: the issuer app (which has the
* card tail) and Apple Wallet (which has the better merchant and the suburb).
* Neither alone is complete, so they must be MERGED, never deduped -- dropping
* either loses a field nothing else supplies. Matched on amount and date rather
* than any id, because the notifications share no identifier.
*/
async function findPairedRow(amount: number, date: string) {
const rows = await queryRaw<{ id: number; meta_id: number | null }>(
`SELECT t.id, em.id AS meta_id
FROM transactions t
LEFT JOIN expense_metadata em ON em.transaction_id = t.id
WHERE t.source = 'wallet'
AND t.statement_id IS NULL
AND t.amount = $1::numeric
AND t.transaction_date BETWEEN $2::date - $3::int AND $2::date + $3::int
AND t.superseded_by_id IS NULL
LIMIT 2`,
[amount, date, PAIR_WINDOW_DAYS]
);
return rows.length === 1 ? rows[0] : null;
}
export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIngestResult> {
const flags: string[] = [];
// An open-world classifier feeding a closed-world writer drops the unknown
// and calls it a refusal (ING-14). `unsure` is a real answer and must not
// silently become a transaction.
if (ev.kind !== "transaction") {
return { transactionId: null, metadataId: null, skipped: "not_a_transaction", flags };
}
if (ev.amount === null || !Number.isFinite(ev.amount) || ev.amount <= 0) {
return { transactionId: null, metadataId: null, skipped: "no_amount", flags };
}
const sourceRef = `evt:${ev.event_id}`;
const existing = await queryRow<{ id: number }>(
`SELECT id FROM transactions WHERE source = 'wallet' AND source_ref = $1`,
[sourceRef]
);
if (existing) {
return { transactionId: existing.id, metadataId: null, skipped: "already_ingested", flags };
}
const date = (ev.occurred_at ?? new Date().toISOString()).slice(0, 10);
// Deterministic parsing of the positional fields. The model decides what the
// amount and merchant ARE; it does not get to parse a fixed-width block.
const desc = parseWestpacDescriptor(ev.notif_body);
const sub = parseWalletSubtitle(ev.notif_subtitle);
const merchant = sub.merchant ?? ev.merchant ?? desc.merchant ?? "Card purchase";
// Two sources for a place, and they are not equal. The Apple Wallet subtitle
// gives "Wyndham Vale, VIC"; the issuer's fixed-width descriptor gives
// "WYNDHAM VAL AU" -- a 12-character truncation in caps. The subtitle wins
// outright, which matters for the pair merge below: whichever half arrives
// second must be able to REPLACE a descriptor-derived place, or the result
// depends on arrival order.
const subtitleLocation = sub.location ?? null;
const location = subtitleLocation
?? ev.location
?? (desc.city && desc.country ? `${desc.city} ${desc.country}` : desc.city);
// Currency: only ever what the notification actually states. A bare "$" is
// recorded as unknown, never assumed AUD -- harmless domestically, a 23x
// error on a THB tap abroad.
const stated = currencyFromAmountText(ev.amount_text)
?? (ev.currency && /^[A-Z]{3}$/.test(ev.currency) ? ev.currency : null);
if (!stated) flags.push("currency_unverified");
if (desc.country && desc.country !== "AU") flags.push(`foreign_country_${desc.country}`);
// Foreign: the native figure is what both sides can agree on exactly, so it
// is what gets stored. `amount` still needs a value (NOT NULL) and takes a
// provisional conversion, flagged, which the statement overwrites.
let amountAud = ev.amount;
let fxAmount: number | null = null;
let fxCode: string | null = null;
if (stated && stated !== "AUD") {
fxAmount = ev.amount;
fxCode = stated;
const rate = await medianRateToAud(stated);
if (rate) {
amountAud = Number((ev.amount * rate).toFixed(2));
flags.push("aud_provisional");
} else {
flags.push("aud_provisional", "no_rate_reference");
}
}
// Merge the paired notification rather than creating a second row for one
// tap. The issuer half brings the card tail; the Wallet half brings the
// merchant and suburb. COALESCE so whichever arrives second only ever fills
// gaps -- it can never blank a field the first half supplied.
const pair = await findPairedRow(amountAud, date);
if (pair) {
await queryRaw(
`UPDATE transactions
SET merchant_name = COALESCE(NULLIF($2,'Card purchase'), merchant_name),
merchant_normalized = COALESCE(NULLIF($2,'Card purchase'), merchant_normalized),
-- A subtitle-derived place overwrites; anything else only fills a
-- gap. Without the $4 branch the merged row depends on which
-- notification happened to arrive first, and the issuer half
-- usually wins by a few seconds -- keeping the worse string.
location = CASE WHEN $4::boolean THEN COALESCE($3, location)
ELSE COALESCE(location, $3) END
WHERE id = $1`,
[pair.id, merchant, location, subtitleLocation !== null]
);
if (pair.meta_id) {
await queryRaw(
`UPDATE expense_metadata
SET card_last4 = COALESCE(card_last4, $2),
payment_method_detail = COALESCE(payment_method_detail, $3),
merchant_normalized = COALESCE(NULLIF($4,'Card purchase'), merchant_normalized)
WHERE id = $1`,
[pair.meta_id, ev.card_tail ?? null, ev.issuer ?? null, merchant]
);
}
return { transactionId: pair.id, metadataId: pair.meta_id, skipped: "merged_pair", flags };
}
const txn = await queryRow<{ id: number }>(
`INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id, location,
foreign_currency_amount, foreign_currency_code, source, source_ref, row_index
) VALUES ($1,$2,$3,$3,'other','card',$4,$4,'debit',$5,$6,$7,$8,'wallet',$9,0)
RETURNING id`,
[date, merchant, amountAud, merchant, DEFAULT_OWNER_ID, location, fxAmount, fxCode, sourceRef]
);
const meta = await queryRow<{ id: number }>(
`INSERT INTO expense_metadata (
transaction_id, source, source_message_id, merchant_normalized, amount,
transaction_date, card_last4, currency, payment_method,
payment_method_detail, line_items, flags, extraction_model
) VALUES ($1,'wallet',$2,$3,$4,$5,$6,$7,'card',$8,'[]'::jsonb,$9::jsonb,'wallet-interpret')
RETURNING id`,
[
txn!.id, sourceRef, merchant, ev.amount, date,
ev.card_tail ?? null, stated, ev.issuer ?? null, JSON.stringify(flags),
]
);
return { transactionId: txn!.id, metadataId: meta!.id, flags };
}