receipts: adopt the wallet row instead of creating a second one (board #252)
ci / lint-test (push) Successful in 51s

Step 2 of wallet-first ingestion. Signals for one purchase arrive in
sequence, not in competition: the notification within seconds of the tap,
the receipt hours or days later. Until now the receipt path always
INSERTed its own transaction, so the same shop became two rows.

The case this is built from is measured, not hypothetical. Woolworths
$103.30 on 2026-08-18: the wallet event landed 09:00:14 on the 18th and
nothing created a transaction until 20:25:06 on the 19th -- 35h25m later,
by this path, from scratch -- discarding what only the notification knew
(the instrument, and the suburb).

findAdoptableTransaction() is deliberately as narrow as findAutoMatches():
exact amount, a few days either side, unclaimed, and UNIQUE. Two
candidates means two plausible purchases, and guessing between them moves
one shop's line items onto another shop's row. It also refuses a row that
already carries line items, because a fully-described row is a different
purchase that happens to share an amount. Returning null is always safe --
the caller falls back to creating its own row, which is the old behaviour.

On adoption, both the transaction UPDATE and the expense_metadata UPDATE
have a column list that IS the ownership boundary, following
order_transaction_bridge's UPGRADE pass: the receipt read the register
tape so it wins on what was bought, and `location` is structurally ABSENT
from the update rather than conditionally skipped, so the notification's
suburb cannot be erased by a bug here. expense_metadata.transaction_id is
UNIQUE, so an adopted row that already has metadata is enriched in place
rather than raising.

5 new integration tests including both refusals. Existing receipt tests
unchanged and passing. npm run test:all green.
This commit is contained in:
2026-08-22 16:17:38 +10:00
parent f31f12799a
commit 8e66e527ae
3 changed files with 267 additions and 8 deletions
@@ -0,0 +1,153 @@
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
import { Pool } from "pg";
import { createPool, resetDB, seedParticipants } from "./helpers";
/**
* A receipt arriving after the wallet notification must ENRICH that row, not
* create a second one.
*
* The case this is built from is real. Woolworths $103.30 on 2026-08-18: the
* wallet event landed at 09:00:14 on the 18th, and nothing created a
* transaction until 20:25:06 on the 19th -- 35h25m later -- at which point the
* receipt path made its own row from scratch. One shop, two rows, and the
* notification's contribution (the instrument, and the suburb) thrown away.
*
* Sources for one purchase arrive in sequence, not in competition, so the rule
* is: the receipt read the register tape and wins on what was bought; anything
* only the notification knew survives untouched.
*/
const TOKEN = "test-receipt-token";
let pool: Pool;
let ownerId: number;
let ingest: typeof import("../../app/api/receipts/ingest/route").POST;
const ingestReq = (body: unknown) =>
({ headers: { get: (h: string) => (h === "x-ingest-token" ? TOKEN : null) }, json: async () => body }) as never;
/** A row shaped the way the wallet lane writes one: statementless, no line
* items, carrying the suburb the notification supplied. */
async function walletRow(amount: number, date: string, merchant: string, location: string) {
const t = await pool.query(
`INSERT INTO transactions
(owner_id, statement_id, transaction_date, description, amount, amount_aud,
transaction_type, category, payment_method, merchant_name, merchant_normalized,
location, source, source_ref, row_index)
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'other', 'card', $3, $3, $5, 'wallet', $6, 0)
RETURNING id`,
[ownerId, date, merchant, amount, location, `evt-${amount}-${date}`]
);
const id = t.rows[0].id as number;
await pool.query(
`INSERT INTO expense_metadata
(transaction_id, source, merchant_normalized, amount, transaction_date,
currency, line_items, payment_method_detail, flags)
VALUES ($1, 'wallet', $2, $3, $4, 'AUD', '[]'::jsonb, 'Visa', '[]'::jsonb)`,
[id, merchant, amount, date]
);
return id;
}
const receipt = (amount: number, date: string) => ({
receipt_uid: `woolworths:3345:${amount}:2026-08-18`,
capture_event_id: 900,
merchant_name: "Woolworths",
store_detail: "Wyndham Vale",
transaction_date: date,
total: amount,
tender_legs: [
{ leg_index: 1, amount, card_last4: "6078", card_product: "VISA PREPAID", class: "gift_card" },
],
line_items: [
{ description: "Milk 2L", qty: 1, unit: "ea", amount: 3.5, category: "dairy" },
{ description: "Bread", qty: 1, unit: "ea", amount: 4.0, category: "bakery" },
],
});
beforeAll(async () => {
process.env.RECEIPT_INGEST_TOKEN = TOKEN;
pool = createPool();
vi.doMock("@/lib/auth", () => ({
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
}));
({ POST: ingest } = await import("../../app/api/receipts/ingest/route"));
});
beforeEach(async () => {
await resetDB(pool);
await pool.query("DELETE FROM expense_metadata");
({ ownerId } = await seedParticipants(pool));
});
const txCount = async () =>
Number((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n);
describe("a receipt meeting the wallet row for the same tap", () => {
it("adopts it instead of creating a second transaction", async () => {
const walletId = await walletRow(103.3, "2026-08-18", "Woolworths", "Wyndham Vale, VIC");
expect(await txCount()).toBe(1);
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
expect(await txCount()).toBe(1);
const meta = await pool.query(
`SELECT transaction_id, jsonb_array_length(line_items) AS items, source,
payment_method_detail, card_last4
FROM expense_metadata`
);
expect(meta.rowCount).toBe(1);
expect(meta.rows[0].transaction_id).toBe(walletId);
expect(meta.rows[0].items).toBe(2);
expect(meta.rows[0].source).toBe("pantry");
// The receipt read the card off the tape; the notification only saw "Visa".
expect(meta.rows[0].payment_method_detail).toBe("VISA PREPAID");
expect(meta.rows[0].card_last4).toBe("6078");
});
it("keeps the suburb only the notification knew", async () => {
await walletRow(103.3, "2026-08-18", "Woolworths", "Wyndham Vale, VIC");
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
const t = await pool.query(`SELECT location, merchant_normalized, category FROM transactions`);
// `location` is absent from the adoption UPDATE's column list on purpose:
// the receipt has no suburb, so it must not be able to erase one.
expect(t.rows[0].location).toBe("Wyndham Vale, VIC");
// ...while what the receipt does know wins.
expect(t.rows[0].merchant_normalized).toBe("Woolworths");
expect(t.rows[0].category).toBe("groceries");
});
it("creates its own row when there is no wallet row to adopt", async () => {
await ingest(ingestReq(receipt(58.4, "2026-08-18")));
expect(await txCount()).toBe(1);
const meta = await pool.query(`SELECT source, jsonb_array_length(line_items) AS items FROM expense_metadata`);
expect(meta.rows[0].source).toBe("pantry");
expect(meta.rows[0].items).toBe(2);
});
it("refuses to adopt when two candidates share the amount", async () => {
// Two taps of the same value in the window: adopting either would move this
// shop's line items onto a purchase it did not describe.
await walletRow(103.3, "2026-08-17", "Woolworths", "Wyndham Vale, VIC");
await walletRow(103.3, "2026-08-18", "Coles", "Manor Lakes, VIC");
expect(await txCount()).toBe(2);
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
// Falls back to creating its own row rather than guessing.
expect(await txCount()).toBe(3);
});
it("refuses to adopt a row that already carries line items", async () => {
const other = await walletRow(103.3, "2026-08-18", "Woolworths", "Wyndham Vale, VIC");
await pool.query(
`UPDATE expense_metadata SET line_items = '[{"description":"Something else","amount":1}]'::jsonb
WHERE transaction_id = $1`, [other]
);
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
// A fully-described row is a different purchase that happens to share an amount.
expect(await txCount()).toBe(2);
});
});
+46
View File
@@ -115,3 +115,49 @@ export async function findAutoMatches(
[ownerId, windowDays]
);
}
/**
* An existing statementless transaction that a newly-arriving receipt or order
* describes -- i.e. the row the wallet lane already created for this purchase.
*
* WHY: signals for one purchase arrive in sequence, not in competition. The
* wallet notification lands within seconds of the tap; the receipt is shared
* hours or days later. Measured on txn 6858: the wallet event arrived 35h25m
* before anything created a transaction, and pantry then created one from
* scratch -- so the same Woolworths shop would have had two rows. The receipt
* must adopt the row that already exists and enrich it.
*
* Deliberately narrow, for the same reason findAutoMatches() is: exact amount,
* a few days either side, unclaimed, and UNIQUE. Two candidates means two
* plausible purchases, and inventing a link between the wrong pair moves one
* shop's line items onto another shop's row. Returning null is always safe --
* the caller falls back to creating its own row, which is today's behaviour.
*/
export async function findAdoptableTransaction(
ownerId: number,
amount: number,
date: string,
windowDays: number = AUTO_MATCH_WINDOW_DAYS
): Promise<number | null> {
const rows = await queryRaw<{ id: number }>(
`SELECT t.id
FROM transactions t
WHERE t.statement_id IS NULL
AND t.owner_id = $1
AND t.amount = $2::numeric
AND t.transaction_date BETWEEN $3::date - $4::int AND $3::date + $4::int
AND t.reconciled_with_id IS NULL
AND t.superseded_by_id IS NULL
-- Never adopt a row that another source already fully described; that
-- is a different purchase that happens to share an amount.
AND NOT EXISTS (
SELECT 1 FROM expense_metadata em
WHERE (em.transaction_id = t.id OR em.matched_transaction_id = t.id)
AND jsonb_typeof(em.line_items) = 'array'
AND jsonb_array_length(em.line_items) > 0
)
LIMIT 2`,
[ownerId, amount, date, windowDays]
);
return rows.length === 1 ? rows[0].id : null;
}
+62 -2
View File
@@ -1,5 +1,6 @@
import { queryRaw, queryRow } from "@/lib/db";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import { findAdoptableTransaction } from "@/lib/auto-reconcile";
/**
* Grocery receipts scanned in pantry-app, arriving as candidate spend.
@@ -154,7 +155,29 @@ export async function processReceiptIngestion(receipt: ParsedReceipt): Promise<R
const paymentMethod = paymentMethodFor(leg.class);
const amount = round2(Number(leg.amount));
const description = receipt.store_detail ? `${receipt.merchant_name} ${receipt.store_detail}` : receipt.merchant_name;
const txn = await queryRow<{ id: number }>(
// Adopt the row the wallet lane already created for this tap, if there is
// one. Signals for a purchase arrive in sequence -- the notification within
// seconds, the receipt hours later -- so creating a second transaction here
// is how one shop ends up as two rows. Measured on txn 6858: the wallet
// event arrived 35h25m before this path created its own row for the same
// $103.30 Woolworths shop. Null means no unambiguous candidate, and the
// fallback is exactly the old behaviour.
const adoptedId = await findAdoptableTransaction(DEFAULT_OWNER_ID, amount, receipt.transaction_date);
const txn = adoptedId
? await queryRow<{ id: number }>(
// The receipt read the register tape; the notification read a push
// banner. So the receipt wins on what was bought and where, and the
// column list here IS the ownership boundary -- `location` is absent
// because the wallet notification's suburb is the better value and
// must survive.
`UPDATE transactions
SET description = $2, merchant_name = $3, merchant_normalized = $3,
category = $4, payment_method = $5
WHERE id = $1 RETURNING id`,
[adoptedId, description, receipt.merchant_name, RECEIPT_CATEGORY, paymentMethod]
)
: await queryRow<{ id: number }>(
`INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id
@@ -163,7 +186,44 @@ export async function processReceiptIngestion(receipt: ParsedReceipt): Promise<R
);
const carriesLines = leg.leg_index === cardLeg?.leg_index;
const meta = await queryRow<{ id: number }>(
// An adopted row may already carry a metadata row (the wallet lane writes
// one). expense_metadata.transaction_id is UNIQUE, so a second INSERT would
// raise -- and must not, because the two describe the same purchase.
// Enrich in place instead, with the same ownership split as the transaction
// update above: everything the receipt read wins, and anything only the
// notification knew (issuer, and the transaction's location) is absent from
// the column list rather than conditionally skipped, so it cannot be
// clobbered by a bug here. Mirrors order_transaction_bridge's UPGRADE pass.
const priorMeta = adoptedId
? await queryRow<{ id: number }>(
`SELECT id FROM expense_metadata WHERE transaction_id = $1`, [adoptedId]
)
: null;
const meta = priorMeta
? await queryRow<{ id: number }>(
`UPDATE expense_metadata
SET source = 'pantry', order_reference = $2, line_items = $3::jsonb,
subtotal = $4, amount = $5, merchant_normalized = $6,
transaction_date = $7, card_last4 = COALESCE($8, card_last4),
currency = 'AUD', flags = flags || $9::jsonb,
payment_method = $10, payment_method_detail = $11,
tender_raw = COALESCE($12, tender_raw),
receipt_sha256 = $13, receipt_group = $14,
extraction_model = 'cloud-budget'
WHERE id = $1 RETURNING id`,
[
priorMeta.id, reference,
JSON.stringify(carriesLines ? receipt.line_items ?? [] : []),
receipt.tax_amount ?? null, amount, receipt.merchant_name,
receipt.transaction_date, leg.card_last4 ?? null,
JSON.stringify(carriesLines ? [...flags, "enriched_prior_row"] : [...flags, "line_items_on_card_leg", "enriched_prior_row"]),
paymentMethod, leg.card_product ?? null, receipt.tender_raw ?? null,
receipt.image_sha256 ?? null, group,
]
)
: await queryRow<{ id: number }>(
`INSERT INTO expense_metadata (
transaction_id, source, order_reference, line_items, subtotal, amount,
merchant_normalized, transaction_date, card_last4, currency, flags,