wallet+intake: adopt the row that is already there, and keep a gift card out of the queue
ci / lint-test (push) Successful in 53s

Three defects found by the 2026-08-18 Woolworths shop reaching the ledger
twice — txn 6858 (pantry receipt, 23 items) and txn 6927 (wallet) were one
$103.30 purchase, and the second parked in the reconcile queue waiting for a
statement line the first had already claimed.

The design says signals for a purchase arrive in sequence — wallet first,
receipt second, statement last — and the receipt lane honours it
(findAdoptableTransaction). Nothing else did.

1. The wallet writer inserted unconditionally. It now adopts a row a receipt
   got to first (findWalletAdoptableTransaction). Deliberately NOT the same
   predicate as the receipt direction: that one refuses a row carrying line
   items, because for a receipt such a row is a different purchase sharing an
   amount; here an itemised row IS the receipt being folded into. Losing that
   guard means the merchant supplies it, so adoption requires merchant
   agreement — a tap that vanishes into an unrelated purchase is worse than a
   visible duplicate. Idempotent across a prompt_version replay via a
   wallet_evt:<id> metadata flag, since an adopted row keeps no wallet
   source_ref of its own.

2. intake_create booked a second row for a tap the wallet lane had already
   recorded. matchTransaction cannot see those rows — its key includes the
   platform and a notification has none — so an Uber Eats screenshot never
   matched the wallet row for the very tap that paid for it. That is the
   NORMAL arrival order, not an edge case.

3. A gift-card tap was written payment_method 'card', which means "a statement
   line is coming"; the queue and findAutoMatches filter on exactly that, so
   it parked forever. The instrument's name survives only in the notification
   title ("Visa Gift Card") — the interpreter reduces it to issuer "Visa",
   which a plain Visa debit card also produces — so notif_title is now carried
   through and a stored-value tap is booked 'credits'. 3 of the first 21 rows.

Also: pantry reports "this receipt is already on my board" in the 200 body and
the forwarder dropped the field, so the one gate that DID fire on the duplicate
was silent (captures 66/69). It now becomes the card. And ambiguity — two
ledger rows for one shop, the exact state behind this bug — no longer reads as
"no match, finance will book it".
This commit is contained in:
2026-08-22 18:10:08 +10:00
parent 9b59edcf48
commit 2c05f10aed
7 changed files with 506 additions and 14 deletions
+69
View File
@@ -105,6 +105,17 @@ async function seedOrder(over: { merchant: string; date: string; amount: number;
} }
return id; return id;
} }
/** A row shaped the way the wallet lane writes one: statementless, no metadata
* at all, no platform, and named by the notification rather than the receipt. */
async function seedWalletRow(over: { merchant: string; date: string; amount: number; owner?: number }) {
const t = await pool.query(
`INSERT INTO transactions (owner_id, statement_id, transaction_date, description, amount, amount_aud, transaction_type, category, merchant_name, merchant_normalized, payment_method, location, source, source_ref, row_index)
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'other', $3, $3, 'card', 'Werribee, VIC', 'wallet', $5, 0) RETURNING id`,
[over.owner ?? ownerId, over.date, over.merchant, over.amount, `evt:${Math.floor(over.amount * 100)}`]
);
return t.rows[0].id as number;
}
/** What `components/order-details.tsx` would actually render for a transaction. */ /** What `components/order-details.tsx` would actually render for a transaction. */
const orderPanel = async (txnId: number) => const orderPanel = async (txnId: number) =>
(await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null; (await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null;
@@ -638,6 +649,33 @@ describe("POST /api/intake — grocery → pantry", () => {
expect(pantryCall.intake).toMatchObject({ platform: null, finance_target_transaction_id: null, finance_action: "create" }); expect(pantryCall.intake).toMatchObject({ platform: null, finance_target_transaction_id: null, finance_action: "create" });
}); });
it("pantry saying the shop is already on its board is SHOWN, not swallowed", async () => {
// Pantry accepts a second capture of one receipt on purpose (the first may
// have been a bad crop) and reports it rather than refusing. Dropping that
// field turned "reported" into silence — and a duplicate nobody sees is a
// duplicate that gets confirmed, which counts the same groceries twice
// because pantry's addItem accumulates. Seen live on captures 66/69.
modelReply = coles;
pantryReply = { status: 200, body: { captureId: 902, items: [1, 2, 3], alreadyCaptured: { captureId: 66, confirmed: true } } };
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
expect(body.text).toMatch(/Already on the pantry board/);
expect(body.text).toMatch(/capture 66 \(already confirmed\)/);
expect(body.text).toMatch(/capture 902/);
});
it("two ledger rows for one shop are reported, not treated as no match", async () => {
// Ambiguity is not a miss. This is the exact state that produced the
// 2026-08-18 Woolworths double-book: the shop was in the ledger twice, the
// matcher returned nothing, and the card said finance would book it.
modelReply = coles;
await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: null });
await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: null });
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
expect(body.text).toMatch(/found more than one row it could be/);
});
it("pantry 409 → duplicate", async () => { it("pantry 409 → duplicate", async () => {
modelReply = coles; modelReply = coles;
pantryReply = { status: 409, body: { error: "This receipt has already been added to the pantry.", captureId: 77 } }; pantryReply = { status: 409, body: { error: "This receipt has already been added to the pantry.", captureId: 77 } };
@@ -711,6 +749,37 @@ describe("Slack taps (C5)", () => {
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/); expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
}); });
it("intake_create adopts the wallet row for the same tap instead of booking it twice", async () => {
// The normal arrival order: the notification lands within seconds of the
// tap, the screenshot is shared later. matchTransaction cannot see that row
// — its key includes the platform and a notification has none — so before
// this, Create booked the purchase a second time.
const wallet = await seedWalletRow({ merchant: "Burgertory", date: "2026-08-19", amount: 22.4 });
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: wallet });
expect((await lineItemsOf(wallet)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
// The suburb the notification supplied is the one thing nothing else knows.
expect((await pool.query(`SELECT location FROM transactions WHERE id = $1`, [wallet])).rows[0].location).toBe("Werribee, VIC");
});
it("intake_create still creates when a same-amount row already carries a receipt", async () => {
// A row that another source fully described is a DIFFERENT purchase that
// happens to share an amount — adopting it would move one order's items
// onto another order's row.
const other = await seedOrder({ merchant: "Someone Else", date: "2026-08-19", amount: 22.4, platform: null });
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [other, JSON.stringify([{ description: "theirs", qty: 1, amount: 22.4 }])]);
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(2);
});
it("two pending cards for one order book it ONCE", async () => { it("two pending cards for one order book it ONCE", async () => {
const a = uuid(); const a = uuid();
const b = uuid(); const b = uuid();
@@ -0,0 +1,176 @@
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
import { Pool } from "pg";
import { createPool, resetDB, seedParticipants } from "./helpers";
/**
* The mirror of receipt-adopts-wallet-row: a wallet notification arriving AFTER
* something else already booked the purchase must fold into that row.
*
* Built from the failure, not from the design. The 2026-08-18 Woolworths shop
* ($103.30, Visa gift card) reached the ledger twice: txn 6858 from the pantry
* receipt on the 19th, and txn 6927 from the wallet lane on the 22nd, when the
* first 21 wallet events were backfilled days after their taps. The sequencing
* the lane is designed around -- wallet first, receipt second -- is a tendency,
* and every writer has to survive it not holding.
*
* Two rules are load-bearing here and are tested separately below:
* - the merchant must agree, because losing the "no line items" guard that
* protects the other direction means amount+date alone could swallow a tap
* into an unrelated purchase of the same value
* - a gift-card tap is `credits`, because `card` means "a statement line is
* coming" and the reconcile queue filters on exactly that
*/
const TOKEN = "test-wallet-token";
let pool: Pool;
let ownerId: number;
let ingest: typeof import("../../app/api/wallet/ingest/route").POST;
const req = (body: unknown) =>
({ headers: { get: (h: string) => (h === "x-ingest-token" ? TOKEN : null) }, json: async () => body }) as never;
/** A row shaped the way the pantry receipt lane writes one: statementless,
* itemised, no location (nothing but a notification supplies a suburb). */
async function receiptRow(amount: number, date: string, merchant: string, opts: { items?: number; meta?: boolean } = {}) {
const { items = 3, meta = true } = opts;
const t = await pool.query(
`INSERT INTO transactions
(owner_id, statement_id, transaction_date, description, amount, amount_aud,
transaction_type, category, payment_method, merchant_name, merchant_normalized, row_index)
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'groceries', 'credits', $3, $3, 0)
RETURNING id`,
[ownerId, date, merchant, amount]
);
const id = t.rows[0].id as number;
if (meta) {
const lineItems = Array.from({ length: items }, (_, i) => ({ description: `item ${i}`, qty: 1, amount: 1 }));
await pool.query(
`INSERT INTO expense_metadata
(transaction_id, source, merchant_normalized, amount, transaction_date, currency, line_items, flags)
VALUES ($1, 'pantry', $2, $3, $4, 'AUD', $5::jsonb, '[]'::jsonb)`,
[id, merchant, amount, date, JSON.stringify(lineItems)]
);
}
return id;
}
/** The real event 80: the Apple Wallet half of a Visa gift-card tap. */
const woolworthsGiftCard = {
event_id: 80, kind: "transaction", direction: "unknown", amount: 103.3,
amount_text: "$103.30", currency: "$?", merchant: "Woolworths", issuer: "Visa",
notif_app: "Wallet", notif_title: "Visa Gift Card",
notif_subtitle: "Woolworths, Wyndham Vale, VIC",
occurred_at: "2026-08-18T09:00:14+10:00",
};
const rows = async () =>
(await pool.query(
`SELECT id, amount, merchant_normalized, location, source, source_ref, payment_method
FROM transactions ORDER BY id`
)).rows;
beforeAll(async () => {
process.env.WALLET_INGEST_TOKEN = TOKEN;
pool = createPool();
vi.doMock("@/lib/auth", () => ({
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
}));
({ POST: ingest } = await import("../../app/api/wallet/ingest/route"));
});
beforeEach(async () => {
await resetDB(pool);
await pool.query("DELETE FROM expense_metadata");
({ ownerId } = await seedParticipants(pool));
});
describe("a wallet notification for a purchase the ledger already holds", () => {
it("adopts the receipt's row instead of creating a second one", async () => {
const existing = await receiptRow(103.3, "2026-08-18", "WOOLWORTHS 3345");
const body = await (await ingest(req(woolworthsGiftCard))).json();
expect(body.skipped).toBe("adopted");
expect(body.transactionId).toBe(existing);
expect(await rows()).toHaveLength(1);
});
it("gives the adopted row the suburb, and leaves what the receipt read alone", async () => {
const existing = await receiptRow(103.3, "2026-08-18", "WOOLWORTHS 3345");
await ingest(req(woolworthsGiftCard));
const [t] = await rows();
// Wallet's own field: nothing else on the estate writes a suburb.
expect(t.location).toBe("Wyndham Vale, VIC");
// The receipt read the register tape. The notification does not get to
// rename the merchant to its own shorter string.
expect(t.merchant_normalized).toBe("WOOLWORTHS 3345");
expect(t.id).toBe(existing);
const [m] = (await pool.query(`SELECT * FROM expense_metadata ORDER BY id`)).rows;
expect(m.source).toBe("pantry");
expect(m.line_items).toHaveLength(3);
expect(m.payment_method_detail).toBe("Visa");
expect(m.flags).toContain("wallet_evt:80");
});
it("is idempotent when the interpreter replays every event", async () => {
// Bumping prompt_version re-interprets and re-posts the whole capture
// history on purpose -- that is the replay mechanism. An adopted row keeps
// no wallet source_ref of its own, so the flag is what makes replay safe.
await ingest(req(woolworthsGiftCard));
const second = await (await ingest(req(woolworthsGiftCard))).json();
expect(second.skipped).toBe("already_ingested");
expect(await rows()).toHaveLength(1);
});
it("writes its own row when the merchant disagrees", async () => {
// Same money, same day, different shop. Folding into it would make the tap
// vanish -- worse than a duplicate, which is at least visible in the queue.
await receiptRow(103.3, "2026-08-18", "Bunnings Werribee");
const body = await (await ingest(req(woolworthsGiftCard))).json();
expect(body.skipped).toBeUndefined();
expect(await rows()).toHaveLength(2);
});
it("writes its own row when two candidates share the amount", async () => {
await receiptRow(103.3, "2026-08-18", "Woolworths Wyndham Vale");
await receiptRow(103.3, "2026-08-18", "Woolworths Point Cook");
const body = await (await ingest(req(woolworthsGiftCard))).json();
expect(body.skipped).toBeUndefined();
expect(await rows()).toHaveLength(3);
});
it("creates the metadata row when the adopted transaction has none", async () => {
const existing = await receiptRow(103.3, "2026-08-18", "WOOLWORTHS 3345", { meta: false });
await ingest(req(woolworthsGiftCard));
const meta = (await pool.query(`SELECT * FROM expense_metadata`)).rows;
expect(meta).toHaveLength(1);
expect(meta[0].transaction_id).toBe(existing);
expect(meta[0].card_last4).toBeNull();
expect(meta[0].flags).toContain("wallet_evt:80");
});
});
describe("a stored-value instrument", () => {
it("is booked as credits, so it never enters the reconcile queue", async () => {
const body = await (await ingest(req(woolworthsGiftCard))).json();
const [t] = await rows();
expect(body.transactionId).toBeTruthy();
// 'card' would park it forever: findAutoMatches and the queue both exclude
// 'credits' and 'cash' precisely because no statement line is coming.
expect(t.payment_method).toBe("credits");
expect(body.flags).toContain("stored_value_instrument");
});
it("leaves an ordinary card tap as card", async () => {
const body = await (await ingest(req({ ...woolworthsGiftCard, notif_title: "Payment", issuer: "Mastercard", notif_app: "Westpac" }))).json();
const [t] = await rows();
expect(t.payment_method).toBe("card");
expect(body.flags).not.toContain("stored_value_instrument");
});
});
+61
View File
@@ -1,4 +1,5 @@
import { queryRaw } from "@/lib/db"; import { queryRaw } from "@/lib/db";
import { merchantMatches } from "@/lib/intake-rules";
/** /**
* Unambiguous statement matches for statementless ("manual") transactions. * Unambiguous statement matches for statementless ("manual") transactions.
@@ -133,6 +134,66 @@ export async function findAutoMatches(
* shop's line items onto another shop's row. Returning null is always safe -- * shop's line items onto another shop's row. Returning null is always safe --
* the caller falls back to creating its own row, which is today's behaviour. * the caller falls back to creating its own row, which is today's behaviour.
*/ */
/**
* The row a newly-arriving WALLET notification describes -- the mirror of
* findAdoptableTransaction, for the case where the signals arrived in the other
* order.
*
* WHY: the sequencing this estate is built on -- wallet first, receipt second,
* statement last -- is a tendency, not a guarantee. The wallet lane can be
* parked (its first 21 rows were backfilled days late), the interpreter runs
* every 15 minutes, the phone can be offline. Whenever a receipt reaches the
* ledger before the notification for the same tap, the wallet writer used to
* insert unconditionally: measured on the 2026-08-18 Woolworths shop, txn 6858
* (pantry, 23 items) and txn 6927 (wallet) were the same $103.30 purchase, and
* the second sat in the reconcile queue waiting for a statement line that the
* first had already claimed.
*
* Deliberately NOT the same predicate as findAdoptableTransaction, in the one
* way that matters: that one refuses a row which already carries line items,
* because for a receipt such a row is a DIFFERENT purchase that happens to
* share an amount. Here it is the opposite -- an itemised row is the receipt we
* are trying to fold into, and refusing it is the bug.
*
* Losing that guard means the merchant has to supply it. A wallet row adopts
* only when the merchant agrees (the same whole-word containment the intake
* matcher uses: "Woolworths" vs "WOOLWORTHS 3345"), because amount-and-date
* alone would let a notification silently disappear into an unrelated purchase
* of the same value -- and a wallet row that is never written is a tap with no
* trace, which is worse than a visible duplicate. When in doubt, return null:
* the caller then writes its own row, which is the pre-existing behaviour.
*/
export async function findWalletAdoptableTransaction(
ownerId: number,
amount: number,
date: string,
merchant: string | null,
windowDays: number = AUTO_MATCH_WINDOW_DAYS
): Promise<{ id: number; metadata_id: number | null } | null> {
// No usable merchant, no adoption. "Card purchase" is wallet-ingestion's own
// fallback for a notification that named no merchant at all.
if (!merchant || merchant === "Card purchase") return null;
const rows = await queryRaw<{ id: number; metadata_id: number | null; merchant: string | null }>(
`SELECT t.id, em.id AS metadata_id,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant
FROM transactions t
LEFT JOIN expense_metadata em ON em.transaction_id = t.id
WHERE t.statement_id IS NULL
AND t.owner_id = $1
AND t.amount = $2::numeric
AND t.transaction_date BETWEEN $3::date - $4::int AND $3::date + $4::int
AND t.reconciled_with_id IS NULL
AND t.superseded_by_id IS NULL
-- A wallet row is the OTHER HALF of this tap, not a purchase to fold
-- into; findPairedRow() merges those, with different rules.
AND COALESCE(t.source, '') <> 'wallet'
LIMIT 5`,
[ownerId, amount, date, windowDays]
);
const hits = rows.filter((r) => merchantMatches(r.merchant, merchant));
return hits.length === 1 ? { id: hits[0].id, metadata_id: hits[0].metadata_id } : null;
}
export async function findAdoptableTransaction( export async function findAdoptableTransaction(
ownerId: number, ownerId: number,
amount: number, amount: number,
+26 -2
View File
@@ -120,8 +120,32 @@ export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks:
return { blocks, text }; return { blocks, text };
} }
export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } { export function forwardedCard(
const text = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " "); c: CardInput & {
captureId: number;
enriching: boolean;
/** Pantry already holds this exact receipt (same barcode tuple). */
alreadyCaptured?: { captureId: number; confirmed: boolean } | null;
/** Finance found more than one row this could be, so it picked none. */
ambiguous?: boolean;
}
): { blocks: Block[]; text: string } {
const what = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""}`.trim();
const items = `${c.itemCount} item${c.itemCount === 1 ? "" : "s"}`;
// The warning comes FIRST and the emoji changes, because this card is read at
// a glance on a phone: a duplicate warning appended to a cheerful "→ pantry
// board" line is a duplicate warning nobody reads. Confirming both copies
// counts the same groceries twice — pantry's addItem accumulates — so the
// card has to say which capture to dismiss, not merely that there are two.
if (c.alreadyCaptured) {
const state = c.alreadyCaptured.confirmed ? "already confirmed" : "still waiting to be confirmed";
const text = `Already on the pantry board — ${what} is capture ${c.alreadyCaptured.captureId} (${state}). This upload is capture ${c.captureId}; dismiss it unless it replaces the first, because confirming both counts the same groceries twice.`.replace(/\s+/g, " ");
return { text, blocks: [section(`:repeat: ${text}`)] };
}
const tail = c.ambiguous
? "— finance found more than one row it could be, so it picked none; reconcile it there"
: `— finance ${c.enriching ? "updates" : "books it"} on confirm`;
const text = `${what} → pantry board (${items}) ${tail}`.replace(/\s+/g, " ");
return { text, blocks: [section(`:shopping_trolley: ${text}`)] }; return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
} }
+28 -3
View File
@@ -20,7 +20,21 @@ export type PantryForwardArgs = {
financeAction: "enrich" | "create"; financeAction: "enrich" | "create";
}; };
export type PantryForwardResult = { captureId: number; itemCount: number | null }; /**
* `alreadyCaptured` is pantry telling us this exact receipt is already on its
* board -- same store, register, receipt number and date, which is the tuple
* both Coles and Woolworths encode in their barcode. Pantry deliberately still
* ACCEPTS the second capture (the first may have been a bad crop being
* replaced) and says so instead of refusing; dropping the field on the floor
* turned "said so" into silence, and a duplicate that is not shown is a
* duplicate that gets confirmed -- which double-counts the groceries, because
* pantry's addItem accumulates.
*/
export type PantryForwardResult = {
captureId: number;
itemCount: number | null;
alreadyCaptured: { captureId: number; confirmed: boolean } | null;
};
export class PantryDuplicateError extends Error { export class PantryDuplicateError extends Error {
constructor( constructor(
@@ -65,12 +79,23 @@ export function pantryForwarder(opts: { baseUrl?: string; token?: string; fetchI
} catch (e) { } catch (e) {
throw new PantryUnavailableError(`pantry unreachable: ${e instanceof Error ? e.message : String(e)}`); throw new PantryUnavailableError(`pantry unreachable: ${e instanceof Error ? e.message : String(e)}`);
} }
const body = (await response.json().catch(() => ({}))) as { captureId?: number; error?: string; items?: unknown[]; reason?: string }; const body = (await response.json().catch(() => ({}))) as {
captureId?: number; error?: string; items?: unknown[]; reason?: string;
alreadyCaptured?: { captureId?: number; confirmed?: boolean } | null;
};
if (response.status === 409) { if (response.status === 409) {
throw new PantryDuplicateError(typeof body.captureId === "number" ? body.captureId : null, body.reason ?? body.error ?? "already captured"); throw new PantryDuplicateError(typeof body.captureId === "number" ? body.captureId : null, body.reason ?? body.error ?? "already captured");
} }
if (!response.ok) throw new PantryUnavailableError(`pantry returned ${response.status}: ${body.error ?? ""}`.trim()); if (!response.ok) throw new PantryUnavailableError(`pantry returned ${response.status}: ${body.error ?? ""}`.trim());
if (typeof body.captureId !== "number") throw new PantryUnavailableError("pantry returned no captureId"); if (typeof body.captureId !== "number") throw new PantryUnavailableError("pantry returned no captureId");
return { captureId: body.captureId, itemCount: Array.isArray(body.items) ? body.items.length : null }; const prior = body.alreadyCaptured;
return {
captureId: body.captureId,
itemCount: Array.isArray(body.items) ? body.items.length : null,
alreadyCaptured:
prior && typeof prior.captureId === "number" && prior.captureId !== body.captureId
? { captureId: prior.captureId, confirmed: prior.confirmed === true }
: null,
};
}; };
} }
+38 -2
View File
@@ -2,6 +2,7 @@ import { randomUUID } from "node:crypto";
import { queryRaw, queryRow, withTransaction } from "@/lib/db"; import { queryRaw, queryRow, withTransaction } from "@/lib/db";
import { EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY } from "@/lib/analytics-sql"; import { EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY } from "@/lib/analytics-sql";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion"; import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import { findAdoptableTransaction } from "@/lib/auto-reconcile";
import { import {
classifyInput, classifyInput,
confabulationReason, confabulationReason,
@@ -376,7 +377,12 @@ export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
/** Grocery: the original files go to pantry's board; finance rides along on its confirm (C3/C4). */ /** Grocery: the original files go to pantry's board; finance rides along on its confirm (C3/C4). */
export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantry: PantryForward): Promise<IntakeResult> { export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantry: PantryForward): Promise<IntakeResult> {
const { match } = await matchTransaction(ctx.header, ctx.platform); // `ambiguous` is not a miss. Two ledger rows for one shop is precisely the
// state that produced the 2026-08-18 Woolworths double-book, and forwarding
// with financeAction 'create' as though nothing matched is what turns it into
// a third row on confirm. The groceries still go to the board — they have to,
// it is where they are worked — but the card must say finance could not pick.
const { match, ambiguous } = await matchTransaction(ctx.header, ctx.platform);
try { try {
const r = await pantry({ const r = await pantry({
files, files,
@@ -387,7 +393,14 @@ export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantr
financeAction: match ? "enrich" : "create", financeAction: match ? "enrich" : "create",
}); });
await updateBatch(ctx.batchId, { kind: "grocery", status: "forwarded_to_pantry", transaction_id: match?.transaction_id ?? null, capture_id: r.captureId, items: ctx.items }); await updateBatch(ctx.batchId, { kind: "grocery", status: "forwarded_to_pantry", transaction_id: match?.transaction_id ?? null, capture_id: r.captureId, items: ctx.items });
const card = forwardedCard({ ...cardInput(ctx), itemCount: r.itemCount ?? ctx.items.length, captureId: r.captureId, enriching: !!match }); const card = forwardedCard({
...cardInput(ctx),
itemCount: r.itemCount ?? ctx.items.length,
captureId: r.captureId,
enriching: !!match,
alreadyCaptured: r.alreadyCaptured,
ambiguous,
});
return finish(ctx, "grocery", "forwarded_to_pantry", card, { capture_id: r.captureId, transaction_id: match?.transaction_id }); return finish(ctx, "grocery", "forwarded_to_pantry", card, { capture_id: r.captureId, transaction_id: match?.transaction_id });
} catch (e) { } catch (e) {
if (e instanceof PantryDuplicateError) { if (e instanceof PantryDuplicateError) {
@@ -447,6 +460,29 @@ export async function createFromBatch(batch: BatchRow, kind: "grocery" | "food")
} }
const ownerId = await ownerForEmail(batch.shared_by); const ownerId = await ownerForEmail(batch.shared_by);
// Before minting a row: is one already here that nothing has described yet?
//
// matchTransaction above cannot see it. Its key includes the platform, and a
// wallet notification has none — so an Uber Eats screenshot never matches the
// wallet row for the very tap that paid for it, and the Create button books
// the purchase a second time. That is the normal arrival order, not an edge
// case: the notification lands within seconds of the tap and the screenshot
// is shared later.
//
// findAdoptableTransaction is the same helper the receipt lane uses and
// carries the same guards — exact amount, ±3 days, unclaimed, UNIQUE, and
// never a row that already carries line items. Its uniqueness check runs
// outside the advisory lock below, so two simultaneous taps could both adopt;
// the second would then find the items in place and fall through to the lock,
// which is where duplicates are actually resolved.
const adopted = await findAdoptableTransaction(ownerId, header.total, header.order_date);
if (adopted) {
await replaceLineItems(adopted, items, { source: "intake", platform: header.platform, batchId: batch.id, currency: header.currency });
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: adopted, items });
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: adopted }), { transaction_id: adopted });
}
const merchant = header.merchant ?? "Unknown merchant"; const merchant = header.merchant ?? "Unknown merchant";
// Not `orderDescription()`: it always prefixes "Order - ", which is right for // Not `orderDescription()`: it always prefixes "Order - ", which is right for
// a delivery receipt and wrong for the in-store till receipt this path also // a delivery receipt and wrong for the in-store till receipt this path also
+108 -7
View File
@@ -1,5 +1,6 @@
import { queryRow, queryRaw } from "@/lib/db"; import { queryRow, queryRaw } from "@/lib/db";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion"; import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import { findWalletAdoptableTransaction } from "@/lib/auto-reconcile";
import { import {
parseWestpacDescriptor, parseWestpacDescriptor,
parseWalletSubtitle, parseWalletSubtitle,
@@ -20,6 +21,12 @@ import {
* (receipt-ingestion.ts adopts it), and the statement confirms it via the * (receipt-ingestion.ts adopts it), and the statement confirms it via the
* reconcile queue, which a wallet row enters unaided: it is statementless with * reconcile queue, which a wallet row enters unaided: it is statementless with
* payment_method 'card'. Nothing here needs to know about statements. * payment_method 'card'. Nothing here needs to know about statements.
*
* But that sequence is a TENDENCY, not a guarantee -- these first 21 rows were
* themselves backfilled days after the taps -- so this also adopts a row a
* receipt got to first (findWalletAdoptableTransaction), and a stored-value
* instrument is written as 'credits' so it never enters a queue no statement
* line will ever resolve.
*/ */
export type WalletEvent = { export type WalletEvent = {
@@ -37,6 +44,8 @@ export type WalletEvent = {
issuer?: string | null; issuer?: string | null;
location?: string | null; location?: string | null;
notif_app?: string | null; notif_app?: string | null;
/** The notification's title line, e.g. "Visa Gift Card" or "Payment". */
notif_title?: string | null;
notif_body?: string | null; notif_body?: string | null;
notif_subtitle?: string | null; notif_subtitle?: string | null;
occurred_at?: string | null; occurred_at?: string | null;
@@ -45,7 +54,7 @@ export type WalletEvent = {
export type WalletIngestResult = { export type WalletIngestResult = {
transactionId: number | null; transactionId: number | null;
metadataId: number | null; metadataId: number | null;
skipped?: "not_a_transaction" | "already_ingested" | "no_amount" | "merged_pair"; skipped?: "not_a_transaction" | "already_ingested" | "no_amount" | "merged_pair" | "adopted";
flags: string[]; flags: string[];
}; };
@@ -103,6 +112,31 @@ async function findPairedRow(amount: number, date: string) {
return rows.length === 1 ? rows[0] : null; return rows.length === 1 ? rows[0] : null;
} }
/**
* A stored-value instrument: a gift card, a prepaid travel card, a store card.
*
* It matters because payment_method decides queue membership. 'card' means "a
* statement line is coming"; findAutoMatches and the reconcile queue both
* exclude 'credits' and 'cash' for exactly that reason. A $103.30 Woolworths
* tap on a Visa gift card written as 'card' parks in the queue forever, being
* offered as an unmatched row against statements that can never contain it --
* three of the first 21 wallet rows (events 80, 92, 101) were gift-card taps.
*
* The evidence is the notification TITLE, which is where Apple Wallet prints
* the instrument's own name ("Visa Gift Card"); the interpreter reduces that to
* issuer "Visa", which a plain Visa debit card also produces, so the title is
* the only place the distinction survives. Read as a prior, not a contract: a
* miss leaves today's behaviour, which is the queue.
*/
const STORED_VALUE = /gift\s*card|prepaid|travel\s*(?:money|card)|store\s*card/i;
/** How an adopted row records which wallet event folded into it. */
const adoptedFlag = (eventId: number) => `wallet_evt:${eventId}`;
export function isStoredValueInstrument(ev: Pick<WalletEvent, "notif_title" | "issuer" | "notif_app">): boolean {
return [ev.notif_title, ev.issuer, ev.notif_app].some((v) => !!v && STORED_VALUE.test(v));
}
export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIngestResult> { export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIngestResult> {
const flags: string[] = []; const flags: string[] = [];
@@ -117,9 +151,21 @@ export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIng
} }
const sourceRef = `evt:${ev.event_id}`; const sourceRef = `evt:${ev.event_id}`;
// Two places this event may already have landed: a row this lane wrote (which
// carries the ref natively), or a row it ADOPTED, whose source_ref belongs to
// whoever created it. The adopted case is marked in the metadata flags rather
// than by overwriting a field another lane owns. Both must be checked here,
// because bumping the interpreter's prompt_version deliberately re-interprets
// and re-posts every event ever captured -- that is the replay mechanism, and
// it must not mint a second row for a purchase that has one.
const existing = await queryRow<{ id: number }>( const existing = await queryRow<{ id: number }>(
`SELECT id FROM transactions WHERE source = 'wallet' AND source_ref = $1`, `SELECT id FROM transactions WHERE source = 'wallet' AND source_ref = $1
[sourceRef] UNION ALL
SELECT em.transaction_id AS id FROM expense_metadata em
JOIN transactions t ON t.id = em.transaction_id
WHERE em.flags @> $2::jsonb AND t.superseded_by_id IS NULL
LIMIT 1`,
[sourceRef, JSON.stringify([adoptedFlag(ev.event_id)])]
); );
if (existing) { if (existing) {
return { transactionId: existing.id, metadataId: null, skipped: "already_ingested", flags }; return { transactionId: existing.id, metadataId: null, skipped: "already_ingested", flags };
@@ -175,6 +221,12 @@ export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIng
} }
} }
// A gift card has no statement line coming, so it must not enter the
// reconcile queue (which filters on exactly this column).
const storedValue = isStoredValueInstrument(ev);
const paymentMethod = storedValue ? "credits" : "card";
if (storedValue) flags.push("stored_value_instrument");
// Merge the paired notification rather than creating a second row for one // Merge the paired notification rather than creating a second row for one
// tap. The issuer half brings the card tail; the Wallet half brings the // tap. The issuer half brings the card tail; the Wallet half brings the
// merchant and suburb. COALESCE so whichever arrives second only ever fills // merchant and suburb. COALESCE so whichever arrives second only ever fills
@@ -207,14 +259,63 @@ export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIng
return { transactionId: pair.id, metadataId: pair.meta_id, skipped: "merged_pair", flags }; return { transactionId: pair.id, metadataId: pair.meta_id, skipped: "merged_pair", flags };
} }
// The receipt got here first. Fold into its row instead of writing a second
// one for the same purchase (see findWalletAdoptableTransaction). The column
// list IS the ownership boundary, the mirror of receipt-ingestion.ts's: the
// receipt read the register tape, so merchant, description, category and the
// items are its business and are absent here; `location` is wallet's own --
// nothing else writes a suburb -- and payment_method is filled only when the
// other lane left it unknown.
const adopted = await findWalletAdoptableTransaction(DEFAULT_OWNER_ID, amountAud, date, merchant);
if (adopted) {
await queryRaw(
`UPDATE transactions
SET location = COALESCE($2, location),
payment_method = COALESCE(payment_method, $3)
WHERE id = $1`,
[adopted.id, location, paymentMethod]
);
let metadataId = adopted.metadata_id;
if (metadataId) {
await queryRaw(
`UPDATE expense_metadata
SET card_last4 = COALESCE(card_last4, $2),
payment_method_detail = COALESCE(payment_method_detail, $3),
currency = COALESCE(currency, $4),
flags = COALESCE(flags, '[]'::jsonb) || $5::jsonb
WHERE id = $1`,
[metadataId, ev.card_tail ?? null, ev.issuer ?? null, stated, JSON.stringify([...flags, adoptedFlag(ev.event_id)])]
);
} else {
// expense_metadata.transaction_id is UNIQUE, so this branch only runs
// when the adopted row genuinely has none -- and it must produce one, or
// the card tail and the flag that makes this idempotent have nowhere to
// live.
const meta = await queryRow<{ id: number }>(
`INSERT INTO expense_metadata (
transaction_id, source, source_message_id, merchant_normalized, amount,
transaction_date, card_last4, currency, payment_method,
payment_method_detail, line_items, flags, extraction_model
) VALUES ($1,'wallet',$2,$3,$4,$5,$6,$7,$8,$9,'[]'::jsonb,$10::jsonb,'wallet-interpret')
RETURNING id`,
[
adopted.id, sourceRef, merchant, ev.amount, date, ev.card_tail ?? null, stated,
paymentMethod, ev.issuer ?? null, JSON.stringify([...flags, adoptedFlag(ev.event_id)]),
]
);
metadataId = meta!.id;
}
return { transactionId: adopted.id, metadataId, skipped: "adopted", flags };
}
const txn = await queryRow<{ id: number }>( const txn = await queryRow<{ id: number }>(
`INSERT INTO transactions ( `INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method, transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id, location, merchant_name, merchant_normalized, transaction_type, owner_id, location,
foreign_currency_amount, foreign_currency_code, source, source_ref, row_index foreign_currency_amount, foreign_currency_code, source, source_ref, row_index
) VALUES ($1,$2,$3,$3,'other','card',$4,$4,'debit',$5,$6,$7,$8,'wallet',$9,0) ) VALUES ($1,$2,$3,$3,'other',$10,$4,$4,'debit',$5,$6,$7,$8,'wallet',$9,0)
RETURNING id`, RETURNING id`,
[date, merchant, amountAud, merchant, DEFAULT_OWNER_ID, location, fxAmount, fxCode, sourceRef] [date, merchant, amountAud, merchant, DEFAULT_OWNER_ID, location, fxAmount, fxCode, sourceRef, paymentMethod]
); );
const meta = await queryRow<{ id: number }>( const meta = await queryRow<{ id: number }>(
@@ -222,11 +323,11 @@ export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIng
transaction_id, source, source_message_id, merchant_normalized, amount, transaction_id, source, source_message_id, merchant_normalized, amount,
transaction_date, card_last4, currency, payment_method, transaction_date, card_last4, currency, payment_method,
payment_method_detail, line_items, flags, extraction_model payment_method_detail, line_items, flags, extraction_model
) VALUES ($1,'wallet',$2,$3,$4,$5,$6,$7,'card',$8,'[]'::jsonb,$9::jsonb,'wallet-interpret') ) VALUES ($1,'wallet',$2,$3,$4,$5,$6,$7,$8,$9,'[]'::jsonb,$10::jsonb,'wallet-interpret')
RETURNING id`, RETURNING id`,
[ [
txn!.id, sourceRef, merchant, ev.amount, date, txn!.id, sourceRef, merchant, ev.amount, date,
ev.card_tail ?? null, stated, ev.issuer ?? null, JSON.stringify(flags), ev.card_tail ?? null, stated, paymentMethod, ev.issuer ?? null, JSON.stringify(flags),
] ]
); );