6 Commits
Author SHA1 Message Date
siddharthd 631e75e58f test(intake): cover the review findings, and CLAUDE.md for the rules they set
ci / lint-test (push) Successful in 59s
Unit: word-boundary merchant matching (Bigals Pizza is not an IGA), mrkdwn
escaping vs the plain_text button label, a foreign total on a card, and the
finish_reason gate.

Integration: the NULL-platform wildcard on both the food and grocery paths,
candidates crossing platforms, EXCLUDE_RECONCILED_SOURCE, the pinned metadata
row, the panel reading the row the write landed on, the once-per-batch undo
baseline, the priceless-upload hold, Create re-matching, two pending cards
booking once, a pantry-resolved sibling, and a USD receipt.

Eleven of the thirteen fail against the pre-fix tree; the exceptions are the
no-batch-id baseline guard (a no-change assertion) and the concurrent-tap one,
which is a race and passed either way.
2026-08-21 17:53:06 +10:00
siddharthd c25db5211b intake: a NULL platform is "no platform", and Create re-checks before it inserts
Six fixes in the router, all of them cases where a stale or over-wide answer
was written rather than re-asked.

- ($3 IS NULL OR em.platform = $3) made an in-store receipt a wildcard: a till
  receipt from the Coles tap matched the Uber Eats Coles order for the same
  day and money, and overwrote its items. In-store now matches only rows with
  no platform of their own.
- findCandidates loses its platform filter entirely. That card is a human
  choice, shown precisely because the exact key missed — the platform is one
  of the things that may be wrong, so the list must widen there, not narrow.
- createFromBatch re-runs the match. The card is minutes old by the time it is
  tapped and the email/order bridge books orders on its own schedule; the
  match it was drawn from is not current.
- ...and checks for a resolved sibling batch. findDuplicateBatch ignores
  unresolved batches by design, so two pending cards for one order each booked
  it.
- INSERT ... ON CONFLICT (source, source_ref) DO NOTHING with a re-select,
  rather than SELECT-then-INSERT, which loses a double tap.
- A non-AUD receipt is booked as order-ingestion.ts books one: amount native,
  amount_aud NULL, foreign_currency_* set. No FX rate exists at intake.

Also: EXCLUDE_RECONCILED_SOURCE in both queries instead of a bare
superseded_by_id check — a reconciled manual twin at the exact key read as an
ambiguity, which is the same spend counted twice. And routeFood refuses to
replace an itemised receipt with an extraction carrying no prices at all: the
reconciliation check passes those (there is nothing to check), so without this
a priceless upload silently destroys a good record. It holds instead.

RouteContext.subtotal was written and never read; it is replaced by currency,
which the cards need.
2026-08-21 17:53:06 +10:00
siddharthd 9a442b7355 intake: escape mrkdwn, and stop printing every total as dollars
Merchant names, transaction descriptions and model output are none of ours:
Slack reads &, < and > as markup, so "Ben & Jerry's" and a description
carrying <http://…|text> rendered as something the card never meant to show.
esc() on every interpolated value in mrkdwn — never on our own literal *bold*,
and never on a button's plain_text label, where entities would print verbatim.

money() also hard-coded "$" while the order nudge already prints "USD 15.00".
The cards now carry the batch's currency.
2026-08-21 17:52:47 +10:00
siddharthd f9638cc5e5 line-items: one metadata row, agreed on by the write and the read
Three defects on one surface, all invisible while a transaction has exactly
one expense_metadata row — which is almost always.

1. ORDER BY (transaction_id = $1) DESC did the opposite of its comment.
   Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at the
   transaction has a NULL transaction_id, so the clause written to prefer the
   owned row preferred the other one — in exactly the case it exists for.
   matchTransaction already spelled out NULLS LAST; now so do this and
   GET /api/transactions/[id]/order, which had no ORDER BY at all and could
   read a different row from the one an enrichment had just written.

2. replaceLineItems takes an optional metadataId. The match resolves a row
   through a platform-filtered join; re-picking independently here can land
   the write on a row the match ruled out.

3. previous_line_items on intake_batches is the undo baseline, recorded once
   per batch under COALESCE and read back. Returning whatever was just read
   hands a pantry retry its own write as the "previous" state, so undo
   restores the thing it undid. Migration 0034 is unapplied anywhere, so the
   column is added there rather than in a 0035.
2026-08-21 17:52:47 +10:00
siddharthd 7424d13858 intake: only a reply that finished is a reply the schema constrained
liteLLMExtractor rejected finish_reason "length" and accepted everything else,
so a content_filter or tool_calls stop was parsed as if the strict json_schema
had held. Reject any reason present and not "stop", naming it.
2026-08-21 17:52:25 +10:00
siddharthd be3d10d4f1 intake: match merchants on whole words, not substrings
merchantKey("Bigals Pizza") contains "iga", so GROCERY_PATTERN filed a pizza
shop as a grocer and merchantMatches matched it against every IGA on the same
date and total. Short names in a substring list collide with ordinary words,
and the collision reads as a correct classification rather than a bug.

GROCERY_PATTERN is now a word-boundary alternation; merchantMatches asks
whether one name's words appear as a contiguous run inside the other's, which
keeps Coles <-> Coles (Wyndham Vale) and Order - Coles (Wyndham Vale).

Also collapses the third PLATFORM_LABEL copy: order-ingestion's was exported
and read nowhere, order-details.tsx had its own, and the intake one — the only
live copy — was missing `uber`. One map, in intake-rules.
2026-08-21 17:52:25 +10:00
13 changed files with 621 additions and 98 deletions
+30 -1
View File
@@ -640,7 +640,36 @@ and the cross-repo contract are in smarthome `plans/intake-router-slice-plan.md`
`intake_use_candidate` / `intake_create` buttons; a transaction is created `intake_use_candidate` / `intake_create` buttons; a transaction is created
only on the explicit tap, ING-10-style (`statement_id NULL`, `owner_id` from only on the explicit tap, ING-10-style (`statement_id NULL`, `owner_id` from
`shared_by` → `participants.email`, else `DEFAULT_OWNER_ID`; `shared_by` → `participants.email`, else `DEFAULT_OWNER_ID`;
`source='intake'`, `source_ref=batch_id` so a second press is a no-op). `source='intake'`, `source_ref=batch_id` so a second press is a no-op —
written as `ON CONFLICT … DO NOTHING`, because a SELECT-then-INSERT loses a
double tap). A non-AUD receipt is booked the way `order-ingestion.ts` books
one: `amount` native, `amount_aud` **NULL**, `foreign_currency_*` set — no FX
rate exists at intake and inventing one is worse than admitting there is none.
- **A NULL platform means "no platform", not "any platform".** An in-store tap
(`coles`, `woolworths`, `other`) matches only rows whose `expense_metadata`
carries no platform of its own. The wildcard reading let a till receipt match
the Uber Eats order for the same shop on the same day and overwrite its items.
The **candidates** query is the opposite and deliberately so: it has no
platform filter at all, because the card is a human choice shown precisely
because the exact key missed — the platform is one of the things that may be
wrong, so the list must widen there, not narrow.
- **The Create tap re-runs the match before it inserts**, and checks for a
resolved sibling batch. The card is minutes old by the time it is pressed and
the email/order bridge books orders on its own schedule; `findDuplicateBatch`
ignores *unresolved* batches by design, so two pending cards for one order
would otherwise each book it.
- **One `expense_metadata` row, agreed on by three queries.** A transaction can
own a row *and* be the `matched_transaction_id` of another; `matchTransaction`
returns the row it resolved through and `replaceLineItems` writes to exactly
that one. All three (match, write, `GET /api/transactions/[id]/order`) order
by `(transaction_id = $1) DESC **NULLS LAST**, id` — the `NULLS LAST` is
load-bearing, because Postgres orders `DESC` as `NULLS FIRST` and the row that
merely *points* at the transaction has a NULL `transaction_id`, so the clause
meant to prefer the owned row preferred the other one.
- **`intake_batches.previous_line_items` is the undo baseline, written once.**
Returning whatever the write just read makes a pantry retry get its own write
back as the "previous" state, so undo restores the thing it undid. Recorded
under `COALESCE(previous_line_items, …)` and read back.
- **Grocery never writes finance here.** The ORIGINAL files go to pantry - **Grocery never writes finance here.** The ORIGINAL files go to pantry
`POST /api/capture` with `intake.finance_target_transaction_id` + `POST /api/capture` with `intake.finance_target_transaction_id` +
`finance_action: enrich|create`; pantry's confirm calls `finance_action: enrich|create`; pantry's confirm calls
@@ -32,10 +32,20 @@ CREATE TABLE IF NOT EXISTS intake_batches (
status text NOT NULL, status text NOT NULL,
transaction_id integer, transaction_id integer,
capture_id integer, capture_id integer,
-- The undo baseline: what `expense_metadata.line_items` held before this
-- batch first replaced it. Recorded here rather than returned from the write,
-- because pantry re-calls the enrich endpoint on every confirm and after a
-- timeout — a caller that hands back whatever it just read gives a retry its
-- OWN write as the "previous" state, and undo then restores the thing it was
-- meant to reverse. Written under COALESCE, so only the first snapshot sticks.
previous_line_items jsonb,
created_at timestamptz NOT NULL DEFAULT now(), created_at timestamptz NOT NULL DEFAULT now(),
resolved_at timestamptz resolved_at timestamptz
); );
-- For a database that already ran an earlier draft of this file.
ALTER TABLE intake_batches ADD COLUMN IF NOT EXISTS previous_line_items jsonb;
COMMENT ON TABLE intake_batches IS COMMENT ON TABLE intake_batches IS
'One row per upload batch through POST /api/intake. Persisted before ' 'One row per upload batch through POST /api/intake. Persisted before '
'extraction so Slack taps can act on it later and re-shares are detected. ' 'extraction so Slack taps can act on it later and re-shares are detected. '
+217 -2
View File
@@ -49,6 +49,7 @@ let fileDir: string;
let intakePOST: (req: NextRequest) => Promise<Response>; let intakePOST: (req: NextRequest) => Promise<Response>;
let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>; let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
let slackPOST: (req: NextRequest) => Promise<Response>; let slackPOST: (req: NextRequest) => Promise<Response>;
let orderGET: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
/** What the next LiteLLM call answers, and what the next pantry call answers. */ /** What the next LiteLLM call answers, and what the next pantry call answers. */
let modelReply: ExtractedOrder | Error = burgertory; let modelReply: ExtractedOrder | Error = burgertory;
@@ -104,6 +105,10 @@ async function seedOrder(over: { merchant: string; date: string; amount: number;
} }
return id; return id;
} }
/** What `components/order-details.tsx` would actually render for a transaction. */
const orderPanel = async (txnId: number) =>
(await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null;
const metaRows = async () => (await pool.query(`SELECT id, platform, line_items FROM expense_metadata ORDER BY id`)).rows as { id: number; platform: string | null; line_items: { description: string }[] }[];
const lineItemsOf = async (txnId: number) => const lineItemsOf = async (txnId: number) =>
(await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items; (await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items;
const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0]; const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0];
@@ -127,9 +132,17 @@ beforeAll(async () => {
participantForSlackUser: () => 1, participantForSlackUser: () => 1,
slackUserForParticipant: () => null, slackUserForParticipant: () => null,
})); }));
// The order-detail panel is the surface an enrichment has to appear on, so
// the read is exercised beside the write rather than assumed.
vi.doMock("@/lib/auth", () => ({ getCurrentUser: async () => ({ id: 1, name: "Owner", email: "o@x" }) }));
vi.doMock("@/lib/queries", async (importOriginal) => ({
...(await importOriginal<Record<string, unknown>>()),
canAccessTransactions: async () => true,
}));
({ POST: intakePOST } = await import("../../app/api/intake/route")); ({ POST: intakePOST } = await import("../../app/api/intake/route"));
({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route")); ({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route"));
({ POST: slackPOST } = await import("../../app/api/slack/interactive/route")); ({ POST: slackPOST } = await import("../../app/api/slack/interactive/route"));
({ GET: orderGET } = await import("../../app/api/transactions/[id]/order/route"));
stubFetch(); stubFetch();
}); });
@@ -210,6 +223,53 @@ describe("PUT /api/transactions/[id]/line-items (C2)", () => {
expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" }); expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" });
}); });
it("the undo baseline is recorded ONCE per batch: a retry gets the original, not its own write", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const original = [{ qty: 1, description: "what the order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(original)]);
const id = uuid();
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
const first = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(first.previous_line_items).toEqual(original);
// Pantry re-calls on every confirm and after a timeout. Before, the retry
// was handed its OWN write back, so undo restored the thing it undid.
const retry = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(retry.previous_line_items).toEqual(original);
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual(original);
await put(txn, { line_items: retry.previous_line_items, source: "pantry", batch_id: id, replace: true });
expect(await lineItemsOf(txn)).toEqual(original);
});
it("without a batch id the baseline is still the row as read — nothing else changes", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(first.previous_line_items).toEqual([]);
const second = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(second.previous_line_items).toEqual(items);
});
it("the panel reads the row the write landed on, whichever order the rows were inserted in", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// Inserted first: the delivery receipt pointing at the statement line.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
[t, uuid()]
);
// Inserted second: a row the transaction owns. An unordered LIMIT 1 reads
// the first one, the write takes the second, and the enrichment reports
// success against a row nothing renders.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, line_items) VALUES ($1, 'pantry', $2, '[]'::jsonb)`,
[t, uuid()]
);
const written = await (await put(t, { line_items: items, source: "intake", replace: true })).json();
const panel = await orderPanel(t);
expect(panel?.line_items).toHaveLength(2);
expect((await metaRows()).find((r) => r.id === written.metadata_id)!.line_items).toHaveLength(2);
});
it("finds the row through matched_transaction_id — a card-settled order has only that", async () => { it("finds the row through matched_transaction_id — a card-settled order has only that", async () => {
const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false }); const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
await pool.query( await pool.query(
@@ -285,10 +345,82 @@ describe("POST /api/intake — food (enrich vs candidates)", () => {
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1); expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
}); });
it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match", async () => { it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match — but it IS offered", async () => {
await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" }); const doordash = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" });
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json(); const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("candidates"); expect(body.status).toBe("candidates");
// The candidates card is a human choice, not a decision. The platform is one
// of the things that may be wrong, so it must not narrow the list — a
// platform-filtered card showed nothing at all here.
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([doordash]);
});
it("a NULL platform is 'no platform', not a wildcard — an in-store share does not overwrite the delivery order", async () => {
await pool.query(`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ('burgertory werribee', 'food')`);
const delivery = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "ubereats" });
const already = [{ qty: 1, description: "what the delivery order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [delivery, JSON.stringify(already)]);
// `other` implies no platform; the delivery row carries one, so it is not
// the match — it is only a candidate.
const body = await (await intakePOST(multipart({ source: "other" }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([delivery]);
expect(await lineItemsOf(delivery)).toEqual(already);
});
it("a reconciled manual row is neither matched nor offered — EXCLUDE_RECONCILED_SOURCE, as everywhere else", async () => {
const live = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const reconciled = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, reconciled]);
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, near]);
// Two rows at the exact key would have read as an ambiguity; one of them is
// the same spend counted twice.
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: live });
});
it("the write lands on the row the MATCH resolved through, not on whichever row sorts first", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// The statement line owns a receipt row of its own (a pantry confirm) …
const grocery = [{ qty: 1, description: "a grocery shop", amount: 22.4 }];
const own = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, merchant_normalized, line_items) VALUES ($1, 'pantry', $2, 'Woolworths', $3::jsonb) RETURNING id`,
[t, uuid(), JSON.stringify(grocery)]
)).rows[0].id as number;
// … and is also the card leg of the Uber Eats order being enriched.
const delivery = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, merchant_normalized, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, 'Burgertory (Werribee)', '[]'::jsonb) RETURNING id`,
[t, uuid()]
)).rows[0].id as number;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: t });
const rows = await metaRows();
expect(rows.find((r) => r.id === delivery)!.line_items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
expect(rows.find((r) => r.id === own)!.line_items).toEqual(grocery);
});
it("an upload with no prices never replaces a receipt that already has items", async () => {
const priceless = { ...burgertory, subtotal: null, total: 22.4, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
modelReply = priceless;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "held", status: "held" });
expect(body.text).toMatch(/left the existing items alone/);
expect(await lineItemsOf(txn)).toEqual(already);
// …but an EMPTY receipt is exactly what this slice exists to fill.
const empty = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-20", amount: 22.4 });
modelReply = { ...priceless, order_date: "2026-08-20" };
const second = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(second).toMatchObject({ status: "enriched", transaction_id: empty });
expect((await lineItemsOf(empty)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
}); });
it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => { it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => {
@@ -391,6 +523,19 @@ describe("POST /api/intake — grocery → pantry", () => {
expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn }); expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
}); });
it("an in-store tap does not hand pantry the delivery order as its finance target", async () => {
modelReply = coles;
// The Uber Eats Coles order the email pipeline already booked.
const delivery = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: "ubereats" });
// The same shop, same day, same money — but shared from the Coles app, so
// it carries no platform and is a different purchase.
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown> };
expect(pantryCall.intake).toMatchObject({ finance_action: "create", finance_target_transaction_id: null, platform: null });
expect(await lineItemsOf(delivery)).toEqual([]);
});
it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => { it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => {
modelReply = coles; modelReply = coles;
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json(); const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
@@ -457,6 +602,76 @@ describe("Slack taps (C5)", () => {
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1); expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
}); });
it("intake_create re-runs the match: an order booked between the card and the tap is enriched, not duplicated", async () => {
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
// The email/order bridge books the same purchase while the card sits in
// Slack. The card's "no match" is minutes stale by the time it is pressed.
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
expect((await lineItemsOf(txn)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
});
it("two pending cards for one order book it ONCE", async () => {
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
expect((await batch(a)).status).toBe("candidates");
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: a }));
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect((await batch(b)).transaction_id).toBe((await batch(a)).transaction_id);
});
it("a sibling already resolved to the pantry board makes Create a duplicate, not a second row", async () => {
// Two shares of one receipt, resolved out of order: nothing the second card
// can match on, because the first one never produced a transaction.
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "other", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
await intakePOST(multipart({ source: "other", batch_id: a }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
expect((await batch(a)).status).toBe("ask");
await slackPOST(slackPress("intake_grocery", { batch_id: a }));
expect((await batch(a)).status).toBe("forwarded_to_pantry");
await slackPOST(slackPress("intake_meal", { batch_id: b }));
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
expect(await batch(b)).toMatchObject({ status: "duplicate", capture_id: 901 });
expect(lastSlackUpdate().text).toMatch(/^Already handled/);
});
it("two taps that land together still book one transaction, and neither reports an error", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: id })), slackPOST(slackPress("intake_create", { batch_id: id }))]);
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
const texts = calls.filter((c) => c.url.includes("hooks.slack.com")).map((c) => (c.body as { text: string }).text);
expect(texts.every((t) => !/Couldn't process/.test(t))).toBe(true);
});
it("a foreign receipt is booked in its own currency, with no invented AUD figure", async () => {
modelReply = { ...burgertory, currency: "USD" };
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
const t = (await pool.query(`SELECT amount::text, amount_aud, foreign_currency_amount::text, foreign_currency_code FROM transactions`)).rows[0];
expect(t).toEqual({ amount: "22.40", amount_aud: null, foreign_currency_amount: "22.40", foreign_currency_code: "USD" });
expect(lastSlackUpdate().text).toMatch(/USD 22\.40/);
});
it("unknown shared_by falls back to the default owner — never a NULL owner", async () => { it("unknown shared_by falls back to the default owner — never a NULL owner", async () => {
const id = uuid(); const id = uuid();
await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf])); await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf]));
+63 -2
View File
@@ -3,6 +3,7 @@ import { readFileSync } from "fs";
import { resolve } from "path"; import { resolve } from "path";
import { import {
classifyInput, classifyInput,
GROCERY_PATTERN,
merchantKey, merchantKey,
merchantMatches, merchantMatches,
platformForSource, platformForSource,
@@ -12,8 +13,8 @@ import {
validateHeader, validateHeader,
type ExtractedOrder, type ExtractedOrder,
} from "../../lib/intake-rules"; } from "../../lib/intake-rules";
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT } from "../../lib/intake-extract"; import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor } from "../../lib/intake-extract";
import { askCard, candidatesCard, enrichedCard } from "../../lib/intake-blocks"; import { askCard, candidatesCard, enrichedCard, type Block } from "../../lib/intake-blocks";
/** /**
* The intake router's pure rules, against the strings real receipts print. * The intake router's pure rules, against the strings real receipts print.
@@ -85,6 +86,17 @@ describe("merchant pattern — grocery | food | ask", () => {
expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false); expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false);
expect(merchantMatches(null, "Coles")).toBe(false); expect(merchantMatches(null, "Coles")).toBe(false);
}); });
it("whole words, not substrings: `Bigals Pizza` contains `iga` and is neither a grocer nor an IGA", () => {
expect(GROCERY_PATTERN.test(merchantKey("Bigals Pizza"))).toBe(false);
expect(resolveMerchantKind("uber_eats", "Bigals Pizza")).toBe("food");
expect(merchantMatches("Bigals Pizza", "IGA")).toBe(false);
// and the grocers a boundary must not cost us
expect(resolveMerchantKind("uber_eats", "IGA Wyndham Vale")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Woolworths Group Limited")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
expect(merchantMatches("IGA Wyndham Vale", "IGA")).toBe(true);
});
}); });
describe("validateHeader — nothing is written from a half-read upload", () => { describe("validateHeader — nothing is written from a half-read upload", () => {
@@ -221,4 +233,53 @@ describe("cards carry the C5 action ids and JSON values", () => {
expect(ids).toContain('"action_id":"intake_grocery"'); expect(ids).toContain('"action_id":"intake_grocery"');
expect(ids).toContain('"action_id":"intake_meal"'); expect(ids).toContain('"action_id":"intake_meal"');
}); });
it("mrkdwn escapes the names it did not write; the plain_text button keeps them raw", () => {
const hostile = { ...c, merchant: "Ben & Jerry's <fake>" };
const { text, blocks } = candidatesCard(hostile, [
{ transaction_id: 42, description: "SQ *BEN & JERRY <AU>", transaction_date: "2026-08-17", amount: "19.90" },
]);
const sections = JSON.stringify(blocks.filter((b) => (b as Block).type === "section"));
expect(sections).toContain("Ben &amp; Jerry's &lt;fake&gt;");
expect(sections).toContain("SQ *BEN &amp; JERRY &lt;AU&gt;");
expect(sections).not.toContain("<fake>");
expect(text).toContain("Ben &amp; Jerry's &lt;fake&gt;");
// The Create button's label is plain_text — Slack shows entities literally
// there, so escaping it would print "Ben &amp; Jerry's" on the button.
const actions = JSON.stringify(blocks.filter((b) => (b as Block).type === "actions"));
expect(actions).toContain("Ben & Jerry's <fake>");
});
it("a foreign receipt is not printed as dollars", () => {
expect(enrichedCard({ ...c, currency: "USD", transactionId: 7 }).text).toBe(
"Added 2 items to your USD 22.40 Uber Eats (Burgertory (Werribee)) order ✓"
);
expect(enrichedCard({ ...c, currency: "AUD", transactionId: 7 }).text).toContain("$22.40");
expect(enrichedCard({ ...c, transactionId: 7 }).text).toContain("$22.40");
});
});
describe("liteLLMExtractor — only a finished reply is accepted", () => {
const good = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD", items: [], adjustments: [] };
const extractor = (finish_reason?: string) =>
liteLLMExtractor({
baseUrl: "http://litellm.test",
key: "k",
fetchImpl: (async () =>
Response.json({
choices: [{ message: { content: JSON.stringify(good) }, ...(finish_reason ? { finish_reason } : {}) }],
})) as unknown as typeof fetch,
});
it("accepts `stop`, and a reply that reports no reason at all", async () => {
expect((await extractor("stop")([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
expect((await extractor()([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
});
it("rejects any other reason, naming it — the strict schema only held for a reply that finished", async () => {
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(ExtractionError);
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/too long/);
await expect(extractor("content_filter")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/content_filter/);
await expect(extractor("tool_calls")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/tool_calls/);
});
}); });
@@ -34,6 +34,14 @@ export async function GET(
-- through matched_transaction_id. Both directions have to resolve or the -- through matched_transaction_id. Both directions have to resolve or the
-- detail is missing on exactly the orders that were paid by card. -- detail is missing on exactly the orders that were paid by card.
WHERE transaction_id = $1 OR matched_transaction_id = $1 WHERE transaction_id = $1 OR matched_transaction_id = $1
-- The same order as replaceLineItems in line-items.ts and matchTransaction
-- in intake-router.ts. Both directions can resolve at once — a statement
-- line owning its own row while a delivery receipt points at it — and an
-- unordered LIMIT 1 then reads a different row from the one an enrichment
-- wrote, so a PUT that reported success showed nothing on the screen.
-- NULLS LAST because DESC is NULLS FIRST and the pointing row's
-- transaction_id is NULL.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`, LIMIT 1`,
[Number(id)] [Number(id)]
); );
+4 -7
View File
@@ -13,12 +13,9 @@ import {
type OrderReceipt, type OrderReceipt,
type OrderRating, type OrderRating,
} from "@/lib/hooks"; } from "@/lib/hooks";
// One platform vocabulary, shared with the intake cards — three copies had
const PLATFORM_LABEL: Record<string, string> = { // already drifted apart (the intake one was missing `uber`).
doordash: "DoorDash", import { platformLabel } from "@/lib/intake-rules";
ubereats: "Uber Eats",
uber: "Uber",
};
/** /**
* Who records verdicts. A two-person household with one primary user: the * Who records verdicts. A two-person household with one primary user: the
@@ -99,7 +96,7 @@ export function OrderDetails({
<p className="text-xs text-zinc-500"> <p className="text-xs text-zinc-500">
Order details Order details
{receipt.platform && ( {receipt.platform && (
<span className="ml-1.5 text-zinc-400">{PLATFORM_LABEL[receipt.platform] ?? receipt.platform}</span> <span className="ml-1.5 text-zinc-400">{platformLabel(receipt.platform)}</span>
)} )}
</p> </p>
{receipt.card_last4 && ( {receipt.card_last4 && (
+2 -1
View File
@@ -58,6 +58,7 @@ const ctxFor = (batch: BatchRow): RouteContext | null =>
platform: batch.header.platform, platform: batch.header.platform,
header: batch.header, header: batch.header,
items: batch.items ?? [], items: batch.items ?? [],
currency: batch.header.currency,
} }
: null; : null;
@@ -85,7 +86,7 @@ export async function handleIntakeAction(
[batch.id, txnId] [batch.id, txnId]
); );
return { return {
...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, transactionId: txnId }), ...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, currency: ctx.currency, transactionId: txnId }),
replace: true, replace: true,
}; };
} }
+39 -16
View File
@@ -23,7 +23,27 @@ export function transactionLink(merchant: string | null): string {
return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`; return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`;
} }
const money = (n: number | string) => `$${Number(n).toFixed(2)}`; /**
* Slack renders `&`, `<` and `>` as markup in mrkdwn (and in the `text`
* fallback). Merchant names, transaction descriptions and model output are
* none of ours — `Ben & Jerry's` must not arrive as `Ben &amp; Jerry's`, and a
* description containing `<http://…|click>` must not become a link the card
* never meant to show. Applied to interpolated values only; our own literal
* `*bold*` and `:emoji:` stay markup.
*/
export function esc(value: string | null | undefined): string {
return (value ?? "").replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
}
/**
* `$` is only right for AUD. An intake batch carries the currency the receipt
* printed, and the order nudge already renders `USD 15.00` rather than
* pretending a foreign order was dollars.
*/
const money = (n: number | string, currency?: string | null) => {
const amount = Number(n).toFixed(2);
return !currency || currency === "AUD" ? `$${amount}` : `${currency} ${amount}`;
};
const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } }); const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } });
const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({ const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({
type: "button", type: "button",
@@ -40,10 +60,12 @@ export type CardInput = {
orderDate: string | null; orderDate: string | null;
platform: string | null; platform: string | null;
itemCount: number; itemCount: number;
currency?: string | null;
}; };
export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } { export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const what = [c.total !== null ? money(c.total) : null, platformLabel(c.platform) || c.merchant, c.platform && c.merchant ? `(${c.merchant})` : null] const merchant = esc(c.merchant);
const what = [c.total !== null ? money(c.total, c.currency) : null, esc(platformLabel(c.platform)) || merchant, c.platform && merchant ? `(${merchant})` : null]
.filter(Boolean) .filter(Boolean)
.join(" "); .join(" ");
const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`; const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`;
@@ -60,7 +82,7 @@ export function enrichedCard(c: CardInput & { transactionId: number }): { blocks
} }
export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } { export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const text = `Created a ${c.total !== null ? money(c.total) + " " : ""}${c.merchant ?? "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`; const text = `Created a ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant) || "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`;
return { return {
text, text,
blocks: [ blocks: [
@@ -74,27 +96,28 @@ export function createdCard(c: CardInput & { transactionId: number }): { blocks:
} }
export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } { export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } {
const label = platformLabel(c.platform) || ""; const label = esc(platformLabel(c.platform));
const head = `No matching ${label ? label + " " : ""}order for *${c.merchant ?? "this merchant"}* ${c.total !== null ? money(c.total) : ""} on ${c.orderDate ?? "?"}.`; const head = `No matching ${label ? label + " " : ""}order for *${esc(c.merchant) || "this merchant"}* ${c.total !== null ? money(c.total, c.currency) : ""} on ${esc(c.orderDate) || "?"}.`;
const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`; const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`;
const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)]; const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)];
for (const cand of candidates) { for (const cand of candidates) {
blocks.push({ blocks.push({
type: "section", type: "section",
text: { type: "mrkdwn", text: `${cand.transaction_date} · ${cand.description ?? "—"} · ${money(cand.amount)}` }, text: { type: "mrkdwn", text: `${esc(cand.transaction_date)} · ${esc(cand.description) || "—"} · ${money(cand.amount)}` },
accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }), accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }),
}); });
} }
blocks.push({ blocks.push({
type: "actions", type: "actions",
block_id: "intake_create", block_id: "intake_create",
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total) + " " : ""}${c.merchant ?? ""} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")], // A button's `text` is plain_text, never mrkdwn — the raw merchant belongs here.
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total, c.currency) + " " : ""}${c.merchant ?? ""} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
}); });
return { blocks, text }; return { blocks, text };
} }
export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } { export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } {
const text = `${c.merchant ?? "Grocery receipt"} ${c.total !== null ? money(c.total) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " "); const text = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " ");
return { text, blocks: [section(`:shopping_trolley: ${text}`)] }; return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
} }
@@ -103,13 +126,13 @@ export function needsHeaderCard(): { blocks: Block[]; text: string } {
return { text, blocks: [section(`:warning: ${text}`)] }; return { text, blocks: [section(`:warning: ${text}`)] };
} }
export function heldCard(reason: string): { blocks: Block[]; text: string } { export function heldCard(reason: string, headline = "Items don't add up to the total — is a shot missing?"): { blocks: Block[]; text: string } {
const text = `Items don't add up to the total — is a shot missing? (${reason})`; const text = `${headline} (${esc(reason)})`;
return { text, blocks: [section(`:warning: ${text}`)] }; return { text, blocks: [section(`:warning: ${text}`)] };
} }
export function askCard(c: CardInput): { blocks: Block[]; text: string } { export function askCard(c: CardInput): { blocks: Block[]; text: string } {
const text = `${c.merchant ?? "This upload"} ${c.total !== null ? money(c.total) : ""} — groceries or a meal?`.replace(/\s+/g, " "); const text = `${esc(c.merchant) || "This upload"} ${c.total !== null ? money(c.total, c.currency) : ""} — groceries or a meal?`.replace(/\s+/g, " ");
return { return {
text, text,
blocks: [ blocks: [
@@ -126,18 +149,18 @@ export function askCard(c: CardInput): { blocks: Block[]; text: string } {
}; };
} }
export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null }): { blocks: Block[]; text: string } { export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null; currency?: string | null }): { blocks: Block[]; text: string } {
const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${c.batchId}` : "an earlier upload"; const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${esc(c.batchId)}` : "an earlier upload";
const text = `Already handled — ${c.merchant ?? "this order"} ${c.total !== null ? money(c.total) : ""} is ${where}. Nothing was changed.`.replace(/\s+/g, " "); const text = `Already handled — ${esc(c.merchant) || "this order"} ${c.total !== null ? money(c.total, c.currency) : ""} is ${where}. Nothing was changed.`.replace(/\s+/g, " ");
return { text, blocks: [section(`:repeat: ${text}`)] }; return { text, blocks: [section(`:repeat: ${text}`)] };
} }
export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } { export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } {
const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${detail})` : ""}`; const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${esc(detail)})` : ""}`;
return { text, blocks: [section(`:no_entry_sign: ${text}`)] }; return { text, blocks: [section(`:no_entry_sign: ${text}`)] };
} }
export function errorCard(detail: string): { blocks: Block[]; text: string } { export function errorCard(detail: string): { blocks: Block[]; text: string } {
const text = `Couldn't process that upload: ${detail}`; const text = `Couldn't process that upload: ${esc(detail)}`;
return { text, blocks: [section(`:x: ${text}`)] }; return { text, blocks: [section(`:x: ${text}`)] };
} }
+13 -3
View File
@@ -185,9 +185,19 @@ export function liteLLMExtractor(opts: {
}); });
if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`); if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`);
const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] }; const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] };
// A truncated reply can still parse once the JSON is closed, which quietly // Only `stop` means the strict schema was satisfied. A truncated reply can
// drops the tail of a long receipt. Losing lines silently is worse than failing. // still parse once the JSON is closed, which quietly drops the tail of a
if (raw.choices?.[0]?.finish_reason === "length") throw new ExtractionError("The receipt was too long to read in one pass."); // long receipt; a `content_filter` or `tool_calls` stop is a different reply
// shape entirely. Accepting anything but `stop` accepts a reply the schema
// never constrained — losing lines silently is worse than failing.
const finish = raw.choices?.[0]?.finish_reason;
if (finish && finish !== "stop") {
throw new ExtractionError(
finish === "length"
? "The receipt was too long to read in one pass."
: `The model stopped before it finished (${finish}).`
);
}
const content = raw.choices?.[0]?.message?.content; const content = raw.choices?.[0]?.message?.content;
if (!content) throw new ExtractionError("LiteLLM returned no content"); if (!content) throw new ExtractionError("LiteLLM returned no content");
try { try {
+142 -38
View File
@@ -1,5 +1,6 @@
import { randomUUID } from "node:crypto"; import { randomUUID } from "node:crypto";
import { queryRaw, queryRow } from "@/lib/db"; import { queryRaw, queryRow } from "@/lib/db";
import { EXCLUDE_RECONCILED_SOURCE } from "@/lib/analytics-sql";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion"; import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import { import {
classifyInput, classifyInput,
@@ -207,15 +208,31 @@ export async function findDuplicateBatch(
return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null; return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null;
} }
type MatchRow = { transaction_id: number; description: string | null; transaction_date: string; amount: string; merchant: string | null; platform: string | null }; type MatchRow = {
transaction_id: number;
description: string | null;
transaction_date: string;
amount: string;
merchant: string | null;
platform: string | null;
/** The `expense_metadata` row the match resolved through — null when there is none. */
metadata_id: number | null;
/** Does that row already carry a receipt? */
has_line_items: boolean;
};
/** /**
* The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and * The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and
* date EXACT, never widened silently. Platform is required when the source * date EXACT, never widened silently. The merchant compares normalised, with
* implies one; an in-store receipt has none and matches on the other three. * whole-word containment ("Coles" vs "Coles (Wyndham Vale)") — that is
* The merchant compares normalised, with containment ("Coles" vs "Coles * spelling, not widening. A unique hit is a match; two hits are an ambiguity
* (Wyndham Vale)") — that is spelling, not widening. A unique hit is a match; * and go to the candidates card like a miss.
* two hits are an ambiguity and go to the candidates card like a miss. *
* A NULL platform means "no platform", not "any platform". An in-store Coles
* receipt carries none, and `$3 IS NULL OR em.platform = $3` let it match an
* Uber Eats Coles order on the same day for the same money — and then overwrite
* that order's items with the till receipt's. The in-store side matches rows
* with no platform of their own; the delivery side still requires its own.
*/ */
export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> { export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> {
if (!header.merchant) return { match: null, ambiguous: false }; if (!header.merchant) return { match: null, ambiguous: false };
@@ -223,13 +240,14 @@ export async function matchTransaction(header: OrderHeader, platform: string | n
`SELECT DISTINCT ON (t.id) `SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text, t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant, COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform em.platform, em.id AS metadata_id,
COALESCE(jsonb_typeof(em.line_items) = 'array' AND jsonb_array_length(em.line_items) > 0, false) AS has_line_items
FROM transactions t FROM transactions t
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE t.superseded_by_id IS NULL WHERE ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date = $1::date AND t.transaction_date = $1::date
AND t.amount = $2::numeric AND t.amount = $2::numeric
AND ($3::text IS NULL OR em.platform = $3) AND (($3::text IS NULL AND em.platform IS NULL) OR em.platform = $3)
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`, ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date, header.total, platform] [header.order_date, header.total, platform]
); );
@@ -238,21 +256,28 @@ export async function matchTransaction(header: OrderHeader, platform: string | n
return { match: null, ambiguous: hits.length > 1 }; return { match: null, ambiguous: hits.length > 1 };
} }
/** Nearest candidates for the card: same platform and merchant, ±3 days, any amount. */ /**
export async function findCandidates(header: OrderHeader, platform: string | null): Promise<Candidate[]> { * Nearest candidates for the card: same merchant, ±3 days, any amount.
*
* Deliberately NOT filtered by platform. The card is a human choice, not a
* decision — the whole reason it is shown is that the exact key missed, so the
* platform is one of the things that may be wrong. A filter here narrows the
* list at exactly the moment it needs to widen, and the row the user was
* looking for is simply absent with nothing saying why.
*/
export async function findCandidates(header: OrderHeader): Promise<Candidate[]> {
if (!header.merchant) return []; if (!header.merchant) return [];
const rows = await queryRaw<MatchRow>( const rows = await queryRaw<MatchRow>(
`SELECT DISTINCT ON (t.id) `SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text, t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant, COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform em.platform, em.id AS metadata_id, false AS has_line_items
FROM transactions t FROM transactions t
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE t.superseded_by_id IS NULL WHERE ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3 AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3
AND ($2::text IS NULL OR em.platform = $2)
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`, ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date, platform] [header.order_date]
); );
return rows return rows
.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant)) .filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant))
@@ -275,7 +300,7 @@ export type RouteContext = {
platform: string | null; platform: string | null;
header: OrderHeader; header: OrderHeader;
items: StoredLineItem[]; items: StoredLineItem[];
subtotal?: number | null; currency?: string | null;
}; };
const cardInput = (ctx: RouteContext): CardInput => ({ const cardInput = (ctx: RouteContext): CardInput => ({
@@ -285,6 +310,7 @@ const cardInput = (ctx: RouteContext): CardInput => ({
orderDate: ctx.header.order_date, orderDate: ctx.header.order_date,
platform: ctx.platform, platform: ctx.platform,
itemCount: ctx.items.length, itemCount: ctx.items.length,
currency: ctx.currency,
}); });
const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({ const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({
@@ -299,15 +325,33 @@ const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card:
...extra, ...extra,
}); });
/**
* Are these items unverifiable — no line carrying any money at all?
*
* That is `reconcileItems`' `basis: "none"`: pasted text with no prices, or an
* extraction that read the names and none of the figures. The reconciliation
* check passes it, correctly, because there is nothing to check. What must not
* follow is replacing a receipt that IS itemised with one nothing verified.
*/
const unverifiable = (items: StoredLineItem[]) => items.every((i) => i.amount === null);
/** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */ /** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */
export async function routeFood(ctx: RouteContext): Promise<IntakeResult> { export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
const { match } = await matchTransaction(ctx.header, ctx.platform); const { match } = await matchTransaction(ctx.header, ctx.platform);
if (match) { if (match) {
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId }); if (unverifiable(ctx.items) && match.has_line_items) {
await updateBatch(ctx.batchId, { status: "held", items: ctx.items });
const card = heldCard(
"this upload carries no prices, so nothing checked it against the total — and that order already has its items",
"I left the existing items alone"
);
return finish(ctx, "held", "held", card, { transaction_id: match.transaction_id });
}
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId, metadataId: match.metadata_id });
await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items }); await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items });
return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id }); return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
} }
const candidates = await findCandidates(ctx.header, ctx.platform); const candidates = await findCandidates(ctx.header);
await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items }); await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items });
return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates }); return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates });
} }
@@ -330,7 +374,7 @@ export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantr
} catch (e) { } catch (e) {
if (e instanceof PantryDuplicateError) { if (e instanceof PantryDuplicateError) {
await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items }); await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items });
return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), { return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, currency: ctx.currency, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), {
capture_id: e.captureId ?? undefined, capture_id: e.captureId ?? undefined,
}); });
} }
@@ -345,38 +389,98 @@ export async function createFromBatch(batch: BatchRow, kind: "grocery" | "food")
if (!batch.header) throw new Error("batch has no header"); if (!batch.header) throw new Error("batch has no header");
const header = batch.header; const header = batch.header;
const items = batch.items ?? []; const items = batch.items ?? [];
const ctx: RouteContext = {
batchId: batch.id,
source: batch.source,
sharedBy: batch.shared_by,
platform: header.platform,
header,
items,
currency: header.currency,
};
// The card was built minutes ago and the tap arrives now. In between, the
// email/order bridge books orders on its own schedule — so re-run the match
// rather than trusting the "no match" the card was drawn from. Creating a
// second row for an order the ledger already holds is the one outcome this
// whole slice exists to avoid.
const { match } = await matchTransaction(header, header.platform);
if (match) {
await replaceLineItems(match.transaction_id, items, { source: "intake", platform: header.platform, batchId: batch.id, metadataId: match.metadata_id });
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: match.transaction_id, items });
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
}
// A sibling batch of the same order — the PDF after the screenshot — may have
// been resolved since. `findDuplicateBatch` deliberately ignores unresolved
// batches, so two pending cards for one order both reach here; without this
// the second tap books the order twice.
const twin = await findDuplicateBatch(batch.id, batch.file_sha256, header);
if (twin) {
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: twin.transaction_id, capture_id: twin.capture_id });
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: twin.transaction_id, captureId: twin.capture_id, batchId: twin.id });
return finish(ctx, kind, "duplicate", card, {
transaction_id: twin.transaction_id ?? undefined,
capture_id: twin.capture_id ?? undefined,
duplicate_of: { batch_id: twin.id, transaction_id: twin.transaction_id, capture_id: twin.capture_id },
});
}
const ownerId = await ownerForEmail(batch.shared_by); const ownerId = await ownerForEmail(batch.shared_by);
const merchant = header.merchant ?? "Unknown merchant"; const merchant = header.merchant ?? "Unknown merchant";
// Not `orderDescription()`: it always prefixes "Order - ", which is right for
// a delivery receipt and wrong for the in-store till receipt this path also
// handles. Not `resolveCategory()` either — it re-derives grocery-vs-food
// from a merchant pattern, and `kind` is the answer the router already
// reached, remembered merchant and Slack tap included.
const description = header.platform ? `Order - ${merchant}` : merchant; const description = header.platform ? `Order - ${merchant}` : merchant;
const category = kind === "grocery" ? "groceries" : "dining"; const category = kind === "grocery" ? "groceries" : "dining";
// source/source_ref make a second press of the same card a no-op rather than a second row. // `amount` is the figure printed on the receipt, in its own currency. No FX
const existing = await queryRow<{ id: number }>(`SELECT id FROM transactions WHERE source = 'intake' AND source_ref = $1`, [batch.id]); // rate exists at intake, so a foreign order leaves `amount_aud` NULL rather
// than asserting a conversion — the same shape order-ingestion.ts writes.
const currency = header.currency ?? "AUD";
const isAud = currency === "AUD";
// source/source_ref make a second press of the same card a no-op rather than a
// second row — and ON CONFLICT, not SELECT-then-INSERT, so two presses that
// arrive together cannot both find nothing and both insert.
const inserted = await queryRow<{ id: number }>(
`INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref,
foreign_currency_amount, foreign_currency_code
) VALUES ($1::date, $2, $3::numeric, $4::numeric, $5, NULL, $6, $6, 'debit', $7, 'intake', $8, $9::numeric, $10)
ON CONFLICT (source, source_ref) WHERE source IS NOT NULL AND source_ref IS NOT NULL
DO NOTHING
RETURNING id`,
[
header.order_date,
description,
header.total,
isAud ? header.total : null,
category,
merchant,
ownerId,
batch.id,
isAud ? null : header.total,
isAud ? null : currency,
]
);
const txnId = const txnId =
existing?.id ?? inserted?.id ??
( (await queryRow<{ id: number }>(`SELECT id FROM transactions WHERE source = 'intake' AND source_ref = $1`, [batch.id]))!.id;
await queryRow<{ id: number }>(
`INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref
) VALUES ($1::date, $2, $3::numeric, $3::numeric, $4, NULL, $5, $5, 'debit', $6, 'intake', $7)
RETURNING id`,
[header.order_date, description, header.total, category, merchant, ownerId, batch.id]
)
)!.id;
await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id }); await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id });
await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId }); await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId });
const ctx: RouteContext = { batchId: batch.id, source: batch.source, sharedBy: batch.shared_by, platform: header.platform, header, items };
return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId }); return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId });
} }
/** The card for a batch as it stands — what a second press of a resolved card gets back. */ /** The card for a batch as it stands — what a second press of a resolved card gets back. */
export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } { export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } {
const h = batch.header; const h = batch.header;
const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0 }; const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0, currency: h?.currency };
if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id }); if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id }); if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id }); if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id });
return duplicateCard({ merchant: base.merchant, total: base.total, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id }); return duplicateCard({ merchant: base.merchant, total: base.total, currency: base.currency, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id });
} }
// ─── the router ───────────────────────────────────────────────────────────── // ─── the router ─────────────────────────────────────────────────────────────
@@ -438,7 +542,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
const dupBySha = await findDuplicateBatch(batchId, shas, null); const dupBySha = await findDuplicateBatch(batchId, shas, null);
if (dupBySha) { if (dupBySha) {
await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id }); await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id });
return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, { return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, currency: dupBySha.header?.currency, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, {
kind: (dupBySha.kind as IntakeKind) ?? "food", kind: (dupBySha.kind as IntakeKind) ?? "food",
duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id }, duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id },
}); });
@@ -478,7 +582,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
const dup = await findDuplicateBatch(batchId, shas, storedHeader); const dup = await findDuplicateBatch(batchId, shas, storedHeader);
if (dup) { if (dup) {
await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id }); await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id });
return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, { return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, currency: storedHeader.currency, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, {
header, header,
items, items,
duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id }, duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id },
@@ -493,7 +597,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
} }
await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" }); await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" });
const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, subtotal: extracted.subtotal }; const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, currency: storedHeader.currency };
// 7. act. // 7. act.
if (kind === "ask") { if (kind === "ask") {
+32 -8
View File
@@ -43,6 +43,7 @@ export function platformForSource(source: IntakeSource): string | null {
export const PLATFORM_LABEL: Record<string, string> = { export const PLATFORM_LABEL: Record<string, string> = {
ubereats: "Uber Eats", ubereats: "Uber Eats",
doordash: "DoorDash", doordash: "DoorDash",
uber: "Uber",
}; };
export function platformLabel(platform: string | null): string { export function platformLabel(platform: string | null): string {
@@ -51,10 +52,16 @@ export function platformLabel(platform: string | null): string {
/** /**
* A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints * A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints
* `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; substring on * `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; a
* the normalised name covers both without a merchant master-data subsystem. * word-boundary alternation on the normalised name covers both without a
* merchant master-data subsystem.
*
* The boundaries are load-bearing, not tidiness: a bare substring made
* `Bigals Pizza` a grocer, because "b-IGA-ls" contains `iga`. Short names in a
* substring list collide with ordinary words, and the collision reads as a
* correct classification.
*/ */
export const GROCERY_PATTERN = /coles|woolworths|woolies|aldi|iga|foodworks|costco/i; export const GROCERY_PATTERN = /\b(coles|woolworths|woolies|aldi|iga|foodworks|costco)\b/i;
/** /**
* The normalised merchant used for matching and for the remembered-kind table. * The normalised merchant used for matching and for the remembered-kind table.
@@ -219,10 +226,27 @@ export function toLineItems(items: ExtractedItem[]): StoredLineItem[] {
}); });
} }
/** Does the stored merchant name denote the extracted one? Exact key, or one contains the other. */ /** Is `needle` a contiguous run of whole words inside `hay`? */
function containsWordRun(hay: string[], needle: string[]): boolean {
if (!needle.length || needle.length > hay.length) return false;
for (let i = 0; i + needle.length <= hay.length; i++) {
if (needle.every((w, j) => hay[i + j] === w)) return true;
}
return false;
}
/**
* Does the stored merchant name denote the extracted one? One name's words must
* appear as a contiguous run inside the other's — so `Coles` matches
* `Coles (Wyndham Vale)` and `Order - Coles (Wyndham Vale)`, which is spelling,
* not widening.
*
* Whole words, never a raw substring: `merchantKey("Bigals Pizza")` contains
* `iga`, so a pizza shop matched every IGA on the same date and total.
*/
export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean { export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean {
const a = merchantKey(stored); const a = merchantKey(stored).split(" ").filter(Boolean);
const b = merchantKey(extracted); const b = merchantKey(extracted).split(" ").filter(Boolean);
if (!a || !b) return false; if (!a.length || !b.length) return false;
return a === b || a.includes(b) || b.includes(a); return containsWordRun(a, b) || containsWordRun(b, a);
} }
+61 -13
View File
@@ -15,6 +15,12 @@ import { queryRow, queryRaw } from "@/lib/db";
* transaction has no row at all (a manual transaction, a statement line), one * transaction has no row at all (a manual transaction, a statement line), one
* is created so the items have somewhere to live; a transaction is NEVER * is created so the items have somewhere to live; a transaction is NEVER
* created here. * created here.
*
* `opts.metadataId` pins WHICH row when the caller already resolved one. A
* statement transaction can own its own `expense_metadata` row AND be the
* `matched_transaction_id` of a delivery row: the match query and this write
* then re-pick independently and can land on different rows, so the enrichment
* reports success against a row nobody reads.
*/ */
export type LineItemsSource = "intake" | "pantry"; export type LineItemsSource = "intake" | "pantry";
@@ -55,10 +61,35 @@ export type ReplaceResult = {
line_items: LineItemRecord[]; line_items: LineItemRecord[];
}; };
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* The undo baseline, recorded once per batch.
*
* Returning the row that was just read makes a RETRY destroy the thing undo
* exists to restore: pantry re-calls this on every confirm and after a timeout,
* and the second call would hand back its own write as the "previous" state.
* The first snapshot a batch ever takes is the only one worth keeping, so it is
* written under COALESCE and read back rather than trusted.
*/
async function undoBaseline(batchId: string | null | undefined, snapshot: unknown[]): Promise<unknown[]> {
// Pantry passes back whatever batch id it was given; only an intake batch has
// a row to record against, and a non-uuid would fail the cast.
if (!batchId || !UUID.test(batchId)) return snapshot;
const row = await queryRow<{ previous_line_items: unknown }>(
`UPDATE intake_batches
SET previous_line_items = COALESCE(previous_line_items, $2::jsonb)
WHERE id = $1::uuid
RETURNING previous_line_items`,
[batchId, JSON.stringify(snapshot)]
);
return Array.isArray(row?.previous_line_items) ? row.previous_line_items : snapshot;
}
export async function replaceLineItems( export async function replaceLineItems(
transactionId: number, transactionId: number,
lineItems: LineItemRecord[], lineItems: LineItemRecord[],
opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null } opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null; metadataId?: number | null }
): Promise<ReplaceResult> { ): Promise<ReplaceResult> {
const txn = await queryRow<{ const txn = await queryRow<{
id: number; id: number;
@@ -73,25 +104,42 @@ export async function replaceLineItems(
); );
if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`); if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`);
// Prefer the row that IS this transaction over one that merely points at it — // The caller's row wins where it still exists; otherwise prefer the row that
// a statement line can be the `matched_transaction_id` of a receipt row and // IS this transaction over one that merely points at it — a statement line
// carry its own row too, and the items belong on the receipt. // can be the `matched_transaction_id` of a receipt row and carry its own row
const existing = await queryRow<{ id: number; line_items: unknown }>( // too, and the items belong on the receipt.
`SELECT id, line_items FROM expense_metadata const pinned = opts.metadataId
WHERE transaction_id = $1 OR matched_transaction_id = $1 ? await queryRow<{ id: number; line_items: unknown }>(
ORDER BY (transaction_id = $1) DESC, id `SELECT id, line_items FROM expense_metadata
LIMIT 1`, WHERE id = $1 AND (transaction_id = $2 OR matched_transaction_id = $2)`,
[transactionId] [opts.metadataId, transactionId]
); )
: null;
const existing =
pinned ??
(await queryRow<{ id: number; line_items: unknown }>(
`SELECT id, line_items FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- NULLS LAST is load-bearing, and its absence inverted this clause.
-- Postgres orders DESC as NULLS FIRST, and the row that merely POINTS
-- at the transaction has a NULL transaction_id — so the comparison
-- meant to prefer the owned row preferred the other one, silently and
-- in exactly the case it was written for. matchTransaction spells it
-- out; this did not, and the two therefore disagreed.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[transactionId]
));
const json = JSON.stringify(lineItems); const json = JSON.stringify(lineItems);
if (existing) { if (existing) {
const previous = await undoBaseline(opts.batchId, Array.isArray(existing.line_items) ? existing.line_items : []);
await queryRaw(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1`, [existing.id, json]); await queryRaw(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1`, [existing.id, json]);
return { return {
transaction_id: transactionId, transaction_id: transactionId,
metadata_id: existing.id, metadata_id: existing.id,
created_metadata: false, created_metadata: false,
previous_line_items: Array.isArray(existing.line_items) ? existing.line_items : [], previous_line_items: previous,
line_items: lineItems, line_items: lineItems,
}; };
} }
@@ -118,7 +166,7 @@ export async function replaceLineItems(
transaction_id: transactionId, transaction_id: transactionId,
metadata_id: created!.id, metadata_id: created!.id,
created_metadata: true, created_metadata: true,
previous_line_items: [], previous_line_items: await undoBaseline(opts.batchId, []),
line_items: lineItems, line_items: lineItems,
}; };
} }
-7
View File
@@ -12,13 +12,6 @@ export const CUTOVER_DATE = "2026-01-09";
*/ */
export const DEFAULT_OWNER_ID = 1; export const DEFAULT_OWNER_ID = 1;
/** Human labels for the platform a receipt came from. */
export const PLATFORM_LABEL: Record<ParsedOrder["platform"], string> = {
doordash: "DoorDash",
ubereats: "Uber Eats",
uber: "Uber",
};
/** /**
* Transaction description. * Transaction description.
* *