Author SHA1 Message Date
siddharthd 057595f4ac wip(intake): router, enrich endpoint, Slack handlers, migration 0034 — UNTESTED checkpoint
ci / lint-test (push) Successful in 44s
Agent was terminated by the session usage limit before running lint/tests or
writing its report. Files are complete drafts against plan §11 (C1, C2, C5):
src/app/api/intake/, src/app/api/transactions/[id]/line-items/, src/lib/intake-*.ts,
src/lib/line-items.ts, prisma/migrations/0034_intake_batches, fixtures + tests.
Next: npm ci && npx prisma generate && npm run lint && npm test && npm run test:integration.
2026-08-21 12:03:48 +10:00
20 changed files with 111 additions and 945 deletions
-83
View File
@@ -622,89 +622,6 @@ and one tap must not flatten it. It says which case it refused on.
Slack reaches this route through an n8n webhook, not directly — see the Slack reaches this route through an n8n webhook, not directly — see the
smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory. smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory.
### Intake router — one upload, every applicable step (2026-08-21)
`POST /api/intake` (token `INTAKE_TOKEN`, multipart: `source`, `shared_by`,
`batch_id`, 1–6 `file`s) is the finance half of the share-sheet intake; design
and the cross-repo contract are in smarthome `plans/intake-router-slice-plan.md`
§11 (C1, C2, C5 live here; C3/C4 are pantry-app; C6 is n8n + compose).
- **Order of operations is the plan's and is load-bearing:** persist
`intake_batches` row → classify each file → ONE LiteLLM call for the whole
batch (`cloud-budget`, `temperature 0`, strict `json_schema`; PDFs go in as
`unpdf` text, never pixels) → header `(merchant, date, total)` → merchant
kind → duplicate check → reconciliation → match → act. `src/lib/intake-rules.ts`
holds the pure decisions, `intake-router.ts` the orchestration.
- **Match is `(platform, merchant, date, total)`, exact, never widened.** A
miss or an ambiguity returns candidates (same merchant ±3 days) with
`intake_use_candidate` / `intake_create` buttons; a transaction is created
only on the explicit tap, ING-10-style (`statement_id NULL`, `owner_id` from
`shared_by` → `participants.email`, else `DEFAULT_OWNER_ID`;
`source='intake'`, `source_ref=batch_id` so a second press is a no-op —
written as `ON CONFLICT … DO NOTHING`, because a SELECT-then-INSERT loses a
double tap). A non-AUD receipt is booked the way `order-ingestion.ts` books
one: `amount` native, `amount_aud` **NULL**, `foreign_currency_*` set — no FX
rate exists at intake and inventing one is worse than admitting there is none.
- **A NULL platform means "no platform", not "any platform".** An in-store tap
(`coles`, `woolworths`, `other`) matches only rows whose `expense_metadata`
carries no platform of its own. The wildcard reading let a till receipt match
the Uber Eats order for the same shop on the same day and overwrite its items.
The **candidates** query is the opposite and deliberately so: it has no
platform filter at all, because the card is a human choice shown precisely
because the exact key missed — the platform is one of the things that may be
wrong, so the list must widen there, not narrow.
- **The Create tap re-runs the match before it inserts**, and checks for a
resolved sibling batch. The card is minutes old by the time it is pressed and
the email/order bridge books orders on its own schedule; `findDuplicateBatch`
ignores *unresolved* batches by design, so two pending cards for one order
would otherwise each book it.
- **One `expense_metadata` row, agreed on by every query that resolves one.** A
transaction can own a row *and* be the `matched_transaction_id` of another;
`matchTransaction` returns the row it resolved through and `replaceLineItems`
writes to exactly that one. Every reader orders by
`(transaction_id = $1) DESC **NULLS LAST**, id` — the match, the write, the
order-detail GET, `getTransactions`' provenance LATERAL, `merchantForTransaction`
and the Slack details modal. The `NULLS LAST` is load-bearing: Postgres orders
`DESC` as `NULLS FIRST` and the row that merely *points* at the transaction has
a NULL `transaction_id`, so the clause meant to prefer the owned row preferred
the other one. Measured 2026-08-21: 667 rows, 147 pointing-only, **zero**
transactions resolving to more than one row — the ordering changes nothing
today and exists so the readers still agree when multiplicity appears.
- **The create is guarded by a transaction-scoped advisory lock on the
PURCHASE**, not the batch. `ON CONFLICT (source, source_ref)` handles a double
tap on one card; two *different* batches for one purchase have different
`source_ref`s and both clear their match and duplicate checks before either
inserts. The lock has to live inside `withTransaction` (`src/lib/db.ts`):
`queryRaw` is `$queryRawUnsafe` on a **pooled** client, so every call is its
own implicit transaction and `pg_advisory_xact_lock` taken in a statement of
its own is released before the next statement runs. The loser reports
`duplicate` pointing at the winner's transaction — the same answer the
sequential path gives.
- **`replaceLineItems` snapshots the undo baseline BEFORE it writes, on both
paths**, and treats an UPDATE that touched no row as a failure rather than a
success. The metadata INSERT is `ON CONFLICT (transaction_id) DO UPDATE` and
carries the batch's **currency** — a hard-coded `'AUD'` made `OrderDetails`
print `$` on a USD order, because it prefers `expense_metadata.currency`.
- **`intake_batches.previous_line_items` is the undo baseline, written once.**
Returning whatever the write just read makes a pantry retry get its own write
back as the "previous" state, so undo restores the thing it undid. Recorded
under `COALESCE(previous_line_items, …)` and read back.
- **Grocery never writes finance here.** The ORIGINAL files go to pantry
`POST /api/capture` with `intake.finance_target_transaction_id` +
`finance_action: enrich|create`; pantry's confirm calls
`PUT /api/transactions/[id]/line-items` back (C4), which is why that route
accepts `RECEIPT_INGEST_TOKEN` too. Files are kept under `INTAKE_FILE_DIR`
keyed by sha256 so a later "Groceries" tap can still forward them.
- **Reconciliation is against the printed item subtotal, not the total.** A
delivery order's total is items + fees − promotions ($118.94 of Coles lines
became $85.74), so checking lines against the total would hold every
delivery order. Tolerance $1 or 2%; a failure is `held`, nothing written.
- **Names stay verbatim.** Burgertory calls its burgers "Michael" and "Joel";
the prompt forbids canonicalising and the stored shape is the order lane's
(`qty`, `description`, `amount`, `options`) so `order-details.tsx` renders it.
- Statuses beyond the contract list: `ask` (the "Groceries or a meal?" holding
state), `held`, `error` (502 — model or pantry down; the batch row says so).
### The shared loan ### The shared loan
The loan is a **separate ledger**, not a shared expense and not a settlement The loan is a **separate ledger**, not a shared expense and not a settlement
-14
View File
@@ -282,18 +282,6 @@ All routes require authentication via `X-Forwarded-User` header (set by Traefik)
| GET / POST | `/api/tags` | List or create tags | | GET / POST | `/api/tags` | List or create tags |
| PATCH / DELETE | `/api/tags/[id]` | Update or delete a tag | | PATCH / DELETE | `/api/tags/[id]` | Update or delete a tag |
**Machine routes** — shared-secret `x-ingest-token` header instead of `X-Forwarded-User`; each fails closed when its token is unset:
| Method | Route | Token | Description |
|--------|-------|-------|-------------|
| POST / PATCH | `/api/orders/ingest` | `ORDER_INGEST_TOKEN` | n8n order receipts → transactions (see smarthome CLAUDE.md "Order Processor") |
| POST | `/api/receipts/ingest` | `RECEIPT_INGEST_TOKEN` | pantry-app confirmed grocery receipts → one transaction per tender leg (ING-10) |
| POST | `/api/intake` | `INTAKE_TOKEN` | **Intake router** (2026-08-21): multipart `source` (`uber_eats\|doordash\|coles\|woolworths\|other`), `shared_by`, `batch_id` (uuid), 1–6 `file`s (image / PDF / text). Persists `intake_batches`, extracts once via LiteLLM `cloud-budget` (PDF → text via `unpdf`), matches `(platform, merchant, date, total)` exactly, then enriches the matched order's `line_items` (food), forwards the original files to pantry `POST /api/capture` (grocery), or replies with candidates / "Groceries or a meal?" / needs-header / held / duplicate. Returns `{kind, status, transaction_id?, capture_id?, candidates?, slack_blocks, text}` |
| PUT | `/api/transactions/[id]/line-items` | `INTAKE_TOKEN` or `RECEIPT_INGEST_TOKEN` | Idempotent replace of `expense_metadata.line_items` for `transaction_id = :id OR matched_transaction_id = :id`; body `{line_items, source: 'intake'\|'pantry', batch_id?, platform?, replace: true}`; creates the metadata row if absent, never a transaction (404). Returns `{previous_line_items, line_items}` — pantry calls it on every confirm and with `previous_line_items` on undo |
| POST | `/api/slack/interactive` | `ORDER_INGEST_TOKEN` + Slack signature | Order-nudge buttons and the intake taps (`intake_use_candidate`, `intake_create`, `intake_grocery`, `intake_meal`) |
Env for the intake router: `INTAKE_TOKEN`, `LITELLM_BASE_URL` + `LITELLM_KEY` (OpenAI-compatible, alias `cloud-budget`), `PANTRY_BASE_URL` (+ the existing `RECEIPT_INGEST_TOKEN` as pantry's machine auth), optional `INTAKE_FILE_DIR` (original files by sha256, default `/app/data/intake` — a later "Groceries" tap re-sends them) and `FINANCE_PUBLIC_URL` (the card's View link, default `https://finance.bosecamp.com`). Design: smarthome `plans/intake-router-slice-plan.md` §11.
--- ---
## Ingestion Pipeline ## Ingestion Pipeline
@@ -359,8 +347,6 @@ docker exec postgres-personal psql -U personal -d personal \
| `0009_split_payments` | `split_payments` | | `0009_split_payments` | `split_payments` |
| `0010_csv_import_reconcile` | `reconciled_with_id`, CSV import support | | `0010_csv_import_reconcile` | `reconciled_with_id`, CSV import support |
| `0011_trips` | `trips`, `trip_id` on `transaction_overrides` (catch-up — was applied directly) | | `0011_trips` | `trips`, `trip_id` on `transaction_overrides` (catch-up — was applied directly) |
| … | |
| `0034_intake_batches` | `intake_batches` (one row per `/api/intake` upload, written before extraction) and `intake_merchant_kinds` (the remembered "Groceries or a meal?" answer per merchant) |
> `paperless_doc_id` on statements and the `uq_statements_paperless_doc_id` index were added directly (not tracked in a migration file). > `paperless_doc_id` on statements and the `uq_statements_paperless_doc_id` index were added directly (not tracked in a migration file).
> `owner_id` on transactions and `statement_id` made nullable were applied directly (March 2026) to support manual transaction entry without a fake statement. > `owner_id` on transactions and `statement_id` made nullable were applied directly (March 2026) to support manual transaction entry without a fake statement.
@@ -32,20 +32,10 @@ CREATE TABLE IF NOT EXISTS intake_batches (
status text NOT NULL, status text NOT NULL,
transaction_id integer, transaction_id integer,
capture_id integer, capture_id integer,
-- The undo baseline: what `expense_metadata.line_items` held before this
-- batch first replaced it. Recorded here rather than returned from the write,
-- because pantry re-calls the enrich endpoint on every confirm and after a
-- timeout — a caller that hands back whatever it just read gives a retry its
-- OWN write as the "previous" state, and undo then restores the thing it was
-- meant to reverse. Written under COALESCE, so only the first snapshot sticks.
previous_line_items jsonb,
created_at timestamptz NOT NULL DEFAULT now(), created_at timestamptz NOT NULL DEFAULT now(),
resolved_at timestamptz resolved_at timestamptz
); );
-- For a database that already ran an earlier draft of this file.
ALTER TABLE intake_batches ADD COLUMN IF NOT EXISTS previous_line_items jsonb;
COMMENT ON TABLE intake_batches IS COMMENT ON TABLE intake_batches IS
'One row per upload batch through POST /api/intake. Persisted before ' 'One row per upload batch through POST /api/intake. Persisted before '
'extraction so Slack taps can act on it later and re-shares are detected. ' 'extraction so Slack taps can act on it later and re-shares are detected. '
-2
View File
@@ -283,8 +283,6 @@ model intake_batches {
status String status String
transaction_id Int? transaction_id Int?
capture_id Int? capture_id Int?
// The undo baseline, written once per batch (see 0034 and line-items.ts).
previous_line_items Json?
created_at DateTime @default(now()) @db.Timestamptz(6) created_at DateTime @default(now()) @db.Timestamptz(6)
resolved_at DateTime? @db.Timestamptz(6) resolved_at DateTime? @db.Timestamptz(6)
} }
-19
View File
@@ -23,25 +23,6 @@ export function mockDbWithPool(p: Pool) {
const result = await p.query(sql, params); const result = await p.query(sql, params);
return result.rows[0] ?? null; return result.rows[0] ?? null;
}, },
// Mirrors withTransaction: one connection, BEGIN/COMMIT around the callback,
// so pg_advisory_xact_lock actually holds across the statements inside it.
withTransaction: async (fn: (q: unknown) => Promise<unknown>) => {
const client = await p.connect();
try {
await client.query("BEGIN");
const result = await fn({
queryRaw: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows,
queryRow: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows[0] ?? null,
});
await client.query("COMMIT");
return result;
} catch (e) {
await client.query("ROLLBACK");
throw e;
} finally {
client.release();
}
},
prisma: p, prisma: p,
})); }));
} }
+2 -282
View File
@@ -49,7 +49,6 @@ let fileDir: string;
let intakePOST: (req: NextRequest) => Promise<Response>; let intakePOST: (req: NextRequest) => Promise<Response>;
let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>; let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
let slackPOST: (req: NextRequest) => Promise<Response>; let slackPOST: (req: NextRequest) => Promise<Response>;
let orderGET: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
/** What the next LiteLLM call answers, and what the next pantry call answers. */ /** What the next LiteLLM call answers, and what the next pantry call answers. */
let modelReply: ExtractedOrder | Error = burgertory; let modelReply: ExtractedOrder | Error = burgertory;
@@ -105,10 +104,6 @@ async function seedOrder(over: { merchant: string; date: string; amount: number;
} }
return id; return id;
} }
/** What `components/order-details.tsx` would actually render for a transaction. */
const orderPanel = async (txnId: number) =>
(await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null;
const metaRows = async () => (await pool.query(`SELECT id, platform, line_items FROM expense_metadata ORDER BY id`)).rows as { id: number; platform: string | null; line_items: { description: string }[] }[];
const lineItemsOf = async (txnId: number) => const lineItemsOf = async (txnId: number) =>
(await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items; (await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items;
const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0]; const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0];
@@ -132,17 +127,9 @@ beforeAll(async () => {
participantForSlackUser: () => 1, participantForSlackUser: () => 1,
slackUserForParticipant: () => null, slackUserForParticipant: () => null,
})); }));
// The order-detail panel is the surface an enrichment has to appear on, so
// the read is exercised beside the write rather than assumed.
vi.doMock("@/lib/auth", () => ({ getCurrentUser: async () => ({ id: 1, name: "Owner", email: "o@x" }) }));
vi.doMock("@/lib/queries", async (importOriginal) => ({
...(await importOriginal<Record<string, unknown>>()),
canAccessTransactions: async () => true,
}));
({ POST: intakePOST } = await import("../../app/api/intake/route")); ({ POST: intakePOST } = await import("../../app/api/intake/route"));
({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route")); ({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route"));
({ POST: slackPOST } = await import("../../app/api/slack/interactive/route")); ({ POST: slackPOST } = await import("../../app/api/slack/interactive/route"));
({ GET: orderGET } = await import("../../app/api/transactions/[id]/order/route"));
stubFetch(); stubFetch();
}); });
@@ -223,82 +210,6 @@ describe("PUT /api/transactions/[id]/line-items (C2)", () => {
expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" }); expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" });
}); });
it("the undo baseline is recorded ONCE per batch: a retry gets the original, not its own write", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const original = [{ qty: 1, description: "what the order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(original)]);
const id = uuid();
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
const first = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(first.previous_line_items).toEqual(original);
// Pantry re-calls on every confirm and after a timeout. Before, the retry
// was handed its OWN write back, so undo restored the thing it undid.
const retry = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(retry.previous_line_items).toEqual(original);
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual(original);
await put(txn, { line_items: retry.previous_line_items, source: "pantry", batch_id: id, replace: true });
expect(await lineItemsOf(txn)).toEqual(original);
});
it("without a batch id the baseline is still the row as read — nothing else changes", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(first.previous_line_items).toEqual([]);
const second = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(second.previous_line_items).toEqual(items);
});
it("a pinned row that no longer exists falls back to the live one, rather than reporting a phantom write", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const live = (await pool.query(`SELECT id FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].id as number;
const { replaceLineItems } = await import("../../lib/line-items");
const r = await replaceLineItems(txn, items, { source: "intake", metadataId: live + 999 });
expect(r.metadata_id).toBe(live);
expect(await lineItemsOf(txn)).toHaveLength(2);
});
it("two writes racing to create the metadata row both resolve to the same one", async () => {
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const both = await Promise.all([
put(txn, { line_items: items, source: "intake", replace: true }),
put(txn, { line_items: items, source: "intake", replace: true }),
]);
expect(both.map((r) => r.status)).toEqual([200, 200]);
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].n).toBe(1);
});
it("the create path records an undo baseline too, before it writes", async () => {
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const id = uuid();
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
await put(txn, { line_items: items, source: "intake", batch_id: id, currency: "AUD", replace: true });
// [] rather than NULL: there was nothing there, and "nothing" is the state
// undo has to be able to restore.
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual([]);
});
it("the panel reads the row the write landed on, whichever order the rows were inserted in", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// Inserted first: the delivery receipt pointing at the statement line.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
[t, uuid()]
);
// Inserted second: a row the transaction owns. An unordered LIMIT 1 reads
// the first one, the write takes the second, and the enrichment reports
// success against a row nothing renders.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, line_items) VALUES ($1, 'pantry', $2, '[]'::jsonb)`,
[t, uuid()]
);
const written = await (await put(t, { line_items: items, source: "intake", replace: true })).json();
const panel = await orderPanel(t);
expect(panel?.line_items).toHaveLength(2);
expect((await metaRows()).find((r) => r.id === written.metadata_id)!.line_items).toHaveLength(2);
});
it("finds the row through matched_transaction_id — a card-settled order has only that", async () => { it("finds the row through matched_transaction_id — a card-settled order has only that", async () => {
const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false }); const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
await pool.query( await pool.query(
@@ -374,82 +285,10 @@ describe("POST /api/intake — food (enrich vs candidates)", () => {
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1); expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
}); });
it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match — but it IS offered", async () => { it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match", async () => {
const doordash = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" }); await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" });
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json(); const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("candidates"); expect(body.status).toBe("candidates");
// The candidates card is a human choice, not a decision. The platform is one
// of the things that may be wrong, so it must not narrow the list — a
// platform-filtered card showed nothing at all here.
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([doordash]);
});
it("a NULL platform is 'no platform', not a wildcard — an in-store share does not overwrite the delivery order", async () => {
await pool.query(`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ('burgertory werribee', 'food')`);
const delivery = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "ubereats" });
const already = [{ qty: 1, description: "what the delivery order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [delivery, JSON.stringify(already)]);
// `other` implies no platform; the delivery row carries one, so it is not
// the match — it is only a candidate.
const body = await (await intakePOST(multipart({ source: "other" }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([delivery]);
expect(await lineItemsOf(delivery)).toEqual(already);
});
it("a reconciled manual row is neither matched nor offered — EXCLUDE_RECONCILED_SOURCE, as everywhere else", async () => {
const live = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const reconciled = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, reconciled]);
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, near]);
// Two rows at the exact key would have read as an ambiguity; one of them is
// the same spend counted twice.
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: live });
});
it("the write lands on the row the MATCH resolved through, not on whichever row sorts first", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// The statement line owns a receipt row of its own (a pantry confirm) …
const grocery = [{ qty: 1, description: "a grocery shop", amount: 22.4 }];
const own = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, merchant_normalized, line_items) VALUES ($1, 'pantry', $2, 'Woolworths', $3::jsonb) RETURNING id`,
[t, uuid(), JSON.stringify(grocery)]
)).rows[0].id as number;
// … and is also the card leg of the Uber Eats order being enriched.
const delivery = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, merchant_normalized, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, 'Burgertory (Werribee)', '[]'::jsonb) RETURNING id`,
[t, uuid()]
)).rows[0].id as number;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: t });
const rows = await metaRows();
expect(rows.find((r) => r.id === delivery)!.line_items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
expect(rows.find((r) => r.id === own)!.line_items).toEqual(grocery);
});
it("an upload with no prices never replaces a receipt that already has items", async () => {
const priceless = { ...burgertory, subtotal: null, total: 22.4, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
modelReply = priceless;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "held", status: "held" });
expect(body.text).toMatch(/left the existing items alone/);
expect(await lineItemsOf(txn)).toEqual(already);
// …but an EMPTY receipt is exactly what this slice exists to fill.
const empty = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-20", amount: 22.4 });
modelReply = { ...priceless, order_date: "2026-08-20" };
const second = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(second).toMatchObject({ status: "enriched", transaction_id: empty });
expect((await lineItemsOf(empty)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
}); });
it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => { it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => {
@@ -552,19 +391,6 @@ describe("POST /api/intake — grocery → pantry", () => {
expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn }); expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
}); });
it("an in-store tap does not hand pantry the delivery order as its finance target", async () => {
modelReply = coles;
// The Uber Eats Coles order the email pipeline already booked.
const delivery = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: "ubereats" });
// The same shop, same day, same money — but shared from the Coles app, so
// it carries no platform and is a different purchase.
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown> };
expect(pantryCall.intake).toMatchObject({ finance_action: "create", finance_target_transaction_id: null, platform: null });
expect(await lineItemsOf(delivery)).toEqual([]);
});
it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => { it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => {
modelReply = coles; modelReply = coles;
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json(); const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
@@ -631,112 +457,6 @@ describe("Slack taps (C5)", () => {
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1); expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
}); });
it("intake_create re-runs the match: an order booked between the card and the tap is enriched, not duplicated", async () => {
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
// The email/order bridge books the same purchase while the card sits in
// Slack. The card's "no match" is minutes stale by the time it is pressed.
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
expect((await lineItemsOf(txn)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
});
it("two pending cards for one order book it ONCE", async () => {
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
expect((await batch(a)).status).toBe("candidates");
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: a }));
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect((await batch(b)).transaction_id).toBe((await batch(a)).transaction_id);
});
it("a sibling already resolved to the pantry board makes Create a duplicate, not a second row", async () => {
// Two shares of one receipt, resolved out of order: nothing the second card
// can match on, because the first one never produced a transaction.
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "other", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
await intakePOST(multipart({ source: "other", batch_id: a }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
expect((await batch(a)).status).toBe("ask");
await slackPOST(slackPress("intake_grocery", { batch_id: a }));
expect((await batch(a)).status).toBe("forwarded_to_pantry");
await slackPOST(slackPress("intake_meal", { batch_id: b }));
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
expect(await batch(b)).toMatchObject({ status: "duplicate", capture_id: 901 });
expect(lastSlackUpdate().text).toMatch(/^Already handled/);
});
it("two taps that land together still book one transaction, and neither reports an error", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: id })), slackPOST(slackPress("intake_create", { batch_id: id }))]);
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
const texts = calls.filter((c) => c.url.includes("hooks.slack.com")).map((c) => (c.body as { text: string }).text);
expect(texts.every((t) => !/Couldn't process/.test(t))).toBe(true);
});
it("a foreign receipt is booked in its own currency, with no invented AUD figure", async () => {
modelReply = { ...burgertory, currency: "USD" };
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
const t = (await pool.query(`SELECT amount::text, amount_aud, foreign_currency_amount::text, foreign_currency_code FROM transactions`)).rows[0];
expect(t).toEqual({ amount: "22.40", amount_aud: null, foreign_currency_amount: "22.40", foreign_currency_code: "USD" });
expect(lastSlackUpdate().text).toMatch(/USD 22\.40/);
// OrderDetails prefers expense_metadata.currency, so a hard-coded 'AUD'
// here printed "$" on a USD order's receipt panel.
expect((await pool.query(`SELECT currency FROM expense_metadata`)).rows[0].currency).toBe("USD");
});
it("the hold applies to the Create tap's re-match, not only to the upload path", async () => {
modelReply = { ...burgertory, subtotal: null, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
// Booked WITH its items between the card and the tap. The re-match walked
// past the guard routeFood applies, so the same upload destroyed the same
// receipt depending on which button reached it.
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect(await lineItemsOf(txn)).toEqual(already);
expect(await batch(id)).toMatchObject({ status: "held" });
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
});
it("two DIFFERENT batches for one purchase, tapped together, book it once", async () => {
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
// Their source_refs differ, so ON CONFLICT cannot save this; both taps
// clear their match and duplicate checks before either inserts.
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: a })), slackPOST(slackPress("intake_create", { batch_id: b }))]);
const rows = (await pool.query(`SELECT id FROM transactions`)).rows;
expect(rows).toHaveLength(1);
expect((await batch(a)).transaction_id).toBe(rows[0].id);
expect((await batch(b)).transaction_id).toBe(rows[0].id);
});
it("unknown shared_by falls back to the default owner — never a NULL owner", async () => { it("unknown shared_by falls back to the default owner — never a NULL owner", async () => {
const id = uuid(); const id = uuid();
await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf])); await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf]));
+2 -78
View File
@@ -3,7 +3,6 @@ import { readFileSync } from "fs";
import { resolve } from "path"; import { resolve } from "path";
import { import {
classifyInput, classifyInput,
GROCERY_PATTERN,
merchantKey, merchantKey,
merchantMatches, merchantMatches,
platformForSource, platformForSource,
@@ -13,8 +12,8 @@ import {
validateHeader, validateHeader,
type ExtractedOrder, type ExtractedOrder,
} from "../../lib/intake-rules"; } from "../../lib/intake-rules";
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor } from "../../lib/intake-extract"; import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT } from "../../lib/intake-extract";
import { askCard, candidatesCard, enrichedCard, type Block } from "../../lib/intake-blocks"; import { askCard, candidatesCard, enrichedCard } from "../../lib/intake-blocks";
/** /**
* The intake router's pure rules, against the strings real receipts print. * The intake router's pure rules, against the strings real receipts print.
@@ -86,17 +85,6 @@ describe("merchant pattern — grocery | food | ask", () => {
expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false); expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false);
expect(merchantMatches(null, "Coles")).toBe(false); expect(merchantMatches(null, "Coles")).toBe(false);
}); });
it("whole words, not substrings: `Bigals Pizza` contains `iga` and is neither a grocer nor an IGA", () => {
expect(GROCERY_PATTERN.test(merchantKey("Bigals Pizza"))).toBe(false);
expect(resolveMerchantKind("uber_eats", "Bigals Pizza")).toBe("food");
expect(merchantMatches("Bigals Pizza", "IGA")).toBe(false);
// and the grocers a boundary must not cost us
expect(resolveMerchantKind("uber_eats", "IGA Wyndham Vale")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Woolworths Group Limited")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
expect(merchantMatches("IGA Wyndham Vale", "IGA")).toBe(true);
});
}); });
describe("validateHeader — nothing is written from a half-read upload", () => { describe("validateHeader — nothing is written from a half-read upload", () => {
@@ -233,68 +221,4 @@ describe("cards carry the C5 action ids and JSON values", () => {
expect(ids).toContain('"action_id":"intake_grocery"'); expect(ids).toContain('"action_id":"intake_grocery"');
expect(ids).toContain('"action_id":"intake_meal"'); expect(ids).toContain('"action_id":"intake_meal"');
}); });
it("every value the card did not write is escaped, plain_text included", () => {
const hostile = { ...c, merchant: "Ben & Jerry's <fake>" };
const { text, blocks } = candidatesCard(hostile, [
{ transaction_id: 42, description: "SQ *BEN & JERRY <AU>", transaction_date: "2026-08-17", amount: "19.90" },
]);
const sections = JSON.stringify(blocks.filter((b) => (b as Block).type === "section"));
expect(sections).toContain("Ben &amp; Jerry's &lt;fake&gt;");
expect(sections).toContain("SQ *BEN &amp; JERRY &lt;AU&gt;");
expect(text).toContain("Ben &amp; Jerry's &lt;fake&gt;");
// plain_text disables formatting, not entity decoding — &, < and > are
// control characters everywhere in a Slack payload.
const whole = JSON.stringify(blocks);
expect(whole).not.toContain("<fake>");
expect(whole).not.toContain("<AU>");
});
it("the currency is model output too, so it is escaped like any other", () => {
// parseExtraction accepts any string for `currency`; unescaped, `<@U123>`
// would be a live mention on every card the batch produces.
const { text } = enrichedCard({ ...c, currency: "<@U123>", transactionId: 7 });
expect(text).toContain("&lt;@U123&gt; 22.40");
expect(text).not.toContain("<@U123>");
});
it("a foreign receipt is not printed as dollars — header and candidate rows alike", () => {
expect(enrichedCard({ ...c, currency: "USD", transactionId: 7 }).text).toBe(
"Added 2 items to your USD 22.40 Uber Eats (Burgertory (Werribee)) order ✓"
);
expect(enrichedCard({ ...c, currency: "AUD", transactionId: 7 }).text).toContain("$22.40");
expect(enrichedCard({ ...c, transactionId: 7 }).text).toContain("$22.40");
// A candidate row is a ledger transaction and carries its own currency; it
// used to print "$" beneath a header reading "USD 22.40".
const { blocks } = candidatesCard({ ...c, currency: "USD" }, [
{ transaction_id: 42, description: "Order - Burgertory", transaction_date: "2026-08-17", amount: "19.90", currency: "USD" },
]);
expect(JSON.stringify(blocks)).toContain("USD 19.90");
});
});
describe("liteLLMExtractor — a finish_reason that is present must be `stop`", () => {
const good = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD", items: [], adjustments: [] };
const extractor = (finish_reason?: string) =>
liteLLMExtractor({
baseUrl: "http://litellm.test",
key: "k",
fetchImpl: (async () =>
Response.json({
choices: [{ message: { content: JSON.stringify(good) }, ...(finish_reason ? { finish_reason } : {}) }],
})) as unknown as typeof fetch,
});
it("accepts `stop`, and — on purpose — a reply that reports no reason at all", async () => {
expect((await extractor("stop")([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
expect((await extractor()([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
});
it("rejects any other reason, naming it — the strict schema only held for a reply that finished", async () => {
// An absent reason is NOT rejected: see the comment in intake-extract.ts.
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(ExtractionError);
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/too long/);
await expect(extractor("content_filter")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/content_filter/);
await expect(extractor("tool_calls")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/tool_calls/);
});
}); });
-10
View File
@@ -234,16 +234,6 @@ async function buildDetailsModal(transactionId: number, participantId: number) {
`SELECT merchant_normalized AS merchant, line_items `SELECT merchant_normalized AS merchant, line_items
FROM expense_metadata FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1 WHERE transaction_id = $1 OR matched_transaction_id = $1
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`, LIMIT 1`,
[transactionId] [transactionId]
); );
@@ -34,14 +34,6 @@ export async function GET(
-- through matched_transaction_id. Both directions have to resolve or the -- through matched_transaction_id. Both directions have to resolve or the
-- detail is missing on exactly the orders that were paid by card. -- detail is missing on exactly the orders that were paid by card.
WHERE transaction_id = $1 OR matched_transaction_id = $1 WHERE transaction_id = $1 OR matched_transaction_id = $1
-- The same order as replaceLineItems in line-items.ts and matchTransaction
-- in intake-router.ts. Both directions can resolve at once — a statement
-- line owning its own row while a delivery receipt points at it — and an
-- unordered LIMIT 1 then reads a different row from the one an enrichment
-- wrote, so a PUT that reported success showed nothing on the screen.
-- NULLS LAST because DESC is NULLS FIRST and the pointing row's
-- transaction_id is NULL.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`, LIMIT 1`,
[Number(id)] [Number(id)]
); );
+7 -4
View File
@@ -13,9 +13,12 @@ import {
type OrderReceipt, type OrderReceipt,
type OrderRating, type OrderRating,
} from "@/lib/hooks"; } from "@/lib/hooks";
// One platform vocabulary, shared with the intake cards — three copies had
// already drifted apart (the intake one was missing `uber`). const PLATFORM_LABEL: Record<string, string> = {
import { platformLabel } from "@/lib/intake-rules"; doordash: "DoorDash",
ubereats: "Uber Eats",
uber: "Uber",
};
/** /**
* Who records verdicts. A two-person household with one primary user: the * Who records verdicts. A two-person household with one primary user: the
@@ -96,7 +99,7 @@ export function OrderDetails({
<p className="text-xs text-zinc-500"> <p className="text-xs text-zinc-500">
Order details Order details
{receipt.platform && ( {receipt.platform && (
<span className="ml-1.5 text-zinc-400">{platformLabel(receipt.platform)}</span> <span className="ml-1.5 text-zinc-400">{PLATFORM_LABEL[receipt.platform] ?? receipt.platform}</span>
)} )}
</p> </p>
{receipt.card_last4 && ( {receipt.card_last4 && (
-30
View File
@@ -20,33 +20,3 @@ export async function queryRow<T>(sql: string, params: unknown[] = []): Promise<
const rows = await prisma.$queryRawUnsafe<T[]>(sql, ...params); const rows = await prisma.$queryRawUnsafe<T[]>(sql, ...params);
return rows.length > 0 ? rows[0] : null; return rows.length > 0 ? rows[0] : null;
} }
/** The same two calls, scoped to one connection inside one transaction. */
export type Queryable = {
queryRaw<T>(sql: string, params?: unknown[]): Promise<T[]>;
queryRow<T>(sql: string, params?: unknown[]): Promise<T | null>;
};
/**
* Run several statements as one transaction, on one connection.
*
* The module-level `queryRaw`/`queryRow` are `$queryRawUnsafe` on a POOLED
* client, so each call is its own implicit transaction on whichever connection
* the pool hands out. That makes a check-then-write pair non-atomic, and it
* makes `pg_advisory_xact_lock` useless outside this helper — the lock is
* released the instant the statement that took it commits, which is before the
* next statement runs. Anything that needs a lock to still be held on its next
* query has to be in here.
*/
export async function withTransaction<T>(fn: (q: Queryable) => Promise<T>): Promise<T> {
return prisma.$transaction(async (tx) => {
const q: Queryable = {
queryRaw: <R>(sql: string, params: unknown[] = []) => tx.$queryRawUnsafe<R[]>(sql, ...params),
queryRow: async <R>(sql: string, params: unknown[] = []) => {
const rows = await tx.$queryRawUnsafe<R[]>(sql, ...params);
return rows.length > 0 ? rows[0] : null;
},
};
return fn(q);
});
}
+2 -3
View File
@@ -58,7 +58,6 @@ const ctxFor = (batch: BatchRow): RouteContext | null =>
platform: batch.header.platform, platform: batch.header.platform,
header: batch.header, header: batch.header,
items: batch.items ?? [], items: batch.items ?? [],
currency: batch.header.currency,
} }
: null; : null;
@@ -80,13 +79,13 @@ export async function handleIntakeAction(
if (actionId === "intake_use_candidate") { if (actionId === "intake_use_candidate") {
if (!Number.isInteger(value.transaction_id)) return note("That button carried no transaction id."); if (!Number.isInteger(value.transaction_id)) return note("That button carried no transaction id.");
const txnId = value.transaction_id as number; const txnId = value.transaction_id as number;
await replaceLineItems(txnId, ctx.items, { source: "intake", platform: ctx.platform, batchId: batch.id, currency: ctx.currency }); await replaceLineItems(txnId, ctx.items, { source: "intake", platform: ctx.platform, batchId: batch.id });
await queryRaw( await queryRaw(
`UPDATE intake_batches SET status = 'enriched', kind = COALESCE(kind, 'food'), transaction_id = $2, resolved_at = now() WHERE id = $1::uuid`, `UPDATE intake_batches SET status = 'enriched', kind = COALESCE(kind, 'food'), transaction_id = $2, resolved_at = now() WHERE id = $1::uuid`,
[batch.id, txnId] [batch.id, txnId]
); );
return { return {
...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, currency: ctx.currency, transactionId: txnId }), ...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, transactionId: txnId }),
replace: true, replace: true,
}; };
} }
+17 -49
View File
@@ -12,7 +12,7 @@ import { platformLabel } from "@/lib/intake-rules";
export type Block = Record<string, unknown>; export type Block = Record<string, unknown>;
export type Candidate = { transaction_id: number; description: string | null; transaction_date: string; amount: string | number; currency?: string | null }; export type Candidate = { transaction_id: number; description: string | null; transaction_date: string; amount: string | number };
export function financePublicUrl(): string { export function financePublicUrl(): string {
return (process.env.FINANCE_PUBLIC_URL || "https://finance.bosecamp.com").replace(/\/$/, ""); return (process.env.FINANCE_PUBLIC_URL || "https://finance.bosecamp.com").replace(/\/$/, "");
@@ -23,29 +23,7 @@ export function transactionLink(merchant: string | null): string {
return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`; return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`;
} }
/** const money = (n: number | string) => `$${Number(n).toFixed(2)}`;
* Slack renders `&`, `<` and `>` as markup in mrkdwn (and in the `text`
* fallback). Merchant names, transaction descriptions and model output are
* none of ours — `Ben & Jerry's` must not arrive as `Ben &amp; Jerry's`, and a
* description containing `<http://…|click>` must not become a link the card
* never meant to show. Applied to interpolated values only; our own literal
* `*bold*` and `:emoji:` stay markup.
*/
export function esc(value: string | null | undefined): string {
return (value ?? "").replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
}
/**
* `$` is only right for AUD. An intake batch carries the currency the receipt
* printed, and the order nudge already renders `USD 15.00` rather than
* pretending a foreign order was dollars.
*/
const money = (n: number | string, currency?: string | null) => {
const amount = Number(n).toFixed(2);
// The currency comes off the model (`parseExtraction` accepts any string), so
// an unescaped `<@U123>` here would be a mention on every card.
return !currency || currency === "AUD" ? `$${amount}` : `${esc(currency)} ${amount}`;
};
const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } }); const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } });
const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({ const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({
type: "button", type: "button",
@@ -62,12 +40,10 @@ export type CardInput = {
orderDate: string | null; orderDate: string | null;
platform: string | null; platform: string | null;
itemCount: number; itemCount: number;
currency?: string | null;
}; };
export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } { export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const merchant = esc(c.merchant); const what = [c.total !== null ? money(c.total) : null, platformLabel(c.platform) || c.merchant, c.platform && c.merchant ? `(${c.merchant})` : null]
const what = [c.total !== null ? money(c.total, c.currency) : null, esc(platformLabel(c.platform)) || merchant, c.platform && merchant ? `(${merchant})` : null]
.filter(Boolean) .filter(Boolean)
.join(" "); .join(" ");
const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`; const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`;
@@ -84,7 +60,7 @@ export function enrichedCard(c: CardInput & { transactionId: number }): { blocks
} }
export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } { export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const text = `Created a ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant) || "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`; const text = `Created a ${c.total !== null ? money(c.total) + " " : ""}${c.merchant ?? "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`;
return { return {
text, text,
blocks: [ blocks: [
@@ -98,30 +74,27 @@ export function createdCard(c: CardInput & { transactionId: number }): { blocks:
} }
export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } { export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } {
const label = esc(platformLabel(c.platform)); const label = platformLabel(c.platform) || "";
const head = `No matching ${label ? label + " " : ""}order for *${esc(c.merchant) || "this merchant"}* ${c.total !== null ? money(c.total, c.currency) : ""} on ${esc(c.orderDate) || "?"}.`; const head = `No matching ${label ? label + " " : ""}order for *${c.merchant ?? "this merchant"}* ${c.total !== null ? money(c.total) : ""} on ${c.orderDate ?? "?"}.`;
const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`; const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`;
const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)]; const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)];
for (const cand of candidates) { for (const cand of candidates) {
blocks.push({ blocks.push({
type: "section", type: "section",
text: { type: "mrkdwn", text: `${esc(cand.transaction_date)} · ${esc(cand.description) || "—"} · ${money(cand.amount, cand.currency)}` }, text: { type: "mrkdwn", text: `${cand.transaction_date} · ${cand.description ?? "—"} · ${money(cand.amount)}` },
accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }), accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }),
}); });
} }
blocks.push({ blocks.push({
type: "actions", type: "actions",
block_id: "intake_create", block_id: "intake_create",
// plain_text disables *formatting*, not entity decoding: `&`, `<` and `>` elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total) + " " : ""}${c.merchant ?? ""} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
// are control characters everywhere in a Slack payload, so the label is
// escaped like any other value we did not write.
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant)} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
}); });
return { blocks, text }; return { blocks, text };
} }
export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } { export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } {
const text = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " "); const text = `${c.merchant ?? "Grocery receipt"} ${c.total !== null ? money(c.total) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " ");
return { text, blocks: [section(`:shopping_trolley: ${text}`)] }; return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
} }
@@ -130,13 +103,13 @@ export function needsHeaderCard(): { blocks: Block[]; text: string } {
return { text, blocks: [section(`:warning: ${text}`)] }; return { text, blocks: [section(`:warning: ${text}`)] };
} }
export function heldCard(reason: string, headline = "Items don't add up to the total — is a shot missing?"): { blocks: Block[]; text: string } { export function heldCard(reason: string): { blocks: Block[]; text: string } {
const text = `${headline} (${esc(reason)})`; const text = `Items don't add up to the total — is a shot missing? (${reason})`;
return { text, blocks: [section(`:warning: ${text}`)] }; return { text, blocks: [section(`:warning: ${text}`)] };
} }
export function askCard(c: CardInput): { blocks: Block[]; text: string } { export function askCard(c: CardInput): { blocks: Block[]; text: string } {
const text = `${esc(c.merchant) || "This upload"} ${c.total !== null ? money(c.total, c.currency) : ""} — groceries or a meal?`.replace(/\s+/g, " "); const text = `${c.merchant ?? "This upload"} ${c.total !== null ? money(c.total) : ""} — groceries or a meal?`.replace(/\s+/g, " ");
return { return {
text, text,
blocks: [ blocks: [
@@ -153,23 +126,18 @@ export function askCard(c: CardInput): { blocks: Block[]; text: string } {
}; };
} }
export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null; currency?: string | null }): { blocks: Block[]; text: string } { export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null }): { blocks: Block[]; text: string } {
const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${esc(c.batchId)}` : "an earlier upload"; const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${c.batchId}` : "an earlier upload";
// The header (merchant, date, total) IS the identity of a purchase here, so a const text = `Already handled — ${c.merchant ?? "this order"} ${c.total !== null ? money(c.total) : ""} is ${where}. Nothing was changed.`.replace(/\s+/g, " ");
// genuine SECOND identical purchase — two of the same coffee, one day, one shop
// — reads as a re-share and is refused. That is the right default (a re-share is
// far commoner than a repeat) but it is a dead end without this sentence: the
// card must say what to do instead, or the ledger quietly loses the second one.
const text = `Already handled — ${esc(c.merchant) || "this order"} ${c.total !== null ? money(c.total, c.currency) : ""} is ${where}. Nothing was changed — if this really is a second, separate purchase, add it in the app.`.replace(/\s+/g, " ");
return { text, blocks: [section(`:repeat: ${text}`)] }; return { text, blocks: [section(`:repeat: ${text}`)] };
} }
export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } { export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } {
const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${esc(detail)})` : ""}`; const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${detail})` : ""}`;
return { text, blocks: [section(`:no_entry_sign: ${text}`)] }; return { text, blocks: [section(`:no_entry_sign: ${text}`)] };
} }
export function errorCard(detail: string): { blocks: Block[]; text: string } { export function errorCard(detail: string): { blocks: Block[]; text: string } {
const text = `Couldn't process that upload: ${esc(detail)}`; const text = `Couldn't process that upload: ${detail}`;
return { text, blocks: [section(`:x: ${text}`)] }; return { text, blocks: [section(`:x: ${text}`)] };
} }
+3 -20
View File
@@ -185,26 +185,9 @@ export function liteLLMExtractor(opts: {
}); });
if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`); if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`);
const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] }; const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] };
// The rule is: a finish_reason that is PRESENT must be `stop`. An absent // A truncated reply can still parse once the JSON is closed, which quietly
// one is accepted, deliberately — providers differ on whether they send the // drops the tail of a long receipt. Losing lines silently is worse than failing.
// field, and failing every call on a provider that omits it would be a if (raw.choices?.[0]?.finish_reason === "length") throw new ExtractionError("The receipt was too long to read in one pass.");
// certain outage in exchange for a case that also requires the model to
// emit syntactically complete JSON. The harm a truncated reply does — a
// partial item list replacing a good one — is closed on the write side
// instead, by the guard in `routeFood`/`createFromBatch`.
//
// `length` is the common one: a truncated reply still parses once the JSON
// happens to close, quietly dropping the tail of a long receipt. Any other
// reason (`content_filter`, `tool_calls`) is a different reply shape that
// the strict schema never constrained.
const finish = raw.choices?.[0]?.finish_reason;
if (finish && finish !== "stop") {
throw new ExtractionError(
finish === "length"
? "The receipt was too long to read in one pass."
: `The model stopped before it finished (${finish}).`
);
}
const content = raw.choices?.[0]?.message?.content; const content = raw.choices?.[0]?.message?.content;
if (!content) throw new ExtractionError("LiteLLM returned no content"); if (!content) throw new ExtractionError("LiteLLM returned no content");
try { try {
+39 -181
View File
@@ -1,6 +1,5 @@
import { randomUUID } from "node:crypto"; import { randomUUID } from "node:crypto";
import { queryRaw, queryRow, withTransaction } from "@/lib/db"; import { queryRaw, queryRow } from "@/lib/db";
import { EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY } from "@/lib/analytics-sql";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion"; import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import { import {
classifyInput, classifyInput,
@@ -208,33 +207,15 @@ export async function findDuplicateBatch(
return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null; return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null;
} }
type MatchRow = { type MatchRow = { transaction_id: number; description: string | null; transaction_date: string; amount: string; merchant: string | null; platform: string | null };
transaction_id: number;
description: string | null;
transaction_date: string;
amount: string;
merchant: string | null;
platform: string | null;
/** What `amount` is denominated in — a candidate row is not always dollars. */
currency: string;
/** The `expense_metadata` row the match resolved through — null when there is none. */
metadata_id: number | null;
/** Does that row already carry a receipt? */
has_line_items: boolean;
};
/** /**
* The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and * The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and
* date EXACT, never widened silently. The merchant compares normalised, with * date EXACT, never widened silently. Platform is required when the source
* whole-word containment ("Coles" vs "Coles (Wyndham Vale)") — that is * implies one; an in-store receipt has none and matches on the other three.
* spelling, not widening. A unique hit is a match; two hits are an ambiguity * The merchant compares normalised, with containment ("Coles" vs "Coles
* and go to the candidates card like a miss. * (Wyndham Vale)") — that is spelling, not widening. A unique hit is a match;
* * two hits are an ambiguity and go to the candidates card like a miss.
* A NULL platform means "no platform", not "any platform". An in-store Coles
* receipt carries none, and `$3 IS NULL OR em.platform = $3` let it match an
* Uber Eats Coles order on the same day for the same money — and then overwrite
* that order's items with the till receipt's. The in-store side matches rows
* with no platform of their own; the delivery side still requires its own.
*/ */
export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> { export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> {
if (!header.merchant) return { match: null, ambiguous: false }; if (!header.merchant) return { match: null, ambiguous: false };
@@ -242,15 +223,13 @@ export async function matchTransaction(header: OrderHeader, platform: string | n
`SELECT DISTINCT ON (t.id) `SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text, t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant, COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency, em.platform
COALESCE(jsonb_typeof(em.line_items) = 'array' AND jsonb_array_length(em.line_items) > 0, false) AS has_line_items
FROM transactions t FROM transactions t
LEFT JOIN statements s ON s.id = t.statement_id
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE ${EXCLUDE_RECONCILED_SOURCE} WHERE t.superseded_by_id IS NULL
AND t.transaction_date = $1::date AND t.transaction_date = $1::date
AND t.amount = $2::numeric AND t.amount = $2::numeric
AND (($3::text IS NULL AND em.platform IS NULL) OR em.platform = $3) AND ($3::text IS NULL OR em.platform = $3)
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`, ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date, header.total, platform] [header.order_date, header.total, platform]
); );
@@ -259,29 +238,21 @@ export async function matchTransaction(header: OrderHeader, platform: string | n
return { match: null, ambiguous: hits.length > 1 }; return { match: null, ambiguous: hits.length > 1 };
} }
/** /** Nearest candidates for the card: same platform and merchant, ±3 days, any amount. */
* Nearest candidates for the card: same merchant, ±3 days, any amount. export async function findCandidates(header: OrderHeader, platform: string | null): Promise<Candidate[]> {
*
* Deliberately NOT filtered by platform. The card is a human choice, not a
* decision — the whole reason it is shown is that the exact key missed, so the
* platform is one of the things that may be wrong. A filter here narrows the
* list at exactly the moment it needs to widen, and the row the user was
* looking for is simply absent with nothing saying why.
*/
export async function findCandidates(header: OrderHeader): Promise<Candidate[]> {
if (!header.merchant) return []; if (!header.merchant) return [];
const rows = await queryRaw<MatchRow>( const rows = await queryRaw<MatchRow>(
`SELECT DISTINCT ON (t.id) `SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text, t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant, COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency, false AS has_line_items em.platform
FROM transactions t FROM transactions t
LEFT JOIN statements s ON s.id = t.statement_id
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE ${EXCLUDE_RECONCILED_SOURCE} WHERE t.superseded_by_id IS NULL
AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3 AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3
AND ($2::text IS NULL OR em.platform = $2)
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`, ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date] [header.order_date, platform]
); );
return rows return rows
.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant)) .filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant))
@@ -292,7 +263,7 @@ export async function findCandidates(header: OrderHeader): Promise<Candidate[]>
return Math.abs(Number(a.amount) - header.total) - Math.abs(Number(b.amount) - header.total); return Math.abs(Number(a.amount) - header.total) - Math.abs(Number(b.amount) - header.total);
}) })
.slice(0, 5) .slice(0, 5)
.map((r) => ({ transaction_id: r.transaction_id, description: r.description, transaction_date: r.transaction_date, amount: r.amount, currency: r.currency })); .map((r) => ({ transaction_id: r.transaction_id, description: r.description, transaction_date: r.transaction_date, amount: r.amount }));
} }
// ─── the two destinations ─────────────────────────────────────────────────── // ─── the two destinations ───────────────────────────────────────────────────
@@ -304,7 +275,7 @@ export type RouteContext = {
platform: string | null; platform: string | null;
header: OrderHeader; header: OrderHeader;
items: StoredLineItem[]; items: StoredLineItem[];
currency?: string | null; subtotal?: number | null;
}; };
const cardInput = (ctx: RouteContext): CardInput => ({ const cardInput = (ctx: RouteContext): CardInput => ({
@@ -314,7 +285,6 @@ const cardInput = (ctx: RouteContext): CardInput => ({
orderDate: ctx.header.order_date, orderDate: ctx.header.order_date,
platform: ctx.platform, platform: ctx.platform,
itemCount: ctx.items.length, itemCount: ctx.items.length,
currency: ctx.currency,
}); });
const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({ const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({
@@ -329,44 +299,15 @@ const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card:
...extra, ...extra,
}); });
/**
* Are these items unverifiable — no line carrying any money at all?
*
* That is `reconcileItems`' `basis: "none"`: pasted text with no prices, or an
* extraction that read the names and none of the figures. The reconciliation
* check passes it, correctly, because there is nothing to check. What must not
* follow is replacing a receipt that IS itemised with one nothing verified.
*/
const unverifiable = (items: StoredLineItem[]) => items.every((i) => i.amount === null);
/**
* The hold, shared by EVERY path that enriches a matched transaction.
*
* It lived only in `routeFood` and `createFromBatch`'s re-match walked straight
* past it — the same upload destroyed the same receipt depending on which
* button reached it. Returns null when there is nothing to protect.
*/
async function holdRatherThanReplace(ctx: RouteContext, match: MatchRow): Promise<IntakeResult | null> {
if (!unverifiable(ctx.items) || !match.has_line_items) return null;
await updateBatch(ctx.batchId, { status: "held", items: ctx.items });
const card = heldCard(
"this upload carries no prices, so nothing checked it against the total — and that order already has its items",
"I left the existing items alone"
);
return finish(ctx, "held", "held", card, { transaction_id: match.transaction_id });
}
/** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */ /** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */
export async function routeFood(ctx: RouteContext): Promise<IntakeResult> { export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
const { match } = await matchTransaction(ctx.header, ctx.platform); const { match } = await matchTransaction(ctx.header, ctx.platform);
if (match) { if (match) {
const held = await holdRatherThanReplace(ctx, match); await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId });
if (held) return held;
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId, metadataId: match.metadata_id, currency: ctx.currency });
await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items }); await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items });
return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id }); return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
} }
const candidates = await findCandidates(ctx.header); const candidates = await findCandidates(ctx.header, ctx.platform);
await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items }); await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items });
return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates }); return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates });
} }
@@ -389,7 +330,7 @@ export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantr
} catch (e) { } catch (e) {
if (e instanceof PantryDuplicateError) { if (e instanceof PantryDuplicateError) {
await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items }); await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items });
return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, currency: ctx.currency, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), { return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), {
capture_id: e.captureId ?? undefined, capture_id: e.captureId ?? undefined,
}); });
} }
@@ -404,121 +345,38 @@ export async function createFromBatch(batch: BatchRow, kind: "grocery" | "food")
if (!batch.header) throw new Error("batch has no header"); if (!batch.header) throw new Error("batch has no header");
const header = batch.header; const header = batch.header;
const items = batch.items ?? []; const items = batch.items ?? [];
const ctx: RouteContext = {
batchId: batch.id,
source: batch.source,
sharedBy: batch.shared_by,
platform: header.platform,
header,
items,
currency: header.currency,
};
// The card was built minutes ago and the tap arrives now. In between, the
// email/order bridge books orders on its own schedule — so re-run the match
// rather than trusting the "no match" the card was drawn from. Creating a
// second row for an order the ledger already holds is the one outcome this
// whole slice exists to avoid.
const { match } = await matchTransaction(header, header.platform);
if (match) {
const held = await holdRatherThanReplace(ctx, match);
if (held) return held;
await replaceLineItems(match.transaction_id, items, { source: "intake", platform: header.platform, batchId: batch.id, metadataId: match.metadata_id, currency: header.currency });
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: match.transaction_id, items });
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
}
// A sibling batch of the same order — the PDF after the screenshot — may have
// been resolved since. `findDuplicateBatch` deliberately ignores unresolved
// batches, so two pending cards for one order both reach here; without this
// the second tap books the order twice.
const twin = await findDuplicateBatch(batch.id, batch.file_sha256, header);
if (twin) {
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: twin.transaction_id, capture_id: twin.capture_id });
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: twin.transaction_id, captureId: twin.capture_id, batchId: twin.id });
return finish(ctx, kind, "duplicate", card, {
transaction_id: twin.transaction_id ?? undefined,
capture_id: twin.capture_id ?? undefined,
duplicate_of: { batch_id: twin.id, transaction_id: twin.transaction_id, capture_id: twin.capture_id },
});
}
const ownerId = await ownerForEmail(batch.shared_by); const ownerId = await ownerForEmail(batch.shared_by);
const merchant = header.merchant ?? "Unknown merchant"; const merchant = header.merchant ?? "Unknown merchant";
// Not `orderDescription()`: it always prefixes "Order - ", which is right for
// a delivery receipt and wrong for the in-store till receipt this path also
// handles. Not `resolveCategory()` either — it re-derives grocery-vs-food
// from a merchant pattern, and `kind` is the answer the router already
// reached, remembered merchant and Slack tap included.
const description = header.platform ? `Order - ${merchant}` : merchant; const description = header.platform ? `Order - ${merchant}` : merchant;
const category = kind === "grocery" ? "groceries" : "dining"; const category = kind === "grocery" ? "groceries" : "dining";
// `amount` is the figure printed on the receipt, in its own currency. No FX // source/source_ref make a second press of the same card a no-op rather than a second row.
// rate exists at intake, so a foreign order leaves `amount_aud` NULL rather const existing = await queryRow<{ id: number }>(`SELECT id FROM transactions WHERE source = 'intake' AND source_ref = $1`, [batch.id]);
// than asserting a conversion — the same shape order-ingestion.ts writes. const txnId =
const currency = header.currency ?? "AUD"; existing?.id ??
const isAud = currency === "AUD"; (
// Check and insert under one lock, on one connection. await queryRow<{ id: number }>(
//
// `source_ref = batch.id` makes a second press of THIS card a no-op, and
// ON CONFLICT makes that safe against a double tap. It cannot help two
// DIFFERENT batches for one purchase — their keys differ — and both taps run
// their match and duplicate checks before either inserts, so both can find
// nothing. The lock is keyed on the purchase, not the batch, which is the
// only thing the two share. It has to be transaction-scoped and therefore
// inside withTransaction: taken by a statement of its own it would be
// released before the next one ran (see db.ts).
const purchaseKey = [header.platform ?? "", merchantKey(merchant), header.order_date, header.total.toFixed(2)].join("|");
const outcome = await withTransaction(async (q) => {
await q.queryRaw(`SELECT pg_advisory_xact_lock(hashtextextended($1, 0))`, [purchaseKey]);
// Mine (a double tap) or another batch's (the race). The ORDER BY is what
// tells them apart when both exist.
const existing = await q.queryRow<{ id: number; source_ref: string }>(
`SELECT id, source_ref FROM transactions
WHERE source = 'intake' AND superseded_by_id IS NULL
AND (source_ref = $1
OR (transaction_date = $2::date AND amount = $3::numeric
AND lower(merchant_normalized) = lower($4)))
ORDER BY (source_ref = $1) DESC NULLS LAST, id
LIMIT 1`,
[batch.id, header.order_date, header.total, merchant]
);
if (existing) return { id: existing.id, mine: existing.source_ref === batch.id };
const row = await q.queryRow<{ id: number }>(
`INSERT INTO transactions ( `INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method, transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref, merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref
foreign_currency_amount, foreign_currency_code ) VALUES ($1::date, $2, $3::numeric, $3::numeric, $4, NULL, $5, $5, 'debit', $6, 'intake', $7)
) VALUES ($1::date, $2, $3::numeric, $4::numeric, $5, NULL, $6, $6, 'debit', $7, 'intake', $8, $9::numeric, $10)
ON CONFLICT (source, source_ref) WHERE source IS NOT NULL AND source_ref IS NOT NULL
DO NOTHING
RETURNING id`, RETURNING id`,
[header.order_date, description, header.total, isAud ? header.total : null, category, merchant, ownerId, batch.id, isAud ? null : header.total, isAud ? null : currency] [header.order_date, description, header.total, category, merchant, ownerId, batch.id]
); )
return { id: row!.id, mine: true }; )!.id;
}); await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id });
// Another batch booked this purchase in the moment between the two taps. Say
// the same thing the sequential case says one second later, and write nothing.
if (!outcome.mine) {
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: outcome.id });
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: outcome.id, captureId: null, batchId: null });
return finish(ctx, kind, "duplicate", card, { transaction_id: outcome.id });
}
const txnId = outcome.id;
await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id, currency });
await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId }); await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId });
const ctx: RouteContext = { batchId: batch.id, source: batch.source, sharedBy: batch.shared_by, platform: header.platform, header, items };
return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId }); return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId });
} }
/** The card for a batch as it stands — what a second press of a resolved card gets back. */ /** The card for a batch as it stands — what a second press of a resolved card gets back. */
export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } { export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } {
const h = batch.header; const h = batch.header;
const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0, currency: h?.currency }; const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0 };
if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id }); if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id }); if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id }); if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id });
return duplicateCard({ merchant: base.merchant, total: base.total, currency: base.currency, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id }); return duplicateCard({ merchant: base.merchant, total: base.total, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id });
} }
// ─── the router ───────────────────────────────────────────────────────────── // ─── the router ─────────────────────────────────────────────────────────────
@@ -580,7 +438,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
const dupBySha = await findDuplicateBatch(batchId, shas, null); const dupBySha = await findDuplicateBatch(batchId, shas, null);
if (dupBySha) { if (dupBySha) {
await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id }); await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id });
return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, currency: dupBySha.header?.currency, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, { return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, {
kind: (dupBySha.kind as IntakeKind) ?? "food", kind: (dupBySha.kind as IntakeKind) ?? "food",
duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id }, duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id },
}); });
@@ -620,7 +478,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
const dup = await findDuplicateBatch(batchId, shas, storedHeader); const dup = await findDuplicateBatch(batchId, shas, storedHeader);
if (dup) { if (dup) {
await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id }); await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id });
return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, currency: storedHeader.currency, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, { return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, {
header, header,
items, items,
duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id }, duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id },
@@ -635,7 +493,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
} }
await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" }); await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" });
const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, currency: storedHeader.currency }; const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, subtotal: extracted.subtotal };
// 7. act. // 7. act.
if (kind === "ask") { if (kind === "ask") {
+8 -32
View File
@@ -43,7 +43,6 @@ export function platformForSource(source: IntakeSource): string | null {
export const PLATFORM_LABEL: Record<string, string> = { export const PLATFORM_LABEL: Record<string, string> = {
ubereats: "Uber Eats", ubereats: "Uber Eats",
doordash: "DoorDash", doordash: "DoorDash",
uber: "Uber",
}; };
export function platformLabel(platform: string | null): string { export function platformLabel(platform: string | null): string {
@@ -52,16 +51,10 @@ export function platformLabel(platform: string | null): string {
/** /**
* A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints * A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints
* `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; a * `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; substring on
* word-boundary alternation on the normalised name covers both without a * the normalised name covers both without a merchant master-data subsystem.
* merchant master-data subsystem.
*
* The boundaries are load-bearing, not tidiness: a bare substring made
* `Bigals Pizza` a grocer, because "b-IGA-ls" contains `iga`. Short names in a
* substring list collide with ordinary words, and the collision reads as a
* correct classification.
*/ */
export const GROCERY_PATTERN = /\b(coles|woolworths|woolies|aldi|iga|foodworks|costco)\b/i; export const GROCERY_PATTERN = /coles|woolworths|woolies|aldi|iga|foodworks|costco/i;
/** /**
* The normalised merchant used for matching and for the remembered-kind table. * The normalised merchant used for matching and for the remembered-kind table.
@@ -226,27 +219,10 @@ export function toLineItems(items: ExtractedItem[]): StoredLineItem[] {
}); });
} }
/** Is `needle` a contiguous run of whole words inside `hay`? */ /** Does the stored merchant name denote the extracted one? Exact key, or one contains the other. */
function containsWordRun(hay: string[], needle: string[]): boolean {
if (!needle.length || needle.length > hay.length) return false;
for (let i = 0; i + needle.length <= hay.length; i++) {
if (needle.every((w, j) => hay[i + j] === w)) return true;
}
return false;
}
/**
* Does the stored merchant name denote the extracted one? One name's words must
* appear as a contiguous run inside the other's — so `Coles` matches
* `Coles (Wyndham Vale)` and `Order - Coles (Wyndham Vale)`, which is spelling,
* not widening.
*
* Whole words, never a raw substring: `merchantKey("Bigals Pizza")` contains
* `iga`, so a pizza shop matched every IGA on the same date and total.
*/
export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean { export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean {
const a = merchantKey(stored).split(" ").filter(Boolean); const a = merchantKey(stored);
const b = merchantKey(extracted).split(" ").filter(Boolean); const b = merchantKey(extracted);
if (!a.length || !b.length) return false; if (!a || !b) return false;
return containsWordRun(a, b) || containsWordRun(b, a); return a === b || a.includes(b) || b.includes(a);
} }
+21 -97
View File
@@ -1,4 +1,4 @@
import { queryRow } from "@/lib/db"; import { queryRow, queryRaw } from "@/lib/db";
/** /**
* Replace the itemised receipt behind a transaction (plan §11 C2). * Replace the itemised receipt behind a transaction (plan §11 C2).
@@ -15,12 +15,6 @@ import { queryRow } from "@/lib/db";
* transaction has no row at all (a manual transaction, a statement line), one * transaction has no row at all (a manual transaction, a statement line), one
* is created so the items have somewhere to live; a transaction is NEVER * is created so the items have somewhere to live; a transaction is NEVER
* created here. * created here.
*
* `opts.metadataId` pins WHICH row when the caller already resolved one. A
* statement transaction can own its own `expense_metadata` row AND be the
* `matched_transaction_id` of a delivery row: the match query and this write
* then re-pick independently and can land on different rows, so the enrichment
* reports success against a row nobody reads.
*/ */
export type LineItemsSource = "intake" | "pantry"; export type LineItemsSource = "intake" | "pantry";
@@ -61,35 +55,10 @@ export type ReplaceResult = {
line_items: LineItemRecord[]; line_items: LineItemRecord[];
}; };
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* The undo baseline, recorded once per batch.
*
* Returning the row that was just read makes a RETRY destroy the thing undo
* exists to restore: pantry re-calls this on every confirm and after a timeout,
* and the second call would hand back its own write as the "previous" state.
* The first snapshot a batch ever takes is the only one worth keeping, so it is
* written under COALESCE and read back rather than trusted.
*/
async function undoBaseline(batchId: string | null | undefined, snapshot: unknown[]): Promise<unknown[]> {
// Pantry passes back whatever batch id it was given; only an intake batch has
// a row to record against, and a non-uuid would fail the cast.
if (!batchId || !UUID.test(batchId)) return snapshot;
const row = await queryRow<{ previous_line_items: unknown }>(
`UPDATE intake_batches
SET previous_line_items = COALESCE(previous_line_items, $2::jsonb)
WHERE id = $1::uuid
RETURNING previous_line_items`,
[batchId, JSON.stringify(snapshot)]
);
return Array.isArray(row?.previous_line_items) ? row.previous_line_items : snapshot;
}
export async function replaceLineItems( export async function replaceLineItems(
transactionId: number, transactionId: number,
lineItems: LineItemRecord[], lineItems: LineItemRecord[],
opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null; metadataId?: number | null; currency?: string | null } opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null }
): Promise<ReplaceResult> { ): Promise<ReplaceResult> {
const txn = await queryRow<{ const txn = await queryRow<{
id: number; id: number;
@@ -104,78 +73,36 @@ export async function replaceLineItems(
); );
if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`); if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`);
// The caller's row wins where it still exists; otherwise prefer the row that // Prefer the row that IS this transaction over one that merely points at it —
// IS this transaction over one that merely points at it — a statement line // a statement line can be the `matched_transaction_id` of a receipt row and
// can be the `matched_transaction_id` of a receipt row and carry its own row // carry its own row too, and the items belong on the receipt.
// too, and the items belong on the receipt.
const pinned = opts.metadataId
? await queryRow<{ id: number; line_items: unknown }>(
`SELECT id, line_items FROM expense_metadata
WHERE id = $1 AND (transaction_id = $2 OR matched_transaction_id = $2)`,
[opts.metadataId, transactionId]
)
: null;
const json = JSON.stringify(lineItems);
/**
* Snapshot the baseline, then write. In that order on BOTH paths: the create
* path used to snapshot after its INSERT, so a crash in between left a retry
* recording the intake's own write as "previous" — undo restoring the thing
* it was meant to reverse, which is the failure `previous_line_items` exists
* to prevent.
*
* Returns null when the row is gone between the read and the write. That
* UPDATE reported success and the batch was marked enriched against a row
* nothing had written to.
*/
const write = async (row: { id: number; line_items: unknown }): Promise<ReplaceResult | null> => {
const previous = await undoBaseline(opts.batchId, Array.isArray(row.line_items) ? row.line_items : []);
const updated = await queryRow<{ id: number }>(
`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1 RETURNING id`,
[row.id, json]
);
if (!updated) return null;
return { transaction_id: transactionId, metadata_id: row.id, created_metadata: false, previous_line_items: previous, line_items: lineItems };
};
if (pinned) {
const done = await write(pinned);
if (done) return done;
}
const existing = await queryRow<{ id: number; line_items: unknown }>( const existing = await queryRow<{ id: number; line_items: unknown }>(
`SELECT id, line_items FROM expense_metadata `SELECT id, line_items FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1 WHERE transaction_id = $1 OR matched_transaction_id = $1
-- NULLS LAST is load-bearing, and its absence inverted this clause. ORDER BY (transaction_id = $1) DESC, id
-- Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at
-- the transaction has a NULL transaction_id — so the comparison meant to
-- prefer the owned row preferred the other one, silently and in exactly
-- the case it was written for. matchTransaction spells it out; this did
-- not, and the two therefore disagreed.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`, LIMIT 1`,
[transactionId] [transactionId]
); );
const json = JSON.stringify(lineItems);
if (existing) { if (existing) {
const done = await write(existing); await queryRaw(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1`, [existing.id, json]);
if (done) return done; return {
transaction_id: transactionId,
metadata_id: existing.id,
created_metadata: false,
previous_line_items: Array.isArray(existing.line_items) ? existing.line_items : [],
line_items: lineItems,
};
} }
// Nothing to update. Snapshot the empty baseline BEFORE creating the row.
const previous = await undoBaseline(opts.batchId, []);
const reference = opts.batchId ? `intake:${opts.batchId}` : `intake:txn:${transactionId}`; const reference = opts.batchId ? `intake:${opts.batchId}` : `intake:txn:${transactionId}`;
// ON CONFLICT because two calls for one transaction can both find no row and const created = await queryRow<{ id: number }>(
// both INSERT; expense_metadata.transaction_id is UNIQUE, so one of them lost
// with a constraint error that reached the user as an error card. `xmax = 0`
// is how a plain insert is told from a conflict-update, so the reported
// `created_metadata` stays true.
const created = await queryRow<{ id: number; inserted: boolean }>(
`INSERT INTO expense_metadata ( `INSERT INTO expense_metadata (
transaction_id, source, platform, order_reference, line_items, transaction_id, source, platform, order_reference, line_items,
merchant_normalized, amount, transaction_date, currency, flags, extraction_model merchant_normalized, amount, transaction_date, currency, flags, extraction_model
) VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7::numeric, $8::date, $9, '[]'::jsonb, 'cloud-budget') ) VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7::numeric, $8::date, 'AUD', '[]'::jsonb, 'cloud-budget')
ON CONFLICT (transaction_id) DO UPDATE SET line_items = EXCLUDED.line_items RETURNING id`,
RETURNING id, (xmax = 0) AS inserted`,
[ [
transactionId, transactionId,
opts.source, opts.source,
@@ -185,16 +112,13 @@ export async function replaceLineItems(
txn.merchant_normalized ?? txn.merchant_name ?? null, txn.merchant_normalized ?? txn.merchant_name ?? null,
txn.amount, txn.amount,
txn.transaction_date, txn.transaction_date,
// Not hard-coded 'AUD': the transaction may carry a foreign figure, and
// OrderDetails prefers this column, so a USD order printed "$".
opts.currency || "AUD",
] ]
); );
return { return {
transaction_id: transactionId, transaction_id: transactionId,
metadata_id: created!.id, metadata_id: created!.id,
created_metadata: created!.inserted, created_metadata: true,
previous_line_items: previous, previous_line_items: [],
line_items: lineItems, line_items: lineItems,
}; };
} }
+7
View File
@@ -12,6 +12,13 @@ export const CUTOVER_DATE = "2026-01-09";
*/ */
export const DEFAULT_OWNER_ID = 1; export const DEFAULT_OWNER_ID = 1;
/** Human labels for the platform a receipt came from. */
export const PLATFORM_LABEL: Record<ParsedOrder["platform"], string> = {
doordash: "DoorDash",
ubereats: "Uber Eats",
uber: "Uber",
};
/** /**
* Transaction description. * Transaction description.
* *
-10
View File
@@ -119,16 +119,6 @@ export async function merchantForTransaction(
const row = await queryRow<{ merchant_normalized: string | null }>( const row = await queryRow<{ merchant_normalized: string | null }>(
`SELECT merchant_normalized FROM expense_metadata `SELECT merchant_normalized FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1 WHERE transaction_id = $1 OR matched_transaction_id = $1
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`, LIMIT 1`,
[transactionId] [transactionId]
); );
-10
View File
@@ -367,16 +367,6 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
SELECT em.route, em.platform SELECT em.route, em.platform
FROM expense_metadata em FROM expense_metadata em
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (em.transaction_id = t.id) DESC NULLS LAST, em.id
LIMIT 1 LIMIT 1
) order_ctx ON true ) order_ctx ON true
LEFT JOIN LATERAL ( LEFT JOIN LATERAL (