Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
057595f4ac |
@@ -622,89 +622,6 @@ and one tap must not flatten it. It says which case it refused on.
|
|||||||
Slack reaches this route through an n8n webhook, not directly — see the
|
Slack reaches this route through an n8n webhook, not directly — see the
|
||||||
smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory.
|
smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory.
|
||||||
|
|
||||||
### Intake router — one upload, every applicable step (2026-08-21)
|
|
||||||
|
|
||||||
`POST /api/intake` (token `INTAKE_TOKEN`, multipart: `source`, `shared_by`,
|
|
||||||
`batch_id`, 1–6 `file`s) is the finance half of the share-sheet intake; design
|
|
||||||
and the cross-repo contract are in smarthome `plans/intake-router-slice-plan.md`
|
|
||||||
§11 (C1, C2, C5 live here; C3/C4 are pantry-app; C6 is n8n + compose).
|
|
||||||
|
|
||||||
- **Order of operations is the plan's and is load-bearing:** persist
|
|
||||||
`intake_batches` row → classify each file → ONE LiteLLM call for the whole
|
|
||||||
batch (`cloud-budget`, `temperature 0`, strict `json_schema`; PDFs go in as
|
|
||||||
`unpdf` text, never pixels) → header `(merchant, date, total)` → merchant
|
|
||||||
kind → duplicate check → reconciliation → match → act. `src/lib/intake-rules.ts`
|
|
||||||
holds the pure decisions, `intake-router.ts` the orchestration.
|
|
||||||
- **Match is `(platform, merchant, date, total)`, exact, never widened.** A
|
|
||||||
miss or an ambiguity returns candidates (same merchant ±3 days) with
|
|
||||||
`intake_use_candidate` / `intake_create` buttons; a transaction is created
|
|
||||||
only on the explicit tap, ING-10-style (`statement_id NULL`, `owner_id` from
|
|
||||||
`shared_by` → `participants.email`, else `DEFAULT_OWNER_ID`;
|
|
||||||
`source='intake'`, `source_ref=batch_id` so a second press is a no-op —
|
|
||||||
written as `ON CONFLICT … DO NOTHING`, because a SELECT-then-INSERT loses a
|
|
||||||
double tap). A non-AUD receipt is booked the way `order-ingestion.ts` books
|
|
||||||
one: `amount` native, `amount_aud` **NULL**, `foreign_currency_*` set — no FX
|
|
||||||
rate exists at intake and inventing one is worse than admitting there is none.
|
|
||||||
- **A NULL platform means "no platform", not "any platform".** An in-store tap
|
|
||||||
(`coles`, `woolworths`, `other`) matches only rows whose `expense_metadata`
|
|
||||||
carries no platform of its own. The wildcard reading let a till receipt match
|
|
||||||
the Uber Eats order for the same shop on the same day and overwrite its items.
|
|
||||||
The **candidates** query is the opposite and deliberately so: it has no
|
|
||||||
platform filter at all, because the card is a human choice shown precisely
|
|
||||||
because the exact key missed — the platform is one of the things that may be
|
|
||||||
wrong, so the list must widen there, not narrow.
|
|
||||||
- **The Create tap re-runs the match before it inserts**, and checks for a
|
|
||||||
resolved sibling batch. The card is minutes old by the time it is pressed and
|
|
||||||
the email/order bridge books orders on its own schedule; `findDuplicateBatch`
|
|
||||||
ignores *unresolved* batches by design, so two pending cards for one order
|
|
||||||
would otherwise each book it.
|
|
||||||
- **One `expense_metadata` row, agreed on by every query that resolves one.** A
|
|
||||||
transaction can own a row *and* be the `matched_transaction_id` of another;
|
|
||||||
`matchTransaction` returns the row it resolved through and `replaceLineItems`
|
|
||||||
writes to exactly that one. Every reader orders by
|
|
||||||
`(transaction_id = $1) DESC **NULLS LAST**, id` — the match, the write, the
|
|
||||||
order-detail GET, `getTransactions`' provenance LATERAL, `merchantForTransaction`
|
|
||||||
and the Slack details modal. The `NULLS LAST` is load-bearing: Postgres orders
|
|
||||||
`DESC` as `NULLS FIRST` and the row that merely *points* at the transaction has
|
|
||||||
a NULL `transaction_id`, so the clause meant to prefer the owned row preferred
|
|
||||||
the other one. Measured 2026-08-21: 667 rows, 147 pointing-only, **zero**
|
|
||||||
transactions resolving to more than one row — the ordering changes nothing
|
|
||||||
today and exists so the readers still agree when multiplicity appears.
|
|
||||||
- **The create is guarded by a transaction-scoped advisory lock on the
|
|
||||||
PURCHASE**, not the batch. `ON CONFLICT (source, source_ref)` handles a double
|
|
||||||
tap on one card; two *different* batches for one purchase have different
|
|
||||||
`source_ref`s and both clear their match and duplicate checks before either
|
|
||||||
inserts. The lock has to live inside `withTransaction` (`src/lib/db.ts`):
|
|
||||||
`queryRaw` is `$queryRawUnsafe` on a **pooled** client, so every call is its
|
|
||||||
own implicit transaction and `pg_advisory_xact_lock` taken in a statement of
|
|
||||||
its own is released before the next statement runs. The loser reports
|
|
||||||
`duplicate` pointing at the winner's transaction — the same answer the
|
|
||||||
sequential path gives.
|
|
||||||
- **`replaceLineItems` snapshots the undo baseline BEFORE it writes, on both
|
|
||||||
paths**, and treats an UPDATE that touched no row as a failure rather than a
|
|
||||||
success. The metadata INSERT is `ON CONFLICT (transaction_id) DO UPDATE` and
|
|
||||||
carries the batch's **currency** — a hard-coded `'AUD'` made `OrderDetails`
|
|
||||||
print `$` on a USD order, because it prefers `expense_metadata.currency`.
|
|
||||||
- **`intake_batches.previous_line_items` is the undo baseline, written once.**
|
|
||||||
Returning whatever the write just read makes a pantry retry get its own write
|
|
||||||
back as the "previous" state, so undo restores the thing it undid. Recorded
|
|
||||||
under `COALESCE(previous_line_items, …)` and read back.
|
|
||||||
- **Grocery never writes finance here.** The ORIGINAL files go to pantry
|
|
||||||
`POST /api/capture` with `intake.finance_target_transaction_id` +
|
|
||||||
`finance_action: enrich|create`; pantry's confirm calls
|
|
||||||
`PUT /api/transactions/[id]/line-items` back (C4), which is why that route
|
|
||||||
accepts `RECEIPT_INGEST_TOKEN` too. Files are kept under `INTAKE_FILE_DIR`
|
|
||||||
keyed by sha256 so a later "Groceries" tap can still forward them.
|
|
||||||
- **Reconciliation is against the printed item subtotal, not the total.** A
|
|
||||||
delivery order's total is items + fees − promotions ($118.94 of Coles lines
|
|
||||||
became $85.74), so checking lines against the total would hold every
|
|
||||||
delivery order. Tolerance $1 or 2%; a failure is `held`, nothing written.
|
|
||||||
- **Names stay verbatim.** Burgertory calls its burgers "Michael" and "Joel";
|
|
||||||
the prompt forbids canonicalising and the stored shape is the order lane's
|
|
||||||
(`qty`, `description`, `amount`, `options`) so `order-details.tsx` renders it.
|
|
||||||
- Statuses beyond the contract list: `ask` (the "Groceries or a meal?" holding
|
|
||||||
state), `held`, `error` (502 — model or pantry down; the batch row says so).
|
|
||||||
|
|
||||||
### The shared loan
|
### The shared loan
|
||||||
|
|
||||||
The loan is a **separate ledger**, not a shared expense and not a settlement
|
The loan is a **separate ledger**, not a shared expense and not a settlement
|
||||||
|
|||||||
@@ -282,18 +282,6 @@ All routes require authentication via `X-Forwarded-User` header (set by Traefik)
|
|||||||
| GET / POST | `/api/tags` | List or create tags |
|
| GET / POST | `/api/tags` | List or create tags |
|
||||||
| PATCH / DELETE | `/api/tags/[id]` | Update or delete a tag |
|
| PATCH / DELETE | `/api/tags/[id]` | Update or delete a tag |
|
||||||
|
|
||||||
**Machine routes** — shared-secret `x-ingest-token` header instead of `X-Forwarded-User`; each fails closed when its token is unset:
|
|
||||||
|
|
||||||
| Method | Route | Token | Description |
|
|
||||||
|--------|-------|-------|-------------|
|
|
||||||
| POST / PATCH | `/api/orders/ingest` | `ORDER_INGEST_TOKEN` | n8n order receipts → transactions (see smarthome CLAUDE.md "Order Processor") |
|
|
||||||
| POST | `/api/receipts/ingest` | `RECEIPT_INGEST_TOKEN` | pantry-app confirmed grocery receipts → one transaction per tender leg (ING-10) |
|
|
||||||
| POST | `/api/intake` | `INTAKE_TOKEN` | **Intake router** (2026-08-21): multipart `source` (`uber_eats\|doordash\|coles\|woolworths\|other`), `shared_by`, `batch_id` (uuid), 1–6 `file`s (image / PDF / text). Persists `intake_batches`, extracts once via LiteLLM `cloud-budget` (PDF → text via `unpdf`), matches `(platform, merchant, date, total)` exactly, then enriches the matched order's `line_items` (food), forwards the original files to pantry `POST /api/capture` (grocery), or replies with candidates / "Groceries or a meal?" / needs-header / held / duplicate. Returns `{kind, status, transaction_id?, capture_id?, candidates?, slack_blocks, text}` |
|
|
||||||
| PUT | `/api/transactions/[id]/line-items` | `INTAKE_TOKEN` or `RECEIPT_INGEST_TOKEN` | Idempotent replace of `expense_metadata.line_items` for `transaction_id = :id OR matched_transaction_id = :id`; body `{line_items, source: 'intake'\|'pantry', batch_id?, platform?, replace: true}`; creates the metadata row if absent, never a transaction (404). Returns `{previous_line_items, line_items}` — pantry calls it on every confirm and with `previous_line_items` on undo |
|
|
||||||
| POST | `/api/slack/interactive` | `ORDER_INGEST_TOKEN` + Slack signature | Order-nudge buttons and the intake taps (`intake_use_candidate`, `intake_create`, `intake_grocery`, `intake_meal`) |
|
|
||||||
|
|
||||||
Env for the intake router: `INTAKE_TOKEN`, `LITELLM_BASE_URL` + `LITELLM_KEY` (OpenAI-compatible, alias `cloud-budget`), `PANTRY_BASE_URL` (+ the existing `RECEIPT_INGEST_TOKEN` as pantry's machine auth), optional `INTAKE_FILE_DIR` (original files by sha256, default `/app/data/intake` — a later "Groceries" tap re-sends them) and `FINANCE_PUBLIC_URL` (the card's View link, default `https://finance.bosecamp.com`). Design: smarthome `plans/intake-router-slice-plan.md` §11.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Ingestion Pipeline
|
## Ingestion Pipeline
|
||||||
@@ -359,8 +347,6 @@ docker exec postgres-personal psql -U personal -d personal \
|
|||||||
| `0009_split_payments` | `split_payments` |
|
| `0009_split_payments` | `split_payments` |
|
||||||
| `0010_csv_import_reconcile` | `reconciled_with_id`, CSV import support |
|
| `0010_csv_import_reconcile` | `reconciled_with_id`, CSV import support |
|
||||||
| `0011_trips` | `trips`, `trip_id` on `transaction_overrides` (catch-up — was applied directly) |
|
| `0011_trips` | `trips`, `trip_id` on `transaction_overrides` (catch-up — was applied directly) |
|
||||||
| … | |
|
|
||||||
| `0034_intake_batches` | `intake_batches` (one row per `/api/intake` upload, written before extraction) and `intake_merchant_kinds` (the remembered "Groceries or a meal?" answer per merchant) |
|
|
||||||
|
|
||||||
> `paperless_doc_id` on statements and the `uq_statements_paperless_doc_id` index were added directly (not tracked in a migration file).
|
> `paperless_doc_id` on statements and the `uq_statements_paperless_doc_id` index were added directly (not tracked in a migration file).
|
||||||
> `owner_id` on transactions and `statement_id` made nullable were applied directly (March 2026) to support manual transaction entry without a fake statement.
|
> `owner_id` on transactions and `statement_id` made nullable were applied directly (March 2026) to support manual transaction entry without a fake statement.
|
||||||
|
|||||||
@@ -32,20 +32,10 @@ CREATE TABLE IF NOT EXISTS intake_batches (
|
|||||||
status text NOT NULL,
|
status text NOT NULL,
|
||||||
transaction_id integer,
|
transaction_id integer,
|
||||||
capture_id integer,
|
capture_id integer,
|
||||||
-- The undo baseline: what `expense_metadata.line_items` held before this
|
|
||||||
-- batch first replaced it. Recorded here rather than returned from the write,
|
|
||||||
-- because pantry re-calls the enrich endpoint on every confirm and after a
|
|
||||||
-- timeout — a caller that hands back whatever it just read gives a retry its
|
|
||||||
-- OWN write as the "previous" state, and undo then restores the thing it was
|
|
||||||
-- meant to reverse. Written under COALESCE, so only the first snapshot sticks.
|
|
||||||
previous_line_items jsonb,
|
|
||||||
created_at timestamptz NOT NULL DEFAULT now(),
|
created_at timestamptz NOT NULL DEFAULT now(),
|
||||||
resolved_at timestamptz
|
resolved_at timestamptz
|
||||||
);
|
);
|
||||||
|
|
||||||
-- For a database that already ran an earlier draft of this file.
|
|
||||||
ALTER TABLE intake_batches ADD COLUMN IF NOT EXISTS previous_line_items jsonb;
|
|
||||||
|
|
||||||
COMMENT ON TABLE intake_batches IS
|
COMMENT ON TABLE intake_batches IS
|
||||||
'One row per upload batch through POST /api/intake. Persisted before '
|
'One row per upload batch through POST /api/intake. Persisted before '
|
||||||
'extraction so Slack taps can act on it later and re-shares are detected. '
|
'extraction so Slack taps can act on it later and re-shares are detected. '
|
||||||
|
|||||||
@@ -283,8 +283,6 @@ model intake_batches {
|
|||||||
status String
|
status String
|
||||||
transaction_id Int?
|
transaction_id Int?
|
||||||
capture_id Int?
|
capture_id Int?
|
||||||
// The undo baseline, written once per batch (see 0034 and line-items.ts).
|
|
||||||
previous_line_items Json?
|
|
||||||
created_at DateTime @default(now()) @db.Timestamptz(6)
|
created_at DateTime @default(now()) @db.Timestamptz(6)
|
||||||
resolved_at DateTime? @db.Timestamptz(6)
|
resolved_at DateTime? @db.Timestamptz(6)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -23,25 +23,6 @@ export function mockDbWithPool(p: Pool) {
|
|||||||
const result = await p.query(sql, params);
|
const result = await p.query(sql, params);
|
||||||
return result.rows[0] ?? null;
|
return result.rows[0] ?? null;
|
||||||
},
|
},
|
||||||
// Mirrors withTransaction: one connection, BEGIN/COMMIT around the callback,
|
|
||||||
// so pg_advisory_xact_lock actually holds across the statements inside it.
|
|
||||||
withTransaction: async (fn: (q: unknown) => Promise<unknown>) => {
|
|
||||||
const client = await p.connect();
|
|
||||||
try {
|
|
||||||
await client.query("BEGIN");
|
|
||||||
const result = await fn({
|
|
||||||
queryRaw: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows,
|
|
||||||
queryRow: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows[0] ?? null,
|
|
||||||
});
|
|
||||||
await client.query("COMMIT");
|
|
||||||
return result;
|
|
||||||
} catch (e) {
|
|
||||||
await client.query("ROLLBACK");
|
|
||||||
throw e;
|
|
||||||
} finally {
|
|
||||||
client.release();
|
|
||||||
}
|
|
||||||
},
|
|
||||||
prisma: p,
|
prisma: p,
|
||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -49,7 +49,6 @@ let fileDir: string;
|
|||||||
let intakePOST: (req: NextRequest) => Promise<Response>;
|
let intakePOST: (req: NextRequest) => Promise<Response>;
|
||||||
let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
|
let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
|
||||||
let slackPOST: (req: NextRequest) => Promise<Response>;
|
let slackPOST: (req: NextRequest) => Promise<Response>;
|
||||||
let orderGET: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
|
|
||||||
|
|
||||||
/** What the next LiteLLM call answers, and what the next pantry call answers. */
|
/** What the next LiteLLM call answers, and what the next pantry call answers. */
|
||||||
let modelReply: ExtractedOrder | Error = burgertory;
|
let modelReply: ExtractedOrder | Error = burgertory;
|
||||||
@@ -105,10 +104,6 @@ async function seedOrder(over: { merchant: string; date: string; amount: number;
|
|||||||
}
|
}
|
||||||
return id;
|
return id;
|
||||||
}
|
}
|
||||||
/** What `components/order-details.tsx` would actually render for a transaction. */
|
|
||||||
const orderPanel = async (txnId: number) =>
|
|
||||||
(await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null;
|
|
||||||
const metaRows = async () => (await pool.query(`SELECT id, platform, line_items FROM expense_metadata ORDER BY id`)).rows as { id: number; platform: string | null; line_items: { description: string }[] }[];
|
|
||||||
const lineItemsOf = async (txnId: number) =>
|
const lineItemsOf = async (txnId: number) =>
|
||||||
(await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items;
|
(await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items;
|
||||||
const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0];
|
const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0];
|
||||||
@@ -132,17 +127,9 @@ beforeAll(async () => {
|
|||||||
participantForSlackUser: () => 1,
|
participantForSlackUser: () => 1,
|
||||||
slackUserForParticipant: () => null,
|
slackUserForParticipant: () => null,
|
||||||
}));
|
}));
|
||||||
// The order-detail panel is the surface an enrichment has to appear on, so
|
|
||||||
// the read is exercised beside the write rather than assumed.
|
|
||||||
vi.doMock("@/lib/auth", () => ({ getCurrentUser: async () => ({ id: 1, name: "Owner", email: "o@x" }) }));
|
|
||||||
vi.doMock("@/lib/queries", async (importOriginal) => ({
|
|
||||||
...(await importOriginal<Record<string, unknown>>()),
|
|
||||||
canAccessTransactions: async () => true,
|
|
||||||
}));
|
|
||||||
({ POST: intakePOST } = await import("../../app/api/intake/route"));
|
({ POST: intakePOST } = await import("../../app/api/intake/route"));
|
||||||
({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route"));
|
({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route"));
|
||||||
({ POST: slackPOST } = await import("../../app/api/slack/interactive/route"));
|
({ POST: slackPOST } = await import("../../app/api/slack/interactive/route"));
|
||||||
({ GET: orderGET } = await import("../../app/api/transactions/[id]/order/route"));
|
|
||||||
stubFetch();
|
stubFetch();
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -223,82 +210,6 @@ describe("PUT /api/transactions/[id]/line-items (C2)", () => {
|
|||||||
expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" });
|
expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" });
|
||||||
});
|
});
|
||||||
|
|
||||||
it("the undo baseline is recorded ONCE per batch: a retry gets the original, not its own write", async () => {
|
|
||||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
const original = [{ qty: 1, description: "what the order already had", amount: 22.4 }];
|
|
||||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(original)]);
|
|
||||||
const id = uuid();
|
|
||||||
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
|
|
||||||
|
|
||||||
const first = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
|
|
||||||
expect(first.previous_line_items).toEqual(original);
|
|
||||||
// Pantry re-calls on every confirm and after a timeout. Before, the retry
|
|
||||||
// was handed its OWN write back, so undo restored the thing it undid.
|
|
||||||
const retry = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
|
|
||||||
expect(retry.previous_line_items).toEqual(original);
|
|
||||||
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual(original);
|
|
||||||
|
|
||||||
await put(txn, { line_items: retry.previous_line_items, source: "pantry", batch_id: id, replace: true });
|
|
||||||
expect(await lineItemsOf(txn)).toEqual(original);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("without a batch id the baseline is still the row as read — nothing else changes", async () => {
|
|
||||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
const first = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
|
|
||||||
expect(first.previous_line_items).toEqual([]);
|
|
||||||
const second = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
|
|
||||||
expect(second.previous_line_items).toEqual(items);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a pinned row that no longer exists falls back to the live one, rather than reporting a phantom write", async () => {
|
|
||||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
const live = (await pool.query(`SELECT id FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].id as number;
|
|
||||||
const { replaceLineItems } = await import("../../lib/line-items");
|
|
||||||
const r = await replaceLineItems(txn, items, { source: "intake", metadataId: live + 999 });
|
|
||||||
expect(r.metadata_id).toBe(live);
|
|
||||||
expect(await lineItemsOf(txn)).toHaveLength(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("two writes racing to create the metadata row both resolve to the same one", async () => {
|
|
||||||
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
|
|
||||||
const both = await Promise.all([
|
|
||||||
put(txn, { line_items: items, source: "intake", replace: true }),
|
|
||||||
put(txn, { line_items: items, source: "intake", replace: true }),
|
|
||||||
]);
|
|
||||||
expect(both.map((r) => r.status)).toEqual([200, 200]);
|
|
||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].n).toBe(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("the create path records an undo baseline too, before it writes", async () => {
|
|
||||||
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
|
|
||||||
const id = uuid();
|
|
||||||
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
|
|
||||||
await put(txn, { line_items: items, source: "intake", batch_id: id, currency: "AUD", replace: true });
|
|
||||||
// [] rather than NULL: there was nothing there, and "nothing" is the state
|
|
||||||
// undo has to be able to restore.
|
|
||||||
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual([]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("the panel reads the row the write landed on, whichever order the rows were inserted in", async () => {
|
|
||||||
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
|
|
||||||
// Inserted first: the delivery receipt pointing at the statement line.
|
|
||||||
await pool.query(
|
|
||||||
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
|
|
||||||
[t, uuid()]
|
|
||||||
);
|
|
||||||
// Inserted second: a row the transaction owns. An unordered LIMIT 1 reads
|
|
||||||
// the first one, the write takes the second, and the enrichment reports
|
|
||||||
// success against a row nothing renders.
|
|
||||||
await pool.query(
|
|
||||||
`INSERT INTO expense_metadata (transaction_id, source, order_reference, line_items) VALUES ($1, 'pantry', $2, '[]'::jsonb)`,
|
|
||||||
[t, uuid()]
|
|
||||||
);
|
|
||||||
const written = await (await put(t, { line_items: items, source: "intake", replace: true })).json();
|
|
||||||
const panel = await orderPanel(t);
|
|
||||||
expect(panel?.line_items).toHaveLength(2);
|
|
||||||
expect((await metaRows()).find((r) => r.id === written.metadata_id)!.line_items).toHaveLength(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("finds the row through matched_transaction_id — a card-settled order has only that", async () => {
|
it("finds the row through matched_transaction_id — a card-settled order has only that", async () => {
|
||||||
const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
|
const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
|
||||||
await pool.query(
|
await pool.query(
|
||||||
@@ -374,82 +285,10 @@ describe("POST /api/intake — food (enrich vs candidates)", () => {
|
|||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match — but it IS offered", async () => {
|
it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match", async () => {
|
||||||
const doordash = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" });
|
await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" });
|
||||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||||
expect(body.status).toBe("candidates");
|
expect(body.status).toBe("candidates");
|
||||||
// The candidates card is a human choice, not a decision. The platform is one
|
|
||||||
// of the things that may be wrong, so it must not narrow the list — a
|
|
||||||
// platform-filtered card showed nothing at all here.
|
|
||||||
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([doordash]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a NULL platform is 'no platform', not a wildcard — an in-store share does not overwrite the delivery order", async () => {
|
|
||||||
await pool.query(`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ('burgertory werribee', 'food')`);
|
|
||||||
const delivery = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "ubereats" });
|
|
||||||
const already = [{ qty: 1, description: "what the delivery order already had", amount: 22.4 }];
|
|
||||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [delivery, JSON.stringify(already)]);
|
|
||||||
|
|
||||||
// `other` implies no platform; the delivery row carries one, so it is not
|
|
||||||
// the match — it is only a candidate.
|
|
||||||
const body = await (await intakePOST(multipart({ source: "other" }, [pdf]))).json();
|
|
||||||
expect(body.status).toBe("candidates");
|
|
||||||
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([delivery]);
|
|
||||||
expect(await lineItemsOf(delivery)).toEqual(already);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a reconciled manual row is neither matched nor offered — EXCLUDE_RECONCILED_SOURCE, as everywhere else", async () => {
|
|
||||||
const live = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
const reconciled = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, reconciled]);
|
|
||||||
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
|
|
||||||
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, near]);
|
|
||||||
|
|
||||||
// Two rows at the exact key would have read as an ambiguity; one of them is
|
|
||||||
// the same spend counted twice.
|
|
||||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
|
||||||
expect(body).toMatchObject({ status: "enriched", transaction_id: live });
|
|
||||||
});
|
|
||||||
|
|
||||||
it("the write lands on the row the MATCH resolved through, not on whichever row sorts first", async () => {
|
|
||||||
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
|
|
||||||
// The statement line owns a receipt row of its own (a pantry confirm) …
|
|
||||||
const grocery = [{ qty: 1, description: "a grocery shop", amount: 22.4 }];
|
|
||||||
const own = (await pool.query(
|
|
||||||
`INSERT INTO expense_metadata (transaction_id, source, order_reference, merchant_normalized, line_items) VALUES ($1, 'pantry', $2, 'Woolworths', $3::jsonb) RETURNING id`,
|
|
||||||
[t, uuid(), JSON.stringify(grocery)]
|
|
||||||
)).rows[0].id as number;
|
|
||||||
// … and is also the card leg of the Uber Eats order being enriched.
|
|
||||||
const delivery = (await pool.query(
|
|
||||||
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, merchant_normalized, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, 'Burgertory (Werribee)', '[]'::jsonb) RETURNING id`,
|
|
||||||
[t, uuid()]
|
|
||||||
)).rows[0].id as number;
|
|
||||||
|
|
||||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
|
||||||
expect(body).toMatchObject({ status: "enriched", transaction_id: t });
|
|
||||||
const rows = await metaRows();
|
|
||||||
expect(rows.find((r) => r.id === delivery)!.line_items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
|
|
||||||
expect(rows.find((r) => r.id === own)!.line_items).toEqual(grocery);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("an upload with no prices never replaces a receipt that already has items", async () => {
|
|
||||||
const priceless = { ...burgertory, subtotal: null, total: 22.4, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
|
|
||||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
|
|
||||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
|
|
||||||
|
|
||||||
modelReply = priceless;
|
|
||||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
|
||||||
expect(body).toMatchObject({ kind: "held", status: "held" });
|
|
||||||
expect(body.text).toMatch(/left the existing items alone/);
|
|
||||||
expect(await lineItemsOf(txn)).toEqual(already);
|
|
||||||
|
|
||||||
// …but an EMPTY receipt is exactly what this slice exists to fill.
|
|
||||||
const empty = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-20", amount: 22.4 });
|
|
||||||
modelReply = { ...priceless, order_date: "2026-08-20" };
|
|
||||||
const second = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
|
||||||
expect(second).toMatchObject({ status: "enriched", transaction_id: empty });
|
|
||||||
expect((await lineItemsOf(empty)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => {
|
it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => {
|
||||||
@@ -552,19 +391,6 @@ describe("POST /api/intake — grocery → pantry", () => {
|
|||||||
expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
|
expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
|
||||||
});
|
});
|
||||||
|
|
||||||
it("an in-store tap does not hand pantry the delivery order as its finance target", async () => {
|
|
||||||
modelReply = coles;
|
|
||||||
// The Uber Eats Coles order the email pipeline already booked.
|
|
||||||
const delivery = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: "ubereats" });
|
|
||||||
// The same shop, same day, same money — but shared from the Coles app, so
|
|
||||||
// it carries no platform and is a different purchase.
|
|
||||||
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
|
||||||
expect(body.status).toBe("forwarded_to_pantry");
|
|
||||||
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown> };
|
|
||||||
expect(pantryCall.intake).toMatchObject({ finance_action: "create", finance_target_transaction_id: null, platform: null });
|
|
||||||
expect(await lineItemsOf(delivery)).toEqual([]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => {
|
it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => {
|
||||||
modelReply = coles;
|
modelReply = coles;
|
||||||
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
||||||
@@ -631,112 +457,6 @@ describe("Slack taps (C5)", () => {
|
|||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("intake_create re-runs the match: an order booked between the card and the tap is enriched, not duplicated", async () => {
|
|
||||||
const id = uuid();
|
|
||||||
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
|
|
||||||
expect(body.status).toBe("candidates");
|
|
||||||
|
|
||||||
// The email/order bridge books the same purchase while the card sits in
|
|
||||||
// Slack. The card's "no match" is minutes stale by the time it is pressed.
|
|
||||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
|
|
||||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
|
||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
|
||||||
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
|
|
||||||
expect((await lineItemsOf(txn)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
|
||||||
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("two pending cards for one order book it ONCE", async () => {
|
|
||||||
const a = uuid();
|
|
||||||
const b = uuid();
|
|
||||||
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
|
|
||||||
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
|
|
||||||
expect((await batch(a)).status).toBe("candidates");
|
|
||||||
expect((await batch(b)).status).toBe("candidates");
|
|
||||||
|
|
||||||
await slackPOST(slackPress("intake_create", { batch_id: a }));
|
|
||||||
await slackPOST(slackPress("intake_create", { batch_id: b }));
|
|
||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
|
||||||
expect((await batch(b)).transaction_id).toBe((await batch(a)).transaction_id);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a sibling already resolved to the pantry board makes Create a duplicate, not a second row", async () => {
|
|
||||||
// Two shares of one receipt, resolved out of order: nothing the second card
|
|
||||||
// can match on, because the first one never produced a transaction.
|
|
||||||
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
|
|
||||||
const a = uuid();
|
|
||||||
const b = uuid();
|
|
||||||
await intakePOST(multipart({ source: "other", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
|
|
||||||
await intakePOST(multipart({ source: "other", batch_id: a }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
|
|
||||||
expect((await batch(a)).status).toBe("ask");
|
|
||||||
|
|
||||||
await slackPOST(slackPress("intake_grocery", { batch_id: a }));
|
|
||||||
expect((await batch(a)).status).toBe("forwarded_to_pantry");
|
|
||||||
|
|
||||||
await slackPOST(slackPress("intake_meal", { batch_id: b }));
|
|
||||||
expect((await batch(b)).status).toBe("candidates");
|
|
||||||
await slackPOST(slackPress("intake_create", { batch_id: b }));
|
|
||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
|
|
||||||
expect(await batch(b)).toMatchObject({ status: "duplicate", capture_id: 901 });
|
|
||||||
expect(lastSlackUpdate().text).toMatch(/^Already handled/);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("two taps that land together still book one transaction, and neither reports an error", async () => {
|
|
||||||
const id = uuid();
|
|
||||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
|
||||||
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: id })), slackPOST(slackPress("intake_create", { batch_id: id }))]);
|
|
||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
|
||||||
const texts = calls.filter((c) => c.url.includes("hooks.slack.com")).map((c) => (c.body as { text: string }).text);
|
|
||||||
expect(texts.every((t) => !/Couldn't process/.test(t))).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a foreign receipt is booked in its own currency, with no invented AUD figure", async () => {
|
|
||||||
modelReply = { ...burgertory, currency: "USD" };
|
|
||||||
const id = uuid();
|
|
||||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
|
||||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
|
||||||
const t = (await pool.query(`SELECT amount::text, amount_aud, foreign_currency_amount::text, foreign_currency_code FROM transactions`)).rows[0];
|
|
||||||
expect(t).toEqual({ amount: "22.40", amount_aud: null, foreign_currency_amount: "22.40", foreign_currency_code: "USD" });
|
|
||||||
expect(lastSlackUpdate().text).toMatch(/USD 22\.40/);
|
|
||||||
// OrderDetails prefers expense_metadata.currency, so a hard-coded 'AUD'
|
|
||||||
// here printed "$" on a USD order's receipt panel.
|
|
||||||
expect((await pool.query(`SELECT currency FROM expense_metadata`)).rows[0].currency).toBe("USD");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("the hold applies to the Create tap's re-match, not only to the upload path", async () => {
|
|
||||||
modelReply = { ...burgertory, subtotal: null, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
|
|
||||||
const id = uuid();
|
|
||||||
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
|
|
||||||
expect(body.status).toBe("candidates");
|
|
||||||
|
|
||||||
// Booked WITH its items between the card and the tap. The re-match walked
|
|
||||||
// past the guard routeFood applies, so the same upload destroyed the same
|
|
||||||
// receipt depending on which button reached it.
|
|
||||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
|
||||||
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
|
|
||||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
|
|
||||||
|
|
||||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
|
||||||
expect(await lineItemsOf(txn)).toEqual(already);
|
|
||||||
expect(await batch(id)).toMatchObject({ status: "held" });
|
|
||||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("two DIFFERENT batches for one purchase, tapped together, book it once", async () => {
|
|
||||||
const a = uuid();
|
|
||||||
const b = uuid();
|
|
||||||
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
|
|
||||||
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
|
|
||||||
// Their source_refs differ, so ON CONFLICT cannot save this; both taps
|
|
||||||
// clear their match and duplicate checks before either inserts.
|
|
||||||
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: a })), slackPOST(slackPress("intake_create", { batch_id: b }))]);
|
|
||||||
const rows = (await pool.query(`SELECT id FROM transactions`)).rows;
|
|
||||||
expect(rows).toHaveLength(1);
|
|
||||||
expect((await batch(a)).transaction_id).toBe(rows[0].id);
|
|
||||||
expect((await batch(b)).transaction_id).toBe(rows[0].id);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("unknown shared_by falls back to the default owner — never a NULL owner", async () => {
|
it("unknown shared_by falls back to the default owner — never a NULL owner", async () => {
|
||||||
const id = uuid();
|
const id = uuid();
|
||||||
await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf]));
|
await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf]));
|
||||||
|
|||||||
@@ -3,7 +3,6 @@ import { readFileSync } from "fs";
|
|||||||
import { resolve } from "path";
|
import { resolve } from "path";
|
||||||
import {
|
import {
|
||||||
classifyInput,
|
classifyInput,
|
||||||
GROCERY_PATTERN,
|
|
||||||
merchantKey,
|
merchantKey,
|
||||||
merchantMatches,
|
merchantMatches,
|
||||||
platformForSource,
|
platformForSource,
|
||||||
@@ -13,8 +12,8 @@ import {
|
|||||||
validateHeader,
|
validateHeader,
|
||||||
type ExtractedOrder,
|
type ExtractedOrder,
|
||||||
} from "../../lib/intake-rules";
|
} from "../../lib/intake-rules";
|
||||||
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor } from "../../lib/intake-extract";
|
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT } from "../../lib/intake-extract";
|
||||||
import { askCard, candidatesCard, enrichedCard, type Block } from "../../lib/intake-blocks";
|
import { askCard, candidatesCard, enrichedCard } from "../../lib/intake-blocks";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The intake router's pure rules, against the strings real receipts print.
|
* The intake router's pure rules, against the strings real receipts print.
|
||||||
@@ -86,17 +85,6 @@ describe("merchant pattern — grocery | food | ask", () => {
|
|||||||
expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false);
|
expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false);
|
||||||
expect(merchantMatches(null, "Coles")).toBe(false);
|
expect(merchantMatches(null, "Coles")).toBe(false);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("whole words, not substrings: `Bigals Pizza` contains `iga` and is neither a grocer nor an IGA", () => {
|
|
||||||
expect(GROCERY_PATTERN.test(merchantKey("Bigals Pizza"))).toBe(false);
|
|
||||||
expect(resolveMerchantKind("uber_eats", "Bigals Pizza")).toBe("food");
|
|
||||||
expect(merchantMatches("Bigals Pizza", "IGA")).toBe(false);
|
|
||||||
// and the grocers a boundary must not cost us
|
|
||||||
expect(resolveMerchantKind("uber_eats", "IGA Wyndham Vale")).toBe("grocery");
|
|
||||||
expect(resolveMerchantKind("uber_eats", "Woolworths Group Limited")).toBe("grocery");
|
|
||||||
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
|
|
||||||
expect(merchantMatches("IGA Wyndham Vale", "IGA")).toBe(true);
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("validateHeader — nothing is written from a half-read upload", () => {
|
describe("validateHeader — nothing is written from a half-read upload", () => {
|
||||||
@@ -233,68 +221,4 @@ describe("cards carry the C5 action ids and JSON values", () => {
|
|||||||
expect(ids).toContain('"action_id":"intake_grocery"');
|
expect(ids).toContain('"action_id":"intake_grocery"');
|
||||||
expect(ids).toContain('"action_id":"intake_meal"');
|
expect(ids).toContain('"action_id":"intake_meal"');
|
||||||
});
|
});
|
||||||
|
|
||||||
it("every value the card did not write is escaped, plain_text included", () => {
|
|
||||||
const hostile = { ...c, merchant: "Ben & Jerry's <fake>" };
|
|
||||||
const { text, blocks } = candidatesCard(hostile, [
|
|
||||||
{ transaction_id: 42, description: "SQ *BEN & JERRY <AU>", transaction_date: "2026-08-17", amount: "19.90" },
|
|
||||||
]);
|
|
||||||
const sections = JSON.stringify(blocks.filter((b) => (b as Block).type === "section"));
|
|
||||||
expect(sections).toContain("Ben & Jerry's <fake>");
|
|
||||||
expect(sections).toContain("SQ *BEN & JERRY <AU>");
|
|
||||||
expect(text).toContain("Ben & Jerry's <fake>");
|
|
||||||
// plain_text disables formatting, not entity decoding — &, < and > are
|
|
||||||
// control characters everywhere in a Slack payload.
|
|
||||||
const whole = JSON.stringify(blocks);
|
|
||||||
expect(whole).not.toContain("<fake>");
|
|
||||||
expect(whole).not.toContain("<AU>");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("the currency is model output too, so it is escaped like any other", () => {
|
|
||||||
// parseExtraction accepts any string for `currency`; unescaped, `<@U123>`
|
|
||||||
// would be a live mention on every card the batch produces.
|
|
||||||
const { text } = enrichedCard({ ...c, currency: "<@U123>", transactionId: 7 });
|
|
||||||
expect(text).toContain("<@U123> 22.40");
|
|
||||||
expect(text).not.toContain("<@U123>");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a foreign receipt is not printed as dollars — header and candidate rows alike", () => {
|
|
||||||
expect(enrichedCard({ ...c, currency: "USD", transactionId: 7 }).text).toBe(
|
|
||||||
"Added 2 items to your USD 22.40 Uber Eats (Burgertory (Werribee)) order ✓"
|
|
||||||
);
|
|
||||||
expect(enrichedCard({ ...c, currency: "AUD", transactionId: 7 }).text).toContain("$22.40");
|
|
||||||
expect(enrichedCard({ ...c, transactionId: 7 }).text).toContain("$22.40");
|
|
||||||
// A candidate row is a ledger transaction and carries its own currency; it
|
|
||||||
// used to print "$" beneath a header reading "USD 22.40".
|
|
||||||
const { blocks } = candidatesCard({ ...c, currency: "USD" }, [
|
|
||||||
{ transaction_id: 42, description: "Order - Burgertory", transaction_date: "2026-08-17", amount: "19.90", currency: "USD" },
|
|
||||||
]);
|
|
||||||
expect(JSON.stringify(blocks)).toContain("USD 19.90");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("liteLLMExtractor — a finish_reason that is present must be `stop`", () => {
|
|
||||||
const good = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD", items: [], adjustments: [] };
|
|
||||||
const extractor = (finish_reason?: string) =>
|
|
||||||
liteLLMExtractor({
|
|
||||||
baseUrl: "http://litellm.test",
|
|
||||||
key: "k",
|
|
||||||
fetchImpl: (async () =>
|
|
||||||
Response.json({
|
|
||||||
choices: [{ message: { content: JSON.stringify(good) }, ...(finish_reason ? { finish_reason } : {}) }],
|
|
||||||
})) as unknown as typeof fetch,
|
|
||||||
});
|
|
||||||
|
|
||||||
it("accepts `stop`, and — on purpose — a reply that reports no reason at all", async () => {
|
|
||||||
expect((await extractor("stop")([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
|
|
||||||
expect((await extractor()([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("rejects any other reason, naming it — the strict schema only held for a reply that finished", async () => {
|
|
||||||
// An absent reason is NOT rejected: see the comment in intake-extract.ts.
|
|
||||||
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(ExtractionError);
|
|
||||||
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/too long/);
|
|
||||||
await expect(extractor("content_filter")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/content_filter/);
|
|
||||||
await expect(extractor("tool_calls")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/tool_calls/);
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -234,16 +234,6 @@ async function buildDetailsModal(transactionId: number, participantId: number) {
|
|||||||
`SELECT merchant_normalized AS merchant, line_items
|
`SELECT merchant_normalized AS merchant, line_items
|
||||||
FROM expense_metadata
|
FROM expense_metadata
|
||||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||||
-- Ordered the same way as replaceLineItems (line-items.ts) and
|
|
||||||
-- matchTransaction (intake-router.ts), so every reader of this row
|
|
||||||
-- resolves the same one a write lands on. NULLS LAST because DESC is
|
|
||||||
-- NULLS FIRST and the row that merely POINTS at the transaction has a
|
|
||||||
-- NULL transaction_id.
|
|
||||||
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
|
|
||||||
-- pointing-only, and ZERO transactions resolving to more than one row by
|
|
||||||
-- either link — so this changes nothing today. It is here so the readers
|
|
||||||
-- still agree on the day multiplicity appears.
|
|
||||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
|
||||||
LIMIT 1`,
|
LIMIT 1`,
|
||||||
[transactionId]
|
[transactionId]
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -34,14 +34,6 @@ export async function GET(
|
|||||||
-- through matched_transaction_id. Both directions have to resolve or the
|
-- through matched_transaction_id. Both directions have to resolve or the
|
||||||
-- detail is missing on exactly the orders that were paid by card.
|
-- detail is missing on exactly the orders that were paid by card.
|
||||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||||
-- The same order as replaceLineItems in line-items.ts and matchTransaction
|
|
||||||
-- in intake-router.ts. Both directions can resolve at once — a statement
|
|
||||||
-- line owning its own row while a delivery receipt points at it — and an
|
|
||||||
-- unordered LIMIT 1 then reads a different row from the one an enrichment
|
|
||||||
-- wrote, so a PUT that reported success showed nothing on the screen.
|
|
||||||
-- NULLS LAST because DESC is NULLS FIRST and the pointing row's
|
|
||||||
-- transaction_id is NULL.
|
|
||||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
|
||||||
LIMIT 1`,
|
LIMIT 1`,
|
||||||
[Number(id)]
|
[Number(id)]
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -13,9 +13,12 @@ import {
|
|||||||
type OrderReceipt,
|
type OrderReceipt,
|
||||||
type OrderRating,
|
type OrderRating,
|
||||||
} from "@/lib/hooks";
|
} from "@/lib/hooks";
|
||||||
// One platform vocabulary, shared with the intake cards — three copies had
|
|
||||||
// already drifted apart (the intake one was missing `uber`).
|
const PLATFORM_LABEL: Record<string, string> = {
|
||||||
import { platformLabel } from "@/lib/intake-rules";
|
doordash: "DoorDash",
|
||||||
|
ubereats: "Uber Eats",
|
||||||
|
uber: "Uber",
|
||||||
|
};
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Who records verdicts. A two-person household with one primary user: the
|
* Who records verdicts. A two-person household with one primary user: the
|
||||||
@@ -96,7 +99,7 @@ export function OrderDetails({
|
|||||||
<p className="text-xs text-zinc-500">
|
<p className="text-xs text-zinc-500">
|
||||||
Order details
|
Order details
|
||||||
{receipt.platform && (
|
{receipt.platform && (
|
||||||
<span className="ml-1.5 text-zinc-400">{platformLabel(receipt.platform)}</span>
|
<span className="ml-1.5 text-zinc-400">{PLATFORM_LABEL[receipt.platform] ?? receipt.platform}</span>
|
||||||
)}
|
)}
|
||||||
</p>
|
</p>
|
||||||
{receipt.card_last4 && (
|
{receipt.card_last4 && (
|
||||||
|
|||||||
@@ -20,33 +20,3 @@ export async function queryRow<T>(sql: string, params: unknown[] = []): Promise<
|
|||||||
const rows = await prisma.$queryRawUnsafe<T[]>(sql, ...params);
|
const rows = await prisma.$queryRawUnsafe<T[]>(sql, ...params);
|
||||||
return rows.length > 0 ? rows[0] : null;
|
return rows.length > 0 ? rows[0] : null;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The same two calls, scoped to one connection inside one transaction. */
|
|
||||||
export type Queryable = {
|
|
||||||
queryRaw<T>(sql: string, params?: unknown[]): Promise<T[]>;
|
|
||||||
queryRow<T>(sql: string, params?: unknown[]): Promise<T | null>;
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Run several statements as one transaction, on one connection.
|
|
||||||
*
|
|
||||||
* The module-level `queryRaw`/`queryRow` are `$queryRawUnsafe` on a POOLED
|
|
||||||
* client, so each call is its own implicit transaction on whichever connection
|
|
||||||
* the pool hands out. That makes a check-then-write pair non-atomic, and it
|
|
||||||
* makes `pg_advisory_xact_lock` useless outside this helper — the lock is
|
|
||||||
* released the instant the statement that took it commits, which is before the
|
|
||||||
* next statement runs. Anything that needs a lock to still be held on its next
|
|
||||||
* query has to be in here.
|
|
||||||
*/
|
|
||||||
export async function withTransaction<T>(fn: (q: Queryable) => Promise<T>): Promise<T> {
|
|
||||||
return prisma.$transaction(async (tx) => {
|
|
||||||
const q: Queryable = {
|
|
||||||
queryRaw: <R>(sql: string, params: unknown[] = []) => tx.$queryRawUnsafe<R[]>(sql, ...params),
|
|
||||||
queryRow: async <R>(sql: string, params: unknown[] = []) => {
|
|
||||||
const rows = await tx.$queryRawUnsafe<R[]>(sql, ...params);
|
|
||||||
return rows.length > 0 ? rows[0] : null;
|
|
||||||
},
|
|
||||||
};
|
|
||||||
return fn(q);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -58,7 +58,6 @@ const ctxFor = (batch: BatchRow): RouteContext | null =>
|
|||||||
platform: batch.header.platform,
|
platform: batch.header.platform,
|
||||||
header: batch.header,
|
header: batch.header,
|
||||||
items: batch.items ?? [],
|
items: batch.items ?? [],
|
||||||
currency: batch.header.currency,
|
|
||||||
}
|
}
|
||||||
: null;
|
: null;
|
||||||
|
|
||||||
@@ -80,13 +79,13 @@ export async function handleIntakeAction(
|
|||||||
if (actionId === "intake_use_candidate") {
|
if (actionId === "intake_use_candidate") {
|
||||||
if (!Number.isInteger(value.transaction_id)) return note("That button carried no transaction id.");
|
if (!Number.isInteger(value.transaction_id)) return note("That button carried no transaction id.");
|
||||||
const txnId = value.transaction_id as number;
|
const txnId = value.transaction_id as number;
|
||||||
await replaceLineItems(txnId, ctx.items, { source: "intake", platform: ctx.platform, batchId: batch.id, currency: ctx.currency });
|
await replaceLineItems(txnId, ctx.items, { source: "intake", platform: ctx.platform, batchId: batch.id });
|
||||||
await queryRaw(
|
await queryRaw(
|
||||||
`UPDATE intake_batches SET status = 'enriched', kind = COALESCE(kind, 'food'), transaction_id = $2, resolved_at = now() WHERE id = $1::uuid`,
|
`UPDATE intake_batches SET status = 'enriched', kind = COALESCE(kind, 'food'), transaction_id = $2, resolved_at = now() WHERE id = $1::uuid`,
|
||||||
[batch.id, txnId]
|
[batch.id, txnId]
|
||||||
);
|
);
|
||||||
return {
|
return {
|
||||||
...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, currency: ctx.currency, transactionId: txnId }),
|
...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, transactionId: txnId }),
|
||||||
replace: true,
|
replace: true,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
+17
-49
@@ -12,7 +12,7 @@ import { platformLabel } from "@/lib/intake-rules";
|
|||||||
|
|
||||||
export type Block = Record<string, unknown>;
|
export type Block = Record<string, unknown>;
|
||||||
|
|
||||||
export type Candidate = { transaction_id: number; description: string | null; transaction_date: string; amount: string | number; currency?: string | null };
|
export type Candidate = { transaction_id: number; description: string | null; transaction_date: string; amount: string | number };
|
||||||
|
|
||||||
export function financePublicUrl(): string {
|
export function financePublicUrl(): string {
|
||||||
return (process.env.FINANCE_PUBLIC_URL || "https://finance.bosecamp.com").replace(/\/$/, "");
|
return (process.env.FINANCE_PUBLIC_URL || "https://finance.bosecamp.com").replace(/\/$/, "");
|
||||||
@@ -23,29 +23,7 @@ export function transactionLink(merchant: string | null): string {
|
|||||||
return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`;
|
return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
const money = (n: number | string) => `$${Number(n).toFixed(2)}`;
|
||||||
* Slack renders `&`, `<` and `>` as markup in mrkdwn (and in the `text`
|
|
||||||
* fallback). Merchant names, transaction descriptions and model output are
|
|
||||||
* none of ours — `Ben & Jerry's` must not arrive as `Ben & Jerry's`, and a
|
|
||||||
* description containing `<http://…|click>` must not become a link the card
|
|
||||||
* never meant to show. Applied to interpolated values only; our own literal
|
|
||||||
* `*bold*` and `:emoji:` stay markup.
|
|
||||||
*/
|
|
||||||
export function esc(value: string | null | undefined): string {
|
|
||||||
return (value ?? "").replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* `$` is only right for AUD. An intake batch carries the currency the receipt
|
|
||||||
* printed, and the order nudge already renders `USD 15.00` rather than
|
|
||||||
* pretending a foreign order was dollars.
|
|
||||||
*/
|
|
||||||
const money = (n: number | string, currency?: string | null) => {
|
|
||||||
const amount = Number(n).toFixed(2);
|
|
||||||
// The currency comes off the model (`parseExtraction` accepts any string), so
|
|
||||||
// an unescaped `<@U123>` here would be a mention on every card.
|
|
||||||
return !currency || currency === "AUD" ? `$${amount}` : `${esc(currency)} ${amount}`;
|
|
||||||
};
|
|
||||||
const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } });
|
const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } });
|
||||||
const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({
|
const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({
|
||||||
type: "button",
|
type: "button",
|
||||||
@@ -62,12 +40,10 @@ export type CardInput = {
|
|||||||
orderDate: string | null;
|
orderDate: string | null;
|
||||||
platform: string | null;
|
platform: string | null;
|
||||||
itemCount: number;
|
itemCount: number;
|
||||||
currency?: string | null;
|
|
||||||
};
|
};
|
||||||
|
|
||||||
export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
|
export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
|
||||||
const merchant = esc(c.merchant);
|
const what = [c.total !== null ? money(c.total) : null, platformLabel(c.platform) || c.merchant, c.platform && c.merchant ? `(${c.merchant})` : null]
|
||||||
const what = [c.total !== null ? money(c.total, c.currency) : null, esc(platformLabel(c.platform)) || merchant, c.platform && merchant ? `(${merchant})` : null]
|
|
||||||
.filter(Boolean)
|
.filter(Boolean)
|
||||||
.join(" ");
|
.join(" ");
|
||||||
const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`;
|
const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`;
|
||||||
@@ -84,7 +60,7 @@ export function enrichedCard(c: CardInput & { transactionId: number }): { blocks
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
|
export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
|
||||||
const text = `Created a ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant) || "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`;
|
const text = `Created a ${c.total !== null ? money(c.total) + " " : ""}${c.merchant ?? "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`;
|
||||||
return {
|
return {
|
||||||
text,
|
text,
|
||||||
blocks: [
|
blocks: [
|
||||||
@@ -98,30 +74,27 @@ export function createdCard(c: CardInput & { transactionId: number }): { blocks:
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } {
|
export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } {
|
||||||
const label = esc(platformLabel(c.platform));
|
const label = platformLabel(c.platform) || "";
|
||||||
const head = `No matching ${label ? label + " " : ""}order for *${esc(c.merchant) || "this merchant"}* ${c.total !== null ? money(c.total, c.currency) : ""} on ${esc(c.orderDate) || "?"}.`;
|
const head = `No matching ${label ? label + " " : ""}order for *${c.merchant ?? "this merchant"}* ${c.total !== null ? money(c.total) : ""} on ${c.orderDate ?? "?"}.`;
|
||||||
const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`;
|
const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`;
|
||||||
const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)];
|
const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)];
|
||||||
for (const cand of candidates) {
|
for (const cand of candidates) {
|
||||||
blocks.push({
|
blocks.push({
|
||||||
type: "section",
|
type: "section",
|
||||||
text: { type: "mrkdwn", text: `${esc(cand.transaction_date)} · ${esc(cand.description) || "—"} · ${money(cand.amount, cand.currency)}` },
|
text: { type: "mrkdwn", text: `${cand.transaction_date} · ${cand.description ?? "—"} · ${money(cand.amount)}` },
|
||||||
accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }),
|
accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
blocks.push({
|
blocks.push({
|
||||||
type: "actions",
|
type: "actions",
|
||||||
block_id: "intake_create",
|
block_id: "intake_create",
|
||||||
// plain_text disables *formatting*, not entity decoding: `&`, `<` and `>`
|
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total) + " " : ""}${c.merchant ?? ""} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
|
||||||
// are control characters everywhere in a Slack payload, so the label is
|
|
||||||
// escaped like any other value we did not write.
|
|
||||||
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant)} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
|
|
||||||
});
|
});
|
||||||
return { blocks, text };
|
return { blocks, text };
|
||||||
}
|
}
|
||||||
|
|
||||||
export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } {
|
export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } {
|
||||||
const text = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " ");
|
const text = `${c.merchant ?? "Grocery receipt"} ${c.total !== null ? money(c.total) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " ");
|
||||||
return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
|
return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -130,13 +103,13 @@ export function needsHeaderCard(): { blocks: Block[]; text: string } {
|
|||||||
return { text, blocks: [section(`:warning: ${text}`)] };
|
return { text, blocks: [section(`:warning: ${text}`)] };
|
||||||
}
|
}
|
||||||
|
|
||||||
export function heldCard(reason: string, headline = "Items don't add up to the total — is a shot missing?"): { blocks: Block[]; text: string } {
|
export function heldCard(reason: string): { blocks: Block[]; text: string } {
|
||||||
const text = `${headline} (${esc(reason)})`;
|
const text = `Items don't add up to the total — is a shot missing? (${reason})`;
|
||||||
return { text, blocks: [section(`:warning: ${text}`)] };
|
return { text, blocks: [section(`:warning: ${text}`)] };
|
||||||
}
|
}
|
||||||
|
|
||||||
export function askCard(c: CardInput): { blocks: Block[]; text: string } {
|
export function askCard(c: CardInput): { blocks: Block[]; text: string } {
|
||||||
const text = `${esc(c.merchant) || "This upload"} ${c.total !== null ? money(c.total, c.currency) : ""} — groceries or a meal?`.replace(/\s+/g, " ");
|
const text = `${c.merchant ?? "This upload"} ${c.total !== null ? money(c.total) : ""} — groceries or a meal?`.replace(/\s+/g, " ");
|
||||||
return {
|
return {
|
||||||
text,
|
text,
|
||||||
blocks: [
|
blocks: [
|
||||||
@@ -153,23 +126,18 @@ export function askCard(c: CardInput): { blocks: Block[]; text: string } {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null; currency?: string | null }): { blocks: Block[]; text: string } {
|
export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null }): { blocks: Block[]; text: string } {
|
||||||
const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${esc(c.batchId)}` : "an earlier upload";
|
const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${c.batchId}` : "an earlier upload";
|
||||||
// The header (merchant, date, total) IS the identity of a purchase here, so a
|
const text = `Already handled — ${c.merchant ?? "this order"} ${c.total !== null ? money(c.total) : ""} is ${where}. Nothing was changed.`.replace(/\s+/g, " ");
|
||||||
// genuine SECOND identical purchase — two of the same coffee, one day, one shop
|
|
||||||
// — reads as a re-share and is refused. That is the right default (a re-share is
|
|
||||||
// far commoner than a repeat) but it is a dead end without this sentence: the
|
|
||||||
// card must say what to do instead, or the ledger quietly loses the second one.
|
|
||||||
const text = `Already handled — ${esc(c.merchant) || "this order"} ${c.total !== null ? money(c.total, c.currency) : ""} is ${where}. Nothing was changed — if this really is a second, separate purchase, add it in the app.`.replace(/\s+/g, " ");
|
|
||||||
return { text, blocks: [section(`:repeat: ${text}`)] };
|
return { text, blocks: [section(`:repeat: ${text}`)] };
|
||||||
}
|
}
|
||||||
|
|
||||||
export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } {
|
export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } {
|
||||||
const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${esc(detail)})` : ""}`;
|
const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${detail})` : ""}`;
|
||||||
return { text, blocks: [section(`:no_entry_sign: ${text}`)] };
|
return { text, blocks: [section(`:no_entry_sign: ${text}`)] };
|
||||||
}
|
}
|
||||||
|
|
||||||
export function errorCard(detail: string): { blocks: Block[]; text: string } {
|
export function errorCard(detail: string): { blocks: Block[]; text: string } {
|
||||||
const text = `Couldn't process that upload: ${esc(detail)}`;
|
const text = `Couldn't process that upload: ${detail}`;
|
||||||
return { text, blocks: [section(`:x: ${text}`)] };
|
return { text, blocks: [section(`:x: ${text}`)] };
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -185,26 +185,9 @@ export function liteLLMExtractor(opts: {
|
|||||||
});
|
});
|
||||||
if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`);
|
if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`);
|
||||||
const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] };
|
const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] };
|
||||||
// The rule is: a finish_reason that is PRESENT must be `stop`. An absent
|
// A truncated reply can still parse once the JSON is closed, which quietly
|
||||||
// one is accepted, deliberately — providers differ on whether they send the
|
// drops the tail of a long receipt. Losing lines silently is worse than failing.
|
||||||
// field, and failing every call on a provider that omits it would be a
|
if (raw.choices?.[0]?.finish_reason === "length") throw new ExtractionError("The receipt was too long to read in one pass.");
|
||||||
// certain outage in exchange for a case that also requires the model to
|
|
||||||
// emit syntactically complete JSON. The harm a truncated reply does — a
|
|
||||||
// partial item list replacing a good one — is closed on the write side
|
|
||||||
// instead, by the guard in `routeFood`/`createFromBatch`.
|
|
||||||
//
|
|
||||||
// `length` is the common one: a truncated reply still parses once the JSON
|
|
||||||
// happens to close, quietly dropping the tail of a long receipt. Any other
|
|
||||||
// reason (`content_filter`, `tool_calls`) is a different reply shape that
|
|
||||||
// the strict schema never constrained.
|
|
||||||
const finish = raw.choices?.[0]?.finish_reason;
|
|
||||||
if (finish && finish !== "stop") {
|
|
||||||
throw new ExtractionError(
|
|
||||||
finish === "length"
|
|
||||||
? "The receipt was too long to read in one pass."
|
|
||||||
: `The model stopped before it finished (${finish}).`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
const content = raw.choices?.[0]?.message?.content;
|
const content = raw.choices?.[0]?.message?.content;
|
||||||
if (!content) throw new ExtractionError("LiteLLM returned no content");
|
if (!content) throw new ExtractionError("LiteLLM returned no content");
|
||||||
try {
|
try {
|
||||||
|
|||||||
+42
-184
@@ -1,6 +1,5 @@
|
|||||||
import { randomUUID } from "node:crypto";
|
import { randomUUID } from "node:crypto";
|
||||||
import { queryRaw, queryRow, withTransaction } from "@/lib/db";
|
import { queryRaw, queryRow } from "@/lib/db";
|
||||||
import { EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY } from "@/lib/analytics-sql";
|
|
||||||
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
|
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
|
||||||
import {
|
import {
|
||||||
classifyInput,
|
classifyInput,
|
||||||
@@ -208,33 +207,15 @@ export async function findDuplicateBatch(
|
|||||||
return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null;
|
return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null;
|
||||||
}
|
}
|
||||||
|
|
||||||
type MatchRow = {
|
type MatchRow = { transaction_id: number; description: string | null; transaction_date: string; amount: string; merchant: string | null; platform: string | null };
|
||||||
transaction_id: number;
|
|
||||||
description: string | null;
|
|
||||||
transaction_date: string;
|
|
||||||
amount: string;
|
|
||||||
merchant: string | null;
|
|
||||||
platform: string | null;
|
|
||||||
/** What `amount` is denominated in — a candidate row is not always dollars. */
|
|
||||||
currency: string;
|
|
||||||
/** The `expense_metadata` row the match resolved through — null when there is none. */
|
|
||||||
metadata_id: number | null;
|
|
||||||
/** Does that row already carry a receipt? */
|
|
||||||
has_line_items: boolean;
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and
|
* The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and
|
||||||
* date EXACT, never widened silently. The merchant compares normalised, with
|
* date EXACT, never widened silently. Platform is required when the source
|
||||||
* whole-word containment ("Coles" vs "Coles (Wyndham Vale)") — that is
|
* implies one; an in-store receipt has none and matches on the other three.
|
||||||
* spelling, not widening. A unique hit is a match; two hits are an ambiguity
|
* The merchant compares normalised, with containment ("Coles" vs "Coles
|
||||||
* and go to the candidates card like a miss.
|
* (Wyndham Vale)") — that is spelling, not widening. A unique hit is a match;
|
||||||
*
|
* two hits are an ambiguity and go to the candidates card like a miss.
|
||||||
* A NULL platform means "no platform", not "any platform". An in-store Coles
|
|
||||||
* receipt carries none, and `$3 IS NULL OR em.platform = $3` let it match an
|
|
||||||
* Uber Eats Coles order on the same day for the same money — and then overwrite
|
|
||||||
* that order's items with the till receipt's. The in-store side matches rows
|
|
||||||
* with no platform of their own; the delivery side still requires its own.
|
|
||||||
*/
|
*/
|
||||||
export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> {
|
export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> {
|
||||||
if (!header.merchant) return { match: null, ambiguous: false };
|
if (!header.merchant) return { match: null, ambiguous: false };
|
||||||
@@ -242,15 +223,13 @@ export async function matchTransaction(header: OrderHeader, platform: string | n
|
|||||||
`SELECT DISTINCT ON (t.id)
|
`SELECT DISTINCT ON (t.id)
|
||||||
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
|
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
|
||||||
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
|
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
|
||||||
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency,
|
em.platform
|
||||||
COALESCE(jsonb_typeof(em.line_items) = 'array' AND jsonb_array_length(em.line_items) > 0, false) AS has_line_items
|
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
LEFT JOIN statements s ON s.id = t.statement_id
|
|
||||||
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||||
WHERE ${EXCLUDE_RECONCILED_SOURCE}
|
WHERE t.superseded_by_id IS NULL
|
||||||
AND t.transaction_date = $1::date
|
AND t.transaction_date = $1::date
|
||||||
AND t.amount = $2::numeric
|
AND t.amount = $2::numeric
|
||||||
AND (($3::text IS NULL AND em.platform IS NULL) OR em.platform = $3)
|
AND ($3::text IS NULL OR em.platform = $3)
|
||||||
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
|
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
|
||||||
[header.order_date, header.total, platform]
|
[header.order_date, header.total, platform]
|
||||||
);
|
);
|
||||||
@@ -259,29 +238,21 @@ export async function matchTransaction(header: OrderHeader, platform: string | n
|
|||||||
return { match: null, ambiguous: hits.length > 1 };
|
return { match: null, ambiguous: hits.length > 1 };
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/** Nearest candidates for the card: same platform and merchant, ±3 days, any amount. */
|
||||||
* Nearest candidates for the card: same merchant, ±3 days, any amount.
|
export async function findCandidates(header: OrderHeader, platform: string | null): Promise<Candidate[]> {
|
||||||
*
|
|
||||||
* Deliberately NOT filtered by platform. The card is a human choice, not a
|
|
||||||
* decision — the whole reason it is shown is that the exact key missed, so the
|
|
||||||
* platform is one of the things that may be wrong. A filter here narrows the
|
|
||||||
* list at exactly the moment it needs to widen, and the row the user was
|
|
||||||
* looking for is simply absent with nothing saying why.
|
|
||||||
*/
|
|
||||||
export async function findCandidates(header: OrderHeader): Promise<Candidate[]> {
|
|
||||||
if (!header.merchant) return [];
|
if (!header.merchant) return [];
|
||||||
const rows = await queryRaw<MatchRow>(
|
const rows = await queryRaw<MatchRow>(
|
||||||
`SELECT DISTINCT ON (t.id)
|
`SELECT DISTINCT ON (t.id)
|
||||||
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
|
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
|
||||||
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
|
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
|
||||||
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency, false AS has_line_items
|
em.platform
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
LEFT JOIN statements s ON s.id = t.statement_id
|
|
||||||
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||||
WHERE ${EXCLUDE_RECONCILED_SOURCE}
|
WHERE t.superseded_by_id IS NULL
|
||||||
AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3
|
AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3
|
||||||
|
AND ($2::text IS NULL OR em.platform = $2)
|
||||||
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
|
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
|
||||||
[header.order_date]
|
[header.order_date, platform]
|
||||||
);
|
);
|
||||||
return rows
|
return rows
|
||||||
.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant))
|
.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant))
|
||||||
@@ -292,7 +263,7 @@ export async function findCandidates(header: OrderHeader): Promise<Candidate[]>
|
|||||||
return Math.abs(Number(a.amount) - header.total) - Math.abs(Number(b.amount) - header.total);
|
return Math.abs(Number(a.amount) - header.total) - Math.abs(Number(b.amount) - header.total);
|
||||||
})
|
})
|
||||||
.slice(0, 5)
|
.slice(0, 5)
|
||||||
.map((r) => ({ transaction_id: r.transaction_id, description: r.description, transaction_date: r.transaction_date, amount: r.amount, currency: r.currency }));
|
.map((r) => ({ transaction_id: r.transaction_id, description: r.description, transaction_date: r.transaction_date, amount: r.amount }));
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── the two destinations ───────────────────────────────────────────────────
|
// ─── the two destinations ───────────────────────────────────────────────────
|
||||||
@@ -304,7 +275,7 @@ export type RouteContext = {
|
|||||||
platform: string | null;
|
platform: string | null;
|
||||||
header: OrderHeader;
|
header: OrderHeader;
|
||||||
items: StoredLineItem[];
|
items: StoredLineItem[];
|
||||||
currency?: string | null;
|
subtotal?: number | null;
|
||||||
};
|
};
|
||||||
|
|
||||||
const cardInput = (ctx: RouteContext): CardInput => ({
|
const cardInput = (ctx: RouteContext): CardInput => ({
|
||||||
@@ -314,7 +285,6 @@ const cardInput = (ctx: RouteContext): CardInput => ({
|
|||||||
orderDate: ctx.header.order_date,
|
orderDate: ctx.header.order_date,
|
||||||
platform: ctx.platform,
|
platform: ctx.platform,
|
||||||
itemCount: ctx.items.length,
|
itemCount: ctx.items.length,
|
||||||
currency: ctx.currency,
|
|
||||||
});
|
});
|
||||||
|
|
||||||
const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({
|
const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({
|
||||||
@@ -329,44 +299,15 @@ const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card:
|
|||||||
...extra,
|
...extra,
|
||||||
});
|
});
|
||||||
|
|
||||||
/**
|
|
||||||
* Are these items unverifiable — no line carrying any money at all?
|
|
||||||
*
|
|
||||||
* That is `reconcileItems`' `basis: "none"`: pasted text with no prices, or an
|
|
||||||
* extraction that read the names and none of the figures. The reconciliation
|
|
||||||
* check passes it, correctly, because there is nothing to check. What must not
|
|
||||||
* follow is replacing a receipt that IS itemised with one nothing verified.
|
|
||||||
*/
|
|
||||||
const unverifiable = (items: StoredLineItem[]) => items.every((i) => i.amount === null);
|
|
||||||
|
|
||||||
/**
|
|
||||||
* The hold, shared by EVERY path that enriches a matched transaction.
|
|
||||||
*
|
|
||||||
* It lived only in `routeFood` and `createFromBatch`'s re-match walked straight
|
|
||||||
* past it — the same upload destroyed the same receipt depending on which
|
|
||||||
* button reached it. Returns null when there is nothing to protect.
|
|
||||||
*/
|
|
||||||
async function holdRatherThanReplace(ctx: RouteContext, match: MatchRow): Promise<IntakeResult | null> {
|
|
||||||
if (!unverifiable(ctx.items) || !match.has_line_items) return null;
|
|
||||||
await updateBatch(ctx.batchId, { status: "held", items: ctx.items });
|
|
||||||
const card = heldCard(
|
|
||||||
"this upload carries no prices, so nothing checked it against the total — and that order already has its items",
|
|
||||||
"I left the existing items alone"
|
|
||||||
);
|
|
||||||
return finish(ctx, "held", "held", card, { transaction_id: match.transaction_id });
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */
|
/** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */
|
||||||
export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
|
export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
|
||||||
const { match } = await matchTransaction(ctx.header, ctx.platform);
|
const { match } = await matchTransaction(ctx.header, ctx.platform);
|
||||||
if (match) {
|
if (match) {
|
||||||
const held = await holdRatherThanReplace(ctx, match);
|
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId });
|
||||||
if (held) return held;
|
|
||||||
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId, metadataId: match.metadata_id, currency: ctx.currency });
|
|
||||||
await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items });
|
await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items });
|
||||||
return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
|
return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
|
||||||
}
|
}
|
||||||
const candidates = await findCandidates(ctx.header);
|
const candidates = await findCandidates(ctx.header, ctx.platform);
|
||||||
await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items });
|
await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items });
|
||||||
return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates });
|
return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates });
|
||||||
}
|
}
|
||||||
@@ -389,7 +330,7 @@ export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantr
|
|||||||
} catch (e) {
|
} catch (e) {
|
||||||
if (e instanceof PantryDuplicateError) {
|
if (e instanceof PantryDuplicateError) {
|
||||||
await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items });
|
await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items });
|
||||||
return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, currency: ctx.currency, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), {
|
return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), {
|
||||||
capture_id: e.captureId ?? undefined,
|
capture_id: e.captureId ?? undefined,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -404,121 +345,38 @@ export async function createFromBatch(batch: BatchRow, kind: "grocery" | "food")
|
|||||||
if (!batch.header) throw new Error("batch has no header");
|
if (!batch.header) throw new Error("batch has no header");
|
||||||
const header = batch.header;
|
const header = batch.header;
|
||||||
const items = batch.items ?? [];
|
const items = batch.items ?? [];
|
||||||
const ctx: RouteContext = {
|
|
||||||
batchId: batch.id,
|
|
||||||
source: batch.source,
|
|
||||||
sharedBy: batch.shared_by,
|
|
||||||
platform: header.platform,
|
|
||||||
header,
|
|
||||||
items,
|
|
||||||
currency: header.currency,
|
|
||||||
};
|
|
||||||
|
|
||||||
// The card was built minutes ago and the tap arrives now. In between, the
|
|
||||||
// email/order bridge books orders on its own schedule — so re-run the match
|
|
||||||
// rather than trusting the "no match" the card was drawn from. Creating a
|
|
||||||
// second row for an order the ledger already holds is the one outcome this
|
|
||||||
// whole slice exists to avoid.
|
|
||||||
const { match } = await matchTransaction(header, header.platform);
|
|
||||||
if (match) {
|
|
||||||
const held = await holdRatherThanReplace(ctx, match);
|
|
||||||
if (held) return held;
|
|
||||||
await replaceLineItems(match.transaction_id, items, { source: "intake", platform: header.platform, batchId: batch.id, metadataId: match.metadata_id, currency: header.currency });
|
|
||||||
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: match.transaction_id, items });
|
|
||||||
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
|
|
||||||
}
|
|
||||||
|
|
||||||
// A sibling batch of the same order — the PDF after the screenshot — may have
|
|
||||||
// been resolved since. `findDuplicateBatch` deliberately ignores unresolved
|
|
||||||
// batches, so two pending cards for one order both reach here; without this
|
|
||||||
// the second tap books the order twice.
|
|
||||||
const twin = await findDuplicateBatch(batch.id, batch.file_sha256, header);
|
|
||||||
if (twin) {
|
|
||||||
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: twin.transaction_id, capture_id: twin.capture_id });
|
|
||||||
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: twin.transaction_id, captureId: twin.capture_id, batchId: twin.id });
|
|
||||||
return finish(ctx, kind, "duplicate", card, {
|
|
||||||
transaction_id: twin.transaction_id ?? undefined,
|
|
||||||
capture_id: twin.capture_id ?? undefined,
|
|
||||||
duplicate_of: { batch_id: twin.id, transaction_id: twin.transaction_id, capture_id: twin.capture_id },
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const ownerId = await ownerForEmail(batch.shared_by);
|
const ownerId = await ownerForEmail(batch.shared_by);
|
||||||
const merchant = header.merchant ?? "Unknown merchant";
|
const merchant = header.merchant ?? "Unknown merchant";
|
||||||
// Not `orderDescription()`: it always prefixes "Order - ", which is right for
|
|
||||||
// a delivery receipt and wrong for the in-store till receipt this path also
|
|
||||||
// handles. Not `resolveCategory()` either — it re-derives grocery-vs-food
|
|
||||||
// from a merchant pattern, and `kind` is the answer the router already
|
|
||||||
// reached, remembered merchant and Slack tap included.
|
|
||||||
const description = header.platform ? `Order - ${merchant}` : merchant;
|
const description = header.platform ? `Order - ${merchant}` : merchant;
|
||||||
const category = kind === "grocery" ? "groceries" : "dining";
|
const category = kind === "grocery" ? "groceries" : "dining";
|
||||||
// `amount` is the figure printed on the receipt, in its own currency. No FX
|
// source/source_ref make a second press of the same card a no-op rather than a second row.
|
||||||
// rate exists at intake, so a foreign order leaves `amount_aud` NULL rather
|
const existing = await queryRow<{ id: number }>(`SELECT id FROM transactions WHERE source = 'intake' AND source_ref = $1`, [batch.id]);
|
||||||
// than asserting a conversion — the same shape order-ingestion.ts writes.
|
const txnId =
|
||||||
const currency = header.currency ?? "AUD";
|
existing?.id ??
|
||||||
const isAud = currency === "AUD";
|
(
|
||||||
// Check and insert under one lock, on one connection.
|
await queryRow<{ id: number }>(
|
||||||
//
|
`INSERT INTO transactions (
|
||||||
// `source_ref = batch.id` makes a second press of THIS card a no-op, and
|
transaction_date, description, amount, amount_aud, category, payment_method,
|
||||||
// ON CONFLICT makes that safe against a double tap. It cannot help two
|
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref
|
||||||
// DIFFERENT batches for one purchase — their keys differ — and both taps run
|
) VALUES ($1::date, $2, $3::numeric, $3::numeric, $4, NULL, $5, $5, 'debit', $6, 'intake', $7)
|
||||||
// their match and duplicate checks before either inserts, so both can find
|
RETURNING id`,
|
||||||
// nothing. The lock is keyed on the purchase, not the batch, which is the
|
[header.order_date, description, header.total, category, merchant, ownerId, batch.id]
|
||||||
// only thing the two share. It has to be transaction-scoped and therefore
|
)
|
||||||
// inside withTransaction: taken by a statement of its own it would be
|
)!.id;
|
||||||
// released before the next one ran (see db.ts).
|
await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id });
|
||||||
const purchaseKey = [header.platform ?? "", merchantKey(merchant), header.order_date, header.total.toFixed(2)].join("|");
|
|
||||||
const outcome = await withTransaction(async (q) => {
|
|
||||||
await q.queryRaw(`SELECT pg_advisory_xact_lock(hashtextextended($1, 0))`, [purchaseKey]);
|
|
||||||
// Mine (a double tap) or another batch's (the race). The ORDER BY is what
|
|
||||||
// tells them apart when both exist.
|
|
||||||
const existing = await q.queryRow<{ id: number; source_ref: string }>(
|
|
||||||
`SELECT id, source_ref FROM transactions
|
|
||||||
WHERE source = 'intake' AND superseded_by_id IS NULL
|
|
||||||
AND (source_ref = $1
|
|
||||||
OR (transaction_date = $2::date AND amount = $3::numeric
|
|
||||||
AND lower(merchant_normalized) = lower($4)))
|
|
||||||
ORDER BY (source_ref = $1) DESC NULLS LAST, id
|
|
||||||
LIMIT 1`,
|
|
||||||
[batch.id, header.order_date, header.total, merchant]
|
|
||||||
);
|
|
||||||
if (existing) return { id: existing.id, mine: existing.source_ref === batch.id };
|
|
||||||
const row = await q.queryRow<{ id: number }>(
|
|
||||||
`INSERT INTO transactions (
|
|
||||||
transaction_date, description, amount, amount_aud, category, payment_method,
|
|
||||||
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref,
|
|
||||||
foreign_currency_amount, foreign_currency_code
|
|
||||||
) VALUES ($1::date, $2, $3::numeric, $4::numeric, $5, NULL, $6, $6, 'debit', $7, 'intake', $8, $9::numeric, $10)
|
|
||||||
ON CONFLICT (source, source_ref) WHERE source IS NOT NULL AND source_ref IS NOT NULL
|
|
||||||
DO NOTHING
|
|
||||||
RETURNING id`,
|
|
||||||
[header.order_date, description, header.total, isAud ? header.total : null, category, merchant, ownerId, batch.id, isAud ? null : header.total, isAud ? null : currency]
|
|
||||||
);
|
|
||||||
return { id: row!.id, mine: true };
|
|
||||||
});
|
|
||||||
|
|
||||||
// Another batch booked this purchase in the moment between the two taps. Say
|
|
||||||
// the same thing the sequential case says one second later, and write nothing.
|
|
||||||
if (!outcome.mine) {
|
|
||||||
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: outcome.id });
|
|
||||||
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: outcome.id, captureId: null, batchId: null });
|
|
||||||
return finish(ctx, kind, "duplicate", card, { transaction_id: outcome.id });
|
|
||||||
}
|
|
||||||
|
|
||||||
const txnId = outcome.id;
|
|
||||||
await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id, currency });
|
|
||||||
await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId });
|
await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId });
|
||||||
|
const ctx: RouteContext = { batchId: batch.id, source: batch.source, sharedBy: batch.shared_by, platform: header.platform, header, items };
|
||||||
return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId });
|
return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId });
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The card for a batch as it stands — what a second press of a resolved card gets back. */
|
/** The card for a batch as it stands — what a second press of a resolved card gets back. */
|
||||||
export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } {
|
export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } {
|
||||||
const h = batch.header;
|
const h = batch.header;
|
||||||
const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0, currency: h?.currency };
|
const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0 };
|
||||||
if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id });
|
if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id });
|
||||||
if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id });
|
if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id });
|
||||||
if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id });
|
if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id });
|
||||||
return duplicateCard({ merchant: base.merchant, total: base.total, currency: base.currency, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id });
|
return duplicateCard({ merchant: base.merchant, total: base.total, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id });
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── the router ─────────────────────────────────────────────────────────────
|
// ─── the router ─────────────────────────────────────────────────────────────
|
||||||
@@ -580,7 +438,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
|
|||||||
const dupBySha = await findDuplicateBatch(batchId, shas, null);
|
const dupBySha = await findDuplicateBatch(batchId, shas, null);
|
||||||
if (dupBySha) {
|
if (dupBySha) {
|
||||||
await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id });
|
await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id });
|
||||||
return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, currency: dupBySha.header?.currency, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, {
|
return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, {
|
||||||
kind: (dupBySha.kind as IntakeKind) ?? "food",
|
kind: (dupBySha.kind as IntakeKind) ?? "food",
|
||||||
duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id },
|
duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id },
|
||||||
});
|
});
|
||||||
@@ -620,7 +478,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
|
|||||||
const dup = await findDuplicateBatch(batchId, shas, storedHeader);
|
const dup = await findDuplicateBatch(batchId, shas, storedHeader);
|
||||||
if (dup) {
|
if (dup) {
|
||||||
await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id });
|
await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id });
|
||||||
return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, currency: storedHeader.currency, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, {
|
return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, {
|
||||||
header,
|
header,
|
||||||
items,
|
items,
|
||||||
duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id },
|
duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id },
|
||||||
@@ -635,7 +493,7 @@ export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<I
|
|||||||
}
|
}
|
||||||
|
|
||||||
await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" });
|
await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" });
|
||||||
const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, currency: storedHeader.currency };
|
const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, subtotal: extracted.subtotal };
|
||||||
|
|
||||||
// 7. act.
|
// 7. act.
|
||||||
if (kind === "ask") {
|
if (kind === "ask") {
|
||||||
|
|||||||
+8
-32
@@ -43,7 +43,6 @@ export function platformForSource(source: IntakeSource): string | null {
|
|||||||
export const PLATFORM_LABEL: Record<string, string> = {
|
export const PLATFORM_LABEL: Record<string, string> = {
|
||||||
ubereats: "Uber Eats",
|
ubereats: "Uber Eats",
|
||||||
doordash: "DoorDash",
|
doordash: "DoorDash",
|
||||||
uber: "Uber",
|
|
||||||
};
|
};
|
||||||
|
|
||||||
export function platformLabel(platform: string | null): string {
|
export function platformLabel(platform: string | null): string {
|
||||||
@@ -52,16 +51,10 @@ export function platformLabel(platform: string | null): string {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints
|
* A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints
|
||||||
* `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; a
|
* `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; substring on
|
||||||
* word-boundary alternation on the normalised name covers both without a
|
* the normalised name covers both without a merchant master-data subsystem.
|
||||||
* merchant master-data subsystem.
|
|
||||||
*
|
|
||||||
* The boundaries are load-bearing, not tidiness: a bare substring made
|
|
||||||
* `Bigals Pizza` a grocer, because "b-IGA-ls" contains `iga`. Short names in a
|
|
||||||
* substring list collide with ordinary words, and the collision reads as a
|
|
||||||
* correct classification.
|
|
||||||
*/
|
*/
|
||||||
export const GROCERY_PATTERN = /\b(coles|woolworths|woolies|aldi|iga|foodworks|costco)\b/i;
|
export const GROCERY_PATTERN = /coles|woolworths|woolies|aldi|iga|foodworks|costco/i;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The normalised merchant used for matching and for the remembered-kind table.
|
* The normalised merchant used for matching and for the remembered-kind table.
|
||||||
@@ -226,27 +219,10 @@ export function toLineItems(items: ExtractedItem[]): StoredLineItem[] {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Is `needle` a contiguous run of whole words inside `hay`? */
|
/** Does the stored merchant name denote the extracted one? Exact key, or one contains the other. */
|
||||||
function containsWordRun(hay: string[], needle: string[]): boolean {
|
|
||||||
if (!needle.length || needle.length > hay.length) return false;
|
|
||||||
for (let i = 0; i + needle.length <= hay.length; i++) {
|
|
||||||
if (needle.every((w, j) => hay[i + j] === w)) return true;
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Does the stored merchant name denote the extracted one? One name's words must
|
|
||||||
* appear as a contiguous run inside the other's — so `Coles` matches
|
|
||||||
* `Coles (Wyndham Vale)` and `Order - Coles (Wyndham Vale)`, which is spelling,
|
|
||||||
* not widening.
|
|
||||||
*
|
|
||||||
* Whole words, never a raw substring: `merchantKey("Bigals Pizza")` contains
|
|
||||||
* `iga`, so a pizza shop matched every IGA on the same date and total.
|
|
||||||
*/
|
|
||||||
export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean {
|
export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean {
|
||||||
const a = merchantKey(stored).split(" ").filter(Boolean);
|
const a = merchantKey(stored);
|
||||||
const b = merchantKey(extracted).split(" ").filter(Boolean);
|
const b = merchantKey(extracted);
|
||||||
if (!a.length || !b.length) return false;
|
if (!a || !b) return false;
|
||||||
return containsWordRun(a, b) || containsWordRun(b, a);
|
return a === b || a.includes(b) || b.includes(a);
|
||||||
}
|
}
|
||||||
|
|||||||
+21
-97
@@ -1,4 +1,4 @@
|
|||||||
import { queryRow } from "@/lib/db";
|
import { queryRow, queryRaw } from "@/lib/db";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Replace the itemised receipt behind a transaction (plan §11 C2).
|
* Replace the itemised receipt behind a transaction (plan §11 C2).
|
||||||
@@ -15,12 +15,6 @@ import { queryRow } from "@/lib/db";
|
|||||||
* transaction has no row at all (a manual transaction, a statement line), one
|
* transaction has no row at all (a manual transaction, a statement line), one
|
||||||
* is created so the items have somewhere to live; a transaction is NEVER
|
* is created so the items have somewhere to live; a transaction is NEVER
|
||||||
* created here.
|
* created here.
|
||||||
*
|
|
||||||
* `opts.metadataId` pins WHICH row when the caller already resolved one. A
|
|
||||||
* statement transaction can own its own `expense_metadata` row AND be the
|
|
||||||
* `matched_transaction_id` of a delivery row: the match query and this write
|
|
||||||
* then re-pick independently and can land on different rows, so the enrichment
|
|
||||||
* reports success against a row nobody reads.
|
|
||||||
*/
|
*/
|
||||||
|
|
||||||
export type LineItemsSource = "intake" | "pantry";
|
export type LineItemsSource = "intake" | "pantry";
|
||||||
@@ -61,35 +55,10 @@ export type ReplaceResult = {
|
|||||||
line_items: LineItemRecord[];
|
line_items: LineItemRecord[];
|
||||||
};
|
};
|
||||||
|
|
||||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* The undo baseline, recorded once per batch.
|
|
||||||
*
|
|
||||||
* Returning the row that was just read makes a RETRY destroy the thing undo
|
|
||||||
* exists to restore: pantry re-calls this on every confirm and after a timeout,
|
|
||||||
* and the second call would hand back its own write as the "previous" state.
|
|
||||||
* The first snapshot a batch ever takes is the only one worth keeping, so it is
|
|
||||||
* written under COALESCE and read back rather than trusted.
|
|
||||||
*/
|
|
||||||
async function undoBaseline(batchId: string | null | undefined, snapshot: unknown[]): Promise<unknown[]> {
|
|
||||||
// Pantry passes back whatever batch id it was given; only an intake batch has
|
|
||||||
// a row to record against, and a non-uuid would fail the cast.
|
|
||||||
if (!batchId || !UUID.test(batchId)) return snapshot;
|
|
||||||
const row = await queryRow<{ previous_line_items: unknown }>(
|
|
||||||
`UPDATE intake_batches
|
|
||||||
SET previous_line_items = COALESCE(previous_line_items, $2::jsonb)
|
|
||||||
WHERE id = $1::uuid
|
|
||||||
RETURNING previous_line_items`,
|
|
||||||
[batchId, JSON.stringify(snapshot)]
|
|
||||||
);
|
|
||||||
return Array.isArray(row?.previous_line_items) ? row.previous_line_items : snapshot;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function replaceLineItems(
|
export async function replaceLineItems(
|
||||||
transactionId: number,
|
transactionId: number,
|
||||||
lineItems: LineItemRecord[],
|
lineItems: LineItemRecord[],
|
||||||
opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null; metadataId?: number | null; currency?: string | null }
|
opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null }
|
||||||
): Promise<ReplaceResult> {
|
): Promise<ReplaceResult> {
|
||||||
const txn = await queryRow<{
|
const txn = await queryRow<{
|
||||||
id: number;
|
id: number;
|
||||||
@@ -104,78 +73,36 @@ export async function replaceLineItems(
|
|||||||
);
|
);
|
||||||
if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`);
|
if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`);
|
||||||
|
|
||||||
// The caller's row wins where it still exists; otherwise prefer the row that
|
// Prefer the row that IS this transaction over one that merely points at it —
|
||||||
// IS this transaction over one that merely points at it — a statement line
|
// a statement line can be the `matched_transaction_id` of a receipt row and
|
||||||
// can be the `matched_transaction_id` of a receipt row and carry its own row
|
// carry its own row too, and the items belong on the receipt.
|
||||||
// too, and the items belong on the receipt.
|
|
||||||
const pinned = opts.metadataId
|
|
||||||
? await queryRow<{ id: number; line_items: unknown }>(
|
|
||||||
`SELECT id, line_items FROM expense_metadata
|
|
||||||
WHERE id = $1 AND (transaction_id = $2 OR matched_transaction_id = $2)`,
|
|
||||||
[opts.metadataId, transactionId]
|
|
||||||
)
|
|
||||||
: null;
|
|
||||||
|
|
||||||
const json = JSON.stringify(lineItems);
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Snapshot the baseline, then write. In that order on BOTH paths: the create
|
|
||||||
* path used to snapshot after its INSERT, so a crash in between left a retry
|
|
||||||
* recording the intake's own write as "previous" — undo restoring the thing
|
|
||||||
* it was meant to reverse, which is the failure `previous_line_items` exists
|
|
||||||
* to prevent.
|
|
||||||
*
|
|
||||||
* Returns null when the row is gone between the read and the write. That
|
|
||||||
* UPDATE reported success and the batch was marked enriched against a row
|
|
||||||
* nothing had written to.
|
|
||||||
*/
|
|
||||||
const write = async (row: { id: number; line_items: unknown }): Promise<ReplaceResult | null> => {
|
|
||||||
const previous = await undoBaseline(opts.batchId, Array.isArray(row.line_items) ? row.line_items : []);
|
|
||||||
const updated = await queryRow<{ id: number }>(
|
|
||||||
`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1 RETURNING id`,
|
|
||||||
[row.id, json]
|
|
||||||
);
|
|
||||||
if (!updated) return null;
|
|
||||||
return { transaction_id: transactionId, metadata_id: row.id, created_metadata: false, previous_line_items: previous, line_items: lineItems };
|
|
||||||
};
|
|
||||||
|
|
||||||
if (pinned) {
|
|
||||||
const done = await write(pinned);
|
|
||||||
if (done) return done;
|
|
||||||
}
|
|
||||||
const existing = await queryRow<{ id: number; line_items: unknown }>(
|
const existing = await queryRow<{ id: number; line_items: unknown }>(
|
||||||
`SELECT id, line_items FROM expense_metadata
|
`SELECT id, line_items FROM expense_metadata
|
||||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||||
-- NULLS LAST is load-bearing, and its absence inverted this clause.
|
ORDER BY (transaction_id = $1) DESC, id
|
||||||
-- Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at
|
|
||||||
-- the transaction has a NULL transaction_id — so the comparison meant to
|
|
||||||
-- prefer the owned row preferred the other one, silently and in exactly
|
|
||||||
-- the case it was written for. matchTransaction spells it out; this did
|
|
||||||
-- not, and the two therefore disagreed.
|
|
||||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
|
||||||
LIMIT 1`,
|
LIMIT 1`,
|
||||||
[transactionId]
|
[transactionId]
|
||||||
);
|
);
|
||||||
|
|
||||||
|
const json = JSON.stringify(lineItems);
|
||||||
if (existing) {
|
if (existing) {
|
||||||
const done = await write(existing);
|
await queryRaw(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1`, [existing.id, json]);
|
||||||
if (done) return done;
|
return {
|
||||||
|
transaction_id: transactionId,
|
||||||
|
metadata_id: existing.id,
|
||||||
|
created_metadata: false,
|
||||||
|
previous_line_items: Array.isArray(existing.line_items) ? existing.line_items : [],
|
||||||
|
line_items: lineItems,
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
// Nothing to update. Snapshot the empty baseline BEFORE creating the row.
|
|
||||||
const previous = await undoBaseline(opts.batchId, []);
|
|
||||||
const reference = opts.batchId ? `intake:${opts.batchId}` : `intake:txn:${transactionId}`;
|
const reference = opts.batchId ? `intake:${opts.batchId}` : `intake:txn:${transactionId}`;
|
||||||
// ON CONFLICT because two calls for one transaction can both find no row and
|
const created = await queryRow<{ id: number }>(
|
||||||
// both INSERT; expense_metadata.transaction_id is UNIQUE, so one of them lost
|
|
||||||
// with a constraint error that reached the user as an error card. `xmax = 0`
|
|
||||||
// is how a plain insert is told from a conflict-update, so the reported
|
|
||||||
// `created_metadata` stays true.
|
|
||||||
const created = await queryRow<{ id: number; inserted: boolean }>(
|
|
||||||
`INSERT INTO expense_metadata (
|
`INSERT INTO expense_metadata (
|
||||||
transaction_id, source, platform, order_reference, line_items,
|
transaction_id, source, platform, order_reference, line_items,
|
||||||
merchant_normalized, amount, transaction_date, currency, flags, extraction_model
|
merchant_normalized, amount, transaction_date, currency, flags, extraction_model
|
||||||
) VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7::numeric, $8::date, $9, '[]'::jsonb, 'cloud-budget')
|
) VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7::numeric, $8::date, 'AUD', '[]'::jsonb, 'cloud-budget')
|
||||||
ON CONFLICT (transaction_id) DO UPDATE SET line_items = EXCLUDED.line_items
|
RETURNING id`,
|
||||||
RETURNING id, (xmax = 0) AS inserted`,
|
|
||||||
[
|
[
|
||||||
transactionId,
|
transactionId,
|
||||||
opts.source,
|
opts.source,
|
||||||
@@ -185,16 +112,13 @@ export async function replaceLineItems(
|
|||||||
txn.merchant_normalized ?? txn.merchant_name ?? null,
|
txn.merchant_normalized ?? txn.merchant_name ?? null,
|
||||||
txn.amount,
|
txn.amount,
|
||||||
txn.transaction_date,
|
txn.transaction_date,
|
||||||
// Not hard-coded 'AUD': the transaction may carry a foreign figure, and
|
|
||||||
// OrderDetails prefers this column, so a USD order printed "$".
|
|
||||||
opts.currency || "AUD",
|
|
||||||
]
|
]
|
||||||
);
|
);
|
||||||
return {
|
return {
|
||||||
transaction_id: transactionId,
|
transaction_id: transactionId,
|
||||||
metadata_id: created!.id,
|
metadata_id: created!.id,
|
||||||
created_metadata: created!.inserted,
|
created_metadata: true,
|
||||||
previous_line_items: previous,
|
previous_line_items: [],
|
||||||
line_items: lineItems,
|
line_items: lineItems,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,6 +12,13 @@ export const CUTOVER_DATE = "2026-01-09";
|
|||||||
*/
|
*/
|
||||||
export const DEFAULT_OWNER_ID = 1;
|
export const DEFAULT_OWNER_ID = 1;
|
||||||
|
|
||||||
|
/** Human labels for the platform a receipt came from. */
|
||||||
|
export const PLATFORM_LABEL: Record<ParsedOrder["platform"], string> = {
|
||||||
|
doordash: "DoorDash",
|
||||||
|
ubereats: "Uber Eats",
|
||||||
|
uber: "Uber",
|
||||||
|
};
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Transaction description.
|
* Transaction description.
|
||||||
*
|
*
|
||||||
|
|||||||
@@ -119,16 +119,6 @@ export async function merchantForTransaction(
|
|||||||
const row = await queryRow<{ merchant_normalized: string | null }>(
|
const row = await queryRow<{ merchant_normalized: string | null }>(
|
||||||
`SELECT merchant_normalized FROM expense_metadata
|
`SELECT merchant_normalized FROM expense_metadata
|
||||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||||
-- Ordered the same way as replaceLineItems (line-items.ts) and
|
|
||||||
-- matchTransaction (intake-router.ts), so every reader of this row
|
|
||||||
-- resolves the same one a write lands on. NULLS LAST because DESC is
|
|
||||||
-- NULLS FIRST and the row that merely POINTS at the transaction has a
|
|
||||||
-- NULL transaction_id.
|
|
||||||
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
|
|
||||||
-- pointing-only, and ZERO transactions resolving to more than one row by
|
|
||||||
-- either link — so this changes nothing today. It is here so the readers
|
|
||||||
-- still agree on the day multiplicity appears.
|
|
||||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
|
||||||
LIMIT 1`,
|
LIMIT 1`,
|
||||||
[transactionId]
|
[transactionId]
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -367,16 +367,6 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
|
|||||||
SELECT em.route, em.platform
|
SELECT em.route, em.platform
|
||||||
FROM expense_metadata em
|
FROM expense_metadata em
|
||||||
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||||
-- Ordered the same way as replaceLineItems (line-items.ts) and
|
|
||||||
-- matchTransaction (intake-router.ts), so every reader of this row
|
|
||||||
-- resolves the same one a write lands on. NULLS LAST because DESC is
|
|
||||||
-- NULLS FIRST and the row that merely POINTS at the transaction has a
|
|
||||||
-- NULL transaction_id.
|
|
||||||
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
|
|
||||||
-- pointing-only, and ZERO transactions resolving to more than one row by
|
|
||||||
-- either link — so this changes nothing today. It is here so the readers
|
|
||||||
-- still agree on the day multiplicity appears.
|
|
||||||
ORDER BY (em.transaction_id = t.id) DESC NULLS LAST, em.id
|
|
||||||
LIMIT 1
|
LIMIT 1
|
||||||
) order_ctx ON true
|
) order_ctx ON true
|
||||||
LEFT JOIN LATERAL (
|
LEFT JOIN LATERAL (
|
||||||
|
|||||||
Reference in New Issue
Block a user