siddharthd/finance-app
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: siddharthd/finance-app:b958d052876a3e3e6165537f460739427d420922
Branches Tags
siddharthd/finance-app:main siddharthd/finance-app:feat/intake-router siddharthd/finance-app:feat/split-scopes siddharthd/finance-app:feat/order-ingestion siddharthd/finance-app:feature/phase6-budget siddharthd/finance-app:feature/phase5-rules
...
compare: siddharthd/finance-app:d88ca2a24c61503b65b9407d436e7b5ace56d8cb
Branches Tags
siddharthd/finance-app:main siddharthd/finance-app:feat/intake-router siddharthd/finance-app:feat/split-scopes siddharthd/finance-app:feat/order-ingestion siddharthd/finance-app:feature/phase6-budget siddharthd/finance-app:feature/phase5-rules
18 Commits
b958d05287 ... d88ca2a24c
Author SHA1 Message Date
siddharthd d88ca2a24c intake: a duplicate card that refuses a purchase must say what to do instead
ci / lint-test (push) Successful in 1m5s
Details 
The header (merchant, date, total) is a purchase's identity here, so a genuine
second identical purchase — two of the same coffee, one shop, one day — reads
as a re-share and is refused. That default is right; a re-share is far commoner
than a repeat. But the card said only "Nothing was changed", which is a dead
end: the money is real and the ledger silently never gets it. Say to add it in
the app.
 2026-08-21 18:27:53 +10:00
siddharthd 91586ce4a4 test(intake): the second review pass, and CLAUDE.md for the rules it set
ci / lint-test (push) Successful in 45s
Details 
Six of the nine fixes have a test that fails against the previous tree: the
metadata row's currency, the Create-path hold, two concurrent batches booking
once, mrkdwn escaping including plain_text, the escaped currency, and
candidate-row currency.

Three do not, and the reason is the same each time — the window needs a crash
or a lost race that a test cannot force. Asserted instead: a stale metadataId
falls back to the live row rather than reporting a phantom write; two writes
racing to create a metadata row both resolve to one; the create path records a
baseline at all. All three passed before as well.
 2026-08-21 18:23:52 +10:00
siddharthd 4b09d4136f intake: lock the purchase, not the batch, and hold on both write paths 
Two different unresolved batches for one purchase could each create a
transaction. ON CONFLICT (source, source_ref) cannot stop it — their source_refs
differ — and both taps clear their match and duplicate checks before either
inserts. The create now takes a transaction-scoped advisory lock keyed on
(platform, merchant, date, total), re-checks inside it, and inserts; the loser
reports `duplicate` against the winner's transaction, the same answer the
sequential path gives.

The lock has to be inside a real transaction, which this app did not have.
queryRaw is $queryRawUnsafe on a POOLED client, so every call is its own
implicit transaction and pg_advisory_xact_lock taken in a statement of its own
is released before the next statement runs — the lock would have looked right
and held nothing. withTransaction() in db.ts is the scope; the integration
helper mirrors it with BEGIN/COMMIT on one pooled connection.

Also: the "hold rather than replace an itemised receipt with an unpriced
upload" guard lived only in routeFood, and createFromBatch's re-match walked
straight past it — the same upload destroyed the same receipt depending on
which button reached it. One function, both callers.
 2026-08-21 18:23:51 +10:00
siddharthd 4f0f0dd285 intake: escape the currency and the button label too 
money() interpolated the extracted currency raw, and parseExtraction accepts
any string for it — so "<@U123>" would have been a live mention on every card
the batch produced. And plain_text disables *formatting*, not entity decoding:
&, < and > are control characters everywhere in a Slack payload, so the Create
button's label is escaped like any other value we did not write. My earlier
note claiming the opposite was wrong.

Candidate rows also carry their transaction's currency (NATIVE_CURRENCY) rather
than always printing "$" under a header reading "USD 22.40".
 2026-08-21 18:23:51 +10:00
siddharthd b788b4e8fa line-items: snapshot before writing, and stop reporting writes that did not happen 
Three faults on the same function, all of them "the write is assumed to have
worked".

- The create path snapshotted the undo baseline AFTER its INSERT. A crash in
  between left the retry recording the intake's own write as "previous", so
  undo restored the thing it was meant to reverse — the exact failure
  previous_line_items exists to prevent. Both paths now snapshot first.
- An UPDATE against a row deleted between the select and the write affects
  nothing and still returned a metadata_id, so the batch was marked enriched
  against a row nobody had written to. The write is now checked and falls
  through to the ordered lookup, then to a create.
- Two calls for one transaction could both find no row and both INSERT;
  expense_metadata.transaction_id is UNIQUE, so one lost with a constraint
  error that reached the user as an error card. ON CONFLICT (transaction_id)
  DO UPDATE, with (xmax = 0) so created_metadata stays truthful.

And the new row's currency comes from the batch instead of a hard-coded 'AUD'.
OrderDetails prefers expense_metadata.currency, so a USD order — booked
correctly on the transaction with foreign_currency_code — printed "$".
 2026-08-21 18:23:32 +10:00
siddharthd b5569b8ccc schema: previous_line_items on the model; say what the finish_reason rule is 
Migration 0034 has the column and the Prisma model did not, so a regenerated
client and the database disagreed.

The extraction comment claimed only a finished reply is accepted while the code
accepts an absent finish_reason. The code is right — failing every call on a
provider that omits an optional field would be a certain outage in exchange for
a case that also requires syntactically complete JSON, and the harm a truncated
reply does is closed on the write side. The prose now states the rule as it is:
a reason that is PRESENT must be `stop`.
 2026-08-21 18:23:32 +10:00
siddharthd d4ff68363a expense_metadata: every reader resolves the row a write lands on 
getTransactions' provenance LATERAL, merchantForTransaction and the Slack
details modal all select on transaction_id = $1 OR matched_transaction_id = $1
with a bare LIMIT 1, so they could read a different row than replaceLineItems
writes. Same ORDER BY as the match, the write and the order-detail GET.

Defensive, not a live bug: measured 2026-08-21, expense_metadata holds 667
rows, 147 of them pointing-only, and ZERO transactions resolve to more than one
row by either link. This changes nothing today. It is here so the readers still
agree on the day multiplicity appears, which is the only condition under which
any of them was ever wrong.
 2026-08-21 18:23:31 +10:00
siddharthd 631e75e58f test(intake): cover the review findings, and CLAUDE.md for the rules they set
ci / lint-test (push) Successful in 59s
Details 
Unit: word-boundary merchant matching (Bigals Pizza is not an IGA), mrkdwn
escaping vs the plain_text button label, a foreign total on a card, and the
finish_reason gate.

Integration: the NULL-platform wildcard on both the food and grocery paths,
candidates crossing platforms, EXCLUDE_RECONCILED_SOURCE, the pinned metadata
row, the panel reading the row the write landed on, the once-per-batch undo
baseline, the priceless-upload hold, Create re-matching, two pending cards
booking once, a pantry-resolved sibling, and a USD receipt.

Eleven of the thirteen fail against the pre-fix tree; the exceptions are the
no-batch-id baseline guard (a no-change assertion) and the concurrent-tap one,
which is a race and passed either way.
 2026-08-21 17:53:06 +10:00
siddharthd c25db5211b intake: a NULL platform is "no platform", and Create re-checks before it inserts 
Six fixes in the router, all of them cases where a stale or over-wide answer
was written rather than re-asked.

- ($3 IS NULL OR em.platform = $3) made an in-store receipt a wildcard: a till
  receipt from the Coles tap matched the Uber Eats Coles order for the same
  day and money, and overwrote its items. In-store now matches only rows with
  no platform of their own.
- findCandidates loses its platform filter entirely. That card is a human
  choice, shown precisely because the exact key missed — the platform is one
  of the things that may be wrong, so the list must widen there, not narrow.
- createFromBatch re-runs the match. The card is minutes old by the time it is
  tapped and the email/order bridge books orders on its own schedule; the
  match it was drawn from is not current.
- ...and checks for a resolved sibling batch. findDuplicateBatch ignores
  unresolved batches by design, so two pending cards for one order each booked
  it.
- INSERT ... ON CONFLICT (source, source_ref) DO NOTHING with a re-select,
  rather than SELECT-then-INSERT, which loses a double tap.
- A non-AUD receipt is booked as order-ingestion.ts books one: amount native,
  amount_aud NULL, foreign_currency_* set. No FX rate exists at intake.

Also: EXCLUDE_RECONCILED_SOURCE in both queries instead of a bare
superseded_by_id check — a reconciled manual twin at the exact key read as an
ambiguity, which is the same spend counted twice. And routeFood refuses to
replace an itemised receipt with an extraction carrying no prices at all: the
reconciliation check passes those (there is nothing to check), so without this
a priceless upload silently destroys a good record. It holds instead.

RouteContext.subtotal was written and never read; it is replaced by currency,
which the cards need.
 2026-08-21 17:53:06 +10:00
siddharthd 9a442b7355 intake: escape mrkdwn, and stop printing every total as dollars 
Merchant names, transaction descriptions and model output are none of ours:
Slack reads &, < and > as markup, so "Ben & Jerry's" and a description
carrying <http://…|text> rendered as something the card never meant to show.
esc() on every interpolated value in mrkdwn — never on our own literal *bold*,
and never on a button's plain_text label, where entities would print verbatim.

money() also hard-coded "$" while the order nudge already prints "USD 15.00".
The cards now carry the batch's currency.
 2026-08-21 17:52:47 +10:00
siddharthd f9638cc5e5 line-items: one metadata row, agreed on by the write and the read 
Three defects on one surface, all invisible while a transaction has exactly
one expense_metadata row — which is almost always.

1. ORDER BY (transaction_id = $1) DESC did the opposite of its comment.
   Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at the
   transaction has a NULL transaction_id, so the clause written to prefer the
   owned row preferred the other one — in exactly the case it exists for.
   matchTransaction already spelled out NULLS LAST; now so do this and
   GET /api/transactions/[id]/order, which had no ORDER BY at all and could
   read a different row from the one an enrichment had just written.

2. replaceLineItems takes an optional metadataId. The match resolves a row
   through a platform-filtered join; re-picking independently here can land
   the write on a row the match ruled out.

3. previous_line_items on intake_batches is the undo baseline, recorded once
   per batch under COALESCE and read back. Returning whatever was just read
   hands a pantry retry its own write as the "previous" state, so undo
   restores the thing it undid. Migration 0034 is unapplied anywhere, so the
   column is added there rather than in a 0035.
 2026-08-21 17:52:47 +10:00
siddharthd 7424d13858 intake: only a reply that finished is a reply the schema constrained 
liteLLMExtractor rejected finish_reason "length" and accepted everything else,
so a content_filter or tool_calls stop was parsed as if the strict json_schema
had held. Reject any reason present and not "stop", naming it.
 2026-08-21 17:52:25 +10:00
siddharthd be3d10d4f1 intake: match merchants on whole words, not substrings 
merchantKey("Bigals Pizza") contains "iga", so GROCERY_PATTERN filed a pizza
shop as a grocer and merchantMatches matched it against every IGA on the same
date and total. Short names in a substring list collide with ordinary words,
and the collision reads as a correct classification rather than a bug.

GROCERY_PATTERN is now a word-boundary alternation; merchantMatches asks
whether one name's words appear as a contiguous run inside the other's, which
keeps Coles <-> Coles (Wyndham Vale) and Order - Coles (Wyndham Vale).

Also collapses the third PLATFORM_LABEL copy: order-ingestion's was exported
and read nowhere, order-details.tsx had its own, and the intake one — the only
live copy — was missing `uber`. One map, in intake-rules.
 2026-08-21 17:52:25 +10:00
siddharthd a4e5f540cc docs: intake router routes, env vars, migration 0034
ci / lint-test (push) Successful in 48s
Details
2026-08-21 12:10:06 +10:00
siddharthd 32a47bf8ee test(intake): rules, PDF text, C2 idempotency, enrich-vs-candidates, pantry hand-off, Slack taps 
Unit: merchant pattern, header validation, reconciliation tolerance, verbatim
line items, one-call message shape, card action ids; unpdf over the two real
Uber Eats exports (Coles $85.74 / 15 lines, Burgertory $22.40 / Michael +
Joel). Integration: fetch stubbed at the LiteLLM and pantry boundaries so the
PDF path, persistence, match and writes run for real against personal_test.
intake-live.test.ts is opt-in (INTAKE_LIVE_TEST=1) and was run green against
cloud-budget on both PDFs.
 2026-08-21 12:10:06 +10:00
siddharthd c7ea516c34 slack: dispatch the four intake taps on action_id (plan §11 C5) 
intake_use_candidate / intake_create / intake_grocery / intake_meal carry JSON
{batch_id, ...} rather than <id>:<verb>, so they branch before the order-nudge
parse. They act on the persisted batch: enrich the chosen transaction, create
ING-10-style (statement_id NULL, owner_id from shared_by, source/source_ref
make a second press a no-op), or remember the merchant's kind and route. A tap
on an already-resolved batch returns its current card and writes nothing. Card
updates go to response_url, as the existing handlers do.
 2026-08-21 12:10:05 +10:00
siddharthd db5c9a02a7 intake: POST /api/intake router + PUT /api/transactions/[id]/line-items (plan §11 C1, C2) 
One upload point, every applicable step. Persist the batch, classify each
file, ONE LiteLLM call (cloud-budget, temperature 0, strict json_schema; PDFs
as unpdf text, never pixels) with an order-item prompt that keeps names
verbatim, header (merchant, date, total), merchant pattern
(coles|woolworths|woolies|aldi|iga|foodworks|costco) -> grocery | food | ask,
exact (platform, merchant, date, total) match against transactions +
expense_metadata, then act:

- food + unique match  -> replace line_items via the C2 path, 'enriched'
- food + miss/ambiguity -> 'candidates' (same merchant ±3d) with
  intake_use_candidate / intake_create buttons; never auto-creates
- grocery              -> original files to pantry POST /api/capture with
  finance_target_transaction_id + finance_action enrich|create (C3);
  409 -> 'duplicate'
- ask / needs_header / held / duplicate / unsupported as replies, not writes

Reconciliation is against the printed item subtotal (a delivery total is
items + fees - promotions: 18.94 of Coles lines settled at 5.74),  or
2%, else 'held'. Originals are kept under INTAKE_FILE_DIR by sha256 so a later
'Groceries' tap can still forward them.

C2 is an idempotent replace on transaction_id = :id OR
matched_transaction_id = :id, accepts INTAKE_TOKEN or RECEIPT_INGEST_TOKEN,
creates the expense_metadata row when absent, 404 when the transaction does
not exist, returns {previous_line_items, line_items} so pantry's undo is the
same call with the previous list.
 2026-08-21 12:10:05 +10:00
siddharthd a538376638 db(0034): intake_batches + intake_merchant_kinds for the intake router 
One row per upload through POST /api/intake, written BEFORE extraction so a
Slack tap minutes later has something to act on and a re-share (same sha256
set, or the same (platform, merchant, date, total) header already resolved)
is detected. Columns are the §11 C1 contract in smarthome
plans/intake-router-slice-plan.md. intake_merchant_kinds holds the remembered
'Groceries or a meal?' answer per normalised merchant (plan §2, Codex #6).

Not applied to production — finance has no migrate service; apply by hand at
deploy, then npm run test:setup.
 2026-08-21 12:10:05 +10:00
29 changed files with 3319 additions and 16 deletions
Expand all files Collapse all files
CLAUDE.md
+83
View File
@@ -622,6 +622,89 @@ and one tap must not flatten it. It says which case it refused on.
Slack reaches this route through an n8n webhook, not directly — see the
smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory.
### Intake router — one upload, every applicable step (2026-08-21)
`POST /api/intake` (token `INTAKE_TOKEN`, multipart: `source`, `shared_by`,
`batch_id`, 1–6 `file`s) is the finance half of the share-sheet intake; design
and the cross-repo contract are in smarthome `plans/intake-router-slice-plan.md`
§11 (C1, C2, C5 live here; C3/C4 are pantry-app; C6 is n8n + compose).
- **Order of operations is the plan's and is load-bearing:** persist
`intake_batches` row → classify each file → ONE LiteLLM call for the whole
batch (`cloud-budget`, `temperature 0`, strict `json_schema`; PDFs go in as
`unpdf` text, never pixels) → header `(merchant, date, total)` → merchant
kind → duplicate check → reconciliation → match → act. `src/lib/intake-rules.ts`
holds the pure decisions, `intake-router.ts` the orchestration.
- **Match is `(platform, merchant, date, total)`, exact, never widened.** A
miss or an ambiguity returns candidates (same merchant ±3 days) with
`intake_use_candidate` / `intake_create` buttons; a transaction is created
only on the explicit tap, ING-10-style (`statement_id NULL`, `owner_id` from
`shared_by` → `participants.email`, else `DEFAULT_OWNER_ID`;
`source='intake'`, `source_ref=batch_id` so a second press is a no-op —
written as `ON CONFLICT … DO NOTHING`, because a SELECT-then-INSERT loses a
double tap). A non-AUD receipt is booked the way `order-ingestion.ts` books
one: `amount` native, `amount_aud` **NULL**, `foreign_currency_*` set — no FX
rate exists at intake and inventing one is worse than admitting there is none.
- **A NULL platform means "no platform", not "any platform".** An in-store tap
(`coles`, `woolworths`, `other`) matches only rows whose `expense_metadata`
carries no platform of its own. The wildcard reading let a till receipt match
the Uber Eats order for the same shop on the same day and overwrite its items.
The **candidates** query is the opposite and deliberately so: it has no
platform filter at all, because the card is a human choice shown precisely
because the exact key missed — the platform is one of the things that may be
wrong, so the list must widen there, not narrow.
- **The Create tap re-runs the match before it inserts**, and checks for a
resolved sibling batch. The card is minutes old by the time it is pressed and
the email/order bridge books orders on its own schedule; `findDuplicateBatch`
ignores *unresolved* batches by design, so two pending cards for one order
would otherwise each book it.
- **One `expense_metadata` row, agreed on by every query that resolves one.** A
transaction can own a row *and* be the `matched_transaction_id` of another;
`matchTransaction` returns the row it resolved through and `replaceLineItems`
writes to exactly that one. Every reader orders by
`(transaction_id = $1) DESC **NULLS LAST**, id` — the match, the write, the
order-detail GET, `getTransactions`' provenance LATERAL, `merchantForTransaction`
and the Slack details modal. The `NULLS LAST` is load-bearing: Postgres orders
`DESC` as `NULLS FIRST` and the row that merely *points* at the transaction has
a NULL `transaction_id`, so the clause meant to prefer the owned row preferred
the other one. Measured 2026-08-21: 667 rows, 147 pointing-only, **zero**
transactions resolving to more than one row — the ordering changes nothing
today and exists so the readers still agree when multiplicity appears.
- **The create is guarded by a transaction-scoped advisory lock on the
PURCHASE**, not the batch. `ON CONFLICT (source, source_ref)` handles a double
tap on one card; two *different* batches for one purchase have different
`source_ref`s and both clear their match and duplicate checks before either
inserts. The lock has to live inside `withTransaction` (`src/lib/db.ts`):
`queryRaw` is `$queryRawUnsafe` on a **pooled** client, so every call is its
own implicit transaction and `pg_advisory_xact_lock` taken in a statement of
its own is released before the next statement runs. The loser reports
`duplicate` pointing at the winner's transaction — the same answer the
sequential path gives.
- **`replaceLineItems` snapshots the undo baseline BEFORE it writes, on both
paths**, and treats an UPDATE that touched no row as a failure rather than a
success. The metadata INSERT is `ON CONFLICT (transaction_id) DO UPDATE` and
carries the batch's **currency** — a hard-coded `'AUD'` made `OrderDetails`
print `$` on a USD order, because it prefers `expense_metadata.currency`.
- **`intake_batches.previous_line_items` is the undo baseline, written once.**
Returning whatever the write just read makes a pantry retry get its own write
back as the "previous" state, so undo restores the thing it undid. Recorded
under `COALESCE(previous_line_items, …)` and read back.
- **Grocery never writes finance here.** The ORIGINAL files go to pantry
`POST /api/capture` with `intake.finance_target_transaction_id` +
`finance_action: enrich|create`; pantry's confirm calls
`PUT /api/transactions/[id]/line-items` back (C4), which is why that route
accepts `RECEIPT_INGEST_TOKEN` too. Files are kept under `INTAKE_FILE_DIR`
keyed by sha256 so a later "Groceries" tap can still forward them.
- **Reconciliation is against the printed item subtotal, not the total.** A
delivery order's total is items + fees − promotions ($118.94 of Coles lines
became $85.74), so checking lines against the total would hold every
delivery order. Tolerance $1 or 2%; a failure is `held`, nothing written.
- **Names stay verbatim.** Burgertory calls its burgers "Michael" and "Joel";
the prompt forbids canonicalising and the stored shape is the order lane's
(`qty`, `description`, `amount`, `options`) so `order-details.tsx` renders it.
- Statuses beyond the contract list: `ask` (the "Groceries or a meal?" holding
state), `held`, `error` (502 — model or pantry down; the batch row says so).
### The shared loan
The loan is a **separate ledger**, not a shared expense and not a settlement
README.md
+14
View File
@@ -282,6 +282,18 @@ All routes require authentication via `X-Forwarded-User` header (set by Traefik)
| GET / POST | `/api/tags` | List or create tags |
| PATCH / DELETE | `/api/tags/[id]` | Update or delete a tag |
**Machine routes** — shared-secret `x-ingest-token` header instead of `X-Forwarded-User`; each fails closed when its token is unset:
| Method | Route | Token | Description |
|--------|-------|-------|-------------|
| POST / PATCH | `/api/orders/ingest` | `ORDER_INGEST_TOKEN` | n8n order receipts → transactions (see smarthome CLAUDE.md "Order Processor") |
| POST | `/api/receipts/ingest` | `RECEIPT_INGEST_TOKEN` | pantry-app confirmed grocery receipts → one transaction per tender leg (ING-10) |
| POST | `/api/intake` | `INTAKE_TOKEN` | **Intake router** (2026-08-21): multipart `source` (`uber_eats\|doordash\|coles\|woolworths\|other`), `shared_by`, `batch_id` (uuid), 1–6 `file`s (image / PDF / text). Persists `intake_batches`, extracts once via LiteLLM `cloud-budget` (PDF → text via `unpdf`), matches `(platform, merchant, date, total)` exactly, then enriches the matched order's `line_items` (food), forwards the original files to pantry `POST /api/capture` (grocery), or replies with candidates / "Groceries or a meal?" / needs-header / held / duplicate. Returns `{kind, status, transaction_id?, capture_id?, candidates?, slack_blocks, text}` |
| PUT | `/api/transactions/[id]/line-items` | `INTAKE_TOKEN` or `RECEIPT_INGEST_TOKEN` | Idempotent replace of `expense_metadata.line_items` for `transaction_id = :id OR matched_transaction_id = :id`; body `{line_items, source: 'intake'\|'pantry', batch_id?, platform?, replace: true}`; creates the metadata row if absent, never a transaction (404). Returns `{previous_line_items, line_items}` — pantry calls it on every confirm and with `previous_line_items` on undo |
| POST | `/api/slack/interactive` | `ORDER_INGEST_TOKEN` + Slack signature | Order-nudge buttons and the intake taps (`intake_use_candidate`, `intake_create`, `intake_grocery`, `intake_meal`) |
Env for the intake router: `INTAKE_TOKEN`, `LITELLM_BASE_URL` + `LITELLM_KEY` (OpenAI-compatible, alias `cloud-budget`), `PANTRY_BASE_URL` (+ the existing `RECEIPT_INGEST_TOKEN` as pantry's machine auth), optional `INTAKE_FILE_DIR` (original files by sha256, default `/app/data/intake` — a later "Groceries" tap re-sends them) and `FINANCE_PUBLIC_URL` (the card's View link, default `https://finance.bosecamp.com`). Design: smarthome `plans/intake-router-slice-plan.md` §11.
---
## Ingestion Pipeline
@@ -347,6 +359,8 @@ docker exec postgres-personal psql -U personal -d personal \
| `0009_split_payments` | `split_payments` |
| `0010_csv_import_reconcile` | `reconciled_with_id`, CSV import support |
| `0011_trips` | `trips`, `trip_id` on `transaction_overrides` (catch-up — was applied directly) |
| … | |
| `0034_intake_batches` | `intake_batches` (one row per `/api/intake` upload, written before extraction) and `intake_merchant_kinds` (the remembered "Groceries or a meal?" answer per merchant) |
> `paperless_doc_id` on statements and the `uq_statements_paperless_doc_id` index were added directly (not tracked in a migration file).
> `owner_id` on transactions and `statement_id` made nullable were applied directly (March 2026) to support manual transaction entry without a fake statement.
package-lock.json
Generated
+19 -1
View File
@@ -16,7 +16,8 @@
"prisma": "^7.4.2",
"react": "19.2.3",
"react-dom": "19.2.3",
"recharts": "^3.8.0"
"recharts": "^3.8.0",
"unpdf": "^1.8.1"
},
"devDependencies": {
"@tailwindcss/postcss": "^4",
@@ -8494,6 +8495,23 @@
"dev": true,
"license": "MIT"
},
"node_modules/unpdf": {
"version": "1.8.1",
"resolved": "https://registry.npmjs.org/unpdf/-/unpdf-1.8.1.tgz",
"integrity": "sha512-xkURhy2SoGpOIH0a1gLHNkASPIQYonadDJs2AQwPEfUakafeD9EA1WTWWsaR++gfTCXJpV27W7tU1nXuk82UKQ==",
"license": "MIT",
"engines": {
"node": ">=22"
},
"peerDependencies": {
"@napi-rs/canvas": "^0.1.69 || ^1.0.0"
},
"peerDependenciesMeta": {
"@napi-rs/canvas": {
"optional": true
}
}
},
"node_modules/unrs-resolver": {
"version": "1.11.1",
"resolved": "https://registry.npmjs.org/unrs-resolver/-/unrs-resolver-1.11.1.tgz",
package.json
+2 -1
View File
@@ -23,7 +23,8 @@
"prisma": "^7.4.2",
"react": "19.2.3",
"react-dom": "19.2.3",
"recharts": "^3.8.0"
"recharts": "^3.8.0",
"unpdf": "^1.8.1"
},
"devDependencies": {
"@tailwindcss/postcss": "^4",
prisma/migrations/0034_intake_batches/migration.sql
+74
View File
@@ -0,0 +1,74 @@
-- Intake router (plans/intake-router-slice-plan.md §11 C1, smarthome repo).
--
-- One upload point for receipts — a screenshot, a PDF export, a pasted body —
-- that reads what it was given and does every step that applies: fill the
-- empty `line_items` on the order the email pipeline already booked, hand a
-- grocery receipt to pantry, or ask when it cannot tell.
--
-- Every batch is persisted BEFORE anything is extracted, for two reasons:
-- 1. The Slack card it produces carries buttons ("Use this", "Create",
-- "Groceries or a meal?") that are pressed minutes later in a fresh
-- request with nothing but the batch id to go on.
-- 2. A re-share of an already-handled order — a second crop, the PDF after
-- the screenshot — must be recognised, and the only durable evidence of
-- the first share is this row (`file_sha256` overlap, or the same
-- (platform, merchant, date, total) header already resolved).
--
-- `id` is the uuid n8n mints per webhook call, so a retried delivery is the
-- same batch rather than a second one. `header` holds the extracted
-- (merchant, order_date, total, platform, ...) and `items` the verbatim item
-- list, so a later tap can enrich or create without re-reading the files.
--
-- Idempotent: safe to re-run.
CREATE TABLE IF NOT EXISTS intake_batches (
id uuid PRIMARY KEY,
source text NOT NULL,
shared_by text,
file_sha256 text[] NOT NULL DEFAULT '{}',
header jsonb,
items jsonb,
kind text,
status text NOT NULL,
transaction_id integer,
capture_id integer,
-- The undo baseline: what `expense_metadata.line_items` held before this
-- batch first replaced it. Recorded here rather than returned from the write,
-- because pantry re-calls the enrich endpoint on every confirm and after a
-- timeout — a caller that hands back whatever it just read gives a retry its
-- OWN write as the "previous" state, and undo then restores the thing it was
-- meant to reverse. Written under COALESCE, so only the first snapshot sticks.
previous_line_items jsonb,
created_at timestamptz NOT NULL DEFAULT now(),
resolved_at timestamptz
);
-- For a database that already ran an earlier draft of this file.
ALTER TABLE intake_batches ADD COLUMN IF NOT EXISTS previous_line_items jsonb;
COMMENT ON TABLE intake_batches IS
'One row per upload batch through POST /api/intake. Persisted before '
'extraction so Slack taps can act on it later and re-shares are detected. '
'kind: grocery|food|ask|held|unsupported. status: received|enriched|created|'
'candidates|forwarded_to_pantry|needs_header|duplicate|unsupported|held|error.';
-- Overlap lookups for the re-share check (`file_sha256 && $1`).
CREATE INDEX IF NOT EXISTS intake_batches_file_sha256_gin
ON intake_batches USING gin (file_sha256);
CREATE INDEX IF NOT EXISTS intake_batches_transaction_id
ON intake_batches (transaction_id) WHERE transaction_id IS NOT NULL;
-- The "Groceries or a meal?" answer, remembered per merchant (plan §2, Codex
-- #6). A tiny answer table, not merchant master-data: the pattern list in code
-- decides the obvious cases and this catches the `Romeo's Food Hall`s. Keyed on
-- the normalised merchant name the router already computes for matching.
CREATE TABLE IF NOT EXISTS intake_merchant_kinds (
merchant_key text PRIMARY KEY,
kind text NOT NULL CHECK (kind IN ('grocery', 'food')),
created_at timestamptz NOT NULL DEFAULT now()
);
COMMENT ON TABLE intake_merchant_kinds IS
'Per-merchant answer to "Groceries or a meal?" from the intake Slack card. '
'Consulted after the grocery pattern list; an unknown merchant asks once.';
prisma/schema.prisma
+27
View File
@@ -268,3 +268,30 @@ model rule_apply_runs {
reverted_at DateTime?
snapshot Json @default("[]")
}
// 0034 — intake router. One row per upload batch, written before extraction so
// Slack taps can act on it later and a re-share is detected. Columns are the
// §11 C1 contract in plans/intake-router-slice-plan.md (smarthome repo).
model intake_batches {
id String @id @db.Uuid
source String
shared_by String?
file_sha256 String[] @default([])
header Json?
items Json?
kind String?
status String
transaction_id Int?
capture_id Int?
// The undo baseline, written once per batch (see 0034 and line-items.ts).
previous_line_items Json?
created_at DateTime @default(now()) @db.Timestamptz(6)
resolved_at DateTime? @db.Timestamptz(6)
}
// 0034 — the remembered answer to "Groceries or a meal?" per merchant.
model intake_merchant_kinds {
merchant_key String @id
kind String
created_at DateTime @default(now()) @db.Timestamptz(6)
}
src/__tests__/fixtures/intake/ubereats-burgertory-2026-08-19.pdf
BIN
View File
Binary file not shown.
src/__tests__/fixtures/intake/ubereats-coles-2026-08-18.pdf
BIN
View File
Binary file not shown.
src/__tests__/integration/helpers.ts
+19
View File
@@ -23,6 +23,25 @@ export function mockDbWithPool(p: Pool) {
const result = await p.query(sql, params);
return result.rows[0] ?? null;
},
// Mirrors withTransaction: one connection, BEGIN/COMMIT around the callback,
// so pg_advisory_xact_lock actually holds across the statements inside it.
withTransaction: async (fn: (q: unknown) => Promise<unknown>) => {
const client = await p.connect();
try {
await client.query("BEGIN");
const result = await fn({
queryRaw: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows,
queryRow: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows[0] ?? null,
});
await client.query("COMMIT");
return result;
} catch (e) {
await client.query("ROLLBACK");
throw e;
} finally {
client.release();
}
},
prisma: p,
}));
}
src/__tests__/integration/intake.test.ts
+793
View File
@@ -0,0 +1,793 @@
import { describe, it, expect, beforeAll, beforeEach, afterAll, vi } from "vitest";
import { Pool } from "pg";
import { readFileSync, mkdtempSync, rmSync } from "fs";
import { tmpdir } from "os";
import { join, resolve } from "path";
import type { NextRequest } from "next/server";
import { createPool, mockDbWithPool, resetDB, seedParticipants } from "./helpers";
import type { ExtractedOrder } from "../../lib/intake-rules";
/**
* The intake router against the real schema (plan §11 C1, C2, C5).
*
* The model is stubbed at the HTTP boundary — `fetch` to LiteLLM answers with a
* canned extraction, `fetch` to pantry with a canned capture — so the PDF text
* path, the persistence, the match and the writes all run for real. The two
* PDFs are the user's own Uber Eats exports; the transactions seeded here are
* shaped like production rows 6846 and 6857 (platform `ubereats`, `line_items`
* empty, the email pipeline's booking of the same two orders).
*/
const INTAKE = "test-intake-token";
const RECEIPT = "test-receipt-token";
const fixtures = resolve(__dirname, "../fixtures/intake");
const BURGERTORY = readFileSync(resolve(fixtures, "ubereats-burgertory-2026-08-19.pdf"));
const COLES = readFileSync(resolve(fixtures, "ubereats-coles-2026-08-18.pdf"));
const burgertory: ExtractedOrder = {
merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD",
items: [
{ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] },
{ quantity: 1, description: "Joel", unit_price: 14, line_total: 14, options: [] },
],
adjustments: [{ label: "Delivery Fee", amount: 5.49 }, { label: "Promotion", amount: -8 }],
};
const coles: ExtractedOrder = {
merchant: "Coles (Wyndham Vale)", order_date: "2026-08-18", total: 85.74, subtotal: 118.94, currency: "AUD",
items: [
{ quantity: 3, description: "Coca-Cola Zero Sugar Soft Drink (2L)", unit_price: 4.4, line_total: 13.2, options: [] },
{ quantity: 1, description: "Coles RSPCA Approved Chicken Thigh Cutlets approx. 1.1kg", unit_price: 11, line_total: 12.76, options: ["Final 1.16 kg $11.00/kg"] },
{ quantity: 1, description: "everything else", unit_price: 92.98, line_total: 92.98, options: [] },
],
adjustments: [],
};
let pool: Pool;
let ownerId: number;
let otherId: number;
let fileDir: string;
let intakePOST: (req: NextRequest) => Promise<Response>;
let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
let slackPOST: (req: NextRequest) => Promise<Response>;
let orderGET: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
/** What the next LiteLLM call answers, and what the next pantry call answers. */
let modelReply: ExtractedOrder | Error = burgertory;
let pantryReply: { status: number; body: unknown } = { status: 200, body: { captureId: 901, items: [1, 2, 3] } };
const calls: { url: string; body: unknown }[] = [];
function stubFetch() {
vi.stubGlobal("fetch", async (url: string | URL, init?: RequestInit) => {
const u = String(url);
if (u.includes("/v1/chat/completions")) {
calls.push({ url: u, body: JSON.parse(String(init?.body)) });
if (modelReply instanceof Error) return new Response("boom", { status: 500 });
return Response.json({ choices: [{ message: { content: JSON.stringify(modelReply) }, finish_reason: "stop" }] });
}
if (u.includes("/api/capture")) {
const form = init?.body as FormData;
calls.push({ url: u, body: { intake: JSON.parse(String(form.get("intake"))), files: form.getAll("image").map((f) => (f as File).name), headers: init?.headers } });
return Response.json(pantryReply.body, { status: pantryReply.status });
}
if (u.includes("hooks.slack.com")) {
calls.push({ url: u, body: JSON.parse(String(init?.body)) });
return new Response("ok");
}
throw new Error(`unexpected fetch ${u}`);
});
}
const multipart = (fields: Record<string, string>, files: { name: string; type: string; bytes: Buffer }[], token: string | null = INTAKE) => {
const fd = new FormData();
for (const [k, v] of Object.entries(fields)) fd.append(k, v);
for (const f of files) fd.append("file", new File([new Uint8Array(f.bytes)], f.name, { type: f.type }));
return new Request("http://finance.test/api/intake", { method: "POST", headers: token ? { "x-ingest-token": token } : {}, body: fd }) as unknown as NextRequest;
};
const jsonReq = (body: unknown, token: string | null) =>
({ headers: { get: (h: string) => (h === "x-ingest-token" ? token : null) }, json: async () => body }) as unknown as NextRequest;
const put = (id: number | string, body: unknown, token: string | null = RECEIPT) => lineItemsPUT(jsonReq(body, token), { params: Promise.resolve({ id: String(id) }) });
const uuid = () => crypto.randomUUID();
/** A transaction shaped like the email pipeline's booking of an order, line_items empty. */
async function seedOrder(over: { merchant: string; date: string; amount: number; platform?: string | null; owner?: number; withMeta?: boolean }) {
const t = await pool.query(
`INSERT INTO transactions (owner_id, statement_id, transaction_date, description, amount, amount_aud, transaction_type, category, merchant_name, merchant_normalized, payment_method, row_index)
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'dining', $5, $5, 'credits', 0) RETURNING id`,
[over.owner ?? ownerId, over.date, `Order - ${over.merchant}`, over.amount, over.merchant]
);
const id = t.rows[0].id as number;
if (over.withMeta !== false) {
await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, platform, order_reference, line_items, merchant_normalized, amount, transaction_date, currency)
VALUES ($1, 'email', $2, $3, '[]'::jsonb, $4, $5, $6, 'AUD')`,
[id, over.platform === undefined ? "ubereats" : over.platform, uuid(), over.merchant, over.amount, over.date]
);
}
return id;
}
/** What `components/order-details.tsx` would actually render for a transaction. */
const orderPanel = async (txnId: number) =>
(await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null;
const metaRows = async () => (await pool.query(`SELECT id, platform, line_items FROM expense_metadata ORDER BY id`)).rows as { id: number; platform: string | null; line_items: { description: string }[] }[];
const lineItemsOf = async (txnId: number) =>
(await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items;
const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0];
beforeAll(async () => {
process.env.INTAKE_TOKEN = INTAKE;
process.env.RECEIPT_INGEST_TOKEN = RECEIPT;
process.env.ORDER_INGEST_TOKEN = "test-order-token";
process.env.LITELLM_BASE_URL = "http://litellm.test";
process.env.LITELLM_KEY = "k";
process.env.PANTRY_BASE_URL = "http://pantry.test";
process.env.FINANCE_PUBLIC_URL = "https://finance.test";
fileDir = mkdtempSync(join(tmpdir(), "intake-test-"));
process.env.INTAKE_FILE_DIR = fileDir;
pool = createPool();
mockDbWithPool(pool);
// The Slack signature gate is exercised by slack-verify.test.ts; here it is
// bypassed so the intake branch of the route is what is under test.
vi.doMock("@/lib/slack-verify", () => ({
verifySlackSignature: () => true,
participantForSlackUser: () => 1,
slackUserForParticipant: () => null,
}));
// The order-detail panel is the surface an enrichment has to appear on, so
// the read is exercised beside the write rather than assumed.
vi.doMock("@/lib/auth", () => ({ getCurrentUser: async () => ({ id: 1, name: "Owner", email: "o@x" }) }));
vi.doMock("@/lib/queries", async (importOriginal) => ({
...(await importOriginal<Record<string, unknown>>()),
canAccessTransactions: async () => true,
}));
({ POST: intakePOST } = await import("../../app/api/intake/route"));
({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route"));
({ POST: slackPOST } = await import("../../app/api/slack/interactive/route"));
({ GET: orderGET } = await import("../../app/api/transactions/[id]/order/route"));
stubFetch();
});
afterAll(() => {
vi.unstubAllGlobals();
rmSync(fileDir, { recursive: true, force: true });
});
beforeEach(async () => {
await resetDB(pool);
await pool.query(`TRUNCATE intake_batches, intake_merchant_kinds`);
({ ownerId, otherId } = await seedParticipants(pool, ["Alice", "Bob"]));
await pool.query(`UPDATE participants SET email = 'bob@example.test' WHERE id = $1`, [otherId]);
modelReply = burgertory;
pantryReply = { status: 200, body: { captureId: 901, items: [1, 2, 3] } };
calls.length = 0;
});
// ─── C2 ───────────────────────────────────────────────────────────────────────
describe("PUT /api/transactions/[id]/line-items (C2)", () => {
const items = [{ qty: 1, description: "Michael", amount: 14, options: ["Milk Bun"] }, { qty: 1, description: "Joel", amount: 14 }];
it("fails closed: no token, wrong token", async () => {
expect((await put(1, { line_items: items, source: "intake", replace: true }, null)).status).toBe(401);
expect((await put(1, { line_items: items, source: "intake", replace: true }, "nope")).status).toBe(401);
});
it("accepts either token", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
expect((await put(id, { line_items: items, source: "pantry", replace: true }, RECEIPT)).status).toBe(200);
expect((await put(id, { line_items: items, source: "intake", replace: true }, INTAKE)).status).toBe(200);
});
it("404 when the transaction does not exist — it never creates one", async () => {
const res = await put(999999, { line_items: items, source: "intake", replace: true });
expect(res.status).toBe(404);
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
});
it("rejects a body that is not a replace, a bad source, or nameless items", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
expect((await put(id, { line_items: items, source: "intake" })).status).toBe(400);
expect((await put(id, { line_items: items, source: "n8n", replace: true })).status).toBe(400);
expect((await put(id, { line_items: [{ qty: 1 }], source: "intake", replace: true })).status).toBe(400);
});
it("idempotent replace: returns previous + new, and a second identical call is a no-op", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = await (await put(id, { line_items: items, source: "intake", replace: true })).json();
expect(first.previous_line_items).toEqual([]);
expect(first.line_items).toHaveLength(2);
expect(first.created_metadata).toBe(false);
expect(await lineItemsOf(id)).toHaveLength(2);
const corrected = [{ qty: 1, description: "Michael (corrected)", amount: 14 }];
const second = await (await put(id, { line_items: corrected, source: "pantry", replace: true })).json();
expect(second.previous_line_items.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(await lineItemsOf(id)).toEqual(corrected);
const third = await (await put(id, { line_items: corrected, source: "pantry", replace: true })).json();
expect(third.previous_line_items).toEqual(corrected);
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata`)).rows[0].n).toBe(1);
});
it("undo shape: PUT previous_line_items back restores the row", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const r = await (await put(id, { line_items: items, source: "pantry", replace: true })).json();
await put(id, { line_items: r.previous_line_items, source: "pantry", replace: true });
expect(await lineItemsOf(id)).toEqual([]);
});
it("creates the expense_metadata row when the transaction has none, with source and platform", async () => {
const id = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const res = await (await put(id, { line_items: items, source: "intake", platform: "ubereats", batch_id: "b1", replace: true })).json();
expect(res.created_metadata).toBe(true);
const em = (await pool.query(`SELECT source, platform, order_reference, merchant_normalized, amount::text FROM expense_metadata WHERE transaction_id = $1`, [id])).rows[0];
expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" });
});
it("the undo baseline is recorded ONCE per batch: a retry gets the original, not its own write", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const original = [{ qty: 1, description: "what the order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(original)]);
const id = uuid();
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
const first = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(first.previous_line_items).toEqual(original);
// Pantry re-calls on every confirm and after a timeout. Before, the retry
// was handed its OWN write back, so undo restored the thing it undid.
const retry = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(retry.previous_line_items).toEqual(original);
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual(original);
await put(txn, { line_items: retry.previous_line_items, source: "pantry", batch_id: id, replace: true });
expect(await lineItemsOf(txn)).toEqual(original);
});
it("without a batch id the baseline is still the row as read — nothing else changes", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(first.previous_line_items).toEqual([]);
const second = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(second.previous_line_items).toEqual(items);
});
it("a pinned row that no longer exists falls back to the live one, rather than reporting a phantom write", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const live = (await pool.query(`SELECT id FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].id as number;
const { replaceLineItems } = await import("../../lib/line-items");
const r = await replaceLineItems(txn, items, { source: "intake", metadataId: live + 999 });
expect(r.metadata_id).toBe(live);
expect(await lineItemsOf(txn)).toHaveLength(2);
});
it("two writes racing to create the metadata row both resolve to the same one", async () => {
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const both = await Promise.all([
put(txn, { line_items: items, source: "intake", replace: true }),
put(txn, { line_items: items, source: "intake", replace: true }),
]);
expect(both.map((r) => r.status)).toEqual([200, 200]);
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].n).toBe(1);
});
it("the create path records an undo baseline too, before it writes", async () => {
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const id = uuid();
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
await put(txn, { line_items: items, source: "intake", batch_id: id, currency: "AUD", replace: true });
// [] rather than NULL: there was nothing there, and "nothing" is the state
// undo has to be able to restore.
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual([]);
});
it("the panel reads the row the write landed on, whichever order the rows were inserted in", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// Inserted first: the delivery receipt pointing at the statement line.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
[t, uuid()]
);
// Inserted second: a row the transaction owns. An unordered LIMIT 1 reads
// the first one, the write takes the second, and the enrichment reports
// success against a row nothing renders.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, line_items) VALUES ($1, 'pantry', $2, '[]'::jsonb)`,
[t, uuid()]
);
const written = await (await put(t, { line_items: items, source: "intake", replace: true })).json();
const panel = await orderPanel(t);
expect(panel?.line_items).toHaveLength(2);
expect((await metaRows()).find((r) => r.id === written.metadata_id)!.line_items).toHaveLength(2);
});
it("finds the row through matched_transaction_id — a card-settled order has only that", async () => {
const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
[statementLine, uuid()]
);
const res = await (await put(statementLine, { line_items: items, source: "intake", replace: true })).json();
expect(res.created_metadata).toBe(false);
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata`)).rows[0].n).toBe(1);
expect(await lineItemsOf(statementLine)).toHaveLength(2);
});
});
// ─── C1 ───────────────────────────────────────────────────────────────────────
describe("POST /api/intake — gates", () => {
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
it("fails closed without INTAKE_TOKEN match", async () => {
expect((await intakePOST(multipart({ source: "uber_eats" }, [pdf], null))).status).toBe(401);
expect((await intakePOST(multipart({ source: "uber_eats" }, [pdf], "nope"))).status).toBe(401);
});
it("400 on a bad source, no files, too many files, a non-uuid batch_id", async () => {
expect((await intakePOST(multipart({ source: "menulog" }, [pdf]))).status).toBe(400);
expect((await intakePOST(multipart({ source: "uber_eats" }, []))).status).toBe(400);
expect((await intakePOST(multipart({ source: "uber_eats" }, Array(7).fill(pdf)))).status).toBe(400);
expect((await intakePOST(multipart({ source: "uber_eats", batch_id: "not-a-uuid" }, [pdf]))).status).toBe(400);
expect((await pool.query(`SELECT count(*)::int AS n FROM intake_batches`)).rows[0].n).toBe(0);
});
it("declines a file type it cannot read, after persisting the batch", async () => {
const id = uuid();
const res = await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [{ name: "clip.mp4", type: "video/mp4", bytes: Buffer.from("x") }]));
const body = await res.json();
expect(body.status).toBe("unsupported");
expect(body.text).toMatch(/not this/);
expect((await batch(id)).status).toBe("unsupported");
expect(calls).toHaveLength(0);
});
});
describe("POST /api/intake — food (enrich vs candidates)", () => {
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
it("food + exact (platform, merchant, date, total) match → enriches the existing order, one model call, PDF as text", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const id = uuid();
const res = await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]));
const body = await res.json();
expect(res.status).toBe(200);
expect(body).toMatchObject({ kind: "food", status: "enriched", transaction_id: txn, batch_id: id });
expect(body.text).toBe("Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓");
expect(JSON.stringify(body.slack_blocks)).toContain("https://finance.test/transactions?q=Burgertory");
const stored = await lineItemsOf(txn);
expect(stored.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(stored[0].options).toEqual(["Milk Bun"]);
// one joint call, the PDF's text in it (not an image)
expect(calls).toHaveLength(1);
const content = calls[0].body as { messages: { content: { type: string; text?: string }[] }[]; model: string; temperature: number; response_format: { type: string } };
expect(content.model).toBe("cloud-budget");
expect(content.temperature).toBe(0);
expect(content.response_format.type).toBe("json_schema");
expect(content.messages[0].content.some((c) => c.type === "image_url")).toBe(false);
expect(content.messages[0].content.some((c) => c.text?.includes("1 Michael $14.00"))).toBe(true);
const row = await batch(id);
expect(row).toMatchObject({ source: "uber_eats", shared_by: "bob@example.test", kind: "food", status: "enriched", transaction_id: txn });
expect(row.header).toMatchObject({ merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, platform: "ubereats" });
expect(row.resolved_at).not.toBeNull();
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
});
it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match — but it IS offered", async () => {
const doordash = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" });
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("candidates");
// The candidates card is a human choice, not a decision. The platform is one
// of the things that may be wrong, so it must not narrow the list — a
// platform-filtered card showed nothing at all here.
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([doordash]);
});
it("a NULL platform is 'no platform', not a wildcard — an in-store share does not overwrite the delivery order", async () => {
await pool.query(`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ('burgertory werribee', 'food')`);
const delivery = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "ubereats" });
const already = [{ qty: 1, description: "what the delivery order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [delivery, JSON.stringify(already)]);
// `other` implies no platform; the delivery row carries one, so it is not
// the match — it is only a candidate.
const body = await (await intakePOST(multipart({ source: "other" }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([delivery]);
expect(await lineItemsOf(delivery)).toEqual(already);
});
it("a reconciled manual row is neither matched nor offered — EXCLUDE_RECONCILED_SOURCE, as everywhere else", async () => {
const live = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const reconciled = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, reconciled]);
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, near]);
// Two rows at the exact key would have read as an ambiguity; one of them is
// the same spend counted twice.
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: live });
});
it("the write lands on the row the MATCH resolved through, not on whichever row sorts first", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// The statement line owns a receipt row of its own (a pantry confirm) …
const grocery = [{ qty: 1, description: "a grocery shop", amount: 22.4 }];
const own = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, merchant_normalized, line_items) VALUES ($1, 'pantry', $2, 'Woolworths', $3::jsonb) RETURNING id`,
[t, uuid(), JSON.stringify(grocery)]
)).rows[0].id as number;
// … and is also the card leg of the Uber Eats order being enriched.
const delivery = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, merchant_normalized, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, 'Burgertory (Werribee)', '[]'::jsonb) RETURNING id`,
[t, uuid()]
)).rows[0].id as number;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: t });
const rows = await metaRows();
expect(rows.find((r) => r.id === delivery)!.line_items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
expect(rows.find((r) => r.id === own)!.line_items).toEqual(grocery);
});
it("an upload with no prices never replaces a receipt that already has items", async () => {
const priceless = { ...burgertory, subtotal: null, total: 22.4, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
modelReply = priceless;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "held", status: "held" });
expect(body.text).toMatch(/left the existing items alone/);
expect(await lineItemsOf(txn)).toEqual(already);
// …but an EMPTY receipt is exactly what this slice exists to fill.
const empty = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-20", amount: 22.4 });
modelReply = { ...priceless, order_date: "2026-08-20" };
const second = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(second).toMatchObject({ status: "enriched", transaction_id: empty });
expect((await lineItemsOf(empty)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
});
it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => {
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-10", amount: 22.4 }); // too far
await seedOrder({ merchant: "Mad Mex", date: "2026-08-19", amount: 22.4 }); // wrong merchant
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([near]);
const blocks = JSON.stringify(body.slack_blocks);
expect(blocks).toContain('"action_id":"intake_use_candidate"');
expect(blocks).toContain('"action_id":"intake_create"');
expect(blocks).toContain(JSON.stringify(JSON.stringify({ batch_id: id, transaction_id: near })));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(3);
expect(await lineItemsOf(near)).toEqual([]);
expect((await batch(id)).items).toHaveLength(2);
});
it("two exact hits is an ambiguity, not a coin toss", async () => {
const a = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const b = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id).sort()).toEqual([a, b].sort());
});
it("missing header (no total) → needs_header, nothing written but the batch row", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
modelReply = { ...burgertory, total: null };
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("needs_header");
expect(body.text).toMatch(/share the top of the screen/);
expect(await lineItemsOf(txn)).toEqual([]);
expect((await batch(id)).status).toBe("needs_header");
});
it("items that do not add up → held, nothing written", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
modelReply = { ...burgertory, items: [burgertory.items[0]] };
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "held", status: "held" });
expect(body.text).toMatch(/is a shot missing/);
expect(await lineItemsOf(txn)).toEqual([]);
});
it("a model failure is a 502 with the batch marked error, not a silent 200", async () => {
modelReply = new Error("down");
const res = await intakePOST(multipart({ source: "uber_eats" }, [pdf]));
expect(res.status).toBe(502);
expect((await res.json()).status).toBe("error");
});
it("re-share of the same file after it was handled → duplicate with a pointer", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: first }, [pdf]));
calls.length = 0;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("duplicate");
expect(body.duplicate_of).toEqual({ batch_id: first, transaction_id: txn, capture_id: null });
expect(body.text).toMatch(/Already handled/);
expect(calls).toHaveLength(0); // no model call spent on a known file
});
it("re-share of the same order under a different file (screenshot after PDF) → duplicate by header", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: first }, [pdf]));
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [{ name: "IMG_1.png", type: "image/png", bytes: Buffer.from("fake-png") }]))).json();
expect(body.status).toBe("duplicate");
expect(body.duplicate_of.transaction_id).toBe(txn);
});
it("a repeated batch_id is refused, not re-run", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
expect((await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).status).toBe(409);
});
});
describe("POST /api/intake — grocery → pantry", () => {
const pdf = { name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES };
it("grocery + match → the ORIGINAL file goes to pantry with the target transaction and action enrich; finance is NOT touched", async () => {
modelReply = coles;
const txn = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74 });
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]))).json();
expect(body).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
expect(body.text).toBe("Coles (Wyndham Vale) $85.74 → pantry board (3 items) — finance updates on confirm");
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown>; files: string[]; headers: Record<string, string> };
expect(pantryCall.headers["x-ingest-token"]).toBe(RECEIPT);
expect(pantryCall.files).toEqual(["Receipt_18Aug2026.pdf"]);
expect(pantryCall.intake).toEqual({ batch_id: id, platform: "ubereats", origin: "intake", shared_by: "bob@example.test", finance_target_transaction_id: txn, finance_action: "enrich" });
expect(await lineItemsOf(txn)).toEqual([]);
expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
});
it("an in-store tap does not hand pantry the delivery order as its finance target", async () => {
modelReply = coles;
// The Uber Eats Coles order the email pipeline already booked.
const delivery = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: "ubereats" });
// The same shop, same day, same money — but shared from the Coles app, so
// it carries no platform and is a different purchase.
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown> };
expect(pantryCall.intake).toMatchObject({ finance_action: "create", finance_target_transaction_id: null, platform: null });
expect(await lineItemsOf(delivery)).toEqual([]);
});
it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => {
modelReply = coles;
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
expect(body.text).toMatch(/finance books it on confirm/);
const pantryCall = calls.find((c) => c.url.includes("/api/capture"))!.body as { intake: Record<string, unknown> };
expect(pantryCall.intake).toMatchObject({ platform: null, finance_target_transaction_id: null, finance_action: "create" });
});
it("pantry 409 → duplicate", async () => {
modelReply = coles;
pantryReply = { status: 409, body: { error: "This receipt has already been added to the pantry.", captureId: 77 } };
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "grocery", status: "duplicate", capture_id: 77 });
});
it("pantry down → 502 error, batch marked error", async () => {
modelReply = coles;
pantryReply = { status: 500, body: { error: "nope" } };
const res = await intakePOST(multipart({ source: "coles" }, [pdf]));
expect(res.status).toBe(502);
expect((await res.json()).status).toBe("error");
});
});
// ─── C5 ───────────────────────────────────────────────────────────────────────
const slackPress = (action_id: string, value: unknown) => {
const payload = { type: "block_actions", user: { id: "U1", name: "alice" }, response_url: "https://hooks.slack.com/r/1", actions: [{ action_id, value: JSON.stringify(value) }] };
const raw = new URLSearchParams({ payload: JSON.stringify(payload) }).toString();
return { headers: { get: (h: string) => (h === "x-ingest-token" ? "test-order-token" : "x") }, text: async () => raw } as unknown as NextRequest;
};
const lastSlackUpdate = () => calls.filter((c) => c.url.includes("hooks.slack.com")).at(-1)?.body as { replace_original?: boolean; text: string; blocks?: unknown[] };
describe("Slack taps (C5)", () => {
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
it("intake_use_candidate enriches the chosen transaction and replaces the card", async () => {
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
const res = await slackPOST(slackPress("intake_use_candidate", { batch_id: id, transaction_id: near }));
expect(res.status).toBe(200);
expect((await lineItemsOf(near)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: near });
expect(lastSlackUpdate()).toMatchObject({ replace_original: true, text: "Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓" });
});
it("intake_create creates the transaction ING-10-style: manual, owner from shared_by, items attached; a second press is a no-op", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
await slackPOST(slackPress("intake_create", { batch_id: id }));
const t = (await pool.query(`SELECT statement_id, owner_id, description, amount::text, category, transaction_date::text, source, source_ref FROM transactions`)).rows;
expect(t).toEqual([{ statement_id: null, owner_id: otherId, description: "Order - Burgertory (Werribee)", amount: "22.40", category: "dining", transaction_date: "2026-08-19", source: "intake", source_ref: id }]);
const em = (await pool.query(`SELECT source, platform, line_items FROM expense_metadata`)).rows[0];
expect(em.source).toBe("intake");
expect(em.platform).toBe("ubereats");
expect(em.line_items.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(lastSlackUpdate().text).toMatch(/^Created a \$22\.40 Burgertory \(Werribee\) transaction with 2 items/);
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
});
it("intake_create re-runs the match: an order booked between the card and the tap is enriched, not duplicated", async () => {
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
// The email/order bridge books the same purchase while the card sits in
// Slack. The card's "no match" is minutes stale by the time it is pressed.
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
expect((await lineItemsOf(txn)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
});
it("two pending cards for one order book it ONCE", async () => {
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
expect((await batch(a)).status).toBe("candidates");
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: a }));
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect((await batch(b)).transaction_id).toBe((await batch(a)).transaction_id);
});
it("a sibling already resolved to the pantry board makes Create a duplicate, not a second row", async () => {
// Two shares of one receipt, resolved out of order: nothing the second card
// can match on, because the first one never produced a transaction.
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "other", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
await intakePOST(multipart({ source: "other", batch_id: a }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
expect((await batch(a)).status).toBe("ask");
await slackPOST(slackPress("intake_grocery", { batch_id: a }));
expect((await batch(a)).status).toBe("forwarded_to_pantry");
await slackPOST(slackPress("intake_meal", { batch_id: b }));
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
expect(await batch(b)).toMatchObject({ status: "duplicate", capture_id: 901 });
expect(lastSlackUpdate().text).toMatch(/^Already handled/);
});
it("two taps that land together still book one transaction, and neither reports an error", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: id })), slackPOST(slackPress("intake_create", { batch_id: id }))]);
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
const texts = calls.filter((c) => c.url.includes("hooks.slack.com")).map((c) => (c.body as { text: string }).text);
expect(texts.every((t) => !/Couldn't process/.test(t))).toBe(true);
});
it("a foreign receipt is booked in its own currency, with no invented AUD figure", async () => {
modelReply = { ...burgertory, currency: "USD" };
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
const t = (await pool.query(`SELECT amount::text, amount_aud, foreign_currency_amount::text, foreign_currency_code FROM transactions`)).rows[0];
expect(t).toEqual({ amount: "22.40", amount_aud: null, foreign_currency_amount: "22.40", foreign_currency_code: "USD" });
expect(lastSlackUpdate().text).toMatch(/USD 22\.40/);
// OrderDetails prefers expense_metadata.currency, so a hard-coded 'AUD'
// here printed "$" on a USD order's receipt panel.
expect((await pool.query(`SELECT currency FROM expense_metadata`)).rows[0].currency).toBe("USD");
});
it("the hold applies to the Create tap's re-match, not only to the upload path", async () => {
modelReply = { ...burgertory, subtotal: null, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
// Booked WITH its items between the card and the tap. The re-match walked
// past the guard routeFood applies, so the same upload destroyed the same
// receipt depending on which button reached it.
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect(await lineItemsOf(txn)).toEqual(already);
expect(await batch(id)).toMatchObject({ status: "held" });
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
});
it("two DIFFERENT batches for one purchase, tapped together, book it once", async () => {
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
// Their source_refs differ, so ON CONFLICT cannot save this; both taps
// clear their match and duplicate checks before either inserts.
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: a })), slackPOST(slackPress("intake_create", { batch_id: b }))]);
const rows = (await pool.query(`SELECT id FROM transactions`)).rows;
expect(rows).toHaveLength(1);
expect((await batch(a)).transaction_id).toBe(rows[0].id);
expect((await batch(b)).transaction_id).toBe(rows[0].id);
});
it("unknown shared_by falls back to the default owner — never a NULL owner", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT owner_id FROM transactions`)).rows[0].owner_id).toBe(1);
});
it("`other` + unrecognised merchant asks; intake_meal routes it as food and remembers the merchant", async () => {
modelReply = { ...burgertory, merchant: "Romeo's Food Hall" };
const txn = await seedOrder({ merchant: "Romeo's Food Hall", date: "2026-08-19", amount: 22.4, platform: null });
const id = uuid();
const body = await (await intakePOST(multipart({ source: "other", batch_id: id }, [pdf]))).json();
expect(body).toMatchObject({ kind: "ask", status: "ask" });
expect(JSON.stringify(body.slack_blocks)).toContain("intake_grocery");
await slackPOST(slackPress("intake_meal", { batch_id: id }));
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
expect(await lineItemsOf(txn)).toHaveLength(2);
expect((await pool.query(`SELECT kind FROM intake_merchant_kinds WHERE merchant_key = 'romeo s food hall'`)).rows[0].kind).toBe("food");
// The answer sticks: the next share of that merchant does not ask.
const again = await (await intakePOST(multipart({ source: "other" }, [{ name: "IMG.png", type: "image/png", bytes: Buffer.from("png2") }]))).json();
expect(again.status).toBe("duplicate"); // same order — but it was routed, not asked
modelReply = { ...burgertory, merchant: "Romeo's Food Hall", order_date: "2026-08-20" };
const next = await (await intakePOST(multipart({ source: "other" }, [{ name: "IMG3.png", type: "image/png", bytes: Buffer.from("png3") }]))).json();
expect(next).toMatchObject({ kind: "food", status: "candidates" });
});
it("intake_grocery forwards the stored original files to pantry", async () => {
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
const id = uuid();
await intakePOST(multipart({ source: "other", batch_id: id }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
expect((await batch(id)).status).toBe("ask");
await slackPOST(slackPress("intake_grocery", { batch_id: id }));
const pantryCall = calls.find((c) => c.url.includes("/api/capture"))!.body as { intake: Record<string, unknown>; files: string[] };
expect(pantryCall.files).toEqual(["Receipt_18Aug2026.pdf"]);
expect(pantryCall.intake).toMatchObject({ batch_id: id, finance_action: "create" });
expect(await batch(id)).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901 });
expect((await pool.query(`SELECT kind FROM intake_merchant_kinds`)).rows[0].kind).toBe("grocery");
});
it("a tap on a resolved batch returns its card and writes nothing; an unknown batch says so", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
await slackPOST(slackPress("intake_create", { batch_id: uuid() }));
expect(lastSlackUpdate()).toMatchObject({ replace_original: false, text: "I can't find that upload any more." });
});
});
src/__tests__/unit/intake-live.test.ts
+37
View File
@@ -0,0 +1,37 @@
import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { resolve } from "path";
import { extractPdfText, liteLLMExtractor } from "../../lib/intake-extract";
import { reconcileItems, toLineItems, validateHeader } from "../../lib/intake-rules";
/**
* Opt-in: the real model over the real Burgertory export.
*
* INTAKE_LIVE_TEST=1 LITELLM_BASE_URL=http://localhost:4200 LITELLM_KEY=... npm test
*
* Skipped otherwise — CI has no LiteLLM, and a unit run must not depend on one.
* What it proves is the thing the mock cannot: that `cloud-budget` keeps the
* names verbatim ("Michael", "Joel", not "Burger") under the order-item prompt.
*/
const live = process.env.INTAKE_LIVE_TEST === "1" && !!process.env.LITELLM_BASE_URL && !!process.env.LITELLM_KEY;
describe.skipIf(!live)("live extraction via LiteLLM (opt-in)", () => {
it("Burgertory PDF: header exact, two items named verbatim, reconciles", async () => {
const text = await extractPdfText(readFileSync(resolve(__dirname, "../fixtures/intake/ubereats-burgertory-2026-08-19.pdf")));
const extracted = await liteLLMExtractor()([{ kind: "text", text, name: "Receipt_19Aug2026.pdf" }]);
const header = validateHeader(extracted);
expect(header.ok && header.header).toEqual({ merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4 });
const items = toLineItems(extracted.items);
expect(items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
expect(reconcileItems(extracted).ok).toBe(true);
}, 120_000);
it("Coles PDF: header exact, 15 items", async () => {
const text = await extractPdfText(readFileSync(resolve(__dirname, "../fixtures/intake/ubereats-coles-2026-08-18.pdf")));
const extracted = await liteLLMExtractor()([{ kind: "text", text, name: "Receipt_18Aug2026.pdf" }]);
const header = validateHeader(extracted);
expect(header.ok && header.header).toEqual({ merchant: "Coles (Wyndham Vale)", order_date: "2026-08-18", total: 85.74 });
expect(toLineItems(extracted.items)).toHaveLength(15);
expect(reconcileItems(extracted).ok).toBe(true);
}, 120_000);
});
src/__tests__/unit/intake-rules.test.ts
+300
View File
@@ -0,0 +1,300 @@
import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { resolve } from "path";
import {
classifyInput,
GROCERY_PATTERN,
merchantKey,
merchantMatches,
platformForSource,
reconcileItems,
resolveMerchantKind,
toLineItems,
validateHeader,
type ExtractedOrder,
} from "../../lib/intake-rules";
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor } from "../../lib/intake-extract";
import { askCard, candidatesCard, enrichedCard, type Block } from "../../lib/intake-blocks";
/**
* The intake router's pure rules, against the strings real receipts print.
* The two PDFs are the user's own Uber Eats exports (plan §3): a Coles grocery
* run and a Burgertory meal whose burgers are named after people.
*/
const fixtures = resolve(__dirname, "../fixtures/intake");
const pdf = (name: string) => readFileSync(resolve(fixtures, name));
describe("classifyInput — input type picks the extractor", () => {
it("PDF by MIME or by name", () => {
expect(classifyInput({ type: "application/pdf", name: "x" })).toBe("pdf");
expect(classifyInput({ type: "application/octet-stream", name: "Receipt_18Aug2026.pdf" })).toBe("pdf");
});
it("images and text", () => {
expect(classifyInput({ type: "image/png", name: "IMG_1.png" })).toBe("image");
expect(classifyInput({ type: "", name: "photo.HEIC" })).toBe("image");
expect(classifyInput({ type: "text/plain", name: "" })).toBe("text");
});
it("declines what it cannot read", () => {
expect(classifyInput({ type: "video/mp4", name: "clip.mp4" })).toBe("unsupported");
expect(classifyInput({ type: "application/zip", name: "a.zip" })).toBe("unsupported");
});
});
describe("platformForSource", () => {
it("maps the tap to expense_metadata.platform vocabulary", () => {
expect(platformForSource("uber_eats")).toBe("ubereats");
expect(platformForSource("doordash")).toBe("doordash");
expect(platformForSource("coles")).toBeNull();
expect(platformForSource("other")).toBeNull();
});
});
describe("merchant pattern — grocery | food | ask", () => {
it("the pattern list is a substring match on the normalised name", () => {
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
expect(resolveMerchantKind("other", "Woolworths Group Limited")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "WOOLIES METRO")).toBe("grocery");
expect(resolveMerchantKind("doordash", "ALDI Werribee")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "IGA Xpress")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "FoodWorks Manor Lakes")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Costco Wholesale")).toBe("grocery");
});
it("a delivery order that is not a grocer is a meal", () => {
expect(resolveMerchantKind("uber_eats", "Burgertory (Werribee)")).toBe("food");
expect(resolveMerchantKind("doordash", "Mad Mex")).toBe("food");
});
it("an empty or unknown merchant asks", () => {
expect(resolveMerchantKind("uber_eats", null)).toBe("ask");
expect(resolveMerchantKind("uber_eats", " ")).toBe("ask");
expect(resolveMerchantKind("doordash", "Unknown")).toBe("ask");
});
it("the Coles/Woolworths taps are ground truth", () => {
expect(resolveMerchantKind("coles", "Romeo's Food Hall")).toBe("grocery");
expect(resolveMerchantKind("woolworths", null)).toBe("grocery");
});
it("`other` with an unrecognised merchant asks; a remembered answer sticks", () => {
expect(resolveMerchantKind("other", "Romeo's Food Hall")).toBe("ask");
expect(resolveMerchantKind("other", "Romeo's Food Hall", "grocery")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Romeo's Food Hall", "grocery")).toBe("grocery");
});
it("merchantKey folds case and punctuation; merchantMatches allows containment only", () => {
expect(merchantKey("Coles (Wyndham Vale)")).toBe("coles wyndham vale");
expect(merchantMatches("Coles (Wyndham Vale)", "coles wyndham vale")).toBe(true);
expect(merchantMatches("Coles (Wyndham Vale)", "Coles")).toBe(true);
expect(merchantMatches("Order - Coles (Wyndham Vale)", "Coles (Wyndham Vale)")).toBe(true);
expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false);
expect(merchantMatches(null, "Coles")).toBe(false);
});
it("whole words, not substrings: `Bigals Pizza` contains `iga` and is neither a grocer nor an IGA", () => {
expect(GROCERY_PATTERN.test(merchantKey("Bigals Pizza"))).toBe(false);
expect(resolveMerchantKind("uber_eats", "Bigals Pizza")).toBe("food");
expect(merchantMatches("Bigals Pizza", "IGA")).toBe(false);
// and the grocers a boundary must not cost us
expect(resolveMerchantKind("uber_eats", "IGA Wyndham Vale")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Woolworths Group Limited")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
expect(merchantMatches("IGA Wyndham Vale", "IGA")).toBe(true);
});
});
describe("validateHeader — nothing is written from a half-read upload", () => {
const base: ExtractedOrder = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, items: [] };
it("accepts a complete header and rounds the total", () => {
const r = validateHeader({ ...base, total: 22.404 });
expect(r.ok && r.header.total).toBe(22.4);
});
it("missing total", () => {
const r = validateHeader({ ...base, total: null });
expect(r.ok).toBe(false);
expect(!r.ok && r.missing).toEqual(["total"]);
});
it("missing or malformed date", () => {
expect(validateHeader({ ...base, order_date: null }).ok).toBe(false);
expect(validateHeader({ ...base, order_date: "19/8/26" }).ok).toBe(false);
});
it("a missing merchant is NOT a missing header — it becomes an ask", () => {
const r = validateHeader({ ...base, merchant: null });
expect(r.ok && r.header.merchant).toBeNull();
});
});
describe("reconcileItems — the missing-screenshot check", () => {
it("reconciles to the printed item subtotal, not the total: fees and promotions are not items", () => {
// The real Coles export: 15 lines summing $118.87 against a printed $118.94 subtotal and an $85.74 total.
const r = reconcileItems({
merchant: "Coles", order_date: "2026-08-18", total: 85.74, subtotal: 118.94,
items: [{ quantity: 1, description: "a", line_total: 118.87 }],
});
expect(r.ok).toBe(true);
expect(r.basis).toBe("subtotal");
});
it("falls back to the total when no subtotal is printed", () => {
const r = reconcileItems({ merchant: "x", order_date: "2026-08-19", total: 28, items: [{ quantity: 1, description: "Michael", line_total: 14 }, { quantity: 1, description: "Joel", line_total: 14 }] });
expect(r.ok).toBe(true);
expect(r.basis).toBe("total");
});
it("holds when the lines fall short — a shot is missing", () => {
const r = reconcileItems({ merchant: "x", order_date: "2026-08-19", total: 28, subtotal: 28, items: [{ quantity: 1, description: "Michael", line_total: 14 }] });
expect(r.ok).toBe(false);
expect(r.reason).toMatch(/14\.00.*28\.00/);
});
it("tolerance is $1 or 2%, whichever is larger", () => {
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, subtotal: 10, items: [{ quantity: 1, description: "a", line_total: 9.05 }] }).ok).toBe(true);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, subtotal: 10, items: [{ quantity: 1, description: "a", line_total: 8.9 }] }).ok).toBe(false);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 200, subtotal: 200, items: [{ quantity: 1, description: "a", line_total: 196.5 }] }).ok).toBe(true);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 200, subtotal: 200, items: [{ quantity: 1, description: "a", line_total: 195 }] }).ok).toBe(false);
});
it("derives a line from quantity × unit price when no line total is printed", () => {
const r = reconcileItems({ merchant: "x", order_date: "d", total: 13.2, subtotal: 13.2, items: [{ quantity: 3, description: "Coke", unit_price: 4.4 }] });
expect(r.ok).toBe(true);
expect(r.items_sum).toBe(13.2);
});
it("no items at all is held; priceless lines (pasted text) are not", () => {
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, items: [] }).ok).toBe(false);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, items: [{ quantity: 1, description: "a" }] }).ok).toBe(true);
});
});
describe("toLineItems — names verbatim, options kept", () => {
it("keeps a burger called Michael called Michael", () => {
const out = toLineItems([{ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] }]);
expect(out).toEqual([{ qty: 1, description: "Michael", amount: 14, unit_price: 14, options: ["Milk Bun"] }]);
});
it("drops nameless lines and defaults quantity to 1", () => {
const out = toLineItems([{ quantity: null, description: " ", line_total: 1 }, { quantity: null, description: "Cucumbers Continental", unit_price: 2.9 }]);
expect(out).toEqual([{ qty: 1, description: "Cucumbers Continental", amount: 2.9, unit_price: 2.9 }]);
});
});
describe("extraction request and reply", () => {
it("one call: the prompt first, then every part in the order shared", () => {
const content = buildMessageContent([
{ kind: "image", mime: "image/png", bytes: Buffer.from("a"), name: "1.png" },
{ kind: "text", text: "Total $22.40", name: "r.pdf" },
{ kind: "image", mime: "image/jpeg", bytes: Buffer.from("b"), name: "2.jpg" },
]) as { type: string; text?: string; image_url?: { url: string } }[];
expect(content[0].text).toMatch(/VERBATIM/);
const images = content.filter((c) => c.type === "image_url").map((c) => c.image_url!.url);
expect(images).toEqual(["data:image/png;base64,YQ==", "data:image/jpeg;base64,Yg=="]);
expect(content.findIndex((c) => c.text?.includes("Total $22.40"))).toBeLessThan(content.findIndex((c) => c.image_url?.url.endsWith("Yg==")));
expect(EXTRACT_RESPONSE_FORMAT.json_schema.strict).toBe(true);
});
it("parses a fenced reply and coerces numbers", () => {
const r = parseExtraction('```json\n{"merchant":"Burgertory (Werribee)","order_date":"2026-08-19","total":"22.40","subtotal":28,"currency":null,"items":[{"quantity":1,"description":"Michael","unit_price":14,"line_total":14,"options":["Milk Bun"]}],"adjustments":[{"label":"Promotion","amount":-8}]}\n```');
expect(r.total).toBe(22.4);
expect(r.currency).toBe("AUD");
expect(r.items[0]).toEqual({ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] });
expect(r.adjustments).toEqual([{ label: "Promotion", amount: -8 }]);
});
});
describe("PDF text — the Uber Eats export is born-digital text", () => {
it("Coles (Wyndham Vale) · 2026-08-18 · $85.74, 15 lines, weighed and substituted", async () => {
const text = await extractPdfText(pdf("ubereats-coles-2026-08-18.pdf"));
expect(text).toContain("Here's your receipt for Coles (Wyndham Vale).");
expect(text).toContain("18 Aug, 2026");
expect(text).toContain("Total $85.74");
expect(text).toContain("Item Subtotal $118.94");
expect(text).toContain("Final 1.16 kg $11.00/kg");
expect(text).toContain("Replaced Coles Beef Slow Cook Chuck Roast");
// 15 item lines: 13 priced per piece, 2 weighed per kg.
expect(text.match(/\$\d+\.\d\d\/pc$/gm)?.length).toBe(13);
expect(text.match(/\$\d+\.\d\d\/kg$/gm)?.length).toBe(2);
});
it("Burgertory (Werribee) · 2026-08-19 · $22.40, burgers named Michael and Joel", async () => {
const text = await extractPdfText(pdf("ubereats-burgertory-2026-08-19.pdf"));
expect(text).toContain("Here's your receipt for Burgertory (Werribee).");
expect(text).toContain("Total $22.40");
expect(text).toContain("1 Michael $14.00");
expect(text).toContain("1 Joel $14.00");
expect(text).toContain("Item Subtotal $28.00");
});
});
describe("cards carry the C5 action ids and JSON values", () => {
const c = { batchId: "11111111-2222-4333-8444-555555555555", merchant: "Burgertory (Werribee)", total: 22.4, orderDate: "2026-08-19", platform: "ubereats", itemCount: 2 };
it("candidates: one Use-this per candidate plus an explicit Create", () => {
const { blocks } = candidatesCard(c, [{ transaction_id: 42, description: "Order - Burgertory (Werribee)", transaction_date: "2026-08-17", amount: "19.90" }]);
const buttons = JSON.stringify(blocks);
expect(buttons).toContain('"action_id":"intake_use_candidate"');
expect(buttons).toContain('"action_id":"intake_create"');
const use = (blocks[1] as { accessory: { value: string } }).accessory.value;
expect(JSON.parse(use)).toEqual({ batch_id: c.batchId, transaction_id: 42 });
});
it("enriched: the confirmation text and a View link", () => {
const { text, blocks } = enrichedCard({ ...c, transactionId: 7 });
expect(text).toBe("Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓");
expect(JSON.stringify(blocks)).toContain("/transactions?q=Burgertory");
});
it("ask: Groceries / A meal", () => {
const ids = JSON.stringify(askCard({ ...c, platform: null }).blocks);
expect(ids).toContain('"action_id":"intake_grocery"');
expect(ids).toContain('"action_id":"intake_meal"');
});
it("every value the card did not write is escaped, plain_text included", () => {
const hostile = { ...c, merchant: "Ben & Jerry's <fake>" };
const { text, blocks } = candidatesCard(hostile, [
{ transaction_id: 42, description: "SQ *BEN & JERRY <AU>", transaction_date: "2026-08-17", amount: "19.90" },
]);
const sections = JSON.stringify(blocks.filter((b) => (b as Block).type === "section"));
expect(sections).toContain("Ben &amp; Jerry's &lt;fake&gt;");
expect(sections).toContain("SQ *BEN &amp; JERRY &lt;AU&gt;");
expect(text).toContain("Ben &amp; Jerry's &lt;fake&gt;");
// plain_text disables formatting, not entity decoding — &, < and > are
// control characters everywhere in a Slack payload.
const whole = JSON.stringify(blocks);
expect(whole).not.toContain("<fake>");
expect(whole).not.toContain("<AU>");
});
it("the currency is model output too, so it is escaped like any other", () => {
// parseExtraction accepts any string for `currency`; unescaped, `<@U123>`
// would be a live mention on every card the batch produces.
const { text } = enrichedCard({ ...c, currency: "<@U123>", transactionId: 7 });
expect(text).toContain("&lt;@U123&gt; 22.40");
expect(text).not.toContain("<@U123>");
});
it("a foreign receipt is not printed as dollars — header and candidate rows alike", () => {
expect(enrichedCard({ ...c, currency: "USD", transactionId: 7 }).text).toBe(
"Added 2 items to your USD 22.40 Uber Eats (Burgertory (Werribee)) order ✓"
);
expect(enrichedCard({ ...c, currency: "AUD", transactionId: 7 }).text).toContain("$22.40");
expect(enrichedCard({ ...c, transactionId: 7 }).text).toContain("$22.40");
// A candidate row is a ledger transaction and carries its own currency; it
// used to print "$" beneath a header reading "USD 22.40".
const { blocks } = candidatesCard({ ...c, currency: "USD" }, [
{ transaction_id: 42, description: "Order - Burgertory", transaction_date: "2026-08-17", amount: "19.90", currency: "USD" },
]);
expect(JSON.stringify(blocks)).toContain("USD 19.90");
});
});
describe("liteLLMExtractor — a finish_reason that is present must be `stop`", () => {
const good = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD", items: [], adjustments: [] };
const extractor = (finish_reason?: string) =>
liteLLMExtractor({
baseUrl: "http://litellm.test",
key: "k",
fetchImpl: (async () =>
Response.json({
choices: [{ message: { content: JSON.stringify(good) }, ...(finish_reason ? { finish_reason } : {}) }],
})) as unknown as typeof fetch,
});
it("accepts `stop`, and — on purpose — a reply that reports no reason at all", async () => {
expect((await extractor("stop")([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
expect((await extractor()([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
});
it("rejects any other reason, naming it — the strict schema only held for a reply that finished", async () => {
// An absent reason is NOT rejected: see the comment in intake-extract.ts.
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(ExtractionError);
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/too long/);
await expect(extractor("content_filter")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/content_filter/);
await expect(extractor("tool_calls")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/tool_calls/);
});
});
src/app/api/intake/route.ts
+67
View File
@@ -0,0 +1,67 @@
import { NextRequest, NextResponse } from "next/server";
import { sha256Of, type StoredFile } from "@/lib/intake-files";
import { liteLLMExtractor } from "@/lib/intake-extract";
import { pantryForwarder } from "@/lib/intake-pantry";
import { isIntakeSource } from "@/lib/intake-rules";
import { MAX_FILES, runIntake } from "@/lib/intake-router";
/**
* The one upload point (plan §11 C1). n8n forwards the iOS Shortcut's
* multipart POST here: `source`, `shared_by`, `batch_id`, and 1–6 files under
* `file`. The router reads what it is given and does every step that applies.
*
* Auth is a shared secret, like the two ingest siblings: this is machine-to-
* machine and there is no browser session to forward. Its own token so the
* Shortcut's credential rotates without touching the order or receipt lanes.
* Fails closed when unset.
*/
function authorised(req: NextRequest): boolean {
const expected = process.env.INTAKE_TOKEN;
if (!expected) return false;
const got = req.headers.get("x-ingest-token");
return !!got && got === expected;
}
const MAX_BYTES = 12_000_000;
export async function POST(req: NextRequest) {
if (!authorised(req)) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
let form: FormData;
try {
form = await req.formData();
} catch {
return NextResponse.json({ error: "multipart/form-data body required" }, { status: 400 });
}
const source = form.get("source");
if (!isIntakeSource(source)) {
return NextResponse.json({ error: "source must be one of uber_eats|doordash|coles|woolworths|other" }, { status: 400 });
}
const sharedBy = typeof form.get("shared_by") === "string" ? String(form.get("shared_by")).trim() || null : null;
const batchIdRaw = form.get("batch_id");
const batchId = typeof batchIdRaw === "string" && /^[0-9a-f-]{36}$/i.test(batchIdRaw) ? batchIdRaw.toLowerCase() : null;
if (batchIdRaw && !batchId) return NextResponse.json({ error: "batch_id must be a uuid" }, { status: 400 });
const files: StoredFile[] = [];
for (const entry of form.getAll("file")) {
if (!(entry instanceof File) || !entry.size) continue;
if (entry.size > MAX_BYTES) return NextResponse.json({ error: `${entry.name} is larger than 12 MB` }, { status: 400 });
const bytes = Buffer.from(await entry.arrayBuffer());
files.push({ name: entry.name || `file-${files.length + 1}`, type: entry.type || "", bytes, sha256: sha256Of(bytes) });
}
if (!files.length) return NextResponse.json({ error: "at least one file is required under field 'file'" }, { status: 400 });
if (files.length > MAX_FILES) return NextResponse.json({ error: `at most ${MAX_FILES} files per batch` }, { status: 400 });
try {
const result = await runIntake(
{ batchId, source, sharedBy, files },
{ extractor: liteLLMExtractor(), pantry: pantryForwarder() }
);
const { http_status, ...body } = result;
return NextResponse.json(body, { status: http_status });
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return NextResponse.json({ error: message }, { status: 500 });
}
}
src/app/api/slack/interactive/route.ts
+28
View File
@@ -6,6 +6,8 @@ import {
slackUserForParticipant,
} from "@/lib/slack-verify";
import { nudgeBlocks, detailsModal, partnerNudgeBlocks } from "@/lib/slack-blocks";
import { handleIntakeAction, isIntakeAction, parseActionValue } from "@/lib/intake-actions";
import { pantryForwarder } from "@/lib/intake-pantry";
import { completeSplit } from "@/lib/splits";
import {
RATINGS,
@@ -73,6 +75,22 @@ export async function POST(req: NextRequest) {
if (payload.type !== "block_actions") return new NextResponse(null, { status: 200 });
const action = payload.actions?.[0];
// Intake taps (plan §11 C5) act on a persisted batch, not a transaction, so
// their `value` is JSON `{batch_id, ...}` rather than `<id>:<verb>`. They are
// dispatched on action_id before the order-nudge parse below, which would
// otherwise read the JSON as a non-numeric transaction id and give up.
if (isIntakeAction(action?.action_id)) {
const reply = await handleIntakeAction(action.action_id, parseActionValue(action?.value), { pantry: pantryForwarder() });
await updateMessage(
payload.response_url,
reply.replace
? { replace_original: true, blocks: reply.blocks, text: reply.text }
: { response_type: "ephemeral", replace_original: false, text: reply.text }
);
return NextResponse.json({ blocks: reply.blocks, text: reply.text });
}
// A button carries `value`; a select carries it on the chosen option. The
// ratings moved to a select because four buttons became four full-width rows
// on Slack mobile, so both shapes have to resolve or rating silently stops
@@ -216,6 +234,16 @@ async function buildDetailsModal(transactionId: number, participantId: number) {
`SELECT merchant_normalized AS merchant, line_items
FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[transactionId]
);
src/app/api/transactions/[id]/line-items/route.ts
+58
View File
@@ -0,0 +1,58 @@
import { NextRequest, NextResponse } from "next/server";
import { LineItemsValidationError, replaceLineItems, TransactionNotFoundError, validateLineItems } from "@/lib/line-items";
/**
* Enrich endpoint (plan §11 C2): idempotent replace of the itemised receipt
* behind a transaction.
*
* Accepts either machine token. Pantry already holds RECEIPT_INGEST_TOKEN for
* the ING-10 lane and calls this on every confirm and on undo; the intake
* router holds INTAKE_TOKEN. No session auth — there is no browser here.
* Fails closed when neither is configured.
*
* Never creates a transaction: 404 when the id does not exist. Creates the
* `expense_metadata` row when the transaction has none.
*/
function authorised(req: NextRequest): boolean {
const got = req.headers.get("x-ingest-token");
if (!got) return false;
const accepted = [process.env.INTAKE_TOKEN, process.env.RECEIPT_INGEST_TOKEN].filter(Boolean);
return accepted.includes(got);
}
export async function PUT(req: NextRequest, { params }: { params: Promise<{ id: string }> }) {
if (!authorised(req)) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
const { id } = await params;
const transactionId = Number(id);
if (!Number.isInteger(transactionId) || transactionId <= 0) return NextResponse.json({ error: "invalid transaction id" }, { status: 400 });
let body: { line_items?: unknown; source?: unknown; batch_id?: unknown; replace?: unknown; platform?: unknown };
try {
body = await req.json();
} catch {
return NextResponse.json({ error: "invalid JSON" }, { status: 400 });
}
if (body.replace !== true) return NextResponse.json({ error: "replace: true is required — this endpoint only replaces" }, { status: 400 });
if (body.source !== "intake" && body.source !== "pantry") return NextResponse.json({ error: "source must be 'intake' or 'pantry'" }, { status: 400 });
const batchId = typeof body.batch_id === "string" && body.batch_id ? body.batch_id : null;
const platform = typeof body.platform === "string" && body.platform ? body.platform : null;
try {
const lineItems = validateLineItems(body.line_items);
const result = await replaceLineItems(transactionId, lineItems, { source: body.source, platform, batchId });
return NextResponse.json({
transaction_id: result.transaction_id,
metadata_id: result.metadata_id,
created_metadata: result.created_metadata,
previous_line_items: result.previous_line_items,
line_items: result.line_items,
});
} catch (e) {
if (e instanceof LineItemsValidationError) return NextResponse.json({ error: e.message }, { status: 400 });
if (e instanceof TransactionNotFoundError) return NextResponse.json({ error: e.message }, { status: 404 });
const message = e instanceof Error ? e.message : String(e);
return NextResponse.json({ error: message }, { status: 500 });
}
}
src/app/api/transactions/[id]/order/route.ts
+8
View File
@@ -34,6 +34,14 @@ export async function GET(
-- through matched_transaction_id. Both directions have to resolve or the
-- detail is missing on exactly the orders that were paid by card.
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- The same order as replaceLineItems in line-items.ts and matchTransaction
-- in intake-router.ts. Both directions can resolve at once — a statement
-- line owning its own row while a delivery receipt points at it — and an
-- unordered LIMIT 1 then reads a different row from the one an enrichment
-- wrote, so a PUT that reported success showed nothing on the screen.
-- NULLS LAST because DESC is NULLS FIRST and the pointing row's
-- transaction_id is NULL.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[Number(id)]
);
src/components/order-details.tsx
+4 -7
View File
@@ -13,12 +13,9 @@ import {
type OrderReceipt,
type OrderRating,
} from "@/lib/hooks";
const PLATFORM_LABEL: Record<string, string> = {
doordash: "DoorDash",
ubereats: "Uber Eats",
uber: "Uber",
};
// One platform vocabulary, shared with the intake cards — three copies had
// already drifted apart (the intake one was missing `uber`).
import { platformLabel } from "@/lib/intake-rules";
/**
* Who records verdicts. A two-person household with one primary user: the
@@ -99,7 +96,7 @@ export function OrderDetails({
<p className="text-xs text-zinc-500">
Order details
{receipt.platform && (
<span className="ml-1.5 text-zinc-400">{PLATFORM_LABEL[receipt.platform] ?? receipt.platform}</span>
<span className="ml-1.5 text-zinc-400">{platformLabel(receipt.platform)}</span>
)}
</p>
{receipt.card_last4 && (
src/lib/db.ts
+30
View File
@@ -20,3 +20,33 @@ export async function queryRow<T>(sql: string, params: unknown[] = []): Promise<
const rows = await prisma.$queryRawUnsafe<T[]>(sql, ...params);
return rows.length > 0 ? rows[0] : null;
}
/** The same two calls, scoped to one connection inside one transaction. */
export type Queryable = {
queryRaw<T>(sql: string, params?: unknown[]): Promise<T[]>;
queryRow<T>(sql: string, params?: unknown[]): Promise<T | null>;
};
/**
* Run several statements as one transaction, on one connection.
*
* The module-level `queryRaw`/`queryRow` are `$queryRawUnsafe` on a POOLED
* client, so each call is its own implicit transaction on whichever connection
* the pool hands out. That makes a check-then-write pair non-atomic, and it
* makes `pg_advisory_xact_lock` useless outside this helper — the lock is
* released the instant the statement that took it commits, which is before the
* next statement runs. Anything that needs a lock to still be held on its next
* query has to be in here.
*/
export async function withTransaction<T>(fn: (q: Queryable) => Promise<T>): Promise<T> {
return prisma.$transaction(async (tx) => {
const q: Queryable = {
queryRaw: <R>(sql: string, params: unknown[] = []) => tx.$queryRawUnsafe<R[]>(sql, ...params),
queryRow: async <R>(sql: string, params: unknown[] = []) => {
const rows = await tx.$queryRawUnsafe<R[]>(sql, ...params);
return rows.length > 0 ? rows[0] : null;
},
};
return fn(q);
});
}
src/lib/intake-actions.ts
+119
View File
@@ -0,0 +1,119 @@
import { loadFiles } from "@/lib/intake-files";
import type { PantryForward } from "@/lib/intake-pantry";
import type { IntakeSource } from "@/lib/intake-rules";
import {
createFromBatch,
loadBatch,
rememberKind,
resolvedCard,
routeFood,
routeGrocery,
type BatchRow,
type RouteContext,
} from "@/lib/intake-router";
import { replaceLineItems, TransactionNotFoundError } from "@/lib/line-items";
import { enrichedCard, errorCard, type Block } from "@/lib/intake-blocks";
import { queryRaw } from "@/lib/db";
/**
* The four intake taps (plan §11 C5), as one function the Slack route calls.
*
* Every tap acts on the persisted batch, never on the card's text. A tap on a
* batch that is already resolved returns its current card rather than acting
* twice — a pressed button that looked dead and got pressed again is how three
* splits were lost on the order nudge, and the same shape applies here.
*/
export const INTAKE_ACTIONS = ["intake_use_candidate", "intake_create", "intake_grocery", "intake_meal"] as const;
export type IntakeActionId = (typeof INTAKE_ACTIONS)[number];
export function isIntakeAction(id: unknown): id is IntakeActionId {
return typeof id === "string" && (INTAKE_ACTIONS as readonly string[]).includes(id);
}
export type IntakeActionValue = { batch_id?: string; transaction_id?: number };
export function parseActionValue(raw: unknown): IntakeActionValue {
if (typeof raw !== "string") return {};
try {
const v = JSON.parse(raw) as IntakeActionValue;
return v && typeof v === "object" ? v : {};
} catch {
return {};
}
}
const note = (text: string): { blocks: Block[]; text: string; replace: boolean } => ({
text,
blocks: [{ type: "section", text: { type: "mrkdwn", text } }],
replace: false,
});
const ctxFor = (batch: BatchRow): RouteContext | null =>
batch.header
? {
batchId: batch.id,
source: batch.source as IntakeSource,
sharedBy: batch.shared_by,
platform: batch.header.platform,
header: batch.header,
items: batch.items ?? [],
currency: batch.header.currency,
}
: null;
export async function handleIntakeAction(
actionId: IntakeActionId,
value: IntakeActionValue,
deps: { pantry: PantryForward }
): Promise<{ blocks: Block[]; text: string; replace: boolean }> {
if (!value.batch_id) return note("That button carried no batch id — I can't tell which upload it was.");
const batch = await loadBatch(value.batch_id);
if (!batch) return note("I can't find that upload any more.");
if (batch.status === "enriched" || batch.status === "created" || batch.status === "forwarded_to_pantry") {
return { ...resolvedCard(batch), replace: true };
}
const ctx = ctxFor(batch);
if (!ctx) return note("That upload never got a readable header — share it again with the top of the screen.");
try {
if (actionId === "intake_use_candidate") {
if (!Number.isInteger(value.transaction_id)) return note("That button carried no transaction id.");
const txnId = value.transaction_id as number;
await replaceLineItems(txnId, ctx.items, { source: "intake", platform: ctx.platform, batchId: batch.id, currency: ctx.currency });
await queryRaw(
`UPDATE intake_batches SET status = 'enriched', kind = COALESCE(kind, 'food'), transaction_id = $2, resolved_at = now() WHERE id = $1::uuid`,
[batch.id, txnId]
);
return {
...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, currency: ctx.currency, transactionId: txnId }),
replace: true,
};
}
if (actionId === "intake_create") {
const kind = batch.kind === "grocery" ? "grocery" : "food";
const r = await createFromBatch(batch, kind);
return { blocks: r.slack_blocks ?? [], text: r.text, replace: true };
}
if (actionId === "intake_meal") {
await rememberKind(ctx.header.merchant, "food");
const r = await routeFood(ctx);
return { blocks: r.slack_blocks ?? [], text: r.text, replace: true };
}
// intake_grocery
await rememberKind(ctx.header.merchant, "grocery");
const files = await loadFiles(batch.file_sha256);
if (!files.length) {
return note("I remembered that as groceries, but the original files are gone — share it again and it will go straight to the pantry board.");
}
const r = await routeGrocery(ctx, files, deps.pantry);
return { blocks: r.slack_blocks ?? [], text: r.text, replace: r.status !== "error" };
} catch (e) {
if (e instanceof TransactionNotFoundError) return note("That transaction is no longer in the ledger.");
const detail = e instanceof Error ? e.message : String(e);
return { ...errorCard(detail), replace: false };
}
}
src/lib/intake-blocks.ts
+175
View File
@@ -0,0 +1,175 @@
import { platformLabel } from "@/lib/intake-rules";
/**
* The intake reply as Block Kit (plan §11 C1/C5). Built here, never in n8n
* expressions, so the button payloads the interactive handler round-trips are
* produced by one piece of code and tested like one.
*
* Button `value`s are JSON `{batch_id, ...}` — the batch row is what a later
* tap acts on. action_ids are the C5 contract: `intake_use_candidate`,
* `intake_create`, `intake_grocery`, `intake_meal`.
*/
export type Block = Record<string, unknown>;
export type Candidate = { transaction_id: number; description: string | null; transaction_date: string; amount: string | number; currency?: string | null };
export function financePublicUrl(): string {
return (process.env.FINANCE_PUBLIC_URL || "https://finance.bosecamp.com").replace(/\/$/, "");
}
/** The transactions page searches on `q`; the merchant is the most selective thing a card knows. */
export function transactionLink(merchant: string | null): string {
return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`;
}
/**
* Slack renders `&`, `<` and `>` as markup in mrkdwn (and in the `text`
* fallback). Merchant names, transaction descriptions and model output are
* none of ours — `Ben & Jerry's` must not arrive as `Ben &amp; Jerry's`, and a
* description containing `<http://…|click>` must not become a link the card
* never meant to show. Applied to interpolated values only; our own literal
* `*bold*` and `:emoji:` stay markup.
*/
export function esc(value: string | null | undefined): string {
return (value ?? "").replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
}
/**
* `$` is only right for AUD. An intake batch carries the currency the receipt
* printed, and the order nudge already renders `USD 15.00` rather than
* pretending a foreign order was dollars.
*/
const money = (n: number | string, currency?: string | null) => {
const amount = Number(n).toFixed(2);
// The currency comes off the model (`parseExtraction` accepts any string), so
// an unescaped `<@U123>` here would be a mention on every card.
return !currency || currency === "AUD" ? `$${amount}` : `${esc(currency)} ${amount}`;
};
const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } });
const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({
type: "button",
action_id,
text: { type: "plain_text", text, emoji: true },
value: JSON.stringify(value),
...(style ? { style } : {}),
});
export type CardInput = {
batchId: string;
merchant: string | null;
total: number | null;
orderDate: string | null;
platform: string | null;
itemCount: number;
currency?: string | null;
};
export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const merchant = esc(c.merchant);
const what = [c.total !== null ? money(c.total, c.currency) : null, esc(platformLabel(c.platform)) || merchant, c.platform && merchant ? `(${merchant})` : null]
.filter(Boolean)
.join(" ");
const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`;
return {
text,
blocks: [
{
type: "section",
text: { type: "mrkdwn", text: `:receipt: ${text}` },
accessory: { type: "button", text: { type: "plain_text", text: "View" }, url: transactionLink(c.merchant), action_id: "intake_view" },
},
],
};
}
export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const text = `Created a ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant) || "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`;
return {
text,
blocks: [
{
type: "section",
text: { type: "mrkdwn", text: `:receipt: ${text}` },
accessory: { type: "button", text: { type: "plain_text", text: "View" }, url: transactionLink(c.merchant), action_id: "intake_view" },
},
],
};
}
export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } {
const label = esc(platformLabel(c.platform));
const head = `No matching ${label ? label + " " : ""}order for *${esc(c.merchant) || "this merchant"}* ${c.total !== null ? money(c.total, c.currency) : ""} on ${esc(c.orderDate) || "?"}.`;
const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`;
const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)];
for (const cand of candidates) {
blocks.push({
type: "section",
text: { type: "mrkdwn", text: `${esc(cand.transaction_date)} · ${esc(cand.description) || "—"} · ${money(cand.amount, cand.currency)}` },
accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }),
});
}
blocks.push({
type: "actions",
block_id: "intake_create",
// plain_text disables *formatting*, not entity decoding: `&`, `<` and `>`
// are control characters everywhere in a Slack payload, so the label is
// escaped like any other value we did not write.
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant)} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
});
return { blocks, text };
}
export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } {
const text = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " ");
return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
}
export function needsHeaderCard(): { blocks: Block[]; text: string } {
const text = "I can see items but no order total or date — share the top of the screen too so I can match it.";
return { text, blocks: [section(`:warning: ${text}`)] };
}
export function heldCard(reason: string, headline = "Items don't add up to the total — is a shot missing?"): { blocks: Block[]; text: string } {
const text = `${headline} (${esc(reason)})`;
return { text, blocks: [section(`:warning: ${text}`)] };
}
export function askCard(c: CardInput): { blocks: Block[]; text: string } {
const text = `${esc(c.merchant) || "This upload"} ${c.total !== null ? money(c.total, c.currency) : ""} — groceries or a meal?`.replace(/\s+/g, " ");
return {
text,
blocks: [
section(`:thinking_face: ${text}`),
{
type: "actions",
block_id: "intake_ask",
elements: [
button("intake_grocery", "Groceries", { batch_id: c.batchId }),
button("intake_meal", "A meal", { batch_id: c.batchId }),
],
},
],
};
}
export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null; currency?: string | null }): { blocks: Block[]; text: string } {
const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${esc(c.batchId)}` : "an earlier upload";
// The header (merchant, date, total) IS the identity of a purchase here, so a
// genuine SECOND identical purchase — two of the same coffee, one day, one shop
// — reads as a re-share and is refused. That is the right default (a re-share is
// far commoner than a repeat) but it is a dead end without this sentence: the
// card must say what to do instead, or the ledger quietly loses the second one.
const text = `Already handled — ${esc(c.merchant) || "this order"} ${c.total !== null ? money(c.total, c.currency) : ""} is ${where}. Nothing was changed — if this really is a second, separate purchase, add it in the app.`.replace(/\s+/g, " ");
return { text, blocks: [section(`:repeat: ${text}`)] };
}
export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } {
const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${esc(detail)})` : ""}`;
return { text, blocks: [section(`:no_entry_sign: ${text}`)] };
}
export function errorCard(detail: string): { blocks: Block[]; text: string } {
const text = `Couldn't process that upload: ${esc(detail)}`;
return { text, blocks: [section(`:x: ${text}`)] };
}
src/lib/intake-extract.ts
+216
View File
@@ -0,0 +1,216 @@
import { extractText, getDocumentProxy } from "unpdf";
import type { ExtractedOrder, ExtractedItem } from "@/lib/intake-rules";
/**
* Extraction for the intake router: one joint model call per batch.
*
* The model EXTRACTS; it does not classify or route. The source tap and the
* merchant pattern decide where the result goes (plan §2 — "a classifier is
* confidently wrong exactly when it matters"). So this prompt asks for the
* header and the lines, verbatim, and nothing else.
*
* One call with every part in order, never one per image: scrolling screenshots
* overlap, and extracting each and concatenating double-counts the row both
* shots contain and strands the lower shot with no header (plan §3, Codex #7).
* Dedupe of the overlap is the model's job; the reconciliation check in
* `intake-rules.ts` is what catches it getting that wrong.
*/
export type InputPart =
| { kind: "image"; mime: string; bytes: Buffer; name: string }
| { kind: "text"; text: string; name: string };
/**
* Digital receipts are text, not pixels. Reading the text beats photographing
* the screen: nothing is misread, and the model parses a page rather than an
* image. Same approach as pantry-app's `pdf-text.ts`.
*/
export async function extractPdfText(bytes: Buffer): Promise<string> {
const document = await getDocumentProxy(new Uint8Array(bytes));
const { text } = await extractText(document, { mergePages: true });
return (Array.isArray(text) ? text.join("\n") : text).trim();
}
export const ORDER_ITEM_PROMPT = `You are reading a purchase receipt: an online food or grocery order (Uber Eats, DoorDash, Coles, Woolworths, ...) or an in-store till receipt. It arrives as one or more screenshots in scroll order, as the text of a PDF, or as pasted text.
Return ONE JSON object with exactly these keys:
{ "merchant", "order_date", "total", "subtotal", "currency", "items": [{ "quantity", "description", "unit_price", "line_total", "options" }], "adjustments": [{ "label", "amount" }] }
Rules:
- merchant: the store or restaurant name exactly as printed, including any branch in parentheses (e.g. "Coles (Wyndham Vale)", "Burgertory (Werribee)"). Never the delivery platform. null if not shown.
- order_date: the order or purchase date printed, as YYYY-MM-DD. null if not shown.
- total: the amount actually charged — the "Total" / amount paid — as a number. null if not shown.
- subtotal: the printed item subtotal before fees and discounts, as a number, or null.
- currency: ISO code if printed (default "AUD").
- items: every purchased line. Keep description VERBATIM as printed — do not rename, canonicalise, translate, shorten or summarise it. A burger called "Michael" stays "Michael"; "Coles RSPCA Approved Chicken Thigh Cutlets approx. 1.1kg" stays exactly that. Put selected options and modifiers (e.g. "Milk Bun", "Chicken") and weighed or substitution notes (e.g. "Final 1.16 kg @ $11.00/kg", "Replaced Coles Beef Slow Cook Chuck Roast approx. 1.3kg") in options, as an array of strings, not in the description.
- quantity: the number ordered (a number; 1 if a line shows none). unit_price: the price per unit as printed, or null. line_total: the money for the whole line as printed, or quantity × unit_price when only a unit price is shown; for weighed lines multiply the final weight by the per-kg price. null when nothing is shown.
- Do not put fees, tips, delivery or service charges, discounts, promotions, savings, tax lines or payment lines in items. List those in adjustments instead, fees positive and discounts negative.
- Multiple screenshots are consecutive scrolls of ONE order and overlap: each physical line appears exactly once in items, never twice.
- Never invent a value. If something is not legible or not present, use null.`;
const NUM_OR_NULL = { anyOf: [{ type: "number" }, { type: "null" }] };
const STR_OR_NULL = { anyOf: [{ type: "string" }, { type: "null" }] };
/** Strict json_schema: every property listed in `required`, nullability via anyOf. */
export const EXTRACT_RESPONSE_FORMAT = {
type: "json_schema",
json_schema: {
name: "order_items",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["merchant", "order_date", "total", "subtotal", "currency", "items", "adjustments"],
properties: {
merchant: STR_OR_NULL,
order_date: STR_OR_NULL,
total: NUM_OR_NULL,
subtotal: NUM_OR_NULL,
currency: STR_OR_NULL,
items: {
type: "array",
items: {
type: "object",
additionalProperties: false,
required: ["quantity", "description", "unit_price", "line_total", "options"],
properties: {
quantity: NUM_OR_NULL,
description: { type: "string" },
unit_price: NUM_OR_NULL,
line_total: NUM_OR_NULL,
options: { type: "array", items: { type: "string" } },
},
},
},
adjustments: {
type: "array",
items: {
type: "object",
additionalProperties: false,
required: ["label", "amount"],
properties: { label: { type: "string" }, amount: { type: "number" } },
},
},
},
},
},
} as const;
/** The OpenAI-format message content: prompt first, then every part in the order shared. */
export function buildMessageContent(parts: InputPart[]): unknown[] {
const content: unknown[] = [{ type: "text", text: ORDER_ITEM_PROMPT }];
parts.forEach((p, i) => {
if (p.kind === "text") {
content.push({ type: "text", text: `\n--- Input ${i + 1} of ${parts.length} (${p.name}) ---\n${p.text}` });
} else {
content.push({ type: "text", text: `\n--- Input ${i + 1} of ${parts.length}: image ${p.name} ---` });
content.push({ type: "image_url", image_url: { url: `data:${p.mime || "image/jpeg"};base64,${p.bytes.toString("base64")}` } });
}
});
return content;
}
const num = (v: unknown): number | null => {
if (typeof v === "number" && Number.isFinite(v)) return v;
if (typeof v === "string" && v.trim()) {
const n = Number(v.replace(/[^0-9.-]/g, ""));
return Number.isFinite(n) ? n : null;
}
return null;
};
/** Tolerant of fences and a stray preamble; strict about the shape it hands on. */
export function parseExtraction(content: string): ExtractedOrder {
const unfenced = content.trim().replace(/^```(?:json)?\s*/i, "").replace(/\s*```$/, "");
const start = unfenced.indexOf("{");
const end = unfenced.lastIndexOf("}");
const candidate = start >= 0 && end > start ? unfenced.slice(start, end + 1) : unfenced;
const raw = JSON.parse(candidate) as Record<string, unknown>;
const items: ExtractedItem[] = Array.isArray(raw.items)
? (raw.items as Record<string, unknown>[])
.filter((i) => i && typeof i.description === "string")
.map((i) => ({
quantity: num(i.quantity),
description: String(i.description),
unit_price: num(i.unit_price),
line_total: num(i.line_total),
options: Array.isArray(i.options) ? i.options.map(String) : [],
}))
: [];
return {
merchant: typeof raw.merchant === "string" ? raw.merchant : null,
order_date: typeof raw.order_date === "string" ? raw.order_date : null,
total: num(raw.total),
subtotal: num(raw.subtotal),
currency: typeof raw.currency === "string" && raw.currency ? raw.currency : "AUD",
items,
adjustments: Array.isArray(raw.adjustments)
? (raw.adjustments as Record<string, unknown>[])
.filter((a) => a && typeof a.label === "string" && num(a.amount) !== null)
.map((a) => ({ label: String(a.label), amount: num(a.amount) as number }))
: [],
};
}
export type Extractor = (parts: InputPart[]) => Promise<ExtractedOrder>;
export class ExtractionError extends Error {}
/**
* LiteLLM, OpenAI-compatible, alias `cloud-budget` — the same alias pantry's
* capture uses, so a model swap is one config line in `litellm-config.yaml`.
* `temperature: 0`; `response_format` is the strict schema above.
*/
export function liteLLMExtractor(opts: {
baseUrl?: string;
key?: string;
model?: string;
fetchImpl?: typeof fetch;
} = {}): Extractor {
return async (parts) => {
const baseUrl = (opts.baseUrl ?? process.env.LITELLM_BASE_URL ?? "").replace(/\/$/, "");
const key = opts.key ?? process.env.LITELLM_KEY;
if (!baseUrl || !key) throw new ExtractionError("LITELLM_BASE_URL / LITELLM_KEY are not configured");
const doFetch = opts.fetchImpl ?? fetch;
const response = await doFetch(`${baseUrl}/v1/chat/completions`, {
method: "POST",
headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
body: JSON.stringify({
model: opts.model ?? "cloud-budget",
temperature: 0,
max_tokens: 16384,
response_format: EXTRACT_RESPONSE_FORMAT,
messages: [{ role: "user", content: buildMessageContent(parts) }],
}),
});
if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`);
const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] };
// The rule is: a finish_reason that is PRESENT must be `stop`. An absent
// one is accepted, deliberately — providers differ on whether they send the
// field, and failing every call on a provider that omits it would be a
// certain outage in exchange for a case that also requires the model to
// emit syntactically complete JSON. The harm a truncated reply does — a
// partial item list replacing a good one — is closed on the write side
// instead, by the guard in `routeFood`/`createFromBatch`.
//
// `length` is the common one: a truncated reply still parses once the JSON
// happens to close, quietly dropping the tail of a long receipt. Any other
// reason (`content_filter`, `tool_calls`) is a different reply shape that
// the strict schema never constrained.
const finish = raw.choices?.[0]?.finish_reason;
if (finish && finish !== "stop") {
throw new ExtractionError(
finish === "length"
? "The receipt was too long to read in one pass."
: `The model stopped before it finished (${finish}).`
);
}
const content = raw.choices?.[0]?.message?.content;
if (!content) throw new ExtractionError("LiteLLM returned no content");
try {
return parseExtraction(content);
} catch (e) {
throw new ExtractionError(`could not parse extraction: ${e instanceof Error ? e.message : String(e)}`);
}
};
}
src/lib/intake-files.ts
+51
View File
@@ -0,0 +1,51 @@
import { createHash } from "node:crypto";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import path from "node:path";
/**
* The attachment store: the original files of a batch, keyed by sha256.
*
* Two reasons it exists. A "Groceries or a meal?" answer arrives minutes after
* the upload, in a fresh request carrying only the batch id, and "groceries"
* means forwarding the ORIGINAL files to pantry — so they have to still be
* here. And plan §8: "the source file is retrievable from the attachment store
* by its sha256".
*
* Plain files under `INTAKE_FILE_DIR` (default `/app/data/intake`); a sidecar
* `.json` keeps the name and MIME type. No database column holds bytes.
*/
export type StoredFile = { name: string; type: string; bytes: Buffer; sha256: string };
export function intakeFileDir(): string {
return process.env.INTAKE_FILE_DIR || "/app/data/intake";
}
export function sha256Of(bytes: Buffer): string {
return createHash("sha256").update(bytes).digest("hex");
}
export async function storeFiles(files: StoredFile[]): Promise<void> {
const dir = intakeFileDir();
await mkdir(dir, { recursive: true });
for (const f of files) {
await writeFile(path.join(dir, `${f.sha256}.bin`), f.bytes);
await writeFile(path.join(dir, `${f.sha256}.json`), JSON.stringify({ name: f.name, type: f.type }));
}
}
/** The files that are still on disk, in the order asked for; missing ones are skipped. */
export async function loadFiles(shas: string[]): Promise<StoredFile[]> {
const dir = intakeFileDir();
const out: StoredFile[] = [];
for (const sha of shas) {
try {
const bytes = await readFile(path.join(dir, `${sha}.bin`));
const meta = JSON.parse(await readFile(path.join(dir, `${sha}.json`), "utf-8")) as { name?: string; type?: string };
out.push({ name: meta.name ?? `${sha}.bin`, type: meta.type ?? "application/octet-stream", bytes, sha256: sha });
} catch {
/* gone — the caller decides what that means */
}
}
return out;
}
src/lib/intake-pantry.ts
+76
View File
@@ -0,0 +1,76 @@
import type { StoredFile } from "@/lib/intake-files";
/**
* finance → pantry hand-off for a grocery upload (plan §11 C3).
*
* The ORIGINAL files go across, not the extracted items: pantry owns grocery
* extraction (canonical names, categories, staples) and its capture board is
* the review surface. What finance adds is the routing decision — which
* transaction the confirm should enrich, or that it should create one — carried
* in the `intake` field so pantry's confirm seam (C4) can suppress the ING-10
* create that would otherwise double-book an order the email already booked.
*/
export type PantryForwardArgs = {
files: StoredFile[];
batchId: string;
platform: string | null;
sharedBy: string | null;
financeTargetTransactionId: number | null;
financeAction: "enrich" | "create";
};
export type PantryForwardResult = { captureId: number; itemCount: number | null };
export class PantryDuplicateError extends Error {
constructor(
public readonly captureId: number | null,
public readonly reason: string
) {
super(reason);
}
}
export class PantryUnavailableError extends Error {}
export type PantryForward = (args: PantryForwardArgs) => Promise<PantryForwardResult>;
export function pantryForwarder(opts: { baseUrl?: string; token?: string; fetchImpl?: typeof fetch } = {}): PantryForward {
return async (args) => {
const baseUrl = (opts.baseUrl ?? process.env.PANTRY_BASE_URL ?? "").replace(/\/$/, "");
const token = opts.token ?? process.env.RECEIPT_INGEST_TOKEN;
if (!baseUrl || !token) throw new PantryUnavailableError("PANTRY_BASE_URL / RECEIPT_INGEST_TOKEN are not configured");
const doFetch = opts.fetchImpl ?? fetch;
const form = new FormData();
for (const f of args.files) {
form.append("image", new Blob([new Uint8Array(f.bytes)], { type: f.type }), f.name);
}
form.append("source", "receipt");
form.append(
"intake",
JSON.stringify({
batch_id: args.batchId,
platform: args.platform,
origin: "intake",
shared_by: args.sharedBy,
finance_target_transaction_id: args.financeTargetTransactionId,
finance_action: args.financeAction,
})
);
let response: Response;
try {
response = await doFetch(`${baseUrl}/api/capture`, { method: "POST", headers: { "x-ingest-token": token }, body: form });
} catch (e) {
throw new PantryUnavailableError(`pantry unreachable: ${e instanceof Error ? e.message : String(e)}`);
}
const body = (await response.json().catch(() => ({}))) as { captureId?: number; error?: string; items?: unknown[]; reason?: string };
if (response.status === 409) {
throw new PantryDuplicateError(typeof body.captureId === "number" ? body.captureId : null, body.reason ?? body.error ?? "already captured");
}
if (!response.ok) throw new PantryUnavailableError(`pantry returned ${response.status}: ${body.error ?? ""}`.trim());
if (typeof body.captureId !== "number") throw new PantryUnavailableError("pantry returned no captureId");
return { captureId: body.captureId, itemCount: Array.isArray(body.items) ? body.items.length : null };
};
}
src/lib/intake-router.ts
+647
View File
@@ -0,0 +1,647 @@
import { randomUUID } from "node:crypto";
import { queryRaw, queryRow, withTransaction } from "@/lib/db";
import { EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY } from "@/lib/analytics-sql";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import {
classifyInput,
merchantKey,
merchantMatches,
platformForSource,
reconcileItems,
resolveMerchantKind,
toLineItems,
validateHeader,
type ExtractedOrder,
type IntakeSource,
type MerchantKind,
type OrderHeader,
type StoredLineItem,
} from "@/lib/intake-rules";
import { extractPdfText, ExtractionError, type Extractor, type InputPart } from "@/lib/intake-extract";
import { storeFiles, type StoredFile } from "@/lib/intake-files";
import { PantryDuplicateError, PantryUnavailableError, type PantryForward } from "@/lib/intake-pantry";
import { replaceLineItems } from "@/lib/line-items";
import {
askCard,
candidatesCard,
createdCard,
duplicateCard,
enrichedCard,
errorCard,
forwardedCard,
heldCard,
needsHeaderCard,
unsupportedCard,
type Block,
type Candidate,
type CardInput,
} from "@/lib/intake-blocks";
/**
* The intake router (plan §11 C1): one upload, every applicable step.
*
* Order of operations matters and is the plan's:
* persist the batch → classify each file → ONE extraction call → header →
* merchant kind → duplicate check → reconciliation → match → act.
*
* The batch row is written before anything else so that a Slack tap minutes
* later has something to act on, and so a re-share is recognisable. Nothing
* else is written until the header is whole: a half-read upload produces a
* reply, not a row (plan §2).
*/
export const MAX_FILES = 6;
export type IntakeKind = "grocery" | "food" | "ask" | "held" | "unsupported";
export type IntakeStatus =
| "received"
| "enriched"
| "created"
| "candidates"
| "forwarded_to_pantry"
| "needs_header"
| "duplicate"
| "unsupported"
| "ask"
| "held"
| "error";
export type IntakeResult = {
kind: IntakeKind;
status: IntakeStatus;
batch_id: string;
transaction_id?: number;
capture_id?: number;
duplicate_of?: { batch_id: string; transaction_id: number | null; capture_id: number | null };
candidates?: Candidate[];
header?: OrderHeader | null;
items?: StoredLineItem[];
slack_blocks: Block[] | null;
text: string;
http_status: number;
};
export type IntakeDeps = { extractor: Extractor; pantry: PantryForward };
export type BatchRow = {
id: string;
source: IntakeSource;
shared_by: string | null;
file_sha256: string[];
header: (OrderHeader & { platform: string | null; subtotal?: number | null; currency?: string | null }) | null;
items: StoredLineItem[] | null;
kind: string | null;
status: IntakeStatus;
transaction_id: number | null;
capture_id: number | null;
};
// ─── batch persistence ──────────────────────────────────────────────────────
export async function loadBatch(id: string): Promise<BatchRow | null> {
return queryRow<BatchRow>(
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
FROM intake_batches WHERE id = $1::uuid`,
[id]
);
}
async function insertBatch(b: { id: string; source: IntakeSource; sharedBy: string | null; shas: string[] }) {
await queryRaw(
`INSERT INTO intake_batches (id, source, shared_by, file_sha256, status)
VALUES ($1::uuid, $2, $3, $4::text[], 'received')`,
[b.id, b.source, b.sharedBy, b.shas]
);
}
async function updateBatch(
id: string,
patch: {
header?: unknown;
items?: unknown;
kind?: string | null;
status: IntakeStatus;
transaction_id?: number | null;
capture_id?: number | null;
}
) {
await queryRaw(
`UPDATE intake_batches
SET header = COALESCE($2::jsonb, header),
items = COALESCE($3::jsonb, items),
kind = COALESCE($4, kind),
status = $5,
transaction_id = COALESCE($6, transaction_id),
capture_id = COALESCE($7, capture_id),
resolved_at = CASE WHEN $5 IN ('enriched','created','forwarded_to_pantry','duplicate') THEN now() ELSE resolved_at END
WHERE id = $1::uuid`,
[
id,
patch.header === undefined ? null : JSON.stringify(patch.header),
patch.items === undefined ? null : JSON.stringify(patch.items),
patch.kind ?? null,
patch.status,
patch.transaction_id ?? null,
patch.capture_id ?? null,
]
);
}
// ─── lookups ────────────────────────────────────────────────────────────────
export async function rememberedKind(merchant: string | null): Promise<"grocery" | "food" | null> {
const key = merchantKey(merchant);
if (!key) return null;
const row = await queryRow<{ kind: "grocery" | "food" }>(`SELECT kind FROM intake_merchant_kinds WHERE merchant_key = $1`, [key]);
return row?.kind ?? null;
}
export async function rememberKind(merchant: string | null, kind: "grocery" | "food"): Promise<void> {
const key = merchantKey(merchant);
if (!key) return;
await queryRaw(
`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ($1, $2)
ON CONFLICT (merchant_key) DO UPDATE SET kind = EXCLUDED.kind`,
[key, kind]
);
}
/** The owner a batch's rows belong to: the sharer's participant, else the default. */
export async function ownerForEmail(email: string | null): Promise<number> {
if (!email) return DEFAULT_OWNER_ID;
const row = await queryRow<{ id: number }>(`SELECT id FROM participants WHERE lower(email) = lower($1) LIMIT 1`, [email]);
return row?.id ?? DEFAULT_OWNER_ID;
}
/**
* A batch already resolved from the same files, or from the same
* (platform, merchant, date, total) header (plan §5, Codex #5). Only RESOLVED
* batches count: a `needs_header` share followed by a fuller one must go
* through, and a pending candidates card re-shared is harmless.
*/
export async function findDuplicateBatch(
exceptId: string,
shas: string[],
header: (OrderHeader & { platform: string | null }) | null
): Promise<BatchRow | null> {
const bySha = await queryRow<BatchRow>(
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
FROM intake_batches
WHERE id <> $1::uuid AND status IN ('enriched','created','forwarded_to_pantry')
AND file_sha256 && $2::text[]
ORDER BY created_at DESC LIMIT 1`,
[exceptId, shas]
);
if (bySha) return bySha;
if (!header || !header.merchant) return null;
const rows = await queryRaw<BatchRow>(
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
FROM intake_batches
WHERE id <> $1::uuid AND status IN ('enriched','created','forwarded_to_pantry')
AND header->>'order_date' = $2
AND (header->>'total')::numeric = $3::numeric
AND COALESCE(header->>'platform', '') = COALESCE($4, '')
ORDER BY created_at DESC`,
[exceptId, header.order_date, header.total, header.platform]
);
return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null;
}
type MatchRow = {
transaction_id: number;
description: string | null;
transaction_date: string;
amount: string;
merchant: string | null;
platform: string | null;
/** What `amount` is denominated in — a candidate row is not always dollars. */
currency: string;
/** The `expense_metadata` row the match resolved through — null when there is none. */
metadata_id: number | null;
/** Does that row already carry a receipt? */
has_line_items: boolean;
};
/**
* The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and
* date EXACT, never widened silently. The merchant compares normalised, with
* whole-word containment ("Coles" vs "Coles (Wyndham Vale)") — that is
* spelling, not widening. A unique hit is a match; two hits are an ambiguity
* and go to the candidates card like a miss.
*
* A NULL platform means "no platform", not "any platform". An in-store Coles
* receipt carries none, and `$3 IS NULL OR em.platform = $3` let it match an
* Uber Eats Coles order on the same day for the same money — and then overwrite
* that order's items with the till receipt's. The in-store side matches rows
* with no platform of their own; the delivery side still requires its own.
*/
export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> {
if (!header.merchant) return { match: null, ambiguous: false };
const rows = await queryRaw<MatchRow>(
`SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency,
COALESCE(jsonb_typeof(em.line_items) = 'array' AND jsonb_array_length(em.line_items) > 0, false) AS has_line_items
FROM transactions t
LEFT JOIN statements s ON s.id = t.statement_id
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date = $1::date
AND t.amount = $2::numeric
AND (($3::text IS NULL AND em.platform IS NULL) OR em.platform = $3)
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date, header.total, platform]
);
const hits = rows.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant));
if (hits.length === 1) return { match: hits[0], ambiguous: false };
return { match: null, ambiguous: hits.length > 1 };
}
/**
* Nearest candidates for the card: same merchant, ±3 days, any amount.
*
* Deliberately NOT filtered by platform. The card is a human choice, not a
* decision — the whole reason it is shown is that the exact key missed, so the
* platform is one of the things that may be wrong. A filter here narrows the
* list at exactly the moment it needs to widen, and the row the user was
* looking for is simply absent with nothing saying why.
*/
export async function findCandidates(header: OrderHeader): Promise<Candidate[]> {
if (!header.merchant) return [];
const rows = await queryRaw<MatchRow>(
`SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency, false AS has_line_items
FROM transactions t
LEFT JOIN statements s ON s.id = t.statement_id
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date]
);
return rows
.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant))
.sort((a, b) => {
const da = Math.abs(Date.parse(a.transaction_date) - Date.parse(header.order_date));
const db = Math.abs(Date.parse(b.transaction_date) - Date.parse(header.order_date));
if (da !== db) return da - db;
return Math.abs(Number(a.amount) - header.total) - Math.abs(Number(b.amount) - header.total);
})
.slice(0, 5)
.map((r) => ({ transaction_id: r.transaction_id, description: r.description, transaction_date: r.transaction_date, amount: r.amount, currency: r.currency }));
}
// ─── the two destinations ───────────────────────────────────────────────────
export type RouteContext = {
batchId: string;
source: IntakeSource;
sharedBy: string | null;
platform: string | null;
header: OrderHeader;
items: StoredLineItem[];
currency?: string | null;
};
const cardInput = (ctx: RouteContext): CardInput => ({
batchId: ctx.batchId,
merchant: ctx.header.merchant,
total: ctx.header.total,
orderDate: ctx.header.order_date,
platform: ctx.platform,
itemCount: ctx.items.length,
currency: ctx.currency,
});
const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({
kind,
status,
batch_id: ctx.batchId,
header: ctx.header,
items: ctx.items,
slack_blocks: card.blocks,
text: card.text,
http_status: 200,
...extra,
});
/**
* Are these items unverifiable — no line carrying any money at all?
*
* That is `reconcileItems`' `basis: "none"`: pasted text with no prices, or an
* extraction that read the names and none of the figures. The reconciliation
* check passes it, correctly, because there is nothing to check. What must not
* follow is replacing a receipt that IS itemised with one nothing verified.
*/
const unverifiable = (items: StoredLineItem[]) => items.every((i) => i.amount === null);
/**
* The hold, shared by EVERY path that enriches a matched transaction.
*
* It lived only in `routeFood` and `createFromBatch`'s re-match walked straight
* past it — the same upload destroyed the same receipt depending on which
* button reached it. Returns null when there is nothing to protect.
*/
async function holdRatherThanReplace(ctx: RouteContext, match: MatchRow): Promise<IntakeResult | null> {
if (!unverifiable(ctx.items) || !match.has_line_items) return null;
await updateBatch(ctx.batchId, { status: "held", items: ctx.items });
const card = heldCard(
"this upload carries no prices, so nothing checked it against the total — and that order already has its items",
"I left the existing items alone"
);
return finish(ctx, "held", "held", card, { transaction_id: match.transaction_id });
}
/** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */
export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
const { match } = await matchTransaction(ctx.header, ctx.platform);
if (match) {
const held = await holdRatherThanReplace(ctx, match);
if (held) return held;
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId, metadataId: match.metadata_id, currency: ctx.currency });
await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items });
return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
}
const candidates = await findCandidates(ctx.header);
await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items });
return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates });
}
/** Grocery: the original files go to pantry's board; finance rides along on its confirm (C3/C4). */
export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantry: PantryForward): Promise<IntakeResult> {
const { match } = await matchTransaction(ctx.header, ctx.platform);
try {
const r = await pantry({
files,
batchId: ctx.batchId,
platform: ctx.platform,
sharedBy: ctx.sharedBy,
financeTargetTransactionId: match?.transaction_id ?? null,
financeAction: match ? "enrich" : "create",
});
await updateBatch(ctx.batchId, { kind: "grocery", status: "forwarded_to_pantry", transaction_id: match?.transaction_id ?? null, capture_id: r.captureId, items: ctx.items });
const card = forwardedCard({ ...cardInput(ctx), itemCount: r.itemCount ?? ctx.items.length, captureId: r.captureId, enriching: !!match });
return finish(ctx, "grocery", "forwarded_to_pantry", card, { capture_id: r.captureId, transaction_id: match?.transaction_id });
} catch (e) {
if (e instanceof PantryDuplicateError) {
await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items });
return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, currency: ctx.currency, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), {
capture_id: e.captureId ?? undefined,
});
}
const detail = e instanceof PantryUnavailableError ? e.message : e instanceof Error ? e.message : String(e);
await updateBatch(ctx.batchId, { kind: "grocery", status: "error", items: ctx.items });
return { ...finish(ctx, "grocery", "error", errorCard(detail)), http_status: 502 };
}
}
/** Create the transaction ING-10-style: manual, `statement_id NULL`, owner set — only ever on an explicit tap. */
export async function createFromBatch(batch: BatchRow, kind: "grocery" | "food"): Promise<IntakeResult> {
if (!batch.header) throw new Error("batch has no header");
const header = batch.header;
const items = batch.items ?? [];
const ctx: RouteContext = {
batchId: batch.id,
source: batch.source,
sharedBy: batch.shared_by,
platform: header.platform,
header,
items,
currency: header.currency,
};
// The card was built minutes ago and the tap arrives now. In between, the
// email/order bridge books orders on its own schedule — so re-run the match
// rather than trusting the "no match" the card was drawn from. Creating a
// second row for an order the ledger already holds is the one outcome this
// whole slice exists to avoid.
const { match } = await matchTransaction(header, header.platform);
if (match) {
const held = await holdRatherThanReplace(ctx, match);
if (held) return held;
await replaceLineItems(match.transaction_id, items, { source: "intake", platform: header.platform, batchId: batch.id, metadataId: match.metadata_id, currency: header.currency });
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: match.transaction_id, items });
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
}
// A sibling batch of the same order — the PDF after the screenshot — may have
// been resolved since. `findDuplicateBatch` deliberately ignores unresolved
// batches, so two pending cards for one order both reach here; without this
// the second tap books the order twice.
const twin = await findDuplicateBatch(batch.id, batch.file_sha256, header);
if (twin) {
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: twin.transaction_id, capture_id: twin.capture_id });
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: twin.transaction_id, captureId: twin.capture_id, batchId: twin.id });
return finish(ctx, kind, "duplicate", card, {
transaction_id: twin.transaction_id ?? undefined,
capture_id: twin.capture_id ?? undefined,
duplicate_of: { batch_id: twin.id, transaction_id: twin.transaction_id, capture_id: twin.capture_id },
});
}
const ownerId = await ownerForEmail(batch.shared_by);
const merchant = header.merchant ?? "Unknown merchant";
// Not `orderDescription()`: it always prefixes "Order - ", which is right for
// a delivery receipt and wrong for the in-store till receipt this path also
// handles. Not `resolveCategory()` either — it re-derives grocery-vs-food
// from a merchant pattern, and `kind` is the answer the router already
// reached, remembered merchant and Slack tap included.
const description = header.platform ? `Order - ${merchant}` : merchant;
const category = kind === "grocery" ? "groceries" : "dining";
// `amount` is the figure printed on the receipt, in its own currency. No FX
// rate exists at intake, so a foreign order leaves `amount_aud` NULL rather
// than asserting a conversion — the same shape order-ingestion.ts writes.
const currency = header.currency ?? "AUD";
const isAud = currency === "AUD";
// Check and insert under one lock, on one connection.
//
// `source_ref = batch.id` makes a second press of THIS card a no-op, and
// ON CONFLICT makes that safe against a double tap. It cannot help two
// DIFFERENT batches for one purchase — their keys differ — and both taps run
// their match and duplicate checks before either inserts, so both can find
// nothing. The lock is keyed on the purchase, not the batch, which is the
// only thing the two share. It has to be transaction-scoped and therefore
// inside withTransaction: taken by a statement of its own it would be
// released before the next one ran (see db.ts).
const purchaseKey = [header.platform ?? "", merchantKey(merchant), header.order_date, header.total.toFixed(2)].join("|");
const outcome = await withTransaction(async (q) => {
await q.queryRaw(`SELECT pg_advisory_xact_lock(hashtextextended($1, 0))`, [purchaseKey]);
// Mine (a double tap) or another batch's (the race). The ORDER BY is what
// tells them apart when both exist.
const existing = await q.queryRow<{ id: number; source_ref: string }>(
`SELECT id, source_ref FROM transactions
WHERE source = 'intake' AND superseded_by_id IS NULL
AND (source_ref = $1
OR (transaction_date = $2::date AND amount = $3::numeric
AND lower(merchant_normalized) = lower($4)))
ORDER BY (source_ref = $1) DESC NULLS LAST, id
LIMIT 1`,
[batch.id, header.order_date, header.total, merchant]
);
if (existing) return { id: existing.id, mine: existing.source_ref === batch.id };
const row = await q.queryRow<{ id: number }>(
`INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref,
foreign_currency_amount, foreign_currency_code
) VALUES ($1::date, $2, $3::numeric, $4::numeric, $5, NULL, $6, $6, 'debit', $7, 'intake', $8, $9::numeric, $10)
ON CONFLICT (source, source_ref) WHERE source IS NOT NULL AND source_ref IS NOT NULL
DO NOTHING
RETURNING id`,
[header.order_date, description, header.total, isAud ? header.total : null, category, merchant, ownerId, batch.id, isAud ? null : header.total, isAud ? null : currency]
);
return { id: row!.id, mine: true };
});
// Another batch booked this purchase in the moment between the two taps. Say
// the same thing the sequential case says one second later, and write nothing.
if (!outcome.mine) {
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: outcome.id });
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: outcome.id, captureId: null, batchId: null });
return finish(ctx, kind, "duplicate", card, { transaction_id: outcome.id });
}
const txnId = outcome.id;
await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id, currency });
await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId });
return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId });
}
/** The card for a batch as it stands — what a second press of a resolved card gets back. */
export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } {
const h = batch.header;
const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0, currency: h?.currency };
if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id });
return duplicateCard({ merchant: base.merchant, total: base.total, currency: base.currency, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id });
}
// ─── the router ─────────────────────────────────────────────────────────────
export type IntakeInput = { batchId?: string | null; source: IntakeSource; sharedBy: string | null; files: StoredFile[] };
export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<IntakeResult> {
const batchId = input.batchId || randomUUID();
const shas = input.files.map((f) => f.sha256);
const platform = platformForSource(input.source);
if (await loadBatch(batchId)) {
return { kind: "unsupported", status: "duplicate", batch_id: batchId, slack_blocks: null, text: `batch ${batchId} was already received`, http_status: 409 };
}
await insertBatch({ id: batchId, source: input.source, sharedBy: input.sharedBy, shas });
const fail = (kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, http = 200, extra: Partial<IntakeResult> = {}): IntakeResult => ({
kind,
status,
batch_id: batchId,
slack_blocks: card.blocks,
text: card.text,
http_status: http,
...extra,
});
// 1. classify each file and build the parts, in the order shared.
const parts: InputPart[] = [];
const usable: StoredFile[] = [];
for (const f of input.files) {
const kind = classifyInput(f);
if (kind === "pdf") {
let text = "";
try {
text = await extractPdfText(f.bytes);
} catch {
/* unreadable PDF — reported below */
}
if (!text) {
await updateBatch(batchId, { kind: "unsupported", status: "unsupported" });
return fail("unsupported", "unsupported", unsupportedCard(`${f.name} has no readable text — try a screenshot of it`));
}
parts.push({ kind: "text", text, name: f.name });
} else if (kind === "image") {
parts.push({ kind: "image", mime: f.type || "image/jpeg", bytes: f.bytes, name: f.name });
} else if (kind === "text") {
parts.push({ kind: "text", text: f.bytes.toString("utf-8"), name: f.name });
} else {
continue;
}
usable.push(f);
}
if (!parts.length) {
await updateBatch(batchId, { kind: "unsupported", status: "unsupported" });
return fail("unsupported", "unsupported", unsupportedCard());
}
// A re-share of the very same files, before spending a model call on it.
const dupBySha = await findDuplicateBatch(batchId, shas, null);
if (dupBySha) {
await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id });
return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, currency: dupBySha.header?.currency, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, {
kind: (dupBySha.kind as IntakeKind) ?? "food",
duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id },
});
}
// Keep the originals: a later "Groceries" tap forwards them to pantry.
try {
await storeFiles(usable);
} catch (e) {
console.warn("intake: could not store files", e instanceof Error ? e.message : e);
}
// 2. ONE extraction call for the whole batch.
let extracted: ExtractedOrder;
try {
extracted = await deps.extractor(parts);
} catch (e) {
const detail = e instanceof ExtractionError ? e.message : e instanceof Error ? e.message : String(e);
await updateBatch(batchId, { status: "error" });
return fail("held", "error", errorCard(detail), 502);
}
const items = toLineItems(extracted.items);
// 3. header.
const headerCheck = validateHeader(extracted);
if (!headerCheck.ok) {
await updateBatch(batchId, { header: { merchant: extracted.merchant, order_date: extracted.order_date, total: extracted.total, platform }, items, status: "needs_header" });
return fail("held", "needs_header", needsHeaderCard(), 200, { items });
}
const header = headerCheck.header;
const storedHeader = { ...header, platform, subtotal: extracted.subtotal ?? null, currency: extracted.currency ?? "AUD" };
// 4. merchant → grocery | food | ask.
const kind: MerchantKind = resolveMerchantKind(input.source, header.merchant, await rememberedKind(header.merchant));
// 5. the same order, already handled under other files.
const dup = await findDuplicateBatch(batchId, shas, storedHeader);
if (dup) {
await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id });
return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, currency: storedHeader.currency, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, {
header,
items,
duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id },
});
}
// 6. do the lines add up? (missing-screenshot check)
const recon = reconcileItems(extracted);
if (!recon.ok) {
await updateBatch(batchId, { header: storedHeader, items, kind, status: "held" });
return fail("held", "held", heldCard(recon.reason ?? "mismatch"), 200, { header, items });
}
await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" });
const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, currency: storedHeader.currency };
// 7. act.
if (kind === "ask") {
await updateBatch(batchId, { kind: "ask", status: "ask" });
return finish(ctx, "ask", "ask", askCard(cardInput(ctx)));
}
if (kind === "grocery") return routeGrocery(ctx, usable, deps.pantry);
return routeFood(ctx);
}
src/lib/intake-rules.ts
+252
View File
@@ -0,0 +1,252 @@
/**
* Intake router — the pure decisions.
*
* Everything here is a function of its arguments: what kind of input a file is,
* which platform a source tap implies, whether a merchant is a grocer, whether a
* header is complete, whether the items add up. Kept free of the database and the
* model so each rule can be unit-tested against the exact strings real receipts
* print. The orchestration that calls these lives in `intake-router.ts`.
*
* Plan: smarthome `plans/intake-router-slice-plan.md` §2, §3, §5, §11 C1.
*/
export const INTAKE_SOURCES = ["uber_eats", "doordash", "coles", "woolworths", "other"] as const;
export type IntakeSource = (typeof INTAKE_SOURCES)[number];
export function isIntakeSource(value: unknown): value is IntakeSource {
return typeof value === "string" && (INTAKE_SOURCES as readonly string[]).includes(value);
}
export type InputKind = "image" | "pdf" | "text" | "unsupported";
/** Input type picks the extractor; it is independent of source and merchant. */
export function classifyInput(file: { type?: string | null; name?: string | null }): InputKind {
const type = (file.type ?? "").toLowerCase();
const name = file.name ?? "";
if (type === "application/pdf" || /\.pdf$/i.test(name)) return "pdf";
if (type.startsWith("image/") || /\.(png|jpe?g|webp|heic|gif)$/i.test(name)) return "image";
if (type.startsWith("text/") || /\.(txt|md)$/i.test(name)) return "text";
return "unsupported";
}
/**
* The platform the source tap implies, in `expense_metadata.platform` vocabulary.
* Null for an in-store receipt or `other`: those rows carry no platform, so the
* match falls back to (merchant, date, total) with no platform constraint.
*/
export function platformForSource(source: IntakeSource): string | null {
if (source === "uber_eats") return "ubereats";
if (source === "doordash") return "doordash";
return null;
}
export const PLATFORM_LABEL: Record<string, string> = {
ubereats: "Uber Eats",
doordash: "DoorDash",
uber: "Uber",
};
export function platformLabel(platform: string | null): string {
return platform ? PLATFORM_LABEL[platform] ?? platform : "";
}
/**
* A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints
* `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; a
* word-boundary alternation on the normalised name covers both without a
* merchant master-data subsystem.
*
* The boundaries are load-bearing, not tidiness: a bare substring made
* `Bigals Pizza` a grocer, because "b-IGA-ls" contains `iga`. Short names in a
* substring list collide with ordinary words, and the collision reads as a
* correct classification.
*/
export const GROCERY_PATTERN = /\b(coles|woolworths|woolies|aldi|iga|foodworks|costco)\b/i;
/**
* The normalised merchant used for matching and for the remembered-kind table.
* Lower-case, punctuation dropped, whitespace collapsed — so `Coles (Wyndham Vale)`
* and `coles wyndham vale` are one key, while `Coles` stays distinct from both.
*/
export function merchantKey(name: string | null | undefined): string {
return (name ?? "")
.toLowerCase()
.replace(/[^\p{L}\p{N}]+/gu, " ")
.trim()
.replace(/\s+/g, " ");
}
export type MerchantKind = "grocery" | "food" | "ask";
const UNKNOWN_MERCHANT = /^(unknown|n ?a|none|null|undefined|merchant)$/;
/**
* Grocery-vs-food is decided by the merchant, not the platform — "Uber Eats"
* covers both a meal and a grocery run. The source tap is ground truth where it
* says so (Coles/Woolworths taps are grocers by definition); the pattern list
* decides the obvious delivery cases; the remembered answer covers the
* `Romeo's Food Hall`s; and `other` with nothing else to go on asks rather than
* silently falling through to "meal" (plan §2, D3 holding state).
*/
export function resolveMerchantKind(
source: IntakeSource,
merchant: string | null | undefined,
remembered: "grocery" | "food" | null = null
): MerchantKind {
if (source === "coles" || source === "woolworths") return "grocery";
const key = merchantKey(merchant);
if (!key || UNKNOWN_MERCHANT.test(key)) return "ask";
if (GROCERY_PATTERN.test(key)) return "grocery";
if (remembered) return remembered;
if (source === "other") return "ask";
return "food";
}
/** What the extractor returns — names verbatim, options kept (plan §3). */
export type ExtractedItem = {
quantity: number | null;
description: string;
unit_price?: number | null;
line_total?: number | null;
options?: string[] | null;
};
export type ExtractedOrder = {
merchant: string | null;
order_date: string | null;
total: number | null;
subtotal?: number | null;
currency?: string | null;
items: ExtractedItem[];
adjustments?: { label: string; amount: number }[] | null;
};
export type OrderHeader = { merchant: string | null; order_date: string; total: number };
/**
* "Missing the header/total shot → nothing is written." A header is the date and
* the total; the merchant may be absent and still route (it becomes an ask).
*/
export function validateHeader(
x: ExtractedOrder
): { ok: true; header: OrderHeader } | { ok: false; missing: string[] } {
const missing: string[] = [];
if (typeof x.order_date !== "string" || !/^\d{4}-\d{2}-\d{2}$/.test(x.order_date)) missing.push("date");
if (typeof x.total !== "number" || !Number.isFinite(x.total) || x.total <= 0) missing.push("total");
if (missing.length) return { ok: false, missing };
return {
ok: true,
header: {
merchant: typeof x.merchant === "string" && x.merchant.trim() ? x.merchant.trim() : null,
order_date: x.order_date as string,
total: round2(x.total as number),
},
};
}
export const round2 = (n: number) => Number(n.toFixed(2));
/** A line's money, as printed or derived; null when the receipt gave neither. */
export function itemAmount(item: ExtractedItem): number | null {
if (typeof item.line_total === "number" && Number.isFinite(item.line_total)) return round2(item.line_total);
if (typeof item.unit_price === "number" && Number.isFinite(item.unit_price)) {
return round2(item.unit_price * (item.quantity ?? 1));
}
return null;
}
export type Reconciliation = {
ok: boolean;
basis: "subtotal" | "total" | "none";
items_sum: number;
reference: number | null;
diff: number;
reason?: string;
};
/**
* The missing-screenshot check (plan §3, Codex #7): the lines must reconcile
* within $1 or 2%, else the batch is held.
*
* Reconciled to the printed ITEM SUBTOTAL when there is one, not the total — a
* delivery order's total is items plus fees minus promotions ($118.94 of items
* became $85.74 on the real Coles export), so against the total every delivery
* order would be held. The total stays the match key; the subtotal is what the
* items can be checked against. Without a printed subtotal the total is the only
* reference there is. A receipt whose lines carry no prices at all (pasted text)
* cannot be checked and is not held for it.
*/
export function reconcileItems(x: ExtractedOrder): Reconciliation {
const amounts = (x.items ?? []).map(itemAmount);
if (!x.items?.length) {
return { ok: false, basis: "none", items_sum: 0, reference: null, diff: 0, reason: "no items" };
}
if (amounts.every((a) => a === null)) {
return { ok: true, basis: "none", items_sum: 0, reference: null, diff: 0 };
}
const itemsSum = round2(amounts.reduce<number>((s, a) => s + (a ?? 0), 0));
const subtotal = typeof x.subtotal === "number" && Number.isFinite(x.subtotal) ? x.subtotal : null;
const basis: Reconciliation["basis"] = subtotal !== null ? "subtotal" : "total";
const reference = subtotal ?? (typeof x.total === "number" ? x.total : null);
if (reference === null) return { ok: false, basis, items_sum: itemsSum, reference: null, diff: 0, reason: "no total" };
const diff = round2(itemsSum - reference);
const tolerance = Math.max(1, 0.02 * reference);
const ok = Math.abs(diff) <= tolerance;
return ok
? { ok, basis, items_sum: itemsSum, reference, diff }
: { ok, basis, items_sum: itemsSum, reference, diff, reason: `items sum to ${itemsSum.toFixed(2)} but the ${basis} is ${reference.toFixed(2)}` };
}
/**
* The stored shape — the same keys the order lane writes and
* `components/order-details.tsx` renders (`qty`, `description`, `amount`,
* `options`). A burger called "Michael" is stored as "Michael".
*/
export type StoredLineItem = {
qty: number;
description: string;
amount: number | null;
unit_price?: number | null;
options?: string[];
};
export function toLineItems(items: ExtractedItem[]): StoredLineItem[] {
return (items ?? [])
.filter((i) => typeof i.description === "string" && i.description.trim())
.map((i) => {
const out: StoredLineItem = {
qty: typeof i.quantity === "number" && i.quantity > 0 ? i.quantity : 1,
description: i.description.trim(),
amount: itemAmount(i),
};
if (typeof i.unit_price === "number" && Number.isFinite(i.unit_price)) out.unit_price = round2(i.unit_price);
const options = (i.options ?? []).map((o) => String(o).trim()).filter(Boolean);
if (options.length) out.options = options;
return out;
});
}
/** Is `needle` a contiguous run of whole words inside `hay`? */
function containsWordRun(hay: string[], needle: string[]): boolean {
if (!needle.length || needle.length > hay.length) return false;
for (let i = 0; i + needle.length <= hay.length; i++) {
if (needle.every((w, j) => hay[i + j] === w)) return true;
}
return false;
}
/**
* Does the stored merchant name denote the extracted one? One name's words must
* appear as a contiguous run inside the other's — so `Coles` matches
* `Coles (Wyndham Vale)` and `Order - Coles (Wyndham Vale)`, which is spelling,
* not widening.
*
* Whole words, never a raw substring: `merchantKey("Bigals Pizza")` contains
* `iga`, so a pizza shop matched every IGA on the same date and total.
*/
export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean {
const a = merchantKey(stored).split(" ").filter(Boolean);
const b = merchantKey(extracted).split(" ").filter(Boolean);
if (!a.length || !b.length) return false;
return containsWordRun(a, b) || containsWordRun(b, a);
}
src/lib/line-items.ts
+200
View File
@@ -0,0 +1,200 @@
import { queryRow } from "@/lib/db";
/**
* Replace the itemised receipt behind a transaction (plan §11 C2).
*
* An idempotent UPDATE, never an INSERT of a second row: `expense_metadata.
* transaction_id` is UNIQUE and the order the email pipeline booked already has
* its row — with `line_items = []`, because the items sat behind an auth-gated
* link. Idempotence is also what lets pantry re-run it after every confirm,
* writing the board's current state (plan §5, Codex #3), and what lets undo
* restore `previous_line_items` through the same call.
*
* The row is found on `transaction_id = :id OR matched_transaction_id = :id` —
* a card-settled order has only the latter (CLAUDE.md trap). When the
* transaction has no row at all (a manual transaction, a statement line), one
* is created so the items have somewhere to live; a transaction is NEVER
* created here.
*
* `opts.metadataId` pins WHICH row when the caller already resolved one. A
* statement transaction can own its own `expense_metadata` row AND be the
* `matched_transaction_id` of a delivery row: the match query and this write
* then re-pick independently and can land on different rows, so the enrichment
* reports success against a row nobody reads.
*/
export type LineItemsSource = "intake" | "pantry";
export class TransactionNotFoundError extends Error {}
export class LineItemsValidationError extends Error {}
export type LineItemRecord = Record<string, unknown> & { description: string };
/**
* Permissive on purpose: the order lane writes `{qty, description, amount,
* options}`, the receipt lane `{description, qty, amount, unit, category}`, and
* the detail panel reads the overlap. What every producer must supply is a
* description — an item with no name renders as nothing.
*/
export function validateLineItems(input: unknown): LineItemRecord[] {
if (!Array.isArray(input)) throw new LineItemsValidationError("line_items must be an array");
return input.map((raw, i) => {
if (!raw || typeof raw !== "object" || Array.isArray(raw)) {
throw new LineItemsValidationError(`line_items[${i}] must be an object`);
}
const item = { ...(raw as Record<string, unknown>) };
if (typeof item.description !== "string" || !item.description.trim()) {
throw new LineItemsValidationError(`line_items[${i}].description is required`);
}
item.description = item.description.trim();
// `quantity` is the extractor's word; `qty` is what the renderer reads.
if (item.qty === undefined && typeof item.quantity === "number") item.qty = item.quantity;
return item as LineItemRecord;
});
}
export type ReplaceResult = {
transaction_id: number;
metadata_id: number;
created_metadata: boolean;
previous_line_items: unknown[];
line_items: LineItemRecord[];
};
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* The undo baseline, recorded once per batch.
*
* Returning the row that was just read makes a RETRY destroy the thing undo
* exists to restore: pantry re-calls this on every confirm and after a timeout,
* and the second call would hand back its own write as the "previous" state.
* The first snapshot a batch ever takes is the only one worth keeping, so it is
* written under COALESCE and read back rather than trusted.
*/
async function undoBaseline(batchId: string | null | undefined, snapshot: unknown[]): Promise<unknown[]> {
// Pantry passes back whatever batch id it was given; only an intake batch has
// a row to record against, and a non-uuid would fail the cast.
if (!batchId || !UUID.test(batchId)) return snapshot;
const row = await queryRow<{ previous_line_items: unknown }>(
`UPDATE intake_batches
SET previous_line_items = COALESCE(previous_line_items, $2::jsonb)
WHERE id = $1::uuid
RETURNING previous_line_items`,
[batchId, JSON.stringify(snapshot)]
);
return Array.isArray(row?.previous_line_items) ? row.previous_line_items : snapshot;
}
export async function replaceLineItems(
transactionId: number,
lineItems: LineItemRecord[],
opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null; metadataId?: number | null; currency?: string | null }
): Promise<ReplaceResult> {
const txn = await queryRow<{
id: number;
merchant_normalized: string | null;
merchant_name: string | null;
amount: string;
transaction_date: string;
}>(
`SELECT id, merchant_normalized, merchant_name, amount::text, transaction_date::text
FROM transactions WHERE id = $1`,
[transactionId]
);
if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`);
// The caller's row wins where it still exists; otherwise prefer the row that
// IS this transaction over one that merely points at it — a statement line
// can be the `matched_transaction_id` of a receipt row and carry its own row
// too, and the items belong on the receipt.
const pinned = opts.metadataId
? await queryRow<{ id: number; line_items: unknown }>(
`SELECT id, line_items FROM expense_metadata
WHERE id = $1 AND (transaction_id = $2 OR matched_transaction_id = $2)`,
[opts.metadataId, transactionId]
)
: null;
const json = JSON.stringify(lineItems);
/**
* Snapshot the baseline, then write. In that order on BOTH paths: the create
* path used to snapshot after its INSERT, so a crash in between left a retry
* recording the intake's own write as "previous" — undo restoring the thing
* it was meant to reverse, which is the failure `previous_line_items` exists
* to prevent.
*
* Returns null when the row is gone between the read and the write. That
* UPDATE reported success and the batch was marked enriched against a row
* nothing had written to.
*/
const write = async (row: { id: number; line_items: unknown }): Promise<ReplaceResult | null> => {
const previous = await undoBaseline(opts.batchId, Array.isArray(row.line_items) ? row.line_items : []);
const updated = await queryRow<{ id: number }>(
`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1 RETURNING id`,
[row.id, json]
);
if (!updated) return null;
return { transaction_id: transactionId, metadata_id: row.id, created_metadata: false, previous_line_items: previous, line_items: lineItems };
};
if (pinned) {
const done = await write(pinned);
if (done) return done;
}
const existing = await queryRow<{ id: number; line_items: unknown }>(
`SELECT id, line_items FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- NULLS LAST is load-bearing, and its absence inverted this clause.
-- Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at
-- the transaction has a NULL transaction_id — so the comparison meant to
-- prefer the owned row preferred the other one, silently and in exactly
-- the case it was written for. matchTransaction spells it out; this did
-- not, and the two therefore disagreed.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[transactionId]
);
if (existing) {
const done = await write(existing);
if (done) return done;
}
// Nothing to update. Snapshot the empty baseline BEFORE creating the row.
const previous = await undoBaseline(opts.batchId, []);
const reference = opts.batchId ? `intake:${opts.batchId}` : `intake:txn:${transactionId}`;
// ON CONFLICT because two calls for one transaction can both find no row and
// both INSERT; expense_metadata.transaction_id is UNIQUE, so one of them lost
// with a constraint error that reached the user as an error card. `xmax = 0`
// is how a plain insert is told from a conflict-update, so the reported
// `created_metadata` stays true.
const created = await queryRow<{ id: number; inserted: boolean }>(
`INSERT INTO expense_metadata (
transaction_id, source, platform, order_reference, line_items,
merchant_normalized, amount, transaction_date, currency, flags, extraction_model
) VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7::numeric, $8::date, $9, '[]'::jsonb, 'cloud-budget')
ON CONFLICT (transaction_id) DO UPDATE SET line_items = EXCLUDED.line_items
RETURNING id, (xmax = 0) AS inserted`,
[
transactionId,
opts.source,
opts.platform ?? null,
reference,
json,
txn.merchant_normalized ?? txn.merchant_name ?? null,
txn.amount,
txn.transaction_date,
// Not hard-coded 'AUD': the transaction may carry a foreign figure, and
// OrderDetails prefers this column, so a USD order printed "$".
opts.currency || "AUD",
]
);
return {
transaction_id: transactionId,
metadata_id: created!.id,
created_metadata: created!.inserted,
previous_line_items: previous,
line_items: lineItems,
};
}
src/lib/order-ingestion.ts
-7
View File
@@ -12,13 +12,6 @@ export const CUTOVER_DATE = "2026-01-09";
*/
export const DEFAULT_OWNER_ID = 1;
/** Human labels for the platform a receipt came from. */
export const PLATFORM_LABEL: Record<ParsedOrder["platform"], string> = {
doordash: "DoorDash",
ubereats: "Uber Eats",
uber: "Uber",
};
/**
* Transaction description.
*
src/lib/order-reviews.ts
+10
View File
@@ -119,6 +119,16 @@ export async function merchantForTransaction(
const row = await queryRow<{ merchant_normalized: string | null }>(
`SELECT merchant_normalized FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[transactionId]
);
src/lib/queries.ts
+10
View File
@@ -367,6 +367,16 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
SELECT em.route, em.platform
FROM expense_metadata em
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (em.transaction_id = t.id) DESC NULLS LAST, em.id
LIMIT 1
) order_ctx ON true
LEFT JOIN LATERAL (