siddharthd/finance-app
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: siddharthd/finance-app:9f1be7433017ced6da58c13d8b9499a1335541fc
Branches Tags
siddharthd/finance-app:main siddharthd/finance-app:feat/intake-router siddharthd/finance-app:feat/split-scopes siddharthd/finance-app:feat/order-ingestion siddharthd/finance-app:feature/phase6-budget siddharthd/finance-app:feature/phase5-rules
...
compare: siddharthd/finance-app:feat/intake-router
Branches Tags
siddharthd/finance-app:main siddharthd/finance-app:feat/intake-router siddharthd/finance-app:feat/split-scopes siddharthd/finance-app:feat/order-ingestion siddharthd/finance-app:feature/phase6-budget siddharthd/finance-app:feature/phase5-rules
21 Commits
9f1be74330 ... feat/intake-router
Author SHA1 Message Date
siddharthd d88ca2a24c intake: a duplicate card that refuses a purchase must say what to do instead
ci / lint-test (push) Successful in 1m5s
Details 
The header (merchant, date, total) is a purchase's identity here, so a genuine
second identical purchase — two of the same coffee, one shop, one day — reads
as a re-share and is refused. That default is right; a re-share is far commoner
than a repeat. But the card said only "Nothing was changed", which is a dead
end: the money is real and the ledger silently never gets it. Say to add it in
the app.
 2026-08-21 18:27:53 +10:00
siddharthd 91586ce4a4 test(intake): the second review pass, and CLAUDE.md for the rules it set
ci / lint-test (push) Successful in 45s
Details 
Six of the nine fixes have a test that fails against the previous tree: the
metadata row's currency, the Create-path hold, two concurrent batches booking
once, mrkdwn escaping including plain_text, the escaped currency, and
candidate-row currency.

Three do not, and the reason is the same each time — the window needs a crash
or a lost race that a test cannot force. Asserted instead: a stale metadataId
falls back to the live row rather than reporting a phantom write; two writes
racing to create a metadata row both resolve to one; the create path records a
baseline at all. All three passed before as well.
 2026-08-21 18:23:52 +10:00
siddharthd 4b09d4136f intake: lock the purchase, not the batch, and hold on both write paths 
Two different unresolved batches for one purchase could each create a
transaction. ON CONFLICT (source, source_ref) cannot stop it — their source_refs
differ — and both taps clear their match and duplicate checks before either
inserts. The create now takes a transaction-scoped advisory lock keyed on
(platform, merchant, date, total), re-checks inside it, and inserts; the loser
reports `duplicate` against the winner's transaction, the same answer the
sequential path gives.

The lock has to be inside a real transaction, which this app did not have.
queryRaw is $queryRawUnsafe on a POOLED client, so every call is its own
implicit transaction and pg_advisory_xact_lock taken in a statement of its own
is released before the next statement runs — the lock would have looked right
and held nothing. withTransaction() in db.ts is the scope; the integration
helper mirrors it with BEGIN/COMMIT on one pooled connection.

Also: the "hold rather than replace an itemised receipt with an unpriced
upload" guard lived only in routeFood, and createFromBatch's re-match walked
straight past it — the same upload destroyed the same receipt depending on
which button reached it. One function, both callers.
 2026-08-21 18:23:51 +10:00
siddharthd 4f0f0dd285 intake: escape the currency and the button label too 
money() interpolated the extracted currency raw, and parseExtraction accepts
any string for it — so "<@U123>" would have been a live mention on every card
the batch produced. And plain_text disables *formatting*, not entity decoding:
&, < and > are control characters everywhere in a Slack payload, so the Create
button's label is escaped like any other value we did not write. My earlier
note claiming the opposite was wrong.

Candidate rows also carry their transaction's currency (NATIVE_CURRENCY) rather
than always printing "$" under a header reading "USD 22.40".
 2026-08-21 18:23:51 +10:00
siddharthd b788b4e8fa line-items: snapshot before writing, and stop reporting writes that did not happen 
Three faults on the same function, all of them "the write is assumed to have
worked".

- The create path snapshotted the undo baseline AFTER its INSERT. A crash in
  between left the retry recording the intake's own write as "previous", so
  undo restored the thing it was meant to reverse — the exact failure
  previous_line_items exists to prevent. Both paths now snapshot first.
- An UPDATE against a row deleted between the select and the write affects
  nothing and still returned a metadata_id, so the batch was marked enriched
  against a row nobody had written to. The write is now checked and falls
  through to the ordered lookup, then to a create.
- Two calls for one transaction could both find no row and both INSERT;
  expense_metadata.transaction_id is UNIQUE, so one lost with a constraint
  error that reached the user as an error card. ON CONFLICT (transaction_id)
  DO UPDATE, with (xmax = 0) so created_metadata stays truthful.

And the new row's currency comes from the batch instead of a hard-coded 'AUD'.
OrderDetails prefers expense_metadata.currency, so a USD order — booked
correctly on the transaction with foreign_currency_code — printed "$".
 2026-08-21 18:23:32 +10:00
siddharthd b5569b8ccc schema: previous_line_items on the model; say what the finish_reason rule is 
Migration 0034 has the column and the Prisma model did not, so a regenerated
client and the database disagreed.

The extraction comment claimed only a finished reply is accepted while the code
accepts an absent finish_reason. The code is right — failing every call on a
provider that omits an optional field would be a certain outage in exchange for
a case that also requires syntactically complete JSON, and the harm a truncated
reply does is closed on the write side. The prose now states the rule as it is:
a reason that is PRESENT must be `stop`.
 2026-08-21 18:23:32 +10:00
siddharthd d4ff68363a expense_metadata: every reader resolves the row a write lands on 
getTransactions' provenance LATERAL, merchantForTransaction and the Slack
details modal all select on transaction_id = $1 OR matched_transaction_id = $1
with a bare LIMIT 1, so they could read a different row than replaceLineItems
writes. Same ORDER BY as the match, the write and the order-detail GET.

Defensive, not a live bug: measured 2026-08-21, expense_metadata holds 667
rows, 147 of them pointing-only, and ZERO transactions resolve to more than one
row by either link. This changes nothing today. It is here so the readers still
agree on the day multiplicity appears, which is the only condition under which
any of them was ever wrong.
 2026-08-21 18:23:31 +10:00
siddharthd 631e75e58f test(intake): cover the review findings, and CLAUDE.md for the rules they set
ci / lint-test (push) Successful in 59s
Details 
Unit: word-boundary merchant matching (Bigals Pizza is not an IGA), mrkdwn
escaping vs the plain_text button label, a foreign total on a card, and the
finish_reason gate.

Integration: the NULL-platform wildcard on both the food and grocery paths,
candidates crossing platforms, EXCLUDE_RECONCILED_SOURCE, the pinned metadata
row, the panel reading the row the write landed on, the once-per-batch undo
baseline, the priceless-upload hold, Create re-matching, two pending cards
booking once, a pantry-resolved sibling, and a USD receipt.

Eleven of the thirteen fail against the pre-fix tree; the exceptions are the
no-batch-id baseline guard (a no-change assertion) and the concurrent-tap one,
which is a race and passed either way.
 2026-08-21 17:53:06 +10:00
siddharthd c25db5211b intake: a NULL platform is "no platform", and Create re-checks before it inserts 
Six fixes in the router, all of them cases where a stale or over-wide answer
was written rather than re-asked.

- ($3 IS NULL OR em.platform = $3) made an in-store receipt a wildcard: a till
  receipt from the Coles tap matched the Uber Eats Coles order for the same
  day and money, and overwrote its items. In-store now matches only rows with
  no platform of their own.
- findCandidates loses its platform filter entirely. That card is a human
  choice, shown precisely because the exact key missed — the platform is one
  of the things that may be wrong, so the list must widen there, not narrow.
- createFromBatch re-runs the match. The card is minutes old by the time it is
  tapped and the email/order bridge books orders on its own schedule; the
  match it was drawn from is not current.
- ...and checks for a resolved sibling batch. findDuplicateBatch ignores
  unresolved batches by design, so two pending cards for one order each booked
  it.
- INSERT ... ON CONFLICT (source, source_ref) DO NOTHING with a re-select,
  rather than SELECT-then-INSERT, which loses a double tap.
- A non-AUD receipt is booked as order-ingestion.ts books one: amount native,
  amount_aud NULL, foreign_currency_* set. No FX rate exists at intake.

Also: EXCLUDE_RECONCILED_SOURCE in both queries instead of a bare
superseded_by_id check — a reconciled manual twin at the exact key read as an
ambiguity, which is the same spend counted twice. And routeFood refuses to
replace an itemised receipt with an extraction carrying no prices at all: the
reconciliation check passes those (there is nothing to check), so without this
a priceless upload silently destroys a good record. It holds instead.

RouteContext.subtotal was written and never read; it is replaced by currency,
which the cards need.
 2026-08-21 17:53:06 +10:00
siddharthd 9a442b7355 intake: escape mrkdwn, and stop printing every total as dollars 
Merchant names, transaction descriptions and model output are none of ours:
Slack reads &, < and > as markup, so "Ben & Jerry's" and a description
carrying <http://…|text> rendered as something the card never meant to show.
esc() on every interpolated value in mrkdwn — never on our own literal *bold*,
and never on a button's plain_text label, where entities would print verbatim.

money() also hard-coded "$" while the order nudge already prints "USD 15.00".
The cards now carry the batch's currency.
 2026-08-21 17:52:47 +10:00
siddharthd f9638cc5e5 line-items: one metadata row, agreed on by the write and the read 
Three defects on one surface, all invisible while a transaction has exactly
one expense_metadata row — which is almost always.

1. ORDER BY (transaction_id = $1) DESC did the opposite of its comment.
   Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at the
   transaction has a NULL transaction_id, so the clause written to prefer the
   owned row preferred the other one — in exactly the case it exists for.
   matchTransaction already spelled out NULLS LAST; now so do this and
   GET /api/transactions/[id]/order, which had no ORDER BY at all and could
   read a different row from the one an enrichment had just written.

2. replaceLineItems takes an optional metadataId. The match resolves a row
   through a platform-filtered join; re-picking independently here can land
   the write on a row the match ruled out.

3. previous_line_items on intake_batches is the undo baseline, recorded once
   per batch under COALESCE and read back. Returning whatever was just read
   hands a pantry retry its own write as the "previous" state, so undo
   restores the thing it undid. Migration 0034 is unapplied anywhere, so the
   column is added there rather than in a 0035.
 2026-08-21 17:52:47 +10:00
siddharthd 7424d13858 intake: only a reply that finished is a reply the schema constrained 
liteLLMExtractor rejected finish_reason "length" and accepted everything else,
so a content_filter or tool_calls stop was parsed as if the strict json_schema
had held. Reject any reason present and not "stop", naming it.
 2026-08-21 17:52:25 +10:00
siddharthd be3d10d4f1 intake: match merchants on whole words, not substrings 
merchantKey("Bigals Pizza") contains "iga", so GROCERY_PATTERN filed a pizza
shop as a grocer and merchantMatches matched it against every IGA on the same
date and total. Short names in a substring list collide with ordinary words,
and the collision reads as a correct classification rather than a bug.

GROCERY_PATTERN is now a word-boundary alternation; merchantMatches asks
whether one name's words appear as a contiguous run inside the other's, which
keeps Coles <-> Coles (Wyndham Vale) and Order - Coles (Wyndham Vale).

Also collapses the third PLATFORM_LABEL copy: order-ingestion's was exported
and read nowhere, order-details.tsx had its own, and the intake one — the only
live copy — was missing `uber`. One map, in intake-rules.
 2026-08-21 17:52:25 +10:00
siddharthd a4e5f540cc docs: intake router routes, env vars, migration 0034
ci / lint-test (push) Successful in 48s
Details
2026-08-21 12:10:06 +10:00
siddharthd 32a47bf8ee test(intake): rules, PDF text, C2 idempotency, enrich-vs-candidates, pantry hand-off, Slack taps 
Unit: merchant pattern, header validation, reconciliation tolerance, verbatim
line items, one-call message shape, card action ids; unpdf over the two real
Uber Eats exports (Coles $85.74 / 15 lines, Burgertory $22.40 / Michael +
Joel). Integration: fetch stubbed at the LiteLLM and pantry boundaries so the
PDF path, persistence, match and writes run for real against personal_test.
intake-live.test.ts is opt-in (INTAKE_LIVE_TEST=1) and was run green against
cloud-budget on both PDFs.
 2026-08-21 12:10:06 +10:00
siddharthd c7ea516c34 slack: dispatch the four intake taps on action_id (plan §11 C5) 
intake_use_candidate / intake_create / intake_grocery / intake_meal carry JSON
{batch_id, ...} rather than <id>:<verb>, so they branch before the order-nudge
parse. They act on the persisted batch: enrich the chosen transaction, create
ING-10-style (statement_id NULL, owner_id from shared_by, source/source_ref
make a second press a no-op), or remember the merchant's kind and route. A tap
on an already-resolved batch returns its current card and writes nothing. Card
updates go to response_url, as the existing handlers do.
 2026-08-21 12:10:05 +10:00
siddharthd db5c9a02a7 intake: POST /api/intake router + PUT /api/transactions/[id]/line-items (plan §11 C1, C2) 
One upload point, every applicable step. Persist the batch, classify each
file, ONE LiteLLM call (cloud-budget, temperature 0, strict json_schema; PDFs
as unpdf text, never pixels) with an order-item prompt that keeps names
verbatim, header (merchant, date, total), merchant pattern
(coles|woolworths|woolies|aldi|iga|foodworks|costco) -> grocery | food | ask,
exact (platform, merchant, date, total) match against transactions +
expense_metadata, then act:

- food + unique match  -> replace line_items via the C2 path, 'enriched'
- food + miss/ambiguity -> 'candidates' (same merchant ±3d) with
  intake_use_candidate / intake_create buttons; never auto-creates
- grocery              -> original files to pantry POST /api/capture with
  finance_target_transaction_id + finance_action enrich|create (C3);
  409 -> 'duplicate'
- ask / needs_header / held / duplicate / unsupported as replies, not writes

Reconciliation is against the printed item subtotal (a delivery total is
items + fees - promotions: 18.94 of Coles lines settled at 5.74),  or
2%, else 'held'. Originals are kept under INTAKE_FILE_DIR by sha256 so a later
'Groceries' tap can still forward them.

C2 is an idempotent replace on transaction_id = :id OR
matched_transaction_id = :id, accepts INTAKE_TOKEN or RECEIPT_INGEST_TOKEN,
creates the expense_metadata row when absent, 404 when the transaction does
not exist, returns {previous_line_items, line_items} so pantry's undo is the
same call with the previous list.
 2026-08-21 12:10:05 +10:00
siddharthd a538376638 db(0034): intake_batches + intake_merchant_kinds for the intake router 
One row per upload through POST /api/intake, written BEFORE extraction so a
Slack tap minutes later has something to act on and a re-share (same sha256
set, or the same (platform, merchant, date, total) header already resolved)
is detected. Columns are the §11 C1 contract in smarthome
plans/intake-router-slice-plan.md. intake_merchant_kinds holds the remembered
'Groceries or a meal?' answer per normalised merchant (plan §2, Codex #6).

Not applied to production — finance has no migrate service; apply by hand at
deploy, then npm run test:setup.
 2026-08-21 12:10:05 +10:00
siddharthd b958d05287 test(db): resolve the integration DSN at run time instead of storing an IP (#161)
ci / lint-test (push) Successful in 1m5s
Details 
setup-test-db.sh wrote .env.test once with the Postgres container's bridge IP
baked in, and Docker reassigns those on recreation. The file rotted silently:
its stored 172.22.0.18 belongs to **gitea** as of today, and an earlier drift
had the suite pointing its TRUNCATE at **postgres-pantry** — which it would
have wiped had the credentials happened to match.

Root-cause fix: stop persisting a host at all.

- scripts/resolve-test-db.sh re-derives the DSN from the live container and
  rewrites .env.test. Resolution order is TEST_DATABASE_URL, then the
  container's published host port, then the bridge IP. `docker port
  postgres-personal` reports 127.0.0.1:5432 since the 2026-08-17 PG17
  migration, so loopback is what gets used here — and a loopback address
  cannot be reassigned to another container, which is the whole point.
  (Note this supersedes the smarthome CLAUDE.md's "postgres-personal publishes
  no host port"; that is now stale.) The bridge-IP fallback is kept for a host
  where the port is not published, and the script never prints the password.
- Wired as `pretest:integration`, so npm runs it before every integration run.
  .env.test becomes a generated artifact rather than stored configuration.
- setup-test-db.sh now only creates the database and copies the schema, then
  delegates to the resolver.
- helpers.ts::assertTestDatabase stays and its message now points at the
  resolver rather than at `docker inspect`. It still earns its place: running
  vitest directly bypasses the pretest hook, and a stale .env.test would then
  still be read.

Docs: CLAUDE.md gains a Testing section carrying this, the prisma-generate
trap for a fresh checkout or worktree, and the frozen-clock rule for tests
whose subject is defined relative to "now". AGENTS.md updated for the same, and
for lint now being blocking.
 2026-08-21 00:52:54 +10:00
siddharthd eaf5d5ac66 test(analytics): guard the three unguarded Priority 0 behaviours (#155) 
docs/ui-information-architecture-review.md claimed Priority 0 was guarded by
analytics-sql.test.ts. That file only ever covered the three SQL *fragments*
(EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY, INVESTMENT_SIGNED). Items 2, 3 and
6 of the Priority 0 list had nothing, and all three fail silently — no error,
no empty screen, just a number that quietly disagrees with the one beside it.

New: src/__tests__/integration/analytics-priority0.test.ts, 18 tests.

- **Item 2, pace/headline parity.** /api/analytics/daily summed over a month
  must equal /api/analytics/monthly's headline to the cent, per category, and
  across every month in the window. Cents rather than exact equality is the
  assertion because the two routes group at different grains (month+category
  vs month+day+category) and both carry 4dp — the displayed grain is what has
  to agree, which is precisely the bug (the chart ending the month a few cents
  off the headline above it). Fixtures are only shapes where a naive gross
  debit sum disagrees: a 50% split, a refund, fee and interest rows, an
  itemised loan repayment, a reconciled source row, family-tagged spend,
  transfers/investment/income, and a row someone else paid that I hold a split
  on. A fourth test asserts the naive sum does NOT match — without it, parity
  between two queries identical by construction would pass on fixtures too
  plain to discriminate.
- **Item 3, fees date range.** Default 12 months with the window reported
  (from=2025-09-01, not 08-31 — toDateStr, not toISOString); the older tail
  excluded; months=0 all-time with a null range rather than a period claim the
  number does not make; a narrower months=3; the transaction-level fee lines
  bounded by the same window as the statement totals, since a bounded total
  beside an unbounded list is the same defect wearing the fix; and a nonsense
  months param clamped rather than producing an Invalid Date window.
- **Item 6, partial-month comparisons.** The in-progress month never enters a
  baseline; a partial month is compared through the same day of each prior
  month; a complete month keeps full totals; top movers truncate both sides;
  year boundary and leap February.

Item 6 was untestable by construction, which is why it had no test: the rules
were closures in budget/page.tsx reading `new Date()` inline. Extracted to
src/lib/partial-month.ts with an injectable clock, and budget/page.tsx now
imports them. The extraction also de-duplicates the baseline filter, which was
written out identically in two places.

The file freezes the clock (vi.useFakeTimers, toFake: ["Date"] only — pg needs
real timers for connection handling). Without it the fixtures drift out of the
routes' rolling windows over time and the assertions go vacuous, which is the
same class of silent failure the file exists to catch.

Mutation-checked, all three: breaking daily's spend semantics fails 3 of the
item-2 tests (the placebo correctly still passes); forcing the fees window to
all-time fails 5; letting the current month into the baseline fails 1; and
dropping the elapsed-day truncation fails 3.
 2026-08-21 00:52:36 +10:00
siddharthd fc9ffaba25 chore(ci): clear the lint debt and make lint blocking (#79) 
CI ran lint with continue-on-error because 23 errors predated it. All are
fixed, so the flag is gone and a typecheck step is added beside it — nothing
else in CI compiles the test files, so a type error in a fixture could only
ever be found on a developer's machine.

Real fixes, no blanket disables and no suppressions:

- react-hooks/set-state-in-effect (6): each was "reset or seed state when an
  input changes", so each becomes a state adjustment during render behind a
  previous-value guard — the documented React pattern for it. Two were also
  latent bugs. transactions/page.tsx depended on `others`, rebuilt by .filter()
  every render, so the effect already ran every render, one commit late.
  shared/page.tsx depended on the `tagIds` array identity, so the row window
  was reset more often than "changing any filter" claims — including after
  "Show more", the one interaction it must not undo; the key now joins by
  value. csv-import-modal reads localStorage once at mount, which is a lazy
  useState initializer, not an effect (SSR-guarded, though the modal is
  mounted from client state and never prerenders).
- react-hooks/static-components (3): SortIcon was declared inside SharedPage,
  making it a new component type on every render. Hoisted to module scope with
  the sort state passed as props.
- @typescript-eslint/no-explicit-any (9): the dynamically imported route
  handlers in two integration tests. Typed as their real signatures with one
  `as unknown as NextRequest` cast at the fixture boundary, so the call sites
  and response bodies get checked instead of opted out.
- react/no-unescaped-entities (5): exact-equivalent entities (&apos;, &quot;),
  so no rendered text changes.

Also fixes the one pre-existing tsc error (unit/order-ingestion.test.ts:511):
`options` is optional, and `?? 0` keeps the assertion failing on an absent
list rather than silencing it with a non-null bang.

9 warnings remain (no-unused-vars, exhaustive-deps). eslint exits 0 on
warnings, so lint blocks on errors only — deliberately, since none of the
remaining warnings should stop a push.
 2026-08-21 00:52:14 +10:00
48 changed files with 4363 additions and 181 deletions
Expand all files Collapse all files
.gitea/workflows/ci.yml
+12 -4
View File
@@ -22,10 +22,18 @@ jobs:
# the normal colour.
- name: Generate Prisma client
run: npx prisma generate
# Advisory until the pre-existing lint debt is cleared (2026-07-19:
# ~20 errors across budget/insights/shared pages) — then make blocking.
- name: Lint (advisory)
# Blocking since #79. It was advisory from 2026-07-19 while 23 errors sat
# across the pages and the integration tests; those are now fixed, so a
# new one must not be able to slip in behind a green tick. Warnings are
# still allowed (eslint exits 0 on them) — there are 9 left, all
# `no-unused-vars` or `exhaustive-deps` advisories, none of which should
# stop a push.
- name: Lint
run: npm run lint
continue-on-error: true
# Typecheck is separate from `next build`: nothing else in CI compiles the
# test files, so a type error in a fixture could only ever be found on a
# developer's machine.
- name: Typecheck
run: npx tsc --noEmit
- name: Unit tests
run: npm test
AGENTS.md
+5 -4
View File
@@ -13,10 +13,11 @@ Keep data flow consistent: API routes call query functions in `src/lib/queries.t
- `npm ci` installs the locked dependency set (Node 22 is used in CI).
- `npm run dev` starts the local Next.js development server.
- `npm run build` creates a production build; `npm start` serves it.
- `npm run lint` runs the Next.js ESLint configuration. Existing lint debt makes CI lint advisory, but new code should pass.
- `npm run lint` runs the Next.js ESLint configuration. **Blocking in CI since #79** — the pre-existing debt is cleared, so it must stay at zero errors (warnings are still allowed).
- `npx tsc --noEmit` typechecks, including the test files. Also blocking in CI.
- `npm test` runs fast unit tests.
- `npm run test:setup` prepares the PostgreSQL test database using `.env.test`.
- `npm run test:integration` runs database-backed tests.
- `npm run test:setup` creates/refreshes the `personal_test` database (run it once, and after any migration).
- `npm run test:integration` runs database-backed tests. It re-resolves the DSN from the live container first (`scripts/resolve-test-db.sh`), so `.env.test` must never carry a hand-written host — see `CLAUDE.md` → Testing.
- `npm run test:all` runs both test suites.
## Coding Style & Naming Conventions
@@ -33,4 +34,4 @@ Add schema changes as the next numbered `prisma/migrations/NNNN_description/migr
## Commits & Pull Requests
History follows concise Conventional Commit-style subjects such as `feat(rules): preview rule changes`, `fix(trips): ...`, and `docs: ...`. Keep commits focused. Pull requests should explain behavior and data-model impact, link related issues, list validation commands, and include screenshots for UI changes. Ensure unit tests and the production build pass; call out any known lint warnings or migration steps.
History follows concise Conventional Commit-style subjects such as `feat(rules): preview rule changes`, `fix(trips): ...`, and `docs: ...`. Keep commits focused. Pull requests should explain behavior and data-model impact, link related issues, list validation commands, and include screenshots for UI changes. Ensure lint, typecheck, unit tests and the production build pass; call out any migration steps.
CLAUDE.md
+126
View File
@@ -59,6 +59,49 @@ docker exec postgres-personal psql -U personal -d personal
docker logs finance -f
```
## Testing
```bash
npm test # unit — pure logic, no database
npm run test:integration # database-backed; resolves the DSN itself (see below)
npm run test:all # both
npm run test:setup # once, and after any migration: creates/refreshes personal_test
npm run lint # blocking in CI since #79 — 0 errors, warnings allowed
npx tsc --noEmit # blocking in CI; nothing else compiles the test files
```
**The integration DSN is resolved at run time — never hand-write a host into
`.env.test`.** `scripts/resolve-test-db.sh` runs automatically as
`pretest:integration` and rewrites that file from the live container before
every run. It prefers the container's **published host port** (`docker port
postgres-personal` → `127.0.0.1:5432`, published since the 2026-08-17 PG17
migration) and falls back to the bridge IP only where that is absent. Set
`TEST_DATABASE_URL` to override it entirely.
This replaces the old "refresh the container IP" instruction, which described
the defect rather than a workaround (#161). `setup-test-db.sh` used to bake the
bridge IP into `.env.test` once, and Docker reassigns those on recreation: by
2026-08-21 the stored `172.22.0.18` belonged to **gitea**, and an earlier drift
had pointed the suite at **postgres-pantry** — whose database it would have
TRUNCATEd had the credentials happened to match. A loopback address cannot be
reassigned to another container, which is why the published port wins.
**Two guards, and both matter.** `helpers.ts::assertTestDatabase` refuses to
truncate anything whose `current_database()` is not `personal_test`, and the
suite runs `fileParallelism: false` because every file shares that one database
and `resetDB()` TRUNCATEs it.
**A fresh checkout or worktree needs `npx prisma generate` first.** The client is
generated into the gitignored `src/generated/prisma`, so without it every test
that reaches `src/lib/db.ts` fails on a missing package — which reads as a code
error rather than a setup one. CI does this explicitly for the same reason.
**Freeze the clock in any test whose subject is defined relative to "now".**
`analytics-priority0.test.ts` uses `vi.useFakeTimers({ toFake: ["Date"] })` —
only `Date`, because `pg` needs real timers for connection handling. Without it
the fixtures drift out of the routes' rolling windows over time and the
assertions quietly stop testing anything.
## Architecture
### Key Files
@@ -579,6 +622,89 @@ and one tap must not flatten it. It says which case it refused on.
Slack reaches this route through an n8n webhook, not directly — see the
smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory.
### Intake router — one upload, every applicable step (2026-08-21)
`POST /api/intake` (token `INTAKE_TOKEN`, multipart: `source`, `shared_by`,
`batch_id`, 1–6 `file`s) is the finance half of the share-sheet intake; design
and the cross-repo contract are in smarthome `plans/intake-router-slice-plan.md`
§11 (C1, C2, C5 live here; C3/C4 are pantry-app; C6 is n8n + compose).
- **Order of operations is the plan's and is load-bearing:** persist
`intake_batches` row → classify each file → ONE LiteLLM call for the whole
batch (`cloud-budget`, `temperature 0`, strict `json_schema`; PDFs go in as
`unpdf` text, never pixels) → header `(merchant, date, total)` → merchant
kind → duplicate check → reconciliation → match → act. `src/lib/intake-rules.ts`
holds the pure decisions, `intake-router.ts` the orchestration.
- **Match is `(platform, merchant, date, total)`, exact, never widened.** A
miss or an ambiguity returns candidates (same merchant ±3 days) with
`intake_use_candidate` / `intake_create` buttons; a transaction is created
only on the explicit tap, ING-10-style (`statement_id NULL`, `owner_id` from
`shared_by` → `participants.email`, else `DEFAULT_OWNER_ID`;
`source='intake'`, `source_ref=batch_id` so a second press is a no-op —
written as `ON CONFLICT … DO NOTHING`, because a SELECT-then-INSERT loses a
double tap). A non-AUD receipt is booked the way `order-ingestion.ts` books
one: `amount` native, `amount_aud` **NULL**, `foreign_currency_*` set — no FX
rate exists at intake and inventing one is worse than admitting there is none.
- **A NULL platform means "no platform", not "any platform".** An in-store tap
(`coles`, `woolworths`, `other`) matches only rows whose `expense_metadata`
carries no platform of its own. The wildcard reading let a till receipt match
the Uber Eats order for the same shop on the same day and overwrite its items.
The **candidates** query is the opposite and deliberately so: it has no
platform filter at all, because the card is a human choice shown precisely
because the exact key missed — the platform is one of the things that may be
wrong, so the list must widen there, not narrow.
- **The Create tap re-runs the match before it inserts**, and checks for a
resolved sibling batch. The card is minutes old by the time it is pressed and
the email/order bridge books orders on its own schedule; `findDuplicateBatch`
ignores *unresolved* batches by design, so two pending cards for one order
would otherwise each book it.
- **One `expense_metadata` row, agreed on by every query that resolves one.** A
transaction can own a row *and* be the `matched_transaction_id` of another;
`matchTransaction` returns the row it resolved through and `replaceLineItems`
writes to exactly that one. Every reader orders by
`(transaction_id = $1) DESC **NULLS LAST**, id` — the match, the write, the
order-detail GET, `getTransactions`' provenance LATERAL, `merchantForTransaction`
and the Slack details modal. The `NULLS LAST` is load-bearing: Postgres orders
`DESC` as `NULLS FIRST` and the row that merely *points* at the transaction has
a NULL `transaction_id`, so the clause meant to prefer the owned row preferred
the other one. Measured 2026-08-21: 667 rows, 147 pointing-only, **zero**
transactions resolving to more than one row — the ordering changes nothing
today and exists so the readers still agree when multiplicity appears.
- **The create is guarded by a transaction-scoped advisory lock on the
PURCHASE**, not the batch. `ON CONFLICT (source, source_ref)` handles a double
tap on one card; two *different* batches for one purchase have different
`source_ref`s and both clear their match and duplicate checks before either
inserts. The lock has to live inside `withTransaction` (`src/lib/db.ts`):
`queryRaw` is `$queryRawUnsafe` on a **pooled** client, so every call is its
own implicit transaction and `pg_advisory_xact_lock` taken in a statement of
its own is released before the next statement runs. The loser reports
`duplicate` pointing at the winner's transaction — the same answer the
sequential path gives.
- **`replaceLineItems` snapshots the undo baseline BEFORE it writes, on both
paths**, and treats an UPDATE that touched no row as a failure rather than a
success. The metadata INSERT is `ON CONFLICT (transaction_id) DO UPDATE` and
carries the batch's **currency** — a hard-coded `'AUD'` made `OrderDetails`
print `$` on a USD order, because it prefers `expense_metadata.currency`.
- **`intake_batches.previous_line_items` is the undo baseline, written once.**
Returning whatever the write just read makes a pantry retry get its own write
back as the "previous" state, so undo restores the thing it undid. Recorded
under `COALESCE(previous_line_items, …)` and read back.
- **Grocery never writes finance here.** The ORIGINAL files go to pantry
`POST /api/capture` with `intake.finance_target_transaction_id` +
`finance_action: enrich|create`; pantry's confirm calls
`PUT /api/transactions/[id]/line-items` back (C4), which is why that route
accepts `RECEIPT_INGEST_TOKEN` too. Files are kept under `INTAKE_FILE_DIR`
keyed by sha256 so a later "Groceries" tap can still forward them.
- **Reconciliation is against the printed item subtotal, not the total.** A
delivery order's total is items + fees − promotions ($118.94 of Coles lines
became $85.74), so checking lines against the total would hold every
delivery order. Tolerance $1 or 2%; a failure is `held`, nothing written.
- **Names stay verbatim.** Burgertory calls its burgers "Michael" and "Joel";
the prompt forbids canonicalising and the stored shape is the order lane's
(`qty`, `description`, `amount`, `options`) so `order-details.tsx` renders it.
- Statuses beyond the contract list: `ask` (the "Groceries or a meal?" holding
state), `held`, `error` (502 — model or pantry down; the batch row says so).
### The shared loan
The loan is a **separate ledger**, not a shared expense and not a settlement
README.md
+14
View File
@@ -282,6 +282,18 @@ All routes require authentication via `X-Forwarded-User` header (set by Traefik)
| GET / POST | `/api/tags` | List or create tags |
| PATCH / DELETE | `/api/tags/[id]` | Update or delete a tag |
**Machine routes** — shared-secret `x-ingest-token` header instead of `X-Forwarded-User`; each fails closed when its token is unset:
| Method | Route | Token | Description |
|--------|-------|-------|-------------|
| POST / PATCH | `/api/orders/ingest` | `ORDER_INGEST_TOKEN` | n8n order receipts → transactions (see smarthome CLAUDE.md "Order Processor") |
| POST | `/api/receipts/ingest` | `RECEIPT_INGEST_TOKEN` | pantry-app confirmed grocery receipts → one transaction per tender leg (ING-10) |
| POST | `/api/intake` | `INTAKE_TOKEN` | **Intake router** (2026-08-21): multipart `source` (`uber_eats\|doordash\|coles\|woolworths\|other`), `shared_by`, `batch_id` (uuid), 1–6 `file`s (image / PDF / text). Persists `intake_batches`, extracts once via LiteLLM `cloud-budget` (PDF → text via `unpdf`), matches `(platform, merchant, date, total)` exactly, then enriches the matched order's `line_items` (food), forwards the original files to pantry `POST /api/capture` (grocery), or replies with candidates / "Groceries or a meal?" / needs-header / held / duplicate. Returns `{kind, status, transaction_id?, capture_id?, candidates?, slack_blocks, text}` |
| PUT | `/api/transactions/[id]/line-items` | `INTAKE_TOKEN` or `RECEIPT_INGEST_TOKEN` | Idempotent replace of `expense_metadata.line_items` for `transaction_id = :id OR matched_transaction_id = :id`; body `{line_items, source: 'intake'\|'pantry', batch_id?, platform?, replace: true}`; creates the metadata row if absent, never a transaction (404). Returns `{previous_line_items, line_items}` — pantry calls it on every confirm and with `previous_line_items` on undo |
| POST | `/api/slack/interactive` | `ORDER_INGEST_TOKEN` + Slack signature | Order-nudge buttons and the intake taps (`intake_use_candidate`, `intake_create`, `intake_grocery`, `intake_meal`) |
Env for the intake router: `INTAKE_TOKEN`, `LITELLM_BASE_URL` + `LITELLM_KEY` (OpenAI-compatible, alias `cloud-budget`), `PANTRY_BASE_URL` (+ the existing `RECEIPT_INGEST_TOKEN` as pantry's machine auth), optional `INTAKE_FILE_DIR` (original files by sha256, default `/app/data/intake` — a later "Groceries" tap re-sends them) and `FINANCE_PUBLIC_URL` (the card's View link, default `https://finance.bosecamp.com`). Design: smarthome `plans/intake-router-slice-plan.md` §11.
---
## Ingestion Pipeline
@@ -347,6 +359,8 @@ docker exec postgres-personal psql -U personal -d personal \
| `0009_split_payments` | `split_payments` |
| `0010_csv_import_reconcile` | `reconciled_with_id`, CSV import support |
| `0011_trips` | `trips`, `trip_id` on `transaction_overrides` (catch-up — was applied directly) |
| … | |
| `0034_intake_batches` | `intake_batches` (one row per `/api/intake` upload, written before extraction) and `intake_merchant_kinds` (the remembered "Groceries or a meal?" answer per merchant) |
> `paperless_doc_id` on statements and the `uq_statements_paperless_doc_id` index were added directly (not tracked in a migration file).
> `owner_id` on transactions and `statement_id` made nullable were applied directly (March 2026) to support manual transaction entry without a fake statement.
docs/ui-information-architecture-review.md
+17 -7
View File
@@ -47,13 +47,23 @@ Also fixed while in here, both found by checking rather than by proposal:
category-day, so the pace chart ended the month a few cents off its own
headline. Both now carry 4dp and round once, at display.
Partially guarded by `src/__tests__/integration/analytics-sql.test.ts`, which
covers the SQL fragments — `EXCLUDE_RECONCILED_SOURCE`, `NATIVE_CURRENCY` and
`INVESTMENT_SIGNED` — and nothing else. **Items 2, 3 and 6 have no test.** Those
are the three where a regression is silent rather than loud: the pace chart would
simply go back to disagreeing with the headline printed directly above it, the
fees figure back to growing forever, and a partial month back to being measured
against complete ones — all without anything failing. Tracked as #155.
Guarded in two files. `src/__tests__/integration/analytics-sql.test.ts` covers
the SQL fragments — `EXCLUDE_RECONCILED_SOURCE`, `NATIVE_CURRENCY` and
`INVESTMENT_SIGNED`. Since 2026-08-21 `analytics-priority0.test.ts` covers items
2, 3 and 6 (#155): the three where a regression is silent rather than loud — the
pace chart simply going back to disagreeing with the headline printed directly
above it, the fees figure back to growing forever, and a partial month back to
being measured against complete ones, none of which fails anything.
Two things were needed to make item 6 testable at all, and both are the point
rather than incidental. The rules lived as closures inside `budget/page.tsx`
reading `new Date()` inline, so they now live in `src/lib/partial-month.ts` with
an injectable clock; and the tests freeze that clock, because a fixture defined
against the real "now" drifts out of the routes' rolling windows and the
assertion goes vacuous — the same class of silent failure, one level up. Item
2's suite carries a placebo assertion for the same reason: parity between two
queries that are identical by construction proves nothing unless some fixture
can actually tell the old and new spend definitions apart.
### Landed after this doc, in the same family (2026-08-02)
package-lock.json
Generated
+19 -1
View File
@@ -16,7 +16,8 @@
"prisma": "^7.4.2",
"react": "19.2.3",
"react-dom": "19.2.3",
"recharts": "^3.8.0"
"recharts": "^3.8.0",
"unpdf": "^1.8.1"
},
"devDependencies": {
"@tailwindcss/postcss": "^4",
@@ -8494,6 +8495,23 @@
"dev": true,
"license": "MIT"
},
"node_modules/unpdf": {
"version": "1.8.1",
"resolved": "https://registry.npmjs.org/unpdf/-/unpdf-1.8.1.tgz",
"integrity": "sha512-xkURhy2SoGpOIH0a1gLHNkASPIQYonadDJs2AQwPEfUakafeD9EA1WTWWsaR++gfTCXJpV27W7tU1nXuk82UKQ==",
"license": "MIT",
"engines": {
"node": ">=22"
},
"peerDependencies": {
"@napi-rs/canvas": "^0.1.69 || ^1.0.0"
},
"peerDependenciesMeta": {
"@napi-rs/canvas": {
"optional": true
}
}
},
"node_modules/unrs-resolver": {
"version": "1.11.1",
"resolved": "https://registry.npmjs.org/unrs-resolver/-/unrs-resolver-1.11.1.tgz",
package.json
+3 -1
View File
@@ -10,6 +10,7 @@
"test": "vitest run --config vitest.config.ts",
"test:watch": "vitest --config vitest.config.ts",
"test:setup": "bash scripts/setup-test-db.sh",
"pretest:integration": "bash scripts/resolve-test-db.sh",
"test:integration": "vitest run --config vitest.integration.config.ts",
"test:all": "npm test && npm run test:integration"
},
@@ -22,7 +23,8 @@
"prisma": "^7.4.2",
"react": "19.2.3",
"react-dom": "19.2.3",
"recharts": "^3.8.0"
"recharts": "^3.8.0",
"unpdf": "^1.8.1"
},
"devDependencies": {
"@tailwindcss/postcss": "^4",
prisma/migrations/0034_intake_batches/migration.sql
+74
View File
@@ -0,0 +1,74 @@
-- Intake router (plans/intake-router-slice-plan.md §11 C1, smarthome repo).
--
-- One upload point for receipts — a screenshot, a PDF export, a pasted body —
-- that reads what it was given and does every step that applies: fill the
-- empty `line_items` on the order the email pipeline already booked, hand a
-- grocery receipt to pantry, or ask when it cannot tell.
--
-- Every batch is persisted BEFORE anything is extracted, for two reasons:
-- 1. The Slack card it produces carries buttons ("Use this", "Create",
-- "Groceries or a meal?") that are pressed minutes later in a fresh
-- request with nothing but the batch id to go on.
-- 2. A re-share of an already-handled order — a second crop, the PDF after
-- the screenshot — must be recognised, and the only durable evidence of
-- the first share is this row (`file_sha256` overlap, or the same
-- (platform, merchant, date, total) header already resolved).
--
-- `id` is the uuid n8n mints per webhook call, so a retried delivery is the
-- same batch rather than a second one. `header` holds the extracted
-- (merchant, order_date, total, platform, ...) and `items` the verbatim item
-- list, so a later tap can enrich or create without re-reading the files.
--
-- Idempotent: safe to re-run.
CREATE TABLE IF NOT EXISTS intake_batches (
id uuid PRIMARY KEY,
source text NOT NULL,
shared_by text,
file_sha256 text[] NOT NULL DEFAULT '{}',
header jsonb,
items jsonb,
kind text,
status text NOT NULL,
transaction_id integer,
capture_id integer,
-- The undo baseline: what `expense_metadata.line_items` held before this
-- batch first replaced it. Recorded here rather than returned from the write,
-- because pantry re-calls the enrich endpoint on every confirm and after a
-- timeout — a caller that hands back whatever it just read gives a retry its
-- OWN write as the "previous" state, and undo then restores the thing it was
-- meant to reverse. Written under COALESCE, so only the first snapshot sticks.
previous_line_items jsonb,
created_at timestamptz NOT NULL DEFAULT now(),
resolved_at timestamptz
);
-- For a database that already ran an earlier draft of this file.
ALTER TABLE intake_batches ADD COLUMN IF NOT EXISTS previous_line_items jsonb;
COMMENT ON TABLE intake_batches IS
'One row per upload batch through POST /api/intake. Persisted before '
'extraction so Slack taps can act on it later and re-shares are detected. '
'kind: grocery|food|ask|held|unsupported. status: received|enriched|created|'
'candidates|forwarded_to_pantry|needs_header|duplicate|unsupported|held|error.';
-- Overlap lookups for the re-share check (`file_sha256 && $1`).
CREATE INDEX IF NOT EXISTS intake_batches_file_sha256_gin
ON intake_batches USING gin (file_sha256);
CREATE INDEX IF NOT EXISTS intake_batches_transaction_id
ON intake_batches (transaction_id) WHERE transaction_id IS NOT NULL;
-- The "Groceries or a meal?" answer, remembered per merchant (plan §2, Codex
-- #6). A tiny answer table, not merchant master-data: the pattern list in code
-- decides the obvious cases and this catches the `Romeo's Food Hall`s. Keyed on
-- the normalised merchant name the router already computes for matching.
CREATE TABLE IF NOT EXISTS intake_merchant_kinds (
merchant_key text PRIMARY KEY,
kind text NOT NULL CHECK (kind IN ('grocery', 'food')),
created_at timestamptz NOT NULL DEFAULT now()
);
COMMENT ON TABLE intake_merchant_kinds IS
'Per-merchant answer to "Groceries or a meal?" from the intake Slack card. '
'Consulted after the grocery pattern list; an unknown merchant asks once.';
prisma/schema.prisma
+27
View File
@@ -268,3 +268,30 @@ model rule_apply_runs {
reverted_at DateTime?
snapshot Json @default("[]")
}
// 0034 — intake router. One row per upload batch, written before extraction so
// Slack taps can act on it later and a re-share is detected. Columns are the
// §11 C1 contract in plans/intake-router-slice-plan.md (smarthome repo).
model intake_batches {
id String @id @db.Uuid
source String
shared_by String?
file_sha256 String[] @default([])
header Json?
items Json?
kind String?
status String
transaction_id Int?
capture_id Int?
// The undo baseline, written once per batch (see 0034 and line-items.ts).
previous_line_items Json?
created_at DateTime @default(now()) @db.Timestamptz(6)
resolved_at DateTime? @db.Timestamptz(6)
}
// 0034 — the remembered answer to "Groceries or a meal?" per merchant.
model intake_merchant_kinds {
merchant_key String @id
kind String
created_at DateTime @default(now()) @db.Timestamptz(6)
}
scripts/resolve-test-db.sh
Executable
+66
View File
@@ -0,0 +1,66 @@
#!/bin/bash
# Resolve the integration-test DSN at run time and (re)write .env.test.
#
# Why this exists (#161): .env.test used to be written ONCE by
# setup-test-db.sh with the Postgres container's bridge IP baked in. Container
# IPs are reassigned on recreation, so the file rotted silently — by
# 2026-08-21 its 172.22.0.18 belonged to *gitea*, and an earlier drift had the
# suite pointing its TRUNCATE at postgres-pantry. It only failed safe because
# the credentials happened not to match.
#
# The fix is to stop persisting a host at all: this runs as `pretest:integration`,
# so the DSN is re-derived from the live container immediately before every run.
#
# Resolution order:
# 1. An explicit TEST_DATABASE_URL in the environment wins (CI, remote DB).
# 2. The container's published host port (`docker port`) — postgres-personal
# publishes 127.0.0.1:5432 since the 2026-08-17 PG17 migration. Preferred
# because a loopback address cannot be reassigned to another container.
# 3. The container's bridge IP, for a host where the port is not published.
#
# helpers.ts::assertTestDatabase is the backstop: it refuses to TRUNCATE
# anything whose current_database() is not `personal_test`.
set -euo pipefail
PG_CONTAINER=${PG_CONTAINER:-postgres-personal}
PG_USER=${PG_USER:-personal}
PG_PASS=${PG_PASS:-personalpassword123}
TEST_DB=${TEST_DB:-personal_test}
ENV_FILE="$(cd "$(dirname "$0")/.." && pwd)/.env.test"
if [ -n "${TEST_DATABASE_URL:-}" ]; then
DSN="$TEST_DATABASE_URL"
SOURCE="TEST_DATABASE_URL"
else
if ! docker inspect "$PG_CONTAINER" >/dev/null 2>&1; then
echo "ERROR: container '$PG_CONTAINER' not found. Start it, or set TEST_DATABASE_URL." >&2
exit 1
fi
# `docker port` prints e.g. "127.0.0.1:5432" (possibly several lines).
PUBLISHED=$(docker port "$PG_CONTAINER" 5432/tcp 2>/dev/null | head -1 || true)
if [ -n "$PUBLISHED" ]; then
HOSTPORT="$PUBLISHED"
SOURCE="published port"
else
IP=$(docker inspect "$PG_CONTAINER" \
--format '{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' 2>/dev/null \
| tr ' ' '\n' | grep -v '^$' | head -1)
if [ -z "$IP" ]; then
echo "ERROR: '$PG_CONTAINER' has neither a published 5432 nor a bridge IP." >&2
exit 1
fi
HOSTPORT="$IP:5432"
SOURCE="bridge IP"
fi
DSN="postgresql://$PG_USER:$PG_PASS@$HOSTPORT/$TEST_DB"
fi
printf 'DATABASE_URL=%s\n' "$DSN" > "$ENV_FILE"
# Never print the password.
echo "resolve-test-db: ${DSN%%://*}://***@${DSN#*@} (via $SOURCE)"
scripts/setup-test-db.sh
+14 -18
View File
@@ -1,25 +1,24 @@
#!/bin/bash
# Creates the personal_test database and writes .env.test
# Run once before integration tests: npm run test:setup
# Creates the personal_test database and copies the production schema into it.
# Run once (and after any migration) before integration tests: npm run test:setup
#
# It no longer writes a host into .env.test. That was the #161 defect: the file
# was written once with the container's bridge IP and then rotted as the IP was
# reassigned. scripts/resolve-test-db.sh now re-derives the DSN before every
# integration run (wired as `pretest:integration`), and this script just calls it
# at the end so a fresh setup leaves a usable .env.test behind.
set -e
PG_CONTAINER=postgres-personal
PG_USER=personal
PG_PASS=personalpassword123
TEST_DB=personal_test
PG_CONTAINER=${PG_CONTAINER:-postgres-personal}
PG_USER=${PG_USER:-personal}
TEST_DB=${TEST_DB:-personal_test}
# Discover the container's bridge IP (accessible from the host on Linux)
PG_HOST=$(docker inspect "$PG_CONTAINER" \
--format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 2>/dev/null | head -1)
if [ -z "$PG_HOST" ]; then
if ! docker inspect "$PG_CONTAINER" >/dev/null 2>&1; then
echo "ERROR: Could not find container '$PG_CONTAINER'"
exit 1
fi
echo "Postgres container at $PG_HOST:5432"
# Create test database (ignore error if it already exists)
docker exec "$PG_CONTAINER" psql -U "$PG_USER" -d postgres \
-c "CREATE DATABASE $TEST_DB;" 2>/dev/null || true
@@ -31,9 +30,6 @@ docker exec "$PG_CONTAINER" pg_dump -U "$PG_USER" --schema-only "$PG_USER" \
echo "Schema ready."
# Write .env.test
cat > "$(dirname "$0")/../.env.test" << EOF
DATABASE_URL=postgresql://$PG_USER:$PG_PASS@$PG_HOST:5432/$TEST_DB
EOF
bash "$(dirname "$0")/resolve-test-db.sh"
echo ".env.test written — integration tests can now run with: npm run test:integration"
echo "Integration tests can now run with: npm run test:integration"
src/__tests__/fixtures/intake/ubereats-burgertory-2026-08-19.pdf
BIN
View File
Binary file not shown.
src/__tests__/fixtures/intake/ubereats-coles-2026-08-18.pdf
BIN
View File
Binary file not shown.
src/__tests__/integration/analytics-priority0.test.ts
+531
View File
@@ -0,0 +1,531 @@
import { describe, it, expect, beforeAll, beforeEach, afterAll, vi } from "vitest";
import type { NextRequest } from "next/server";
import { createPool, mockDbWithPool, resetDB } from "./helpers";
import {
comparisonBasis,
baselineMonths,
baselineAverage,
spendThrough,
prevMonth,
} from "../../lib/partial-month";
/**
* The three Priority 0 behaviours that shipped without a test (#155).
*
* `docs/ui-information-architecture-review.md` claimed Priority 0 was guarded;
* `analytics-sql.test.ts` only ever covered the three SQL *fragments*
* (EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY, INVESTMENT_SIGNED). Items 2, 3
* and 6 of that list had nothing:
*
* 2. the spend-pace series must use the same spend semantics as the headline
* printed directly above it;
* 3. fees and interest must be bounded by an explicit, reported period;
* 6. a month in progress must never be compared against complete ones.
*
* All three fail *silently* when they regress — no error, no empty screen, just
* a number that quietly disagrees with the one beside it. That is precisely the
* class the review flagged and the reason these are worth the fixture cost.
*
* The clock is frozen for every test in this file. Two of the three behaviours
* are defined relative to "now" (which month is in progress, how far back the
* default fees window reaches), so against the real clock the fixtures would
* drift out of the routes' windows and the assertions would go vacuous — which
* is the failure mode this file exists to prevent, one level up. Only `Date` is
* faked: `pg` uses real timers for its connection handling.
*/
const pool = createPool();
mockDbWithPool(pool);
/** Frozen wall clock: the 15th, so the current month is genuinely half-done. */
const NOW = new Date(2026, 7, 15, 10, 0, 0); // 2026-08-15, local time
const CURRENT_MONTH = "2026-08";
const ELAPSED = 15;
let monthlyGET: (req: NextRequest) => Promise<Response>;
let dailyGET: (req: NextRequest) => Promise<Response>;
let feesGET: (req: NextRequest) => Promise<Response>;
let ownerId: number;
// One cast at the boundary: these are the two properties the analytics routes
// read (`url` for the query string, `headers` for the auth header, which is
// mocked out anyway), not a whole NextRequest.
const req = (path: string) =>
({ url: `http://localhost${path}`, headers: { get: () => null } }) as unknown as NextRequest;
const json = async (res: Response) => (await res.json()) as Record<string, unknown>;
beforeAll(async () => {
vi.useFakeTimers({ toFake: ["Date"] });
vi.setSystemTime(NOW);
vi.doMock("@/lib/auth", () => ({
getCurrentUser: async () => ({ id: ownerId, name: "Owner", email: "o@x" }),
}));
({ GET: monthlyGET } = await import("../../app/api/analytics/monthly/route"));
({ GET: dailyGET } = await import("../../app/api/analytics/daily/route"));
({ GET: feesGET } = await import("../../app/api/analytics/fees/route"));
});
afterAll(() => {
vi.useRealTimers();
});
beforeEach(async () => {
await resetDB(pool);
const a = await pool.query(`INSERT INTO participants (name) VALUES ('Owner') RETURNING id`);
const b = await pool.query(`INSERT INTO participants (name) VALUES ('Other') RETURNING id`);
ownerId = a.rows[0].id;
otherId = b.rows[0].id;
});
let otherId: number;
/** A manual transaction owned by `owner`. Returns its id. */
async function txn(over: {
date: string;
amount: number;
type?: string;
category?: string;
owner?: number;
description?: string;
interest_amount?: number;
reconciled_with_id?: number;
}): Promise<number> {
const r = await pool.query(
`INSERT INTO transactions
(owner_id, statement_id, transaction_date, description, amount,
transaction_type, category, interest_amount, reconciled_with_id, row_index)
VALUES ($1, NULL, $2, $3, $4, $5, $6, $7, $8, 0) RETURNING id`,
[
over.owner ?? ownerId,
over.date,
over.description ?? `fixture ${over.date} ${over.amount}`,
over.amount,
over.type ?? "debit",
over.category ?? "groceries",
over.interest_amount ?? null,
over.reconciled_with_id ?? null,
]
);
return r.rows[0].id as number;
}
async function split(txnId: number, participantId: number, pct: number) {
await pool.query(
`INSERT INTO transaction_splits (transaction_id, participant_id, share_percent)
VALUES ($1, $2, $3)`,
[txnId, participantId, pct]
);
}
async function tag(txnId: number, name: string) {
const t = await pool.query(
`INSERT INTO tags (name) VALUES ($1) ON CONFLICT (name) DO UPDATE SET name = EXCLUDED.name RETURNING id`,
[name]
);
await pool.query(
`INSERT INTO transaction_tags (transaction_id, tag_id) VALUES ($1, $2)`,
[txnId, t.rows[0].id]
);
}
// ───────────────────────────────────────────────────────────────────────────
// Item 2 — the pace chart and the headline must agree
// ───────────────────────────────────────────────────────────────────────────
/**
* The pace chart used to sum gross `amount_aud ?? amount` over
* `transaction_type = 'debit'` in the browser. Measured on live data it ended
* July at $4,747.31 against a headline of $3,597.10 — a 32% overstatement of
* the number printed directly above it, with both labelled "spend".
*
* `/api/analytics/daily` fixed that by building the series from the same
* fragments as `/monthly`. Nothing has been asserting it since, and the two
* routes are separate files: the drift they were merged to prevent can
* reappear the moment one is edited alone.
*
* Every fixture below is a shape where the naive gross-debit sum and the real
* spend definition disagree. The last test proves that — without it, parity
* could hold simply because the fixtures were too plain to discriminate, and
* the test would pass against a route that had regressed.
*/
describe("Priority 0 item 2 — spend pace matches the headline", () => {
/** Everything the two routes are supposed to treat identically. */
async function seedDivergentMonth(month: string) {
const d = (day: number) => `${month}-${String(day).padStart(2, "0")}`;
// Plain debit — both definitions agree on this one.
await txn({ date: d(2), amount: 100, category: "groceries" });
// Half mine. Gross says 200; my share is 100.
const shared = await txn({ date: d(3), amount: 200, category: "groceries" });
await split(shared, ownerId, 50);
await split(shared, otherId, 50);
// A refund. Ignored entirely by a debit-only sum; must cancel spend here.
await txn({ date: d(5), amount: 40, type: "refund", category: "groceries" });
// Fee and interest rows: spend, but not type 'debit'.
await txn({ date: d(6), amount: 15, type: "fee", category: "fees" });
await txn({ date: d(7), amount: 25, type: "interest", category: "fees" });
// An itemised loan repayment: type 'payment', so a debit-only sum skips it,
// but its interest portion is real spend and its principal is not.
await txn({
date: d(8),
amount: 1200,
type: "payment",
category: "loan",
interest_amount: 300,
});
// The manual row a statement line superseded — double count if counted.
const survivor = await txn({ date: d(9), amount: 60, category: "dining" });
await txn({ date: d(9), amount: 60, category: "dining", reconciled_with_id: survivor });
// Categories that are not spend at all.
await txn({ date: d(10), amount: 5000, category: "transfers" });
await txn({ date: d(11), amount: 2000, category: "investment" });
await txn({ date: d(12), amount: 3000, type: "credit", category: "income" });
// A `family`-tagged row: excluded from budget spend by EXCLUDE_NON_BUDGET_TAGS.
const family = await txn({ date: d(13), amount: 500, category: "groceries" });
await tag(family, "family");
// Someone else's row I hold a split on — mine only to the extent of the split.
const theirs = await txn({ date: d(14), amount: 80, category: "dining", owner: otherId });
await split(theirs, ownerId, 25);
await split(theirs, otherId, 75);
}
/** Sum the sparse daily map for one month. */
const sumDaily = (daily: Record<string, Record<number, number>>, month: string) =>
Object.values(daily[month] ?? {}).reduce((s, v) => s + v, 0);
it("the daily series sums to the monthly headline, to the cent", async () => {
await seedDivergentMonth("2026-06");
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
const daily = await json(await dailyGET(req("/api/analytics/daily?months=6")));
const headline = (monthly.totals as Record<string, { spent: number }>)["2026-06"].spent;
const pace = sumDaily(daily.daily as Record<string, Record<number, number>>, "2026-06");
// Cents, not exact float equality: the two routes group at different grains
// (month+category vs month+day+category) and both carry 4dp, so they can
// differ below a cent by construction. The displayed grain is what has to
// agree — that IS the bug, the chart ending the month a few cents off the
// headline it sits under.
expect(Math.round(pace * 100) / 100).toBe(headline);
});
it("agrees per category, not just in total", async () => {
// A total can match while two categories are wrong in opposite directions,
// and the category drill-down is a real surface on this page.
await seedDivergentMonth("2026-06");
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
const daily = await json(await dailyGET(req("/api/analytics/daily?months=6")));
const byCategory = (daily.byCategory as Record<string, Record<string, Record<number, number>>>)["2026-06"];
const rows = monthly.rows as { category: string; spent: Record<string, number> }[];
expect(Object.keys(byCategory).sort()).toEqual(
rows.filter((r) => r.spent["2026-06"] !== undefined).map((r) => r.category).sort()
);
for (const r of rows) {
const expected = r.spent["2026-06"];
if (expected === undefined) continue;
const got = Object.values(byCategory[r.category] ?? {}).reduce((s, v) => s + v, 0);
expect([r.category, Math.round(got * 100) / 100]).toEqual([
r.category,
Math.round(expected * 100) / 100,
]);
}
});
it("holds across every month in the window, not only the seeded one", async () => {
await seedDivergentMonth("2026-05");
await seedDivergentMonth("2026-06");
await seedDivergentMonth("2026-07");
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
const daily = await json(await dailyGET(req("/api/analytics/daily?months=6")));
const totals = monthly.totals as Record<string, { spent: number }>;
const dailyMap = daily.daily as Record<string, Record<number, number>>;
for (const m of monthly.months as string[]) {
expect([m, Math.round(sumDaily(dailyMap, m) * 100) / 100]).toEqual([m, totals[m].spent]);
}
});
it("the fixtures actually discriminate — a gross debit sum does NOT agree", async () => {
// The placebo. Parity between two identical-by-construction queries is easy
// to assert and worthless if no fixture can tell the definitions apart.
// This pins the gap the old client-side series had.
await seedDivergentMonth("2026-06");
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
const headline = (monthly.totals as Record<string, { spent: number }>)["2026-06"].spent;
const naive = await pool.query<{ total: string }>(
`SELECT COALESCE(SUM(COALESCE(amount_aud, amount)), 0)::numeric(14,2) AS total
FROM transactions
WHERE transaction_type = 'debit'
AND transaction_date >= '2026-06-01' AND transaction_date < '2026-07-01'`
);
expect(Number(naive.rows[0].total)).toBeGreaterThan(headline);
});
});
// ───────────────────────────────────────────────────────────────────────────
// Item 3 — fees and interest are bounded by an explicit, reported period
// ───────────────────────────────────────────────────────────────────────────
/**
* `/api/analytics/fees` used to aggregate every statement ever imported with no
* date filter, and the UI printed the result with no period label — so a
* lifetime-to-date total read as a current-period one and grew forever. It was
* overstating the last 12 months by roughly $2,700 of fees.
*
* The regression is silent in the worst way: the number is always plausible and
* always too big, and it gets worse the longer the app is used.
*/
describe("Priority 0 item 3 — fees and interest carry a date range", () => {
/** A statement dated by the period it covers, carrying fees and interest. */
async function statement(billingEnd: string, fees: number, interest: number, bank = "Fixture Bank") {
const r = await pool.query(
`INSERT INTO statements
(filename, bank_name, account_number, billing_end_date, currency,
fees_charged, interest_charged, owner_id)
VALUES ($1, $2, '0000', $3, 'AUD', $4, $5, $6) RETURNING id`,
[`fees-${billingEnd}-${bank}.pdf`, bank, billingEnd, fees, interest, ownerId]
);
return r.rows[0].id as number;
}
beforeEach(async () => {
// Inside the default 12-month window (opens 2025-09-01 from a 2026-08-15 now).
await statement("2026-07-31", 10, 5);
await statement("2025-10-31", 20, 7);
// Outside it — the tail that used to be silently included.
await statement("2024-06-30", 1000, 500);
await statement("2019-01-31", 2500, 900);
});
it("defaults to the last 12 months and says so", async () => {
const body = await json(await feesGET(req("/api/analytics/fees")));
const period = body.period as { months: number; from: string; to: string; all_time: boolean };
expect(period.months).toBe(12);
expect(period.all_time).toBe(false);
// The window opens on the 1st of the month 11 months back, and toDateStr()
// must not shift it a day earlier — `toISOString()` on a local-midnight
// Date does exactly that in any timezone east of Greenwich, which is how
// every analytics window used to be a day early.
expect(period.from).toBe("2025-09-01");
expect(period.to).toBe("2026-09-01");
expect(body.total_fees).toBe(30);
expect(body.total_interest).toBe(12);
});
it("excludes the older tail — the $2,700-of-fees overstatement", async () => {
const body = await json(await feesGET(req("/api/analytics/fees")));
// 3530 of fees exist; only 30 are in the last 12 months.
expect(body.total_fees).toBe(30);
expect(body.total_fees).not.toBe(3530);
});
it("months=0 asks for all time deliberately, and is labelled as such", async () => {
const body = await json(await feesGET(req("/api/analytics/fees?months=0")));
const period = body.period as { months: number; from: null; to: null; all_time: boolean };
expect(period.all_time).toBe(true);
// A null range is the honest answer for an unbounded total; a from/to here
// would be a period claim the number does not actually make.
expect(period.from).toBeNull();
expect(period.to).toBeNull();
expect(body.total_fees).toBe(3530);
expect(body.total_interest).toBe(1412);
});
it("honours a narrower explicit window", async () => {
const body = await json(await feesGET(req("/api/analytics/fees?months=3")));
const period = body.period as { months: number; from: string };
expect(period.months).toBe(3);
expect(period.from).toBe("2026-06-01");
// Only the 2026-07-31 statement falls inside three months.
expect(body.total_fees).toBe(10);
expect(body.total_interest).toBe(5);
});
it("bounds the transaction-level fee lines by the same window", async () => {
// Two halves of the same answer, and only the statement half was ever
// date-filtered in the original. A bounded total beside an unbounded list
// is the same defect wearing the fix.
await txn({ date: "2026-07-10", amount: 12, type: "fee", category: "fees" });
await txn({ date: "2020-02-10", amount: 99, type: "fee", category: "fees" });
const bounded = await json(await feesGET(req("/api/analytics/fees")));
const all = await json(await feesGET(req("/api/analytics/fees?months=0")));
expect((bounded.transactions as unknown[]).length).toBe(1);
expect((all.transactions as unknown[]).length).toBe(2);
});
it("clamps a nonsense months parameter instead of unbounding the total", async () => {
// A NaN reaching `months` would make `new Date(y, NaN, 1)` an Invalid Date
// and the window meaningless. Falling back to 12 keeps the default claim.
const body = await json(await feesGET(req("/api/analytics/fees?months=banana")));
expect((body.period as { months: number }).months).toBe(12);
expect(body.total_fees).toBe(30);
});
});
// ───────────────────────────────────────────────────────────────────────────
// Item 6 — a month in progress is never compared against complete ones
// ───────────────────────────────────────────────────────────────────────────
/**
* Two rules, both silent when broken (see src/lib/partial-month.ts):
*
* 1. the in-progress month never enters a baseline;
* 2. when it is the month selected, both sides of every comparison are
* truncated to the same day.
*
* These are driven off real `/api/analytics/daily` output rather than a
* hand-built map, because rule 2 depends on that route's day-of-month
* granularity — a route returning month totals only would make the rule
* unimplementable while every pure-function test still passed.
*/
describe("Priority 0 item 6 — partial-month comparisons", () => {
/** `perDay` dollars on each of days 1..n of `month`. */
async function evenMonth(month: string, perDay: number, days: number) {
for (let d = 1; d <= days; d++) {
await txn({ date: `${month}-${String(d).padStart(2, "0")}`, amount: perDay, category: "groceries" });
}
}
async function dailyMap() {
const body = await json(await dailyGET(req("/api/analytics/daily?months=12")));
return body.daily as Record<string, Record<number, number>>;
}
it("recognises the in-progress month and the day to compare through", () => {
expect(comparisonBasis(CURRENT_MONTH, NOW)).toEqual({ isPartial: true, compareDay: ELAPSED });
// A complete month compares through its last day, so the same code path
// serves both cases and `spendThrough` becomes a no-op.
expect(comparisonBasis("2026-06", NOW)).toEqual({ isPartial: false, compareDay: 30 });
expect(comparisonBasis("2026-02", NOW)).toEqual({ isPartial: false, compareDay: 28 });
});
it("keeps the in-progress month out of the baseline", () => {
const totals: Record<string, { spent: number }> = {
"2026-05": { spent: 100 },
"2026-06": { spent: 100 },
"2026-07": { spent: 100 },
[CURRENT_MONTH]: { spent: 40 }, // half a month of spend
};
const months = ["2026-05", "2026-06", "2026-07", CURRENT_MONTH];
// Baseline for a complete month: every other complete month with data.
expect(baselineMonths(months, "2026-07", totals, NOW)).toEqual(["2026-05", "2026-06"]);
// Baseline for the month in progress: itself excluded twice over.
expect(baselineMonths(months, CURRENT_MONTH, totals, NOW)).toEqual([
"2026-05",
"2026-06",
"2026-07",
]);
// The partial month is never in it. Averaging its 40 in would drag a
// baseline of 100 down to 85 and make an ordinary month look extravagant.
expect(baselineMonths(months, "2026-07", totals, NOW)).not.toContain(CURRENT_MONTH);
});
it("drops a month with no data rather than averaging a zero into the baseline", () => {
const totals: Record<string, { spent: number }> = { "2026-05": { spent: 100 }, "2026-06": { spent: 0 } };
expect(baselineMonths(["2026-05", "2026-06"], "2026-07", totals, NOW)).toEqual(["2026-05"]);
});
it("compares the month in progress through the same day of each prior month", async () => {
// $10/day everywhere. Prior months ran the full month; the current one has
// only reached the 15th, so it holds $150 against their $300-odd.
await evenMonth("2026-06", 10, 30);
await evenMonth("2026-07", 10, 31);
await evenMonth(CURRENT_MONTH, 10, ELAPSED);
const daily = await dailyMap();
const totals: Record<string, { spent: number }> = {
"2026-06": { spent: 300 },
"2026-07": { spent: 310 },
[CURRENT_MONTH]: { spent: 150 },
};
const months = ["2026-06", "2026-07", CURRENT_MONTH];
const { isPartial, compareDay } = comparisonBasis(CURRENT_MONTH, NOW);
const baseline = baselineMonths(months, CURRENT_MONTH, totals, NOW);
const mine = spendThrough(daily[CURRENT_MONTH], compareDay);
const avg = baselineAverage(baseline, { isPartial, compareDay, daily, totals });
// Truncated on both sides, this month is exactly typical.
expect(mine).toBe(150);
expect(avg).toBe(150);
expect(Math.round(((mine - avg) / avg) * 100)).toBe(0);
// Untruncated — the bug — it reads as 51% below average on identical
// spending, purely because half the month has not happened yet.
const naiveAvg = baseline.reduce((s, m) => s + totals[m].spent, 0) / baseline.length;
expect(Math.round(((mine - naiveAvg) / naiveAvg) * 100)).toBe(-51);
});
it("leaves a complete month compared on full totals", async () => {
await evenMonth("2026-06", 10, 30);
await evenMonth("2026-07", 20, 31);
const daily = await dailyMap();
const totals: Record<string, { spent: number }> = { "2026-06": { spent: 300 }, "2026-07": { spent: 620 } };
const months = ["2026-06", "2026-07"];
const { isPartial, compareDay } = comparisonBasis("2026-07", NOW);
const baseline = baselineMonths(months, "2026-07", totals, NOW);
expect(isPartial).toBe(false);
// Truncation must not quietly apply to a finished month: June's full 30
// days are the right comparison for July's full 31.
expect(baselineAverage(baseline, { isPartial, compareDay, daily, totals })).toBe(300);
});
it("truncates the top-movers comparison on both sides", async () => {
// The movers row asks "biggest category moves vs last month". With one side
// full and the other half-done, every category reads as a large decrease.
await evenMonth("2026-07", 10, 31); // $310 in July
await evenMonth(CURRENT_MONTH, 10, ELAPSED); // $150 so far in August
const body = await json(await dailyGET(req("/api/analytics/daily?months=12")));
const byCategory = body.byCategory as Record<string, Record<string, Record<number, number>>>;
const { compareDay } = comparisonBasis(CURRENT_MONTH, NOW);
const pm = prevMonth(CURRENT_MONTH);
expect(pm).toBe("2026-07");
const now = spendThrough(byCategory[CURRENT_MONTH]?.groceries, compareDay);
const before = spendThrough(byCategory[pm]?.groceries, compareDay);
expect(now - before).toBe(0);
// Against July's full month it would report a $160 fall that never happened.
const fullBefore = Object.values(byCategory[pm].groceries).reduce((s, v) => s + v, 0);
expect(now - fullBefore).toBe(-160);
});
it("crosses a year boundary correctly", () => {
expect(prevMonth("2026-01")).toBe("2025-12");
expect(comparisonBasis("2025-12", NOW)).toEqual({ isPartial: false, compareDay: 31 });
});
it("handles February in a leap year", () => {
// 2028 is a leap year; a hardcoded 28 would silently drop the 29th.
expect(comparisonBasis("2028-02", new Date(2028, 5, 1)).compareDay).toBe(29);
});
});
src/__tests__/integration/helpers.ts
+33 -8
View File
@@ -23,6 +23,25 @@ export function mockDbWithPool(p: Pool) {
const result = await p.query(sql, params);
return result.rows[0] ?? null;
},
// Mirrors withTransaction: one connection, BEGIN/COMMIT around the callback,
// so pg_advisory_xact_lock actually holds across the statements inside it.
withTransaction: async (fn: (q: unknown) => Promise<unknown>) => {
const client = await p.connect();
try {
await client.query("BEGIN");
const result = await fn({
queryRaw: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows,
queryRow: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows[0] ?? null,
});
await client.query("COMMIT");
return result;
} catch (e) {
await client.query("ROLLBACK");
throw e;
} finally {
client.release();
}
},
prisma: p,
}));
}
@@ -30,12 +49,17 @@ export function mockDbWithPool(p: Pool) {
/**
* Refuse to truncate anything that is not the test database.
*
* `DATABASE_URL` in `.env.test` names the Postgres container by IP, and
* container IPs move on recreation: 172.22.0.47 stopped being
* `postgres-personal` and became `postgres-pantry`, so the suite spent a while
* pointing its TRUNCATE at another app's database. It only failed safe because
* the credentials happened not to match — had they matched, this would have
* wiped pantry-app.
* `.env.test` used to name the Postgres container by bridge IP, and Docker
* reassigns those on recreation: 172.22.0.47 stopped being `postgres-personal`
* and became `postgres-pantry`, so the suite spent a while pointing its
* TRUNCATE at another app's database. It only failed safe because the
* credentials happened not to match — had they matched, this would have wiped
* pantry-app.
*
* `scripts/resolve-test-db.sh` (wired as `pretest:integration`) now re-derives
* the DSN from the live container before every run, preferring the published
* loopback port. This check stays anyway: running vitest directly bypasses the
* pretest hook, and a stale `.env.test` would still be read.
*
* Checked once per process, before the first truncate.
*/
@@ -47,8 +71,9 @@ async function assertTestDatabase(pool: Pool) {
if (db !== "personal_test") {
throw new Error(
`Refusing to truncate: connected to "${db}", expected "personal_test". ` +
`Check DATABASE_URL in .env.test — the Postgres container IP may have changed ` +
`(docker inspect postgres-personal --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}').`
`Re-resolve the DSN with \`bash scripts/resolve-test-db.sh\` (or just run ` +
`\`npm run test:integration\`, which does it for you), and run ` +
`\`npm run test:setup\` if personal_test does not exist yet.`
);
}
targetVerified = true;
src/__tests__/integration/intake.test.ts
+793
View File
@@ -0,0 +1,793 @@
import { describe, it, expect, beforeAll, beforeEach, afterAll, vi } from "vitest";
import { Pool } from "pg";
import { readFileSync, mkdtempSync, rmSync } from "fs";
import { tmpdir } from "os";
import { join, resolve } from "path";
import type { NextRequest } from "next/server";
import { createPool, mockDbWithPool, resetDB, seedParticipants } from "./helpers";
import type { ExtractedOrder } from "../../lib/intake-rules";
/**
* The intake router against the real schema (plan §11 C1, C2, C5).
*
* The model is stubbed at the HTTP boundary — `fetch` to LiteLLM answers with a
* canned extraction, `fetch` to pantry with a canned capture — so the PDF text
* path, the persistence, the match and the writes all run for real. The two
* PDFs are the user's own Uber Eats exports; the transactions seeded here are
* shaped like production rows 6846 and 6857 (platform `ubereats`, `line_items`
* empty, the email pipeline's booking of the same two orders).
*/
const INTAKE = "test-intake-token";
const RECEIPT = "test-receipt-token";
const fixtures = resolve(__dirname, "../fixtures/intake");
const BURGERTORY = readFileSync(resolve(fixtures, "ubereats-burgertory-2026-08-19.pdf"));
const COLES = readFileSync(resolve(fixtures, "ubereats-coles-2026-08-18.pdf"));
const burgertory: ExtractedOrder = {
merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD",
items: [
{ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] },
{ quantity: 1, description: "Joel", unit_price: 14, line_total: 14, options: [] },
],
adjustments: [{ label: "Delivery Fee", amount: 5.49 }, { label: "Promotion", amount: -8 }],
};
const coles: ExtractedOrder = {
merchant: "Coles (Wyndham Vale)", order_date: "2026-08-18", total: 85.74, subtotal: 118.94, currency: "AUD",
items: [
{ quantity: 3, description: "Coca-Cola Zero Sugar Soft Drink (2L)", unit_price: 4.4, line_total: 13.2, options: [] },
{ quantity: 1, description: "Coles RSPCA Approved Chicken Thigh Cutlets approx. 1.1kg", unit_price: 11, line_total: 12.76, options: ["Final 1.16 kg $11.00/kg"] },
{ quantity: 1, description: "everything else", unit_price: 92.98, line_total: 92.98, options: [] },
],
adjustments: [],
};
let pool: Pool;
let ownerId: number;
let otherId: number;
let fileDir: string;
let intakePOST: (req: NextRequest) => Promise<Response>;
let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
let slackPOST: (req: NextRequest) => Promise<Response>;
let orderGET: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
/** What the next LiteLLM call answers, and what the next pantry call answers. */
let modelReply: ExtractedOrder | Error = burgertory;
let pantryReply: { status: number; body: unknown } = { status: 200, body: { captureId: 901, items: [1, 2, 3] } };
const calls: { url: string; body: unknown }[] = [];
function stubFetch() {
vi.stubGlobal("fetch", async (url: string | URL, init?: RequestInit) => {
const u = String(url);
if (u.includes("/v1/chat/completions")) {
calls.push({ url: u, body: JSON.parse(String(init?.body)) });
if (modelReply instanceof Error) return new Response("boom", { status: 500 });
return Response.json({ choices: [{ message: { content: JSON.stringify(modelReply) }, finish_reason: "stop" }] });
}
if (u.includes("/api/capture")) {
const form = init?.body as FormData;
calls.push({ url: u, body: { intake: JSON.parse(String(form.get("intake"))), files: form.getAll("image").map((f) => (f as File).name), headers: init?.headers } });
return Response.json(pantryReply.body, { status: pantryReply.status });
}
if (u.includes("hooks.slack.com")) {
calls.push({ url: u, body: JSON.parse(String(init?.body)) });
return new Response("ok");
}
throw new Error(`unexpected fetch ${u}`);
});
}
const multipart = (fields: Record<string, string>, files: { name: string; type: string; bytes: Buffer }[], token: string | null = INTAKE) => {
const fd = new FormData();
for (const [k, v] of Object.entries(fields)) fd.append(k, v);
for (const f of files) fd.append("file", new File([new Uint8Array(f.bytes)], f.name, { type: f.type }));
return new Request("http://finance.test/api/intake", { method: "POST", headers: token ? { "x-ingest-token": token } : {}, body: fd }) as unknown as NextRequest;
};
const jsonReq = (body: unknown, token: string | null) =>
({ headers: { get: (h: string) => (h === "x-ingest-token" ? token : null) }, json: async () => body }) as unknown as NextRequest;
const put = (id: number | string, body: unknown, token: string | null = RECEIPT) => lineItemsPUT(jsonReq(body, token), { params: Promise.resolve({ id: String(id) }) });
const uuid = () => crypto.randomUUID();
/** A transaction shaped like the email pipeline's booking of an order, line_items empty. */
async function seedOrder(over: { merchant: string; date: string; amount: number; platform?: string | null; owner?: number; withMeta?: boolean }) {
const t = await pool.query(
`INSERT INTO transactions (owner_id, statement_id, transaction_date, description, amount, amount_aud, transaction_type, category, merchant_name, merchant_normalized, payment_method, row_index)
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'dining', $5, $5, 'credits', 0) RETURNING id`,
[over.owner ?? ownerId, over.date, `Order - ${over.merchant}`, over.amount, over.merchant]
);
const id = t.rows[0].id as number;
if (over.withMeta !== false) {
await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, platform, order_reference, line_items, merchant_normalized, amount, transaction_date, currency)
VALUES ($1, 'email', $2, $3, '[]'::jsonb, $4, $5, $6, 'AUD')`,
[id, over.platform === undefined ? "ubereats" : over.platform, uuid(), over.merchant, over.amount, over.date]
);
}
return id;
}
/** What `components/order-details.tsx` would actually render for a transaction. */
const orderPanel = async (txnId: number) =>
(await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null;
const metaRows = async () => (await pool.query(`SELECT id, platform, line_items FROM expense_metadata ORDER BY id`)).rows as { id: number; platform: string | null; line_items: { description: string }[] }[];
const lineItemsOf = async (txnId: number) =>
(await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items;
const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0];
beforeAll(async () => {
process.env.INTAKE_TOKEN = INTAKE;
process.env.RECEIPT_INGEST_TOKEN = RECEIPT;
process.env.ORDER_INGEST_TOKEN = "test-order-token";
process.env.LITELLM_BASE_URL = "http://litellm.test";
process.env.LITELLM_KEY = "k";
process.env.PANTRY_BASE_URL = "http://pantry.test";
process.env.FINANCE_PUBLIC_URL = "https://finance.test";
fileDir = mkdtempSync(join(tmpdir(), "intake-test-"));
process.env.INTAKE_FILE_DIR = fileDir;
pool = createPool();
mockDbWithPool(pool);
// The Slack signature gate is exercised by slack-verify.test.ts; here it is
// bypassed so the intake branch of the route is what is under test.
vi.doMock("@/lib/slack-verify", () => ({
verifySlackSignature: () => true,
participantForSlackUser: () => 1,
slackUserForParticipant: () => null,
}));
// The order-detail panel is the surface an enrichment has to appear on, so
// the read is exercised beside the write rather than assumed.
vi.doMock("@/lib/auth", () => ({ getCurrentUser: async () => ({ id: 1, name: "Owner", email: "o@x" }) }));
vi.doMock("@/lib/queries", async (importOriginal) => ({
...(await importOriginal<Record<string, unknown>>()),
canAccessTransactions: async () => true,
}));
({ POST: intakePOST } = await import("../../app/api/intake/route"));
({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route"));
({ POST: slackPOST } = await import("../../app/api/slack/interactive/route"));
({ GET: orderGET } = await import("../../app/api/transactions/[id]/order/route"));
stubFetch();
});
afterAll(() => {
vi.unstubAllGlobals();
rmSync(fileDir, { recursive: true, force: true });
});
beforeEach(async () => {
await resetDB(pool);
await pool.query(`TRUNCATE intake_batches, intake_merchant_kinds`);
({ ownerId, otherId } = await seedParticipants(pool, ["Alice", "Bob"]));
await pool.query(`UPDATE participants SET email = 'bob@example.test' WHERE id = $1`, [otherId]);
modelReply = burgertory;
pantryReply = { status: 200, body: { captureId: 901, items: [1, 2, 3] } };
calls.length = 0;
});
// ─── C2 ───────────────────────────────────────────────────────────────────────
describe("PUT /api/transactions/[id]/line-items (C2)", () => {
const items = [{ qty: 1, description: "Michael", amount: 14, options: ["Milk Bun"] }, { qty: 1, description: "Joel", amount: 14 }];
it("fails closed: no token, wrong token", async () => {
expect((await put(1, { line_items: items, source: "intake", replace: true }, null)).status).toBe(401);
expect((await put(1, { line_items: items, source: "intake", replace: true }, "nope")).status).toBe(401);
});
it("accepts either token", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
expect((await put(id, { line_items: items, source: "pantry", replace: true }, RECEIPT)).status).toBe(200);
expect((await put(id, { line_items: items, source: "intake", replace: true }, INTAKE)).status).toBe(200);
});
it("404 when the transaction does not exist — it never creates one", async () => {
const res = await put(999999, { line_items: items, source: "intake", replace: true });
expect(res.status).toBe(404);
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
});
it("rejects a body that is not a replace, a bad source, or nameless items", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
expect((await put(id, { line_items: items, source: "intake" })).status).toBe(400);
expect((await put(id, { line_items: items, source: "n8n", replace: true })).status).toBe(400);
expect((await put(id, { line_items: [{ qty: 1 }], source: "intake", replace: true })).status).toBe(400);
});
it("idempotent replace: returns previous + new, and a second identical call is a no-op", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = await (await put(id, { line_items: items, source: "intake", replace: true })).json();
expect(first.previous_line_items).toEqual([]);
expect(first.line_items).toHaveLength(2);
expect(first.created_metadata).toBe(false);
expect(await lineItemsOf(id)).toHaveLength(2);
const corrected = [{ qty: 1, description: "Michael (corrected)", amount: 14 }];
const second = await (await put(id, { line_items: corrected, source: "pantry", replace: true })).json();
expect(second.previous_line_items.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(await lineItemsOf(id)).toEqual(corrected);
const third = await (await put(id, { line_items: corrected, source: "pantry", replace: true })).json();
expect(third.previous_line_items).toEqual(corrected);
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata`)).rows[0].n).toBe(1);
});
it("undo shape: PUT previous_line_items back restores the row", async () => {
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const r = await (await put(id, { line_items: items, source: "pantry", replace: true })).json();
await put(id, { line_items: r.previous_line_items, source: "pantry", replace: true });
expect(await lineItemsOf(id)).toEqual([]);
});
it("creates the expense_metadata row when the transaction has none, with source and platform", async () => {
const id = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const res = await (await put(id, { line_items: items, source: "intake", platform: "ubereats", batch_id: "b1", replace: true })).json();
expect(res.created_metadata).toBe(true);
const em = (await pool.query(`SELECT source, platform, order_reference, merchant_normalized, amount::text FROM expense_metadata WHERE transaction_id = $1`, [id])).rows[0];
expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" });
});
it("the undo baseline is recorded ONCE per batch: a retry gets the original, not its own write", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const original = [{ qty: 1, description: "what the order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(original)]);
const id = uuid();
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
const first = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(first.previous_line_items).toEqual(original);
// Pantry re-calls on every confirm and after a timeout. Before, the retry
// was handed its OWN write back, so undo restored the thing it undid.
const retry = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
expect(retry.previous_line_items).toEqual(original);
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual(original);
await put(txn, { line_items: retry.previous_line_items, source: "pantry", batch_id: id, replace: true });
expect(await lineItemsOf(txn)).toEqual(original);
});
it("without a batch id the baseline is still the row as read — nothing else changes", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(first.previous_line_items).toEqual([]);
const second = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
expect(second.previous_line_items).toEqual(items);
});
it("a pinned row that no longer exists falls back to the live one, rather than reporting a phantom write", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const live = (await pool.query(`SELECT id FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].id as number;
const { replaceLineItems } = await import("../../lib/line-items");
const r = await replaceLineItems(txn, items, { source: "intake", metadataId: live + 999 });
expect(r.metadata_id).toBe(live);
expect(await lineItemsOf(txn)).toHaveLength(2);
});
it("two writes racing to create the metadata row both resolve to the same one", async () => {
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const both = await Promise.all([
put(txn, { line_items: items, source: "intake", replace: true }),
put(txn, { line_items: items, source: "intake", replace: true }),
]);
expect(both.map((r) => r.status)).toEqual([200, 200]);
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].n).toBe(1);
});
it("the create path records an undo baseline too, before it writes", async () => {
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
const id = uuid();
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
await put(txn, { line_items: items, source: "intake", batch_id: id, currency: "AUD", replace: true });
// [] rather than NULL: there was nothing there, and "nothing" is the state
// undo has to be able to restore.
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual([]);
});
it("the panel reads the row the write landed on, whichever order the rows were inserted in", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// Inserted first: the delivery receipt pointing at the statement line.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
[t, uuid()]
);
// Inserted second: a row the transaction owns. An unordered LIMIT 1 reads
// the first one, the write takes the second, and the enrichment reports
// success against a row nothing renders.
await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, line_items) VALUES ($1, 'pantry', $2, '[]'::jsonb)`,
[t, uuid()]
);
const written = await (await put(t, { line_items: items, source: "intake", replace: true })).json();
const panel = await orderPanel(t);
expect(panel?.line_items).toHaveLength(2);
expect((await metaRows()).find((r) => r.id === written.metadata_id)!.line_items).toHaveLength(2);
});
it("finds the row through matched_transaction_id — a card-settled order has only that", async () => {
const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
[statementLine, uuid()]
);
const res = await (await put(statementLine, { line_items: items, source: "intake", replace: true })).json();
expect(res.created_metadata).toBe(false);
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata`)).rows[0].n).toBe(1);
expect(await lineItemsOf(statementLine)).toHaveLength(2);
});
});
// ─── C1 ───────────────────────────────────────────────────────────────────────
describe("POST /api/intake — gates", () => {
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
it("fails closed without INTAKE_TOKEN match", async () => {
expect((await intakePOST(multipart({ source: "uber_eats" }, [pdf], null))).status).toBe(401);
expect((await intakePOST(multipart({ source: "uber_eats" }, [pdf], "nope"))).status).toBe(401);
});
it("400 on a bad source, no files, too many files, a non-uuid batch_id", async () => {
expect((await intakePOST(multipart({ source: "menulog" }, [pdf]))).status).toBe(400);
expect((await intakePOST(multipart({ source: "uber_eats" }, []))).status).toBe(400);
expect((await intakePOST(multipart({ source: "uber_eats" }, Array(7).fill(pdf)))).status).toBe(400);
expect((await intakePOST(multipart({ source: "uber_eats", batch_id: "not-a-uuid" }, [pdf]))).status).toBe(400);
expect((await pool.query(`SELECT count(*)::int AS n FROM intake_batches`)).rows[0].n).toBe(0);
});
it("declines a file type it cannot read, after persisting the batch", async () => {
const id = uuid();
const res = await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [{ name: "clip.mp4", type: "video/mp4", bytes: Buffer.from("x") }]));
const body = await res.json();
expect(body.status).toBe("unsupported");
expect(body.text).toMatch(/not this/);
expect((await batch(id)).status).toBe("unsupported");
expect(calls).toHaveLength(0);
});
});
describe("POST /api/intake — food (enrich vs candidates)", () => {
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
it("food + exact (platform, merchant, date, total) match → enriches the existing order, one model call, PDF as text", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const id = uuid();
const res = await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]));
const body = await res.json();
expect(res.status).toBe(200);
expect(body).toMatchObject({ kind: "food", status: "enriched", transaction_id: txn, batch_id: id });
expect(body.text).toBe("Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓");
expect(JSON.stringify(body.slack_blocks)).toContain("https://finance.test/transactions?q=Burgertory");
const stored = await lineItemsOf(txn);
expect(stored.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(stored[0].options).toEqual(["Milk Bun"]);
// one joint call, the PDF's text in it (not an image)
expect(calls).toHaveLength(1);
const content = calls[0].body as { messages: { content: { type: string; text?: string }[] }[]; model: string; temperature: number; response_format: { type: string } };
expect(content.model).toBe("cloud-budget");
expect(content.temperature).toBe(0);
expect(content.response_format.type).toBe("json_schema");
expect(content.messages[0].content.some((c) => c.type === "image_url")).toBe(false);
expect(content.messages[0].content.some((c) => c.text?.includes("1 Michael $14.00"))).toBe(true);
const row = await batch(id);
expect(row).toMatchObject({ source: "uber_eats", shared_by: "bob@example.test", kind: "food", status: "enriched", transaction_id: txn });
expect(row.header).toMatchObject({ merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, platform: "ubereats" });
expect(row.resolved_at).not.toBeNull();
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
});
it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match — but it IS offered", async () => {
const doordash = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" });
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("candidates");
// The candidates card is a human choice, not a decision. The platform is one
// of the things that may be wrong, so it must not narrow the list — a
// platform-filtered card showed nothing at all here.
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([doordash]);
});
it("a NULL platform is 'no platform', not a wildcard — an in-store share does not overwrite the delivery order", async () => {
await pool.query(`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ('burgertory werribee', 'food')`);
const delivery = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "ubereats" });
const already = [{ qty: 1, description: "what the delivery order already had", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [delivery, JSON.stringify(already)]);
// `other` implies no platform; the delivery row carries one, so it is not
// the match — it is only a candidate.
const body = await (await intakePOST(multipart({ source: "other" }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([delivery]);
expect(await lineItemsOf(delivery)).toEqual(already);
});
it("a reconciled manual row is neither matched nor offered — EXCLUDE_RECONCILED_SOURCE, as everywhere else", async () => {
const live = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const reconciled = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, reconciled]);
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, near]);
// Two rows at the exact key would have read as an ambiguity; one of them is
// the same spend counted twice.
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: live });
});
it("the write lands on the row the MATCH resolved through, not on whichever row sorts first", async () => {
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
// The statement line owns a receipt row of its own (a pantry confirm) …
const grocery = [{ qty: 1, description: "a grocery shop", amount: 22.4 }];
const own = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, source, order_reference, merchant_normalized, line_items) VALUES ($1, 'pantry', $2, 'Woolworths', $3::jsonb) RETURNING id`,
[t, uuid(), JSON.stringify(grocery)]
)).rows[0].id as number;
// … and is also the card leg of the Uber Eats order being enriched.
const delivery = (await pool.query(
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, merchant_normalized, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, 'Burgertory (Werribee)', '[]'::jsonb) RETURNING id`,
[t, uuid()]
)).rows[0].id as number;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ status: "enriched", transaction_id: t });
const rows = await metaRows();
expect(rows.find((r) => r.id === delivery)!.line_items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
expect(rows.find((r) => r.id === own)!.line_items).toEqual(grocery);
});
it("an upload with no prices never replaces a receipt that already has items", async () => {
const priceless = { ...burgertory, subtotal: null, total: 22.4, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
modelReply = priceless;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "held", status: "held" });
expect(body.text).toMatch(/left the existing items alone/);
expect(await lineItemsOf(txn)).toEqual(already);
// …but an EMPTY receipt is exactly what this slice exists to fill.
const empty = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-20", amount: 22.4 });
modelReply = { ...priceless, order_date: "2026-08-20" };
const second = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(second).toMatchObject({ status: "enriched", transaction_id: empty });
expect((await lineItemsOf(empty)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
});
it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => {
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-10", amount: 22.4 }); // too far
await seedOrder({ merchant: "Mad Mex", date: "2026-08-19", amount: 22.4 }); // wrong merchant
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([near]);
const blocks = JSON.stringify(body.slack_blocks);
expect(blocks).toContain('"action_id":"intake_use_candidate"');
expect(blocks).toContain('"action_id":"intake_create"');
expect(blocks).toContain(JSON.stringify(JSON.stringify({ batch_id: id, transaction_id: near })));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(3);
expect(await lineItemsOf(near)).toEqual([]);
expect((await batch(id)).items).toHaveLength(2);
});
it("two exact hits is an ambiguity, not a coin toss", async () => {
const a = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const b = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("candidates");
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id).sort()).toEqual([a, b].sort());
});
it("missing header (no total) → needs_header, nothing written but the batch row", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
modelReply = { ...burgertory, total: null };
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("needs_header");
expect(body.text).toMatch(/share the top of the screen/);
expect(await lineItemsOf(txn)).toEqual([]);
expect((await batch(id)).status).toBe("needs_header");
});
it("items that do not add up → held, nothing written", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
modelReply = { ...burgertory, items: [burgertory.items[0]] };
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "held", status: "held" });
expect(body.text).toMatch(/is a shot missing/);
expect(await lineItemsOf(txn)).toEqual([]);
});
it("a model failure is a 502 with the batch marked error, not a silent 200", async () => {
modelReply = new Error("down");
const res = await intakePOST(multipart({ source: "uber_eats" }, [pdf]));
expect(res.status).toBe(502);
expect((await res.json()).status).toBe("error");
});
it("re-share of the same file after it was handled → duplicate with a pointer", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: first }, [pdf]));
calls.length = 0;
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
expect(body.status).toBe("duplicate");
expect(body.duplicate_of).toEqual({ batch_id: first, transaction_id: txn, capture_id: null });
expect(body.text).toMatch(/Already handled/);
expect(calls).toHaveLength(0); // no model call spent on a known file
});
it("re-share of the same order under a different file (screenshot after PDF) → duplicate by header", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const first = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: first }, [pdf]));
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [{ name: "IMG_1.png", type: "image/png", bytes: Buffer.from("fake-png") }]))).json();
expect(body.status).toBe("duplicate");
expect(body.duplicate_of.transaction_id).toBe(txn);
});
it("a repeated batch_id is refused, not re-run", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
expect((await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).status).toBe(409);
});
});
describe("POST /api/intake — grocery → pantry", () => {
const pdf = { name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES };
it("grocery + match → the ORIGINAL file goes to pantry with the target transaction and action enrich; finance is NOT touched", async () => {
modelReply = coles;
const txn = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74 });
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]))).json();
expect(body).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
expect(body.text).toBe("Coles (Wyndham Vale) $85.74 → pantry board (3 items) — finance updates on confirm");
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown>; files: string[]; headers: Record<string, string> };
expect(pantryCall.headers["x-ingest-token"]).toBe(RECEIPT);
expect(pantryCall.files).toEqual(["Receipt_18Aug2026.pdf"]);
expect(pantryCall.intake).toEqual({ batch_id: id, platform: "ubereats", origin: "intake", shared_by: "bob@example.test", finance_target_transaction_id: txn, finance_action: "enrich" });
expect(await lineItemsOf(txn)).toEqual([]);
expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
});
it("an in-store tap does not hand pantry the delivery order as its finance target", async () => {
modelReply = coles;
// The Uber Eats Coles order the email pipeline already booked.
const delivery = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: "ubereats" });
// The same shop, same day, same money — but shared from the Coles app, so
// it carries no platform and is a different purchase.
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown> };
expect(pantryCall.intake).toMatchObject({ finance_action: "create", finance_target_transaction_id: null, platform: null });
expect(await lineItemsOf(delivery)).toEqual([]);
});
it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => {
modelReply = coles;
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body.status).toBe("forwarded_to_pantry");
expect(body.text).toMatch(/finance books it on confirm/);
const pantryCall = calls.find((c) => c.url.includes("/api/capture"))!.body as { intake: Record<string, unknown> };
expect(pantryCall.intake).toMatchObject({ platform: null, finance_target_transaction_id: null, finance_action: "create" });
});
it("pantry 409 → duplicate", async () => {
modelReply = coles;
pantryReply = { status: 409, body: { error: "This receipt has already been added to the pantry.", captureId: 77 } };
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
expect(body).toMatchObject({ kind: "grocery", status: "duplicate", capture_id: 77 });
});
it("pantry down → 502 error, batch marked error", async () => {
modelReply = coles;
pantryReply = { status: 500, body: { error: "nope" } };
const res = await intakePOST(multipart({ source: "coles" }, [pdf]));
expect(res.status).toBe(502);
expect((await res.json()).status).toBe("error");
});
});
// ─── C5 ───────────────────────────────────────────────────────────────────────
const slackPress = (action_id: string, value: unknown) => {
const payload = { type: "block_actions", user: { id: "U1", name: "alice" }, response_url: "https://hooks.slack.com/r/1", actions: [{ action_id, value: JSON.stringify(value) }] };
const raw = new URLSearchParams({ payload: JSON.stringify(payload) }).toString();
return { headers: { get: (h: string) => (h === "x-ingest-token" ? "test-order-token" : "x") }, text: async () => raw } as unknown as NextRequest;
};
const lastSlackUpdate = () => calls.filter((c) => c.url.includes("hooks.slack.com")).at(-1)?.body as { replace_original?: boolean; text: string; blocks?: unknown[] };
describe("Slack taps (C5)", () => {
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
it("intake_use_candidate enriches the chosen transaction and replaces the card", async () => {
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
const res = await slackPOST(slackPress("intake_use_candidate", { batch_id: id, transaction_id: near }));
expect(res.status).toBe(200);
expect((await lineItemsOf(near)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: near });
expect(lastSlackUpdate()).toMatchObject({ replace_original: true, text: "Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓" });
});
it("intake_create creates the transaction ING-10-style: manual, owner from shared_by, items attached; a second press is a no-op", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
await slackPOST(slackPress("intake_create", { batch_id: id }));
const t = (await pool.query(`SELECT statement_id, owner_id, description, amount::text, category, transaction_date::text, source, source_ref FROM transactions`)).rows;
expect(t).toEqual([{ statement_id: null, owner_id: otherId, description: "Order - Burgertory (Werribee)", amount: "22.40", category: "dining", transaction_date: "2026-08-19", source: "intake", source_ref: id }]);
const em = (await pool.query(`SELECT source, platform, line_items FROM expense_metadata`)).rows[0];
expect(em.source).toBe("intake");
expect(em.platform).toBe("ubereats");
expect(em.line_items.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(lastSlackUpdate().text).toMatch(/^Created a \$22\.40 Burgertory \(Werribee\) transaction with 2 items/);
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
});
it("intake_create re-runs the match: an order booked between the card and the tap is enriched, not duplicated", async () => {
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
// The email/order bridge books the same purchase while the card sits in
// Slack. The card's "no match" is minutes stale by the time it is pressed.
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
expect((await lineItemsOf(txn)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
});
it("two pending cards for one order book it ONCE", async () => {
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
expect((await batch(a)).status).toBe("candidates");
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: a }));
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect((await batch(b)).transaction_id).toBe((await batch(a)).transaction_id);
});
it("a sibling already resolved to the pantry board makes Create a duplicate, not a second row", async () => {
// Two shares of one receipt, resolved out of order: nothing the second card
// can match on, because the first one never produced a transaction.
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "other", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
await intakePOST(multipart({ source: "other", batch_id: a }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
expect((await batch(a)).status).toBe("ask");
await slackPOST(slackPress("intake_grocery", { batch_id: a }));
expect((await batch(a)).status).toBe("forwarded_to_pantry");
await slackPOST(slackPress("intake_meal", { batch_id: b }));
expect((await batch(b)).status).toBe("candidates");
await slackPOST(slackPress("intake_create", { batch_id: b }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
expect(await batch(b)).toMatchObject({ status: "duplicate", capture_id: 901 });
expect(lastSlackUpdate().text).toMatch(/^Already handled/);
});
it("two taps that land together still book one transaction, and neither reports an error", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: id })), slackPOST(slackPress("intake_create", { batch_id: id }))]);
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
const texts = calls.filter((c) => c.url.includes("hooks.slack.com")).map((c) => (c.body as { text: string }).text);
expect(texts.every((t) => !/Couldn't process/.test(t))).toBe(true);
});
it("a foreign receipt is booked in its own currency, with no invented AUD figure", async () => {
modelReply = { ...burgertory, currency: "USD" };
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
const t = (await pool.query(`SELECT amount::text, amount_aud, foreign_currency_amount::text, foreign_currency_code FROM transactions`)).rows[0];
expect(t).toEqual({ amount: "22.40", amount_aud: null, foreign_currency_amount: "22.40", foreign_currency_code: "USD" });
expect(lastSlackUpdate().text).toMatch(/USD 22\.40/);
// OrderDetails prefers expense_metadata.currency, so a hard-coded 'AUD'
// here printed "$" on a USD order's receipt panel.
expect((await pool.query(`SELECT currency FROM expense_metadata`)).rows[0].currency).toBe("USD");
});
it("the hold applies to the Create tap's re-match, not only to the upload path", async () => {
modelReply = { ...burgertory, subtotal: null, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
const id = uuid();
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
expect(body.status).toBe("candidates");
// Booked WITH its items between the card and the tap. The re-match walked
// past the guard routeFood applies, so the same upload destroyed the same
// receipt depending on which button reached it.
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect(await lineItemsOf(txn)).toEqual(already);
expect(await batch(id)).toMatchObject({ status: "held" });
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
});
it("two DIFFERENT batches for one purchase, tapped together, book it once", async () => {
const a = uuid();
const b = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
// Their source_refs differ, so ON CONFLICT cannot save this; both taps
// clear their match and duplicate checks before either inserts.
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: a })), slackPOST(slackPress("intake_create", { batch_id: b }))]);
const rows = (await pool.query(`SELECT id FROM transactions`)).rows;
expect(rows).toHaveLength(1);
expect((await batch(a)).transaction_id).toBe(rows[0].id);
expect((await batch(b)).transaction_id).toBe(rows[0].id);
});
it("unknown shared_by falls back to the default owner — never a NULL owner", async () => {
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT owner_id FROM transactions`)).rows[0].owner_id).toBe(1);
});
it("`other` + unrecognised merchant asks; intake_meal routes it as food and remembers the merchant", async () => {
modelReply = { ...burgertory, merchant: "Romeo's Food Hall" };
const txn = await seedOrder({ merchant: "Romeo's Food Hall", date: "2026-08-19", amount: 22.4, platform: null });
const id = uuid();
const body = await (await intakePOST(multipart({ source: "other", batch_id: id }, [pdf]))).json();
expect(body).toMatchObject({ kind: "ask", status: "ask" });
expect(JSON.stringify(body.slack_blocks)).toContain("intake_grocery");
await slackPOST(slackPress("intake_meal", { batch_id: id }));
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
expect(await lineItemsOf(txn)).toHaveLength(2);
expect((await pool.query(`SELECT kind FROM intake_merchant_kinds WHERE merchant_key = 'romeo s food hall'`)).rows[0].kind).toBe("food");
// The answer sticks: the next share of that merchant does not ask.
const again = await (await intakePOST(multipart({ source: "other" }, [{ name: "IMG.png", type: "image/png", bytes: Buffer.from("png2") }]))).json();
expect(again.status).toBe("duplicate"); // same order — but it was routed, not asked
modelReply = { ...burgertory, merchant: "Romeo's Food Hall", order_date: "2026-08-20" };
const next = await (await intakePOST(multipart({ source: "other" }, [{ name: "IMG3.png", type: "image/png", bytes: Buffer.from("png3") }]))).json();
expect(next).toMatchObject({ kind: "food", status: "candidates" });
});
it("intake_grocery forwards the stored original files to pantry", async () => {
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
const id = uuid();
await intakePOST(multipart({ source: "other", batch_id: id }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
expect((await batch(id)).status).toBe("ask");
await slackPOST(slackPress("intake_grocery", { batch_id: id }));
const pantryCall = calls.find((c) => c.url.includes("/api/capture"))!.body as { intake: Record<string, unknown>; files: string[] };
expect(pantryCall.files).toEqual(["Receipt_18Aug2026.pdf"]);
expect(pantryCall.intake).toMatchObject({ batch_id: id, finance_action: "create" });
expect(await batch(id)).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901 });
expect((await pool.query(`SELECT kind FROM intake_merchant_kinds`)).rows[0].kind).toBe("grocery");
});
it("a tap on a resolved batch returns its card and writes nothing; an unknown batch says so", async () => {
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
const id = uuid();
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
await slackPOST(slackPress("intake_create", { batch_id: id }));
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
await slackPOST(slackPress("intake_create", { batch_id: uuid() }));
expect(lastSlackUpdate()).toMatchObject({ replace_original: false, text: "I can't find that upload any more." });
});
});
src/__tests__/integration/order-ingest-api.test.ts
+12 -2
View File
@@ -1,6 +1,7 @@
import { describe, it, expect, beforeAll } from "vitest";
import { readFileSync } from "fs";
import { resolve } from "path";
import type { NextRequest } from "next/server";
import { queryRaw } from "../../lib/db";
/**
@@ -13,13 +14,22 @@ const dir = resolve(__dirname, "../fixtures/orders/real");
const html = (f: string) => readFileSync(resolve(dir, `${f}.html`), "utf-8");
const TOKEN = "test-ingest-token";
let POST: any;
// The handler is imported dynamically (after the env var is set), so its type
// is declared here rather than inferred. `unknown` for the request is
// deliberate: these fixtures are the two properties the route actually reads,
// not a whole NextRequest, and widening to `any` would stop the compiler
// checking the call sites too.
let POST: (req: NextRequest) => Promise<Response>;
// One cast, at the boundary, instead of `any` everywhere downstream: the
// fixture is the two properties the route actually reads, and everything past
// this line stays type-checked.
const req = (body: unknown, token: string | null = TOKEN) =>
({
headers: { get: (h: string) => (h === "x-ingest-token" ? token : null) },
json: async () => body,
}) as any;
}) as unknown as NextRequest;
beforeAll(async () => {
process.env.ORDER_INGEST_TOKEN = TOKEN;
src/__tests__/integration/order-reviews.test.ts
+62 -35
View File
@@ -1,4 +1,5 @@
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
import type { NextRequest } from "next/server";
import { createPool, mockDbWithPool, resetDB } from "./helpers";
/**
@@ -12,15 +13,41 @@ import { createPool, mockDbWithPool, resetDB } from "./helpers";
const pool = createPool();
mockDbWithPool(pool);
let GET: any;
let PUT: any;
/** What the review route returns, to the extent these tests read it. */
type ItemVerdict = { item: string; verdict: string };
type Review = {
participant_id: number;
rating: string | null;
note: string | null;
order_again: boolean | null;
item_verdicts: ItemVerdict[];
};
type ReviewBody = {
reviews: Review[];
splits: { participant_id: number; share_percent: number }[];
merchant: {
warn: boolean;
note: string | null;
counts: Record<string, number>;
history: { transaction_id: number }[];
items: { item: string; loved: number; never: number }[];
} | null;
};
type Ctx = { params: Promise<{ id: string }> };
// Declared rather than inferred because the route is imported dynamically,
// after vi.doMock. `unknown` for the request keeps the fixture honest — it is
// the two properties the handler reads, not a NextRequest.
let GET: (req: NextRequest, ctx: Ctx) => Promise<Response>;
let PUT: (req: NextRequest, ctx: Ctx) => Promise<Response>;
let ownerId: number;
let otherId: number;
let txnId: number;
const req = (body?: unknown) =>
({ headers: { get: () => null }, json: async () => body }) as any;
const params = (id: number) => ({ params: Promise.resolve({ id: String(id) }) });
({ headers: { get: () => null }, json: async () => body }) as unknown as NextRequest;
const params = (id: number): Ctx => ({ params: Promise.resolve({ id: String(id) }) });
const bodyOf = async (res: Response): Promise<ReviewBody> => (await res.json()) as ReviewBody;
beforeAll(async () => {
vi.doMock("@/lib/auth", () => ({
@@ -59,25 +86,25 @@ describe("order verdicts — one per person", () => {
it("keeps both people's verdicts on the same order", async () => {
await PUT(req({ participant_id: ownerId, rating: "loved" }), params(txnId));
const res = await PUT(req({ participant_id: otherId, rating: "never" }), params(txnId));
const body = await res.json();
const body = await bodyOf(res);
// The bug this guards: a UNIQUE on transaction_id alone made the second
// save overwrite the first, and the disagreement is the useful part.
expect(body.reviews).toHaveLength(2);
expect(body.reviews.find((r: any) => r.participant_id === ownerId).rating).toBe("loved");
expect(body.reviews.find((r: any) => r.participant_id === otherId).rating).toBe("never");
expect(body.reviews.find((r) => r.participant_id === ownerId)?.rating).toBe("loved");
expect(body.reviews.find((r) => r.participant_id === otherId)?.rating).toBe("never");
});
it("defaults the verdict to the signed-in user, not the owner", async () => {
const res = await PUT(req({ rating: "ok" }), params(txnId));
const body = await res.json();
const body = await bodyOf(res);
expect(body.reviews[0].participant_id).toBe(ownerId);
});
it("revising a verdict updates rather than duplicating", async () => {
await PUT(req({ participant_id: ownerId, rating: "loved" }), params(txnId));
const res = await PUT(req({ participant_id: ownerId, rating: "never" }), params(txnId));
const body = await res.json();
const body = await bodyOf(res);
expect(body.reviews).toHaveLength(1);
expect(body.reviews[0].rating).toBe("never");
});
@@ -88,12 +115,12 @@ describe("order verdicts — one per person", () => {
});
it("derives order_again from the rating", async () => {
let body = await (await PUT(req({ rating: "never" }), params(txnId))).json();
let body = await bodyOf(await PUT(req({ rating: "never" }), params(txnId)));
expect(body.reviews[0].order_again).toBe(false);
body = await (await PUT(req({ rating: "ok" }), params(txnId))).json();
body = await bodyOf(await PUT(req({ rating: "ok" }), params(txnId)));
expect(body.reviews[0].order_again).toBe(true);
// "bad" is also a no — you would not choose it again.
body = await (await PUT(req({ rating: "bad" }), params(txnId))).json();
body = await bodyOf(await PUT(req({ rating: "bad" }), params(txnId)));
expect(body.reviews[0].order_again).toBe(false);
});
@@ -104,13 +131,13 @@ describe("order verdicts — one per person", () => {
const older = await seedOrder("Thai Palace", "2026-06-01");
await PUT(req({ participant_id: ownerId, rating: "bad" }), params(older));
let body = await (await GET(req(), params(txnId))).json();
expect(body.merchant.counts.bad).toBe(1);
expect(body.merchant.warn).toBe(false);
let body = await bodyOf(await GET(req(), params(txnId)));
expect(body.merchant!.counts.bad).toBe(1);
expect(body.merchant!.warn).toBe(false);
await PUT(req({ participant_id: ownerId, rating: "never" }), params(older));
body = await (await GET(req(), params(txnId))).json();
expect(body.merchant.warn).toBe(true);
body = await bodyOf(await GET(req(), params(txnId)));
expect(body.merchant!.warn).toBe(true);
});
});
@@ -125,7 +152,7 @@ describe("item verdicts", () => {
);
// No item_verdicts key at all — the shape a note-only form sends.
const res = await PUT(req({ rating: "liked", note: "slow delivery" }), params(txnId));
const body = await res.json();
const body = await bodyOf(res);
expect(body.reviews[0].item_verdicts).toEqual([
{ item: "Pad Thai", verdict: "loved" },
]);
@@ -138,7 +165,7 @@ describe("item verdicts", () => {
params(txnId)
);
const res = await PUT(req({ rating: "liked", item_verdicts: [] }), params(txnId));
const body = await res.json();
const body = await bodyOf(res);
expect(body.reviews[0].item_verdicts).toEqual([]);
});
@@ -154,7 +181,7 @@ describe("item verdicts", () => {
}),
params(txnId)
);
const body = await res.json();
const body = await bodyOf(res);
expect(body.reviews[0].rating).toBe("ok");
expect(body.reviews[0].item_verdicts).toEqual([
{ item: "Pad Thai", verdict: "loved" },
@@ -167,10 +194,10 @@ describe("merchant history", () => {
const older = await seedOrder("Thai Palace", "2026-06-01");
await PUT(req({ participant_id: ownerId, rating: "never", note: "cold" }), params(older));
const body = await (await GET(req(), params(txnId))).json();
expect(body.merchant.warn).toBe(true);
expect(body.merchant.counts.never).toBe(1);
expect(body.merchant.history.map((h: any) => h.transaction_id)).toEqual([older]);
const body = await bodyOf(await GET(req(), params(txnId)));
expect(body.merchant!.warn).toBe(true);
expect(body.merchant!.counts.never).toBe(1);
expect(body.merchant!.history.map((h) => h.transaction_id)).toEqual([older]);
});
it("treats the same restaurant as one merchant across platforms", async () => {
@@ -181,17 +208,17 @@ describe("merchant history", () => {
const shouty = await seedOrder("THAI PALACE", "2026-06-01");
await PUT(req({ participant_id: ownerId, rating: "never" }), params(shouty));
const body = await (await GET(req(), params(txnId))).json();
expect(body.merchant.warn).toBe(true);
expect(body.merchant.history.map((h: any) => h.transaction_id)).toContain(shouty);
const body = await bodyOf(await GET(req(), params(txnId)));
expect(body.merchant!.warn).toBe(true);
expect(body.merchant!.history.map((h) => h.transaction_id)).toContain(shouty);
});
it("does not carry a verdict across different merchants", async () => {
const other = await seedOrder("Pizza Place", "2026-06-01");
await PUT(req({ participant_id: ownerId, rating: "never" }), params(other));
const body = await (await GET(req(), params(txnId))).json();
expect(body.merchant.warn).toBe(false);
const body = await bodyOf(await GET(req(), params(txnId)));
expect(body.merchant!.warn).toBe(false);
});
it("pools item opinions across the merchant's orders", async () => {
@@ -206,10 +233,10 @@ describe("merchant history", () => {
params(older2)
);
const body = await (await GET(req(), params(txnId))).json();
const body = await bodyOf(await GET(req(), params(txnId)));
// Case-folded: the same dish comes back capitalised differently between
// receipts, and two entries for one dish is not a track record.
expect(body.merchant.items).toEqual([{ item: "Pad Thai", loved: 2, never: 0 }]);
expect(body.merchant!.items).toEqual([{ item: "Pad Thai", loved: 2, never: 0 }]);
});
it("keeps item opinions from reviews that have no overall rating", async () => {
@@ -218,14 +245,14 @@ describe("merchant history", () => {
req({ item_verdicts: [{ item: "Satay", verdict: "never" }] }),
params(older)
);
const body = await (await GET(req(), params(txnId))).json();
expect(body.merchant.items).toEqual([{ item: "Satay", loved: 0, never: 1 }]);
const body = await bodyOf(await GET(req(), params(txnId)));
expect(body.merchant!.items).toEqual([{ item: "Satay", loved: 0, never: 1 }]);
});
});
describe("share state", () => {
it("reports splits so the panel can show shared vs just me", async () => {
let body = await (await GET(req(), params(txnId))).json();
let body = await bodyOf(await GET(req(), params(txnId)));
expect(body.splits).toEqual([]);
await pool.query(
@@ -233,7 +260,7 @@ describe("share state", () => {
VALUES ($1, $2, 50)`,
[txnId, otherId]
);
body = await (await GET(req(), params(txnId))).json();
body = await bodyOf(await GET(req(), params(txnId)));
expect(body.splits).toHaveLength(1);
expect(body.splits[0].participant_id).toBe(otherId);
});
src/__tests__/unit/intake-live.test.ts
+37
View File
@@ -0,0 +1,37 @@
import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { resolve } from "path";
import { extractPdfText, liteLLMExtractor } from "../../lib/intake-extract";
import { reconcileItems, toLineItems, validateHeader } from "../../lib/intake-rules";
/**
* Opt-in: the real model over the real Burgertory export.
*
* INTAKE_LIVE_TEST=1 LITELLM_BASE_URL=http://localhost:4200 LITELLM_KEY=... npm test
*
* Skipped otherwise — CI has no LiteLLM, and a unit run must not depend on one.
* What it proves is the thing the mock cannot: that `cloud-budget` keeps the
* names verbatim ("Michael", "Joel", not "Burger") under the order-item prompt.
*/
const live = process.env.INTAKE_LIVE_TEST === "1" && !!process.env.LITELLM_BASE_URL && !!process.env.LITELLM_KEY;
describe.skipIf(!live)("live extraction via LiteLLM (opt-in)", () => {
it("Burgertory PDF: header exact, two items named verbatim, reconciles", async () => {
const text = await extractPdfText(readFileSync(resolve(__dirname, "../fixtures/intake/ubereats-burgertory-2026-08-19.pdf")));
const extracted = await liteLLMExtractor()([{ kind: "text", text, name: "Receipt_19Aug2026.pdf" }]);
const header = validateHeader(extracted);
expect(header.ok && header.header).toEqual({ merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4 });
const items = toLineItems(extracted.items);
expect(items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
expect(reconcileItems(extracted).ok).toBe(true);
}, 120_000);
it("Coles PDF: header exact, 15 items", async () => {
const text = await extractPdfText(readFileSync(resolve(__dirname, "../fixtures/intake/ubereats-coles-2026-08-18.pdf")));
const extracted = await liteLLMExtractor()([{ kind: "text", text, name: "Receipt_18Aug2026.pdf" }]);
const header = validateHeader(extracted);
expect(header.ok && header.header).toEqual({ merchant: "Coles (Wyndham Vale)", order_date: "2026-08-18", total: 85.74 });
expect(toLineItems(extracted.items)).toHaveLength(15);
expect(reconcileItems(extracted).ok).toBe(true);
}, 120_000);
});
src/__tests__/unit/intake-rules.test.ts
+300
View File
@@ -0,0 +1,300 @@
import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { resolve } from "path";
import {
classifyInput,
GROCERY_PATTERN,
merchantKey,
merchantMatches,
platformForSource,
reconcileItems,
resolveMerchantKind,
toLineItems,
validateHeader,
type ExtractedOrder,
} from "../../lib/intake-rules";
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor } from "../../lib/intake-extract";
import { askCard, candidatesCard, enrichedCard, type Block } from "../../lib/intake-blocks";
/**
* The intake router's pure rules, against the strings real receipts print.
* The two PDFs are the user's own Uber Eats exports (plan §3): a Coles grocery
* run and a Burgertory meal whose burgers are named after people.
*/
const fixtures = resolve(__dirname, "../fixtures/intake");
const pdf = (name: string) => readFileSync(resolve(fixtures, name));
describe("classifyInput — input type picks the extractor", () => {
it("PDF by MIME or by name", () => {
expect(classifyInput({ type: "application/pdf", name: "x" })).toBe("pdf");
expect(classifyInput({ type: "application/octet-stream", name: "Receipt_18Aug2026.pdf" })).toBe("pdf");
});
it("images and text", () => {
expect(classifyInput({ type: "image/png", name: "IMG_1.png" })).toBe("image");
expect(classifyInput({ type: "", name: "photo.HEIC" })).toBe("image");
expect(classifyInput({ type: "text/plain", name: "" })).toBe("text");
});
it("declines what it cannot read", () => {
expect(classifyInput({ type: "video/mp4", name: "clip.mp4" })).toBe("unsupported");
expect(classifyInput({ type: "application/zip", name: "a.zip" })).toBe("unsupported");
});
});
describe("platformForSource", () => {
it("maps the tap to expense_metadata.platform vocabulary", () => {
expect(platformForSource("uber_eats")).toBe("ubereats");
expect(platformForSource("doordash")).toBe("doordash");
expect(platformForSource("coles")).toBeNull();
expect(platformForSource("other")).toBeNull();
});
});
describe("merchant pattern — grocery | food | ask", () => {
it("the pattern list is a substring match on the normalised name", () => {
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
expect(resolveMerchantKind("other", "Woolworths Group Limited")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "WOOLIES METRO")).toBe("grocery");
expect(resolveMerchantKind("doordash", "ALDI Werribee")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "IGA Xpress")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "FoodWorks Manor Lakes")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Costco Wholesale")).toBe("grocery");
});
it("a delivery order that is not a grocer is a meal", () => {
expect(resolveMerchantKind("uber_eats", "Burgertory (Werribee)")).toBe("food");
expect(resolveMerchantKind("doordash", "Mad Mex")).toBe("food");
});
it("an empty or unknown merchant asks", () => {
expect(resolveMerchantKind("uber_eats", null)).toBe("ask");
expect(resolveMerchantKind("uber_eats", " ")).toBe("ask");
expect(resolveMerchantKind("doordash", "Unknown")).toBe("ask");
});
it("the Coles/Woolworths taps are ground truth", () => {
expect(resolveMerchantKind("coles", "Romeo's Food Hall")).toBe("grocery");
expect(resolveMerchantKind("woolworths", null)).toBe("grocery");
});
it("`other` with an unrecognised merchant asks; a remembered answer sticks", () => {
expect(resolveMerchantKind("other", "Romeo's Food Hall")).toBe("ask");
expect(resolveMerchantKind("other", "Romeo's Food Hall", "grocery")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Romeo's Food Hall", "grocery")).toBe("grocery");
});
it("merchantKey folds case and punctuation; merchantMatches allows containment only", () => {
expect(merchantKey("Coles (Wyndham Vale)")).toBe("coles wyndham vale");
expect(merchantMatches("Coles (Wyndham Vale)", "coles wyndham vale")).toBe(true);
expect(merchantMatches("Coles (Wyndham Vale)", "Coles")).toBe(true);
expect(merchantMatches("Order - Coles (Wyndham Vale)", "Coles (Wyndham Vale)")).toBe(true);
expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false);
expect(merchantMatches(null, "Coles")).toBe(false);
});
it("whole words, not substrings: `Bigals Pizza` contains `iga` and is neither a grocer nor an IGA", () => {
expect(GROCERY_PATTERN.test(merchantKey("Bigals Pizza"))).toBe(false);
expect(resolveMerchantKind("uber_eats", "Bigals Pizza")).toBe("food");
expect(merchantMatches("Bigals Pizza", "IGA")).toBe(false);
// and the grocers a boundary must not cost us
expect(resolveMerchantKind("uber_eats", "IGA Wyndham Vale")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Woolworths Group Limited")).toBe("grocery");
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
expect(merchantMatches("IGA Wyndham Vale", "IGA")).toBe(true);
});
});
describe("validateHeader — nothing is written from a half-read upload", () => {
const base: ExtractedOrder = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, items: [] };
it("accepts a complete header and rounds the total", () => {
const r = validateHeader({ ...base, total: 22.404 });
expect(r.ok && r.header.total).toBe(22.4);
});
it("missing total", () => {
const r = validateHeader({ ...base, total: null });
expect(r.ok).toBe(false);
expect(!r.ok && r.missing).toEqual(["total"]);
});
it("missing or malformed date", () => {
expect(validateHeader({ ...base, order_date: null }).ok).toBe(false);
expect(validateHeader({ ...base, order_date: "19/8/26" }).ok).toBe(false);
});
it("a missing merchant is NOT a missing header — it becomes an ask", () => {
const r = validateHeader({ ...base, merchant: null });
expect(r.ok && r.header.merchant).toBeNull();
});
});
describe("reconcileItems — the missing-screenshot check", () => {
it("reconciles to the printed item subtotal, not the total: fees and promotions are not items", () => {
// The real Coles export: 15 lines summing $118.87 against a printed $118.94 subtotal and an $85.74 total.
const r = reconcileItems({
merchant: "Coles", order_date: "2026-08-18", total: 85.74, subtotal: 118.94,
items: [{ quantity: 1, description: "a", line_total: 118.87 }],
});
expect(r.ok).toBe(true);
expect(r.basis).toBe("subtotal");
});
it("falls back to the total when no subtotal is printed", () => {
const r = reconcileItems({ merchant: "x", order_date: "2026-08-19", total: 28, items: [{ quantity: 1, description: "Michael", line_total: 14 }, { quantity: 1, description: "Joel", line_total: 14 }] });
expect(r.ok).toBe(true);
expect(r.basis).toBe("total");
});
it("holds when the lines fall short — a shot is missing", () => {
const r = reconcileItems({ merchant: "x", order_date: "2026-08-19", total: 28, subtotal: 28, items: [{ quantity: 1, description: "Michael", line_total: 14 }] });
expect(r.ok).toBe(false);
expect(r.reason).toMatch(/14\.00.*28\.00/);
});
it("tolerance is $1 or 2%, whichever is larger", () => {
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, subtotal: 10, items: [{ quantity: 1, description: "a", line_total: 9.05 }] }).ok).toBe(true);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, subtotal: 10, items: [{ quantity: 1, description: "a", line_total: 8.9 }] }).ok).toBe(false);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 200, subtotal: 200, items: [{ quantity: 1, description: "a", line_total: 196.5 }] }).ok).toBe(true);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 200, subtotal: 200, items: [{ quantity: 1, description: "a", line_total: 195 }] }).ok).toBe(false);
});
it("derives a line from quantity × unit price when no line total is printed", () => {
const r = reconcileItems({ merchant: "x", order_date: "d", total: 13.2, subtotal: 13.2, items: [{ quantity: 3, description: "Coke", unit_price: 4.4 }] });
expect(r.ok).toBe(true);
expect(r.items_sum).toBe(13.2);
});
it("no items at all is held; priceless lines (pasted text) are not", () => {
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, items: [] }).ok).toBe(false);
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, items: [{ quantity: 1, description: "a" }] }).ok).toBe(true);
});
});
describe("toLineItems — names verbatim, options kept", () => {
it("keeps a burger called Michael called Michael", () => {
const out = toLineItems([{ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] }]);
expect(out).toEqual([{ qty: 1, description: "Michael", amount: 14, unit_price: 14, options: ["Milk Bun"] }]);
});
it("drops nameless lines and defaults quantity to 1", () => {
const out = toLineItems([{ quantity: null, description: " ", line_total: 1 }, { quantity: null, description: "Cucumbers Continental", unit_price: 2.9 }]);
expect(out).toEqual([{ qty: 1, description: "Cucumbers Continental", amount: 2.9, unit_price: 2.9 }]);
});
});
describe("extraction request and reply", () => {
it("one call: the prompt first, then every part in the order shared", () => {
const content = buildMessageContent([
{ kind: "image", mime: "image/png", bytes: Buffer.from("a"), name: "1.png" },
{ kind: "text", text: "Total $22.40", name: "r.pdf" },
{ kind: "image", mime: "image/jpeg", bytes: Buffer.from("b"), name: "2.jpg" },
]) as { type: string; text?: string; image_url?: { url: string } }[];
expect(content[0].text).toMatch(/VERBATIM/);
const images = content.filter((c) => c.type === "image_url").map((c) => c.image_url!.url);
expect(images).toEqual(["data:image/png;base64,YQ==", "data:image/jpeg;base64,Yg=="]);
expect(content.findIndex((c) => c.text?.includes("Total $22.40"))).toBeLessThan(content.findIndex((c) => c.image_url?.url.endsWith("Yg==")));
expect(EXTRACT_RESPONSE_FORMAT.json_schema.strict).toBe(true);
});
it("parses a fenced reply and coerces numbers", () => {
const r = parseExtraction('```json\n{"merchant":"Burgertory (Werribee)","order_date":"2026-08-19","total":"22.40","subtotal":28,"currency":null,"items":[{"quantity":1,"description":"Michael","unit_price":14,"line_total":14,"options":["Milk Bun"]}],"adjustments":[{"label":"Promotion","amount":-8}]}\n```');
expect(r.total).toBe(22.4);
expect(r.currency).toBe("AUD");
expect(r.items[0]).toEqual({ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] });
expect(r.adjustments).toEqual([{ label: "Promotion", amount: -8 }]);
});
});
describe("PDF text — the Uber Eats export is born-digital text", () => {
it("Coles (Wyndham Vale) · 2026-08-18 · $85.74, 15 lines, weighed and substituted", async () => {
const text = await extractPdfText(pdf("ubereats-coles-2026-08-18.pdf"));
expect(text).toContain("Here's your receipt for Coles (Wyndham Vale).");
expect(text).toContain("18 Aug, 2026");
expect(text).toContain("Total $85.74");
expect(text).toContain("Item Subtotal $118.94");
expect(text).toContain("Final 1.16 kg $11.00/kg");
expect(text).toContain("Replaced Coles Beef Slow Cook Chuck Roast");
// 15 item lines: 13 priced per piece, 2 weighed per kg.
expect(text.match(/\$\d+\.\d\d\/pc$/gm)?.length).toBe(13);
expect(text.match(/\$\d+\.\d\d\/kg$/gm)?.length).toBe(2);
});
it("Burgertory (Werribee) · 2026-08-19 · $22.40, burgers named Michael and Joel", async () => {
const text = await extractPdfText(pdf("ubereats-burgertory-2026-08-19.pdf"));
expect(text).toContain("Here's your receipt for Burgertory (Werribee).");
expect(text).toContain("Total $22.40");
expect(text).toContain("1 Michael $14.00");
expect(text).toContain("1 Joel $14.00");
expect(text).toContain("Item Subtotal $28.00");
});
});
describe("cards carry the C5 action ids and JSON values", () => {
const c = { batchId: "11111111-2222-4333-8444-555555555555", merchant: "Burgertory (Werribee)", total: 22.4, orderDate: "2026-08-19", platform: "ubereats", itemCount: 2 };
it("candidates: one Use-this per candidate plus an explicit Create", () => {
const { blocks } = candidatesCard(c, [{ transaction_id: 42, description: "Order - Burgertory (Werribee)", transaction_date: "2026-08-17", amount: "19.90" }]);
const buttons = JSON.stringify(blocks);
expect(buttons).toContain('"action_id":"intake_use_candidate"');
expect(buttons).toContain('"action_id":"intake_create"');
const use = (blocks[1] as { accessory: { value: string } }).accessory.value;
expect(JSON.parse(use)).toEqual({ batch_id: c.batchId, transaction_id: 42 });
});
it("enriched: the confirmation text and a View link", () => {
const { text, blocks } = enrichedCard({ ...c, transactionId: 7 });
expect(text).toBe("Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓");
expect(JSON.stringify(blocks)).toContain("/transactions?q=Burgertory");
});
it("ask: Groceries / A meal", () => {
const ids = JSON.stringify(askCard({ ...c, platform: null }).blocks);
expect(ids).toContain('"action_id":"intake_grocery"');
expect(ids).toContain('"action_id":"intake_meal"');
});
it("every value the card did not write is escaped, plain_text included", () => {
const hostile = { ...c, merchant: "Ben & Jerry's <fake>" };
const { text, blocks } = candidatesCard(hostile, [
{ transaction_id: 42, description: "SQ *BEN & JERRY <AU>", transaction_date: "2026-08-17", amount: "19.90" },
]);
const sections = JSON.stringify(blocks.filter((b) => (b as Block).type === "section"));
expect(sections).toContain("Ben &amp; Jerry's &lt;fake&gt;");
expect(sections).toContain("SQ *BEN &amp; JERRY &lt;AU&gt;");
expect(text).toContain("Ben &amp; Jerry's &lt;fake&gt;");
// plain_text disables formatting, not entity decoding — &, < and > are
// control characters everywhere in a Slack payload.
const whole = JSON.stringify(blocks);
expect(whole).not.toContain("<fake>");
expect(whole).not.toContain("<AU>");
});
it("the currency is model output too, so it is escaped like any other", () => {
// parseExtraction accepts any string for `currency`; unescaped, `<@U123>`
// would be a live mention on every card the batch produces.
const { text } = enrichedCard({ ...c, currency: "<@U123>", transactionId: 7 });
expect(text).toContain("&lt;@U123&gt; 22.40");
expect(text).not.toContain("<@U123>");
});
it("a foreign receipt is not printed as dollars — header and candidate rows alike", () => {
expect(enrichedCard({ ...c, currency: "USD", transactionId: 7 }).text).toBe(
"Added 2 items to your USD 22.40 Uber Eats (Burgertory (Werribee)) order ✓"
);
expect(enrichedCard({ ...c, currency: "AUD", transactionId: 7 }).text).toContain("$22.40");
expect(enrichedCard({ ...c, transactionId: 7 }).text).toContain("$22.40");
// A candidate row is a ledger transaction and carries its own currency; it
// used to print "$" beneath a header reading "USD 22.40".
const { blocks } = candidatesCard({ ...c, currency: "USD" }, [
{ transaction_id: 42, description: "Order - Burgertory", transaction_date: "2026-08-17", amount: "19.90", currency: "USD" },
]);
expect(JSON.stringify(blocks)).toContain("USD 19.90");
});
});
describe("liteLLMExtractor — a finish_reason that is present must be `stop`", () => {
const good = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD", items: [], adjustments: [] };
const extractor = (finish_reason?: string) =>
liteLLMExtractor({
baseUrl: "http://litellm.test",
key: "k",
fetchImpl: (async () =>
Response.json({
choices: [{ message: { content: JSON.stringify(good) }, ...(finish_reason ? { finish_reason } : {}) }],
})) as unknown as typeof fetch,
});
it("accepts `stop`, and — on purpose — a reply that reports no reason at all", async () => {
expect((await extractor("stop")([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
expect((await extractor()([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
});
it("rejects any other reason, naming it — the strict schema only held for a reply that finished", async () => {
// An absent reason is NOT rejected: see the comment in intake-extract.ts.
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(ExtractionError);
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/too long/);
await expect(extractor("content_filter")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/content_filter/);
await expect(extractor("tool_calls")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/tool_calls/);
});
});
src/__tests__/unit/order-ingestion.test.ts
+3 -1
View File
@@ -508,7 +508,9 @@ describe("HTML entities in line items", () => {
// The ampersand decodes, and the bullet becomes a boundary, not text.
expect(item!.description).toContain("Sweet & Sour Crunch");
expect(item!.description).not.toContain("•");
expect(item!.options.length).toBeGreaterThan(0);
// `options` is optional on the parsed item; `?? 0` keeps the assertion honest
// (an absent list fails) instead of silencing the check with a non-null bang.
expect(item!.options?.length ?? 0).toBeGreaterThan(0);
});
it("leaves a literal &amp;bull; alone rather than turning it into a bullet", () => {
src/app/api/intake/route.ts
+67
View File
@@ -0,0 +1,67 @@
import { NextRequest, NextResponse } from "next/server";
import { sha256Of, type StoredFile } from "@/lib/intake-files";
import { liteLLMExtractor } from "@/lib/intake-extract";
import { pantryForwarder } from "@/lib/intake-pantry";
import { isIntakeSource } from "@/lib/intake-rules";
import { MAX_FILES, runIntake } from "@/lib/intake-router";
/**
* The one upload point (plan §11 C1). n8n forwards the iOS Shortcut's
* multipart POST here: `source`, `shared_by`, `batch_id`, and 1–6 files under
* `file`. The router reads what it is given and does every step that applies.
*
* Auth is a shared secret, like the two ingest siblings: this is machine-to-
* machine and there is no browser session to forward. Its own token so the
* Shortcut's credential rotates without touching the order or receipt lanes.
* Fails closed when unset.
*/
function authorised(req: NextRequest): boolean {
const expected = process.env.INTAKE_TOKEN;
if (!expected) return false;
const got = req.headers.get("x-ingest-token");
return !!got && got === expected;
}
const MAX_BYTES = 12_000_000;
export async function POST(req: NextRequest) {
if (!authorised(req)) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
let form: FormData;
try {
form = await req.formData();
} catch {
return NextResponse.json({ error: "multipart/form-data body required" }, { status: 400 });
}
const source = form.get("source");
if (!isIntakeSource(source)) {
return NextResponse.json({ error: "source must be one of uber_eats|doordash|coles|woolworths|other" }, { status: 400 });
}
const sharedBy = typeof form.get("shared_by") === "string" ? String(form.get("shared_by")).trim() || null : null;
const batchIdRaw = form.get("batch_id");
const batchId = typeof batchIdRaw === "string" && /^[0-9a-f-]{36}$/i.test(batchIdRaw) ? batchIdRaw.toLowerCase() : null;
if (batchIdRaw && !batchId) return NextResponse.json({ error: "batch_id must be a uuid" }, { status: 400 });
const files: StoredFile[] = [];
for (const entry of form.getAll("file")) {
if (!(entry instanceof File) || !entry.size) continue;
if (entry.size > MAX_BYTES) return NextResponse.json({ error: `${entry.name} is larger than 12 MB` }, { status: 400 });
const bytes = Buffer.from(await entry.arrayBuffer());
files.push({ name: entry.name || `file-${files.length + 1}`, type: entry.type || "", bytes, sha256: sha256Of(bytes) });
}
if (!files.length) return NextResponse.json({ error: "at least one file is required under field 'file'" }, { status: 400 });
if (files.length > MAX_FILES) return NextResponse.json({ error: `at most ${MAX_FILES} files per batch` }, { status: 400 });
try {
const result = await runIntake(
{ batchId, source, sharedBy, files },
{ extractor: liteLLMExtractor(), pantry: pantryForwarder() }
);
const { http_status, ...body } = result;
return NextResponse.json(body, { status: http_status });
} catch (e) {
const message = e instanceof Error ? e.message : String(e);
return NextResponse.json({ error: message }, { status: 500 });
}
}
src/app/api/slack/interactive/route.ts
+28
View File
@@ -6,6 +6,8 @@ import {
slackUserForParticipant,
} from "@/lib/slack-verify";
import { nudgeBlocks, detailsModal, partnerNudgeBlocks } from "@/lib/slack-blocks";
import { handleIntakeAction, isIntakeAction, parseActionValue } from "@/lib/intake-actions";
import { pantryForwarder } from "@/lib/intake-pantry";
import { completeSplit } from "@/lib/splits";
import {
RATINGS,
@@ -73,6 +75,22 @@ export async function POST(req: NextRequest) {
if (payload.type !== "block_actions") return new NextResponse(null, { status: 200 });
const action = payload.actions?.[0];
// Intake taps (plan §11 C5) act on a persisted batch, not a transaction, so
// their `value` is JSON `{batch_id, ...}` rather than `<id>:<verb>`. They are
// dispatched on action_id before the order-nudge parse below, which would
// otherwise read the JSON as a non-numeric transaction id and give up.
if (isIntakeAction(action?.action_id)) {
const reply = await handleIntakeAction(action.action_id, parseActionValue(action?.value), { pantry: pantryForwarder() });
await updateMessage(
payload.response_url,
reply.replace
? { replace_original: true, blocks: reply.blocks, text: reply.text }
: { response_type: "ephemeral", replace_original: false, text: reply.text }
);
return NextResponse.json({ blocks: reply.blocks, text: reply.text });
}
// A button carries `value`; a select carries it on the chosen option. The
// ratings moved to a select because four buttons became four full-width rows
// on Slack mobile, so both shapes have to resolve or rating silently stops
@@ -216,6 +234,16 @@ async function buildDetailsModal(transactionId: number, participantId: number) {
`SELECT merchant_normalized AS merchant, line_items
FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[transactionId]
);
src/app/api/transactions/[id]/line-items/route.ts
+58
View File
@@ -0,0 +1,58 @@
import { NextRequest, NextResponse } from "next/server";
import { LineItemsValidationError, replaceLineItems, TransactionNotFoundError, validateLineItems } from "@/lib/line-items";
/**
* Enrich endpoint (plan §11 C2): idempotent replace of the itemised receipt
* behind a transaction.
*
* Accepts either machine token. Pantry already holds RECEIPT_INGEST_TOKEN for
* the ING-10 lane and calls this on every confirm and on undo; the intake
* router holds INTAKE_TOKEN. No session auth — there is no browser here.
* Fails closed when neither is configured.
*
* Never creates a transaction: 404 when the id does not exist. Creates the
* `expense_metadata` row when the transaction has none.
*/
function authorised(req: NextRequest): boolean {
const got = req.headers.get("x-ingest-token");
if (!got) return false;
const accepted = [process.env.INTAKE_TOKEN, process.env.RECEIPT_INGEST_TOKEN].filter(Boolean);
return accepted.includes(got);
}
export async function PUT(req: NextRequest, { params }: { params: Promise<{ id: string }> }) {
if (!authorised(req)) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
const { id } = await params;
const transactionId = Number(id);
if (!Number.isInteger(transactionId) || transactionId <= 0) return NextResponse.json({ error: "invalid transaction id" }, { status: 400 });
let body: { line_items?: unknown; source?: unknown; batch_id?: unknown; replace?: unknown; platform?: unknown };
try {
body = await req.json();
} catch {
return NextResponse.json({ error: "invalid JSON" }, { status: 400 });
}
if (body.replace !== true) return NextResponse.json({ error: "replace: true is required — this endpoint only replaces" }, { status: 400 });
if (body.source !== "intake" && body.source !== "pantry") return NextResponse.json({ error: "source must be 'intake' or 'pantry'" }, { status: 400 });
const batchId = typeof body.batch_id === "string" && body.batch_id ? body.batch_id : null;
const platform = typeof body.platform === "string" && body.platform ? body.platform : null;
try {
const lineItems = validateLineItems(body.line_items);
const result = await replaceLineItems(transactionId, lineItems, { source: body.source, platform, batchId });
return NextResponse.json({
transaction_id: result.transaction_id,
metadata_id: result.metadata_id,
created_metadata: result.created_metadata,
previous_line_items: result.previous_line_items,
line_items: result.line_items,
});
} catch (e) {
if (e instanceof LineItemsValidationError) return NextResponse.json({ error: e.message }, { status: 400 });
if (e instanceof TransactionNotFoundError) return NextResponse.json({ error: e.message }, { status: 404 });
const message = e instanceof Error ? e.message : String(e);
return NextResponse.json({ error: message }, { status: 500 });
}
}
src/app/api/transactions/[id]/order/route.ts
+8
View File
@@ -34,6 +34,14 @@ export async function GET(
-- through matched_transaction_id. Both directions have to resolve or the
-- detail is missing on exactly the orders that were paid by card.
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- The same order as replaceLineItems in line-items.ts and matchTransaction
-- in intake-router.ts. Both directions can resolve at once — a statement
-- line owning its own row while a delivery receipt points at it — and an
-- unordered LIMIT 1 then reads a different row from the one an enrichment
-- wrote, so a PUT that reported success showed nothing on the screen.
-- NULLS LAST because DESC is NULLS FIRST and the pointing row's
-- transaction_id is NULL.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[Number(id)]
);
src/app/budget/page.tsx
+25 -42
View File
@@ -19,16 +19,20 @@ import { useQueryClient } from "@tanstack/react-query";
import { useMonthlyAnalytics, useDailySpend, useTransactions, useUpdateTransaction } from "@/lib/hooks";
import { formatCategory, CATEGORIES } from "@/lib/categories";
import { CATEGORY_COLORS, CHART, TOOLTIP_STYLE } from "@/lib/category-colors";
// The partial-month comparison rules live in their own module so they can be
// tested. They used to be closures here reading `new Date()` inline, which is
// why Priority 0 item 6 had no test (#155). See src/lib/partial-month.ts.
import {
currentMonthStr,
prevMonth,
daysInMonthOf,
elapsedDays,
spendThrough,
comparisonBasis,
baselineMonths,
baselineAverage,
} from "@/lib/partial-month";
function currentMonthStr(): string {
const now = new Date();
return `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, "0")}`;
}
function prevMonth(m: string): string {
const [year, month] = m.split("-").map(Number);
const d = new Date(year, month - 2, 1);
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
}
function formatMonth(m: string): string {
const [year, month] = m.split("-");
return new Date(Number(year), Number(month) - 1, 1).toLocaleString("default", { month: "long", year: "numeric" });
@@ -37,26 +41,6 @@ function formatShortMonth(m: string): string {
const [year, month] = m.split("-");
return new Date(Number(year), Number(month) - 1, 1).toLocaleString("default", { month: "short" });
}
function daysInMonthOf(m: string): number {
const [year, month] = m.split("-").map(Number);
return new Date(year, month, 0).getDate();
}
/**
* How much of `month` has actually happened. A month in progress is only
* complete up to today; every earlier month is complete.
*/
function elapsedDays(m: string): number {
return m === currentMonthStr() ? new Date().getDate() : daysInMonthOf(m);
}
/** Spend in `month` from day 1 through `throughDay` inclusive. */
function spendThrough(days: Record<number, number> | undefined, throughDay: number): number {
if (!days) return 0;
let sum = 0;
for (const [d, v] of Object.entries(days)) {
if (Number(d) <= throughDay) sum += v;
}
return sum;
}
function fmt(n: number): string { return `$${Math.round(n).toLocaleString()}`; }
function fmtExact(n: number): string { return `$${n.toFixed(2)}`; }
function fmtSigned(n: number): string { return `${n >= 0 ? "+" : "−"}$${Math.abs(n) >= 100 ? Math.round(Math.abs(n)).toLocaleString() : Math.abs(n).toFixed(0)}`; }
@@ -221,8 +205,7 @@ export default function AnalyticsPage() {
// A month in progress is only comparable to prior months through the same day.
// Comparing 27 days of July against a full June always flattered July.
const compareDay = elapsedDays(selectedMonth);
const selectedIsPartial = selectedMonth === currentMonthStr();
const { compareDay, isPartial: selectedIsPartial } = comparisonBasis(selectedMonth);
// Category rows for selected month
const categoryRows = useMemo(() => {
@@ -299,8 +282,10 @@ export default function AnalyticsPage() {
// Only complete months with data form the baseline. A month still in
// progress has no spend recorded past today, so including it would pull the
// typical curve down by however much of it has not happened yet.
const priorMonths = (analytics?.months ?? []).filter(
(m) => m !== selectedMonth && m !== currentMonthStr() && (analytics?.totals[m]?.spent || 0) > 0
const priorMonths = baselineMonths(
analytics?.months ?? [],
selectedMonth,
analytics?.totals ?? {}
);
let cum = 0;
@@ -342,18 +327,16 @@ export default function AnalyticsPage() {
// baseline, because most of it has not happened. And when the month in
// progress is the one selected, its running total has to be measured against
// the same slice of each prior month, not against their full totals.
const otherMonths = months.filter(
(m) => m !== selectedMonth && m !== currentMonthStr() && (analytics.totals[m]?.spent || 0) > 0
);
const otherMonths = baselineMonths(months, selectedMonth, analytics.totals);
const comparableSpend = selectedIsPartial
? spendThrough(dailyData?.daily?.[selectedMonth], compareDay)
: totals.spent;
const avgSpend = otherMonths.length
? otherMonths.reduce(
(s, m) => s + (selectedIsPartial ? spendThrough(dailyData?.daily?.[m], compareDay) : analytics.totals[m]?.spent || 0),
0
) / otherMonths.length
: 0;
const avgSpend = baselineAverage(otherMonths, {
isPartial: selectedIsPartial,
compareDay,
daily: dailyData?.daily,
totals: analytics.totals,
});
const avgDeltaPct = avgSpend > 0 ? Math.round(((comparableSpend - avgSpend) / avgSpend) * 100) : 0;
const throughQualifier = selectedIsPartial ? ` through day ${compareDay}` : "";
const heroSentence =
src/app/reconcile/page.tsx
+1 -1
View File
@@ -141,7 +141,7 @@ export default function ReconcilePage() {
<div className="border border-zinc-800 rounded-xl overflow-hidden">
<div className="px-4 py-3 bg-zinc-900/60 border-b border-zinc-800">
<p className="text-sm font-medium text-zinc-400">No statement matches found ({noMatches.length})</p>
<p className="text-xs text-zinc-600 mt-0.5">These may not have hit a statement yet, or the statement hasn't been imported.</p>
<p className="text-xs text-zinc-600 mt-0.5">These may not have hit a statement yet, or the statement hasn&apos;t been imported.</p>
</div>
<div className="divide-y divide-zinc-800">
{noMatches.map((tx) => (
src/app/shared/page.tsx
+26 -9
View File
@@ -432,6 +432,19 @@ const SharedTxRow = memo(function SharedTxRow({
// ── Main page ─────────────────────────────────────────────────────────────────
type SortCol = "transaction_date" | "created_at" | "amount";
/**
* Sort direction marker for a column header.
*
* Declared at module scope and given the sort state as props, rather than
* closing over it inside SharedPage: a component defined during render is a
* new type on every render, so React unmounts and remounts it each time
* instead of updating it.
*/
function SortIcon({ col, sortCol, sortDir }: { col: SortCol; sortCol: SortCol; sortDir: "asc" | "desc" }) {
if (sortCol !== col) return <span className="text-zinc-600 ml-0.5">↕</span>;
return <span className="ml-0.5">{sortDir === "desc" ? "↓" : "↑"}</span>;
}
export default function SharedPage() {
const [tagIds, setTagIds] = useState<string[]>([]);
const [participantId, setParticipantId] = useState<number | undefined>(undefined);
@@ -478,10 +491,6 @@ export default function SharedPage() {
else { setSortCol(col); setSortDir("desc"); }
}
function SortIcon({ col }: { col: SortCol }) {
if (sortCol !== col) return <span className="text-zinc-600 ml-0.5">↕</span>;
return <span className="ml-0.5">{sortDir === "desc" ? "↓" : "↑"}</span>;
}
const { data: balances = [], isLoading: balLoading } = useParticipantBalances(realTagIds);
const { data: allTags = [] } = useTags();
const { data: me } = useCurrentUser();
@@ -504,9 +513,17 @@ export default function SharedPage() {
// table was what made expanding a receipt freeze the browser. "Show more"
// extends the window; changing any filter resets it.
const [visibleCount, setVisibleCount] = useState(100);
useEffect(() => {
// Reset during render behind a previous-value guard rather than in an effect.
// Note the key joins `tagIds` by value: as a raw dependency it was a fresh
// array on many renders, so the window was being reset more often than
// "changing any filter" claims — including after "Show more", which is the
// one interaction this must not undo.
const windowKey = `${search}|${sortCol}|${sortDir}|${tagIds.join(",")}|${participantId ?? ""}`;
const [lastWindowKey, setLastWindowKey] = useState(windowKey);
if (windowKey !== lastWindowKey) {
setLastWindowKey(windowKey);
setVisibleCount(100);
}, [search, sortCol, sortDir, tagIds, participantId]);
}
const visible = (transactions as SharedTransactionRow[]).slice(0, visibleCount);
// Receipt expansion, same pattern as the transactions page. The shared
@@ -667,13 +684,13 @@ export default function SharedPage() {
className="text-left px-4 py-2 text-xs text-zinc-500 font-medium cursor-pointer hover:text-white whitespace-nowrap"
onClick={() => toggleSort("transaction_date")}
>
Date <SortIcon col="transaction_date" />
Date <SortIcon col="transaction_date" sortCol={sortCol} sortDir={sortDir} />
</th>
<th
className="text-left px-4 py-2 text-xs text-zinc-500 font-medium cursor-pointer hover:text-white whitespace-nowrap"
onClick={() => toggleSort("created_at")}
>
Imported <SortIcon col="created_at" />
Imported <SortIcon col="created_at" sortCol={sortCol} sortDir={sortDir} />
</th>
<th className="text-left px-4 py-2 text-xs text-zinc-500 font-medium sticky left-0 z-10 bg-zinc-900 border-r border-zinc-800/80">Description</th>
<th className="text-left px-4 py-2 text-xs text-zinc-500 font-medium">Category</th>
@@ -681,7 +698,7 @@ export default function SharedPage() {
className="text-right px-4 py-2 text-xs text-zinc-500 font-medium cursor-pointer hover:text-white"
onClick={() => toggleSort("amount")}
>
Amount <SortIcon col="amount" />
Amount <SortIcon col="amount" sortCol={sortCol} sortDir={sortDir} />
</th>
{/* Paid by sits next to Splits deliberately: together they are the
two halves of the question this page exists to answer — whose
src/app/tags/page.tsx
+1 -1
View File
@@ -59,7 +59,7 @@ function ConvertModal({
<div className="px-6 pt-5 pb-4 border-b border-zinc-800">
<div className="flex items-center gap-2">
<span className="w-3 h-3 rounded-full flex-shrink-0" style={{ backgroundColor: tag.color }} />
<h3 className="font-semibold text-sm text-zinc-300">Convert "{tag.name}" to Trip</h3>
<h3 className="font-semibold text-sm text-zinc-300">Convert &quot;{tag.name}&quot; to Trip</h3>
</div>
<p className="text-xs text-zinc-500 mt-1">
Creates a trip with the same name and color, and assigns all {tag.transaction_count} tagged transaction{tag.transaction_count !== 1 ? "s" : ""} to it.
src/app/transactions/page.tsx
+12 -6
View File
@@ -148,7 +148,7 @@ function SaveAsRulePrompt({
Automatically apply this {field === "category" ? "category" : "merchant name"} to future matching transactions.
</p>
<div className="bg-zinc-900 rounded-lg px-3 py-2 text-xs text-zinc-300 mb-3 space-y-1">
<p><span className="text-zinc-500">If</span> {conditionField === "merchant_normalized" ? "merchant" : "description"} {conditionOperator} <span className="text-white">"{conditionValue}"</span></p>
<p><span className="text-zinc-500">If</span> {conditionField === "merchant_normalized" ? "merchant" : "description"} {conditionOperator} <span className="text-white">&quot;{conditionValue}&quot;</span></p>
<p>
<span className="text-zinc-500">Then</span>{" "}
{field === "category"
@@ -250,11 +250,17 @@ function MarkAsPaymentModal({
const [participantId, setParticipantId] = useState<number | "">(others[0]?.id ?? "");
useEffect(() => {
if (participantId === "" && others.length > 0) {
setParticipantId(others[0].id);
}
}, [others]);
// Seed the picker once the participant list arrives — the initial state is
// computed before the query resolves, so it starts empty.
//
// Adjusted during render rather than in an effect, which is also strictly
// more correct: `others` is rebuilt by .filter() on every render, so as a
// dependency it never compared equal and the effect ran every render anyway
// — just one commit later, painting an empty select first. The condition
// clears itself on the first non-empty list, so this cannot loop.
if (participantId === "" && others.length > 0) {
setParticipantId(others[0].id);
}
// For credits/refunds the default direction is "they paid me"
const [direction, setDirection] = useState<"received" | "sent">(
SPEND_TYPES.has(transaction.transaction_type) ? "sent" : "received"
src/components/csv-import-modal.tsx
+13 -3
View File
@@ -1,6 +1,6 @@
"use client";
import { useState, useRef, useEffect } from "react";
import { useState, useRef } from "react";
import { CATEGORIES, formatCategory } from "@/lib/categories";
import { useImportCSV, useStatementCoverage } from "@/lib/hooks";
import {
@@ -54,13 +54,23 @@ export function CsvImportModal({ onClose }: { onClose: () => void }) {
const [savePreset, setSavePreset] = useState(false);
const [editedRows, setEditedRows] = useState<ParsedTransaction[]>([]);
const [error, setError] = useState("");
const [presets, setPresets] = useState<BankPreset[]>([]);
// Read once, at mount, from localStorage. A lazy initializer rather than an
// effect: it runs exactly once per mount — the same semantics as
// `useEffect(…, [])` — but before the first paint, so the preset dropdown
// does not appear empty and then populate. Nothing in this component writes
// `presets` afterwards, so there is no state to keep in sync.
//
// The `typeof window` guard is for safety only: this modal is mounted from
// client state and so is never server-rendered. If that ever changes, it
// degrades to an empty list instead of throwing on `localStorage`.
const [presets] = useState<BankPreset[]>(() =>
typeof window === "undefined" ? [] : loadBankPresets()
);
const [insertedCount, setInsertedCount] = useState(0);
const [coveredRows, setCoveredRows] = useState<ParsedTransaction[]>([]);
const [showCovered, setShowCovered] = useState(false);
const coverage = useStatementCoverage();
useEffect(() => { setPresets(loadBankPresets()); }, []);
function handleFile(file: File) {
setError("");
src/components/order-details.tsx
+4 -7
View File
@@ -13,12 +13,9 @@ import {
type OrderReceipt,
type OrderRating,
} from "@/lib/hooks";
const PLATFORM_LABEL: Record<string, string> = {
doordash: "DoorDash",
ubereats: "Uber Eats",
uber: "Uber",
};
// One platform vocabulary, shared with the intake cards — three copies had
// already drifted apart (the intake one was missing `uber`).
import { platformLabel } from "@/lib/intake-rules";
/**
* Who records verdicts. A two-person household with one primary user: the
@@ -99,7 +96,7 @@ export function OrderDetails({
<p className="text-xs text-zinc-500">
Order details
{receipt.platform && (
<span className="ml-1.5 text-zinc-400">{PLATFORM_LABEL[receipt.platform] ?? receipt.platform}</span>
<span className="ml-1.5 text-zinc-400">{platformLabel(receipt.platform)}</span>
)}
</p>
{receipt.card_last4 && (
src/components/rule-preview-modal.tsx
+12 -4
View File
@@ -1,6 +1,6 @@
"use client";
import { useEffect, useMemo, useState } from "react";
import { useMemo, useState } from "react";
import { useRuleMatches, useApplyRuleToSelection, type RuleMatchChange } from "@/lib/hooks";
import { formatCategory } from "@/lib/categories";
@@ -39,9 +39,17 @@ export function RulePreviewModal({
// Default to everything the rule would change — the common case is "yes, all
// of these", with deselection as the exception.
useEffect(() => {
if (data?.transactions) setSelected(new Set(data.transactions.map((t) => t.id)));
}, [data]);
//
// Adjusted during render behind a previous-value guard rather than in an
// effect. In an effect the modal painted once with nothing ticked and then
// again with everything ticked; here the seeded selection is part of the
// first commit. The guard is on `data` identity, so a user deselecting a row
// is not undone on the next render — only a genuinely new preview reseeds.
const [seededFrom, setSeededFrom] = useState<typeof data>(undefined);
if (data !== seededFrom) {
setSeededFrom(data);
setSelected(new Set(data?.transactions?.map((t) => t.id) ?? []));
}
const rows = data?.transactions ?? [];
const allSelected = rows.length > 0 && selected.size === rows.length;
src/components/sidebar.tsx
+12 -3
View File
@@ -82,10 +82,19 @@ export function Sidebar() {
const pathname = usePathname();
const [open, setOpen] = useState(false);
// Close drawer on route change
useEffect(() => {
// Close the drawer on route change.
//
// Adjusting state during render behind a previous-value guard — the
// documented React pattern for "reset state when a prop changes" — rather
// than in an effect: React re-runs this component immediately and never
// commits the intermediate DOM, so there is no cascading render. Putting it
// in each Link's onClick is the other candidate fix and is worse here: it
// misses browser back/forward, where the route changes with no click.
const [lastPathname, setLastPathname] = useState(pathname);
if (pathname !== lastPathname) {
setLastPathname(pathname);
setOpen(false);
}, [pathname]);
}
// Prevent body scroll when drawer is open
useEffect(() => {
src/components/split-modal.tsx
+27 -17
View File
@@ -1,6 +1,6 @@
"use client";
import { useState, useEffect } from "react";
import { useState } from "react";
import { useParticipants, useSetSplits, useTransactionSplits, useBulkAction, useCreateRule, useCurrentUser } from "@/lib/hooks";
interface Split {
@@ -33,22 +33,32 @@ export function SplitModal({ transactionId, transactionIds, amount, description,
const [saveAsRule, setSaveAsRule] = useState(false);
const [ruleSaved, setRuleSaved] = useState(false);
// Initialise: bulk always defaults to 100% Me; single loads existing splits
useEffect(() => {
if (!participants || participants.length === 0 || !currentUser) return;
if (isBulk) {
setSplitsState([{ participant_id: currentUser.id, share_percent: 100 }]);
} else if (existingSplits && existingSplits.length > 0) {
setSplitsState(
existingSplits.map((s: { participant_id: number; share_percent: number }) => ({
participant_id: s.participant_id,
share_percent: Number(s.share_percent),
}))
);
} else {
setSplitsState([{ participant_id: currentUser.id, share_percent: 100 }]);
}
}, [existingSplits, participants, isBulk, currentUser]);
// Initialise: bulk always defaults to 100% Me; single loads existing splits.
//
// Seeded during render behind a previous-value guard rather than in an
// effect. Both queries resolve after the first paint, so as an effect this
// rendered an empty split table and then replaced it — and, worse, it re-ran
// on every identity change of `participants`/`existingSplits`, which would
// discard edits in progress if either query refetched while the modal was
// open. Keying on the inputs that actually define the seed fixes both.
const seedKey =
!participants || participants.length === 0 || !currentUser
? null
: `${isBulk ? "bulk" : singleId}:${currentUser.id}:${
existingSplits ? JSON.stringify(existingSplits) : "none"
}`;
const [seededFrom, setSeededFrom] = useState<string | null>(null);
if (seedKey !== null && seedKey !== seededFrom) {
setSeededFrom(seedKey);
setSplitsState(
!isBulk && existingSplits && existingSplits.length > 0
? existingSplits.map((s: { participant_id: number; share_percent: number }) => ({
participant_id: s.participant_id,
share_percent: Number(s.share_percent),
}))
: [{ participant_id: currentUser!.id, share_percent: 100 }]
);
}
const total = splits.reduce((sum, s) => sum + s.share_percent, 0);
src/lib/db.ts
+30
View File
@@ -20,3 +20,33 @@ export async function queryRow<T>(sql: string, params: unknown[] = []): Promise<
const rows = await prisma.$queryRawUnsafe<T[]>(sql, ...params);
return rows.length > 0 ? rows[0] : null;
}
/** The same two calls, scoped to one connection inside one transaction. */
export type Queryable = {
queryRaw<T>(sql: string, params?: unknown[]): Promise<T[]>;
queryRow<T>(sql: string, params?: unknown[]): Promise<T | null>;
};
/**
* Run several statements as one transaction, on one connection.
*
* The module-level `queryRaw`/`queryRow` are `$queryRawUnsafe` on a POOLED
* client, so each call is its own implicit transaction on whichever connection
* the pool hands out. That makes a check-then-write pair non-atomic, and it
* makes `pg_advisory_xact_lock` useless outside this helper — the lock is
* released the instant the statement that took it commits, which is before the
* next statement runs. Anything that needs a lock to still be held on its next
* query has to be in here.
*/
export async function withTransaction<T>(fn: (q: Queryable) => Promise<T>): Promise<T> {
return prisma.$transaction(async (tx) => {
const q: Queryable = {
queryRaw: <R>(sql: string, params: unknown[] = []) => tx.$queryRawUnsafe<R[]>(sql, ...params),
queryRow: async <R>(sql: string, params: unknown[] = []) => {
const rows = await tx.$queryRawUnsafe<R[]>(sql, ...params);
return rows.length > 0 ? rows[0] : null;
},
};
return fn(q);
});
}
src/lib/intake-actions.ts
+119
View File
@@ -0,0 +1,119 @@
import { loadFiles } from "@/lib/intake-files";
import type { PantryForward } from "@/lib/intake-pantry";
import type { IntakeSource } from "@/lib/intake-rules";
import {
createFromBatch,
loadBatch,
rememberKind,
resolvedCard,
routeFood,
routeGrocery,
type BatchRow,
type RouteContext,
} from "@/lib/intake-router";
import { replaceLineItems, TransactionNotFoundError } from "@/lib/line-items";
import { enrichedCard, errorCard, type Block } from "@/lib/intake-blocks";
import { queryRaw } from "@/lib/db";
/**
* The four intake taps (plan §11 C5), as one function the Slack route calls.
*
* Every tap acts on the persisted batch, never on the card's text. A tap on a
* batch that is already resolved returns its current card rather than acting
* twice — a pressed button that looked dead and got pressed again is how three
* splits were lost on the order nudge, and the same shape applies here.
*/
export const INTAKE_ACTIONS = ["intake_use_candidate", "intake_create", "intake_grocery", "intake_meal"] as const;
export type IntakeActionId = (typeof INTAKE_ACTIONS)[number];
export function isIntakeAction(id: unknown): id is IntakeActionId {
return typeof id === "string" && (INTAKE_ACTIONS as readonly string[]).includes(id);
}
export type IntakeActionValue = { batch_id?: string; transaction_id?: number };
export function parseActionValue(raw: unknown): IntakeActionValue {
if (typeof raw !== "string") return {};
try {
const v = JSON.parse(raw) as IntakeActionValue;
return v && typeof v === "object" ? v : {};
} catch {
return {};
}
}
const note = (text: string): { blocks: Block[]; text: string; replace: boolean } => ({
text,
blocks: [{ type: "section", text: { type: "mrkdwn", text } }],
replace: false,
});
const ctxFor = (batch: BatchRow): RouteContext | null =>
batch.header
? {
batchId: batch.id,
source: batch.source as IntakeSource,
sharedBy: batch.shared_by,
platform: batch.header.platform,
header: batch.header,
items: batch.items ?? [],
currency: batch.header.currency,
}
: null;
export async function handleIntakeAction(
actionId: IntakeActionId,
value: IntakeActionValue,
deps: { pantry: PantryForward }
): Promise<{ blocks: Block[]; text: string; replace: boolean }> {
if (!value.batch_id) return note("That button carried no batch id — I can't tell which upload it was.");
const batch = await loadBatch(value.batch_id);
if (!batch) return note("I can't find that upload any more.");
if (batch.status === "enriched" || batch.status === "created" || batch.status === "forwarded_to_pantry") {
return { ...resolvedCard(batch), replace: true };
}
const ctx = ctxFor(batch);
if (!ctx) return note("That upload never got a readable header — share it again with the top of the screen.");
try {
if (actionId === "intake_use_candidate") {
if (!Number.isInteger(value.transaction_id)) return note("That button carried no transaction id.");
const txnId = value.transaction_id as number;
await replaceLineItems(txnId, ctx.items, { source: "intake", platform: ctx.platform, batchId: batch.id, currency: ctx.currency });
await queryRaw(
`UPDATE intake_batches SET status = 'enriched', kind = COALESCE(kind, 'food'), transaction_id = $2, resolved_at = now() WHERE id = $1::uuid`,
[batch.id, txnId]
);
return {
...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, currency: ctx.currency, transactionId: txnId }),
replace: true,
};
}
if (actionId === "intake_create") {
const kind = batch.kind === "grocery" ? "grocery" : "food";
const r = await createFromBatch(batch, kind);
return { blocks: r.slack_blocks ?? [], text: r.text, replace: true };
}
if (actionId === "intake_meal") {
await rememberKind(ctx.header.merchant, "food");
const r = await routeFood(ctx);
return { blocks: r.slack_blocks ?? [], text: r.text, replace: true };
}
// intake_grocery
await rememberKind(ctx.header.merchant, "grocery");
const files = await loadFiles(batch.file_sha256);
if (!files.length) {
return note("I remembered that as groceries, but the original files are gone — share it again and it will go straight to the pantry board.");
}
const r = await routeGrocery(ctx, files, deps.pantry);
return { blocks: r.slack_blocks ?? [], text: r.text, replace: r.status !== "error" };
} catch (e) {
if (e instanceof TransactionNotFoundError) return note("That transaction is no longer in the ledger.");
const detail = e instanceof Error ? e.message : String(e);
return { ...errorCard(detail), replace: false };
}
}
src/lib/intake-blocks.ts
+175
View File
@@ -0,0 +1,175 @@
import { platformLabel } from "@/lib/intake-rules";
/**
* The intake reply as Block Kit (plan §11 C1/C5). Built here, never in n8n
* expressions, so the button payloads the interactive handler round-trips are
* produced by one piece of code and tested like one.
*
* Button `value`s are JSON `{batch_id, ...}` — the batch row is what a later
* tap acts on. action_ids are the C5 contract: `intake_use_candidate`,
* `intake_create`, `intake_grocery`, `intake_meal`.
*/
export type Block = Record<string, unknown>;
export type Candidate = { transaction_id: number; description: string | null; transaction_date: string; amount: string | number; currency?: string | null };
export function financePublicUrl(): string {
return (process.env.FINANCE_PUBLIC_URL || "https://finance.bosecamp.com").replace(/\/$/, "");
}
/** The transactions page searches on `q`; the merchant is the most selective thing a card knows. */
export function transactionLink(merchant: string | null): string {
return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`;
}
/**
* Slack renders `&`, `<` and `>` as markup in mrkdwn (and in the `text`
* fallback). Merchant names, transaction descriptions and model output are
* none of ours — `Ben & Jerry's` must not arrive as `Ben &amp; Jerry's`, and a
* description containing `<http://…|click>` must not become a link the card
* never meant to show. Applied to interpolated values only; our own literal
* `*bold*` and `:emoji:` stay markup.
*/
export function esc(value: string | null | undefined): string {
return (value ?? "").replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
}
/**
* `$` is only right for AUD. An intake batch carries the currency the receipt
* printed, and the order nudge already renders `USD 15.00` rather than
* pretending a foreign order was dollars.
*/
const money = (n: number | string, currency?: string | null) => {
const amount = Number(n).toFixed(2);
// The currency comes off the model (`parseExtraction` accepts any string), so
// an unescaped `<@U123>` here would be a mention on every card.
return !currency || currency === "AUD" ? `$${amount}` : `${esc(currency)} ${amount}`;
};
const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } });
const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({
type: "button",
action_id,
text: { type: "plain_text", text, emoji: true },
value: JSON.stringify(value),
...(style ? { style } : {}),
});
export type CardInput = {
batchId: string;
merchant: string | null;
total: number | null;
orderDate: string | null;
platform: string | null;
itemCount: number;
currency?: string | null;
};
export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const merchant = esc(c.merchant);
const what = [c.total !== null ? money(c.total, c.currency) : null, esc(platformLabel(c.platform)) || merchant, c.platform && merchant ? `(${merchant})` : null]
.filter(Boolean)
.join(" ");
const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`;
return {
text,
blocks: [
{
type: "section",
text: { type: "mrkdwn", text: `:receipt: ${text}` },
accessory: { type: "button", text: { type: "plain_text", text: "View" }, url: transactionLink(c.merchant), action_id: "intake_view" },
},
],
};
}
export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
const text = `Created a ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant) || "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`;
return {
text,
blocks: [
{
type: "section",
text: { type: "mrkdwn", text: `:receipt: ${text}` },
accessory: { type: "button", text: { type: "plain_text", text: "View" }, url: transactionLink(c.merchant), action_id: "intake_view" },
},
],
};
}
export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } {
const label = esc(platformLabel(c.platform));
const head = `No matching ${label ? label + " " : ""}order for *${esc(c.merchant) || "this merchant"}* ${c.total !== null ? money(c.total, c.currency) : ""} on ${esc(c.orderDate) || "?"}.`;
const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`;
const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)];
for (const cand of candidates) {
blocks.push({
type: "section",
text: { type: "mrkdwn", text: `${esc(cand.transaction_date)} · ${esc(cand.description) || "—"} · ${money(cand.amount, cand.currency)}` },
accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }),
});
}
blocks.push({
type: "actions",
block_id: "intake_create",
// plain_text disables *formatting*, not entity decoding: `&`, `<` and `>`
// are control characters everywhere in a Slack payload, so the label is
// escaped like any other value we did not write.
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant)} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
});
return { blocks, text };
}
export function forwardedCard(c: CardInput & { captureId: number; enriching: boolean }): { blocks: Block[]; text: string } {
const text = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""} → pantry board (${c.itemCount} item${c.itemCount === 1 ? "" : "s"}) — finance ${c.enriching ? "updates" : "books it"} on confirm`.replace(/\s+/g, " ");
return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
}
export function needsHeaderCard(): { blocks: Block[]; text: string } {
const text = "I can see items but no order total or date — share the top of the screen too so I can match it.";
return { text, blocks: [section(`:warning: ${text}`)] };
}
export function heldCard(reason: string, headline = "Items don't add up to the total — is a shot missing?"): { blocks: Block[]; text: string } {
const text = `${headline} (${esc(reason)})`;
return { text, blocks: [section(`:warning: ${text}`)] };
}
export function askCard(c: CardInput): { blocks: Block[]; text: string } {
const text = `${esc(c.merchant) || "This upload"} ${c.total !== null ? money(c.total, c.currency) : ""} — groceries or a meal?`.replace(/\s+/g, " ");
return {
text,
blocks: [
section(`:thinking_face: ${text}`),
{
type: "actions",
block_id: "intake_ask",
elements: [
button("intake_grocery", "Groceries", { batch_id: c.batchId }),
button("intake_meal", "A meal", { batch_id: c.batchId }),
],
},
],
};
}
export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null; currency?: string | null }): { blocks: Block[]; text: string } {
const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${esc(c.batchId)}` : "an earlier upload";
// The header (merchant, date, total) IS the identity of a purchase here, so a
// genuine SECOND identical purchase — two of the same coffee, one day, one shop
// — reads as a re-share and is refused. That is the right default (a re-share is
// far commoner than a repeat) but it is a dead end without this sentence: the
// card must say what to do instead, or the ledger quietly loses the second one.
const text = `Already handled — ${esc(c.merchant) || "this order"} ${c.total !== null ? money(c.total, c.currency) : ""} is ${where}. Nothing was changed — if this really is a second, separate purchase, add it in the app.`.replace(/\s+/g, " ");
return { text, blocks: [section(`:repeat: ${text}`)] };
}
export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } {
const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${esc(detail)})` : ""}`;
return { text, blocks: [section(`:no_entry_sign: ${text}`)] };
}
export function errorCard(detail: string): { blocks: Block[]; text: string } {
const text = `Couldn't process that upload: ${esc(detail)}`;
return { text, blocks: [section(`:x: ${text}`)] };
}
src/lib/intake-extract.ts
+216
View File
@@ -0,0 +1,216 @@
import { extractText, getDocumentProxy } from "unpdf";
import type { ExtractedOrder, ExtractedItem } from "@/lib/intake-rules";
/**
* Extraction for the intake router: one joint model call per batch.
*
* The model EXTRACTS; it does not classify or route. The source tap and the
* merchant pattern decide where the result goes (plan §2 — "a classifier is
* confidently wrong exactly when it matters"). So this prompt asks for the
* header and the lines, verbatim, and nothing else.
*
* One call with every part in order, never one per image: scrolling screenshots
* overlap, and extracting each and concatenating double-counts the row both
* shots contain and strands the lower shot with no header (plan §3, Codex #7).
* Dedupe of the overlap is the model's job; the reconciliation check in
* `intake-rules.ts` is what catches it getting that wrong.
*/
export type InputPart =
| { kind: "image"; mime: string; bytes: Buffer; name: string }
| { kind: "text"; text: string; name: string };
/**
* Digital receipts are text, not pixels. Reading the text beats photographing
* the screen: nothing is misread, and the model parses a page rather than an
* image. Same approach as pantry-app's `pdf-text.ts`.
*/
export async function extractPdfText(bytes: Buffer): Promise<string> {
const document = await getDocumentProxy(new Uint8Array(bytes));
const { text } = await extractText(document, { mergePages: true });
return (Array.isArray(text) ? text.join("\n") : text).trim();
}
export const ORDER_ITEM_PROMPT = `You are reading a purchase receipt: an online food or grocery order (Uber Eats, DoorDash, Coles, Woolworths, ...) or an in-store till receipt. It arrives as one or more screenshots in scroll order, as the text of a PDF, or as pasted text.
Return ONE JSON object with exactly these keys:
{ "merchant", "order_date", "total", "subtotal", "currency", "items": [{ "quantity", "description", "unit_price", "line_total", "options" }], "adjustments": [{ "label", "amount" }] }
Rules:
- merchant: the store or restaurant name exactly as printed, including any branch in parentheses (e.g. "Coles (Wyndham Vale)", "Burgertory (Werribee)"). Never the delivery platform. null if not shown.
- order_date: the order or purchase date printed, as YYYY-MM-DD. null if not shown.
- total: the amount actually charged — the "Total" / amount paid — as a number. null if not shown.
- subtotal: the printed item subtotal before fees and discounts, as a number, or null.
- currency: ISO code if printed (default "AUD").
- items: every purchased line. Keep description VERBATIM as printed — do not rename, canonicalise, translate, shorten or summarise it. A burger called "Michael" stays "Michael"; "Coles RSPCA Approved Chicken Thigh Cutlets approx. 1.1kg" stays exactly that. Put selected options and modifiers (e.g. "Milk Bun", "Chicken") and weighed or substitution notes (e.g. "Final 1.16 kg @ $11.00/kg", "Replaced Coles Beef Slow Cook Chuck Roast approx. 1.3kg") in options, as an array of strings, not in the description.
- quantity: the number ordered (a number; 1 if a line shows none). unit_price: the price per unit as printed, or null. line_total: the money for the whole line as printed, or quantity × unit_price when only a unit price is shown; for weighed lines multiply the final weight by the per-kg price. null when nothing is shown.
- Do not put fees, tips, delivery or service charges, discounts, promotions, savings, tax lines or payment lines in items. List those in adjustments instead, fees positive and discounts negative.
- Multiple screenshots are consecutive scrolls of ONE order and overlap: each physical line appears exactly once in items, never twice.
- Never invent a value. If something is not legible or not present, use null.`;
const NUM_OR_NULL = { anyOf: [{ type: "number" }, { type: "null" }] };
const STR_OR_NULL = { anyOf: [{ type: "string" }, { type: "null" }] };
/** Strict json_schema: every property listed in `required`, nullability via anyOf. */
export const EXTRACT_RESPONSE_FORMAT = {
type: "json_schema",
json_schema: {
name: "order_items",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["merchant", "order_date", "total", "subtotal", "currency", "items", "adjustments"],
properties: {
merchant: STR_OR_NULL,
order_date: STR_OR_NULL,
total: NUM_OR_NULL,
subtotal: NUM_OR_NULL,
currency: STR_OR_NULL,
items: {
type: "array",
items: {
type: "object",
additionalProperties: false,
required: ["quantity", "description", "unit_price", "line_total", "options"],
properties: {
quantity: NUM_OR_NULL,
description: { type: "string" },
unit_price: NUM_OR_NULL,
line_total: NUM_OR_NULL,
options: { type: "array", items: { type: "string" } },
},
},
},
adjustments: {
type: "array",
items: {
type: "object",
additionalProperties: false,
required: ["label", "amount"],
properties: { label: { type: "string" }, amount: { type: "number" } },
},
},
},
},
},
} as const;
/** The OpenAI-format message content: prompt first, then every part in the order shared. */
export function buildMessageContent(parts: InputPart[]): unknown[] {
const content: unknown[] = [{ type: "text", text: ORDER_ITEM_PROMPT }];
parts.forEach((p, i) => {
if (p.kind === "text") {
content.push({ type: "text", text: `\n--- Input ${i + 1} of ${parts.length} (${p.name}) ---\n${p.text}` });
} else {
content.push({ type: "text", text: `\n--- Input ${i + 1} of ${parts.length}: image ${p.name} ---` });
content.push({ type: "image_url", image_url: { url: `data:${p.mime || "image/jpeg"};base64,${p.bytes.toString("base64")}` } });
}
});
return content;
}
const num = (v: unknown): number | null => {
if (typeof v === "number" && Number.isFinite(v)) return v;
if (typeof v === "string" && v.trim()) {
const n = Number(v.replace(/[^0-9.-]/g, ""));
return Number.isFinite(n) ? n : null;
}
return null;
};
/** Tolerant of fences and a stray preamble; strict about the shape it hands on. */
export function parseExtraction(content: string): ExtractedOrder {
const unfenced = content.trim().replace(/^```(?:json)?\s*/i, "").replace(/\s*```$/, "");
const start = unfenced.indexOf("{");
const end = unfenced.lastIndexOf("}");
const candidate = start >= 0 && end > start ? unfenced.slice(start, end + 1) : unfenced;
const raw = JSON.parse(candidate) as Record<string, unknown>;
const items: ExtractedItem[] = Array.isArray(raw.items)
? (raw.items as Record<string, unknown>[])
.filter((i) => i && typeof i.description === "string")
.map((i) => ({
quantity: num(i.quantity),
description: String(i.description),
unit_price: num(i.unit_price),
line_total: num(i.line_total),
options: Array.isArray(i.options) ? i.options.map(String) : [],
}))
: [];
return {
merchant: typeof raw.merchant === "string" ? raw.merchant : null,
order_date: typeof raw.order_date === "string" ? raw.order_date : null,
total: num(raw.total),
subtotal: num(raw.subtotal),
currency: typeof raw.currency === "string" && raw.currency ? raw.currency : "AUD",
items,
adjustments: Array.isArray(raw.adjustments)
? (raw.adjustments as Record<string, unknown>[])
.filter((a) => a && typeof a.label === "string" && num(a.amount) !== null)
.map((a) => ({ label: String(a.label), amount: num(a.amount) as number }))
: [],
};
}
export type Extractor = (parts: InputPart[]) => Promise<ExtractedOrder>;
export class ExtractionError extends Error {}
/**
* LiteLLM, OpenAI-compatible, alias `cloud-budget` — the same alias pantry's
* capture uses, so a model swap is one config line in `litellm-config.yaml`.
* `temperature: 0`; `response_format` is the strict schema above.
*/
export function liteLLMExtractor(opts: {
baseUrl?: string;
key?: string;
model?: string;
fetchImpl?: typeof fetch;
} = {}): Extractor {
return async (parts) => {
const baseUrl = (opts.baseUrl ?? process.env.LITELLM_BASE_URL ?? "").replace(/\/$/, "");
const key = opts.key ?? process.env.LITELLM_KEY;
if (!baseUrl || !key) throw new ExtractionError("LITELLM_BASE_URL / LITELLM_KEY are not configured");
const doFetch = opts.fetchImpl ?? fetch;
const response = await doFetch(`${baseUrl}/v1/chat/completions`, {
method: "POST",
headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
body: JSON.stringify({
model: opts.model ?? "cloud-budget",
temperature: 0,
max_tokens: 16384,
response_format: EXTRACT_RESPONSE_FORMAT,
messages: [{ role: "user", content: buildMessageContent(parts) }],
}),
});
if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`);
const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] };
// The rule is: a finish_reason that is PRESENT must be `stop`. An absent
// one is accepted, deliberately — providers differ on whether they send the
// field, and failing every call on a provider that omits it would be a
// certain outage in exchange for a case that also requires the model to
// emit syntactically complete JSON. The harm a truncated reply does — a
// partial item list replacing a good one — is closed on the write side
// instead, by the guard in `routeFood`/`createFromBatch`.
//
// `length` is the common one: a truncated reply still parses once the JSON
// happens to close, quietly dropping the tail of a long receipt. Any other
// reason (`content_filter`, `tool_calls`) is a different reply shape that
// the strict schema never constrained.
const finish = raw.choices?.[0]?.finish_reason;
if (finish && finish !== "stop") {
throw new ExtractionError(
finish === "length"
? "The receipt was too long to read in one pass."
: `The model stopped before it finished (${finish}).`
);
}
const content = raw.choices?.[0]?.message?.content;
if (!content) throw new ExtractionError("LiteLLM returned no content");
try {
return parseExtraction(content);
} catch (e) {
throw new ExtractionError(`could not parse extraction: ${e instanceof Error ? e.message : String(e)}`);
}
};
}
src/lib/intake-files.ts
+51
View File
@@ -0,0 +1,51 @@
import { createHash } from "node:crypto";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import path from "node:path";
/**
* The attachment store: the original files of a batch, keyed by sha256.
*
* Two reasons it exists. A "Groceries or a meal?" answer arrives minutes after
* the upload, in a fresh request carrying only the batch id, and "groceries"
* means forwarding the ORIGINAL files to pantry — so they have to still be
* here. And plan §8: "the source file is retrievable from the attachment store
* by its sha256".
*
* Plain files under `INTAKE_FILE_DIR` (default `/app/data/intake`); a sidecar
* `.json` keeps the name and MIME type. No database column holds bytes.
*/
export type StoredFile = { name: string; type: string; bytes: Buffer; sha256: string };
export function intakeFileDir(): string {
return process.env.INTAKE_FILE_DIR || "/app/data/intake";
}
export function sha256Of(bytes: Buffer): string {
return createHash("sha256").update(bytes).digest("hex");
}
export async function storeFiles(files: StoredFile[]): Promise<void> {
const dir = intakeFileDir();
await mkdir(dir, { recursive: true });
for (const f of files) {
await writeFile(path.join(dir, `${f.sha256}.bin`), f.bytes);
await writeFile(path.join(dir, `${f.sha256}.json`), JSON.stringify({ name: f.name, type: f.type }));
}
}
/** The files that are still on disk, in the order asked for; missing ones are skipped. */
export async function loadFiles(shas: string[]): Promise<StoredFile[]> {
const dir = intakeFileDir();
const out: StoredFile[] = [];
for (const sha of shas) {
try {
const bytes = await readFile(path.join(dir, `${sha}.bin`));
const meta = JSON.parse(await readFile(path.join(dir, `${sha}.json`), "utf-8")) as { name?: string; type?: string };
out.push({ name: meta.name ?? `${sha}.bin`, type: meta.type ?? "application/octet-stream", bytes, sha256: sha });
} catch {
/* gone — the caller decides what that means */
}
}
return out;
}
src/lib/intake-pantry.ts
+76
View File
@@ -0,0 +1,76 @@
import type { StoredFile } from "@/lib/intake-files";
/**
* finance → pantry hand-off for a grocery upload (plan §11 C3).
*
* The ORIGINAL files go across, not the extracted items: pantry owns grocery
* extraction (canonical names, categories, staples) and its capture board is
* the review surface. What finance adds is the routing decision — which
* transaction the confirm should enrich, or that it should create one — carried
* in the `intake` field so pantry's confirm seam (C4) can suppress the ING-10
* create that would otherwise double-book an order the email already booked.
*/
export type PantryForwardArgs = {
files: StoredFile[];
batchId: string;
platform: string | null;
sharedBy: string | null;
financeTargetTransactionId: number | null;
financeAction: "enrich" | "create";
};
export type PantryForwardResult = { captureId: number; itemCount: number | null };
export class PantryDuplicateError extends Error {
constructor(
public readonly captureId: number | null,
public readonly reason: string
) {
super(reason);
}
}
export class PantryUnavailableError extends Error {}
export type PantryForward = (args: PantryForwardArgs) => Promise<PantryForwardResult>;
export function pantryForwarder(opts: { baseUrl?: string; token?: string; fetchImpl?: typeof fetch } = {}): PantryForward {
return async (args) => {
const baseUrl = (opts.baseUrl ?? process.env.PANTRY_BASE_URL ?? "").replace(/\/$/, "");
const token = opts.token ?? process.env.RECEIPT_INGEST_TOKEN;
if (!baseUrl || !token) throw new PantryUnavailableError("PANTRY_BASE_URL / RECEIPT_INGEST_TOKEN are not configured");
const doFetch = opts.fetchImpl ?? fetch;
const form = new FormData();
for (const f of args.files) {
form.append("image", new Blob([new Uint8Array(f.bytes)], { type: f.type }), f.name);
}
form.append("source", "receipt");
form.append(
"intake",
JSON.stringify({
batch_id: args.batchId,
platform: args.platform,
origin: "intake",
shared_by: args.sharedBy,
finance_target_transaction_id: args.financeTargetTransactionId,
finance_action: args.financeAction,
})
);
let response: Response;
try {
response = await doFetch(`${baseUrl}/api/capture`, { method: "POST", headers: { "x-ingest-token": token }, body: form });
} catch (e) {
throw new PantryUnavailableError(`pantry unreachable: ${e instanceof Error ? e.message : String(e)}`);
}
const body = (await response.json().catch(() => ({}))) as { captureId?: number; error?: string; items?: unknown[]; reason?: string };
if (response.status === 409) {
throw new PantryDuplicateError(typeof body.captureId === "number" ? body.captureId : null, body.reason ?? body.error ?? "already captured");
}
if (!response.ok) throw new PantryUnavailableError(`pantry returned ${response.status}: ${body.error ?? ""}`.trim());
if (typeof body.captureId !== "number") throw new PantryUnavailableError("pantry returned no captureId");
return { captureId: body.captureId, itemCount: Array.isArray(body.items) ? body.items.length : null };
};
}
src/lib/intake-router.ts
+647
View File
@@ -0,0 +1,647 @@
import { randomUUID } from "node:crypto";
import { queryRaw, queryRow, withTransaction } from "@/lib/db";
import { EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY } from "@/lib/analytics-sql";
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
import {
classifyInput,
merchantKey,
merchantMatches,
platformForSource,
reconcileItems,
resolveMerchantKind,
toLineItems,
validateHeader,
type ExtractedOrder,
type IntakeSource,
type MerchantKind,
type OrderHeader,
type StoredLineItem,
} from "@/lib/intake-rules";
import { extractPdfText, ExtractionError, type Extractor, type InputPart } from "@/lib/intake-extract";
import { storeFiles, type StoredFile } from "@/lib/intake-files";
import { PantryDuplicateError, PantryUnavailableError, type PantryForward } from "@/lib/intake-pantry";
import { replaceLineItems } from "@/lib/line-items";
import {
askCard,
candidatesCard,
createdCard,
duplicateCard,
enrichedCard,
errorCard,
forwardedCard,
heldCard,
needsHeaderCard,
unsupportedCard,
type Block,
type Candidate,
type CardInput,
} from "@/lib/intake-blocks";
/**
* The intake router (plan §11 C1): one upload, every applicable step.
*
* Order of operations matters and is the plan's:
* persist the batch → classify each file → ONE extraction call → header →
* merchant kind → duplicate check → reconciliation → match → act.
*
* The batch row is written before anything else so that a Slack tap minutes
* later has something to act on, and so a re-share is recognisable. Nothing
* else is written until the header is whole: a half-read upload produces a
* reply, not a row (plan §2).
*/
export const MAX_FILES = 6;
export type IntakeKind = "grocery" | "food" | "ask" | "held" | "unsupported";
export type IntakeStatus =
| "received"
| "enriched"
| "created"
| "candidates"
| "forwarded_to_pantry"
| "needs_header"
| "duplicate"
| "unsupported"
| "ask"
| "held"
| "error";
export type IntakeResult = {
kind: IntakeKind;
status: IntakeStatus;
batch_id: string;
transaction_id?: number;
capture_id?: number;
duplicate_of?: { batch_id: string; transaction_id: number | null; capture_id: number | null };
candidates?: Candidate[];
header?: OrderHeader | null;
items?: StoredLineItem[];
slack_blocks: Block[] | null;
text: string;
http_status: number;
};
export type IntakeDeps = { extractor: Extractor; pantry: PantryForward };
export type BatchRow = {
id: string;
source: IntakeSource;
shared_by: string | null;
file_sha256: string[];
header: (OrderHeader & { platform: string | null; subtotal?: number | null; currency?: string | null }) | null;
items: StoredLineItem[] | null;
kind: string | null;
status: IntakeStatus;
transaction_id: number | null;
capture_id: number | null;
};
// ─── batch persistence ──────────────────────────────────────────────────────
export async function loadBatch(id: string): Promise<BatchRow | null> {
return queryRow<BatchRow>(
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
FROM intake_batches WHERE id = $1::uuid`,
[id]
);
}
async function insertBatch(b: { id: string; source: IntakeSource; sharedBy: string | null; shas: string[] }) {
await queryRaw(
`INSERT INTO intake_batches (id, source, shared_by, file_sha256, status)
VALUES ($1::uuid, $2, $3, $4::text[], 'received')`,
[b.id, b.source, b.sharedBy, b.shas]
);
}
async function updateBatch(
id: string,
patch: {
header?: unknown;
items?: unknown;
kind?: string | null;
status: IntakeStatus;
transaction_id?: number | null;
capture_id?: number | null;
}
) {
await queryRaw(
`UPDATE intake_batches
SET header = COALESCE($2::jsonb, header),
items = COALESCE($3::jsonb, items),
kind = COALESCE($4, kind),
status = $5,
transaction_id = COALESCE($6, transaction_id),
capture_id = COALESCE($7, capture_id),
resolved_at = CASE WHEN $5 IN ('enriched','created','forwarded_to_pantry','duplicate') THEN now() ELSE resolved_at END
WHERE id = $1::uuid`,
[
id,
patch.header === undefined ? null : JSON.stringify(patch.header),
patch.items === undefined ? null : JSON.stringify(patch.items),
patch.kind ?? null,
patch.status,
patch.transaction_id ?? null,
patch.capture_id ?? null,
]
);
}
// ─── lookups ────────────────────────────────────────────────────────────────
export async function rememberedKind(merchant: string | null): Promise<"grocery" | "food" | null> {
const key = merchantKey(merchant);
if (!key) return null;
const row = await queryRow<{ kind: "grocery" | "food" }>(`SELECT kind FROM intake_merchant_kinds WHERE merchant_key = $1`, [key]);
return row?.kind ?? null;
}
export async function rememberKind(merchant: string | null, kind: "grocery" | "food"): Promise<void> {
const key = merchantKey(merchant);
if (!key) return;
await queryRaw(
`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ($1, $2)
ON CONFLICT (merchant_key) DO UPDATE SET kind = EXCLUDED.kind`,
[key, kind]
);
}
/** The owner a batch's rows belong to: the sharer's participant, else the default. */
export async function ownerForEmail(email: string | null): Promise<number> {
if (!email) return DEFAULT_OWNER_ID;
const row = await queryRow<{ id: number }>(`SELECT id FROM participants WHERE lower(email) = lower($1) LIMIT 1`, [email]);
return row?.id ?? DEFAULT_OWNER_ID;
}
/**
* A batch already resolved from the same files, or from the same
* (platform, merchant, date, total) header (plan §5, Codex #5). Only RESOLVED
* batches count: a `needs_header` share followed by a fuller one must go
* through, and a pending candidates card re-shared is harmless.
*/
export async function findDuplicateBatch(
exceptId: string,
shas: string[],
header: (OrderHeader & { platform: string | null }) | null
): Promise<BatchRow | null> {
const bySha = await queryRow<BatchRow>(
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
FROM intake_batches
WHERE id <> $1::uuid AND status IN ('enriched','created','forwarded_to_pantry')
AND file_sha256 && $2::text[]
ORDER BY created_at DESC LIMIT 1`,
[exceptId, shas]
);
if (bySha) return bySha;
if (!header || !header.merchant) return null;
const rows = await queryRaw<BatchRow>(
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
FROM intake_batches
WHERE id <> $1::uuid AND status IN ('enriched','created','forwarded_to_pantry')
AND header->>'order_date' = $2
AND (header->>'total')::numeric = $3::numeric
AND COALESCE(header->>'platform', '') = COALESCE($4, '')
ORDER BY created_at DESC`,
[exceptId, header.order_date, header.total, header.platform]
);
return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null;
}
type MatchRow = {
transaction_id: number;
description: string | null;
transaction_date: string;
amount: string;
merchant: string | null;
platform: string | null;
/** What `amount` is denominated in — a candidate row is not always dollars. */
currency: string;
/** The `expense_metadata` row the match resolved through — null when there is none. */
metadata_id: number | null;
/** Does that row already carry a receipt? */
has_line_items: boolean;
};
/**
* The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and
* date EXACT, never widened silently. The merchant compares normalised, with
* whole-word containment ("Coles" vs "Coles (Wyndham Vale)") — that is
* spelling, not widening. A unique hit is a match; two hits are an ambiguity
* and go to the candidates card like a miss.
*
* A NULL platform means "no platform", not "any platform". An in-store Coles
* receipt carries none, and `$3 IS NULL OR em.platform = $3` let it match an
* Uber Eats Coles order on the same day for the same money — and then overwrite
* that order's items with the till receipt's. The in-store side matches rows
* with no platform of their own; the delivery side still requires its own.
*/
export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> {
if (!header.merchant) return { match: null, ambiguous: false };
const rows = await queryRaw<MatchRow>(
`SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency,
COALESCE(jsonb_typeof(em.line_items) = 'array' AND jsonb_array_length(em.line_items) > 0, false) AS has_line_items
FROM transactions t
LEFT JOIN statements s ON s.id = t.statement_id
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date = $1::date
AND t.amount = $2::numeric
AND (($3::text IS NULL AND em.platform IS NULL) OR em.platform = $3)
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date, header.total, platform]
);
const hits = rows.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant));
if (hits.length === 1) return { match: hits[0], ambiguous: false };
return { match: null, ambiguous: hits.length > 1 };
}
/**
* Nearest candidates for the card: same merchant, ±3 days, any amount.
*
* Deliberately NOT filtered by platform. The card is a human choice, not a
* decision — the whole reason it is shown is that the exact key missed, so the
* platform is one of the things that may be wrong. A filter here narrows the
* list at exactly the moment it needs to widen, and the row the user was
* looking for is simply absent with nothing saying why.
*/
export async function findCandidates(header: OrderHeader): Promise<Candidate[]> {
if (!header.merchant) return [];
const rows = await queryRaw<MatchRow>(
`SELECT DISTINCT ON (t.id)
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency, false AS has_line_items
FROM transactions t
LEFT JOIN statements s ON s.id = t.statement_id
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
WHERE ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
[header.order_date]
);
return rows
.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant))
.sort((a, b) => {
const da = Math.abs(Date.parse(a.transaction_date) - Date.parse(header.order_date));
const db = Math.abs(Date.parse(b.transaction_date) - Date.parse(header.order_date));
if (da !== db) return da - db;
return Math.abs(Number(a.amount) - header.total) - Math.abs(Number(b.amount) - header.total);
})
.slice(0, 5)
.map((r) => ({ transaction_id: r.transaction_id, description: r.description, transaction_date: r.transaction_date, amount: r.amount, currency: r.currency }));
}
// ─── the two destinations ───────────────────────────────────────────────────
export type RouteContext = {
batchId: string;
source: IntakeSource;
sharedBy: string | null;
platform: string | null;
header: OrderHeader;
items: StoredLineItem[];
currency?: string | null;
};
const cardInput = (ctx: RouteContext): CardInput => ({
batchId: ctx.batchId,
merchant: ctx.header.merchant,
total: ctx.header.total,
orderDate: ctx.header.order_date,
platform: ctx.platform,
itemCount: ctx.items.length,
currency: ctx.currency,
});
const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({
kind,
status,
batch_id: ctx.batchId,
header: ctx.header,
items: ctx.items,
slack_blocks: card.blocks,
text: card.text,
http_status: 200,
...extra,
});
/**
* Are these items unverifiable — no line carrying any money at all?
*
* That is `reconcileItems`' `basis: "none"`: pasted text with no prices, or an
* extraction that read the names and none of the figures. The reconciliation
* check passes it, correctly, because there is nothing to check. What must not
* follow is replacing a receipt that IS itemised with one nothing verified.
*/
const unverifiable = (items: StoredLineItem[]) => items.every((i) => i.amount === null);
/**
* The hold, shared by EVERY path that enriches a matched transaction.
*
* It lived only in `routeFood` and `createFromBatch`'s re-match walked straight
* past it — the same upload destroyed the same receipt depending on which
* button reached it. Returns null when there is nothing to protect.
*/
async function holdRatherThanReplace(ctx: RouteContext, match: MatchRow): Promise<IntakeResult | null> {
if (!unverifiable(ctx.items) || !match.has_line_items) return null;
await updateBatch(ctx.batchId, { status: "held", items: ctx.items });
const card = heldCard(
"this upload carries no prices, so nothing checked it against the total — and that order already has its items",
"I left the existing items alone"
);
return finish(ctx, "held", "held", card, { transaction_id: match.transaction_id });
}
/** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */
export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
const { match } = await matchTransaction(ctx.header, ctx.platform);
if (match) {
const held = await holdRatherThanReplace(ctx, match);
if (held) return held;
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId, metadataId: match.metadata_id, currency: ctx.currency });
await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items });
return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
}
const candidates = await findCandidates(ctx.header);
await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items });
return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates });
}
/** Grocery: the original files go to pantry's board; finance rides along on its confirm (C3/C4). */
export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantry: PantryForward): Promise<IntakeResult> {
const { match } = await matchTransaction(ctx.header, ctx.platform);
try {
const r = await pantry({
files,
batchId: ctx.batchId,
platform: ctx.platform,
sharedBy: ctx.sharedBy,
financeTargetTransactionId: match?.transaction_id ?? null,
financeAction: match ? "enrich" : "create",
});
await updateBatch(ctx.batchId, { kind: "grocery", status: "forwarded_to_pantry", transaction_id: match?.transaction_id ?? null, capture_id: r.captureId, items: ctx.items });
const card = forwardedCard({ ...cardInput(ctx), itemCount: r.itemCount ?? ctx.items.length, captureId: r.captureId, enriching: !!match });
return finish(ctx, "grocery", "forwarded_to_pantry", card, { capture_id: r.captureId, transaction_id: match?.transaction_id });
} catch (e) {
if (e instanceof PantryDuplicateError) {
await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items });
return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, currency: ctx.currency, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), {
capture_id: e.captureId ?? undefined,
});
}
const detail = e instanceof PantryUnavailableError ? e.message : e instanceof Error ? e.message : String(e);
await updateBatch(ctx.batchId, { kind: "grocery", status: "error", items: ctx.items });
return { ...finish(ctx, "grocery", "error", errorCard(detail)), http_status: 502 };
}
}
/** Create the transaction ING-10-style: manual, `statement_id NULL`, owner set — only ever on an explicit tap. */
export async function createFromBatch(batch: BatchRow, kind: "grocery" | "food"): Promise<IntakeResult> {
if (!batch.header) throw new Error("batch has no header");
const header = batch.header;
const items = batch.items ?? [];
const ctx: RouteContext = {
batchId: batch.id,
source: batch.source,
sharedBy: batch.shared_by,
platform: header.platform,
header,
items,
currency: header.currency,
};
// The card was built minutes ago and the tap arrives now. In between, the
// email/order bridge books orders on its own schedule — so re-run the match
// rather than trusting the "no match" the card was drawn from. Creating a
// second row for an order the ledger already holds is the one outcome this
// whole slice exists to avoid.
const { match } = await matchTransaction(header, header.platform);
if (match) {
const held = await holdRatherThanReplace(ctx, match);
if (held) return held;
await replaceLineItems(match.transaction_id, items, { source: "intake", platform: header.platform, batchId: batch.id, metadataId: match.metadata_id, currency: header.currency });
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: match.transaction_id, items });
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
}
// A sibling batch of the same order — the PDF after the screenshot — may have
// been resolved since. `findDuplicateBatch` deliberately ignores unresolved
// batches, so two pending cards for one order both reach here; without this
// the second tap books the order twice.
const twin = await findDuplicateBatch(batch.id, batch.file_sha256, header);
if (twin) {
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: twin.transaction_id, capture_id: twin.capture_id });
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: twin.transaction_id, captureId: twin.capture_id, batchId: twin.id });
return finish(ctx, kind, "duplicate", card, {
transaction_id: twin.transaction_id ?? undefined,
capture_id: twin.capture_id ?? undefined,
duplicate_of: { batch_id: twin.id, transaction_id: twin.transaction_id, capture_id: twin.capture_id },
});
}
const ownerId = await ownerForEmail(batch.shared_by);
const merchant = header.merchant ?? "Unknown merchant";
// Not `orderDescription()`: it always prefixes "Order - ", which is right for
// a delivery receipt and wrong for the in-store till receipt this path also
// handles. Not `resolveCategory()` either — it re-derives grocery-vs-food
// from a merchant pattern, and `kind` is the answer the router already
// reached, remembered merchant and Slack tap included.
const description = header.platform ? `Order - ${merchant}` : merchant;
const category = kind === "grocery" ? "groceries" : "dining";
// `amount` is the figure printed on the receipt, in its own currency. No FX
// rate exists at intake, so a foreign order leaves `amount_aud` NULL rather
// than asserting a conversion — the same shape order-ingestion.ts writes.
const currency = header.currency ?? "AUD";
const isAud = currency === "AUD";
// Check and insert under one lock, on one connection.
//
// `source_ref = batch.id` makes a second press of THIS card a no-op, and
// ON CONFLICT makes that safe against a double tap. It cannot help two
// DIFFERENT batches for one purchase — their keys differ — and both taps run
// their match and duplicate checks before either inserts, so both can find
// nothing. The lock is keyed on the purchase, not the batch, which is the
// only thing the two share. It has to be transaction-scoped and therefore
// inside withTransaction: taken by a statement of its own it would be
// released before the next one ran (see db.ts).
const purchaseKey = [header.platform ?? "", merchantKey(merchant), header.order_date, header.total.toFixed(2)].join("|");
const outcome = await withTransaction(async (q) => {
await q.queryRaw(`SELECT pg_advisory_xact_lock(hashtextextended($1, 0))`, [purchaseKey]);
// Mine (a double tap) or another batch's (the race). The ORDER BY is what
// tells them apart when both exist.
const existing = await q.queryRow<{ id: number; source_ref: string }>(
`SELECT id, source_ref FROM transactions
WHERE source = 'intake' AND superseded_by_id IS NULL
AND (source_ref = $1
OR (transaction_date = $2::date AND amount = $3::numeric
AND lower(merchant_normalized) = lower($4)))
ORDER BY (source_ref = $1) DESC NULLS LAST, id
LIMIT 1`,
[batch.id, header.order_date, header.total, merchant]
);
if (existing) return { id: existing.id, mine: existing.source_ref === batch.id };
const row = await q.queryRow<{ id: number }>(
`INSERT INTO transactions (
transaction_date, description, amount, amount_aud, category, payment_method,
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref,
foreign_currency_amount, foreign_currency_code
) VALUES ($1::date, $2, $3::numeric, $4::numeric, $5, NULL, $6, $6, 'debit', $7, 'intake', $8, $9::numeric, $10)
ON CONFLICT (source, source_ref) WHERE source IS NOT NULL AND source_ref IS NOT NULL
DO NOTHING
RETURNING id`,
[header.order_date, description, header.total, isAud ? header.total : null, category, merchant, ownerId, batch.id, isAud ? null : header.total, isAud ? null : currency]
);
return { id: row!.id, mine: true };
});
// Another batch booked this purchase in the moment between the two taps. Say
// the same thing the sequential case says one second later, and write nothing.
if (!outcome.mine) {
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: outcome.id });
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: outcome.id, captureId: null, batchId: null });
return finish(ctx, kind, "duplicate", card, { transaction_id: outcome.id });
}
const txnId = outcome.id;
await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id, currency });
await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId });
return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId });
}
/** The card for a batch as it stands — what a second press of a resolved card gets back. */
export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } {
const h = batch.header;
const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0, currency: h?.currency };
if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id });
if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id });
return duplicateCard({ merchant: base.merchant, total: base.total, currency: base.currency, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id });
}
// ─── the router ─────────────────────────────────────────────────────────────
export type IntakeInput = { batchId?: string | null; source: IntakeSource; sharedBy: string | null; files: StoredFile[] };
export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<IntakeResult> {
const batchId = input.batchId || randomUUID();
const shas = input.files.map((f) => f.sha256);
const platform = platformForSource(input.source);
if (await loadBatch(batchId)) {
return { kind: "unsupported", status: "duplicate", batch_id: batchId, slack_blocks: null, text: `batch ${batchId} was already received`, http_status: 409 };
}
await insertBatch({ id: batchId, source: input.source, sharedBy: input.sharedBy, shas });
const fail = (kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, http = 200, extra: Partial<IntakeResult> = {}): IntakeResult => ({
kind,
status,
batch_id: batchId,
slack_blocks: card.blocks,
text: card.text,
http_status: http,
...extra,
});
// 1. classify each file and build the parts, in the order shared.
const parts: InputPart[] = [];
const usable: StoredFile[] = [];
for (const f of input.files) {
const kind = classifyInput(f);
if (kind === "pdf") {
let text = "";
try {
text = await extractPdfText(f.bytes);
} catch {
/* unreadable PDF — reported below */
}
if (!text) {
await updateBatch(batchId, { kind: "unsupported", status: "unsupported" });
return fail("unsupported", "unsupported", unsupportedCard(`${f.name} has no readable text — try a screenshot of it`));
}
parts.push({ kind: "text", text, name: f.name });
} else if (kind === "image") {
parts.push({ kind: "image", mime: f.type || "image/jpeg", bytes: f.bytes, name: f.name });
} else if (kind === "text") {
parts.push({ kind: "text", text: f.bytes.toString("utf-8"), name: f.name });
} else {
continue;
}
usable.push(f);
}
if (!parts.length) {
await updateBatch(batchId, { kind: "unsupported", status: "unsupported" });
return fail("unsupported", "unsupported", unsupportedCard());
}
// A re-share of the very same files, before spending a model call on it.
const dupBySha = await findDuplicateBatch(batchId, shas, null);
if (dupBySha) {
await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id });
return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, currency: dupBySha.header?.currency, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, {
kind: (dupBySha.kind as IntakeKind) ?? "food",
duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id },
});
}
// Keep the originals: a later "Groceries" tap forwards them to pantry.
try {
await storeFiles(usable);
} catch (e) {
console.warn("intake: could not store files", e instanceof Error ? e.message : e);
}
// 2. ONE extraction call for the whole batch.
let extracted: ExtractedOrder;
try {
extracted = await deps.extractor(parts);
} catch (e) {
const detail = e instanceof ExtractionError ? e.message : e instanceof Error ? e.message : String(e);
await updateBatch(batchId, { status: "error" });
return fail("held", "error", errorCard(detail), 502);
}
const items = toLineItems(extracted.items);
// 3. header.
const headerCheck = validateHeader(extracted);
if (!headerCheck.ok) {
await updateBatch(batchId, { header: { merchant: extracted.merchant, order_date: extracted.order_date, total: extracted.total, platform }, items, status: "needs_header" });
return fail("held", "needs_header", needsHeaderCard(), 200, { items });
}
const header = headerCheck.header;
const storedHeader = { ...header, platform, subtotal: extracted.subtotal ?? null, currency: extracted.currency ?? "AUD" };
// 4. merchant → grocery | food | ask.
const kind: MerchantKind = resolveMerchantKind(input.source, header.merchant, await rememberedKind(header.merchant));
// 5. the same order, already handled under other files.
const dup = await findDuplicateBatch(batchId, shas, storedHeader);
if (dup) {
await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id });
return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, currency: storedHeader.currency, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, {
header,
items,
duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id },
});
}
// 6. do the lines add up? (missing-screenshot check)
const recon = reconcileItems(extracted);
if (!recon.ok) {
await updateBatch(batchId, { header: storedHeader, items, kind, status: "held" });
return fail("held", "held", heldCard(recon.reason ?? "mismatch"), 200, { header, items });
}
await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" });
const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, currency: storedHeader.currency };
// 7. act.
if (kind === "ask") {
await updateBatch(batchId, { kind: "ask", status: "ask" });
return finish(ctx, "ask", "ask", askCard(cardInput(ctx)));
}
if (kind === "grocery") return routeGrocery(ctx, usable, deps.pantry);
return routeFood(ctx);
}
src/lib/intake-rules.ts
+252
View File
@@ -0,0 +1,252 @@
/**
* Intake router — the pure decisions.
*
* Everything here is a function of its arguments: what kind of input a file is,
* which platform a source tap implies, whether a merchant is a grocer, whether a
* header is complete, whether the items add up. Kept free of the database and the
* model so each rule can be unit-tested against the exact strings real receipts
* print. The orchestration that calls these lives in `intake-router.ts`.
*
* Plan: smarthome `plans/intake-router-slice-plan.md` §2, §3, §5, §11 C1.
*/
export const INTAKE_SOURCES = ["uber_eats", "doordash", "coles", "woolworths", "other"] as const;
export type IntakeSource = (typeof INTAKE_SOURCES)[number];
export function isIntakeSource(value: unknown): value is IntakeSource {
return typeof value === "string" && (INTAKE_SOURCES as readonly string[]).includes(value);
}
export type InputKind = "image" | "pdf" | "text" | "unsupported";
/** Input type picks the extractor; it is independent of source and merchant. */
export function classifyInput(file: { type?: string | null; name?: string | null }): InputKind {
const type = (file.type ?? "").toLowerCase();
const name = file.name ?? "";
if (type === "application/pdf" || /\.pdf$/i.test(name)) return "pdf";
if (type.startsWith("image/") || /\.(png|jpe?g|webp|heic|gif)$/i.test(name)) return "image";
if (type.startsWith("text/") || /\.(txt|md)$/i.test(name)) return "text";
return "unsupported";
}
/**
* The platform the source tap implies, in `expense_metadata.platform` vocabulary.
* Null for an in-store receipt or `other`: those rows carry no platform, so the
* match falls back to (merchant, date, total) with no platform constraint.
*/
export function platformForSource(source: IntakeSource): string | null {
if (source === "uber_eats") return "ubereats";
if (source === "doordash") return "doordash";
return null;
}
export const PLATFORM_LABEL: Record<string, string> = {
ubereats: "Uber Eats",
doordash: "DoorDash",
uber: "Uber",
};
export function platformLabel(platform: string | null): string {
return platform ? PLATFORM_LABEL[platform] ?? platform : "";
}
/**
* A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints
* `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; a
* word-boundary alternation on the normalised name covers both without a
* merchant master-data subsystem.
*
* The boundaries are load-bearing, not tidiness: a bare substring made
* `Bigals Pizza` a grocer, because "b-IGA-ls" contains `iga`. Short names in a
* substring list collide with ordinary words, and the collision reads as a
* correct classification.
*/
export const GROCERY_PATTERN = /\b(coles|woolworths|woolies|aldi|iga|foodworks|costco)\b/i;
/**
* The normalised merchant used for matching and for the remembered-kind table.
* Lower-case, punctuation dropped, whitespace collapsed — so `Coles (Wyndham Vale)`
* and `coles wyndham vale` are one key, while `Coles` stays distinct from both.
*/
export function merchantKey(name: string | null | undefined): string {
return (name ?? "")
.toLowerCase()
.replace(/[^\p{L}\p{N}]+/gu, " ")
.trim()
.replace(/\s+/g, " ");
}
export type MerchantKind = "grocery" | "food" | "ask";
const UNKNOWN_MERCHANT = /^(unknown|n ?a|none|null|undefined|merchant)$/;
/**
* Grocery-vs-food is decided by the merchant, not the platform — "Uber Eats"
* covers both a meal and a grocery run. The source tap is ground truth where it
* says so (Coles/Woolworths taps are grocers by definition); the pattern list
* decides the obvious delivery cases; the remembered answer covers the
* `Romeo's Food Hall`s; and `other` with nothing else to go on asks rather than
* silently falling through to "meal" (plan §2, D3 holding state).
*/
export function resolveMerchantKind(
source: IntakeSource,
merchant: string | null | undefined,
remembered: "grocery" | "food" | null = null
): MerchantKind {
if (source === "coles" || source === "woolworths") return "grocery";
const key = merchantKey(merchant);
if (!key || UNKNOWN_MERCHANT.test(key)) return "ask";
if (GROCERY_PATTERN.test(key)) return "grocery";
if (remembered) return remembered;
if (source === "other") return "ask";
return "food";
}
/** What the extractor returns — names verbatim, options kept (plan §3). */
export type ExtractedItem = {
quantity: number | null;
description: string;
unit_price?: number | null;
line_total?: number | null;
options?: string[] | null;
};
export type ExtractedOrder = {
merchant: string | null;
order_date: string | null;
total: number | null;
subtotal?: number | null;
currency?: string | null;
items: ExtractedItem[];
adjustments?: { label: string; amount: number }[] | null;
};
export type OrderHeader = { merchant: string | null; order_date: string; total: number };
/**
* "Missing the header/total shot → nothing is written." A header is the date and
* the total; the merchant may be absent and still route (it becomes an ask).
*/
export function validateHeader(
x: ExtractedOrder
): { ok: true; header: OrderHeader } | { ok: false; missing: string[] } {
const missing: string[] = [];
if (typeof x.order_date !== "string" || !/^\d{4}-\d{2}-\d{2}$/.test(x.order_date)) missing.push("date");
if (typeof x.total !== "number" || !Number.isFinite(x.total) || x.total <= 0) missing.push("total");
if (missing.length) return { ok: false, missing };
return {
ok: true,
header: {
merchant: typeof x.merchant === "string" && x.merchant.trim() ? x.merchant.trim() : null,
order_date: x.order_date as string,
total: round2(x.total as number),
},
};
}
export const round2 = (n: number) => Number(n.toFixed(2));
/** A line's money, as printed or derived; null when the receipt gave neither. */
export function itemAmount(item: ExtractedItem): number | null {
if (typeof item.line_total === "number" && Number.isFinite(item.line_total)) return round2(item.line_total);
if (typeof item.unit_price === "number" && Number.isFinite(item.unit_price)) {
return round2(item.unit_price * (item.quantity ?? 1));
}
return null;
}
export type Reconciliation = {
ok: boolean;
basis: "subtotal" | "total" | "none";
items_sum: number;
reference: number | null;
diff: number;
reason?: string;
};
/**
* The missing-screenshot check (plan §3, Codex #7): the lines must reconcile
* within $1 or 2%, else the batch is held.
*
* Reconciled to the printed ITEM SUBTOTAL when there is one, not the total — a
* delivery order's total is items plus fees minus promotions ($118.94 of items
* became $85.74 on the real Coles export), so against the total every delivery
* order would be held. The total stays the match key; the subtotal is what the
* items can be checked against. Without a printed subtotal the total is the only
* reference there is. A receipt whose lines carry no prices at all (pasted text)
* cannot be checked and is not held for it.
*/
export function reconcileItems(x: ExtractedOrder): Reconciliation {
const amounts = (x.items ?? []).map(itemAmount);
if (!x.items?.length) {
return { ok: false, basis: "none", items_sum: 0, reference: null, diff: 0, reason: "no items" };
}
if (amounts.every((a) => a === null)) {
return { ok: true, basis: "none", items_sum: 0, reference: null, diff: 0 };
}
const itemsSum = round2(amounts.reduce<number>((s, a) => s + (a ?? 0), 0));
const subtotal = typeof x.subtotal === "number" && Number.isFinite(x.subtotal) ? x.subtotal : null;
const basis: Reconciliation["basis"] = subtotal !== null ? "subtotal" : "total";
const reference = subtotal ?? (typeof x.total === "number" ? x.total : null);
if (reference === null) return { ok: false, basis, items_sum: itemsSum, reference: null, diff: 0, reason: "no total" };
const diff = round2(itemsSum - reference);
const tolerance = Math.max(1, 0.02 * reference);
const ok = Math.abs(diff) <= tolerance;
return ok
? { ok, basis, items_sum: itemsSum, reference, diff }
: { ok, basis, items_sum: itemsSum, reference, diff, reason: `items sum to ${itemsSum.toFixed(2)} but the ${basis} is ${reference.toFixed(2)}` };
}
/**
* The stored shape — the same keys the order lane writes and
* `components/order-details.tsx` renders (`qty`, `description`, `amount`,
* `options`). A burger called "Michael" is stored as "Michael".
*/
export type StoredLineItem = {
qty: number;
description: string;
amount: number | null;
unit_price?: number | null;
options?: string[];
};
export function toLineItems(items: ExtractedItem[]): StoredLineItem[] {
return (items ?? [])
.filter((i) => typeof i.description === "string" && i.description.trim())
.map((i) => {
const out: StoredLineItem = {
qty: typeof i.quantity === "number" && i.quantity > 0 ? i.quantity : 1,
description: i.description.trim(),
amount: itemAmount(i),
};
if (typeof i.unit_price === "number" && Number.isFinite(i.unit_price)) out.unit_price = round2(i.unit_price);
const options = (i.options ?? []).map((o) => String(o).trim()).filter(Boolean);
if (options.length) out.options = options;
return out;
});
}
/** Is `needle` a contiguous run of whole words inside `hay`? */
function containsWordRun(hay: string[], needle: string[]): boolean {
if (!needle.length || needle.length > hay.length) return false;
for (let i = 0; i + needle.length <= hay.length; i++) {
if (needle.every((w, j) => hay[i + j] === w)) return true;
}
return false;
}
/**
* Does the stored merchant name denote the extracted one? One name's words must
* appear as a contiguous run inside the other's — so `Coles` matches
* `Coles (Wyndham Vale)` and `Order - Coles (Wyndham Vale)`, which is spelling,
* not widening.
*
* Whole words, never a raw substring: `merchantKey("Bigals Pizza")` contains
* `iga`, so a pizza shop matched every IGA on the same date and total.
*/
export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean {
const a = merchantKey(stored).split(" ").filter(Boolean);
const b = merchantKey(extracted).split(" ").filter(Boolean);
if (!a.length || !b.length) return false;
return containsWordRun(a, b) || containsWordRun(b, a);
}
src/lib/line-items.ts
+200
View File
@@ -0,0 +1,200 @@
import { queryRow } from "@/lib/db";
/**
* Replace the itemised receipt behind a transaction (plan §11 C2).
*
* An idempotent UPDATE, never an INSERT of a second row: `expense_metadata.
* transaction_id` is UNIQUE and the order the email pipeline booked already has
* its row — with `line_items = []`, because the items sat behind an auth-gated
* link. Idempotence is also what lets pantry re-run it after every confirm,
* writing the board's current state (plan §5, Codex #3), and what lets undo
* restore `previous_line_items` through the same call.
*
* The row is found on `transaction_id = :id OR matched_transaction_id = :id` —
* a card-settled order has only the latter (CLAUDE.md trap). When the
* transaction has no row at all (a manual transaction, a statement line), one
* is created so the items have somewhere to live; a transaction is NEVER
* created here.
*
* `opts.metadataId` pins WHICH row when the caller already resolved one. A
* statement transaction can own its own `expense_metadata` row AND be the
* `matched_transaction_id` of a delivery row: the match query and this write
* then re-pick independently and can land on different rows, so the enrichment
* reports success against a row nobody reads.
*/
export type LineItemsSource = "intake" | "pantry";
export class TransactionNotFoundError extends Error {}
export class LineItemsValidationError extends Error {}
export type LineItemRecord = Record<string, unknown> & { description: string };
/**
* Permissive on purpose: the order lane writes `{qty, description, amount,
* options}`, the receipt lane `{description, qty, amount, unit, category}`, and
* the detail panel reads the overlap. What every producer must supply is a
* description — an item with no name renders as nothing.
*/
export function validateLineItems(input: unknown): LineItemRecord[] {
if (!Array.isArray(input)) throw new LineItemsValidationError("line_items must be an array");
return input.map((raw, i) => {
if (!raw || typeof raw !== "object" || Array.isArray(raw)) {
throw new LineItemsValidationError(`line_items[${i}] must be an object`);
}
const item = { ...(raw as Record<string, unknown>) };
if (typeof item.description !== "string" || !item.description.trim()) {
throw new LineItemsValidationError(`line_items[${i}].description is required`);
}
item.description = item.description.trim();
// `quantity` is the extractor's word; `qty` is what the renderer reads.
if (item.qty === undefined && typeof item.quantity === "number") item.qty = item.quantity;
return item as LineItemRecord;
});
}
export type ReplaceResult = {
transaction_id: number;
metadata_id: number;
created_metadata: boolean;
previous_line_items: unknown[];
line_items: LineItemRecord[];
};
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* The undo baseline, recorded once per batch.
*
* Returning the row that was just read makes a RETRY destroy the thing undo
* exists to restore: pantry re-calls this on every confirm and after a timeout,
* and the second call would hand back its own write as the "previous" state.
* The first snapshot a batch ever takes is the only one worth keeping, so it is
* written under COALESCE and read back rather than trusted.
*/
async function undoBaseline(batchId: string | null | undefined, snapshot: unknown[]): Promise<unknown[]> {
// Pantry passes back whatever batch id it was given; only an intake batch has
// a row to record against, and a non-uuid would fail the cast.
if (!batchId || !UUID.test(batchId)) return snapshot;
const row = await queryRow<{ previous_line_items: unknown }>(
`UPDATE intake_batches
SET previous_line_items = COALESCE(previous_line_items, $2::jsonb)
WHERE id = $1::uuid
RETURNING previous_line_items`,
[batchId, JSON.stringify(snapshot)]
);
return Array.isArray(row?.previous_line_items) ? row.previous_line_items : snapshot;
}
export async function replaceLineItems(
transactionId: number,
lineItems: LineItemRecord[],
opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null; metadataId?: number | null; currency?: string | null }
): Promise<ReplaceResult> {
const txn = await queryRow<{
id: number;
merchant_normalized: string | null;
merchant_name: string | null;
amount: string;
transaction_date: string;
}>(
`SELECT id, merchant_normalized, merchant_name, amount::text, transaction_date::text
FROM transactions WHERE id = $1`,
[transactionId]
);
if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`);
// The caller's row wins where it still exists; otherwise prefer the row that
// IS this transaction over one that merely points at it — a statement line
// can be the `matched_transaction_id` of a receipt row and carry its own row
// too, and the items belong on the receipt.
const pinned = opts.metadataId
? await queryRow<{ id: number; line_items: unknown }>(
`SELECT id, line_items FROM expense_metadata
WHERE id = $1 AND (transaction_id = $2 OR matched_transaction_id = $2)`,
[opts.metadataId, transactionId]
)
: null;
const json = JSON.stringify(lineItems);
/**
* Snapshot the baseline, then write. In that order on BOTH paths: the create
* path used to snapshot after its INSERT, so a crash in between left a retry
* recording the intake's own write as "previous" — undo restoring the thing
* it was meant to reverse, which is the failure `previous_line_items` exists
* to prevent.
*
* Returns null when the row is gone between the read and the write. That
* UPDATE reported success and the batch was marked enriched against a row
* nothing had written to.
*/
const write = async (row: { id: number; line_items: unknown }): Promise<ReplaceResult | null> => {
const previous = await undoBaseline(opts.batchId, Array.isArray(row.line_items) ? row.line_items : []);
const updated = await queryRow<{ id: number }>(
`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1 RETURNING id`,
[row.id, json]
);
if (!updated) return null;
return { transaction_id: transactionId, metadata_id: row.id, created_metadata: false, previous_line_items: previous, line_items: lineItems };
};
if (pinned) {
const done = await write(pinned);
if (done) return done;
}
const existing = await queryRow<{ id: number; line_items: unknown }>(
`SELECT id, line_items FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- NULLS LAST is load-bearing, and its absence inverted this clause.
-- Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at
-- the transaction has a NULL transaction_id — so the comparison meant to
-- prefer the owned row preferred the other one, silently and in exactly
-- the case it was written for. matchTransaction spells it out; this did
-- not, and the two therefore disagreed.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[transactionId]
);
if (existing) {
const done = await write(existing);
if (done) return done;
}
// Nothing to update. Snapshot the empty baseline BEFORE creating the row.
const previous = await undoBaseline(opts.batchId, []);
const reference = opts.batchId ? `intake:${opts.batchId}` : `intake:txn:${transactionId}`;
// ON CONFLICT because two calls for one transaction can both find no row and
// both INSERT; expense_metadata.transaction_id is UNIQUE, so one of them lost
// with a constraint error that reached the user as an error card. `xmax = 0`
// is how a plain insert is told from a conflict-update, so the reported
// `created_metadata` stays true.
const created = await queryRow<{ id: number; inserted: boolean }>(
`INSERT INTO expense_metadata (
transaction_id, source, platform, order_reference, line_items,
merchant_normalized, amount, transaction_date, currency, flags, extraction_model
) VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7::numeric, $8::date, $9, '[]'::jsonb, 'cloud-budget')
ON CONFLICT (transaction_id) DO UPDATE SET line_items = EXCLUDED.line_items
RETURNING id, (xmax = 0) AS inserted`,
[
transactionId,
opts.source,
opts.platform ?? null,
reference,
json,
txn.merchant_normalized ?? txn.merchant_name ?? null,
txn.amount,
txn.transaction_date,
// Not hard-coded 'AUD': the transaction may carry a foreign figure, and
// OrderDetails prefers this column, so a USD order printed "$".
opts.currency || "AUD",
]
);
return {
transaction_id: transactionId,
metadata_id: created!.id,
created_metadata: created!.inserted,
previous_line_items: previous,
line_items: lineItems,
};
}
src/lib/order-ingestion.ts
-7
View File
@@ -12,13 +12,6 @@ export const CUTOVER_DATE = "2026-01-09";
*/
export const DEFAULT_OWNER_ID = 1;
/** Human labels for the platform a receipt came from. */
export const PLATFORM_LABEL: Record<ParsedOrder["platform"], string> = {
doordash: "DoorDash",
ubereats: "Uber Eats",
uber: "Uber",
};
/**
* Transaction description.
*
src/lib/order-reviews.ts
+10
View File
@@ -119,6 +119,16 @@ export async function merchantForTransaction(
const row = await queryRow<{ merchant_normalized: string | null }>(
`SELECT merchant_normalized FROM expense_metadata
WHERE transaction_id = $1 OR matched_transaction_id = $1
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
LIMIT 1`,
[transactionId]
);
src/lib/partial-month.ts
+135
View File
@@ -0,0 +1,135 @@
/**
* Partial-month comparison rules (Priority 0, item 6).
*
* A month in progress is not comparable to a complete one, and getting that
* wrong is silent: nothing fails, the current month simply looks thrifty
* because most of it has not happened yet. Two distinct rules follow from it,
* and both are applied in three places on the analytics page:
*
* 1. **The in-progress month is never part of a baseline.** Averaging it in
* drags the "typical" figure down by however much of the month is still in
* the future.
* 2. **When the in-progress month is the one being looked at, every month it
* is compared against is truncated to the same day.** Its running total is
* measured against the same slice of each prior month, not their full
* totals.
*
* This module exists so those rules can be tested. They previously lived as
* closures inside `src/app/budget/page.tsx` reading `new Date()` directly, so
* "is this month in progress" depended on the wall clock at run time and no
* test could assert either rule without waiting for the calendar. Every
* function here takes `now` explicitly, defaulting to the real clock, which is
* the whole reason the behaviour is now guardable.
*
* Pair with `/api/analytics/daily`: rule 2 needs day-of-month granularity,
* which is why that route exists.
*/
/** `YYYY-MM` for a Date, in local time. */
export function monthStrOf(d: Date): string {
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
}
/** The month currently in progress. */
export function currentMonthStr(now: Date = new Date()): string {
return monthStrOf(now);
}
/** The calendar month before `m`, as `YYYY-MM`. */
export function prevMonth(m: string): string {
const [year, month] = m.split("-").map(Number);
return monthStrOf(new Date(year, month - 2, 1));
}
/** Total days in month `m`. */
export function daysInMonthOf(m: string): number {
const [year, month] = m.split("-").map(Number);
return new Date(year, month, 0).getDate();
}
/**
* How much of `m` has actually happened. A month in progress is complete only
* up to today; every earlier month is complete.
*/
export function elapsedDays(m: string, now: Date = new Date()): number {
return m === currentMonthStr(now) ? now.getDate() : daysInMonthOf(m);
}
/** Spend in a month from day 1 through `throughDay` inclusive. */
export function spendThrough(
days: Record<number, number> | undefined,
throughDay: number
): number {
if (!days) return 0;
let sum = 0;
for (const [d, v] of Object.entries(days)) {
if (Number(d) <= throughDay) sum += v;
}
return sum;
}
/**
* Whether `selectedMonth` is still in progress, and the day both sides of a
* comparison must be truncated to.
*
* For a complete month the day is its last, which makes `spendThrough` a no-op
* — so callers can use the same code path either way.
*/
export function comparisonBasis(
selectedMonth: string,
now: Date = new Date()
): { isPartial: boolean; compareDay: number } {
return {
isPartial: selectedMonth === currentMonthStr(now),
compareDay: elapsedDays(selectedMonth, now),
};
}
/**
* The months that may form a baseline for `selectedMonth`.
*
* Excludes the selected month itself, the month in progress (rule 1), and any
* month with no spend recorded — an empty month is absence of data, not a
* thrifty month, and averaging zeroes in understates the baseline.
*/
export function baselineMonths(
months: readonly string[],
selectedMonth: string,
totals: Record<string, { spent: number }>,
now: Date = new Date()
): string[] {
const current = currentMonthStr(now);
return months.filter(
(m) => m !== selectedMonth && m !== current && (totals[m]?.spent || 0) > 0
);
}
/**
* Mean spend across `baseline`, truncated to `compareDay` when the selected
* month is partial.
*
* `daily` is the sparse `{ [month]: { [day]: amount } }` map from
* `/api/analytics/daily`; `totals` supplies whole-month figures for the
* complete-month case.
*/
export function baselineAverage(
baseline: readonly string[],
{
isPartial,
compareDay,
daily,
totals,
}: {
isPartial: boolean;
compareDay: number;
daily: Record<string, Record<number, number>> | undefined;
totals: Record<string, { spent: number }>;
}
): number {
if (!baseline.length) return 0;
const sum = baseline.reduce(
(s, m) => s + (isPartial ? spendThrough(daily?.[m], compareDay) : totals[m]?.spent || 0),
0
);
return sum / baseline.length;
}
src/lib/queries.ts
+10
View File
@@ -367,6 +367,16 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
SELECT em.route, em.platform
FROM expense_metadata em
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
-- Ordered the same way as replaceLineItems (line-items.ts) and
-- matchTransaction (intake-router.ts), so every reader of this row
-- resolves the same one a write lands on. NULLS LAST because DESC is
-- NULLS FIRST and the row that merely POINTS at the transaction has a
-- NULL transaction_id.
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
-- pointing-only, and ZERO transactions resolving to more than one row by
-- either link — so this changes nothing today. It is here so the readers
-- still agree on the day multiplicity appears.
ORDER BY (em.transaction_id = t.id) DESC NULLS LAST, em.id
LIMIT 1
) order_ctx ON true
LEFT JOIN LATERAL (