Compare commits
43
Commits
9f1be74330
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c25c82932d | ||
|
|
a1c2eb736f | ||
|
|
069034951e | ||
|
|
e956f39ffc | ||
|
|
d75589e128 | ||
|
|
ce0bf0a775 | ||
|
|
a3e4e1102a | ||
|
|
22fde30b26 | ||
|
|
48c6b2395f | ||
|
|
5076f27f68 | ||
|
|
10cf0aaf21 | ||
|
|
2251060ac6 | ||
|
|
2c05f10aed | ||
|
|
9b59edcf48 | ||
|
|
3d0d75519a | ||
|
|
8d1875225f | ||
|
|
8e66e527ae | ||
|
|
f31f12799a | ||
|
|
5e355c7e85 | ||
|
|
669f9119b9 | ||
|
|
258fe3b617 | ||
|
|
0823244e4a | ||
|
|
d88ca2a24c | ||
|
|
91586ce4a4 | ||
|
|
4b09d4136f | ||
|
|
4f0f0dd285 | ||
|
|
b788b4e8fa | ||
|
|
b5569b8ccc | ||
|
|
d4ff68363a | ||
|
|
631e75e58f | ||
|
|
c25db5211b | ||
|
|
9a442b7355 | ||
|
|
f9638cc5e5 | ||
|
|
7424d13858 | ||
|
|
be3d10d4f1 | ||
|
|
a4e5f540cc | ||
|
|
32a47bf8ee | ||
|
|
c7ea516c34 | ||
|
|
db5c9a02a7 | ||
|
|
a538376638 | ||
|
|
b958d05287 | ||
|
|
eaf5d5ac66 | ||
|
|
fc9ffaba25 |
+12
-4
@@ -22,10 +22,18 @@ jobs:
|
||||
# the normal colour.
|
||||
- name: Generate Prisma client
|
||||
run: npx prisma generate
|
||||
# Advisory until the pre-existing lint debt is cleared (2026-07-19:
|
||||
# ~20 errors across budget/insights/shared pages) — then make blocking.
|
||||
- name: Lint (advisory)
|
||||
# Blocking since #79. It was advisory from 2026-07-19 while 23 errors sat
|
||||
# across the pages and the integration tests; those are now fixed, so a
|
||||
# new one must not be able to slip in behind a green tick. Warnings are
|
||||
# still allowed (eslint exits 0 on them) — there are 9 left, all
|
||||
# `no-unused-vars` or `exhaustive-deps` advisories, none of which should
|
||||
# stop a push.
|
||||
- name: Lint
|
||||
run: npm run lint
|
||||
continue-on-error: true
|
||||
# Typecheck is separate from `next build`: nothing else in CI compiles the
|
||||
# test files, so a type error in a fixture could only ever be found on a
|
||||
# developer's machine.
|
||||
- name: Typecheck
|
||||
run: npx tsc --noEmit
|
||||
- name: Unit tests
|
||||
run: npm test
|
||||
|
||||
@@ -13,10 +13,11 @@ Keep data flow consistent: API routes call query functions in `src/lib/queries.t
|
||||
- `npm ci` installs the locked dependency set (Node 22 is used in CI).
|
||||
- `npm run dev` starts the local Next.js development server.
|
||||
- `npm run build` creates a production build; `npm start` serves it.
|
||||
- `npm run lint` runs the Next.js ESLint configuration. Existing lint debt makes CI lint advisory, but new code should pass.
|
||||
- `npm run lint` runs the Next.js ESLint configuration. **Blocking in CI since #79** — the pre-existing debt is cleared, so it must stay at zero errors (warnings are still allowed).
|
||||
- `npx tsc --noEmit` typechecks, including the test files. Also blocking in CI.
|
||||
- `npm test` runs fast unit tests.
|
||||
- `npm run test:setup` prepares the PostgreSQL test database using `.env.test`.
|
||||
- `npm run test:integration` runs database-backed tests.
|
||||
- `npm run test:setup` creates/refreshes the `personal_test` database (run it once, and after any migration).
|
||||
- `npm run test:integration` runs database-backed tests. It re-resolves the DSN from the live container first (`scripts/resolve-test-db.sh`), so `.env.test` must never carry a hand-written host — see `CLAUDE.md` → Testing.
|
||||
- `npm run test:all` runs both test suites.
|
||||
|
||||
## Coding Style & Naming Conventions
|
||||
@@ -33,4 +34,4 @@ Add schema changes as the next numbered `prisma/migrations/NNNN_description/migr
|
||||
|
||||
## Commits & Pull Requests
|
||||
|
||||
History follows concise Conventional Commit-style subjects such as `feat(rules): preview rule changes`, `fix(trips): ...`, and `docs: ...`. Keep commits focused. Pull requests should explain behavior and data-model impact, link related issues, list validation commands, and include screenshots for UI changes. Ensure unit tests and the production build pass; call out any known lint warnings or migration steps.
|
||||
History follows concise Conventional Commit-style subjects such as `feat(rules): preview rule changes`, `fix(trips): ...`, and `docs: ...`. Keep commits focused. Pull requests should explain behavior and data-model impact, link related issues, list validation commands, and include screenshots for UI changes. Ensure lint, typecheck, unit tests and the production build pass; call out any migration steps.
|
||||
|
||||
@@ -59,6 +59,56 @@ docker exec postgres-personal psql -U personal -d personal
|
||||
docker logs finance -f
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
npm test # unit — pure logic, no database
|
||||
npm run test:integration # database-backed; resolves the DSN itself (see below)
|
||||
npm run test:all # both
|
||||
npm run test:setup # once, and after any migration: drops and rebuilds personal_test
|
||||
npm run lint # blocking in CI since #79 — 0 errors, warnings allowed
|
||||
npx tsc --noEmit # blocking in CI; nothing else compiles the test files
|
||||
```
|
||||
|
||||
**The integration DSN is resolved at run time — never hand-write a host into
|
||||
`.env.test`.** `scripts/resolve-test-db.sh` runs automatically as
|
||||
`pretest:integration` and rewrites that file from the live container before
|
||||
every run. It prefers the container's **published host port** (`docker port
|
||||
postgres-personal` → `127.0.0.1:5432`, published since the 2026-08-17 PG17
|
||||
migration) and falls back to the bridge IP only where that is absent. Set
|
||||
`TEST_DATABASE_URL` to override it entirely.
|
||||
|
||||
This replaces the old "refresh the container IP" instruction, which described
|
||||
the defect rather than a workaround (#161). `setup-test-db.sh` used to bake the
|
||||
bridge IP into `.env.test` once, and Docker reassigns those on recreation: by
|
||||
2026-08-21 the stored `172.22.0.18` belonged to **gitea**, and an earlier drift
|
||||
had pointed the suite at **postgres-pantry** — whose database it would have
|
||||
TRUNCATEd had the credentials happened to match. A loopback address cannot be
|
||||
reassigned to another container, which is why the published port wins.
|
||||
|
||||
**`test:setup` really rebuilds now (2026-10-03).** It used to replay the
|
||||
schema dump over the existing database, where every `CREATE TABLE` failed with
|
||||
"already exists" and the old shape survived — so no migration ever reached
|
||||
`personal_test`, and new-column code failed every test with a misleading
|
||||
`column does not exist`. It now `DROP DATABASE … WITH (FORCE)`s first, refusing
|
||||
any name not ending `_test`, and loads with `ON_ERROR_STOP`.
|
||||
|
||||
**Two guards, and both matter.** `helpers.ts::assertTestDatabase` refuses to
|
||||
truncate anything whose `current_database()` is not `personal_test`, and the
|
||||
suite runs `fileParallelism: false` because every file shares that one database
|
||||
and `resetDB()` TRUNCATEs it.
|
||||
|
||||
**A fresh checkout or worktree needs `npx prisma generate` first.** The client is
|
||||
generated into the gitignored `src/generated/prisma`, so without it every test
|
||||
that reaches `src/lib/db.ts` fails on a missing package — which reads as a code
|
||||
error rather than a setup one. CI does this explicitly for the same reason.
|
||||
|
||||
**Freeze the clock in any test whose subject is defined relative to "now".**
|
||||
`analytics-priority0.test.ts` uses `vi.useFakeTimers({ toFake: ["Date"] })` —
|
||||
only `Date`, because `pg` needs real timers for connection handling. Without it
|
||||
the fixtures drift out of the routes' rolling windows over time and the
|
||||
assertions quietly stop testing anything.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Key Files
|
||||
@@ -133,6 +183,215 @@ docker exec postgres-personal psql -U personal -d personal < prisma/migrations/<
|
||||
- `statements` — one row per billing period per bank account
|
||||
- `transactions` — line items; `statement_id` is nullable (NULL = manual entry); `reconciled_with_id` links a manual tx to its matched statement tx; `payment_method` (migration 0016, `credits` added by 0018) is `card | cash | bank_transfer | credits | other`, NULL = unknown, and the TypeScript mirror is `src/lib/payment-methods.ts`; `source` / `source_ref` / `source_account` (migration 0028) identify a row that came from an account feed rather than a statement or a hand entry — `source_ref` is the provider's own id and carries a partial unique index, which is the only thing making a re-import idempotent
|
||||
|
||||
### The change log (migration 0037)
|
||||
|
||||
`transaction_audit` is an **append-only** row-level record of every change to
|
||||
`transactions`, `transaction_splits`, `transaction_tags`, `transaction_overrides`
|
||||
and `expense_metadata`. Never `UPDATE` or `DELETE` from it. There is deliberately
|
||||
**no foreign key** to `transactions` — the log has to outlive the row it describes.
|
||||
|
||||
```sql
|
||||
-- everything that ever happened to one transaction
|
||||
SELECT changed_at, table_name, op, old_data, new_data, app_name
|
||||
FROM transaction_audit WHERE transaction_id = 6961 ORDER BY changed_at;
|
||||
|
||||
-- one logical edit, all the rows it touched (a reconcile moves 4 children at once)
|
||||
SELECT * FROM transaction_audit WHERE txid = <txid> ORDER BY id;
|
||||
```
|
||||
|
||||
**Why it exists.** On 2026-08-23 the n8n CC Statement Processor stamped
|
||||
`statement_id` onto seven statementless wallet/pantry rows without setting
|
||||
`reconciled_with_id`, double-counting $245.90 and hiding those rows from the
|
||||
reconcile queue (`WHERE statement_id IS NULL`). Answering *"was anything lost?"*
|
||||
took restoring the 02:30 `pg_dump` into a scratch database and diffing it column
|
||||
by column, because nothing recorded what changed.
|
||||
|
||||
**Why it is a trigger and not application code.** The ledger has at least five
|
||||
independent writers — this app (Prisma), n8n (raw SQL in a Postgres node),
|
||||
pantry-app, the ingestion-engine runner (psycopg), and ad-hoc `docker exec psql`.
|
||||
That bug was an n8n raw-SQL `UPDATE`; an app-level audit would not have seen it.
|
||||
|
||||
**Why not Postgres' own logs.** Checked live: `wal_level=replica` (WAL cannot be
|
||||
decoded into row changes), `wal_keep_size=0` and `archive_mode=off` (recycled
|
||||
almost immediately), `log_statement=none`, `track_commit_timestamp=off`, no
|
||||
`pgaudit`. Postgres records nothing usable and has no built-in audit or temporal
|
||||
tables.
|
||||
|
||||
Three details worth knowing before relying on it:
|
||||
- **`old_data`/`new_data` hold only the columns that actually changed** on an
|
||||
`UPDATE`. A whole-row copy would bury a one-column edit, and
|
||||
`expense_metadata.line_items` alone can be kilobytes.
|
||||
- **A no-op `UPDATE` writes nothing** (the upsert paths issue plenty).
|
||||
- **Attribution is partial and that is understood.** Every client connects as the
|
||||
same `personal` role, so `db_user` never discriminates. `app_name` is only as
|
||||
good as what each client sets; `client_addr` separates containers and is NULL
|
||||
for `docker exec` over the unix socket — itself a useful signal. `txid` is the
|
||||
reliable grouping key.
|
||||
|
||||
`TRUNCATE` does not fire row-level triggers, so the integration suite's table
|
||||
wipes produce no audit rows.
|
||||
|
||||
### Resolving a duplicate by hand (`POST /api/transactions/<id>/merge`)
|
||||
|
||||
`<id>` is the row that gets **hidden**; `other_id` is the one that **survives**.
|
||||
|
||||
| mode | requires | sets | for |
|
||||
|---|---|---|---|
|
||||
| `reconcile` | loser statementless, survivor a statement row | `reconciled_with_id` | the everyday case |
|
||||
| `supersede` | nothing | `superseded_by_id` | two rows on the same footing, e.g. a statement imported twice |
|
||||
|
||||
**There is no delete, deliberately.** Every child of `transactions` is
|
||||
`ON DELETE CASCADE`, so removing a row destroys the splits, tags and metadata
|
||||
that make it legible. Both modes keep the row and its children and merely stop
|
||||
it counting. Do not add a delete endpoint.
|
||||
|
||||
Both modes **move curation onto the survivor first**, through the single
|
||||
`moveCurationWithin()` in `lib/reconcile.ts` — because both hide the source row
|
||||
from every figure, and a split stranded on a hidden row stops counting in
|
||||
silence. That is also why the two callers share one implementation: the file
|
||||
header warns that a second copy would eventually move a different set.
|
||||
|
||||
Refusals (all deliberate, all tested in `merge-supersede.test.ts`): a row already
|
||||
reconciled or superseded on **either** side → 409 (re-pointing it strands the
|
||||
curation moved on the first pass); superseding under a row that is itself
|
||||
superseded → 409; a row that is not the caller's → 404. Ownership is checked with
|
||||
`COALESCE(t.owner_id, s.owner_id)` — a statement row carries `owner_id` on the
|
||||
statement, so checking `t.owner_id` alone rejects every statement row as "not
|
||||
found".
|
||||
|
||||
**Why it had to exist.** `superseded_by_id` was read everywhere (analytics and
|
||||
auto-reconcile both exclude it) but written **nowhere** in application code — set
|
||||
exactly once, by migration 0023. The reconcile queue was the only merge path, and
|
||||
it only offers candidates inside an amount band.
|
||||
|
||||
**The band is ±5%, not ±1% (2026-08-23).** A foreign card purchase is charged the
|
||||
notified amount **plus the card's foreign transaction fee** — 3% on the Westpac
|
||||
Altitude — so a wallet row and its statement line can never match exactly.
|
||||
`SP PEAKDESIGN` was notified at **$42.44** and charged **$43.71** (×1.03), which
|
||||
put the one row that *was* its twin outside the old band: the picker could not
|
||||
offer the only correct answer. Widening is safe because the list is capped at 5
|
||||
by the `rn` window and ordered by amount closeness, so a looser band changes
|
||||
*which* five appear, never how many, and a person confirms each. Measured on the
|
||||
live queue at the time of the change: candidate counts went **0 → 0** on all 12
|
||||
rows, i.e. no new noise. **The auto-matcher stays exact** — see `findAutoMatches()`.
|
||||
|
||||
### A charge that never posted (`POST /api/transactions/<id>/void`, migration 0040)
|
||||
|
||||
The third way a row stops counting, for a case with **no survivor**: a card
|
||||
notification is sent for an *authorisation*, and when the merchant releases the
|
||||
hold no statement line ever arrives. Txn 7951 — a $1,040.41 Airbnb hold on Amex,
|
||||
booking cancelled the next day — sat in the reconcile queue with nothing to
|
||||
match and counted as spend everywhere. `reconcile` needs a statement twin,
|
||||
`supersede` needs a survivor (and `transactions_no_self_supersede` blocks
|
||||
pointing a row at itself), so it gets its own columns: `voided_at` +
|
||||
`void_reason` (reason mandatory, by CHECK). Body `{reason}`; `DELETE` undoes it.
|
||||
UI: **Never posted** on every row of `/reconcile`.
|
||||
|
||||
- **In `EXCLUDE_RECONCILED_SOURCE`**, so every "count each purchase once" query
|
||||
drops it without being edited; the queue, `findAutoMatches`, both adoption
|
||||
helpers, annual charges and the manual reconcile POST filter it explicitly.
|
||||
- **Statementless only** — `transactions_void_statementless` enforces it in the
|
||||
database. A statement line is the bank's record of a posted charge; a wrong
|
||||
one is a supersede.
|
||||
- **Refused** on a reconciled row (it posted), a superseded row, and a row with a
|
||||
*settled* split (someone already paid their share — voiding would quietly turn
|
||||
that into a credit). `merge` refuses a voided row on either side.
|
||||
- **The wallet lane deliberately still SEES voided rows.** `alreadyIngested()` and
|
||||
the issuer/Wallet pairing in `wallet-ingestion.ts` do not filter `voided_at`:
|
||||
the voided row is the sink that stops a late twin notification, or the next
|
||||
`prompt_version` re-post, from booking the hold again. Same for the intake
|
||||
router's idempotency lookup. Do not "fix" those.
|
||||
- **Outside this repo, nothing filters it**: Hermes' `agent_ro` views
|
||||
(`cashflow_monthly` etc.) and ingestion-engine's `instalment_matcher` already
|
||||
ignore `superseded_by_id`/`reconciled_with_id` too, so a voided row is no worse
|
||||
there than any other hidden row.
|
||||
|
||||
### Upcoming annual charges (`GET /api/annual-charges`, migration 0038)
|
||||
|
||||
Card annual fees, loan package fees and annual subscriptions, derived from
|
||||
**payments alone** — independent of the email lane, because a bank announces an
|
||||
annual fee on the statement and nowhere else.
|
||||
|
||||
**`transaction_type = 'fee'` is the universe, not a description match.** The
|
||||
extractor already classifies fees structurally (131 rows), and *every* card and
|
||||
loan annual fee on this ledger is typed that way. The first version of this
|
||||
filtered `transaction_type = 'debit'` and returned **none** of them. The fees are
|
||||
not debits.
|
||||
|
||||
**Two signals, kept apart because they are not equally trustworthy:**
|
||||
|
||||
| confidence | basis |
|
||||
|---|---|
|
||||
| `confirmed` | typed a fee **and** repeated on an annual cadence |
|
||||
| `likely` | typed a fee, seen once, and *named* annual/card/package — the kind is certain, the cadence inferred |
|
||||
| `possible` | no fee wording; a merchant whose every gap is annual **and** whose amount is stable |
|
||||
|
||||
**A naive "two charges ~365 days apart" is useless** — measured on this ledger it
|
||||
ranks a bakery first with **202** such pairs, then a donation, Patreon, Coles.
|
||||
Frequency is the confound. Even "every gap is annual" admits a Target visited
|
||||
twice ($30, then $198). The `possible` tier therefore also requires amount
|
||||
stability (≤20%, which absorbs VicRoads' ~3%/yr CPI drift) and ≥2 observations.
|
||||
It still carries real false positives (H&M, Superdry) — that is what the label is
|
||||
for, and why it must never render like a bank fact.
|
||||
|
||||
**`expected_not_seen` is the more interesting status.** The estimate passed and
|
||||
nothing landed, which means the account stopped being imported, or closed, or the
|
||||
charge arrived under a different label. `account_days_stale` is the
|
||||
discriminator: Westpac at 0 days stale is trustworthy; ANZ at 392 is simply
|
||||
missing data. Never report these as upcoming.
|
||||
|
||||
**Account continuity (`account_links`, migration 0038).** One card can appear
|
||||
under two identities: Citi Prestige `5258…0253` (to 2025-11-13) became NAB MyCard
|
||||
`1015…6227` (from 2025-11-14), same 14th-to-13th cycle, no gap. Without the link
|
||||
the $700 fee reads as two dead one-offs instead of one due again ~Nov 2026. The
|
||||
table is **hand-asserted with evidence** (`source_documents` ids), not inferred —
|
||||
a contiguous-cycle heuristic would join two genuinely different accounts as
|
||||
readily as one renamed one. The resolver walks it forward recursively;
|
||||
`uq_account_links_from` forbids a fork and `account_links_no_self` a self-link, so
|
||||
it terminates. **Display the resolved current bank**, not the one that printed the
|
||||
last fee — naming the retired brand sends you to the wrong bank.
|
||||
|
||||
Known limitation: the merchant signal fragments on label drift. VicRoads rego was
|
||||
paid 2024/2025 as `VICROADS` and 2026 as `VICROADS ONLINE PAYM`, so the series
|
||||
splits and the 2025 estimate shows as `expected_not_seen`. Not fuzzy-matched on
|
||||
purpose — see the rejected pg_trgm merchant grouping.
|
||||
|
||||
### The wallet lane (`/api/wallet/ingest`, `lib/wallet-ingestion.ts`)
|
||||
|
||||
iOS notifications → `capture.wallet_events` → n8n **Wallet Interpret**
|
||||
(`CAp1s5cuzJk2mDaI`, local gemma, every 15 min) → this route. A row per tap,
|
||||
statementless, `source='wallet'`, `source_ref='evt:<id>'`; the statement later
|
||||
confirms it through the reconcile queue. Fixed 2026-10-03 after a month abroad
|
||||
(ticket #266); the rules below are each the fix for something that broke.
|
||||
|
||||
- **Storage convention differs from the order lane.** `amount` = `amount_aud` =
|
||||
a *provisional AUD* figure; the native figure is `foreign_currency_amount`.
|
||||
`NATIVE_CURRENCY` reads `amount_aud IS NOT NULL` as "amount is AUD" — without
|
||||
that an INR 524 tap displayed as "INR 9.42".
|
||||
- **Provisional rate:** ledger median over *statement* rows (≥5, carries the
|
||||
card's FX fee) → else ECB via Frankfurter (`rate_ecb`) → else `0` with
|
||||
`no_rate_reference`. Never the native figure as AUD (672006 VND once became
|
||||
A$672,006).
|
||||
- **One tap = two notifications** (issuer app + Wallet, 2–8 s apart), merged.
|
||||
A pair must be **≤10 minutes apart AND from different apps**
|
||||
(`expense_metadata.notified_at` / `notif_app`, migration 0039). The old
|
||||
amount-within-±1-day rule merged the UNICEF $50 into an unrelated $50 charge.
|
||||
- **The whole check→pair→adopt→insert runs under one advisory lock**
|
||||
(`hashtextextended('wallet-ingest',0)`), on the transaction's own connection.
|
||||
n8n posts a poll's items concurrently; without it 11 taps became 22 rows.
|
||||
Prisma cannot deserialise `void` — the lock SELECT needs `::text`.
|
||||
- **A merged half records `wallet_evt:<id>` in the survivor's flags**, so a
|
||||
replay is idempotent. Reconciled wallet rows carry **no** `expense_metadata`
|
||||
(reconcile moves it to the statement twin) — the flag check follows it there.
|
||||
- **Skipped, not booked:** account notices (`overdrawn`, `to avoid fees`, …) and
|
||||
transit-card drawdowns (`Your new/current balance is` — the Octopus *load* is
|
||||
the spend). The fixed-width descriptor parser runs on Westpac bodies only.
|
||||
- **Dated in Australia/Melbourne**, like the statements (UTC slicing filed 28 of
|
||||
134 rows a day early).
|
||||
- **Replay a few events by relabelling their interpretations**
|
||||
(`UPDATE capture.wallet_interpretations SET prompt_version='v1-<why>' WHERE …`),
|
||||
not by bumping `prompt_version`, which re-posts every event ever captured.
|
||||
|
||||
### Cash and reconciliation
|
||||
|
||||
`payment_method` in (`cash`, `credits`) excludes a transaction from
|
||||
@@ -579,6 +838,108 @@ and one tap must not flatten it. It says which case it refused on.
|
||||
Slack reaches this route through an n8n webhook, not directly — see the
|
||||
smarthome repo's CLAUDE.md and the `slack-interactive-via-n8n` memory.
|
||||
|
||||
### Intake router — one upload, every applicable step (2026-08-21)
|
||||
|
||||
`POST /api/intake` (token `INTAKE_TOKEN`, multipart: `source`, `shared_by`,
|
||||
`batch_id`, 1–6 `file`s) is the finance half of the share-sheet intake; design
|
||||
and the cross-repo contract are in smarthome `plans/intake-router-slice-plan.md`
|
||||
§11 (C1, C2, C5 live here; C3/C4 are pantry-app; C6 is n8n + compose).
|
||||
|
||||
- **Order of operations is the plan's and is load-bearing:** persist
|
||||
`intake_batches` row → classify each file → ONE LiteLLM call for the whole
|
||||
batch (`cloud-budget`, `temperature 0`, strict `json_schema`; PDFs go in as
|
||||
`unpdf` text, never pixels) → header `(merchant, date, total)` → merchant
|
||||
kind → duplicate check → reconciliation → match → act. `src/lib/intake-rules.ts`
|
||||
holds the pure decisions, `intake-router.ts` the orchestration.
|
||||
- **Match is `(platform, merchant, date, total)`, exact, never widened.** A
|
||||
miss or an ambiguity returns candidates (same merchant ±3 days) with
|
||||
`intake_use_candidate` / `intake_create` buttons; a transaction is created
|
||||
only on the explicit tap, ING-10-style (`statement_id NULL`, `owner_id` from
|
||||
`shared_by` → `participants.email`, else `DEFAULT_OWNER_ID`;
|
||||
`source='intake'`, `source_ref=batch_id` so a second press is a no-op —
|
||||
written as `ON CONFLICT … DO NOTHING`, because a SELECT-then-INSERT loses a
|
||||
double tap). A non-AUD receipt is booked the way `order-ingestion.ts` books
|
||||
one: `amount` native, `amount_aud` **NULL**, `foreign_currency_*` set — no FX
|
||||
rate exists at intake and inventing one is worse than admitting there is none.
|
||||
- **A NULL platform means "no platform", not "any platform".** An in-store tap
|
||||
(`coles`, `woolworths`, `other`) matches only rows whose `expense_metadata`
|
||||
carries no platform of its own. The wildcard reading let a till receipt match
|
||||
the Uber Eats order for the same shop on the same day and overwrite its items.
|
||||
The **candidates** query is the opposite and deliberately so: it has no
|
||||
platform filter at all, because the card is a human choice shown precisely
|
||||
because the exact key missed — the platform is one of the things that may be
|
||||
wrong, so the list must widen there, not narrow.
|
||||
- **The Create tap re-runs the match before it inserts**, and checks for a
|
||||
resolved sibling batch. The card is minutes old by the time it is pressed and
|
||||
the email/order bridge books orders on its own schedule; `findDuplicateBatch`
|
||||
ignores *unresolved* batches by design, so two pending cards for one order
|
||||
would otherwise each book it.
|
||||
- **One `expense_metadata` row, agreed on by every query that resolves one.** A
|
||||
transaction can own a row *and* be the `matched_transaction_id` of another;
|
||||
`matchTransaction` returns the row it resolved through and `replaceLineItems`
|
||||
writes to exactly that one. Every reader orders by
|
||||
`(transaction_id = $1) DESC **NULLS LAST**, id` — the match, the write, the
|
||||
order-detail GET, `getTransactions`' provenance LATERAL, `merchantForTransaction`
|
||||
and the Slack details modal. The `NULLS LAST` is load-bearing: Postgres orders
|
||||
`DESC` as `NULLS FIRST` and the row that merely *points* at the transaction has
|
||||
a NULL `transaction_id`, so the clause meant to prefer the owned row preferred
|
||||
the other one. Measured 2026-08-21: 667 rows, 147 pointing-only, **zero**
|
||||
transactions resolving to more than one row — the ordering changes nothing
|
||||
today and exists so the readers still agree when multiplicity appears.
|
||||
- **The create is guarded by a transaction-scoped advisory lock on the
|
||||
PURCHASE**, not the batch. `ON CONFLICT (source, source_ref)` handles a double
|
||||
tap on one card; two *different* batches for one purchase have different
|
||||
`source_ref`s and both clear their match and duplicate checks before either
|
||||
inserts. The lock has to live inside `withTransaction` (`src/lib/db.ts`):
|
||||
`queryRaw` is `$queryRawUnsafe` on a **pooled** client, so every call is its
|
||||
own implicit transaction and `pg_advisory_xact_lock` taken in a statement of
|
||||
its own is released before the next statement runs. The loser reports
|
||||
`duplicate` pointing at the winner's transaction — the same answer the
|
||||
sequential path gives.
|
||||
- **`replaceLineItems` snapshots the undo baseline BEFORE it writes, on both
|
||||
paths**, and treats an UPDATE that touched no row as a failure rather than a
|
||||
success. The metadata INSERT is `ON CONFLICT (transaction_id) DO UPDATE` and
|
||||
carries the batch's **currency** — a hard-coded `'AUD'` made `OrderDetails`
|
||||
print `$` on a USD order, because it prefers `expense_metadata.currency`.
|
||||
- **`intake_batches.previous_line_items` is the undo baseline, written once.**
|
||||
Returning whatever the write just read makes a pantry retry get its own write
|
||||
back as the "previous" state, so undo restores the thing it undid. Recorded
|
||||
under `COALESCE(previous_line_items, …)` and read back.
|
||||
- **Grocery never writes finance here.** The ORIGINAL files go to pantry
|
||||
`POST /api/capture` with `intake.finance_target_transaction_id` +
|
||||
`finance_action: enrich|create`; pantry's confirm calls
|
||||
`PUT /api/transactions/[id]/line-items` back (C4), which is why that route
|
||||
accepts `RECEIPT_INGEST_TOKEN` too. Files are kept under `INTAKE_FILE_DIR`
|
||||
keyed by sha256 so a later "Groceries" tap can still forward them.
|
||||
- **Reconciliation is against the printed item subtotal, not the total.** A
|
||||
delivery order's total is items + fees − promotions ($118.94 of Coles lines
|
||||
became $85.74), so checking lines against the total would hold every
|
||||
delivery order. Tolerance $1 or 2%; a failure is `held`, nothing written.
|
||||
- **Nothing that cannot be a receipt reaches the model, and the prompt names no
|
||||
real shop.** On 2026-08-22 an iOS Shortcut coerced a shared photo to text, so
|
||||
a batch arrived as the five characters `Image` — and the model returned a
|
||||
whole receipt: `Burgertory (Braybrook)`, two "The Pride" lines, $41.15. None
|
||||
of it existed; "Burgertory" was in `ORDER_ITEM_PROMPT` as the example of how
|
||||
to print a branch in parentheses. Only the absent date kept it out of the
|
||||
matcher, and the invented header and items were still stored on the batch as
|
||||
`needs_header`. Three layers now, in order: `textCouldBeReceipt()` rejects a
|
||||
TEXT part with fewer than two words, no digit, or under 40 characters with no
|
||||
money figure (`TOTAL $17.00` passes — length alone is *not* the test)
|
||||
**before** any model call; the prompt's merchant examples are
|
||||
`PROMPT_PLACEHOLDER_MERCHANTS` format placeholders and it is told to return
|
||||
nulls for a non-receipt; and `confabulationReason()` catches what the
|
||||
pre-check cannot see — a PDF that renders to junk — plus any answer echoing a
|
||||
placeholder. All three file `unsupported`, never `needs_header`: a
|
||||
confabulated header is not a partial one. Same shape as the ingestion
|
||||
engine's 2026-08-05 empty-body defect — **a stage that can be handed nothing
|
||||
must say so, because the model downstream always answers, and it answers out
|
||||
of its prompt.**
|
||||
- **Names stay verbatim.** Burgertory calls its burgers "Michael" and "Joel";
|
||||
the prompt forbids canonicalising and the stored shape is the order lane's
|
||||
(`qty`, `description`, `amount`, `options`) so `order-details.tsx` renders it.
|
||||
- Statuses beyond the contract list: `ask` (the "Groceries or a meal?" holding
|
||||
state), `held`, `error` (502 — model or pantry down; the batch row says so).
|
||||
|
||||
### The shared loan
|
||||
|
||||
The loan is a **separate ledger**, not a shared expense and not a settlement
|
||||
|
||||
@@ -282,6 +282,18 @@ All routes require authentication via `X-Forwarded-User` header (set by Traefik)
|
||||
| GET / POST | `/api/tags` | List or create tags |
|
||||
| PATCH / DELETE | `/api/tags/[id]` | Update or delete a tag |
|
||||
|
||||
**Machine routes** — shared-secret `x-ingest-token` header instead of `X-Forwarded-User`; each fails closed when its token is unset:
|
||||
|
||||
| Method | Route | Token | Description |
|
||||
|--------|-------|-------|-------------|
|
||||
| POST / PATCH | `/api/orders/ingest` | `ORDER_INGEST_TOKEN` | n8n order receipts → transactions (see smarthome CLAUDE.md "Order Processor") |
|
||||
| POST | `/api/receipts/ingest` | `RECEIPT_INGEST_TOKEN` | pantry-app confirmed grocery receipts → one transaction per tender leg (ING-10) |
|
||||
| POST | `/api/intake` | `INTAKE_TOKEN` | **Intake router** (2026-08-21): multipart `source` (`uber_eats\|doordash\|coles\|woolworths\|other`), `shared_by`, `batch_id` (uuid), 1–6 `file`s (image / PDF / text). Persists `intake_batches`, extracts once via LiteLLM `cloud-budget` (PDF → text via `unpdf`), matches `(platform, merchant, date, total)` exactly, then enriches the matched order's `line_items` (food), forwards the original files to pantry `POST /api/capture` (grocery), or replies with candidates / "Groceries or a meal?" / needs-header / held / duplicate. Returns `{kind, status, transaction_id?, capture_id?, candidates?, slack_blocks, text}` |
|
||||
| PUT | `/api/transactions/[id]/line-items` | `INTAKE_TOKEN` or `RECEIPT_INGEST_TOKEN` | Idempotent replace of `expense_metadata.line_items` for `transaction_id = :id OR matched_transaction_id = :id`; body `{line_items, source: 'intake'\|'pantry', batch_id?, platform?, replace: true}`; creates the metadata row if absent, never a transaction (404). Returns `{previous_line_items, line_items}` — pantry calls it on every confirm and with `previous_line_items` on undo |
|
||||
| POST | `/api/slack/interactive` | `ORDER_INGEST_TOKEN` + Slack signature | Order-nudge buttons and the intake taps (`intake_use_candidate`, `intake_create`, `intake_grocery`, `intake_meal`) |
|
||||
|
||||
Env for the intake router: `INTAKE_TOKEN`, `LITELLM_BASE_URL` + `LITELLM_KEY` (OpenAI-compatible, alias `cloud-budget`), `PANTRY_BASE_URL` (+ the existing `RECEIPT_INGEST_TOKEN` as pantry's machine auth), optional `INTAKE_FILE_DIR` (original files by sha256, default `/app/data/intake` — a later "Groceries" tap re-sends them) and `FINANCE_PUBLIC_URL` (the card's View link, default `https://finance.bosecamp.com`). Design: smarthome `plans/intake-router-slice-plan.md` §11.
|
||||
|
||||
---
|
||||
|
||||
## Ingestion Pipeline
|
||||
@@ -347,6 +359,8 @@ docker exec postgres-personal psql -U personal -d personal \
|
||||
| `0009_split_payments` | `split_payments` |
|
||||
| `0010_csv_import_reconcile` | `reconciled_with_id`, CSV import support |
|
||||
| `0011_trips` | `trips`, `trip_id` on `transaction_overrides` (catch-up — was applied directly) |
|
||||
| … | |
|
||||
| `0034_intake_batches` | `intake_batches` (one row per `/api/intake` upload, written before extraction) and `intake_merchant_kinds` (the remembered "Groceries or a meal?" answer per merchant) |
|
||||
|
||||
> `paperless_doc_id` on statements and the `uq_statements_paperless_doc_id` index were added directly (not tracked in a migration file).
|
||||
> `owner_id` on transactions and `statement_id` made nullable were applied directly (March 2026) to support manual transaction entry without a fake statement.
|
||||
|
||||
@@ -47,13 +47,23 @@ Also fixed while in here, both found by checking rather than by proposal:
|
||||
category-day, so the pace chart ended the month a few cents off its own
|
||||
headline. Both now carry 4dp and round once, at display.
|
||||
|
||||
Partially guarded by `src/__tests__/integration/analytics-sql.test.ts`, which
|
||||
covers the SQL fragments — `EXCLUDE_RECONCILED_SOURCE`, `NATIVE_CURRENCY` and
|
||||
`INVESTMENT_SIGNED` — and nothing else. **Items 2, 3 and 6 have no test.** Those
|
||||
are the three where a regression is silent rather than loud: the pace chart would
|
||||
simply go back to disagreeing with the headline printed directly above it, the
|
||||
fees figure back to growing forever, and a partial month back to being measured
|
||||
against complete ones — all without anything failing. Tracked as #155.
|
||||
Guarded in two files. `src/__tests__/integration/analytics-sql.test.ts` covers
|
||||
the SQL fragments — `EXCLUDE_RECONCILED_SOURCE`, `NATIVE_CURRENCY` and
|
||||
`INVESTMENT_SIGNED`. Since 2026-08-21 `analytics-priority0.test.ts` covers items
|
||||
2, 3 and 6 (#155): the three where a regression is silent rather than loud — the
|
||||
pace chart simply going back to disagreeing with the headline printed directly
|
||||
above it, the fees figure back to growing forever, and a partial month back to
|
||||
being measured against complete ones, none of which fails anything.
|
||||
|
||||
Two things were needed to make item 6 testable at all, and both are the point
|
||||
rather than incidental. The rules lived as closures inside `budget/page.tsx`
|
||||
reading `new Date()` inline, so they now live in `src/lib/partial-month.ts` with
|
||||
an injectable clock; and the tests freeze that clock, because a fixture defined
|
||||
against the real "now" drifts out of the routes' rolling windows and the
|
||||
assertion goes vacuous — the same class of silent failure, one level up. Item
|
||||
2's suite carries a placebo assertion for the same reason: parity between two
|
||||
queries that are identical by construction proves nothing unless some fixture
|
||||
can actually tell the old and new spend definitions apart.
|
||||
|
||||
### Landed after this doc, in the same family (2026-08-02)
|
||||
|
||||
|
||||
Generated
+19
-1
@@ -16,7 +16,8 @@
|
||||
"prisma": "^7.4.2",
|
||||
"react": "19.2.3",
|
||||
"react-dom": "19.2.3",
|
||||
"recharts": "^3.8.0"
|
||||
"recharts": "^3.8.0",
|
||||
"unpdf": "^1.8.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@tailwindcss/postcss": "^4",
|
||||
@@ -8494,6 +8495,23 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/unpdf": {
|
||||
"version": "1.8.1",
|
||||
"resolved": "https://registry.npmjs.org/unpdf/-/unpdf-1.8.1.tgz",
|
||||
"integrity": "sha512-xkURhy2SoGpOIH0a1gLHNkASPIQYonadDJs2AQwPEfUakafeD9EA1WTWWsaR++gfTCXJpV27W7tU1nXuk82UKQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=22"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@napi-rs/canvas": "^0.1.69 || ^1.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@napi-rs/canvas": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/unrs-resolver": {
|
||||
"version": "1.11.1",
|
||||
"resolved": "https://registry.npmjs.org/unrs-resolver/-/unrs-resolver-1.11.1.tgz",
|
||||
|
||||
+3
-1
@@ -10,6 +10,7 @@
|
||||
"test": "vitest run --config vitest.config.ts",
|
||||
"test:watch": "vitest --config vitest.config.ts",
|
||||
"test:setup": "bash scripts/setup-test-db.sh",
|
||||
"pretest:integration": "bash scripts/resolve-test-db.sh",
|
||||
"test:integration": "vitest run --config vitest.integration.config.ts",
|
||||
"test:all": "npm test && npm run test:integration"
|
||||
},
|
||||
@@ -22,7 +23,8 @@
|
||||
"prisma": "^7.4.2",
|
||||
"react": "19.2.3",
|
||||
"react-dom": "19.2.3",
|
||||
"recharts": "^3.8.0"
|
||||
"recharts": "^3.8.0",
|
||||
"unpdf": "^1.8.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@tailwindcss/postcss": "^4",
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
-- Intake router (plans/intake-router-slice-plan.md §11 C1, smarthome repo).
|
||||
--
|
||||
-- One upload point for receipts — a screenshot, a PDF export, a pasted body —
|
||||
-- that reads what it was given and does every step that applies: fill the
|
||||
-- empty `line_items` on the order the email pipeline already booked, hand a
|
||||
-- grocery receipt to pantry, or ask when it cannot tell.
|
||||
--
|
||||
-- Every batch is persisted BEFORE anything is extracted, for two reasons:
|
||||
-- 1. The Slack card it produces carries buttons ("Use this", "Create",
|
||||
-- "Groceries or a meal?") that are pressed minutes later in a fresh
|
||||
-- request with nothing but the batch id to go on.
|
||||
-- 2. A re-share of an already-handled order — a second crop, the PDF after
|
||||
-- the screenshot — must be recognised, and the only durable evidence of
|
||||
-- the first share is this row (`file_sha256` overlap, or the same
|
||||
-- (platform, merchant, date, total) header already resolved).
|
||||
--
|
||||
-- `id` is the uuid n8n mints per webhook call, so a retried delivery is the
|
||||
-- same batch rather than a second one. `header` holds the extracted
|
||||
-- (merchant, order_date, total, platform, ...) and `items` the verbatim item
|
||||
-- list, so a later tap can enrich or create without re-reading the files.
|
||||
--
|
||||
-- Idempotent: safe to re-run.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS intake_batches (
|
||||
id uuid PRIMARY KEY,
|
||||
source text NOT NULL,
|
||||
shared_by text,
|
||||
file_sha256 text[] NOT NULL DEFAULT '{}',
|
||||
header jsonb,
|
||||
items jsonb,
|
||||
kind text,
|
||||
status text NOT NULL,
|
||||
transaction_id integer,
|
||||
capture_id integer,
|
||||
-- The undo baseline: what `expense_metadata.line_items` held before this
|
||||
-- batch first replaced it. Recorded here rather than returned from the write,
|
||||
-- because pantry re-calls the enrich endpoint on every confirm and after a
|
||||
-- timeout — a caller that hands back whatever it just read gives a retry its
|
||||
-- OWN write as the "previous" state, and undo then restores the thing it was
|
||||
-- meant to reverse. Written under COALESCE, so only the first snapshot sticks.
|
||||
previous_line_items jsonb,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
resolved_at timestamptz
|
||||
);
|
||||
|
||||
-- For a database that already ran an earlier draft of this file.
|
||||
ALTER TABLE intake_batches ADD COLUMN IF NOT EXISTS previous_line_items jsonb;
|
||||
|
||||
COMMENT ON TABLE intake_batches IS
|
||||
'One row per upload batch through POST /api/intake. Persisted before '
|
||||
'extraction so Slack taps can act on it later and re-shares are detected. '
|
||||
'kind: grocery|food|ask|held|unsupported. status: received|enriched|created|'
|
||||
'candidates|forwarded_to_pantry|needs_header|duplicate|unsupported|held|error.';
|
||||
|
||||
-- Overlap lookups for the re-share check (`file_sha256 && $1`).
|
||||
CREATE INDEX IF NOT EXISTS intake_batches_file_sha256_gin
|
||||
ON intake_batches USING gin (file_sha256);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS intake_batches_transaction_id
|
||||
ON intake_batches (transaction_id) WHERE transaction_id IS NOT NULL;
|
||||
|
||||
-- The "Groceries or a meal?" answer, remembered per merchant (plan §2, Codex
|
||||
-- #6). A tiny answer table, not merchant master-data: the pattern list in code
|
||||
-- decides the obvious cases and this catches the `Romeo's Food Hall`s. Keyed on
|
||||
-- the normalised merchant name the router already computes for matching.
|
||||
CREATE TABLE IF NOT EXISTS intake_merchant_kinds (
|
||||
merchant_key text PRIMARY KEY,
|
||||
kind text NOT NULL CHECK (kind IN ('grocery', 'food')),
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
COMMENT ON TABLE intake_merchant_kinds IS
|
||||
'Per-merchant answer to "Groceries or a meal?" from the intake Slack card. '
|
||||
'Consulted after the grocery pattern list; an unknown merchant asks once.';
|
||||
@@ -0,0 +1,73 @@
|
||||
-- transaction_search: resolve expense_metadata by BOTH links, without fanning out.
|
||||
--
|
||||
-- THE DEFECT (measured 2026-08-22): the view joined only on
|
||||
-- `em.transaction_id = t.id`, so it silently omitted every card-settled order,
|
||||
-- which carries only `matched_transaction_id`. 148 transactions are affected --
|
||||
-- Glovo orders in Milan, Uber Eats, all `source='order-bridge'` -- and for each
|
||||
-- of them the view reports NULL payment_method / order_reference / line_items
|
||||
-- while the data sits in the table.
|
||||
--
|
||||
-- That is not cosmetic. This view is the agent read surface: Hermes'
|
||||
-- postgres-personal MCP selects payment_method and order_reference from it
|
||||
-- (personal-agent-gateway/mcp-servers/postgres-personal/queries.py), and
|
||||
-- home-dashboard reads it via dash_ro. CLAUDE.md states the rule the view broke:
|
||||
-- "That lookup must match transaction_id OR matched_transaction_id: a
|
||||
-- card-settled order has only the latter."
|
||||
--
|
||||
-- NOT a fan-out fix, contrary to a first reading. The original join CANNOT fan
|
||||
-- out: expense_metadata_transaction_id_key is UNIQUE on transaction_id, so it
|
||||
-- matched at most one row by construction. Sums taken over this view were never
|
||||
-- inflated.
|
||||
--
|
||||
-- But broadening the join is exactly what INTRODUCES that risk, which is why
|
||||
-- LATERAL is used rather than a second OR'd condition. transaction_id and
|
||||
-- matched_transaction_id are enforced by two SEPARATE unique indexes
|
||||
-- (expense_metadata_transaction_id_key, uq_expense_matched_txn), so one
|
||||
-- transaction can legally be pointed at by two DIFFERENT rows -- one via each
|
||||
-- link. Zero such transactions exist today; the LATERAL means that stays a
|
||||
-- non-event rather than a silent duplicate in the agent's spending totals.
|
||||
--
|
||||
-- Tie-break matches every other reader (getTransactions, replaceLineItems,
|
||||
-- matchTransaction, /api/transactions/[id]/order), so the row this view resolves
|
||||
-- is the row the UI shows.
|
||||
--
|
||||
-- Verified before applying: row count unchanged at 4760 (= count(transactions)),
|
||||
-- sum(amount) over debits unchanged at 1514412.89 -- amount comes from
|
||||
-- transactions, never from the metadata row -- and the only difference is the
|
||||
-- 148 rows that gain the metadata they always should have had.
|
||||
--
|
||||
-- CREATE OR REPLACE VIEW DROPS GRANTS. dash_ro (home-dashboard) and agent_ro
|
||||
-- (Hermes) both read this view and losing either breaks a surface with no error
|
||||
-- at replace time, so both are re-granted below.
|
||||
|
||||
CREATE OR REPLACE VIEW transaction_search AS
|
||||
SELECT t.id,
|
||||
t.transaction_date,
|
||||
t.description,
|
||||
COALESCE(t.merchant_normalized, t.merchant_name) AS merchant,
|
||||
t.category,
|
||||
t.transaction_type,
|
||||
COALESCE(t.amount_aud, t.amount) AS amount,
|
||||
t.foreign_currency_amount,
|
||||
t.foreign_currency_code,
|
||||
t.location,
|
||||
s.bank_name,
|
||||
s.card_name,
|
||||
s.account_number,
|
||||
s.statement_type,
|
||||
s.billing_end_date,
|
||||
em.payment_method,
|
||||
em.order_reference,
|
||||
em.line_items
|
||||
FROM transactions t
|
||||
LEFT JOIN statements s ON s.id = t.statement_id
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT em2.payment_method, em2.order_reference, em2.line_items
|
||||
FROM expense_metadata em2
|
||||
WHERE em2.transaction_id = t.id OR em2.matched_transaction_id = t.id
|
||||
ORDER BY (em2.transaction_id = t.id) DESC NULLS LAST, em2.id
|
||||
LIMIT 1
|
||||
) em ON true;
|
||||
|
||||
GRANT SELECT ON transaction_search TO dash_ro;
|
||||
GRANT SELECT ON transaction_search TO agent_ro;
|
||||
@@ -0,0 +1,73 @@
|
||||
-- Normalise foreign_currency_code to ISO-4217, and clear two sentinel values.
|
||||
--
|
||||
-- WHY THIS MATTERS: exact-code equality is the whole basis of FX reconciliation.
|
||||
-- ingestion-engine's order_transaction_bridge matches foreign orders with
|
||||
-- AND t.foreign_currency_code = %(ccy)s
|
||||
-- AND t.foreign_currency_amount = %(total)s
|
||||
-- so a row labelled 'BAHT' can never match a THB order, and a row labelled
|
||||
-- 'null' can never match anything. 226 of 1,632 coded rows (14%) were
|
||||
-- unmatchable for this reason alone, which silently degrades a shipped job.
|
||||
--
|
||||
-- THE SENTINELS. 187 rows carry the literal 4-character STRING 'null' rather
|
||||
-- than SQL NULL, and 42 of those also carry foreign_currency_amount = -1.00
|
||||
-- exactly. Neither is data: they are "no value" markers that survived a truthy
|
||||
-- guard. Root cause is in the CC statement processors' Parse node, which does
|
||||
-- foreign_currency_code: t.foreign_currency_code || null
|
||||
-- and a non-empty string is truthy, so a model that answers with the STRING
|
||||
-- "null" (rather than JSON null) passes straight through. Same for -1. Fixed
|
||||
-- workflow-side in the same change; this migration cleans what already landed.
|
||||
-- Not purely historical -- 155 rows in 2026-03, but also 8 in May, 3 in July
|
||||
-- and 21 in August 2026.
|
||||
--
|
||||
-- MAPPINGS WERE VERIFIED AGAINST IMPLIED RATE, NOT TRUSTED FROM THE LABEL,
|
||||
-- because a row whose rate contradicts its claimed currency is a broken ledger
|
||||
-- row rather than a mislabelled one (the trap order_transaction_bridge's rate
|
||||
-- guard exists for: 2 of its first 73 matches were broken rows, not bad
|
||||
-- matches). Median amount/foreign_currency_amount per raw code vs the ISO
|
||||
-- median over statement rows (n>=5):
|
||||
-- BAHT 0.0433 vs THB 0.0470 INDIAN RUPEE 0.0183 vs INR 0.0180
|
||||
-- U.A.E. DIRHAM 0.4287 vs AED 0.4184 US DOLLAR 1.4054 vs USD 1.5513
|
||||
-- NZ DOLLAR 0.8243 vs NZD 0.9311 EURO 1.6342 vs EUR 1.6964
|
||||
-- Every label agrees with its rate. foreign_currency_amount is untouched by the
|
||||
-- relabels -- only the code was ever wrong, so this is a relabel, never a
|
||||
-- recompute.
|
||||
--
|
||||
-- EXPECTED ROW COUNTS (verified pre-flight, includes superseded rows):
|
||||
-- code 'null' -> NULL 187 BAHT -> THB 13
|
||||
-- amount -1.00 -> NULL 42 U.A.E. DIRHAM -> AED 10
|
||||
-- NZ DOLLAR -> NZD 6 US DOLLAR -> USD 4
|
||||
-- INDIAN RUPEE -> INR 4 EURO -> EUR 2
|
||||
-- Pre-flight also confirmed: no -1.00 amount exists outside the 'null'-code
|
||||
-- rows, and no other non-ISO code exists.
|
||||
|
||||
BEGIN;
|
||||
|
||||
-- Sentinels first: the -1.00 amounts live only on 'null'-code rows, so clear
|
||||
-- them before that code is nulled and the set becomes unfindable.
|
||||
UPDATE transactions SET foreign_currency_amount = NULL
|
||||
WHERE foreign_currency_amount = -1.00 AND foreign_currency_code = 'null';
|
||||
|
||||
UPDATE transactions SET foreign_currency_code = NULL
|
||||
WHERE foreign_currency_code = 'null';
|
||||
|
||||
-- Word-form relabels.
|
||||
UPDATE transactions SET foreign_currency_code = 'THB' WHERE foreign_currency_code = 'BAHT';
|
||||
UPDATE transactions SET foreign_currency_code = 'AED' WHERE foreign_currency_code = 'U.A.E. DIRHAM';
|
||||
UPDATE transactions SET foreign_currency_code = 'NZD' WHERE foreign_currency_code = 'NZ DOLLAR';
|
||||
UPDATE transactions SET foreign_currency_code = 'USD' WHERE foreign_currency_code = 'US DOLLAR';
|
||||
UPDATE transactions SET foreign_currency_code = 'INR' WHERE foreign_currency_code = 'INDIAN RUPEE';
|
||||
UPDATE transactions SET foreign_currency_code = 'EUR' WHERE foreign_currency_code = 'EURO';
|
||||
|
||||
-- Postcondition: every remaining code is ISO-4217 shaped. Fails the whole
|
||||
-- transaction rather than leaving a partial normalisation behind.
|
||||
DO $$
|
||||
DECLARE bad int;
|
||||
BEGIN
|
||||
SELECT count(*) INTO bad FROM transactions
|
||||
WHERE foreign_currency_code IS NOT NULL AND foreign_currency_code !~ '^[A-Z]{3}$';
|
||||
IF bad <> 0 THEN
|
||||
RAISE EXCEPTION 'normalisation incomplete: % non-ISO codes remain', bad;
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
COMMIT;
|
||||
@@ -0,0 +1,152 @@
|
||||
-- Append-only change log for the ledger and its curation children.
|
||||
--
|
||||
-- WHY THIS EXISTS: on 2026-08-23 the CC Statement Processor's `Run Reconciliation`
|
||||
-- node stamped `statement_id` onto seven statementless wallet/pantry rows without
|
||||
-- setting `reconciled_with_id` or superseding the statement's own extracted line.
|
||||
-- The purchase counted twice, and because finance-app's reconcile queue is
|
||||
-- `WHERE statement_id IS NULL`, the stamp also hid those rows from the only thing
|
||||
-- that could repair them. Answering "was anything lost?" required restoring the
|
||||
-- 02:30 pg_dump into a scratch database and diffing it column by column, because
|
||||
-- NOTHING in this estate records what changed. That is the gap this closes.
|
||||
--
|
||||
-- WHY AT THE DATABASE AND NOT IN THE APP: the ledger has at least five independent
|
||||
-- writers -- finance-app (Prisma), n8n (raw SQL in a Postgres node), pantry-app,
|
||||
-- the ingestion-engine runner (psycopg), and ad-hoc `docker exec psql`. The 4a bug
|
||||
-- was an n8n raw-SQL UPDATE; an application-level audit would not have seen it at
|
||||
-- all. A trigger cannot be bypassed by choice of client.
|
||||
--
|
||||
-- WHY NOT POSTGRES' OWN LOGS: checked live on this instance -- wal_level=replica
|
||||
-- (so WAL cannot be logically decoded into row changes), wal_keep_size=0 and
|
||||
-- archive_mode=off (so WAL is recycled almost immediately), log_statement=none,
|
||||
-- logging_collector=off, track_commit_timestamp=off, and pgaudit is not installed.
|
||||
-- Postgres was recording nothing usable. It has no built-in audit or temporal
|
||||
-- tables; this always has to be built.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS transaction_audit (
|
||||
id bigserial PRIMARY KEY,
|
||||
changed_at timestamptz NOT NULL DEFAULT now(),
|
||||
-- Groups every row touched by one logical edit. `applyReconcileMatch` moves
|
||||
-- overrides, tags, splits and metadata in a single transaction; without this
|
||||
-- they read as four unrelated events.
|
||||
txid bigint NOT NULL DEFAULT txid_current(),
|
||||
table_name text NOT NULL,
|
||||
op text NOT NULL CHECK (op IN ('INSERT','UPDATE','DELETE')),
|
||||
-- Full primary key as an object, because transaction_tags is composite
|
||||
-- (transaction_id, tag_id) and the others are a bare `id`.
|
||||
row_key jsonb NOT NULL,
|
||||
-- Denormalised so "everything that ever happened to transaction N" is one
|
||||
-- index scan rather than a five-way union.
|
||||
transaction_id bigint,
|
||||
-- On UPDATE these hold ONLY the columns that actually changed. Storing whole
|
||||
-- rows would bury a one-column edit in forty unchanged fields, and
|
||||
-- expense_metadata.line_items alone can be kilobytes.
|
||||
old_data jsonb,
|
||||
new_data jsonb,
|
||||
-- Attribution is partial and that is understood: every client connects as the
|
||||
-- same `personal` role. app_name is only as good as what each client sets;
|
||||
-- client_addr distinguishes containers and is NULL for `docker exec` over the
|
||||
-- unix socket, which is itself a useful signal.
|
||||
db_user text NOT NULL DEFAULT current_user,
|
||||
app_name text,
|
||||
client_addr inet
|
||||
);
|
||||
|
||||
COMMENT ON TABLE transaction_audit IS
|
||||
'Append-only row-level change log for transactions + curation children. Never UPDATE or DELETE from this table.';
|
||||
|
||||
CREATE INDEX IF NOT EXISTS transaction_audit_txn_time
|
||||
ON transaction_audit (transaction_id, changed_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS transaction_audit_time
|
||||
ON transaction_audit (changed_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS transaction_audit_txid
|
||||
ON transaction_audit (txid);
|
||||
CREATE INDEX IF NOT EXISTS transaction_audit_table_time
|
||||
ON transaction_audit (table_name, changed_at DESC);
|
||||
|
||||
-- TG_ARGV[0]: comma-separated candidate columns for the owning transaction id;
|
||||
-- the first non-null wins (expense_metadata carries the id in
|
||||
-- transaction_id OR matched_transaction_id -- a card-settled order
|
||||
-- only ever has the latter).
|
||||
-- TG_ARGV[1]: comma-separated primary key columns.
|
||||
CREATE OR REPLACE FUNCTION log_transaction_change() RETURNS trigger
|
||||
LANGUAGE plpgsql SECURITY DEFINER
|
||||
SET search_path = public, pg_temp
|
||||
AS $fn$
|
||||
DECLARE
|
||||
v_full jsonb;
|
||||
v_old jsonb;
|
||||
v_new jsonb;
|
||||
v_key jsonb := '{}'::jsonb;
|
||||
v_txn bigint;
|
||||
k text;
|
||||
BEGIN
|
||||
IF TG_OP = 'INSERT' THEN
|
||||
v_new := to_jsonb(NEW);
|
||||
v_full := v_new;
|
||||
ELSIF TG_OP = 'DELETE' THEN
|
||||
v_old := to_jsonb(OLD);
|
||||
v_full := v_old;
|
||||
ELSE
|
||||
v_old := to_jsonb(OLD);
|
||||
v_new := to_jsonb(NEW);
|
||||
v_full := v_new;
|
||||
-- Reduce to the changed columns only. IS DISTINCT FROM so a NULL->NULL
|
||||
-- comparison counts as unchanged rather than evaluating to NULL.
|
||||
FOR k IN SELECT jsonb_object_keys(v_old) LOOP
|
||||
IF (v_old -> k) IS NOT DISTINCT FROM (v_new -> k) THEN
|
||||
v_old := v_old - k;
|
||||
v_new := v_new - k;
|
||||
END IF;
|
||||
END LOOP;
|
||||
-- A no-op UPDATE is noise, and finance-app's upserts issue plenty of them.
|
||||
IF v_old = '{}'::jsonb THEN
|
||||
RETURN NULL;
|
||||
END IF;
|
||||
END IF;
|
||||
|
||||
FOREACH k IN ARRAY string_to_array(TG_ARGV[1], ',') LOOP
|
||||
v_key := v_key || jsonb_build_object(k, v_full -> k);
|
||||
END LOOP;
|
||||
|
||||
FOREACH k IN ARRAY string_to_array(TG_ARGV[0], ',') LOOP
|
||||
IF v_txn IS NULL AND (v_full ->> k) IS NOT NULL THEN
|
||||
v_txn := (v_full ->> k)::bigint;
|
||||
END IF;
|
||||
END LOOP;
|
||||
|
||||
INSERT INTO transaction_audit
|
||||
(table_name, op, row_key, transaction_id, old_data, new_data, app_name, client_addr)
|
||||
VALUES
|
||||
(TG_TABLE_NAME, TG_OP, v_key, v_txn, v_old, v_new,
|
||||
NULLIF(current_setting('application_name', true), ''), inet_client_addr());
|
||||
|
||||
RETURN NULL; -- AFTER trigger; return value is ignored
|
||||
END
|
||||
$fn$;
|
||||
|
||||
-- AFTER, so a statement that raises never leaves a phantom audit row; the audit
|
||||
-- write is in the same transaction as the change and rolls back with it.
|
||||
DROP TRIGGER IF EXISTS trg_audit_transactions ON transactions;
|
||||
CREATE TRIGGER trg_audit_transactions
|
||||
AFTER INSERT OR UPDATE OR DELETE ON transactions
|
||||
FOR EACH ROW EXECUTE FUNCTION log_transaction_change('id', 'id');
|
||||
|
||||
DROP TRIGGER IF EXISTS trg_audit_transaction_splits ON transaction_splits;
|
||||
CREATE TRIGGER trg_audit_transaction_splits
|
||||
AFTER INSERT OR UPDATE OR DELETE ON transaction_splits
|
||||
FOR EACH ROW EXECUTE FUNCTION log_transaction_change('transaction_id', 'id');
|
||||
|
||||
DROP TRIGGER IF EXISTS trg_audit_transaction_tags ON transaction_tags;
|
||||
CREATE TRIGGER trg_audit_transaction_tags
|
||||
AFTER INSERT OR UPDATE OR DELETE ON transaction_tags
|
||||
FOR EACH ROW EXECUTE FUNCTION log_transaction_change('transaction_id', 'transaction_id,tag_id');
|
||||
|
||||
DROP TRIGGER IF EXISTS trg_audit_transaction_overrides ON transaction_overrides;
|
||||
CREATE TRIGGER trg_audit_transaction_overrides
|
||||
AFTER INSERT OR UPDATE OR DELETE ON transaction_overrides
|
||||
FOR EACH ROW EXECUTE FUNCTION log_transaction_change('transaction_id', 'id');
|
||||
|
||||
DROP TRIGGER IF EXISTS trg_audit_expense_metadata ON expense_metadata;
|
||||
CREATE TRIGGER trg_audit_expense_metadata
|
||||
AFTER INSERT OR UPDATE OR DELETE ON expense_metadata
|
||||
FOR EACH ROW EXECUTE FUNCTION log_transaction_change('transaction_id,matched_transaction_id', 'id');
|
||||
@@ -0,0 +1,76 @@
|
||||
-- Account continuity across a rebrand, reissue or issuer change.
|
||||
--
|
||||
-- WHY THIS EXISTS: one card can appear in `statements` under two completely
|
||||
-- different identities. The live case is the Citi Prestige card, which moved to
|
||||
-- NAB and was rebranded MyCard:
|
||||
--
|
||||
-- Citibank 5258073330340253 statements ... 2025-11-13
|
||||
-- NAB 1015001000116227 statements 2025-11-14 ...
|
||||
--
|
||||
-- Same 14th-to-13th billing cycle, no gap, one continuous product. Nothing in
|
||||
-- the schema linked them, so anything grouping by (bank_name, account) sees a
|
||||
-- dead account and a new one with no history. For the annual-fee detector that
|
||||
-- is not cosmetic: the $700 fee charged 2024-12-13 and 2025-11-13 reads as
|
||||
-- "stopped" when it is due again around November 2026 under the new name.
|
||||
--
|
||||
-- WHY A TABLE AND NOT AN INFERENCE: contiguous-cycle heuristics would join two
|
||||
-- genuinely different accounts as readily as one renamed one, and a wrong join
|
||||
-- merges two real fee series. There are only a handful of these events in a
|
||||
-- lifetime, each announced by mail, so each link is asserted once with the
|
||||
-- document that proves it. `evidence` is not decoration -- it is how a future
|
||||
-- reader checks the claim instead of trusting it.
|
||||
--
|
||||
-- NOT `account_owner_mappings`: that answers "whose account is this" and keys on
|
||||
-- the raw (bank_name, account_number). This answers "is this the same account as
|
||||
-- that one", and keys on the normalised form.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS account_links (
|
||||
id bigserial PRIMARY KEY,
|
||||
-- Normalised the same way statements.account_number_key is generated
|
||||
-- (non-alphanumerics stripped), so these join without per-call cleanup.
|
||||
-- Amex-style masks keep their X's: they record which digits were redacted.
|
||||
from_bank_name text NOT NULL,
|
||||
from_account_key text NOT NULL,
|
||||
to_bank_name text NOT NULL,
|
||||
to_account_key text NOT NULL,
|
||||
-- When the successor took over. The predecessor's last statement and the
|
||||
-- successor's first are the observable boundary; this is the announced date
|
||||
-- where mail gives one.
|
||||
effective_date date,
|
||||
reason text NOT NULL
|
||||
CHECK (reason IN ('rebrand', 'reissue', 'issuer_change', 'migration')),
|
||||
-- Provenance for the claim, e.g. 'source_documents:32919'. Free text because
|
||||
-- the evidence is not always a document -- sometimes it is a statement pair.
|
||||
evidence text,
|
||||
notes text,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
|
||||
-- One successor per predecessor: a chain, never a fork. A fork would make
|
||||
-- "the current identity of this account" ambiguous, and the resolver walks
|
||||
-- this forward assuming it is not.
|
||||
CONSTRAINT uq_account_links_from UNIQUE (from_bank_name, from_account_key),
|
||||
-- A link to itself would make that walk non-terminating.
|
||||
CONSTRAINT account_links_no_self CHECK (
|
||||
from_bank_name IS DISTINCT FROM to_bank_name
|
||||
OR from_account_key IS DISTINCT FROM to_account_key
|
||||
)
|
||||
);
|
||||
|
||||
COMMENT ON TABLE account_links IS
|
||||
'Predecessor -> successor account identity across rebrands/reissues. Hand-asserted, each row carrying the evidence for its claim.';
|
||||
|
||||
CREATE INDEX IF NOT EXISTS account_links_to
|
||||
ON account_links (to_bank_name, to_account_key);
|
||||
|
||||
-- The one link the mail record proves today.
|
||||
-- source_documents 32919 (2025-10-01) "your Citi Prestige Card account is moving to MyCard"
|
||||
-- source_documents 32618 (2025-11-10) "MyCard is coming soon"
|
||||
-- source_documents 32539 (2025-11-24) "The new MyCard App and MyCard Online are now live"
|
||||
-- Statement boundary: Citibank ends 2025-11-13, NAB begins 2025-11-14.
|
||||
INSERT INTO account_links
|
||||
(from_bank_name, from_account_key, to_bank_name, to_account_key, effective_date, reason, evidence, notes)
|
||||
VALUES
|
||||
('Citibank', '5258073330340253', 'NAB', '1015001000116227', '2025-11-14', 'issuer_change',
|
||||
'source_documents:32919,32618,32539',
|
||||
'Citi Prestige -> MyCard Prestige. Billing cycle (14th-13th) continues unbroken across the boundary.')
|
||||
ON CONFLICT (from_bank_name, from_account_key) DO NOTHING;
|
||||
@@ -0,0 +1,26 @@
|
||||
-- When, and from which app, a wallet notification arrived.
|
||||
--
|
||||
-- WHY: one tap emits two notifications 2-8 seconds apart -- the issuer app
|
||||
-- (card tail) and Apple Wallet (merchant, suburb) -- and wallet-ingestion.ts
|
||||
-- merges them. It used to recognise the pair by amount alone, within +/-1 DAY,
|
||||
-- which is wide enough to fold two different purchases together: on 2026-10-01
|
||||
-- an Amex "$50.00 at A MEDIUM" notification merged into the UNICEF $50 donation
|
||||
-- from 17 hours earlier and renamed it. A real pair is seconds apart and comes
|
||||
-- from two DIFFERENT apps; two notifications from one app are two charges.
|
||||
-- Neither fact was stored anywhere on the ledger side, so the matcher could not
|
||||
-- ask. These columns let it.
|
||||
--
|
||||
-- Backfilled from capture.wallet_events through source_message_id ('evt:<id>'),
|
||||
-- which every wallet-written metadata row carries.
|
||||
|
||||
ALTER TABLE expense_metadata
|
||||
ADD COLUMN IF NOT EXISTS notified_at timestamptz,
|
||||
ADD COLUMN IF NOT EXISTS notif_app text;
|
||||
|
||||
UPDATE expense_metadata em
|
||||
SET notified_at = e.received_at,
|
||||
notif_app = e.notif_app
|
||||
FROM capture.wallet_events e
|
||||
WHERE em.source = 'wallet'
|
||||
AND em.source_message_id = 'evt:' || e.id
|
||||
AND em.notified_at IS NULL;
|
||||
@@ -0,0 +1,43 @@
|
||||
-- A statementless row for a charge that never posted.
|
||||
--
|
||||
-- WHY: the wallet lane books every card notification as a purchase, and a card
|
||||
-- notification is sent for an AUTHORISATION, not a settlement. When the
|
||||
-- merchant releases the hold the bank never posts the charge, so no statement
|
||||
-- line ever arrives: the row sits in the reconcile queue forever and counts as
|
||||
-- spend in every figure. Txn 7951: Airbnb HMXAHAW49Y, $1,040.41 held on Amex
|
||||
-- 1000 on 2026-09-24, booking cancelled 25 Sep inside the free window, hold
|
||||
-- released ("We removed the $1,040.41 (AUD) payment hold"). Statement 219
|
||||
-- (11 Sep - 10 Oct) has no such line and reconciles to its header without it.
|
||||
--
|
||||
-- Neither existing exclusion fits. reconciled_with_id needs a statement twin
|
||||
-- and superseded_by_id needs a surviving row (and may not point at itself,
|
||||
-- transactions_no_self_supersede). There is no survivor: nothing happened.
|
||||
--
|
||||
-- Not a DELETE: every child of transactions cascades, and the row is also the
|
||||
-- sink the wallet lane checks before re-posting an event. Deleting it would let
|
||||
-- the next prompt_version bump re-create it.
|
||||
|
||||
ALTER TABLE transactions
|
||||
ADD COLUMN IF NOT EXISTS voided_at timestamptz,
|
||||
ADD COLUMN IF NOT EXISTS void_reason text;
|
||||
|
||||
COMMENT ON COLUMN transactions.voided_at IS
|
||||
'Set when this statementless row records a charge that never posted (e.g. a released authorisation hold). Excluded from every figure and from the reconcile queue, kept for its children and audit trail. NULL = live.';
|
||||
|
||||
-- A reason is mandatory: a void with no explanation is indistinguishable from
|
||||
-- a mistake six months later.
|
||||
ALTER TABLE transactions
|
||||
ADD CONSTRAINT transactions_void_has_reason
|
||||
CHECK ((voided_at IS NULL AND void_reason IS NULL)
|
||||
OR (voided_at IS NOT NULL AND btrim(coalesce(void_reason, '')) <> ''));
|
||||
|
||||
-- Statement rows cannot be voided. A statement line is the bank's own record
|
||||
-- of a posted charge, and the statement's rows must keep summing to its
|
||||
-- header; a wrong statement row is a supersede, not a void.
|
||||
ALTER TABLE transactions
|
||||
ADD CONSTRAINT transactions_void_statementless
|
||||
CHECK (voided_at IS NULL OR statement_id IS NULL);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_transactions_voided
|
||||
ON transactions (voided_at)
|
||||
WHERE voided_at IS NOT NULL;
|
||||
+33
-1
@@ -190,7 +190,9 @@ model transactions {
|
||||
owner_id Int?
|
||||
reconciled_with_id Int?
|
||||
superseded_by_id Int?
|
||||
principal_amount Decimal? @db.Decimal(12, 2)
|
||||
voided_at DateTime? @db.Timestamptz(6) // a charge that never posted (migration 0040)
|
||||
void_reason String?
|
||||
principal_amount Decimal? @db.Decimal(12, 2)
|
||||
interest_amount Decimal? @db.Decimal(12, 2)
|
||||
source String? // feed this row came from, e.g. "frollo"; NULL = statement/manual (migration 0028)
|
||||
source_ref String? // the provider's own transaction id — idempotency key for re-imports
|
||||
@@ -237,6 +239,9 @@ model expense_metadata {
|
||||
tender_raw String?
|
||||
receipt_sha256 String?
|
||||
receipt_group String?
|
||||
// 0039 — when/where a wallet notification arrived; the pair merge needs both.
|
||||
notified_at DateTime? @db.Timestamptz(6)
|
||||
notif_app String?
|
||||
transaction transactions? @relation(fields: [transaction_id], references: [id], onDelete: Cascade)
|
||||
|
||||
@@unique([source, order_reference], name: "uq_expense_source_order")
|
||||
@@ -268,3 +273,30 @@ model rule_apply_runs {
|
||||
reverted_at DateTime?
|
||||
snapshot Json @default("[]")
|
||||
}
|
||||
|
||||
// 0034 — intake router. One row per upload batch, written before extraction so
|
||||
// Slack taps can act on it later and a re-share is detected. Columns are the
|
||||
// §11 C1 contract in plans/intake-router-slice-plan.md (smarthome repo).
|
||||
model intake_batches {
|
||||
id String @id @db.Uuid
|
||||
source String
|
||||
shared_by String?
|
||||
file_sha256 String[] @default([])
|
||||
header Json?
|
||||
items Json?
|
||||
kind String?
|
||||
status String
|
||||
transaction_id Int?
|
||||
capture_id Int?
|
||||
// The undo baseline, written once per batch (see 0034 and line-items.ts).
|
||||
previous_line_items Json?
|
||||
created_at DateTime @default(now()) @db.Timestamptz(6)
|
||||
resolved_at DateTime? @db.Timestamptz(6)
|
||||
}
|
||||
|
||||
// 0034 — the remembered answer to "Groceries or a meal?" per merchant.
|
||||
model intake_merchant_kinds {
|
||||
merchant_key String @id
|
||||
kind String
|
||||
created_at DateTime @default(now()) @db.Timestamptz(6)
|
||||
}
|
||||
|
||||
Executable
+66
@@ -0,0 +1,66 @@
|
||||
#!/bin/bash
|
||||
# Resolve the integration-test DSN at run time and (re)write .env.test.
|
||||
#
|
||||
# Why this exists (#161): .env.test used to be written ONCE by
|
||||
# setup-test-db.sh with the Postgres container's bridge IP baked in. Container
|
||||
# IPs are reassigned on recreation, so the file rotted silently — by
|
||||
# 2026-08-21 its 172.22.0.18 belonged to *gitea*, and an earlier drift had the
|
||||
# suite pointing its TRUNCATE at postgres-pantry. It only failed safe because
|
||||
# the credentials happened not to match.
|
||||
#
|
||||
# The fix is to stop persisting a host at all: this runs as `pretest:integration`,
|
||||
# so the DSN is re-derived from the live container immediately before every run.
|
||||
#
|
||||
# Resolution order:
|
||||
# 1. An explicit TEST_DATABASE_URL in the environment wins (CI, remote DB).
|
||||
# 2. The container's published host port (`docker port`) — postgres-personal
|
||||
# publishes 127.0.0.1:5432 since the 2026-08-17 PG17 migration. Preferred
|
||||
# because a loopback address cannot be reassigned to another container.
|
||||
# 3. The container's bridge IP, for a host where the port is not published.
|
||||
#
|
||||
# helpers.ts::assertTestDatabase is the backstop: it refuses to TRUNCATE
|
||||
# anything whose current_database() is not `personal_test`.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
PG_CONTAINER=${PG_CONTAINER:-postgres-personal}
|
||||
PG_USER=${PG_USER:-personal}
|
||||
PG_PASS=${PG_PASS:-personalpassword123}
|
||||
TEST_DB=${TEST_DB:-personal_test}
|
||||
|
||||
ENV_FILE="$(cd "$(dirname "$0")/.." && pwd)/.env.test"
|
||||
|
||||
if [ -n "${TEST_DATABASE_URL:-}" ]; then
|
||||
DSN="$TEST_DATABASE_URL"
|
||||
SOURCE="TEST_DATABASE_URL"
|
||||
else
|
||||
if ! docker inspect "$PG_CONTAINER" >/dev/null 2>&1; then
|
||||
echo "ERROR: container '$PG_CONTAINER' not found. Start it, or set TEST_DATABASE_URL." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# `docker port` prints e.g. "127.0.0.1:5432" (possibly several lines).
|
||||
PUBLISHED=$(docker port "$PG_CONTAINER" 5432/tcp 2>/dev/null | head -1 || true)
|
||||
|
||||
if [ -n "$PUBLISHED" ]; then
|
||||
HOSTPORT="$PUBLISHED"
|
||||
SOURCE="published port"
|
||||
else
|
||||
IP=$(docker inspect "$PG_CONTAINER" \
|
||||
--format '{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' 2>/dev/null \
|
||||
| tr ' ' '\n' | grep -v '^$' | head -1)
|
||||
if [ -z "$IP" ]; then
|
||||
echo "ERROR: '$PG_CONTAINER' has neither a published 5432 nor a bridge IP." >&2
|
||||
exit 1
|
||||
fi
|
||||
HOSTPORT="$IP:5432"
|
||||
SOURCE="bridge IP"
|
||||
fi
|
||||
|
||||
DSN="postgresql://$PG_USER:$PG_PASS@$HOSTPORT/$TEST_DB"
|
||||
fi
|
||||
|
||||
printf 'DATABASE_URL=%s\n' "$DSN" > "$ENV_FILE"
|
||||
|
||||
# Never print the password.
|
||||
echo "resolve-test-db: ${DSN%%://*}://***@${DSN#*@} (via $SOURCE)"
|
||||
+28
-23
@@ -1,39 +1,44 @@
|
||||
#!/bin/bash
|
||||
# Creates the personal_test database and writes .env.test
|
||||
# Run once before integration tests: npm run test:setup
|
||||
# Creates the personal_test database and copies the production schema into it.
|
||||
# Run once (and after any migration) before integration tests: npm run test:setup
|
||||
#
|
||||
# It no longer writes a host into .env.test. That was the #161 defect: the file
|
||||
# was written once with the container's bridge IP and then rotted as the IP was
|
||||
# reassigned. scripts/resolve-test-db.sh now re-derives the DSN before every
|
||||
# integration run (wired as `pretest:integration`), and this script just calls it
|
||||
# at the end so a fresh setup leaves a usable .env.test behind.
|
||||
|
||||
set -e
|
||||
|
||||
PG_CONTAINER=postgres-personal
|
||||
PG_USER=personal
|
||||
PG_PASS=personalpassword123
|
||||
TEST_DB=personal_test
|
||||
PG_CONTAINER=${PG_CONTAINER:-postgres-personal}
|
||||
PG_USER=${PG_USER:-personal}
|
||||
TEST_DB=${TEST_DB:-personal_test}
|
||||
|
||||
# Discover the container's bridge IP (accessible from the host on Linux)
|
||||
PG_HOST=$(docker inspect "$PG_CONTAINER" \
|
||||
--format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 2>/dev/null | head -1)
|
||||
|
||||
if [ -z "$PG_HOST" ]; then
|
||||
if ! docker inspect "$PG_CONTAINER" >/dev/null 2>&1; then
|
||||
echo "ERROR: Could not find container '$PG_CONTAINER'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Postgres container at $PG_HOST:5432"
|
||||
# Drop and recreate, so the schema is rebuilt rather than replayed. Replaying
|
||||
# the dump over an existing database was a silent no-op for every migration:
|
||||
# each CREATE TABLE failed with "already exists" and the old shape survived, so
|
||||
# a new column (0039's expense_metadata.notified_at, 2026-10-03) never reached
|
||||
# the test database. The guard is because this now DROPs: it refuses anything
|
||||
# not named *_test, whatever TEST_DB was set to.
|
||||
case "$TEST_DB" in
|
||||
*_test) ;;
|
||||
*) echo "ERROR: refusing to drop '$TEST_DB' -- the test database must be named *_test"; exit 1 ;;
|
||||
esac
|
||||
docker exec "$PG_CONTAINER" psql -U "$PG_USER" -d postgres -q \
|
||||
-c "DROP DATABASE IF EXISTS $TEST_DB WITH (FORCE);" \
|
||||
-c "CREATE DATABASE $TEST_DB;"
|
||||
|
||||
# Create test database (ignore error if it already exists)
|
||||
docker exec "$PG_CONTAINER" psql -U "$PG_USER" -d postgres \
|
||||
-c "CREATE DATABASE $TEST_DB;" 2>/dev/null || true
|
||||
|
||||
# Wipe and rebuild schema from production (schema only, no data)
|
||||
echo "Copying schema from $PG_USER to $TEST_DB..."
|
||||
docker exec "$PG_CONTAINER" pg_dump -U "$PG_USER" --schema-only "$PG_USER" \
|
||||
| docker exec -i "$PG_CONTAINER" psql -U "$PG_USER" -d "$TEST_DB" -q
|
||||
| docker exec -i "$PG_CONTAINER" psql -U "$PG_USER" -d "$TEST_DB" -q -v ON_ERROR_STOP=1
|
||||
|
||||
echo "Schema ready."
|
||||
|
||||
# Write .env.test
|
||||
cat > "$(dirname "$0")/../.env.test" << EOF
|
||||
DATABASE_URL=postgresql://$PG_USER:$PG_PASS@$PG_HOST:5432/$TEST_DB
|
||||
EOF
|
||||
bash "$(dirname "$0")/resolve-test-db.sh"
|
||||
|
||||
echo ".env.test written — integration tests can now run with: npm run test:integration"
|
||||
echo "Integration tests can now run with: npm run test:integration"
|
||||
|
||||
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,531 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, afterAll, vi } from "vitest";
|
||||
import type { NextRequest } from "next/server";
|
||||
import { createPool, mockDbWithPool, resetDB } from "./helpers";
|
||||
import {
|
||||
comparisonBasis,
|
||||
baselineMonths,
|
||||
baselineAverage,
|
||||
spendThrough,
|
||||
prevMonth,
|
||||
} from "../../lib/partial-month";
|
||||
|
||||
/**
|
||||
* The three Priority 0 behaviours that shipped without a test (#155).
|
||||
*
|
||||
* `docs/ui-information-architecture-review.md` claimed Priority 0 was guarded;
|
||||
* `analytics-sql.test.ts` only ever covered the three SQL *fragments*
|
||||
* (EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY, INVESTMENT_SIGNED). Items 2, 3
|
||||
* and 6 of that list had nothing:
|
||||
*
|
||||
* 2. the spend-pace series must use the same spend semantics as the headline
|
||||
* printed directly above it;
|
||||
* 3. fees and interest must be bounded by an explicit, reported period;
|
||||
* 6. a month in progress must never be compared against complete ones.
|
||||
*
|
||||
* All three fail *silently* when they regress — no error, no empty screen, just
|
||||
* a number that quietly disagrees with the one beside it. That is precisely the
|
||||
* class the review flagged and the reason these are worth the fixture cost.
|
||||
*
|
||||
* The clock is frozen for every test in this file. Two of the three behaviours
|
||||
* are defined relative to "now" (which month is in progress, how far back the
|
||||
* default fees window reaches), so against the real clock the fixtures would
|
||||
* drift out of the routes' windows and the assertions would go vacuous — which
|
||||
* is the failure mode this file exists to prevent, one level up. Only `Date` is
|
||||
* faked: `pg` uses real timers for its connection handling.
|
||||
*/
|
||||
|
||||
const pool = createPool();
|
||||
mockDbWithPool(pool);
|
||||
|
||||
/** Frozen wall clock: the 15th, so the current month is genuinely half-done. */
|
||||
const NOW = new Date(2026, 7, 15, 10, 0, 0); // 2026-08-15, local time
|
||||
const CURRENT_MONTH = "2026-08";
|
||||
const ELAPSED = 15;
|
||||
|
||||
let monthlyGET: (req: NextRequest) => Promise<Response>;
|
||||
let dailyGET: (req: NextRequest) => Promise<Response>;
|
||||
let feesGET: (req: NextRequest) => Promise<Response>;
|
||||
let ownerId: number;
|
||||
|
||||
// One cast at the boundary: these are the two properties the analytics routes
|
||||
// read (`url` for the query string, `headers` for the auth header, which is
|
||||
// mocked out anyway), not a whole NextRequest.
|
||||
const req = (path: string) =>
|
||||
({ url: `http://localhost${path}`, headers: { get: () => null } }) as unknown as NextRequest;
|
||||
|
||||
const json = async (res: Response) => (await res.json()) as Record<string, unknown>;
|
||||
|
||||
beforeAll(async () => {
|
||||
vi.useFakeTimers({ toFake: ["Date"] });
|
||||
vi.setSystemTime(NOW);
|
||||
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
getCurrentUser: async () => ({ id: ownerId, name: "Owner", email: "o@x" }),
|
||||
}));
|
||||
({ GET: monthlyGET } = await import("../../app/api/analytics/monthly/route"));
|
||||
({ GET: dailyGET } = await import("../../app/api/analytics/daily/route"));
|
||||
({ GET: feesGET } = await import("../../app/api/analytics/fees/route"));
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
const a = await pool.query(`INSERT INTO participants (name) VALUES ('Owner') RETURNING id`);
|
||||
const b = await pool.query(`INSERT INTO participants (name) VALUES ('Other') RETURNING id`);
|
||||
ownerId = a.rows[0].id;
|
||||
otherId = b.rows[0].id;
|
||||
});
|
||||
|
||||
let otherId: number;
|
||||
|
||||
/** A manual transaction owned by `owner`. Returns its id. */
|
||||
async function txn(over: {
|
||||
date: string;
|
||||
amount: number;
|
||||
type?: string;
|
||||
category?: string;
|
||||
owner?: number;
|
||||
description?: string;
|
||||
interest_amount?: number;
|
||||
reconciled_with_id?: number;
|
||||
}): Promise<number> {
|
||||
const r = await pool.query(
|
||||
`INSERT INTO transactions
|
||||
(owner_id, statement_id, transaction_date, description, amount,
|
||||
transaction_type, category, interest_amount, reconciled_with_id, row_index)
|
||||
VALUES ($1, NULL, $2, $3, $4, $5, $6, $7, $8, 0) RETURNING id`,
|
||||
[
|
||||
over.owner ?? ownerId,
|
||||
over.date,
|
||||
over.description ?? `fixture ${over.date} ${over.amount}`,
|
||||
over.amount,
|
||||
over.type ?? "debit",
|
||||
over.category ?? "groceries",
|
||||
over.interest_amount ?? null,
|
||||
over.reconciled_with_id ?? null,
|
||||
]
|
||||
);
|
||||
return r.rows[0].id as number;
|
||||
}
|
||||
|
||||
async function split(txnId: number, participantId: number, pct: number) {
|
||||
await pool.query(
|
||||
`INSERT INTO transaction_splits (transaction_id, participant_id, share_percent)
|
||||
VALUES ($1, $2, $3)`,
|
||||
[txnId, participantId, pct]
|
||||
);
|
||||
}
|
||||
|
||||
async function tag(txnId: number, name: string) {
|
||||
const t = await pool.query(
|
||||
`INSERT INTO tags (name) VALUES ($1) ON CONFLICT (name) DO UPDATE SET name = EXCLUDED.name RETURNING id`,
|
||||
[name]
|
||||
);
|
||||
await pool.query(
|
||||
`INSERT INTO transaction_tags (transaction_id, tag_id) VALUES ($1, $2)`,
|
||||
[txnId, t.rows[0].id]
|
||||
);
|
||||
}
|
||||
|
||||
// ───────────────────────────────────────────────────────────────────────────
|
||||
// Item 2 — the pace chart and the headline must agree
|
||||
// ───────────────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* The pace chart used to sum gross `amount_aud ?? amount` over
|
||||
* `transaction_type = 'debit'` in the browser. Measured on live data it ended
|
||||
* July at $4,747.31 against a headline of $3,597.10 — a 32% overstatement of
|
||||
* the number printed directly above it, with both labelled "spend".
|
||||
*
|
||||
* `/api/analytics/daily` fixed that by building the series from the same
|
||||
* fragments as `/monthly`. Nothing has been asserting it since, and the two
|
||||
* routes are separate files: the drift they were merged to prevent can
|
||||
* reappear the moment one is edited alone.
|
||||
*
|
||||
* Every fixture below is a shape where the naive gross-debit sum and the real
|
||||
* spend definition disagree. The last test proves that — without it, parity
|
||||
* could hold simply because the fixtures were too plain to discriminate, and
|
||||
* the test would pass against a route that had regressed.
|
||||
*/
|
||||
describe("Priority 0 item 2 — spend pace matches the headline", () => {
|
||||
/** Everything the two routes are supposed to treat identically. */
|
||||
async function seedDivergentMonth(month: string) {
|
||||
const d = (day: number) => `${month}-${String(day).padStart(2, "0")}`;
|
||||
|
||||
// Plain debit — both definitions agree on this one.
|
||||
await txn({ date: d(2), amount: 100, category: "groceries" });
|
||||
|
||||
// Half mine. Gross says 200; my share is 100.
|
||||
const shared = await txn({ date: d(3), amount: 200, category: "groceries" });
|
||||
await split(shared, ownerId, 50);
|
||||
await split(shared, otherId, 50);
|
||||
|
||||
// A refund. Ignored entirely by a debit-only sum; must cancel spend here.
|
||||
await txn({ date: d(5), amount: 40, type: "refund", category: "groceries" });
|
||||
|
||||
// Fee and interest rows: spend, but not type 'debit'.
|
||||
await txn({ date: d(6), amount: 15, type: "fee", category: "fees" });
|
||||
await txn({ date: d(7), amount: 25, type: "interest", category: "fees" });
|
||||
|
||||
// An itemised loan repayment: type 'payment', so a debit-only sum skips it,
|
||||
// but its interest portion is real spend and its principal is not.
|
||||
await txn({
|
||||
date: d(8),
|
||||
amount: 1200,
|
||||
type: "payment",
|
||||
category: "loan",
|
||||
interest_amount: 300,
|
||||
});
|
||||
|
||||
// The manual row a statement line superseded — double count if counted.
|
||||
const survivor = await txn({ date: d(9), amount: 60, category: "dining" });
|
||||
await txn({ date: d(9), amount: 60, category: "dining", reconciled_with_id: survivor });
|
||||
|
||||
// Categories that are not spend at all.
|
||||
await txn({ date: d(10), amount: 5000, category: "transfers" });
|
||||
await txn({ date: d(11), amount: 2000, category: "investment" });
|
||||
await txn({ date: d(12), amount: 3000, type: "credit", category: "income" });
|
||||
|
||||
// A `family`-tagged row: excluded from budget spend by EXCLUDE_NON_BUDGET_TAGS.
|
||||
const family = await txn({ date: d(13), amount: 500, category: "groceries" });
|
||||
await tag(family, "family");
|
||||
|
||||
// Someone else's row I hold a split on — mine only to the extent of the split.
|
||||
const theirs = await txn({ date: d(14), amount: 80, category: "dining", owner: otherId });
|
||||
await split(theirs, ownerId, 25);
|
||||
await split(theirs, otherId, 75);
|
||||
}
|
||||
|
||||
/** Sum the sparse daily map for one month. */
|
||||
const sumDaily = (daily: Record<string, Record<number, number>>, month: string) =>
|
||||
Object.values(daily[month] ?? {}).reduce((s, v) => s + v, 0);
|
||||
|
||||
it("the daily series sums to the monthly headline, to the cent", async () => {
|
||||
await seedDivergentMonth("2026-06");
|
||||
|
||||
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
|
||||
const daily = await json(await dailyGET(req("/api/analytics/daily?months=6")));
|
||||
|
||||
const headline = (monthly.totals as Record<string, { spent: number }>)["2026-06"].spent;
|
||||
const pace = sumDaily(daily.daily as Record<string, Record<number, number>>, "2026-06");
|
||||
|
||||
// Cents, not exact float equality: the two routes group at different grains
|
||||
// (month+category vs month+day+category) and both carry 4dp, so they can
|
||||
// differ below a cent by construction. The displayed grain is what has to
|
||||
// agree — that IS the bug, the chart ending the month a few cents off the
|
||||
// headline it sits under.
|
||||
expect(Math.round(pace * 100) / 100).toBe(headline);
|
||||
});
|
||||
|
||||
it("agrees per category, not just in total", async () => {
|
||||
// A total can match while two categories are wrong in opposite directions,
|
||||
// and the category drill-down is a real surface on this page.
|
||||
await seedDivergentMonth("2026-06");
|
||||
|
||||
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
|
||||
const daily = await json(await dailyGET(req("/api/analytics/daily?months=6")));
|
||||
|
||||
const byCategory = (daily.byCategory as Record<string, Record<string, Record<number, number>>>)["2026-06"];
|
||||
const rows = monthly.rows as { category: string; spent: Record<string, number> }[];
|
||||
|
||||
expect(Object.keys(byCategory).sort()).toEqual(
|
||||
rows.filter((r) => r.spent["2026-06"] !== undefined).map((r) => r.category).sort()
|
||||
);
|
||||
|
||||
for (const r of rows) {
|
||||
const expected = r.spent["2026-06"];
|
||||
if (expected === undefined) continue;
|
||||
const got = Object.values(byCategory[r.category] ?? {}).reduce((s, v) => s + v, 0);
|
||||
expect([r.category, Math.round(got * 100) / 100]).toEqual([
|
||||
r.category,
|
||||
Math.round(expected * 100) / 100,
|
||||
]);
|
||||
}
|
||||
});
|
||||
|
||||
it("holds across every month in the window, not only the seeded one", async () => {
|
||||
await seedDivergentMonth("2026-05");
|
||||
await seedDivergentMonth("2026-06");
|
||||
await seedDivergentMonth("2026-07");
|
||||
|
||||
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
|
||||
const daily = await json(await dailyGET(req("/api/analytics/daily?months=6")));
|
||||
const totals = monthly.totals as Record<string, { spent: number }>;
|
||||
const dailyMap = daily.daily as Record<string, Record<number, number>>;
|
||||
|
||||
for (const m of monthly.months as string[]) {
|
||||
expect([m, Math.round(sumDaily(dailyMap, m) * 100) / 100]).toEqual([m, totals[m].spent]);
|
||||
}
|
||||
});
|
||||
|
||||
it("the fixtures actually discriminate — a gross debit sum does NOT agree", async () => {
|
||||
// The placebo. Parity between two identical-by-construction queries is easy
|
||||
// to assert and worthless if no fixture can tell the definitions apart.
|
||||
// This pins the gap the old client-side series had.
|
||||
await seedDivergentMonth("2026-06");
|
||||
|
||||
const monthly = await json(await monthlyGET(req("/api/analytics/monthly?months=6")));
|
||||
const headline = (monthly.totals as Record<string, { spent: number }>)["2026-06"].spent;
|
||||
|
||||
const naive = await pool.query<{ total: string }>(
|
||||
`SELECT COALESCE(SUM(COALESCE(amount_aud, amount)), 0)::numeric(14,2) AS total
|
||||
FROM transactions
|
||||
WHERE transaction_type = 'debit'
|
||||
AND transaction_date >= '2026-06-01' AND transaction_date < '2026-07-01'`
|
||||
);
|
||||
|
||||
expect(Number(naive.rows[0].total)).toBeGreaterThan(headline);
|
||||
});
|
||||
});
|
||||
|
||||
// ───────────────────────────────────────────────────────────────────────────
|
||||
// Item 3 — fees and interest are bounded by an explicit, reported period
|
||||
// ───────────────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* `/api/analytics/fees` used to aggregate every statement ever imported with no
|
||||
* date filter, and the UI printed the result with no period label — so a
|
||||
* lifetime-to-date total read as a current-period one and grew forever. It was
|
||||
* overstating the last 12 months by roughly $2,700 of fees.
|
||||
*
|
||||
* The regression is silent in the worst way: the number is always plausible and
|
||||
* always too big, and it gets worse the longer the app is used.
|
||||
*/
|
||||
describe("Priority 0 item 3 — fees and interest carry a date range", () => {
|
||||
/** A statement dated by the period it covers, carrying fees and interest. */
|
||||
async function statement(billingEnd: string, fees: number, interest: number, bank = "Fixture Bank") {
|
||||
const r = await pool.query(
|
||||
`INSERT INTO statements
|
||||
(filename, bank_name, account_number, billing_end_date, currency,
|
||||
fees_charged, interest_charged, owner_id)
|
||||
VALUES ($1, $2, '0000', $3, 'AUD', $4, $5, $6) RETURNING id`,
|
||||
[`fees-${billingEnd}-${bank}.pdf`, bank, billingEnd, fees, interest, ownerId]
|
||||
);
|
||||
return r.rows[0].id as number;
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
// Inside the default 12-month window (opens 2025-09-01 from a 2026-08-15 now).
|
||||
await statement("2026-07-31", 10, 5);
|
||||
await statement("2025-10-31", 20, 7);
|
||||
// Outside it — the tail that used to be silently included.
|
||||
await statement("2024-06-30", 1000, 500);
|
||||
await statement("2019-01-31", 2500, 900);
|
||||
});
|
||||
|
||||
it("defaults to the last 12 months and says so", async () => {
|
||||
const body = await json(await feesGET(req("/api/analytics/fees")));
|
||||
const period = body.period as { months: number; from: string; to: string; all_time: boolean };
|
||||
|
||||
expect(period.months).toBe(12);
|
||||
expect(period.all_time).toBe(false);
|
||||
// The window opens on the 1st of the month 11 months back, and toDateStr()
|
||||
// must not shift it a day earlier — `toISOString()` on a local-midnight
|
||||
// Date does exactly that in any timezone east of Greenwich, which is how
|
||||
// every analytics window used to be a day early.
|
||||
expect(period.from).toBe("2025-09-01");
|
||||
expect(period.to).toBe("2026-09-01");
|
||||
|
||||
expect(body.total_fees).toBe(30);
|
||||
expect(body.total_interest).toBe(12);
|
||||
});
|
||||
|
||||
it("excludes the older tail — the $2,700-of-fees overstatement", async () => {
|
||||
const body = await json(await feesGET(req("/api/analytics/fees")));
|
||||
// 3530 of fees exist; only 30 are in the last 12 months.
|
||||
expect(body.total_fees).toBe(30);
|
||||
expect(body.total_fees).not.toBe(3530);
|
||||
});
|
||||
|
||||
it("months=0 asks for all time deliberately, and is labelled as such", async () => {
|
||||
const body = await json(await feesGET(req("/api/analytics/fees?months=0")));
|
||||
const period = body.period as { months: number; from: null; to: null; all_time: boolean };
|
||||
|
||||
expect(period.all_time).toBe(true);
|
||||
// A null range is the honest answer for an unbounded total; a from/to here
|
||||
// would be a period claim the number does not actually make.
|
||||
expect(period.from).toBeNull();
|
||||
expect(period.to).toBeNull();
|
||||
|
||||
expect(body.total_fees).toBe(3530);
|
||||
expect(body.total_interest).toBe(1412);
|
||||
});
|
||||
|
||||
it("honours a narrower explicit window", async () => {
|
||||
const body = await json(await feesGET(req("/api/analytics/fees?months=3")));
|
||||
const period = body.period as { months: number; from: string };
|
||||
expect(period.months).toBe(3);
|
||||
expect(period.from).toBe("2026-06-01");
|
||||
// Only the 2026-07-31 statement falls inside three months.
|
||||
expect(body.total_fees).toBe(10);
|
||||
expect(body.total_interest).toBe(5);
|
||||
});
|
||||
|
||||
it("bounds the transaction-level fee lines by the same window", async () => {
|
||||
// Two halves of the same answer, and only the statement half was ever
|
||||
// date-filtered in the original. A bounded total beside an unbounded list
|
||||
// is the same defect wearing the fix.
|
||||
await txn({ date: "2026-07-10", amount: 12, type: "fee", category: "fees" });
|
||||
await txn({ date: "2020-02-10", amount: 99, type: "fee", category: "fees" });
|
||||
|
||||
const bounded = await json(await feesGET(req("/api/analytics/fees")));
|
||||
const all = await json(await feesGET(req("/api/analytics/fees?months=0")));
|
||||
|
||||
expect((bounded.transactions as unknown[]).length).toBe(1);
|
||||
expect((all.transactions as unknown[]).length).toBe(2);
|
||||
});
|
||||
|
||||
it("clamps a nonsense months parameter instead of unbounding the total", async () => {
|
||||
// A NaN reaching `months` would make `new Date(y, NaN, 1)` an Invalid Date
|
||||
// and the window meaningless. Falling back to 12 keeps the default claim.
|
||||
const body = await json(await feesGET(req("/api/analytics/fees?months=banana")));
|
||||
expect((body.period as { months: number }).months).toBe(12);
|
||||
expect(body.total_fees).toBe(30);
|
||||
});
|
||||
});
|
||||
|
||||
// ───────────────────────────────────────────────────────────────────────────
|
||||
// Item 6 — a month in progress is never compared against complete ones
|
||||
// ───────────────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Two rules, both silent when broken (see src/lib/partial-month.ts):
|
||||
*
|
||||
* 1. the in-progress month never enters a baseline;
|
||||
* 2. when it is the month selected, both sides of every comparison are
|
||||
* truncated to the same day.
|
||||
*
|
||||
* These are driven off real `/api/analytics/daily` output rather than a
|
||||
* hand-built map, because rule 2 depends on that route's day-of-month
|
||||
* granularity — a route returning month totals only would make the rule
|
||||
* unimplementable while every pure-function test still passed.
|
||||
*/
|
||||
describe("Priority 0 item 6 — partial-month comparisons", () => {
|
||||
/** `perDay` dollars on each of days 1..n of `month`. */
|
||||
async function evenMonth(month: string, perDay: number, days: number) {
|
||||
for (let d = 1; d <= days; d++) {
|
||||
await txn({ date: `${month}-${String(d).padStart(2, "0")}`, amount: perDay, category: "groceries" });
|
||||
}
|
||||
}
|
||||
|
||||
async function dailyMap() {
|
||||
const body = await json(await dailyGET(req("/api/analytics/daily?months=12")));
|
||||
return body.daily as Record<string, Record<number, number>>;
|
||||
}
|
||||
|
||||
it("recognises the in-progress month and the day to compare through", () => {
|
||||
expect(comparisonBasis(CURRENT_MONTH, NOW)).toEqual({ isPartial: true, compareDay: ELAPSED });
|
||||
// A complete month compares through its last day, so the same code path
|
||||
// serves both cases and `spendThrough` becomes a no-op.
|
||||
expect(comparisonBasis("2026-06", NOW)).toEqual({ isPartial: false, compareDay: 30 });
|
||||
expect(comparisonBasis("2026-02", NOW)).toEqual({ isPartial: false, compareDay: 28 });
|
||||
});
|
||||
|
||||
it("keeps the in-progress month out of the baseline", () => {
|
||||
const totals: Record<string, { spent: number }> = {
|
||||
"2026-05": { spent: 100 },
|
||||
"2026-06": { spent: 100 },
|
||||
"2026-07": { spent: 100 },
|
||||
[CURRENT_MONTH]: { spent: 40 }, // half a month of spend
|
||||
};
|
||||
const months = ["2026-05", "2026-06", "2026-07", CURRENT_MONTH];
|
||||
|
||||
// Baseline for a complete month: every other complete month with data.
|
||||
expect(baselineMonths(months, "2026-07", totals, NOW)).toEqual(["2026-05", "2026-06"]);
|
||||
// Baseline for the month in progress: itself excluded twice over.
|
||||
expect(baselineMonths(months, CURRENT_MONTH, totals, NOW)).toEqual([
|
||||
"2026-05",
|
||||
"2026-06",
|
||||
"2026-07",
|
||||
]);
|
||||
// The partial month is never in it. Averaging its 40 in would drag a
|
||||
// baseline of 100 down to 85 and make an ordinary month look extravagant.
|
||||
expect(baselineMonths(months, "2026-07", totals, NOW)).not.toContain(CURRENT_MONTH);
|
||||
});
|
||||
|
||||
it("drops a month with no data rather than averaging a zero into the baseline", () => {
|
||||
const totals: Record<string, { spent: number }> = { "2026-05": { spent: 100 }, "2026-06": { spent: 0 } };
|
||||
expect(baselineMonths(["2026-05", "2026-06"], "2026-07", totals, NOW)).toEqual(["2026-05"]);
|
||||
});
|
||||
|
||||
it("compares the month in progress through the same day of each prior month", async () => {
|
||||
// $10/day everywhere. Prior months ran the full month; the current one has
|
||||
// only reached the 15th, so it holds $150 against their $300-odd.
|
||||
await evenMonth("2026-06", 10, 30);
|
||||
await evenMonth("2026-07", 10, 31);
|
||||
await evenMonth(CURRENT_MONTH, 10, ELAPSED);
|
||||
|
||||
const daily = await dailyMap();
|
||||
const totals: Record<string, { spent: number }> = {
|
||||
"2026-06": { spent: 300 },
|
||||
"2026-07": { spent: 310 },
|
||||
[CURRENT_MONTH]: { spent: 150 },
|
||||
};
|
||||
const months = ["2026-06", "2026-07", CURRENT_MONTH];
|
||||
const { isPartial, compareDay } = comparisonBasis(CURRENT_MONTH, NOW);
|
||||
const baseline = baselineMonths(months, CURRENT_MONTH, totals, NOW);
|
||||
|
||||
const mine = spendThrough(daily[CURRENT_MONTH], compareDay);
|
||||
const avg = baselineAverage(baseline, { isPartial, compareDay, daily, totals });
|
||||
|
||||
// Truncated on both sides, this month is exactly typical.
|
||||
expect(mine).toBe(150);
|
||||
expect(avg).toBe(150);
|
||||
expect(Math.round(((mine - avg) / avg) * 100)).toBe(0);
|
||||
|
||||
// Untruncated — the bug — it reads as 51% below average on identical
|
||||
// spending, purely because half the month has not happened yet.
|
||||
const naiveAvg = baseline.reduce((s, m) => s + totals[m].spent, 0) / baseline.length;
|
||||
expect(Math.round(((mine - naiveAvg) / naiveAvg) * 100)).toBe(-51);
|
||||
});
|
||||
|
||||
it("leaves a complete month compared on full totals", async () => {
|
||||
await evenMonth("2026-06", 10, 30);
|
||||
await evenMonth("2026-07", 20, 31);
|
||||
|
||||
const daily = await dailyMap();
|
||||
const totals: Record<string, { spent: number }> = { "2026-06": { spent: 300 }, "2026-07": { spent: 620 } };
|
||||
const months = ["2026-06", "2026-07"];
|
||||
const { isPartial, compareDay } = comparisonBasis("2026-07", NOW);
|
||||
const baseline = baselineMonths(months, "2026-07", totals, NOW);
|
||||
|
||||
expect(isPartial).toBe(false);
|
||||
// Truncation must not quietly apply to a finished month: June's full 30
|
||||
// days are the right comparison for July's full 31.
|
||||
expect(baselineAverage(baseline, { isPartial, compareDay, daily, totals })).toBe(300);
|
||||
});
|
||||
|
||||
it("truncates the top-movers comparison on both sides", async () => {
|
||||
// The movers row asks "biggest category moves vs last month". With one side
|
||||
// full and the other half-done, every category reads as a large decrease.
|
||||
await evenMonth("2026-07", 10, 31); // $310 in July
|
||||
await evenMonth(CURRENT_MONTH, 10, ELAPSED); // $150 so far in August
|
||||
|
||||
const body = await json(await dailyGET(req("/api/analytics/daily?months=12")));
|
||||
const byCategory = body.byCategory as Record<string, Record<string, Record<number, number>>>;
|
||||
const { compareDay } = comparisonBasis(CURRENT_MONTH, NOW);
|
||||
const pm = prevMonth(CURRENT_MONTH);
|
||||
expect(pm).toBe("2026-07");
|
||||
|
||||
const now = spendThrough(byCategory[CURRENT_MONTH]?.groceries, compareDay);
|
||||
const before = spendThrough(byCategory[pm]?.groceries, compareDay);
|
||||
expect(now - before).toBe(0);
|
||||
|
||||
// Against July's full month it would report a $160 fall that never happened.
|
||||
const fullBefore = Object.values(byCategory[pm].groceries).reduce((s, v) => s + v, 0);
|
||||
expect(now - fullBefore).toBe(-160);
|
||||
});
|
||||
|
||||
it("crosses a year boundary correctly", () => {
|
||||
expect(prevMonth("2026-01")).toBe("2025-12");
|
||||
expect(comparisonBasis("2025-12", NOW)).toEqual({ isPartial: false, compareDay: 31 });
|
||||
});
|
||||
|
||||
it("handles February in a leap year", () => {
|
||||
// 2028 is a leap year; a hardcoded 28 would silently drop the 29th.
|
||||
expect(comparisonBasis("2028-02", new Date(2028, 5, 1)).compareDay).toBe(29);
|
||||
});
|
||||
});
|
||||
@@ -105,6 +105,19 @@ describe("NATIVE_CURRENCY", () => {
|
||||
expect(unconverted).toBe(true);
|
||||
});
|
||||
|
||||
it("labels a statement-less wallet row AUD, because its amount already is", async () => {
|
||||
// Wallet rows store a provisional AUD figure in amount AND amount_aud, and
|
||||
// the native figure in foreign_currency_amount. Reading the foreign code
|
||||
// here printed "INR 9.42" for an INR 524 tap (txn 8162).
|
||||
const id = await scratchTxn(
|
||||
"transaction_date, description, amount, amount_aud, transaction_type, payment_method, source, source_ref, foreign_currency_amount, foreign_currency_code",
|
||||
"'2026-10-03','Wallet foreign fixture', 9.42, 9.42, 'debit', 'card', 'wallet', 'evt:fixture-native-ccy', 524.00, 'INR'"
|
||||
);
|
||||
const { ccy, unconverted } = await currencyOf(id);
|
||||
expect(ccy).toBe("AUD");
|
||||
expect(unconverted).toBe(false);
|
||||
});
|
||||
|
||||
it("prefers the statement's currency over the foreign-charge record", async () => {
|
||||
// Opposite convention: on an AUD statement, `amount` is AUD and
|
||||
// foreign_currency_code merely notes what was originally charged. Reading
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants, mockDbWithPool } from "./helpers";
|
||||
|
||||
/**
|
||||
* Upcoming annual charges from payments alone.
|
||||
*
|
||||
* The exclusions carry this file. Anything can be made to find a card annual
|
||||
* fee; the difficulty is not also reporting the monthly account-servicing fee,
|
||||
* the per-transfer Wise charge, and the restaurant visited twice a year apart.
|
||||
* Measured on the real ledger, a naive "two charges ~365 days apart" ranks a
|
||||
* bakery first with 202 such pairs.
|
||||
*/
|
||||
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let getUpcomingAnnualCharges: typeof import("../../lib/annual-charges").getUpcomingAnnualCharges;
|
||||
|
||||
let acctSeq = 0;
|
||||
|
||||
/** A statement plus one transaction on it, for a named account. */
|
||||
async function charge(
|
||||
bank: string,
|
||||
acct: string,
|
||||
date: string,
|
||||
description: string,
|
||||
amount: number,
|
||||
type = "fee"
|
||||
) {
|
||||
const key = `${bank}:${acct}`;
|
||||
let stId = stmtCache.get(`${key}:${date}`);
|
||||
if (!stId) {
|
||||
const st = await pool.query(
|
||||
`INSERT INTO statements (filename, account_number, bank_name, billing_start_date, billing_end_date, owner_id)
|
||||
VALUES ($4, $1, $2, $3::date - 20, $3::date + 10, $5) RETURNING id`,
|
||||
[acct, bank, date, `s${acctSeq++}.pdf`, ownerId]
|
||||
);
|
||||
stId = st.rows[0].id as number;
|
||||
stmtCache.set(`${key}:${date}`, stId);
|
||||
}
|
||||
await pool.query(
|
||||
`INSERT INTO transactions (statement_id, transaction_date, description, amount, transaction_type, owner_id)
|
||||
VALUES ($1,$2,$3,$4,$5,$6)`,
|
||||
[stId, date, description, amount, type, ownerId]
|
||||
);
|
||||
}
|
||||
let stmtCache = new Map<string, number>();
|
||||
|
||||
/** A statementless merchant purchase, for the non-fee signal. */
|
||||
async function purchase(date: string, merchant: string, amount: number) {
|
||||
await pool.query(
|
||||
`INSERT INTO transactions (statement_id, transaction_date, description, amount, transaction_type, owner_id, merchant_normalized, row_index)
|
||||
VALUES (NULL,$1,$2,$3,'debit',$4,$2,0)`,
|
||||
[date, merchant, amount, ownerId]
|
||||
);
|
||||
}
|
||||
|
||||
const find = <T extends { label: string }>(rows: T[], label: string) =>
|
||||
rows.find((r) => r.label === label);
|
||||
|
||||
beforeAll(async () => {
|
||||
pool = createPool();
|
||||
mockDbWithPool(pool);
|
||||
({ getUpcomingAnnualCharges } = await import("../../lib/annual-charges"));
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
await pool.query("DELETE FROM account_links");
|
||||
stmtCache = new Map();
|
||||
({ ownerId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
describe("what it finds", () => {
|
||||
it("confirms a card annual fee that repeated a year later", async () => {
|
||||
await charge("American Express", "1000", "2025-06-10", "ANNUAL CARD FEE", 875);
|
||||
await charge("American Express", "1000", "2026-06-10", "ANNUAL CARD FEE", 1750);
|
||||
const row = find(await getUpcomingAnnualCharges(ownerId), "ANNUAL CARD FEE");
|
||||
expect(row?.confidence).toBe("confirmed");
|
||||
expect(row?.observations).toBe(2);
|
||||
// The amount is NOT averaged: this fee doubled, and the next one to plan for
|
||||
// is the latest, not the mean of a rise.
|
||||
expect(Number(row?.last_amount)).toBe(1750);
|
||||
expect(row?.next_due_estimate.startsWith("2027-06")).toBe(true);
|
||||
});
|
||||
|
||||
it("accepts a fee seen only once when the bank names it annual", async () => {
|
||||
await charge("Westpac", "8032", "2025-10-21", "CARD FEE", 295);
|
||||
const row = find(await getUpcomingAnnualCharges(ownerId), "CARD FEE");
|
||||
// One charge cannot show a cadence, but the kind is not in doubt.
|
||||
expect(row?.confidence).toBe("likely");
|
||||
expect(row?.next_due_estimate).toBe("2026-10-21");
|
||||
});
|
||||
|
||||
it("finds an annual merchant charge with no fee wording, if the amount is stable", async () => {
|
||||
await purchase("2025-02-03", "AMAZON PRIME", 79);
|
||||
await purchase("2026-02-01", "AMAZON PRIME", 79);
|
||||
const row = find(await getUpcomingAnnualCharges(ownerId), "AMAZON PRIME");
|
||||
expect(row?.confidence).toBe("possible");
|
||||
});
|
||||
});
|
||||
|
||||
describe("what it must NOT report", () => {
|
||||
it("ignores a monthly account fee", async () => {
|
||||
for (const d of ["2026-01-31", "2026-02-28", "2026-03-31", "2026-04-30"]) {
|
||||
await charge("ANZ", "5626", d, "ACCOUNT SERVICING FEE", 5);
|
||||
}
|
||||
expect(find(await getUpcomingAnnualCharges(ownerId), "ACCOUNT SERVICING FEE")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("ignores per-transaction fees, which never repeat under one label", async () => {
|
||||
await charge("Wise Australia Pty Ltd.", "9001", "2026-05-20", "WISE CHARGES FOR: TRANSFER-2143509822", 30.67);
|
||||
await charge("Wise Australia Pty Ltd.", "9001", "2026-06-04", "WISE CHARGES FOR: TRANSFER-2171403032", 19.51);
|
||||
const rows = await getUpcomingAnnualCharges(ownerId);
|
||||
expect(rows.filter((r) => r.label.startsWith("WISE CHARGES"))).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("ignores a merchant visited twice a year apart for wildly different amounts", async () => {
|
||||
// The exact shape that defeats a pure cadence test: Target, $30 then $198.
|
||||
await purchase("2025-07-21", "TARGET", 30);
|
||||
await purchase("2026-07-11", "TARGET", 198);
|
||||
expect(find(await getUpcomingAnnualCharges(ownerId), "TARGET")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("ignores a merchant seen only once", async () => {
|
||||
await purchase("2026-04-03", "UBER ONE MEMBERSHIP", 96);
|
||||
expect(find(await getUpcomingAnnualCharges(ownerId), "UBER ONE MEMBERSHIP")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("account identity and honesty about gaps", () => {
|
||||
it("treats a renamed account as one continuous fee series", async () => {
|
||||
// Citi Prestige -> NAB MyCard: same card, two bank names, two numbers.
|
||||
await charge("Citibank", "5258073330340253", "2024-12-13", "Annual Fee", 700);
|
||||
await charge("Citibank", "5258073330340253", "2025-11-13", "Annual Fee", 700);
|
||||
await pool.query(
|
||||
`INSERT INTO account_links (from_bank_name, from_account_key, to_bank_name, to_account_key, effective_date, reason)
|
||||
VALUES ('Citibank','5258073330340253','NAB','1015001000116227','2025-11-14','issuer_change')`
|
||||
);
|
||||
// A statement under the NEW identity, so the link has somewhere to resolve to.
|
||||
await charge("NAB", "1015001000116227", "2026-08-13", "Purchase", 29.99, "debit");
|
||||
|
||||
const row = find(await getUpcomingAnnualCharges(ownerId), "ANNUAL FEE");
|
||||
expect(row?.observations).toBe(2);
|
||||
expect(row?.confidence).toBe("confirmed");
|
||||
// Names the account's CURRENT bank -- the next fee comes from there, and
|
||||
// sending someone to the retired brand is the bug this link exists to stop.
|
||||
expect(row?.bank_name).toBe("NAB");
|
||||
});
|
||||
|
||||
it("marks a passed estimate as expected_not_seen rather than upcoming", async () => {
|
||||
await charge("HSBC", "0351", "2024-03-06", "ANNUAL CHARGE", 450);
|
||||
await charge("HSBC", "0351", "2025-03-06", "ANNUAL CHARGE", 450);
|
||||
const row = find(await getUpcomingAnnualCharges(ownerId), "ANNUAL CHARGE");
|
||||
expect(row?.status).toBe("expected_not_seen");
|
||||
// And it reports how stale the account is, which is what separates
|
||||
// "it stopped being charged" from "we stopped importing the statements".
|
||||
expect(row?.account_days_stale).toBeGreaterThan(300);
|
||||
});
|
||||
|
||||
it("sorts upcoming ahead of expected_not_seen", async () => {
|
||||
await charge("HSBC", "0351", "2024-03-06", "ANNUAL CHARGE", 450);
|
||||
await charge("HSBC", "0351", "2025-03-06", "ANNUAL CHARGE", 450);
|
||||
await charge("Westpac", "8032", "2026-08-01", "CARD FEE", 295);
|
||||
const rows = await getUpcomingAnnualCharges(ownerId);
|
||||
const firstMissing = rows.findIndex((r) => r.status === "expected_not_seen");
|
||||
const lastUpcoming = rows.map((r) => r.status).lastIndexOf("upcoming");
|
||||
expect(lastUpcoming).toBeLessThan(firstMissing);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,271 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants, insertTransaction } from "./helpers";
|
||||
|
||||
/**
|
||||
* The automatic pass over the reconcile queue.
|
||||
*
|
||||
* This exists because the wallet lane writes a transaction for every tap
|
||||
* (DECISIONS.md ING-11), adding ~63 rows a month to a queue already ~177 deep
|
||||
* and worked by hand. Hiding those rows instead is explicitly ruled out --
|
||||
* see the awaitsStatementLine() removal note in queries.ts.
|
||||
*
|
||||
* The cases that must NOT match matter more than the ones that must: this runs
|
||||
* unattended, and a wrong pairing merges one purchase's curation onto another
|
||||
* purchase's row. So most of what follows is about refusal.
|
||||
*/
|
||||
|
||||
const MACHINE_TOKEN = "test-wallet-token";
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let autoReconcile: typeof import("../../app/api/transactions/reconcile/route").PATCH;
|
||||
|
||||
/** Browser-session caller: no machine token, auth comes from the mocked session. */
|
||||
const userReq = (url = "http://x/api/transactions/reconcile") =>
|
||||
({ url, headers: { get: () => null } }) as never;
|
||||
|
||||
/** Scheduled caller: machine token, no session, sweeps every owner. */
|
||||
const machineReq = (url = "http://x/api/transactions/reconcile") =>
|
||||
({ url, headers: { get: (h: string) => (h === "x-ingest-token" ? MACHINE_TOKEN : null) } }) as never;
|
||||
|
||||
async function statementLine(
|
||||
amount: number,
|
||||
date: string,
|
||||
description: string,
|
||||
opts: { owner?: number; type?: string; fxAmount?: number; fxCode?: string } = {}
|
||||
) {
|
||||
const owner = opts.owner ?? ownerId;
|
||||
const st = await pool.query(
|
||||
// Account number varies by owner: uq_statement_identity is
|
||||
// (bank_name, account_number, billing_end_date), so two owners sharing one
|
||||
// account collide rather than modelling two real cards.
|
||||
`INSERT INTO statements (filename, account_number, bank_name, billing_start_date, billing_end_date, owner_id)
|
||||
VALUES ('s.pdf', $3, 'Test Bank', $1::date - 20, $1::date + 10, $2) RETURNING id`,
|
||||
[date, owner, `1234-5678-9012-${String(owner).padStart(4, "0")}`]
|
||||
);
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions
|
||||
(statement_id, transaction_date, description, amount, transaction_type, owner_id,
|
||||
foreign_currency_amount, foreign_currency_code)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING id`,
|
||||
[st.rows[0].id, date, description, amount, opts.type ?? "debit", owner,
|
||||
opts.fxAmount ?? null, opts.fxCode ?? null]
|
||||
);
|
||||
return t.rows[0].id as number;
|
||||
}
|
||||
|
||||
async function walletRow(
|
||||
amount: number,
|
||||
date: string,
|
||||
opts: { fxAmount?: number; fxCode?: string; paymentMethod?: string } = {}
|
||||
) {
|
||||
const id = await insertTransaction(pool, ownerId, {
|
||||
amount, transaction_date: date, description: "Wallet tap",
|
||||
});
|
||||
await pool.query(
|
||||
`UPDATE transactions
|
||||
SET foreign_currency_amount = $2, foreign_currency_code = $3, payment_method = $4
|
||||
WHERE id = $1`,
|
||||
[id, opts.fxAmount ?? null, opts.fxCode ?? null, opts.paymentMethod ?? null]
|
||||
);
|
||||
return id;
|
||||
}
|
||||
|
||||
const reconciledWith = async (id: number) =>
|
||||
(await pool.query(`SELECT reconciled_with_id FROM transactions WHERE id = $1`, [id]))
|
||||
.rows[0].reconciled_with_id as number | null;
|
||||
|
||||
beforeAll(async () => {
|
||||
process.env.WALLET_INGEST_TOKEN = MACHINE_TOKEN;
|
||||
pool = createPool();
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
|
||||
}));
|
||||
({ PATCH: autoReconcile } = await import("../../app/api/transactions/reconcile/route"));
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
await pool.query("DELETE FROM expense_metadata");
|
||||
({ ownerId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
describe("pairs it should apply", () => {
|
||||
it("matches an exact amount two days apart and moves the link", async () => {
|
||||
const manual = await walletRow(13.5, "2026-08-13");
|
||||
const stmt = await statementLine(13.5, "2026-08-15", "BREADWORLD BAKERY");
|
||||
|
||||
const body = await (await autoReconcile(userReq())).json();
|
||||
expect(body.reconciled).toBe(1);
|
||||
expect(await reconciledWith(manual)).toBe(stmt);
|
||||
});
|
||||
|
||||
it("matches a foreign row on the NATIVE pair, not on the AUD amount", async () => {
|
||||
// The bank converts at its own rate, so the wallet row cannot predict the
|
||||
// AUD figure. It can only know THB 500. Deliberately give the statement a
|
||||
// wildly different AUD amount to prove `amount` is not what matched.
|
||||
const manual = await walletRow(999.99, "2026-08-13", { fxAmount: 500, fxCode: "THB" });
|
||||
const stmt = await statementLine(22.85, "2026-08-14", "7-ELEVEN BANGKOK", {
|
||||
fxAmount: 500, fxCode: "THB",
|
||||
});
|
||||
|
||||
const body = await (await autoReconcile(userReq())).json();
|
||||
expect(body.reconciled).toBe(1);
|
||||
expect(await reconciledWith(manual)).toBe(stmt);
|
||||
});
|
||||
|
||||
it("carries a SETTLED split across without resurrecting the debt", async () => {
|
||||
const manual = await walletRow(50, "2026-08-13");
|
||||
const stmt = await statementLine(50, "2026-08-14", "SHARED DINNER");
|
||||
await pool.query(
|
||||
`INSERT INTO transaction_splits (transaction_id, participant_id, share_percent, settled, settled_at)
|
||||
VALUES ($1, $2, 50, true, now())`,
|
||||
[manual, ownerId]
|
||||
);
|
||||
|
||||
expect((await autoReconcile(userReq())).status ?? 200).toBe(200);
|
||||
|
||||
const moved = await pool.query(
|
||||
`SELECT settled, settled_at FROM transaction_splits WHERE transaction_id = $1`, [stmt]
|
||||
);
|
||||
expect(moved.rowCount).toBe(1);
|
||||
// The whole point: a delete-and-recreate defaults `settled` to false, which
|
||||
// silently marks a paid debt owed again. Nothing errors when it regresses.
|
||||
expect(moved.rows[0].settled).toBe(true);
|
||||
expect(moved.rows[0].settled_at).not.toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("pairs it must refuse", () => {
|
||||
it("refuses two identical amounts in the same window (ambiguous both ways)", async () => {
|
||||
// Two $13.50 bakery taps in one week is the ordinary case, not an edge one.
|
||||
const a = await walletRow(13.5, "2026-08-13");
|
||||
const b = await walletRow(13.5, "2026-08-14");
|
||||
await statementLine(13.5, "2026-08-14", "BREADWORLD ONE");
|
||||
await statementLine(13.5, "2026-08-15", "BREADWORLD TWO");
|
||||
|
||||
const body = await (await autoReconcile(userReq())).json();
|
||||
expect(body.reconciled).toBe(0);
|
||||
expect(await reconciledWith(a)).toBeNull();
|
||||
expect(await reconciledWith(b)).toBeNull();
|
||||
});
|
||||
|
||||
it("refuses a near-miss amount — a person should look at that", async () => {
|
||||
const manual = await walletRow(13.5, "2026-08-13");
|
||||
await statementLine(13.55, "2026-08-14", "BREADWORLD BAKERY");
|
||||
|
||||
expect((await (await autoReconcile(userReq())).json()).reconciled).toBe(0);
|
||||
expect(await reconciledWith(manual)).toBeNull();
|
||||
});
|
||||
|
||||
it("refuses a match outside the day window", async () => {
|
||||
const manual = await walletRow(13.5, "2026-08-13");
|
||||
await statementLine(13.5, "2026-08-25", "BREADWORLD BAKERY");
|
||||
|
||||
expect((await (await autoReconcile(userReq())).json()).reconciled).toBe(0);
|
||||
expect(await reconciledWith(manual)).toBeNull();
|
||||
});
|
||||
|
||||
it("refuses a statement row already claimed by another manual row", async () => {
|
||||
const claimed = await statementLine(40, "2026-08-14", "ALREADY TAKEN");
|
||||
const first = await walletRow(40, "2026-08-13");
|
||||
await pool.query(`UPDATE transactions SET reconciled_with_id = $2 WHERE id = $1`, [first, claimed]);
|
||||
const second = await walletRow(40, "2026-08-14");
|
||||
|
||||
expect((await (await autoReconcile(userReq())).json()).reconciled).toBe(0);
|
||||
expect(await reconciledWith(second)).toBeNull();
|
||||
});
|
||||
|
||||
it("refuses a credits-funded row — no card leg is ever coming", async () => {
|
||||
const manual = await walletRow(30, "2026-08-13", { paymentMethod: "credits" });
|
||||
await statementLine(30, "2026-08-14", "COINCIDENTAL AMOUNT");
|
||||
|
||||
expect((await (await autoReconcile(userReq())).json()).reconciled).toBe(0);
|
||||
expect(await reconciledWith(manual)).toBeNull();
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
describe("an AUD row against a foreign statement line", () => {
|
||||
it("DOES match on the AUD figure — this is what every real pair looks like", async () => {
|
||||
// Checked against live data: all 48 reconciliations a human has made are
|
||||
// this exact shape -- manual row with no currency, statement row carrying
|
||||
// EUR or CHF, paired on exact AUD. Overseas receipts and orders record what
|
||||
// was charged in AUD; only the wallet lane records natively. An earlier
|
||||
// version of this rule required both sides to be non-foreign, on the
|
||||
// reasoning that a currency mismatch is ambiguous. It matched 0 of the 48.
|
||||
const manual = await walletRow(22.85, "2026-08-13");
|
||||
const stmt = await statementLine(22.85, "2026-08-14", "7-ELEVEN BANGKOK", {
|
||||
fxAmount: 500, fxCode: "THB",
|
||||
});
|
||||
|
||||
expect((await (await autoReconcile(userReq())).json()).reconciled).toBe(1);
|
||||
expect(await reconciledWith(manual)).toBe(stmt);
|
||||
});
|
||||
|
||||
it("still refuses when a second candidate makes it ambiguous", async () => {
|
||||
// The coincidence this guards against -- a domestic tap whose amount equals
|
||||
// a foreign charge's AUD figure -- is caught by unique-both-ways, not by a
|
||||
// currency test.
|
||||
const manual = await walletRow(22.85, "2026-08-13");
|
||||
await statementLine(22.85, "2026-08-14", "7-ELEVEN BANGKOK", { fxAmount: 500, fxCode: "THB" });
|
||||
await statementLine(22.85, "2026-08-13", "MELBOURNE CAFE");
|
||||
|
||||
expect((await (await autoReconcile(userReq())).json()).reconciled).toBe(0);
|
||||
expect(await reconciledWith(manual)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("dry run", () => {
|
||||
it("reports what it would do and writes nothing", async () => {
|
||||
const manual = await walletRow(13.5, "2026-08-13");
|
||||
await statementLine(13.5, "2026-08-14", "BREADWORLD BAKERY");
|
||||
|
||||
const body = await (await autoReconcile(userReq("http://x/api/transactions/reconcile?dry=1"))).json();
|
||||
expect(body.dry_run).toBe(true);
|
||||
expect(body.would_reconcile).toBe(1);
|
||||
expect(await reconciledWith(manual)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("the scheduled sweep", () => {
|
||||
it("runs on the machine token with no session at all", async () => {
|
||||
const manual = await walletRow(13.5, "2026-08-13");
|
||||
const stmt = await statementLine(13.5, "2026-08-14", "BREADWORLD BAKERY");
|
||||
|
||||
const body = await (await autoReconcile(machineReq())).json();
|
||||
expect(body.reconciled).toBe(1);
|
||||
expect(await reconciledWith(manual)).toBe(stmt);
|
||||
});
|
||||
|
||||
it("covers EVERY owner, not just one", async () => {
|
||||
// 169 of the 177 rows outstanding belong to the second participant, so a
|
||||
// sweep scoped to a single user would leave almost all of the queue alone.
|
||||
const other = (await pool.query(
|
||||
`INSERT INTO participants (name) VALUES ('Carol') RETURNING id`
|
||||
)).rows[0].id as number;
|
||||
|
||||
const mine = await walletRow(11, "2026-08-13");
|
||||
const theirTx = (await pool.query(
|
||||
`INSERT INTO transactions (owner_id, statement_id, transaction_date, description,
|
||||
amount, transaction_type, category, row_index)
|
||||
VALUES ($1, NULL, '2026-08-13', 'Theirs', 22, 'debit', 'other', 0) RETURNING id`, [other]
|
||||
)).rows[0].id as number;
|
||||
|
||||
await statementLine(11, "2026-08-14", "MINE");
|
||||
await statementLine(22, "2026-08-14", "THEIRS", { owner: other });
|
||||
|
||||
const body = await (await autoReconcile(machineReq())).json();
|
||||
expect(body.reconciled).toBe(2);
|
||||
expect(await reconciledWith(mine)).not.toBeNull();
|
||||
expect(await reconciledWith(theirTx)).not.toBeNull();
|
||||
});
|
||||
|
||||
it("rejects a wrong machine token and falls through to session auth", async () => {
|
||||
const bad = ({ url: "http://x/", headers: { get: () => "not-the-token" } }) as never;
|
||||
// Session is mocked, so this still succeeds -- but as the session user only,
|
||||
// proving the bad token did not grant the all-owner sweep.
|
||||
const body = await (await autoReconcile(bad)).json();
|
||||
expect(body.reconciled).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -23,6 +23,25 @@ export function mockDbWithPool(p: Pool) {
|
||||
const result = await p.query(sql, params);
|
||||
return result.rows[0] ?? null;
|
||||
},
|
||||
// Mirrors withTransaction: one connection, BEGIN/COMMIT around the callback,
|
||||
// so pg_advisory_xact_lock actually holds across the statements inside it.
|
||||
withTransaction: async (fn: (q: unknown) => Promise<unknown>) => {
|
||||
const client = await p.connect();
|
||||
try {
|
||||
await client.query("BEGIN");
|
||||
const result = await fn({
|
||||
queryRaw: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows,
|
||||
queryRow: async (sql: string, params: unknown[] = []) => (await client.query(sql, params)).rows[0] ?? null,
|
||||
});
|
||||
await client.query("COMMIT");
|
||||
return result;
|
||||
} catch (e) {
|
||||
await client.query("ROLLBACK");
|
||||
throw e;
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
},
|
||||
prisma: p,
|
||||
}));
|
||||
}
|
||||
@@ -30,12 +49,17 @@ export function mockDbWithPool(p: Pool) {
|
||||
/**
|
||||
* Refuse to truncate anything that is not the test database.
|
||||
*
|
||||
* `DATABASE_URL` in `.env.test` names the Postgres container by IP, and
|
||||
* container IPs move on recreation: 172.22.0.47 stopped being
|
||||
* `postgres-personal` and became `postgres-pantry`, so the suite spent a while
|
||||
* pointing its TRUNCATE at another app's database. It only failed safe because
|
||||
* the credentials happened not to match — had they matched, this would have
|
||||
* wiped pantry-app.
|
||||
* `.env.test` used to name the Postgres container by bridge IP, and Docker
|
||||
* reassigns those on recreation: 172.22.0.47 stopped being `postgres-personal`
|
||||
* and became `postgres-pantry`, so the suite spent a while pointing its
|
||||
* TRUNCATE at another app's database. It only failed safe because the
|
||||
* credentials happened not to match — had they matched, this would have wiped
|
||||
* pantry-app.
|
||||
*
|
||||
* `scripts/resolve-test-db.sh` (wired as `pretest:integration`) now re-derives
|
||||
* the DSN from the live container before every run, preferring the published
|
||||
* loopback port. This check stays anyway: running vitest directly bypasses the
|
||||
* pretest hook, and a stale `.env.test` would still be read.
|
||||
*
|
||||
* Checked once per process, before the first truncate.
|
||||
*/
|
||||
@@ -47,8 +71,9 @@ async function assertTestDatabase(pool: Pool) {
|
||||
if (db !== "personal_test") {
|
||||
throw new Error(
|
||||
`Refusing to truncate: connected to "${db}", expected "personal_test". ` +
|
||||
`Check DATABASE_URL in .env.test — the Postgres container IP may have changed ` +
|
||||
`(docker inspect postgres-personal --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}').`
|
||||
`Re-resolve the DSN with \`bash scripts/resolve-test-db.sh\` (or just run ` +
|
||||
`\`npm run test:integration\`, which does it for you), and run ` +
|
||||
`\`npm run test:setup\` if personal_test does not exist yet.`
|
||||
);
|
||||
}
|
||||
targetVerified = true;
|
||||
@@ -59,6 +84,11 @@ export async function resetDB(pool: Pool) {
|
||||
await assertTestDatabase(pool);
|
||||
await pool.query(`
|
||||
TRUNCATE
|
||||
-- The audit triggers (migration 0037) fire on every INSERT the suite makes.
|
||||
-- TRUNCATE does not fire row-level triggers, so nothing here is logged --
|
||||
-- but without this line the log is never cleared either, and the table
|
||||
-- grows by roughly a thousand rows per full run, forever.
|
||||
transaction_audit,
|
||||
split_payments,
|
||||
transaction_splits,
|
||||
transaction_tags,
|
||||
|
||||
@@ -0,0 +1,926 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, afterAll, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { readFileSync, mkdtempSync, rmSync } from "fs";
|
||||
import { tmpdir } from "os";
|
||||
import { join, resolve } from "path";
|
||||
import type { NextRequest } from "next/server";
|
||||
import { createPool, mockDbWithPool, resetDB, seedParticipants } from "./helpers";
|
||||
import type { ExtractedOrder } from "../../lib/intake-rules";
|
||||
|
||||
/**
|
||||
* The intake router against the real schema (plan §11 C1, C2, C5).
|
||||
*
|
||||
* The model is stubbed at the HTTP boundary — `fetch` to LiteLLM answers with a
|
||||
* canned extraction, `fetch` to pantry with a canned capture — so the PDF text
|
||||
* path, the persistence, the match and the writes all run for real. The two
|
||||
* PDFs are the user's own Uber Eats exports; the transactions seeded here are
|
||||
* shaped like production rows 6846 and 6857 (platform `ubereats`, `line_items`
|
||||
* empty, the email pipeline's booking of the same two orders).
|
||||
*/
|
||||
|
||||
const INTAKE = "test-intake-token";
|
||||
const RECEIPT = "test-receipt-token";
|
||||
const fixtures = resolve(__dirname, "../fixtures/intake");
|
||||
const BURGERTORY = readFileSync(resolve(fixtures, "ubereats-burgertory-2026-08-19.pdf"));
|
||||
const COLES = readFileSync(resolve(fixtures, "ubereats-coles-2026-08-18.pdf"));
|
||||
|
||||
const burgertory: ExtractedOrder = {
|
||||
merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD",
|
||||
items: [
|
||||
{ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] },
|
||||
{ quantity: 1, description: "Joel", unit_price: 14, line_total: 14, options: [] },
|
||||
],
|
||||
adjustments: [{ label: "Delivery Fee", amount: 5.49 }, { label: "Promotion", amount: -8 }],
|
||||
};
|
||||
const coles: ExtractedOrder = {
|
||||
merchant: "Coles (Wyndham Vale)", order_date: "2026-08-18", total: 85.74, subtotal: 118.94, currency: "AUD",
|
||||
items: [
|
||||
{ quantity: 3, description: "Coca-Cola Zero Sugar Soft Drink (2L)", unit_price: 4.4, line_total: 13.2, options: [] },
|
||||
{ quantity: 1, description: "Coles RSPCA Approved Chicken Thigh Cutlets approx. 1.1kg", unit_price: 11, line_total: 12.76, options: ["Final 1.16 kg $11.00/kg"] },
|
||||
{ quantity: 1, description: "everything else", unit_price: 92.98, line_total: 92.98, options: [] },
|
||||
],
|
||||
adjustments: [],
|
||||
};
|
||||
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let otherId: number;
|
||||
let fileDir: string;
|
||||
let intakePOST: (req: NextRequest) => Promise<Response>;
|
||||
let lineItemsPUT: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
|
||||
let slackPOST: (req: NextRequest) => Promise<Response>;
|
||||
let orderGET: (req: NextRequest, ctx: { params: Promise<{ id: string }> }) => Promise<Response>;
|
||||
|
||||
/** What the next LiteLLM call answers, and what the next pantry call answers. */
|
||||
let modelReply: ExtractedOrder | Error = burgertory;
|
||||
let pantryReply: { status: number; body: unknown } = { status: 200, body: { captureId: 901, items: [1, 2, 3] } };
|
||||
const calls: { url: string; body: unknown }[] = [];
|
||||
|
||||
function stubFetch() {
|
||||
vi.stubGlobal("fetch", async (url: string | URL, init?: RequestInit) => {
|
||||
const u = String(url);
|
||||
if (u.includes("/v1/chat/completions")) {
|
||||
calls.push({ url: u, body: JSON.parse(String(init?.body)) });
|
||||
if (modelReply instanceof Error) return new Response("boom", { status: 500 });
|
||||
return Response.json({ choices: [{ message: { content: JSON.stringify(modelReply) }, finish_reason: "stop" }] });
|
||||
}
|
||||
if (u.includes("/api/capture")) {
|
||||
const form = init?.body as FormData;
|
||||
calls.push({ url: u, body: { intake: JSON.parse(String(form.get("intake"))), files: form.getAll("image").map((f) => (f as File).name), headers: init?.headers } });
|
||||
return Response.json(pantryReply.body, { status: pantryReply.status });
|
||||
}
|
||||
if (u.includes("hooks.slack.com")) {
|
||||
calls.push({ url: u, body: JSON.parse(String(init?.body)) });
|
||||
return new Response("ok");
|
||||
}
|
||||
throw new Error(`unexpected fetch ${u}`);
|
||||
});
|
||||
}
|
||||
|
||||
const multipart = (fields: Record<string, string>, files: { name: string; type: string; bytes: Buffer }[], token: string | null = INTAKE) => {
|
||||
const fd = new FormData();
|
||||
for (const [k, v] of Object.entries(fields)) fd.append(k, v);
|
||||
for (const f of files) fd.append("file", new File([new Uint8Array(f.bytes)], f.name, { type: f.type }));
|
||||
return new Request("http://finance.test/api/intake", { method: "POST", headers: token ? { "x-ingest-token": token } : {}, body: fd }) as unknown as NextRequest;
|
||||
};
|
||||
const jsonReq = (body: unknown, token: string | null) =>
|
||||
({ headers: { get: (h: string) => (h === "x-ingest-token" ? token : null) }, json: async () => body }) as unknown as NextRequest;
|
||||
const put = (id: number | string, body: unknown, token: string | null = RECEIPT) => lineItemsPUT(jsonReq(body, token), { params: Promise.resolve({ id: String(id) }) });
|
||||
const uuid = () => crypto.randomUUID();
|
||||
|
||||
/** A transaction shaped like the email pipeline's booking of an order, line_items empty. */
|
||||
async function seedOrder(over: { merchant: string; date: string; amount: number; platform?: string | null; owner?: number; withMeta?: boolean }) {
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions (owner_id, statement_id, transaction_date, description, amount, amount_aud, transaction_type, category, merchant_name, merchant_normalized, payment_method, row_index)
|
||||
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'dining', $5, $5, 'credits', 0) RETURNING id`,
|
||||
[over.owner ?? ownerId, over.date, `Order - ${over.merchant}`, over.amount, over.merchant]
|
||||
);
|
||||
const id = t.rows[0].id as number;
|
||||
if (over.withMeta !== false) {
|
||||
await pool.query(
|
||||
`INSERT INTO expense_metadata (transaction_id, source, platform, order_reference, line_items, merchant_normalized, amount, transaction_date, currency)
|
||||
VALUES ($1, 'email', $2, $3, '[]'::jsonb, $4, $5, $6, 'AUD')`,
|
||||
[id, over.platform === undefined ? "ubereats" : over.platform, uuid(), over.merchant, over.amount, over.date]
|
||||
);
|
||||
}
|
||||
return id;
|
||||
}
|
||||
/** A row shaped the way the wallet lane writes one: statementless, no metadata
|
||||
* at all, no platform, and named by the notification rather than the receipt. */
|
||||
async function seedWalletRow(over: { merchant: string; date: string; amount: number; owner?: number }) {
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions (owner_id, statement_id, transaction_date, description, amount, amount_aud, transaction_type, category, merchant_name, merchant_normalized, payment_method, location, source, source_ref, row_index)
|
||||
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'other', $3, $3, 'card', 'Werribee, VIC', 'wallet', $5, 0) RETURNING id`,
|
||||
[over.owner ?? ownerId, over.date, over.merchant, over.amount, `evt:${Math.floor(over.amount * 100)}`]
|
||||
);
|
||||
return t.rows[0].id as number;
|
||||
}
|
||||
|
||||
/** What `components/order-details.tsx` would actually render for a transaction. */
|
||||
const orderPanel = async (txnId: number) =>
|
||||
(await (await orderGET({ headers: { get: () => null } } as unknown as NextRequest, { params: Promise.resolve({ id: String(txnId) }) })).json()) as { line_items?: unknown[] } | null;
|
||||
const metaRows = async () => (await pool.query(`SELECT id, platform, line_items FROM expense_metadata ORDER BY id`)).rows as { id: number; platform: string | null; line_items: { description: string }[] }[];
|
||||
const lineItemsOf = async (txnId: number) =>
|
||||
(await pool.query(`SELECT line_items FROM expense_metadata WHERE transaction_id = $1 OR matched_transaction_id = $1`, [txnId])).rows[0]?.line_items;
|
||||
const batch = async (id: string) => (await pool.query(`SELECT * FROM intake_batches WHERE id = $1`, [id])).rows[0];
|
||||
|
||||
beforeAll(async () => {
|
||||
process.env.INTAKE_TOKEN = INTAKE;
|
||||
process.env.RECEIPT_INGEST_TOKEN = RECEIPT;
|
||||
process.env.ORDER_INGEST_TOKEN = "test-order-token";
|
||||
process.env.LITELLM_BASE_URL = "http://litellm.test";
|
||||
process.env.LITELLM_KEY = "k";
|
||||
process.env.PANTRY_BASE_URL = "http://pantry.test";
|
||||
process.env.FINANCE_PUBLIC_URL = "https://finance.test";
|
||||
fileDir = mkdtempSync(join(tmpdir(), "intake-test-"));
|
||||
process.env.INTAKE_FILE_DIR = fileDir;
|
||||
pool = createPool();
|
||||
mockDbWithPool(pool);
|
||||
// The Slack signature gate is exercised by slack-verify.test.ts; here it is
|
||||
// bypassed so the intake branch of the route is what is under test.
|
||||
vi.doMock("@/lib/slack-verify", () => ({
|
||||
verifySlackSignature: () => true,
|
||||
participantForSlackUser: () => 1,
|
||||
slackUserForParticipant: () => null,
|
||||
}));
|
||||
// The order-detail panel is the surface an enrichment has to appear on, so
|
||||
// the read is exercised beside the write rather than assumed.
|
||||
vi.doMock("@/lib/auth", () => ({ getCurrentUser: async () => ({ id: 1, name: "Owner", email: "o@x" }) }));
|
||||
vi.doMock("@/lib/queries", async (importOriginal) => ({
|
||||
...(await importOriginal<Record<string, unknown>>()),
|
||||
canAccessTransactions: async () => true,
|
||||
}));
|
||||
({ POST: intakePOST } = await import("../../app/api/intake/route"));
|
||||
({ PUT: lineItemsPUT } = await import("../../app/api/transactions/[id]/line-items/route"));
|
||||
({ POST: slackPOST } = await import("../../app/api/slack/interactive/route"));
|
||||
({ GET: orderGET } = await import("../../app/api/transactions/[id]/order/route"));
|
||||
stubFetch();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
vi.unstubAllGlobals();
|
||||
rmSync(fileDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
await pool.query(`TRUNCATE intake_batches, intake_merchant_kinds`);
|
||||
({ ownerId, otherId } = await seedParticipants(pool, ["Alice", "Bob"]));
|
||||
await pool.query(`UPDATE participants SET email = 'bob@example.test' WHERE id = $1`, [otherId]);
|
||||
modelReply = burgertory;
|
||||
pantryReply = { status: 200, body: { captureId: 901, items: [1, 2, 3] } };
|
||||
calls.length = 0;
|
||||
});
|
||||
|
||||
// ─── C2 ───────────────────────────────────────────────────────────────────────
|
||||
|
||||
describe("PUT /api/transactions/[id]/line-items (C2)", () => {
|
||||
const items = [{ qty: 1, description: "Michael", amount: 14, options: ["Milk Bun"] }, { qty: 1, description: "Joel", amount: 14 }];
|
||||
|
||||
it("fails closed: no token, wrong token", async () => {
|
||||
expect((await put(1, { line_items: items, source: "intake", replace: true }, null)).status).toBe(401);
|
||||
expect((await put(1, { line_items: items, source: "intake", replace: true }, "nope")).status).toBe(401);
|
||||
});
|
||||
|
||||
it("accepts either token", async () => {
|
||||
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
expect((await put(id, { line_items: items, source: "pantry", replace: true }, RECEIPT)).status).toBe(200);
|
||||
expect((await put(id, { line_items: items, source: "intake", replace: true }, INTAKE)).status).toBe(200);
|
||||
});
|
||||
|
||||
it("404 when the transaction does not exist — it never creates one", async () => {
|
||||
const res = await put(999999, { line_items: items, source: "intake", replace: true });
|
||||
expect(res.status).toBe(404);
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
|
||||
});
|
||||
|
||||
it("rejects a body that is not a replace, a bad source, or nameless items", async () => {
|
||||
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
expect((await put(id, { line_items: items, source: "intake" })).status).toBe(400);
|
||||
expect((await put(id, { line_items: items, source: "n8n", replace: true })).status).toBe(400);
|
||||
expect((await put(id, { line_items: [{ qty: 1 }], source: "intake", replace: true })).status).toBe(400);
|
||||
});
|
||||
|
||||
it("idempotent replace: returns previous + new, and a second identical call is a no-op", async () => {
|
||||
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const first = await (await put(id, { line_items: items, source: "intake", replace: true })).json();
|
||||
expect(first.previous_line_items).toEqual([]);
|
||||
expect(first.line_items).toHaveLength(2);
|
||||
expect(first.created_metadata).toBe(false);
|
||||
expect(await lineItemsOf(id)).toHaveLength(2);
|
||||
|
||||
const corrected = [{ qty: 1, description: "Michael (corrected)", amount: 14 }];
|
||||
const second = await (await put(id, { line_items: corrected, source: "pantry", replace: true })).json();
|
||||
expect(second.previous_line_items.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
expect(await lineItemsOf(id)).toEqual(corrected);
|
||||
|
||||
const third = await (await put(id, { line_items: corrected, source: "pantry", replace: true })).json();
|
||||
expect(third.previous_line_items).toEqual(corrected);
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata`)).rows[0].n).toBe(1);
|
||||
});
|
||||
|
||||
it("undo shape: PUT previous_line_items back restores the row", async () => {
|
||||
const id = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const r = await (await put(id, { line_items: items, source: "pantry", replace: true })).json();
|
||||
await put(id, { line_items: r.previous_line_items, source: "pantry", replace: true });
|
||||
expect(await lineItemsOf(id)).toEqual([]);
|
||||
});
|
||||
|
||||
it("creates the expense_metadata row when the transaction has none, with source and platform", async () => {
|
||||
const id = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
|
||||
const res = await (await put(id, { line_items: items, source: "intake", platform: "ubereats", batch_id: "b1", replace: true })).json();
|
||||
expect(res.created_metadata).toBe(true);
|
||||
const em = (await pool.query(`SELECT source, platform, order_reference, merchant_normalized, amount::text FROM expense_metadata WHERE transaction_id = $1`, [id])).rows[0];
|
||||
expect(em).toEqual({ source: "intake", platform: "ubereats", order_reference: "intake:b1", merchant_normalized: "Coles", amount: "85.74" });
|
||||
});
|
||||
|
||||
it("the undo baseline is recorded ONCE per batch: a retry gets the original, not its own write", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const original = [{ qty: 1, description: "what the order already had", amount: 22.4 }];
|
||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(original)]);
|
||||
const id = uuid();
|
||||
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
|
||||
|
||||
const first = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
|
||||
expect(first.previous_line_items).toEqual(original);
|
||||
// Pantry re-calls on every confirm and after a timeout. Before, the retry
|
||||
// was handed its OWN write back, so undo restored the thing it undid.
|
||||
const retry = await (await put(txn, { line_items: items, source: "pantry", batch_id: id, replace: true })).json();
|
||||
expect(retry.previous_line_items).toEqual(original);
|
||||
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual(original);
|
||||
|
||||
await put(txn, { line_items: retry.previous_line_items, source: "pantry", batch_id: id, replace: true });
|
||||
expect(await lineItemsOf(txn)).toEqual(original);
|
||||
});
|
||||
|
||||
it("without a batch id the baseline is still the row as read — nothing else changes", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const first = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
|
||||
expect(first.previous_line_items).toEqual([]);
|
||||
const second = await (await put(txn, { line_items: items, source: "pantry", replace: true })).json();
|
||||
expect(second.previous_line_items).toEqual(items);
|
||||
});
|
||||
|
||||
it("a pinned row that no longer exists falls back to the live one, rather than reporting a phantom write", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const live = (await pool.query(`SELECT id FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].id as number;
|
||||
const { replaceLineItems } = await import("../../lib/line-items");
|
||||
const r = await replaceLineItems(txn, items, { source: "intake", metadataId: live + 999 });
|
||||
expect(r.metadata_id).toBe(live);
|
||||
expect(await lineItemsOf(txn)).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("two writes racing to create the metadata row both resolve to the same one", async () => {
|
||||
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
|
||||
const both = await Promise.all([
|
||||
put(txn, { line_items: items, source: "intake", replace: true }),
|
||||
put(txn, { line_items: items, source: "intake", replace: true }),
|
||||
]);
|
||||
expect(both.map((r) => r.status)).toEqual([200, 200]);
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata WHERE transaction_id = $1`, [txn])).rows[0].n).toBe(1);
|
||||
});
|
||||
|
||||
it("the create path records an undo baseline too, before it writes", async () => {
|
||||
const txn = await seedOrder({ merchant: "Coles", date: "2026-08-18", amount: 85.74, withMeta: false });
|
||||
const id = uuid();
|
||||
await pool.query(`INSERT INTO intake_batches (id, source, file_sha256, status) VALUES ($1::uuid, 'other', '{}', 'received')`, [id]);
|
||||
await put(txn, { line_items: items, source: "intake", batch_id: id, currency: "AUD", replace: true });
|
||||
// [] rather than NULL: there was nothing there, and "nothing" is the state
|
||||
// undo has to be able to restore.
|
||||
expect((await pool.query(`SELECT previous_line_items FROM intake_batches WHERE id = $1`, [id])).rows[0].previous_line_items).toEqual([]);
|
||||
});
|
||||
|
||||
it("the panel reads the row the write landed on, whichever order the rows were inserted in", async () => {
|
||||
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
|
||||
// Inserted first: the delivery receipt pointing at the statement line.
|
||||
await pool.query(
|
||||
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
|
||||
[t, uuid()]
|
||||
);
|
||||
// Inserted second: a row the transaction owns. An unordered LIMIT 1 reads
|
||||
// the first one, the write takes the second, and the enrichment reports
|
||||
// success against a row nothing renders.
|
||||
await pool.query(
|
||||
`INSERT INTO expense_metadata (transaction_id, source, order_reference, line_items) VALUES ($1, 'pantry', $2, '[]'::jsonb)`,
|
||||
[t, uuid()]
|
||||
);
|
||||
const written = await (await put(t, { line_items: items, source: "intake", replace: true })).json();
|
||||
const panel = await orderPanel(t);
|
||||
expect(panel?.line_items).toHaveLength(2);
|
||||
expect((await metaRows()).find((r) => r.id === written.metadata_id)!.line_items).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("finds the row through matched_transaction_id — a card-settled order has only that", async () => {
|
||||
const statementLine = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
|
||||
await pool.query(
|
||||
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, '[]'::jsonb)`,
|
||||
[statementLine, uuid()]
|
||||
);
|
||||
const res = await (await put(statementLine, { line_items: items, source: "intake", replace: true })).json();
|
||||
expect(res.created_metadata).toBe(false);
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM expense_metadata`)).rows[0].n).toBe(1);
|
||||
expect(await lineItemsOf(statementLine)).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── C1 ───────────────────────────────────────────────────────────────────────
|
||||
|
||||
describe("POST /api/intake — gates", () => {
|
||||
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
|
||||
|
||||
it("fails closed without INTAKE_TOKEN match", async () => {
|
||||
expect((await intakePOST(multipart({ source: "uber_eats" }, [pdf], null))).status).toBe(401);
|
||||
expect((await intakePOST(multipart({ source: "uber_eats" }, [pdf], "nope"))).status).toBe(401);
|
||||
});
|
||||
|
||||
it("400 on a bad source, no files, too many files, a non-uuid batch_id", async () => {
|
||||
expect((await intakePOST(multipart({ source: "menulog" }, [pdf]))).status).toBe(400);
|
||||
expect((await intakePOST(multipart({ source: "uber_eats" }, []))).status).toBe(400);
|
||||
expect((await intakePOST(multipart({ source: "uber_eats" }, Array(7).fill(pdf)))).status).toBe(400);
|
||||
expect((await intakePOST(multipart({ source: "uber_eats", batch_id: "not-a-uuid" }, [pdf]))).status).toBe(400);
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM intake_batches`)).rows[0].n).toBe(0);
|
||||
});
|
||||
|
||||
it("declines a file type it cannot read, after persisting the batch", async () => {
|
||||
const id = uuid();
|
||||
const res = await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [{ name: "clip.mp4", type: "video/mp4", bytes: Buffer.from("x") }]));
|
||||
const body = await res.json();
|
||||
expect(body.status).toBe("unsupported");
|
||||
expect(body.text).toMatch(/not this/);
|
||||
expect((await batch(id)).status).toBe("unsupported");
|
||||
expect(calls).toHaveLength(0);
|
||||
});
|
||||
|
||||
// 2026-08-22, live batch 3d2ffbe8…: an iOS Shortcut coerced the shared photo
|
||||
// to text, so the whole upload was the five characters `Image`. The model was
|
||||
// asked anyway and returned Burgertory (Braybrook), two "The Pride" lines and
|
||||
// a $41.15 total — none of which existed. Only the absent date kept it out of
|
||||
// the matcher, and the invented header was stored on the batch as
|
||||
// `needs_header`. The model must never be asked.
|
||||
it("a text part that cannot be a receipt is declined BEFORE any model call", async () => {
|
||||
const id = uuid();
|
||||
modelReply = { ...burgertory, merchant: "Burgertory (Braybrook)", order_date: null, total: 41.15 };
|
||||
const res = await intakePOST(
|
||||
multipart({ source: "other", batch_id: id }, [{ name: "shared.txt", type: "text/plain", bytes: Buffer.from("Image") }])
|
||||
);
|
||||
const body = await res.json();
|
||||
expect(body).toMatchObject({ kind: "unsupported", status: "unsupported" });
|
||||
expect(body.text).toMatch(/single word/);
|
||||
expect(body.text).toMatch(/"Image"/);
|
||||
// The model was never asked — the only thing that makes confabulation impossible.
|
||||
expect(calls).toHaveLength(0);
|
||||
const row = await batch(id);
|
||||
expect(row.status).toBe("unsupported");
|
||||
expect(row.kind).toBe("unsupported");
|
||||
expect(row.header).toBeNull();
|
||||
expect(row.items).toBeNull();
|
||||
expect(row.transaction_id).toBeNull();
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
|
||||
});
|
||||
|
||||
it("a junk text part beside a real receipt does not sink the batch", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const body = await (
|
||||
await intakePOST(
|
||||
multipart({ source: "uber_eats" }, [
|
||||
{ name: "shared.txt", type: "text/plain", bytes: Buffer.from("Image") },
|
||||
{ name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY },
|
||||
])
|
||||
)
|
||||
).json();
|
||||
expect(body.status).toBe("enriched");
|
||||
expect(await lineItemsOf(txn)).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("an order extracted from an unreadable PDF is unsupported, not needs_header", async () => {
|
||||
const id = uuid();
|
||||
// A PDF that renders to a scrap of junk still reaches the model — the
|
||||
// pre-check only inspects text parts. `confabulationReason` is the belt.
|
||||
vi.spyOn(await import("../../lib/intake-extract"), "extractPdfText").mockResolvedValueOnce("Image");
|
||||
modelReply = { ...burgertory, merchant: "Burgertory (Braybrook)" };
|
||||
const body = await (
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [{ name: "x.pdf", type: "application/pdf", bytes: BURGERTORY }]))
|
||||
).json();
|
||||
expect(body).toMatchObject({ kind: "unsupported", status: "unsupported" });
|
||||
expect(body.text).toMatch(/nothing shared could be read/);
|
||||
const row = await batch(id);
|
||||
expect(row.status).toBe("unsupported");
|
||||
expect(row.header).toBeNull();
|
||||
expect(row.items).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("POST /api/intake — food (enrich vs candidates)", () => {
|
||||
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
|
||||
|
||||
it("food + exact (platform, merchant, date, total) match → enriches the existing order, one model call, PDF as text", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const id = uuid();
|
||||
const res = await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]));
|
||||
const body = await res.json();
|
||||
expect(res.status).toBe(200);
|
||||
expect(body).toMatchObject({ kind: "food", status: "enriched", transaction_id: txn, batch_id: id });
|
||||
expect(body.text).toBe("Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓");
|
||||
expect(JSON.stringify(body.slack_blocks)).toContain("https://finance.test/transactions?q=Burgertory");
|
||||
|
||||
const stored = await lineItemsOf(txn);
|
||||
expect(stored.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
expect(stored[0].options).toEqual(["Milk Bun"]);
|
||||
|
||||
// one joint call, the PDF's text in it (not an image)
|
||||
expect(calls).toHaveLength(1);
|
||||
const content = calls[0].body as { messages: { content: { type: string; text?: string }[] }[]; model: string; temperature: number; response_format: { type: string } };
|
||||
expect(content.model).toBe("cloud-budget");
|
||||
expect(content.temperature).toBe(0);
|
||||
expect(content.response_format.type).toBe("json_schema");
|
||||
expect(content.messages[0].content.some((c) => c.type === "image_url")).toBe(false);
|
||||
expect(content.messages[0].content.some((c) => c.text?.includes("1 Michael $14.00"))).toBe(true);
|
||||
|
||||
const row = await batch(id);
|
||||
expect(row).toMatchObject({ source: "uber_eats", shared_by: "bob@example.test", kind: "food", status: "enriched", transaction_id: txn });
|
||||
expect(row.header).toMatchObject({ merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, platform: "ubereats" });
|
||||
expect(row.resolved_at).not.toBeNull();
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
});
|
||||
|
||||
it("the platform is part of the key: the same merchant/date/total on DoorDash is not the match — but it IS offered", async () => {
|
||||
const doordash = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "doordash" });
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(body.status).toBe("candidates");
|
||||
// The candidates card is a human choice, not a decision. The platform is one
|
||||
// of the things that may be wrong, so it must not narrow the list — a
|
||||
// platform-filtered card showed nothing at all here.
|
||||
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([doordash]);
|
||||
});
|
||||
|
||||
it("a NULL platform is 'no platform', not a wildcard — an in-store share does not overwrite the delivery order", async () => {
|
||||
await pool.query(`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ('burgertory werribee', 'food')`);
|
||||
const delivery = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4, platform: "ubereats" });
|
||||
const already = [{ qty: 1, description: "what the delivery order already had", amount: 22.4 }];
|
||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [delivery, JSON.stringify(already)]);
|
||||
|
||||
// `other` implies no platform; the delivery row carries one, so it is not
|
||||
// the match — it is only a candidate.
|
||||
const body = await (await intakePOST(multipart({ source: "other" }, [pdf]))).json();
|
||||
expect(body.status).toBe("candidates");
|
||||
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([delivery]);
|
||||
expect(await lineItemsOf(delivery)).toEqual(already);
|
||||
});
|
||||
|
||||
it("a reconciled manual row is neither matched nor offered — EXCLUDE_RECONCILED_SOURCE, as everywhere else", async () => {
|
||||
const live = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const reconciled = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, reconciled]);
|
||||
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
|
||||
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [live, near]);
|
||||
|
||||
// Two rows at the exact key would have read as an ambiguity; one of them is
|
||||
// the same spend counted twice.
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(body).toMatchObject({ status: "enriched", transaction_id: live });
|
||||
});
|
||||
|
||||
it("the write lands on the row the MATCH resolved through, not on whichever row sorts first", async () => {
|
||||
const t = await seedOrder({ merchant: "UBER *EATS", date: "2026-08-19", amount: 22.4, withMeta: false });
|
||||
// The statement line owns a receipt row of its own (a pantry confirm) …
|
||||
const grocery = [{ qty: 1, description: "a grocery shop", amount: 22.4 }];
|
||||
const own = (await pool.query(
|
||||
`INSERT INTO expense_metadata (transaction_id, source, order_reference, merchant_normalized, line_items) VALUES ($1, 'pantry', $2, 'Woolworths', $3::jsonb) RETURNING id`,
|
||||
[t, uuid(), JSON.stringify(grocery)]
|
||||
)).rows[0].id as number;
|
||||
// … and is also the card leg of the Uber Eats order being enriched.
|
||||
const delivery = (await pool.query(
|
||||
`INSERT INTO expense_metadata (transaction_id, matched_transaction_id, source, platform, order_reference, merchant_normalized, line_items) VALUES (NULL, $1, 'email', 'ubereats', $2, 'Burgertory (Werribee)', '[]'::jsonb) RETURNING id`,
|
||||
[t, uuid()]
|
||||
)).rows[0].id as number;
|
||||
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(body).toMatchObject({ status: "enriched", transaction_id: t });
|
||||
const rows = await metaRows();
|
||||
expect(rows.find((r) => r.id === delivery)!.line_items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
expect(rows.find((r) => r.id === own)!.line_items).toEqual(grocery);
|
||||
});
|
||||
|
||||
it("an upload with no prices never replaces a receipt that already has items", async () => {
|
||||
const priceless = { ...burgertory, subtotal: null, total: 22.4, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
|
||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
|
||||
|
||||
modelReply = priceless;
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(body).toMatchObject({ kind: "held", status: "held" });
|
||||
expect(body.text).toMatch(/left the existing items alone/);
|
||||
expect(await lineItemsOf(txn)).toEqual(already);
|
||||
|
||||
// …but an EMPTY receipt is exactly what this slice exists to fill.
|
||||
const empty = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-20", amount: 22.4 });
|
||||
modelReply = { ...priceless, order_date: "2026-08-20" };
|
||||
const second = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(second).toMatchObject({ status: "enriched", transaction_id: empty });
|
||||
expect((await lineItemsOf(empty)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
});
|
||||
|
||||
it("food + no match → candidates (same merchant ±3 days, any amount) with Use-this / Create buttons; NEVER auto-creates", async () => {
|
||||
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
|
||||
await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-10", amount: 22.4 }); // too far
|
||||
await seedOrder({ merchant: "Mad Mex", date: "2026-08-19", amount: 22.4 }); // wrong merchant
|
||||
const id = uuid();
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
|
||||
expect(body.status).toBe("candidates");
|
||||
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id)).toEqual([near]);
|
||||
const blocks = JSON.stringify(body.slack_blocks);
|
||||
expect(blocks).toContain('"action_id":"intake_use_candidate"');
|
||||
expect(blocks).toContain('"action_id":"intake_create"');
|
||||
expect(blocks).toContain(JSON.stringify(JSON.stringify({ batch_id: id, transaction_id: near })));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(3);
|
||||
expect(await lineItemsOf(near)).toEqual([]);
|
||||
expect((await batch(id)).items).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("two exact hits is an ambiguity, not a coin toss", async () => {
|
||||
const a = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const b = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(body.status).toBe("candidates");
|
||||
expect(body.candidates.map((c: { transaction_id: number }) => c.transaction_id).sort()).toEqual([a, b].sort());
|
||||
});
|
||||
|
||||
it("missing header (no total) → needs_header, nothing written but the batch row", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
modelReply = { ...burgertory, total: null };
|
||||
const id = uuid();
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
|
||||
expect(body.status).toBe("needs_header");
|
||||
// Asserts BOTH halves of needsHeaderCard(): that it names what is actually
|
||||
// missing (total, not a blanket "total or date"), and that a uber_eats
|
||||
// source gets the share-more-of-the-screen advice rather than the in-store
|
||||
// Share-button advice. 258fe3b changed this text and updated the unit tests
|
||||
// but not this one.
|
||||
expect(body.text).toMatch(/I can see items but no total/);
|
||||
expect(body.text).toMatch(/share the part of the screen that shows it/);
|
||||
expect(await lineItemsOf(txn)).toEqual([]);
|
||||
expect((await batch(id)).status).toBe("needs_header");
|
||||
});
|
||||
|
||||
it("items that do not add up → held, nothing written", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
modelReply = { ...burgertory, items: [burgertory.items[0]] };
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(body).toMatchObject({ kind: "held", status: "held" });
|
||||
expect(body.text).toMatch(/is a shot missing/);
|
||||
expect(await lineItemsOf(txn)).toEqual([]);
|
||||
});
|
||||
|
||||
it("a model failure is a 502 with the batch marked error, not a silent 200", async () => {
|
||||
modelReply = new Error("down");
|
||||
const res = await intakePOST(multipart({ source: "uber_eats" }, [pdf]));
|
||||
expect(res.status).toBe(502);
|
||||
expect((await res.json()).status).toBe("error");
|
||||
});
|
||||
|
||||
it("re-share of the same file after it was handled → duplicate with a pointer", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const first = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: first }, [pdf]));
|
||||
calls.length = 0;
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [pdf]))).json();
|
||||
expect(body.status).toBe("duplicate");
|
||||
expect(body.duplicate_of).toEqual({ batch_id: first, transaction_id: txn, capture_id: null });
|
||||
expect(body.text).toMatch(/Already handled/);
|
||||
expect(calls).toHaveLength(0); // no model call spent on a known file
|
||||
});
|
||||
|
||||
it("re-share of the same order under a different file (screenshot after PDF) → duplicate by header", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const first = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: first }, [pdf]));
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats" }, [{ name: "IMG_1.png", type: "image/png", bytes: Buffer.from("fake-png") }]))).json();
|
||||
expect(body.status).toBe("duplicate");
|
||||
expect(body.duplicate_of.transaction_id).toBe(txn);
|
||||
});
|
||||
|
||||
it("a repeated batch_id is refused, not re-run", async () => {
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
||||
expect((await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).status).toBe(409);
|
||||
});
|
||||
});
|
||||
|
||||
describe("POST /api/intake — grocery → pantry", () => {
|
||||
const pdf = { name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES };
|
||||
|
||||
it("grocery + match → the ORIGINAL file goes to pantry with the target transaction and action enrich; finance is NOT touched", async () => {
|
||||
modelReply = coles;
|
||||
const txn = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74 });
|
||||
const id = uuid();
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]))).json();
|
||||
expect(body).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
|
||||
expect(body.text).toBe("Coles (Wyndham Vale) $85.74 → pantry board (3 items) — finance updates on confirm");
|
||||
|
||||
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown>; files: string[]; headers: Record<string, string> };
|
||||
expect(pantryCall.headers["x-ingest-token"]).toBe(RECEIPT);
|
||||
expect(pantryCall.files).toEqual(["Receipt_18Aug2026.pdf"]);
|
||||
expect(pantryCall.intake).toEqual({ batch_id: id, platform: "ubereats", origin: "intake", shared_by: "bob@example.test", finance_target_transaction_id: txn, finance_action: "enrich" });
|
||||
|
||||
expect(await lineItemsOf(txn)).toEqual([]);
|
||||
expect((await batch(id))).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901, transaction_id: txn });
|
||||
});
|
||||
|
||||
it("an in-store tap does not hand pantry the delivery order as its finance target", async () => {
|
||||
modelReply = coles;
|
||||
// The Uber Eats Coles order the email pipeline already booked.
|
||||
const delivery = await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: "ubereats" });
|
||||
// The same shop, same day, same money — but shared from the Coles app, so
|
||||
// it carries no platform and is a different purchase.
|
||||
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
||||
expect(body.status).toBe("forwarded_to_pantry");
|
||||
const pantryCall = calls.find((c) => c.url === "http://pantry.test/api/capture")!.body as { intake: Record<string, unknown> };
|
||||
expect(pantryCall.intake).toMatchObject({ finance_action: "create", finance_target_transaction_id: null, platform: null });
|
||||
expect(await lineItemsOf(delivery)).toEqual([]);
|
||||
});
|
||||
|
||||
it("grocery + no match → pantry with finance_action create (ING-10 as today)", async () => {
|
||||
modelReply = coles;
|
||||
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
||||
expect(body.status).toBe("forwarded_to_pantry");
|
||||
expect(body.text).toMatch(/finance books it on confirm/);
|
||||
const pantryCall = calls.find((c) => c.url.includes("/api/capture"))!.body as { intake: Record<string, unknown> };
|
||||
expect(pantryCall.intake).toMatchObject({ platform: null, finance_target_transaction_id: null, finance_action: "create" });
|
||||
});
|
||||
|
||||
it("pantry saying the shop is already on its board is SHOWN, not swallowed", async () => {
|
||||
// Pantry accepts a second capture of one receipt on purpose (the first may
|
||||
// have been a bad crop) and reports it rather than refusing. Dropping that
|
||||
// field turned "reported" into silence — and a duplicate nobody sees is a
|
||||
// duplicate that gets confirmed, which counts the same groceries twice
|
||||
// because pantry's addItem accumulates. Seen live on captures 66/69.
|
||||
modelReply = coles;
|
||||
pantryReply = { status: 200, body: { captureId: 902, items: [1, 2, 3], alreadyCaptured: { captureId: 66, confirmed: true } } };
|
||||
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
||||
expect(body.status).toBe("forwarded_to_pantry");
|
||||
expect(body.text).toMatch(/Already on the pantry board/);
|
||||
expect(body.text).toMatch(/capture 66 \(already confirmed\)/);
|
||||
expect(body.text).toMatch(/capture 902/);
|
||||
});
|
||||
|
||||
it("two ledger rows for one shop are reported, not treated as no match", async () => {
|
||||
// Ambiguity is not a miss. This is the exact state that produced the
|
||||
// 2026-08-18 Woolworths double-book: the shop was in the ledger twice, the
|
||||
// matcher returned nothing, and the card said finance would book it.
|
||||
modelReply = coles;
|
||||
await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: null });
|
||||
await seedOrder({ merchant: "Coles (Wyndham Vale)", date: "2026-08-18", amount: 85.74, platform: null });
|
||||
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
||||
expect(body.status).toBe("forwarded_to_pantry");
|
||||
expect(body.text).toMatch(/found more than one row it could be/);
|
||||
});
|
||||
|
||||
it("pantry 409 → duplicate", async () => {
|
||||
modelReply = coles;
|
||||
pantryReply = { status: 409, body: { error: "This receipt has already been added to the pantry.", captureId: 77 } };
|
||||
const body = await (await intakePOST(multipart({ source: "coles" }, [pdf]))).json();
|
||||
expect(body).toMatchObject({ kind: "grocery", status: "duplicate", capture_id: 77 });
|
||||
});
|
||||
|
||||
it("pantry down → 502 error, batch marked error", async () => {
|
||||
modelReply = coles;
|
||||
pantryReply = { status: 500, body: { error: "nope" } };
|
||||
const res = await intakePOST(multipart({ source: "coles" }, [pdf]));
|
||||
expect(res.status).toBe(502);
|
||||
expect((await res.json()).status).toBe("error");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── C5 ───────────────────────────────────────────────────────────────────────
|
||||
|
||||
const slackPress = (action_id: string, value: unknown) => {
|
||||
const payload = { type: "block_actions", user: { id: "U1", name: "alice" }, response_url: "https://hooks.slack.com/r/1", actions: [{ action_id, value: JSON.stringify(value) }] };
|
||||
const raw = new URLSearchParams({ payload: JSON.stringify(payload) }).toString();
|
||||
return { headers: { get: (h: string) => (h === "x-ingest-token" ? "test-order-token" : "x") }, text: async () => raw } as unknown as NextRequest;
|
||||
};
|
||||
const lastSlackUpdate = () => calls.filter((c) => c.url.includes("hooks.slack.com")).at(-1)?.body as { replace_original?: boolean; text: string; blocks?: unknown[] };
|
||||
|
||||
describe("Slack taps (C5)", () => {
|
||||
const pdf = { name: "Receipt_19Aug2026.pdf", type: "application/pdf", bytes: BURGERTORY };
|
||||
|
||||
it("intake_use_candidate enriches the chosen transaction and replaces the card", async () => {
|
||||
const near = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-17", amount: 19.9 });
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
||||
const res = await slackPOST(slackPress("intake_use_candidate", { batch_id: id, transaction_id: near }));
|
||||
expect(res.status).toBe(200);
|
||||
expect((await lineItemsOf(near)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: near });
|
||||
expect(lastSlackUpdate()).toMatchObject({ replace_original: true, text: "Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓" });
|
||||
});
|
||||
|
||||
it("intake_create creates the transaction ING-10-style: manual, owner from shared_by, items attached; a second press is a no-op", async () => {
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", shared_by: "bob@example.test", batch_id: id }, [pdf]));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
|
||||
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
const t = (await pool.query(`SELECT statement_id, owner_id, description, amount::text, category, transaction_date::text, source, source_ref FROM transactions`)).rows;
|
||||
expect(t).toEqual([{ statement_id: null, owner_id: otherId, description: "Order - Burgertory (Werribee)", amount: "22.40", category: "dining", transaction_date: "2026-08-19", source: "intake", source_ref: id }]);
|
||||
const em = (await pool.query(`SELECT source, platform, line_items FROM expense_metadata`)).rows[0];
|
||||
expect(em.source).toBe("intake");
|
||||
expect(em.platform).toBe("ubereats");
|
||||
expect(em.line_items.map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
expect(lastSlackUpdate().text).toMatch(/^Created a \$22\.40 Burgertory \(Werribee\) transaction with 2 items/);
|
||||
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
});
|
||||
|
||||
it("intake_create re-runs the match: an order booked between the card and the tap is enriched, not duplicated", async () => {
|
||||
const id = uuid();
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
|
||||
expect(body.status).toBe("candidates");
|
||||
|
||||
// The email/order bridge books the same purchase while the card sits in
|
||||
// Slack. The card's "no match" is minutes stale by the time it is pressed.
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
|
||||
expect((await lineItemsOf(txn)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
|
||||
});
|
||||
|
||||
it("intake_create adopts the wallet row for the same tap instead of booking it twice", async () => {
|
||||
// The normal arrival order: the notification lands within seconds of the
|
||||
// tap, the screenshot is shared later. matchTransaction cannot see that row
|
||||
// — its key includes the platform and a notification has none — so before
|
||||
// this, Create booked the purchase a second time.
|
||||
const wallet = await seedWalletRow({ merchant: "Burgertory", date: "2026-08-19", amount: 22.4 });
|
||||
const id = uuid();
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
|
||||
expect(body.status).toBe("candidates");
|
||||
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: wallet });
|
||||
expect((await lineItemsOf(wallet)).map((i: { description: string }) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
// The suburb the notification supplied is the one thing nothing else knows.
|
||||
expect((await pool.query(`SELECT location FROM transactions WHERE id = $1`, [wallet])).rows[0].location).toBe("Werribee, VIC");
|
||||
});
|
||||
|
||||
it("intake_create still creates when a same-amount row already carries a receipt", async () => {
|
||||
// A row that another source fully described is a DIFFERENT purchase that
|
||||
// happens to share an amount — adopting it would move one order's items
|
||||
// onto another order's row.
|
||||
const other = await seedOrder({ merchant: "Someone Else", date: "2026-08-19", amount: 22.4, platform: null });
|
||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [other, JSON.stringify([{ description: "theirs", qty: 1, amount: 22.4 }])]);
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(2);
|
||||
});
|
||||
|
||||
it("two pending cards for one order book it ONCE", async () => {
|
||||
const a = uuid();
|
||||
const b = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
|
||||
expect((await batch(a)).status).toBe("candidates");
|
||||
expect((await batch(b)).status).toBe("candidates");
|
||||
|
||||
await slackPOST(slackPress("intake_create", { batch_id: a }));
|
||||
await slackPOST(slackPress("intake_create", { batch_id: b }));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
expect((await batch(b)).transaction_id).toBe((await batch(a)).transaction_id);
|
||||
});
|
||||
|
||||
it("a sibling already resolved to the pantry board makes Create a duplicate, not a second row", async () => {
|
||||
// Two shares of one receipt, resolved out of order: nothing the second card
|
||||
// can match on, because the first one never produced a transaction.
|
||||
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
|
||||
const a = uuid();
|
||||
const b = uuid();
|
||||
await intakePOST(multipart({ source: "other", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
|
||||
await intakePOST(multipart({ source: "other", batch_id: a }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
|
||||
expect((await batch(a)).status).toBe("ask");
|
||||
|
||||
await slackPOST(slackPress("intake_grocery", { batch_id: a }));
|
||||
expect((await batch(a)).status).toBe("forwarded_to_pantry");
|
||||
|
||||
await slackPOST(slackPress("intake_meal", { batch_id: b }));
|
||||
expect((await batch(b)).status).toBe("candidates");
|
||||
await slackPOST(slackPress("intake_create", { batch_id: b }));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(0);
|
||||
expect(await batch(b)).toMatchObject({ status: "duplicate", capture_id: 901 });
|
||||
expect(lastSlackUpdate().text).toMatch(/^Already handled/);
|
||||
});
|
||||
|
||||
it("two taps that land together still book one transaction, and neither reports an error", async () => {
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
||||
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: id })), slackPOST(slackPress("intake_create", { batch_id: id }))]);
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
const texts = calls.filter((c) => c.url.includes("hooks.slack.com")).map((c) => (c.body as { text: string }).text);
|
||||
expect(texts.every((t) => !/Couldn't process/.test(t))).toBe(true);
|
||||
});
|
||||
|
||||
it("a foreign receipt is booked in its own currency, with no invented AUD figure", async () => {
|
||||
modelReply = { ...burgertory, currency: "USD" };
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
const t = (await pool.query(`SELECT amount::text, amount_aud, foreign_currency_amount::text, foreign_currency_code FROM transactions`)).rows[0];
|
||||
expect(t).toEqual({ amount: "22.40", amount_aud: null, foreign_currency_amount: "22.40", foreign_currency_code: "USD" });
|
||||
expect(lastSlackUpdate().text).toMatch(/USD 22\.40/);
|
||||
// OrderDetails prefers expense_metadata.currency, so a hard-coded 'AUD'
|
||||
// here printed "$" on a USD order's receipt panel.
|
||||
expect((await pool.query(`SELECT currency FROM expense_metadata`)).rows[0].currency).toBe("USD");
|
||||
});
|
||||
|
||||
it("the hold applies to the Create tap's re-match, not only to the upload path", async () => {
|
||||
modelReply = { ...burgertory, subtotal: null, items: burgertory.items.map((i) => ({ ...i, unit_price: null, line_total: null })) };
|
||||
const id = uuid();
|
||||
const body = await (await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]))).json();
|
||||
expect(body.status).toBe("candidates");
|
||||
|
||||
// Booked WITH its items between the card and the tap. The re-match walked
|
||||
// past the guard routeFood applies, so the same upload destroyed the same
|
||||
// receipt depending on which button reached it.
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const already = [{ qty: 1, description: "the itemised receipt already there", amount: 22.4 }];
|
||||
await pool.query(`UPDATE expense_metadata SET line_items = $2::jsonb WHERE transaction_id = $1`, [txn, JSON.stringify(already)]);
|
||||
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
expect(await lineItemsOf(txn)).toEqual(already);
|
||||
expect(await batch(id)).toMatchObject({ status: "held" });
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
});
|
||||
|
||||
it("two DIFFERENT batches for one purchase, tapped together, book it once", async () => {
|
||||
const a = uuid();
|
||||
const b = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: a }, [pdf]));
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: b }, [{ name: "shot.png", type: "image/png", bytes: Buffer.from("shot") }]));
|
||||
// Their source_refs differ, so ON CONFLICT cannot save this; both taps
|
||||
// clear their match and duplicate checks before either inserts.
|
||||
await Promise.all([slackPOST(slackPress("intake_create", { batch_id: a })), slackPOST(slackPress("intake_create", { batch_id: b }))]);
|
||||
const rows = (await pool.query(`SELECT id FROM transactions`)).rows;
|
||||
expect(rows).toHaveLength(1);
|
||||
expect((await batch(a)).transaction_id).toBe(rows[0].id);
|
||||
expect((await batch(b)).transaction_id).toBe(rows[0].id);
|
||||
});
|
||||
|
||||
it("unknown shared_by falls back to the default owner — never a NULL owner", async () => {
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", shared_by: "nobody@example.test", batch_id: id }, [pdf]));
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
expect((await pool.query(`SELECT owner_id FROM transactions`)).rows[0].owner_id).toBe(1);
|
||||
});
|
||||
|
||||
it("`other` + unrecognised merchant asks; intake_meal routes it as food and remembers the merchant", async () => {
|
||||
modelReply = { ...burgertory, merchant: "Romeo's Food Hall" };
|
||||
const txn = await seedOrder({ merchant: "Romeo's Food Hall", date: "2026-08-19", amount: 22.4, platform: null });
|
||||
const id = uuid();
|
||||
const body = await (await intakePOST(multipart({ source: "other", batch_id: id }, [pdf]))).json();
|
||||
expect(body).toMatchObject({ kind: "ask", status: "ask" });
|
||||
expect(JSON.stringify(body.slack_blocks)).toContain("intake_grocery");
|
||||
|
||||
await slackPOST(slackPress("intake_meal", { batch_id: id }));
|
||||
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
|
||||
expect(await lineItemsOf(txn)).toHaveLength(2);
|
||||
expect((await pool.query(`SELECT kind FROM intake_merchant_kinds WHERE merchant_key = 'romeo s food hall'`)).rows[0].kind).toBe("food");
|
||||
|
||||
// The answer sticks: the next share of that merchant does not ask.
|
||||
const again = await (await intakePOST(multipart({ source: "other" }, [{ name: "IMG.png", type: "image/png", bytes: Buffer.from("png2") }]))).json();
|
||||
expect(again.status).toBe("duplicate"); // same order — but it was routed, not asked
|
||||
modelReply = { ...burgertory, merchant: "Romeo's Food Hall", order_date: "2026-08-20" };
|
||||
const next = await (await intakePOST(multipart({ source: "other" }, [{ name: "IMG3.png", type: "image/png", bytes: Buffer.from("png3") }]))).json();
|
||||
expect(next).toMatchObject({ kind: "food", status: "candidates" });
|
||||
});
|
||||
|
||||
it("intake_grocery forwards the stored original files to pantry", async () => {
|
||||
modelReply = { ...coles, merchant: "Romeo's Food Hall" };
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "other", batch_id: id }, [{ name: "Receipt_18Aug2026.pdf", type: "application/pdf", bytes: COLES }]));
|
||||
expect((await batch(id)).status).toBe("ask");
|
||||
await slackPOST(slackPress("intake_grocery", { batch_id: id }));
|
||||
const pantryCall = calls.find((c) => c.url.includes("/api/capture"))!.body as { intake: Record<string, unknown>; files: string[] };
|
||||
expect(pantryCall.files).toEqual(["Receipt_18Aug2026.pdf"]);
|
||||
expect(pantryCall.intake).toMatchObject({ batch_id: id, finance_action: "create" });
|
||||
expect(await batch(id)).toMatchObject({ kind: "grocery", status: "forwarded_to_pantry", capture_id: 901 });
|
||||
expect((await pool.query(`SELECT kind FROM intake_merchant_kinds`)).rows[0].kind).toBe("grocery");
|
||||
});
|
||||
|
||||
it("a tap on a resolved batch returns its card and writes nothing; an unknown batch says so", async () => {
|
||||
const txn = await seedOrder({ merchant: "Burgertory (Werribee)", date: "2026-08-19", amount: 22.4 });
|
||||
const id = uuid();
|
||||
await intakePOST(multipart({ source: "uber_eats", batch_id: id }, [pdf]));
|
||||
await slackPOST(slackPress("intake_create", { batch_id: id }));
|
||||
expect((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n).toBe(1);
|
||||
expect(lastSlackUpdate().text).toMatch(/^Added 2 items/);
|
||||
expect(await batch(id)).toMatchObject({ status: "enriched", transaction_id: txn });
|
||||
|
||||
await slackPOST(slackPress("intake_create", { batch_id: uuid() }));
|
||||
expect(lastSlackUpdate()).toMatchObject({ replace_original: false, text: "I can't find that upload any more." });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,175 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants, insertTransaction } from "./helpers";
|
||||
|
||||
/**
|
||||
* Manually resolving a duplicate the automation cannot see.
|
||||
*
|
||||
* Until this route existed there was no way to merge or remove a transaction at
|
||||
* all: `superseded_by_id` was read everywhere and written nowhere in
|
||||
* application code, having been set exactly once by migration 0023. The only
|
||||
* merge path was the reconcile queue, whose candidate list is bounded by an
|
||||
* amount band -- so SP PEAKDESIGN (notified $42.44, charged $43.71, the card's
|
||||
* 3% foreign transaction fee) had no offerable twin.
|
||||
*
|
||||
* The refusals matter more than the happy paths, exactly as in
|
||||
* auto-reconcile.test.ts: this writes to columns that hide a row from every
|
||||
* figure, so a wrong call does not error, it quietly stops counting money.
|
||||
*/
|
||||
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let otherId: number;
|
||||
let merge: typeof import("../../app/api/transactions/[id]/merge/route").POST;
|
||||
|
||||
const req = (body: unknown) => ({ json: async () => body, headers: { get: () => null } }) as never;
|
||||
const call = (id: number, body: unknown) => merge(req(body), { params: Promise.resolve({ id: String(id) }) });
|
||||
|
||||
// uq_statement_identity is (bank_name, account_number, billing_end_date), so
|
||||
// every statement in a test needs its own account or the second insert collides.
|
||||
let acctSeq = 0;
|
||||
|
||||
async function statementLine(amount: number, date: string, description: string) {
|
||||
const st = await pool.query(
|
||||
`INSERT INTO statements (filename, account_number, bank_name, billing_start_date, billing_end_date, owner_id)
|
||||
VALUES ('s.pdf', $3, 'Test Bank', $1::date - 20, $1::date + 10, $2) RETURNING id`,
|
||||
[date, ownerId, `4444-0000-0000-${String(1000 + acctSeq++)}`]
|
||||
);
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions (statement_id, transaction_date, description, amount, transaction_type, owner_id)
|
||||
VALUES ($1, $2, $3, $4, 'debit', $5) RETURNING id`,
|
||||
[st.rows[0].id, date, description, amount, ownerId]
|
||||
);
|
||||
return t.rows[0].id as number;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
pool = createPool();
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
|
||||
}));
|
||||
({ POST: merge } = await import("../../app/api/transactions/[id]/merge/route"));
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
await pool.query("DELETE FROM expense_metadata");
|
||||
({ ownerId, otherId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
describe("reconcile mode", () => {
|
||||
it("folds a wallet row into its statement twin even though the amounts differ by the FX fee", async () => {
|
||||
const wallet = await insertTransaction(pool, ownerId, {
|
||||
description: "SP PEAKDESIGN", amount: 42.44, transaction_date: "2026-08-16",
|
||||
});
|
||||
const stmt = await statementLine(43.71, "2026-08-15", "SP PEAKDESIGN SAN FRANCISCO USA");
|
||||
|
||||
const res = await call(wallet, { other_id: stmt, mode: "reconcile" });
|
||||
expect(res.status ?? 200).toBe(200);
|
||||
|
||||
const r = await pool.query(`SELECT reconciled_with_id FROM transactions WHERE id = $1`, [wallet]);
|
||||
expect(r.rows[0].reconciled_with_id).toBe(stmt);
|
||||
});
|
||||
|
||||
it("refuses when the target is not a statement row", async () => {
|
||||
const a = await insertTransaction(pool, ownerId, { amount: 10 });
|
||||
const b = await insertTransaction(pool, ownerId, { amount: 10 });
|
||||
const res = await call(a, { other_id: b, mode: "reconcile" });
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
|
||||
it("refuses when the source is itself a statement row", async () => {
|
||||
const s1 = await statementLine(10, "2026-08-01", "A");
|
||||
const s2 = await statementLine(10, "2026-08-02", "B");
|
||||
const res = await call(s1, { other_id: s2, mode: "reconcile" });
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
});
|
||||
|
||||
describe("supersede mode", () => {
|
||||
it("carries splits onto the survivor before hiding the loser", async () => {
|
||||
// The whole reason supersede moves curation: `superseded_by_id` removes the
|
||||
// row from every figure, so a split left behind stops counting silently and
|
||||
// a settled debt reappears as owed.
|
||||
const loser = await insertTransaction(pool, ownerId, { amount: 80, description: "DUP" });
|
||||
const survivor = await insertTransaction(pool, ownerId, { amount: 80, description: "KEEP" });
|
||||
await pool.query(
|
||||
`INSERT INTO transaction_splits (transaction_id, participant_id, share_percent, settled)
|
||||
VALUES ($1,$2,50,true), ($1,$3,50,false)`,
|
||||
[loser, ownerId, otherId]
|
||||
);
|
||||
|
||||
const res = await call(loser, { other_id: survivor, mode: "supersede" });
|
||||
expect(res.status ?? 200).toBe(200);
|
||||
|
||||
const onLoser = await pool.query(`SELECT count(*)::int c FROM transaction_splits WHERE transaction_id=$1`, [loser]);
|
||||
const onSurvivor = await pool.query(
|
||||
`SELECT participant_id, share_percent, settled FROM transaction_splits
|
||||
WHERE transaction_id=$1 ORDER BY participant_id`, [survivor]
|
||||
);
|
||||
expect(onLoser.rows[0].c).toBe(0);
|
||||
expect(onSurvivor.rows).toHaveLength(2);
|
||||
// settled must survive the move -- a delete-and-recreate defaults it to false.
|
||||
expect(onSurvivor.rows.map((r) => r.settled)).toContain(true);
|
||||
|
||||
const marked = await pool.query(`SELECT superseded_by_id FROM transactions WHERE id=$1`, [loser]);
|
||||
expect(marked.rows[0].superseded_by_id).toBe(survivor);
|
||||
});
|
||||
|
||||
it("works between two statement rows, which reconcile cannot express", async () => {
|
||||
const s1 = await statementLine(25, "2026-08-01", "IMPORTED TWICE");
|
||||
const s2 = await statementLine(25, "2026-08-01", "IMPORTED TWICE");
|
||||
const res = await call(s1, { other_id: s2, mode: "supersede" });
|
||||
expect(res.status ?? 200).toBe(200);
|
||||
const r = await pool.query(`SELECT superseded_by_id FROM transactions WHERE id=$1`, [s1]);
|
||||
expect(r.rows[0].superseded_by_id).toBe(s2);
|
||||
});
|
||||
|
||||
it("refuses to hide a statement row behind a statementless one", async () => {
|
||||
// The wrong direction, and silently wrong: a wallet row holds the amount
|
||||
// BEFORE the card's foreign transaction fee, so burying the statement line
|
||||
// under it leaves the ledger holding $42.44 for a $43.71 charge.
|
||||
const stmt = await statementLine(43.71, "2026-08-15", "SP PEAKDESIGN SAN FRANCISCO USA");
|
||||
const wallet = await insertTransaction(pool, ownerId, {
|
||||
description: "SP PEAKDESIGN", amount: 42.44, transaction_date: "2026-08-16",
|
||||
});
|
||||
const res = await call(stmt, { other_id: wallet, mode: "supersede" });
|
||||
expect(res.status).toBe(400);
|
||||
// and the statement row is untouched
|
||||
const r = await pool.query(`SELECT superseded_by_id FROM transactions WHERE id=$1`, [stmt]);
|
||||
expect(r.rows[0].superseded_by_id).toBeNull();
|
||||
});
|
||||
|
||||
it("refuses to bury a row under one that is already superseded", async () => {
|
||||
const a = await insertTransaction(pool, ownerId, { amount: 5 });
|
||||
const b = await insertTransaction(pool, ownerId, { amount: 5 });
|
||||
const c = await insertTransaction(pool, ownerId, { amount: 5 });
|
||||
await call(b, { other_id: c, mode: "supersede" }); // b -> c
|
||||
const res = await call(a, { other_id: b, mode: "supersede" }); // a -> b, but b is buried
|
||||
expect(res.status).toBe(409);
|
||||
});
|
||||
});
|
||||
|
||||
describe("refusals common to both modes", () => {
|
||||
it("refuses a transaction merging into itself", async () => {
|
||||
const a = await insertTransaction(pool, ownerId, { amount: 1 });
|
||||
const res = await call(a, { other_id: a, mode: "supersede" });
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
|
||||
it("refuses a row that is already reconciled", async () => {
|
||||
const wallet = await insertTransaction(pool, ownerId, { amount: 12, transaction_date: "2026-08-10" });
|
||||
const stmt = await statementLine(12, "2026-08-10", "X");
|
||||
await call(wallet, { other_id: stmt, mode: "reconcile" });
|
||||
const other = await insertTransaction(pool, ownerId, { amount: 12 });
|
||||
const res = await call(wallet, { other_id: other, mode: "supersede" });
|
||||
expect(res.status).toBe(409);
|
||||
});
|
||||
|
||||
it("refuses a transaction belonging to someone else", async () => {
|
||||
const mine = await insertTransaction(pool, ownerId, { amount: 7 });
|
||||
const theirs = await insertTransaction(pool, otherId, { amount: 7 });
|
||||
const res = await call(mine, { other_id: theirs, mode: "supersede" });
|
||||
expect(res.status).toBe(404);
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, it, expect, beforeAll } from "vitest";
|
||||
import { readFileSync } from "fs";
|
||||
import { resolve } from "path";
|
||||
import type { NextRequest } from "next/server";
|
||||
import { queryRaw } from "../../lib/db";
|
||||
|
||||
/**
|
||||
@@ -13,13 +14,22 @@ const dir = resolve(__dirname, "../fixtures/orders/real");
|
||||
const html = (f: string) => readFileSync(resolve(dir, `${f}.html`), "utf-8");
|
||||
|
||||
const TOKEN = "test-ingest-token";
|
||||
let POST: any;
|
||||
|
||||
// The handler is imported dynamically (after the env var is set), so its type
|
||||
// is declared here rather than inferred. `unknown` for the request is
|
||||
// deliberate: these fixtures are the two properties the route actually reads,
|
||||
// not a whole NextRequest, and widening to `any` would stop the compiler
|
||||
// checking the call sites too.
|
||||
let POST: (req: NextRequest) => Promise<Response>;
|
||||
|
||||
// One cast, at the boundary, instead of `any` everywhere downstream: the
|
||||
// fixture is the two properties the route actually reads, and everything past
|
||||
// this line stays type-checked.
|
||||
const req = (body: unknown, token: string | null = TOKEN) =>
|
||||
({
|
||||
headers: { get: (h: string) => (h === "x-ingest-token" ? token : null) },
|
||||
json: async () => body,
|
||||
}) as any;
|
||||
}) as unknown as NextRequest;
|
||||
|
||||
beforeAll(async () => {
|
||||
process.env.ORDER_INGEST_TOKEN = TOKEN;
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import type { NextRequest } from "next/server";
|
||||
import { createPool, mockDbWithPool, resetDB } from "./helpers";
|
||||
|
||||
/**
|
||||
@@ -12,15 +13,41 @@ import { createPool, mockDbWithPool, resetDB } from "./helpers";
|
||||
const pool = createPool();
|
||||
mockDbWithPool(pool);
|
||||
|
||||
let GET: any;
|
||||
let PUT: any;
|
||||
/** What the review route returns, to the extent these tests read it. */
|
||||
type ItemVerdict = { item: string; verdict: string };
|
||||
type Review = {
|
||||
participant_id: number;
|
||||
rating: string | null;
|
||||
note: string | null;
|
||||
order_again: boolean | null;
|
||||
item_verdicts: ItemVerdict[];
|
||||
};
|
||||
type ReviewBody = {
|
||||
reviews: Review[];
|
||||
splits: { participant_id: number; share_percent: number }[];
|
||||
merchant: {
|
||||
warn: boolean;
|
||||
note: string | null;
|
||||
counts: Record<string, number>;
|
||||
history: { transaction_id: number }[];
|
||||
items: { item: string; loved: number; never: number }[];
|
||||
} | null;
|
||||
};
|
||||
type Ctx = { params: Promise<{ id: string }> };
|
||||
|
||||
// Declared rather than inferred because the route is imported dynamically,
|
||||
// after vi.doMock. `unknown` for the request keeps the fixture honest — it is
|
||||
// the two properties the handler reads, not a NextRequest.
|
||||
let GET: (req: NextRequest, ctx: Ctx) => Promise<Response>;
|
||||
let PUT: (req: NextRequest, ctx: Ctx) => Promise<Response>;
|
||||
let ownerId: number;
|
||||
let otherId: number;
|
||||
let txnId: number;
|
||||
|
||||
const req = (body?: unknown) =>
|
||||
({ headers: { get: () => null }, json: async () => body }) as any;
|
||||
const params = (id: number) => ({ params: Promise.resolve({ id: String(id) }) });
|
||||
({ headers: { get: () => null }, json: async () => body }) as unknown as NextRequest;
|
||||
const params = (id: number): Ctx => ({ params: Promise.resolve({ id: String(id) }) });
|
||||
const bodyOf = async (res: Response): Promise<ReviewBody> => (await res.json()) as ReviewBody;
|
||||
|
||||
beforeAll(async () => {
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
@@ -59,25 +86,25 @@ describe("order verdicts — one per person", () => {
|
||||
it("keeps both people's verdicts on the same order", async () => {
|
||||
await PUT(req({ participant_id: ownerId, rating: "loved" }), params(txnId));
|
||||
const res = await PUT(req({ participant_id: otherId, rating: "never" }), params(txnId));
|
||||
const body = await res.json();
|
||||
const body = await bodyOf(res);
|
||||
|
||||
// The bug this guards: a UNIQUE on transaction_id alone made the second
|
||||
// save overwrite the first, and the disagreement is the useful part.
|
||||
expect(body.reviews).toHaveLength(2);
|
||||
expect(body.reviews.find((r: any) => r.participant_id === ownerId).rating).toBe("loved");
|
||||
expect(body.reviews.find((r: any) => r.participant_id === otherId).rating).toBe("never");
|
||||
expect(body.reviews.find((r) => r.participant_id === ownerId)?.rating).toBe("loved");
|
||||
expect(body.reviews.find((r) => r.participant_id === otherId)?.rating).toBe("never");
|
||||
});
|
||||
|
||||
it("defaults the verdict to the signed-in user, not the owner", async () => {
|
||||
const res = await PUT(req({ rating: "ok" }), params(txnId));
|
||||
const body = await res.json();
|
||||
const body = await bodyOf(res);
|
||||
expect(body.reviews[0].participant_id).toBe(ownerId);
|
||||
});
|
||||
|
||||
it("revising a verdict updates rather than duplicating", async () => {
|
||||
await PUT(req({ participant_id: ownerId, rating: "loved" }), params(txnId));
|
||||
const res = await PUT(req({ participant_id: ownerId, rating: "never" }), params(txnId));
|
||||
const body = await res.json();
|
||||
const body = await bodyOf(res);
|
||||
expect(body.reviews).toHaveLength(1);
|
||||
expect(body.reviews[0].rating).toBe("never");
|
||||
});
|
||||
@@ -88,12 +115,12 @@ describe("order verdicts — one per person", () => {
|
||||
});
|
||||
|
||||
it("derives order_again from the rating", async () => {
|
||||
let body = await (await PUT(req({ rating: "never" }), params(txnId))).json();
|
||||
let body = await bodyOf(await PUT(req({ rating: "never" }), params(txnId)));
|
||||
expect(body.reviews[0].order_again).toBe(false);
|
||||
body = await (await PUT(req({ rating: "ok" }), params(txnId))).json();
|
||||
body = await bodyOf(await PUT(req({ rating: "ok" }), params(txnId)));
|
||||
expect(body.reviews[0].order_again).toBe(true);
|
||||
// "bad" is also a no — you would not choose it again.
|
||||
body = await (await PUT(req({ rating: "bad" }), params(txnId))).json();
|
||||
body = await bodyOf(await PUT(req({ rating: "bad" }), params(txnId)));
|
||||
expect(body.reviews[0].order_again).toBe(false);
|
||||
});
|
||||
|
||||
@@ -104,13 +131,13 @@ describe("order verdicts — one per person", () => {
|
||||
const older = await seedOrder("Thai Palace", "2026-06-01");
|
||||
await PUT(req({ participant_id: ownerId, rating: "bad" }), params(older));
|
||||
|
||||
let body = await (await GET(req(), params(txnId))).json();
|
||||
expect(body.merchant.counts.bad).toBe(1);
|
||||
expect(body.merchant.warn).toBe(false);
|
||||
let body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.merchant!.counts.bad).toBe(1);
|
||||
expect(body.merchant!.warn).toBe(false);
|
||||
|
||||
await PUT(req({ participant_id: ownerId, rating: "never" }), params(older));
|
||||
body = await (await GET(req(), params(txnId))).json();
|
||||
expect(body.merchant.warn).toBe(true);
|
||||
body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.merchant!.warn).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -125,7 +152,7 @@ describe("item verdicts", () => {
|
||||
);
|
||||
// No item_verdicts key at all — the shape a note-only form sends.
|
||||
const res = await PUT(req({ rating: "liked", note: "slow delivery" }), params(txnId));
|
||||
const body = await res.json();
|
||||
const body = await bodyOf(res);
|
||||
expect(body.reviews[0].item_verdicts).toEqual([
|
||||
{ item: "Pad Thai", verdict: "loved" },
|
||||
]);
|
||||
@@ -138,7 +165,7 @@ describe("item verdicts", () => {
|
||||
params(txnId)
|
||||
);
|
||||
const res = await PUT(req({ rating: "liked", item_verdicts: [] }), params(txnId));
|
||||
const body = await res.json();
|
||||
const body = await bodyOf(res);
|
||||
expect(body.reviews[0].item_verdicts).toEqual([]);
|
||||
});
|
||||
|
||||
@@ -154,7 +181,7 @@ describe("item verdicts", () => {
|
||||
}),
|
||||
params(txnId)
|
||||
);
|
||||
const body = await res.json();
|
||||
const body = await bodyOf(res);
|
||||
expect(body.reviews[0].rating).toBe("ok");
|
||||
expect(body.reviews[0].item_verdicts).toEqual([
|
||||
{ item: "Pad Thai", verdict: "loved" },
|
||||
@@ -167,10 +194,10 @@ describe("merchant history", () => {
|
||||
const older = await seedOrder("Thai Palace", "2026-06-01");
|
||||
await PUT(req({ participant_id: ownerId, rating: "never", note: "cold" }), params(older));
|
||||
|
||||
const body = await (await GET(req(), params(txnId))).json();
|
||||
expect(body.merchant.warn).toBe(true);
|
||||
expect(body.merchant.counts.never).toBe(1);
|
||||
expect(body.merchant.history.map((h: any) => h.transaction_id)).toEqual([older]);
|
||||
const body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.merchant!.warn).toBe(true);
|
||||
expect(body.merchant!.counts.never).toBe(1);
|
||||
expect(body.merchant!.history.map((h) => h.transaction_id)).toEqual([older]);
|
||||
});
|
||||
|
||||
it("treats the same restaurant as one merchant across platforms", async () => {
|
||||
@@ -181,17 +208,17 @@ describe("merchant history", () => {
|
||||
const shouty = await seedOrder("THAI PALACE", "2026-06-01");
|
||||
await PUT(req({ participant_id: ownerId, rating: "never" }), params(shouty));
|
||||
|
||||
const body = await (await GET(req(), params(txnId))).json();
|
||||
expect(body.merchant.warn).toBe(true);
|
||||
expect(body.merchant.history.map((h: any) => h.transaction_id)).toContain(shouty);
|
||||
const body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.merchant!.warn).toBe(true);
|
||||
expect(body.merchant!.history.map((h) => h.transaction_id)).toContain(shouty);
|
||||
});
|
||||
|
||||
it("does not carry a verdict across different merchants", async () => {
|
||||
const other = await seedOrder("Pizza Place", "2026-06-01");
|
||||
await PUT(req({ participant_id: ownerId, rating: "never" }), params(other));
|
||||
|
||||
const body = await (await GET(req(), params(txnId))).json();
|
||||
expect(body.merchant.warn).toBe(false);
|
||||
const body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.merchant!.warn).toBe(false);
|
||||
});
|
||||
|
||||
it("pools item opinions across the merchant's orders", async () => {
|
||||
@@ -206,10 +233,10 @@ describe("merchant history", () => {
|
||||
params(older2)
|
||||
);
|
||||
|
||||
const body = await (await GET(req(), params(txnId))).json();
|
||||
const body = await bodyOf(await GET(req(), params(txnId)));
|
||||
// Case-folded: the same dish comes back capitalised differently between
|
||||
// receipts, and two entries for one dish is not a track record.
|
||||
expect(body.merchant.items).toEqual([{ item: "Pad Thai", loved: 2, never: 0 }]);
|
||||
expect(body.merchant!.items).toEqual([{ item: "Pad Thai", loved: 2, never: 0 }]);
|
||||
});
|
||||
|
||||
it("keeps item opinions from reviews that have no overall rating", async () => {
|
||||
@@ -218,14 +245,14 @@ describe("merchant history", () => {
|
||||
req({ item_verdicts: [{ item: "Satay", verdict: "never" }] }),
|
||||
params(older)
|
||||
);
|
||||
const body = await (await GET(req(), params(txnId))).json();
|
||||
expect(body.merchant.items).toEqual([{ item: "Satay", loved: 0, never: 1 }]);
|
||||
const body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.merchant!.items).toEqual([{ item: "Satay", loved: 0, never: 1 }]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("share state", () => {
|
||||
it("reports splits so the panel can show shared vs just me", async () => {
|
||||
let body = await (await GET(req(), params(txnId))).json();
|
||||
let body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.splits).toEqual([]);
|
||||
|
||||
await pool.query(
|
||||
@@ -233,7 +260,7 @@ describe("share state", () => {
|
||||
VALUES ($1, $2, 50)`,
|
||||
[txnId, otherId]
|
||||
);
|
||||
body = await (await GET(req(), params(txnId))).json();
|
||||
body = await bodyOf(await GET(req(), params(txnId)));
|
||||
expect(body.splits).toHaveLength(1);
|
||||
expect(body.splits[0].participant_id).toBe(otherId);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants } from "./helpers";
|
||||
|
||||
/**
|
||||
* A receipt arriving after the wallet notification must ENRICH that row, not
|
||||
* create a second one.
|
||||
*
|
||||
* The case this is built from is real. Woolworths $103.30 on 2026-08-18: the
|
||||
* wallet event landed at 09:00:14 on the 18th, and nothing created a
|
||||
* transaction until 20:25:06 on the 19th -- 35h25m later -- at which point the
|
||||
* receipt path made its own row from scratch. One shop, two rows, and the
|
||||
* notification's contribution (the instrument, and the suburb) thrown away.
|
||||
*
|
||||
* Sources for one purchase arrive in sequence, not in competition, so the rule
|
||||
* is: the receipt read the register tape and wins on what was bought; anything
|
||||
* only the notification knew survives untouched.
|
||||
*/
|
||||
|
||||
const TOKEN = "test-receipt-token";
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let ingest: typeof import("../../app/api/receipts/ingest/route").POST;
|
||||
|
||||
const ingestReq = (body: unknown) =>
|
||||
({ headers: { get: (h: string) => (h === "x-ingest-token" ? TOKEN : null) }, json: async () => body }) as never;
|
||||
|
||||
/** A row shaped the way the wallet lane writes one: statementless, no line
|
||||
* items, carrying the suburb the notification supplied. */
|
||||
async function walletRow(amount: number, date: string, merchant: string, location: string) {
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions
|
||||
(owner_id, statement_id, transaction_date, description, amount, amount_aud,
|
||||
transaction_type, category, payment_method, merchant_name, merchant_normalized,
|
||||
location, source, source_ref, row_index)
|
||||
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'other', 'card', $3, $3, $5, 'wallet', $6, 0)
|
||||
RETURNING id`,
|
||||
[ownerId, date, merchant, amount, location, `evt-${amount}-${date}`]
|
||||
);
|
||||
const id = t.rows[0].id as number;
|
||||
await pool.query(
|
||||
`INSERT INTO expense_metadata
|
||||
(transaction_id, source, merchant_normalized, amount, transaction_date,
|
||||
currency, line_items, payment_method_detail, flags)
|
||||
VALUES ($1, 'wallet', $2, $3, $4, 'AUD', '[]'::jsonb, 'Visa', '[]'::jsonb)`,
|
||||
[id, merchant, amount, date]
|
||||
);
|
||||
return id;
|
||||
}
|
||||
|
||||
const receipt = (amount: number, date: string) => ({
|
||||
receipt_uid: `woolworths:3345:${amount}:2026-08-18`,
|
||||
capture_event_id: 900,
|
||||
merchant_name: "Woolworths",
|
||||
store_detail: "Wyndham Vale",
|
||||
transaction_date: date,
|
||||
total: amount,
|
||||
tender_legs: [
|
||||
{ leg_index: 1, amount, card_last4: "6078", card_product: "VISA PREPAID", class: "gift_card" },
|
||||
],
|
||||
line_items: [
|
||||
{ description: "Milk 2L", qty: 1, unit: "ea", amount: 3.5, category: "dairy" },
|
||||
{ description: "Bread", qty: 1, unit: "ea", amount: 4.0, category: "bakery" },
|
||||
],
|
||||
});
|
||||
|
||||
beforeAll(async () => {
|
||||
process.env.RECEIPT_INGEST_TOKEN = TOKEN;
|
||||
pool = createPool();
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
|
||||
}));
|
||||
({ POST: ingest } = await import("../../app/api/receipts/ingest/route"));
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
await pool.query("DELETE FROM expense_metadata");
|
||||
({ ownerId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
const txCount = async () =>
|
||||
Number((await pool.query(`SELECT count(*)::int AS n FROM transactions`)).rows[0].n);
|
||||
|
||||
describe("a receipt meeting the wallet row for the same tap", () => {
|
||||
it("adopts it instead of creating a second transaction", async () => {
|
||||
const walletId = await walletRow(103.3, "2026-08-18", "Woolworths", "Wyndham Vale, VIC");
|
||||
expect(await txCount()).toBe(1);
|
||||
|
||||
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
|
||||
|
||||
expect(await txCount()).toBe(1);
|
||||
const meta = await pool.query(
|
||||
`SELECT transaction_id, jsonb_array_length(line_items) AS items, source,
|
||||
payment_method_detail, card_last4
|
||||
FROM expense_metadata`
|
||||
);
|
||||
expect(meta.rowCount).toBe(1);
|
||||
expect(meta.rows[0].transaction_id).toBe(walletId);
|
||||
expect(meta.rows[0].items).toBe(2);
|
||||
expect(meta.rows[0].source).toBe("pantry");
|
||||
// The receipt read the card off the tape; the notification only saw "Visa".
|
||||
expect(meta.rows[0].payment_method_detail).toBe("VISA PREPAID");
|
||||
expect(meta.rows[0].card_last4).toBe("6078");
|
||||
});
|
||||
|
||||
it("keeps the suburb only the notification knew", async () => {
|
||||
await walletRow(103.3, "2026-08-18", "Woolworths", "Wyndham Vale, VIC");
|
||||
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
|
||||
|
||||
const t = await pool.query(`SELECT location, merchant_normalized, category FROM transactions`);
|
||||
// `location` is absent from the adoption UPDATE's column list on purpose:
|
||||
// the receipt has no suburb, so it must not be able to erase one.
|
||||
expect(t.rows[0].location).toBe("Wyndham Vale, VIC");
|
||||
// ...while what the receipt does know wins.
|
||||
expect(t.rows[0].merchant_normalized).toBe("Woolworths");
|
||||
expect(t.rows[0].category).toBe("groceries");
|
||||
});
|
||||
|
||||
it("creates its own row when there is no wallet row to adopt", async () => {
|
||||
await ingest(ingestReq(receipt(58.4, "2026-08-18")));
|
||||
expect(await txCount()).toBe(1);
|
||||
const meta = await pool.query(`SELECT source, jsonb_array_length(line_items) AS items FROM expense_metadata`);
|
||||
expect(meta.rows[0].source).toBe("pantry");
|
||||
expect(meta.rows[0].items).toBe(2);
|
||||
});
|
||||
|
||||
it("refuses to adopt when two candidates share the amount", async () => {
|
||||
// Two taps of the same value in the window: adopting either would move this
|
||||
// shop's line items onto a purchase it did not describe.
|
||||
await walletRow(103.3, "2026-08-17", "Woolworths", "Wyndham Vale, VIC");
|
||||
await walletRow(103.3, "2026-08-18", "Coles", "Manor Lakes, VIC");
|
||||
expect(await txCount()).toBe(2);
|
||||
|
||||
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
|
||||
|
||||
// Falls back to creating its own row rather than guessing.
|
||||
expect(await txCount()).toBe(3);
|
||||
});
|
||||
|
||||
it("refuses to adopt a row that already carries line items", async () => {
|
||||
const other = await walletRow(103.3, "2026-08-18", "Woolworths", "Wyndham Vale, VIC");
|
||||
await pool.query(
|
||||
`UPDATE expense_metadata SET line_items = '[{"description":"Something else","amount":1}]'::jsonb
|
||||
WHERE transaction_id = $1`, [other]
|
||||
);
|
||||
|
||||
await ingest(ingestReq(receipt(103.3, "2026-08-18")));
|
||||
|
||||
// A fully-described row is a different purchase that happens to share an amount.
|
||||
expect(await txCount()).toBe(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,86 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants, insertTransaction } from "./helpers";
|
||||
import { getPendingReconciliations } from "../../lib/queries";
|
||||
|
||||
/**
|
||||
* The reconcile queue must not offer what is already resolved.
|
||||
*
|
||||
* A superseded row is hidden from every figure (EXCLUDE_RECONCILED_SOURCE), so
|
||||
* listing it is not merely noise: on 2026-10-03 the queue showed the duplicate
|
||||
* wallet half 7037, superseded under its twin 7038, as a second "Digidirect"
|
||||
* with the same statement line offered as its match. Accepting it there would
|
||||
* have spent that line on a hidden row and left the real one unmatched forever.
|
||||
* findAutoMatches() always had both filters; this query had neither.
|
||||
*/
|
||||
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let acctSeq = 0;
|
||||
|
||||
async function statementLine(amount: number, date: string, description: string) {
|
||||
const st = await pool.query(
|
||||
`INSERT INTO statements (filename, account_number, bank_name, billing_start_date, billing_end_date, owner_id)
|
||||
VALUES ('q.pdf', $3, 'Test Bank', $1::date - 20, $1::date + 10, $2) RETURNING id`,
|
||||
[date, ownerId, `5555-0000-0000-${String(1000 + acctSeq++)}`]
|
||||
);
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions (statement_id, transaction_date, description, amount, transaction_type, owner_id)
|
||||
VALUES ($1, $2, $3, $4, 'debit', $5) RETURNING id`,
|
||||
[st.rows[0].id, date, description, amount, ownerId]
|
||||
);
|
||||
return t.rows[0].id as number;
|
||||
}
|
||||
|
||||
beforeAll(() => { pool = createPool(); });
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
({ ownerId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
describe("getPendingReconciliations", () => {
|
||||
it("does not list a superseded manual row", async () => {
|
||||
const survivor = await insertTransaction(pool, ownerId, { description: "DIGIDIRECT", amount: 1180.05, transaction_date: "2026-08-25" });
|
||||
const hidden = await insertTransaction(pool, ownerId, { description: "Digidirect", amount: 1180.05, transaction_date: "2026-08-25" });
|
||||
await pool.query(`UPDATE transactions SET superseded_by_id = $1 WHERE id = $2`, [survivor, hidden]);
|
||||
const line = await statementLine(1180.05, "2026-08-25", "SP DIGIDIRECT STRATHFIELD");
|
||||
|
||||
const queue = await getPendingReconciliations(ownerId);
|
||||
const ids = queue.map((r) => r.id);
|
||||
expect(ids).toContain(survivor);
|
||||
expect(ids).not.toContain(hidden);
|
||||
// The survivor is still offered its line.
|
||||
const row = queue.find((r) => r.id === survivor)!;
|
||||
expect(row.matches.map((m) => m.id)).toContain(line);
|
||||
});
|
||||
|
||||
it("does not offer a superseded statement row as a match", async () => {
|
||||
const manual = await insertTransaction(pool, ownerId, { description: "Grocer", amount: 42.0, transaction_date: "2026-08-10" });
|
||||
const live = await statementLine(42.0, "2026-08-10", "GROCER");
|
||||
const dupe = await statementLine(42.0, "2026-08-10", "GROCER");
|
||||
await pool.query(`UPDATE transactions SET superseded_by_id = $1 WHERE id = $2`, [live, dupe]);
|
||||
|
||||
const [row] = (await getPendingReconciliations(ownerId)).filter((r) => r.id === manual);
|
||||
const offered = row.matches.map((m) => m.id);
|
||||
expect(offered).toContain(live);
|
||||
expect(offered).not.toContain(dupe);
|
||||
});
|
||||
|
||||
it("offers the exact native twin of a foreign row even when its provisional AUD is far off", async () => {
|
||||
// Live shape, 2026-10-09: wallet INR 112 at a provisional A$2.01; CommBank
|
||||
// charged A$1.68 -- 16% under, outside the +-5% band.
|
||||
const manual = await insertTransaction(pool, ownerId, { description: "SWIGGY LIMITED", amount: 2.01, transaction_date: "2026-10-04" });
|
||||
await pool.query(`UPDATE transactions SET foreign_currency_code = 'INR', foreign_currency_amount = 112 WHERE id = $1`, [manual]);
|
||||
const twin = await statementLine(1.68, "2026-10-04", "Swiggy Limited Bengaluru");
|
||||
const otherRupees = await statementLine(1.68, "2026-10-04", "Swiggy Limited Bengaluru");
|
||||
await pool.query(`UPDATE transactions SET foreign_currency_code = 'INR', foreign_currency_amount = 112 WHERE id = $1`, [twin]);
|
||||
await pool.query(`UPDATE transactions SET foreign_currency_code = 'INR', foreign_currency_amount = 113 WHERE id = $1`, [otherRupees]);
|
||||
|
||||
const [row] = (await getPendingReconciliations(ownerId)).filter((r) => r.id === manual);
|
||||
const offered = row.matches.map((m) => m.id);
|
||||
expect(offered).toContain(twin);
|
||||
// A different native figure at the same AUD is not admitted by this path.
|
||||
expect(offered).not.toContain(otherRupees);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,167 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants, insertTransaction } from "./helpers";
|
||||
import { EXCLUDE_RECONCILED_SOURCE } from "@/lib/analytics-sql";
|
||||
|
||||
/**
|
||||
* A charge that never posted (migration 0040).
|
||||
*
|
||||
* Txn 7951: a $1,040.41 Airbnb authorisation hold on Amex, booking cancelled
|
||||
* the next day, hold released. The wallet lane booked the notification as a
|
||||
* purchase and no statement line ever arrived, so it sat in the reconcile queue
|
||||
* forever and counted as spend everywhere. A void has no survivor, which is why
|
||||
* neither reconcile nor supersede could express it.
|
||||
*
|
||||
* As in merge-supersede.test.ts, the refusals matter as much as the happy path:
|
||||
* a void quietly stops money counting.
|
||||
*/
|
||||
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let otherId: number;
|
||||
let voidRoute: typeof import("../../app/api/transactions/[id]/void/route");
|
||||
let merge: typeof import("../../app/api/transactions/[id]/merge/route").POST;
|
||||
|
||||
const req = (body: unknown) => ({ json: async () => body, headers: { get: () => null } }) as never;
|
||||
const params = (id: number) => ({ params: Promise.resolve({ id: String(id) }) });
|
||||
const doVoid = (id: number, reason: unknown = "hold released, booking cancelled") =>
|
||||
voidRoute.POST(req({ reason }), params(id));
|
||||
|
||||
let acctSeq = 0;
|
||||
async function statementLine(amount: number, date: string, description: string) {
|
||||
const st = await pool.query(
|
||||
`INSERT INTO statements (filename, account_number, bank_name, billing_start_date, billing_end_date, owner_id)
|
||||
VALUES ('s.pdf', $3, 'Test Bank', $1::date - 20, $1::date + 10, $2) RETURNING id`,
|
||||
[date, ownerId, `5555-0000-0000-${String(1000 + acctSeq++)}`]
|
||||
);
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions (statement_id, transaction_date, description, amount, transaction_type, owner_id)
|
||||
VALUES ($1, $2, $3, $4, 'debit', $5) RETURNING id`,
|
||||
[st.rows[0].id, date, description, amount, ownerId]
|
||||
);
|
||||
return t.rows[0].id as number;
|
||||
}
|
||||
|
||||
/** What every "count each purchase once" query sees. */
|
||||
async function countedIds(): Promise<number[]> {
|
||||
const r = await pool.query(
|
||||
`SELECT t.id FROM transactions t WHERE ${EXCLUDE_RECONCILED_SOURCE} ORDER BY t.id`
|
||||
);
|
||||
return r.rows.map((x) => x.id);
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
pool = createPool();
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
|
||||
}));
|
||||
voidRoute = await import("../../app/api/transactions/[id]/void/route");
|
||||
({ POST: merge } = await import("../../app/api/transactions/[id]/merge/route"));
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
await pool.query("DELETE FROM expense_metadata");
|
||||
({ ownerId, otherId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
describe("voiding a charge that never posted", () => {
|
||||
it("drops the row out of every figure and the reconcile queue, but keeps it", async () => {
|
||||
const { getPendingReconciliations } = await import("@/lib/queries");
|
||||
const hold = await insertTransaction(pool, ownerId, { description: "AIRBNB", amount: 1040.41 });
|
||||
const real = await insertTransaction(pool, ownerId, { description: "AIRBNB", amount: 529.05 });
|
||||
|
||||
expect((await getPendingReconciliations(ownerId)).map((t) => t.id)).toContain(hold);
|
||||
|
||||
const res = await doVoid(hold);
|
||||
expect(res.status ?? 200).toBe(200);
|
||||
|
||||
expect(await countedIds()).toEqual([real]);
|
||||
expect((await getPendingReconciliations(ownerId)).map((t) => t.id)).not.toContain(hold);
|
||||
const r = await pool.query(`SELECT voided_at, void_reason FROM transactions WHERE id = $1`, [hold]);
|
||||
expect(r.rows[0].voided_at).not.toBeNull();
|
||||
expect(r.rows[0].void_reason).toBe("hold released, booking cancelled");
|
||||
});
|
||||
|
||||
it("can be undone", async () => {
|
||||
const hold = await insertTransaction(pool, ownerId, { amount: 1040.41 });
|
||||
await doVoid(hold);
|
||||
const res = await voidRoute.DELETE(req({}), params(hold));
|
||||
expect(res.status ?? 200).toBe(200);
|
||||
expect(await countedIds()).toEqual([hold]);
|
||||
});
|
||||
|
||||
it("requires a reason", async () => {
|
||||
const hold = await insertTransaction(pool, ownerId, { amount: 10 });
|
||||
expect((await doVoid(hold, " ")).status).toBe(400);
|
||||
expect((await doVoid(hold, null)).status).toBe(400);
|
||||
expect(await countedIds()).toEqual([hold]);
|
||||
});
|
||||
|
||||
it("refuses a statement line -- that charge posted", async () => {
|
||||
const stmt = await statementLine(50, "2026-09-10", "SOMETHING");
|
||||
expect((await doVoid(stmt)).status).toBe(409);
|
||||
expect(await countedIds()).toEqual([stmt]);
|
||||
});
|
||||
|
||||
it("refuses a reconciled row -- it found its statement line, so it posted", async () => {
|
||||
const wallet = await insertTransaction(pool, ownerId, { amount: 50, transaction_date: "2026-09-10" });
|
||||
const stmt = await statementLine(50, "2026-09-10", "SOMETHING");
|
||||
await pool.query(`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`, [stmt, wallet]);
|
||||
expect((await doVoid(wallet)).status).toBe(409);
|
||||
});
|
||||
|
||||
it("refuses a row with a settled split -- someone already paid their share", async () => {
|
||||
const hold = await insertTransaction(pool, ownerId, { amount: 80 });
|
||||
await pool.query(
|
||||
`INSERT INTO transaction_splits (transaction_id, participant_id, share_percent, settled)
|
||||
VALUES ($1, $2, 50, true)`,
|
||||
[hold, otherId]
|
||||
);
|
||||
expect((await doVoid(hold)).status).toBe(409);
|
||||
expect(await countedIds()).toEqual([hold]);
|
||||
});
|
||||
|
||||
it("is enforced by the database, not only the route", async () => {
|
||||
const stmt = await statementLine(50, "2026-09-10", "SOMETHING");
|
||||
await expect(
|
||||
pool.query(`UPDATE transactions SET voided_at = now(), void_reason = 'x' WHERE id = $1`, [stmt])
|
||||
).rejects.toThrow(/transactions_void_statementless/);
|
||||
|
||||
const manual = await insertTransaction(pool, ownerId, { amount: 5 });
|
||||
await expect(
|
||||
pool.query(`UPDATE transactions SET voided_at = now() WHERE id = $1`, [manual])
|
||||
).rejects.toThrow(/transactions_void_has_reason/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the matchers never use a voided row", () => {
|
||||
it("auto-reconcile does not pair a voided row with a statement line of the same amount", async () => {
|
||||
const { findAutoMatches } = await import("@/lib/auto-reconcile");
|
||||
const hold = await insertTransaction(pool, ownerId, { amount: 120, transaction_date: "2026-09-10" });
|
||||
await statementLine(120, "2026-09-11", "SOMETHING");
|
||||
expect((await findAutoMatches(ownerId)).map((m) => m.manual_id)).toContain(hold);
|
||||
|
||||
await doVoid(hold);
|
||||
expect(await findAutoMatches(ownerId)).toEqual([]);
|
||||
});
|
||||
|
||||
it("a receipt for a rebooking at the same amount adopts the live row, not the released hold", async () => {
|
||||
const { findAdoptableTransaction } = await import("@/lib/auto-reconcile");
|
||||
const hold = await insertTransaction(pool, ownerId, { amount: 300, transaction_date: "2026-09-10" });
|
||||
const rebooked = await insertTransaction(pool, ownerId, { amount: 300, transaction_date: "2026-09-11" });
|
||||
// Two candidates: the adopter refuses rather than guess.
|
||||
expect(await findAdoptableTransaction(ownerId, 300, "2026-09-11")).toBeNull();
|
||||
|
||||
await doVoid(hold);
|
||||
expect(await findAdoptableTransaction(ownerId, 300, "2026-09-11")).toBe(rebooked);
|
||||
});
|
||||
|
||||
it("merge refuses a voided row on either side", async () => {
|
||||
const hold = await insertTransaction(pool, ownerId, { amount: 70, transaction_date: "2026-09-10" });
|
||||
const stmt = await statementLine(70, "2026-09-10", "SOMETHING");
|
||||
await doVoid(hold);
|
||||
const res = await merge(req({ other_id: stmt, mode: "reconcile" }), params(hold));
|
||||
expect(res.status).toBe(409);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,176 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants } from "./helpers";
|
||||
|
||||
/**
|
||||
* The mirror of receipt-adopts-wallet-row: a wallet notification arriving AFTER
|
||||
* something else already booked the purchase must fold into that row.
|
||||
*
|
||||
* Built from the failure, not from the design. The 2026-08-18 Woolworths shop
|
||||
* ($103.30, Visa gift card) reached the ledger twice: txn 6858 from the pantry
|
||||
* receipt on the 19th, and txn 6927 from the wallet lane on the 22nd, when the
|
||||
* first 21 wallet events were backfilled days after their taps. The sequencing
|
||||
* the lane is designed around -- wallet first, receipt second -- is a tendency,
|
||||
* and every writer has to survive it not holding.
|
||||
*
|
||||
* Two rules are load-bearing here and are tested separately below:
|
||||
* - the merchant must agree, because losing the "no line items" guard that
|
||||
* protects the other direction means amount+date alone could swallow a tap
|
||||
* into an unrelated purchase of the same value
|
||||
* - a gift-card tap is `credits`, because `card` means "a statement line is
|
||||
* coming" and the reconcile queue filters on exactly that
|
||||
*/
|
||||
|
||||
const TOKEN = "test-wallet-token";
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let ingest: typeof import("../../app/api/wallet/ingest/route").POST;
|
||||
|
||||
const req = (body: unknown) =>
|
||||
({ headers: { get: (h: string) => (h === "x-ingest-token" ? TOKEN : null) }, json: async () => body }) as never;
|
||||
|
||||
/** A row shaped the way the pantry receipt lane writes one: statementless,
|
||||
* itemised, no location (nothing but a notification supplies a suburb). */
|
||||
async function receiptRow(amount: number, date: string, merchant: string, opts: { items?: number; meta?: boolean } = {}) {
|
||||
const { items = 3, meta = true } = opts;
|
||||
const t = await pool.query(
|
||||
`INSERT INTO transactions
|
||||
(owner_id, statement_id, transaction_date, description, amount, amount_aud,
|
||||
transaction_type, category, payment_method, merchant_name, merchant_normalized, row_index)
|
||||
VALUES ($1, NULL, $2, $3, $4, $4, 'debit', 'groceries', 'credits', $3, $3, 0)
|
||||
RETURNING id`,
|
||||
[ownerId, date, merchant, amount]
|
||||
);
|
||||
const id = t.rows[0].id as number;
|
||||
if (meta) {
|
||||
const lineItems = Array.from({ length: items }, (_, i) => ({ description: `item ${i}`, qty: 1, amount: 1 }));
|
||||
await pool.query(
|
||||
`INSERT INTO expense_metadata
|
||||
(transaction_id, source, merchant_normalized, amount, transaction_date, currency, line_items, flags)
|
||||
VALUES ($1, 'pantry', $2, $3, $4, 'AUD', $5::jsonb, '[]'::jsonb)`,
|
||||
[id, merchant, amount, date, JSON.stringify(lineItems)]
|
||||
);
|
||||
}
|
||||
return id;
|
||||
}
|
||||
|
||||
/** The real event 80: the Apple Wallet half of a Visa gift-card tap. */
|
||||
const woolworthsGiftCard = {
|
||||
event_id: 80, kind: "transaction", direction: "unknown", amount: 103.3,
|
||||
amount_text: "$103.30", currency: "$?", merchant: "Woolworths", issuer: "Visa",
|
||||
notif_app: "Wallet", notif_title: "Visa Gift Card",
|
||||
notif_subtitle: "Woolworths, Wyndham Vale, VIC",
|
||||
occurred_at: "2026-08-18T09:00:14+10:00",
|
||||
};
|
||||
|
||||
const rows = async () =>
|
||||
(await pool.query(
|
||||
`SELECT id, amount, merchant_normalized, location, source, source_ref, payment_method
|
||||
FROM transactions ORDER BY id`
|
||||
)).rows;
|
||||
|
||||
beforeAll(async () => {
|
||||
process.env.WALLET_INGEST_TOKEN = TOKEN;
|
||||
pool = createPool();
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
|
||||
}));
|
||||
({ POST: ingest } = await import("../../app/api/wallet/ingest/route"));
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDB(pool);
|
||||
await pool.query("DELETE FROM expense_metadata");
|
||||
({ ownerId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
describe("a wallet notification for a purchase the ledger already holds", () => {
|
||||
it("adopts the receipt's row instead of creating a second one", async () => {
|
||||
const existing = await receiptRow(103.3, "2026-08-18", "WOOLWORTHS 3345");
|
||||
|
||||
const body = await (await ingest(req(woolworthsGiftCard))).json();
|
||||
|
||||
expect(body.skipped).toBe("adopted");
|
||||
expect(body.transactionId).toBe(existing);
|
||||
expect(await rows()).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("gives the adopted row the suburb, and leaves what the receipt read alone", async () => {
|
||||
const existing = await receiptRow(103.3, "2026-08-18", "WOOLWORTHS 3345");
|
||||
await ingest(req(woolworthsGiftCard));
|
||||
|
||||
const [t] = await rows();
|
||||
// Wallet's own field: nothing else on the estate writes a suburb.
|
||||
expect(t.location).toBe("Wyndham Vale, VIC");
|
||||
// The receipt read the register tape. The notification does not get to
|
||||
// rename the merchant to its own shorter string.
|
||||
expect(t.merchant_normalized).toBe("WOOLWORTHS 3345");
|
||||
expect(t.id).toBe(existing);
|
||||
|
||||
const [m] = (await pool.query(`SELECT * FROM expense_metadata ORDER BY id`)).rows;
|
||||
expect(m.source).toBe("pantry");
|
||||
expect(m.line_items).toHaveLength(3);
|
||||
expect(m.payment_method_detail).toBe("Visa");
|
||||
expect(m.flags).toContain("wallet_evt:80");
|
||||
});
|
||||
|
||||
it("is idempotent when the interpreter replays every event", async () => {
|
||||
// Bumping prompt_version re-interprets and re-posts the whole capture
|
||||
// history on purpose -- that is the replay mechanism. An adopted row keeps
|
||||
// no wallet source_ref of its own, so the flag is what makes replay safe.
|
||||
await ingest(req(woolworthsGiftCard));
|
||||
const second = await (await ingest(req(woolworthsGiftCard))).json();
|
||||
expect(second.skipped).toBe("already_ingested");
|
||||
expect(await rows()).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("writes its own row when the merchant disagrees", async () => {
|
||||
// Same money, same day, different shop. Folding into it would make the tap
|
||||
// vanish -- worse than a duplicate, which is at least visible in the queue.
|
||||
await receiptRow(103.3, "2026-08-18", "Bunnings Werribee");
|
||||
const body = await (await ingest(req(woolworthsGiftCard))).json();
|
||||
|
||||
expect(body.skipped).toBeUndefined();
|
||||
expect(await rows()).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("writes its own row when two candidates share the amount", async () => {
|
||||
await receiptRow(103.3, "2026-08-18", "Woolworths Wyndham Vale");
|
||||
await receiptRow(103.3, "2026-08-18", "Woolworths Point Cook");
|
||||
const body = await (await ingest(req(woolworthsGiftCard))).json();
|
||||
|
||||
expect(body.skipped).toBeUndefined();
|
||||
expect(await rows()).toHaveLength(3);
|
||||
});
|
||||
|
||||
it("creates the metadata row when the adopted transaction has none", async () => {
|
||||
const existing = await receiptRow(103.3, "2026-08-18", "WOOLWORTHS 3345", { meta: false });
|
||||
await ingest(req(woolworthsGiftCard));
|
||||
|
||||
const meta = (await pool.query(`SELECT * FROM expense_metadata`)).rows;
|
||||
expect(meta).toHaveLength(1);
|
||||
expect(meta[0].transaction_id).toBe(existing);
|
||||
expect(meta[0].card_last4).toBeNull();
|
||||
expect(meta[0].flags).toContain("wallet_evt:80");
|
||||
});
|
||||
});
|
||||
|
||||
describe("a stored-value instrument", () => {
|
||||
it("is booked as credits, so it never enters the reconcile queue", async () => {
|
||||
const body = await (await ingest(req(woolworthsGiftCard))).json();
|
||||
const [t] = await rows();
|
||||
|
||||
expect(body.transactionId).toBeTruthy();
|
||||
// 'card' would park it forever: findAutoMatches and the queue both exclude
|
||||
// 'credits' and 'cash' precisely because no statement line is coming.
|
||||
expect(t.payment_method).toBe("credits");
|
||||
expect(body.flags).toContain("stored_value_instrument");
|
||||
});
|
||||
|
||||
it("leaves an ordinary card tap as card", async () => {
|
||||
const body = await (await ingest(req({ ...woolworthsGiftCard, notif_title: "Payment", issuer: "Mastercard", notif_app: "Westpac" }))).json();
|
||||
const [t] = await rows();
|
||||
expect(t.payment_method).toBe("card");
|
||||
expect(body.flags).not.toContain("stored_value_instrument");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,403 @@
|
||||
import { describe, it, expect, beforeAll, beforeEach, vi } from "vitest";
|
||||
import { Pool } from "pg";
|
||||
import { createPool, resetDB, seedParticipants } from "./helpers";
|
||||
|
||||
/**
|
||||
* Wallet notifications becoming transactions.
|
||||
*
|
||||
* Every fixture is a real captured event from capture.wallet_events
|
||||
* (2026-08-13 .. 08-22). The rules being tested are the ones the design turned
|
||||
* on, so each has a reason rather than being a shape check:
|
||||
*
|
||||
* - a transaction for EVERY tap, because overseas is where live spend matters
|
||||
* - the pair is MERGED, never deduped: the issuer half has the card tail, the
|
||||
* Wallet half has the merchant and suburb, neither alone is complete
|
||||
* - the NATIVE amount is what gets stored for a foreign charge, because the
|
||||
* AUD figure is the bank's own conversion and cannot be predicted
|
||||
* - a bare "$" is recorded as unknown, never assumed AUD
|
||||
*/
|
||||
|
||||
const TOKEN = "test-wallet-token";
|
||||
let pool: Pool;
|
||||
let ownerId: number;
|
||||
let ingest: typeof import("../../app/api/wallet/ingest/route").POST;
|
||||
|
||||
const req = (body: unknown) =>
|
||||
({ headers: { get: (h: string) => (h === "x-ingest-token" ? TOKEN : null) }, json: async () => body }) as never;
|
||||
|
||||
const noAuth = (body: unknown) =>
|
||||
({ headers: { get: () => null }, json: async () => body }) as never;
|
||||
|
||||
// --- real captured events -------------------------------------------------
|
||||
const breadworldBank = {
|
||||
event_id: 64, kind: "transaction", direction: "debit", amount: 13.5,
|
||||
amount_text: "$13.50", currency: "$?", merchant: "Breadworld Bakery Cafe",
|
||||
card_tail: "8032", issuer: "Mastercard", notif_app: "Westpac",
|
||||
notif_body: "$13.50 paid from Mastercard ending...8032. Breadworld Bakery Cafe WYNDHAM VAL AU",
|
||||
occurred_at: "2026-08-13T08:01:28+10:00",
|
||||
};
|
||||
const breadworldWallet = {
|
||||
event_id: 65, kind: "transaction", direction: "unknown", amount: 13.5,
|
||||
amount_text: "$13.50", currency: "$?", merchant: "Breadworld Bakery Cafe",
|
||||
notif_app: "Wallet", notif_subtitle: "Breadworld Bakery Cafe, Wyndham Vale, VIC",
|
||||
occurred_at: "2026-08-13T08:01:35+10:00",
|
||||
};
|
||||
const amexRewardPay = {
|
||||
event_id: 87, kind: "transaction", direction: "debit", amount: 5103.49,
|
||||
amount_text: "A$5,103.49", currency: "AUD", merchant: "REWARDPAY",
|
||||
card_tail: "01000", issuer: "American Express", notif_app: "Amex AU",
|
||||
notif_body: "You have made a A$5,103.49 charge on your American Express® Card ending in 01000 at REWARDPAY.",
|
||||
occurred_at: "2026-08-20T11:54:43+10:00",
|
||||
};
|
||||
const ebayOrderPlaced = {
|
||||
event_id: 66, kind: "not_transaction", direction: "unknown", amount: null,
|
||||
notif_app: "Wallet", notif_body: "eBay has received your order.",
|
||||
occurred_at: "2026-08-13T09:45:55+10:00",
|
||||
};
|
||||
|
||||
/** Only the rows this lane wrote -- some tests seed statement history for the
|
||||
* rate reference, and those must not be mistaken for wallet output. */
|
||||
const txns = async () =>
|
||||
(await pool.query(
|
||||
`SELECT id, amount, amount_aud, merchant_normalized, location, source, source_ref,
|
||||
foreign_currency_amount, foreign_currency_code, payment_method, statement_id
|
||||
FROM transactions WHERE source = 'wallet' ORDER BY id`
|
||||
)).rows;
|
||||
|
||||
const metas = async () =>
|
||||
(await pool.query(`SELECT * FROM expense_metadata ORDER BY id`)).rows;
|
||||
|
||||
beforeAll(async () => {
|
||||
process.env.WALLET_INGEST_TOKEN = TOKEN;
|
||||
pool = createPool();
|
||||
vi.doMock("@/lib/auth", () => ({
|
||||
getCurrentUser: async () => ({ id: ownerId, name: "Alice", email: "alice@example.com" }),
|
||||
}));
|
||||
({ POST: ingest } = await import("../../app/api/wallet/ingest/route"));
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
// No network in tests. The ECB fallback treats any fetch failure as "no
|
||||
// rate", which is what every pre-existing currency test assumes; the one test
|
||||
// about the fallback stubs a rate of its own.
|
||||
vi.stubGlobal("fetch", async () => { throw new Error("no network in tests"); });
|
||||
await resetDB(pool);
|
||||
await pool.query("DELETE FROM expense_metadata");
|
||||
({ ownerId } = await seedParticipants(pool));
|
||||
});
|
||||
|
||||
describe("auth", () => {
|
||||
it("fails closed without the token", async () => {
|
||||
expect((await ingest(noAuth(breadworldBank))).status).toBe(401);
|
||||
expect(await txns()).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("what reaches the ledger", () => {
|
||||
it("creates a statementless card transaction for a tap", async () => {
|
||||
const body = await (await ingest(req(breadworldBank))).json();
|
||||
expect(body.transactionId).toBeTruthy();
|
||||
|
||||
const [t] = await txns();
|
||||
expect(Number(t.amount)).toBe(13.5);
|
||||
expect(t.source).toBe("wallet");
|
||||
expect(t.source_ref).toBe("evt:64");
|
||||
// Statementless + card is exactly what puts it in the reconcile queue, with
|
||||
// no extra wiring: that is how the statement eventually confirms it.
|
||||
expect(t.statement_id).toBeNull();
|
||||
expect(t.payment_method).toBe("card");
|
||||
});
|
||||
|
||||
it("ignores an eBay order-lifecycle notification", async () => {
|
||||
// 10 of the 13 non-transactions in the capture window are these.
|
||||
const body = await (await ingest(req(ebayOrderPlaced))).json();
|
||||
expect(body.skipped).toBe("not_a_transaction");
|
||||
expect(await txns()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("is idempotent on the event id", async () => {
|
||||
await ingest(req(breadworldBank));
|
||||
const second = await (await ingest(req(breadworldBank))).json();
|
||||
expect(second.skipped).toBe("already_ingested");
|
||||
expect(await txns()).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the notification pair", () => {
|
||||
it("merges the two halves into ONE transaction, keeping both contributions", async () => {
|
||||
await ingest(req(breadworldBank)); // has the card tail
|
||||
await ingest(req(breadworldWallet)); // has the suburb
|
||||
|
||||
const rows = await txns();
|
||||
expect(rows).toHaveLength(1);
|
||||
// The Wallet half supplies the suburb; the issuer half never had one.
|
||||
expect(rows[0].location).toBe("Wyndham Vale, VIC");
|
||||
|
||||
const [m] = await metas();
|
||||
// ...and the issuer half's card tail survives the merge.
|
||||
expect(m.card_last4).toBe("8032");
|
||||
expect(m.payment_method_detail).toBe("Mastercard");
|
||||
});
|
||||
|
||||
it("merges in either arrival order", async () => {
|
||||
await ingest(req(breadworldWallet));
|
||||
await ingest(req(breadworldBank));
|
||||
|
||||
expect(await txns()).toHaveLength(1);
|
||||
const [m] = await metas();
|
||||
expect(m.card_last4).toBe("8032");
|
||||
const [t] = await txns();
|
||||
expect(t.location).toBe("Wyndham Vale, VIC");
|
||||
});
|
||||
});
|
||||
|
||||
describe("the notification pair is recognised by time and app, not amount alone", () => {
|
||||
it("makes ONE row when both halves are posted at the same moment", async () => {
|
||||
// n8n posts a poll's items concurrently, and both halves of a tap usually
|
||||
// land in the same poll. Before the lock, each ran its pair lookup before
|
||||
// the other inserted: 11 taps became 22 rows, identical created_at on each.
|
||||
await Promise.all([ingest(req(breadworldBank)), ingest(req(breadworldWallet))]);
|
||||
expect(await txns()).toHaveLength(1);
|
||||
const [m] = await metas();
|
||||
expect(m.card_last4).toBe("8032");
|
||||
});
|
||||
|
||||
it("keeps two same-amount charges hours apart as two purchases", async () => {
|
||||
// Event 275 ($50 at A MEDIUM) merged into the UNICEF $50 donation from 17
|
||||
// hours earlier and renamed it, because the old window was +/-1 day.
|
||||
await ingest(req({
|
||||
...amexRewardPay, event_id: 267, amount: 50, amount_text: "A$50.00", merchant: "UNICEF",
|
||||
notif_body: "You have made a A$50.00 charge on your American Express® Card ending in 01000 at UNICEF.",
|
||||
occurred_at: "2026-10-01T11:02:40+10:00",
|
||||
}));
|
||||
await ingest(req({
|
||||
...amexRewardPay, event_id: 275, amount: 50, amount_text: "$50.00", merchant: "A MEDIUM",
|
||||
notif_body: "You have made a $50.00 charge on your American Express® Card ending in 01000 at A MEDIUM.",
|
||||
occurred_at: "2026-10-02T04:06:49+10:00",
|
||||
}));
|
||||
const rows = await txns();
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows.map((r) => r.merchant_normalized).sort()).toEqual(["A MEDIUM", "UNICEF"]);
|
||||
});
|
||||
|
||||
it("never pairs two notifications from the SAME app, however close", async () => {
|
||||
// A pair is the issuer half plus the Wallet half. Two from one app are two
|
||||
// charges -- two $20 coffees bought together, say.
|
||||
await ingest(req(breadworldBank));
|
||||
await ingest(req({ ...breadworldBank, event_id: 640, occurred_at: "2026-08-13T08:01:40+10:00" }));
|
||||
expect(await txns()).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("records a merged half, so a replay cannot merge it again", async () => {
|
||||
await ingest(req(breadworldBank));
|
||||
await ingest(req(breadworldWallet));
|
||||
const again = await (await ingest(req(breadworldWallet))).json();
|
||||
expect(again.skipped).toBe("already_ingested");
|
||||
const [m] = await metas();
|
||||
expect(m.flags).toContain("wallet_evt:65");
|
||||
});
|
||||
});
|
||||
|
||||
describe("notifications that name an amount but are not purchases", () => {
|
||||
it("skips an overdraft notice", async () => {
|
||||
// Event 235 became a $21.21 purchase named "To avoid fees and interest,".
|
||||
const body = await (await ingest(req({
|
||||
event_id: 235, kind: "transaction", direction: "debit", amount: 21.21,
|
||||
amount_text: "$21.21", currency: "$?", merchant: null, notif_app: "CommBank",
|
||||
notif_body: "Your Smart Access 1910 is overdrawn by $21.21. To avoid fees and interest, bring your account to positive by midnight 21 Sep (Syd/Melb time).",
|
||||
occurred_at: "2026-09-22T09:00:18+10:00",
|
||||
}))).json();
|
||||
expect(body.skipped).toBe("account_notice");
|
||||
expect(await txns()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("skips a transit-card fare, because the card's load is the spend", async () => {
|
||||
// Event 200: the Octopus load is on the bank statement; each fare draws it
|
||||
// down. 11 of 13 fares were already not_transaction; this makes the other 2 agree.
|
||||
const body = await (await ingest(req({
|
||||
event_id: 200, kind: "transaction", direction: "debit", amount: 105,
|
||||
amount_text: "HKD 105.00", currency: "HKD", merchant: "Octopus", notif_app: "Wallet",
|
||||
notif_title: "Octopus", notif_subtitle: "HKD 105.00 for Train",
|
||||
notif_body: "Your new balance is -HKD 12.60.",
|
||||
occurred_at: "2026-09-14T02:00:37+10:00",
|
||||
}))).json();
|
||||
expect(body.skipped).toBe("stored_value_drawdown");
|
||||
expect(await txns()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("does not read a Westpac descriptor out of another bank's message", async () => {
|
||||
await ingest(req({
|
||||
event_id: 2350, kind: "transaction", direction: "debit", amount: 9.5,
|
||||
amount_text: "$9.50", currency: "$?", merchant: null, notif_app: "CommBank",
|
||||
notif_body: "Payment of $9.50 made. Thanks for banking with us.",
|
||||
occurred_at: "2026-09-22T09:00:18+10:00",
|
||||
}));
|
||||
const [t] = await txns();
|
||||
expect(t.merchant_normalized).toBe("Card purchase");
|
||||
});
|
||||
});
|
||||
|
||||
describe("dating", () => {
|
||||
it("dates a tap in Melbourne local time, as the statements do", async () => {
|
||||
// 21:30 UTC on 30 Sep is 07:30 on 1 Oct in Melbourne. Slicing the ISO
|
||||
// string filed it a day early.
|
||||
await ingest(req({ ...breadworldBank, occurred_at: "2026-09-30T21:30:00Z" }));
|
||||
const { rows } = await pool.query(
|
||||
`SELECT transaction_date::text AS d FROM transactions WHERE source = 'wallet'`
|
||||
);
|
||||
expect(rows[0].d).toBe("2026-10-01");
|
||||
});
|
||||
});
|
||||
|
||||
describe("currency", () => {
|
||||
it("falls back to the ECB rate when the ledger has no history", async () => {
|
||||
// Hong Kong: no HKD statement row existed, so every tap stored A$0.00.
|
||||
const calls: string[] = [];
|
||||
vi.stubGlobal("fetch", async (url: string) => {
|
||||
calls.push(String(url));
|
||||
return { ok: true, json: async () => ({ rates: { AUD: 0.19 } }) };
|
||||
});
|
||||
await ingest(req({
|
||||
event_id: 167, kind: "transaction", direction: "debit", amount: 310.1,
|
||||
amount_text: "HKD 310.10", currency: "HKD", merchant: "Wonder Taxi", notif_app: "Wallet",
|
||||
notif_subtitle: "Wonder Taxi, Tsim Sha Tsui East, Kowloon",
|
||||
occurred_at: "2026-09-10T08:54:00+10:00",
|
||||
}));
|
||||
const [t] = await txns();
|
||||
expect(Number(t.amount)).toBeCloseTo(58.92, 2);
|
||||
expect(Number(t.foreign_currency_amount)).toBe(310.1);
|
||||
expect(calls[0]).toContain("/2026-09-10?from=HKD&to=AUD");
|
||||
const [m] = await metas();
|
||||
// Flagged as the mid-rate, not the bank's: it lacks the card's FX fee.
|
||||
expect(m.flags).toEqual(expect.arrayContaining(["aud_provisional", "rate_ecb"]));
|
||||
expect(m.flags).not.toContain("no_rate_reference");
|
||||
});
|
||||
|
||||
it("records AUD when Amex states it explicitly", async () => {
|
||||
await ingest(req(amexRewardPay));
|
||||
const [t] = await txns();
|
||||
expect(t.foreign_currency_code).toBeNull();
|
||||
expect(Number(t.amount)).toBe(5103.49);
|
||||
const [m] = await metas();
|
||||
expect(m.currency).toBe("AUD");
|
||||
expect(m.flags).not.toContain("currency_unverified");
|
||||
});
|
||||
|
||||
it("flags a bare dollar sign rather than assuming AUD", async () => {
|
||||
// 14 of 21 captured transaction rows are a bare "$". Assuming AUD is
|
||||
// harmless at home and a 23x error on a THB tap abroad.
|
||||
await ingest(req(breadworldBank));
|
||||
const [m] = await metas();
|
||||
expect(m.currency).toBeNull();
|
||||
expect(m.flags).toContain("currency_unverified");
|
||||
});
|
||||
|
||||
it("stores a foreign charge on the NATIVE pair, with a provisional AUD", async () => {
|
||||
// Seed enough history for the median rate to exist (n >= 5).
|
||||
const st = await pool.query(
|
||||
`INSERT INTO statements (filename, account_number, bank_name, billing_start_date, billing_end_date, owner_id)
|
||||
VALUES ('s.pdf','1','Test Bank','2026-01-01','2026-01-31',$1) RETURNING id`, [ownerId]
|
||||
);
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await pool.query(
|
||||
`INSERT INTO transactions (statement_id, transaction_date, description, amount,
|
||||
transaction_type, owner_id, foreign_currency_amount, foreign_currency_code)
|
||||
VALUES ($1,'2026-01-10','BANGKOK',4.57,'debit',$2,100,'THB')`, [st.rows[0].id, ownerId]
|
||||
);
|
||||
}
|
||||
|
||||
await ingest(req({
|
||||
event_id: 999, kind: "transaction", direction: "debit", amount: 500,
|
||||
amount_text: "฿500", merchant: "7-Eleven", notif_app: "Wallet",
|
||||
notif_subtitle: "7-Eleven, Bangkok", occurred_at: "2026-08-20T12:00:00+10:00",
|
||||
}));
|
||||
|
||||
const [t] = await txns();
|
||||
expect(Number(t.foreign_currency_amount)).toBe(500);
|
||||
expect(t.foreign_currency_code).toBe("THB");
|
||||
// The native figure is exact on both sides, so it is what the statement
|
||||
// will match on. amount is only ever provisional here.
|
||||
expect(Number(t.amount)).toBeCloseTo(22.85, 1);
|
||||
const [m] = await metas();
|
||||
expect(m.flags).toContain("aud_provisional");
|
||||
});
|
||||
|
||||
it("still records a foreign charge when there is no rate history", async () => {
|
||||
await ingest(req({
|
||||
event_id: 998, kind: "transaction", direction: "debit", amount: 42,
|
||||
amount_text: "€42.00", merchant: "Somewhere", notif_app: "Wallet",
|
||||
occurred_at: "2026-08-20T12:00:00+10:00",
|
||||
}));
|
||||
const [t] = await txns();
|
||||
expect(t.foreign_currency_code).toBe("EUR");
|
||||
const [m] = await metas();
|
||||
// Says so, rather than inventing a rate.
|
||||
expect(m.flags).toContain("no_rate_reference");
|
||||
// And never stores the native figure as AUD (672006 VND became A$672,006).
|
||||
expect(Number(t.amount)).toBe(0);
|
||||
expect(Number(t.foreign_currency_amount)).toBe(42);
|
||||
});
|
||||
|
||||
it("derives the rate from statement rows only, never from wallet rows", async () => {
|
||||
// Six statementless rows at a nonsense 1:1 rate must not count as history.
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await pool.query(
|
||||
`INSERT INTO transactions (transaction_date, description, amount, transaction_type,
|
||||
owner_id, foreign_currency_amount, foreign_currency_code, source, source_ref)
|
||||
VALUES ('2026-01-10','VN',1000,'debit',$1,1000,'VND','wallet',$2)`,
|
||||
[ownerId, `evt:seed${i}`]
|
||||
);
|
||||
}
|
||||
await ingest(req({
|
||||
event_id: 997, kind: "transaction", direction: "debit", amount: 672006,
|
||||
amount_text: "672006 VND", currency: "VND", merchant: "EVISA VIETNAM", notif_app: "CommBank",
|
||||
occurred_at: "2026-09-05T11:22:00+10:00",
|
||||
}));
|
||||
const t = (await txns()).find((r) => r.source_ref === "evt:997")!;
|
||||
expect(Number(t.amount)).toBe(0);
|
||||
const [m] = (await metas()).filter((r) => r.source_message_id === "evt:997");
|
||||
expect(m.flags).toContain("no_rate_reference");
|
||||
});
|
||||
|
||||
it("pairs foreign halves on the native figure, not the placeholder amount", async () => {
|
||||
await ingest(req({
|
||||
event_id: 996, kind: "transaction", direction: "debit", amount: 42,
|
||||
amount_text: "€42.00", merchant: "Cafe A", notif_app: "Wallet",
|
||||
occurred_at: "2026-08-20T12:00:00+10:00",
|
||||
}));
|
||||
await ingest(req({
|
||||
event_id: 995, kind: "transaction", direction: "debit", amount: 17,
|
||||
amount_text: "€17.00", merchant: "Cafe B", notif_app: "Wallet",
|
||||
occurred_at: "2026-08-20T12:05:00+10:00",
|
||||
}));
|
||||
// Both have amount 0; they are different purchases and must stay two rows.
|
||||
expect((await txns()).filter((r) => r.foreign_currency_code === "EUR")).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("location", () => {
|
||||
it("takes the suburb from the Wallet subtitle", async () => {
|
||||
await ingest(req(breadworldWallet));
|
||||
const [t] = await txns();
|
||||
expect(t.location).toBe("Wyndham Vale, VIC");
|
||||
});
|
||||
|
||||
it("falls back to the fixed-width descriptor, country included", async () => {
|
||||
// The issuer half has no subtitle; the country code is positional.
|
||||
await ingest(req({
|
||||
...breadworldBank, event_id: 75, amount: 42.44,
|
||||
merchant: "SP PEAKDESIGN",
|
||||
// The model's own answer for this row, included because every real
|
||||
// interpretation carries one -- omitting it is what let a wrong
|
||||
// precedence order pass this test the first time, and ship
|
||||
// "SAN FRANCISOUS" into the ledger.
|
||||
location: "SAN FRANCISOUS",
|
||||
notif_body: "$42.44 paid from Mastercard ending...8032. SP PEAKDESIGN SAN FRANCISOUS",
|
||||
}));
|
||||
const [t] = await txns();
|
||||
expect(t.location).toBe("SAN FRANCISO US");
|
||||
const [m] = await metas();
|
||||
// The only foreign merchant in the whole capture window — worth surfacing.
|
||||
expect(m.flags).toContain("foreign_country_US");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,37 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { readFileSync } from "fs";
|
||||
import { resolve } from "path";
|
||||
import { extractPdfText, liteLLMExtractor } from "../../lib/intake-extract";
|
||||
import { reconcileItems, toLineItems, validateHeader } from "../../lib/intake-rules";
|
||||
|
||||
/**
|
||||
* Opt-in: the real model over the real Burgertory export.
|
||||
*
|
||||
* INTAKE_LIVE_TEST=1 LITELLM_BASE_URL=http://localhost:4200 LITELLM_KEY=... npm test
|
||||
*
|
||||
* Skipped otherwise — CI has no LiteLLM, and a unit run must not depend on one.
|
||||
* What it proves is the thing the mock cannot: that `cloud-budget` keeps the
|
||||
* names verbatim ("Michael", "Joel", not "Burger") under the order-item prompt.
|
||||
*/
|
||||
const live = process.env.INTAKE_LIVE_TEST === "1" && !!process.env.LITELLM_BASE_URL && !!process.env.LITELLM_KEY;
|
||||
|
||||
describe.skipIf(!live)("live extraction via LiteLLM (opt-in)", () => {
|
||||
it("Burgertory PDF: header exact, two items named verbatim, reconciles", async () => {
|
||||
const text = await extractPdfText(readFileSync(resolve(__dirname, "../fixtures/intake/ubereats-burgertory-2026-08-19.pdf")));
|
||||
const extracted = await liteLLMExtractor()([{ kind: "text", text, name: "Receipt_19Aug2026.pdf" }]);
|
||||
const header = validateHeader(extracted);
|
||||
expect(header.ok && header.header).toEqual({ merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4 });
|
||||
const items = toLineItems(extracted.items);
|
||||
expect(items.map((i) => i.description)).toEqual(["Michael", "Joel"]);
|
||||
expect(reconcileItems(extracted).ok).toBe(true);
|
||||
}, 120_000);
|
||||
|
||||
it("Coles PDF: header exact, 15 items", async () => {
|
||||
const text = await extractPdfText(readFileSync(resolve(__dirname, "../fixtures/intake/ubereats-coles-2026-08-18.pdf")));
|
||||
const extracted = await liteLLMExtractor()([{ kind: "text", text, name: "Receipt_18Aug2026.pdf" }]);
|
||||
const header = validateHeader(extracted);
|
||||
expect(header.ok && header.header).toEqual({ merchant: "Coles (Wyndham Vale)", order_date: "2026-08-18", total: 85.74 });
|
||||
expect(toLineItems(extracted.items)).toHaveLength(15);
|
||||
expect(reconcileItems(extracted).ok).toBe(true);
|
||||
}, 120_000);
|
||||
});
|
||||
@@ -0,0 +1,419 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { readFileSync } from "fs";
|
||||
import { resolve } from "path";
|
||||
import {
|
||||
classifyInput,
|
||||
confabulationReason,
|
||||
GROCERY_PATTERN,
|
||||
merchantKey,
|
||||
merchantMatches,
|
||||
platformForSource,
|
||||
PROMPT_PLACEHOLDER_MERCHANTS,
|
||||
quoteShared,
|
||||
reconcileItems,
|
||||
resolveMerchantKind,
|
||||
textCouldBeReceipt,
|
||||
toLineItems,
|
||||
validateHeader,
|
||||
type ExtractedOrder,
|
||||
} from "../../lib/intake-rules";
|
||||
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor, ORDER_ITEM_PROMPT } from "../../lib/intake-extract";
|
||||
import { askCard, candidatesCard, enrichedCard, type Block } from "../../lib/intake-blocks";
|
||||
|
||||
/**
|
||||
* The intake router's pure rules, against the strings real receipts print.
|
||||
* The two PDFs are the user's own Uber Eats exports (plan §3): a Coles grocery
|
||||
* run and a Burgertory meal whose burgers are named after people.
|
||||
*/
|
||||
|
||||
const fixtures = resolve(__dirname, "../fixtures/intake");
|
||||
const pdf = (name: string) => readFileSync(resolve(fixtures, name));
|
||||
|
||||
describe("classifyInput — input type picks the extractor", () => {
|
||||
it("PDF by MIME or by name", () => {
|
||||
expect(classifyInput({ type: "application/pdf", name: "x" })).toBe("pdf");
|
||||
expect(classifyInput({ type: "application/octet-stream", name: "Receipt_18Aug2026.pdf" })).toBe("pdf");
|
||||
});
|
||||
it("images and text", () => {
|
||||
expect(classifyInput({ type: "image/png", name: "IMG_1.png" })).toBe("image");
|
||||
expect(classifyInput({ type: "", name: "photo.HEIC" })).toBe("image");
|
||||
expect(classifyInput({ type: "text/plain", name: "" })).toBe("text");
|
||||
});
|
||||
it("declines what it cannot read", () => {
|
||||
expect(classifyInput({ type: "video/mp4", name: "clip.mp4" })).toBe("unsupported");
|
||||
expect(classifyInput({ type: "application/zip", name: "a.zip" })).toBe("unsupported");
|
||||
});
|
||||
});
|
||||
|
||||
describe("platformForSource", () => {
|
||||
it("maps the tap to expense_metadata.platform vocabulary", () => {
|
||||
expect(platformForSource("uber_eats")).toBe("ubereats");
|
||||
expect(platformForSource("doordash")).toBe("doordash");
|
||||
expect(platformForSource("coles")).toBeNull();
|
||||
expect(platformForSource("other")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("merchant pattern — grocery | food | ask", () => {
|
||||
it("the pattern list is a substring match on the normalised name", () => {
|
||||
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
|
||||
expect(resolveMerchantKind("other", "Woolworths Group Limited")).toBe("grocery");
|
||||
expect(resolveMerchantKind("uber_eats", "WOOLIES METRO")).toBe("grocery");
|
||||
expect(resolveMerchantKind("doordash", "ALDI Werribee")).toBe("grocery");
|
||||
expect(resolveMerchantKind("uber_eats", "IGA Xpress")).toBe("grocery");
|
||||
expect(resolveMerchantKind("uber_eats", "FoodWorks Manor Lakes")).toBe("grocery");
|
||||
expect(resolveMerchantKind("uber_eats", "Costco Wholesale")).toBe("grocery");
|
||||
});
|
||||
it("a delivery order that is not a grocer is a meal", () => {
|
||||
expect(resolveMerchantKind("uber_eats", "Burgertory (Werribee)")).toBe("food");
|
||||
expect(resolveMerchantKind("doordash", "Mad Mex")).toBe("food");
|
||||
});
|
||||
it("an empty or unknown merchant asks", () => {
|
||||
expect(resolveMerchantKind("uber_eats", null)).toBe("ask");
|
||||
expect(resolveMerchantKind("uber_eats", " ")).toBe("ask");
|
||||
expect(resolveMerchantKind("doordash", "Unknown")).toBe("ask");
|
||||
});
|
||||
it("the Coles/Woolworths taps are ground truth", () => {
|
||||
expect(resolveMerchantKind("coles", "Romeo's Food Hall")).toBe("grocery");
|
||||
expect(resolveMerchantKind("woolworths", null)).toBe("grocery");
|
||||
});
|
||||
it("`other` with an unrecognised merchant asks; a remembered answer sticks", () => {
|
||||
expect(resolveMerchantKind("other", "Romeo's Food Hall")).toBe("ask");
|
||||
expect(resolveMerchantKind("other", "Romeo's Food Hall", "grocery")).toBe("grocery");
|
||||
expect(resolveMerchantKind("uber_eats", "Romeo's Food Hall", "grocery")).toBe("grocery");
|
||||
});
|
||||
it("merchantKey folds case and punctuation; merchantMatches allows containment only", () => {
|
||||
expect(merchantKey("Coles (Wyndham Vale)")).toBe("coles wyndham vale");
|
||||
expect(merchantMatches("Coles (Wyndham Vale)", "coles wyndham vale")).toBe(true);
|
||||
expect(merchantMatches("Coles (Wyndham Vale)", "Coles")).toBe(true);
|
||||
expect(merchantMatches("Order - Coles (Wyndham Vale)", "Coles (Wyndham Vale)")).toBe(true);
|
||||
expect(merchantMatches("Burgertory (Werribee)", "Coles")).toBe(false);
|
||||
expect(merchantMatches(null, "Coles")).toBe(false);
|
||||
});
|
||||
|
||||
it("whole words, not substrings: `Bigals Pizza` contains `iga` and is neither a grocer nor an IGA", () => {
|
||||
expect(GROCERY_PATTERN.test(merchantKey("Bigals Pizza"))).toBe(false);
|
||||
expect(resolveMerchantKind("uber_eats", "Bigals Pizza")).toBe("food");
|
||||
expect(merchantMatches("Bigals Pizza", "IGA")).toBe(false);
|
||||
// and the grocers a boundary must not cost us
|
||||
expect(resolveMerchantKind("uber_eats", "IGA Wyndham Vale")).toBe("grocery");
|
||||
expect(resolveMerchantKind("uber_eats", "Woolworths Group Limited")).toBe("grocery");
|
||||
expect(resolveMerchantKind("uber_eats", "Coles (Wyndham Vale)")).toBe("grocery");
|
||||
expect(merchantMatches("IGA Wyndham Vale", "IGA")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("validateHeader — nothing is written from a half-read upload", () => {
|
||||
const base: ExtractedOrder = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, items: [] };
|
||||
it("accepts a complete header and rounds the total", () => {
|
||||
const r = validateHeader({ ...base, total: 22.404 });
|
||||
expect(r.ok && r.header.total).toBe(22.4);
|
||||
});
|
||||
it("missing total", () => {
|
||||
const r = validateHeader({ ...base, total: null });
|
||||
expect(r.ok).toBe(false);
|
||||
expect(!r.ok && r.missing).toEqual(["total"]);
|
||||
});
|
||||
it("missing or malformed date", () => {
|
||||
expect(validateHeader({ ...base, order_date: null }).ok).toBe(false);
|
||||
expect(validateHeader({ ...base, order_date: "19/8/26" }).ok).toBe(false);
|
||||
});
|
||||
it("a missing merchant is NOT a missing header — it becomes an ask", () => {
|
||||
const r = validateHeader({ ...base, merchant: null });
|
||||
expect(r.ok && r.header.merchant).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("reconcileItems — the missing-screenshot check", () => {
|
||||
it("reconciles to the printed item subtotal, not the total: fees and promotions are not items", () => {
|
||||
// The real Coles export: 15 lines summing $118.87 against a printed $118.94 subtotal and an $85.74 total.
|
||||
const r = reconcileItems({
|
||||
merchant: "Coles", order_date: "2026-08-18", total: 85.74, subtotal: 118.94,
|
||||
items: [{ quantity: 1, description: "a", line_total: 118.87 }],
|
||||
});
|
||||
expect(r.ok).toBe(true);
|
||||
expect(r.basis).toBe("subtotal");
|
||||
});
|
||||
it("falls back to the total when no subtotal is printed", () => {
|
||||
const r = reconcileItems({ merchant: "x", order_date: "2026-08-19", total: 28, items: [{ quantity: 1, description: "Michael", line_total: 14 }, { quantity: 1, description: "Joel", line_total: 14 }] });
|
||||
expect(r.ok).toBe(true);
|
||||
expect(r.basis).toBe("total");
|
||||
});
|
||||
it("holds when the lines fall short — a shot is missing", () => {
|
||||
const r = reconcileItems({ merchant: "x", order_date: "2026-08-19", total: 28, subtotal: 28, items: [{ quantity: 1, description: "Michael", line_total: 14 }] });
|
||||
expect(r.ok).toBe(false);
|
||||
expect(r.reason).toMatch(/14\.00.*28\.00/);
|
||||
});
|
||||
it("tolerance is $1 or 2%, whichever is larger", () => {
|
||||
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, subtotal: 10, items: [{ quantity: 1, description: "a", line_total: 9.05 }] }).ok).toBe(true);
|
||||
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, subtotal: 10, items: [{ quantity: 1, description: "a", line_total: 8.9 }] }).ok).toBe(false);
|
||||
expect(reconcileItems({ merchant: "x", order_date: "d", total: 200, subtotal: 200, items: [{ quantity: 1, description: "a", line_total: 196.5 }] }).ok).toBe(true);
|
||||
expect(reconcileItems({ merchant: "x", order_date: "d", total: 200, subtotal: 200, items: [{ quantity: 1, description: "a", line_total: 195 }] }).ok).toBe(false);
|
||||
});
|
||||
it("derives a line from quantity × unit price when no line total is printed", () => {
|
||||
const r = reconcileItems({ merchant: "x", order_date: "d", total: 13.2, subtotal: 13.2, items: [{ quantity: 3, description: "Coke", unit_price: 4.4 }] });
|
||||
expect(r.ok).toBe(true);
|
||||
expect(r.items_sum).toBe(13.2);
|
||||
});
|
||||
it("no items at all is held; priceless lines (pasted text) are not", () => {
|
||||
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, items: [] }).ok).toBe(false);
|
||||
expect(reconcileItems({ merchant: "x", order_date: "d", total: 10, items: [{ quantity: 1, description: "a" }] }).ok).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("toLineItems — names verbatim, options kept", () => {
|
||||
it("keeps a burger called Michael called Michael", () => {
|
||||
const out = toLineItems([{ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] }]);
|
||||
expect(out).toEqual([{ qty: 1, description: "Michael", amount: 14, unit_price: 14, options: ["Milk Bun"] }]);
|
||||
});
|
||||
it("drops nameless lines and defaults quantity to 1", () => {
|
||||
const out = toLineItems([{ quantity: null, description: " ", line_total: 1 }, { quantity: null, description: "Cucumbers Continental", unit_price: 2.9 }]);
|
||||
expect(out).toEqual([{ qty: 1, description: "Cucumbers Continental", amount: 2.9, unit_price: 2.9 }]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("extraction request and reply", () => {
|
||||
it("one call: the prompt first, then every part in the order shared", () => {
|
||||
const content = buildMessageContent([
|
||||
{ kind: "image", mime: "image/png", bytes: Buffer.from("a"), name: "1.png" },
|
||||
{ kind: "text", text: "Total $22.40", name: "r.pdf" },
|
||||
{ kind: "image", mime: "image/jpeg", bytes: Buffer.from("b"), name: "2.jpg" },
|
||||
]) as { type: string; text?: string; image_url?: { url: string } }[];
|
||||
expect(content[0].text).toMatch(/VERBATIM/);
|
||||
const images = content.filter((c) => c.type === "image_url").map((c) => c.image_url!.url);
|
||||
expect(images).toEqual(["data:image/png;base64,YQ==", "data:image/jpeg;base64,Yg=="]);
|
||||
expect(content.findIndex((c) => c.text?.includes("Total $22.40"))).toBeLessThan(content.findIndex((c) => c.image_url?.url.endsWith("Yg==")));
|
||||
expect(EXTRACT_RESPONSE_FORMAT.json_schema.strict).toBe(true);
|
||||
});
|
||||
it("parses a fenced reply and coerces numbers", () => {
|
||||
const r = parseExtraction('```json\n{"merchant":"Burgertory (Werribee)","order_date":"2026-08-19","total":"22.40","subtotal":28,"currency":null,"items":[{"quantity":1,"description":"Michael","unit_price":14,"line_total":14,"options":["Milk Bun"]}],"adjustments":[{"label":"Promotion","amount":-8}]}\n```');
|
||||
expect(r.total).toBe(22.4);
|
||||
expect(r.currency).toBe("AUD");
|
||||
expect(r.items[0]).toEqual({ quantity: 1, description: "Michael", unit_price: 14, line_total: 14, options: ["Milk Bun"] });
|
||||
expect(r.adjustments).toEqual([{ label: "Promotion", amount: -8 }]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("PDF text — the Uber Eats export is born-digital text", () => {
|
||||
it("Coles (Wyndham Vale) · 2026-08-18 · $85.74, 15 lines, weighed and substituted", async () => {
|
||||
const text = await extractPdfText(pdf("ubereats-coles-2026-08-18.pdf"));
|
||||
expect(text).toContain("Here's your receipt for Coles (Wyndham Vale).");
|
||||
expect(text).toContain("18 Aug, 2026");
|
||||
expect(text).toContain("Total $85.74");
|
||||
expect(text).toContain("Item Subtotal $118.94");
|
||||
expect(text).toContain("Final 1.16 kg $11.00/kg");
|
||||
expect(text).toContain("Replaced Coles Beef Slow Cook Chuck Roast");
|
||||
// 15 item lines: 13 priced per piece, 2 weighed per kg.
|
||||
expect(text.match(/\$\d+\.\d\d\/pc$/gm)?.length).toBe(13);
|
||||
expect(text.match(/\$\d+\.\d\d\/kg$/gm)?.length).toBe(2);
|
||||
});
|
||||
it("Burgertory (Werribee) · 2026-08-19 · $22.40, burgers named Michael and Joel", async () => {
|
||||
const text = await extractPdfText(pdf("ubereats-burgertory-2026-08-19.pdf"));
|
||||
expect(text).toContain("Here's your receipt for Burgertory (Werribee).");
|
||||
expect(text).toContain("Total $22.40");
|
||||
expect(text).toContain("1 Michael $14.00");
|
||||
expect(text).toContain("1 Joel $14.00");
|
||||
expect(text).toContain("Item Subtotal $28.00");
|
||||
});
|
||||
});
|
||||
|
||||
describe("cards carry the C5 action ids and JSON values", () => {
|
||||
const c = { batchId: "11111111-2222-4333-8444-555555555555", merchant: "Burgertory (Werribee)", total: 22.4, orderDate: "2026-08-19", platform: "ubereats", itemCount: 2 };
|
||||
it("candidates: one Use-this per candidate plus an explicit Create", () => {
|
||||
const { blocks } = candidatesCard(c, [{ transaction_id: 42, description: "Order - Burgertory (Werribee)", transaction_date: "2026-08-17", amount: "19.90" }]);
|
||||
const buttons = JSON.stringify(blocks);
|
||||
expect(buttons).toContain('"action_id":"intake_use_candidate"');
|
||||
expect(buttons).toContain('"action_id":"intake_create"');
|
||||
const use = (blocks[1] as { accessory: { value: string } }).accessory.value;
|
||||
expect(JSON.parse(use)).toEqual({ batch_id: c.batchId, transaction_id: 42 });
|
||||
});
|
||||
it("enriched: the confirmation text and a View link", () => {
|
||||
const { text, blocks } = enrichedCard({ ...c, transactionId: 7 });
|
||||
expect(text).toBe("Added 2 items to your $22.40 Uber Eats (Burgertory (Werribee)) order ✓");
|
||||
expect(JSON.stringify(blocks)).toContain("/transactions?q=Burgertory");
|
||||
});
|
||||
it("ask: Groceries / A meal", () => {
|
||||
const ids = JSON.stringify(askCard({ ...c, platform: null }).blocks);
|
||||
expect(ids).toContain('"action_id":"intake_grocery"');
|
||||
expect(ids).toContain('"action_id":"intake_meal"');
|
||||
});
|
||||
|
||||
it("every value the card did not write is escaped, plain_text included", () => {
|
||||
const hostile = { ...c, merchant: "Ben & Jerry's <fake>" };
|
||||
const { text, blocks } = candidatesCard(hostile, [
|
||||
{ transaction_id: 42, description: "SQ *BEN & JERRY <AU>", transaction_date: "2026-08-17", amount: "19.90" },
|
||||
]);
|
||||
const sections = JSON.stringify(blocks.filter((b) => (b as Block).type === "section"));
|
||||
expect(sections).toContain("Ben & Jerry's <fake>");
|
||||
expect(sections).toContain("SQ *BEN & JERRY <AU>");
|
||||
expect(text).toContain("Ben & Jerry's <fake>");
|
||||
// plain_text disables formatting, not entity decoding — &, < and > are
|
||||
// control characters everywhere in a Slack payload.
|
||||
const whole = JSON.stringify(blocks);
|
||||
expect(whole).not.toContain("<fake>");
|
||||
expect(whole).not.toContain("<AU>");
|
||||
});
|
||||
|
||||
it("the currency is model output too, so it is escaped like any other", () => {
|
||||
// parseExtraction accepts any string for `currency`; unescaped, `<@U123>`
|
||||
// would be a live mention on every card the batch produces.
|
||||
const { text } = enrichedCard({ ...c, currency: "<@U123>", transactionId: 7 });
|
||||
expect(text).toContain("<@U123> 22.40");
|
||||
expect(text).not.toContain("<@U123>");
|
||||
});
|
||||
|
||||
it("a foreign receipt is not printed as dollars — header and candidate rows alike", () => {
|
||||
expect(enrichedCard({ ...c, currency: "USD", transactionId: 7 }).text).toBe(
|
||||
"Added 2 items to your USD 22.40 Uber Eats (Burgertory (Werribee)) order ✓"
|
||||
);
|
||||
expect(enrichedCard({ ...c, currency: "AUD", transactionId: 7 }).text).toContain("$22.40");
|
||||
expect(enrichedCard({ ...c, transactionId: 7 }).text).toContain("$22.40");
|
||||
// A candidate row is a ledger transaction and carries its own currency; it
|
||||
// used to print "$" beneath a header reading "USD 22.40".
|
||||
const { blocks } = candidatesCard({ ...c, currency: "USD" }, [
|
||||
{ transaction_id: 42, description: "Order - Burgertory", transaction_date: "2026-08-17", amount: "19.90", currency: "USD" },
|
||||
]);
|
||||
expect(JSON.stringify(blocks)).toContain("USD 19.90");
|
||||
});
|
||||
});
|
||||
|
||||
describe("liteLLMExtractor — a finish_reason that is present must be `stop`", () => {
|
||||
const good = { merchant: "Burgertory (Werribee)", order_date: "2026-08-19", total: 22.4, subtotal: 28, currency: "AUD", items: [], adjustments: [] };
|
||||
const extractor = (finish_reason?: string) =>
|
||||
liteLLMExtractor({
|
||||
baseUrl: "http://litellm.test",
|
||||
key: "k",
|
||||
fetchImpl: (async () =>
|
||||
Response.json({
|
||||
choices: [{ message: { content: JSON.stringify(good) }, ...(finish_reason ? { finish_reason } : {}) }],
|
||||
})) as unknown as typeof fetch,
|
||||
});
|
||||
|
||||
it("accepts `stop`, and — on purpose — a reply that reports no reason at all", async () => {
|
||||
expect((await extractor("stop")([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
|
||||
expect((await extractor()([{ kind: "text", text: "x", name: "x" }])).total).toBe(22.4);
|
||||
});
|
||||
|
||||
it("rejects any other reason, naming it — the strict schema only held for a reply that finished", async () => {
|
||||
// An absent reason is NOT rejected: see the comment in intake-extract.ts.
|
||||
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(ExtractionError);
|
||||
await expect(extractor("length")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/too long/);
|
||||
await expect(extractor("content_filter")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/content_filter/);
|
||||
await expect(extractor("tool_calls")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/tool_calls/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("textCouldBeReceipt — nothing that cannot be a receipt reaches the model", () => {
|
||||
const why = (t: string | null | undefined) => {
|
||||
const r = textCouldBeReceipt(t);
|
||||
return r.ok ? null : r.reason;
|
||||
};
|
||||
|
||||
it("the live defect: the phone sent the WORD `Image`, not the image", () => {
|
||||
expect(textCouldBeReceipt("Image").ok).toBe(false);
|
||||
expect(why("Image")).toMatch(/single word/);
|
||||
expect(why("Image\n")).toMatch(/single word/);
|
||||
});
|
||||
|
||||
it("a filename is not a receipt, digits in it notwithstanding", () => {
|
||||
expect(why("IMG_0042.jpeg")).toMatch(/single word/);
|
||||
expect(why("Receipt_19Aug2026.pdf")).toMatch(/single word/);
|
||||
});
|
||||
|
||||
it("empty, blank and whitespace-only", () => {
|
||||
expect(why("")).toMatch(/empty/);
|
||||
expect(why(" \n\t ")).toMatch(/empty/);
|
||||
expect(why(null)).toMatch(/empty/);
|
||||
expect(why(undefined)).toMatch(/empty/);
|
||||
});
|
||||
|
||||
it("prose with no numbers cannot be a receipt", () => {
|
||||
expect(why("Shared from Photos")).toMatch(/no numbers/);
|
||||
expect(why("here is the receipt for dinner last night, sorry it took so long")).toMatch(/no numbers/);
|
||||
});
|
||||
|
||||
it("short but with a printed amount goes through — this door accepts a pasted line", () => {
|
||||
expect(textCouldBeReceipt("TOTAL $17.00").ok).toBe(true);
|
||||
expect(textCouldBeReceipt("Total 17.00").ok).toBe(true);
|
||||
expect(textCouldBeReceipt("Gesamt 12,50 EUR").ok).toBe(true);
|
||||
});
|
||||
|
||||
it("short with digits but no money figure does not — a screenshot name is not a purchase", () => {
|
||||
expect(why("Screenshot 2026 08 22")).toMatch(/too short/);
|
||||
expect(why("2 items")).toMatch(/too short/);
|
||||
});
|
||||
|
||||
it("a real receipt body passes", () => {
|
||||
expect(
|
||||
textCouldBeReceipt("Burgertory (Werribee)\n19 Aug 2026\n1x Michael 14.00\n1x Joel 14.00\nSubtotal 28.00\nTotal 22.40").ok
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("quoteShared flattens and truncates what goes on the card", () => {
|
||||
expect(quoteShared("Image")).toBe("Image");
|
||||
expect(quoteShared(" a \n b ")).toBe("a b");
|
||||
expect(quoteShared("x".repeat(200))).toHaveLength(80);
|
||||
expect(quoteShared("x".repeat(200)).endsWith("…")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("confabulationReason — the belt to that pre-check", () => {
|
||||
const nothing = { textParts: ["Image"], hasImages: false };
|
||||
const readable = { textParts: ["Total $22.40 Burgertory (Werribee) 2026-08-19"], hasImages: false };
|
||||
const order = { merchant: "Burgertory (Braybrook)", items: [{ description: "The Pride" }] };
|
||||
|
||||
it("flags an order extracted from nothing readable", () => {
|
||||
expect(confabulationReason(nothing, order)).toMatch(/nothing shared could be read/);
|
||||
});
|
||||
|
||||
it("does not flag one extracted from something readable", () => {
|
||||
expect(confabulationReason(readable, order)).toBeNull();
|
||||
});
|
||||
|
||||
it("does not flag an image share — the pixels are what we asked for", () => {
|
||||
expect(confabulationReason({ textParts: [], hasImages: true }, order)).toBeNull();
|
||||
});
|
||||
|
||||
it("an empty answer to an unreadable share is honest, not a confabulation", () => {
|
||||
expect(confabulationReason(nothing, { merchant: null, items: [] })).toBeNull();
|
||||
});
|
||||
|
||||
it("the prompt's own placeholder coming back as a merchant is always a confabulation", () => {
|
||||
for (const p of PROMPT_PLACEHOLDER_MERCHANTS) {
|
||||
expect(confabulationReason(readable, { merchant: p, items: [] })).toMatch(/prompt's own example/);
|
||||
expect(confabulationReason({ textParts: [], hasImages: true }, { merchant: p, items: [] })).toMatch(/prompt's own example/);
|
||||
}
|
||||
});
|
||||
|
||||
it("a real merchant that happens to be named in the prompt's platform list is NOT flagged", () => {
|
||||
expect(confabulationReason(readable, { merchant: "Coles (Wyndham Vale)", items: [] })).toBeNull();
|
||||
expect(confabulationReason(readable, { merchant: "Woolworths", items: [] })).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("the prompt carries no real merchant to echo", () => {
|
||||
it("names no shop it could hand back as data", () => {
|
||||
expect(ORDER_ITEM_PROMPT).not.toMatch(/Burgertory/);
|
||||
expect(ORDER_ITEM_PROMPT).not.toMatch(/Wyndham Vale/);
|
||||
expect(ORDER_ITEM_PROMPT).toContain(PROMPT_PLACEHOLDER_MERCHANTS[0]);
|
||||
});
|
||||
|
||||
it("tells the model what to return when the input is not a receipt", () => {
|
||||
expect(ORDER_ITEM_PROMPT).toMatch(/NOT a receipt/);
|
||||
expect(ORDER_ITEM_PROMPT).toMatch(/Do not construct a plausible order/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("needsHeaderCard", () => {
|
||||
it("names the missing field and sends a dateless in-store screenshot to the app's Share button", async () => {
|
||||
const { needsHeaderCard } = await import("@/lib/intake-blocks");
|
||||
const dateless = needsHeaderCard(["date"], "woolworths").text;
|
||||
expect(dateless).toContain("no date");
|
||||
expect(dateless).toContain("Share button");
|
||||
expect(dateless).not.toContain("total");
|
||||
const delivery = needsHeaderCard(["total"], "uber_eats").text;
|
||||
expect(delivery).toContain("no total");
|
||||
expect(delivery).toContain("part of the screen");
|
||||
expect(needsHeaderCard().text).toContain("total or date");
|
||||
});
|
||||
});
|
||||
@@ -508,7 +508,9 @@ describe("HTML entities in line items", () => {
|
||||
// The ampersand decodes, and the bullet becomes a boundary, not text.
|
||||
expect(item!.description).toContain("Sweet & Sour Crunch");
|
||||
expect(item!.description).not.toContain("•");
|
||||
expect(item!.options.length).toBeGreaterThan(0);
|
||||
// `options` is optional on the parsed item; `?? 0` keeps the assertion honest
|
||||
// (an absent list fails) instead of silencing the check with a non-null bang.
|
||||
expect(item!.options?.length ?? 0).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("leaves a literal &bull; alone rather than turning it into a bullet", () => {
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
parseWestpacDescriptor,
|
||||
parseWalletSubtitle,
|
||||
currencyFromAmountText,
|
||||
} from "@/lib/wallet-descriptor";
|
||||
|
||||
/**
|
||||
* Every input below is a real captured notification body or subtitle from
|
||||
* capture.wallet_events (2026-08-13 .. 08-22), not an invented fixture. The
|
||||
* model handled this one fixed-width format three different ways on this exact
|
||||
* data, which is why the parsing moved into code.
|
||||
*/
|
||||
|
||||
describe("parseWestpacDescriptor", () => {
|
||||
it("reads merchant, city and country from the fixed-width block", () => {
|
||||
expect(parseWestpacDescriptor(
|
||||
"$13.50 paid from Mastercard ending...8032. Breadworld Bakery Cafe WYNDHAM VAL AU"
|
||||
)).toEqual({ merchant: "Breadworld Bakery Cafe", city: "WYNDHAM VAL", country: "AU" });
|
||||
});
|
||||
|
||||
it("handles a city padded to the full field width", () => {
|
||||
expect(parseWestpacDescriptor(
|
||||
"$54.50 paid from Mastercard ending...8032. PLA DONATION MELBOURNE AU"
|
||||
)).toEqual({ merchant: "PLA DONATION", city: "MELBOURNE", country: "AU" });
|
||||
});
|
||||
|
||||
it("splits a 12-char city that runs into the country code with no space", () => {
|
||||
// The case that defeated the model: "SAN FRANCISO" exactly fills the city
|
||||
// field, so "US" follows immediately. Read as one token it becomes the
|
||||
// nonsense location "SAN FRANCISOUS" and the country is lost -- on the only
|
||||
// foreign merchant in the whole capture window.
|
||||
expect(parseWestpacDescriptor(
|
||||
"$42.44 paid from Mastercard ending...8032. SP PEAKDESIGN SAN FRANCISOUS"
|
||||
)).toEqual({ merchant: "SP PEAKDESIGN", city: "SAN FRANCISO", country: "US" });
|
||||
});
|
||||
|
||||
it("reads mixed-case cities the model inconsistently stripped", () => {
|
||||
expect(parseWestpacDescriptor(
|
||||
"$20.99 paid from Mastercard ending...8032. Spotify P45AE035CB Sydney AU"
|
||||
)).toEqual({ merchant: "Spotify P45AE035CB", city: "Sydney", country: "AU" });
|
||||
});
|
||||
|
||||
it("degrades to nulls rather than guessing", () => {
|
||||
expect(parseWestpacDescriptor(null)).toEqual({ merchant: null, city: null, country: null });
|
||||
expect(parseWestpacDescriptor("")).toEqual({ merchant: null, city: null, country: null });
|
||||
// No sentence terminator: not this format at all.
|
||||
expect(parseWestpacDescriptor("You have a new notification"))
|
||||
.toEqual({ merchant: null, city: null, country: null });
|
||||
});
|
||||
|
||||
it("does not invent a country from a short merchant-only tail", () => {
|
||||
// "...IT" here is the end of a word, not Italy. A wrong country on a
|
||||
// foreign transaction is worse than no country.
|
||||
const r = parseWestpacDescriptor("$9.00 paid from Mastercard ending...8032. SOME MERCHANT");
|
||||
expect(r.country).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseWalletSubtitle", () => {
|
||||
it("separates merchant from suburb and state", () => {
|
||||
expect(parseWalletSubtitle("Breadworld Bakery Cafe, Wyndham Vale, VIC"))
|
||||
.toEqual({ merchant: "Breadworld Bakery Cafe", location: "Wyndham Vale, VIC" });
|
||||
});
|
||||
|
||||
it("returns no location for an online merchant", () => {
|
||||
// 2 of the 10 Wallet subtitles captured have no place at all.
|
||||
expect(parseWalletSubtitle("Seatsaero")).toEqual({ merchant: "Seatsaero", location: null });
|
||||
expect(parseWalletSubtitle("Prism Accounting Bookkeeping Services"))
|
||||
.toEqual({ merchant: "Prism Accounting Bookkeeping Services", location: null });
|
||||
});
|
||||
|
||||
it("keeps a merchant name containing its own comma out of the location", () => {
|
||||
expect(parseWalletSubtitle("Big Daddy`s Fresh Ma, Manor Lakes, VIC").location)
|
||||
.toBe("Manor Lakes, VIC");
|
||||
});
|
||||
|
||||
it("handles absent input", () => {
|
||||
expect(parseWalletSubtitle(null)).toEqual({ merchant: null, location: null });
|
||||
expect(parseWalletSubtitle(" ")).toEqual({ merchant: null, location: null });
|
||||
});
|
||||
});
|
||||
|
||||
describe("currencyFromAmountText", () => {
|
||||
it("reads Amex's explicit marker", () => {
|
||||
expect(currencyFromAmountText("A$5,103.49")).toBe("AUD");
|
||||
expect(currencyFromAmountText("A$13.50")).toBe("AUD");
|
||||
});
|
||||
|
||||
it("refuses to guess a bare dollar sign", () => {
|
||||
// 14 of 21 captured transaction rows are this. Assuming AUD is harmless at
|
||||
// home and a 23x error on a THB tap abroad, so the row goes to the
|
||||
// reconcile queue with no currency instead.
|
||||
expect(currencyFromAmountText("$13.50")).toBeNull();
|
||||
expect(currencyFromAmountText("$103.30")).toBeNull();
|
||||
});
|
||||
|
||||
it("reads unambiguous symbols when a card does state one", () => {
|
||||
expect(currencyFromAmountText("€24.00")).toBe("EUR");
|
||||
expect(currencyFromAmountText("฿500")).toBe("THB");
|
||||
expect(currencyFromAmountText("S$18.20")).toBe("SGD");
|
||||
expect(currencyFromAmountText("NZ$40.00")).toBe("NZD");
|
||||
});
|
||||
|
||||
it("handles absent input", () => {
|
||||
expect(currencyFromAmountText(null)).toBeNull();
|
||||
expect(currencyFromAmountText("")).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,44 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { getCurrentUser } from "@/lib/auth";
|
||||
import { getUpcomingAnnualCharges } from "@/lib/annual-charges";
|
||||
|
||||
/**
|
||||
* Upcoming annual charges, from payments alone.
|
||||
*
|
||||
* Lives here rather than in home-dashboard so the detection sits beside the
|
||||
* ledger it queries, is covered by this repo's integration suite, and can be
|
||||
* reused by a future Slack nudge without moving. home-dashboard reads it over
|
||||
* networks_internal the way its other widgets read their sources.
|
||||
*
|
||||
* Machine access uses the same shared secret as the dashboard's other reads;
|
||||
* a browser session scopes to that user instead.
|
||||
*/
|
||||
export async function GET(req: NextRequest) {
|
||||
// Its OWN token, with no fallback to WALLET_INGEST_TOKEN. home-dashboard's
|
||||
// compose states the rule plainly -- "this app never holds write creds" -- and
|
||||
// WALLET_INGEST_TOKEN unlocks POST /api/wallet/ingest and the estate-wide
|
||||
// reconcile sweep. Handing that to a read-only surface to save minting a
|
||||
// secret would quietly convert a dashboard into a writer. Unset means closed.
|
||||
const token = process.env.DASH_READ_TOKEN;
|
||||
const supplied = req.headers.get("x-ingest-token");
|
||||
const machine = !!token && !!supplied && supplied === token;
|
||||
|
||||
let ownerId: number | null = null;
|
||||
if (machine) {
|
||||
// A machine caller names the owner explicitly rather than defaulting to 1:
|
||||
// silently assuming an owner is how an ingested order ended up invisible in
|
||||
// every scoped view while sitting in the table.
|
||||
const q = req.nextUrl.searchParams.get("owner_id");
|
||||
ownerId = q !== null && Number.isInteger(Number(q)) ? Number(q) : null;
|
||||
if (ownerId === null) {
|
||||
return NextResponse.json({ error: "owner_id is required for machine access" }, { status: 400 });
|
||||
}
|
||||
} else {
|
||||
const user = await getCurrentUser(req);
|
||||
if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 403 });
|
||||
ownerId = user.id;
|
||||
}
|
||||
|
||||
const charges = await getUpcomingAnnualCharges(ownerId);
|
||||
return NextResponse.json({ charges, count: charges.length });
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { sha256Of, type StoredFile } from "@/lib/intake-files";
|
||||
import { liteLLMExtractor } from "@/lib/intake-extract";
|
||||
import { pantryForwarder } from "@/lib/intake-pantry";
|
||||
import { isIntakeSource } from "@/lib/intake-rules";
|
||||
import { MAX_FILES, runIntake } from "@/lib/intake-router";
|
||||
|
||||
/**
|
||||
* The one upload point (plan §11 C1). n8n forwards the iOS Shortcut's
|
||||
* multipart POST here: `source`, `shared_by`, `batch_id`, and 1–6 files under
|
||||
* `file`. The router reads what it is given and does every step that applies.
|
||||
*
|
||||
* Auth is a shared secret, like the two ingest siblings: this is machine-to-
|
||||
* machine and there is no browser session to forward. Its own token so the
|
||||
* Shortcut's credential rotates without touching the order or receipt lanes.
|
||||
* Fails closed when unset.
|
||||
*/
|
||||
function authorised(req: NextRequest): boolean {
|
||||
const expected = process.env.INTAKE_TOKEN;
|
||||
if (!expected) return false;
|
||||
const got = req.headers.get("x-ingest-token");
|
||||
return !!got && got === expected;
|
||||
}
|
||||
|
||||
const MAX_BYTES = 12_000_000;
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
if (!authorised(req)) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
|
||||
|
||||
let form: FormData;
|
||||
try {
|
||||
form = await req.formData();
|
||||
} catch {
|
||||
return NextResponse.json({ error: "multipart/form-data body required" }, { status: 400 });
|
||||
}
|
||||
|
||||
const source = form.get("source");
|
||||
if (!isIntakeSource(source)) {
|
||||
return NextResponse.json({ error: "source must be one of uber_eats|doordash|coles|woolworths|other" }, { status: 400 });
|
||||
}
|
||||
const sharedBy = typeof form.get("shared_by") === "string" ? String(form.get("shared_by")).trim() || null : null;
|
||||
const batchIdRaw = form.get("batch_id");
|
||||
const batchId = typeof batchIdRaw === "string" && /^[0-9a-f-]{36}$/i.test(batchIdRaw) ? batchIdRaw.toLowerCase() : null;
|
||||
if (batchIdRaw && !batchId) return NextResponse.json({ error: "batch_id must be a uuid" }, { status: 400 });
|
||||
|
||||
const files: StoredFile[] = [];
|
||||
for (const entry of form.getAll("file")) {
|
||||
if (!(entry instanceof File) || !entry.size) continue;
|
||||
if (entry.size > MAX_BYTES) return NextResponse.json({ error: `${entry.name} is larger than 12 MB` }, { status: 400 });
|
||||
const bytes = Buffer.from(await entry.arrayBuffer());
|
||||
files.push({ name: entry.name || `file-${files.length + 1}`, type: entry.type || "", bytes, sha256: sha256Of(bytes) });
|
||||
}
|
||||
if (!files.length) return NextResponse.json({ error: "at least one file is required under field 'file'" }, { status: 400 });
|
||||
if (files.length > MAX_FILES) return NextResponse.json({ error: `at most ${MAX_FILES} files per batch` }, { status: 400 });
|
||||
|
||||
try {
|
||||
const result = await runIntake(
|
||||
{ batchId, source, sharedBy, files },
|
||||
{ extractor: liteLLMExtractor(), pantry: pantryForwarder() }
|
||||
);
|
||||
const { http_status, ...body } = result;
|
||||
return NextResponse.json(body, { status: http_status });
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,8 @@ import {
|
||||
slackUserForParticipant,
|
||||
} from "@/lib/slack-verify";
|
||||
import { nudgeBlocks, detailsModal, partnerNudgeBlocks } from "@/lib/slack-blocks";
|
||||
import { handleIntakeAction, isIntakeAction, parseActionValue } from "@/lib/intake-actions";
|
||||
import { pantryForwarder } from "@/lib/intake-pantry";
|
||||
import { completeSplit } from "@/lib/splits";
|
||||
import {
|
||||
RATINGS,
|
||||
@@ -73,6 +75,22 @@ export async function POST(req: NextRequest) {
|
||||
if (payload.type !== "block_actions") return new NextResponse(null, { status: 200 });
|
||||
|
||||
const action = payload.actions?.[0];
|
||||
|
||||
// Intake taps (plan §11 C5) act on a persisted batch, not a transaction, so
|
||||
// their `value` is JSON `{batch_id, ...}` rather than `<id>:<verb>`. They are
|
||||
// dispatched on action_id before the order-nudge parse below, which would
|
||||
// otherwise read the JSON as a non-numeric transaction id and give up.
|
||||
if (isIntakeAction(action?.action_id)) {
|
||||
const reply = await handleIntakeAction(action.action_id, parseActionValue(action?.value), { pantry: pantryForwarder() });
|
||||
await updateMessage(
|
||||
payload.response_url,
|
||||
reply.replace
|
||||
? { replace_original: true, blocks: reply.blocks, text: reply.text }
|
||||
: { response_type: "ephemeral", replace_original: false, text: reply.text }
|
||||
);
|
||||
return NextResponse.json({ blocks: reply.blocks, text: reply.text });
|
||||
}
|
||||
|
||||
// A button carries `value`; a select carries it on the chosen option. The
|
||||
// ratings moved to a select because four buttons became four full-width rows
|
||||
// on Slack mobile, so both shapes have to resolve or rating silently stops
|
||||
@@ -216,6 +234,16 @@ async function buildDetailsModal(transactionId: number, participantId: number) {
|
||||
`SELECT merchant_normalized AS merchant, line_items
|
||||
FROM expense_metadata
|
||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||
-- Ordered the same way as replaceLineItems (line-items.ts) and
|
||||
-- matchTransaction (intake-router.ts), so every reader of this row
|
||||
-- resolves the same one a write lands on. NULLS LAST because DESC is
|
||||
-- NULLS FIRST and the row that merely POINTS at the transaction has a
|
||||
-- NULL transaction_id.
|
||||
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
|
||||
-- pointing-only, and ZERO transactions resolving to more than one row by
|
||||
-- either link — so this changes nothing today. It is here so the readers
|
||||
-- still agree on the day multiplicity appears.
|
||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
||||
LIMIT 1`,
|
||||
[transactionId]
|
||||
);
|
||||
@@ -359,12 +387,21 @@ async function nudgeState(transactionId: number) {
|
||||
currency: string | null;
|
||||
amount: string;
|
||||
}>(
|
||||
`SELECT em.merchant_normalized AS merchant,
|
||||
COALESCE(em.currency, 'AUD') AS currency,
|
||||
// LATERAL + the shared tie-break, not a bare join: `queryRow` returns
|
||||
// rows[0] and Postgres guarantees no ordering, so a second metadata row on
|
||||
// one transaction would flip the merchant and currency on every card
|
||||
// refresh at random. Every sibling reader already resolves this way.
|
||||
`SELECT order_ctx.merchant_normalized AS merchant,
|
||||
COALESCE(order_ctx.currency, 'AUD') AS currency,
|
||||
t.amount
|
||||
FROM transactions t
|
||||
LEFT JOIN expense_metadata em
|
||||
ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT em.merchant_normalized, em.currency
|
||||
FROM expense_metadata em
|
||||
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||
ORDER BY (em.transaction_id = t.id) DESC NULLS LAST, em.id
|
||||
LIMIT 1
|
||||
) order_ctx ON true
|
||||
WHERE t.id = $1`,
|
||||
[transactionId]
|
||||
);
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { LineItemsValidationError, replaceLineItems, TransactionNotFoundError, validateLineItems } from "@/lib/line-items";
|
||||
|
||||
/**
|
||||
* Enrich endpoint (plan §11 C2): idempotent replace of the itemised receipt
|
||||
* behind a transaction.
|
||||
*
|
||||
* Accepts either machine token. Pantry already holds RECEIPT_INGEST_TOKEN for
|
||||
* the ING-10 lane and calls this on every confirm and on undo; the intake
|
||||
* router holds INTAKE_TOKEN. No session auth — there is no browser here.
|
||||
* Fails closed when neither is configured.
|
||||
*
|
||||
* Never creates a transaction: 404 when the id does not exist. Creates the
|
||||
* `expense_metadata` row when the transaction has none.
|
||||
*/
|
||||
function authorised(req: NextRequest): boolean {
|
||||
const got = req.headers.get("x-ingest-token");
|
||||
if (!got) return false;
|
||||
const accepted = [process.env.INTAKE_TOKEN, process.env.RECEIPT_INGEST_TOKEN].filter(Boolean);
|
||||
return accepted.includes(got);
|
||||
}
|
||||
|
||||
export async function PUT(req: NextRequest, { params }: { params: Promise<{ id: string }> }) {
|
||||
if (!authorised(req)) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
|
||||
|
||||
const { id } = await params;
|
||||
const transactionId = Number(id);
|
||||
if (!Number.isInteger(transactionId) || transactionId <= 0) return NextResponse.json({ error: "invalid transaction id" }, { status: 400 });
|
||||
|
||||
let body: { line_items?: unknown; source?: unknown; batch_id?: unknown; replace?: unknown; platform?: unknown };
|
||||
try {
|
||||
body = await req.json();
|
||||
} catch {
|
||||
return NextResponse.json({ error: "invalid JSON" }, { status: 400 });
|
||||
}
|
||||
|
||||
if (body.replace !== true) return NextResponse.json({ error: "replace: true is required — this endpoint only replaces" }, { status: 400 });
|
||||
if (body.source !== "intake" && body.source !== "pantry") return NextResponse.json({ error: "source must be 'intake' or 'pantry'" }, { status: 400 });
|
||||
const batchId = typeof body.batch_id === "string" && body.batch_id ? body.batch_id : null;
|
||||
const platform = typeof body.platform === "string" && body.platform ? body.platform : null;
|
||||
|
||||
try {
|
||||
const lineItems = validateLineItems(body.line_items);
|
||||
const result = await replaceLineItems(transactionId, lineItems, { source: body.source, platform, batchId });
|
||||
return NextResponse.json({
|
||||
transaction_id: result.transaction_id,
|
||||
metadata_id: result.metadata_id,
|
||||
created_metadata: result.created_metadata,
|
||||
previous_line_items: result.previous_line_items,
|
||||
line_items: result.line_items,
|
||||
});
|
||||
} catch (e) {
|
||||
if (e instanceof LineItemsValidationError) return NextResponse.json({ error: e.message }, { status: 400 });
|
||||
if (e instanceof TransactionNotFoundError) return NextResponse.json({ error: e.message }, { status: 404 });
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,155 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { getCurrentUser } from "@/lib/auth";
|
||||
import { queryRaw } from "@/lib/db";
|
||||
import { applyReconcileMatch, supersedeTransaction } from "@/lib/reconcile";
|
||||
|
||||
/**
|
||||
* Manually resolve a duplicate pair the automation cannot see.
|
||||
*
|
||||
* WHY THIS EXISTS: until now there was no way to merge or remove a transaction
|
||||
* at all. `superseded_by_id` was READ everywhere -- analytics excludes it,
|
||||
* auto-reconcile excludes it -- but written nowhere in application code; it had
|
||||
* only ever been set once, by migration 0023. The reconcile queue was the sole
|
||||
* merge path, and it only offers candidates within a narrow amount band, so a
|
||||
* pair outside it (SP PEAKDESIGN: notified $42.44, charged $43.71 -- the card's
|
||||
* 3% foreign transaction fee) was unresolvable from the UI.
|
||||
*
|
||||
* There is deliberately NO delete. Every child of `transactions` is
|
||||
* ON DELETE CASCADE, so removing a row destroys its splits, tags and metadata
|
||||
* curation. Both modes here keep the row and its children and merely stop it
|
||||
* counting:
|
||||
*
|
||||
* reconcile -- a statementLESS row folded into its statement twin
|
||||
* (`reconciled_with_id`). The everyday case.
|
||||
* supersede -- two rows on the same footing, e.g. a statement imported twice
|
||||
* (`superseded_by_id`). No constraint on either side's origin.
|
||||
*
|
||||
* Both move curation onto the survivor FIRST, via one shared implementation in
|
||||
* lib/reconcile.ts, because both hide the source row from every figure and
|
||||
* splits stranded on a hidden row silently stop counting.
|
||||
*/
|
||||
export async function POST(req: NextRequest, { params }: { params: Promise<{ id: string }> }) {
|
||||
const user = await getCurrentUser(req);
|
||||
if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 403 });
|
||||
|
||||
const { id } = await params;
|
||||
const loserId = Number(id);
|
||||
if (!Number.isInteger(loserId)) {
|
||||
return NextResponse.json({ error: "invalid transaction id" }, { status: 400 });
|
||||
}
|
||||
|
||||
let body: { other_id?: number; mode?: string };
|
||||
try {
|
||||
body = await req.json();
|
||||
} catch {
|
||||
return NextResponse.json({ error: "invalid JSON" }, { status: 400 });
|
||||
}
|
||||
|
||||
const otherId = Number(body.other_id);
|
||||
const mode = body.mode ?? "reconcile";
|
||||
if (!Number.isInteger(otherId)) {
|
||||
return NextResponse.json({ error: "other_id is required" }, { status: 400 });
|
||||
}
|
||||
if (loserId === otherId) {
|
||||
return NextResponse.json({ error: "a transaction cannot merge into itself" }, { status: 400 });
|
||||
}
|
||||
if (mode !== "reconcile" && mode !== "supersede") {
|
||||
return NextResponse.json({ error: "mode must be 'reconcile' or 'supersede'" }, { status: 400 });
|
||||
}
|
||||
|
||||
// Both rows must be the caller's. A statement row carries owner_id on the
|
||||
// statement rather than the transaction, which is why this reads the same
|
||||
// COALESCE every scoped query uses -- checking t.owner_id alone would reject
|
||||
// every statement row as "not found".
|
||||
const rows = await queryRaw<{
|
||||
id: number;
|
||||
statement_id: number | null;
|
||||
reconciled_with_id: number | null;
|
||||
superseded_by_id: number | null;
|
||||
voided_at: Date | null;
|
||||
}>(
|
||||
`SELECT t.id, t.statement_id, t.reconciled_with_id, t.superseded_by_id, t.voided_at
|
||||
FROM transactions t
|
||||
LEFT JOIN statements s ON s.id = t.statement_id
|
||||
WHERE t.id = ANY($1::int[]) AND COALESCE(t.owner_id, s.owner_id) = $2`,
|
||||
[[loserId, otherId], user.id]
|
||||
);
|
||||
if (rows.length !== 2) {
|
||||
return NextResponse.json({ error: "one or both transactions not found" }, { status: 404 });
|
||||
}
|
||||
const loser = rows.find((r) => r.id === loserId)!;
|
||||
const other = rows.find((r) => r.id === otherId)!;
|
||||
|
||||
// Refuse a row that is already folded into something else, in either
|
||||
// direction. Re-pointing it would strand the curation moved on the first
|
||||
// pass -- and silently, since both columns hide the row from every figure.
|
||||
for (const [label, row] of [["source", loser], ["target", other]] as const) {
|
||||
if (row.reconciled_with_id !== null) {
|
||||
return NextResponse.json(
|
||||
{ error: `${label} transaction ${row.id} is already reconciled with ${row.reconciled_with_id}` },
|
||||
{ status: 409 }
|
||||
);
|
||||
}
|
||||
if (row.superseded_by_id !== null) {
|
||||
return NextResponse.json(
|
||||
{ error: `${label} transaction ${row.id} is already superseded by ${row.superseded_by_id}` },
|
||||
{ status: 409 }
|
||||
);
|
||||
}
|
||||
// A voided row is a charge that never posted. As a source it is already
|
||||
// out of every figure; as a survivor it would take the other row out too.
|
||||
if (row.voided_at !== null) {
|
||||
return NextResponse.json(
|
||||
{ error: `${label} transaction ${row.id} is voided (a charge that never posted)` },
|
||||
{ status: 409 }
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
if (mode === "reconcile") {
|
||||
// applyReconcileMatch's contract, enforced here so a misuse is a 400 and
|
||||
// not a row quietly linked the wrong way round.
|
||||
if (loser.statement_id !== null) {
|
||||
return NextResponse.json(
|
||||
{ error: "reconcile expects the source to be a statementless row; use mode 'supersede'" },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
if (other.statement_id === null) {
|
||||
return NextResponse.json(
|
||||
{ error: "reconcile expects the target to be a statement row; use mode 'supersede'" },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
await applyReconcileMatch(loserId, otherId);
|
||||
} else {
|
||||
// Never hide a statement row behind a statementless one. The statement is
|
||||
// the bank's own record; the statementless row is a notification, a
|
||||
// receipt or a hand entry, and is the side that can be wrong -- a wallet
|
||||
// row carries the amount BEFORE the card's foreign transaction fee, so
|
||||
// burying the statement line under it would leave the ledger holding
|
||||
// $42.44 for a purchase the bank charged $43.71 for, with the statement
|
||||
// no longer reconciling against its own rows. The correct move is the
|
||||
// other direction, which is exactly what `reconcile` does.
|
||||
//
|
||||
// Statement-under-statement stays allowed: that is the imported-twice
|
||||
// pair migration 0023 existed for, and neither side is more true.
|
||||
if (loser.statement_id !== null && other.statement_id === null) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error:
|
||||
"Refusing to hide a statement row behind a statementless one — the statement is the bank's record. " +
|
||||
`Open transaction ${otherId} instead and fold it into this one.`,
|
||||
},
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
await supersedeTransaction(loserId, otherId);
|
||||
}
|
||||
} catch (e) {
|
||||
return NextResponse.json({ error: e instanceof Error ? e.message : String(e) }, { status: 400 });
|
||||
}
|
||||
|
||||
return NextResponse.json({ ok: true, mode, source: loserId, target: otherId });
|
||||
}
|
||||
@@ -34,6 +34,14 @@ export async function GET(
|
||||
-- through matched_transaction_id. Both directions have to resolve or the
|
||||
-- detail is missing on exactly the orders that were paid by card.
|
||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||
-- The same order as replaceLineItems in line-items.ts and matchTransaction
|
||||
-- in intake-router.ts. Both directions can resolve at once — a statement
|
||||
-- line owning its own row while a delivery receipt points at it — and an
|
||||
-- unordered LIMIT 1 then reads a different row from the one an enrichment
|
||||
-- wrote, so a PUT that reported success showed nothing on the screen.
|
||||
-- NULLS LAST because DESC is NULLS FIRST and the pointing row's
|
||||
-- transaction_id is NULL.
|
||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
||||
LIMIT 1`,
|
||||
[Number(id)]
|
||||
);
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { getCurrentUser } from "@/lib/auth";
|
||||
import { queryRaw } from "@/lib/db";
|
||||
import { unvoidTransaction, voidTransaction } from "@/lib/reconcile";
|
||||
|
||||
/**
|
||||
* POST /api/transactions/<id>/void { reason } -- a charge that never posted
|
||||
* DELETE /api/transactions/<id>/void -- undo it
|
||||
*
|
||||
* For a statementless row whose charge never reached a statement: an
|
||||
* authorisation hold the merchant released (txn 7951, a cancelled Airbnb
|
||||
* booking). The row stays, with its children and audit trail, and drops out of
|
||||
* every figure and the reconcile queue. Rules and refusals: `voidTransaction`
|
||||
* in lib/reconcile.ts.
|
||||
*/
|
||||
|
||||
async function ownedRow(req: NextRequest, idParam: string) {
|
||||
const user = await getCurrentUser(req);
|
||||
if (!user) return { error: NextResponse.json({ error: "Unauthorized" }, { status: 403 }) };
|
||||
const id = Number(idParam);
|
||||
if (!Number.isInteger(id)) {
|
||||
return { error: NextResponse.json({ error: "invalid transaction id" }, { status: 400 }) };
|
||||
}
|
||||
// Same owner scope as every other route; a statement row carries owner_id on
|
||||
// the statement. (Statement rows are refused later, but must 400, not 404.)
|
||||
const rows = await queryRaw<{ id: number }>(
|
||||
`SELECT t.id FROM transactions t
|
||||
LEFT JOIN statements s ON s.id = t.statement_id
|
||||
WHERE t.id = $1 AND COALESCE(t.owner_id, s.owner_id) = $2`,
|
||||
[id, user.id]
|
||||
);
|
||||
if (rows.length === 0) {
|
||||
return { error: NextResponse.json({ error: "transaction not found" }, { status: 404 }) };
|
||||
}
|
||||
return { id };
|
||||
}
|
||||
|
||||
export async function POST(req: NextRequest, { params }: { params: Promise<{ id: string }> }) {
|
||||
const owned = await ownedRow(req, (await params).id);
|
||||
if (owned.error) return owned.error;
|
||||
|
||||
let body: { reason?: unknown };
|
||||
try {
|
||||
body = await req.json();
|
||||
} catch {
|
||||
return NextResponse.json({ error: "invalid JSON" }, { status: 400 });
|
||||
}
|
||||
const reason = typeof body.reason === "string" ? body.reason.trim() : "";
|
||||
if (!reason) return NextResponse.json({ error: "reason is required" }, { status: 400 });
|
||||
|
||||
try {
|
||||
await voidTransaction(owned.id, reason);
|
||||
} catch (e) {
|
||||
return NextResponse.json({ error: e instanceof Error ? e.message : String(e) }, { status: 409 });
|
||||
}
|
||||
return NextResponse.json({ ok: true, id: owned.id, voided: true });
|
||||
}
|
||||
|
||||
export async function DELETE(req: NextRequest, { params }: { params: Promise<{ id: string }> }) {
|
||||
const owned = await ownedRow(req, (await params).id);
|
||||
if (owned.error) return owned.error;
|
||||
await unvoidTransaction(owned.id);
|
||||
return NextResponse.json({ ok: true, id: owned.id, voided: false });
|
||||
}
|
||||
@@ -1,6 +1,8 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { getCurrentUser } from "@/lib/auth";
|
||||
import { prisma, queryRaw } from "@/lib/db";
|
||||
import { queryRaw } from "@/lib/db";
|
||||
import { applyReconcileMatch } from "@/lib/reconcile";
|
||||
import { findAutoMatches } from "@/lib/auto-reconcile";
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
const user = await getCurrentUser(req);
|
||||
@@ -17,7 +19,7 @@ export async function POST(req: NextRequest) {
|
||||
// Verify all manual_ids belong to this user
|
||||
const manualIds = body.matches.map((m) => m.manual_id);
|
||||
const owned = await queryRaw<{ id: number }>(
|
||||
`SELECT id FROM transactions WHERE id = ANY($1::int[]) AND statement_id IS NULL AND owner_id = $2`,
|
||||
`SELECT id FROM transactions WHERE id = ANY($1::int[]) AND statement_id IS NULL AND voided_at IS NULL AND owner_id = $2`,
|
||||
[manualIds, user.id]
|
||||
);
|
||||
if (owned.length !== manualIds.length) {
|
||||
@@ -25,97 +27,71 @@ export async function POST(req: NextRequest) {
|
||||
}
|
||||
|
||||
let reconciled = 0;
|
||||
|
||||
for (const { manual_id, statement_tx_id } of body.matches) {
|
||||
await prisma.$transaction(async (tx) => {
|
||||
// Copy overrides: manual → statement tx
|
||||
const override = await tx.transaction_overrides.findUnique({
|
||||
where: { transaction_id: manual_id },
|
||||
});
|
||||
if (override) {
|
||||
await tx.transaction_overrides.upsert({
|
||||
where: { transaction_id: statement_tx_id },
|
||||
update: {
|
||||
category_override: override.category_override,
|
||||
merchant_normalized: override.merchant_normalized,
|
||||
notes: override.notes,
|
||||
my_share_percent: override.my_share_percent,
|
||||
updated_at: new Date(),
|
||||
},
|
||||
create: {
|
||||
transaction_id: statement_tx_id,
|
||||
category_override: override.category_override,
|
||||
merchant_normalized: override.merchant_normalized,
|
||||
notes: override.notes,
|
||||
my_share_percent: override.my_share_percent,
|
||||
},
|
||||
});
|
||||
await tx.transaction_overrides.deleteMany({ where: { transaction_id: manual_id } });
|
||||
}
|
||||
|
||||
// Move tags: manual → statement tx
|
||||
const tags = await tx.transaction_tags.findMany({ where: { transaction_id: manual_id } });
|
||||
if (tags.length) {
|
||||
await tx.transaction_tags.createMany({
|
||||
data: tags.map((t) => ({ transaction_id: statement_tx_id, tag_id: t.tag_id })),
|
||||
skipDuplicates: true,
|
||||
});
|
||||
await tx.transaction_tags.deleteMany({ where: { transaction_id: manual_id } });
|
||||
}
|
||||
|
||||
// Move splits: manual → statement tx
|
||||
const splits = await tx.transaction_splits.findMany({ where: { transaction_id: manual_id } });
|
||||
if (splits.length) {
|
||||
await tx.transaction_splits.createMany({
|
||||
data: splits.map((s) => ({
|
||||
transaction_id: statement_tx_id,
|
||||
participant_id: s.participant_id,
|
||||
share_percent: s.share_percent,
|
||||
})),
|
||||
skipDuplicates: true,
|
||||
});
|
||||
await tx.transaction_splits.deleteMany({ where: { transaction_id: manual_id } });
|
||||
}
|
||||
|
||||
// Move provenance: manual → statement tx.
|
||||
//
|
||||
// Overrides, tags and splits above were always carried across; expense_metadata was
|
||||
// the one child left behind, which did not matter while every metadata row came from
|
||||
// an email that had created its own transaction. It matters now: a scanned grocery
|
||||
// receipt puts its line items here, and reconciliation hides the manual row from
|
||||
// every figure — so without this the shop's contents disappear at exactly the moment
|
||||
// the statement line appears, and `COLES 0556 MANOR LAKES` stays as unreadable as it
|
||||
// was before the receipt was ever scanned.
|
||||
//
|
||||
// transaction_id is UNIQUE, so a statement row that already has metadata (an emailed
|
||||
// or Paperless copy got there first) keeps it. The pantry row stays attached to the
|
||||
// reconciled manual transaction and is flagged, rather than raising a constraint
|
||||
// violation or silently overwriting the other source.
|
||||
const moved = await tx.$executeRawUnsafe(
|
||||
`UPDATE expense_metadata SET transaction_id = $1
|
||||
WHERE transaction_id = $2
|
||||
AND NOT EXISTS (SELECT 1 FROM expense_metadata other WHERE other.transaction_id = $1)`,
|
||||
statement_tx_id,
|
||||
manual_id
|
||||
);
|
||||
if (moved === 0) {
|
||||
await tx.$executeRawUnsafe(
|
||||
`UPDATE expense_metadata
|
||||
SET flags = coalesce(flags, '[]'::jsonb) || '["metadata_collision_on_reconcile"]'::jsonb
|
||||
WHERE transaction_id = $1`,
|
||||
manual_id
|
||||
);
|
||||
}
|
||||
|
||||
// Mark manual tx as reconciled (link to statement tx)
|
||||
await tx.$executeRawUnsafe(
|
||||
`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`,
|
||||
statement_tx_id,
|
||||
manual_id
|
||||
);
|
||||
});
|
||||
await applyReconcileMatch(manual_id, statement_tx_id);
|
||||
reconciled++;
|
||||
}
|
||||
|
||||
return NextResponse.json({ reconciled });
|
||||
}
|
||||
|
||||
/**
|
||||
* Automatic pass over the reconcile queue.
|
||||
*
|
||||
* Applies only pairs that are unambiguous by the strict rule in
|
||||
* `findAutoMatches` -- exact amount (or exact native pair for a foreign row),
|
||||
* within a few days, and unique in BOTH directions. Everything else is left in
|
||||
* the queue for a person, which is the point: this exists to keep the queue
|
||||
* workable as the wallet lane adds ~63 rows a month, not to clear it.
|
||||
*
|
||||
* Validated against the 48 reconciliations a human had already made: the rule
|
||||
* reproduces all 48 and proposes nothing they did not.
|
||||
*
|
||||
* `?dry=1` returns what it WOULD do without writing, which is how this should
|
||||
* be run the first time after any change to the matching rule.
|
||||
*/
|
||||
export async function PATCH(req: NextRequest) {
|
||||
// Two callers, two auth modes. A person in the browser reconciles their own
|
||||
// queue; the scheduled sweep has no session and carries the machine token,
|
||||
// and must cover EVERY owner with a queue -- 169 of the 177 rows outstanding
|
||||
// belong to the second participant, so a sweep scoped to one user would leave
|
||||
// almost all of it untouched.
|
||||
const machineToken = req.headers.get("x-ingest-token");
|
||||
const machineOk = !!machineToken && machineToken === process.env.WALLET_INGEST_TOKEN;
|
||||
|
||||
let ownerIds: number[];
|
||||
if (machineOk) {
|
||||
ownerIds = (await queryRaw<{ owner_id: number }>(
|
||||
`SELECT DISTINCT owner_id FROM transactions
|
||||
WHERE statement_id IS NULL AND owner_id IS NOT NULL
|
||||
AND reconciled_with_id IS NULL AND superseded_by_id IS NULL`
|
||||
)).map((r) => r.owner_id);
|
||||
} else {
|
||||
const user = await getCurrentUser(req);
|
||||
if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 403 });
|
||||
ownerIds = [user.id];
|
||||
}
|
||||
|
||||
const dry = new URL(req.url).searchParams.get("dry") === "1";
|
||||
const matches = (await Promise.all(ownerIds.map((id) => findAutoMatches(id)))).flat();
|
||||
|
||||
if (dry) {
|
||||
return NextResponse.json({ dry_run: true, would_reconcile: matches.length, matches });
|
||||
}
|
||||
|
||||
let reconciled = 0;
|
||||
const failed: { manual_id: number; error: string }[] = [];
|
||||
for (const m of matches) {
|
||||
try {
|
||||
await applyReconcileMatch(m.manual_id, m.statement_tx_id);
|
||||
reconciled++;
|
||||
} catch (e) {
|
||||
// One bad pair must not abandon the rest -- each merge is its own
|
||||
// transaction, so a failure here leaves that pair untouched and in the
|
||||
// queue rather than half-applied.
|
||||
failed.push({ manual_id: m.manual_id, error: e instanceof Error ? e.message : String(e) });
|
||||
}
|
||||
}
|
||||
|
||||
return NextResponse.json({ reconciled, examined: matches.length, failed });
|
||||
}
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { processWalletIngestion, type WalletEvent } from "@/lib/wallet-ingestion";
|
||||
|
||||
/**
|
||||
* Machine ingest endpoint for interpreted iOS Wallet notifications.
|
||||
*
|
||||
* Sibling to /api/orders/ingest and /api/receipts/ingest, and deliberately
|
||||
* shaped like them: a shared secret rather than a browser session, because n8n
|
||||
* calls this app-to-app.
|
||||
*
|
||||
* Its own token so the wallet lane's credential can be rotated without touching
|
||||
* the order flow, which runs on a schedule nobody is watching.
|
||||
*
|
||||
* n8n is transport only. The interpretation already happened (Wallet Interpret,
|
||||
* CAp1s5cuzJk2mDaI, local model on Unraid because card tails are PII); this
|
||||
* endpoint owns every decision about what reaches the ledger.
|
||||
*/
|
||||
function authorised(req: NextRequest): boolean {
|
||||
const expected = process.env.WALLET_INGEST_TOKEN;
|
||||
if (!expected) return false; // fail closed when unconfigured
|
||||
const got = req.headers.get("x-ingest-token");
|
||||
return !!got && got === expected;
|
||||
}
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
if (!authorised(req)) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
|
||||
|
||||
let body: WalletEvent;
|
||||
try {
|
||||
body = await req.json();
|
||||
} catch {
|
||||
return NextResponse.json({ error: "invalid JSON" }, { status: 400 });
|
||||
}
|
||||
|
||||
if (!Number.isInteger(body?.event_id) || typeof body?.kind !== "string") {
|
||||
return NextResponse.json({ error: "event_id (int) and kind are required" }, { status: 400 });
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await processWalletIngestion(body);
|
||||
return NextResponse.json(result);
|
||||
} catch (e) {
|
||||
// A 500 here alerts #log-alerts through the workflow error handler, which
|
||||
// is what should happen: a write that fails is a template or schema change,
|
||||
// not something to swallow.
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
+25
-42
@@ -19,16 +19,20 @@ import { useQueryClient } from "@tanstack/react-query";
|
||||
import { useMonthlyAnalytics, useDailySpend, useTransactions, useUpdateTransaction } from "@/lib/hooks";
|
||||
import { formatCategory, CATEGORIES } from "@/lib/categories";
|
||||
import { CATEGORY_COLORS, CHART, TOOLTIP_STYLE } from "@/lib/category-colors";
|
||||
// The partial-month comparison rules live in their own module so they can be
|
||||
// tested. They used to be closures here reading `new Date()` inline, which is
|
||||
// why Priority 0 item 6 had no test (#155). See src/lib/partial-month.ts.
|
||||
import {
|
||||
currentMonthStr,
|
||||
prevMonth,
|
||||
daysInMonthOf,
|
||||
elapsedDays,
|
||||
spendThrough,
|
||||
comparisonBasis,
|
||||
baselineMonths,
|
||||
baselineAverage,
|
||||
} from "@/lib/partial-month";
|
||||
|
||||
function currentMonthStr(): string {
|
||||
const now = new Date();
|
||||
return `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, "0")}`;
|
||||
}
|
||||
function prevMonth(m: string): string {
|
||||
const [year, month] = m.split("-").map(Number);
|
||||
const d = new Date(year, month - 2, 1);
|
||||
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
|
||||
}
|
||||
function formatMonth(m: string): string {
|
||||
const [year, month] = m.split("-");
|
||||
return new Date(Number(year), Number(month) - 1, 1).toLocaleString("default", { month: "long", year: "numeric" });
|
||||
@@ -37,26 +41,6 @@ function formatShortMonth(m: string): string {
|
||||
const [year, month] = m.split("-");
|
||||
return new Date(Number(year), Number(month) - 1, 1).toLocaleString("default", { month: "short" });
|
||||
}
|
||||
function daysInMonthOf(m: string): number {
|
||||
const [year, month] = m.split("-").map(Number);
|
||||
return new Date(year, month, 0).getDate();
|
||||
}
|
||||
/**
|
||||
* How much of `month` has actually happened. A month in progress is only
|
||||
* complete up to today; every earlier month is complete.
|
||||
*/
|
||||
function elapsedDays(m: string): number {
|
||||
return m === currentMonthStr() ? new Date().getDate() : daysInMonthOf(m);
|
||||
}
|
||||
/** Spend in `month` from day 1 through `throughDay` inclusive. */
|
||||
function spendThrough(days: Record<number, number> | undefined, throughDay: number): number {
|
||||
if (!days) return 0;
|
||||
let sum = 0;
|
||||
for (const [d, v] of Object.entries(days)) {
|
||||
if (Number(d) <= throughDay) sum += v;
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
function fmt(n: number): string { return `$${Math.round(n).toLocaleString()}`; }
|
||||
function fmtExact(n: number): string { return `$${n.toFixed(2)}`; }
|
||||
function fmtSigned(n: number): string { return `${n >= 0 ? "+" : "−"}$${Math.abs(n) >= 100 ? Math.round(Math.abs(n)).toLocaleString() : Math.abs(n).toFixed(0)}`; }
|
||||
@@ -221,8 +205,7 @@ export default function AnalyticsPage() {
|
||||
|
||||
// A month in progress is only comparable to prior months through the same day.
|
||||
// Comparing 27 days of July against a full June always flattered July.
|
||||
const compareDay = elapsedDays(selectedMonth);
|
||||
const selectedIsPartial = selectedMonth === currentMonthStr();
|
||||
const { compareDay, isPartial: selectedIsPartial } = comparisonBasis(selectedMonth);
|
||||
|
||||
// Category rows for selected month
|
||||
const categoryRows = useMemo(() => {
|
||||
@@ -299,8 +282,10 @@ export default function AnalyticsPage() {
|
||||
// Only complete months with data form the baseline. A month still in
|
||||
// progress has no spend recorded past today, so including it would pull the
|
||||
// typical curve down by however much of it has not happened yet.
|
||||
const priorMonths = (analytics?.months ?? []).filter(
|
||||
(m) => m !== selectedMonth && m !== currentMonthStr() && (analytics?.totals[m]?.spent || 0) > 0
|
||||
const priorMonths = baselineMonths(
|
||||
analytics?.months ?? [],
|
||||
selectedMonth,
|
||||
analytics?.totals ?? {}
|
||||
);
|
||||
|
||||
let cum = 0;
|
||||
@@ -342,18 +327,16 @@ export default function AnalyticsPage() {
|
||||
// baseline, because most of it has not happened. And when the month in
|
||||
// progress is the one selected, its running total has to be measured against
|
||||
// the same slice of each prior month, not against their full totals.
|
||||
const otherMonths = months.filter(
|
||||
(m) => m !== selectedMonth && m !== currentMonthStr() && (analytics.totals[m]?.spent || 0) > 0
|
||||
);
|
||||
const otherMonths = baselineMonths(months, selectedMonth, analytics.totals);
|
||||
const comparableSpend = selectedIsPartial
|
||||
? spendThrough(dailyData?.daily?.[selectedMonth], compareDay)
|
||||
: totals.spent;
|
||||
const avgSpend = otherMonths.length
|
||||
? otherMonths.reduce(
|
||||
(s, m) => s + (selectedIsPartial ? spendThrough(dailyData?.daily?.[m], compareDay) : analytics.totals[m]?.spent || 0),
|
||||
0
|
||||
) / otherMonths.length
|
||||
: 0;
|
||||
const avgSpend = baselineAverage(otherMonths, {
|
||||
isPartial: selectedIsPartial,
|
||||
compareDay,
|
||||
daily: dailyData?.daily,
|
||||
totals: analytics.totals,
|
||||
});
|
||||
const avgDeltaPct = avgSpend > 0 ? Math.round(((comparableSpend - avgSpend) / avgSpend) * 100) : 0;
|
||||
const throughQualifier = selectedIsPartial ? ` through day ${compareDay}` : "";
|
||||
const heroSentence =
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import { usePendingReconciliations, useReconcile } from "@/lib/hooks";
|
||||
import { usePendingReconciliations, useReconcile, useVoidTransaction } from "@/lib/hooks";
|
||||
import type { ManualTxWithMatches, PotentialMatch } from "@/lib/queries";
|
||||
|
||||
function formatDate(d: string | Date) {
|
||||
@@ -141,7 +141,7 @@ export default function ReconcilePage() {
|
||||
<div className="border border-zinc-800 rounded-xl overflow-hidden">
|
||||
<div className="px-4 py-3 bg-zinc-900/60 border-b border-zinc-800">
|
||||
<p className="text-sm font-medium text-zinc-400">No statement matches found ({noMatches.length})</p>
|
||||
<p className="text-xs text-zinc-600 mt-0.5">These may not have hit a statement yet, or the statement hasn't been imported.</p>
|
||||
<p className="text-xs text-zinc-600 mt-0.5">These may not have hit a statement yet, or the statement hasn't been imported.</p>
|
||||
</div>
|
||||
<div className="divide-y divide-zinc-800">
|
||||
{noMatches.map((tx) => (
|
||||
@@ -156,6 +156,7 @@ export default function ReconcilePage() {
|
||||
<span key={tag.id} className="w-2 h-2 rounded-full" style={{ backgroundColor: tag.color }} title={tag.name} />
|
||||
))}
|
||||
</div>
|
||||
<VoidButton id={tx.id} amount={tx.amount} onError={setError} />
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
@@ -243,6 +244,7 @@ function ReconcileRow({
|
||||
Clear
|
||||
</button>
|
||||
)}
|
||||
<VoidButton id={tx.id} amount={tx.amount} />
|
||||
{!isSkipped && (
|
||||
<button onClick={onSkip} className="text-xs text-zinc-500 hover:text-zinc-300 px-2 py-1 rounded hover:bg-zinc-800">
|
||||
Skip (no match)
|
||||
@@ -296,3 +298,34 @@ function MatchRow({
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* "Never posted": a released authorisation hold, e.g. a booking cancelled
|
||||
* before the charge settled. Drops the row out of every figure and this queue
|
||||
* (migration 0040); the row and its history are kept.
|
||||
*/
|
||||
function VoidButton({ id, amount, onError }: { id: number; amount: number; onError?: (msg: string) => void }) {
|
||||
const voidTx = useVoidTransaction();
|
||||
async function handle() {
|
||||
const reason = window.prompt(
|
||||
`Mark this $${Number(amount).toFixed(2)} charge as never posted? It will stop counting everywhere.\n\nReason (e.g. "hold released, booking cancelled"):`
|
||||
);
|
||||
if (!reason?.trim()) return;
|
||||
try {
|
||||
await voidTx.mutateAsync({ id, reason });
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : "Void failed";
|
||||
if (onError) onError(msg); else window.alert(msg);
|
||||
}
|
||||
}
|
||||
return (
|
||||
<button
|
||||
onClick={handle}
|
||||
disabled={voidTx.isPending}
|
||||
title="The charge never posted (e.g. a released hold) — stop counting it"
|
||||
className="text-xs text-zinc-500 hover:text-red-300 px-2 py-1 rounded hover:bg-zinc-800 disabled:opacity-50 flex-shrink-0"
|
||||
>
|
||||
{voidTx.isPending ? "Voiding…" : "Never posted"}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -463,7 +463,7 @@ export default function RulesPage() {
|
||||
>
|
||||
{expandedRun === run.id ? "▾" : "▸"}
|
||||
</button>
|
||||
<span className="text-zinc-300">{new Date(run.applied_at).toLocaleString()}</span>
|
||||
<span className="text-zinc-300">{new Date(run.applied_at).toLocaleString("en-AU")}</span>
|
||||
{/* Which rule ran matters most: a merchant rename and a
|
||||
50/50 split of everything look identical as counts. */}
|
||||
<span className="text-zinc-200">
|
||||
@@ -480,7 +480,7 @@ export default function RulesPage() {
|
||||
)}
|
||||
</div>
|
||||
{run.reverted_at ? (
|
||||
<span className="text-xs text-zinc-500">reverted {new Date(run.reverted_at).toLocaleString()}</span>
|
||||
<span className="text-xs text-zinc-500">reverted {new Date(run.reverted_at).toLocaleString("en-AU")}</span>
|
||||
) : (
|
||||
<button
|
||||
onClick={() => {
|
||||
|
||||
+26
-9
@@ -432,6 +432,19 @@ const SharedTxRow = memo(function SharedTxRow({
|
||||
// ── Main page ─────────────────────────────────────────────────────────────────
|
||||
type SortCol = "transaction_date" | "created_at" | "amount";
|
||||
|
||||
/**
|
||||
* Sort direction marker for a column header.
|
||||
*
|
||||
* Declared at module scope and given the sort state as props, rather than
|
||||
* closing over it inside SharedPage: a component defined during render is a
|
||||
* new type on every render, so React unmounts and remounts it each time
|
||||
* instead of updating it.
|
||||
*/
|
||||
function SortIcon({ col, sortCol, sortDir }: { col: SortCol; sortCol: SortCol; sortDir: "asc" | "desc" }) {
|
||||
if (sortCol !== col) return <span className="text-zinc-600 ml-0.5">↕</span>;
|
||||
return <span className="ml-0.5">{sortDir === "desc" ? "↓" : "↑"}</span>;
|
||||
}
|
||||
|
||||
export default function SharedPage() {
|
||||
const [tagIds, setTagIds] = useState<string[]>([]);
|
||||
const [participantId, setParticipantId] = useState<number | undefined>(undefined);
|
||||
@@ -478,10 +491,6 @@ export default function SharedPage() {
|
||||
else { setSortCol(col); setSortDir("desc"); }
|
||||
}
|
||||
|
||||
function SortIcon({ col }: { col: SortCol }) {
|
||||
if (sortCol !== col) return <span className="text-zinc-600 ml-0.5">↕</span>;
|
||||
return <span className="ml-0.5">{sortDir === "desc" ? "↓" : "↑"}</span>;
|
||||
}
|
||||
const { data: balances = [], isLoading: balLoading } = useParticipantBalances(realTagIds);
|
||||
const { data: allTags = [] } = useTags();
|
||||
const { data: me } = useCurrentUser();
|
||||
@@ -504,9 +513,17 @@ export default function SharedPage() {
|
||||
// table was what made expanding a receipt freeze the browser. "Show more"
|
||||
// extends the window; changing any filter resets it.
|
||||
const [visibleCount, setVisibleCount] = useState(100);
|
||||
useEffect(() => {
|
||||
// Reset during render behind a previous-value guard rather than in an effect.
|
||||
// Note the key joins `tagIds` by value: as a raw dependency it was a fresh
|
||||
// array on many renders, so the window was being reset more often than
|
||||
// "changing any filter" claims — including after "Show more", which is the
|
||||
// one interaction this must not undo.
|
||||
const windowKey = `${search}|${sortCol}|${sortDir}|${tagIds.join(",")}|${participantId ?? ""}`;
|
||||
const [lastWindowKey, setLastWindowKey] = useState(windowKey);
|
||||
if (windowKey !== lastWindowKey) {
|
||||
setLastWindowKey(windowKey);
|
||||
setVisibleCount(100);
|
||||
}, [search, sortCol, sortDir, tagIds, participantId]);
|
||||
}
|
||||
const visible = (transactions as SharedTransactionRow[]).slice(0, visibleCount);
|
||||
|
||||
// Receipt expansion, same pattern as the transactions page. The shared
|
||||
@@ -667,13 +684,13 @@ export default function SharedPage() {
|
||||
className="text-left px-4 py-2 text-xs text-zinc-500 font-medium cursor-pointer hover:text-white whitespace-nowrap"
|
||||
onClick={() => toggleSort("transaction_date")}
|
||||
>
|
||||
Date <SortIcon col="transaction_date" />
|
||||
Date <SortIcon col="transaction_date" sortCol={sortCol} sortDir={sortDir} />
|
||||
</th>
|
||||
<th
|
||||
className="text-left px-4 py-2 text-xs text-zinc-500 font-medium cursor-pointer hover:text-white whitespace-nowrap"
|
||||
onClick={() => toggleSort("created_at")}
|
||||
>
|
||||
Imported <SortIcon col="created_at" />
|
||||
Imported <SortIcon col="created_at" sortCol={sortCol} sortDir={sortDir} />
|
||||
</th>
|
||||
<th className="text-left px-4 py-2 text-xs text-zinc-500 font-medium sticky left-0 z-10 bg-zinc-900 border-r border-zinc-800/80">Description</th>
|
||||
<th className="text-left px-4 py-2 text-xs text-zinc-500 font-medium">Category</th>
|
||||
@@ -681,7 +698,7 @@ export default function SharedPage() {
|
||||
className="text-right px-4 py-2 text-xs text-zinc-500 font-medium cursor-pointer hover:text-white"
|
||||
onClick={() => toggleSort("amount")}
|
||||
>
|
||||
Amount <SortIcon col="amount" />
|
||||
Amount <SortIcon col="amount" sortCol={sortCol} sortDir={sortDir} />
|
||||
</th>
|
||||
{/* Paid by sits next to Splits deliberately: together they are the
|
||||
two halves of the question this page exists to answer — whose
|
||||
|
||||
@@ -59,7 +59,7 @@ function ConvertModal({
|
||||
<div className="px-6 pt-5 pb-4 border-b border-zinc-800">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="w-3 h-3 rounded-full flex-shrink-0" style={{ backgroundColor: tag.color }} />
|
||||
<h3 className="font-semibold text-sm text-zinc-300">Convert "{tag.name}" to Trip</h3>
|
||||
<h3 className="font-semibold text-sm text-zinc-300">Convert "{tag.name}" to Trip</h3>
|
||||
</div>
|
||||
<p className="text-xs text-zinc-500 mt-1">
|
||||
Creates a trip with the same name and color, and assigns all {tag.transaction_count} tagged transaction{tag.transaction_count !== 1 ? "s" : ""} to it.
|
||||
|
||||
@@ -148,7 +148,7 @@ function SaveAsRulePrompt({
|
||||
Automatically apply this {field === "category" ? "category" : "merchant name"} to future matching transactions.
|
||||
</p>
|
||||
<div className="bg-zinc-900 rounded-lg px-3 py-2 text-xs text-zinc-300 mb-3 space-y-1">
|
||||
<p><span className="text-zinc-500">If</span> {conditionField === "merchant_normalized" ? "merchant" : "description"} {conditionOperator} <span className="text-white">"{conditionValue}"</span></p>
|
||||
<p><span className="text-zinc-500">If</span> {conditionField === "merchant_normalized" ? "merchant" : "description"} {conditionOperator} <span className="text-white">"{conditionValue}"</span></p>
|
||||
<p>
|
||||
<span className="text-zinc-500">Then</span>{" "}
|
||||
{field === "category"
|
||||
@@ -250,11 +250,17 @@ function MarkAsPaymentModal({
|
||||
|
||||
const [participantId, setParticipantId] = useState<number | "">(others[0]?.id ?? "");
|
||||
|
||||
useEffect(() => {
|
||||
if (participantId === "" && others.length > 0) {
|
||||
setParticipantId(others[0].id);
|
||||
}
|
||||
}, [others]);
|
||||
// Seed the picker once the participant list arrives — the initial state is
|
||||
// computed before the query resolves, so it starts empty.
|
||||
//
|
||||
// Adjusted during render rather than in an effect, which is also strictly
|
||||
// more correct: `others` is rebuilt by .filter() on every render, so as a
|
||||
// dependency it never compared equal and the effect ran every render anyway
|
||||
// — just one commit later, painting an empty select first. The condition
|
||||
// clears itself on the first non-empty list, so this cannot loop.
|
||||
if (participantId === "" && others.length > 0) {
|
||||
setParticipantId(others[0].id);
|
||||
}
|
||||
// For credits/refunds the default direction is "they paid me"
|
||||
const [direction, setDirection] = useState<"received" | "sent">(
|
||||
SPEND_TYPES.has(transaction.transaction_type) ? "sent" : "received"
|
||||
@@ -1079,7 +1085,18 @@ function TransactionsContent() {
|
||||
) : (
|
||||
<>
|
||||
{formatAmount(t.amount_aud ?? t.amount, t.transaction_type)}
|
||||
{t.currency && t.currency !== "AUD" && (
|
||||
{/*
|
||||
The native figure: `foreign_currency_amount` where one
|
||||
was recorded (a statement or wallet row, whose `amount`
|
||||
is AUD), else `amount` on a non-AUD statement. Printing
|
||||
`amount` under the foreign code put the AUD figure
|
||||
behind an INR label on every wallet row.
|
||||
*/}
|
||||
{t.foreign_currency_code && t.foreign_currency_code !== "AUD" && t.foreign_currency_amount !== null ? (
|
||||
<div className="text-[10px] text-zinc-500 mt-0.5">
|
||||
{formatAmount(t.foreign_currency_amount, t.transaction_type, t.foreign_currency_code)}
|
||||
</div>
|
||||
) : t.currency && t.currency !== "AUD" && (
|
||||
<div className="text-[10px] text-zinc-500 mt-0.5">
|
||||
{formatAmount(t.amount, t.transaction_type, t.currency)}
|
||||
</div>
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useRef, useEffect } from "react";
|
||||
import { useState, useRef } from "react";
|
||||
import { CATEGORIES, formatCategory } from "@/lib/categories";
|
||||
import { useImportCSV, useStatementCoverage } from "@/lib/hooks";
|
||||
import {
|
||||
@@ -54,13 +54,23 @@ export function CsvImportModal({ onClose }: { onClose: () => void }) {
|
||||
const [savePreset, setSavePreset] = useState(false);
|
||||
const [editedRows, setEditedRows] = useState<ParsedTransaction[]>([]);
|
||||
const [error, setError] = useState("");
|
||||
const [presets, setPresets] = useState<BankPreset[]>([]);
|
||||
// Read once, at mount, from localStorage. A lazy initializer rather than an
|
||||
// effect: it runs exactly once per mount — the same semantics as
|
||||
// `useEffect(…, [])` — but before the first paint, so the preset dropdown
|
||||
// does not appear empty and then populate. Nothing in this component writes
|
||||
// `presets` afterwards, so there is no state to keep in sync.
|
||||
//
|
||||
// The `typeof window` guard is for safety only: this modal is mounted from
|
||||
// client state and so is never server-rendered. If that ever changes, it
|
||||
// degrades to an empty list instead of throwing on `localStorage`.
|
||||
const [presets] = useState<BankPreset[]>(() =>
|
||||
typeof window === "undefined" ? [] : loadBankPresets()
|
||||
);
|
||||
const [insertedCount, setInsertedCount] = useState(0);
|
||||
const [coveredRows, setCoveredRows] = useState<ParsedTransaction[]>([]);
|
||||
const [showCovered, setShowCovered] = useState(false);
|
||||
const coverage = useStatementCoverage();
|
||||
|
||||
useEffect(() => { setPresets(loadBankPresets()); }, []);
|
||||
|
||||
function handleFile(file: File) {
|
||||
setError("");
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import { useEffect, useState } from "react";
|
||||
import {
|
||||
useUpdateTransaction,
|
||||
useMergeTransaction,
|
||||
useTags,
|
||||
useAddTransactionTag,
|
||||
useRemoveTransactionTag,
|
||||
@@ -93,6 +94,210 @@ function InlineTags({ transactionId, initialTags }: { transactionId: number; ini
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Determines which merge mode the backend will accept for (loser, candidate).
|
||||
* Mirrors the constraint enforced server-side in the merge route: reconcile
|
||||
* requires a statementless loser folding into a statement survivor; every
|
||||
* other pairing is a supersede.
|
||||
*/
|
||||
function mergeModeFor(loser: TransactionRow, candidate: TransactionRow): "reconcile" | "supersede" {
|
||||
return loser.statement_id === null && candidate.statement_id !== null ? "reconcile" : "supersede";
|
||||
}
|
||||
|
||||
/**
|
||||
* Collapsed by default, at the bottom, visually separate — this is the one
|
||||
* control on the modal that changes which rows this transaction can ever be
|
||||
* again. `transaction` is always the row that will be hidden (the loser);
|
||||
* the row picked here is the survivor (`other_id`).
|
||||
*/
|
||||
function DuplicateSection({ transaction, onMerged }: { transaction: TransactionRow; onMerged: () => void }) {
|
||||
const [expanded, setExpanded] = useState(false);
|
||||
const [query, setQuery] = useState("");
|
||||
const [results, setResults] = useState<TransactionRow[]>([]);
|
||||
const [searching, setSearching] = useState(false);
|
||||
const [selected, setSelected] = useState<TransactionRow | null>(null);
|
||||
const mergeTxn = useMergeTransaction();
|
||||
|
||||
useEffect(() => {
|
||||
if (!expanded || selected) return;
|
||||
const q = query.trim();
|
||||
// No synchronous setState here: an empty box is handled at render time, and
|
||||
// `searching` is raised inside the timeout. Clearing state here instead
|
||||
// cascades a render on every keystroke (react-hooks lint catches it).
|
||||
if (q.length === 0) return;
|
||||
const handle = setTimeout(() => {
|
||||
setSearching(true);
|
||||
fetch(`/api/transactions?search=${encodeURIComponent(q)}&limit=8`)
|
||||
.then((res) => res.json())
|
||||
.then((body: { data?: TransactionRow[] }) => {
|
||||
setResults((body.data ?? []).filter((r) => r.id !== transaction.id));
|
||||
})
|
||||
.catch(() => setResults([]))
|
||||
.finally(() => setSearching(false));
|
||||
}, 300);
|
||||
return () => clearTimeout(handle);
|
||||
}, [query, expanded, selected, transaction.id]);
|
||||
|
||||
function reset() {
|
||||
setExpanded(false);
|
||||
setQuery("");
|
||||
setResults([]);
|
||||
setSelected(null);
|
||||
mergeTxn.reset();
|
||||
}
|
||||
|
||||
async function handleConfirm() {
|
||||
if (!selected) return;
|
||||
try {
|
||||
await mergeTxn.mutateAsync({
|
||||
id: transaction.id,
|
||||
other_id: selected.id,
|
||||
mode: mergeModeFor(transaction, selected),
|
||||
});
|
||||
onMerged();
|
||||
} catch {
|
||||
// Surfaced below via mergeTxn.error — section stays open so they can retry.
|
||||
}
|
||||
}
|
||||
|
||||
const mode = selected ? mergeModeFor(transaction, selected) : null;
|
||||
// Hiding a statement row behind a statementless one is always backwards — the
|
||||
// bank's record is the side that survives. The API refuses it too; catching it
|
||||
// here just means the button explains itself instead of returning a 400.
|
||||
const wrongDirection =
|
||||
selected !== null && transaction.statement_id !== null && selected.statement_id === null;
|
||||
|
||||
return (
|
||||
<div className="pt-4 border-t border-zinc-800">
|
||||
{!expanded ? (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setExpanded(true)}
|
||||
className="text-xs text-zinc-500 hover:text-zinc-300"
|
||||
>
|
||||
Mark as duplicate…
|
||||
</button>
|
||||
) : (
|
||||
<div className="space-y-2.5">
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="font-semibold text-xs text-zinc-400">Mark as duplicate</p>
|
||||
<button
|
||||
type="button"
|
||||
onClick={reset}
|
||||
className="text-xs text-zinc-500 hover:text-zinc-300"
|
||||
>
|
||||
Cancel
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{!selected ? (
|
||||
<>
|
||||
<input
|
||||
type="text"
|
||||
value={query}
|
||||
onChange={(e) => setQuery(e.target.value)}
|
||||
placeholder="Search for the surviving transaction…"
|
||||
autoFocus
|
||||
className="w-full bg-zinc-800 border border-zinc-700 rounded px-2 py-1.5 text-sm"
|
||||
/>
|
||||
{searching && <p className="text-xs text-zinc-500">Searching…</p>}
|
||||
{!searching && query.trim().length > 0 && results.length === 0 && (
|
||||
<p className="text-xs text-zinc-600 italic">No matches</p>
|
||||
)}
|
||||
{/* Gated on the query too: results are no longer cleared when the
|
||||
box empties, so without this the last search's rows linger. */}
|
||||
{query.trim().length > 0 && results.length > 0 && (
|
||||
<div className="space-y-1 max-h-48 overflow-y-auto">
|
||||
{results.map((r) => (
|
||||
<button
|
||||
key={r.id}
|
||||
type="button"
|
||||
onClick={() => setSelected(r)}
|
||||
className="w-full text-left px-2 py-1.5 rounded bg-zinc-800/50 hover:bg-zinc-800 border border-zinc-700/50"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<span className="text-sm text-zinc-200 truncate">{r.description}</span>
|
||||
<span
|
||||
className={`text-sm font-mono shrink-0 ${SPEND_TYPES.has(r.transaction_type) ? "text-red-400" : "text-green-400"}`}
|
||||
>
|
||||
{formatAmount(r.amount, r.transaction_type)}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex items-center justify-between mt-0.5">
|
||||
<span className="text-xs text-zinc-500">
|
||||
{new Date(r.transaction_date).toLocaleDateString("en-AU", { day: "numeric", month: "short", year: "numeric" })}
|
||||
</span>
|
||||
<span className="text-xs text-zinc-500">
|
||||
{r.statement_id !== null ? "Statement row" : "Statementless"}
|
||||
</span>
|
||||
</div>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
) : (
|
||||
<div className="space-y-2">
|
||||
<div className="px-2 py-1.5 rounded bg-zinc-800/50 border border-zinc-700/50">
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<span className="text-sm text-zinc-200 truncate">{selected.description}</span>
|
||||
<span
|
||||
className={`text-sm font-mono shrink-0 ${SPEND_TYPES.has(selected.transaction_type) ? "text-red-400" : "text-green-400"}`}
|
||||
>
|
||||
{formatAmount(selected.amount, selected.transaction_type)}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex items-center justify-between mt-0.5">
|
||||
<span className="text-xs text-zinc-500">
|
||||
{new Date(selected.transaction_date).toLocaleDateString("en-AU", { day: "numeric", month: "short", year: "numeric" })}
|
||||
</span>
|
||||
<span className="text-xs text-zinc-500">
|
||||
{selected.statement_id !== null ? "Statement row" : "Statementless"}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
{wrongDirection ? (
|
||||
<p className="text-xs text-red-400">
|
||||
This is the wrong way round — the statement is the bank's record, so it
|
||||
has to be the row that survives. Close this and open{" "}
|
||||
<span className="text-zinc-300">{selected.description}</span> instead, then
|
||||
fold that one into this.
|
||||
</p>
|
||||
) : (
|
||||
<p className="text-xs text-amber-400/90">
|
||||
{mode === "reconcile"
|
||||
? "This row will be folded into the statement row above."
|
||||
: "This row will be marked a duplicate of the row above."}
|
||||
</p>
|
||||
)}
|
||||
{mergeTxn.error && (
|
||||
<p className="text-red-400 text-xs">{(mergeTxn.error as Error).message}</p>
|
||||
)}
|
||||
<div className="flex gap-2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setSelected(null)}
|
||||
className="px-3 py-1.5 bg-zinc-800 hover:bg-zinc-700 text-zinc-300 rounded-lg text-xs"
|
||||
>
|
||||
Choose a different row
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={handleConfirm}
|
||||
disabled={mergeTxn.isPending || wrongDirection}
|
||||
className="px-3 py-1.5 bg-amber-700 hover:bg-amber-600 disabled:opacity-50 text-white rounded-lg text-xs"
|
||||
>
|
||||
{mergeTxn.isPending ? "Merging…" : mode === "reconcile" ? "Fold into statement row" : "Mark as duplicate"}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function EditTransactionModal({
|
||||
transaction,
|
||||
onClose,
|
||||
@@ -416,6 +621,8 @@ export function EditTransactionModal({
|
||||
|
||||
<OrderDetails transactionId={transaction.id} currency={transaction.currency ?? null} />
|
||||
|
||||
<DuplicateSection transaction={transaction} onMerged={onClose} />
|
||||
|
||||
</div>
|
||||
|
||||
{/* Footer */}
|
||||
|
||||
@@ -13,12 +13,9 @@ import {
|
||||
type OrderReceipt,
|
||||
type OrderRating,
|
||||
} from "@/lib/hooks";
|
||||
|
||||
const PLATFORM_LABEL: Record<string, string> = {
|
||||
doordash: "DoorDash",
|
||||
ubereats: "Uber Eats",
|
||||
uber: "Uber",
|
||||
};
|
||||
// One platform vocabulary, shared with the intake cards — three copies had
|
||||
// already drifted apart (the intake one was missing `uber`).
|
||||
import { platformLabel } from "@/lib/intake-rules";
|
||||
|
||||
/**
|
||||
* Who records verdicts. A two-person household with one primary user: the
|
||||
@@ -99,7 +96,7 @@ export function OrderDetails({
|
||||
<p className="text-xs text-zinc-500">
|
||||
Order details
|
||||
{receipt.platform && (
|
||||
<span className="ml-1.5 text-zinc-400">{PLATFORM_LABEL[receipt.platform] ?? receipt.platform}</span>
|
||||
<span className="ml-1.5 text-zinc-400">{platformLabel(receipt.platform)}</span>
|
||||
)}
|
||||
</p>
|
||||
{receipt.card_last4 && (
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { useMemo, useState } from "react";
|
||||
import { useRuleMatches, useApplyRuleToSelection, type RuleMatchChange } from "@/lib/hooks";
|
||||
import { formatCategory } from "@/lib/categories";
|
||||
|
||||
@@ -39,9 +39,17 @@ export function RulePreviewModal({
|
||||
|
||||
// Default to everything the rule would change — the common case is "yes, all
|
||||
// of these", with deselection as the exception.
|
||||
useEffect(() => {
|
||||
if (data?.transactions) setSelected(new Set(data.transactions.map((t) => t.id)));
|
||||
}, [data]);
|
||||
//
|
||||
// Adjusted during render behind a previous-value guard rather than in an
|
||||
// effect. In an effect the modal painted once with nothing ticked and then
|
||||
// again with everything ticked; here the seeded selection is part of the
|
||||
// first commit. The guard is on `data` identity, so a user deselecting a row
|
||||
// is not undone on the next render — only a genuinely new preview reseeds.
|
||||
const [seededFrom, setSeededFrom] = useState<typeof data>(undefined);
|
||||
if (data !== seededFrom) {
|
||||
setSeededFrom(data);
|
||||
setSelected(new Set(data?.transactions?.map((t) => t.id) ?? []));
|
||||
}
|
||||
|
||||
const rows = data?.transactions ?? [];
|
||||
const allSelected = rows.length > 0 && selected.size === rows.length;
|
||||
|
||||
@@ -82,10 +82,19 @@ export function Sidebar() {
|
||||
const pathname = usePathname();
|
||||
const [open, setOpen] = useState(false);
|
||||
|
||||
// Close drawer on route change
|
||||
useEffect(() => {
|
||||
// Close the drawer on route change.
|
||||
//
|
||||
// Adjusting state during render behind a previous-value guard — the
|
||||
// documented React pattern for "reset state when a prop changes" — rather
|
||||
// than in an effect: React re-runs this component immediately and never
|
||||
// commits the intermediate DOM, so there is no cascading render. Putting it
|
||||
// in each Link's onClick is the other candidate fix and is worse here: it
|
||||
// misses browser back/forward, where the route changes with no click.
|
||||
const [lastPathname, setLastPathname] = useState(pathname);
|
||||
if (pathname !== lastPathname) {
|
||||
setLastPathname(pathname);
|
||||
setOpen(false);
|
||||
}, [pathname]);
|
||||
}
|
||||
|
||||
// Prevent body scroll when drawer is open
|
||||
useEffect(() => {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useEffect } from "react";
|
||||
import { useState } from "react";
|
||||
import { useParticipants, useSetSplits, useTransactionSplits, useBulkAction, useCreateRule, useCurrentUser } from "@/lib/hooks";
|
||||
|
||||
interface Split {
|
||||
@@ -33,22 +33,32 @@ export function SplitModal({ transactionId, transactionIds, amount, description,
|
||||
const [saveAsRule, setSaveAsRule] = useState(false);
|
||||
const [ruleSaved, setRuleSaved] = useState(false);
|
||||
|
||||
// Initialise: bulk always defaults to 100% Me; single loads existing splits
|
||||
useEffect(() => {
|
||||
if (!participants || participants.length === 0 || !currentUser) return;
|
||||
if (isBulk) {
|
||||
setSplitsState([{ participant_id: currentUser.id, share_percent: 100 }]);
|
||||
} else if (existingSplits && existingSplits.length > 0) {
|
||||
setSplitsState(
|
||||
existingSplits.map((s: { participant_id: number; share_percent: number }) => ({
|
||||
participant_id: s.participant_id,
|
||||
share_percent: Number(s.share_percent),
|
||||
}))
|
||||
);
|
||||
} else {
|
||||
setSplitsState([{ participant_id: currentUser.id, share_percent: 100 }]);
|
||||
}
|
||||
}, [existingSplits, participants, isBulk, currentUser]);
|
||||
// Initialise: bulk always defaults to 100% Me; single loads existing splits.
|
||||
//
|
||||
// Seeded during render behind a previous-value guard rather than in an
|
||||
// effect. Both queries resolve after the first paint, so as an effect this
|
||||
// rendered an empty split table and then replaced it — and, worse, it re-ran
|
||||
// on every identity change of `participants`/`existingSplits`, which would
|
||||
// discard edits in progress if either query refetched while the modal was
|
||||
// open. Keying on the inputs that actually define the seed fixes both.
|
||||
const seedKey =
|
||||
!participants || participants.length === 0 || !currentUser
|
||||
? null
|
||||
: `${isBulk ? "bulk" : singleId}:${currentUser.id}:${
|
||||
existingSplits ? JSON.stringify(existingSplits) : "none"
|
||||
}`;
|
||||
const [seededFrom, setSeededFrom] = useState<string | null>(null);
|
||||
if (seedKey !== null && seedKey !== seededFrom) {
|
||||
setSeededFrom(seedKey);
|
||||
setSplitsState(
|
||||
!isBulk && existingSplits && existingSplits.length > 0
|
||||
? existingSplits.map((s: { participant_id: number; share_percent: number }) => ({
|
||||
participant_id: s.participant_id,
|
||||
share_percent: Number(s.share_percent),
|
||||
}))
|
||||
: [{ participant_id: currentUser!.id, share_percent: 100 }]
|
||||
);
|
||||
}
|
||||
|
||||
const total = splits.reduce((sum, s) => sum + s.share_percent, 0);
|
||||
|
||||
|
||||
@@ -71,9 +71,17 @@ export const STATEMENTS_JOIN = `LEFT JOIN statements s ON s.id = t.statement_id`
|
||||
* without being edited. Anything summing transactions without this fragment
|
||||
* still double-counts — that is the same gap that let reconciled rows into the
|
||||
* analytics routes in the first place.
|
||||
*
|
||||
* **3. The charge that never posted** (`voided_at`, migration 0040).
|
||||
*
|
||||
* A card notification is sent for an authorisation, and a released hold never
|
||||
* reaches a statement (txn 7951: a $1,040.41 Airbnb hold on a booking cancelled
|
||||
* the next day). There is no survivor to point at, so it gets its own column —
|
||||
* and it lives in this fragment for the same reason case 2 does.
|
||||
*/
|
||||
export const EXCLUDE_RECONCILED_SOURCE = `NOT (t.statement_id IS NULL AND t.reconciled_with_id IS NOT NULL)
|
||||
AND t.superseded_by_id IS NULL`;
|
||||
AND t.superseded_by_id IS NULL
|
||||
AND t.voided_at IS NULL`;
|
||||
|
||||
/**
|
||||
* The currency `t.amount` is actually denominated in.
|
||||
@@ -89,10 +97,17 @@ export const EXCLUDE_RECONCILED_SOURCE = `NOT (t.statement_id IS NULL AND t.reco
|
||||
* figure and `foreign_currency_code` names it (order-ingestion.ts leaves
|
||||
* `amount_aud` NULL rather than asserting an FX rate it does not have).
|
||||
*
|
||||
* - A wallet row also has no statement, but follows the STATEMENT convention:
|
||||
* `amount` = `amount_aud` = a provisional AUD figure, and the native figure
|
||||
* lives in `foreign_currency_amount` (wallet-ingestion.ts). `amount_aud`
|
||||
* being present is what says `amount` is already AUD. Without that arm, an
|
||||
* INR 524 tap stored as A$9.42 rendered as "INR 9.42" -- the AUD number
|
||||
* under the foreign label (txn 8162, 2026-10-03).
|
||||
*
|
||||
* Reading `s.currency` alone labels every foreign order row as AUD.
|
||||
* `s` must be the statements alias in scope.
|
||||
*/
|
||||
export const NATIVE_CURRENCY = `COALESCE(s.currency, t.foreign_currency_code, 'AUD')`;
|
||||
export const NATIVE_CURRENCY = `COALESCE(s.currency, CASE WHEN t.amount_aud IS NULL THEN t.foreign_currency_code END, 'AUD')`;
|
||||
|
||||
/**
|
||||
* True when a row's AUD value is unknown: it is denominated in something other
|
||||
|
||||
@@ -0,0 +1,275 @@
|
||||
import { queryRaw } from "@/lib/db";
|
||||
|
||||
/**
|
||||
* Upcoming annual charges, derived from payments alone.
|
||||
*
|
||||
* Independent of the email lane on purpose: a recurring CHARGE is visible in the
|
||||
* ledger whether or not anything was ever emailed about it, which is the only
|
||||
* way a card annual fee or a loan package fee shows up at all -- banks announce
|
||||
* those on the statement, not in a renewal notice.
|
||||
*
|
||||
* WHAT DOES NOT WORK, measured before this was written: "two charges about 365
|
||||
* days apart" is useless on its own. Ranked that way the top hit on this ledger
|
||||
* is a bakery with 202 such pairs, then a donation, Patreon, Coles -- merchants
|
||||
* used often enough that some pair is inevitably a year apart. Frequency is the
|
||||
* confound. Even "every gap is annual" still surfaces a Target visited twice
|
||||
* ($30, then $198) and a Specsavers ($24, then $224). Two visits a year apart
|
||||
* are not a fee.
|
||||
*
|
||||
* So there are two signals, and they are kept apart because they are not equally
|
||||
* trustworthy -- a guess must never render beside a certainty as if it were one:
|
||||
*
|
||||
* confirmed the bank called it a fee AND it has repeated on an annual cadence
|
||||
* likely the bank called it a fee, seen once -- the kind is certain, the
|
||||
* cadence is inferred (an annual fee charged once is still annual)
|
||||
* possible no fee wording; a merchant whose every gap is annual AND whose
|
||||
* amount is stable. Catches Amazon Prime, Uber One, VicRoads rego,
|
||||
* none of which carry fee wording. Also where false positives live.
|
||||
*/
|
||||
|
||||
export type Confidence = "confirmed" | "likely" | "possible";
|
||||
|
||||
/**
|
||||
* `expected_not_seen` is not a lesser version of `upcoming` -- it is the more
|
||||
* interesting row. The estimate has passed and no charge landed, which means one
|
||||
* of three things, and `account_days_stale` is what tells them apart:
|
||||
* the account stopped being imported (HSBC: ledger ends 2026-01-07, while the
|
||||
* mail shows statements to 2026-02-07)
|
||||
* the account genuinely closed
|
||||
* the charge landed under a different label (VicRoads paid 2026-03-16 as
|
||||
* "VICROADS ONLINE PAYM", not "VICROADS")
|
||||
* Reporting it as upcoming would be a lie; hiding it would lose the signal.
|
||||
*/
|
||||
export type ChargeStatus = "upcoming" | "expected_not_seen";
|
||||
|
||||
export type AnnualCharge = {
|
||||
label: string;
|
||||
status: ChargeStatus;
|
||||
bank_name: string | null;
|
||||
account_number: string | null;
|
||||
confidence: Confidence;
|
||||
observations: number;
|
||||
last_charged: string;
|
||||
last_amount: string;
|
||||
amount_low: string;
|
||||
amount_high: string;
|
||||
median_gap_days: number | null;
|
||||
next_due_estimate: string;
|
||||
/** Ledger liveness for the account: a fee that "stopped" may just be an
|
||||
* account that stopped being imported. Null for merchant-signal rows, which
|
||||
* are not tied to one account. */
|
||||
account_last_statement: string | null;
|
||||
account_days_stale: number | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* `transaction_type = 'fee'` is the universe, not a description match.
|
||||
*
|
||||
* The statement extractor already classifies fees structurally -- 131 rows
|
||||
* across the ledger -- and every card and loan annual fee on this ledger is
|
||||
* typed that way. An earlier version of this filtered `transaction_type =
|
||||
* 'debit'` and returned NONE of them, which is the whole reason this comment
|
||||
* exists: the fees are not debits.
|
||||
*
|
||||
* That universe is broader than "annual", though. It also holds ANZ's monthly
|
||||
* ACCOUNT SERVICING FEE, Zip's MONTHLY ACCOUNT SERVICE FEE, CommBank's MONTHLY
|
||||
* FEE, Citibank LATE PAYMENT FEE, HSBC OVERSEAS TRANSACTION FEE and every Wise
|
||||
* per-transfer charge. Cadence separates them: an annual fee repeats about a
|
||||
* year apart, the rest repeat monthly, on an event, or never (Wise labels each
|
||||
* transfer uniquely, so they can never group to two observations at all).
|
||||
*/
|
||||
const FEE_TYPE = "fee";
|
||||
|
||||
/**
|
||||
* Wording that establishes the KIND when there is only one observation.
|
||||
*
|
||||
* A single charge cannot demonstrate a cadence, but a bank calling something an
|
||||
* annual fee has already told us what it is -- Westpac's CARD FEE $295 and AMP's
|
||||
* Professional Pack Annual Fee each appear once and are certainly annual.
|
||||
* Deliberately narrow: bare "fee" would readmit every per-transaction charge.
|
||||
*/
|
||||
const ANNUAL_WORDING =
|
||||
"(annual|yearly)[[:space:]]*(card[[:space:]]*)?(fee|charge)" +
|
||||
"|card[[:space:]]+fee" +
|
||||
"|(package|membership|facility)[[:space:]]+fee";
|
||||
|
||||
/**
|
||||
* Resolve every statement to a canonical account identity, following
|
||||
* account_links forward. One card can appear under two bank names and two
|
||||
* numbers after an issuer change (Citi Prestige -> NAB MyCard); without this the
|
||||
* $700 fee charged 2024-12-13 and 2025-11-13 reads as two dead one-offs.
|
||||
*
|
||||
* Recursive because the table permits a chain (A->B->C); `uq_account_links_from`
|
||||
* forbids a fork, and `account_links_no_self` a self-link, so the walk
|
||||
* terminates. The depth guard is belt-and-braces against a cycle someone inserts
|
||||
* by hand (A->B, B->A), which no constraint can catch.
|
||||
*/
|
||||
const RESOLVED_ACCOUNTS = `
|
||||
WITH RECURSIVE chain AS (
|
||||
SELECT s.id AS statement_id, s.bank_name, s.account_number,
|
||||
s.bank_name AS cur_bank, s.account_number_key AS cur_key, 0 AS depth
|
||||
FROM statements s
|
||||
UNION ALL
|
||||
SELECT c.statement_id, c.bank_name, c.account_number,
|
||||
l.to_bank_name, l.to_account_key, c.depth + 1
|
||||
FROM chain c
|
||||
JOIN account_links l
|
||||
ON l.from_bank_name = c.cur_bank AND l.from_account_key = c.cur_key
|
||||
WHERE c.depth < 10
|
||||
),
|
||||
resolved AS (
|
||||
SELECT DISTINCT ON (statement_id)
|
||||
statement_id, bank_name, account_number, cur_bank AS acct_bank, cur_key AS acct_key
|
||||
FROM chain ORDER BY statement_id, depth DESC
|
||||
)`;
|
||||
|
||||
export async function getUpcomingAnnualCharges(ownerId: number): Promise<AnnualCharge[]> {
|
||||
// Signal 1 -- the bank's own fee wording, grouped by resolved account identity
|
||||
// and by the fee's wording so two different fees on one card stay apart
|
||||
// (ANZ charges "ANNUAL FEE" $120 and "ADDITIONAL CARD FEE" $10 on the same
|
||||
// statement, and they are not the same obligation).
|
||||
const feeRows = await queryRaw<AnnualCharge>(
|
||||
`${RESOLVED_ACCOUNTS},
|
||||
fees AS (
|
||||
SELECT r.acct_bank, r.acct_key, r.bank_name, r.account_number,
|
||||
upper(trim(regexp_replace(t.description, '[[:space:]]+', ' ', 'g'))) AS label,
|
||||
t.transaction_date, t.amount
|
||||
FROM transactions t
|
||||
JOIN resolved r ON r.statement_id = t.statement_id
|
||||
JOIN statements s ON s.id = t.statement_id
|
||||
WHERE t.superseded_by_id IS NULL AND t.reconciled_with_id IS NULL
|
||||
AND t.amount > 0
|
||||
AND COALESCE(t.owner_id, s.owner_id) = $1
|
||||
-- Structural first, wording only as a widening: a bank that types a
|
||||
-- fee as a plain debit still gets picked up if it names it annual.
|
||||
AND (t.transaction_type = $3 OR t.description ~* $2)
|
||||
),
|
||||
gapped AS (
|
||||
SELECT f.*,
|
||||
transaction_date - lag(transaction_date)
|
||||
OVER (PARTITION BY acct_key, label ORDER BY transaction_date) AS gap
|
||||
FROM fees f
|
||||
),
|
||||
agg AS (
|
||||
SELECT acct_key, acct_bank, label,
|
||||
count(*)::int AS observations,
|
||||
max(transaction_date) AS last_charged,
|
||||
min(amount) AS amount_low, max(amount) AS amount_high,
|
||||
percentile_cont(0.5) WITHIN GROUP (ORDER BY gap)
|
||||
FILTER (WHERE gap IS NOT NULL) AS median_gap,
|
||||
count(*) FILTER (WHERE gap IS NOT NULL) AS gaps,
|
||||
count(*) FILTER (WHERE gap BETWEEN 300 AND 430) AS annual_gaps,
|
||||
(array_agg(bank_name ORDER BY transaction_date DESC))[1] AS bank_name,
|
||||
(array_agg(account_number ORDER BY transaction_date DESC))[1] AS account_number,
|
||||
(array_agg(amount ORDER BY transaction_date DESC))[1] AS last_amount
|
||||
FROM gapped GROUP BY acct_key, acct_bank, label
|
||||
),
|
||||
liveness AS (
|
||||
SELECT r.acct_key, max(s.billing_end_date) AS last_statement
|
||||
FROM statements s JOIN resolved r ON r.statement_id = s.id
|
||||
GROUP BY r.acct_key
|
||||
)
|
||||
SELECT a.label,
|
||||
-- The account's CURRENT identity, not the bank that happened to
|
||||
-- print the last fee. That $700 fee last appeared under "Citibank"
|
||||
-- in Nov 2025, but the account is NAB/MyCard now and the next one
|
||||
-- will come from there; naming the dead brand would send you to the
|
||||
-- wrong bank.
|
||||
a.acct_bank AS bank_name, a.account_number,
|
||||
CASE WHEN (a.last_charged + COALESCE(round(a.median_gap)::int, 365)) >= CURRENT_DATE
|
||||
THEN 'upcoming' ELSE 'expected_not_seen' END AS status,
|
||||
-- One observation cannot prove a cadence, but the bank naming it an
|
||||
-- annual fee already tells us the kind. Repeated on an annual gap it
|
||||
-- is confirmed; seen once it is likely and says so.
|
||||
CASE WHEN a.gaps > 0 AND a.annual_gaps = a.gaps THEN 'confirmed'
|
||||
ELSE 'likely' END AS confidence,
|
||||
a.observations,
|
||||
a.last_charged::text AS last_charged,
|
||||
a.last_amount::text AS last_amount,
|
||||
a.amount_low::text AS amount_low,
|
||||
a.amount_high::text AS amount_high,
|
||||
round(a.median_gap)::int AS median_gap_days,
|
||||
(a.last_charged + COALESCE(round(a.median_gap)::int, 365))::text AS next_due_estimate,
|
||||
l.last_statement::text AS account_last_statement,
|
||||
(CURRENT_DATE - l.last_statement)::int AS account_days_stale
|
||||
FROM agg a LEFT JOIN liveness l ON l.acct_key = a.acct_key
|
||||
-- Two ways to earn a place, and nothing else gets one. Filtered here, not
|
||||
-- in the UI, so the API can never be read wrongly by a future caller.
|
||||
WHERE
|
||||
-- Repeated on an annual cadence. Excludes ANZ's monthly ACCOUNT
|
||||
-- SERVICING FEE and Zip's MONTHLY ACCOUNT SERVICE FEE, whose gaps are
|
||||
-- ~30 days, and Citibank's LATE PAYMENT FEE, which is event-driven.
|
||||
(a.gaps > 0 AND a.annual_gaps = a.gaps AND a.median_gap >= 300)
|
||||
-- Or seen once, but named as an annual/card/package fee. Westpac's CARD
|
||||
-- FEE and AMP's Professional Pack Annual Fee are each a single charge and
|
||||
-- unmistakably annual. A Wise per-transfer fee is also a single charge
|
||||
-- and says nothing of the sort, which is what keeps it out.
|
||||
OR (a.gaps = 0 AND a.label ~* $2)`,
|
||||
[ownerId, ANNUAL_WORDING, FEE_TYPE]
|
||||
);
|
||||
|
||||
// Signal 2 -- merchant-charged annual items with no fee wording. Requires
|
||||
// BOTH that every gap is annual AND that the amount is stable: VicRoads rego
|
||||
// rises about 3% a year ($825.23 -> $852.88 -> $875.48), so equality is too
|
||||
// strict, while no amount test at all readmits the twice-visited restaurant.
|
||||
// Keyed on merchant, never on card -- that rego was paid on three different
|
||||
// banks, and a card-keyed view would see three unrelated one-offs.
|
||||
const merchantRows = await queryRaw<AnnualCharge>(
|
||||
`WITH t AS (
|
||||
SELECT upper(trim(COALESCE(t.merchant_normalized, t.merchant_name, t.description))) AS label,
|
||||
t.transaction_date, t.amount
|
||||
FROM transactions t
|
||||
LEFT JOIN statements s ON s.id = t.statement_id
|
||||
WHERE t.superseded_by_id IS NULL AND t.reconciled_with_id IS NULL
|
||||
AND t.voided_at IS NULL
|
||||
AND t.transaction_type = 'debit' AND t.amount > 0
|
||||
AND COALESCE(t.owner_id, s.owner_id) = $1
|
||||
AND t.transaction_type = 'debit' AND t.description !~* $2
|
||||
),
|
||||
gapped AS (
|
||||
SELECT t.*, transaction_date - lag(transaction_date)
|
||||
OVER (PARTITION BY label ORDER BY transaction_date) AS gap
|
||||
FROM t
|
||||
),
|
||||
agg AS (
|
||||
SELECT label, count(*)::int AS observations,
|
||||
max(transaction_date) AS last_charged,
|
||||
min(amount) AS amount_low, max(amount) AS amount_high,
|
||||
percentile_cont(0.5) WITHIN GROUP (ORDER BY gap)
|
||||
FILTER (WHERE gap IS NOT NULL) AS median_gap,
|
||||
count(*) FILTER (WHERE gap IS NOT NULL) AS gaps,
|
||||
count(*) FILTER (WHERE gap BETWEEN 300 AND 430) AS annual_gaps,
|
||||
(array_agg(amount ORDER BY transaction_date DESC))[1] AS last_amount
|
||||
FROM gapped GROUP BY label
|
||||
)
|
||||
SELECT label, NULL::text AS bank_name, NULL::text AS account_number,
|
||||
CASE WHEN (last_charged + round(median_gap)::int) >= CURRENT_DATE
|
||||
THEN 'upcoming' ELSE 'expected_not_seen' END AS status,
|
||||
'possible' AS confidence, observations,
|
||||
last_charged::text AS last_charged,
|
||||
last_amount::text AS last_amount,
|
||||
amount_low::text AS amount_low,
|
||||
amount_high::text AS amount_high,
|
||||
round(median_gap)::int AS median_gap_days,
|
||||
(last_charged + round(median_gap)::int)::text AS next_due_estimate,
|
||||
NULL::text AS account_last_statement,
|
||||
NULL::int AS account_days_stale
|
||||
FROM agg
|
||||
WHERE gaps >= 1 AND annual_gaps = gaps
|
||||
-- Amount stability is what separates a subscription from a restaurant.
|
||||
-- 20% absorbs CPI drift and an FX-priced renewal without admitting a
|
||||
-- $30-then-$198 pair.
|
||||
AND amount_high <= amount_low * 1.20
|
||||
-- Two observations is the floor for a cadence claim, and this lane has no
|
||||
-- wording to fall back on the way the fee lane does.
|
||||
AND observations >= 2`,
|
||||
[ownerId, ANNUAL_WORDING]
|
||||
);
|
||||
|
||||
// Upcoming first and soonest-first within that, so the thing you can still
|
||||
// act on is at the top; the expected-but-missing rows follow as a review list.
|
||||
const rank = (c: AnnualCharge) => (c.status === "upcoming" ? 0 : 1);
|
||||
return [...feeRows, ...merchantRows].sort(
|
||||
(a, b) => rank(a) - rank(b) || a.next_due_estimate.localeCompare(b.next_due_estimate)
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,235 @@
|
||||
import { queryRaw, type Queryable } from "@/lib/db";
|
||||
import { merchantMatches } from "@/lib/intake-rules";
|
||||
|
||||
/**
|
||||
* Unambiguous statement matches for statementless ("manual") transactions.
|
||||
*
|
||||
* WHY THIS EXISTS
|
||||
* ---------------
|
||||
* The reconcile queue is worked by hand and is already ~177 deep. The wallet
|
||||
* lane writes a transaction for every tap (DECISIONS.md ING-11, 2026-08-22),
|
||||
* which at the observed rate adds ~63 rows a month, ~767 a year. Without an
|
||||
* automatic pass for the easy cases that queue stops being workable, and the
|
||||
* tempting alternative -- filtering the new rows back out of it -- is
|
||||
* explicitly ruled out by the `awaitsStatementLine()` removal note in
|
||||
* queries.ts: "an imported row that has no statement belongs in the queue;
|
||||
* volume is solved upstream, never downstream by hiding what is there."
|
||||
*
|
||||
* WHAT COUNTS AS UNAMBIGUOUS
|
||||
* --------------------------
|
||||
* Deliberately much stricter than the queue's own candidate ranking, which
|
||||
* offers the top 5 within +/-1% and +/-3 days for a human to choose between.
|
||||
* A machine gets one shot and no judgement, so:
|
||||
*
|
||||
* - EXACT amount equality, not the queue's +/-1% band. A near-miss is exactly
|
||||
* the case a person should look at.
|
||||
* - Same owner, and the statement row must not already be reconciled against.
|
||||
* - UNIQUE BOTH WAYS: the manual row must have exactly one candidate, AND
|
||||
* that candidate must be claimed by exactly one manual row. This is the
|
||||
* order_transaction_bridge rule, and it is what stops two identical $13.50
|
||||
* bakery taps in one window from being paired arbitrarily.
|
||||
*
|
||||
* Foreign rows match on the NATIVE pair (foreign_currency_amount +
|
||||
* foreign_currency_code), never on `amount`: `amount` is the AUD the bank
|
||||
* charged after applying its own rate, so a wallet row cannot predict it, while
|
||||
* the native figure is exact on both sides. Same reasoning as
|
||||
* order_transaction_bridge's FX path.
|
||||
*
|
||||
* This function only PROPOSES. Applying a pair goes through the same merge the
|
||||
* manual route uses, so overrides/tags/splits/metadata move identically.
|
||||
*/
|
||||
|
||||
export type AutoMatch = {
|
||||
manual_id: number;
|
||||
statement_tx_id: number;
|
||||
amount: string;
|
||||
manual_date: string;
|
||||
statement_date: string;
|
||||
description: string;
|
||||
basis: "aud" | "native";
|
||||
};
|
||||
|
||||
/** Days either side of the manual row's date to consider. Authorisation and
|
||||
* settlement are rarely the same day; 3 matches the queue's own window and
|
||||
* the wallet reconciliation check's default. */
|
||||
export const AUTO_MATCH_WINDOW_DAYS = 3;
|
||||
|
||||
export async function findAutoMatches(
|
||||
ownerId: number,
|
||||
windowDays: number = AUTO_MATCH_WINDOW_DAYS
|
||||
): Promise<AutoMatch[]> {
|
||||
return queryRaw<AutoMatch>(
|
||||
`WITH pairs AS (
|
||||
SELECT m.id AS manual_id,
|
||||
t.id AS statement_tx_id,
|
||||
t.amount::text AS amount,
|
||||
m.transaction_date::text AS manual_date,
|
||||
t.transaction_date::text AS statement_date,
|
||||
t.description AS description,
|
||||
CASE WHEN m.foreign_currency_code IS NOT NULL
|
||||
AND m.foreign_currency_amount IS NOT NULL
|
||||
THEN 'native' ELSE 'aud' END AS basis
|
||||
FROM transactions m
|
||||
JOIN transactions t
|
||||
ON t.statement_id IS NOT NULL
|
||||
AND t.transaction_type = 'debit'
|
||||
AND t.transaction_date BETWEEN m.transaction_date - $2::int
|
||||
AND m.transaction_date + $2::int
|
||||
AND (
|
||||
-- Foreign: compare what both sides actually recorded natively.
|
||||
(m.foreign_currency_code IS NOT NULL
|
||||
AND m.foreign_currency_amount IS NOT NULL
|
||||
AND t.foreign_currency_code = m.foreign_currency_code
|
||||
AND t.foreign_currency_amount = m.foreign_currency_amount)
|
||||
OR -- AUD side: exact equality, never a tolerance band. The
|
||||
-- statement row MAY be foreign here, and that is not a mistake:
|
||||
-- all 48 reconciliations a human has made on this ledger are
|
||||
-- exactly this shape -- a manual row carrying no currency,
|
||||
-- matched on exact AUD against a statement row that does carry
|
||||
-- one (EUR, CHF). Overseas receipts and orders record what was
|
||||
-- charged in AUD; only the wallet lane records natively, which
|
||||
-- is why the branch above exists. Requiring both sides to be
|
||||
-- non-foreign was tried and rejected: it matched 0 of those 48.
|
||||
-- The coincidence risk (a domestic tap whose amount happens to
|
||||
-- equal a foreign charge's AUD figure) is already covered by
|
||||
-- unique-both-ways, which refuses as soon as a second candidate
|
||||
-- exists.
|
||||
(m.foreign_currency_code IS NULL
|
||||
AND t.amount = m.amount)
|
||||
)
|
||||
JOIN statements s ON s.id = t.statement_id
|
||||
WHERE m.statement_id IS NULL
|
||||
AND m.owner_id = $1
|
||||
AND m.reconciled_with_id IS NULL
|
||||
AND m.superseded_by_id IS NULL
|
||||
AND m.voided_at IS NULL
|
||||
AND (m.payment_method IS NULL OR m.payment_method NOT IN ('cash', 'credits'))
|
||||
AND t.superseded_by_id IS NULL
|
||||
AND COALESCE(t.owner_id, s.owner_id) = $1
|
||||
-- The statement row must be unclaimed.
|
||||
AND NOT EXISTS (SELECT 1 FROM transactions mt WHERE mt.reconciled_with_id = t.id)
|
||||
)
|
||||
SELECT manual_id, statement_tx_id, amount, manual_date, statement_date, description, basis
|
||||
FROM pairs p
|
||||
WHERE (SELECT count(*) FROM pairs q WHERE q.manual_id = p.manual_id) = 1
|
||||
AND (SELECT count(*) FROM pairs q WHERE q.statement_tx_id = p.statement_tx_id) = 1
|
||||
ORDER BY manual_id`,
|
||||
[ownerId, windowDays]
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* An existing statementless transaction that a newly-arriving receipt or order
|
||||
* describes -- i.e. the row the wallet lane already created for this purchase.
|
||||
*
|
||||
* WHY: signals for one purchase arrive in sequence, not in competition. The
|
||||
* wallet notification lands within seconds of the tap; the receipt is shared
|
||||
* hours or days later. Measured on txn 6858: the wallet event arrived 35h25m
|
||||
* before anything created a transaction, and pantry then created one from
|
||||
* scratch -- so the same Woolworths shop would have had two rows. The receipt
|
||||
* must adopt the row that already exists and enrich it.
|
||||
*
|
||||
* Deliberately narrow, for the same reason findAutoMatches() is: exact amount,
|
||||
* a few days either side, unclaimed, and UNIQUE. Two candidates means two
|
||||
* plausible purchases, and inventing a link between the wrong pair moves one
|
||||
* shop's line items onto another shop's row. Returning null is always safe --
|
||||
* the caller falls back to creating its own row, which is today's behaviour.
|
||||
*/
|
||||
/**
|
||||
* The row a newly-arriving WALLET notification describes -- the mirror of
|
||||
* findAdoptableTransaction, for the case where the signals arrived in the other
|
||||
* order.
|
||||
*
|
||||
* WHY: the sequencing this estate is built on -- wallet first, receipt second,
|
||||
* statement last -- is a tendency, not a guarantee. The wallet lane can be
|
||||
* parked (its first 21 rows were backfilled days late), the interpreter runs
|
||||
* every 15 minutes, the phone can be offline. Whenever a receipt reaches the
|
||||
* ledger before the notification for the same tap, the wallet writer used to
|
||||
* insert unconditionally: measured on the 2026-08-18 Woolworths shop, txn 6858
|
||||
* (pantry, 23 items) and txn 6927 (wallet) were the same $103.30 purchase, and
|
||||
* the second sat in the reconcile queue waiting for a statement line that the
|
||||
* first had already claimed.
|
||||
*
|
||||
* Deliberately NOT the same predicate as findAdoptableTransaction, in the one
|
||||
* way that matters: that one refuses a row which already carries line items,
|
||||
* because for a receipt such a row is a DIFFERENT purchase that happens to
|
||||
* share an amount. Here it is the opposite -- an itemised row is the receipt we
|
||||
* are trying to fold into, and refusing it is the bug.
|
||||
*
|
||||
* Losing that guard means the merchant has to supply it. A wallet row adopts
|
||||
* only when the merchant agrees (the same whole-word containment the intake
|
||||
* matcher uses: "Woolworths" vs "WOOLWORTHS 3345"), because amount-and-date
|
||||
* alone would let a notification silently disappear into an unrelated purchase
|
||||
* of the same value -- and a wallet row that is never written is a tap with no
|
||||
* trace, which is worse than a visible duplicate. When in doubt, return null:
|
||||
* the caller then writes its own row, which is the pre-existing behaviour.
|
||||
*/
|
||||
export async function findWalletAdoptableTransaction(
|
||||
ownerId: number,
|
||||
amount: number,
|
||||
date: string,
|
||||
merchant: string | null,
|
||||
windowDays: number = AUTO_MATCH_WINDOW_DAYS,
|
||||
// wallet-ingestion calls this while holding its advisory lock, and must pass
|
||||
// that transaction's connection: borrowing a second pooled one per request
|
||||
// is how a burst of concurrent posts exhausts the pool and deadlocks.
|
||||
db: Pick<Queryable, "queryRaw"> = { queryRaw }
|
||||
): Promise<{ id: number; metadata_id: number | null } | null> {
|
||||
// No usable merchant, no adoption. "Card purchase" is wallet-ingestion's own
|
||||
// fallback for a notification that named no merchant at all.
|
||||
if (!merchant || merchant === "Card purchase") return null;
|
||||
const rows = await db.queryRaw<{ id: number; metadata_id: number | null; merchant: string | null }>(
|
||||
`SELECT t.id, em.id AS metadata_id,
|
||||
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant
|
||||
FROM transactions t
|
||||
LEFT JOIN expense_metadata em ON em.transaction_id = t.id
|
||||
WHERE t.statement_id IS NULL
|
||||
AND t.owner_id = $1
|
||||
AND t.amount = $2::numeric
|
||||
AND t.transaction_date BETWEEN $3::date - $4::int AND $3::date + $4::int
|
||||
AND t.reconciled_with_id IS NULL
|
||||
AND t.superseded_by_id IS NULL
|
||||
-- A voided row is a charge that never posted; a real charge for the same
|
||||
-- amount (a rebooking after a released hold) must not fold into it.
|
||||
AND t.voided_at IS NULL
|
||||
-- A wallet row is the OTHER HALF of this tap, not a purchase to fold
|
||||
-- into; findPairedRow() merges those, with different rules.
|
||||
AND COALESCE(t.source, '') <> 'wallet'
|
||||
LIMIT 5`,
|
||||
[ownerId, amount, date, windowDays]
|
||||
);
|
||||
const hits = rows.filter((r) => merchantMatches(r.merchant, merchant));
|
||||
return hits.length === 1 ? { id: hits[0].id, metadata_id: hits[0].metadata_id } : null;
|
||||
}
|
||||
|
||||
export async function findAdoptableTransaction(
|
||||
ownerId: number,
|
||||
amount: number,
|
||||
date: string,
|
||||
windowDays: number = AUTO_MATCH_WINDOW_DAYS
|
||||
): Promise<number | null> {
|
||||
const rows = await queryRaw<{ id: number }>(
|
||||
`SELECT t.id
|
||||
FROM transactions t
|
||||
WHERE t.statement_id IS NULL
|
||||
AND t.owner_id = $1
|
||||
AND t.amount = $2::numeric
|
||||
AND t.transaction_date BETWEEN $3::date - $4::int AND $3::date + $4::int
|
||||
AND t.reconciled_with_id IS NULL
|
||||
AND t.superseded_by_id IS NULL
|
||||
-- A voided row is a charge that never posted; a real charge for the same
|
||||
-- amount (a rebooking after a released hold) must not fold into it.
|
||||
AND t.voided_at IS NULL
|
||||
-- Never adopt a row that another source already fully described; that
|
||||
-- is a different purchase that happens to share an amount.
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM expense_metadata em
|
||||
WHERE (em.transaction_id = t.id OR em.matched_transaction_id = t.id)
|
||||
AND jsonb_typeof(em.line_items) = 'array'
|
||||
AND jsonb_array_length(em.line_items) > 0
|
||||
)
|
||||
LIMIT 2`,
|
||||
[ownerId, amount, date, windowDays]
|
||||
);
|
||||
return rows.length === 1 ? rows[0].id : null;
|
||||
}
|
||||
@@ -20,3 +20,38 @@ export async function queryRow<T>(sql: string, params: unknown[] = []): Promise<
|
||||
const rows = await prisma.$queryRawUnsafe<T[]>(sql, ...params);
|
||||
return rows.length > 0 ? rows[0] : null;
|
||||
}
|
||||
|
||||
/** The same two calls, scoped to one connection inside one transaction. */
|
||||
export type Queryable = {
|
||||
queryRaw<T>(sql: string, params?: unknown[]): Promise<T[]>;
|
||||
queryRow<T>(sql: string, params?: unknown[]): Promise<T | null>;
|
||||
};
|
||||
|
||||
/**
|
||||
* Run several statements as one transaction, on one connection.
|
||||
*
|
||||
* The module-level `queryRaw`/`queryRow` are `$queryRawUnsafe` on a POOLED
|
||||
* client, so each call is its own implicit transaction on whichever connection
|
||||
* the pool hands out. That makes a check-then-write pair non-atomic, and it
|
||||
* makes `pg_advisory_xact_lock` useless outside this helper — the lock is
|
||||
* released the instant the statement that took it commits, which is before the
|
||||
* next statement runs. Anything that needs a lock to still be held on its next
|
||||
* query has to be in here.
|
||||
*/
|
||||
export async function withTransaction<T>(
|
||||
fn: (q: Queryable) => Promise<T>,
|
||||
// Prisma's defaults (maxWait 2s, timeout 5s) count time spent WAITING on an
|
||||
// advisory lock, so a caller that serialises a burst on one needs more.
|
||||
opts?: { maxWait?: number; timeout?: number }
|
||||
): Promise<T> {
|
||||
return prisma.$transaction(async (tx) => {
|
||||
const q: Queryable = {
|
||||
queryRaw: <R>(sql: string, params: unknown[] = []) => tx.$queryRawUnsafe<R[]>(sql, ...params),
|
||||
queryRow: async <R>(sql: string, params: unknown[] = []) => {
|
||||
const rows = await tx.$queryRawUnsafe<R[]>(sql, ...params);
|
||||
return rows.length > 0 ? rows[0] : null;
|
||||
},
|
||||
};
|
||||
return fn(q);
|
||||
}, opts);
|
||||
}
|
||||
|
||||
@@ -177,6 +177,40 @@ export function useUpdateTransaction() {
|
||||
});
|
||||
}
|
||||
|
||||
export function useMergeTransaction() {
|
||||
const qc = useQueryClient();
|
||||
return useMutation({
|
||||
mutationFn: async ({
|
||||
id,
|
||||
other_id,
|
||||
mode,
|
||||
}: {
|
||||
id: number;
|
||||
other_id: number;
|
||||
mode: "reconcile" | "supersede";
|
||||
}) => {
|
||||
const res = await fetch(`/api/transactions/${id}/merge`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ other_id, mode }),
|
||||
});
|
||||
if (!res.ok) {
|
||||
const body = await res.json().catch(() => ({}));
|
||||
throw new Error(body.error || "Failed to merge transaction");
|
||||
}
|
||||
return res.json();
|
||||
},
|
||||
onSuccess: () => {
|
||||
qc.invalidateQueries({ queryKey: ["transactions"] });
|
||||
qc.invalidateQueries({ queryKey: ["transaction"] });
|
||||
qc.invalidateQueries({ queryKey: ["splits"] });
|
||||
qc.invalidateQueries({ queryKey: ["analytics"] });
|
||||
qc.invalidateQueries({ queryKey: ["participant-balances"] });
|
||||
qc.invalidateQueries({ queryKey: ["shared-transactions"] });
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function useBulkAction() {
|
||||
const qc = useQueryClient();
|
||||
return useMutation({
|
||||
@@ -1128,6 +1162,27 @@ export function useReconcile() {
|
||||
});
|
||||
}
|
||||
|
||||
/** Mark a statementless row as a charge that never posted (migration 0040). */
|
||||
export function useVoidTransaction() {
|
||||
const qc = useQueryClient();
|
||||
return useMutation({
|
||||
mutationFn: async ({ id, reason }: { id: number; reason: string }) => {
|
||||
const res = await fetch(`/api/transactions/${id}/void`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ reason }),
|
||||
});
|
||||
if (!res.ok) throw new Error((await res.json()).error || "Void failed");
|
||||
return res.json();
|
||||
},
|
||||
onSuccess: () => {
|
||||
qc.invalidateQueries({ queryKey: ["transactions"] });
|
||||
qc.invalidateQueries({ queryKey: ["reconcile-pending"] });
|
||||
qc.invalidateQueries({ queryKey: ["analytics"] });
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function useMerchantTransactions(merchant: string | null) {
|
||||
return useQuery<{ transactions: MerchantTxnRow[] }>({
|
||||
queryKey: ["analytics", "merchant-txns", merchant],
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
import { loadFiles } from "@/lib/intake-files";
|
||||
import type { PantryForward } from "@/lib/intake-pantry";
|
||||
import type { IntakeSource } from "@/lib/intake-rules";
|
||||
import {
|
||||
createFromBatch,
|
||||
loadBatch,
|
||||
rememberKind,
|
||||
resolvedCard,
|
||||
routeFood,
|
||||
routeGrocery,
|
||||
type BatchRow,
|
||||
type RouteContext,
|
||||
} from "@/lib/intake-router";
|
||||
import { replaceLineItems, TransactionNotFoundError } from "@/lib/line-items";
|
||||
import { enrichedCard, errorCard, type Block } from "@/lib/intake-blocks";
|
||||
import { queryRaw } from "@/lib/db";
|
||||
|
||||
/**
|
||||
* The four intake taps (plan §11 C5), as one function the Slack route calls.
|
||||
*
|
||||
* Every tap acts on the persisted batch, never on the card's text. A tap on a
|
||||
* batch that is already resolved returns its current card rather than acting
|
||||
* twice — a pressed button that looked dead and got pressed again is how three
|
||||
* splits were lost on the order nudge, and the same shape applies here.
|
||||
*/
|
||||
|
||||
export const INTAKE_ACTIONS = ["intake_use_candidate", "intake_create", "intake_grocery", "intake_meal"] as const;
|
||||
export type IntakeActionId = (typeof INTAKE_ACTIONS)[number];
|
||||
|
||||
export function isIntakeAction(id: unknown): id is IntakeActionId {
|
||||
return typeof id === "string" && (INTAKE_ACTIONS as readonly string[]).includes(id);
|
||||
}
|
||||
|
||||
export type IntakeActionValue = { batch_id?: string; transaction_id?: number };
|
||||
|
||||
export function parseActionValue(raw: unknown): IntakeActionValue {
|
||||
if (typeof raw !== "string") return {};
|
||||
try {
|
||||
const v = JSON.parse(raw) as IntakeActionValue;
|
||||
return v && typeof v === "object" ? v : {};
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
const note = (text: string): { blocks: Block[]; text: string; replace: boolean } => ({
|
||||
text,
|
||||
blocks: [{ type: "section", text: { type: "mrkdwn", text } }],
|
||||
replace: false,
|
||||
});
|
||||
|
||||
const ctxFor = (batch: BatchRow): RouteContext | null =>
|
||||
batch.header
|
||||
? {
|
||||
batchId: batch.id,
|
||||
source: batch.source as IntakeSource,
|
||||
sharedBy: batch.shared_by,
|
||||
platform: batch.header.platform,
|
||||
header: batch.header,
|
||||
items: batch.items ?? [],
|
||||
currency: batch.header.currency,
|
||||
}
|
||||
: null;
|
||||
|
||||
export async function handleIntakeAction(
|
||||
actionId: IntakeActionId,
|
||||
value: IntakeActionValue,
|
||||
deps: { pantry: PantryForward }
|
||||
): Promise<{ blocks: Block[]; text: string; replace: boolean }> {
|
||||
if (!value.batch_id) return note("That button carried no batch id — I can't tell which upload it was.");
|
||||
const batch = await loadBatch(value.batch_id);
|
||||
if (!batch) return note("I can't find that upload any more.");
|
||||
if (batch.status === "enriched" || batch.status === "created" || batch.status === "forwarded_to_pantry") {
|
||||
return { ...resolvedCard(batch), replace: true };
|
||||
}
|
||||
const ctx = ctxFor(batch);
|
||||
if (!ctx) return note("That upload never got a readable header — share it again with the top of the screen.");
|
||||
|
||||
try {
|
||||
if (actionId === "intake_use_candidate") {
|
||||
if (!Number.isInteger(value.transaction_id)) return note("That button carried no transaction id.");
|
||||
const txnId = value.transaction_id as number;
|
||||
await replaceLineItems(txnId, ctx.items, { source: "intake", platform: ctx.platform, batchId: batch.id, currency: ctx.currency });
|
||||
await queryRaw(
|
||||
`UPDATE intake_batches SET status = 'enriched', kind = COALESCE(kind, 'food'), transaction_id = $2, resolved_at = now() WHERE id = $1::uuid`,
|
||||
[batch.id, txnId]
|
||||
);
|
||||
return {
|
||||
...enrichedCard({ batchId: batch.id, merchant: ctx.header.merchant, total: ctx.header.total, orderDate: ctx.header.order_date, platform: ctx.platform, itemCount: ctx.items.length, currency: ctx.currency, transactionId: txnId }),
|
||||
replace: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (actionId === "intake_create") {
|
||||
const kind = batch.kind === "grocery" ? "grocery" : "food";
|
||||
const r = await createFromBatch(batch, kind);
|
||||
return { blocks: r.slack_blocks ?? [], text: r.text, replace: true };
|
||||
}
|
||||
|
||||
if (actionId === "intake_meal") {
|
||||
await rememberKind(ctx.header.merchant, "food");
|
||||
const r = await routeFood(ctx);
|
||||
return { blocks: r.slack_blocks ?? [], text: r.text, replace: true };
|
||||
}
|
||||
|
||||
// intake_grocery
|
||||
await rememberKind(ctx.header.merchant, "grocery");
|
||||
const files = await loadFiles(batch.file_sha256);
|
||||
if (!files.length) {
|
||||
return note("I remembered that as groceries, but the original files are gone — share it again and it will go straight to the pantry board.");
|
||||
}
|
||||
const r = await routeGrocery(ctx, files, deps.pantry);
|
||||
return { blocks: r.slack_blocks ?? [], text: r.text, replace: r.status !== "error" };
|
||||
} catch (e) {
|
||||
if (e instanceof TransactionNotFoundError) return note("That transaction is no longer in the ledger.");
|
||||
const detail = e instanceof Error ? e.message : String(e);
|
||||
return { ...errorCard(detail), replace: false };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
import { platformLabel } from "@/lib/intake-rules";
|
||||
|
||||
/**
|
||||
* The intake reply as Block Kit (plan §11 C1/C5). Built here, never in n8n
|
||||
* expressions, so the button payloads the interactive handler round-trips are
|
||||
* produced by one piece of code and tested like one.
|
||||
*
|
||||
* Button `value`s are JSON `{batch_id, ...}` — the batch row is what a later
|
||||
* tap acts on. action_ids are the C5 contract: `intake_use_candidate`,
|
||||
* `intake_create`, `intake_grocery`, `intake_meal`.
|
||||
*/
|
||||
|
||||
export type Block = Record<string, unknown>;
|
||||
|
||||
export type Candidate = { transaction_id: number; description: string | null; transaction_date: string; amount: string | number; currency?: string | null };
|
||||
|
||||
export function financePublicUrl(): string {
|
||||
return (process.env.FINANCE_PUBLIC_URL || "https://finance.bosecamp.com").replace(/\/$/, "");
|
||||
}
|
||||
|
||||
/** The transactions page searches on `q`; the merchant is the most selective thing a card knows. */
|
||||
export function transactionLink(merchant: string | null): string {
|
||||
return `${financePublicUrl()}/transactions?q=${encodeURIComponent(merchant ?? "")}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Slack renders `&`, `<` and `>` as markup in mrkdwn (and in the `text`
|
||||
* fallback). Merchant names, transaction descriptions and model output are
|
||||
* none of ours — `Ben & Jerry's` must not arrive as `Ben & Jerry's`, and a
|
||||
* description containing `<http://…|click>` must not become a link the card
|
||||
* never meant to show. Applied to interpolated values only; our own literal
|
||||
* `*bold*` and `:emoji:` stay markup.
|
||||
*/
|
||||
export function esc(value: string | null | undefined): string {
|
||||
return (value ?? "").replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
|
||||
}
|
||||
|
||||
/**
|
||||
* `$` is only right for AUD. An intake batch carries the currency the receipt
|
||||
* printed, and the order nudge already renders `USD 15.00` rather than
|
||||
* pretending a foreign order was dollars.
|
||||
*/
|
||||
const money = (n: number | string, currency?: string | null) => {
|
||||
const amount = Number(n).toFixed(2);
|
||||
// The currency comes off the model (`parseExtraction` accepts any string), so
|
||||
// an unescaped `<@U123>` here would be a mention on every card.
|
||||
return !currency || currency === "AUD" ? `$${amount}` : `${esc(currency)} ${amount}`;
|
||||
};
|
||||
const section = (text: string): Block => ({ type: "section", text: { type: "mrkdwn", text } });
|
||||
const button = (action_id: string, text: string, value: unknown, style?: "primary" | "danger"): Block => ({
|
||||
type: "button",
|
||||
action_id,
|
||||
text: { type: "plain_text", text, emoji: true },
|
||||
value: JSON.stringify(value),
|
||||
...(style ? { style } : {}),
|
||||
});
|
||||
|
||||
export type CardInput = {
|
||||
batchId: string;
|
||||
merchant: string | null;
|
||||
total: number | null;
|
||||
orderDate: string | null;
|
||||
platform: string | null;
|
||||
itemCount: number;
|
||||
currency?: string | null;
|
||||
};
|
||||
|
||||
export function enrichedCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
|
||||
const merchant = esc(c.merchant);
|
||||
const what = [c.total !== null ? money(c.total, c.currency) : null, esc(platformLabel(c.platform)) || merchant, c.platform && merchant ? `(${merchant})` : null]
|
||||
.filter(Boolean)
|
||||
.join(" ");
|
||||
const text = `Added ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} to your ${what} order ✓`;
|
||||
return {
|
||||
text,
|
||||
blocks: [
|
||||
{
|
||||
type: "section",
|
||||
text: { type: "mrkdwn", text: `:receipt: ${text}` },
|
||||
accessory: { type: "button", text: { type: "plain_text", text: "View" }, url: transactionLink(c.merchant), action_id: "intake_view" },
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
export function createdCard(c: CardInput & { transactionId: number }): { blocks: Block[]; text: string } {
|
||||
const text = `Created a ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant) || "new"} transaction with ${c.itemCount} item${c.itemCount === 1 ? "" : "s"} ✓`;
|
||||
return {
|
||||
text,
|
||||
blocks: [
|
||||
{
|
||||
type: "section",
|
||||
text: { type: "mrkdwn", text: `:receipt: ${text}` },
|
||||
accessory: { type: "button", text: { type: "plain_text", text: "View" }, url: transactionLink(c.merchant), action_id: "intake_view" },
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
export function candidatesCard(c: CardInput, candidates: Candidate[]): { blocks: Block[]; text: string } {
|
||||
const label = esc(platformLabel(c.platform));
|
||||
const head = `No matching ${label ? label + " " : ""}order for *${esc(c.merchant) || "this merchant"}* ${c.total !== null ? money(c.total, c.currency) : ""} on ${esc(c.orderDate) || "?"}.`;
|
||||
const text = `${head.replace(/\*/g, "")} ${candidates.length ? "Pick one below or create it." : "Create it?"}`;
|
||||
const blocks: Block[] = [section(`:mag: ${head}${candidates.length ? "\nIs it one of these?" : ""}`)];
|
||||
for (const cand of candidates) {
|
||||
blocks.push({
|
||||
type: "section",
|
||||
text: { type: "mrkdwn", text: `${esc(cand.transaction_date)} · ${esc(cand.description) || "—"} · ${money(cand.amount, cand.currency)}` },
|
||||
accessory: button("intake_use_candidate", "Use this", { batch_id: c.batchId, transaction_id: cand.transaction_id }),
|
||||
});
|
||||
}
|
||||
blocks.push({
|
||||
type: "actions",
|
||||
block_id: "intake_create",
|
||||
// plain_text disables *formatting*, not entity decoding: `&`, `<` and `>`
|
||||
// are control characters everywhere in a Slack payload, so the label is
|
||||
// escaped like any other value we did not write.
|
||||
elements: [button("intake_create", `Create new ${c.total !== null ? money(c.total, c.currency) + " " : ""}${esc(c.merchant)} transaction`.replace(/\s+/g, " ").trim(), { batch_id: c.batchId }, "primary")],
|
||||
});
|
||||
return { blocks, text };
|
||||
}
|
||||
|
||||
export function forwardedCard(
|
||||
c: CardInput & {
|
||||
captureId: number;
|
||||
enriching: boolean;
|
||||
/** Pantry already holds this exact receipt (same barcode tuple). */
|
||||
alreadyCaptured?: { captureId: number; confirmed: boolean } | null;
|
||||
/** Finance found more than one row this could be, so it picked none. */
|
||||
ambiguous?: boolean;
|
||||
}
|
||||
): { blocks: Block[]; text: string } {
|
||||
const what = `${esc(c.merchant) || "Grocery receipt"} ${c.total !== null ? money(c.total, c.currency) : ""}`.trim();
|
||||
const items = `${c.itemCount} item${c.itemCount === 1 ? "" : "s"}`;
|
||||
// The warning comes FIRST and the emoji changes, because this card is read at
|
||||
// a glance on a phone: a duplicate warning appended to a cheerful "→ pantry
|
||||
// board" line is a duplicate warning nobody reads. Confirming both copies
|
||||
// counts the same groceries twice — pantry's addItem accumulates — so the
|
||||
// card has to say which capture to dismiss, not merely that there are two.
|
||||
if (c.alreadyCaptured) {
|
||||
const state = c.alreadyCaptured.confirmed ? "already confirmed" : "still waiting to be confirmed";
|
||||
const text = `Already on the pantry board — ${what} is capture ${c.alreadyCaptured.captureId} (${state}). This upload is capture ${c.captureId}; dismiss it unless it replaces the first, because confirming both counts the same groceries twice.`.replace(/\s+/g, " ");
|
||||
return { text, blocks: [section(`:repeat: ${text}`)] };
|
||||
}
|
||||
const tail = c.ambiguous
|
||||
? "— finance found more than one row it could be, so it picked none; reconcile it there"
|
||||
: `— finance ${c.enriching ? "updates" : "books it"} on confirm`;
|
||||
const text = `${what} → pantry board (${items}) ${tail}`.replace(/\s+/g, " ");
|
||||
return { text, blocks: [section(`:shopping_trolley: ${text}`)] };
|
||||
}
|
||||
|
||||
/**
|
||||
* Names what is missing, because the advice differs. A delivery-app screenshot
|
||||
* that lost its header needs the top of the screen; a Woolworths eReceipt
|
||||
* SCREENSHOT has no date anywhere on it (seen 2026-08-22 — store, total, items,
|
||||
* payment, and not one date), while the app's own Share button sends the raw
|
||||
* receipt WITH the date. So for a dateless in-store receipt the right advice is
|
||||
* the Share button, or pantry's own upload — not "share more of the screen".
|
||||
*/
|
||||
export function needsHeaderCard(missing: string[] = ["total", "date"], source?: string | null): { blocks: Block[]; text: string } {
|
||||
const what = missing.length ? missing.join(" or ") : "total or date";
|
||||
const inStore = source === "coles" || source === "woolworths";
|
||||
const hint =
|
||||
missing.length === 1 && missing[0] === "date" && inStore
|
||||
? "an app screenshot carries no date — use the app's Share button instead, or upload the picture to pantry directly"
|
||||
: "share the part of the screen that shows it";
|
||||
const text = `I can see items but no ${esc(what)} — ${hint}, so I can match it.`;
|
||||
return { text, blocks: [section(`:warning: ${text}`)] };
|
||||
}
|
||||
|
||||
export function heldCard(reason: string, headline = "Items don't add up to the total — is a shot missing?"): { blocks: Block[]; text: string } {
|
||||
const text = `${headline} (${esc(reason)})`;
|
||||
return { text, blocks: [section(`:warning: ${text}`)] };
|
||||
}
|
||||
|
||||
export function askCard(c: CardInput): { blocks: Block[]; text: string } {
|
||||
const text = `${esc(c.merchant) || "This upload"} ${c.total !== null ? money(c.total, c.currency) : ""} — groceries or a meal?`.replace(/\s+/g, " ");
|
||||
return {
|
||||
text,
|
||||
blocks: [
|
||||
section(`:thinking_face: ${text}`),
|
||||
{
|
||||
type: "actions",
|
||||
block_id: "intake_ask",
|
||||
elements: [
|
||||
button("intake_grocery", "Groceries", { batch_id: c.batchId }),
|
||||
button("intake_meal", "A meal", { batch_id: c.batchId }),
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
export function duplicateCard(c: { merchant: string | null; total: number | null; transactionId: number | null; captureId: number | null; batchId: string | null; currency?: string | null }): { blocks: Block[]; text: string } {
|
||||
const where = c.transactionId ? `transaction ${c.transactionId}` : c.captureId ? `pantry capture ${c.captureId}` : c.batchId ? `batch ${esc(c.batchId)}` : "an earlier upload";
|
||||
// The header (merchant, date, total) IS the identity of a purchase here, so a
|
||||
// genuine SECOND identical purchase — two of the same coffee, one day, one shop
|
||||
// — reads as a re-share and is refused. That is the right default (a re-share is
|
||||
// far commoner than a repeat) but it is a dead end without this sentence: the
|
||||
// card must say what to do instead, or the ledger quietly loses the second one.
|
||||
const text = `Already handled — ${esc(c.merchant) || "this order"} ${c.total !== null ? money(c.total, c.currency) : ""} is ${where}. Nothing was changed — if this really is a second, separate purchase, add it in the app.`.replace(/\s+/g, " ");
|
||||
return { text, blocks: [section(`:repeat: ${text}`)] };
|
||||
}
|
||||
|
||||
export function unsupportedCard(detail?: string): { blocks: Block[]; text: string } {
|
||||
const text = `I can read receipts, statements and order screens — not this.${detail ? ` (${esc(detail)})` : ""}`;
|
||||
return { text, blocks: [section(`:no_entry_sign: ${text}`)] };
|
||||
}
|
||||
|
||||
export function errorCard(detail: string): { blocks: Block[]; text: string } {
|
||||
const text = `Couldn't process that upload: ${esc(detail)}`;
|
||||
return { text, blocks: [section(`:x: ${text}`)] };
|
||||
}
|
||||
@@ -0,0 +1,225 @@
|
||||
import { extractText, getDocumentProxy } from "unpdf";
|
||||
import { PROMPT_PLACEHOLDER_MERCHANTS, type ExtractedOrder, type ExtractedItem } from "@/lib/intake-rules";
|
||||
|
||||
/**
|
||||
* Extraction for the intake router: one joint model call per batch.
|
||||
*
|
||||
* The model EXTRACTS; it does not classify or route. The source tap and the
|
||||
* merchant pattern decide where the result goes (plan §2 — "a classifier is
|
||||
* confidently wrong exactly when it matters"). So this prompt asks for the
|
||||
* header and the lines, verbatim, and nothing else.
|
||||
*
|
||||
* One call with every part in order, never one per image: scrolling screenshots
|
||||
* overlap, and extracting each and concatenating double-counts the row both
|
||||
* shots contain and strands the lower shot with no header (plan §3, Codex #7).
|
||||
* Dedupe of the overlap is the model's job; the reconciliation check in
|
||||
* `intake-rules.ts` is what catches it getting that wrong.
|
||||
*/
|
||||
|
||||
export type InputPart =
|
||||
| { kind: "image"; mime: string; bytes: Buffer; name: string }
|
||||
| { kind: "text"; text: string; name: string };
|
||||
|
||||
/**
|
||||
* Digital receipts are text, not pixels. Reading the text beats photographing
|
||||
* the screen: nothing is misread, and the model parses a page rather than an
|
||||
* image. Same approach as pantry-app's `pdf-text.ts`.
|
||||
*/
|
||||
export async function extractPdfText(bytes: Buffer): Promise<string> {
|
||||
const document = await getDocumentProxy(new Uint8Array(bytes));
|
||||
const { text } = await extractText(document, { mergePages: true });
|
||||
return (Array.isArray(text) ? text.join("\n") : text).trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* The merchant examples are FORMAT PLACEHOLDERS, never real shops
|
||||
* (`PROMPT_PLACEHOLDER_MERCHANTS`). Given an input with nothing in it, a model
|
||||
* still answers, and it answers out of its prompt: on 2026-08-22 the word
|
||||
* `Image` produced a complete Burgertory receipt because the prompt named
|
||||
* Burgertory. The "not a receipt" rule below and the placeholder names are the
|
||||
* two halves of closing that.
|
||||
*/
|
||||
export const ORDER_ITEM_PROMPT = `You are reading a purchase receipt: an online food or grocery order (Uber Eats, DoorDash, Coles, Woolworths, ...) or an in-store till receipt. It arrives as one or more screenshots in scroll order, as the text of a PDF, or as pasted text.
|
||||
|
||||
Return ONE JSON object with exactly these keys:
|
||||
{ "merchant", "order_date", "total", "subtotal", "currency", "items": [{ "quantity", "description", "unit_price", "line_total", "options" }], "adjustments": [{ "label", "amount" }] }
|
||||
|
||||
Rules:
|
||||
- If what you are given is NOT a receipt or an order — it is blank, a filename, a single word such as "Image", an app screen with no purchase on it, or anything else you cannot read a purchase off — return { "merchant": null, "order_date": null, "total": null, "subtotal": null, "currency": null, "items": [], "adjustments": [] }. Do not construct a plausible order. Nothing in these instructions is data: never return an example from this prompt as if it were printed on the input.
|
||||
- merchant: the store or restaurant name exactly as printed, including any branch in parentheses (format: "${PROMPT_PLACEHOLDER_MERCHANTS[0]}"). Never the delivery platform. null if not shown.
|
||||
- order_date: the order or purchase date printed, as YYYY-MM-DD. null if not shown.
|
||||
- total: the amount actually charged — the "Total" / amount paid — as a number. null if not shown.
|
||||
- subtotal: the printed item subtotal before fees and discounts, as a number, or null.
|
||||
- currency: ISO code if printed (default "AUD").
|
||||
- items: every purchased line. Keep description VERBATIM as printed — do not rename, canonicalise, translate, shorten or summarise it. A burger called "Michael" stays "Michael"; "Coles RSPCA Approved Chicken Thigh Cutlets approx. 1.1kg" stays exactly that. Put selected options and modifiers (e.g. "Milk Bun", "Chicken") and weighed or substitution notes (e.g. "Final 1.16 kg @ $11.00/kg", "Replaced Coles Beef Slow Cook Chuck Roast approx. 1.3kg") in options, as an array of strings, not in the description.
|
||||
- quantity: the number ordered (a number; 1 if a line shows none). unit_price: the price per unit as printed, or null. line_total: the money for the whole line as printed, or quantity × unit_price when only a unit price is shown; for weighed lines multiply the final weight by the per-kg price. null when nothing is shown.
|
||||
- Do not put fees, tips, delivery or service charges, discounts, promotions, savings, tax lines or payment lines in items. List those in adjustments instead, fees positive and discounts negative.
|
||||
- Multiple screenshots are consecutive scrolls of ONE order and overlap: each physical line appears exactly once in items, never twice.
|
||||
- Never invent a value. If something is not legible or not present, use null.`;
|
||||
|
||||
const NUM_OR_NULL = { anyOf: [{ type: "number" }, { type: "null" }] };
|
||||
const STR_OR_NULL = { anyOf: [{ type: "string" }, { type: "null" }] };
|
||||
|
||||
/** Strict json_schema: every property listed in `required`, nullability via anyOf. */
|
||||
export const EXTRACT_RESPONSE_FORMAT = {
|
||||
type: "json_schema",
|
||||
json_schema: {
|
||||
name: "order_items",
|
||||
strict: true,
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
required: ["merchant", "order_date", "total", "subtotal", "currency", "items", "adjustments"],
|
||||
properties: {
|
||||
merchant: STR_OR_NULL,
|
||||
order_date: STR_OR_NULL,
|
||||
total: NUM_OR_NULL,
|
||||
subtotal: NUM_OR_NULL,
|
||||
currency: STR_OR_NULL,
|
||||
items: {
|
||||
type: "array",
|
||||
items: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
required: ["quantity", "description", "unit_price", "line_total", "options"],
|
||||
properties: {
|
||||
quantity: NUM_OR_NULL,
|
||||
description: { type: "string" },
|
||||
unit_price: NUM_OR_NULL,
|
||||
line_total: NUM_OR_NULL,
|
||||
options: { type: "array", items: { type: "string" } },
|
||||
},
|
||||
},
|
||||
},
|
||||
adjustments: {
|
||||
type: "array",
|
||||
items: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
required: ["label", "amount"],
|
||||
properties: { label: { type: "string" }, amount: { type: "number" } },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
|
||||
/** The OpenAI-format message content: prompt first, then every part in the order shared. */
|
||||
export function buildMessageContent(parts: InputPart[]): unknown[] {
|
||||
const content: unknown[] = [{ type: "text", text: ORDER_ITEM_PROMPT }];
|
||||
parts.forEach((p, i) => {
|
||||
if (p.kind === "text") {
|
||||
content.push({ type: "text", text: `\n--- Input ${i + 1} of ${parts.length} (${p.name}) ---\n${p.text}` });
|
||||
} else {
|
||||
content.push({ type: "text", text: `\n--- Input ${i + 1} of ${parts.length}: image ${p.name} ---` });
|
||||
content.push({ type: "image_url", image_url: { url: `data:${p.mime || "image/jpeg"};base64,${p.bytes.toString("base64")}` } });
|
||||
}
|
||||
});
|
||||
return content;
|
||||
}
|
||||
|
||||
const num = (v: unknown): number | null => {
|
||||
if (typeof v === "number" && Number.isFinite(v)) return v;
|
||||
if (typeof v === "string" && v.trim()) {
|
||||
const n = Number(v.replace(/[^0-9.-]/g, ""));
|
||||
return Number.isFinite(n) ? n : null;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
/** Tolerant of fences and a stray preamble; strict about the shape it hands on. */
|
||||
export function parseExtraction(content: string): ExtractedOrder {
|
||||
const unfenced = content.trim().replace(/^```(?:json)?\s*/i, "").replace(/\s*```$/, "");
|
||||
const start = unfenced.indexOf("{");
|
||||
const end = unfenced.lastIndexOf("}");
|
||||
const candidate = start >= 0 && end > start ? unfenced.slice(start, end + 1) : unfenced;
|
||||
const raw = JSON.parse(candidate) as Record<string, unknown>;
|
||||
const items: ExtractedItem[] = Array.isArray(raw.items)
|
||||
? (raw.items as Record<string, unknown>[])
|
||||
.filter((i) => i && typeof i.description === "string")
|
||||
.map((i) => ({
|
||||
quantity: num(i.quantity),
|
||||
description: String(i.description),
|
||||
unit_price: num(i.unit_price),
|
||||
line_total: num(i.line_total),
|
||||
options: Array.isArray(i.options) ? i.options.map(String) : [],
|
||||
}))
|
||||
: [];
|
||||
return {
|
||||
merchant: typeof raw.merchant === "string" ? raw.merchant : null,
|
||||
order_date: typeof raw.order_date === "string" ? raw.order_date : null,
|
||||
total: num(raw.total),
|
||||
subtotal: num(raw.subtotal),
|
||||
currency: typeof raw.currency === "string" && raw.currency ? raw.currency : "AUD",
|
||||
items,
|
||||
adjustments: Array.isArray(raw.adjustments)
|
||||
? (raw.adjustments as Record<string, unknown>[])
|
||||
.filter((a) => a && typeof a.label === "string" && num(a.amount) !== null)
|
||||
.map((a) => ({ label: String(a.label), amount: num(a.amount) as number }))
|
||||
: [],
|
||||
};
|
||||
}
|
||||
|
||||
export type Extractor = (parts: InputPart[]) => Promise<ExtractedOrder>;
|
||||
|
||||
export class ExtractionError extends Error {}
|
||||
|
||||
/**
|
||||
* LiteLLM, OpenAI-compatible, alias `cloud-budget` — the same alias pantry's
|
||||
* capture uses, so a model swap is one config line in `litellm-config.yaml`.
|
||||
* `temperature: 0`; `response_format` is the strict schema above.
|
||||
*/
|
||||
export function liteLLMExtractor(opts: {
|
||||
baseUrl?: string;
|
||||
key?: string;
|
||||
model?: string;
|
||||
fetchImpl?: typeof fetch;
|
||||
} = {}): Extractor {
|
||||
return async (parts) => {
|
||||
const baseUrl = (opts.baseUrl ?? process.env.LITELLM_BASE_URL ?? "").replace(/\/$/, "");
|
||||
const key = opts.key ?? process.env.LITELLM_KEY;
|
||||
if (!baseUrl || !key) throw new ExtractionError("LITELLM_BASE_URL / LITELLM_KEY are not configured");
|
||||
const doFetch = opts.fetchImpl ?? fetch;
|
||||
const response = await doFetch(`${baseUrl}/v1/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model: opts.model ?? "cloud-budget",
|
||||
temperature: 0,
|
||||
max_tokens: 16384,
|
||||
response_format: EXTRACT_RESPONSE_FORMAT,
|
||||
messages: [{ role: "user", content: buildMessageContent(parts) }],
|
||||
}),
|
||||
});
|
||||
if (!response.ok) throw new ExtractionError(`LiteLLM returned ${response.status}`);
|
||||
const raw = (await response.json()) as { choices?: { message?: { content?: string }; finish_reason?: string }[] };
|
||||
// The rule is: a finish_reason that is PRESENT must be `stop`. An absent
|
||||
// one is accepted, deliberately — providers differ on whether they send the
|
||||
// field, and failing every call on a provider that omits it would be a
|
||||
// certain outage in exchange for a case that also requires the model to
|
||||
// emit syntactically complete JSON. The harm a truncated reply does — a
|
||||
// partial item list replacing a good one — is closed on the write side
|
||||
// instead, by the guard in `routeFood`/`createFromBatch`.
|
||||
//
|
||||
// `length` is the common one: a truncated reply still parses once the JSON
|
||||
// happens to close, quietly dropping the tail of a long receipt. Any other
|
||||
// reason (`content_filter`, `tool_calls`) is a different reply shape that
|
||||
// the strict schema never constrained.
|
||||
const finish = raw.choices?.[0]?.finish_reason;
|
||||
if (finish && finish !== "stop") {
|
||||
throw new ExtractionError(
|
||||
finish === "length"
|
||||
? "The receipt was too long to read in one pass."
|
||||
: `The model stopped before it finished (${finish}).`
|
||||
);
|
||||
}
|
||||
const content = raw.choices?.[0]?.message?.content;
|
||||
if (!content) throw new ExtractionError("LiteLLM returned no content");
|
||||
try {
|
||||
return parseExtraction(content);
|
||||
} catch (e) {
|
||||
throw new ExtractionError(`could not parse extraction: ${e instanceof Error ? e.message : String(e)}`);
|
||||
}
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
/**
|
||||
* The attachment store: the original files of a batch, keyed by sha256.
|
||||
*
|
||||
* Two reasons it exists. A "Groceries or a meal?" answer arrives minutes after
|
||||
* the upload, in a fresh request carrying only the batch id, and "groceries"
|
||||
* means forwarding the ORIGINAL files to pantry — so they have to still be
|
||||
* here. And plan §8: "the source file is retrievable from the attachment store
|
||||
* by its sha256".
|
||||
*
|
||||
* Plain files under `INTAKE_FILE_DIR` (default `/app/data/intake`); a sidecar
|
||||
* `.json` keeps the name and MIME type. No database column holds bytes.
|
||||
*/
|
||||
|
||||
export type StoredFile = { name: string; type: string; bytes: Buffer; sha256: string };
|
||||
|
||||
export function intakeFileDir(): string {
|
||||
return process.env.INTAKE_FILE_DIR || "/app/data/intake";
|
||||
}
|
||||
|
||||
export function sha256Of(bytes: Buffer): string {
|
||||
return createHash("sha256").update(bytes).digest("hex");
|
||||
}
|
||||
|
||||
export async function storeFiles(files: StoredFile[]): Promise<void> {
|
||||
const dir = intakeFileDir();
|
||||
await mkdir(dir, { recursive: true });
|
||||
for (const f of files) {
|
||||
await writeFile(path.join(dir, `${f.sha256}.bin`), f.bytes);
|
||||
await writeFile(path.join(dir, `${f.sha256}.json`), JSON.stringify({ name: f.name, type: f.type }));
|
||||
}
|
||||
}
|
||||
|
||||
/** The files that are still on disk, in the order asked for; missing ones are skipped. */
|
||||
export async function loadFiles(shas: string[]): Promise<StoredFile[]> {
|
||||
const dir = intakeFileDir();
|
||||
const out: StoredFile[] = [];
|
||||
for (const sha of shas) {
|
||||
try {
|
||||
const bytes = await readFile(path.join(dir, `${sha}.bin`));
|
||||
const meta = JSON.parse(await readFile(path.join(dir, `${sha}.json`), "utf-8")) as { name?: string; type?: string };
|
||||
out.push({ name: meta.name ?? `${sha}.bin`, type: meta.type ?? "application/octet-stream", bytes, sha256: sha });
|
||||
} catch {
|
||||
/* gone — the caller decides what that means */
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
import type { StoredFile } from "@/lib/intake-files";
|
||||
|
||||
/**
|
||||
* finance → pantry hand-off for a grocery upload (plan §11 C3).
|
||||
*
|
||||
* The ORIGINAL files go across, not the extracted items: pantry owns grocery
|
||||
* extraction (canonical names, categories, staples) and its capture board is
|
||||
* the review surface. What finance adds is the routing decision — which
|
||||
* transaction the confirm should enrich, or that it should create one — carried
|
||||
* in the `intake` field so pantry's confirm seam (C4) can suppress the ING-10
|
||||
* create that would otherwise double-book an order the email already booked.
|
||||
*/
|
||||
|
||||
export type PantryForwardArgs = {
|
||||
files: StoredFile[];
|
||||
batchId: string;
|
||||
platform: string | null;
|
||||
sharedBy: string | null;
|
||||
financeTargetTransactionId: number | null;
|
||||
financeAction: "enrich" | "create";
|
||||
};
|
||||
|
||||
/**
|
||||
* `alreadyCaptured` is pantry telling us this exact receipt is already on its
|
||||
* board -- same store, register, receipt number and date, which is the tuple
|
||||
* both Coles and Woolworths encode in their barcode. Pantry deliberately still
|
||||
* ACCEPTS the second capture (the first may have been a bad crop being
|
||||
* replaced) and says so instead of refusing; dropping the field on the floor
|
||||
* turned "said so" into silence, and a duplicate that is not shown is a
|
||||
* duplicate that gets confirmed -- which double-counts the groceries, because
|
||||
* pantry's addItem accumulates.
|
||||
*/
|
||||
export type PantryForwardResult = {
|
||||
captureId: number;
|
||||
itemCount: number | null;
|
||||
alreadyCaptured: { captureId: number; confirmed: boolean } | null;
|
||||
};
|
||||
|
||||
export class PantryDuplicateError extends Error {
|
||||
constructor(
|
||||
public readonly captureId: number | null,
|
||||
public readonly reason: string
|
||||
) {
|
||||
super(reason);
|
||||
}
|
||||
}
|
||||
|
||||
export class PantryUnavailableError extends Error {}
|
||||
|
||||
export type PantryForward = (args: PantryForwardArgs) => Promise<PantryForwardResult>;
|
||||
|
||||
export function pantryForwarder(opts: { baseUrl?: string; token?: string; fetchImpl?: typeof fetch } = {}): PantryForward {
|
||||
return async (args) => {
|
||||
const baseUrl = (opts.baseUrl ?? process.env.PANTRY_BASE_URL ?? "").replace(/\/$/, "");
|
||||
const token = opts.token ?? process.env.RECEIPT_INGEST_TOKEN;
|
||||
if (!baseUrl || !token) throw new PantryUnavailableError("PANTRY_BASE_URL / RECEIPT_INGEST_TOKEN are not configured");
|
||||
const doFetch = opts.fetchImpl ?? fetch;
|
||||
|
||||
const form = new FormData();
|
||||
for (const f of args.files) {
|
||||
form.append("image", new Blob([new Uint8Array(f.bytes)], { type: f.type }), f.name);
|
||||
}
|
||||
form.append("source", "receipt");
|
||||
form.append(
|
||||
"intake",
|
||||
JSON.stringify({
|
||||
batch_id: args.batchId,
|
||||
platform: args.platform,
|
||||
origin: "intake",
|
||||
shared_by: args.sharedBy,
|
||||
finance_target_transaction_id: args.financeTargetTransactionId,
|
||||
finance_action: args.financeAction,
|
||||
})
|
||||
);
|
||||
|
||||
let response: Response;
|
||||
try {
|
||||
response = await doFetch(`${baseUrl}/api/capture`, { method: "POST", headers: { "x-ingest-token": token }, body: form });
|
||||
} catch (e) {
|
||||
throw new PantryUnavailableError(`pantry unreachable: ${e instanceof Error ? e.message : String(e)}`);
|
||||
}
|
||||
const body = (await response.json().catch(() => ({}))) as {
|
||||
captureId?: number; error?: string; items?: unknown[]; reason?: string;
|
||||
alreadyCaptured?: { captureId?: number; confirmed?: boolean } | null;
|
||||
};
|
||||
if (response.status === 409) {
|
||||
throw new PantryDuplicateError(typeof body.captureId === "number" ? body.captureId : null, body.reason ?? body.error ?? "already captured");
|
||||
}
|
||||
if (!response.ok) throw new PantryUnavailableError(`pantry returned ${response.status}: ${body.error ?? ""}`.trim());
|
||||
if (typeof body.captureId !== "number") throw new PantryUnavailableError("pantry returned no captureId");
|
||||
const prior = body.alreadyCaptured;
|
||||
return {
|
||||
captureId: body.captureId,
|
||||
itemCount: Array.isArray(body.items) ? body.items.length : null,
|
||||
alreadyCaptured:
|
||||
prior && typeof prior.captureId === "number" && prior.captureId !== body.captureId
|
||||
? { captureId: prior.captureId, confirmed: prior.confirmed === true }
|
||||
: null,
|
||||
};
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,731 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { queryRaw, queryRow, withTransaction } from "@/lib/db";
|
||||
import { EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY } from "@/lib/analytics-sql";
|
||||
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
|
||||
import { findAdoptableTransaction } from "@/lib/auto-reconcile";
|
||||
import {
|
||||
classifyInput,
|
||||
confabulationReason,
|
||||
merchantKey,
|
||||
quoteShared,
|
||||
textCouldBeReceipt,
|
||||
merchantMatches,
|
||||
platformForSource,
|
||||
reconcileItems,
|
||||
resolveMerchantKind,
|
||||
toLineItems,
|
||||
validateHeader,
|
||||
type ExtractedOrder,
|
||||
type IntakeSource,
|
||||
type MerchantKind,
|
||||
type OrderHeader,
|
||||
type StoredLineItem,
|
||||
} from "@/lib/intake-rules";
|
||||
import { extractPdfText, ExtractionError, type Extractor, type InputPart } from "@/lib/intake-extract";
|
||||
import { storeFiles, type StoredFile } from "@/lib/intake-files";
|
||||
import { PantryDuplicateError, PantryUnavailableError, type PantryForward } from "@/lib/intake-pantry";
|
||||
import { replaceLineItems } from "@/lib/line-items";
|
||||
import {
|
||||
askCard,
|
||||
candidatesCard,
|
||||
createdCard,
|
||||
duplicateCard,
|
||||
enrichedCard,
|
||||
errorCard,
|
||||
forwardedCard,
|
||||
heldCard,
|
||||
needsHeaderCard,
|
||||
unsupportedCard,
|
||||
type Block,
|
||||
type Candidate,
|
||||
type CardInput,
|
||||
} from "@/lib/intake-blocks";
|
||||
|
||||
/**
|
||||
* The intake router (plan §11 C1): one upload, every applicable step.
|
||||
*
|
||||
* Order of operations matters and is the plan's:
|
||||
* persist the batch → classify each file → ONE extraction call → header →
|
||||
* merchant kind → duplicate check → reconciliation → match → act.
|
||||
*
|
||||
* The batch row is written before anything else so that a Slack tap minutes
|
||||
* later has something to act on, and so a re-share is recognisable. Nothing
|
||||
* else is written until the header is whole: a half-read upload produces a
|
||||
* reply, not a row (plan §2).
|
||||
*/
|
||||
|
||||
export const MAX_FILES = 6;
|
||||
|
||||
export type IntakeKind = "grocery" | "food" | "ask" | "held" | "unsupported";
|
||||
export type IntakeStatus =
|
||||
| "received"
|
||||
| "enriched"
|
||||
| "created"
|
||||
| "candidates"
|
||||
| "forwarded_to_pantry"
|
||||
| "needs_header"
|
||||
| "duplicate"
|
||||
| "unsupported"
|
||||
| "ask"
|
||||
| "held"
|
||||
| "error";
|
||||
|
||||
export type IntakeResult = {
|
||||
kind: IntakeKind;
|
||||
status: IntakeStatus;
|
||||
batch_id: string;
|
||||
transaction_id?: number;
|
||||
capture_id?: number;
|
||||
duplicate_of?: { batch_id: string; transaction_id: number | null; capture_id: number | null };
|
||||
candidates?: Candidate[];
|
||||
header?: OrderHeader | null;
|
||||
items?: StoredLineItem[];
|
||||
slack_blocks: Block[] | null;
|
||||
text: string;
|
||||
http_status: number;
|
||||
};
|
||||
|
||||
export type IntakeDeps = { extractor: Extractor; pantry: PantryForward };
|
||||
|
||||
export type BatchRow = {
|
||||
id: string;
|
||||
source: IntakeSource;
|
||||
shared_by: string | null;
|
||||
file_sha256: string[];
|
||||
header: (OrderHeader & { platform: string | null; subtotal?: number | null; currency?: string | null }) | null;
|
||||
items: StoredLineItem[] | null;
|
||||
kind: string | null;
|
||||
status: IntakeStatus;
|
||||
transaction_id: number | null;
|
||||
capture_id: number | null;
|
||||
};
|
||||
|
||||
|
||||
// ─── batch persistence ──────────────────────────────────────────────────────
|
||||
|
||||
export async function loadBatch(id: string): Promise<BatchRow | null> {
|
||||
return queryRow<BatchRow>(
|
||||
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
|
||||
FROM intake_batches WHERE id = $1::uuid`,
|
||||
[id]
|
||||
);
|
||||
}
|
||||
|
||||
async function insertBatch(b: { id: string; source: IntakeSource; sharedBy: string | null; shas: string[] }) {
|
||||
await queryRaw(
|
||||
`INSERT INTO intake_batches (id, source, shared_by, file_sha256, status)
|
||||
VALUES ($1::uuid, $2, $3, $4::text[], 'received')`,
|
||||
[b.id, b.source, b.sharedBy, b.shas]
|
||||
);
|
||||
}
|
||||
|
||||
async function updateBatch(
|
||||
id: string,
|
||||
patch: {
|
||||
header?: unknown;
|
||||
items?: unknown;
|
||||
kind?: string | null;
|
||||
status: IntakeStatus;
|
||||
transaction_id?: number | null;
|
||||
capture_id?: number | null;
|
||||
}
|
||||
) {
|
||||
await queryRaw(
|
||||
`UPDATE intake_batches
|
||||
SET header = COALESCE($2::jsonb, header),
|
||||
items = COALESCE($3::jsonb, items),
|
||||
kind = COALESCE($4, kind),
|
||||
status = $5,
|
||||
transaction_id = COALESCE($6, transaction_id),
|
||||
capture_id = COALESCE($7, capture_id),
|
||||
resolved_at = CASE WHEN $5 IN ('enriched','created','forwarded_to_pantry','duplicate') THEN now() ELSE resolved_at END
|
||||
WHERE id = $1::uuid`,
|
||||
[
|
||||
id,
|
||||
patch.header === undefined ? null : JSON.stringify(patch.header),
|
||||
patch.items === undefined ? null : JSON.stringify(patch.items),
|
||||
patch.kind ?? null,
|
||||
patch.status,
|
||||
patch.transaction_id ?? null,
|
||||
patch.capture_id ?? null,
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
// ─── lookups ────────────────────────────────────────────────────────────────
|
||||
|
||||
export async function rememberedKind(merchant: string | null): Promise<"grocery" | "food" | null> {
|
||||
const key = merchantKey(merchant);
|
||||
if (!key) return null;
|
||||
const row = await queryRow<{ kind: "grocery" | "food" }>(`SELECT kind FROM intake_merchant_kinds WHERE merchant_key = $1`, [key]);
|
||||
return row?.kind ?? null;
|
||||
}
|
||||
|
||||
export async function rememberKind(merchant: string | null, kind: "grocery" | "food"): Promise<void> {
|
||||
const key = merchantKey(merchant);
|
||||
if (!key) return;
|
||||
await queryRaw(
|
||||
`INSERT INTO intake_merchant_kinds (merchant_key, kind) VALUES ($1, $2)
|
||||
ON CONFLICT (merchant_key) DO UPDATE SET kind = EXCLUDED.kind`,
|
||||
[key, kind]
|
||||
);
|
||||
}
|
||||
|
||||
/** The owner a batch's rows belong to: the sharer's participant, else the default. */
|
||||
export async function ownerForEmail(email: string | null): Promise<number> {
|
||||
if (!email) return DEFAULT_OWNER_ID;
|
||||
const row = await queryRow<{ id: number }>(`SELECT id FROM participants WHERE lower(email) = lower($1) LIMIT 1`, [email]);
|
||||
return row?.id ?? DEFAULT_OWNER_ID;
|
||||
}
|
||||
|
||||
/**
|
||||
* A batch already resolved from the same files, or from the same
|
||||
* (platform, merchant, date, total) header (plan §5, Codex #5). Only RESOLVED
|
||||
* batches count: a `needs_header` share followed by a fuller one must go
|
||||
* through, and a pending candidates card re-shared is harmless.
|
||||
*/
|
||||
export async function findDuplicateBatch(
|
||||
exceptId: string,
|
||||
shas: string[],
|
||||
header: (OrderHeader & { platform: string | null }) | null
|
||||
): Promise<BatchRow | null> {
|
||||
const bySha = await queryRow<BatchRow>(
|
||||
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
|
||||
FROM intake_batches
|
||||
WHERE id <> $1::uuid AND status IN ('enriched','created','forwarded_to_pantry')
|
||||
AND file_sha256 && $2::text[]
|
||||
ORDER BY created_at DESC LIMIT 1`,
|
||||
[exceptId, shas]
|
||||
);
|
||||
if (bySha) return bySha;
|
||||
if (!header || !header.merchant) return null;
|
||||
const rows = await queryRaw<BatchRow>(
|
||||
`SELECT id::text, source, shared_by, file_sha256, header, items, kind, status, transaction_id, capture_id
|
||||
FROM intake_batches
|
||||
WHERE id <> $1::uuid AND status IN ('enriched','created','forwarded_to_pantry')
|
||||
AND header->>'order_date' = $2
|
||||
AND (header->>'total')::numeric = $3::numeric
|
||||
AND COALESCE(header->>'platform', '') = COALESCE($4, '')
|
||||
ORDER BY created_at DESC`,
|
||||
[exceptId, header.order_date, header.total, header.platform]
|
||||
);
|
||||
return rows.find((r) => merchantMatches(r.header?.merchant, header.merchant)) ?? null;
|
||||
}
|
||||
|
||||
type MatchRow = {
|
||||
transaction_id: number;
|
||||
description: string | null;
|
||||
transaction_date: string;
|
||||
amount: string;
|
||||
merchant: string | null;
|
||||
platform: string | null;
|
||||
/** What `amount` is denominated in — a candidate row is not always dollars. */
|
||||
currency: string;
|
||||
/** The `expense_metadata` row the match resolved through — null when there is none. */
|
||||
metadata_id: number | null;
|
||||
/** Does that row already carry a receipt? */
|
||||
has_line_items: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* The match rule (plan §5, D2): `(platform, merchant, date, total)`, amount and
|
||||
* date EXACT, never widened silently. The merchant compares normalised, with
|
||||
* whole-word containment ("Coles" vs "Coles (Wyndham Vale)") — that is
|
||||
* spelling, not widening. A unique hit is a match; two hits are an ambiguity
|
||||
* and go to the candidates card like a miss.
|
||||
*
|
||||
* A NULL platform means "no platform", not "any platform". An in-store Coles
|
||||
* receipt carries none, and `$3 IS NULL OR em.platform = $3` let it match an
|
||||
* Uber Eats Coles order on the same day for the same money — and then overwrite
|
||||
* that order's items with the till receipt's. The in-store side matches rows
|
||||
* with no platform of their own; the delivery side still requires its own.
|
||||
*/
|
||||
export async function matchTransaction(header: OrderHeader, platform: string | null): Promise<{ match: MatchRow | null; ambiguous: boolean }> {
|
||||
if (!header.merchant) return { match: null, ambiguous: false };
|
||||
const rows = await queryRaw<MatchRow>(
|
||||
`SELECT DISTINCT ON (t.id)
|
||||
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
|
||||
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
|
||||
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency,
|
||||
COALESCE(jsonb_typeof(em.line_items) = 'array' AND jsonb_array_length(em.line_items) > 0, false) AS has_line_items
|
||||
FROM transactions t
|
||||
LEFT JOIN statements s ON s.id = t.statement_id
|
||||
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||
WHERE ${EXCLUDE_RECONCILED_SOURCE}
|
||||
AND t.transaction_date = $1::date
|
||||
AND t.amount = $2::numeric
|
||||
AND (($3::text IS NULL AND em.platform IS NULL) OR em.platform = $3)
|
||||
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
|
||||
[header.order_date, header.total, platform]
|
||||
);
|
||||
const hits = rows.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant));
|
||||
if (hits.length === 1) return { match: hits[0], ambiguous: false };
|
||||
return { match: null, ambiguous: hits.length > 1 };
|
||||
}
|
||||
|
||||
/**
|
||||
* Nearest candidates for the card: same merchant, ±3 days, any amount.
|
||||
*
|
||||
* Deliberately NOT filtered by platform. The card is a human choice, not a
|
||||
* decision — the whole reason it is shown is that the exact key missed, so the
|
||||
* platform is one of the things that may be wrong. A filter here narrows the
|
||||
* list at exactly the moment it needs to widen, and the row the user was
|
||||
* looking for is simply absent with nothing saying why.
|
||||
*/
|
||||
export async function findCandidates(header: OrderHeader): Promise<Candidate[]> {
|
||||
if (!header.merchant) return [];
|
||||
const rows = await queryRaw<MatchRow>(
|
||||
`SELECT DISTINCT ON (t.id)
|
||||
t.id AS transaction_id, t.description, t.transaction_date::text, t.amount::text,
|
||||
COALESCE(em.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) AS merchant,
|
||||
em.platform, em.id AS metadata_id, ${NATIVE_CURRENCY} AS currency, false AS has_line_items
|
||||
FROM transactions t
|
||||
LEFT JOIN statements s ON s.id = t.statement_id
|
||||
LEFT JOIN expense_metadata em ON em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||
WHERE ${EXCLUDE_RECONCILED_SOURCE}
|
||||
AND t.transaction_date BETWEEN $1::date - 3 AND $1::date + 3
|
||||
ORDER BY t.id, (em.transaction_id = t.id) DESC NULLS LAST`,
|
||||
[header.order_date]
|
||||
);
|
||||
return rows
|
||||
.filter((r) => merchantMatches(r.merchant, header.merchant) || merchantMatches(r.description, header.merchant))
|
||||
.sort((a, b) => {
|
||||
const da = Math.abs(Date.parse(a.transaction_date) - Date.parse(header.order_date));
|
||||
const db = Math.abs(Date.parse(b.transaction_date) - Date.parse(header.order_date));
|
||||
if (da !== db) return da - db;
|
||||
return Math.abs(Number(a.amount) - header.total) - Math.abs(Number(b.amount) - header.total);
|
||||
})
|
||||
.slice(0, 5)
|
||||
.map((r) => ({ transaction_id: r.transaction_id, description: r.description, transaction_date: r.transaction_date, amount: r.amount, currency: r.currency }));
|
||||
}
|
||||
|
||||
// ─── the two destinations ───────────────────────────────────────────────────
|
||||
|
||||
export type RouteContext = {
|
||||
batchId: string;
|
||||
source: IntakeSource;
|
||||
sharedBy: string | null;
|
||||
platform: string | null;
|
||||
header: OrderHeader;
|
||||
items: StoredLineItem[];
|
||||
currency?: string | null;
|
||||
};
|
||||
|
||||
const cardInput = (ctx: RouteContext): CardInput => ({
|
||||
batchId: ctx.batchId,
|
||||
merchant: ctx.header.merchant,
|
||||
total: ctx.header.total,
|
||||
orderDate: ctx.header.order_date,
|
||||
platform: ctx.platform,
|
||||
itemCount: ctx.items.length,
|
||||
currency: ctx.currency,
|
||||
});
|
||||
|
||||
const finish = (ctx: RouteContext, kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, extra: Partial<IntakeResult> = {}): IntakeResult => ({
|
||||
kind,
|
||||
status,
|
||||
batch_id: ctx.batchId,
|
||||
header: ctx.header,
|
||||
items: ctx.items,
|
||||
slack_blocks: card.blocks,
|
||||
text: card.text,
|
||||
http_status: 200,
|
||||
...extra,
|
||||
});
|
||||
|
||||
/**
|
||||
* Are these items unverifiable — no line carrying any money at all?
|
||||
*
|
||||
* That is `reconcileItems`' `basis: "none"`: pasted text with no prices, or an
|
||||
* extraction that read the names and none of the figures. The reconciliation
|
||||
* check passes it, correctly, because there is nothing to check. What must not
|
||||
* follow is replacing a receipt that IS itemised with one nothing verified.
|
||||
*/
|
||||
const unverifiable = (items: StoredLineItem[]) => items.every((i) => i.amount === null);
|
||||
|
||||
/**
|
||||
* The hold, shared by EVERY path that enriches a matched transaction.
|
||||
*
|
||||
* It lived only in `routeFood` and `createFromBatch`'s re-match walked straight
|
||||
* past it — the same upload destroyed the same receipt depending on which
|
||||
* button reached it. Returns null when there is nothing to protect.
|
||||
*/
|
||||
async function holdRatherThanReplace(ctx: RouteContext, match: MatchRow): Promise<IntakeResult | null> {
|
||||
if (!unverifiable(ctx.items) || !match.has_line_items) return null;
|
||||
await updateBatch(ctx.batchId, { status: "held", items: ctx.items });
|
||||
const card = heldCard(
|
||||
"this upload carries no prices, so nothing checked it against the total — and that order already has its items",
|
||||
"I left the existing items alone"
|
||||
);
|
||||
return finish(ctx, "held", "held", card, { transaction_id: match.transaction_id });
|
||||
}
|
||||
|
||||
/** Food: enrich the matched order's line_items, or show candidates. NEVER auto-creates. */
|
||||
export async function routeFood(ctx: RouteContext): Promise<IntakeResult> {
|
||||
const { match } = await matchTransaction(ctx.header, ctx.platform);
|
||||
if (match) {
|
||||
const held = await holdRatherThanReplace(ctx, match);
|
||||
if (held) return held;
|
||||
await replaceLineItems(match.transaction_id, ctx.items, { source: "intake", platform: ctx.platform, batchId: ctx.batchId, metadataId: match.metadata_id, currency: ctx.currency });
|
||||
await updateBatch(ctx.batchId, { kind: "food", status: "enriched", transaction_id: match.transaction_id, items: ctx.items });
|
||||
return finish(ctx, "food", "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
|
||||
}
|
||||
const candidates = await findCandidates(ctx.header);
|
||||
await updateBatch(ctx.batchId, { kind: "food", status: "candidates", items: ctx.items });
|
||||
return finish(ctx, "food", "candidates", candidatesCard(cardInput(ctx), candidates), { candidates });
|
||||
}
|
||||
|
||||
/** Grocery: the original files go to pantry's board; finance rides along on its confirm (C3/C4). */
|
||||
export async function routeGrocery(ctx: RouteContext, files: StoredFile[], pantry: PantryForward): Promise<IntakeResult> {
|
||||
// `ambiguous` is not a miss. Two ledger rows for one shop is precisely the
|
||||
// state that produced the 2026-08-18 Woolworths double-book, and forwarding
|
||||
// with financeAction 'create' as though nothing matched is what turns it into
|
||||
// a third row on confirm. The groceries still go to the board — they have to,
|
||||
// it is where they are worked — but the card must say finance could not pick.
|
||||
const { match, ambiguous } = await matchTransaction(ctx.header, ctx.platform);
|
||||
try {
|
||||
const r = await pantry({
|
||||
files,
|
||||
batchId: ctx.batchId,
|
||||
platform: ctx.platform,
|
||||
sharedBy: ctx.sharedBy,
|
||||
financeTargetTransactionId: match?.transaction_id ?? null,
|
||||
financeAction: match ? "enrich" : "create",
|
||||
});
|
||||
await updateBatch(ctx.batchId, { kind: "grocery", status: "forwarded_to_pantry", transaction_id: match?.transaction_id ?? null, capture_id: r.captureId, items: ctx.items });
|
||||
const card = forwardedCard({
|
||||
...cardInput(ctx),
|
||||
itemCount: r.itemCount ?? ctx.items.length,
|
||||
captureId: r.captureId,
|
||||
enriching: !!match,
|
||||
alreadyCaptured: r.alreadyCaptured,
|
||||
ambiguous,
|
||||
});
|
||||
return finish(ctx, "grocery", "forwarded_to_pantry", card, { capture_id: r.captureId, transaction_id: match?.transaction_id });
|
||||
} catch (e) {
|
||||
if (e instanceof PantryDuplicateError) {
|
||||
await updateBatch(ctx.batchId, { kind: "grocery", status: "duplicate", capture_id: e.captureId, items: ctx.items });
|
||||
return finish(ctx, "grocery", "duplicate", duplicateCard({ merchant: ctx.header.merchant, total: ctx.header.total, currency: ctx.currency, transactionId: match?.transaction_id ?? null, captureId: e.captureId, batchId: null }), {
|
||||
capture_id: e.captureId ?? undefined,
|
||||
});
|
||||
}
|
||||
const detail = e instanceof PantryUnavailableError ? e.message : e instanceof Error ? e.message : String(e);
|
||||
await updateBatch(ctx.batchId, { kind: "grocery", status: "error", items: ctx.items });
|
||||
return { ...finish(ctx, "grocery", "error", errorCard(detail)), http_status: 502 };
|
||||
}
|
||||
}
|
||||
|
||||
/** Create the transaction ING-10-style: manual, `statement_id NULL`, owner set — only ever on an explicit tap. */
|
||||
export async function createFromBatch(batch: BatchRow, kind: "grocery" | "food"): Promise<IntakeResult> {
|
||||
if (!batch.header) throw new Error("batch has no header");
|
||||
const header = batch.header;
|
||||
const items = batch.items ?? [];
|
||||
const ctx: RouteContext = {
|
||||
batchId: batch.id,
|
||||
source: batch.source,
|
||||
sharedBy: batch.shared_by,
|
||||
platform: header.platform,
|
||||
header,
|
||||
items,
|
||||
currency: header.currency,
|
||||
};
|
||||
|
||||
// The card was built minutes ago and the tap arrives now. In between, the
|
||||
// email/order bridge books orders on its own schedule — so re-run the match
|
||||
// rather than trusting the "no match" the card was drawn from. Creating a
|
||||
// second row for an order the ledger already holds is the one outcome this
|
||||
// whole slice exists to avoid.
|
||||
const { match } = await matchTransaction(header, header.platform);
|
||||
if (match) {
|
||||
const held = await holdRatherThanReplace(ctx, match);
|
||||
if (held) return held;
|
||||
await replaceLineItems(match.transaction_id, items, { source: "intake", platform: header.platform, batchId: batch.id, metadataId: match.metadata_id, currency: header.currency });
|
||||
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: match.transaction_id, items });
|
||||
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: match.transaction_id }), { transaction_id: match.transaction_id });
|
||||
}
|
||||
|
||||
// A sibling batch of the same order — the PDF after the screenshot — may have
|
||||
// been resolved since. `findDuplicateBatch` deliberately ignores unresolved
|
||||
// batches, so two pending cards for one order both reach here; without this
|
||||
// the second tap books the order twice.
|
||||
const twin = await findDuplicateBatch(batch.id, batch.file_sha256, header);
|
||||
if (twin) {
|
||||
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: twin.transaction_id, capture_id: twin.capture_id });
|
||||
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: twin.transaction_id, captureId: twin.capture_id, batchId: twin.id });
|
||||
return finish(ctx, kind, "duplicate", card, {
|
||||
transaction_id: twin.transaction_id ?? undefined,
|
||||
capture_id: twin.capture_id ?? undefined,
|
||||
duplicate_of: { batch_id: twin.id, transaction_id: twin.transaction_id, capture_id: twin.capture_id },
|
||||
});
|
||||
}
|
||||
|
||||
const ownerId = await ownerForEmail(batch.shared_by);
|
||||
|
||||
// Before minting a row: is one already here that nothing has described yet?
|
||||
//
|
||||
// matchTransaction above cannot see it. Its key includes the platform, and a
|
||||
// wallet notification has none — so an Uber Eats screenshot never matches the
|
||||
// wallet row for the very tap that paid for it, and the Create button books
|
||||
// the purchase a second time. That is the normal arrival order, not an edge
|
||||
// case: the notification lands within seconds of the tap and the screenshot
|
||||
// is shared later.
|
||||
//
|
||||
// findAdoptableTransaction is the same helper the receipt lane uses and
|
||||
// carries the same guards — exact amount, ±3 days, unclaimed, UNIQUE, and
|
||||
// never a row that already carries line items. Its uniqueness check runs
|
||||
// outside the advisory lock below, so two simultaneous taps could both adopt;
|
||||
// the second would then find the items in place and fall through to the lock,
|
||||
// which is where duplicates are actually resolved.
|
||||
const adopted = await findAdoptableTransaction(ownerId, header.total, header.order_date);
|
||||
if (adopted) {
|
||||
await replaceLineItems(adopted, items, { source: "intake", platform: header.platform, batchId: batch.id, currency: header.currency });
|
||||
await updateBatch(batch.id, { kind, status: "enriched", transaction_id: adopted, items });
|
||||
return finish(ctx, kind, "enriched", enrichedCard({ ...cardInput(ctx), transactionId: adopted }), { transaction_id: adopted });
|
||||
}
|
||||
|
||||
const merchant = header.merchant ?? "Unknown merchant";
|
||||
// Not `orderDescription()`: it always prefixes "Order - ", which is right for
|
||||
// a delivery receipt and wrong for the in-store till receipt this path also
|
||||
// handles. Not `resolveCategory()` either — it re-derives grocery-vs-food
|
||||
// from a merchant pattern, and `kind` is the answer the router already
|
||||
// reached, remembered merchant and Slack tap included.
|
||||
const description = header.platform ? `Order - ${merchant}` : merchant;
|
||||
const category = kind === "grocery" ? "groceries" : "dining";
|
||||
// `amount` is the figure printed on the receipt, in its own currency. No FX
|
||||
// rate exists at intake, so a foreign order leaves `amount_aud` NULL rather
|
||||
// than asserting a conversion — the same shape order-ingestion.ts writes.
|
||||
const currency = header.currency ?? "AUD";
|
||||
const isAud = currency === "AUD";
|
||||
// Check and insert under one lock, on one connection.
|
||||
//
|
||||
// `source_ref = batch.id` makes a second press of THIS card a no-op, and
|
||||
// ON CONFLICT makes that safe against a double tap. It cannot help two
|
||||
// DIFFERENT batches for one purchase — their keys differ — and both taps run
|
||||
// their match and duplicate checks before either inserts, so both can find
|
||||
// nothing. The lock is keyed on the purchase, not the batch, which is the
|
||||
// only thing the two share. It has to be transaction-scoped and therefore
|
||||
// inside withTransaction: taken by a statement of its own it would be
|
||||
// released before the next one ran (see db.ts).
|
||||
const purchaseKey = [header.platform ?? "", merchantKey(merchant), header.order_date, header.total.toFixed(2)].join("|");
|
||||
const outcome = await withTransaction(async (q) => {
|
||||
await q.queryRaw(`SELECT pg_advisory_xact_lock(hashtextextended($1, 0))`, [purchaseKey]);
|
||||
// Mine (a double tap) or another batch's (the race). The ORDER BY is what
|
||||
// tells them apart when both exist.
|
||||
const existing = await q.queryRow<{ id: number; source_ref: string }>(
|
||||
`SELECT id, source_ref FROM transactions
|
||||
WHERE source = 'intake' AND superseded_by_id IS NULL
|
||||
AND (source_ref = $1
|
||||
OR (transaction_date = $2::date AND amount = $3::numeric
|
||||
AND lower(merchant_normalized) = lower($4)))
|
||||
ORDER BY (source_ref = $1) DESC NULLS LAST, id
|
||||
LIMIT 1`,
|
||||
[batch.id, header.order_date, header.total, merchant]
|
||||
);
|
||||
if (existing) return { id: existing.id, mine: existing.source_ref === batch.id };
|
||||
const row = await q.queryRow<{ id: number }>(
|
||||
`INSERT INTO transactions (
|
||||
transaction_date, description, amount, amount_aud, category, payment_method,
|
||||
merchant_name, merchant_normalized, transaction_type, owner_id, source, source_ref,
|
||||
foreign_currency_amount, foreign_currency_code
|
||||
) VALUES ($1::date, $2, $3::numeric, $4::numeric, $5, NULL, $6, $6, 'debit', $7, 'intake', $8, $9::numeric, $10)
|
||||
ON CONFLICT (source, source_ref) WHERE source IS NOT NULL AND source_ref IS NOT NULL
|
||||
DO NOTHING
|
||||
RETURNING id`,
|
||||
[header.order_date, description, header.total, isAud ? header.total : null, category, merchant, ownerId, batch.id, isAud ? null : header.total, isAud ? null : currency]
|
||||
);
|
||||
return { id: row!.id, mine: true };
|
||||
});
|
||||
|
||||
// Another batch booked this purchase in the moment between the two taps. Say
|
||||
// the same thing the sequential case says one second later, and write nothing.
|
||||
if (!outcome.mine) {
|
||||
await updateBatch(batch.id, { kind, status: "duplicate", transaction_id: outcome.id });
|
||||
const card = duplicateCard({ merchant: header.merchant, total: header.total, currency: header.currency, transactionId: outcome.id, captureId: null, batchId: null });
|
||||
return finish(ctx, kind, "duplicate", card, { transaction_id: outcome.id });
|
||||
}
|
||||
|
||||
const txnId = outcome.id;
|
||||
await replaceLineItems(txnId, items, { source: "intake", platform: header.platform, batchId: batch.id, currency });
|
||||
await updateBatch(batch.id, { kind, status: "created", transaction_id: txnId });
|
||||
return finish(ctx, kind, "created", createdCard({ ...cardInput(ctx), transactionId: txnId }), { transaction_id: txnId });
|
||||
}
|
||||
|
||||
/** The card for a batch as it stands — what a second press of a resolved card gets back. */
|
||||
export function resolvedCard(batch: BatchRow): { blocks: Block[]; text: string } {
|
||||
const h = batch.header;
|
||||
const base: CardInput = { batchId: batch.id, merchant: h?.merchant ?? null, total: h?.total ?? null, orderDate: h?.order_date ?? null, platform: h?.platform ?? null, itemCount: batch.items?.length ?? 0, currency: h?.currency };
|
||||
if (batch.status === "enriched" && batch.transaction_id) return enrichedCard({ ...base, transactionId: batch.transaction_id });
|
||||
if (batch.status === "created" && batch.transaction_id) return createdCard({ ...base, transactionId: batch.transaction_id });
|
||||
if (batch.status === "forwarded_to_pantry" && batch.capture_id) return forwardedCard({ ...base, captureId: batch.capture_id, enriching: !!batch.transaction_id });
|
||||
return duplicateCard({ merchant: base.merchant, total: base.total, currency: base.currency, transactionId: batch.transaction_id, captureId: batch.capture_id, batchId: batch.id });
|
||||
}
|
||||
|
||||
// ─── the router ─────────────────────────────────────────────────────────────
|
||||
|
||||
export type IntakeInput = { batchId?: string | null; source: IntakeSource; sharedBy: string | null; files: StoredFile[] };
|
||||
|
||||
export async function runIntake(input: IntakeInput, deps: IntakeDeps): Promise<IntakeResult> {
|
||||
const batchId = input.batchId || randomUUID();
|
||||
const shas = input.files.map((f) => f.sha256);
|
||||
const platform = platformForSource(input.source);
|
||||
|
||||
if (await loadBatch(batchId)) {
|
||||
return { kind: "unsupported", status: "duplicate", batch_id: batchId, slack_blocks: null, text: `batch ${batchId} was already received`, http_status: 409 };
|
||||
}
|
||||
await insertBatch({ id: batchId, source: input.source, sharedBy: input.sharedBy, shas });
|
||||
|
||||
const fail = (kind: IntakeKind, status: IntakeStatus, card: { blocks: Block[]; text: string }, http = 200, extra: Partial<IntakeResult> = {}): IntakeResult => ({
|
||||
kind,
|
||||
status,
|
||||
batch_id: batchId,
|
||||
slack_blocks: card.blocks,
|
||||
text: card.text,
|
||||
http_status: http,
|
||||
...extra,
|
||||
});
|
||||
|
||||
// 1. classify each file and build the parts, in the order shared.
|
||||
const parts: InputPart[] = [];
|
||||
const usable: StoredFile[] = [];
|
||||
/** Text parts dropped before the model saw them — see `textCouldBeReceipt`. */
|
||||
const rejectedText: { name: string; text: string; reason: string }[] = [];
|
||||
for (const f of input.files) {
|
||||
const kind = classifyInput(f);
|
||||
if (kind === "pdf") {
|
||||
let text = "";
|
||||
try {
|
||||
text = await extractPdfText(f.bytes);
|
||||
} catch {
|
||||
/* unreadable PDF — reported below */
|
||||
}
|
||||
if (!text) {
|
||||
await updateBatch(batchId, { kind: "unsupported", status: "unsupported" });
|
||||
return fail("unsupported", "unsupported", unsupportedCard(`${f.name} has no readable text — try a screenshot of it`));
|
||||
}
|
||||
parts.push({ kind: "text", text, name: f.name });
|
||||
} else if (kind === "image") {
|
||||
parts.push({ kind: "image", mime: f.type || "image/jpeg", bytes: f.bytes, name: f.name });
|
||||
} else if (kind === "text") {
|
||||
// Nothing that cannot be a receipt reaches the model. A share that
|
||||
// arrives as the word "Image" (the phone coerced the photo to text) is
|
||||
// not a thin receipt, and asking anyway produces a whole invented one —
|
||||
// 2026-08-22, `textCouldBeReceipt`. Dropped like an unreadable file type
|
||||
// rather than failing the batch, so one junk text part beside a good
|
||||
// screenshot still goes through; if nothing usable is left the card below
|
||||
// quotes what the phone actually sent.
|
||||
const text = f.bytes.toString("utf-8");
|
||||
const check = textCouldBeReceipt(text);
|
||||
if (!check.ok) {
|
||||
rejectedText.push({ name: f.name, text, reason: check.reason });
|
||||
continue;
|
||||
}
|
||||
parts.push({ kind: "text", text, name: f.name });
|
||||
} else {
|
||||
continue;
|
||||
}
|
||||
usable.push(f);
|
||||
}
|
||||
if (!parts.length) {
|
||||
await updateBatch(batchId, { kind: "unsupported", status: "unsupported" });
|
||||
const dropped = rejectedText[0];
|
||||
return fail(
|
||||
"unsupported",
|
||||
"unsupported",
|
||||
unsupportedCard(
|
||||
dropped
|
||||
? `the share arrived as text and ${dropped.reason}: "${quoteShared(dropped.text)}" — send the picture or the PDF itself`
|
||||
: undefined
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
// A re-share of the very same files, before spending a model call on it.
|
||||
const dupBySha = await findDuplicateBatch(batchId, shas, null);
|
||||
if (dupBySha) {
|
||||
await updateBatch(batchId, { status: "duplicate", transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id });
|
||||
return fail("unsupported", "duplicate", duplicateCard({ merchant: dupBySha.header?.merchant ?? null, total: dupBySha.header?.total ?? null, currency: dupBySha.header?.currency, transactionId: dupBySha.transaction_id, captureId: dupBySha.capture_id, batchId: dupBySha.id }), 200, {
|
||||
kind: (dupBySha.kind as IntakeKind) ?? "food",
|
||||
duplicate_of: { batch_id: dupBySha.id, transaction_id: dupBySha.transaction_id, capture_id: dupBySha.capture_id },
|
||||
});
|
||||
}
|
||||
|
||||
// Keep the originals: a later "Groceries" tap forwards them to pantry.
|
||||
try {
|
||||
await storeFiles(usable);
|
||||
} catch (e) {
|
||||
console.warn("intake: could not store files", e instanceof Error ? e.message : e);
|
||||
}
|
||||
|
||||
// 2. ONE extraction call for the whole batch.
|
||||
let extracted: ExtractedOrder;
|
||||
try {
|
||||
extracted = await deps.extractor(parts);
|
||||
} catch (e) {
|
||||
const detail = e instanceof ExtractionError ? e.message : e instanceof Error ? e.message : String(e);
|
||||
await updateBatch(batchId, { status: "error" });
|
||||
return fail("held", "error", errorCard(detail), 502);
|
||||
}
|
||||
const items = toLineItems(extracted.items);
|
||||
|
||||
// 2b. did it answer out of the upload, or out of its prompt?
|
||||
const confab = confabulationReason(
|
||||
{
|
||||
textParts: parts.filter((p): p is Extract<InputPart, { kind: "text" }> => p.kind === "text").map((p) => p.text),
|
||||
hasImages: parts.some((p) => p.kind === "image"),
|
||||
},
|
||||
extracted
|
||||
);
|
||||
if (confab) {
|
||||
await updateBatch(batchId, { kind: "unsupported", status: "unsupported" });
|
||||
return fail("unsupported", "unsupported", unsupportedCard(confab));
|
||||
}
|
||||
|
||||
// 3. header.
|
||||
//
|
||||
// This path stores the partial header and the items on the batch row, which
|
||||
// is right for a genuine screenshot missing its top — the next tap has
|
||||
// something to work from. It is wrong for a confabulation, and on 2026-08-22
|
||||
// it stored one: an invented Burgertory header with two invented lines, held
|
||||
// back from the matcher only by an absent date. Nothing here could tell the
|
||||
// two apart, which is why the pre-check above and 2b now keep confabulations
|
||||
// from reaching this point at all rather than trying to filter them here.
|
||||
const headerCheck = validateHeader(extracted);
|
||||
if (!headerCheck.ok) {
|
||||
await updateBatch(batchId, { header: { merchant: extracted.merchant, order_date: extracted.order_date, total: extracted.total, platform }, items, status: "needs_header" });
|
||||
return fail("held", "needs_header", needsHeaderCard(headerCheck.missing, input.source), 200, { items });
|
||||
}
|
||||
const header = headerCheck.header;
|
||||
const storedHeader = { ...header, platform, subtotal: extracted.subtotal ?? null, currency: extracted.currency ?? "AUD" };
|
||||
|
||||
// 4. merchant → grocery | food | ask.
|
||||
const kind: MerchantKind = resolveMerchantKind(input.source, header.merchant, await rememberedKind(header.merchant));
|
||||
|
||||
// 5. the same order, already handled under other files.
|
||||
const dup = await findDuplicateBatch(batchId, shas, storedHeader);
|
||||
if (dup) {
|
||||
await updateBatch(batchId, { header: storedHeader, items, kind, status: "duplicate", transaction_id: dup.transaction_id, capture_id: dup.capture_id });
|
||||
return fail(kind, "duplicate", duplicateCard({ merchant: header.merchant, total: header.total, currency: storedHeader.currency, transactionId: dup.transaction_id, captureId: dup.capture_id, batchId: dup.id }), 200, {
|
||||
header,
|
||||
items,
|
||||
duplicate_of: { batch_id: dup.id, transaction_id: dup.transaction_id, capture_id: dup.capture_id },
|
||||
});
|
||||
}
|
||||
|
||||
// 6. do the lines add up? (missing-screenshot check)
|
||||
const recon = reconcileItems(extracted);
|
||||
if (!recon.ok) {
|
||||
await updateBatch(batchId, { header: storedHeader, items, kind, status: "held" });
|
||||
return fail("held", "held", heldCard(recon.reason ?? "mismatch"), 200, { header, items });
|
||||
}
|
||||
|
||||
await updateBatch(batchId, { header: storedHeader, items, kind, status: "received" });
|
||||
const ctx: RouteContext = { batchId, source: input.source, sharedBy: input.sharedBy, platform, header, items, currency: storedHeader.currency };
|
||||
|
||||
// 7. act.
|
||||
if (kind === "ask") {
|
||||
await updateBatch(batchId, { kind: "ask", status: "ask" });
|
||||
return finish(ctx, "ask", "ask", askCard(cardInput(ctx)));
|
||||
}
|
||||
if (kind === "grocery") return routeGrocery(ctx, usable, deps.pantry);
|
||||
return routeFood(ctx);
|
||||
}
|
||||
@@ -0,0 +1,343 @@
|
||||
/**
|
||||
* Intake router — the pure decisions.
|
||||
*
|
||||
* Everything here is a function of its arguments: what kind of input a file is,
|
||||
* which platform a source tap implies, whether a merchant is a grocer, whether a
|
||||
* header is complete, whether the items add up. Kept free of the database and the
|
||||
* model so each rule can be unit-tested against the exact strings real receipts
|
||||
* print. The orchestration that calls these lives in `intake-router.ts`.
|
||||
*
|
||||
* Plan: smarthome `plans/intake-router-slice-plan.md` §2, §3, §5, §11 C1.
|
||||
*/
|
||||
|
||||
export const INTAKE_SOURCES = ["uber_eats", "doordash", "coles", "woolworths", "other"] as const;
|
||||
export type IntakeSource = (typeof INTAKE_SOURCES)[number];
|
||||
|
||||
export function isIntakeSource(value: unknown): value is IntakeSource {
|
||||
return typeof value === "string" && (INTAKE_SOURCES as readonly string[]).includes(value);
|
||||
}
|
||||
|
||||
export type InputKind = "image" | "pdf" | "text" | "unsupported";
|
||||
|
||||
/** Input type picks the extractor; it is independent of source and merchant. */
|
||||
export function classifyInput(file: { type?: string | null; name?: string | null }): InputKind {
|
||||
const type = (file.type ?? "").toLowerCase();
|
||||
const name = file.name ?? "";
|
||||
if (type === "application/pdf" || /\.pdf$/i.test(name)) return "pdf";
|
||||
if (type.startsWith("image/") || /\.(png|jpe?g|webp|heic|gif)$/i.test(name)) return "image";
|
||||
if (type.startsWith("text/") || /\.(txt|md)$/i.test(name)) return "text";
|
||||
return "unsupported";
|
||||
}
|
||||
|
||||
/**
|
||||
* The platform the source tap implies, in `expense_metadata.platform` vocabulary.
|
||||
* Null for an in-store receipt or `other`: those rows carry no platform, so the
|
||||
* match falls back to (merchant, date, total) with no platform constraint.
|
||||
*/
|
||||
export function platformForSource(source: IntakeSource): string | null {
|
||||
if (source === "uber_eats") return "ubereats";
|
||||
if (source === "doordash") return "doordash";
|
||||
return null;
|
||||
}
|
||||
|
||||
export const PLATFORM_LABEL: Record<string, string> = {
|
||||
ubereats: "Uber Eats",
|
||||
doordash: "DoorDash",
|
||||
uber: "Uber",
|
||||
};
|
||||
|
||||
export function platformLabel(platform: string | null): string {
|
||||
return platform ? PLATFORM_LABEL[platform] ?? platform : "";
|
||||
}
|
||||
|
||||
/**
|
||||
* A small pattern list, not an exact lookup (plan §2, Codex #6). Uber Eats prints
|
||||
* `Coles (Wyndham Vale)`, a PDF may print `Woolworths Group Limited`; a
|
||||
* word-boundary alternation on the normalised name covers both without a
|
||||
* merchant master-data subsystem.
|
||||
*
|
||||
* The boundaries are load-bearing, not tidiness: a bare substring made
|
||||
* `Bigals Pizza` a grocer, because "b-IGA-ls" contains `iga`. Short names in a
|
||||
* substring list collide with ordinary words, and the collision reads as a
|
||||
* correct classification.
|
||||
*/
|
||||
export const GROCERY_PATTERN = /\b(coles|woolworths|woolies|aldi|iga|foodworks|costco)\b/i;
|
||||
|
||||
/**
|
||||
* The normalised merchant used for matching and for the remembered-kind table.
|
||||
* Lower-case, punctuation dropped, whitespace collapsed — so `Coles (Wyndham Vale)`
|
||||
* and `coles wyndham vale` are one key, while `Coles` stays distinct from both.
|
||||
*/
|
||||
export function merchantKey(name: string | null | undefined): string {
|
||||
return (name ?? "")
|
||||
.toLowerCase()
|
||||
.replace(/[^\p{L}\p{N}]+/gu, " ")
|
||||
.trim()
|
||||
.replace(/\s+/g, " ");
|
||||
}
|
||||
|
||||
export type MerchantKind = "grocery" | "food" | "ask";
|
||||
|
||||
const UNKNOWN_MERCHANT = /^(unknown|n ?a|none|null|undefined|merchant)$/;
|
||||
|
||||
/**
|
||||
* Grocery-vs-food is decided by the merchant, not the platform — "Uber Eats"
|
||||
* covers both a meal and a grocery run. The source tap is ground truth where it
|
||||
* says so (Coles/Woolworths taps are grocers by definition); the pattern list
|
||||
* decides the obvious delivery cases; the remembered answer covers the
|
||||
* `Romeo's Food Hall`s; and `other` with nothing else to go on asks rather than
|
||||
* silently falling through to "meal" (plan §2, D3 holding state).
|
||||
*/
|
||||
export function resolveMerchantKind(
|
||||
source: IntakeSource,
|
||||
merchant: string | null | undefined,
|
||||
remembered: "grocery" | "food" | null = null
|
||||
): MerchantKind {
|
||||
if (source === "coles" || source === "woolworths") return "grocery";
|
||||
const key = merchantKey(merchant);
|
||||
if (!key || UNKNOWN_MERCHANT.test(key)) return "ask";
|
||||
if (GROCERY_PATTERN.test(key)) return "grocery";
|
||||
if (remembered) return remembered;
|
||||
if (source === "other") return "ask";
|
||||
return "food";
|
||||
}
|
||||
|
||||
/** What the extractor returns — names verbatim, options kept (plan §3). */
|
||||
export type ExtractedItem = {
|
||||
quantity: number | null;
|
||||
description: string;
|
||||
unit_price?: number | null;
|
||||
line_total?: number | null;
|
||||
options?: string[] | null;
|
||||
};
|
||||
|
||||
export type ExtractedOrder = {
|
||||
merchant: string | null;
|
||||
order_date: string | null;
|
||||
total: number | null;
|
||||
subtotal?: number | null;
|
||||
currency?: string | null;
|
||||
items: ExtractedItem[];
|
||||
adjustments?: { label: string; amount: number }[] | null;
|
||||
};
|
||||
|
||||
export type OrderHeader = { merchant: string | null; order_date: string; total: number };
|
||||
|
||||
/**
|
||||
* "Missing the header/total shot → nothing is written." A header is the date and
|
||||
* the total; the merchant may be absent and still route (it becomes an ask).
|
||||
*/
|
||||
export function validateHeader(
|
||||
x: ExtractedOrder
|
||||
): { ok: true; header: OrderHeader } | { ok: false; missing: string[] } {
|
||||
const missing: string[] = [];
|
||||
if (typeof x.order_date !== "string" || !/^\d{4}-\d{2}-\d{2}$/.test(x.order_date)) missing.push("date");
|
||||
if (typeof x.total !== "number" || !Number.isFinite(x.total) || x.total <= 0) missing.push("total");
|
||||
if (missing.length) return { ok: false, missing };
|
||||
return {
|
||||
ok: true,
|
||||
header: {
|
||||
merchant: typeof x.merchant === "string" && x.merchant.trim() ? x.merchant.trim() : null,
|
||||
order_date: x.order_date as string,
|
||||
total: round2(x.total as number),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export const round2 = (n: number) => Number(n.toFixed(2));
|
||||
|
||||
/** A line's money, as printed or derived; null when the receipt gave neither. */
|
||||
export function itemAmount(item: ExtractedItem): number | null {
|
||||
if (typeof item.line_total === "number" && Number.isFinite(item.line_total)) return round2(item.line_total);
|
||||
if (typeof item.unit_price === "number" && Number.isFinite(item.unit_price)) {
|
||||
return round2(item.unit_price * (item.quantity ?? 1));
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export type Reconciliation = {
|
||||
ok: boolean;
|
||||
basis: "subtotal" | "total" | "none";
|
||||
items_sum: number;
|
||||
reference: number | null;
|
||||
diff: number;
|
||||
reason?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* The missing-screenshot check (plan §3, Codex #7): the lines must reconcile
|
||||
* within $1 or 2%, else the batch is held.
|
||||
*
|
||||
* Reconciled to the printed ITEM SUBTOTAL when there is one, not the total — a
|
||||
* delivery order's total is items plus fees minus promotions ($118.94 of items
|
||||
* became $85.74 on the real Coles export), so against the total every delivery
|
||||
* order would be held. The total stays the match key; the subtotal is what the
|
||||
* items can be checked against. Without a printed subtotal the total is the only
|
||||
* reference there is. A receipt whose lines carry no prices at all (pasted text)
|
||||
* cannot be checked and is not held for it.
|
||||
*/
|
||||
export function reconcileItems(x: ExtractedOrder): Reconciliation {
|
||||
const amounts = (x.items ?? []).map(itemAmount);
|
||||
if (!x.items?.length) {
|
||||
return { ok: false, basis: "none", items_sum: 0, reference: null, diff: 0, reason: "no items" };
|
||||
}
|
||||
if (amounts.every((a) => a === null)) {
|
||||
return { ok: true, basis: "none", items_sum: 0, reference: null, diff: 0 };
|
||||
}
|
||||
const itemsSum = round2(amounts.reduce<number>((s, a) => s + (a ?? 0), 0));
|
||||
const subtotal = typeof x.subtotal === "number" && Number.isFinite(x.subtotal) ? x.subtotal : null;
|
||||
const basis: Reconciliation["basis"] = subtotal !== null ? "subtotal" : "total";
|
||||
const reference = subtotal ?? (typeof x.total === "number" ? x.total : null);
|
||||
if (reference === null) return { ok: false, basis, items_sum: itemsSum, reference: null, diff: 0, reason: "no total" };
|
||||
const diff = round2(itemsSum - reference);
|
||||
const tolerance = Math.max(1, 0.02 * reference);
|
||||
const ok = Math.abs(diff) <= tolerance;
|
||||
return ok
|
||||
? { ok, basis, items_sum: itemsSum, reference, diff }
|
||||
: { ok, basis, items_sum: itemsSum, reference, diff, reason: `items sum to ${itemsSum.toFixed(2)} but the ${basis} is ${reference.toFixed(2)}` };
|
||||
}
|
||||
|
||||
/**
|
||||
* The stored shape — the same keys the order lane writes and
|
||||
* `components/order-details.tsx` renders (`qty`, `description`, `amount`,
|
||||
* `options`). A burger called "Michael" is stored as "Michael".
|
||||
*/
|
||||
export type StoredLineItem = {
|
||||
qty: number;
|
||||
description: string;
|
||||
amount: number | null;
|
||||
unit_price?: number | null;
|
||||
options?: string[];
|
||||
};
|
||||
|
||||
export function toLineItems(items: ExtractedItem[]): StoredLineItem[] {
|
||||
return (items ?? [])
|
||||
.filter((i) => typeof i.description === "string" && i.description.trim())
|
||||
.map((i) => {
|
||||
const out: StoredLineItem = {
|
||||
qty: typeof i.quantity === "number" && i.quantity > 0 ? i.quantity : 1,
|
||||
description: i.description.trim(),
|
||||
amount: itemAmount(i),
|
||||
};
|
||||
if (typeof i.unit_price === "number" && Number.isFinite(i.unit_price)) out.unit_price = round2(i.unit_price);
|
||||
const options = (i.options ?? []).map((o) => String(o).trim()).filter(Boolean);
|
||||
if (options.length) out.options = options;
|
||||
return out;
|
||||
});
|
||||
}
|
||||
|
||||
/** Is `needle` a contiguous run of whole words inside `hay`? */
|
||||
function containsWordRun(hay: string[], needle: string[]): boolean {
|
||||
if (!needle.length || needle.length > hay.length) return false;
|
||||
for (let i = 0; i + needle.length <= hay.length; i++) {
|
||||
if (needle.every((w, j) => hay[i + j] === w)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Does the stored merchant name denote the extracted one? One name's words must
|
||||
* appear as a contiguous run inside the other's — so `Coles` matches
|
||||
* `Coles (Wyndham Vale)` and `Order - Coles (Wyndham Vale)`, which is spelling,
|
||||
* not widening.
|
||||
*
|
||||
* Whole words, never a raw substring: `merchantKey("Bigals Pizza")` contains
|
||||
* `iga`, so a pizza shop matched every IGA on the same date and total.
|
||||
*/
|
||||
export function merchantMatches(stored: string | null | undefined, extracted: string | null | undefined): boolean {
|
||||
const a = merchantKey(stored).split(" ").filter(Boolean);
|
||||
const b = merchantKey(extracted).split(" ").filter(Boolean);
|
||||
if (!a.length || !b.length) return false;
|
||||
return containsWordRun(a, b) || containsWordRun(b, a);
|
||||
}
|
||||
|
||||
// ─── is this text even a receipt? (2026-08-22) ───────────────────────────────
|
||||
|
||||
/**
|
||||
* The merchant names the extraction prompt uses to show the FORMAT. They are
|
||||
* placeholders rather than real shops on purpose.
|
||||
*
|
||||
* On 2026-08-22 an iOS Shortcut coerced a shared photo to text, so the intake
|
||||
* router received one text part whose entire content was the five characters
|
||||
* `Image`. The model was handed that, and returned a whole receipt:
|
||||
* `Burgertory (Braybrook)`, two "The Pride" lines at $22.80, a $41.15 total.
|
||||
* Nothing on the receipt existed — "Burgertory" was in the prompt, as the
|
||||
* example of how to print a branch in parentheses. Only the absent date kept it
|
||||
* out of the matcher; the invented header and items were stored on the batch.
|
||||
*
|
||||
* Same shape as the ingestion engine's 2026-08-05 empty-body defect: a stage
|
||||
* that can be handed nothing has to say so, because the model downstream will
|
||||
* always answer, and it answers out of its prompt.
|
||||
*
|
||||
* Exported from here rather than written into the prompt string so the prompt
|
||||
* and the guard that rejects an echo of it cannot drift apart.
|
||||
*/
|
||||
export const PROMPT_PLACEHOLDER_MERCHANTS = ["<Store> (<Suburb>)"] as const;
|
||||
|
||||
/**
|
||||
* Below this a text part needs a money figure to be believable. NOT a floor on
|
||||
* its own: `TOTAL $17.00` is twelve characters and is exactly the kind of
|
||||
* pasted line this door exists to accept.
|
||||
*/
|
||||
export const MIN_RECEIPT_TEXT_CHARS = 40;
|
||||
|
||||
/** `$17.00`, `17.00`, `12,50` — a printed amount, not merely a digit. */
|
||||
const MONEY_SHAPE = /(?:[$€£¥]\s*\d|\d[.,]\d{2}(?!\d))/;
|
||||
|
||||
export type TextCheck = { ok: true } | { ok: false; reason: string };
|
||||
|
||||
/**
|
||||
* Can this text possibly be a receipt? Deterministic, before any model call.
|
||||
*
|
||||
* A receipt has more than one word and at least one number. Everything the
|
||||
* phone substitutes when a share goes wrong — the literal word `Image`, a bare
|
||||
* filename like `IMG_0042.jpeg`, an empty body — fails one of those, and none
|
||||
* of them fails in a way a model will admit to.
|
||||
*
|
||||
* Applies to TEXT parts only. A PDF is checked for readable text separately and
|
||||
* an image is the thing we actually asked for.
|
||||
*/
|
||||
export function textCouldBeReceipt(raw: string | null | undefined): TextCheck {
|
||||
const text = (raw ?? "").trim();
|
||||
if (!text) return { ok: false, reason: "it was empty" };
|
||||
if (!/\s/.test(text)) return { ok: false, reason: "it is a single word" };
|
||||
if (!/\d/.test(text)) return { ok: false, reason: "it has no numbers in it" };
|
||||
if (text.length < MIN_RECEIPT_TEXT_CHARS && !MONEY_SHAPE.test(text)) {
|
||||
return { ok: false, reason: "it is too short to be a receipt" };
|
||||
}
|
||||
return { ok: true };
|
||||
}
|
||||
|
||||
/** What the phone actually sent, short enough to put on a card. */
|
||||
export function quoteShared(raw: string, max = 80): string {
|
||||
const flat = (raw ?? "").replace(/\s+/g, " ").trim();
|
||||
return flat.length > max ? `${flat.slice(0, max - 1)}…` : flat;
|
||||
}
|
||||
|
||||
/**
|
||||
* Belt to `textCouldBeReceipt`'s braces: did the extractor answer out of its
|
||||
* prompt rather than out of the upload?
|
||||
*
|
||||
* Two tells, both cheap and both deterministic:
|
||||
* - the merchant IS one of the prompt's format placeholders; or
|
||||
* - nothing readable was sent at all (no images, and no text part that could
|
||||
* be a receipt) and yet a merchant or items came back.
|
||||
*
|
||||
* The second is reachable even with the pre-check in place, because the
|
||||
* pre-check only inspects text parts: a PDF that extracts to a few characters
|
||||
* of junk still reaches the model. This is `unsupported`, never `needs_header`
|
||||
* — a confabulated header is not a partial one.
|
||||
*/
|
||||
export function confabulationReason(
|
||||
input: { textParts: string[]; hasImages: boolean },
|
||||
extracted: { merchant: string | null; items: unknown[] }
|
||||
): string | null {
|
||||
const key = merchantKey(extracted.merchant);
|
||||
if (key && PROMPT_PLACEHOLDER_MERCHANTS.some((p) => merchantKey(p) === key)) {
|
||||
return "the merchant came back as the prompt's own example rather than a name off the receipt";
|
||||
}
|
||||
if (input.hasImages) return null;
|
||||
if (input.textParts.some((t) => textCouldBeReceipt(t).ok)) return null;
|
||||
if (!extracted.merchant && !extracted.items.length) return null;
|
||||
return "nothing shared could be read as a receipt, yet an order came back";
|
||||
}
|
||||
@@ -0,0 +1,200 @@
|
||||
import { queryRow } from "@/lib/db";
|
||||
|
||||
/**
|
||||
* Replace the itemised receipt behind a transaction (plan §11 C2).
|
||||
*
|
||||
* An idempotent UPDATE, never an INSERT of a second row: `expense_metadata.
|
||||
* transaction_id` is UNIQUE and the order the email pipeline booked already has
|
||||
* its row — with `line_items = []`, because the items sat behind an auth-gated
|
||||
* link. Idempotence is also what lets pantry re-run it after every confirm,
|
||||
* writing the board's current state (plan §5, Codex #3), and what lets undo
|
||||
* restore `previous_line_items` through the same call.
|
||||
*
|
||||
* The row is found on `transaction_id = :id OR matched_transaction_id = :id` —
|
||||
* a card-settled order has only the latter (CLAUDE.md trap). When the
|
||||
* transaction has no row at all (a manual transaction, a statement line), one
|
||||
* is created so the items have somewhere to live; a transaction is NEVER
|
||||
* created here.
|
||||
*
|
||||
* `opts.metadataId` pins WHICH row when the caller already resolved one. A
|
||||
* statement transaction can own its own `expense_metadata` row AND be the
|
||||
* `matched_transaction_id` of a delivery row: the match query and this write
|
||||
* then re-pick independently and can land on different rows, so the enrichment
|
||||
* reports success against a row nobody reads.
|
||||
*/
|
||||
|
||||
export type LineItemsSource = "intake" | "pantry";
|
||||
|
||||
export class TransactionNotFoundError extends Error {}
|
||||
export class LineItemsValidationError extends Error {}
|
||||
|
||||
export type LineItemRecord = Record<string, unknown> & { description: string };
|
||||
|
||||
/**
|
||||
* Permissive on purpose: the order lane writes `{qty, description, amount,
|
||||
* options}`, the receipt lane `{description, qty, amount, unit, category}`, and
|
||||
* the detail panel reads the overlap. What every producer must supply is a
|
||||
* description — an item with no name renders as nothing.
|
||||
*/
|
||||
export function validateLineItems(input: unknown): LineItemRecord[] {
|
||||
if (!Array.isArray(input)) throw new LineItemsValidationError("line_items must be an array");
|
||||
return input.map((raw, i) => {
|
||||
if (!raw || typeof raw !== "object" || Array.isArray(raw)) {
|
||||
throw new LineItemsValidationError(`line_items[${i}] must be an object`);
|
||||
}
|
||||
const item = { ...(raw as Record<string, unknown>) };
|
||||
if (typeof item.description !== "string" || !item.description.trim()) {
|
||||
throw new LineItemsValidationError(`line_items[${i}].description is required`);
|
||||
}
|
||||
item.description = item.description.trim();
|
||||
// `quantity` is the extractor's word; `qty` is what the renderer reads.
|
||||
if (item.qty === undefined && typeof item.quantity === "number") item.qty = item.quantity;
|
||||
return item as LineItemRecord;
|
||||
});
|
||||
}
|
||||
|
||||
export type ReplaceResult = {
|
||||
transaction_id: number;
|
||||
metadata_id: number;
|
||||
created_metadata: boolean;
|
||||
previous_line_items: unknown[];
|
||||
line_items: LineItemRecord[];
|
||||
};
|
||||
|
||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
||||
|
||||
/**
|
||||
* The undo baseline, recorded once per batch.
|
||||
*
|
||||
* Returning the row that was just read makes a RETRY destroy the thing undo
|
||||
* exists to restore: pantry re-calls this on every confirm and after a timeout,
|
||||
* and the second call would hand back its own write as the "previous" state.
|
||||
* The first snapshot a batch ever takes is the only one worth keeping, so it is
|
||||
* written under COALESCE and read back rather than trusted.
|
||||
*/
|
||||
async function undoBaseline(batchId: string | null | undefined, snapshot: unknown[]): Promise<unknown[]> {
|
||||
// Pantry passes back whatever batch id it was given; only an intake batch has
|
||||
// a row to record against, and a non-uuid would fail the cast.
|
||||
if (!batchId || !UUID.test(batchId)) return snapshot;
|
||||
const row = await queryRow<{ previous_line_items: unknown }>(
|
||||
`UPDATE intake_batches
|
||||
SET previous_line_items = COALESCE(previous_line_items, $2::jsonb)
|
||||
WHERE id = $1::uuid
|
||||
RETURNING previous_line_items`,
|
||||
[batchId, JSON.stringify(snapshot)]
|
||||
);
|
||||
return Array.isArray(row?.previous_line_items) ? row.previous_line_items : snapshot;
|
||||
}
|
||||
|
||||
export async function replaceLineItems(
|
||||
transactionId: number,
|
||||
lineItems: LineItemRecord[],
|
||||
opts: { source: LineItemsSource; platform?: string | null; batchId?: string | null; metadataId?: number | null; currency?: string | null }
|
||||
): Promise<ReplaceResult> {
|
||||
const txn = await queryRow<{
|
||||
id: number;
|
||||
merchant_normalized: string | null;
|
||||
merchant_name: string | null;
|
||||
amount: string;
|
||||
transaction_date: string;
|
||||
}>(
|
||||
`SELECT id, merchant_normalized, merchant_name, amount::text, transaction_date::text
|
||||
FROM transactions WHERE id = $1`,
|
||||
[transactionId]
|
||||
);
|
||||
if (!txn) throw new TransactionNotFoundError(`transaction ${transactionId} does not exist`);
|
||||
|
||||
// The caller's row wins where it still exists; otherwise prefer the row that
|
||||
// IS this transaction over one that merely points at it — a statement line
|
||||
// can be the `matched_transaction_id` of a receipt row and carry its own row
|
||||
// too, and the items belong on the receipt.
|
||||
const pinned = opts.metadataId
|
||||
? await queryRow<{ id: number; line_items: unknown }>(
|
||||
`SELECT id, line_items FROM expense_metadata
|
||||
WHERE id = $1 AND (transaction_id = $2 OR matched_transaction_id = $2)`,
|
||||
[opts.metadataId, transactionId]
|
||||
)
|
||||
: null;
|
||||
|
||||
const json = JSON.stringify(lineItems);
|
||||
|
||||
/**
|
||||
* Snapshot the baseline, then write. In that order on BOTH paths: the create
|
||||
* path used to snapshot after its INSERT, so a crash in between left a retry
|
||||
* recording the intake's own write as "previous" — undo restoring the thing
|
||||
* it was meant to reverse, which is the failure `previous_line_items` exists
|
||||
* to prevent.
|
||||
*
|
||||
* Returns null when the row is gone between the read and the write. That
|
||||
* UPDATE reported success and the batch was marked enriched against a row
|
||||
* nothing had written to.
|
||||
*/
|
||||
const write = async (row: { id: number; line_items: unknown }): Promise<ReplaceResult | null> => {
|
||||
const previous = await undoBaseline(opts.batchId, Array.isArray(row.line_items) ? row.line_items : []);
|
||||
const updated = await queryRow<{ id: number }>(
|
||||
`UPDATE expense_metadata SET line_items = $2::jsonb WHERE id = $1 RETURNING id`,
|
||||
[row.id, json]
|
||||
);
|
||||
if (!updated) return null;
|
||||
return { transaction_id: transactionId, metadata_id: row.id, created_metadata: false, previous_line_items: previous, line_items: lineItems };
|
||||
};
|
||||
|
||||
if (pinned) {
|
||||
const done = await write(pinned);
|
||||
if (done) return done;
|
||||
}
|
||||
const existing = await queryRow<{ id: number; line_items: unknown }>(
|
||||
`SELECT id, line_items FROM expense_metadata
|
||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||
-- NULLS LAST is load-bearing, and its absence inverted this clause.
|
||||
-- Postgres orders DESC as NULLS FIRST, and the row that merely POINTS at
|
||||
-- the transaction has a NULL transaction_id — so the comparison meant to
|
||||
-- prefer the owned row preferred the other one, silently and in exactly
|
||||
-- the case it was written for. matchTransaction spells it out; this did
|
||||
-- not, and the two therefore disagreed.
|
||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
||||
LIMIT 1`,
|
||||
[transactionId]
|
||||
);
|
||||
if (existing) {
|
||||
const done = await write(existing);
|
||||
if (done) return done;
|
||||
}
|
||||
|
||||
// Nothing to update. Snapshot the empty baseline BEFORE creating the row.
|
||||
const previous = await undoBaseline(opts.batchId, []);
|
||||
const reference = opts.batchId ? `intake:${opts.batchId}` : `intake:txn:${transactionId}`;
|
||||
// ON CONFLICT because two calls for one transaction can both find no row and
|
||||
// both INSERT; expense_metadata.transaction_id is UNIQUE, so one of them lost
|
||||
// with a constraint error that reached the user as an error card. `xmax = 0`
|
||||
// is how a plain insert is told from a conflict-update, so the reported
|
||||
// `created_metadata` stays true.
|
||||
const created = await queryRow<{ id: number; inserted: boolean }>(
|
||||
`INSERT INTO expense_metadata (
|
||||
transaction_id, source, platform, order_reference, line_items,
|
||||
merchant_normalized, amount, transaction_date, currency, flags, extraction_model
|
||||
) VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7::numeric, $8::date, $9, '[]'::jsonb, 'cloud-budget')
|
||||
ON CONFLICT (transaction_id) DO UPDATE SET line_items = EXCLUDED.line_items
|
||||
RETURNING id, (xmax = 0) AS inserted`,
|
||||
[
|
||||
transactionId,
|
||||
opts.source,
|
||||
opts.platform ?? null,
|
||||
reference,
|
||||
json,
|
||||
txn.merchant_normalized ?? txn.merchant_name ?? null,
|
||||
txn.amount,
|
||||
txn.transaction_date,
|
||||
// Not hard-coded 'AUD': the transaction may carry a foreign figure, and
|
||||
// OrderDetails prefers this column, so a USD order printed "$".
|
||||
opts.currency || "AUD",
|
||||
]
|
||||
);
|
||||
return {
|
||||
transaction_id: transactionId,
|
||||
metadata_id: created!.id,
|
||||
created_metadata: created!.inserted,
|
||||
previous_line_items: previous,
|
||||
line_items: lineItems,
|
||||
};
|
||||
}
|
||||
@@ -12,13 +12,6 @@ export const CUTOVER_DATE = "2026-01-09";
|
||||
*/
|
||||
export const DEFAULT_OWNER_ID = 1;
|
||||
|
||||
/** Human labels for the platform a receipt came from. */
|
||||
export const PLATFORM_LABEL: Record<ParsedOrder["platform"], string> = {
|
||||
doordash: "DoorDash",
|
||||
ubereats: "Uber Eats",
|
||||
uber: "Uber",
|
||||
};
|
||||
|
||||
/**
|
||||
* Transaction description.
|
||||
*
|
||||
|
||||
@@ -526,7 +526,11 @@ export function parseOrderHTML(html: string, meta: MessageMeta): ParsedOrder {
|
||||
} else {
|
||||
// DoorDash carries no order id anywhere in the receipt, so the provider
|
||||
// message id is the only stable identity available. That is correct for
|
||||
// ingestion idempotency: one receipt is one order.
|
||||
// ingestion idempotency: one receipt is one order — PROVIDED every caller
|
||||
// sends the same id for the same email. Two lanes post here (n8n Order
|
||||
// Processor and ingestion-engine order_handoff); both must send the bare
|
||||
// RFC Message-ID. n8n sent Gmail's own id until 2026-10-03 and one order
|
||||
// became txns 7268 + 7269.
|
||||
if (!meta.messageId) {
|
||||
throw new OrderParseError("no order id in body and no messageId supplied");
|
||||
}
|
||||
|
||||
@@ -119,6 +119,16 @@ export async function merchantForTransaction(
|
||||
const row = await queryRow<{ merchant_normalized: string | null }>(
|
||||
`SELECT merchant_normalized FROM expense_metadata
|
||||
WHERE transaction_id = $1 OR matched_transaction_id = $1
|
||||
-- Ordered the same way as replaceLineItems (line-items.ts) and
|
||||
-- matchTransaction (intake-router.ts), so every reader of this row
|
||||
-- resolves the same one a write lands on. NULLS LAST because DESC is
|
||||
-- NULLS FIRST and the row that merely POINTS at the transaction has a
|
||||
-- NULL transaction_id.
|
||||
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
|
||||
-- pointing-only, and ZERO transactions resolving to more than one row by
|
||||
-- either link — so this changes nothing today. It is here so the readers
|
||||
-- still agree on the day multiplicity appears.
|
||||
ORDER BY (transaction_id = $1) DESC NULLS LAST, id
|
||||
LIMIT 1`,
|
||||
[transactionId]
|
||||
);
|
||||
@@ -157,9 +167,24 @@ export async function merchantVerdict(
|
||||
FROM order_reviews r
|
||||
JOIN transactions t ON t.id = r.transaction_id
|
||||
JOIN participants p ON p.id = r.participant_id
|
||||
JOIN expense_metadata em
|
||||
ON em.transaction_id = r.transaction_id
|
||||
OR em.matched_transaction_id = r.transaction_id
|
||||
-- (No backticks in here: this SQL lives in a TS template literal and a
|
||||
-- backtick ends the string -- TS1005 on a line that looks like a comment.)
|
||||
-- LATERAL, not a plain JOIN. A plain join on
|
||||
-- transaction_id OR matched_transaction_id fans out: one review
|
||||
-- joined to N metadata rows appears N times in the result, and the loops
|
||||
-- below count every appearance -- inflating the loved/never tallies
|
||||
-- that drive the pre-order warning. LIMIT 50 applies to the fanned-out
|
||||
-- set too, so duplicates would also push older reviews out of the
|
||||
-- window. Same tie-break as every other reader, so the merchant matched
|
||||
-- here is the merchant the UI shows.
|
||||
JOIN LATERAL (
|
||||
SELECT em2.merchant_normalized
|
||||
FROM expense_metadata em2
|
||||
WHERE em2.transaction_id = r.transaction_id
|
||||
OR em2.matched_transaction_id = r.transaction_id
|
||||
ORDER BY (em2.transaction_id = r.transaction_id) DESC NULLS LAST, em2.id
|
||||
LIMIT 1
|
||||
) em ON true
|
||||
-- Case-folded: the platforms capitalise the same restaurant differently
|
||||
-- ("TEG Kebabs & Biryani" on Uber Eats, "TEG KEBABS & BIRYANI" on
|
||||
-- DoorDash). An exact match split one restaurant's history in two, so a
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
/**
|
||||
* Partial-month comparison rules (Priority 0, item 6).
|
||||
*
|
||||
* A month in progress is not comparable to a complete one, and getting that
|
||||
* wrong is silent: nothing fails, the current month simply looks thrifty
|
||||
* because most of it has not happened yet. Two distinct rules follow from it,
|
||||
* and both are applied in three places on the analytics page:
|
||||
*
|
||||
* 1. **The in-progress month is never part of a baseline.** Averaging it in
|
||||
* drags the "typical" figure down by however much of the month is still in
|
||||
* the future.
|
||||
* 2. **When the in-progress month is the one being looked at, every month it
|
||||
* is compared against is truncated to the same day.** Its running total is
|
||||
* measured against the same slice of each prior month, not their full
|
||||
* totals.
|
||||
*
|
||||
* This module exists so those rules can be tested. They previously lived as
|
||||
* closures inside `src/app/budget/page.tsx` reading `new Date()` directly, so
|
||||
* "is this month in progress" depended on the wall clock at run time and no
|
||||
* test could assert either rule without waiting for the calendar. Every
|
||||
* function here takes `now` explicitly, defaulting to the real clock, which is
|
||||
* the whole reason the behaviour is now guardable.
|
||||
*
|
||||
* Pair with `/api/analytics/daily`: rule 2 needs day-of-month granularity,
|
||||
* which is why that route exists.
|
||||
*/
|
||||
|
||||
/** `YYYY-MM` for a Date, in local time. */
|
||||
export function monthStrOf(d: Date): string {
|
||||
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
/** The month currently in progress. */
|
||||
export function currentMonthStr(now: Date = new Date()): string {
|
||||
return monthStrOf(now);
|
||||
}
|
||||
|
||||
/** The calendar month before `m`, as `YYYY-MM`. */
|
||||
export function prevMonth(m: string): string {
|
||||
const [year, month] = m.split("-").map(Number);
|
||||
return monthStrOf(new Date(year, month - 2, 1));
|
||||
}
|
||||
|
||||
/** Total days in month `m`. */
|
||||
export function daysInMonthOf(m: string): number {
|
||||
const [year, month] = m.split("-").map(Number);
|
||||
return new Date(year, month, 0).getDate();
|
||||
}
|
||||
|
||||
/**
|
||||
* How much of `m` has actually happened. A month in progress is complete only
|
||||
* up to today; every earlier month is complete.
|
||||
*/
|
||||
export function elapsedDays(m: string, now: Date = new Date()): number {
|
||||
return m === currentMonthStr(now) ? now.getDate() : daysInMonthOf(m);
|
||||
}
|
||||
|
||||
/** Spend in a month from day 1 through `throughDay` inclusive. */
|
||||
export function spendThrough(
|
||||
days: Record<number, number> | undefined,
|
||||
throughDay: number
|
||||
): number {
|
||||
if (!days) return 0;
|
||||
let sum = 0;
|
||||
for (const [d, v] of Object.entries(days)) {
|
||||
if (Number(d) <= throughDay) sum += v;
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether `selectedMonth` is still in progress, and the day both sides of a
|
||||
* comparison must be truncated to.
|
||||
*
|
||||
* For a complete month the day is its last, which makes `spendThrough` a no-op
|
||||
* — so callers can use the same code path either way.
|
||||
*/
|
||||
export function comparisonBasis(
|
||||
selectedMonth: string,
|
||||
now: Date = new Date()
|
||||
): { isPartial: boolean; compareDay: number } {
|
||||
return {
|
||||
isPartial: selectedMonth === currentMonthStr(now),
|
||||
compareDay: elapsedDays(selectedMonth, now),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The months that may form a baseline for `selectedMonth`.
|
||||
*
|
||||
* Excludes the selected month itself, the month in progress (rule 1), and any
|
||||
* month with no spend recorded — an empty month is absence of data, not a
|
||||
* thrifty month, and averaging zeroes in understates the baseline.
|
||||
*/
|
||||
export function baselineMonths(
|
||||
months: readonly string[],
|
||||
selectedMonth: string,
|
||||
totals: Record<string, { spent: number }>,
|
||||
now: Date = new Date()
|
||||
): string[] {
|
||||
const current = currentMonthStr(now);
|
||||
return months.filter(
|
||||
(m) => m !== selectedMonth && m !== current && (totals[m]?.spent || 0) > 0
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mean spend across `baseline`, truncated to `compareDay` when the selected
|
||||
* month is partial.
|
||||
*
|
||||
* `daily` is the sparse `{ [month]: { [day]: amount } }` map from
|
||||
* `/api/analytics/daily`; `totals` supplies whole-month figures for the
|
||||
* complete-month case.
|
||||
*/
|
||||
export function baselineAverage(
|
||||
baseline: readonly string[],
|
||||
{
|
||||
isPartial,
|
||||
compareDay,
|
||||
daily,
|
||||
totals,
|
||||
}: {
|
||||
isPartial: boolean;
|
||||
compareDay: number;
|
||||
daily: Record<string, Record<number, number>> | undefined;
|
||||
totals: Record<string, { spent: number }>;
|
||||
}
|
||||
): number {
|
||||
if (!baseline.length) return 0;
|
||||
const sum = baseline.reduce(
|
||||
(s, m) => s + (isPartial ? spendThrough(daily?.[m], compareDay) : totals[m]?.spent || 0),
|
||||
0
|
||||
);
|
||||
return sum / baseline.length;
|
||||
}
|
||||
+52
-1
@@ -367,6 +367,16 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
|
||||
SELECT em.route, em.platform
|
||||
FROM expense_metadata em
|
||||
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||
-- Ordered the same way as replaceLineItems (line-items.ts) and
|
||||
-- matchTransaction (intake-router.ts), so every reader of this row
|
||||
-- resolves the same one a write lands on. NULLS LAST because DESC is
|
||||
-- NULLS FIRST and the row that merely POINTS at the transaction has a
|
||||
-- NULL transaction_id.
|
||||
-- Measured 2026-08-21: 667 expense_metadata rows, 147 of them
|
||||
-- pointing-only, and ZERO transactions resolving to more than one row by
|
||||
-- either link — so this changes nothing today. It is here so the readers
|
||||
-- still agree on the day multiplicity appears.
|
||||
ORDER BY (em.transaction_id = t.id) DESC NULLS LAST, em.id
|
||||
LIMIT 1
|
||||
) order_ctx ON true
|
||||
LEFT JOIN LATERAL (
|
||||
@@ -841,6 +851,15 @@ export async function getPendingReconciliations(ownerId: number): Promise<Manual
|
||||
WHERE ts.transaction_id = t.id
|
||||
) txn_splits ON true
|
||||
WHERE t.statement_id IS NULL AND t.owner_id = $1 AND t.reconciled_with_id IS NULL
|
||||
-- A superseded row is already resolved, just not by reconciling. Without
|
||||
-- this the queue kept offering it: a duplicate wallet half hidden under
|
||||
-- its twin (txn 7037, 2026-10-03) sat beside that twin with the same
|
||||
-- statement line as its match, and accepting it on the hidden row would
|
||||
-- have used the line up and stranded the real one.
|
||||
AND t.superseded_by_id IS NULL
|
||||
-- A voided row is resolved too: the charge never posted, so no statement
|
||||
-- line will ever arrive for it (migration 0040).
|
||||
AND t.voided_at IS NULL
|
||||
AND ${needsCardMatch("t")}
|
||||
ORDER BY t.transaction_date DESC, t.row_index ASC`,
|
||||
[ownerId]
|
||||
@@ -877,12 +896,39 @@ export async function getPendingReconciliations(ownerId: number): Promise<Manual
|
||||
FROM transactions m
|
||||
JOIN transactions t ON t.statement_id IS NOT NULL
|
||||
AND t.transaction_date BETWEEN m.transaction_date - 3 AND m.transaction_date + 3
|
||||
AND t.amount BETWEEN m.amount * 0.99 AND m.amount * 1.01
|
||||
-- +-5%, NOT +-1%. A foreign card purchase is charged at the notified
|
||||
-- amount PLUS the card's foreign transaction fee (3% on the Westpac
|
||||
-- Altitude), so the wallet row and its statement line can never match
|
||||
-- exactly and the old +-1% band excluded the statement row outright:
|
||||
-- SP PEAKDESIGN was notified at $42.44 and charged $43.71 (x1.03),
|
||||
-- so the one row that WAS its twin was the one the picker could not
|
||||
-- offer. Widening is safe here because this list is capped at 5 by
|
||||
-- the rn window and ordered by amount closeness; a looser band changes
|
||||
-- which five appear, never how many, and a person confirms each one.
|
||||
-- The AUTO-matcher stays exact on purpose; see findAutoMatches().
|
||||
AND (t.amount BETWEEN m.amount * 0.95 AND m.amount * 1.05
|
||||
-- Or the exact native pair, whatever the AUD figures say. A
|
||||
-- wallet row's AUD is provisional: in Oct 2026 it priced INR at
|
||||
-- 0.0180 while CommBank charged ~0.0149, 20% under, so the band
|
||||
-- excluded every rupee twin. findAutoMatches() matches on this
|
||||
-- pair already, but refuses when two lines share it inside its
|
||||
-- window (INR 112 on 10/4 and again on 10/7) -- and those are
|
||||
-- exactly the rows that land here for a person to choose.
|
||||
OR (m.foreign_currency_code IS NOT NULL
|
||||
AND t.foreign_currency_code = m.foreign_currency_code
|
||||
AND t.foreign_currency_amount = m.foreign_currency_amount))
|
||||
JOIN statements s ON s.id = t.statement_id
|
||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||
WHERE m.statement_id IS NULL
|
||||
AND m.owner_id = $1
|
||||
AND m.reconciled_with_id IS NULL
|
||||
AND m.superseded_by_id IS NULL
|
||||
AND m.voided_at IS NULL
|
||||
-- Nor offer a superseded STATEMENT row as the match: the ANZ
|
||||
-- statements 107/142/143 left 31 of those, and a manual row
|
||||
-- reconciled to one inherits its exclusion from every figure.
|
||||
-- findAutoMatches() has had both filters all along.
|
||||
AND t.superseded_by_id IS NULL
|
||||
AND ${needsCardMatch("m")}
|
||||
AND COALESCE(t.owner_id, s.owner_id) = $1
|
||||
AND NOT EXISTS (
|
||||
@@ -980,6 +1026,11 @@ export async function getSharedTransactions(ownerId: number, tagIds?: number[],
|
||||
SELECT em.platform
|
||||
FROM expense_metadata em
|
||||
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
|
||||
-- Same tie-break as getTransactions / replaceLineItems / matchTransaction,
|
||||
-- so every reader resolves the row a write lands on. Without the ORDER BY
|
||||
-- this LIMIT 1 is bounded but arbitrary: it would disagree with the main
|
||||
-- transaction list for the same transaction the day multiplicity appears.
|
||||
ORDER BY (em.transaction_id = t.id) DESC NULLS LAST, em.id
|
||||
LIMIT 1
|
||||
) order_ctx ON true
|
||||
LEFT JOIN transactions src ON src.reconciled_with_id = t.id AND src.statement_id IS NULL
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { queryRaw, queryRow } from "@/lib/db";
|
||||
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
|
||||
import { findAdoptableTransaction } from "@/lib/auto-reconcile";
|
||||
|
||||
/**
|
||||
* Grocery receipts scanned in pantry-app, arriving as candidate spend.
|
||||
@@ -154,16 +155,75 @@ export async function processReceiptIngestion(receipt: ParsedReceipt): Promise<R
|
||||
const paymentMethod = paymentMethodFor(leg.class);
|
||||
const amount = round2(Number(leg.amount));
|
||||
const description = receipt.store_detail ? `${receipt.merchant_name} ${receipt.store_detail}` : receipt.merchant_name;
|
||||
const txn = await queryRow<{ id: number }>(
|
||||
`INSERT INTO transactions (
|
||||
transaction_date, description, amount, amount_aud, category, payment_method,
|
||||
merchant_name, merchant_normalized, transaction_type, owner_id
|
||||
) VALUES ($1,$2,$3,$3,$4,$5,$6,$6,'debit',$7) RETURNING id`,
|
||||
[receipt.transaction_date, description, amount, RECEIPT_CATEGORY, paymentMethod, receipt.merchant_name, DEFAULT_OWNER_ID]
|
||||
);
|
||||
|
||||
// Adopt the row the wallet lane already created for this tap, if there is
|
||||
// one. Signals for a purchase arrive in sequence -- the notification within
|
||||
// seconds, the receipt hours later -- so creating a second transaction here
|
||||
// is how one shop ends up as two rows. Measured on txn 6858: the wallet
|
||||
// event arrived 35h25m before this path created its own row for the same
|
||||
// $103.30 Woolworths shop. Null means no unambiguous candidate, and the
|
||||
// fallback is exactly the old behaviour.
|
||||
const adoptedId = await findAdoptableTransaction(DEFAULT_OWNER_ID, amount, receipt.transaction_date);
|
||||
const txn = adoptedId
|
||||
? await queryRow<{ id: number }>(
|
||||
// The receipt read the register tape; the notification read a push
|
||||
// banner. So the receipt wins on what was bought and where, and the
|
||||
// column list here IS the ownership boundary -- `location` is absent
|
||||
// because the wallet notification's suburb is the better value and
|
||||
// must survive.
|
||||
`UPDATE transactions
|
||||
SET description = $2, merchant_name = $3, merchant_normalized = $3,
|
||||
category = $4, payment_method = $5
|
||||
WHERE id = $1 RETURNING id`,
|
||||
[adoptedId, description, receipt.merchant_name, RECEIPT_CATEGORY, paymentMethod]
|
||||
)
|
||||
: await queryRow<{ id: number }>(
|
||||
`INSERT INTO transactions (
|
||||
transaction_date, description, amount, amount_aud, category, payment_method,
|
||||
merchant_name, merchant_normalized, transaction_type, owner_id
|
||||
) VALUES ($1,$2,$3,$3,$4,$5,$6,$6,'debit',$7) RETURNING id`,
|
||||
[receipt.transaction_date, description, amount, RECEIPT_CATEGORY, paymentMethod, receipt.merchant_name, DEFAULT_OWNER_ID]
|
||||
);
|
||||
|
||||
const carriesLines = leg.leg_index === cardLeg?.leg_index;
|
||||
const meta = await queryRow<{ id: number }>(
|
||||
|
||||
// An adopted row may already carry a metadata row (the wallet lane writes
|
||||
// one). expense_metadata.transaction_id is UNIQUE, so a second INSERT would
|
||||
// raise -- and must not, because the two describe the same purchase.
|
||||
// Enrich in place instead, with the same ownership split as the transaction
|
||||
// update above: everything the receipt read wins, and anything only the
|
||||
// notification knew (issuer, and the transaction's location) is absent from
|
||||
// the column list rather than conditionally skipped, so it cannot be
|
||||
// clobbered by a bug here. Mirrors order_transaction_bridge's UPGRADE pass.
|
||||
const priorMeta = adoptedId
|
||||
? await queryRow<{ id: number }>(
|
||||
`SELECT id FROM expense_metadata WHERE transaction_id = $1`, [adoptedId]
|
||||
)
|
||||
: null;
|
||||
|
||||
const meta = priorMeta
|
||||
? await queryRow<{ id: number }>(
|
||||
`UPDATE expense_metadata
|
||||
SET source = 'pantry', order_reference = $2, line_items = $3::jsonb,
|
||||
subtotal = $4, amount = $5, merchant_normalized = $6,
|
||||
transaction_date = $7, card_last4 = COALESCE($8, card_last4),
|
||||
currency = 'AUD', flags = flags || $9::jsonb,
|
||||
payment_method = $10, payment_method_detail = $11,
|
||||
tender_raw = COALESCE($12, tender_raw),
|
||||
receipt_sha256 = $13, receipt_group = $14,
|
||||
extraction_model = 'cloud-budget'
|
||||
WHERE id = $1 RETURNING id`,
|
||||
[
|
||||
priorMeta.id, reference,
|
||||
JSON.stringify(carriesLines ? receipt.line_items ?? [] : []),
|
||||
receipt.tax_amount ?? null, amount, receipt.merchant_name,
|
||||
receipt.transaction_date, leg.card_last4 ?? null,
|
||||
JSON.stringify(carriesLines ? [...flags, "enriched_prior_row"] : [...flags, "line_items_on_card_leg", "enriched_prior_row"]),
|
||||
paymentMethod, leg.card_product ?? null, receipt.tender_raw ?? null,
|
||||
receipt.image_sha256 ?? null, group,
|
||||
]
|
||||
)
|
||||
: await queryRow<{ id: number }>(
|
||||
`INSERT INTO expense_metadata (
|
||||
transaction_id, source, order_reference, line_items, subtotal, amount,
|
||||
merchant_normalized, transaction_date, card_last4, currency, flags,
|
||||
|
||||
@@ -0,0 +1,248 @@
|
||||
import { prisma } from "@/lib/db";
|
||||
|
||||
/**
|
||||
* Merge a statementless ("manual") transaction into its statement twin.
|
||||
*
|
||||
* Extracted from POST /api/transactions/reconcile so the manual queue and the
|
||||
* automatic pass cannot drift apart. There must be exactly one implementation
|
||||
* of this: it moves curation (overrides, tags, splits) and provenance
|
||||
* (expense_metadata) off a row that reconciliation is about to hide from every
|
||||
* figure, and a second copy of that logic would eventually move a different set.
|
||||
*
|
||||
* The manual row is NOT deleted -- it is linked via `reconciled_with_id`.
|
||||
* Deleting it would cascade its children away; superseding/linking keeps the
|
||||
* history and lets the row stay visible as provenance.
|
||||
*/
|
||||
export async function applyReconcileMatch(
|
||||
manual_id: number,
|
||||
statement_tx_id: number
|
||||
): Promise<void> {
|
||||
await prisma.$transaction(async (tx) => {
|
||||
await moveCurationWithin(tx, manual_id, statement_tx_id);
|
||||
|
||||
// Mark manual tx as reconciled (link to statement tx)
|
||||
await tx.$executeRawUnsafe(
|
||||
`UPDATE transactions SET reconciled_with_id = $1 WHERE id = $2`,
|
||||
statement_tx_id,
|
||||
manual_id
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Move overrides, tags, splits and expense_metadata from one transaction to
|
||||
* another, inside a caller-supplied transaction.
|
||||
*
|
||||
* Callers: `applyReconcileMatch` (then sets `reconciled_with_id`) and
|
||||
* `supersedeTransaction` (then sets `superseded_by_id`). Both hide the source
|
||||
* row from every figure, so both must carry its curation across first; keeping
|
||||
* one implementation is what the file header insists on.
|
||||
*/
|
||||
async function moveCurationWithin(
|
||||
tx: Parameters<Parameters<typeof prisma.$transaction>[0]>[0],
|
||||
manual_id: number,
|
||||
statement_tx_id: number
|
||||
): Promise<void> {
|
||||
{
|
||||
// Copy overrides: manual → statement tx
|
||||
const override = await tx.transaction_overrides.findUnique({
|
||||
where: { transaction_id: manual_id },
|
||||
});
|
||||
if (override) {
|
||||
await tx.transaction_overrides.upsert({
|
||||
where: { transaction_id: statement_tx_id },
|
||||
update: {
|
||||
category_override: override.category_override,
|
||||
merchant_normalized: override.merchant_normalized,
|
||||
notes: override.notes,
|
||||
my_share_percent: override.my_share_percent,
|
||||
updated_at: new Date(),
|
||||
},
|
||||
create: {
|
||||
transaction_id: statement_tx_id,
|
||||
category_override: override.category_override,
|
||||
merchant_normalized: override.merchant_normalized,
|
||||
notes: override.notes,
|
||||
my_share_percent: override.my_share_percent,
|
||||
},
|
||||
});
|
||||
await tx.transaction_overrides.deleteMany({ where: { transaction_id: manual_id } });
|
||||
}
|
||||
|
||||
// Move tags: manual → statement tx
|
||||
const tags = await tx.transaction_tags.findMany({ where: { transaction_id: manual_id } });
|
||||
if (tags.length) {
|
||||
await tx.transaction_tags.createMany({
|
||||
data: tags.map((t) => ({ transaction_id: statement_tx_id, tag_id: t.tag_id })),
|
||||
skipDuplicates: true,
|
||||
});
|
||||
await tx.transaction_tags.deleteMany({ where: { transaction_id: manual_id } });
|
||||
}
|
||||
|
||||
// Move splits: manual → statement tx.
|
||||
//
|
||||
// `settled` and `settled_at` MUST be carried. This is a delete-and-recreate,
|
||||
// and the column defaults to false — so omitting it silently marks every
|
||||
// settled split unsettled again and resurrects a debt that was already
|
||||
// paid. Nothing errors and nothing in the UI says the flag was reset; it
|
||||
// just reappears as owed. No queue row carries a settled split today, but
|
||||
// the auto-matcher runs this path unattended, so a latent version of this
|
||||
// would surface as money, not as an exception.
|
||||
const splits = await tx.transaction_splits.findMany({ where: { transaction_id: manual_id } });
|
||||
if (splits.length) {
|
||||
await tx.transaction_splits.createMany({
|
||||
data: splits.map((s) => ({
|
||||
transaction_id: statement_tx_id,
|
||||
participant_id: s.participant_id,
|
||||
share_percent: s.share_percent,
|
||||
settled: s.settled ?? false,
|
||||
settled_at: s.settled_at,
|
||||
})),
|
||||
skipDuplicates: true,
|
||||
});
|
||||
await tx.transaction_splits.deleteMany({ where: { transaction_id: manual_id } });
|
||||
}
|
||||
|
||||
// Move provenance: manual → statement tx.
|
||||
//
|
||||
// Overrides, tags and splits above were always carried across; expense_metadata was
|
||||
// the one child left behind, which did not matter while every metadata row came from
|
||||
// an email that had created its own transaction. It matters now: a scanned grocery
|
||||
// receipt puts its line items here, and reconciliation hides the manual row from
|
||||
// every figure — so without this the shop's contents disappear at exactly the moment
|
||||
// the statement line appears, and COLES 0556 MANOR LAKES stays as unreadable as it
|
||||
// was before the receipt was ever scanned.
|
||||
//
|
||||
// transaction_id is UNIQUE, so a statement row that already has metadata (an emailed
|
||||
// or Paperless copy got there first) keeps it. The manual row's metadata stays
|
||||
// attached to the reconciled manual transaction and is flagged, rather than raising a
|
||||
// constraint violation or silently overwriting the other source.
|
||||
const moved = await tx.$executeRawUnsafe(
|
||||
`UPDATE expense_metadata SET transaction_id = $1
|
||||
WHERE transaction_id = $2
|
||||
AND NOT EXISTS (SELECT 1 FROM expense_metadata other WHERE other.transaction_id = $1)`,
|
||||
statement_tx_id,
|
||||
manual_id
|
||||
);
|
||||
if (moved === 0) {
|
||||
await tx.$executeRawUnsafe(
|
||||
`UPDATE expense_metadata
|
||||
SET flags = coalesce(flags, '[]'::jsonb) || '["metadata_collision_on_reconcile"]'::jsonb
|
||||
WHERE transaction_id = $1`,
|
||||
manual_id
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Manually mark `loser_id` as superseded by `survivor_id`.
|
||||
*
|
||||
* The escape hatch for a duplicate that `applyReconcileMatch` cannot express:
|
||||
* that one merges a statementLESS row into its statement twin, so it has
|
||||
* nothing to say about two rows on the SAME footing -- the pair left by a
|
||||
* statement imported twice (migration 0023 superseded 31 such pairs, $42,040.68,
|
||||
* by hand), or two feed rows for one purchase.
|
||||
*
|
||||
* Curation moves FIRST, using the same block `applyReconcileMatch` runs, because
|
||||
* `superseded_by_id` hides the row from every figure (`analytics-sql.ts`) and
|
||||
* splits or tags left on it would silently stop counting. That is why this lives
|
||||
* beside its sibling and calls into the same movement rather than re-deriving
|
||||
* it: the header of this file warns that a second copy would eventually move a
|
||||
* different set, and a supersede that forgot splits would resurrect a debt in
|
||||
* exactly the way `applyReconcileMatch`'s own comment describes.
|
||||
*
|
||||
* NOT a delete. Every child of `transactions` is ON DELETE CASCADE, so removing
|
||||
* the row would destroy the splits, tags and metadata that make it legible --
|
||||
* and a duplicate is evidence of how the ledger was assembled, which is worth
|
||||
* keeping. `transactions_no_self_supersede` blocks a row pointing at itself; the
|
||||
* survivor is re-checked here anyway so the error is a 400 and not a 500.
|
||||
*/
|
||||
export async function supersedeTransaction(
|
||||
loser_id: number,
|
||||
survivor_id: number
|
||||
): Promise<void> {
|
||||
if (loser_id === survivor_id) throw new Error("A transaction cannot supersede itself");
|
||||
await prisma.$transaction(async (tx) => {
|
||||
// Refuse to bury a row underneath one that is itself already buried, which
|
||||
// would make the curation we just moved unreachable in a second hop.
|
||||
const survivor = await tx.transactions.findUnique({
|
||||
where: { id: survivor_id },
|
||||
select: { superseded_by_id: true },
|
||||
});
|
||||
if (!survivor) throw new Error(`Survivor transaction ${survivor_id} not found`);
|
||||
if (survivor.superseded_by_id !== null) {
|
||||
throw new Error(`Transaction ${survivor_id} is itself superseded; pick the surviving row`);
|
||||
}
|
||||
await moveCurationWithin(tx, loser_id, survivor_id);
|
||||
await tx.$executeRawUnsafe(
|
||||
`UPDATE transactions SET superseded_by_id = $1 WHERE id = $2`,
|
||||
survivor_id,
|
||||
loser_id
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark a statementless row as a charge that never posted (migration 0040).
|
||||
*
|
||||
* The case neither sibling above can express: a card notification is sent for
|
||||
* an AUTHORISATION, and when the merchant releases the hold no statement line
|
||||
* ever arrives. Txn 7951 was a $1,040.41 Airbnb hold on a booking cancelled the
|
||||
* next day -- it sat in the reconcile queue with nothing to match and counted as
|
||||
* spend in every figure. There is no survivor to move curation onto, so nothing
|
||||
* moves: the row and its children stay, and `EXCLUDE_RECONCILED_SOURCE` stops
|
||||
* them counting.
|
||||
*
|
||||
* Refusals, all deliberate:
|
||||
* - a statement row: that is the bank's record of a posted charge, and the
|
||||
* statement must keep summing to its header (`transactions_void_statementless`
|
||||
* enforces this in the database too). A wrong statement row is a supersede.
|
||||
* - a row already reconciled or superseded: it is already out of every figure,
|
||||
* and a reconciled one evidently DID post.
|
||||
* - a row with a SETTLED split: someone has already paid their share of it.
|
||||
* Voiding would silently turn that payment into a credit on their balance;
|
||||
* that needs a person to look at it, not a side effect.
|
||||
*/
|
||||
export async function voidTransaction(id: number, reason: string): Promise<void> {
|
||||
const why = reason.trim();
|
||||
if (!why) throw new Error("A void needs a reason");
|
||||
await prisma.$transaction(async (tx) => {
|
||||
const rows = await tx.$queryRawUnsafe<
|
||||
{ statement_id: number | null; reconciled_with_id: number | null; superseded_by_id: number | null; settled: number }[]
|
||||
>(
|
||||
`SELECT t.statement_id, t.reconciled_with_id, t.superseded_by_id,
|
||||
(SELECT count(*)::int FROM transaction_splits ts
|
||||
WHERE ts.transaction_id = t.id AND ts.settled) AS settled
|
||||
FROM transactions t WHERE t.id = $1 FOR UPDATE`,
|
||||
id
|
||||
);
|
||||
const row = rows[0];
|
||||
if (!row) throw new Error(`Transaction ${id} not found`);
|
||||
if (row.statement_id !== null) {
|
||||
throw new Error(`Transaction ${id} is a statement line; a posted charge cannot be voided (supersede it instead)`);
|
||||
}
|
||||
if (row.reconciled_with_id !== null) {
|
||||
throw new Error(`Transaction ${id} is reconciled with ${row.reconciled_with_id}, so it posted`);
|
||||
}
|
||||
if (row.superseded_by_id !== null) {
|
||||
throw new Error(`Transaction ${id} is already superseded by ${row.superseded_by_id}`);
|
||||
}
|
||||
if (row.settled > 0) {
|
||||
throw new Error(`Transaction ${id} has ${row.settled} settled split(s); resolve those before voiding`);
|
||||
}
|
||||
await tx.$executeRawUnsafe(
|
||||
`UPDATE transactions SET voided_at = now(), void_reason = $1 WHERE id = $2 AND voided_at IS NULL`,
|
||||
why,
|
||||
id
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/** Undo `voidTransaction` -- e.g. the "released" hold posted after all. */
|
||||
export async function unvoidTransaction(id: number): Promise<void> {
|
||||
await prisma.$executeRawUnsafe(
|
||||
`UPDATE transactions SET voided_at = NULL, void_reason = NULL WHERE id = $1`,
|
||||
id
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
/**
|
||||
* Deterministic parsing of the bits of a wallet notification that have a fixed
|
||||
* shape.
|
||||
*
|
||||
* Everything here was previously left to the interpreting model, which handled
|
||||
* ONE fixed-width format three different ways on the same day's data: it kept
|
||||
* the country code on "WYNDHAM VAL AU", stripped it on "Sydney" and "Perth",
|
||||
* and swallowed "SAN FRANCISOUS" whole into the location field. A fixed-width
|
||||
* field is not a judgement call, and the same argument that put the order
|
||||
* parsers in code rather than in a Gemini call applies: a regex fails loudly on
|
||||
* a template change, where a model returns a plausible wrong answer.
|
||||
*
|
||||
* The model still decides what the amount and merchant ARE -- it is better at
|
||||
* that than a pattern, because five cards emit three shapes. This only handles
|
||||
* the parts that are positional.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Westpac writes the merchant/city/country block as a fixed-width statement
|
||||
* descriptor: 27-char merchant, 12-char city, 2-char ISO country. Verified
|
||||
* across every Westpac event captured:
|
||||
*
|
||||
* Breadworld Bakery Cafe WYNDHAM VAL AU
|
||||
* PLA DONATION MELBOURNE AU
|
||||
* SP PEAKDESIGN SAN FRANCISOUS <- "SAN FRANCISO" + "US"
|
||||
*
|
||||
* The last example is why this is positional rather than a split on whitespace:
|
||||
* a 12-character city fills its field exactly and runs straight into the
|
||||
* country code with no separator.
|
||||
*/
|
||||
export type WestpacDescriptor = {
|
||||
merchant: string | null;
|
||||
city: string | null;
|
||||
/** ISO-3166 alpha-2, upper-case. Null when the tail is not a country code. */
|
||||
country: string | null;
|
||||
};
|
||||
|
||||
const MERCHANT_WIDTH = 27;
|
||||
const CITY_WIDTH = 12;
|
||||
|
||||
/**
|
||||
* `body` is the whole notification body, e.g.
|
||||
* "$13.50 paid from Mastercard ending...8032. Breadworld Bakery Cafe WYNDHAM VAL AU"
|
||||
* Returns nulls rather than throwing: a template change must degrade to "we
|
||||
* don't know", never to a wrong country on a foreign transaction.
|
||||
*/
|
||||
export function parseWestpacDescriptor(body: string | null | undefined): WestpacDescriptor {
|
||||
const none: WestpacDescriptor = { merchant: null, city: null, country: null };
|
||||
if (!body) return none;
|
||||
|
||||
// The descriptor is whatever follows the first sentence ("... ending...8032. ").
|
||||
// [\s\S] rather than the `s` flag: the tsconfig target predates es2018.
|
||||
const m = body.match(/^[\s\S]*?\.\s([\s\S]*)$/);
|
||||
if (!m) return none;
|
||||
const tail = m[1].replace(/\s+$/, "");
|
||||
if (!tail) return none;
|
||||
|
||||
const merchant = tail.slice(0, MERCHANT_WIDTH).trim() || null;
|
||||
const rest = tail.slice(MERCHANT_WIDTH);
|
||||
if (!rest) return { merchant, city: null, country: null };
|
||||
|
||||
// Country is the final two characters of the fixed block, but only when the
|
||||
// block is actually long enough to contain one -- a short tail is a merchant
|
||||
// name with no location, not a merchant whose last two letters are a country.
|
||||
const block = rest.slice(0, CITY_WIDTH + 2);
|
||||
if (block.length < CITY_WIDTH + 2) {
|
||||
return { merchant, city: block.trim() || null, country: null };
|
||||
}
|
||||
const city = block.slice(0, CITY_WIDTH).trim() || null;
|
||||
const tailTwo = block.slice(CITY_WIDTH).trim().toUpperCase();
|
||||
const country = /^[A-Z]{2}$/.test(tailTwo) ? tailTwo : null;
|
||||
return { merchant, city, country: country };
|
||||
}
|
||||
|
||||
/**
|
||||
* The Apple Wallet notification's subtitle, which is comma-delimited and clean:
|
||||
* "Breadworld Bakery Cafe, Wyndham Vale, VIC" -> location "Wyndham Vale, VIC"
|
||||
* "Seatsaero" -> no location (online merchant)
|
||||
*
|
||||
* Only the first comma separates merchant from place; the rest is the place.
|
||||
*/
|
||||
export function parseWalletSubtitle(subtitle: string | null | undefined): {
|
||||
merchant: string | null;
|
||||
location: string | null;
|
||||
} {
|
||||
if (!subtitle || !subtitle.trim()) return { merchant: null, location: null };
|
||||
const parts = subtitle.split(",").map((p) => p.trim()).filter(Boolean);
|
||||
if (parts.length <= 1) return { merchant: parts[0] ?? null, location: null };
|
||||
return { merchant: parts[0], location: parts.slice(1).join(", ") };
|
||||
}
|
||||
|
||||
/**
|
||||
* Currency marker, if the notification states one.
|
||||
*
|
||||
* Only Amex does: it writes "A$5,103.49". Westpac, CommBank and Apple Wallet
|
||||
* write a bare "$", which is honestly recorded as unknown rather than assumed
|
||||
* to be AUD -- 14 of 21 captured transaction rows. Domestically that assumption
|
||||
* would be harmless; abroad it is a 23x error on a THB tap, so it is not made
|
||||
* here. The statement settles it within weeks, and the reconcile queue holds
|
||||
* the row until then.
|
||||
*/
|
||||
export function currencyFromAmountText(amountText: string | null | undefined): string | null {
|
||||
if (!amountText) return null;
|
||||
const t = amountText.trim();
|
||||
if (/^A\$/i.test(t)) return "AUD";
|
||||
if (/^(NZ\$)/i.test(t)) return "NZD";
|
||||
if (/^US\$/i.test(t)) return "USD";
|
||||
if (/^€/.test(t)) return "EUR";
|
||||
if (/^£/.test(t)) return "GBP";
|
||||
if (/^¥/.test(t)) return "JPY";
|
||||
if (/^฿/.test(t)) return "THB";
|
||||
if (/^S\$/i.test(t)) return "SGD";
|
||||
if (/^₹/.test(t)) return "INR";
|
||||
// CommBank's approval prompts write the ISO code after the figure
|
||||
// ("672006 VND", "1340.00 INR"); some issuers put it first ("USD 10.00").
|
||||
const iso = t.match(/^[\d.,\s]+([A-Z]{3})$/) ?? t.match(/^([A-Z]{3})\s*[\d.,]+$/);
|
||||
if (iso) return iso[1];
|
||||
// A bare "$" is genuinely ambiguous -- do not guess.
|
||||
return null;
|
||||
}
|
||||
@@ -0,0 +1,481 @@
|
||||
import { queryRow, withTransaction, type Queryable } from "@/lib/db";
|
||||
import { DEFAULT_OWNER_ID } from "@/lib/order-ingestion";
|
||||
import { findWalletAdoptableTransaction } from "@/lib/auto-reconcile";
|
||||
import {
|
||||
parseWestpacDescriptor,
|
||||
parseWalletSubtitle,
|
||||
currencyFromAmountText,
|
||||
} from "@/lib/wallet-descriptor";
|
||||
|
||||
/**
|
||||
* Wallet notifications -> transactions.
|
||||
*
|
||||
* A transaction is created for EVERY tap (DECISIONS.md ING-11, 2026-08-22),
|
||||
* because overseas trips are where live spend matters most: the statement lag
|
||||
* is the same 6-35 days abroad but knowing what you have spent matters far more
|
||||
* in an unfamiliar currency.
|
||||
*
|
||||
* SEQUENCING IS THE WHOLE DESIGN. Wallet is the FIRST signal, not a late one --
|
||||
* measured on txn 6858, the notification arrived 35h25m before anything created
|
||||
* a transaction. So this creates the row, the receipt enriches it
|
||||
* (receipt-ingestion.ts adopts it), and the statement confirms it via the
|
||||
* reconcile queue, which a wallet row enters unaided: it is statementless with
|
||||
* payment_method 'card'. Nothing here needs to know about statements.
|
||||
*
|
||||
* But that sequence is a TENDENCY, not a guarantee -- these first 21 rows were
|
||||
* themselves backfilled days after the taps -- so this also adopts a row a
|
||||
* receipt got to first (findWalletAdoptableTransaction), and a stored-value
|
||||
* instrument is written as 'credits' so it never enters a queue no statement
|
||||
* line will ever resolve.
|
||||
*/
|
||||
|
||||
export type WalletEvent = {
|
||||
/** capture.wallet_events.id -- the idempotency key. */
|
||||
event_id: number;
|
||||
/** 'transaction' | 'not_transaction' | 'unsure' from the interpreter. */
|
||||
kind: string;
|
||||
direction?: string | null;
|
||||
amount: number | null;
|
||||
/** The exact substring the model read the amount from, e.g. "A$5,103.49". */
|
||||
amount_text?: string | null;
|
||||
currency?: string | null;
|
||||
merchant?: string | null;
|
||||
card_tail?: string | null;
|
||||
issuer?: string | null;
|
||||
location?: string | null;
|
||||
notif_app?: string | null;
|
||||
/** The notification's title line, e.g. "Visa Gift Card" or "Payment". */
|
||||
notif_title?: string | null;
|
||||
notif_body?: string | null;
|
||||
notif_subtitle?: string | null;
|
||||
occurred_at?: string | null;
|
||||
};
|
||||
|
||||
export type WalletIngestResult = {
|
||||
transactionId: number | null;
|
||||
metadataId: number | null;
|
||||
skipped?:
|
||||
| "not_a_transaction" | "already_ingested" | "no_amount" | "merged_pair" | "adopted"
|
||||
| "account_notice" | "stored_value_drawdown";
|
||||
flags: string[];
|
||||
};
|
||||
|
||||
/**
|
||||
* How far apart the two halves of ONE tap can arrive. Measured 2-8 seconds on
|
||||
* every captured pair; ten minutes leaves room for a phone that was offline,
|
||||
* and is still nowhere near the +/-1 DAY the matcher used to allow -- which
|
||||
* folded the UNICEF $50 donation and an unrelated $50 bar charge 17 hours later
|
||||
* into one row (event 275 into txn 8091, 2026-10-01).
|
||||
*/
|
||||
const PAIR_WINDOW_MINUTES = 10;
|
||||
|
||||
/** The ledger is dated the way the bank statements are: Melbourne local. */
|
||||
const LEDGER_TZ = "Australia/Melbourne";
|
||||
|
||||
/**
|
||||
* The calendar date of an instant, in LEDGER_TZ.
|
||||
*
|
||||
* Slicing the ISO string took whatever offset n8n serialised -- UTC in
|
||||
* practice -- so a Melbourne-evening tap landed on the previous day (28 of the
|
||||
* first 134 rows). Statements, which this row is reconciled against, date in
|
||||
* Australian local time.
|
||||
*/
|
||||
export function ledgerDate(occurredAt: string | null | undefined, now: Date = new Date()): string {
|
||||
const d = occurredAt ? new Date(occurredAt) : now;
|
||||
const at = Number.isNaN(d.getTime()) ? now : d;
|
||||
// en-CA formats as YYYY-MM-DD.
|
||||
return new Intl.DateTimeFormat("en-CA", {
|
||||
timeZone: LEDGER_TZ, year: "numeric", month: "2-digit", day: "2-digit",
|
||||
}).format(at);
|
||||
}
|
||||
|
||||
/**
|
||||
* A ledger-derived AUD rate for a currency: the median of what the bank
|
||||
* actually charged divided by the native figure, over statement rows.
|
||||
*
|
||||
* Same source order_transaction_bridge uses for its plausibility guard. This is
|
||||
* only ever a PROVISIONAL value -- transactions.amount is NOT NULL and always
|
||||
* AUD, and a notification cannot know the AUD figure because the bank sets the
|
||||
* rate at settlement. The statement overwrites it at reconcile time.
|
||||
* Returns null when there is not enough history to be worth anything.
|
||||
*/
|
||||
async function medianRateToAud(code: string): Promise<number | null> {
|
||||
const row = await queryRow<{ rate: string | null }>(
|
||||
`SELECT percentile_cont(0.5) WITHIN GROUP (
|
||||
ORDER BY amount / NULLIF(foreign_currency_amount, 0)
|
||||
)::text AS rate
|
||||
FROM transactions
|
||||
WHERE foreign_currency_code = $1
|
||||
AND foreign_currency_amount > 0
|
||||
AND superseded_by_id IS NULL
|
||||
-- Statement rows only: a wallet row's amount is itself a provisional
|
||||
-- guess, and one stored before this filter (txn 7316, 672006 VND at a
|
||||
-- 1:1 "rate") would otherwise feed the next estimate.
|
||||
AND statement_id IS NOT NULL
|
||||
HAVING count(*) >= 5`,
|
||||
[code]
|
||||
);
|
||||
const rate = row?.rate ? Number(row.rate) : null;
|
||||
return rate && Number.isFinite(rate) && rate > 0 ? rate : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The ECB reference rate for the day, via Frankfurter (free, no key; the same
|
||||
* source the statement processor converts with).
|
||||
*
|
||||
* The fallback when the ledger has no history for a currency. Hong Kong was the
|
||||
* first trip to need it: no HKD statement row existed, so every HKD tap stored
|
||||
* A$0.00 and the trip read as free until the statement landed weeks later. The
|
||||
* ledger median is still preferred -- it carries the card's ~3% foreign fee,
|
||||
* which the ECB mid-rate does not -- so this one is flagged as such.
|
||||
*
|
||||
* Any failure is null, never a throw: no rate is a state this lane already
|
||||
* handles (a flagged 0), and a slow FX API must not lose the purchase.
|
||||
*/
|
||||
async function ecbRateToAud(code: string, date: string): Promise<number | null> {
|
||||
try {
|
||||
const res = await fetch(
|
||||
`https://api.frankfurter.dev/v1/${date}?from=${encodeURIComponent(code)}&to=AUD`,
|
||||
{ signal: AbortSignal.timeout(5000) }
|
||||
);
|
||||
if (!res.ok) return null;
|
||||
const body = (await res.json()) as { rates?: { AUD?: number } };
|
||||
const rate = body?.rates?.AUD;
|
||||
return typeof rate === "number" && Number.isFinite(rate) && rate > 0 ? rate : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The other half of a paired notification, if it has already been ingested.
|
||||
*
|
||||
* One tap emits TWO notifications ~2-8s apart: the issuer app (which has the
|
||||
* card tail) and Apple Wallet (which has the better merchant and the suburb).
|
||||
* Neither alone is complete, so they must be MERGED, never deduped -- dropping
|
||||
* either loses a field nothing else supplies. The notifications share no
|
||||
* identifier, so a pair is recognised by what a pair IS: the same amount,
|
||||
* minutes apart, from two different apps. Two notifications from one app are
|
||||
* two charges, however close together.
|
||||
*/
|
||||
async function findPairedRow(
|
||||
db: Queryable,
|
||||
amount: number,
|
||||
notifiedAt: string,
|
||||
notifApp: string | null,
|
||||
fx: { amount: number; code: string } | null = null
|
||||
) {
|
||||
// A foreign row pairs on its native pair: its `amount` is a provisional
|
||||
// conversion (or a 0 placeholder), so two unrelated foreign taps could share it.
|
||||
const rows = await db.queryRaw<{ id: number; meta_id: number }>(
|
||||
`SELECT t.id, em.id AS meta_id
|
||||
FROM transactions t
|
||||
JOIN expense_metadata em ON em.transaction_id = t.id
|
||||
WHERE t.source = 'wallet'
|
||||
AND t.statement_id IS NULL
|
||||
AND t.superseded_by_id IS NULL
|
||||
AND CASE WHEN $4::text IS NULL
|
||||
THEN t.amount = $1::numeric AND t.foreign_currency_code IS NULL
|
||||
ELSE t.foreign_currency_code = $4::text
|
||||
AND t.foreign_currency_amount = $5::numeric END
|
||||
AND em.notified_at BETWEEN $2::timestamptz - make_interval(mins => $3::int)
|
||||
AND $2::timestamptz + make_interval(mins => $3::int)
|
||||
AND em.notif_app IS DISTINCT FROM $6::text
|
||||
LIMIT 2`,
|
||||
[amount, notifiedAt, PAIR_WINDOW_MINUTES, fx?.code ?? null, fx?.amount ?? null, notifApp]
|
||||
);
|
||||
return rows.length === 1 ? rows[0] : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* A stored-value instrument: a gift card, a prepaid travel card, a store card.
|
||||
*
|
||||
* It matters because payment_method decides queue membership. 'card' means "a
|
||||
* statement line is coming"; findAutoMatches and the reconcile queue both
|
||||
* exclude 'credits' and 'cash' for exactly that reason. A $103.30 Woolworths
|
||||
* tap on a Visa gift card written as 'card' parks in the queue forever, being
|
||||
* offered as an unmatched row against statements that can never contain it --
|
||||
* three of the first 21 wallet rows (events 80, 92, 101) were gift-card taps.
|
||||
*
|
||||
* The evidence is the notification TITLE, which is where Apple Wallet prints
|
||||
* the instrument's own name ("Visa Gift Card"); the interpreter reduces that to
|
||||
* issuer "Visa", which a plain Visa debit card also produces, so the title is
|
||||
* the only place the distinction survives. Read as a prior, not a contract: a
|
||||
* miss leaves today's behaviour, which is the queue.
|
||||
*/
|
||||
const STORED_VALUE = /gift\s*card|prepaid|travel\s*(?:money|card)|store\s*card/i;
|
||||
|
||||
/**
|
||||
* A transit card reporting a drawdown of its own balance ("HKD 7.40 for Train"
|
||||
* / "Your new balance is HKD 169.50.").
|
||||
*
|
||||
* Unlike a gift card, a transit card is loaded FROM a bank card -- that load is
|
||||
* on the statement and is the spend. Each fare is that money being used, so
|
||||
* booking it again counts it twice. The interpreter already called 11 of 13
|
||||
* Octopus fares not_transaction; the two it did not (events 191, 200) became
|
||||
* rows, so the outcome depended on the model's mood. This makes it a rule. The
|
||||
* body wording is the signal because it is what every such card says; the
|
||||
* issuer name is not (Octopus, Opal, myki, Suica each name themselves).
|
||||
*/
|
||||
const BALANCE_DRAWDOWN = /\byour (?:new|current) balance is\b/i;
|
||||
|
||||
/**
|
||||
* Account notices that state an amount without money moving: "Your Smart Access
|
||||
* 1910 is overdrawn by $21.21. To avoid fees and interest, ..." (event 235)
|
||||
* became a $21.21 purchase named "To avoid fees and interest,". The model is
|
||||
* right that the text names an amount; it is not a payment.
|
||||
*/
|
||||
const ACCOUNT_NOTICE = /\boverdrawn\b|\bto avoid fees\b|\bavailable balance\b|\blow balance\b/i;
|
||||
|
||||
/** How an adopted or merged row records which wallet event folded into it. */
|
||||
const adoptedFlag = (eventId: number) => `wallet_evt:${eventId}`;
|
||||
|
||||
export function isStoredValueInstrument(ev: Pick<WalletEvent, "notif_title" | "issuer" | "notif_app">): boolean {
|
||||
return [ev.notif_title, ev.issuer, ev.notif_app].some((v) => !!v && STORED_VALUE.test(v));
|
||||
}
|
||||
|
||||
/** Already landed: a row this lane wrote, or one it adopted or merged into. */
|
||||
async function alreadyIngested(db: Pick<Queryable, "queryRow">, eventId: number) {
|
||||
return db.queryRow<{ id: number }>(
|
||||
`SELECT id FROM transactions WHERE source = 'wallet' AND source_ref = $1
|
||||
UNION ALL
|
||||
SELECT em.transaction_id AS id FROM expense_metadata em
|
||||
JOIN transactions t ON t.id = em.transaction_id
|
||||
WHERE em.flags @> $2::jsonb AND t.superseded_by_id IS NULL
|
||||
LIMIT 1`,
|
||||
[`evt:${eventId}`, JSON.stringify([adoptedFlag(eventId)])]
|
||||
);
|
||||
}
|
||||
|
||||
export async function processWalletIngestion(ev: WalletEvent): Promise<WalletIngestResult> {
|
||||
const flags: string[] = [];
|
||||
|
||||
// An open-world classifier feeding a closed-world writer drops the unknown
|
||||
// and calls it a refusal (ING-14). `unsure` is a real answer and must not
|
||||
// silently become a transaction.
|
||||
if (ev.kind !== "transaction") {
|
||||
return { transactionId: null, metadataId: null, skipped: "not_a_transaction", flags };
|
||||
}
|
||||
if (ev.amount === null || !Number.isFinite(ev.amount) || ev.amount <= 0) {
|
||||
return { transactionId: null, metadataId: null, skipped: "no_amount", flags };
|
||||
}
|
||||
const text = [ev.notif_subtitle, ev.notif_body].filter(Boolean).join("\n");
|
||||
if (ACCOUNT_NOTICE.test(text)) {
|
||||
return { transactionId: null, metadataId: null, skipped: "account_notice", flags };
|
||||
}
|
||||
if (BALANCE_DRAWDOWN.test(ev.notif_body ?? "")) {
|
||||
return { transactionId: null, metadataId: null, skipped: "stored_value_drawdown", flags };
|
||||
}
|
||||
|
||||
// Two places this event may already have landed: a row this lane wrote (which
|
||||
// carries the ref natively), or a row it ADOPTED or MERGED into, whose
|
||||
// source_ref belongs to whoever created it -- marked in the metadata flags
|
||||
// rather than by overwriting a field another lane owns. Both must be checked,
|
||||
// because bumping the interpreter's prompt_version deliberately re-interprets
|
||||
// and re-posts every event ever captured -- that is the replay mechanism, and
|
||||
// it must not mint a second row for a purchase that has one. Checked here
|
||||
// without the lock to skip the FX lookup on a replay, and again under it.
|
||||
const seen = await alreadyIngested({ queryRow }, ev.event_id);
|
||||
if (seen) return { transactionId: seen.id, metadataId: null, skipped: "already_ingested", flags };
|
||||
|
||||
const sourceRef = `evt:${ev.event_id}`;
|
||||
const notifiedAt = ev.occurred_at ?? new Date().toISOString();
|
||||
const date = ledgerDate(ev.occurred_at);
|
||||
|
||||
// Deterministic parsing of the positional fields. The model decides what the
|
||||
// amount and merchant ARE; it does not get to parse a fixed-width block.
|
||||
// Only Westpac writes that block: run over a CommBank overdraft notice it
|
||||
// produced the merchant "To avoid fees and interest," (event 235).
|
||||
const desc = /westpac/i.test(ev.notif_app ?? "")
|
||||
? parseWestpacDescriptor(ev.notif_body)
|
||||
: { merchant: null, city: null, country: null };
|
||||
const sub = parseWalletSubtitle(ev.notif_subtitle);
|
||||
const merchant = sub.merchant ?? ev.merchant ?? desc.merchant ?? "Card purchase";
|
||||
// Two sources for a place, and they are not equal. The Apple Wallet subtitle
|
||||
// gives "Wyndham Vale, VIC"; the issuer's fixed-width descriptor gives
|
||||
// "WYNDHAM VAL AU" -- a 12-character truncation in caps. The subtitle wins
|
||||
// outright, which matters for the pair merge below: whichever half arrives
|
||||
// second must be able to REPLACE a descriptor-derived place, or the result
|
||||
// depends on arrival order.
|
||||
const subtitleLocation = sub.location ?? null;
|
||||
// Precedence: the Wallet subtitle, then the DETERMINISTIC descriptor parse,
|
||||
// and only then the model's own answer. Getting this order wrong defeats the
|
||||
// point of parsing positionally at all -- the model returned the mangled
|
||||
// "SAN FRANCISOUS" and the padded "MELBOURNE AU" for exactly these rows,
|
||||
// which is what wallet-descriptor.ts exists to correct.
|
||||
const descriptorLocation = desc.city && desc.country
|
||||
? `${desc.city} ${desc.country}`
|
||||
: desc.city;
|
||||
const location = subtitleLocation ?? descriptorLocation ?? ev.location ?? null;
|
||||
|
||||
// Currency: only ever what the notification actually states. A bare "$" is
|
||||
// recorded as unknown, never assumed AUD -- harmless domestically, a 23x
|
||||
// error on a THB tap abroad.
|
||||
const stated = currencyFromAmountText(ev.amount_text)
|
||||
?? (ev.currency && /^[A-Z]{3}$/.test(ev.currency) ? ev.currency : null);
|
||||
if (!stated) flags.push("currency_unverified");
|
||||
if (desc.country && desc.country !== "AU") flags.push(`foreign_country_${desc.country}`);
|
||||
|
||||
// Foreign: the native figure is what both sides can agree on exactly, so it
|
||||
// is what gets stored. `amount` still needs a value (NOT NULL) and takes a
|
||||
// provisional conversion, flagged, which the statement overwrites. Resolved
|
||||
// BEFORE the lock: the ECB fallback is a network call, and the lock
|
||||
// serialises every wallet post behind whatever runs inside it.
|
||||
let amountAud = ev.amount;
|
||||
let fxAmount: number | null = null;
|
||||
let fxCode: string | null = null;
|
||||
if (stated && stated !== "AUD") {
|
||||
fxAmount = ev.amount;
|
||||
fxCode = stated;
|
||||
const ledgerRate = await medianRateToAud(stated);
|
||||
const rate = ledgerRate ?? (await ecbRateToAud(stated, date));
|
||||
if (rate) {
|
||||
amountAud = Number((ev.amount * rate).toFixed(2));
|
||||
flags.push("aud_provisional");
|
||||
if (!ledgerRate) flags.push("rate_ecb");
|
||||
} else {
|
||||
// No rate from either source. The native figure must NOT fall through as
|
||||
// AUD -- 672006 VND once landed as A$672,006. Zero is a flagged
|
||||
// placeholder; the statement supplies the real AUD at reconcile time,
|
||||
// which matches on the native pair, not on this column.
|
||||
amountAud = 0;
|
||||
flags.push("aud_provisional", "no_rate_reference");
|
||||
}
|
||||
}
|
||||
|
||||
// A gift card has no statement line coming, so it must not enter the
|
||||
// reconcile queue (which filters on exactly this column).
|
||||
const storedValue = isStoredValueInstrument(ev);
|
||||
const paymentMethod = storedValue ? "credits" : "card";
|
||||
if (storedValue) flags.push("stored_value_instrument");
|
||||
|
||||
// Everything from the idempotency re-check to the INSERT holds one lock.
|
||||
// n8n's HTTP node posts a poll's items CONCURRENTLY, and both halves of a tap
|
||||
// usually land in the same 15-minute poll -- so each ran findPairedRow before
|
||||
// the other had inserted, neither found its partner, and one tap became two
|
||||
// rows: 11 pairs, A$3,841.40, every pair with an identical created_at. A
|
||||
// single lock for the whole lane, not one per amount: wallet volume is a few
|
||||
// dozen a day, and a coarse lock has no key to get wrong.
|
||||
return withTransaction(async (db) => {
|
||||
await db.queryRaw(`SELECT pg_advisory_xact_lock(hashtextextended('wallet-ingest', 0))::text`);
|
||||
|
||||
const existing = await alreadyIngested(db, ev.event_id);
|
||||
if (existing) {
|
||||
return { transactionId: existing.id, metadataId: null, skipped: "already_ingested" as const, flags };
|
||||
}
|
||||
|
||||
// Merge the paired notification rather than creating a second row for one
|
||||
// tap. The issuer half brings the card tail; the Wallet half brings the
|
||||
// merchant and suburb. COALESCE so whichever arrives second only ever fills
|
||||
// gaps -- it can never blank a field the first half supplied.
|
||||
const pair = await findPairedRow(
|
||||
db, amountAud, notifiedAt, ev.notif_app ?? null,
|
||||
fxCode && fxAmount !== null ? { amount: fxAmount, code: fxCode } : null
|
||||
);
|
||||
if (pair) {
|
||||
await db.queryRaw(
|
||||
`UPDATE transactions
|
||||
SET merchant_name = COALESCE(NULLIF($2,'Card purchase'), merchant_name),
|
||||
merchant_normalized = COALESCE(NULLIF($2,'Card purchase'), merchant_normalized),
|
||||
-- A subtitle-derived place overwrites; anything else only fills a
|
||||
-- gap. Without the $4 branch the merged row depends on which
|
||||
-- notification happened to arrive first, and the issuer half
|
||||
-- usually wins by a few seconds -- keeping the worse string.
|
||||
location = CASE WHEN $4::boolean THEN COALESCE($3, location)
|
||||
ELSE COALESCE(location, $3) END
|
||||
WHERE id = $1`,
|
||||
[pair.id, merchant, location, subtitleLocation !== null]
|
||||
);
|
||||
// The merged half records itself, or a replay finds nothing and merges
|
||||
// it again (or, once the partner is superseded, mints a new row).
|
||||
await db.queryRaw(
|
||||
`UPDATE expense_metadata
|
||||
SET card_last4 = COALESCE(card_last4, $2),
|
||||
payment_method_detail = COALESCE(payment_method_detail, $3),
|
||||
merchant_normalized = COALESCE(NULLIF($4,'Card purchase'), merchant_normalized),
|
||||
flags = COALESCE(flags, '[]'::jsonb) || $5::jsonb
|
||||
WHERE id = $1`,
|
||||
[pair.meta_id, ev.card_tail ?? null, ev.issuer ?? null, merchant, JSON.stringify([adoptedFlag(ev.event_id)])]
|
||||
);
|
||||
return { transactionId: pair.id, metadataId: pair.meta_id, skipped: "merged_pair" as const, flags };
|
||||
}
|
||||
|
||||
// The receipt got here first. Fold into its row instead of writing a second
|
||||
// one for the same purchase (see findWalletAdoptableTransaction). The column
|
||||
// list IS the ownership boundary, the mirror of receipt-ingestion.ts's: the
|
||||
// receipt read the register tape, so merchant, description, category and the
|
||||
// items are its business and are absent here; `location` is wallet's own --
|
||||
// nothing else writes a suburb -- and payment_method is filled only when the
|
||||
// other lane left it unknown.
|
||||
const adopted = await findWalletAdoptableTransaction(DEFAULT_OWNER_ID, amountAud, date, merchant, undefined, db);
|
||||
if (adopted) {
|
||||
await db.queryRaw(
|
||||
`UPDATE transactions
|
||||
SET location = COALESCE($2, location),
|
||||
payment_method = COALESCE(payment_method, $3)
|
||||
WHERE id = $1`,
|
||||
[adopted.id, location, paymentMethod]
|
||||
);
|
||||
let metadataId = adopted.metadata_id;
|
||||
if (metadataId) {
|
||||
await db.queryRaw(
|
||||
`UPDATE expense_metadata
|
||||
SET card_last4 = COALESCE(card_last4, $2),
|
||||
payment_method_detail = COALESCE(payment_method_detail, $3),
|
||||
currency = COALESCE(currency, $4),
|
||||
flags = COALESCE(flags, '[]'::jsonb) || $5::jsonb
|
||||
WHERE id = $1`,
|
||||
[metadataId, ev.card_tail ?? null, ev.issuer ?? null, stated, JSON.stringify([...flags, adoptedFlag(ev.event_id)])]
|
||||
);
|
||||
} else {
|
||||
// expense_metadata.transaction_id is UNIQUE, so this branch only runs
|
||||
// when the adopted row genuinely has none -- and it must produce one, or
|
||||
// the card tail and the flag that makes this idempotent have nowhere to
|
||||
// live.
|
||||
const meta = await db.queryRow<{ id: number }>(
|
||||
`INSERT INTO expense_metadata (
|
||||
transaction_id, source, source_message_id, merchant_normalized, amount,
|
||||
transaction_date, card_last4, currency, payment_method,
|
||||
payment_method_detail, line_items, flags, extraction_model,
|
||||
notified_at, notif_app
|
||||
) VALUES ($1,'wallet',$2,$3,$4,$5,$6,$7,$8,$9,'[]'::jsonb,$10::jsonb,'wallet-interpret',$11,$12)
|
||||
RETURNING id`,
|
||||
[
|
||||
adopted.id, sourceRef, merchant, ev.amount, date, ev.card_tail ?? null, stated,
|
||||
paymentMethod, ev.issuer ?? null, JSON.stringify([...flags, adoptedFlag(ev.event_id)]),
|
||||
notifiedAt, ev.notif_app ?? null,
|
||||
]
|
||||
);
|
||||
metadataId = meta!.id;
|
||||
}
|
||||
return { transactionId: adopted.id, metadataId, skipped: "adopted" as const, flags };
|
||||
}
|
||||
|
||||
const txn = await db.queryRow<{ id: number }>(
|
||||
`INSERT INTO transactions (
|
||||
transaction_date, description, amount, amount_aud, category, payment_method,
|
||||
merchant_name, merchant_normalized, transaction_type, owner_id, location,
|
||||
foreign_currency_amount, foreign_currency_code, source, source_ref, row_index
|
||||
) VALUES ($1,$2,$3,$3,'other',$10,$4,$4,'debit',$5,$6,$7,$8,'wallet',$9,0)
|
||||
RETURNING id`,
|
||||
[date, merchant, amountAud, merchant, DEFAULT_OWNER_ID, location, fxAmount, fxCode, sourceRef, paymentMethod]
|
||||
);
|
||||
|
||||
const meta = await db.queryRow<{ id: number }>(
|
||||
`INSERT INTO expense_metadata (
|
||||
transaction_id, source, source_message_id, merchant_normalized, amount,
|
||||
transaction_date, card_last4, currency, payment_method,
|
||||
payment_method_detail, line_items, flags, extraction_model,
|
||||
notified_at, notif_app
|
||||
) VALUES ($1,'wallet',$2,$3,$4,$5,$6,$7,$8,$9,'[]'::jsonb,$10::jsonb,'wallet-interpret',$11,$12)
|
||||
RETURNING id`,
|
||||
[
|
||||
txn!.id, sourceRef, merchant, ev.amount, date,
|
||||
ev.card_tail ?? null, stated, paymentMethod, ev.issuer ?? null, JSON.stringify(flags),
|
||||
notifiedAt, ev.notif_app ?? null,
|
||||
]
|
||||
);
|
||||
|
||||
return { transactionId: txn!.id, metadataId: meta!.id, flags };
|
||||
}, { maxWait: 10_000, timeout: 30_000 });
|
||||
}
|
||||
Reference in New Issue
Block a user