c25c82932d9c41eacb1245991d1f0b9201b7eb9a
57
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  siddharthdandClaude Opus 5.5
|
c25c82932d |
void: a statementless row for a charge that never posted (migration 0040)
ci / lint-test (push) Successful in 54s
A card notification is sent for an authorisation; a released hold never reaches a statement. Txn 7951 (a $1,040.41 Airbnb hold, booking cancelled the next day) sat in the reconcile queue forever and counted as spend. Neither reconcile (needs a statement twin) nor supersede (needs a survivor) can express it. - transactions.voided_at + void_reason; CHECKs: reason mandatory, statementless only - EXCLUDE_RECONCILED_SOURCE excludes voided rows; reconcile queue, findAutoMatches, both adoption helpers, annual charges and the manual reconcile POST filter it; merge refuses a voided row on either side - POST/DELETE /api/transactions/<id>/void; refuses reconciled, superseded and settled-split rows - "Never posted" button on /reconcile - wallet-ingestion deliberately still sees voided rows, so a late twin or a prompt_version re-post cannot rebook the hold Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
siddharthd
|
069034951e |
reconcile queue: offer a foreign row's exact native twin regardless of provisional AUD
ci / lint-test (push) Successful in 1m4s
Wallet INR taps were priced at a provisional 0.0180 AUD/INR; CommBank charged ~0.0149, so every rupee twin sat ~20% outside the picker's +-5% band. The auto-matcher pairs on the native figure but refuses ambiguity (INR 112 twice in three days, INR 175 three times), which left those rows offered nothing at all. |
||
|
siddharthdandClaude Opus 5.5
|
ce0bf0a775 |
reconcile queue: never list or offer a superseded row
ci / lint-test (push) Successful in 58s
getPendingReconciliations filtered neither the manual side nor the candidate statement side on superseded_by_id. After the 2026-10-03 wallet cleanup the queue still showed duplicate half 7037 (superseded under 7038) beside its twin, with the same Digidirect statement line offered to both -- accepting it on the hidden row would have spent the line and stranded the real one. The candidate side could likewise offer one of the 31 superseded ANZ rows. findAutoMatches() always had both filters. First tests for this query; both fail against the old SQL. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
siddharthdandClaude Opus 5.5
|
a3e4e1102a |
wallet: lock the ingest, pair on time and app, skip notices, ECB fallback
ci / lint-test (push) Successful in 54s
Four defects found reviewing a month of overseas wallet rows: - Race: n8n posts a poll's items concurrently and both halves of a tap usually share a poll, so each ran findPairedRow before the other inserted. 11 taps became 22 rows (A$3,841.40), identical created_at on every pair. One advisory lock now covers re-check -> pair -> adopt -> insert, on the transaction's own connection (a second pooled one per request deadlocks a burst). Mutation-tested: without the lock the concurrent test fails 3/3. - Pair window: amount within +/-1 DAY folded the UNICEF $50 donation and an unrelated $50 Amex charge 17h later into one row and renamed it. A pair is now minutes apart AND from two different apps. Needs when/where a notification arrived: migration 0039 adds expense_metadata.notified_at / notif_app, backfilled from capture.wallet_events (144 rows). A merged half now records wallet_evt:<id>, so a replay cannot merge it twice. - Not purchases: a CommBank overdraft notice became a $21.21 purchase named "To avoid fees and interest," (the Westpac fixed-width parser ran on a CommBank body). Account notices and transit-card balance drawdowns are skipped; the descriptor parser only runs on Westpac. - FX: with no ledger history (Hong Kong) every tap stored A$0.00. The ECB rate via Frankfurter is the fallback, flagged rate_ecb; the ledger median stays first because it carries the card's FX fee. Also: dates in Australia/Melbourne like the statements (UTC slicing filed 28 of 134 rows a day early), and test:setup now actually rebuilds personal_test -- replaying the dump over it never applied a migration. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
siddharthdandClaude Opus 5.5
|
22fde30b26 |
currency: a wallet row's amount is AUD, so show its native figure, not the AUD one under a foreign label
ci / lint-test (push) Successful in 1m41s
Wallet rows store the provisional AUD figure in amount and amount_aud and the native figure in foreign_currency_amount. NATIVE_CURRENCY labelled every statement-less row with foreign_currency_code (the order-lane convention, where amount IS native and amount_aud is NULL), so an INR 524 tap rendered as "$9.42 / INR 9.42" (txn 8162). amount_aud being present now marks amount as AUD. Measured live: exactly the 38 foreign wallet rows change label, nothing else. The transactions list's second line prints foreign_currency_amount where one exists, which also surfaces the native figure on statement rows that carry one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
siddharthdandClaude Opus 5
|
48c6b2395f |
wallet: never store a native figure as AUD; rate from statement rows only
ci / lint-test (push) Successful in 51s
A CommBank approval prompt for 672006 VND had no VND rate history, and the
no_rate_reference branch left the native figure in transactions.amount -- the
ledger carried A$672,006 for six days (txn 7316, repaired to 35.95).
- no rate history: amount = 0, still flagged aud_provisional/no_rate_reference;
reconcile matches on the native pair, so the statement supplies the AUD
- medianRateToAud reads statement rows only; a wallet row's amount is itself
a guess and a 1:1 row would have poisoned the next estimate
- foreign halves pair on the native pair, since several can share amount 0
- currencyFromAmountText reads ISO codes around the figure ("672006 VND")
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
siddharthd
|
5076f27f68 |
annual charges: surface upcoming card/loan fees from payments alone
ci / lint-test (push) Successful in 56s
Answers "what annual payments are coming?" without touching the email lane --
a bank announces an annual fee on the statement and nowhere else.
WHAT DOES NOT WORK, measured before writing it: "two charges ~365 days apart"
ranks a BAKERY first on this ledger, with 202 such pairs, then a donation,
Patreon, Coles. Frequency is the confound. Tightening to "every gap is annual"
still admits a Target visited twice ($30, then $198).
transaction_type='fee' is the universe, not a description match. The extractor
already classifies fees structurally (131 rows) and every card and loan annual
fee here is typed that way -- the first cut of this filtered
transaction_type='debit' and returned NONE of them. The fees are not debits.
Three confidence tiers, kept apart so a guess never renders like a bank fact:
confirmed typed a fee AND repeated on an annual cadence
likely typed a fee, seen once, and NAMED annual/card/package -- one charge
cannot show a cadence but the kind is not in doubt
possible no fee wording; merchant whose every gap is annual AND whose amount
is stable (<=20%, absorbing VicRoads' ~3%/yr drift). Catches Amazon
Prime and rego; also where the false positives live, hence the label.
expected_not_seen is the more interesting status: the estimate passed and
nothing landed. account_days_stale is the discriminator -- Westpac at 0 days is
trustworthy, ANZ at 392 is simply missing data. Reporting those as upcoming
would be a lie.
account_links (migration 0038) fixes account continuity. Citi Prestige
5258...0253 became NAB MyCard 1015...6227 -- same card, same 14th-to-13th cycle,
no gap -- so the $700 fee charged 2024-12-13 and 2025-11-13 read as two dead
one-offs instead of one due again ~Nov 2026. Hand-asserted with source_documents
ids as evidence rather than inferred: a contiguous-cycle heuristic would join two
genuinely different accounts as readily as one renamed one. Display resolves to
the CURRENT bank; naming the retired brand sends you to the wrong bank.
The GET takes its own DASH_READ_TOKEN with no fallback to WALLET_INGEST_TOKEN --
that one unlocks wallet ingest and the estate-wide reconcile sweep, and
home-dashboard's compose states the rule: it never holds a write credential.
Known limit: the merchant signal fragments on label drift (VicRoads paid as
VICROADS then VICROADS ONLINE PAYM). Not fuzzy-matched, per the rejected pg_trgm
merchant grouping.
339 integration + 196 unit green, tsc clean. Board #265.
|
||
|
siddharthd
|
10cf0aaf21 |
transactions: merge/supersede a duplicate by hand, and widen the picker band
ci / lint-test (push) Successful in 55s
Two gaps, found because SP PEAKDESIGN was sitting duplicated with no way to
resolve it.
1. THE PICKER COULD NOT OFFER THE RIGHT ANSWER. The manual reconcile queue's
candidate band was `amount * 0.99 .. 1.01`. A foreign card purchase is
charged the notified amount PLUS the card's foreign transaction fee -- 3% on
the Westpac Altitude -- so a wallet row and its statement line can never
match exactly: notified $42.44, charged $43.71, x1.03. The one row that WAS
its twin was the single row outside the band. Widened to +-5%. Safe because
the list is capped at 5 by the rn window and ordered by amount closeness, so
a looser band changes which five appear, never how many, and a person
confirms each. Measured on the live queue: candidate counts 0 -> 0 on all 12
rows, i.e. no new noise. The auto-matcher stays exact.
2. THERE WAS NO MANUAL MERGE OR DELETE AT ALL. `superseded_by_id` was read
everywhere (analytics and auto-reconcile both exclude it) and written
NOWHERE in application code -- set exactly once, by migration 0023.
POST /api/transactions/<id>/merge takes {other_id, mode}: `reconcile` folds a
statementless row into its statement twin, `supersede` handles two rows on the
same footing (a statement imported twice). Deliberately NO delete -- children
are ON DELETE CASCADE, so deleting destroys the splits, tags and metadata that
make a row legible. Both modes keep the row and merely stop it counting.
Both move curation onto the survivor FIRST, through one extracted
moveCurationWithin() rather than a second copy: reconcile.ts's own header warns
that a duplicate would eventually move a different set, and a split stranded on
a hidden row stops counting in silence.
Refuses, all tested: hiding a statement row behind a statementless one (the
bank's record is the side that survives -- burying it under a wallet row leaves
the ledger holding the pre-fee amount); either side already reconciled or
superseded (409, since re-pointing strands the curation moved on the first
pass); burying under an already-superseded row; self-merge; wrong mode for the
pair; someone else's row (ownership via COALESCE(t.owner_id, s.owner_id),
because a statement row carries owner_id on the statement).
UI: a collapsed "Mark as duplicate..." section at the bottom of the edit modal,
debounced search over the existing transactions endpoint, statement-vs-
statementless badges, mode chosen automatically and stated in plain language,
and the wrong-direction case explained with the button disabled rather than
letting it 400.
Also: resetDB now truncates transaction_audit -- the 0037 triggers fire on every
insert the suite makes, so without it the test DB grew ~1,100 rows per full run,
forever (1,125 -> 2).
329 integration + 196 unit green, tsc clean, lint error-free, build compiles.
Board #260.
|
||
|
siddharthd
|
2c05f10aed |
wallet+intake: adopt the row that is already there, and keep a gift card out of the queue
ci / lint-test (push) Successful in 53s
Three defects found by the 2026-08-18 Woolworths shop reaching the ledger
twice — txn 6858 (pantry receipt, 23 items) and txn 6927 (wallet) were one
$103.30 purchase, and the second parked in the reconcile queue waiting for a
statement line the first had already claimed.
The design says signals for a purchase arrive in sequence — wallet first,
receipt second, statement last — and the receipt lane honours it
(findAdoptableTransaction). Nothing else did.
1. The wallet writer inserted unconditionally. It now adopts a row a receipt
got to first (findWalletAdoptableTransaction). Deliberately NOT the same
predicate as the receipt direction: that one refuses a row carrying line
items, because for a receipt such a row is a different purchase sharing an
amount; here an itemised row IS the receipt being folded into. Losing that
guard means the merchant supplies it, so adoption requires merchant
agreement — a tap that vanishes into an unrelated purchase is worse than a
visible duplicate. Idempotent across a prompt_version replay via a
wallet_evt:<id> metadata flag, since an adopted row keeps no wallet
source_ref of its own.
2. intake_create booked a second row for a tap the wallet lane had already
recorded. matchTransaction cannot see those rows — its key includes the
platform and a notification has none — so an Uber Eats screenshot never
matched the wallet row for the very tap that paid for it. That is the
NORMAL arrival order, not an edge case.
3. A gift-card tap was written payment_method 'card', which means "a statement
line is coming"; the queue and findAutoMatches filter on exactly that, so
it parked forever. The instrument's name survives only in the notification
title ("Visa Gift Card") — the interpreter reduces it to issuer "Visa",
which a plain Visa debit card also produces — so notif_title is now carried
through and a stored-value tap is booked 'credits'. 3 of the first 21 rows.
Also: pantry reports "this receipt is already on my board" in the 200 body and
the forwarder dropped the field, so the one gate that DID fire on the duplicate
was silent (captures 66/69). It now becomes the card. And ambiguity — two
ledger rows for one shop, the exact state behind this bug — no longer reads as
"no match, finance will book it".
|
||
|
siddharthd
|
9b59edcf48 |
reconcile: machine auth for the scheduled sweep, across all owners
ci / lint-test (push) Successful in 46s
I built the auto-reconcile endpoint and then left nothing calling it — exactly the gap I had criticised reconcilePendingOrders for. Found by tracing the pipeline end to end rather than by anything failing. PATCH now takes two callers. A person in the browser reconciles their own queue via the session; the scheduled sweep has no session, carries the machine token, and sweeps EVERY owner with a queue. That distinction is load-bearing: 169 of the 177 rows outstanding belong to the second participant, so a sweep scoped to the session user would leave almost all of the queue untouched. A wrong machine token does not fail the request outright — it falls through to session auth, so the all-owner sweep stays gated while a browser caller is unaffected. Verified live: 403 with a bad token and no session, dry-run OK with the real one. 3 new tests. Two test-harness bugs fixed while adding them: the request stub had no `headers` at all, and the statement helper reused one account number, so two owners collided on uq_statement_identity rather than modelling two real cards. |
||
|
siddharthd
|
3d0d75519a |
wallet: the deterministic parse must beat the model's location
ci / lint-test (push) Successful in 46s
Found by reading the backfilled rows rather than by a test. The precedence was subtitle -> MODEL -> descriptor, so the model's answer won wherever it had one, which is always. That defeats the entire point of parsing the descriptor positionally: the ledger got "SAN FRANCISOUS" for the only foreign merchant in the capture window, plus the padded "MELBOURNE AU", and the bare "Sydney"/"Perth" with their country codes dropped. Correct order is subtitle -> deterministic descriptor -> model. After the fix those rows read SAN FRANCISO US, MELBOURNE AU, Sydney AU, Perth AU, and a subtitle-derived place still wins where one exists. The test that should have caught this passed because its fixture omitted `location`, so the fallback chain was never exercised — every real interpretation carries one. Fixture now includes the model's actual mangled answer for that row, so the test fails if the order regresses. Re-ran the backfill on the corrected code: 21 transactions, $6,284.37 unchanged, 13 with a location. The 21 previously-written rows were deleted first rather than superseded, which is normally forbidden on transactions — checked before doing it that they carried zero splits, tags and overrides, being machine-written minutes earlier by this same lane with nothing yet attached. |
||
|
siddharthd
|
8d1875225f |
wallet: a transaction for every tap (board #252, ING-11)
ci / lint-test (push) Successful in 52s
Step 3, and the last app-side piece. POST /api/wallet/ingest takes an
interpreted notification and writes a statementless card transaction plus
one expense_metadata row. Sibling to /api/orders/ingest and
/api/receipts/ingest, same shared-secret shape, its own token so the
wallet credential rotates without touching the order flow.
Nothing here knows about statements, and that is the point: statementless
+ payment_method 'card' is exactly the reconcile queue's predicate, so a
wallet row enters it unaided and the auto-matcher confirms it later.
THE PAIR IS MERGED, NEVER DEDUPED. One tap emits two notifications ~2-8s
apart: the issuer app has the card tail, Apple Wallet has the merchant and
the suburb, and neither alone is complete. Matched on amount and date
because the two share no identifier. My first version COALESCEd every
field, which made the result depend on arrival order — the issuer half
usually lands first, so the truncated all-caps "WYNDHAM VAL AU" beat the
subtitle's "Wyndham Vale, VIC". A subtitle-derived place now overwrites; a
descriptor-derived one only fills a gap. The test that caught this asserts
both arrival orders.
CURRENCY IS NEVER ASSUMED. Only Amex states one ("A$5,103.49"); Westpac,
CommBank and Apple Wallet write a bare "$", which is 14 of 21 captured
rows. Recorded as unknown and flagged, not assumed AUD — harmless at home,
a 23x error on a THB tap abroad. When a currency IS stated and is not AUD,
the NATIVE figure goes to foreign_currency_amount, because the AUD amount
is the bank's own conversion and a notification cannot predict it, while
the native pair is exact on both sides and is what the statement will
match on. transactions.amount is NOT NULL so it takes a provisional
conversion from the ledger-derived median rate, flagged aud_provisional,
overwritten at reconcile. No rate history means no_rate_reference rather
than an invented number.
wallet-descriptor.ts moves the positional parsing out of the model, which
handled ONE fixed-width format three ways on the same day's data: kept the
country code on "WYNDHAM VAL AU", stripped it on "Sydney" and "Perth", and
swallowed "SAN FRANCISOUS" whole — that being "SAN FRANCISO" truncated to
the 12-char city field plus "US", on the only foreign merchant in the
entire capture window. Every unit-test input is a real captured string.
14 unit + 12 integration tests. npm run test:all green (196 + 304), lint
clean.
|
||
|
siddharthd
|
8e66e527ae |
receipts: adopt the wallet row instead of creating a second one (board #252)
ci / lint-test (push) Successful in 51s
Step 2 of wallet-first ingestion. Signals for one purchase arrive in sequence, not in competition: the notification within seconds of the tap, the receipt hours or days later. Until now the receipt path always INSERTed its own transaction, so the same shop became two rows. The case this is built from is measured, not hypothetical. Woolworths $103.30 on 2026-08-18: the wallet event landed 09:00:14 on the 18th and nothing created a transaction until 20:25:06 on the 19th -- 35h25m later, by this path, from scratch -- discarding what only the notification knew (the instrument, and the suburb). findAdoptableTransaction() is deliberately as narrow as findAutoMatches(): exact amount, a few days either side, unclaimed, and UNIQUE. Two candidates means two plausible purchases, and guessing between them moves one shop's line items onto another shop's row. It also refuses a row that already carries line items, because a fully-described row is a different purchase that happens to share an amount. Returning null is always safe -- the caller falls back to creating its own row, which is the old behaviour. On adoption, both the transaction UPDATE and the expense_metadata UPDATE have a column list that IS the ownership boundary, following order_transaction_bridge's UPGRADE pass: the receipt read the register tape so it wins on what was bought, and `location` is structurally ABSENT from the update rather than conditionally skipped, so the notification's suburb cannot be erased by a bug here. expense_metadata.transaction_id is UNIQUE, so an adopted row that already has metadata is enriched in place rather than raising. 5 new integration tests including both refusals. Existing receipt tests unchanged and passing. npm run test:all green. |
||
|
siddharthd
|
f31f12799a |
reconcile: automatic pass for unambiguous statement matches (board #252)
ci / lint-test (push) Successful in 56s
Prerequisite for the wallet lane writing a transaction per tap. The queue is worked by hand and already ~177 deep; wallet adds ~63 rows a month, ~767 a year. Filtering those rows back out is explicitly ruled out by the awaitsStatementLine() removal note: "an imported row that has no statement belongs in the queue; volume is solved upstream, never downstream by hiding what is there." The rule is deliberately stricter than the queue's own candidate ranking (top 5 within +/-1% and +/-3 days, for a human to choose between). A machine gets one shot and no judgement, so: exact amount, never a tolerance band; and unique BOTH ways, so two identical $13.50 bakery taps in one window are refused rather than paired arbitrarily. Foreign rows match on the native pair, because `amount` is the bank's own conversion and a wallet row cannot predict it. VALIDATED AGAINST REAL DECISIONS, and that overturned my reasoning. I first required both sides to be non-foreign, arguing a currency mismatch is ambiguous. Checked against the 48 reconciliations a human has actually made: that version matched ZERO of them. All 48 are exactly the shape it refused — a manual row carrying no currency, matched on exact AUD against a statement row that does carry one (EUR, CHF) — because overseas receipts and orders record what was charged in AUD, and only the wallet lane records natively. Reverted; the rule now reproduces all 48 and proposes nothing they did not. The coincidence it was meant to guard against is already covered by unique-both-ways. Extracted the merge into src/lib/reconcile.ts so the manual route and the automatic pass cannot drift — there must be exactly one implementation of something that moves curation off a row reconciliation is about to hide. Fixed a latent bug in that merge while extracting it: splits were recreated WITHOUT `settled`/`settled_at`, and the column defaults to false, so reconciling a settled split silently marked a paid debt owed again. No queue row carries one today, but this path is about to run unattended, where that surfaces as money rather than as an exception. PATCH ?dry=1 reports without writing. Each pair is its own transaction, so one bad pair leaves the rest untouched and in the queue. 11 new integration tests, most of them about refusal. Live dry-run proposes 0 pairs for either owner, which is correct: nothing in the queue has an exact statement twin yet. npm run test:all green (182 + 287). |
||
|
siddharthd
|
669f9119b9 |
enrichment readers: resolve one metadata row, deterministically (board #253)
ci / lint-test (push) Successful in 53s
Four readers of expense_metadata that only the 1:1 UNIQUE constraints keep
correct, found while designing wallet-first ingestion. Each fails silently
rather than erroring, so none of them would announce itself.
merchantVerdict (order-reviews.ts) was a genuine fan-out: a plain JOIN on
transaction_id OR matched_transaction_id, no per-transaction cap, feeding
`for (const r of rows) counts[r.rating] += 1`. One review joined to N
metadata rows is counted N times, inflating the loved/never tallies that
drive the pre-order warning — and its LIMIT 50 applies to the fanned-out
set, so duplicates would also push older reviews out of the window. Now a
LATERAL, so the merchant matched is the merchant the UI shows.
nudgeState (slack/interactive) had no LIMIT and no ORDER BY, feeding
queryRow, which returns rows[0]. Postgres guarantees no ordering, so the
merchant and currency on every Slack card refresh could flip at random.
It was the only "resolve both links" site missing the convention its
siblings all carry.
getSharedTransactions had LATERAL LIMIT 1 with no ORDER BY — bounded, but
arbitrary, and free to disagree with the main transaction list about the
same transaction.
transaction_search (migration 0035): NOT the fan-out I first claimed. It
joined only on em.transaction_id, which is UNIQUE, so it could not
multiply rows and sums over it were never inflated. The real defect is
narrower and worse: it OMITTED matched_transaction_id, so every
card-settled order was invisible to it — 148 transactions (Glovo Milan,
Uber Eats, all source='order-bridge') reporting NULL payment_method /
order_reference / line_items while the data sat in the table. This view is
the agent read surface (Hermes' MCP selects those columns from it) and
home-dashboard reads it via dash_ro, so the omission was estate-wide.
Broadening the join is what introduces fan-out risk — transaction_id and
matched_transaction_id are two SEPARATE unique indexes, so two different
rows can legally point at one transaction — hence LATERAL rather than a
second OR. Verified before applying: 4760 rows unchanged, debit sum
unchanged at 1514412.89, only the 148 rows differ. Grants re-granted and
re-verified, since CREATE OR REPLACE VIEW drops them.
Also fixes an integration assertion
|
||
|
siddharthd
|
0823244e4a |
intake: the word "Image" is not a receipt, and the prompt must not be data
ci / lint-test (push) Successful in 52s
Batch 3d2ffbe8 arrived as one text part whose entire content was the five characters `Image` — an iOS Shortcut coerced the shared photo to text. The model was asked anyway and answered out of its prompt: Burgertory (Braybrook), two "The Pride" lines at $22.80, a $41.15 total. Nothing on it existed. Burgertory was in ORDER_ITEM_PROMPT as the example of how to print a branch in parentheses. Only the missing date kept it out of the matcher; the invented header and items were stored on the batch regardless. Three layers, none of them a model: 1. textCouldBeReceipt() — deterministic, before any model call. A TEXT part with fewer than two words, no digit, or under 40 characters and no money figure cannot be a receipt. Length is NOT the test on its own: `TOTAL $17.00` is twelve characters and is exactly what this door accepts. A rejected part is dropped like an unreadable file type, so one junk part beside a good screenshot still goes through; if nothing usable is left the card quotes what the phone actually sent. PDFs and images untouched. 2. The prompt names no real shop — merchant examples are PROMPT_PLACEHOLDER_MERCHANTS, exported from intake-rules so the prompt and the guard cannot drift — and it is told what to return when the input is not a receipt at all. 3. confabulationReason() — the belt. Reachable where the pre-check cannot look: a PDF that renders to a scrap of junk still reaches the model. Also rejects any answer whose merchant IS a placeholder. All three file `unsupported`, never `needs_header`. A confabulated header is not a partial one, and the needs_header path stores what it is given. Same shape as the ingestion engine's 2026-08-05 empty-body defect: a stage that can be handed nothing has to say so, because the model downstream will always answer. Tests: 16 unit cases over the two pure functions and the prompt; three integration cases — `Image` is declined with zero model calls and nothing but `unsupported` on the batch row, a junk part beside a real receipt still enriches, and a junk PDF is unsupported rather than needs_header. The first and third fail on the tree before this commit. |
||
|
siddharthd
|
91586ce4a4 |
test(intake): the second review pass, and CLAUDE.md for the rules it set
ci / lint-test (push) Successful in 45s
Six of the nine fixes have a test that fails against the previous tree: the metadata row's currency, the Create-path hold, two concurrent batches booking once, mrkdwn escaping including plain_text, the escaped currency, and candidate-row currency. Three do not, and the reason is the same each time — the window needs a crash or a lost race that a test cannot force. Asserted instead: a stale metadataId falls back to the live row rather than reporting a phantom write; two writes racing to create a metadata row both resolve to one; the create path records a baseline at all. All three passed before as well. |
||
|
siddharthd
|
631e75e58f |
test(intake): cover the review findings, and CLAUDE.md for the rules they set
ci / lint-test (push) Successful in 59s
Unit: word-boundary merchant matching (Bigals Pizza is not an IGA), mrkdwn escaping vs the plain_text button label, a foreign total on a card, and the finish_reason gate. Integration: the NULL-platform wildcard on both the food and grocery paths, candidates crossing platforms, EXCLUDE_RECONCILED_SOURCE, the pinned metadata row, the panel reading the row the write landed on, the once-per-batch undo baseline, the priceless-upload hold, Create re-matching, two pending cards booking once, a pantry-resolved sibling, and a USD receipt. Eleven of the thirteen fail against the pre-fix tree; the exceptions are the no-batch-id baseline guard (a no-change assertion) and the concurrent-tap one, which is a race and passed either way. |
||
|
siddharthd
|
32a47bf8ee |
test(intake): rules, PDF text, C2 idempotency, enrich-vs-candidates, pantry hand-off, Slack taps
Unit: merchant pattern, header validation, reconciliation tolerance, verbatim line items, one-call message shape, card action ids; unpdf over the two real Uber Eats exports (Coles $85.74 / 15 lines, Burgertory $22.40 / Michael + Joel). Integration: fetch stubbed at the LiteLLM and pantry boundaries so the PDF path, persistence, match and writes run for real against personal_test. intake-live.test.ts is opt-in (INTAKE_LIVE_TEST=1) and was run green against cloud-budget on both PDFs. |
||
|
siddharthd
|
b958d05287 |
test(db): resolve the integration DSN at run time instead of storing an IP (#161)
ci / lint-test (push) Successful in 1m5s
setup-test-db.sh wrote .env.test once with the Postgres container's bridge IP baked in, and Docker reassigns those on recreation. The file rotted silently: its stored 172.22.0.18 belongs to **gitea** as of today, and an earlier drift had the suite pointing its TRUNCATE at **postgres-pantry** — which it would have wiped had the credentials happened to match. Root-cause fix: stop persisting a host at all. - scripts/resolve-test-db.sh re-derives the DSN from the live container and rewrites .env.test. Resolution order is TEST_DATABASE_URL, then the container's published host port, then the bridge IP. `docker port postgres-personal` reports 127.0.0.1:5432 since the 2026-08-17 PG17 migration, so loopback is what gets used here — and a loopback address cannot be reassigned to another container, which is the whole point. (Note this supersedes the smarthome CLAUDE.md's "postgres-personal publishes no host port"; that is now stale.) The bridge-IP fallback is kept for a host where the port is not published, and the script never prints the password. - Wired as `pretest:integration`, so npm runs it before every integration run. .env.test becomes a generated artifact rather than stored configuration. - setup-test-db.sh now only creates the database and copies the schema, then delegates to the resolver. - helpers.ts::assertTestDatabase stays and its message now points at the resolver rather than at `docker inspect`. It still earns its place: running vitest directly bypasses the pretest hook, and a stale .env.test would then still be read. Docs: CLAUDE.md gains a Testing section carrying this, the prisma-generate trap for a fresh checkout or worktree, and the frozen-clock rule for tests whose subject is defined relative to "now". AGENTS.md updated for the same, and for lint now being blocking. |
||
|
siddharthd
|
eaf5d5ac66 |
test(analytics): guard the three unguarded Priority 0 behaviours (#155)
docs/ui-information-architecture-review.md claimed Priority 0 was guarded by analytics-sql.test.ts. That file only ever covered the three SQL *fragments* (EXCLUDE_RECONCILED_SOURCE, NATIVE_CURRENCY, INVESTMENT_SIGNED). Items 2, 3 and 6 of the Priority 0 list had nothing, and all three fail silently — no error, no empty screen, just a number that quietly disagrees with the one beside it. New: src/__tests__/integration/analytics-priority0.test.ts, 18 tests. - **Item 2, pace/headline parity.** /api/analytics/daily summed over a month must equal /api/analytics/monthly's headline to the cent, per category, and across every month in the window. Cents rather than exact equality is the assertion because the two routes group at different grains (month+category vs month+day+category) and both carry 4dp — the displayed grain is what has to agree, which is precisely the bug (the chart ending the month a few cents off the headline above it). Fixtures are only shapes where a naive gross debit sum disagrees: a 50% split, a refund, fee and interest rows, an itemised loan repayment, a reconciled source row, family-tagged spend, transfers/investment/income, and a row someone else paid that I hold a split on. A fourth test asserts the naive sum does NOT match — without it, parity between two queries identical by construction would pass on fixtures too plain to discriminate. - **Item 3, fees date range.** Default 12 months with the window reported (from=2025-09-01, not 08-31 — toDateStr, not toISOString); the older tail excluded; months=0 all-time with a null range rather than a period claim the number does not make; a narrower months=3; the transaction-level fee lines bounded by the same window as the statement totals, since a bounded total beside an unbounded list is the same defect wearing the fix; and a nonsense months param clamped rather than producing an Invalid Date window. - **Item 6, partial-month comparisons.** The in-progress month never enters a baseline; a partial month is compared through the same day of each prior month; a complete month keeps full totals; top movers truncate both sides; year boundary and leap February. Item 6 was untestable by construction, which is why it had no test: the rules were closures in budget/page.tsx reading `new Date()` inline. Extracted to src/lib/partial-month.ts with an injectable clock, and budget/page.tsx now imports them. The extraction also de-duplicates the baseline filter, which was written out identically in two places. The file freezes the clock (vi.useFakeTimers, toFake: ["Date"] only — pg needs real timers for connection handling). Without it the fixtures drift out of the routes' rolling windows over time and the assertions go vacuous, which is the same class of silent failure the file exists to catch. Mutation-checked, all three: breaking daily's spend semantics fails 3 of the item-2 tests (the placebo correctly still passes); forcing the fees window to all-time fails 5; letting the current month into the baseline fails 1; and dropping the elapsed-day truncation fails 3. |
||
|
siddharthd
|
fc9ffaba25 |
chore(ci): clear the lint debt and make lint blocking (#79)
CI ran lint with continue-on-error because 23 errors predated it. All are fixed, so the flag is gone and a typecheck step is added beside it — nothing else in CI compiles the test files, so a type error in a fixture could only ever be found on a developer's machine. Real fixes, no blanket disables and no suppressions: - react-hooks/set-state-in-effect (6): each was "reset or seed state when an input changes", so each becomes a state adjustment during render behind a previous-value guard — the documented React pattern for it. Two were also latent bugs. transactions/page.tsx depended on `others`, rebuilt by .filter() every render, so the effect already ran every render, one commit late. shared/page.tsx depended on the `tagIds` array identity, so the row window was reset more often than "changing any filter" claims — including after "Show more", the one interaction it must not undo; the key now joins by value. csv-import-modal reads localStorage once at mount, which is a lazy useState initializer, not an effect (SSR-guarded, though the modal is mounted from client state and never prerenders). - react-hooks/static-components (3): SortIcon was declared inside SharedPage, making it a new component type on every render. Hoisted to module scope with the sort state passed as props. - @typescript-eslint/no-explicit-any (9): the dynamically imported route handlers in two integration tests. Typed as their real signatures with one `as unknown as NextRequest` cast at the fixture boundary, so the call sites and response bodies get checked instead of opted out. - react/no-unescaped-entities (5): exact-equivalent entities (', "), so no rendered text changes. Also fixes the one pre-existing tsc error (unit/order-ingestion.test.ts:511): `options` is optional, and `?? 0` keeps the assertion failing on an absent list rather than silencing it with a non-null bang. 9 warnings remain (no-unused-vars, exhaustive-deps). eslint exits 0 on warnings, so lint blocks on errors only — deliberately, since none of the remaining warnings should stop a push. |
||
|
siddharthd
|
9f1be74330 |
transactions: let a gift card be a payment method you can pick
ci / lint-test (push) Successful in 43s
`credits` has been a valid payment_method since migration 0018, which added it so the order pipeline could file gift-card-funded orders. It reached the database, `needsCardMatch()` (which holds those rows out of reconciliation) and `bankLabel()` (which renders them "Gift Card"). It never reached the two places a human touches it. So entering a gift-card purchase by hand was impossible: the dropdown stopped at "Other", the row took the NULL default, and NULL is *reconcilable* — it joined the pending queue to be offered card matches within 3 days and 1% that cannot exist, because the balance already paid the bill. Accepting one is the destructive case the cash exclusion documents: reconciled manual rows are filtered out of every query, so the spend vanishes while an unrelated card row claims to be it. - `src/lib/payment-methods.ts` — one canonical list, mirroring the `statement-types.ts` pattern, plus the TS mirror of `needsCardMatch()`. - `PATCH /api/transactions/[id]` — the `VALID_METHODS` literal omitted `credits`, so the API answered 400 for a value the CHECK constraint admits and 372 live rows already carry. It validates against the list now. - `POST /api/transactions` — validated at all, so a bad value is a 400 naming the field rather than a 500 carrying a Postgres error. - Add modal — offers every method, and explains the consequence when you pick one that suppresses reconciliation. Relabelled "Paid by" → "Payment method", which the edit modal uses for the *owner*; they are different questions. - Edit modal — had no payment-method field at all, so an existing row wrongly in the queue could not be corrected. Manual rows only: the queue is `statement_id IS NULL`, so it decides nothing on a statement row. The test asserts the list against `pg_get_constraintdef` and evaluates `needsCardMatch()` as real SQL rather than re-implementing it, so the next migration that widens the constraint fails the suite instead of drifting for two months. Mutation-tested by dropping `credits`. Unit 130 pass, integration 206 pass (the integration DB needed `npm run test:setup` first — `.env.test` still named a stale container IP). |
||
|
siddharthd
|
2cfaf8e8c8 |
statements: derive account identity, fix the new-bank alert (0033)
ci / lint-test (push) Successful in 42s
account_number fragments exactly like bank_name did, from the same cause: ANZ's Access Advantage arrived as both '4085-56264' (4 statements) and '408556264' (1). uq_statement_identity keyed on it, so re-importing one period under the other spelling evaded the duplicate check. account_number_key is GENERATED ALWAYS — the number with separators stripped — and the unique index moves onto it. Derived rather than rewritten because the raw number is the readable one and some of it is structure: Up stores '633-123 / 176540052', a BSB and an account number, and flattening it would lose a distinction a human reads at a glance to fix a machine problem. The Amex mask and case are preserved; 'X' records which digits were redacted. Not the cause of the documented 31-row / $42,040.68 ANZ duplication — 107/142/143 overlap on different end dates, which that index cannot catch at any spelling. Adds statement_identity_drift: one account under two bank names, or one bank under two spellings of one number. Non-empty means fragmentation the normaliser could not know about. Currently one row, the ANZ pair above, now unified by key. Also documents the N8N regression this series caused. Check Known Bank compared the raw Gemini name against the canonical column before the insert, so nothing ever matched and every Zip statement was tagged pending and held for Slack approval — a stall, not just a noisy alert. Fixed in workflow FysADdFwEtwONQl4 by comparing like with like through normalize_bank_name(), passing currency so a genuinely new national entity still alerts. Verified live: 12s after vs 30-53s before. |
||
|
siddharthd
|
95d8544752 |
transactions: change who paid, on a row and on a statement
ci / lint-test (push) Successful in 46s
Owner was write-once for every ingestion path — a pantry receipt hardcodes DEFAULT_OWNER_ID and there was not one `UPDATE ... SET owner_id` in src/ — so a shop the other person paid for was permanently filed as yours. PATCH /api/transactions/[id] now takes owner_id, for manual rows only. A statement row returns 400 statement_owned and points at the statements page: its effective owner is COALESCE(t.owner_id, s.owner_id), so writing it there would either no-op or detach one row from the account it came from. PATCH /api/statements/[id] is new. The statements page has had an owner dropdown since it was built, wired to a route with no PATCH handler — every change 405'd, and because useUpdateStatement never checked res.ok it failed silently and the select snapped back on refetch. It writes both tables: 2,194 statement rows carry their own owner_id against 1,803 that inherit, so updating `statements` alone moves less than half and splits one account's history between two people. The guard is the point. Access is "owner OR holds a split", so handing a row over while holding no split removes it from your list and 404s every route that could put it back — only the new owner can undo it. That is 409 would_lose_access, and the modal offers both ways forward: add my split first, or give it away anyway. Taking a row onto your own ledger is never blocked, and claiming a row you cannot see is a 404 before any owner logic runs. Splits are deliberately not rewritten. They record shares, not direction, so a 50/50 flips from "they owe me" to "I owe them" untouched, settled included. Also adds the missing res.ok check to useUpdateTransaction, without which every rejection resolved as success: the modal closed, the list refetched, and the edit silently vanished. 14 new integration tests; 203 integration + 130 unit green. |
||
|
siddharthd
|
6c14b493ef |
analytics: your share of what someone else paid is your spend
ci / lint-test (push) Successful in 1m25s
Every spend analytic gated on `OWNER_SCOPE = $1` and scaled by `mySplitOf` *within* that gate, so ownership was a precondition for an expense being yours. Half a grocery shop Sonu paid for counted as zero — in monthly, daily, merchants, subscriptions, fees and the budget page. 167 rows / $3,210.91 across Jan-Jul 2026, worst in April (+$1,354.83, the Europe trips), while getParticipantBalances booked the matching debt correctly. The app could say you owed her for a shop while insisting you had not spent anything on it. New MY_SPEND_SCOPE(): owner = me OR I hold a split. OWNER_SCOPE stays on the things that measure an *account* rather than a person — the income and investment lines, and the statement-level fee rollup. myShare had to change with it, and widening the gate alone would have been worse than the bug: its `100 - everyone else` fallback is the payer's remainder, so on someone else's unsplit row it returns 100 and moves their whole bill onto you. It now branches on ownership — my row resolves as before; their row takes an explicit split row only, absent meaning 0. That 0 is what makes the wider gate safe. my_share_percent is deliberately not read on someone else's row: one unscoped column, writable by anyone who can see the row, so "my" can only mean the owner's. All 402 rows carrying one today are owner-side. MY_SHARE_PCT mirrors myShare for the transactions list, which has no viewer-scoped ts join; a test asserts the two agree across seven fixture shapes. No historical restatement — every non-owner split is 2026-dated, and the 1,266 pre-2026 SplitMyExpenses splits are all on rows you own. Also fixes a latent failure in the NATIVE_CURRENCY test, which inserted a statement relying on participant id 1 existing (owner_id is NOT NULL DEFAULT 1 with an FK) and only passed when a sibling file had left one behind. It now owns its fixture. 15 new integration tests; 189 integration + 130 unit green. |
||
|
siddharthd
|
2d341e24a0 |
A negative trip balance is over-coverage, not a bill
ci / lint-test (push) Successful in 49s
You were right and my label was wrong. A payment is allocated to a trip as a lump sum, and the grouped-payment allocation gave each trip enough to clear the payer's gross share. So netting the other side off leaves a fully-paid trip negative by exactly what the payment over-covered: Europe reads -$802.75 because Sonu paid $8,004.04 against a net share of $7,201.30. That surplus is already carried in the overall balance, where she still owes $5,313.38 — nothing is owed to her, and "you owe them" said the opposite. The arithmetic never changed; only what the page claims it means. paid_to_me is the discriminator: negative with a payment into the scope is over-coverage, negative with no payment is genuinely owed because the viewer's share of the other person's spending exceeds theirs. Both cases now tested. All three of today's negatives are the first kind. The trip table gains an Overall balance column from the unscoped participant balances, because a single trip's figure cannot tell you whether to pay anyone, and the footnote now says settlement is always against the overall figure. 278 passing, build clean. |
||
|
siddharthd
|
6db2345c49 |
Net the trip debt to one figure, and show payer, category and search on Shared
ci / lint-test (push) Successful in 48s
Shared view: the query already returned owner_name and effective_category, the table just never rendered them. Paid by sits next to Splits because together they are whose money went out and whose share it was. Search is client-side — this endpoint returns all 1,267 split rows in one request with no pagination, so there is nothing for a round-trip to narrow, and the sort was already client-side. It matches description, merchant, notes, category and payer, but not participant names: the dropdown does that, and "sonu" matching every row she is split on would read as broken. Trip owed collapses to one settle-up figure per person, with the breakdown beside it so the net is auditable rather than asserted. I argued against netting a few hours ago and was wrong. The claim was that the grouped-payment allocation cleared each trip against the one-directional gross, so netting would redefine that debt after the fact. The rows say otherwise: Europe's $802.75 is 56 transactions Sonu actually paid across Rome, Venice, the Dolomites, Bellagio, Lucerne and Paris on which I hold 25%, and paid_by_me is $0.00 on every row of every trip because nothing has ever been recorded going from me to her. Her side looked settled only because the allocation derived her payment split from her gross, so it lands on zero by construction. The one-directional view was hiding a live obligation, not protecting an allocation. Nets now: Auckland Sonu +$1,077.25, Europe Sonu -$802.75, Sonu + Sunny -$936.34, Europe Molina -$816.16. Also correcting an error in my own reporting: I said Auckland's mirror was $0.00. It is $428.39 — 17 Auckland rows Sonu paid that I hold a split on. Two ad-hoc verification queries mis-joined on a nullable scope column and under-reported the mirror side. The app code was never affected and the owed column is still byte-identical. The footnote now states the trap the netting exposes: a debt settled by a payment left on the household tab still reads as outstanding on the trip. Payment 5 (Molina to Sonu, $1,605.49) is exactly that case and is left alone as a data decision. 277 passing, build clean. |
||
|
siddharthd
|
cb7665ded1 |
Let everyone on a trip see it, and give payments their scope back
ci / lint-test (push) Successful in 52s
Trips were scoped to trips.owner_id, so Sonu saw no trips at all — despite having paid for 104 of the tagged rows herself. Her own spending was invisible on the only page organised around it. A participant is now anyone with a split on, who paid for, or whose payment is scoped to, a transaction tagged to the trip. Derived, not stored. A trip_participants table was designed and rejected: the expenses already carry the fact, and two records of one fact drift apart. Deriving it also excludes Singapore + Bangkok 2026 from Sonu for free, which a table would have to be kept in sync to do. Siddharth 4 trips, Sonu 3, Molina 1. Everything about a trip is shared except delete. Both trip foreign keys are ON DELETE SET NULL, so deleting Europe 2026 untags 210 transactions and NULLs the trip scope on 6 payments — where the hand-derived Europe-first allocation lives, which nothing recomputes. That stays with the owner. Trip owed now returns both directions and nets neither. An obligation lives on a row someone else paid for, so a viewer-as-payer figure can never hold it, and Sonu's Europe read "you are owed $2,408.24" while omitting the $8,004.04 she owed. Collapsing the two into a signed net is the tempting next step and would have corrupted the scope allocation: the grouped-payment allocation cleared each trip against the one-directional gross, so redefining the debt afterwards turns $8,004.04 already allocated into an $802.75 over-allocation with household understated by the same amount. Verified byte-identical — Auckland $1,505.64, Europe Molina -$816.16, Europe Sonu $0.00, Sonu + Sunny $0.00. getTransactions gained trip_all_rows so a participant sees the whole trip. It is opt-in and not implied by trip_id, because the same endpoint backs the main transactions list and its trip filter must keep owner scoping. Participation is re-checked in SQL, so passing the flag for someone else's trip returns nothing. Payments can finally say what they settle. trip_id has existed since migration 0022 but POST never read it and GET never returned it, so every payment made in the app landed on household and the 9 trip-scoped rows were hand-written SQL. "Both" needs no new shape — one row per scope sharing a linked_transaction_id. Three write paths had no authorisation at all and were reachable by any participant: assignTransactionsToTrip checked nothing, DELETE on a payment deleted by bare id, and POST accepted any from/to pair. All three now check. Also fixes the test suite, which was pointing at postgres-pantry: container IPs move on recreation and 172.22.0.47 stopped being postgres-personal. It only failed safe because the credentials did not match — resetDB now refuses to truncate anything not named personal_test. 22 new tests, 276 passing, build clean. |
||
|
siddharthd
|
c70d2b1fac |
Sign the investments line so withdrawals net against contributions
ci / lint-test (push) Successful in 50s
A withdrawal from a fund is a disinvestment, not income: units convert back to cash and net worth is unchanged. Summed unsigned it read as more money invested. March 2026 showed $38,615.34 of investing in a month that was net -$11,384.66, because a $25,000 Raiz withdrawal was added to an $8,563.80 IBKR deposit instead of cancelling it. Each credit costs twice — once for being added, once for not being subtracted — so the error is double the credit: $50,000 in March, $3,000 in May 2025, $53,000 across the window. Since net = income - spent - investments, March's net of -$55,918.73 should read -$5,918.73. Filing withdrawals as income is the other tempting answer and is worse: it books an asset disposal as earnings and feeds the same figure into net with a flattering sign. Same reason the Up item sales in Known Gaps do not belong on the income line. What this cannot resolve: part of a withdrawal genuinely is income — the capital gain. The bank descriptor is one gross figure with no cost base, so it cannot be decomposed from statement data. Netting tracks cash committed against cash returned and leaves the gain for holdings data to surface; it does not assert the gain is zero. The budget page gates the Invested card on `!== 0` rather than `> 0` — a net-disinvesting month is real data, not an empty one — and renders negative months in amber so the sign is not hidden by matching digits. |
||
|
siddharthd
|
f6c500b27a |
Hide transfers in the transactions view by default
ci / lint-test (push) Successful in 51s
Transfers move money between your own accounts; at 433 of 3,996 rows (~11%) they crowd out the rows that represent actual spending. getTransactions gains `exclude_categories`, opt-in per caller and deliberately not defaulted in queries.ts: the rules preview and the bulk rule-apply path both read candidate rows through getTransactions, and a default exclusion there would silently shrink what a rule can see and reach — invisibly, since a rule that matches nothing looks the same as a rule with nothing to do. Two behaviours the filter needs, both tested: - An explicit category pick beats the exclusion. Selecting "Transfers" while the default is on subtracts it from the hidden list instead of returning zero rows and reading as "you have no transfers". - COALESCE the effective category to '' before `<> ALL`. NULL <> ALL(...) is NULL, not true, so an uncategorised row would disappear from a filter that never named its category — the trap EXCLUDE_NON_SPEND already documents. The default is off when the view is scoped to a statement: that is a reconciliation view, the row count has to match the statement, and a credit-card payment is the row you went there to check. |
||
|
siddharthd
|
a465b147a3 |
Read receipt lines under the keys the receipt panel renders
ci / lint-test (push) Successful in 48s
The order-details panel reads qty/description/amount from expense_metadata.line_items. A grocery shop is a receipt like any other, so it stores the same keys rather than name/quantity/line_total — otherwise the rows arrive complete and display blank, which is exactly the failure ING-8 names. unit and category are the two fields a grocery line has and a delivery line does not; nothing renders them yet and the category composition will. |
||
|
siddharthd
|
17028c79ff |
Accept grocery receipts scanned in pantry as candidate spend
ci / lint-test (push) Successful in 48s
Adds /api/receipts/ingest as a sibling to the order lane, sharing its shape but making one decision differently: nothing is parked. An order can wait for its statement because it is already visible as an email; a gift-card grocery shop is visible nowhere at all, so a scan that produces no transaction produces nothing anyone can see. Every payment becomes a manual row immediately and the existing pending-reconciliation queue resolves the ones with a card leg coming. One transaction per tender leg. A $114.57 shop settled $40.75 gift card + $73.82 Mastercard has a statement line for $73.82 only. A single row marked credits is excluded from the queue while that line double-counts; marked card it is searched for at 1% of $114.57 and never matches. Either way the shop books $188.39. Per-leg rows make each amount the settled amount, so the matcher works untouched. Reconciliation now carries expense_metadata across. It already moved overrides, tags and splits from the manual row to the statement row and left metadata behind, which did not matter while metadata only came from an email that made its own transaction. It matters now that it carries a shop's line items: unmoved, the contents vanish at exactly the moment the statement line appears, and COLES 0556 MANOR LAKES stays as unreadable as before anything was scanned. transaction_id is UNIQUE, so a statement row that already has metadata keeps it and the pantry row is flagged rather than raising a constraint violation. Also regenerates the Prisma model. card_last4, currency, flags, reconciled_at, matched_transaction_id, platform and route have been in the database since migrations 0019/0020 and were absent from schema.prisma — regenerating the client from it would have dropped columns the order lane writes on every ingest. 23 integration tests against the real schema, built from the three receipts that drove the design. Existing suites unchanged: 104 unit, 144 integration. |
||
|
siddharthd
|
22e4a1ead0 |
fix(splits): make every split account for 100%
ci / lint-test (push) Successful in 1m39s
A 50/50 arrangement was stored as a single row saying "Sonu 50%". The
arithmetic was never wrong — `myShare` resolves the payer's share as
`100 - SUM(everyone else)`, so balances and per-user spend were correct
throughout. It was still a bug, because a ledger is read as well as
computed: on screen that row is a 50% share against a blank, which looks
like half the money is unallocated and is indistinguishable from a split
somebody abandoned half-finished.
It also leaked. `getSharedTransactions` filters by participant with an
EXISTS on an explicit split row, so filtering the Shared view by the
payer silently dropped every transaction where their share was only ever
implied.
Four write paths could produce it, three of them unguarded:
- the Slack nudge's share button, which inserted one row
- `POST /api/transactions`, where the add form shows an amber total
under 100 but saves anyway — this is how Lawn Mowing and Hedge
Pruning were stored
- `applyRuleActions`, where ten of the fourteen live split rules name
only the other person
`completeSplit` is now the single place that writes the remainder, and
every one of those paths ends in it. The remainder goes to the
transaction's *owner*, never to "me": the owner's row on their own
transaction is excluded from both halves of the balance query, so it
cannot create, enlarge or discharge a debt, whereas a row for me on
someone else's transaction is a real obligation. That distinction is
what makes this safe to apply to existing data.
Also fixes the order panel's "Shared 50/50" toggle, which was inert in
both directions: it posted a lone 50% row to share (rejected — must
total 100%) and an empty array to un-share (rejected — array required),
because no way to clear a split existed. DELETE on the splits route is
that way.
Backfill: 7 rows, verified against a row-level dump diff — 2549 -> 2556
rows, none removed, none modified — and participant balances byte
identical before and after (Molina 19556.07, Sonu 20913.35). Every split
in the database now totals 100%.
Not done: a database-level constraint. Enforcing the sum needs a
deferred constraint trigger, and the rule path commits its DELETE and
INSERT as separate statements, so the trigger would reject the
intermediate state. Making it work means wrapping every write path in a
transaction, which is a larger change than the defect warrants.
|
||
|
siddharthd
|
3144cf3176 |
feat(orders): record credits orders from before the cutover
ci / lint-test (push) Successful in 58s
I1 refused any credits-funded order dated before 2026-01-09, storing
nothing at all — no transaction and no metadata, so the receipt was
discarded rather than kept as history.
Its reason was splits, not spend: before the cutover shared expenses
lived in SplitMyExpenses, and re-importing them would double-charge
against carryover transaction 2348. That reason expired with
|
||
|
siddharthd
|
d458228625 |
feat(orders): a fifth verdict, 'bad', between ok and never again
ci / lint-test (push) Successful in 47s
The jump from "OK" to "Never again" is too big and most disappointments live in the gap (user, 2026-07-28) — so a merely poor meal either flattered itself as OK or got blacklisted. Only 'never' raises the warning on a future order. A blacklist that fires for every mediocre delivery is one nobody reads, so 'bad' records the disappointment without triggering the alarm. Both set order_again = false — you would not choose either again — and that split between "would I order it" and "warn me about it" is the point of the extra level. Migration widens the CHECK; nothing is removed, so no existing row needs mapping. |
||
|
siddharthd
|
0595d49d5c |
fix(orders): the restaurant is the merchant, not the courier
ci / lint-test (push) Successful in 43s
Reverses a change made on request. The platform in the headline
fragmented the merchant: the same restaurant read as two, depending on
who carried the bag, and that is not a distinction anyone rating the food
cares about. It also already has a home — the expandable Order details
panel renders expense_metadata.platform next to its heading, which is
where the user asked for it.
The fragmentation was worse than cosmetic. merchantVerdict joined on an
exact merchant_normalized, and the platforms capitalise differently
("TEG Kebabs & Biryani" on Uber Eats, "TEG KEBABS & BIRYANI" on
DoorDash), so one restaurant kept two separate histories and a "never
again" recorded through one app never warned in the other — silently
defeating the point of the memory. Now case-folded; verified on real
data, where the DoorDash order sees 1 prior verdict against 0 before.
81 existing descriptions backfilled in one transaction, dry-run first and
dumped beforehand. The regex is anchored to the end so suburb parens
survive: "Order - Coles (Wyndham Vale) (Uber Eats)" becomes
"Order - Coles (Wyndham Vale)", not "Order - Coles".
|
||
|
siddharthd
|
0cb46a087b |
feat(orders): record what we thought of an order, per person
The ledger already knew we had ordered from a place; it did not know the food was bad. Orders got repeated from places we disliked because nobody remembered by the time the next one went in. That is what the receipt ingestion was for (ING-9) and the last piece was missing: order_reviews existed as a table with no API, no UI and no writes. Four levels, not three. "Loved" and "liked" are both "would order again" but only one is worth a detour, and "ok" is not a recommendation. A verdict belongs to a person, not to an order. A shared meal produces two opinions and they routinely disagree — that disagreement is the useful part, and the old UNIQUE on transaction_id alone could not hold it. Now UNIQUE (transaction_id, participant_id), and the default is the signed-in user rather than the owner: Sonu authenticates through the same Traefik OAuth as participant 4, so an owner default would have filed her verdict under his name. Per-item opinions key on the item DESCRIPTION, not its index. An index is meaningless across orders; "the Pad Thai here is good" is the signal that has to survive into the next order from the same merchant. Only the two poles are offered — a per-item "ok" answers neither of the questions you ask at order time. Sharing is recorded as a real 50/50 split, not a decorative flag. The split already IS the record that an order was shared, and two records of one fact drift apart. An ABSENT item_verdicts means "leave them alone"; an empty array clears them. Without that distinction a note-only save silently wipes every per-item opinion — the same shape as the bug that reset `settled` on split rewrites, and just as invisible on screen. Mutation-tested: making keepItems a no-op fails exactly one test. mockDbWithPool gained queryRow. Omitting an export from the mock makes it undefined at the call site, which fails as "not a function" and reads like a code bug rather than a test-harness gap. |
||
|
siddharthd
|
788219b9fd |
fix(splits): the cutover date, not a boolean, is what gates a balance
ci / lint-test (push) Successful in 46s
Nothing dated before 2026-01-09 can be owed, because carryover transaction 2348
already carries the entire pre-cutover balance as a single figure. ACTIVE_OBLIGATION
now says so directly.
This inverts which mechanism is load-bearing, and that is the point. Until now
the only thing keeping $37,233.28 of paid debt out of the balances was
transaction_splits.settled -- a boolean that any delete-and-recreate write path
resets to false, as the split modal did until commit
|
||
|
siddharthd
|
dbfbd5196d |
fix(transactions): supersede rows imported twice instead of deleting them
ci / lint-test (push) Successful in 48s
Statements 107, 142 and 143 bill overlapping periods on one ANZ account, so 31 transactions -- $42,040.68 -- are in the ledger twice. They are marked superseded, not deleted. Every child of transactions is ON DELETE CASCADE (splits, tags, overrides, expense_metadata, order_reviews), so deleting "the duplicate" destroys whatever curation sits on it, and which member of a pair holds that curation is an accident of import order: here 1 pair carries splits and 6 carry overrides, all on the surviving side, but nothing guarantees that. Superseding keeps the row, keeps its children, and makes a mistake one UPDATE to undo rather than a restore from backup. reconciled_with_id could not be reused. Its predicate is scoped to statement_id IS NULL on purpose -- a statement line pointing at something else is the survivor, not the duplicate -- and here both rows are statement lines. The exclusion goes into EXCLUDE_RECONCILED_SOURCE rather than into a new fragment, so every query already asking "count each purchase once" gets it without being edited. The trip cost queries did not use that fragment at all and now do; verified a no-op on current data (0 trip-tagged rows are either reconciled sources or duplicates), but they were one import away from double-counting. Most of the $42k is transfers and investments, which spend already excludes. The damage was elsewhere: duplicated rows in the list, and rules re-splitting a duplicate -- txn 3807 is one of these 31 and was a candidate for splitting earlier today. Balances are unchanged: no duplicate carried a split. |
||
|
siddharthd
|
d5589b2980 |
feat(statements): flag billing periods that overlap another statement
ci / lint-test (push) Successful in 46s
An account cannot be billed twice for the same day, so an overlap means those
transactions are in the ledger twice. ANZ statements 107 and 143 overlap by 118
days and put roughly $42,000 of duplicate rows in; nothing anywhere said so.
Two details decide whether this catches the real case:
- Account numbers compare with non-digits stripped. The duplicate got in
because the existing key compared raw text and ANZ wrote the same account
as 408556264 on one statement and 4085-56264 on the other.
- The range is half-open. These statements are issued back-to-back with one
period ending the day the next starts, so inclusive bounds flagged 5 pairs
of which 3 were consecutive and fine. Half-open leaves exactly the 2 real
ones.
NULL bounds are excluded rather than handed to daterange, where NULL means
unbounded and an undated statement would overlap all of history.
Detection only. It does not refuse the import or touch the duplicate rows --
cleaning those is separate, and must supersede rather than delete because every
child of transactions is ON DELETE CASCADE and the curation sits on the
duplicate side.
Both subtleties have a test, and both fail if you undo them.
|
||
|
siddharthd
|
8c21893cc2 |
fix(trips): money that came back is not what the trip cost
ci / lint-test (push) Successful in 1m28s
Every trip figure filtered on transaction_type IN ('debit','fee','interest'),
which drops refunds and credits outright. A partly-refunded booking therefore
read at its full price and the refund subtracted nothing, anywhere: the
headline total_spend, the category breakdown, the daily chart, top merchants
and the tag breakdown were all gross.
This is the same defect the general analytics fixed once already, which is why
NET_SPEND_ROWS and SPEND_SIGNED exist -- a refunded Expedia purchase read as
$2,888.92 of spend until they did. Trip analytics never adopted them. Doing so
now costs one predicate and one expression per query.
getTrips/getTripById needed the trips alias moved to `tr`: the fragments assume
`t` is `transactions`, and hand-inlining a copy rather than renaming is exactly
how the reconciled-row exclusion drifted out of the analytics routes before.
On Europe 2026 this is $821.12 -- a LuxuryEscapes booking with two part-credits
against it, and a FreeNow hold adjustment. Fully cancelled bookings are a
different case and are handled by untagging both legs from the trip by hand,
because a trip never incurred a cost it cancelled.
No balance moves: the owed query already excludes credits and a refund carries
no split. There is a test asserting exactly that, and it passes with or without
this change -- it is a guard, not a proof. The three that do prove it fail
without it.
|
||
|
siddharthd
|
4fcb135805 |
fix(trips): a trip figure must only count what the owner is owed
ci / lint-test (push) Successful in 46s
The per-trip owed number shipped in
|
||
|
siddharthd
|
689fadc8b9 |
feat(shared): give a payment a tab to settle
A payment has only ever recorded from, to, amount and date. That is why the per-trip owed figure did not exist — getTripAnalytics said so where the number should have been: "split_payments carries no trip attribution, so a payment cannot be assigned to a trip. Settlement is a property of the whole relationship." Every trip therefore read 100% unsettled, including trips paid in full. It is also why the Shared page silently drops payments under a tag filter. With one global pool there was nothing honest to subtract, so it showed gross splits under the same label. A tag is a view; a scope is a ledger. The scope is a trip, not a new settlement_contexts table. trips already has owner_id, dates and archived, and transaction_overrides.trip_id already decides membership. A second grouping beside it would be two unsynchronised scopes over the same rows, with no invariant saying which governs. NULL means the ongoing household tab, which never closes. settled answers a different question and the two must not be collapsed: trip_id is which tab, settled is whether the obligation is still live. Critically, a live obligation is NOT settled by flipping the flag — it is settled by recording the payment, and the balance nets to zero on its own. Doing both would subtract the settlement twice. So settled is written only by the historical import, for repayments made on a platform we no longer run, and there is deliberately no "mark settled" action. Both owed figures now exclude settled splits and the trip figure nets its own payments. Spend analytics (myShare/mySplitOf) deliberately still count settled rows: my half of a 2025 grocery shop is my spend whether or not the other half was ever repaid, and filtering them would re-inflate exactly the figures importing settled history exists to correct. Also drops /api/participants/[id]/balance. It had no consumers, no owner scoping, no debit/credit signs and no EXCLUDE_RECONCILED_SOURCE — a fourth balance implementation that disagreed with the others and would have imported three bugs if anything had aligned to it. getTripAnalytics had no test at all. It has five now, including the one that matters: a household payment must not make a trip look paid. Verified by mutation — neutering the settled filter fails three, and dropping the trip filter on payments fails that one. |
||
|
siddharthd
|
a4ab543a6c |
fix(analytics): make the displayed numbers mean what they say
ci / lint-test (push) Failing after 44s
Six metric-integrity defects from the UI/IA review, plus two found while verifying the review's own claims against the code. The reconciled-row exclusion existed only in queries.ts. Every analytics route counted the superseded manual rows as spend — 48 rows, $4,474.79 of double count, invisible precisely because the transaction list looked right. It is now one fragment both sides import. The spend-pace chart computed its own totals in the browser: gross amounts, debits only, no personal share, no refunds, fees, interest or itemised loan repayments. On live data it ended July at $4,747.31 under a headline reading $3,597.10 — and its own baseline line was drawn from the split-adjusted monthly totals, so the two series in one chart disagreed with each other. Both now come from /api/analytics/daily, built from the same fragments as the headline. Fees aggregated every statement ever imported with no date filter, under a heading with no period, so a lifetime figure read as a current one and grew forever. Now bounded, labelled, and selectable. Comparisons no longer measure a month in progress against complete ones: the in-progress month is out of every baseline, and a selected current month is compared through the same day. Two the review did not catch: - Every analytics window was a day early. toISOString() on a local-midnight Date converts backwards through UTC. Surfaced only once fees started reporting the range it had used. - /monthly rounded per category, /daily per category-day, so the pace chart ended a few cents off the headline above it. Shared currency needed amending rather than applying. Reading s.currency would have labelled every order row AUD, since an order receipt has no statement and carries its own currency — the opposite convention from a foreign charge on an AUD statement, where amount IS AUD. NATIVE_CURRENCY's COALESCE order keeps the two apart. Balances also now count rows whose AUD value is genuinely unknown instead of netting a foreign figure against AUD ones. Latent today: no foreign transaction is currently split. Tag-filtered balance cards no longer claim "owes you". With a filter on, payments are deliberately not subtracted, so the figure is a split total and settling against it would record a payment for a debt that never was. Split-coverage warnings deliberately omitted (user decision). |
||
|
siddharthd
|
3bb67f370d |
feat(orders): show where an Uber trip went, in the list
ci / lint-test (push) Failing after 41s
Five rows all reading "Order - Uber Trip" are indistinguishable — the list gives you a date and an amount and nothing to tell one ride from another (user, 2026-07-27). Where the trip went is exactly what separates them, and it was already stored on expense_metadata.route since this morning; nothing in the list read it. getTransactions now joins the receipt (both directions — transaction_id OR matched_transaction_id, since a card-settled order points at the statement line instead) and the description cell renders "Terminal 2, Melbourne Airport (MEL) → 19 Lady Penrhyn Dr" in the same italic sub-line notes use. Two deliberate limits: - **A note the user wrote always wins.** This only fills an empty sub-line; it never occupies the notes field, which is theirs. - **Deliveries are excluded.** Their merchant already identifies them, so the restaurant's street address would be clutter on every food order. Gated on platform = 'uber'. The summary keeps the first two comma-segments of each address — a truncation, not a guess about geography. Uber puts the venue or street first, which is the identifying part; the full stops with their times stay in the title attribute. |
||
|
siddharthd
|
c656f5d26b |
feat(orders): read Uber trips, and reject the charge summary that duplicates them
ci / lint-test (push) Failing after 45s
Local rides are paid with credits (only overseas ones go on a card), so trips
belong to this slice and were simply never fetched — the Graph query searched
"order with Uber", the Eats subject. Captured 15 real messages from the mailbox
via a dry-run before touching anything, which found two defects that no amount
of reasoning about the template would have:
**Uber sends two mails per trip.** A "charge summary" when the ride ends, then
the real receipt when payment settles — same subject, same total. The summary
carries no tripReference, so order_reference fell back to `msg:<message-id>`
and I7 could not dedupe it against the receipt that follows. Every trip would
have been recorded twice. It says so itself ("This is not a payment receipt ...
You will receive a trip receipt when the payment is processed"), so it is now a
NotAReceiptError — 200 and silent, like every other expected non-receipt.
**Trip receipts label neither end of the journey.** Delivery receipts write
"1:20 pm - Pick-up"; trips print the time alone. The split regex put the time
into `label` and left `time` null. Time is now read properly, and a two-stop
trip is labelled Pick-up/Drop-off positionally — only where the receipt was
silent, so a template that does label its stops keeps its own wording.
Verified against all 15 captured messages: 7 trips recorded, 5 charge summaries
and 3 promotions skipped, 0 failures, no duplicate references. Two of the seven
are AUD credits-funded ($84.78 + $47.97) and would become transactions; the
five NZD ones are card-settled and correctly create provenance only (I5).
Fixtures ut-00 (local credits trip), ut-01 (overseas card trip) and ut-summary
(the charge summary) are captured mail, not written by hand.
|
||
|
siddharthd
|
4febf38292 |
test(orders): clean statement fixtures before ingest, not after
ci / lint-test (push) Failing after 40s
These tests insert a Westpac statement and a `DD *DOORDASH ...` charge, and only removed them at the end of the test — so they survived into the next run, where `reconcileCardLeg` could match one at ingest time and resolve an order that was meant to park `awaiting_card_statement`. That is a real ordering bug in the fixtures regardless. It is my best explanation for the intermittent failure in "parks an unresolvable split", but I could not reproduce it: seeding the exact leftover row and running the old code passed anyway. So this is hygiene with a plausible mechanism, not a confirmed fix — if that test fails again, this was not the cause. |
||
|
siddharthd
|
b6cd62f7b5 |
feat(orders): show the receipt in the transaction detail panel
ci / lint-test (push) Failing after 47s
`expense_metadata` has held the itemised receipt since ingestion started and nothing in the UI ever read it. A transaction that came from a DoorDash or Uber Eats receipt showed a merchant and an amount, with the item list and the delivery addresses sitting unread in the row behind it (user, 2026-07-27). Adds GET /api/transactions/[id]/order and an "Order details" section in the edit modal: line items with their options, pick-up/delivery stops with times and addresses, the card tail when one was involved, and the provider's own order reference. Two details that matter: - The lookup resolves from **both** sides — `transaction_id` OR `matched_transaction_id`. A card-settled order creates no transaction of its own (I5); the receipt points at the statement line instead. Matching only on transaction_id would have left the panel blank on exactly the card-paid orders, which are the ones whose detail is hardest to find elsewhere. - An empty item list says so in words rather than rendering nothing. Uber itemises groceries but not restaurant orders, and orders ingested before the Uber item parser existed have none either — a blank section reads as a bug when it is usually the receipt. Read-only. This is what a provider sent; editing it would make provenance mean nothing. |
||
|
siddharthd
|
df4b875b82 |
feat(orders): make an ingested order legible in the transactions view
ci / lint-test (push) Failing after 43s
Four things the view could not tell you, all from reading the rows (user,
2026-07-27).
**Which platform.** The parser has always known — it has to, to read the
template — and then discarded it. "Order - Burger Corner" gives no way to know
whether to open DoorDash or Uber Eats for the detail, and restaurants exist on
both. Now stored on expense_metadata and named in the description:
"Order - Burger Corner (Uber Eats)". Migration 0021 recovers it for the 101
backfilled rows from the order_reference shape — DoorDash receipts carry no id
of their own so ingestion synthesises `msg:<message-id>`, Uber carries a real
trip UUID, which makes the discriminator exact.
**Bank said "Manual".** That label is derived, not stored, and "Manual" reads
as "hand-entered, still awaiting a card line to match". A gift-card order has
no card line coming, ever. It now reads "Gift Card", and — the part that
actually mattered — credits joins cash in needsCardMatch(), so these stop
sitting in the pending-reconciliation queue. All 81 were queued against a match
that could not exist.
**Uber line items were never parsed.** 67 of 101 orders had none. Uber itemises
groceries but not restaurant orders, so some of that is genuine; the rest was
simply unread. Its markup is better than DoorDash's — every cell carries a
data-testid with the item's uuid, so qty/title/amount bind by id rather than by
column position. Sold-out items (0.00) are kept: they are why a total is lower
than what was ordered.
**Uber prints pick-up and delivery addresses on every receipt** and they were
thrown away. Captured as `route` [{label, time, address}], de-duplicated
because the template renders the whole block twice for narrow screens. Wording
is kept as printed ("Pick-up" on some receipts, "Pickup" on others) rather than
normalised, so a template change stays visible. This is the same block a *trip*
receipt uses for start and destination — rides are not ingested today, but the
reader will not need changing when they are.
Also stores source_email_subject/from, which order ingestion had left null on
columns that already existed.
Verified against the captured corpus: route on all 6 Uber fixtures, 5/5 items
on the GLOMARK grocery receipt including the sold-out one. Production data
updated by smarthome:docker/scripts/order-presentation-2026-07-27.sql
(81 descriptions, `backfill` tag, re-run clean). `route` and Uber line items
are parsed from here on only — recovering them for already-ingested orders
means re-reading the mail, which I7 idempotency refuses by design.
|