Being sent to a web app to answer "was this shared?" is enough friction
that the question stops getting answered — which is the exact failure the
nudge exists to prevent. So the buttons now act in place: pressing
"Shared 50/50" writes the transaction_splits row and edits the message,
and the app is never opened.
Slack does NOT reach this route directly. It posts to an n8n webhook that
forwards the raw body and signature headers here (user's suggestion).
That is the better shape: n8n already terminates public webhooks, so the
app keeps its blanket OAuth chain and gains no internet-facing
unauthenticated route, and no Traefik change is needed. n8n cannot verify
the signature itself — its Code sandbox has no `require`, so no `crypto`.
Two independent gates, both failing closed: the shared x-ingest-token
(came from n8n) and Slack's v0 signature over the raw body (came from
Slack, not replayed within 5 minutes). An unset signing secret rejects
everything rather than waving it through, because the alternative turns a
misconfigured deploy into an open write endpoint.
An unmapped Slack user is refused rather than defaulted to the owner. In
a two-person household a wrong attribution is not a rounding error, it is
the other person's opinion recorded under your name.
Block Kit is built in the app (ingest returns slack_blocks) rather than
in n8n expressions: a template string is untestable, and this shape has
to stay in step with what the interactive endpoint re-renders after each
press. Null when there is no transaction yet — a card-settled order is
parked until its statement arrives, so there is nothing to split or rate.
Also: /transactions now honours ?q=, so the link lands on the row instead
of the top of an unfiltered ledger.
The Slack order nudge links here. Without it the link drops you at the
top of an unfiltered ledger and the merchant has to be found by hand,
which is how a nudge stops being opened.
The ledger already knew we had ordered from a place; it did not know the
food was bad. Orders got repeated from places we disliked because nobody
remembered by the time the next one went in. That is what the receipt
ingestion was for (ING-9) and the last piece was missing: order_reviews
existed as a table with no API, no UI and no writes.
Four levels, not three. "Loved" and "liked" are both "would order again"
but only one is worth a detour, and "ok" is not a recommendation.
A verdict belongs to a person, not to an order. A shared meal produces
two opinions and they routinely disagree — that disagreement is the
useful part, and the old UNIQUE on transaction_id alone could not hold
it. Now UNIQUE (transaction_id, participant_id), and the default is the
signed-in user rather than the owner: Sonu authenticates through the same
Traefik OAuth as participant 4, so an owner default would have filed her
verdict under his name.
Per-item opinions key on the item DESCRIPTION, not its index. An index is
meaningless across orders; "the Pad Thai here is good" is the signal that
has to survive into the next order from the same merchant. Only the two
poles are offered — a per-item "ok" answers neither of the questions you
ask at order time.
Sharing is recorded as a real 50/50 split, not a decorative flag. The
split already IS the record that an order was shared, and two records of
one fact drift apart.
An ABSENT item_verdicts means "leave them alone"; an empty array clears
them. Without that distinction a note-only save silently wipes every
per-item opinion — the same shape as the bug that reset `settled` on
split rewrites, and just as invisible on screen. Mutation-tested: making
keepItems a no-op fails exactly one test.
mockDbWithPool gained queryRow. Omitting an export from the mock makes it
undefined at the call site, which fails as "not a function" and reads
like a code bug rather than a test-harness gap.
Every warning in it was inverted by this week's work:
- "settled is dead data, false on every row" -- there are now 1,266 settled
splits across 657 pre-2026 transactions.
- "do not fix getParticipantBalances to exclude settled splits" -- it now
excludes them, via ACTIVE_OBLIGATION, and must.
- "settlement cannot be attributed per trip" -- migration 0022 added
split_payments.trip_id and it is attributed.
- "splits exist from 2026-01-09 only" -- pre-2026 transactions are now split
deliberately, to stop them inflating spend.
Replaced with what is actually true, including the rule that matters most: the
cutover DATE is the primary balance gate and the settled flag only refines it,
so pre-2026 expenses can be split freely.
Nothing dated before 2026-01-09 can be owed, because carryover transaction 2348
already carries the entire pre-cutover balance as a single figure. ACTIVE_OBLIGATION
now says so directly.
This inverts which mechanism is load-bearing, and that is the point. Until now
the only thing keeping $37,233.28 of paid debt out of the balances was
transaction_splits.settled -- a boolean that any delete-and-recreate write path
resets to false, as the split modal did until commit 6add958. Losing the flag on
a pre-cutover row now costs nothing: the date still excludes it. The flag matters
only on or after the cutover, marking the few settled outside this app.
It also makes splitting history safe to do freely. A split on a 2024 grocery
shop can now describe how the expense was shared -- which is what stops it
inflating spend -- without asserting a debt that was settled years ago. That was
the whole reason not to split pre-2026 expenses, and it no longer applies.
The bound is inclusive because transaction 2348 is itself dated 2026-01-09; an
exclusive one would drop the carryover and with it the entire pre-cutover
balance.
Corrects one live figure: a Woolworths on 2026-01-06 was split 50/50 three days
before the cutover, double-counting $7.55 against the carryover. Sonu
$5,428.08 -> $5,420.53.
The test fixture default moved to 2026-06-15 -- it was 2024-06-15, which is now
pre-cutover and made every balance fixture read zero. That the suite caught this
is the guard working.
The route replaces every split for a transaction rather than editing in place,
so the recreated rows took the column default settled=false. Opening the split
modal on a historical transaction and saving it therefore converted a
discharged obligation into a live one, with nothing on screen saying so.
That is not theoretical. 657 pre-2026 transactions now carry settled splits
imported from SplitMyExpenses -- $37,233.28 of balance that the carryover
(transaction 2348) already accounts for. Editing one would double-count its
share against a debt that was paid years ago.
Now carries settled and settled_at across the rewrite, per participant, the
same way the rules revert route already does. Changing someone's percentage
does not re-open the obligation: it was settled outside this app and stays
settled. A participant who was not on the transaction before is a genuinely new
obligation and correctly starts unsettled.
rule-actions.ts was already safe here -- it upserts ON CONFLICT DO UPDATE SET
share_percent, so it never touches the flag.
scripts/split_csv_match.py needs psycopg2, so scripts/ now has a venv beside it.
This repo deploys from its working tree, so an untracked .venv would be swept
into the Docker build context.
Sonu's two "transfer" payments are split by scope, Europe first and the
remainder to household, chronologically so each settles what was outstanding
when it was made. Both Europe tabs now read $0.00.
No schema change was needed and that is the point worth writing down:
split_payments has no unique constraint on linked_transaction_id, so one bank
transfer carries one row per scope and the rows re-add to the transfer --
verified, 4111 sums to $3,779.33 and 4121 to $4,794.06.
Her overall balance is unchanged at $5,428.08. Allocation moves money between
tabs, never between people; that invariance is the check to repeat on any
future re-allocation.
Trip 3, 2026-04-12 to 2026-04-28, 124 rows, $9,914.24. It was marked only by
tag 5 and so was invisible to every trip figure.
It reads at first like a sharing scope overlapping the group trip, because the
tag's earliest row is 17 March. It is not: that is a single advance booking
(Ticketmaster Nanterre), 8 more rows fall on the 12 April handover day and were
already held out of Europe 2026, and the remaining 115 run 13-28 April. A clean
sequential leg.
Also records that grouped payments need no schema change -- split_payments has
no unique constraint on linked_transaction_id, so one transfer can carry a row
per scope -- and that the 23 Apr Qantas booking is the flight to Bangkok
starting a solo leg, which is why it stays out of this trip.
The 2026-07-26 document was a proposal marked "nothing built". Everything it
described as broken is now fixed, and the fix is not the one it proposed, so
leaving it in place would misdescribe the system to whoever reads it next.
Records what the code now does: settled as the single balance gate, settled and
trip_id as orthogonal axes, settling up by recording a payment rather than
flipping a flag, and the reasons duplicates are superseded rather than deleted.
Keeps the loan design intact and clearly marked as still a proposal -- it was
never built and nothing in this work touched it.
Also records that the proposal's own recommendation not to restate history from
the CSVs was overturned, and why it was wrong: it measured the value in
balances, where it is nil, and missed it in spend, where it is $35,259.
Adds --write to the matcher. Wrote 1,242 split rows across 657 transactions.
Imported settled, and that is the whole design. These obligations were
discharged years ago on a platform we no longer run, and their residual is
already carried by transaction 2348. Writing them unsettled would re-open
roughly $40k of debts that were paid. ACTIVE_OBLIGATION keeps settled splits
out of every owed figure while myShare/mySplitOf still count them, which is
exactly the asymmetry this needs: the import exists to correct historical
SPEND, not to move a balance.
Effect: $35,259 leaves my historical spend -- $13,088 in 2024, $22,117 in
2025 -- because a $200 grocery shop that was always half hers no longer reads
as $200 of mine. Balances are byte-identical before and after (Molina
-1226.72/145, Sonu 5428.08/419), which is the assertion that matters.
Shares are written as the CSV computed them, so a 50/50 row can land as
50.01/49.99. That is faithful rather than tidy; no transaction exceeds 100%.
Rehearsed on the 37-row Rome file first (24 rows) and verified before the full
run -- both the balances and one split read back through the API.
Matches the five CSV exports against transactions already in the ledger and
reports what it would do. Writes nothing -- importing is a separate step, and
rehearsing it first is what catches the defects that tests do not.
What it found, and why the number is what it is: 676 of 1,536 shareable rows
match (44%). The ceiling is ledger coverage, not matcher quality. The CSVs
describe 678 shared expenses in 2024 alone; the ledger holds 591 rows for the
whole of that year, 3 to 72 a month, which is far less than a household
actually spends. Most 2024 CSV rows have no transaction to attach a split to
and never will. South Korea April 2024 matches 4 of 158 for that reason.
Three decisions are encoded deliberately:
- Date format is decided per FILE, not per row. The household export writes
D/M/YYYY and the four trip exports write ISO, and 474 rows parse validly
under both readings -- per-row guessing silently swaps January and February
for some rows and not others.
- A person's column is their net balance impact, not their share. The payer
is whoever is positive; the other's share is |their negative| / cost. So a
+cost/-cost row means the other party owes 100%, not that the expense was
unshared -- the reading that would fake an arrangement change.
- Matching is one-to-one, best pair first. The NZ trip has two identical
$10.16 Uber rows against three ledger rows and four PayMyPark rows in the
same shape; without this a ledger row is claimed repeatedly and the second
CSV row looks matched while being unrepresented.
Statements 107, 142 and 143 bill overlapping periods on one ANZ account, so 31
transactions -- $42,040.68 -- are in the ledger twice.
They are marked superseded, not deleted. Every child of transactions is ON
DELETE CASCADE (splits, tags, overrides, expense_metadata, order_reviews), so
deleting "the duplicate" destroys whatever curation sits on it, and which
member of a pair holds that curation is an accident of import order: here 1
pair carries splits and 6 carry overrides, all on the surviving side, but
nothing guarantees that. Superseding keeps the row, keeps its children, and
makes a mistake one UPDATE to undo rather than a restore from backup.
reconciled_with_id could not be reused. Its predicate is scoped to
statement_id IS NULL on purpose -- a statement line pointing at something else
is the survivor, not the duplicate -- and here both rows are statement lines.
The exclusion goes into EXCLUDE_RECONCILED_SOURCE rather than into a new
fragment, so every query already asking "count each purchase once" gets it
without being edited. The trip cost queries did not use that fragment at all
and now do; verified a no-op on current data (0 trip-tagged rows are either
reconciled sources or duplicates), but they were one import away from
double-counting.
Most of the $42k is transfers and investments, which spend already excludes.
The damage was elsewhere: duplicated rows in the list, and rules re-splitting a
duplicate -- txn 3807 is one of these 31 and was a candidate for splitting
earlier today.
Balances are unchanged: no duplicate carried a split.
An account cannot be billed twice for the same day, so an overlap means those
transactions are in the ledger twice. ANZ statements 107 and 143 overlap by 118
days and put roughly $42,000 of duplicate rows in; nothing anywhere said so.
Two details decide whether this catches the real case:
- Account numbers compare with non-digits stripped. The duplicate got in
because the existing key compared raw text and ANZ wrote the same account
as 408556264 on one statement and 4085-56264 on the other.
- The range is half-open. These statements are issued back-to-back with one
period ending the day the next starts, so inclusive bounds flagged 5 pairs
of which 3 were consecutive and fine. Half-open leaves exactly the 2 real
ones.
NULL bounds are excluded rather than handed to daterange, where NULL means
unbounded and an undated statement would overlap all of history.
Detection only. It does not refuse the import or touch the duplicate rows --
cleaning those is separate, and must supersede rather than delete because every
child of transactions is ON DELETE CASCADE and the curation sits on the
duplicate side.
Both subtleties have a test, and both fail if you undo them.
A participant who has overpaid a trip showed as "$-816.16" under a column
headed "Outstanding on this trip". A negative outstanding reads as a bug
rather than as "they are ahead", so the sign is now spelled out: magnitude
plus one of all square / owes you / ahead — you owe them, coloured the same
way Shared colours the same three states.
Also corrects the footer, which had gone stale and was now simply false. It
said settlement could not be computed per trip because payments carried no
trip attribution. Migration 0022 added split_payments.trip_id and the figures
above it have been net of trip-scoped payments since. What a reader needs to
know is the opposite of what it said: household-tab payments are the ones NOT
counted here.
Total Spend is every payer's trip-tagged spending; the split figures directly
below it are scoped to the owner. Two lenses on one screen read as one unless
the card says which it is. The number is unchanged and deliberate -- a trip
cost what the group put into it -- so this is a label, not a fix.
Every trip figure filtered on transaction_type IN ('debit','fee','interest'),
which drops refunds and credits outright. A partly-refunded booking therefore
read at its full price and the refund subtracted nothing, anywhere: the
headline total_spend, the category breakdown, the daily chart, top merchants
and the tag breakdown were all gross.
This is the same defect the general analytics fixed once already, which is why
NET_SPEND_ROWS and SPEND_SIGNED exist -- a refunded Expedia purchase read as
$2,888.92 of spend until they did. Trip analytics never adopted them. Doing so
now costs one predicate and one expression per query.
getTrips/getTripById needed the trips alias moved to `tr`: the fragments assume
`t` is `transactions`, and hand-inlining a copy rather than renaming is exactly
how the reconciled-row exclusion drifted out of the analytics routes before.
On Europe 2026 this is $821.12 -- a LuxuryEscapes booking with two part-credits
against it, and a FreeNow hold adjustment. Fully cancelled bookings are a
different case and are handled by untagging both legs from the trip by hand,
because a trip never incurred a cost it cancelled.
No balance moves: the owed query already excludes credits and a refund carries
no split. There is a test asserting exactly that, and it passes with or without
this change -- it is a guard, not a proof. The three that do prove it fail
without it.
The per-trip owed number shipped in 689fadc counted every split on every
trip transaction regardless of who paid, so it silently mixed debts owed to
different people under one label.
On Europe 2026 that meant Molina "owed" $21,572.12, of which $1,605.49 was
her share of rows Sonu paid for — a real debt, but between the other two
participants, and one they had already settled directly (split_payments id
5, Molina -> Sonu, exactly $1,605.49). A participant's own share of a row
they themselves paid for was in there too, which is nobody's debt at all.
Both sides needed scoping, not just one: the owed side to rows this owner
paid for, and the paid side to payments made to this owner. Scoping only
the first would have let a Molina -> Sonu payment reduce what Molina owes
the owner.
The corrected figures reproduce a number derived independently, months of
data apart: Molina now reads -$816.16 on Europe, matching her known
overpayment to the cent ($19,966.63 of splits against $20,782.79 paid).
Sonu goes from $8,793.10 to $1,084.61, and the owner correctly disappears
from a list of people who owe the owner.
Found by checking a household total against what the app had been showing
all along — the query was gross, gave a number about twice the real one,
and I had quoted it as "owed". Worth stating plainly: the defect was not in
the number the app displayed, it was in the number I computed to explain it.
The pipeline has been red on every run since at least ae0c34f. npm ci
installs dependencies but the Prisma client is generated into
src/generated/prisma, which is gitignored — so a fresh CI checkout has no
client and anything importing src/lib/db.ts fails with 'Cannot find package
@/generated/prisma/client' before a single assertion runs.
prisma generate reads only the schema, so it needs no database and no
secrets.
Worth noting why this went unnoticed for a dozen commits: a pipeline that
is always red carries no signal, so it stopped being read.
a4ab543 landed six hours ago and this branch rewrote one of the queries it
had just fixed, quietly dropping both of its guarantees.
That commit made EXCLUDE_RECONCILED_SOURCE "one fragment both sides import"
because an inlined copy is how the reconciled-row exclusion drifted out of
the analytics routes and double-counted 48 rows / $4,474.79. The trip owed
query here had hand-inlined its own copy — the fragment assumes the alias
`t` and this query used `tx`, so the path of least resistance was to
re-create exactly the divergence that was being removed. Aliased to `t` so
the fragments apply directly.
The same commit made balances count rows whose AUD value is unknown rather
than netting a foreign figure against AUD ones. The trip figure had no
equivalent — on the query where it matters most, because a trip is where
foreign rows actually live. A Europe total silently mixing EUR into AUD is
the whole failure that fix was written to prevent.
The column header still read "Share of this trip" while the number is now
net of payments, which is the same class of drift a4ab543 set out to fix.
It reads "Outstanding on this trip", carries the approx/unconverted caveat
the Shared cards use, and greys a settled zero.
A payment has only ever recorded from, to, amount and date. That is why
the per-trip owed figure did not exist — getTripAnalytics said so where
the number should have been: "split_payments carries no trip attribution,
so a payment cannot be assigned to a trip. Settlement is a property of the
whole relationship." Every trip therefore read 100% unsettled, including
trips paid in full.
It is also why the Shared page silently drops payments under a tag filter.
With one global pool there was nothing honest to subtract, so it showed
gross splits under the same label. A tag is a view; a scope is a ledger.
The scope is a trip, not a new settlement_contexts table. trips already
has owner_id, dates and archived, and transaction_overrides.trip_id
already decides membership. A second grouping beside it would be two
unsynchronised scopes over the same rows, with no invariant saying which
governs. NULL means the ongoing household tab, which never closes.
settled answers a different question and the two must not be collapsed:
trip_id is which tab, settled is whether the obligation is still live.
Critically, a live obligation is NOT settled by flipping the flag — it is
settled by recording the payment, and the balance nets to zero on its own.
Doing both would subtract the settlement twice. So settled is written only
by the historical import, for repayments made on a platform we no longer
run, and there is deliberately no "mark settled" action.
Both owed figures now exclude settled splits and the trip figure nets its
own payments. Spend analytics (myShare/mySplitOf) deliberately still count
settled rows: my half of a 2025 grocery shop is my spend whether or not the
other half was ever repaid, and filtering them would re-inflate exactly the
figures importing settled history exists to correct.
Also drops /api/participants/[id]/balance. It had no consumers, no owner
scoping, no debit/credit signs and no EXCLUDE_RECONCILED_SOURCE — a fourth
balance implementation that disagreed with the others and would have
imported three bugs if anything had aligned to it.
getTripAnalytics had no test at all. It has five now, including the one
that matters: a household payment must not make a trip look paid. Verified
by mutation — neutering the settled filter fails three, and dropping the
trip filter on payments fails that one.
Six metric-integrity defects from the UI/IA review, plus two found while
verifying the review's own claims against the code.
The reconciled-row exclusion existed only in queries.ts. Every analytics
route counted the superseded manual rows as spend — 48 rows, $4,474.79 of
double count, invisible precisely because the transaction list looked
right. It is now one fragment both sides import.
The spend-pace chart computed its own totals in the browser: gross
amounts, debits only, no personal share, no refunds, fees, interest or
itemised loan repayments. On live data it ended July at $4,747.31 under a
headline reading $3,597.10 — and its own baseline line was drawn from the
split-adjusted monthly totals, so the two series in one chart disagreed
with each other. Both now come from /api/analytics/daily, built from the
same fragments as the headline.
Fees aggregated every statement ever imported with no date filter, under
a heading with no period, so a lifetime figure read as a current one and
grew forever. Now bounded, labelled, and selectable.
Comparisons no longer measure a month in progress against complete ones:
the in-progress month is out of every baseline, and a selected current
month is compared through the same day.
Two the review did not catch:
- Every analytics window was a day early. toISOString() on a
local-midnight Date converts backwards through UTC. Surfaced only once
fees started reporting the range it had used.
- /monthly rounded per category, /daily per category-day, so the pace
chart ended a few cents off the headline above it.
Shared currency needed amending rather than applying. Reading s.currency
would have labelled every order row AUD, since an order receipt has no
statement and carries its own currency — the opposite convention from a
foreign charge on an AUD statement, where amount IS AUD. NATIVE_CURRENCY's
COALESCE order keeps the two apart. Balances also now count rows whose AUD
value is genuinely unknown instead of netting a foreign figure against AUD
ones. Latent today: no foreign transaction is currently split.
Tag-filtered balance cards no longer claim "owes you". With a filter on,
payments are deliberately not subtracted, so the figure is a split total
and settling against it would record a payment for a debt that never was.
Split-coverage warnings deliberately omitted (user decision).
Enrichment is the point of the ingestion pipeline (DECISIONS ING-8) — a bank
statement gives a date, an amount and a mangled descriptor, and everything that
makes a transaction understandable arrives by email. It was all reachable only
by opening the edit modal, which is a strange place to look for "what was in
this order".
Rows with a receipt behind them get a disclosure arrow in the description cell;
clicking expands an inline panel with the line items, the pick-up and drop-off
stops, the card tail and the provider's reference. Several rows can be open at
once — the point is comparing orders without losing your place.
The arrow appears only where `order_platform` is set. Putting one on every
transaction would promise detail that mostly does not exist.
OrderDetails moves out of edit-transaction-modal.tsx into its own component so
both surfaces render the same thing; `bare` drops the modal's top border when
it sits in a table row.
Five rows all reading "Order - Uber Trip" are indistinguishable — the list gives
you a date and an amount and nothing to tell one ride from another (user,
2026-07-27). Where the trip went is exactly what separates them, and it was
already stored on expense_metadata.route since this morning; nothing in the list
read it.
getTransactions now joins the receipt (both directions — transaction_id OR
matched_transaction_id, since a card-settled order points at the statement line
instead) and the description cell renders "Terminal 2, Melbourne Airport (MEL)
→ 19 Lady Penrhyn Dr" in the same italic sub-line notes use.
Two deliberate limits:
- **A note the user wrote always wins.** This only fills an empty sub-line; it
never occupies the notes field, which is theirs.
- **Deliveries are excluded.** Their merchant already identifies them, so the
restaurant's street address would be clutter on every food order. Gated on
platform = 'uber'.
The summary keeps the first two comma-segments of each address — a truncation,
not a guess about geography. Uber puts the venue or street first, which is the
identifying part; the full stops with their times stay in the title attribute.
Trip rows read "Order - Uber Trip (Uber)". The suffix exists so you can tell
where to go and look; when the merchant is literally "Uber Trip" it says
nothing. What identifies a trip is its two addresses, and those are in the
Order details panel. Existing rows updated in prod.
Local rides are paid with credits (only overseas ones go on a card), so trips
belong to this slice and were simply never fetched — the Graph query searched
"order with Uber", the Eats subject. Captured 15 real messages from the mailbox
via a dry-run before touching anything, which found two defects that no amount
of reasoning about the template would have:
**Uber sends two mails per trip.** A "charge summary" when the ride ends, then
the real receipt when payment settles — same subject, same total. The summary
carries no tripReference, so order_reference fell back to `msg:<message-id>`
and I7 could not dedupe it against the receipt that follows. Every trip would
have been recorded twice. It says so itself ("This is not a payment receipt ...
You will receive a trip receipt when the payment is processed"), so it is now a
NotAReceiptError — 200 and silent, like every other expected non-receipt.
**Trip receipts label neither end of the journey.** Delivery receipts write
"1:20 pm - Pick-up"; trips print the time alone. The split regex put the time
into `label` and left `time` null. Time is now read properly, and a two-stop
trip is labelled Pick-up/Drop-off positionally — only where the receipt was
silent, so a template that does label its stops keeps its own wording.
Verified against all 15 captured messages: 7 trips recorded, 5 charge summaries
and 3 promotions skipped, 0 failures, no duplicate references. Two of the seven
are AUD credits-funded ($84.78 + $47.97) and would become transactions; the
five NZD ones are card-settled and correctly create provenance only (I5).
Fixtures ut-00 (local credits trip), ut-01 (overseas card trip) and ut-summary
(the charge summary) are captured mail, not written by hand.
These tests insert a Westpac statement and a `DD *DOORDASH ...` charge, and
only removed them at the end of the test — so they survived into the next run,
where `reconcileCardLeg` could match one at ingest time and resolve an order
that was meant to park `awaiting_card_statement`.
That is a real ordering bug in the fixtures regardless. It is my best
explanation for the intermittent failure in "parks an unresolvable split",
but I could not reproduce it: seeding the exact leftover row and running the
old code passed anyway. So this is hygiene with a plausible mechanism, not a
confirmed fix — if that test fails again, this was not the cause.
`expense_metadata` has held the itemised receipt since ingestion started and
nothing in the UI ever read it. A transaction that came from a DoorDash or Uber
Eats receipt showed a merchant and an amount, with the item list and the
delivery addresses sitting unread in the row behind it (user, 2026-07-27).
Adds GET /api/transactions/[id]/order and an "Order details" section in the
edit modal: line items with their options, pick-up/delivery stops with times
and addresses, the card tail when one was involved, and the provider's own
order reference.
Two details that matter:
- The lookup resolves from **both** sides — `transaction_id` OR
`matched_transaction_id`. A card-settled order creates no transaction of its
own (I5); the receipt points at the statement line instead. Matching only on
transaction_id would have left the panel blank on exactly the card-paid
orders, which are the ones whose detail is hardest to find elsewhere.
- An empty item list says so in words rather than rendering nothing. Uber
itemises groceries but not restaurant orders, and orders ingested before the
Uber item parser existed have none either — a blank section reads as a bug
when it is usually the receipt.
Read-only. This is what a provider sent; editing it would make provenance mean
nothing.
Four things the view could not tell you, all from reading the rows (user,
2026-07-27).
**Which platform.** The parser has always known — it has to, to read the
template — and then discarded it. "Order - Burger Corner" gives no way to know
whether to open DoorDash or Uber Eats for the detail, and restaurants exist on
both. Now stored on expense_metadata and named in the description:
"Order - Burger Corner (Uber Eats)". Migration 0021 recovers it for the 101
backfilled rows from the order_reference shape — DoorDash receipts carry no id
of their own so ingestion synthesises `msg:<message-id>`, Uber carries a real
trip UUID, which makes the discriminator exact.
**Bank said "Manual".** That label is derived, not stored, and "Manual" reads
as "hand-entered, still awaiting a card line to match". A gift-card order has
no card line coming, ever. It now reads "Gift Card", and — the part that
actually mattered — credits joins cash in needsCardMatch(), so these stop
sitting in the pending-reconciliation queue. All 81 were queued against a match
that could not exist.
**Uber line items were never parsed.** 67 of 101 orders had none. Uber itemises
groceries but not restaurant orders, so some of that is genuine; the rest was
simply unread. Its markup is better than DoorDash's — every cell carries a
data-testid with the item's uuid, so qty/title/amount bind by id rather than by
column position. Sold-out items (0.00) are kept: they are why a total is lower
than what was ordered.
**Uber prints pick-up and delivery addresses on every receipt** and they were
thrown away. Captured as `route` [{label, time, address}], de-duplicated
because the template renders the whole block twice for narrow screens. Wording
is kept as printed ("Pick-up" on some receipts, "Pickup" on others) rather than
normalised, so a template change stays visible. This is the same block a *trip*
receipt uses for start and destination — rides are not ingested today, but the
reader will not need changing when they are.
Also stores source_email_subject/from, which order ingestion had left null on
columns that already existed.
Verified against the captured corpus: route on all 6 Uber fixtures, 5/5 items
on the GLOMARK grocery receipt including the sold-out one. Production data
updated by smarthome:docker/scripts/order-presentation-2026-07-27.sql
(81 descriptions, `backfill` tag, re-run clean). `route` and Uber line items
are parsed from here on only — recovering them for already-ingested orders
means re-reading the mail, which I7 idempotency refuses by design.
Both were found by looking at the data after the live backfill, not by the
suite — 105 tests were green while 85 rows were invisible and 4 were double
counted.
owner_id was NULL on every ingested order. Analytics scope on
COALESCE(t.owner_id, s.owner_id), and an ingested order carries no statement,
so the coalesce resolved to NULL and matched no owner. The rows existed in
`transactions` and appeared in no view in the app. Ingestion now sets
DEFAULT_OWNER_ID, and a regression test asserts the row survives the same
COALESCE scoping the UI uses.
[Family] orders are card-settled, not credits-funded. Their receipts name the
payer ("Payments Siddharth LKR 3,783.20") and no instrument, which an earlier
version read as credits. The card statement carries all four of them (CBA
...3893, exact foreign_currency_amount matches), so creating a transaction
duplicated spend already recorded — the double-count I5 exists to prevent.
They now record provenance only; the statement line is the transaction and is
what carries the `family` tag that keeps them out of budgets.
Production data corrected separately by
smarthome:docker/scripts/fix-order-backfill-2026-07-27.sql.
Also: reconciliation tests no longer assert global row counts.
reconcilePendingOrders() scans every pending row, so leftovers from other
files moved the totals — the source of an intermittent failure that only
appeared on the first run after a source edit.
Backfill dry-run over 130 real messages surfaced one 422: 'payments sum to 1.17
but receipt states 16.50'. The receipt is a mixed Uber payment —
Uber Cash $1.17 + Westpac ••••8032 $15.33 — and the card regex only matched
Visa|MasterCard|American Express|Amex, so an issuer-named leg was dropped
entirely. validateOrderTotals correctly refused it rather than recording $1.17
as the cost of a $16.50 order.
Anchors on the ••••NNNN masking instead, which also covers the form already
seen in the corpus ('Mastercard ••••3893 (CBA Ultimate) CHF 51.23'). Fixture
and regression test added.
Also repoints .env.test at the current postgres-personal container IP and
documents why: the container publishes no host port, so the address changes on
every recreate and the whole integration suite fails with connection errors
until it is refreshed.
None of these were caught by 105 green tests, because the code they live in was
barely tested and the HTTP path was not tested at all.
1. reconcilePendingOrders hardcoded category 'dining', so any order resolved
through the deferred path booked as dining regardless of merchant — a
Woolworths grocery order that parks and later reconciles was misfiled.
That reintroduced, through the back door, exactly the misfiling
resolveCategory() exists to prevent. Now calls it.
2. reconcileCardLeg never marked a statement line as consumed, so two orders on
the same card inside the +/-4 day window both bound to the same charge and
each booked its own credits remainder — double-counting spend. At 10-15
orders a month on one card that is not a corner case. Migration 0020 adds
matched_transaction_id with a unique index; the matcher now excludes lines
already claimed.
3. The ingest API returned HTTP 200 for every parse failure, and the Slack
alert fires only on non-200. So the single most likely production failure —
a provider template change breaking every order at once — was completely
silent. Split into NotAReceiptError (promotions, delivery updates, refund
and adjustment notices: 200, silent, expected traffic) and OrderParseError
(it IS a receipt and would not parse: 422, alerts).
Also: order_reference now anchors on Uber's own tripReference cell rather than
'first UUID in the document'. I had claimed to verify that the first UUID was
always the order UUID; that check compared against zero samples and was
vacuous. tripReference is present in all 29 captured receipts and, for ue-00,
equals the UUID the PDF redirect resolves to. The positional fallback remains
but only flags when there is genuine ambiguity.
Adds the API route's first tests — auth gate and error taxonomy — plus
anchoring regressions. 63 unit + 53 integration green on five consecutive runs;
corpus holds at 63/65.
Closes the three gaps left after the parser rebuild.
Refund amendments. ue-05 is a real refund: 'Previous total $49.94 / Refund
-$4.21 / New Total $45.73'. Uber reuses the order UUID across the receipt and
the amendment, so the two can be matched. The transaction is reduced in place
rather than offset with a second row — the order is one event whose cost
changed, and a compensating row would misreport both the meal count and the
merchant's spend. When the original was never ingested, nothing is invented.
[Family] orders now import instead of parking. Their payment line names the
payer, not an instrument ('Payments Siddharth LKR 3,783.20'), so no split is
recoverable and there is no card leg to reconcile against — they would have sat
pending forever, which fails the actual requirement to import and tag them.
Treated as credits, flagged as an assumption. Safe because the family tag
removes them from every budget regardless of instrument, and the LKR amount is
preserved with amount_aud left NULL rather than asserting an FX rate.
Ingest API. n8n now POSTs each message to /api/orders/ingest instead of parsing
in a Code node — the n8n sandbox has no require or fs, so a parser there cannot
be tested against the fixture corpus, which is the one thing that makes this
parser trustworthy. Auth is a shared secret, since machine callers have no
Traefik session header. Rejections return 422 and record nothing.
60 unit + 45 integration green on three consecutive runs; 63/65 corpus holds.
Ingestion now runs on the rebuilt parser. Three substantive changes.
Deferred card reconciliation. A 'MasterCard 8032 and/or credits' receipt never
states the split, but the card leg lands on the statement — Subway's $29.08
order shows $13.06 on 8032, so $16.02 was credits. For a live order that
statement is weeks away, so the split cannot be settled at ingest time. Such
orders are now parked with provenance and no transaction, and
reconcilePendingOrders() resolves them once the statement arrives. Backfill
takes the same path and resolves immediately. Migration 0019 adds the columns
that make an order resumable; applied to personal_test only, prod untouched.
Payment detection bug, found by the new tests: the old regex delimited the
'Paid with' line on a double space, which whitespace collapsing removes. Every
card and mixed receipt fell through to the credits branch — the Woolworths
receipt booked $60.93 of credits spend that never happened.
Category resolution reversed deliberately. Correction 1 said never default to
dining; the implementation of that sent everything unrecognised to 'other', and
knowing six merchants meant Carl's Jr, Taco Bell, Chilli India, Oporto, Schnitz
and Souvlaki GR all landed there. Grocers are an enumerable set and restaurants
are not, so match groceries explicitly and let the residual be dining.
Tests rebuilt on real captured receipts; the synthetic fixtures are deleted.
60 unit + 41 integration green on three consecutive runs.
The previous parser was written against synthetic fixtures shaped to match the
code. It invented a table layout DoorDash does not send, generated
order_reference from Math.random(), read the order date from a 'Date:' string
present in no real message, and detected [Family] by searching the body for the
substring 'family'. Its tests passed because the fixtures were built to satisfy
it. Against 36 real DoorDash and 29 real Uber Eats receipts it does not work.
Rebuilt from the real corpus. 63 of 65 now parse and validate; the 2 rejected
are correctly rejected — one is an order-adjustment notice and one a refund,
neither of which is a receipt.
Corrects an inherited diagnosis: the Mad Mex 'Discounts -$24.09' was recorded
as an HTML-flattening artefact masking a 'true discount of $9.45'. Parsing the
table cells structurally returns the same figures and no $9.45 exists anywhere
in the message — DoorDash genuinely prints a Discounts line that equals subtotal
plus service fee, and the components fail to reconcile on 32 of 36 receipts. So
the breakdown is stored as provenance and never gated on; validation instead
cross-checks the two independently stated totals and the payment line, which is
the number that becomes money.
Real-world cases the corpus forced, none of which were in the spec: [Family]
orders are LKR purchases for family in Sri Lanka (reading them as dollars
inflates ~200x), Swiss orders arrive in CHF, grocery 'Final receipt' mails carry
no Total Charged row, and a declined payment is printed alongside the successful
retry and must be skipped or it records money that never moved.
order_reference now comes from the Uber order UUID embedded in the body, or the
provider message id where DoorDash supplies no order id at all — never random,
so re-ingestion is genuinely idempotent.
Tests 12 & 13 asserted that every 'ShopBack Gift Cards' row becomes a transfer.
Resolved against the ShopBack purchase emails, 3 of the 14 matching rows are
Airbnb, 1 Shell, 1 Amazon — the bank descriptor's trailing token is a sequence
counter, not a brand code, so the description cannot identify what was bought.
Only $313.66 of $3,411.16 was ever reclassifiable.
Withdrawn rather than narrowed: making it safe needs ShopBack purchase-email
ingestion, brand resolution and an approval gate, to correctly handle 2
transactions in 20 months. Those two rows get handled by hand.
The integration suite shares one personal_test database and helpers.resetDB()
TRUNCATEs it with CASCADE, which reaches expense_metadata via the transactions
FK. With file parallelism on, queries.test.ts and participants.test.ts were
truncating rows out from under order-ingestion.test.ts mid-test, so a different
set of assertions failed on every run — including I6 (credits), I7
(idempotency) and I11 ([Family]), the three invariants the suite exists to
prove. Serialise the files.
Suite was reported 31/31 green; observed 29/31 then 28/31 on consecutive runs.
Now 31/31 on three consecutive runs.
One-off analysis, nothing built. Realistic baseline $4,140/mo -> $24,800 for
six months, against $89,770 already accessible ($81,017 loan redraw + $8,753
offset).
Records four corrections the raw data needs before any restatement:
misfiled Raiz/Vanguard/moomoo debits counted as spend, `other` credits read as
negative spend, `government` conflating ATO with rates/rego, and `fees` being
mostly annual.
CLAUDE.md gains two traps found while doing it: partial split coverage inside a
category is usually correct rather than a gap (only shared utilities and
subscriptions are split), and the loan repayment is voluntarily above contracted
($2,500 vs $1,190.54 per fortnight) with the difference recoverable via redraw.
CLAUDE.md gains the traps a new session would otherwise re-discover:
- Rules: a zero-condition rule matches everything (rule 43 would split all ~3,700
transactions); preview-then-apply-by-id is the safe pattern and why it beats
auto-applying on ingestion; how run provenance works.
- Shared expenses: transaction_splits.settled is dead data; getParticipantBalances
is correct and must not be 'fixed'; settlement cannot be attributed per trip.
- The shared loan: separate ledger, fixed 50% with a tracked receivable, why the
share must not be derived from actual payments, and why interest stays as spend.
- Extraction: balance assertions are the check that works, do not derive
opening_balance or add a totals assertion (both would be tautological), Gemini
invents summary fields it was not given, empty statements must not throw, FX is
per-date, and CSV comparisons need millisecond ordering.
The design doc records Phase 0 as done - including that the original Phase 0 plan
was wrong, since reading the code first is what prevented breaking a working
balance page.
Known Gaps lists what is open: the unbuilt phases, 11 failing assertions, the
uncategorised Up rows, and the CSVs sitting in 030490e's history.
The trip view showed Total Owed / Settled / Unsettled per participant, with the
last two derived from transaction_splits.settled. Nothing sets that flag - its
only writer was /api/splits/settle, which no UI calls - so it is false on all 673
splits and every trip reported 100% unsettled, including trips already paid in
full. Molina has paid $20,782.79 against $19,556.07 of splits and the Europe trip
still showed her entire share outstanding.
A correct per-trip figure is not computable either: split_payments records only
from, to, amount and date, so a payment cannot be attributed to a trip. The trip
view now shows each participant's share and points at Shared for what is actually
owed, which is where settlement genuinely lives.
Also removes /api/splits/settle. It was unreachable from the UI but live on its
URL, and a single call with participant_id would mark every one of that person's
splits settled - writing a flag nothing reads. Settlement will be reintroduced
against settlement contexts (docs/shared-expenses-design.md).
getParticipantBalances is deliberately untouched: it computes splits minus
payments, which is coherent. Excluding settled splits there while still
subtracting the payments that settled them would double-count.
The loan is a separate ledger, not a settlement context: a contribution must
never be able to settle a dinner.
The share is fixed at 50%, not derived from actual payments. During Sonu's leave
the obligation did not change, only the payment did - a percentage-of-actual
model would silently redefine her share as 30% and make the shortfall vanish. So
the model needs an expected schedule alongside actual contributions, with the
difference as a tracked receivable. Currently $4,000.00 over Jul 2025 - Jun 2026.
On interest: recorded the mechanics (it is debited to the loan and repaid as part
of the balance - the reconciliation is exact) alongside the counter-argument that
$16,523.64 left and bought nothing, which is what an expense is. Recommends
keeping it as spend with a fixed/discretionary grouping to address the real
concern, but flags it as a judgement call rather than settling it.
Three problems that look separate are one: the app records money moving, and
separately records who owes whom, and the two never meet.
Documents what is broken with evidence - two half-built settlement models,
settlements existing twice unlinked, and Sonu's $37,980 of loan contributions
sitting unrecognised as generic transfers - then proposes settlement contexts,
payments as transactions rather than a side table, and loan co-ownership.
Nothing built. Five open questions, two of which are decisions about the
arrangement rather than the software.
Apply History listed only counts - '13 matches · 13 transactions' - which reads
identically whether the run renamed a merchant or split every transaction with
another participant. Revert is destructive, so that is not enough to decide on.
Two additions. Migration 0017 records rule_id, rule_name and source on each run:
rule_name is denormalised so history stays readable after a rule is edited or
deleted, and there is no FK so deleting a rule cannot cascade away the audit
trail. Both write paths now populate it - the condition-matched run and the
selection-based quick action.
And rows expand to show the run's snapshot set against current values: which
transactions were touched and what changed on each. Rows changed by something
else since the run are called out, because reverting restores the pre-run value
and would discard that later edit.
Runs recorded before this show 'Unknown rule' - the rule they came from is not
recoverable.
Selecting a rule now shows the transactions it would alter, so a subset can be
ticked and applied rather than trusting a bulk run. The apply step takes explicit
transaction ids (the existing bulk apply_rule path), so what you tick is exactly
what changes - a rule whose conditions are too broad cannot reach further than
the preview showed.
Matches are split into 'would change' and 'already correct'. A merchant
normalisation rule matching 400 rows where 380 already hold the right value is 20
changes and 380 rows of noise; only the 20 are listed.
Preview is offered for every rule including manual_only quick actions, which
previously had no way to see their reach at all. A rule with no conditions
matches every transaction - that is how apply already behaves, so the preview
reports it prominently rather than hiding it.
Applies still snapshot to rule_apply_runs, so they remain revertable.
A git add -A in 030490e swept the Wise CSV exports into the repo and they were
pushed. They contain account numbers, an IBAN, payer and payee names and full
transaction detail.
This removes them from HEAD and ignores the directory. It does NOT purge them
from history — 030490e still contains them.
getPendingReconciliations treated every unreconciled manual transaction as
awaiting a matching statement row. Cash never appears on a statement, so a cash
entry sat in the queue indefinitely being offered matches within 3 days and 1%
on amount - and accepting one is silently destructive: reconciled manual rows
are filtered out of every query, so the cash spend disappears while the card
transaction it matched claims to be that same spend.
Migration 0016 adds transactions.payment_method (card | cash | bank_transfer |
other, NULL = unknown) with a CHECK constraint and a partial index. The notCash()
fragment excludes cash from both halves of the reconciliation query - the pending
list and the candidate match subquery, which aliases the manual row as m.
Only cash is excluded. Bank transfers do appear on a statement now that
transaction accounts are imported, and NULL means unknown, so both stay
candidates and every pre-existing row behaves exactly as before.
ATM withdrawals deliberately stay categorised as spend rather than transfers.
Treating them as transfers is only correct if every cash purchase is logged;
with partial logging it silently deletes the unlogged remainder from spend.
siddharthd