Split-shipment orders are charged per shipment; the spine bridge now
writes one row per shipment sharing (source, order_reference). Meal-lane
idempotency semantics unchanged (index scope excludes only
source='order-bridge'); bridge rows get their own unique index on
(source, source_message_id). Applied 2026-08-10.
The CSV importer has no idempotency and structurally cannot have any:
batchInsertCSVTransactions assigns row_index = MAX(row_index) + 1, and
row_index is the fifth column of uq_transaction_identity, so the constraint
is guaranteed a fresh value on every run. The same file imported twice
produces two full sets of rows and nothing objects.
Tolerable for the hand-driven bank CSV this was built for. Not tolerable for
any recurring feed, whose windows overlap by design — and not cleanable
afterwards, since every child of transactions is ON DELETE CASCADE, so a
duplicate must be superseded rather than deleted (0023). ~$42k of re-imported
statement rows already show how that goes.
source_ref carries the provider's own key verbatim, enforced by a partial
unique index rather than an application-side ON CONFLICT that a refactor can
quietly drop.
Found while evaluating Frollo (DECISIONS.md ING-11 in the smarthome repo).
That feed is not being ingested — 88% of it duplicates existing statement
coverage — but this gap is real independently of it.
Adds /api/receipts/ingest as a sibling to the order lane, sharing its shape but
making one decision differently: nothing is parked. An order can wait for its
statement because it is already visible as an email; a gift-card grocery shop is
visible nowhere at all, so a scan that produces no transaction produces nothing
anyone can see. Every payment becomes a manual row immediately and the existing
pending-reconciliation queue resolves the ones with a card leg coming.
One transaction per tender leg. A $114.57 shop settled $40.75 gift card +
$73.82 Mastercard has a statement line for $73.82 only. A single row marked
credits is excluded from the queue while that line double-counts; marked card it
is searched for at 1% of $114.57 and never matches. Either way the shop books
$188.39. Per-leg rows make each amount the settled amount, so the matcher works
untouched.
Reconciliation now carries expense_metadata across. It already moved overrides,
tags and splits from the manual row to the statement row and left metadata
behind, which did not matter while metadata only came from an email that made
its own transaction. It matters now that it carries a shop's line items:
unmoved, the contents vanish at exactly the moment the statement line appears,
and COLES 0556 MANOR LAKES stays as unreadable as before anything was scanned.
transaction_id is UNIQUE, so a statement row that already has metadata keeps it
and the pantry row is flagged rather than raising a constraint violation.
Also regenerates the Prisma model. card_last4, currency, flags, reconciled_at,
matched_transaction_id, platform and route have been in the database since
migrations 0019/0020 and were absent from schema.prisma — regenerating the
client from it would have dropped columns the order lane writes on every ingest.
23 integration tests against the real schema, built from the three receipts that
drove the design. Existing suites unchanged: 104 unit, 144 integration.
I1 refused any credits-funded order dated before 2026-01-09, storing
nothing at all — no transaction and no metadata, so the receipt was
discarded rather than kept as history.
Its reason was splits, not spend: before the cutover shared expenses
lived in SplitMyExpenses, and re-importing them would double-charge
against carryover transaction 2348. That reason expired with 788219b,
where ACTIVE_OBLIGATION became `settled = false AND transaction_date >=
'2026-01-09'`. A pre-cutover split can no longer assert a debt, so a
pre-cutover order cannot move a balance however it is recorded — and
ingestion writes no splits at any date, which now has a test of its own.
What the guard was still doing was hiding ordinary history: 275 orders,
$9,799.96 of meals and rides across 2020-2025, invisible only because the
money came from a gift-card balance instead of a card.
The funding side stays as it is, deliberately. Some of those orders were
paid from ShopBack gift cards that are themselves booked as expenses, so
that portion is counted twice. The exposure is bounded at $3,411.16 over
14 loads and is probably smaller: the descriptors name no brand — "ShopBack
Gift Cards SQ" is a batch code, and the card could be Amazon, Airbnb or
Shell as easily as DoorDash — and six are categorised `gifts`, which may
be real presents rather than self-funding. Reclassifying them on a guess
would corrupt correct data to fix a double-count that cannot be shown.
Only the ShopBack purchase emails can settle it, joined on total paid.
The jump from "OK" to "Never again" is too big and most
disappointments live in the gap (user, 2026-07-28) — so a merely poor
meal either flattered itself as OK or got blacklisted.
Only 'never' raises the warning on a future order. A blacklist that fires
for every mediocre delivery is one nobody reads, so 'bad' records the
disappointment without triggering the alarm. Both set order_again = false
— you would not choose either again — and that split between "would I
order it" and "warn me about it" is the point of the extra level.
Migration widens the CHECK; nothing is removed, so no existing row needs
mapping.
The ledger already knew we had ordered from a place; it did not know the
food was bad. Orders got repeated from places we disliked because nobody
remembered by the time the next one went in. That is what the receipt
ingestion was for (ING-9) and the last piece was missing: order_reviews
existed as a table with no API, no UI and no writes.
Four levels, not three. "Loved" and "liked" are both "would order again"
but only one is worth a detour, and "ok" is not a recommendation.
A verdict belongs to a person, not to an order. A shared meal produces
two opinions and they routinely disagree — that disagreement is the
useful part, and the old UNIQUE on transaction_id alone could not hold
it. Now UNIQUE (transaction_id, participant_id), and the default is the
signed-in user rather than the owner: Sonu authenticates through the same
Traefik OAuth as participant 4, so an owner default would have filed her
verdict under his name.
Per-item opinions key on the item DESCRIPTION, not its index. An index is
meaningless across orders; "the Pad Thai here is good" is the signal that
has to survive into the next order from the same merchant. Only the two
poles are offered — a per-item "ok" answers neither of the questions you
ask at order time.
Sharing is recorded as a real 50/50 split, not a decorative flag. The
split already IS the record that an order was shared, and two records of
one fact drift apart.
An ABSENT item_verdicts means "leave them alone"; an empty array clears
them. Without that distinction a note-only save silently wipes every
per-item opinion — the same shape as the bug that reset `settled` on
split rewrites, and just as invisible on screen. Mutation-tested: making
keepItems a no-op fails exactly one test.
mockDbWithPool gained queryRow. Omitting an export from the mock makes it
undefined at the call site, which fails as "not a function" and reads
like a code bug rather than a test-harness gap.
Statements 107, 142 and 143 bill overlapping periods on one ANZ account, so 31
transactions -- $42,040.68 -- are in the ledger twice.
They are marked superseded, not deleted. Every child of transactions is ON
DELETE CASCADE (splits, tags, overrides, expense_metadata, order_reviews), so
deleting "the duplicate" destroys whatever curation sits on it, and which
member of a pair holds that curation is an accident of import order: here 1
pair carries splits and 6 carry overrides, all on the surviving side, but
nothing guarantees that. Superseding keeps the row, keeps its children, and
makes a mistake one UPDATE to undo rather than a restore from backup.
reconciled_with_id could not be reused. Its predicate is scoped to
statement_id IS NULL on purpose -- a statement line pointing at something else
is the survivor, not the duplicate -- and here both rows are statement lines.
The exclusion goes into EXCLUDE_RECONCILED_SOURCE rather than into a new
fragment, so every query already asking "count each purchase once" gets it
without being edited. The trip cost queries did not use that fragment at all
and now do; verified a no-op on current data (0 trip-tagged rows are either
reconciled sources or duplicates), but they were one import away from
double-counting.
Most of the $42k is transfers and investments, which spend already excludes.
The damage was elsewhere: duplicated rows in the list, and rules re-splitting a
duplicate -- txn 3807 is one of these 31 and was a candidate for splitting
earlier today.
Balances are unchanged: no duplicate carried a split.
A payment has only ever recorded from, to, amount and date. That is why
the per-trip owed figure did not exist — getTripAnalytics said so where
the number should have been: "split_payments carries no trip attribution,
so a payment cannot be assigned to a trip. Settlement is a property of the
whole relationship." Every trip therefore read 100% unsettled, including
trips paid in full.
It is also why the Shared page silently drops payments under a tag filter.
With one global pool there was nothing honest to subtract, so it showed
gross splits under the same label. A tag is a view; a scope is a ledger.
The scope is a trip, not a new settlement_contexts table. trips already
has owner_id, dates and archived, and transaction_overrides.trip_id
already decides membership. A second grouping beside it would be two
unsynchronised scopes over the same rows, with no invariant saying which
governs. NULL means the ongoing household tab, which never closes.
settled answers a different question and the two must not be collapsed:
trip_id is which tab, settled is whether the obligation is still live.
Critically, a live obligation is NOT settled by flipping the flag — it is
settled by recording the payment, and the balance nets to zero on its own.
Doing both would subtract the settlement twice. So settled is written only
by the historical import, for repayments made on a platform we no longer
run, and there is deliberately no "mark settled" action.
Both owed figures now exclude settled splits and the trip figure nets its
own payments. Spend analytics (myShare/mySplitOf) deliberately still count
settled rows: my half of a 2025 grocery shop is my spend whether or not the
other half was ever repaid, and filtering them would re-inflate exactly the
figures importing settled history exists to correct.
Also drops /api/participants/[id]/balance. It had no consumers, no owner
scoping, no debit/credit signs and no EXCLUDE_RECONCILED_SOURCE — a fourth
balance implementation that disagreed with the others and would have
imported three bugs if anything had aligned to it.
getTripAnalytics had no test at all. It has five now, including the one
that matters: a household payment must not make a trip look paid. Verified
by mutation — neutering the settled filter fails three, and dropping the
trip filter on payments fails that one.
Four things the view could not tell you, all from reading the rows (user,
2026-07-27).
**Which platform.** The parser has always known — it has to, to read the
template — and then discarded it. "Order - Burger Corner" gives no way to know
whether to open DoorDash or Uber Eats for the detail, and restaurants exist on
both. Now stored on expense_metadata and named in the description:
"Order - Burger Corner (Uber Eats)". Migration 0021 recovers it for the 101
backfilled rows from the order_reference shape — DoorDash receipts carry no id
of their own so ingestion synthesises `msg:<message-id>`, Uber carries a real
trip UUID, which makes the discriminator exact.
**Bank said "Manual".** That label is derived, not stored, and "Manual" reads
as "hand-entered, still awaiting a card line to match". A gift-card order has
no card line coming, ever. It now reads "Gift Card", and — the part that
actually mattered — credits joins cash in needsCardMatch(), so these stop
sitting in the pending-reconciliation queue. All 81 were queued against a match
that could not exist.
**Uber line items were never parsed.** 67 of 101 orders had none. Uber itemises
groceries but not restaurant orders, so some of that is genuine; the rest was
simply unread. Its markup is better than DoorDash's — every cell carries a
data-testid with the item's uuid, so qty/title/amount bind by id rather than by
column position. Sold-out items (0.00) are kept: they are why a total is lower
than what was ordered.
**Uber prints pick-up and delivery addresses on every receipt** and they were
thrown away. Captured as `route` [{label, time, address}], de-duplicated
because the template renders the whole block twice for narrow screens. Wording
is kept as printed ("Pick-up" on some receipts, "Pickup" on others) rather than
normalised, so a template change stays visible. This is the same block a *trip*
receipt uses for start and destination — rides are not ingested today, but the
reader will not need changing when they are.
Also stores source_email_subject/from, which order ingestion had left null on
columns that already existed.
Verified against the captured corpus: route on all 6 Uber fixtures, 5/5 items
on the GLOMARK grocery receipt including the sold-out one. Production data
updated by smarthome:docker/scripts/order-presentation-2026-07-27.sql
(81 descriptions, `backfill` tag, re-run clean). `route` and Uber line items
are parsed from here on only — recovering them for already-ingested orders
means re-reading the mail, which I7 idempotency refuses by design.
None of these were caught by 105 green tests, because the code they live in was
barely tested and the HTTP path was not tested at all.
1. reconcilePendingOrders hardcoded category 'dining', so any order resolved
through the deferred path booked as dining regardless of merchant — a
Woolworths grocery order that parks and later reconciles was misfiled.
That reintroduced, through the back door, exactly the misfiling
resolveCategory() exists to prevent. Now calls it.
2. reconcileCardLeg never marked a statement line as consumed, so two orders on
the same card inside the +/-4 day window both bound to the same charge and
each booked its own credits remainder — double-counting spend. At 10-15
orders a month on one card that is not a corner case. Migration 0020 adds
matched_transaction_id with a unique index; the matcher now excludes lines
already claimed.
3. The ingest API returned HTTP 200 for every parse failure, and the Slack
alert fires only on non-200. So the single most likely production failure —
a provider template change breaking every order at once — was completely
silent. Split into NotAReceiptError (promotions, delivery updates, refund
and adjustment notices: 200, silent, expected traffic) and OrderParseError
(it IS a receipt and would not parse: 422, alerts).
Also: order_reference now anchors on Uber's own tripReference cell rather than
'first UUID in the document'. I had claimed to verify that the first UUID was
always the order UUID; that check compared against zero samples and was
vacuous. tripReference is present in all 29 captured receipts and, for ue-00,
equals the UUID the PDF redirect resolves to. The positional fallback remains
but only flags when there is genuine ambiguity.
Adds the API route's first tests — auth gate and error taxonomy — plus
anchoring regressions. 63 unit + 53 integration green on five consecutive runs;
corpus holds at 63/65.
Ingestion now runs on the rebuilt parser. Three substantive changes.
Deferred card reconciliation. A 'MasterCard 8032 and/or credits' receipt never
states the split, but the card leg lands on the statement — Subway's $29.08
order shows $13.06 on 8032, so $16.02 was credits. For a live order that
statement is weeks away, so the split cannot be settled at ingest time. Such
orders are now parked with provenance and no transaction, and
reconcilePendingOrders() resolves them once the statement arrives. Backfill
takes the same path and resolves immediately. Migration 0019 adds the columns
that make an order resumable; applied to personal_test only, prod untouched.
Payment detection bug, found by the new tests: the old regex delimited the
'Paid with' line on a double space, which whitespace collapsing removes. Every
card and mixed receipt fell through to the credits branch — the Woolworths
receipt booked $60.93 of credits spend that never happened.
Category resolution reversed deliberately. Correction 1 said never default to
dining; the implementation of that sent everything unrecognised to 'other', and
knowing six merchants meant Carl's Jr, Taco Bell, Chilli India, Oporto, Schnitz
and Souvlaki GR all landed there. Grocers are an enumerable set and restaurants
are not, so match groceries explicitly and let the residual be dining.
Tests rebuilt on real captured receipts; the synthetic fixtures are deleted.
60 unit + 41 integration green on three consecutive runs.
Apply History listed only counts - '13 matches · 13 transactions' - which reads
identically whether the run renamed a merchant or split every transaction with
another participant. Revert is destructive, so that is not enough to decide on.
Two additions. Migration 0017 records rule_id, rule_name and source on each run:
rule_name is denormalised so history stays readable after a rule is edited or
deleted, and there is no FK so deleting a rule cannot cascade away the audit
trail. Both write paths now populate it - the condition-matched run and the
selection-based quick action.
And rows expand to show the run's snapshot set against current values: which
transactions were touched and what changed on each. Rows changed by something
else since the run are called out, because reverting restores the pre-run value
and would discard that later edit.
Runs recorded before this show 'Unknown rule' - the rule they came from is not
recoverable.
getPendingReconciliations treated every unreconciled manual transaction as
awaiting a matching statement row. Cash never appears on a statement, so a cash
entry sat in the queue indefinitely being offered matches within 3 days and 1%
on amount - and accepting one is silently destructive: reconciled manual rows
are filtered out of every query, so the cash spend disappears while the card
transaction it matched claims to be that same spend.
Migration 0016 adds transactions.payment_method (card | cash | bank_transfer |
other, NULL = unknown) with a CHECK constraint and a partial index. The notCash()
fragment excludes cash from both halves of the reconciliation query - the pending
list and the candidate match subquery, which aliases the manual row as m.
Only cash is excluded. Bank transfers do appear on a statement now that
transaction accounts are imported, and NULL means unknown, so both stay
candidates and every pre-existing row behaves exactly as before.
ATM withdrawals deliberately stay categorised as spend rather than transfers.
Treating them as transfers is only correct if every cash purchase is logged;
with partial logging it silently deletes the unlogged remainder from spend.
Four related fixes to spend correctness.
Balance assertions. Nothing checked that a statement's transactions add up to
its closing balance. getStatements now computes opening + movement - closing
and the statements page flags any statement that does not reconcile. Sign
depends on what the balance means: on a credit card or loan it is what you owe,
so spending increases it; on a transaction or offset account it is what you
hold. 11 statements currently fail, $4,177 unexplained - including two adjacent
ANZ statements off by exactly +/-$230.38, a transaction filed against the wrong
one.
Category normalisation (migration 0015). Categories arrive from Gemini (which
writes straight to Postgres from N8N), CSV import and manual edits, so the rule
belongs in the database - same reasoning as normalize_statement_type in 0013.
Adds normalize_category(), triggers on transactions and transaction_overrides,
and CHECK constraints. Backfilled 122 rows: 19 'payment' ($42,569) to transfers,
14 'refund' recovered to the merchant's usual category, and title-case duplicates
folded into their canonical spelling - 'Shopping' and 'shopping' had been
counted as separate categories by every GROUP BY.
Refund netting. Monthly analytics counted only debits, so a refund was counted
nowhere: excluded from spend by type, and not income by category. A $2,888.92
Expedia purchase refunded in full eight days later still read as $2,888.92 of
spend. SPEND_SIGNED and NET_SPEND_ROWS bring refunds in as negatives; 'income'
joins the excluded categories so incoming money cannot leak in as negative
spend. $36,773 across 93 rows now nets correctly.
Drill-down share. The insights drill-down used my_share_percent ?? 100, which
ignored transaction_splits entirely and repeated the bug fixed in ab00f8c one
layer up. getTransactions now returns my_share_pct and my_amount resolved
server-side, and the table shows gross alongside your share.
A loan repayment is not an expense. A $3,000 mortgage repayment is roughly
$1,200 of principal (equity — a balance-sheet move) and $1,800 of interest (the
only part that is genuinely spend).
Migration 0014:
- transactions.principal_amount / interest_amount, populated only when the lender
itemises the split on the repayment row
- statements.interest_rate, scheduled_repayment, repayment_frequency,
redraw_available, loan_term_months
- normalize_repayment_frequency() + trigger, so "Fortnightly", "Bi-Weekly" and
"Every 2 weeks" all land on 'fortnightly'
Two statement shapes are handled. Where the loan statement lists repayments and
"Interest Charged" as separate rows (the common Australian case), transaction_type
already does the work. Where a lender itemises the split on the repayment row,
that row is typed 'payment' and would be skipped entirely — losing the interest.
New SPEND_ROWS / SPEND_BASE fragments in analytics-sql.ts count such rows at
interest_amount instead of amount.
Adds the loan_interest category (+ colour, and the missing fees colour).
Verified against the live DB with a synthetic ANZ home loan statement: a $3,000
itemised repayment plus a $10 service fee moved April spend by exactly $1,810,
with the $1,200 principal excluded and still retained on the row. Test data
removed and the figure confirmed back at its original value.
Groundwork for importing bank and loan statements alongside credit cards.
statement_type was whatever free text Gemini put in account_type ('Credit Card',
'credit card', 'credit_card', 'Business Card', 'ACCESS ADVANTAGE',
'multi-currency account'). The UI coped only by doing .includes("card"), which
breaks as soon as bank and loan statements arrive.
- Migration 0013 adds normalize_statement_type() + a BEFORE INSERT/UPDATE
trigger and a CHECK constraint over credit_card|transaction|savings|loan|
offset|investment|other. The trigger means the N8N workflow keeps working
unchanged while it still sends free text. Raw value stays in account_type.
Backfilled 99 existing rows.
- src/lib/statement-types.ts mirrors the vocabulary for the UI; statements page
now filters by the real types and headlines balance vs amount due per type.
Analytics were scoped with INNER JOIN statements + s.owner_id, which silently
dropped all 180 manual/CSV transactions (statement_id IS NULL) from every
report. Switched all six routes to LEFT JOIN + COALESCE(t.owner_id, s.owner_id)
via shared fragments in src/lib/analytics-sql.ts, so the transfers/investment
exclusion that stops card-payment double counting stays consistent. Also
extended that exclusion to trip analytics, which had none.
Drive-by: /api/analytics/subscriptions was returning 500 on an unserialisable
BigInt from COUNT(*) + 1.
Verified against the live DB: monthly spend picks up the previously invisible
manual transactions (Apr 9,849.91 -> 14,286.70) and all four analytics
endpoints return 200.
A rule flagged manual_only never runs in the apply-all pass; instead it shows
as a button in the transactions bulk bar and applies its actions to the current
selection (conditions ignored — the selection is the condition). Recorded as a
rule_apply_run, so it reverts from Rules -> Apply History like any other run.
Motivation: tagging Home + splitting 50/50 with Sonu was two bulk actions every
time. Now it is one click, and any other combo can be defined the same way.
Extracts the action-application and snapshot logic from the apply route into
src/lib/rule-actions.ts so both callers share one implementation — splits upsert
rather than delete+reinsert, so settled flags survive.
- prisma/migrations/0011_trips: trips table, trip_id on overrides, partial
index — idempotent; matches DDL already applied to prod. Applied to
personal_test (integration suite was failing on missing trips relation).
- README/CLAUDE.md: deployment is now push-to-deploy via Komodo
(deploy-finance Procedure, Gitea webhook); compose command is fallback.
- README: migrations table completed through 0011; note that the container
must only be reachable via Traefik (header-trust auth).
- Add reconciled_with_id column to transactions (links manual → statement tx)
- CSV import wizard: 4-step modal (upload → map columns → review → done)
- Handles any bank format via column mapping with localStorage presets
- Single signed or separate debit/credit column modes
- Editable preview table before committing
- Auto-tags all imported rows with 'csv-import'
- Batch reconcile page: shows all unreconciled manual transactions with
potential statement matches (date ±3 days, amount ±1%) pre-fetched
- Select matches across multiple rows, apply all at once
- Copies overrides/tags/splits from manual → statement tx atomically
- Manual tx marked reconciled (linked), hidden from main transactions view
- Transactions with no matches shown separately
- Import CSV button on transactions page
- Reconcile nav item in sidebar
- New split_payments table records actual payments between participants
- Balance = total split obligations - total payments (splits never marked settled)
- Record Payment modal per participant: direction toggle, amount pre-filled with balance, date, notes
- Payment history inline on each balance card with +/- display and delete
- Per-transaction Settle button removed; Action column removed from shared table
- Splits always show the true cost breakdown regardless of payment state
- Add POST /api/transactions to create manual transactions (statement_id=NULL, owner_id set directly)
- Queries switch from JOIN to LEFT JOIN statements so manual transactions are visible
- COALESCE(t.owner_id, s.owner_id) throughout for owner resolution
- Add "Manual" bank filter option in getTransactions
- Search extended to include merchant_normalized override
- Split data fetched via lateral subquery on every transaction row
- getParticipantBalances rewritten as UNION for bidirectional net balances
(credits/refunds negate, split from either side of the relationship)
- getSharedTransactions: remove my_share_percent from SELECT (fixes GROUP BY error),
WHERE rewritten as two distinct cases (owner with others split vs participant on others' txn)
- getTransactions: OR EXISTS condition so split participants see shared transactions
- add-transaction-modal component for creating manual transactions with splits
- 0008_my_share_percent migration adds my_share_percent to transaction_overrides
Add rules engine with CRUD API, condition/action evaluation, and apply-all endpoint.
UI: rule builder form with field/operator/value conditions, tag multi-select, apply button with result stats.
- tags table: name, color; transaction_tags junction table
- GET/POST /api/tags; DELETE /api/tags/:id
- POST/DELETE /api/transactions/:id/tags for per-transaction tagging
- Bulk tag/untag via /api/transactions/bulk (action: tag/untag)
- Tags returned inline with transaction list via LATERAL join
- Tag filter on Transactions page
- Bulk "Tag as..." in bulk action bar
- Tag pills + "+" picker on each transaction row
- /tags page: create with color picker, list with counts, delete
siddharthd