2c666236b2fdf857859acc95115f351444d9d613
93
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 siddharthd
|
2c666236b2 |
orders: share the merchant tidy, and fix two bugs the corpus run found (board 212)
ci / lint-test (push) Successful in 41s
The tidy existed but lived inside the list page, so the DETAIL page never had it: following a row through showed "Apple" becoming "Apple Pty Ltd." Moved to src/lib/merchant-label.ts and applied on both surfaces. Running it over all 1,451 distinct merchant names — rather than over examples — turned up two real defects: Coburger & Co -> "Coburger &" eBay Commerce Australia Pty Ltd. -> "EBay Commerce Australia" The first strips a suffix that is part of the brand and leaves a dangling connector. The second is the capitalisation rule firing on "starts with a lower-case letter" when what it means is "is a bare domain" — mangling a brand that is deliberately lower-cased. Both are now guarded, and re-running over the full corpus reports 170 names tidied and 0 defects. PRESENTATION ONLY. It merges nothing and must never decide two rows are the same merchant. Amazon.in stays Amazon.in — a different marketplace with different currency and geography, not a name variant — and there is a test asserting it can never equal amazon.com.au. Kogan.com, GOG.com, AliExpress.com and Catch.com.au are real brand names containing a TLD and are left alone. Real unification is the merchant-alias bridge, ticket 176, which has already merged what can be merged safely. The five names that read alike after tidying (ebay/eBay/eBay Inc., menulog/Menulog Pty Ltd, deliveroo/Deliveroo, grab/Grab, paypal/PayPal) are genuinely one merchant each, so reading alike is correct. Lower-case platform SLUGS are deliberately not capitalised: "ebay" -> "Ebay" is wrong, and getting it right is the registry's job, not the stylesheet's. |
||
|
siddharthd
|
215303161b |
orders: bring the lifecycle-only toggle's numbers up to date (board 210)
ci / lint-test (push) Successful in 42s
The count moved three times on 2026-08-12 and the history is the point:
755 when the predicate was written
904 after board 219 turned unstated zeros into NULLs — those 149 rows had
been failing the "no amount" test on a technicality
699 after 210a retired 205 Amazon "share your experience" mails, which were
never lifecycle events at all
620 after 210b joined 79 phantoms to their real parent, placebo-verified at
a 1.9% error rate
What is left is the corpus coverage ceiling rather than a defect — a mail whose
parent is not in the corpus has nothing to join to — and the tooltip now says
that instead of describing the whole class as an open ingestion defect.
|
||
|
siddharthd
|
3b5e495ca4 |
orders phase 2: order_transaction_links, and move every reader to it (board 205)
ci / lint-test (push) Successful in 41s
Phase 1 read linkage out of expense_metadata, which is shaped as ONE ROW PER
TRANSACTION — transaction_id UNIQUE, matched_transaction_id partial-unique — so
every multiplicity it expresses is smuggled through a string key (0029 keys
split shipments <entity_key>#f<fact_id>). A BNPL plan needs four rows against
one order and has no such trick available.
Migration 0030 adds order_transaction_links (many-to-many, keyed on entity_key
rather than entities.id: finance-app does not model the spine and must not hold
an FK across a boundary a re-extraction can decompile) and backfills all 63
existing bridge rows.
THE UNIQUENESS RULE WAS WRONG FIRST TIME, in the most on-brand way available.
It read (entity_key, leg_kind, COALESCE(leg_index, 0)), so two shipment legs of
the same order — both leg_index NULL — collapsed to one key and ON CONFLICT DO
NOTHING dropped one SILENTLY. Caught only because the backfill reported 62
against 63 candidates. The row it ate was order_amazon_249-4859367-0690246's
$130.00 second shipment, the same order named in migration 0029's comment as
the reason split shipments need distinct keys at all. What identifies a leg
depends on its kind: an instalment by its INDEX, a shipment by its FACT, a
whole-order charge by neither.
ALL FOUR READERS MOVE TOGETHER, links first with expense_metadata as fallback:
- LINK_LATERAL (list txn_count/first_txn_id, and the has_transaction facets
that read it)
- the detail page's transactions query
- /api/transactions/[id]/order
- the order_ctx lateral in queries.ts
Moving fewer is not a smaller change, it is an inconsistent one: the matcher's
four links for order_ebay_14-11714-95953 come with no expense_metadata row, so
a half-move would show four instalments on the detail page while the list said
txn_count = 0 and put the order on the wrong side of BOTH has_transaction
filters. Verified after: detail 4 legs, list txn_count 4, has_transaction=yes
includes it, =no excludes it.
UNION not UNION ALL on transaction_id — after the backfill the same charge is
legitimately in both stores and counting it twice would show "2 charges" on a
single-payment order.
order_platform is deliberately NOT coalesced with the link's platform. It gates
the receipt disclosure arrow on /transactions, and a BNPL leg has no receipt
behind it — filling it in put an arrow on four Afterpay rows that expand to
nothing, which is the exact promise the arrow exists to avoid making. Separate
leg fields carry the sub-line instead ("1 of 4 - DJI Air 3 Fly More Combo").
order-details.tsx now also requires a real receipt (platform present) before
rendering, because the endpoint can answer with a link alone.
Two invariant views, both empty and expected to stay so: order_link_orphans
(spine re-keying silently orphans a TEXT key — bridge links are rebuildable,
`manual` ones are lost curation) and order_link_drift (the two stores
disagreeing). The plan's suggested fix for drift — widening the bridge's NOT
EXISTS guard to "neither store has it" — was NOT taken: the expense_metadata
INSERT has no ON CONFLICT, so re-attempting an order that already has a receipt
row would duplicate it. Detecting is cheap; a non-idempotent re-write is not.
Unchanged: order_feed 6,265, order_spend AUD 4,895 / $442,651.80. Links 75.
|
||
|
siddharthd
|
6b9b5fe518 |
orders: badge the annual summary receipts (board 211)
ci / lint-test (push) Successful in 40s
Migration 033 adds entity_orders.summarises_period; order_spend drops those rows and order_feed keeps them. Unbadged, a $2,376 annual tax receipt restating twelve monthly donations renders as the largest single purchase of the year, so both surfaces say what it is: - /orders — a "year summary" chip beside the auth badges - /orders/<key> — a banner in the same shape as the settled-rail one, saying the monthly debits are in the ledger under their own dates The amount is deliberately still shown. It is the year's giving total and the reason the row is worth keeping; it just belongs to no total on the page. The detail page already falls back to o.order_total when order_spend has no row, which is exactly the case now — verified: gross_total null, order_total 2125.5. |
||
|
siddharthd
|
231ef3411a |
orders: record why line items may not exempt a row from the phantom filter
ci / lint-test (push) Successful in 40s
Board 219 nulled 263 unstated zero totals in the spine, which moved 149 rows into the lifecycle-only hide predicate — they had been failing its "no amount" test on a technicality, an unstated 0 they should never have carried, while matching the phantom signature on every other count. Tried exempting rows that carry line items, on the theory that naming what was bought proves a purchase. It surfaced 442 rows and the first window checked showed why it is wrong: the Amazon "Ordered:" mail that mints a msg- phantom carries the item list too, so Xelsluthe and YIWENTEC each rendered TWICE again — the exact complaint this page was fixed for today. Reverted, and the reason recorded in place so it is not re-attempted. Items do not separate an orphan purchase from an item-carrying echo. Only deduplication does, and that is board 210. No behaviour change from the previous commit; comments and counts only. |
||
|
siddharthd
|
699e4a2ddd |
orders: cast cadence order_count to int — BigInt broke every recurring merchant
ci / lint-test (push) Successful in 43s
order_merchant_cadence.order_count is a count(*), so BIGINT, and JSON.stringify throws on it: 'Do not know how to serialize a BigInt', a 500 carrying no SQL. It fires only where a cadence EXISTS, so the failure hid behind the merchant_entity_id fault and reappeared the moment that was fixed — St. Ali (40 orders, 14-day cadence) still 500'd while Amazon opened fine. |
||
|
siddharthd
|
79940202f1 |
orders: fix the 42703 that broke EVERY order detail page
getOrderDetail joined order_merchant_cadence on merchant_entity_id. Migration 021 rekeyed that view to merchant_key (merchant_entity_id is NULL on ~9% of the feed, and duplicate merchant entities split one shop's history); 027 and 029 moved order_feed and order_spend across, and this query was missed. Every /orders/<key> request has since failed with column rec.merchant_entity_id does not exist The reason nobody saw a 500 is the second half of this commit: the page collapsed every failure into 'That order could not be found.' A server fault wearing the costume of a data condition reads as an empty spine and gets investigated in the wrong repo. The hook now carries the status and only a genuine 404 says the order is missing. |
||
|
siddharthd
|
1b34cd65d8 |
orders: restore the design the prototype had, and stop repeating the row
ci / lint-test (push) Successful in 49s
Rebuilding the prototype in React quietly dropped most of what made it read well. Side by side, the live page had lost the year strip entirely — which was both the hero and the context for the date range — along with the masthead totals, the A$ prefix on amounts, and the Fraunces wordmark. The trust marker rendered as a tofu box. Restored, with the strip clickable so a bar scopes to that year. Also three content fixes, all visible the moment anyone looked at a real page. The title and the manifest were printing the same text twice on 1,964 of 4,649 itemised rows — 42% — because on a single-item order the order title IS the line item. The title now earns its line only when it says something the items do not. Food line items carry the full customisation: one Subway order runs to 450 characters naming every topping, and it swamped everything around it. Rows show a short form, cut at the first bracket where there is one, since "Footlong (Italian Herb…" is the product and the bracket is the customisation. Rows now expand, following the disclosure pattern the transactions page already uses, and the query fetches twelve items rather than three so opening one costs no round trip. Merchant names get a presentational tidy — corporate suffixes dropped, a lower-cased domain capitalised. This never merges two merchants: the estate holds amazon.com.au, Amazon.in and Amazon Services Australia as three distinct entities, and Amazon.in must stay separate because it is a different marketplace rather than a name variant. Actually unifying them is the merchant-alias bridge. |
||
|
siddharthd
|
c05c4b5a35 |
orders: show the context, not just the row
ci / lint-test (push) Successful in 44s
The first cut rendered entity_orders and nothing else, and it read like a table dump: a courier tracking notice and a subscription payment both presented as retail purchases, merchants shown as sender-domain slugs, and no indication that the same coffee roaster had been billing fortnightly for two years. Rows now carry what the ingestion layer already knew. A kind badge from content_class distinguishes a delivery notice, an invoice, a booking and a subscription from an actual purchase — NULL stays unbadged rather than being labelled a purchase, because 48% of rows predate the interpretation index and "unknown" is not "order". display_name shows the resolved merchant (98% of rows) instead of the slug. A recurrence badge marks merchants billing on a cadence, which is derived from the gaps between orders rather than stated anywhere in the mail. And the order's own title now appears when it differs from the merchant, so the manifest is not the only detail on the row. Adds the services lane for bookings, invoices and subscriptions — things with no goods and no delivery, where the retail columns are dead space. |
||
|
siddharthd
|
b26d526e83 |
orders: a browse surface for the purchase history the ledger cannot show
ci / lint-test (push) Successful in 59s
The spine holds ~6,300 purchase orders back to 2006, ~4,600 of them itemised, and 61 reach a transaction. Everything else has been visible only through SQL. This adds /orders and /orders/[entityKey] over it. The point of the page is the manifest. /transactions can only ever say "AMAZON AU MARKETPLACE SYDNEY"; a row here says what was in the box, which is the one thing the ledger structurally cannot carry. Five lanes, because the shapes genuinely differ — retail ends refunded or returned 14.1% of the time against food's 5.1%, food has no meaningful ETA where grocery has one on 54.8% of orders, digital never ships at all. The lane comes from order_lane() in migration 018 rather than a column, because slug 'uber' carries 394 taxi rides and 485 Eats orders. Defaults to this year: 449 orders rather than 6,283. Twenty-one years is the archive, not the working set. Three things the data forced. Unknown amounts render "not stated", never $0.00, because 1,648 orders have no amount and a zero would be false. A full reversal strikes the figure through; a partial refund does not, since striking $191.40 when $13.33 came back is a lie — the charge stays primary and the credit sits under it with the net. And rows with no amount, no reference and one lifecycle event are hidden by default, which lifts amount coverage from 74% to 84%; the toggle says on its face that it is a workaround for board 210 rather than a fix. Reads are raw SQL in lib/order-feed.ts rather than queries.ts: the spine is written by the ingestion-engine, is not in prisma/schema.prisma and never will be, and mixing it into a file where everything is Prisma-modelled would destroy that invariant. /orders is gated by an explicit viewer allowlist. Not because the three people listed need protecting from each other — everything here is on one person's cards — but because a participant is an accounting entity and any participant row with an email is a login. Adding someone to split a holiday must not silently hand them the purchase history. Verified live: gate returns 403 for non-participants and for a missing identity header; search "drone" finds the DJI order through its line items; the detail page renders its three lifecycle events and its Afterpay settlement sibling; and a bridged Amazon order shows both split-shipment charges. |
||
|
siddharthd
|
0e5422919e |
shared page: expandable receipt details on split transactions
ci / lint-test (push) Successful in 57s
The item list existed only on the owner's transactions page — the person a meal or order was split WITH could see the amount but never what was in it. getSharedTransactions now carries the same order_platform LATERAL the transactions page uses, and rows with a receipt get the same disclosure arrow + OrderDetails expansion. The shared viewer is a split participant, so canAccessTransactions already authorises the order API for them; meal rows keep the verdict UI (the partner records their own), bridge rows stay review-free. |
||
|
siddharthd
|
7f8e363b5b |
order details: hide the review surface for spine-bridged retail rows
ci / lint-test (push) Successful in 58s
Verdicts are the meal lane's feedback loop (loved/never steers the next restaurant order and the nudge card warns on re-order). Bridge rows (source='order-bridge': Amazon, eBay, Officeworks...) have no such loop, so ItemVerdictToggle and OrderVerdict stay hidden there. The order API now returns 'source'; OrderReceipt.platform widened to string — the bridge writes real platform slugs beyond the meal trio. |
||
|
siddharthd
|
9e4b518f57 |
Split a trip into its two economies, so travel stops being a 60% slab
ci / lint-test (push) Successful in 48s
travel dominated every trip page and said nothing. The tempting fix is a finer travel taxonomy, which needs a hand-maintained merchant list — the trap #19 already describes — and it is also the wrong diagnosis. travel is the only category that spans both phases of a trip. Every other one is 100% on-the-ground: on Europe 2026, dining, transport, entertainment, groceries and shopping are all exactly $0.00 before departure. The chart was not bad, it was two economies stacked into one, and travel was the only thing visible in the union. So split on start_date and use the axis that carries information in each phase. Booked ahead ($22,050.51, 57%) is all flights and stays, so merchant is the axis — Agoda $4,490, Air India $3,454, Luxury Escapes $3,284. On the ground ($16,946.94) travel falls to $8,241 among dining $4,452 and transport $2,938, and category is finally worth charting. The hero is the ratio, not a lone total, with the on-ground daily rate beside it — the only figure comparable between trips, since totals are not: Europe $677.88/day against Auckland $83.39. A trip with near-zero committed spend says so, because Sonu + Sunny's $184.84 is a filing artefact (both legs' bookings sit on the first trip), not a cheap trip. Two dataviz rules this page was breaking. Category bars now use one copper hue with the name as a direct label: the per-bar rainbow double-encoded identity the label already carries, and the trip subset fails CVD validation on this surface (other vs shopping at delta-E 5.0 protan, below the floor of 6). And the hero figure drops the serif and tabular-nums, which read as decoration at that size. The phase bar is two ordinal steps of one hue, validated with --ordinal against the card surface, with a 2px gap so the boundary is an edge. 278 passing, build clean. Data verified against the database directly; I could not render the page in a browser to eyeball the layout. |
||
|
siddharthd
|
cb7665ded1 |
Let everyone on a trip see it, and give payments their scope back
ci / lint-test (push) Successful in 52s
Trips were scoped to trips.owner_id, so Sonu saw no trips at all — despite having paid for 104 of the tagged rows herself. Her own spending was invisible on the only page organised around it. A participant is now anyone with a split on, who paid for, or whose payment is scoped to, a transaction tagged to the trip. Derived, not stored. A trip_participants table was designed and rejected: the expenses already carry the fact, and two records of one fact drift apart. Deriving it also excludes Singapore + Bangkok 2026 from Sonu for free, which a table would have to be kept in sync to do. Siddharth 4 trips, Sonu 3, Molina 1. Everything about a trip is shared except delete. Both trip foreign keys are ON DELETE SET NULL, so deleting Europe 2026 untags 210 transactions and NULLs the trip scope on 6 payments — where the hand-derived Europe-first allocation lives, which nothing recomputes. That stays with the owner. Trip owed now returns both directions and nets neither. An obligation lives on a row someone else paid for, so a viewer-as-payer figure can never hold it, and Sonu's Europe read "you are owed $2,408.24" while omitting the $8,004.04 she owed. Collapsing the two into a signed net is the tempting next step and would have corrupted the scope allocation: the grouped-payment allocation cleared each trip against the one-directional gross, so redefining the debt afterwards turns $8,004.04 already allocated into an $802.75 over-allocation with household understated by the same amount. Verified byte-identical — Auckland $1,505.64, Europe Molina -$816.16, Europe Sonu $0.00, Sonu + Sunny $0.00. getTransactions gained trip_all_rows so a participant sees the whole trip. It is opt-in and not implied by trip_id, because the same endpoint backs the main transactions list and its trip filter must keep owner scoping. Participation is re-checked in SQL, so passing the flag for someone else's trip returns nothing. Payments can finally say what they settle. trip_id has existed since migration 0022 but POST never read it and GET never returned it, so every payment made in the app landed on household and the 9 trip-scoped rows were hand-written SQL. "Both" needs no new shape — one row per scope sharing a linked_transaction_id. Three write paths had no authorisation at all and were reachable by any participant: assignTransactionsToTrip checked nothing, DELETE on a payment deleted by bare id, and POST accepted any from/to pair. All three now check. Also fixes the test suite, which was pointing at postgres-pantry: container IPs move on recreation and 172.22.0.47 stopped being postgres-personal. It only failed safe because the credentials did not match — resetDB now refuses to truncate anything not named personal_test. 22 new tests, 276 passing, build clean. |
||
|
siddharthd
|
c70d2b1fac |
Sign the investments line so withdrawals net against contributions
ci / lint-test (push) Successful in 50s
A withdrawal from a fund is a disinvestment, not income: units convert back to cash and net worth is unchanged. Summed unsigned it read as more money invested. March 2026 showed $38,615.34 of investing in a month that was net -$11,384.66, because a $25,000 Raiz withdrawal was added to an $8,563.80 IBKR deposit instead of cancelling it. Each credit costs twice — once for being added, once for not being subtracted — so the error is double the credit: $50,000 in March, $3,000 in May 2025, $53,000 across the window. Since net = income - spent - investments, March's net of -$55,918.73 should read -$5,918.73. Filing withdrawals as income is the other tempting answer and is worse: it books an asset disposal as earnings and feeds the same figure into net with a flattering sign. Same reason the Up item sales in Known Gaps do not belong on the income line. What this cannot resolve: part of a withdrawal genuinely is income — the capital gain. The bank descriptor is one gross figure with no cost base, so it cannot be decomposed from statement data. Netting tracks cash committed against cash returned and leaves the gain for holdings data to surface; it does not assert the gain is zero. The budget page gates the Invested card on `!== 0` rather than `> 0` — a net-disinvesting month is real data, not an empty one — and renders negative months in amber so the sign is not hidden by matching digits. |
||
|
siddharthd
|
f6c500b27a |
Hide transfers in the transactions view by default
ci / lint-test (push) Successful in 51s
Transfers move money between your own accounts; at 433 of 3,996 rows (~11%) they crowd out the rows that represent actual spending. getTransactions gains `exclude_categories`, opt-in per caller and deliberately not defaulted in queries.ts: the rules preview and the bulk rule-apply path both read candidate rows through getTransactions, and a default exclusion there would silently shrink what a rule can see and reach — invisibly, since a rule that matches nothing looks the same as a rule with nothing to do. Two behaviours the filter needs, both tested: - An explicit category pick beats the exclusion. Selecting "Transfers" while the default is on subtracts it from the hidden list instead of returning zero rows and reading as "you have no transfers". - COALESCE the effective category to '' before `<> ALL`. NULL <> ALL(...) is NULL, not true, so an uncategorised row would disappear from a filter that never named its category — the trap EXCLUDE_NON_SPEND already documents. The default is off when the view is scoped to a statement: that is a reconciliation view, the row count has to match the statement, and a credit-card payment is the row you went there to check. |
||
|
siddharthd
|
a465b147a3 |
Read receipt lines under the keys the receipt panel renders
ci / lint-test (push) Successful in 48s
The order-details panel reads qty/description/amount from expense_metadata.line_items. A grocery shop is a receipt like any other, so it stores the same keys rather than name/quantity/line_total — otherwise the rows arrive complete and display blank, which is exactly the failure ING-8 names. unit and category are the two fields a grocery line has and a delivery line does not; nothing renders them yet and the category composition will. |
||
|
siddharthd
|
17028c79ff |
Accept grocery receipts scanned in pantry as candidate spend
ci / lint-test (push) Successful in 48s
Adds /api/receipts/ingest as a sibling to the order lane, sharing its shape but making one decision differently: nothing is parked. An order can wait for its statement because it is already visible as an email; a gift-card grocery shop is visible nowhere at all, so a scan that produces no transaction produces nothing anyone can see. Every payment becomes a manual row immediately and the existing pending-reconciliation queue resolves the ones with a card leg coming. One transaction per tender leg. A $114.57 shop settled $40.75 gift card + $73.82 Mastercard has a statement line for $73.82 only. A single row marked credits is excluded from the queue while that line double-counts; marked card it is searched for at 1% of $114.57 and never matches. Either way the shop books $188.39. Per-leg rows make each amount the settled amount, so the matcher works untouched. Reconciliation now carries expense_metadata across. It already moved overrides, tags and splits from the manual row to the statement row and left metadata behind, which did not matter while metadata only came from an email that made its own transaction. It matters now that it carries a shop's line items: unmoved, the contents vanish at exactly the moment the statement line appears, and COLES 0556 MANOR LAKES stays as unreadable as before anything was scanned. transaction_id is UNIQUE, so a statement row that already has metadata keeps it and the pantry row is flagged rather than raising a constraint violation. Also regenerates the Prisma model. card_last4, currency, flags, reconciled_at, matched_transaction_id, platform and route have been in the database since migrations 0019/0020 and were absent from schema.prisma — regenerating the client from it would have dropped columns the order lane writes on every ingest. 23 integration tests against the real schema, built from the three receipts that drove the design. Existing suites unchanged: 104 unit, 144 integration. |
||
|
siddharthd
|
22e4a1ead0 |
fix(splits): make every split account for 100%
ci / lint-test (push) Successful in 1m39s
A 50/50 arrangement was stored as a single row saying "Sonu 50%". The
arithmetic was never wrong — `myShare` resolves the payer's share as
`100 - SUM(everyone else)`, so balances and per-user spend were correct
throughout. It was still a bug, because a ledger is read as well as
computed: on screen that row is a 50% share against a blank, which looks
like half the money is unallocated and is indistinguishable from a split
somebody abandoned half-finished.
It also leaked. `getSharedTransactions` filters by participant with an
EXISTS on an explicit split row, so filtering the Shared view by the
payer silently dropped every transaction where their share was only ever
implied.
Four write paths could produce it, three of them unguarded:
- the Slack nudge's share button, which inserted one row
- `POST /api/transactions`, where the add form shows an amber total
under 100 but saves anyway — this is how Lawn Mowing and Hedge
Pruning were stored
- `applyRuleActions`, where ten of the fourteen live split rules name
only the other person
`completeSplit` is now the single place that writes the remainder, and
every one of those paths ends in it. The remainder goes to the
transaction's *owner*, never to "me": the owner's row on their own
transaction is excluded from both halves of the balance query, so it
cannot create, enlarge or discharge a debt, whereas a row for me on
someone else's transaction is a real obligation. That distinction is
what makes this safe to apply to existing data.
Also fixes the order panel's "Shared 50/50" toggle, which was inert in
both directions: it posted a lone 50% row to share (rejected — must
total 100%) and an empty array to un-share (rejected — array required),
because no way to clear a split existed. DELETE on the splits route is
that way.
Backfill: 7 rows, verified against a row-level dump diff — 2549 -> 2556
rows, none removed, none modified — and participant balances byte
identical before and after (Molina 19556.07, Sonu 20913.35). Every split
in the database now totals 100%.
Not done: a database-level constraint. Enforcing the sum needs a
deferred constraint trigger, and the rule path commits its DELETE and
INSERT as separate statements, so the trigger would reject the
intermediate state. Making it work means wrapping every write path in a
transaction, which is a larger change than the defect warrants.
|
||
|
siddharthd
|
dd0462a5f9 |
fix(orders): decode • so item options separate again
ci / lint-test (push) Successful in 53s
`•` was missing from the entity table, and that was not cosmetic. DoorDash separates an item's name from its options with a bullet and parseDoorDashLineItems splits on the literal "•" — so left encoded, the split never happened and the line collapsed into the description: "Bucket and Side Pack (Meal Deals) • Hot Bucket • Chips" with options []. The entity showed on screen and the structure behind it was gone. Numeric entities are now decoded generically rather than one at a time, which is how $ came to be listed individually while its neighbours were not, and & resolves last so a literal "&bull;" stays as written instead of turning into a bullet. Repaired the 47 stored rows by re-parsing the captured email behind each one rather than string-replacing the entity, since a replacement would have fixed the display and left options [] underneath. Rehearsed first: all 47 re-parsed, all 47 gained options, 0 line items lost, 0 amounts changed. Old values kept in dump/rollback-line-items-20260728-223737.json. Pre-existing — 22 rows predate today — but the pre-cutover backfill more than doubled the affected rows, which is what surfaced it. Verified in the Order details panel, not in SQL. |
||
|
siddharthd
|
3144cf3176 |
feat(orders): record credits orders from before the cutover
ci / lint-test (push) Successful in 58s
I1 refused any credits-funded order dated before 2026-01-09, storing
nothing at all — no transaction and no metadata, so the receipt was
discarded rather than kept as history.
Its reason was splits, not spend: before the cutover shared expenses
lived in SplitMyExpenses, and re-importing them would double-charge
against carryover transaction 2348. That reason expired with
|
||
|
siddharthd
|
fe104a9618 |
fix(orders): read both legs of an Uber payment
ci / lint-test (push) Successful in 53s
Uber Cash IS credits, and the payment line had become unreadable in the
newer layout: "Payments Uber Cash 10/17/25 8:50 PM A$54.87" carries a
timestamp between the label and the amount, and writes the currency as a
prefix. Both defeated the pattern, so credits_amount stayed null and the
order was filed as card-settled — sent looking for a card leg that does
not exist, found nothing, and left as an orphan with no transaction and
no card tail to match on. 118 captured messages sit in that state, and
every one of them is pre-cutover, so reading them correctly means I1
skips them rather than storing enrichment that points at nothing.
The card leg had the same blind spot, hidden behind the first: the gap
between the mask and the amount was [^\d]{0,40}, which a timestamp
breaks. While BOTH legs were unreadable a mixed payment still looked
consistent — the order read as card-settled for the full total. Fixing
only credits reads half an order, which validateOrderTotals correctly
refuses. Found exactly that way: four messages that validated before
began failing "payments sum to 34.92 but receipt states 43.60".
Legs are summed rather than taken first, because one order can be charged
in instalments and an instrument can carry no mask at all (PayPal). But a
leg that already equals the stated total IS the payment, not an
instalment: a Dubai trip prints an AED 17.67 authorisation and then the
AED 577.83 settled charge, and adding the hold overstates the trip. A
mixed credits+card order is unaffected — neither leg equals the total
there, which is why it needs summing.
A/B over all 776 captures: 468 parsed by both, zero change to any amount,
currency or existing card tail, 0 lost, 118 credits figures newly read.
Mutation-tested: dropping the exact-leg rule fails 2 tests, dropping the
Uber Cash read fails 7.
|
||
|
siddharthd
|
63aaf1eb21 |
fix(orders): read A$ totals and credits-funded receipts
ci / lint-test (push) Successful in 51s
Two parse bugs that between them made 218 of 776 captured messages
unreadable. Neither was the "old template" they were filed as.
Uber writes the currency three ways and only two were handled. "Total
A$54.87" is what it sends for ordinary Australian orders — A$ misses
[A-Z]{3} by one character — so 98 of 275 Uber Eats mails and 78 of 275
trip mails failed with "no Total found" while the amount sat in plain
sight. Most were 2024-2025, i.e. current mail. NZ$, US$, S$, HK$, C$ and
a bare rupee/euro/pound symbol are handled the same way. A bare "$" is
still left unresolved on purpose: a dozen currencies use it, and the
body-wide scan that reads the receipt's own stated code should win.
A DoorDash order paid from credits states "Total Charged $0.00"
truthfully, above a real subtotal. Read literally that is a $0 order, and
validateOrderTotals rejected 88 of them as non-positive — discarding
exactly the credit-funded spend this pipeline exists to surface. The
order's value is its subtotal; recording zero would show the order and
hide what it cost. Guarded on the receipt actually saying credits, so an
empty mail still fails rather than inheriting a stray subtotal, and the
header cross-check stands down for these or it rejects the figure the
parser deliberately overrode.
Measured A/B over all 776 captures: 254 parsed by both parsers with zero
change to any amount or currency, 0 lost, 214 newly readable. Both fixes
mutation-tested — reverting the regex fails 4 tests, removing the credits
branch fails 3.
Fixtures are real captured receipts, per the 2026-07-26 rewrite: the
earlier synthetic suite passed while the parser could not read a real
email.
No history replay — I7 idempotency refuses re-reads and that needs an
explicit update mode. This fixes ingest from here on.
|
||
|
siddharthd
|
d458228625 |
feat(orders): a fifth verdict, 'bad', between ok and never again
ci / lint-test (push) Successful in 47s
The jump from "OK" to "Never again" is too big and most disappointments live in the gap (user, 2026-07-28) — so a merely poor meal either flattered itself as OK or got blacklisted. Only 'never' raises the warning on a future order. A blacklist that fires for every mediocre delivery is one nobody reads, so 'bad' records the disappointment without triggering the alarm. Both set order_again = false — you would not choose either again — and that split between "would I order it" and "warn me about it" is the point of the extra level. Migration widens the CHECK; nothing is removed, so no existing row needs mapping. |
||
|
siddharthd
|
9f0f38449b |
feat(slack): ask the other person too, and put rating before sharing
ci / lint-test (push) Successful in 43s
Two halves of the same requirement, one of which was quietly missing. Sharing split the money but never reached her: she is not in #smarthome, so the card whose caption said "both verdicts welcome" was one she could not see. Now a share DMs her a card of her own. A DM rather than adding her to the channel, so her surface stays "orders that concern me" instead of the whole house's ops feed. She was already in SLACK_USER_MAP, so her press files under participant 4. Only on the press that turns sharing ON, and only when someone else did the sharing. Re-notifying on every later rating press would turn one shared meal into a stream of DMs, which is how a nudge gets muted. Her card carries no share button: she is being told it was shared, not asked to decide, and two people toggling one split from separate copies of a card is a race with no upside. The app still holds no Slack bot token — it returns a notify instruction and n8n sends it, the same shape as the modal open. If SLACK_USER_MAP has no id for her the DM is skipped silently: the split is correct and complete either way, and failing the press over an unaddressable nudge would be the worse trade. Card reordered to rate -> details -> share. You judge the food, then decide who pays for it; asking "was this shared?" first inverts the order a person thinks in. The status caption moved under the share button it describes rather than sitting orphaned mid-card. |
||
|
siddharthd
|
0595d49d5c |
fix(orders): the restaurant is the merchant, not the courier
ci / lint-test (push) Successful in 43s
Reverses a change made on request. The platform in the headline
fragmented the merchant: the same restaurant read as two, depending on
who carried the bag, and that is not a distinction anyone rating the food
cares about. It also already has a home — the expandable Order details
panel renders expense_metadata.platform next to its heading, which is
where the user asked for it.
The fragmentation was worse than cosmetic. merchantVerdict joined on an
exact merchant_normalized, and the platforms capitalise differently
("TEG Kebabs & Biryani" on Uber Eats, "TEG KEBABS & BIRYANI" on
DoorDash), so one restaurant kept two separate histories and a "never
again" recorded through one app never warned in the other — silently
defeating the point of the memory. Now case-folded; verified on real
data, where the DoorDash order sees 1 prior verdict against 0 before.
81 existing descriptions backfilled in one transaction, dry-run first and
dumped beforehand. The regex is anchored to the end so suburb parens
survive: "Order - Coles (Wyndham Vale) (Uber Eats)" becomes
"Order - Coles (Wyndham Vale)", not "Order - Coles".
|
||
|
siddharthd
|
14d6b40578 |
feat(slack): per-item verdicts and a note, in a Slack modal
ci / lint-test (push) Successful in 42s
The card can rate an order but cannot ask which dish or why: a message cannot collect free text, and an actions block caps at 25 elements while item counts vary per receipt. A modal is the only Slack-native answer, and it stays inside Slack — no browser, no app, which is the whole reason it exists rather than a link. The overall rating deliberately stays on the card. That is the thing done every time and it should cost one tap; this is for when something was notably good or bad. finance-app holds no Slack bot token by design, so it returns the view and n8n — which already has the credential — calls views.open. One copy of the token, no new secret, no compose change. Item text travels in private_metadata because a submission returns block ids and values, never labels, so there is otherwise no way back to which dish a radio button referred to. Capped at 20 rows: a grocery order runs long and nobody scrolls a modal to rate a tin of tomatoes. The modal does NOT write the rating. A form that silently reset a decision the user did not revisit is the same class of bug as the split rewrite that dropped `settled`. |
||
|
siddharthd
|
8fbcbc5f83 |
fix(slack): one select instead of four rating buttons
ci / lint-test (push) Successful in 47s
Slack's mobile client gives every button in an actions block its own full-width row, so the four ratings rendered as four stacked bars and the nudge filled the screen (user, with a screenshot). A select is one row and still one decision. The interactive route now resolves both shapes — a button carries `value`, a select carries it on `selected_option`. Reading only the former would have left sharing working while rating silently did nothing. |
||
|
siddharthd
|
aaa36dd75e |
feat(slack): answer the order nudge in Slack, without opening the app
ci / lint-test (push) Successful in 40s
Being sent to a web app to answer "was this shared?" is enough friction that the question stops getting answered — which is the exact failure the nudge exists to prevent. So the buttons now act in place: pressing "Shared 50/50" writes the transaction_splits row and edits the message, and the app is never opened. Slack does NOT reach this route directly. It posts to an n8n webhook that forwards the raw body and signature headers here (user's suggestion). That is the better shape: n8n already terminates public webhooks, so the app keeps its blanket OAuth chain and gains no internet-facing unauthenticated route, and no Traefik change is needed. n8n cannot verify the signature itself — its Code sandbox has no `require`, so no `crypto`. Two independent gates, both failing closed: the shared x-ingest-token (came from n8n) and Slack's v0 signature over the raw body (came from Slack, not replayed within 5 minutes). An unset signing secret rejects everything rather than waving it through, because the alternative turns a misconfigured deploy into an open write endpoint. An unmapped Slack user is refused rather than defaulted to the owner. In a two-person household a wrong attribution is not a rounding error, it is the other person's opinion recorded under your name. Block Kit is built in the app (ingest returns slack_blocks) rather than in n8n expressions: a template string is untestable, and this shape has to stay in step with what the interactive endpoint re-renders after each press. Null when there is no transaction yet — a card-settled order is parked until its statement arrives, so there is nothing to split or rate. Also: /transactions now honours ?q=, so the link lands on the row instead of the top of an unfiltered ledger. |
||
|
siddharthd
|
0cb46a087b |
feat(orders): record what we thought of an order, per person
The ledger already knew we had ordered from a place; it did not know the food was bad. Orders got repeated from places we disliked because nobody remembered by the time the next one went in. That is what the receipt ingestion was for (ING-9) and the last piece was missing: order_reviews existed as a table with no API, no UI and no writes. Four levels, not three. "Loved" and "liked" are both "would order again" but only one is worth a detour, and "ok" is not a recommendation. A verdict belongs to a person, not to an order. A shared meal produces two opinions and they routinely disagree — that disagreement is the useful part, and the old UNIQUE on transaction_id alone could not hold it. Now UNIQUE (transaction_id, participant_id), and the default is the signed-in user rather than the owner: Sonu authenticates through the same Traefik OAuth as participant 4, so an owner default would have filed her verdict under his name. Per-item opinions key on the item DESCRIPTION, not its index. An index is meaningless across orders; "the Pad Thai here is good" is the signal that has to survive into the next order from the same merchant. Only the two poles are offered — a per-item "ok" answers neither of the questions you ask at order time. Sharing is recorded as a real 50/50 split, not a decorative flag. The split already IS the record that an order was shared, and two records of one fact drift apart. An ABSENT item_verdicts means "leave them alone"; an empty array clears them. Without that distinction a note-only save silently wipes every per-item opinion — the same shape as the bug that reset `settled` on split rewrites, and just as invisible on screen. Mutation-tested: making keepItems a no-op fails exactly one test. mockDbWithPool gained queryRow. Omitting an export from the mock makes it undefined at the call site, which fails as "not a function" and reads like a code bug rather than a test-harness gap. |
||
|
siddharthd
|
788219b9fd |
fix(splits): the cutover date, not a boolean, is what gates a balance
ci / lint-test (push) Successful in 46s
Nothing dated before 2026-01-09 can be owed, because carryover transaction 2348
already carries the entire pre-cutover balance as a single figure. ACTIVE_OBLIGATION
now says so directly.
This inverts which mechanism is load-bearing, and that is the point. Until now
the only thing keeping $37,233.28 of paid debt out of the balances was
transaction_splits.settled -- a boolean that any delete-and-recreate write path
resets to false, as the split modal did until commit
|
||
|
siddharthd
|
dbfbd5196d |
fix(transactions): supersede rows imported twice instead of deleting them
ci / lint-test (push) Successful in 48s
Statements 107, 142 and 143 bill overlapping periods on one ANZ account, so 31 transactions -- $42,040.68 -- are in the ledger twice. They are marked superseded, not deleted. Every child of transactions is ON DELETE CASCADE (splits, tags, overrides, expense_metadata, order_reviews), so deleting "the duplicate" destroys whatever curation sits on it, and which member of a pair holds that curation is an accident of import order: here 1 pair carries splits and 6 carry overrides, all on the surviving side, but nothing guarantees that. Superseding keeps the row, keeps its children, and makes a mistake one UPDATE to undo rather than a restore from backup. reconciled_with_id could not be reused. Its predicate is scoped to statement_id IS NULL on purpose -- a statement line pointing at something else is the survivor, not the duplicate -- and here both rows are statement lines. The exclusion goes into EXCLUDE_RECONCILED_SOURCE rather than into a new fragment, so every query already asking "count each purchase once" gets it without being edited. The trip cost queries did not use that fragment at all and now do; verified a no-op on current data (0 trip-tagged rows are either reconciled sources or duplicates), but they were one import away from double-counting. Most of the $42k is transfers and investments, which spend already excludes. The damage was elsewhere: duplicated rows in the list, and rules re-splitting a duplicate -- txn 3807 is one of these 31 and was a candidate for splitting earlier today. Balances are unchanged: no duplicate carried a split. |
||
|
siddharthd
|
d5589b2980 |
feat(statements): flag billing periods that overlap another statement
ci / lint-test (push) Successful in 46s
An account cannot be billed twice for the same day, so an overlap means those
transactions are in the ledger twice. ANZ statements 107 and 143 overlap by 118
days and put roughly $42,000 of duplicate rows in; nothing anywhere said so.
Two details decide whether this catches the real case:
- Account numbers compare with non-digits stripped. The duplicate got in
because the existing key compared raw text and ANZ wrote the same account
as 408556264 on one statement and 4085-56264 on the other.
- The range is half-open. These statements are issued back-to-back with one
period ending the day the next starts, so inclusive bounds flagged 5 pairs
of which 3 were consecutive and fine. Half-open leaves exactly the 2 real
ones.
NULL bounds are excluded rather than handed to daterange, where NULL means
unbounded and an undated statement would overlap all of history.
Detection only. It does not refuse the import or touch the duplicate rows --
cleaning those is separate, and must supersede rather than delete because every
child of transactions is ON DELETE CASCADE and the curation sits on the
duplicate side.
Both subtleties have a test, and both fail if you undo them.
|
||
|
siddharthd
|
8c21893cc2 |
fix(trips): money that came back is not what the trip cost
ci / lint-test (push) Successful in 1m28s
Every trip figure filtered on transaction_type IN ('debit','fee','interest'),
which drops refunds and credits outright. A partly-refunded booking therefore
read at its full price and the refund subtracted nothing, anywhere: the
headline total_spend, the category breakdown, the daily chart, top merchants
and the tag breakdown were all gross.
This is the same defect the general analytics fixed once already, which is why
NET_SPEND_ROWS and SPEND_SIGNED exist -- a refunded Expedia purchase read as
$2,888.92 of spend until they did. Trip analytics never adopted them. Doing so
now costs one predicate and one expression per query.
getTrips/getTripById needed the trips alias moved to `tr`: the fragments assume
`t` is `transactions`, and hand-inlining a copy rather than renaming is exactly
how the reconciled-row exclusion drifted out of the analytics routes before.
On Europe 2026 this is $821.12 -- a LuxuryEscapes booking with two part-credits
against it, and a FreeNow hold adjustment. Fully cancelled bookings are a
different case and are handled by untagging both legs from the trip by hand,
because a trip never incurred a cost it cancelled.
No balance moves: the owed query already excludes credits and a refund carries
no split. There is a test asserting exactly that, and it passes with or without
this change -- it is a guard, not a proof. The three that do prove it fail
without it.
|
||
|
siddharthd
|
4fcb135805 |
fix(trips): a trip figure must only count what the owner is owed
ci / lint-test (push) Successful in 46s
The per-trip owed number shipped in
|
||
|
siddharthd
|
ae23b03d5d |
fix(trips): carry the currency and reconcile rules into the trip figure
|
||
|
siddharthd
|
689fadc8b9 |
feat(shared): give a payment a tab to settle
A payment has only ever recorded from, to, amount and date. That is why the per-trip owed figure did not exist — getTripAnalytics said so where the number should have been: "split_payments carries no trip attribution, so a payment cannot be assigned to a trip. Settlement is a property of the whole relationship." Every trip therefore read 100% unsettled, including trips paid in full. It is also why the Shared page silently drops payments under a tag filter. With one global pool there was nothing honest to subtract, so it showed gross splits under the same label. A tag is a view; a scope is a ledger. The scope is a trip, not a new settlement_contexts table. trips already has owner_id, dates and archived, and transaction_overrides.trip_id already decides membership. A second grouping beside it would be two unsynchronised scopes over the same rows, with no invariant saying which governs. NULL means the ongoing household tab, which never closes. settled answers a different question and the two must not be collapsed: trip_id is which tab, settled is whether the obligation is still live. Critically, a live obligation is NOT settled by flipping the flag — it is settled by recording the payment, and the balance nets to zero on its own. Doing both would subtract the settlement twice. So settled is written only by the historical import, for repayments made on a platform we no longer run, and there is deliberately no "mark settled" action. Both owed figures now exclude settled splits and the trip figure nets its own payments. Spend analytics (myShare/mySplitOf) deliberately still count settled rows: my half of a 2025 grocery shop is my spend whether or not the other half was ever repaid, and filtering them would re-inflate exactly the figures importing settled history exists to correct. Also drops /api/participants/[id]/balance. It had no consumers, no owner scoping, no debit/credit signs and no EXCLUDE_RECONCILED_SOURCE — a fourth balance implementation that disagreed with the others and would have imported three bugs if anything had aligned to it. getTripAnalytics had no test at all. It has five now, including the one that matters: a household payment must not make a trip look paid. Verified by mutation — neutering the settled filter fails three, and dropping the trip filter on payments fails that one. |
||
|
siddharthd
|
a4ab543a6c |
fix(analytics): make the displayed numbers mean what they say
ci / lint-test (push) Failing after 44s
Six metric-integrity defects from the UI/IA review, plus two found while verifying the review's own claims against the code. The reconciled-row exclusion existed only in queries.ts. Every analytics route counted the superseded manual rows as spend — 48 rows, $4,474.79 of double count, invisible precisely because the transaction list looked right. It is now one fragment both sides import. The spend-pace chart computed its own totals in the browser: gross amounts, debits only, no personal share, no refunds, fees, interest or itemised loan repayments. On live data it ended July at $4,747.31 under a headline reading $3,597.10 — and its own baseline line was drawn from the split-adjusted monthly totals, so the two series in one chart disagreed with each other. Both now come from /api/analytics/daily, built from the same fragments as the headline. Fees aggregated every statement ever imported with no date filter, under a heading with no period, so a lifetime figure read as a current one and grew forever. Now bounded, labelled, and selectable. Comparisons no longer measure a month in progress against complete ones: the in-progress month is out of every baseline, and a selected current month is compared through the same day. Two the review did not catch: - Every analytics window was a day early. toISOString() on a local-midnight Date converts backwards through UTC. Surfaced only once fees started reporting the range it had used. - /monthly rounded per category, /daily per category-day, so the pace chart ended a few cents off the headline above it. Shared currency needed amending rather than applying. Reading s.currency would have labelled every order row AUD, since an order receipt has no statement and carries its own currency — the opposite convention from a foreign charge on an AUD statement, where amount IS AUD. NATIVE_CURRENCY's COALESCE order keeps the two apart. Balances also now count rows whose AUD value is genuinely unknown instead of netting a foreign figure against AUD ones. Latent today: no foreign transaction is currently split. Tag-filtered balance cards no longer claim "owes you". With a filter on, payments are deliberately not subtracted, so the figure is a split total and settling against it would record a payment for a debt that never was. Split-coverage warnings deliberately omitted (user decision). |
||
|
siddharthd
|
3bb67f370d |
feat(orders): show where an Uber trip went, in the list
ci / lint-test (push) Failing after 41s
Five rows all reading "Order - Uber Trip" are indistinguishable — the list gives you a date and an amount and nothing to tell one ride from another (user, 2026-07-27). Where the trip went is exactly what separates them, and it was already stored on expense_metadata.route since this morning; nothing in the list read it. getTransactions now joins the receipt (both directions — transaction_id OR matched_transaction_id, since a card-settled order points at the statement line instead) and the description cell renders "Terminal 2, Melbourne Airport (MEL) → 19 Lady Penrhyn Dr" in the same italic sub-line notes use. Two deliberate limits: - **A note the user wrote always wins.** This only fills an empty sub-line; it never occupies the notes field, which is theirs. - **Deliveries are excluded.** Their merchant already identifies them, so the restaurant's street address would be clutter on every food order. Gated on platform = 'uber'. The summary keeps the first two comma-segments of each address — a truncation, not a guess about geography. Uber puts the venue or street first, which is the identifying part; the full stops with their times stay in the title attribute. |
||
|
siddharthd
|
6161ddc9de |
fix(orders): don't restate a platform the merchant already names
ci / lint-test (push) Failing after 41s
Trip rows read "Order - Uber Trip (Uber)". The suffix exists so you can tell where to go and look; when the merchant is literally "Uber Trip" it says nothing. What identifies a trip is its two addresses, and those are in the Order details panel. Existing rows updated in prod. |
||
|
siddharthd
|
c656f5d26b |
feat(orders): read Uber trips, and reject the charge summary that duplicates them
ci / lint-test (push) Failing after 45s
Local rides are paid with credits (only overseas ones go on a card), so trips
belong to this slice and were simply never fetched — the Graph query searched
"order with Uber", the Eats subject. Captured 15 real messages from the mailbox
via a dry-run before touching anything, which found two defects that no amount
of reasoning about the template would have:
**Uber sends two mails per trip.** A "charge summary" when the ride ends, then
the real receipt when payment settles — same subject, same total. The summary
carries no tripReference, so order_reference fell back to `msg:<message-id>`
and I7 could not dedupe it against the receipt that follows. Every trip would
have been recorded twice. It says so itself ("This is not a payment receipt ...
You will receive a trip receipt when the payment is processed"), so it is now a
NotAReceiptError — 200 and silent, like every other expected non-receipt.
**Trip receipts label neither end of the journey.** Delivery receipts write
"1:20 pm - Pick-up"; trips print the time alone. The split regex put the time
into `label` and left `time` null. Time is now read properly, and a two-stop
trip is labelled Pick-up/Drop-off positionally — only where the receipt was
silent, so a template that does label its stops keeps its own wording.
Verified against all 15 captured messages: 7 trips recorded, 5 charge summaries
and 3 promotions skipped, 0 failures, no duplicate references. Two of the seven
are AUD credits-funded ($84.78 + $47.97) and would become transactions; the
five NZD ones are card-settled and correctly create provenance only (I5).
Fixtures ut-00 (local credits trip), ut-01 (overseas card trip) and ut-summary
(the charge summary) are captured mail, not written by hand.
|
||
|
siddharthd
|
b6cd62f7b5 |
feat(orders): show the receipt in the transaction detail panel
ci / lint-test (push) Failing after 47s
`expense_metadata` has held the itemised receipt since ingestion started and nothing in the UI ever read it. A transaction that came from a DoorDash or Uber Eats receipt showed a merchant and an amount, with the item list and the delivery addresses sitting unread in the row behind it (user, 2026-07-27). Adds GET /api/transactions/[id]/order and an "Order details" section in the edit modal: line items with their options, pick-up/delivery stops with times and addresses, the card tail when one was involved, and the provider's own order reference. Two details that matter: - The lookup resolves from **both** sides — `transaction_id` OR `matched_transaction_id`. A card-settled order creates no transaction of its own (I5); the receipt points at the statement line instead. Matching only on transaction_id would have left the panel blank on exactly the card-paid orders, which are the ones whose detail is hardest to find elsewhere. - An empty item list says so in words rather than rendering nothing. Uber itemises groceries but not restaurant orders, and orders ingested before the Uber item parser existed have none either — a blank section reads as a bug when it is usually the receipt. Read-only. This is what a provider sent; editing it would make provenance mean nothing. |
||
|
siddharthd
|
df4b875b82 |
feat(orders): make an ingested order legible in the transactions view
ci / lint-test (push) Failing after 43s
Four things the view could not tell you, all from reading the rows (user,
2026-07-27).
**Which platform.** The parser has always known — it has to, to read the
template — and then discarded it. "Order - Burger Corner" gives no way to know
whether to open DoorDash or Uber Eats for the detail, and restaurants exist on
both. Now stored on expense_metadata and named in the description:
"Order - Burger Corner (Uber Eats)". Migration 0021 recovers it for the 101
backfilled rows from the order_reference shape — DoorDash receipts carry no id
of their own so ingestion synthesises `msg:<message-id>`, Uber carries a real
trip UUID, which makes the discriminator exact.
**Bank said "Manual".** That label is derived, not stored, and "Manual" reads
as "hand-entered, still awaiting a card line to match". A gift-card order has
no card line coming, ever. It now reads "Gift Card", and — the part that
actually mattered — credits joins cash in needsCardMatch(), so these stop
sitting in the pending-reconciliation queue. All 81 were queued against a match
that could not exist.
**Uber line items were never parsed.** 67 of 101 orders had none. Uber itemises
groceries but not restaurant orders, so some of that is genuine; the rest was
simply unread. Its markup is better than DoorDash's — every cell carries a
data-testid with the item's uuid, so qty/title/amount bind by id rather than by
column position. Sold-out items (0.00) are kept: they are why a total is lower
than what was ordered.
**Uber prints pick-up and delivery addresses on every receipt** and they were
thrown away. Captured as `route` [{label, time, address}], de-duplicated
because the template renders the whole block twice for narrow screens. Wording
is kept as printed ("Pick-up" on some receipts, "Pickup" on others) rather than
normalised, so a template change stays visible. This is the same block a *trip*
receipt uses for start and destination — rides are not ingested today, but the
reader will not need changing when they are.
Also stores source_email_subject/from, which order ingestion had left null on
columns that already existed.
Verified against the captured corpus: route on all 6 Uber fixtures, 5/5 items
on the GLOMARK grocery receipt including the sold-out one. Production data
updated by smarthome:docker/scripts/order-presentation-2026-07-27.sql
(81 descriptions, `backfill` tag, re-run clean). `route` and Uber line items
are parsed from here on only — recovering them for already-ingested orders
means re-reading the mail, which I7 idempotency refuses by design.
|
||
|
siddharthd
|
ae0c34fce7 |
fix(orders): two defects the backfill exposed that tests could not
ci / lint-test (push) Failing after 1m26s
Both were found by looking at the data after the live backfill, not by the
suite — 105 tests were green while 85 rows were invisible and 4 were double
counted.
owner_id was NULL on every ingested order. Analytics scope on
COALESCE(t.owner_id, s.owner_id), and an ingested order carries no statement,
so the coalesce resolved to NULL and matched no owner. The rows existed in
`transactions` and appeared in no view in the app. Ingestion now sets
DEFAULT_OWNER_ID, and a regression test asserts the row survives the same
COALESCE scoping the UI uses.
[Family] orders are card-settled, not credits-funded. Their receipts name the
payer ("Payments Siddharth LKR 3,783.20") and no instrument, which an earlier
version read as credits. The card statement carries all four of them (CBA
...3893, exact foreign_currency_amount matches), so creating a transaction
duplicated spend already recorded — the double-count I5 exists to prevent.
They now record provenance only; the statement line is the transaction and is
what carries the `family` tag that keeps them out of budgets.
Production data corrected separately by
smarthome:docker/scripts/fix-order-backfill-2026-07-27.sql.
Also: reconciliation tests no longer assert global row counts.
reconcilePendingOrders() scans every pending row, so leftovers from other
files moved the totals — the source of an intermittent failure that only
appeared on the first run after a source edit.
|
||
|
siddharthd
|
5db42f086f |
fix(orders): match the card leg by masking, not by card brand
ci / lint-test (push) Failing after 45s
Backfill dry-run over 130 real messages surfaced one 422: 'payments sum to 1.17
but receipt states 16.50'. The receipt is a mixed Uber payment —
Uber Cash $1.17 + Westpac ••••8032 $15.33 — and the card regex only matched
Visa|MasterCard|American Express|Amex, so an issuer-named leg was dropped
entirely. validateOrderTotals correctly refused it rather than recording $1.17
as the cost of a $16.50 order.
Anchors on the ••••NNNN masking instead, which also covers the form already
seen in the corpus ('Mastercard ••••3893 (CBA Ultimate) CHF 51.23'). Fixture
and regression test added.
Also repoints .env.test at the current postgres-personal container IP and
documents why: the container publishes no host port, so the address changes on
every recreate and the whole integration suite fails with connection errors
until it is refreshed.
|
||
|
siddharthd
|
1103397397 |
fix(orders): three defects found reviewing my own branch
ci / lint-test (push) Failing after 1m27s
None of these were caught by 105 green tests, because the code they live in was barely tested and the HTTP path was not tested at all. 1. reconcilePendingOrders hardcoded category 'dining', so any order resolved through the deferred path booked as dining regardless of merchant — a Woolworths grocery order that parks and later reconciles was misfiled. That reintroduced, through the back door, exactly the misfiling resolveCategory() exists to prevent. Now calls it. 2. reconcileCardLeg never marked a statement line as consumed, so two orders on the same card inside the +/-4 day window both bound to the same charge and each booked its own credits remainder — double-counting spend. At 10-15 orders a month on one card that is not a corner case. Migration 0020 adds matched_transaction_id with a unique index; the matcher now excludes lines already claimed. 3. The ingest API returned HTTP 200 for every parse failure, and the Slack alert fires only on non-200. So the single most likely production failure — a provider template change breaking every order at once — was completely silent. Split into NotAReceiptError (promotions, delivery updates, refund and adjustment notices: 200, silent, expected traffic) and OrderParseError (it IS a receipt and would not parse: 422, alerts). Also: order_reference now anchors on Uber's own tripReference cell rather than 'first UUID in the document'. I had claimed to verify that the first UUID was always the order UUID; that check compared against zero samples and was vacuous. tripReference is present in all 29 captured receipts and, for ue-00, equals the UUID the PDF redirect resolves to. The positional fallback remains but only flags when there is genuine ambiguity. Adds the API route's first tests — auth gate and error taxonomy — plus anchoring regressions. 63 unit + 53 integration green on five consecutive runs; corpus holds at 63/65. |
||
|
siddharthd
|
a9e251d969 |
feat(orders): amendments, family imports, and the ingest API
Closes the three gaps left after the parser rebuild.
Refund amendments. ue-05 is a real refund: 'Previous total $49.94 / Refund
-$4.21 / New Total $45.73'. Uber reuses the order UUID across the receipt and
the amendment, so the two can be matched. The transaction is reduced in place
rather than offset with a second row — the order is one event whose cost
changed, and a compensating row would misreport both the meal count and the
merchant's spend. When the original was never ingested, nothing is invented.
[Family] orders now import instead of parking. Their payment line names the
payer, not an instrument ('Payments Siddharth LKR 3,783.20'), so no split is
recoverable and there is no card leg to reconcile against — they would have sat
pending forever, which fails the actual requirement to import and tag them.
Treated as credits, flagged as an assumption. Safe because the family tag
removes them from every budget regardless of instrument, and the LKR amount is
preserved with amount_aud left NULL rather than asserting an FX rate.
Ingest API. n8n now POSTs each message to /api/orders/ingest instead of parsing
in a Code node — the n8n sandbox has no require or fs, so a parser there cannot
be tested against the fixture corpus, which is the one thing that makes this
parser trustworthy. Auth is a shared secret, since machine callers have no
Traefik session header. Rejections return 422 and record nothing.
60 unit + 45 integration green on three consecutive runs; 63/65 corpus holds.
|
||
|
siddharthd
|
c82a22767f |
feat(orders): wire the real parser in, defer card reconciliation
Ingestion now runs on the rebuilt parser. Three substantive changes. Deferred card reconciliation. A 'MasterCard 8032 and/or credits' receipt never states the split, but the card leg lands on the statement — Subway's $29.08 order shows $13.06 on 8032, so $16.02 was credits. For a live order that statement is weeks away, so the split cannot be settled at ingest time. Such orders are now parked with provenance and no transaction, and reconcilePendingOrders() resolves them once the statement arrives. Backfill takes the same path and resolves immediately. Migration 0019 adds the columns that make an order resumable; applied to personal_test only, prod untouched. Payment detection bug, found by the new tests: the old regex delimited the 'Paid with' line on a double space, which whitespace collapsing removes. Every card and mixed receipt fell through to the credits branch — the Woolworths receipt booked $60.93 of credits spend that never happened. Category resolution reversed deliberately. Correction 1 said never default to dining; the implementation of that sent everything unrecognised to 'other', and knowing six merchants meant Carl's Jr, Taco Bell, Chilli India, Oporto, Schnitz and Souvlaki GR all landed there. Grocers are an enumerable set and restaurants are not, so match groceries explicitly and let the residual be dining. Tests rebuilt on real captured receipts; the synthetic fixtures are deleted. 60 unit + 41 integration green on three consecutive runs. |
||
|
siddharthd
|
33db7d05ef |
feat(orders): rebuild the receipt parser against real captured email
The previous parser was written against synthetic fixtures shaped to match the code. It invented a table layout DoorDash does not send, generated order_reference from Math.random(), read the order date from a 'Date:' string present in no real message, and detected [Family] by searching the body for the substring 'family'. Its tests passed because the fixtures were built to satisfy it. Against 36 real DoorDash and 29 real Uber Eats receipts it does not work. Rebuilt from the real corpus. 63 of 65 now parse and validate; the 2 rejected are correctly rejected — one is an order-adjustment notice and one a refund, neither of which is a receipt. Corrects an inherited diagnosis: the Mad Mex 'Discounts -$24.09' was recorded as an HTML-flattening artefact masking a 'true discount of $9.45'. Parsing the table cells structurally returns the same figures and no $9.45 exists anywhere in the message — DoorDash genuinely prints a Discounts line that equals subtotal plus service fee, and the components fail to reconcile on 32 of 36 receipts. So the breakdown is stored as provenance and never gated on; validation instead cross-checks the two independently stated totals and the payment line, which is the number that becomes money. Real-world cases the corpus forced, none of which were in the spec: [Family] orders are LKR purchases for family in Sri Lanka (reading them as dollars inflates ~200x), Swiss orders arrive in CHF, grocery 'Final receipt' mails carry no Total Charged row, and a declined payment is printed alongside the successful retry and must be skipped or it records money that never moved. order_reference now comes from the Uber order UUID embedded in the body, or the provider message id where DoorDash supplies no order id at all — never random, so re-ingestion is genuinely idempotent. |
||
|
siddharthd
|
775e5cc08f | feat(orders): add Uber Eats and Uber Rides support, fare parsing, and category resolution |