14 Commits

Include renames

Author	SHA1	Message	Date
siddharthd	689fadc8b9	feat(shared): give a payment a tab to settle ... A payment has only ever recorded from, to, amount and date. That is why the per-trip owed figure did not exist — getTripAnalytics said so where the number should have been: "split_payments carries no trip attribution, so a payment cannot be assigned to a trip. Settlement is a property of the whole relationship." Every trip therefore read 100% unsettled, including trips paid in full. It is also why the Shared page silently drops payments under a tag filter. With one global pool there was nothing honest to subtract, so it showed gross splits under the same label. A tag is a view; a scope is a ledger. The scope is a trip, not a new settlement_contexts table. trips already has owner_id, dates and archived, and transaction_overrides.trip_id already decides membership. A second grouping beside it would be two unsynchronised scopes over the same rows, with no invariant saying which governs. NULL means the ongoing household tab, which never closes. settled answers a different question and the two must not be collapsed: trip_id is which tab, settled is whether the obligation is still live. Critically, a live obligation is NOT settled by flipping the flag — it is settled by recording the payment, and the balance nets to zero on its own. Doing both would subtract the settlement twice. So settled is written only by the historical import, for repayments made on a platform we no longer run, and there is deliberately no "mark settled" action. Both owed figures now exclude settled splits and the trip figure nets its own payments. Spend analytics (myShare/mySplitOf) deliberately still count settled rows: my half of a 2025 grocery shop is my spend whether or not the other half was ever repaid, and filtering them would re-inflate exactly the figures importing settled history exists to correct. Also drops /api/participants/[id]/balance. It had no consumers, no owner scoping, no debit/credit signs and no EXCLUDE_RECONCILED_SOURCE — a fourth balance implementation that disagreed with the others and would have imported three bugs if anything had aligned to it. getTripAnalytics had no test at all. It has five now, including the one that matters: a household payment must not make a trip look paid. Verified by mutation — neutering the settled filter fails three, and dropping the trip filter on payments fails that one.	2026-07-27 23:12:05 +10:00
siddharthd	a4ab543a6c	fix(analytics): make the displayed numbers mean what they say ... Six metric-integrity defects from the UI/IA review, plus two found while verifying the review's own claims against the code. The reconciled-row exclusion existed only in queries.ts. Every analytics route counted the superseded manual rows as spend — 48 rows, $4,474.79 of double count, invisible precisely because the transaction list looked right. It is now one fragment both sides import. The spend-pace chart computed its own totals in the browser: gross amounts, debits only, no personal share, no refunds, fees, interest or itemised loan repayments. On live data it ended July at $4,747.31 under a headline reading $3,597.10 — and its own baseline line was drawn from the split-adjusted monthly totals, so the two series in one chart disagreed with each other. Both now come from /api/analytics/daily, built from the same fragments as the headline. Fees aggregated every statement ever imported with no date filter, under a heading with no period, so a lifetime figure read as a current one and grew forever. Now bounded, labelled, and selectable. Comparisons no longer measure a month in progress against complete ones: the in-progress month is out of every baseline, and a selected current month is compared through the same day. Two the review did not catch: - Every analytics window was a day early. toISOString() on a local-midnight Date converts backwards through UTC. Surfaced only once fees started reporting the range it had used. - /monthly rounded per category, /daily per category-day, so the pace chart ended a few cents off the headline above it. Shared currency needed amending rather than applying. Reading s.currency would have labelled every order row AUD, since an order receipt has no statement and carries its own currency — the opposite convention from a foreign charge on an AUD statement, where amount IS AUD. NATIVE_CURRENCY's COALESCE order keeps the two apart. Balances also now count rows whose AUD value is genuinely unknown instead of netting a foreign figure against AUD ones. Latent today: no foreign transaction is currently split. Tag-filtered balance cards no longer claim "owes you". With a filter on, payments are deliberately not subtracted, so the figure is a split total and settling against it would record a payment for a debt that never was. Split-coverage warnings deliberately omitted (user decision).	2026-07-27 17:10:27 +10:00
siddharthd	3bb67f370d	feat(orders): show where an Uber trip went, in the list ... Five rows all reading "Order - Uber Trip" are indistinguishable — the list gives you a date and an amount and nothing to tell one ride from another (user, 2026-07-27). Where the trip went is exactly what separates them, and it was already stored on expense_metadata.route since this morning; nothing in the list read it. getTransactions now joins the receipt (both directions — transaction_id OR matched_transaction_id, since a card-settled order points at the statement line instead) and the description cell renders "Terminal 2, Melbourne Airport (MEL) → 19 Lady Penrhyn Dr" in the same italic sub-line notes use. Two deliberate limits: - **A note the user wrote always wins.** This only fills an empty sub-line; it never occupies the notes field, which is theirs. - **Deliveries are excluded.** Their merchant already identifies them, so the restaurant's street address would be clutter on every food order. Gated on platform = 'uber'. The summary keeps the first two comma-segments of each address — a truncation, not a guess about geography. Uber puts the venue or street first, which is the identifying part; the full stops with their times stay in the title attribute.	2026-07-27 11:43:55 +10:00
siddharthd	c656f5d26b	feat(orders): read Uber trips, and reject the charge summary that duplicates them ... Local rides are paid with credits (only overseas ones go on a card), so trips belong to this slice and were simply never fetched — the Graph query searched "order with Uber", the Eats subject. Captured 15 real messages from the mailbox via a dry-run before touching anything, which found two defects that no amount of reasoning about the template would have: **Uber sends two mails per trip.** A "charge summary" when the ride ends, then the real receipt when payment settles — same subject, same total. The summary carries no tripReference, so order_reference fell back to `msg:<message-id>` and I7 could not dedupe it against the receipt that follows. Every trip would have been recorded twice. It says so itself ("This is not a payment receipt ... You will receive a trip receipt when the payment is processed"), so it is now a NotAReceiptError — 200 and silent, like every other expected non-receipt. **Trip receipts label neither end of the journey.** Delivery receipts write "1:20 pm - Pick-up"; trips print the time alone. The split regex put the time into `label` and left `time` null. Time is now read properly, and a two-stop trip is labelled Pick-up/Drop-off positionally — only where the receipt was silent, so a template that does label its stops keeps its own wording. Verified against all 15 captured messages: 7 trips recorded, 5 charge summaries and 3 promotions skipped, 0 failures, no duplicate references. Two of the seven are AUD credits-funded ($84.78 + $47.97) and would become transactions; the five NZD ones are card-settled and correctly create provenance only (I5). Fixtures ut-00 (local credits trip), ut-01 (overseas card trip) and ut-summary (the charge summary) are captured mail, not written by hand.	2026-07-27 11:21:13 +10:00
siddharthd	4febf38292	test(orders): clean statement fixtures before ingest, not after ... These tests insert a Westpac statement and a `DD *DOORDASH ...` charge, and only removed them at the end of the test — so they survived into the next run, where `reconcileCardLeg` could match one at ingest time and resolve an order that was meant to park `awaiting_card_statement`. That is a real ordering bug in the fixtures regardless. It is my best explanation for the intermittent failure in "parks an unresolvable split", but I could not reproduce it: seeding the exact leftover row and running the old code passed anyway. So this is hygiene with a plausible mechanism, not a confirmed fix — if that test fails again, this was not the cause.	2026-07-27 11:01:23 +10:00
siddharthd	b6cd62f7b5	feat(orders): show the receipt in the transaction detail panel ... `expense_metadata` has held the itemised receipt since ingestion started and nothing in the UI ever read it. A transaction that came from a DoorDash or Uber Eats receipt showed a merchant and an amount, with the item list and the delivery addresses sitting unread in the row behind it (user, 2026-07-27). Adds GET /api/transactions/[id]/order and an "Order details" section in the edit modal: line items with their options, pick-up/delivery stops with times and addresses, the card tail when one was involved, and the provider's own order reference. Two details that matter: - The lookup resolves from **both** sides — `transaction_id` OR `matched_transaction_id`. A card-settled order creates no transaction of its own (I5); the receipt points at the statement line instead. Matching only on transaction_id would have left the panel blank on exactly the card-paid orders, which are the ones whose detail is hardest to find elsewhere. - An empty item list says so in words rather than rendering nothing. Uber itemises groceries but not restaurant orders, and orders ingested before the Uber item parser existed have none either — a blank section reads as a bug when it is usually the receipt. Read-only. This is what a provider sent; editing it would make provenance mean nothing.	2026-07-27 10:55:57 +10:00
siddharthd	df4b875b82	feat(orders): make an ingested order legible in the transactions view ... Four things the view could not tell you, all from reading the rows (user, 2026-07-27). **Which platform.** The parser has always known — it has to, to read the template — and then discarded it. "Order - Burger Corner" gives no way to know whether to open DoorDash or Uber Eats for the detail, and restaurants exist on both. Now stored on expense_metadata and named in the description: "Order - Burger Corner (Uber Eats)". Migration 0021 recovers it for the 101 backfilled rows from the order_reference shape — DoorDash receipts carry no id of their own so ingestion synthesises `msg:<message-id>`, Uber carries a real trip UUID, which makes the discriminator exact. **Bank said "Manual".** That label is derived, not stored, and "Manual" reads as "hand-entered, still awaiting a card line to match". A gift-card order has no card line coming, ever. It now reads "Gift Card", and — the part that actually mattered — credits joins cash in needsCardMatch(), so these stop sitting in the pending-reconciliation queue. All 81 were queued against a match that could not exist. **Uber line items were never parsed.** 67 of 101 orders had none. Uber itemises groceries but not restaurant orders, so some of that is genuine; the rest was simply unread. Its markup is better than DoorDash's — every cell carries a data-testid with the item's uuid, so qty/title/amount bind by id rather than by column position. Sold-out items (0.00) are kept: they are why a total is lower than what was ordered. **Uber prints pick-up and delivery addresses on every receipt** and they were thrown away. Captured as `route` [{label, time, address}], de-duplicated because the template renders the whole block twice for narrow screens. Wording is kept as printed ("Pick-up" on some receipts, "Pickup" on others) rather than normalised, so a template change stays visible. This is the same block a *trip* receipt uses for start and destination — rides are not ingested today, but the reader will not need changing when they are. Also stores source_email_subject/from, which order ingestion had left null on columns that already existed. Verified against the captured corpus: route on all 6 Uber fixtures, 5/5 items on the GLOMARK grocery receipt including the sold-out one. Production data updated by smarthome:docker/scripts/order-presentation-2026-07-27.sql (81 descriptions, `backfill` tag, re-run clean). `route` and Uber line items are parsed from here on only — recovering them for already-ingested orders means re-reading the mail, which I7 idempotency refuses by design.	2026-07-27 10:51:30 +10:00
siddharthd	ae0c34fce7	fix(orders): two defects the backfill exposed that tests could not ... Both were found by looking at the data after the live backfill, not by the suite — 105 tests were green while 85 rows were invisible and 4 were double counted. owner_id was NULL on every ingested order. Analytics scope on COALESCE(t.owner_id, s.owner_id), and an ingested order carries no statement, so the coalesce resolved to NULL and matched no owner. The rows existed in `transactions` and appeared in no view in the app. Ingestion now sets DEFAULT_OWNER_ID, and a regression test asserts the row survives the same COALESCE scoping the UI uses. [Family] orders are card-settled, not credits-funded. Their receipts name the payer ("Payments Siddharth LKR 3,783.20") and no instrument, which an earlier version read as credits. The card statement carries all four of them (CBA ...3893, exact foreign_currency_amount matches), so creating a transaction duplicated spend already recorded — the double-count I5 exists to prevent. They now record provenance only; the statement line is the transaction and is what carries the `family` tag that keeps them out of budgets. Production data corrected separately by smarthome:docker/scripts/fix-order-backfill-2026-07-27.sql. Also: reconciliation tests no longer assert global row counts. reconcilePendingOrders() scans every pending row, so leftovers from other files moved the totals — the source of an intermittent failure that only appeared on the first run after a source edit.	2026-07-27 10:41:10 +10:00
siddharthd	1103397397	fix(orders): three defects found reviewing my own branch ... None of these were caught by 105 green tests, because the code they live in was barely tested and the HTTP path was not tested at all. 1. reconcilePendingOrders hardcoded category 'dining', so any order resolved through the deferred path booked as dining regardless of merchant — a Woolworths grocery order that parks and later reconciles was misfiled. That reintroduced, through the back door, exactly the misfiling resolveCategory() exists to prevent. Now calls it. 2. reconcileCardLeg never marked a statement line as consumed, so two orders on the same card inside the +/-4 day window both bound to the same charge and each booked its own credits remainder — double-counting spend. At 10-15 orders a month on one card that is not a corner case. Migration 0020 adds matched_transaction_id with a unique index; the matcher now excludes lines already claimed. 3. The ingest API returned HTTP 200 for every parse failure, and the Slack alert fires only on non-200. So the single most likely production failure — a provider template change breaking every order at once — was completely silent. Split into NotAReceiptError (promotions, delivery updates, refund and adjustment notices: 200, silent, expected traffic) and OrderParseError (it IS a receipt and would not parse: 422, alerts). Also: order_reference now anchors on Uber's own tripReference cell rather than 'first UUID in the document'. I had claimed to verify that the first UUID was always the order UUID; that check compared against zero samples and was vacuous. tripReference is present in all 29 captured receipts and, for ue-00, equals the UUID the PDF redirect resolves to. The positional fallback remains but only flags when there is genuine ambiguity. Adds the API route's first tests — auth gate and error taxonomy — plus anchoring regressions. 63 unit + 53 integration green on five consecutive runs; corpus holds at 63/65.	2026-07-27 01:15:39 +10:00
siddharthd	a9e251d969	feat(orders): amendments, family imports, and the ingest API ... Closes the three gaps left after the parser rebuild. Refund amendments. ue-05 is a real refund: 'Previous total $49.94 / Refund -$4.21 / New Total $45.73'. Uber reuses the order UUID across the receipt and the amendment, so the two can be matched. The transaction is reduced in place rather than offset with a second row — the order is one event whose cost changed, and a compensating row would misreport both the meal count and the merchant's spend. When the original was never ingested, nothing is invented. [Family] orders now import instead of parking. Their payment line names the payer, not an instrument ('Payments Siddharth LKR 3,783.20'), so no split is recoverable and there is no card leg to reconcile against — they would have sat pending forever, which fails the actual requirement to import and tag them. Treated as credits, flagged as an assumption. Safe because the family tag removes them from every budget regardless of instrument, and the LKR amount is preserved with amount_aud left NULL rather than asserting an FX rate. Ingest API. n8n now POSTs each message to /api/orders/ingest instead of parsing in a Code node — the n8n sandbox has no require or fs, so a parser there cannot be tested against the fixture corpus, which is the one thing that makes this parser trustworthy. Auth is a shared secret, since machine callers have no Traefik session header. Rejections return 422 and record nothing. 60 unit + 45 integration green on three consecutive runs; 63/65 corpus holds.	2026-07-27 00:48:20 +10:00
siddharthd	c82a22767f	feat(orders): wire the real parser in, defer card reconciliation ... Ingestion now runs on the rebuilt parser. Three substantive changes. Deferred card reconciliation. A 'MasterCard 8032 and/or credits' receipt never states the split, but the card leg lands on the statement — Subway's $29.08 order shows $13.06 on 8032, so $16.02 was credits. For a live order that statement is weeks away, so the split cannot be settled at ingest time. Such orders are now parked with provenance and no transaction, and reconcilePendingOrders() resolves them once the statement arrives. Backfill takes the same path and resolves immediately. Migration 0019 adds the columns that make an order resumable; applied to personal_test only, prod untouched. Payment detection bug, found by the new tests: the old regex delimited the 'Paid with' line on a double space, which whitespace collapsing removes. Every card and mixed receipt fell through to the credits branch — the Woolworths receipt booked $60.93 of credits spend that never happened. Category resolution reversed deliberately. Correction 1 said never default to dining; the implementation of that sent everything unrecognised to 'other', and knowing six merchants meant Carl's Jr, Taco Bell, Chilli India, Oporto, Schnitz and Souvlaki GR all landed there. Grocers are an enumerable set and restaurants are not, so match groceries explicitly and let the residual be dining. Tests rebuilt on real captured receipts; the synthetic fixtures are deleted. 60 unit + 41 integration green on three consecutive runs.	2026-07-26 22:43:10 +10:00
siddharthd	6d3b6e1a9d	feat(orders): withdraw the ShopBack transfer guard ... Tests 12 & 13 asserted that every 'ShopBack Gift Cards' row becomes a transfer. Resolved against the ShopBack purchase emails, 3 of the 14 matching rows are Airbnb, 1 Shell, 1 Amazon — the bank descriptor's trailing token is a sequence counter, not a brand code, so the description cannot identify what was bought. Only $313.66 of $3,411.16 was ever reclassifiable. Withdrawn rather than narrowed: making it safe needs ShopBack purchase-email ingestion, brand resolution and an approval gate, to correctly handle 2 transactions in 20 months. Those two rows get handled by hand.	2026-07-26 22:09:31 +10:00
siddharthd	bbb90238e4	feat(orders): implement order ingestion pipeline, review ratings schema, and [Family] exclusion	2026-07-26 18:52:34 +10:00
siddharthd	1296555f17	test: add unit and integration test suites ... - Extract evaluateCondition + rule types into src/lib/rules.ts for testability - 48 unit tests for evaluateCondition (all fields/operators) and formatCategory - 21 integration tests for getTransactions filters and getParticipantBalances - Vitest configs for unit (vitest.config.ts) and integration (vitest.integration.config.ts) - setup-test-db.sh creates personal_test DB from production schema via pg_dump - Use vi.doMock + dynamic import pattern to isolate test DB from Prisma singleton	2026-04-01 19:59:29 +11:00