diff --git a/docs/shared-expenses-design.md b/docs/shared-expenses-design.md index 2fde17e..f90b672 100644 --- a/docs/shared-expenses-design.md +++ b/docs/shared-expenses-design.md @@ -1,57 +1,151 @@ -# Shared expenses, settlement, and the shared loan — design proposal +# Shared expenses and settlement -Status: **proposal, nothing built**. Written 2026-07-26 for review. +Status: **built and live**, as of 2026-07-28. The loan section at the end is +still a proposal — nothing there is built. -## Why this exists - -Three questions have no answer in the current model: - -1. Which splits does a settlement payment settle? -2. Is the Europe trip settled, separately from the ongoing household tab? -3. Whose expense is a $2,500 loan repayment when Sonu funds part of it? - -They look like three problems. They are one: **the app records money moving, and -separately records who owes whom, and the two never meet.** +This replaces the 2026-07-26 proposal. That document described three problems +and proposed a `settlement_contexts` table to solve them. The problems were +real; the table was not built, and the reasoning for not building it is +recorded under [What was rejected](#what-was-rejected). --- -## What is actually broken +## The one rule -### Two settlement models, neither finished +**Spend and owed are different questions asked of the same table, and the line +between them is `transaction_splits.settled`.** -| Model | Where | State | +| | Counts settled splits? | Why | |---|---|---| -| Running tab | `split_payments` (from, to, amount, date) | **in use** — 8 payments, $37,881.10 | -| Per-split flag | `transaction_splits.settled` | **never used** — all 673 splits are `false` | +| **Spend** (`myShare`, `mySplitOf`) | **Yes** | Half a 2025 grocery shop was my expense whether or not the other half was ever repaid. | +| **Owed** (balances, trip figures) | **No** | A discharged obligation is not outstanding. | -They are honoured inconsistently: +Getting this backwards in either direction is the failure the model exists to +prevent. Filtering settled rows out of spend would re-inflate exactly the +figures that importing settled history exists to correct. -- `getParticipantBalances` (the shared page) ignores `settled` entirely -- `/api/participants/[id]/balance` filters on `settled = false` -- `getTripAnalytics` reports settled/unsettled **from the unused flag** +The predicate is `ACTIVE_OBLIGATION` in `src/lib/analytics-sql.ts`. -The third is a live bug. Every trip shows 100% unsettled forever, even though -Molina has paid $20,782.79 against $19,556.07 of splits and is square. +## Two orthogonal axes -### Settlements exist twice, unlinked +`settled` and `trip_id` answer different questions and neither implies the +other: -Four of the eight recorded payments match an offset-account credit exactly: +- **`transaction_splits.settled`** — *is this obligation still live?* +- **`split_payments.trip_id`** — *which tab does this payment settle?* + NULL means the ongoing household tab. -| Payment date | From | Amount | Offset transaction | -|---|---|---:|---| -| 2026-02-02 | Molina | 7,500.00 | `Transfer from - MEGHALEE BOSE mummy Pa…` | -| 2026-04-12 | Sonu | 3,779.33 | `Transfer from - MEGHALEE BOSE transfer` | -| 2026-04-21 | Molina | 1,685.24 | `Transfer from - MEGHALEE BOSE mummy split` | -| 2026-05-16 | Sonu | 4,794.06 | `Transfer from - MEGHALEE BOSE transfer` | +A trip can be fully paid while the household tab runs a balance, and vice +versa. Before migration 0022 there was one global pool and this could not be +expressed, so every trip reported 100% unsettled forever — including trips paid +in full. -The same money is a `split_payments` row *and* a `transactions` row. -`split_payments.linked_transaction_id` exists but only 1 of 8 rows uses it. So a -settlement is bookkeeping that happens to resemble a bank credit, rather than -being that credit. +## How settling up actually works -### The shared loan is invisible +**By recording a payment.** There is deliberately no "mark settled" action +anywhere in the app. -Sonu's contributions are already in the ledger and unrecognised: +`settled` marks obligations discharged *outside* this app — the imported +SplitMyExpenses history, whose repayments happened on a platform we no longer +run and which therefore have no `split_payments` row here. A live obligation is +settled by recording the payment, and the balance nets to zero on its own. + +Doing both would subtract the settlement twice: the splits leave the sum *and* +the payment is deducted, driving the balance negative by the amount repaid. + +## What is built + +| Piece | Where | Note | +|---|---|---| +| `settled` as the single balance gate | `ACTIVE_OBLIGATION` | Applied in both arms of the balances UNION and in the trip owed query | +| Payment scope | `split_payments.trip_id` (migration 0022) | Household payments do not settle a trip, and vice versa | +| Owner-scoped owed | `OWNER_SCOPE` in the trip owed query | Without it, a debt between the *other two* participants was reported as owed to the owner — $1,605.49 on Europe 2026 | +| Direction on screen | `/trips/[id]`, `/shared` | all square / owes you / ahead — you owe them | +| Historical splits | `scripts/split_csv_match.py` | 1,242 rows across 657 transactions, all `settled` | +| Duplicate suppression | `transactions.superseded_by_id` (migration 0023) | 31 rows, $42,040.68 | +| Overlap detection | `STATEMENT_OVERLAPS` → statements page | Red badge; catches the cause rather than the symptom | + +## The historical import + +The five SplitMyExpenses CSVs are the record of how expenses were shared before +this app existed. 676 of 1,536 shareable rows matched (44%), and 1,242 split +rows were written as `settled = true`. + +**The deliverable is historical spend, not balances.** $35,259 left my spend — +$13,088 in 2024 and $22,117 in 2025 — because a $200 grocery shop that was +always half hers no longer reads as $200 of mine. Balances were byte-identical +before and after, which is the assertion that mattered. + +Three things the matcher has to get right, each of which has bitten: + +1. **Date format is decided per file.** The household export writes D/M/YYYY and + the four trip exports write ISO; 474 rows parse validly under both readings. + Guessing per row silently swaps January and February for some rows and not + others. +2. **A person's column is net balance impact, not their share.** The payer is + whoever is positive; the other's share is `|their negative| / cost`. So a + `+cost / -cost` row means the other party owes **100%** — not that the + expense was unshared, which is the reading that fakes an arrangement change. +3. **Matching is one-to-one, best pair first.** The NZ trip has two identical + $10.16 Uber rows against three ledger rows; without this a ledger row is + claimed repeatedly while the second CSV row looks matched and is not. + +**The 44% is a coverage ceiling, not a matcher weakness.** The CSVs describe 678 +shared expenses in 2024; the ledger holds 591 rows for all of 2024, 3 to 72 a +month, far less than a household actually spends. South Korea April 2024 matches +4 of 158. Chasing a higher rate is chasing transactions that were never +imported. + +### A reversed recommendation + +The 2026-07-26 proposal said, under *What I would not do*: "**Do not** restate +history from the SplitMyExpenses CSVs… the value is low: those balances are +settled and will not change." + +That was overturned on 2026-07-28, and it was wrong in an instructive way: it +measured the value in *balances*, where it is indeed nil, and missed the value +in *spend*, where it is $35,259. Importing as `settled` gets the second without +touching the first. The "combining problem" it cited is real and is why the +match rate is capped — but a partial restatement of spend beats none, and rows +that cannot be matched simply keep their current treatment. + +## What was rejected + +**`settlement_contexts` as a table.** The need was real — a payment must say +what it settles. But trips already exist and already carry membership on +`transaction_overrides.trip_id`, so scope is a read of existing data rather +than a new grouping key. One nullable column on `split_payments` expressed it. + +A general context table would have meant a new entity to create and maintain +before a payment could be recorded, in a two-person household with two trips. + +**Deleting duplicate transactions.** Every child of `transactions` is +`ON DELETE CASCADE`, and which member of a duplicate pair holds the curation is +an accident of import order. Duplicates are superseded instead: the row stays, +keeps its children, and points at the row that replaces it. + +**Reusing `reconciled_with_id` for duplicates.** Its predicate is scoped to +`statement_id IS NULL` on purpose — a statement line pointing at something else +is the survivor, not the duplicate. In the duplicate-import case both rows are +statement lines, so that predicate can never hide either. + +## Scale note + +This is a home app for one user, occasionally two, and the second user consumes +the splits view and little else. Reviews of this subsystem have repeatedly +proposed enterprise-grade reconciliation, lineage and audit machinery; the +*findings* are often right and the *sizing* is not. A one-column solution a +person can hold in their head beats a correct-but-unmaintainable one here. + +--- + +## Still a proposal: the shared loan + +**Nothing in this section is built.** + +Sonu's contributions are in the ledger and unrecognised. All are categorised +`transfers` — correct for spend, but it makes a loan contribution and an expense +settlement indistinguishable: | Pattern in offset credits | Rows | Total | Meaning | |---|---:|---:|---| @@ -59,162 +153,15 @@ Sonu's contributions are already in the ledger and unrecognised: | `…mummy…` | 6 | $29,721.24 | Molina's money, forwarded by Sonu | | other Meghalee | 15 | $71,130.27 | Sonu's own settlements | -All are categorised `transfers` — correct for spend purposes, but it means a -loan contribution and an expense settlement are indistinguishable. - -Meanwhile the loan itself, over the 12 imported months: - -| | | -|---|---:| -| Principal repaid (`investment`, excluded from spend) | $63,500.00 | -| Interest charged (`loan_interest`, the only part counted as spend) | $16,523.64 | - -At roughly $25,000/year of `emi` against ~$80,000 of annual repayments, Sonu -funds about **31%** — of both the equity being built and the interest being paid. -Today 100% of the interest counts as your spend and 100% of the equity as yours. - ---- - -## The core problem - -The model conflates two different things: - -- **Money movement** — a credit landed in the offset account -- **Obligation** — someone owed someone else, and now owes less - -A settlement is both. A loan contribution is both. Right now movement lives in -`transactions` and obligation lives in `transaction_splits` / `split_payments`, -with nothing joining them. That is why a payment cannot say what it settles: it -was never attached to anything in the first place. - ---- - -## Proposed model - -### 1. Settlement contexts - -Splits belong to something that is settled **as a unit**. Payments name which -unit they settle. Balance is computed per context, not globally. - -| Context | Splits | Settled by | State | -|---|---|---|---| -| Household (default) | ongoing | periodic payments | running tab | -| Europe 2026 | trip-bound | lump sum | closeable | -| Pre-2026 (SplitMyExpenses) | historical | settled elsewhere | **born closed** | - -A closed context still contributes to analytics — you see your true share — but -contributes nothing to what anyone owes. - -This answers all three opening questions, and it dissolves the `splitFrom` date -cutoff: pre-2026 splits can be applied retroactively **because they are born -into a closed context**, so they fix the analytics without creating debt. No date -guard needed, no risk of resurrecting settled obligations. - -Mechanically: `settlement_contexts` table; `transaction_splits.context_id`; -`split_payments.context_id`. `transaction_splits.settled` becomes derived -("is my context closed?") or is dropped. - -### 2. Payments are transactions, not a side table - -A settlement is the offset-account credit. `split_payments` becomes a thin -attribution layer over a real transaction rather than a parallel record of it: - -- Populate `linked_transaction_id` on all existing payments where a match exists -- On ingestion, an incoming credit that looks like a settlement is *proposed* as - one for confirmation, rather than silently becoming `transfers` -- A payment with no matching transaction (cash, or an account not imported) - stays as a manual row — the model must tolerate that - -### 3. The shared loan — a separate ledger - -Not a settlement context. The loan is a jointly funded asset with its own -obligation, and mixing it with expense settlement would let a contribution -accidentally settle a dinner. - -- `emi` credits are recognised as **contributions**, not generic transfers -- A **contribution schedule** states what is owed per period (50% of the - repayment), independent of what was actually paid -- The running difference is a **receivable** — currently $4,000.00 - -The schedule matters: during Sonu's leave the obligation did not change, only the -payment did. A percentage-of-actual model would silently redefine her share as -30% and make the shortfall disappear. - ---- - -## Migration path - -### Phase 0 — DONE (2026-07-26, commit `3f04cbd`) - -Stop the trip view reporting a settlement breakdown it cannot compute. - -The original plan was "make `getTripAnalytics` and `getParticipantBalances` -agree". **That plan was wrong** and reading the code before building is what -caught it: - -- `getParticipantBalances` is *not* buggy. It computes `splits − payments`, - which is coherent. Excluding settled splits there while still subtracting the - payments that settled them would have double-counted and broken a working page. -- The real defect was narrower: the trip view showed Settled/Unsettled from - `transaction_splits.settled`, which nothing sets. A correct per-trip figure is - not computable at all, because `split_payments` has no trip attribution. - -So the fix was **subtractive**: the trip view now shows each participant's share -and points at Shared for what is actually owed. - -Also removed `/api/splits/settle` — unreachable from the UI but live on its URL, -where one call with `participant_id` would mark every one of that person's splits -settled, writing a flag nothing reads. - -`transaction_splits.settled` / `settled_at` still exist and are now pure dead -data. Phase 1 either repurposes them ("is my context closed?") or drops them. - -### Phase 1 — settlement contexts - -Add contexts; put every existing split in "Household"; every payment likewise. -Balance queries group by context. Touches `queries.ts` (both balance CTEs), -`shared/page.tsx`, `trips/[id]/page.tsx`, `split-payments/route.ts`. ~1 day. - -### Phase 2 — link payments to transactions - -Backfill `linked_transaction_id` for the four exact matches; flag the other four -for manual linking. On ingestion, propose a matching credit as a settlement -rather than silently categorising it `transfers`. ~half a day. - -### Phase 3 — retroactive pre-2026 split - -Create the "Pre-2026" closed context. Apply household split rules into it via the -rule preview (`/api/rules/[id]/matches`, built 2026-07-26) — fixes ~$97,627 of -the trailing 12 months currently shown as 100% yours. Then delete the `splitFrom` -cutoff entirely. - -**Validate the ratio first.** This assumes today's 50/50 held through 2025. The -SplitMyExpenses CSVs should be used to *check* that assumption — not to -reconcile, since transactions were sometimes combined and exact matching is -impossible. - -### Phase 4 — loan ledger - -Contribution schedule, contributions recognised from `emi` credits, running -receivable. Independent of contexts — the loan is a separate ledger. ~1–2 days. - ---- - -## Decisions taken (2026-07-26) - ### The loan is separate from shared expenses Different obligations, different rhythms, different nature: one funds an asset, -the other funds consumption. They do not share a settlement context and a -contribution is never a settlement. +the other funds consumption. They do not share a settlement scope, and a +contribution must never settle a dinner. -### The share is 50/50, fixed — with the shortfall tracked +### The share is 50/50 fixed, with the shortfall tracked -Not derived from actual payments, which fluctuate. Sonu's obligation is half the -repayment; the difference between obligation and actual is a **receivable**, and -it is the interesting number. - -Over 2025-07-01 → 2026-06-30: +Not derived from actual payments, which fluctuate. Over 2025-07-01 → 2026-06-30: | | | |---|---:| @@ -223,65 +170,44 @@ Over 2025-07-01 → 2026-06-30: | Actually contributed (26 payments) | $27,750.00 | | **Shortfall** | **$4,000.00** | -She never missed a fortnight; the rate changed: - -| Rate | Payments | Period | -|---|---:|---| -| $1,250 | 15 | Aug 2025 – Feb 2026 (the correct 50%) | -| $1,000 | 3 | Jul 2025 (pre-adjustment) | -| $750 | 8 | Mar – Jun 2026 (leave) | +She never missed a fortnight; the rate changed — $1,250 × 15 (Aug 2025–Feb +2026, the correct 50%), $1,000 × 3 (Jul 2025, pre-adjustment), $750 × 8 +(Mar–Jun 2026, leave). So the model needs a **contribution schedule** (expected per period) alongside -actual contributions, with the running difference as a tracked balance. A flat -percentage cannot express "obligation unchanged, payment temporarily reduced, -difference owed". +actual contributions, with the running difference as a tracked receivable. A +flat percentage-of-actual cannot express "obligation unchanged, payment +temporarily reduced, difference owed" — it would silently redefine her share as +30% and make the shortfall disappear. ### Interest: recommended as expense, pending final call -The mechanics are as described — interest is debited to the loan and repayments -pay down the combined balance. Reconciles exactly: +Over 12 months $63,500 of cash left and debt fell by $44,127.36. The $16,523.64 +difference bought nothing and is not recoverable — an expense by definition. +Excluding it leaves the balance sheet unable to reconcile cash out against +equity gained, and understates annual cost by ~10%. - 134: 31,000.00 − 8,553.27 = 22,446.73 = balance reduction - 133: 32,500.00 − 7,970.37 − 2,849.00 = 21,680.63 = balance reduction +The legitimate concern is that interest is non-discretionary. The answer is a +fixed-commitments grouping alongside rent, insurance and utilities — a +presentation change, not an exclusion. -But mechanics are not the same as economics. Over 12 months $63,500 of cash left -and debt fell by $44,127.36. The $16,523.64 difference bought nothing and is not -recoverable — that is an expense by definition. Excluding it leaves the balance -sheet unable to reconcile cash out against equity gained, and understates annual -cost by ~10%. - -The legitimate concern is that interest is **non-discretionary**. The answer to -that is a fixed-commitments grouping alongside rent, insurance and utilities — -a presentation change, not an exclusion. - -**Recommendation: keep `loan_interest` as spend, add a fixed/discretionary -split.** Flagged rather than settled: it is a judgement about what "spend" means -in your own reporting. +**Do not** model the loan as a recurring split: that would put $2,500 a +fortnight of principal into spend, the error migration 0014 exists to prevent. ## Open questions 1. **Does equity need tracking per person?** If Sonu accrues a share of the principal, that is a balance-sheet item the app has no concept of. Probably - belongs in the net-worth view rather than here. - -4. **Attribution of forwarded payments.** `mummy` in the description reliably + belongs in a net-worth view rather than here. +2. **Attribution of forwarded payments.** `mummy` in the description reliably marks Molina's money in all six known cases, but it is a description match on - a free-text field. Acceptable as a *suggestion* requiring confirmation; not as - an automatic rule. - -5. **Retroactive split ratios.** Applying today's household rules to 2025 - assumes the arrangement has not changed. The SplitMyExpenses CSVs could give - real historical shares, but transactions were sometimes combined, so matching - is imperfect. Recommendation: use today's ratios, accept the approximation — - the goal is a truer analytics picture, not a restated ledger. - ---- - -## What I would not do - -- **Do not** restate history from the SplitMyExpenses CSVs. The combining problem - makes exact reconciliation impossible, and the value is low: those balances are - settled and will not change. -- **Do not** make the loan a shared *expense*. It is a funded asset. Modelling it - as a recurring split would put $2,500 a fortnight of principal into spend, - which is the error migration 0014 was written to prevent. + a free-text field. Acceptable as a *suggestion* requiring confirmation, not + as an automatic rule. +3. **Untracked travel, Apr–May 2026.** France ($7,020.99, 68 rows) and Singapore + ($2,242.05, 40 rows) between 13 April and 9 May 2026 belong to no trip. The + France rows begin the day after Europe 2026's end date, so this is either an + extension of that trip or one or two new ones. +4. **Two unattributable Sonu payments.** #3 ($3,779.33, 12 Apr) and #8 + ($4,794.06, 16 May) are narrated only as "transfer". Both currently sit on + the household tab; scoping them elsewhere moves money between tabs without + changing her overall balance.