feat(loans): principal/interest split so repayments stop distorting spend
ci / lint-test (push) Successful in 41s
ci / lint-test (push) Successful in 41s
A loan repayment is not an expense. A $3,000 mortgage repayment is roughly $1,200 of principal (equity — a balance-sheet move) and $1,800 of interest (the only part that is genuinely spend). Migration 0014: - transactions.principal_amount / interest_amount, populated only when the lender itemises the split on the repayment row - statements.interest_rate, scheduled_repayment, repayment_frequency, redraw_available, loan_term_months - normalize_repayment_frequency() + trigger, so "Fortnightly", "Bi-Weekly" and "Every 2 weeks" all land on 'fortnightly' Two statement shapes are handled. Where the loan statement lists repayments and "Interest Charged" as separate rows (the common Australian case), transaction_type already does the work. Where a lender itemises the split on the repayment row, that row is typed 'payment' and would be skipped entirely — losing the interest. New SPEND_ROWS / SPEND_BASE fragments in analytics-sql.ts count such rows at interest_amount instead of amount. Adds the loan_interest category (+ colour, and the missing fees colour). Verified against the live DB with a synthetic ANZ home loan statement: a $3,000 itemised repayment plus a $10 service fee moved April spend by exactly $1,810, with the $1,200 principal excluded and still retained on the row. Test data removed and the figure confirmed back at its original value.
This commit is contained in:
@@ -152,6 +152,37 @@ Card'`) and the DB normalises it on write. The raw extracted value is preserved
|
|||||||
The TypeScript mirror is `src/lib/statement-types.ts` — keep the list, the SQL
|
The TypeScript mirror is `src/lib/statement-types.ts` — keep the list, the SQL
|
||||||
function, and the CHECK constraint in sync when adding a type.
|
function, and the CHECK constraint in sync when adding a type.
|
||||||
|
|
||||||
|
### Loans
|
||||||
|
|
||||||
|
A loan repayment is **not** an expense. It is part principal (equity, a
|
||||||
|
balance-sheet move) and part interest (the only part that is spend). Migration
|
||||||
|
0014 adds:
|
||||||
|
|
||||||
|
- `transactions.principal_amount` / `interest_amount` — populated only when the
|
||||||
|
lender itemises the split on the repayment row itself
|
||||||
|
- `statements.interest_rate`, `scheduled_repayment`, `repayment_frequency`,
|
||||||
|
`redraw_available`, `loan_term_months`
|
||||||
|
|
||||||
|
Two statement shapes, both handled:
|
||||||
|
|
||||||
|
1. **Separate rows** (the common Australian case) — the loan statement lists
|
||||||
|
repayments and "Interest Charged" separately. `transaction_type` alone is
|
||||||
|
enough: `interest` rows count as spend, `payment` rows don't. No split columns
|
||||||
|
needed.
|
||||||
|
2. **Itemised repayment row** — some lenders print principal and interest on the
|
||||||
|
repayment line. That row is typed `payment`, so it would be skipped entirely
|
||||||
|
and its interest lost. The `SPEND_ROWS` / `SPEND_BASE` fragments in
|
||||||
|
`analytics-sql.ts` handle it: a row with a non-null `interest_amount` counts
|
||||||
|
as spend, valued at `interest_amount` rather than `amount`.
|
||||||
|
|
||||||
|
The N8N `Parse Gemini Result` node only accepts a split when both parts are
|
||||||
|
present *and* they sum to the row amount (±2c) — a half-extracted split would
|
||||||
|
silently misreport spend, so it is discarded rather than trusted.
|
||||||
|
|
||||||
|
Loan interest uses the `loan_interest` category; principal repayments use
|
||||||
|
`investment` (excluded from spend, surfaced on the investments line in monthly
|
||||||
|
analytics).
|
||||||
|
|
||||||
### Prisma
|
### Prisma
|
||||||
|
|
||||||
The schema at `prisma/schema.prisma` covers all tables. The generated client (gitignored) must be regenerated after schema changes:
|
The schema at `prisma/schema.prisma` covers all tables. The generated client (gitignored) must be regenerated after schema changes:
|
||||||
|
|||||||
@@ -0,0 +1,85 @@
|
|||||||
|
-- Loan statement support.
|
||||||
|
--
|
||||||
|
-- Migration 0013 taught the system that a statement can be a loan; this gives
|
||||||
|
-- loans somewhere to put the data that only loans have.
|
||||||
|
--
|
||||||
|
-- The core accounting problem: a loan repayment is not an expense. A $3,000
|
||||||
|
-- mortgage repayment is roughly $1,200 of principal (a balance-sheet move that
|
||||||
|
-- builds equity) and $1,800 of interest (the only part that is genuinely spend).
|
||||||
|
-- Counting the whole repayment as spending overstates expenses badly.
|
||||||
|
--
|
||||||
|
-- Two statement shapes are handled:
|
||||||
|
-- (a) The common Australian case — the loan statement lists repayments and
|
||||||
|
-- "Interest Charged" as separate rows. transaction_type already carries
|
||||||
|
-- this: 'interest' rows count as spend, 'payment' rows do not.
|
||||||
|
-- (b) Some lenders itemise principal and interest on the repayment row itself.
|
||||||
|
-- That's what principal_amount / interest_amount are for: when
|
||||||
|
-- interest_amount is set, analytics count that instead of the full amount.
|
||||||
|
--
|
||||||
|
-- Idempotent: safe to re-run.
|
||||||
|
|
||||||
|
-- Per-transaction principal/interest split (shape (b) above).
|
||||||
|
ALTER TABLE transactions
|
||||||
|
ADD COLUMN IF NOT EXISTS principal_amount NUMERIC(12,2),
|
||||||
|
ADD COLUMN IF NOT EXISTS interest_amount NUMERIC(12,2);
|
||||||
|
|
||||||
|
COMMENT ON COLUMN transactions.principal_amount IS
|
||||||
|
'Principal portion of a loan repayment, when the statement itemises it. Not spend.';
|
||||||
|
COMMENT ON COLUMN transactions.interest_amount IS
|
||||||
|
'Interest portion of a loan repayment, when the statement itemises it. This is the part that counts as spend.';
|
||||||
|
|
||||||
|
-- Partial index: only loan repayment rows carry a split.
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_transactions_interest_amount
|
||||||
|
ON transactions (interest_amount)
|
||||||
|
WHERE interest_amount IS NOT NULL;
|
||||||
|
|
||||||
|
-- Loan-level terms, read off the statement header.
|
||||||
|
ALTER TABLE statements
|
||||||
|
ADD COLUMN IF NOT EXISTS interest_rate NUMERIC(6,3),
|
||||||
|
ADD COLUMN IF NOT EXISTS scheduled_repayment NUMERIC(12,2),
|
||||||
|
ADD COLUMN IF NOT EXISTS repayment_frequency TEXT,
|
||||||
|
ADD COLUMN IF NOT EXISTS redraw_available NUMERIC(12,2),
|
||||||
|
ADD COLUMN IF NOT EXISTS loan_term_months INTEGER;
|
||||||
|
|
||||||
|
COMMENT ON COLUMN statements.interest_rate IS 'Annual interest rate as a percentage, e.g. 6.140';
|
||||||
|
COMMENT ON COLUMN statements.redraw_available IS 'Funds available to redraw (loans) — not the same as available_credit on a card.';
|
||||||
|
|
||||||
|
-- Free text varies by lender ("Monthly", "Fortnightly"); normalise the common
|
||||||
|
-- spellings rather than constraining, so an unexpected value never blocks an import.
|
||||||
|
ALTER TABLE statements DROP CONSTRAINT IF EXISTS statements_repayment_frequency_check;
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION normalize_repayment_frequency(raw TEXT)
|
||||||
|
RETURNS TEXT AS $$
|
||||||
|
DECLARE
|
||||||
|
v TEXT := lower(trim(coalesce(raw, '')));
|
||||||
|
BEGIN
|
||||||
|
IF v = '' THEN RETURN NULL; END IF;
|
||||||
|
-- Check fortnightly spellings before the bare '%week%' match below.
|
||||||
|
IF v LIKE '%fortnight%' OR v LIKE '%bi-week%' OR v LIKE '%biweek%'
|
||||||
|
OR v LIKE '%2 week%' OR v LIKE '%two week%' OR v LIKE '%14 day%'
|
||||||
|
THEN RETURN 'fortnightly'; END IF;
|
||||||
|
IF v LIKE '%month%' THEN RETURN 'monthly'; END IF;
|
||||||
|
IF v LIKE '%week%' THEN RETURN 'weekly'; END IF;
|
||||||
|
IF v LIKE '%quarter%' THEN RETURN 'quarterly'; END IF;
|
||||||
|
IF v LIKE '%annual%' OR v LIKE '%year%' THEN RETURN 'annually'; END IF;
|
||||||
|
RETURN v;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql IMMUTABLE;
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION statements_normalize_loan_fields_trigger()
|
||||||
|
RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
NEW.repayment_frequency := normalize_repayment_frequency(NEW.repayment_frequency);
|
||||||
|
RETURN NEW;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
DROP TRIGGER IF EXISTS trg_statements_normalize_loan_fields ON statements;
|
||||||
|
CREATE TRIGGER trg_statements_normalize_loan_fields
|
||||||
|
BEFORE INSERT OR UPDATE OF repayment_frequency ON statements
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION statements_normalize_loan_fields_trigger();
|
||||||
|
|
||||||
|
UPDATE statements
|
||||||
|
SET repayment_frequency = normalize_repayment_frequency(repayment_frequency)
|
||||||
|
WHERE repayment_frequency IS NOT NULL
|
||||||
|
AND repayment_frequency IS DISTINCT FROM normalize_repayment_frequency(repayment_frequency);
|
||||||
@@ -151,6 +151,11 @@ model statements {
|
|||||||
account_holder_name String?
|
account_holder_name String?
|
||||||
exchange_rate_to_aud Decimal? @db.Decimal(10, 6)
|
exchange_rate_to_aud Decimal? @db.Decimal(10, 6)
|
||||||
paperless_doc_id Int? @unique
|
paperless_doc_id Int? @unique
|
||||||
|
interest_rate Decimal? @db.Decimal(6, 3)
|
||||||
|
scheduled_repayment Decimal? @db.Decimal(12, 2)
|
||||||
|
repayment_frequency String?
|
||||||
|
redraw_available Decimal? @db.Decimal(12, 2)
|
||||||
|
loan_term_months Int?
|
||||||
transactions transactions[]
|
transactions transactions[]
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -172,6 +177,8 @@ model transactions {
|
|||||||
amount_aud Decimal? @db.Decimal(12, 2)
|
amount_aud Decimal? @db.Decimal(12, 2)
|
||||||
owner_id Int?
|
owner_id Int?
|
||||||
reconciled_with_id Int?
|
reconciled_with_id Int?
|
||||||
|
principal_amount Decimal? @db.Decimal(12, 2)
|
||||||
|
interest_amount Decimal? @db.Decimal(12, 2)
|
||||||
statement statements? @relation(fields: [statement_id], references: [id], onDelete: Cascade)
|
statement statements? @relation(fields: [statement_id], references: [id], onDelete: Cascade)
|
||||||
reconciled_with transactions? @relation("reconciled", fields: [reconciled_with_id], references: [id], onDelete: SetNull)
|
reconciled_with transactions? @relation("reconciled", fields: [reconciled_with_id], references: [id], onDelete: SetNull)
|
||||||
reconciled_by transactions[] @relation("reconciled")
|
reconciled_by transactions[] @relation("reconciled")
|
||||||
|
|||||||
@@ -6,6 +6,8 @@ import {
|
|||||||
STATEMENTS_JOIN,
|
STATEMENTS_JOIN,
|
||||||
EFFECTIVE_CATEGORY,
|
EFFECTIVE_CATEGORY,
|
||||||
EXCLUDE_NON_SPEND,
|
EXCLUDE_NON_SPEND,
|
||||||
|
SPEND_ROWS,
|
||||||
|
SPEND_BASE,
|
||||||
} from "@/lib/analytics-sql";
|
} from "@/lib/analytics-sql";
|
||||||
|
|
||||||
export async function GET(req: NextRequest) {
|
export async function GET(req: NextRequest) {
|
||||||
@@ -34,9 +36,9 @@ export async function GET(req: NextRequest) {
|
|||||||
${EFFECTIVE_CATEGORY} as category,
|
${EFFECTIVE_CATEGORY} as category,
|
||||||
SUM(
|
SUM(
|
||||||
CASE
|
CASE
|
||||||
WHEN ts.share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * ts.share_percent / 100
|
WHEN ts.share_percent IS NOT NULL THEN (${SPEND_BASE}) * ts.share_percent / 100
|
||||||
WHEN o.my_share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * o.my_share_percent / 100
|
WHEN o.my_share_percent IS NOT NULL THEN (${SPEND_BASE}) * o.my_share_percent / 100
|
||||||
ELSE COALESCE(t.amount_aud, t.amount)
|
ELSE (${SPEND_BASE})
|
||||||
END
|
END
|
||||||
)::numeric(12,2) as total_spent,
|
)::numeric(12,2) as total_spent,
|
||||||
COUNT(*)::int as transaction_count
|
COUNT(*)::int as transaction_count
|
||||||
@@ -45,7 +47,7 @@ export async function GET(req: NextRequest) {
|
|||||||
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
||||||
${STATEMENTS_JOIN}
|
${STATEMENTS_JOIN}
|
||||||
WHERE ${OWNER_SCOPE} = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('debit', 'fee', 'interest')
|
AND ${SPEND_ROWS}
|
||||||
AND ${EXCLUDE_NON_SPEND}
|
AND ${EXCLUDE_NON_SPEND}
|
||||||
AND t.transaction_date >= $2
|
AND t.transaction_date >= $2
|
||||||
AND t.transaction_date < $3
|
AND t.transaction_date < $3
|
||||||
|
|||||||
@@ -180,6 +180,20 @@ export default function StatementsPage() {
|
|||||||
{s.card_name && (
|
{s.card_name && (
|
||||||
<div className="text-xs text-zinc-500 truncate max-w-[160px]">{s.card_name}</div>
|
<div className="text-xs text-zinc-500 truncate max-w-[160px]">{s.card_name}</div>
|
||||||
)}
|
)}
|
||||||
|
{stmtType === "loan" && (s.interest_rate || s.scheduled_repayment) && (
|
||||||
|
<div className="text-xs text-zinc-500 truncate max-w-[160px]">
|
||||||
|
{[
|
||||||
|
s.interest_rate ? `${Number(s.interest_rate).toFixed(2)}% p.a.` : null,
|
||||||
|
s.scheduled_repayment
|
||||||
|
? `${formatAmount(s.scheduled_repayment)}${
|
||||||
|
s.repayment_frequency ? ` ${s.repayment_frequency}` : ""
|
||||||
|
}`
|
||||||
|
: null,
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(" · ")}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
<Link
|
<Link
|
||||||
href={`/transactions?statement_id=${s.id}`}
|
href={`/transactions?statement_id=${s.id}`}
|
||||||
className="sm:hidden text-xs text-indigo-400 hover:text-indigo-300 mt-1 inline-block"
|
className="sm:hidden text-xs text-indigo-400 hover:text-indigo-300 mt-1 inline-block"
|
||||||
|
|||||||
@@ -21,6 +21,24 @@ export const STATEMENTS_JOIN = `LEFT JOIN statements s ON s.id = t.statement_id`
|
|||||||
/** Transaction types that represent money going out. */
|
/** Transaction types that represent money going out. */
|
||||||
export const SPEND_TYPES = `('debit', 'fee', 'interest')`;
|
export const SPEND_TYPES = `('debit', 'fee', 'interest')`;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Rows that count towards spend.
|
||||||
|
*
|
||||||
|
* The `interest_amount IS NOT NULL` arm is for loan repayments: when a lender
|
||||||
|
* itemises principal and interest on a single repayment row, that row is usually
|
||||||
|
* typed 'payment' (money reducing the loan balance) and would otherwise be
|
||||||
|
* skipped — but its interest portion is real spend.
|
||||||
|
*/
|
||||||
|
export const SPEND_ROWS = `(t.transaction_type IN ('debit', 'fee', 'interest') OR t.interest_amount IS NOT NULL)`;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The amount of a row that counts as spend, before split adjustment.
|
||||||
|
*
|
||||||
|
* For an itemised loan repayment only the interest portion is an expense; the
|
||||||
|
* principal builds equity and is a balance-sheet move, not spending.
|
||||||
|
*/
|
||||||
|
export const SPEND_BASE = `CASE WHEN t.interest_amount IS NOT NULL THEN t.interest_amount ELSE COALESCE(t.amount_aud, t.amount) END`;
|
||||||
|
|
||||||
/** Effective category, honouring overrides. Never NULL. */
|
/** Effective category, honouring overrides. Never NULL. */
|
||||||
export const EFFECTIVE_CATEGORY = `COALESCE(o.category_override, t.category, 'other')`;
|
export const EFFECTIVE_CATEGORY = `COALESCE(o.category_override, t.category, 'other')`;
|
||||||
|
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ export const CATEGORIES = [
|
|||||||
"transfers",
|
"transfers",
|
||||||
"income",
|
"income",
|
||||||
"investment",
|
"investment",
|
||||||
|
"loan_interest",
|
||||||
"personal_care",
|
"personal_care",
|
||||||
"pets",
|
"pets",
|
||||||
"gifts",
|
"gifts",
|
||||||
|
|||||||
@@ -17,6 +17,8 @@ export const CATEGORY_COLORS: Record<string, string> = {
|
|||||||
transfers: "#6b7280",
|
transfers: "#6b7280",
|
||||||
income: "#34d399",
|
income: "#34d399",
|
||||||
investment: "#818cf8",
|
investment: "#818cf8",
|
||||||
|
loan_interest: "#9f1239",
|
||||||
|
fees: "#f87171",
|
||||||
personal_care: "#fb7185",
|
personal_care: "#fb7185",
|
||||||
pets: "#86efac",
|
pets: "#86efac",
|
||||||
gifts: "#fcd34d",
|
gifts: "#fcd34d",
|
||||||
|
|||||||
@@ -22,6 +22,9 @@ export interface TransactionRow {
|
|||||||
category: string;
|
category: string;
|
||||||
row_index: number;
|
row_index: number;
|
||||||
created_at: string;
|
created_at: string;
|
||||||
|
// loan repayment split — set only when the lender itemises it (migration 0014)
|
||||||
|
principal_amount: number | null;
|
||||||
|
interest_amount: number | null;
|
||||||
// override fields
|
// override fields
|
||||||
category_override: string | null;
|
category_override: string | null;
|
||||||
merchant_override: string | null;
|
merchant_override: string | null;
|
||||||
@@ -64,6 +67,12 @@ export interface StatementRow {
|
|||||||
credit_limit: number | null;
|
credit_limit: number | null;
|
||||||
currency: string;
|
currency: string;
|
||||||
statement_type: string | null;
|
statement_type: string | null;
|
||||||
|
// Loan statements only (see migration 0014)
|
||||||
|
interest_rate: number | null;
|
||||||
|
scheduled_repayment: number | null;
|
||||||
|
repayment_frequency: string | null;
|
||||||
|
redraw_available: number | null;
|
||||||
|
loan_term_months: number | null;
|
||||||
tier_used: string | null;
|
tier_used: string | null;
|
||||||
owner_id: number;
|
owner_id: number;
|
||||||
owner_name: string;
|
owner_name: string;
|
||||||
|
|||||||
Reference in New Issue
Block a user