fix(analytics): make the displayed numbers mean what they say
ci / lint-test (push) Failing after 44s

Six metric-integrity defects from the UI/IA review, plus two found while
verifying the review's own claims against the code.

The reconciled-row exclusion existed only in queries.ts. Every analytics
route counted the superseded manual rows as spend — 48 rows, $4,474.79 of
double count, invisible precisely because the transaction list looked
right. It is now one fragment both sides import.

The spend-pace chart computed its own totals in the browser: gross
amounts, debits only, no personal share, no refunds, fees, interest or
itemised loan repayments. On live data it ended July at $4,747.31 under a
headline reading $3,597.10 — and its own baseline line was drawn from the
split-adjusted monthly totals, so the two series in one chart disagreed
with each other. Both now come from /api/analytics/daily, built from the
same fragments as the headline.

Fees aggregated every statement ever imported with no date filter, under
a heading with no period, so a lifetime figure read as a current one and
grew forever. Now bounded, labelled, and selectable.

Comparisons no longer measure a month in progress against complete ones:
the in-progress month is out of every baseline, and a selected current
month is compared through the same day.

Two the review did not catch:

- Every analytics window was a day early. toISOString() on a
  local-midnight Date converts backwards through UTC. Surfaced only once
  fees started reporting the range it had used.
- /monthly rounded per category, /daily per category-day, so the pace
  chart ended a few cents off the headline above it.

Shared currency needed amending rather than applying. Reading s.currency
would have labelled every order row AUD, since an order receipt has no
statement and carries its own currency — the opposite convention from a
foreign charge on an AUD statement, where amount IS AUD. NATIVE_CURRENCY's
COALESCE order keeps the two apart. Balances also now count rows whose AUD
value is genuinely unknown instead of netting a foreign figure against AUD
ones. Latent today: no foreign transaction is currently split.

Tag-filtered balance cards no longer claim "owes you". With a filter on,
payments are deliberately not subtracted, so the figure is a split total
and settling against it would record a payment for a debt that never was.

Split-coverage warnings deliberately omitted (user decision).
This commit is contained in:
2026-07-27 17:10:27 +10:00
parent 5ee5ee24cf
commit a4ab543a6c
14 changed files with 643 additions and 90 deletions
+87
View File
@@ -0,0 +1,87 @@
import { NextRequest, NextResponse } from "next/server";
import { getCurrentUser } from "@/lib/auth";
import { queryRaw } from "@/lib/db";
import {
OWNER_SCOPE,
STATEMENTS_JOIN,
EXCLUDE_NON_SPEND,
EXCLUDE_RECONCILED_SOURCE,
EFFECTIVE_CATEGORY,
NET_SPEND_ROWS,
SPEND_SIGNED,
mySplitOf,
toDateStr,
} from "@/lib/analytics-sql";
/**
* Daily net spend, by month, day-of-month and category.
*
* This exists so the spend-pace chart stops computing its own totals. It used to
* sum gross `amount_aud ?? amount` over `transaction_type = 'debit'` in the
* browser, which meant it ignored personal share, refunds, fees, interest, and
* itemised loan repayments — every rule the headline applies. The two numbers
* could disagree while both were labelled "spend", and the chart's own baseline
* line was drawn from the split-adjusted monthly totals, so the two series in
* one chart were on different bases.
*
* Day-of-month granularity is also what lets the page compare a partial current
* month against prior months *through the same day*, instead of against their
* full-month totals — which always made a month in progress look thrifty.
*
* Same fragments as /api/analytics/monthly. If that route's semantics change,
* this one changes with it.
*/
export async function GET(req: NextRequest) {
const user = await getCurrentUser(req);
if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 403 });
const { searchParams } = new URL(req.url);
const monthCount = Math.min(Math.max(Number(searchParams.get("months") || "12"), 1), 24);
const now = new Date();
const endDate = new Date(now.getFullYear(), now.getMonth() + 1, 1);
const startDate = new Date(now.getFullYear(), now.getMonth() - monthCount + 1, 1);
const rows = await queryRaw<{ month: string; day: number; category: string; spent: string }>(
`SELECT
TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM') as month,
EXTRACT(DAY FROM t.transaction_date::date)::int as day,
${EFFECTIVE_CATEGORY} as category,
-- 4dp, not 2. This is grouped finer than /monthly (by day as well as
-- category), so rounding each bucket to cents and summing accumulates a
-- different error than rounding per category does — the pace chart ended
-- the month a few cents off the headline it sits under. Round once, at
-- display time.
SUM(${mySplitOf(SPEND_SIGNED)})::numeric(14,4) as spent
FROM transactions t
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
${STATEMENTS_JOIN}
WHERE ${OWNER_SCOPE} = $1
AND ${NET_SPEND_ROWS}
AND ${EXCLUDE_NON_SPEND}
AND ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date >= $2
AND t.transaction_date < $3
GROUP BY 1, 2, 3
ORDER BY 1, 2`,
[user.id, toDateStr(startDate), toDateStr(endDate)]
);
// Sparse by design — a day with no spend has no entry, and the client treats
// a missing day as zero. Emitting 31 zeroes per month per category would
// dominate the payload.
const daily: Record<string, Record<number, number>> = {};
const byCategory: Record<string, Record<string, Record<number, number>>> = {};
for (const r of rows) {
const spent = Number(r.spent);
const m = (daily[r.month] ??= {});
m[r.day] = (m[r.day] ?? 0) + spent;
const c = ((byCategory[r.month] ??= {})[r.category] ??= {});
c[r.day] = (c[r.day] ?? 0) + spent;
}
return NextResponse.json({ daily, byCategory });
}
+38 -4
View File
@@ -1,12 +1,35 @@
import { NextRequest, NextResponse } from "next/server";
import { getCurrentUser } from "@/lib/auth";
import { queryRaw } from "@/lib/db";
import { OWNER_SCOPE, STATEMENTS_JOIN, mySplitOf } from "@/lib/analytics-sql";
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_RECONCILED_SOURCE, mySplitOf, toDateStr } from "@/lib/analytics-sql";
/**
* Fees and interest over an explicit window.
*
* This used to aggregate every statement ever imported with no date filter, and
* the UI printed the result with no period label — so a lifetime-to-date total
* read as a current-period one, and grew forever. `months=0` asks for all time
* deliberately, which is a different claim from asking for it by accident.
*/
export async function GET(req: NextRequest) {
const user = await getCurrentUser(req);
if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 403 });
const { searchParams } = new URL(req.url);
const monthsParam = Number(searchParams.get("months") ?? "12");
const months = Number.isFinite(monthsParam) ? Math.min(Math.max(monthsParam, 0), 120) : 12;
const allTime = months === 0;
const now = new Date();
const from = new Date(now.getFullYear(), now.getMonth() - months + 1, 1);
const fromStr = toDateStr(from);
const toStr = toDateStr(new Date(now.getFullYear(), now.getMonth() + 1, 1));
// A statement is dated by the period it covers, not by when it was imported.
const stmtWindow = allTime ? "" : `AND billing_end_date >= $2 AND billing_end_date < $3`;
const txnWindow = allTime ? "" : `AND t.transaction_date >= $2 AND t.transaction_date < $3`;
const windowParams = allTime ? [] : [fromStr, toStr];
// Statement-level fees and interest (aggregated by Gemini from the PDF)
const stmtRows = await queryRaw<{
bank_name: string;
@@ -19,10 +42,11 @@ export async function GET(req: NextRequest) {
SUM(COALESCE(interest_charged, 0))::numeric(12,2) AS interest
FROM statements
WHERE owner_id = $1
${stmtWindow}
GROUP BY bank_name
HAVING SUM(COALESCE(fees_charged, 0)) + SUM(COALESCE(interest_charged, 0)) > 0
ORDER BY (SUM(COALESCE(fees_charged, 0)) + SUM(COALESCE(interest_charged, 0))) DESC`,
[user.id]
[user.id, ...windowParams]
);
// Transaction-level fee and interest line items (split-adjusted)
@@ -49,8 +73,10 @@ export async function GET(req: NextRequest) {
${STATEMENTS_JOIN}
WHERE ${OWNER_SCOPE} = $1
AND t.transaction_type IN ('fee', 'interest')
AND ${EXCLUDE_RECONCILED_SOURCE}
${txnWindow}
ORDER BY t.transaction_date DESC`,
[user.id]
[user.id, ...windowParams]
);
const by_bank = stmtRows.map((r) => ({
@@ -69,5 +95,13 @@ export async function GET(req: NextRequest) {
const total_fees = by_bank.reduce((s, r) => s + r.fees, 0);
const total_interest = by_bank.reduce((s, r) => s + r.interest, 0);
return NextResponse.json({ by_bank, transactions, total_fees, total_interest });
return NextResponse.json({
by_bank,
transactions,
total_fees,
total_interest,
// The period is part of the answer — the client must be able to say what
// window these totals cover rather than implying "now".
period: { months, from: allTime ? null : fromStr, to: allTime ? null : toStr, all_time: allTime },
});
}
@@ -1,7 +1,8 @@
import { NextRequest, NextResponse } from "next/server";
import { getCurrentUser } from "@/lib/auth";
import { queryRaw } from "@/lib/db";
import { OWNER_SCOPE, STATEMENTS_JOIN, EFFECTIVE_CATEGORY, mySplitOf } from "@/lib/analytics-sql";
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_RECONCILED_SOURCE, EFFECTIVE_CATEGORY, mySplitOf } from "@/lib/analytics-sql";
import { bankLabel } from "@/lib/queries";
const MY_AMOUNT = mySplitOf(`COALESCE(t.amount_aud, t.amount)`);
@@ -39,7 +40,7 @@ export async function GET(
END::numeric(10,2) as my_amount,
t.transaction_type,
${EFFECTIVE_CATEGORY} as category,
COALESCE(s.bank_name, 'Manual') as bank_name,
${bankLabel()} as bank_name,
t.statement_id
FROM transactions t
${STATEMENTS_JOIN}
@@ -48,6 +49,7 @@ export async function GET(
WHERE ${OWNER_SCOPE} = $1
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) = $2
AND ${EXCLUDE_RECONCILED_SOURCE}
ORDER BY t.transaction_date DESC
LIMIT 500
`, [user.id, decoded]);
+4 -2
View File
@@ -1,7 +1,7 @@
import { NextRequest, NextResponse } from "next/server";
import { getCurrentUser } from "@/lib/auth";
import { queryRaw } from "@/lib/db";
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_NON_SPEND, EFFECTIVE_CATEGORY, mySplitOf } from "@/lib/analytics-sql";
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_NON_SPEND, EXCLUDE_RECONCILED_SOURCE, EFFECTIVE_CATEGORY, mySplitOf, toDateStr } from "@/lib/analytics-sql";
// Split-adjusted amount helper (positive for spend, negative for refunds)
const MY_AMOUNT = mySplitOf(`COALESCE(t.amount_aud, t.amount)`);
@@ -21,7 +21,7 @@ export async function GET(req: NextRequest) {
const cutoff = new Date();
cutoff.setMonth(cutoff.getMonth() - months);
const fromDate = cutoff.toISOString().slice(0, 10);
const fromDate = toDateStr(cutoff);
// Merchant aggregates — net spend (debits + fees - refunds/credits)
const rows = await queryRaw<{
@@ -69,6 +69,7 @@ export async function GET(req: NextRequest) {
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
AND t.transaction_date >= $2
AND ${EXCLUDE_NON_SPEND}
AND ${EXCLUDE_RECONCILED_SOURCE}
GROUP BY 1
HAVING SUM(${SPEND_EXPR}) > 0
ORDER BY net_spend DESC
@@ -95,6 +96,7 @@ export async function GET(req: NextRequest) {
AND t.transaction_date >= $2
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) = ANY($3)
AND ${EXCLUDE_NON_SPEND}
AND ${EXCLUDE_RECONCILED_SOURCE}
GROUP BY 1, 2
ORDER BY 1, 2
`, [user.id, fromDate, topMerchants]);
+10 -3
View File
@@ -6,9 +6,11 @@ import {
STATEMENTS_JOIN,
EFFECTIVE_CATEGORY,
EXCLUDE_NON_SPEND,
EXCLUDE_RECONCILED_SOURCE,
NET_SPEND_ROWS,
SPEND_SIGNED,
mySplitOf,
toDateStr,
} from "@/lib/analytics-sql";
export async function GET(req: NextRequest) {
@@ -22,8 +24,8 @@ export async function GET(req: NextRequest) {
const endDate = new Date(now.getFullYear(), now.getMonth() + 1, 1);
const startDate = new Date(now.getFullYear(), now.getMonth() - monthCount + 1, 1);
const startStr = startDate.toISOString().slice(0, 10);
const endStr = endDate.toISOString().slice(0, 10);
const startStr = toDateStr(startDate);
const endStr = toDateStr(endDate);
// Expenses: debits excluding transfers and investments, split-adjusted
const spendRows = await queryRaw<{
@@ -35,7 +37,9 @@ export async function GET(req: NextRequest) {
`SELECT
TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM') as month,
${EFFECTIVE_CATEGORY} as category,
SUM(${mySplitOf(SPEND_SIGNED)})::numeric(12,2) as total_spent,
-- 4dp so the month total is summed from unrounded parts; every consumer
-- rounds for display. See the note in /api/analytics/daily.
SUM(${mySplitOf(SPEND_SIGNED)})::numeric(14,4) as total_spent,
COUNT(*)::int as transaction_count
FROM transactions t
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
@@ -44,6 +48,7 @@ export async function GET(req: NextRequest) {
WHERE ${OWNER_SCOPE} = $1
AND ${NET_SPEND_ROWS}
AND ${EXCLUDE_NON_SPEND}
AND ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date >= $2
AND t.transaction_date < $3
GROUP BY 1, 2
@@ -67,6 +72,7 @@ export async function GET(req: NextRequest) {
WHERE ${OWNER_SCOPE} = $1
AND t.transaction_type IN ('credit', 'payment')
AND ${EFFECTIVE_CATEGORY} = 'income'
AND ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date >= $2
AND t.transaction_date < $3
GROUP BY 1
@@ -89,6 +95,7 @@ export async function GET(req: NextRequest) {
${STATEMENTS_JOIN}
WHERE ${OWNER_SCOPE} = $1
AND ${EFFECTIVE_CATEGORY} = 'investment'
AND ${EXCLUDE_RECONCILED_SOURCE}
AND t.transaction_date >= $2
AND t.transaction_date < $3
GROUP BY 1
+2 -1
View File
@@ -1,7 +1,7 @@
import { NextRequest, NextResponse } from "next/server";
import { getCurrentUser } from "@/lib/auth";
import { queryRaw } from "@/lib/db";
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_NON_SPEND, EFFECTIVE_CATEGORY, mySplitOf } from "@/lib/analytics-sql";
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_NON_SPEND, EXCLUDE_RECONCILED_SOURCE, EFFECTIVE_CATEGORY, mySplitOf } from "@/lib/analytics-sql";
export async function GET(req: NextRequest) {
const user = await getCurrentUser(req);
@@ -31,6 +31,7 @@ export async function GET(req: NextRequest) {
WHERE ${OWNER_SCOPE} = $1
AND t.transaction_type IN ('debit', 'fee')
AND ${EXCLUDE_NON_SPEND}
AND ${EXCLUDE_RECONCILED_SOURCE}
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) IS NOT NULL
),
merchant_with_lag AS (