feat(statements+analytics): normalise statement_type; fix analytics scoping
ci / lint-test (push) Successful in 38s
ci / lint-test (push) Successful in 38s
Groundwork for importing bank and loan statements alongside credit cards.
statement_type was whatever free text Gemini put in account_type ('Credit Card',
'credit card', 'credit_card', 'Business Card', 'ACCESS ADVANTAGE',
'multi-currency account'). The UI coped only by doing .includes("card"), which
breaks as soon as bank and loan statements arrive.
- Migration 0013 adds normalize_statement_type() + a BEFORE INSERT/UPDATE
trigger and a CHECK constraint over credit_card|transaction|savings|loan|
offset|investment|other. The trigger means the N8N workflow keeps working
unchanged while it still sends free text. Raw value stays in account_type.
Backfilled 99 existing rows.
- src/lib/statement-types.ts mirrors the vocabulary for the UI; statements page
now filters by the real types and headlines balance vs amount due per type.
Analytics were scoped with INNER JOIN statements + s.owner_id, which silently
dropped all 180 manual/CSV transactions (statement_id IS NULL) from every
report. Switched all six routes to LEFT JOIN + COALESCE(t.owner_id, s.owner_id)
via shared fragments in src/lib/analytics-sql.ts, so the transfers/investment
exclusion that stops card-payment double counting stays consistent. Also
extended that exclusion to trip analytics, which had none.
Drive-by: /api/analytics/subscriptions was returning 500 on an unserialisable
BigInt from COUNT(*) + 1.
Verified against the live DB: monthly spend picks up the previously invisible
manual transactions (Apr 9,849.91 -> 14,286.70) and all four analytics
endpoints return 200.
This commit is contained in:
@@ -127,6 +127,31 @@ Two files only:
|
|||||||
- Owner filter pattern: `WHERE COALESCE(t.owner_id, s.owner_id) = $1`
|
- Owner filter pattern: `WHERE COALESCE(t.owner_id, s.owner_id) = $1`
|
||||||
- Bank name pattern: `COALESCE(s.bank_name, 'Manual') as bank_name`
|
- Bank name pattern: `COALESCE(s.bank_name, 'Manual') as bank_name`
|
||||||
|
|
||||||
|
Analytics queries must import the fragments from `src/lib/analytics-sql.ts`
|
||||||
|
(`STATEMENTS_JOIN`, `OWNER_SCOPE`, `EFFECTIVE_CATEGORY`, `EXCLUDE_NON_SPEND`)
|
||||||
|
rather than hand-rolling them. Two failure modes they exist to prevent:
|
||||||
|
|
||||||
|
- An `INNER JOIN statements` + `WHERE s.owner_id = $1` silently drops every
|
||||||
|
manual/CSV transaction (`statement_id IS NULL`).
|
||||||
|
- Spend must exclude the `transfers` and `investment` categories. Once bank
|
||||||
|
statements are imported alongside card statements, a credit-card payment
|
||||||
|
appears twice — as a debit leaving the bank account and as the underlying
|
||||||
|
purchases on the card statement. Excluding `transfers` is what nets it out.
|
||||||
|
Use the `EXCLUDE_NON_SPEND` fragment: a bare `category NOT IN (...)` evaluates
|
||||||
|
to NULL for uncategorised rows and drops them from totals.
|
||||||
|
|
||||||
|
### Statement types
|
||||||
|
|
||||||
|
`statements.statement_type` is constrained to `credit_card | transaction |
|
||||||
|
savings | loan | offset | investment | other`. Migration 0013 added a
|
||||||
|
`normalize_statement_type()` SQL function plus a BEFORE INSERT/UPDATE trigger, so
|
||||||
|
the N8N workflow can keep sending raw free text (`'ACCESS ADVANTAGE'`, `'Business
|
||||||
|
Card'`) and the DB normalises it on write. The raw extracted value is preserved in
|
||||||
|
`account_type`.
|
||||||
|
|
||||||
|
The TypeScript mirror is `src/lib/statement-types.ts` — keep the list, the SQL
|
||||||
|
function, and the CHECK constraint in sync when adding a type.
|
||||||
|
|
||||||
### Prisma
|
### Prisma
|
||||||
|
|
||||||
The schema at `prisma/schema.prisma` covers all tables. The generated client (gitignored) must be regenerated after schema changes:
|
The schema at `prisma/schema.prisma` covers all tables. The generated client (gitignored) must be regenerated after schema changes:
|
||||||
|
|||||||
@@ -0,0 +1,99 @@
|
|||||||
|
-- Normalise statements.statement_type to a fixed vocabulary.
|
||||||
|
--
|
||||||
|
-- Before this migration the column held whatever free text Gemini put in
|
||||||
|
-- `account_type` ('Credit Card', 'credit card', 'credit_card', 'Business Card',
|
||||||
|
-- 'ACCESS ADVANTAGE', 'multi-currency account', ...). The UI coped only by
|
||||||
|
-- doing `.includes("card")`, which breaks as soon as bank and loan statements
|
||||||
|
-- arrive.
|
||||||
|
--
|
||||||
|
-- The raw extracted text is preserved in `account_type` — this only touches
|
||||||
|
-- `statement_type`. A BEFORE trigger normalises on write, so the N8N ingestion
|
||||||
|
-- workflow keeps working unchanged while it still sends free text.
|
||||||
|
--
|
||||||
|
-- Idempotent: safe to re-run.
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION normalize_statement_type(raw TEXT)
|
||||||
|
RETURNS TEXT AS $$
|
||||||
|
DECLARE
|
||||||
|
v TEXT := lower(trim(coalesce(raw, '')));
|
||||||
|
BEGIN
|
||||||
|
IF v = '' THEN
|
||||||
|
RETURN 'other';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Already canonical
|
||||||
|
IF v IN ('credit_card', 'transaction', 'savings', 'loan', 'offset', 'investment', 'other') THEN
|
||||||
|
RETURN v;
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Offset before loan: "Mortgage Offset" is an offset account, not a loan.
|
||||||
|
IF v LIKE '%offset%' THEN
|
||||||
|
RETURN 'offset';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Loans before transaction: "home loan account" must not match '%account%'.
|
||||||
|
IF v LIKE '%loan%' OR v LIKE '%mortgage%' THEN
|
||||||
|
RETURN 'loan';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Cards: covers 'Credit Card', 'credit card', 'Business Card', 'Charge Card'
|
||||||
|
IF v LIKE '%card%' THEN
|
||||||
|
RETURN 'credit_card';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
IF v LIKE '%saving%' OR v LIKE '%term deposit%' THEN
|
||||||
|
RETURN 'savings';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
IF v LIKE '%invest%' OR v LIKE '%share%' OR v LIKE '%broker%' THEN
|
||||||
|
RETURN 'investment';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Everyday transaction accounts. Bank-specific product names go here; the
|
||||||
|
-- generic keywords catch the rest.
|
||||||
|
IF v LIKE '%access advantage%' -- ANZ
|
||||||
|
OR v LIKE '%complete access%' -- Westpac
|
||||||
|
OR v LIKE '%smart access%' -- CommBank
|
||||||
|
OR v LIKE '%netbank%' -- CommBank
|
||||||
|
OR v LIKE '%classic banking%' -- NAB
|
||||||
|
OR v LIKE '%multi-currency%' -- Wise
|
||||||
|
OR v LIKE '%multi currency%'
|
||||||
|
OR v LIKE '%transaction%'
|
||||||
|
OR v LIKE '%everyday%'
|
||||||
|
OR v LIKE '%cheque%'
|
||||||
|
OR v LIKE '%current account%'
|
||||||
|
OR v LIKE '%debit%'
|
||||||
|
THEN
|
||||||
|
RETURN 'transaction';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
RETURN 'other';
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql IMMUTABLE;
|
||||||
|
|
||||||
|
-- Normalise on write so the N8N workflow can keep sending raw account_type.
|
||||||
|
CREATE OR REPLACE FUNCTION statements_normalize_type_trigger()
|
||||||
|
RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
NEW.statement_type := normalize_statement_type(NEW.statement_type);
|
||||||
|
RETURN NEW;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
DROP TRIGGER IF EXISTS trg_statements_normalize_type ON statements;
|
||||||
|
CREATE TRIGGER trg_statements_normalize_type
|
||||||
|
BEFORE INSERT OR UPDATE OF statement_type ON statements
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION statements_normalize_type_trigger();
|
||||||
|
|
||||||
|
-- Backfill existing rows.
|
||||||
|
UPDATE statements
|
||||||
|
SET statement_type = normalize_statement_type(statement_type)
|
||||||
|
WHERE statement_type IS DISTINCT FROM normalize_statement_type(statement_type);
|
||||||
|
|
||||||
|
-- Guard the vocabulary. Safe because the trigger runs first on every write.
|
||||||
|
ALTER TABLE statements DROP CONSTRAINT IF EXISTS statements_statement_type_check;
|
||||||
|
ALTER TABLE statements
|
||||||
|
ADD CONSTRAINT statements_statement_type_check
|
||||||
|
CHECK (statement_type IN ('credit_card', 'transaction', 'savings', 'loan', 'offset', 'investment', 'other'));
|
||||||
|
|
||||||
|
ALTER TABLE statements ALTER COLUMN statement_type SET DEFAULT 'other';
|
||||||
@@ -137,7 +137,7 @@ model statements {
|
|||||||
event_created Boolean? @default(false)
|
event_created Boolean? @default(false)
|
||||||
tier_used String?
|
tier_used String?
|
||||||
created_at DateTime? @default(now())
|
created_at DateTime? @default(now())
|
||||||
statement_type String @default("credit_card")
|
statement_type String @default("other")
|
||||||
currency String? @default("AUD")
|
currency String? @default("AUD")
|
||||||
opening_balance Decimal? @db.Decimal(12, 2)
|
opening_balance Decimal? @db.Decimal(12, 2)
|
||||||
closing_balance Decimal? @db.Decimal(12, 2)
|
closing_balance Decimal? @db.Decimal(12, 2)
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { NextRequest, NextResponse } from "next/server";
|
import { NextRequest, NextResponse } from "next/server";
|
||||||
import { getCurrentUser } from "@/lib/auth";
|
import { getCurrentUser } from "@/lib/auth";
|
||||||
import { queryRaw } from "@/lib/db";
|
import { queryRaw } from "@/lib/db";
|
||||||
|
import { OWNER_SCOPE, STATEMENTS_JOIN } from "@/lib/analytics-sql";
|
||||||
|
|
||||||
export async function GET(req: NextRequest) {
|
export async function GET(req: NextRequest) {
|
||||||
const user = await getCurrentUser(req);
|
const user = await getCurrentUser(req);
|
||||||
@@ -45,12 +46,12 @@ export async function GET(req: NextRequest) {
|
|||||||
WHEN o.my_share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * o.my_share_percent / 100
|
WHEN o.my_share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * o.my_share_percent / 100
|
||||||
ELSE COALESCE(t.amount_aud, t.amount)
|
ELSE COALESCE(t.amount_aud, t.amount)
|
||||||
END::numeric(12,2) AS my_amount,
|
END::numeric(12,2) AS my_amount,
|
||||||
s.bank_name
|
COALESCE(s.bank_name, 'Manual') AS bank_name
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('fee', 'interest')
|
AND t.transaction_type IN ('fee', 'interest')
|
||||||
ORDER BY t.transaction_date DESC`,
|
ORDER BY t.transaction_date DESC`,
|
||||||
[user.id]
|
[user.id]
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { NextRequest, NextResponse } from "next/server";
|
import { NextRequest, NextResponse } from "next/server";
|
||||||
import { getCurrentUser } from "@/lib/auth";
|
import { getCurrentUser } from "@/lib/auth";
|
||||||
import { queryRaw } from "@/lib/db";
|
import { queryRaw } from "@/lib/db";
|
||||||
|
import { OWNER_SCOPE, STATEMENTS_JOIN } from "@/lib/analytics-sql";
|
||||||
|
|
||||||
export async function GET(
|
export async function GET(
|
||||||
req: NextRequest,
|
req: NextRequest,
|
||||||
@@ -38,13 +39,13 @@ export async function GET(
|
|||||||
END::numeric(10,2) as my_amount,
|
END::numeric(10,2) as my_amount,
|
||||||
t.transaction_type,
|
t.transaction_type,
|
||||||
COALESCE(o.category_override, t.category) as category,
|
COALESCE(o.category_override, t.category) as category,
|
||||||
s.bank_name,
|
COALESCE(s.bank_name, 'Manual') as bank_name,
|
||||||
t.statement_id
|
t.statement_id
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
|
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
|
||||||
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) = $2
|
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) = $2
|
||||||
ORDER BY t.transaction_date DESC
|
ORDER BY t.transaction_date DESC
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { NextRequest, NextResponse } from "next/server";
|
import { NextRequest, NextResponse } from "next/server";
|
||||||
import { getCurrentUser } from "@/lib/auth";
|
import { getCurrentUser } from "@/lib/auth";
|
||||||
import { queryRaw } from "@/lib/db";
|
import { queryRaw } from "@/lib/db";
|
||||||
|
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_NON_SPEND } from "@/lib/analytics-sql";
|
||||||
|
|
||||||
// Split-adjusted amount helper (positive for spend, negative for refunds)
|
// Split-adjusted amount helper (positive for spend, negative for refunds)
|
||||||
const MY_AMOUNT = `CASE WHEN ts.share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * ts.share_percent / 100 WHEN o.my_share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * o.my_share_percent / 100 ELSE COALESCE(t.amount_aud, t.amount) END`;
|
const MY_AMOUNT = `CASE WHEN ts.share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * ts.share_percent / 100 WHEN o.my_share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * o.my_share_percent / 100 ELSE COALESCE(t.amount_aud, t.amount) END`;
|
||||||
@@ -61,13 +62,13 @@ export async function GET(req: NextRequest) {
|
|||||||
MAX(t.transaction_date)::text as last_seen,
|
MAX(t.transaction_date)::text as last_seen,
|
||||||
COUNT(DISTINCT TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM'))::int as months_active
|
COUNT(DISTINCT TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM'))::int as months_active
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
|
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
|
||||||
AND t.transaction_date >= $2
|
AND t.transaction_date >= $2
|
||||||
AND COALESCE(o.category_override, t.category) NOT IN ('transfers', 'investment')
|
AND ${EXCLUDE_NON_SPEND}
|
||||||
GROUP BY 1
|
GROUP BY 1
|
||||||
HAVING SUM(${SPEND_EXPR}) > 0
|
HAVING SUM(${SPEND_EXPR}) > 0
|
||||||
ORDER BY net_spend DESC
|
ORDER BY net_spend DESC
|
||||||
@@ -86,14 +87,14 @@ export async function GET(req: NextRequest) {
|
|||||||
TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM') as month,
|
TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM') as month,
|
||||||
SUM(${SPEND_EXPR})::numeric(10,2) as total
|
SUM(${SPEND_EXPR})::numeric(10,2) as total
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
|
AND t.transaction_type IN ('debit', 'fee', 'interest', 'refund', 'credit')
|
||||||
AND t.transaction_date >= $2
|
AND t.transaction_date >= $2
|
||||||
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) = ANY($3)
|
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name, t.description) = ANY($3)
|
||||||
AND COALESCE(o.category_override, t.category) NOT IN ('transfers', 'investment')
|
AND ${EXCLUDE_NON_SPEND}
|
||||||
GROUP BY 1, 2
|
GROUP BY 1, 2
|
||||||
ORDER BY 1, 2
|
ORDER BY 1, 2
|
||||||
`, [user.id, fromDate, topMerchants]);
|
`, [user.id, fromDate, topMerchants]);
|
||||||
|
|||||||
@@ -1,6 +1,12 @@
|
|||||||
import { NextRequest, NextResponse } from "next/server";
|
import { NextRequest, NextResponse } from "next/server";
|
||||||
import { getCurrentUser } from "@/lib/auth";
|
import { getCurrentUser } from "@/lib/auth";
|
||||||
import { queryRaw } from "@/lib/db";
|
import { queryRaw } from "@/lib/db";
|
||||||
|
import {
|
||||||
|
OWNER_SCOPE,
|
||||||
|
STATEMENTS_JOIN,
|
||||||
|
EFFECTIVE_CATEGORY,
|
||||||
|
EXCLUDE_NON_SPEND,
|
||||||
|
} from "@/lib/analytics-sql";
|
||||||
|
|
||||||
export async function GET(req: NextRequest) {
|
export async function GET(req: NextRequest) {
|
||||||
const user = await getCurrentUser(req);
|
const user = await getCurrentUser(req);
|
||||||
@@ -25,7 +31,7 @@ export async function GET(req: NextRequest) {
|
|||||||
}>(
|
}>(
|
||||||
`SELECT
|
`SELECT
|
||||||
TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM') as month,
|
TO_CHAR(DATE_TRUNC('month', t.transaction_date::date), 'YYYY-MM') as month,
|
||||||
COALESCE(o.category_override, t.category) as category,
|
${EFFECTIVE_CATEGORY} as category,
|
||||||
SUM(
|
SUM(
|
||||||
CASE
|
CASE
|
||||||
WHEN ts.share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * ts.share_percent / 100
|
WHEN ts.share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * ts.share_percent / 100
|
||||||
@@ -37,10 +43,10 @@ export async function GET(req: NextRequest) {
|
|||||||
FROM transactions t
|
FROM transactions t
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('debit', 'fee', 'interest')
|
AND t.transaction_type IN ('debit', 'fee', 'interest')
|
||||||
AND COALESCE(o.category_override, t.category) NOT IN ('transfers', 'investment')
|
AND ${EXCLUDE_NON_SPEND}
|
||||||
AND t.transaction_date >= $2
|
AND t.transaction_date >= $2
|
||||||
AND t.transaction_date < $3
|
AND t.transaction_date < $3
|
||||||
GROUP BY 1, 2
|
GROUP BY 1, 2
|
||||||
@@ -60,10 +66,10 @@ export async function GET(req: NextRequest) {
|
|||||||
COUNT(*)::int as transaction_count
|
COUNT(*)::int as transaction_count
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('credit', 'payment')
|
AND t.transaction_type IN ('credit', 'payment')
|
||||||
AND COALESCE(o.category_override, t.category) = 'income'
|
AND ${EFFECTIVE_CATEGORY} = 'income'
|
||||||
AND t.transaction_date >= $2
|
AND t.transaction_date >= $2
|
||||||
AND t.transaction_date < $3
|
AND t.transaction_date < $3
|
||||||
GROUP BY 1
|
GROUP BY 1
|
||||||
@@ -83,9 +89,9 @@ export async function GET(req: NextRequest) {
|
|||||||
COUNT(*)::int as transaction_count
|
COUNT(*)::int as transaction_count
|
||||||
FROM transactions t
|
FROM transactions t
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND COALESCE(o.category_override, t.category) = 'investment'
|
AND ${EFFECTIVE_CATEGORY} = 'investment'
|
||||||
AND t.transaction_date >= $2
|
AND t.transaction_date >= $2
|
||||||
AND t.transaction_date < $3
|
AND t.transaction_date < $3
|
||||||
GROUP BY 1
|
GROUP BY 1
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { NextRequest, NextResponse } from "next/server";
|
import { NextRequest, NextResponse } from "next/server";
|
||||||
import { getCurrentUser } from "@/lib/auth";
|
import { getCurrentUser } from "@/lib/auth";
|
||||||
import { queryRaw } from "@/lib/db";
|
import { queryRaw } from "@/lib/db";
|
||||||
|
import { OWNER_SCOPE, STATEMENTS_JOIN, EXCLUDE_NON_SPEND, EFFECTIVE_CATEGORY } from "@/lib/analytics-sql";
|
||||||
|
|
||||||
export async function GET(req: NextRequest) {
|
export async function GET(req: NextRequest) {
|
||||||
const user = await getCurrentUser(req);
|
const user = await getCurrentUser(req);
|
||||||
@@ -20,7 +21,7 @@ export async function GET(req: NextRequest) {
|
|||||||
`WITH merchant_txns AS (
|
`WITH merchant_txns AS (
|
||||||
SELECT
|
SELECT
|
||||||
COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) AS merchant,
|
COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) AS merchant,
|
||||||
COALESCE(o.category_override, t.category) AS category,
|
${EFFECTIVE_CATEGORY} AS category,
|
||||||
t.transaction_date,
|
t.transaction_date,
|
||||||
CASE
|
CASE
|
||||||
WHEN ts.share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * ts.share_percent / 100
|
WHEN ts.share_percent IS NOT NULL THEN COALESCE(t.amount_aud, t.amount) * ts.share_percent / 100
|
||||||
@@ -30,9 +31,10 @@ export async function GET(req: NextRequest) {
|
|||||||
FROM transactions t
|
FROM transactions t
|
||||||
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
|
||||||
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
LEFT JOIN transaction_splits ts ON ts.transaction_id = t.id AND ts.participant_id = $1
|
||||||
JOIN statements s ON s.id = t.statement_id
|
${STATEMENTS_JOIN}
|
||||||
WHERE s.owner_id = $1
|
WHERE ${OWNER_SCOPE} = $1
|
||||||
AND t.transaction_type IN ('debit', 'fee')
|
AND t.transaction_type IN ('debit', 'fee')
|
||||||
|
AND ${EXCLUDE_NON_SPEND}
|
||||||
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) IS NOT NULL
|
AND COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) IS NOT NULL
|
||||||
),
|
),
|
||||||
merchant_with_lag AS (
|
merchant_with_lag AS (
|
||||||
@@ -48,7 +50,7 @@ export async function GET(req: NextRequest) {
|
|||||||
SELECT
|
SELECT
|
||||||
merchant,
|
merchant,
|
||||||
MODE() WITHIN GROUP (ORDER BY category) AS category,
|
MODE() WITHIN GROUP (ORDER BY category) AS category,
|
||||||
COUNT(*) + 1 AS occurrences,
|
(COUNT(*) + 1)::int AS occurrences,
|
||||||
AVG(my_amount)::numeric(12,2) AS avg_amount,
|
AVG(my_amount)::numeric(12,2) AS avg_amount,
|
||||||
MIN(transaction_date) AS first_seen,
|
MIN(transaction_date) AS first_seen,
|
||||||
MAX(transaction_date) AS last_seen,
|
MAX(transaction_date) AS last_seen,
|
||||||
|
|||||||
+19
-10
@@ -3,6 +3,12 @@
|
|||||||
import { useState, useMemo } from "react";
|
import { useState, useMemo } from "react";
|
||||||
import Link from "next/link";
|
import Link from "next/link";
|
||||||
import { useStatements, useParticipants, useUpdateStatement } from "@/lib/hooks";
|
import { useStatements, useParticipants, useUpdateStatement } from "@/lib/hooks";
|
||||||
|
import {
|
||||||
|
STATEMENT_TYPES,
|
||||||
|
STATEMENT_TYPE_LABELS,
|
||||||
|
asStatementType,
|
||||||
|
isLiability,
|
||||||
|
} from "@/lib/statement-types";
|
||||||
|
|
||||||
function formatDate(d: string | null) {
|
function formatDate(d: string | null) {
|
||||||
if (!d) return "—";
|
if (!d) return "—";
|
||||||
@@ -40,7 +46,7 @@ export default function StatementsPage() {
|
|||||||
const updateStatement = useUpdateStatement();
|
const updateStatement = useUpdateStatement();
|
||||||
|
|
||||||
const [bankFilter, setBankFilter] = useState("");
|
const [bankFilter, setBankFilter] = useState("");
|
||||||
const [typeFilter, setTypeFilter] = useState<"all" | "card" | "bank">("all");
|
const [typeFilter, setTypeFilter] = useState<"all" | (typeof STATEMENT_TYPES)[number]>("all");
|
||||||
const [ownerFilter, setOwnerFilter] = useState("");
|
const [ownerFilter, setOwnerFilter] = useState("");
|
||||||
const [yearFilter, setYearFilter] = useState("");
|
const [yearFilter, setYearFilter] = useState("");
|
||||||
|
|
||||||
@@ -63,10 +69,8 @@ export default function StatementsPage() {
|
|||||||
const filtered = useMemo(() => {
|
const filtered = useMemo(() => {
|
||||||
if (!statements) return [];
|
if (!statements) return [];
|
||||||
return statements.filter((s) => {
|
return statements.filter((s) => {
|
||||||
const isCard = s.statement_type?.toLowerCase().includes("card") ?? false;
|
|
||||||
if (bankFilter && s.bank_name !== bankFilter) return false;
|
if (bankFilter && s.bank_name !== bankFilter) return false;
|
||||||
if (typeFilter === "card" && !isCard) return false;
|
if (typeFilter !== "all" && asStatementType(s.statement_type) !== typeFilter) return false;
|
||||||
if (typeFilter === "bank" && isCard) return false;
|
|
||||||
if (ownerFilter && String(s.owner_id) !== ownerFilter) return false;
|
if (ownerFilter && String(s.owner_id) !== ownerFilter) return false;
|
||||||
if (yearFilter && s.billing_end_date?.slice(0, 4) !== yearFilter) return false;
|
if (yearFilter && s.billing_end_date?.slice(0, 4) !== yearFilter) return false;
|
||||||
return true;
|
return true;
|
||||||
@@ -98,8 +102,9 @@ export default function StatementsPage() {
|
|||||||
|
|
||||||
<select value={typeFilter} onChange={(e) => setTypeFilter(e.target.value as typeof typeFilter)} className={selectCls}>
|
<select value={typeFilter} onChange={(e) => setTypeFilter(e.target.value as typeof typeFilter)} className={selectCls}>
|
||||||
<option value="all">All types</option>
|
<option value="all">All types</option>
|
||||||
<option value="card">Credit card</option>
|
{STATEMENT_TYPES.map((t) => (
|
||||||
<option value="bank">Bank account</option>
|
<option key={t} value={t}>{STATEMENT_TYPE_LABELS[t]}</option>
|
||||||
|
))}
|
||||||
</select>
|
</select>
|
||||||
|
|
||||||
{participants && participants.length > 1 && (
|
{participants && participants.length > 1 && (
|
||||||
@@ -152,10 +157,14 @@ export default function StatementsPage() {
|
|||||||
</thead>
|
</thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
{filtered.map((s, idx) => {
|
{filtered.map((s, idx) => {
|
||||||
const isCreditCard = s.statement_type?.toLowerCase().includes("card") ?? false;
|
const stmtType = asStatementType(s.statement_type);
|
||||||
const displayAmount = isCreditCard ? s.total_amount_due : s.closing_balance;
|
const owed = isLiability(s.statement_type);
|
||||||
|
// Cards headline the amount due; everything else (including loans,
|
||||||
|
// where the balance is what's still owed) headlines the balance.
|
||||||
|
const displayAmount =
|
||||||
|
stmtType === "credit_card" ? s.total_amount_due : s.closing_balance;
|
||||||
const amount = Number(displayAmount);
|
const amount = Number(displayAmount);
|
||||||
const amountColor = isCreditCard
|
const amountColor = owed
|
||||||
? "text-red-400"
|
? "text-red-400"
|
||||||
: amount >= 0
|
: amount >= 0
|
||||||
? "text-green-400"
|
? "text-green-400"
|
||||||
@@ -185,7 +194,7 @@ export default function StatementsPage() {
|
|||||||
{formatPeriod(s.billing_start_date, s.billing_end_date)}
|
{formatPeriod(s.billing_start_date, s.billing_end_date)}
|
||||||
</td>
|
</td>
|
||||||
<td className="px-4 py-3 text-zinc-400 whitespace-nowrap">
|
<td className="px-4 py-3 text-zinc-400 whitespace-nowrap">
|
||||||
{isCreditCard ? formatDate(s.payment_due_date) : formatDate(s.billing_end_date)}
|
{formatDate(s.payment_due_date ?? s.billing_end_date)}
|
||||||
</td>
|
</td>
|
||||||
<td className="px-4 py-3 text-zinc-500 text-xs">
|
<td className="px-4 py-3 text-zinc-500 text-xs">
|
||||||
{s.currency}
|
{s.currency}
|
||||||
|
|||||||
@@ -0,0 +1,32 @@
|
|||||||
|
// Shared SQL fragments for analytics queries, so spend/income semantics stay
|
||||||
|
// identical across routes.
|
||||||
|
//
|
||||||
|
// Two rules every analytics query must follow:
|
||||||
|
//
|
||||||
|
// 1. Join `statements` with LEFT JOIN and scope on COALESCE(t.owner_id, s.owner_id).
|
||||||
|
// An INNER JOIN silently drops every manual/CSV transaction (statement_id IS
|
||||||
|
// NULL) — which is most of the reconciliation and cash-spend data.
|
||||||
|
// 2. Exclude `transfers` and `investment` from spend. Once bank statements are
|
||||||
|
// imported, a credit-card payment appears twice: once as a debit leaving the
|
||||||
|
// bank account and again as the underlying purchases on the card statement.
|
||||||
|
// Categorising the money movement as `transfers` and excluding it here is what
|
||||||
|
// stops the double count. Investments are a balance-sheet move, not spend.
|
||||||
|
|
||||||
|
/** Owner scoping that works for both statement-linked and manual transactions. */
|
||||||
|
export const OWNER_SCOPE = `COALESCE(t.owner_id, s.owner_id)`;
|
||||||
|
|
||||||
|
/** Join clause to pair with OWNER_SCOPE. */
|
||||||
|
export const STATEMENTS_JOIN = `LEFT JOIN statements s ON s.id = t.statement_id`;
|
||||||
|
|
||||||
|
/** Transaction types that represent money going out. */
|
||||||
|
export const SPEND_TYPES = `('debit', 'fee', 'interest')`;
|
||||||
|
|
||||||
|
/** Effective category, honouring overrides. Never NULL. */
|
||||||
|
export const EFFECTIVE_CATEGORY = `COALESCE(o.category_override, t.category, 'other')`;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Predicate excluding money-movement categories.
|
||||||
|
* The COALESCE matters: a bare `category NOT IN (...)` evaluates to NULL for
|
||||||
|
* uncategorised rows, which silently drops them from spend totals.
|
||||||
|
*/
|
||||||
|
export const EXCLUDE_NON_SPEND = `${EFFECTIVE_CATEGORY} NOT IN ('transfers', 'investment')`;
|
||||||
+11
-2
@@ -670,7 +670,9 @@ export async function getTrips(ownerId: number): Promise<TripRow[]> {
|
|||||||
SELECT
|
SELECT
|
||||||
t.*,
|
t.*,
|
||||||
COALESCE(SUM(
|
COALESCE(SUM(
|
||||||
CASE WHEN tx.transaction_type IN ('debit','fee','interest') THEN COALESCE(tx.amount_aud, tx.amount) ELSE 0 END
|
CASE WHEN tx.transaction_type IN ('debit','fee','interest')
|
||||||
|
AND COALESCE(o.category_override, tx.category, 'other') NOT IN ('transfers', 'investment')
|
||||||
|
THEN COALESCE(tx.amount_aud, tx.amount) ELSE 0 END
|
||||||
), 0)::float AS total_spend,
|
), 0)::float AS total_spend,
|
||||||
COUNT(o.transaction_id)::int AS transaction_count
|
COUNT(o.transaction_id)::int AS transaction_count
|
||||||
FROM trips t
|
FROM trips t
|
||||||
@@ -687,7 +689,9 @@ export async function getTripById(id: number, ownerId: number): Promise<TripRow
|
|||||||
SELECT
|
SELECT
|
||||||
t.*,
|
t.*,
|
||||||
COALESCE(SUM(
|
COALESCE(SUM(
|
||||||
CASE WHEN tx.transaction_type IN ('debit','fee','interest') THEN COALESCE(tx.amount_aud, tx.amount) ELSE 0 END
|
CASE WHEN tx.transaction_type IN ('debit','fee','interest')
|
||||||
|
AND COALESCE(o.category_override, tx.category, 'other') NOT IN ('transfers', 'investment')
|
||||||
|
THEN COALESCE(tx.amount_aud, tx.amount) ELSE 0 END
|
||||||
), 0)::float AS total_spend,
|
), 0)::float AS total_spend,
|
||||||
COUNT(o.transaction_id)::int AS transaction_count
|
COUNT(o.transaction_id)::int AS transaction_count
|
||||||
FROM trips t
|
FROM trips t
|
||||||
@@ -713,6 +717,7 @@ export async function getTripAnalytics(tripId: number, ownerId: number): Promise
|
|||||||
JOIN transactions tx ON tx.id = o.transaction_id
|
JOIN transactions tx ON tx.id = o.transaction_id
|
||||||
WHERE o.trip_id = $1
|
WHERE o.trip_id = $1
|
||||||
AND tx.transaction_type IN ('debit','fee','interest')
|
AND tx.transaction_type IN ('debit','fee','interest')
|
||||||
|
AND COALESCE(o.category_override, tx.category, 'other') NOT IN ('transfers', 'investment')
|
||||||
GROUP BY 1
|
GROUP BY 1
|
||||||
ORDER BY 2 DESC
|
ORDER BY 2 DESC
|
||||||
`, [tripId]),
|
`, [tripId]),
|
||||||
@@ -725,6 +730,7 @@ export async function getTripAnalytics(tripId: number, ownerId: number): Promise
|
|||||||
JOIN transactions tx ON tx.id = o.transaction_id
|
JOIN transactions tx ON tx.id = o.transaction_id
|
||||||
WHERE o.trip_id = $1
|
WHERE o.trip_id = $1
|
||||||
AND tx.transaction_type IN ('debit','fee','interest')
|
AND tx.transaction_type IN ('debit','fee','interest')
|
||||||
|
AND COALESCE(o.category_override, tx.category, 'other') NOT IN ('transfers', 'investment')
|
||||||
GROUP BY 1
|
GROUP BY 1
|
||||||
ORDER BY 1
|
ORDER BY 1
|
||||||
`, [tripId]),
|
`, [tripId]),
|
||||||
@@ -738,6 +744,7 @@ export async function getTripAnalytics(tripId: number, ownerId: number): Promise
|
|||||||
JOIN transactions tx ON tx.id = o.transaction_id
|
JOIN transactions tx ON tx.id = o.transaction_id
|
||||||
WHERE o.trip_id = $1
|
WHERE o.trip_id = $1
|
||||||
AND tx.transaction_type IN ('debit','fee','interest')
|
AND tx.transaction_type IN ('debit','fee','interest')
|
||||||
|
AND COALESCE(o.category_override, tx.category, 'other') NOT IN ('transfers', 'investment')
|
||||||
GROUP BY 1
|
GROUP BY 1
|
||||||
ORDER BY 2 DESC
|
ORDER BY 2 DESC
|
||||||
LIMIT 10
|
LIMIT 10
|
||||||
@@ -754,6 +761,7 @@ export async function getTripAnalytics(tripId: number, ownerId: number): Promise
|
|||||||
JOIN tags tg ON tg.id = tt.tag_id
|
JOIN tags tg ON tg.id = tt.tag_id
|
||||||
WHERE o.trip_id = $1
|
WHERE o.trip_id = $1
|
||||||
AND tx.transaction_type IN ('debit','fee','interest')
|
AND tx.transaction_type IN ('debit','fee','interest')
|
||||||
|
AND COALESCE(o.category_override, tx.category, 'other') NOT IN ('transfers', 'investment')
|
||||||
GROUP BY tg.id
|
GROUP BY tg.id
|
||||||
ORDER BY 4 DESC
|
ORDER BY 4 DESC
|
||||||
`, [tripId]),
|
`, [tripId]),
|
||||||
@@ -771,6 +779,7 @@ export async function getTripAnalytics(tripId: number, ownerId: number): Promise
|
|||||||
JOIN participants p ON p.id = ts.participant_id
|
JOIN participants p ON p.id = ts.participant_id
|
||||||
WHERE o.trip_id = $1
|
WHERE o.trip_id = $1
|
||||||
AND tx.transaction_type IN ('debit','fee','interest')
|
AND tx.transaction_type IN ('debit','fee','interest')
|
||||||
|
AND COALESCE(o.category_override, tx.category, 'other') NOT IN ('transfers', 'investment')
|
||||||
GROUP BY p.id
|
GROUP BY p.id
|
||||||
ORDER BY 3 DESC
|
ORDER BY 3 DESC
|
||||||
`, [tripId]),
|
`, [tripId]),
|
||||||
|
|||||||
@@ -0,0 +1,45 @@
|
|||||||
|
// Canonical statement types. Mirrors the CHECK constraint and the
|
||||||
|
// normalize_statement_type() SQL function in migration 0013 — keep them in sync.
|
||||||
|
export const STATEMENT_TYPES = [
|
||||||
|
"credit_card",
|
||||||
|
"transaction",
|
||||||
|
"savings",
|
||||||
|
"loan",
|
||||||
|
"offset",
|
||||||
|
"investment",
|
||||||
|
"other",
|
||||||
|
] as const;
|
||||||
|
|
||||||
|
export type StatementType = (typeof STATEMENT_TYPES)[number];
|
||||||
|
|
||||||
|
export const STATEMENT_TYPE_LABELS: Record<StatementType, string> = {
|
||||||
|
credit_card: "Credit Card",
|
||||||
|
transaction: "Transaction",
|
||||||
|
savings: "Savings",
|
||||||
|
loan: "Loan",
|
||||||
|
offset: "Offset",
|
||||||
|
investment: "Investment",
|
||||||
|
other: "Other",
|
||||||
|
};
|
||||||
|
|
||||||
|
export function isStatementType(v: string | null | undefined): v is StatementType {
|
||||||
|
return !!v && (STATEMENT_TYPES as readonly string[]).includes(v);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Falls back to "other" for anything unrecognised (e.g. rows predating 0013). */
|
||||||
|
export function asStatementType(v: string | null | undefined): StatementType {
|
||||||
|
return isStatementType(v) ? v : "other";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function formatStatementType(v: string | null | undefined): string {
|
||||||
|
return STATEMENT_TYPE_LABELS[asStatementType(v)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Types where the headline figure is "amount owed" rather than "balance held",
|
||||||
|
* and where a payment due date is meaningful.
|
||||||
|
*/
|
||||||
|
export function isLiability(v: string | null | undefined): boolean {
|
||||||
|
const t = asStatementType(v);
|
||||||
|
return t === "credit_card" || t === "loan";
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user