db: append-only change log for the ledger (migration 0037)
ci / lint-test (push) Successful in 57s

`transaction_audit` records every INSERT/UPDATE/DELETE on transactions,
transaction_splits, transaction_tags, transaction_overrides and
expense_metadata, via AFTER row triggers.

WHY: on 2026-08-23 the n8n CC Statement Processor stamped `statement_id` onto
seven statementless wallet/pantry rows without setting `reconciled_with_id`,
double-counting $245.90 and hiding those rows from the reconcile queue
(`WHERE statement_id IS NULL`). Answering "was anything lost?" required
restoring the 02:30 pg_dump into a scratch database and diffing it column by
column, because nothing in the estate recorded what changed.

Trigger rather than application code because the ledger has at least five
independent writers -- this app (Prisma), n8n (raw SQL), pantry-app, the
ingestion runner (psycopg) and ad-hoc `docker exec psql`. That bug was an n8n
raw-SQL UPDATE; an app-level audit would not have seen it.

Not Postgres' own logs: wal_level=replica, wal_keep_size=0, archive_mode=off,
log_statement=none, track_commit_timestamp=off, no pgaudit -- verified live.
Postgres was recording nothing usable, and has no built-in audit tables.

UPDATEs store only the columns that changed (a whole-row copy buries a
one-column edit, and line_items can be kilobytes); no-op UPDATEs write nothing;
txid groups the rows of one logical edit, which matters because
applyReconcileMatch moves four children in a single transaction. No FK to
transactions -- the log must outlive the row. TRUNCATE does not fire row
triggers, so the test suite's wipes are silent.

Rehearsed against a scratch restore first: changed-column-only diffing, no-op
suppression, composite PK (transaction_tags), and the
matched_transaction_id fallback (a card-settled order has only that) all
verified before applying. Applied to personal and personal_test. Board #259.
This commit is contained in:
2026-08-23 15:57:44 +10:00
parent 2c05f10aed
commit 2251060ac6
2 changed files with 200 additions and 0 deletions
+48
View File
@@ -176,6 +176,54 @@ docker exec postgres-personal psql -U personal -d personal < prisma/migrations/<
- `statements` — one row per billing period per bank account
- `transactions` — line items; `statement_id` is nullable (NULL = manual entry); `reconciled_with_id` links a manual tx to its matched statement tx; `payment_method` (migration 0016, `credits` added by 0018) is `card | cash | bank_transfer | credits | other`, NULL = unknown, and the TypeScript mirror is `src/lib/payment-methods.ts`; `source` / `source_ref` / `source_account` (migration 0028) identify a row that came from an account feed rather than a statement or a hand entry — `source_ref` is the provider's own id and carries a partial unique index, which is the only thing making a re-import idempotent
### The change log (migration 0037)
`transaction_audit` is an **append-only** row-level record of every change to
`transactions`, `transaction_splits`, `transaction_tags`, `transaction_overrides`
and `expense_metadata`. Never `UPDATE` or `DELETE` from it. There is deliberately
**no foreign key** to `transactions` — the log has to outlive the row it describes.
```sql
-- everything that ever happened to one transaction
SELECT changed_at, table_name, op, old_data, new_data, app_name
FROM transaction_audit WHERE transaction_id = 6961 ORDER BY changed_at;
-- one logical edit, all the rows it touched (a reconcile moves 4 children at once)
SELECT * FROM transaction_audit WHERE txid = <txid> ORDER BY id;
```
**Why it exists.** On 2026-08-23 the n8n CC Statement Processor stamped
`statement_id` onto seven statementless wallet/pantry rows without setting
`reconciled_with_id`, double-counting $245.90 and hiding those rows from the
reconcile queue (`WHERE statement_id IS NULL`). Answering *"was anything lost?"*
took restoring the 02:30 `pg_dump` into a scratch database and diffing it column
by column, because nothing recorded what changed.
**Why it is a trigger and not application code.** The ledger has at least five
independent writers — this app (Prisma), n8n (raw SQL in a Postgres node),
pantry-app, the ingestion-engine runner (psycopg), and ad-hoc `docker exec psql`.
That bug was an n8n raw-SQL `UPDATE`; an app-level audit would not have seen it.
**Why not Postgres' own logs.** Checked live: `wal_level=replica` (WAL cannot be
decoded into row changes), `wal_keep_size=0` and `archive_mode=off` (recycled
almost immediately), `log_statement=none`, `track_commit_timestamp=off`, no
`pgaudit`. Postgres records nothing usable and has no built-in audit or temporal
tables.
Three details worth knowing before relying on it:
- **`old_data`/`new_data` hold only the columns that actually changed** on an
`UPDATE`. A whole-row copy would bury a one-column edit, and
`expense_metadata.line_items` alone can be kilobytes.
- **A no-op `UPDATE` writes nothing** (the upsert paths issue plenty).
- **Attribution is partial and that is understood.** Every client connects as the
same `personal` role, so `db_user` never discriminates. `app_name` is only as
good as what each client sets; `client_addr` separates containers and is NULL
for `docker exec` over the unix socket — itself a useful signal. `txid` is the
reliable grouping key.
`TRUNCATE` does not fire row-level triggers, so the integration suite's table
wipes produce no audit rows.
### Cash and reconciliation
`payment_method` in (`cash`, `credits`) excludes a transaction from