`transaction_audit` records every INSERT/UPDATE/DELETE on transactions, transaction_splits, transaction_tags, transaction_overrides and expense_metadata, via AFTER row triggers. WHY: on 2026-08-23 the n8n CC Statement Processor stamped `statement_id` onto seven statementless wallet/pantry rows without setting `reconciled_with_id`, double-counting $245.90 and hiding those rows from the reconcile queue (`WHERE statement_id IS NULL`). Answering "was anything lost?" required restoring the 02:30 pg_dump into a scratch database and diffing it column by column, because nothing in the estate recorded what changed. Trigger rather than application code because the ledger has at least five independent writers -- this app (Prisma), n8n (raw SQL), pantry-app, the ingestion runner (psycopg) and ad-hoc `docker exec psql`. That bug was an n8n raw-SQL UPDATE; an app-level audit would not have seen it. Not Postgres' own logs: wal_level=replica, wal_keep_size=0, archive_mode=off, log_statement=none, track_commit_timestamp=off, no pgaudit -- verified live. Postgres was recording nothing usable, and has no built-in audit tables. UPDATEs store only the columns that changed (a whole-row copy buries a one-column edit, and line_items can be kilobytes); no-op UPDATEs write nothing; txid groups the rows of one logical edit, which matters because applyReconcileMatch moves four children in a single transaction. No FK to transactions -- the log must outlive the row. TRUNCATE does not fire row triggers, so the test suite's wipes are silent. Rehearsed against a scratch restore first: changed-column-only diffing, no-op suppression, composite PK (transaction_tags), and the matched_transaction_id fallback (a card-settled order has only that) all verified before applying. Applied to personal and personal_test. Board #259.
This commit is contained in:
@@ -176,6 +176,54 @@ docker exec postgres-personal psql -U personal -d personal < prisma/migrations/<
|
||||
- `statements` — one row per billing period per bank account
|
||||
- `transactions` — line items; `statement_id` is nullable (NULL = manual entry); `reconciled_with_id` links a manual tx to its matched statement tx; `payment_method` (migration 0016, `credits` added by 0018) is `card | cash | bank_transfer | credits | other`, NULL = unknown, and the TypeScript mirror is `src/lib/payment-methods.ts`; `source` / `source_ref` / `source_account` (migration 0028) identify a row that came from an account feed rather than a statement or a hand entry — `source_ref` is the provider's own id and carries a partial unique index, which is the only thing making a re-import idempotent
|
||||
|
||||
### The change log (migration 0037)
|
||||
|
||||
`transaction_audit` is an **append-only** row-level record of every change to
|
||||
`transactions`, `transaction_splits`, `transaction_tags`, `transaction_overrides`
|
||||
and `expense_metadata`. Never `UPDATE` or `DELETE` from it. There is deliberately
|
||||
**no foreign key** to `transactions` — the log has to outlive the row it describes.
|
||||
|
||||
```sql
|
||||
-- everything that ever happened to one transaction
|
||||
SELECT changed_at, table_name, op, old_data, new_data, app_name
|
||||
FROM transaction_audit WHERE transaction_id = 6961 ORDER BY changed_at;
|
||||
|
||||
-- one logical edit, all the rows it touched (a reconcile moves 4 children at once)
|
||||
SELECT * FROM transaction_audit WHERE txid = <txid> ORDER BY id;
|
||||
```
|
||||
|
||||
**Why it exists.** On 2026-08-23 the n8n CC Statement Processor stamped
|
||||
`statement_id` onto seven statementless wallet/pantry rows without setting
|
||||
`reconciled_with_id`, double-counting $245.90 and hiding those rows from the
|
||||
reconcile queue (`WHERE statement_id IS NULL`). Answering *"was anything lost?"*
|
||||
took restoring the 02:30 `pg_dump` into a scratch database and diffing it column
|
||||
by column, because nothing recorded what changed.
|
||||
|
||||
**Why it is a trigger and not application code.** The ledger has at least five
|
||||
independent writers — this app (Prisma), n8n (raw SQL in a Postgres node),
|
||||
pantry-app, the ingestion-engine runner (psycopg), and ad-hoc `docker exec psql`.
|
||||
That bug was an n8n raw-SQL `UPDATE`; an app-level audit would not have seen it.
|
||||
|
||||
**Why not Postgres' own logs.** Checked live: `wal_level=replica` (WAL cannot be
|
||||
decoded into row changes), `wal_keep_size=0` and `archive_mode=off` (recycled
|
||||
almost immediately), `log_statement=none`, `track_commit_timestamp=off`, no
|
||||
`pgaudit`. Postgres records nothing usable and has no built-in audit or temporal
|
||||
tables.
|
||||
|
||||
Three details worth knowing before relying on it:
|
||||
- **`old_data`/`new_data` hold only the columns that actually changed** on an
|
||||
`UPDATE`. A whole-row copy would bury a one-column edit, and
|
||||
`expense_metadata.line_items` alone can be kilobytes.
|
||||
- **A no-op `UPDATE` writes nothing** (the upsert paths issue plenty).
|
||||
- **Attribution is partial and that is understood.** Every client connects as the
|
||||
same `personal` role, so `db_user` never discriminates. `app_name` is only as
|
||||
good as what each client sets; `client_addr` separates containers and is NULL
|
||||
for `docker exec` over the unix socket — itself a useful signal. `txid` is the
|
||||
reliable grouping key.
|
||||
|
||||
`TRUNCATE` does not fire row-level triggers, so the integration suite's table
|
||||
wipes produce no audit rows.
|
||||
|
||||
### Cash and reconciliation
|
||||
|
||||
`payment_method` in (`cash`, `credits`) excludes a transaction from
|
||||
|
||||
Reference in New Issue
Block a user