fix(transactions): supersede rows imported twice instead of deleting them
ci / lint-test (push) Successful in 48s
ci / lint-test (push) Successful in 48s
Statements 107, 142 and 143 bill overlapping periods on one ANZ account, so 31 transactions -- $42,040.68 -- are in the ledger twice. They are marked superseded, not deleted. Every child of transactions is ON DELETE CASCADE (splits, tags, overrides, expense_metadata, order_reviews), so deleting "the duplicate" destroys whatever curation sits on it, and which member of a pair holds that curation is an accident of import order: here 1 pair carries splits and 6 carry overrides, all on the surviving side, but nothing guarantees that. Superseding keeps the row, keeps its children, and makes a mistake one UPDATE to undo rather than a restore from backup. reconciled_with_id could not be reused. Its predicate is scoped to statement_id IS NULL on purpose -- a statement line pointing at something else is the survivor, not the duplicate -- and here both rows are statement lines. The exclusion goes into EXCLUDE_RECONCILED_SOURCE rather than into a new fragment, so every query already asking "count each purchase once" gets it without being edited. The trip cost queries did not use that fragment at all and now do; verified a no-op on current data (0 trip-tagged rows are either reconciled sources or duplicates), but they were one import away from double-counting. Most of the $42k is transfers and investments, which spend already excludes. The damage was elsewhere: duplicated rows in the list, and rules re-splitting a duplicate -- txn 3807 is one of these 31 and was a candidate for splitting earlier today. Balances are unchanged: no duplicate carried a split.
This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
-- A transaction imported twice cannot simply be deleted.
|
||||
--
|
||||
-- Every child of `transactions` is ON DELETE CASCADE — splits, tags, overrides,
|
||||
-- expense_metadata, order_reviews. Deleting a row said to be "the duplicate"
|
||||
-- therefore destroys whatever curation happens to sit on it, silently and
|
||||
-- unrecoverably. The curation is not reliably on the surviving side either: of
|
||||
-- the 31 known duplicate pairs, one carries splits and six carry overrides, and
|
||||
-- which member holds them is an accident of import order.
|
||||
--
|
||||
-- So a duplicate is superseded, never removed. The row stays, keeps its
|
||||
-- children, and points at the row that replaces it. Reversing a mistake is then
|
||||
-- one UPDATE rather than a restore from backup.
|
||||
--
|
||||
-- This is the statement-vs-statement case. `reconciled_with_id` already covers
|
||||
-- manual-vs-statement, and deliberately cannot be reused: the predicate that
|
||||
-- hides a reconciled row is scoped to `statement_id IS NULL`, because a
|
||||
-- statement line pointing at something else is the survivor, not the duplicate.
|
||||
-- Both of these rows are statement lines.
|
||||
|
||||
ALTER TABLE transactions
|
||||
ADD COLUMN IF NOT EXISTS superseded_by_id integer
|
||||
REFERENCES transactions(id) ON DELETE SET NULL;
|
||||
|
||||
COMMENT ON COLUMN transactions.superseded_by_id IS
|
||||
'This row was imported twice; the named row is the one that counts. Excluded from every figure, kept for its children and its audit trail. NULL = live.';
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_transactions_superseded
|
||||
ON transactions (superseded_by_id)
|
||||
WHERE superseded_by_id IS NOT NULL;
|
||||
|
||||
-- A row cannot supersede itself, and a survivor cannot itself be superseded
|
||||
-- (that would hide both members of the pair and lose the amount entirely).
|
||||
ALTER TABLE transactions
|
||||
DROP CONSTRAINT IF EXISTS transactions_no_self_supersede;
|
||||
ALTER TABLE transactions
|
||||
ADD CONSTRAINT transactions_no_self_supersede
|
||||
CHECK (superseded_by_id IS NULL OR superseded_by_id <> id);
|
||||
@@ -183,11 +183,14 @@ model transactions {
|
||||
payment_method String? // card | cash | bank_transfer | other; NULL = unknown (migration 0016)
|
||||
owner_id Int?
|
||||
reconciled_with_id Int?
|
||||
superseded_by_id Int?
|
||||
principal_amount Decimal? @db.Decimal(12, 2)
|
||||
interest_amount Decimal? @db.Decimal(12, 2)
|
||||
statement statements? @relation(fields: [statement_id], references: [id], onDelete: Cascade)
|
||||
reconciled_with transactions? @relation("reconciled", fields: [reconciled_with_id], references: [id], onDelete: SetNull)
|
||||
reconciled_by transactions[] @relation("reconciled")
|
||||
superseded_by transactions? @relation("superseded", fields: [superseded_by_id], references: [id], onDelete: SetNull)
|
||||
supersedes transactions[] @relation("superseded")
|
||||
expense_metadata expense_metadata?
|
||||
order_review order_reviews?
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user