wallet: lock the ingest, pair on time and app, skip notices, ECB fallback
ci / lint-test (push) Successful in 54s
ci / lint-test (push) Successful in 54s
Four defects found reviewing a month of overseas wallet rows: - Race: n8n posts a poll's items concurrently and both halves of a tap usually share a poll, so each ran findPairedRow before the other inserted. 11 taps became 22 rows (A$3,841.40), identical created_at on every pair. One advisory lock now covers re-check -> pair -> adopt -> insert, on the transaction's own connection (a second pooled one per request deadlocks a burst). Mutation-tested: without the lock the concurrent test fails 3/3. - Pair window: amount within +/-1 DAY folded the UNICEF $50 donation and an unrelated $50 Amex charge 17h later into one row and renamed it. A pair is now minutes apart AND from two different apps. Needs when/where a notification arrived: migration 0039 adds expense_metadata.notified_at / notif_app, backfilled from capture.wallet_events (144 rows). A merged half now records wallet_evt:<id>, so a replay cannot merge it twice. - Not purchases: a CommBank overdraft notice became a $21.21 purchase named "To avoid fees and interest," (the Westpac fixed-width parser ran on a CommBank body). Account notices and transit-card balance drawdowns are skipped; the descriptor parser only runs on Westpac. - FX: with no ledger history (Hong Kong) every tap stored A$0.00. The ECB rate via Frankfurter is the fallback, flagged rate_ecb; the ledger median stays first because it carries the card's FX fee. Also: dates in Australia/Melbourne like the statements (UTC slicing filed 28 of 134 rows a day early), and test:setup now actually rebuilds personal_test -- replaying the dump over it never applied a migration. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -19,14 +19,23 @@ if ! docker inspect "$PG_CONTAINER" >/dev/null 2>&1; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Create test database (ignore error if it already exists)
|
||||
docker exec "$PG_CONTAINER" psql -U "$PG_USER" -d postgres \
|
||||
-c "CREATE DATABASE $TEST_DB;" 2>/dev/null || true
|
||||
# Drop and recreate, so the schema is rebuilt rather than replayed. Replaying
|
||||
# the dump over an existing database was a silent no-op for every migration:
|
||||
# each CREATE TABLE failed with "already exists" and the old shape survived, so
|
||||
# a new column (0039's expense_metadata.notified_at, 2026-10-03) never reached
|
||||
# the test database. The guard is because this now DROPs: it refuses anything
|
||||
# not named *_test, whatever TEST_DB was set to.
|
||||
case "$TEST_DB" in
|
||||
*_test) ;;
|
||||
*) echo "ERROR: refusing to drop '$TEST_DB' -- the test database must be named *_test"; exit 1 ;;
|
||||
esac
|
||||
docker exec "$PG_CONTAINER" psql -U "$PG_USER" -d postgres -q \
|
||||
-c "DROP DATABASE IF EXISTS $TEST_DB WITH (FORCE);" \
|
||||
-c "CREATE DATABASE $TEST_DB;"
|
||||
|
||||
# Wipe and rebuild schema from production (schema only, no data)
|
||||
echo "Copying schema from $PG_USER to $TEST_DB..."
|
||||
docker exec "$PG_CONTAINER" pg_dump -U "$PG_USER" --schema-only "$PG_USER" \
|
||||
| docker exec -i "$PG_CONTAINER" psql -U "$PG_USER" -d "$TEST_DB" -q
|
||||
| docker exec -i "$PG_CONTAINER" psql -U "$PG_USER" -d "$TEST_DB" -q -v ON_ERROR_STOP=1
|
||||
|
||||
echo "Schema ready."
|
||||
|
||||
|
||||
Reference in New Issue
Block a user