statements: keep one bank's national entities apart (migration 0032)
ci / lint-test (push) Successful in 44s

Fixes 0031. Every merge it made was provably one account — each split
pair shared an account_number — but the PREFIX generalises past that
evidence. Citibank Australia and Citibank India are different banks with
different accounts and currency, and `citi%` flattens them into one.
Older Citi India statements are queued to load, so this would have merged
them on arrival. Not Citi-specific: `hsbc%` and `%american express%`
carry the identical defect.

Currency alone cannot be the discriminator: Wise holds AUD, EUR and USD
accounts under one provider, so a blanket "non-AUD is a different bank"
rule would shatter it into three. Wise stays unmapped for exactly that
reason, now a documented property rather than an accident.

The country comes from the name when the name states it and from the
currency otherwise, with AUD as home taking no suffix — so nothing
renames: the backfill touched 0 rows.

  'Citibank'       + AUD -> 'Citibank'        (unchanged)
  'Citibank India' + INR -> 'Citibank India'
  'Citibank N.A.'  + INR -> 'Citibank India'  (currency carries it)
  'CITIBANK INDIA' + AUD -> 'Citibank India'  (name beats a wrong currency)

'N.A.' is deliberately not read as a country marker: it means "National
Association", a US legal form printed on Citibank letterhead worldwide,
including India. Reading it as US is precisely the mistake this fixes.

The trigger now fires on UPDATE OF bank_name, currency, since the country
is partly derived from the currency.
This commit is contained in:
2026-08-15 16:52:14 +10:00
parent 0bb4329249
commit c3b623052d
@@ -0,0 +1,164 @@
-- Keep one institution's national entities apart. Fixes 0031.
--
-- 0031 merged spellings by prefix — `citi%` -> 'Citibank'. Every merge it made
-- was provably one account (each split pair shared an account_number), but the
-- PREFIX generalises past that evidence: Citibank Australia and Citibank India
-- are different banks, different accounts, different currency, and `citi%`
-- flattens them into one. Older Citi India statements are queued to load, so
-- this would have merged them on arrival.
--
-- It is not a Citi problem. `hsbc%` and `%american express%` carry the identical
-- defect — all three are global banks operating as separate national entities.
--
-- WHY NOT DISCRIMINATE ON CURRENCY ALONE: Wise holds AUD, EUR and USD accounts
-- under one provider, so a blanket "non-AUD means a different bank" rule would
-- shatter it into three. Wise stays unmapped for exactly this reason — mapping
-- a genuinely multi-currency provider through this function would be wrong, and
-- that is now a documented property rather than an accident.
--
-- THE RULE: the country comes from the name when the name states it, and from
-- the currency otherwise. AUD is home and takes no suffix, so nothing renames
-- today — all 21 existing statements keep the name 0031 gave them.
--
-- 'Citibank' + AUD -> 'Citibank' (unchanged)
-- 'Citi' + AUD -> 'Citibank' (unchanged)
-- 'Citibank Australia'+ AUD -> 'Citibank'
-- 'Citibank India' + INR -> 'Citibank India'
-- 'Citibank N.A.' + INR -> 'Citibank India' (currency carries it)
-- 'HSBC' + AUD -> 'HSBC' (unchanged)
--
-- Degradation is safe but not silent: an India statement whose currency was
-- extracted as AUD and whose name omits the country WILL merge. The tell is a
-- second account_number appearing under one bank_name with a different
-- currency — see the check in CLAUDE.md.
--
-- Idempotent: safe to re-run.
-- 'N.A.' is deliberately absent: it means "National Association", a US legal
-- form that appears on Citibank letterhead worldwide, including India. Reading
-- it as a country marker is precisely the mistake this migration exists to fix.
CREATE OR REPLACE FUNCTION bank_country_suffix(v TEXT, cur TEXT)
RETURNS TEXT AS $$
DECLARE
c TEXT := upper(trim(coalesce(cur, '')));
BEGIN
-- The name wins when it states a country: it is the source's own assertion,
-- and it survives a currency that failed to extract.
IF v LIKE '%australia%' OR v LIKE '% au%' THEN RETURN ''; END IF; -- home
IF v LIKE '%india%' THEN RETURN ' India'; END IF;
IF v LIKE '%singapore%' THEN RETURN ' Singapore'; END IF;
IF v LIKE '%hong kong%' THEN RETURN ' Hong Kong'; END IF;
IF v LIKE '%new zealand%' THEN RETURN ' NZ'; END IF;
IF v LIKE '%united kingdom%' OR v LIKE '% uk%' THEN RETURN ' UK'; END IF;
IF v LIKE '%emirates%' OR v LIKE '% uae%' THEN RETURN ' UAE'; END IF;
-- Otherwise infer from the statement's own currency. AUD is home.
IF c = '' OR c = 'AUD' THEN RETURN ''; END IF;
IF c = 'INR' THEN RETURN ' India'; END IF;
IF c = 'SGD' THEN RETURN ' Singapore'; END IF;
IF c = 'HKD' THEN RETURN ' Hong Kong'; END IF;
IF c = 'NZD' THEN RETURN ' NZ'; END IF;
IF c = 'GBP' THEN RETURN ' UK'; END IF;
IF c = 'AED' THEN RETURN ' UAE'; END IF;
IF c = 'USD' THEN RETURN ' US'; END IF;
-- EUR spans countries and cannot name one, so fall back to the code itself.
-- A distinguishable name beats a wrong one.
RETURN ' ' || c;
END;
$$ LANGUAGE plpgsql IMMUTABLE;
-- `currency` is a second parameter with a default so any existing single-arg
-- call keeps working; the trigger passes NEW.currency.
CREATE OR REPLACE FUNCTION normalize_bank_name(raw TEXT, cur TEXT DEFAULT NULL)
RETURNS TEXT AS $$
DECLARE
v TEXT;
BEGIN
IF raw IS NULL THEN
RETURN NULL;
END IF;
v := lower(regexp_replace(trim(raw), '\s+', ' ', 'g'));
IF v = '' THEN
RETURN NULL;
END IF;
-- ---- Merges: institutions seen under more than one spelling ----
-- Each carries a country suffix, empty for AUD, so one bank's national
-- entities stay apart while its spellings collapse.
IF v LIKE 'zip%' THEN
RETURN 'Zip' || bank_country_suffix(v, cur);
END IF;
IF v LIKE '%american express%' OR v LIKE 'amex%' THEN
RETURN 'American Express' || bank_country_suffix(v, cur);
END IF;
IF v LIKE 'citi%' THEN
RETURN 'Citibank' || bank_country_suffix(v, cur);
END IF;
IF v LIKE '%national australia bank%' OR v = 'nab' THEN
RETURN 'NAB';
END IF;
-- ---- Defensive: map future legal-entity variants onto the name in use ----
IF v LIKE '%westpac%' THEN
RETURN 'Westpac' || bank_country_suffix(v, cur);
END IF;
IF v = 'anz' OR v LIKE 'anz %' OR v LIKE '%australia and new zealand banking%' THEN
RETURN 'ANZ' || bank_country_suffix(v, cur);
END IF;
IF v LIKE '%commonwealth bank%' OR v LIKE 'commbank%' OR v = 'cba' THEN
RETURN 'Commonwealth Bank';
END IF;
IF v LIKE 'hsbc%' THEN
RETURN 'HSBC' || bank_country_suffix(v, cur);
END IF;
IF v = 'ing' OR v LIKE 'ing %' THEN
RETURN 'ING' || bank_country_suffix(v, cur);
END IF;
IF v = 'amp' OR v LIKE 'amp %' THEN
RETURN 'AMP Bank';
END IF;
IF v = 'up' OR v LIKE 'up bank%' THEN
RETURN 'Up';
END IF;
-- Unknown institution: keep exactly what was extracted, tidied only.
-- Wise reaches here, which is what keeps its AUD/EUR/USD accounts together.
RETURN regexp_replace(trim(raw), '\s+', ' ', 'g');
END;
$$ LANGUAGE plpgsql IMMUTABLE;
CREATE OR REPLACE FUNCTION statements_normalize_bank_name_trigger()
RETURNS TRIGGER AS $$
BEGIN
NEW.bank_name := normalize_bank_name(NEW.bank_name, NEW.currency);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Re-point the trigger: it must fire when `currency` changes too, since the
-- country is now partly derived from it.
DROP TRIGGER IF EXISTS trg_statements_normalize_bank_name ON statements;
CREATE TRIGGER trg_statements_normalize_bank_name
BEFORE INSERT OR UPDATE OF bank_name, currency ON statements
FOR EACH ROW EXECUTE FUNCTION statements_normalize_bank_name_trigger();
-- Backfill. Expected to touch nothing: every existing statement is AUD except
-- Wise, which is unmapped.
UPDATE statements
SET bank_name = normalize_bank_name(bank_name, currency)
WHERE bank_name IS DISTINCT FROM normalize_bank_name(bank_name, currency);