orders phase 2: order_transaction_links, and move every reader to it (board 205)
ci / lint-test (push) Successful in 41s

Phase 1 read linkage out of expense_metadata, which is shaped as ONE ROW PER
TRANSACTION — transaction_id UNIQUE, matched_transaction_id partial-unique — so
every multiplicity it expresses is smuggled through a string key (0029 keys
split shipments <entity_key>#f<fact_id>). A BNPL plan needs four rows against
one order and has no such trick available.

Migration 0030 adds order_transaction_links (many-to-many, keyed on entity_key
rather than entities.id: finance-app does not model the spine and must not hold
an FK across a boundary a re-extraction can decompile) and backfills all 63
existing bridge rows.

THE UNIQUENESS RULE WAS WRONG FIRST TIME, in the most on-brand way available.
It read (entity_key, leg_kind, COALESCE(leg_index, 0)), so two shipment legs of
the same order — both leg_index NULL — collapsed to one key and ON CONFLICT DO
NOTHING dropped one SILENTLY. Caught only because the backfill reported 62
against 63 candidates. The row it ate was order_amazon_249-4859367-0690246's
$130.00 second shipment, the same order named in migration 0029's comment as
the reason split shipments need distinct keys at all. What identifies a leg
depends on its kind: an instalment by its INDEX, a shipment by its FACT, a
whole-order charge by neither.

ALL FOUR READERS MOVE TOGETHER, links first with expense_metadata as fallback:

  - LINK_LATERAL (list txn_count/first_txn_id, and the has_transaction facets
    that read it)
  - the detail page's transactions query
  - /api/transactions/[id]/order
  - the order_ctx lateral in queries.ts

Moving fewer is not a smaller change, it is an inconsistent one: the matcher's
four links for order_ebay_14-11714-95953 come with no expense_metadata row, so
a half-move would show four instalments on the detail page while the list said
txn_count = 0 and put the order on the wrong side of BOTH has_transaction
filters. Verified after: detail 4 legs, list txn_count 4, has_transaction=yes
includes it, =no excludes it.

UNION not UNION ALL on transaction_id — after the backfill the same charge is
legitimately in both stores and counting it twice would show "2 charges" on a
single-payment order.

order_platform is deliberately NOT coalesced with the link's platform. It gates
the receipt disclosure arrow on /transactions, and a BNPL leg has no receipt
behind it — filling it in put an arrow on four Afterpay rows that expand to
nothing, which is the exact promise the arrow exists to avoid making. Separate
leg fields carry the sub-line instead ("1 of 4 - DJI Air 3 Fly More Combo").
order-details.tsx now also requires a real receipt (platform present) before
rendering, because the endpoint can answer with a link alone.

Two invariant views, both empty and expected to stay so: order_link_orphans
(spine re-keying silently orphans a TEXT key — bridge links are rebuildable,
`manual` ones are lost curation) and order_link_drift (the two stores
disagreeing). The plan's suggested fix for drift — widening the bridge's NOT
EXISTS guard to "neither store has it" — was NOT taken: the expense_metadata
INSERT has no ON CONFLICT, so re-attempting an order that already has a receipt
row would duplicate it. Detecting is cheap; a non-idempotent re-write is not.

Unchanged: order_feed 6,265, order_spend AUD 4,895 / $442,651.80. Links 75.
This commit is contained in:
2026-08-12 20:29:42 +10:00
parent 6b9b5fe518
commit 3b5e495ca4
7 changed files with 279 additions and 20 deletions
+67 -17
View File
@@ -205,18 +205,41 @@ function buildWhere(filters: OrderFilters) {
}
/**
* Ledger linkage. Phase 1 reads expense_metadata; phase 2 swaps this lateral
* AND the detail query AND /api/transactions/[id]/order together — moving only
* one of the three leaves the list contradicting the detail page.
* Ledger linkage — phase 2 (board 205).
*
* `order_transaction_links` is the source of truth: it is many-to-many, so a
* BNPL plan is four rows against one order. `expense_metadata` remains as a
* FALLBACK because `jobs/order_transaction_bridge.py` still writes there on
* every hourly tick; until a bridge row is mirrored into links, dropping the
* fallback would make freshly bridged orders read as unpaid.
*
* UNION, not UNION ALL, on transaction_id: after the 0030 backfill the same
* charge is legitimately present in both stores, and counting it twice would
* show "2 charges" on a single-payment order.
*
* All four readers move together — this lateral, the detail query below,
* /api/transactions/[id]/order, and the order_ctx lateral in queries.ts.
* Moving fewer is not a smaller change, it is an inconsistent one: once the
* matcher inserts four links for order_ebay_14-11714-95953 that order still
* has no expense_metadata row, so the detail page would show four instalments
* while the list showed txn_count = 0 and the has_transaction facet put it on
* the wrong side of both filters.
*/
const LINK_LATERAL = `
LEFT JOIN LATERAL (
SELECT count(*)::int AS txn_count,
min(em.matched_transaction_id) AS first_txn_id
FROM expense_metadata em
WHERE em.source = 'order-bridge'
AND (em.source_message_id = f.entity_key
OR em.source_message_id LIKE f.entity_key || '#f%')
SELECT count(*)::int AS txn_count, min(txn_id) AS first_txn_id
FROM (
SELECT l.transaction_id AS txn_id
FROM order_transaction_links l
WHERE l.entity_key = f.entity_key
UNION
SELECT COALESCE(em.matched_transaction_id, em.transaction_id)
FROM expense_metadata em
WHERE em.source = 'order-bridge'
AND (em.source_message_id = f.entity_key
OR em.source_message_id LIKE f.entity_key || '#f%')
AND COALESCE(em.matched_transaction_id, em.transaction_id) IS NOT NULL
) both_stores
) link ON true`;
const FROM_CLAUSE = `
@@ -360,6 +383,12 @@ export interface OrderLinkedTxn {
description: string;
amount: string;
source_message_id: string;
/** 'charge' | 'shipment' | 'instalment' | 'refund' | 'fee'. A plan's legs are
* four 'instalment' rows; a whole-order card charge is one 'charge'. */
leg_kind: string;
/** Human position, "2 of 4" — populated for instalments only. */
leg_index: number | null;
leg_count: number | null;
}
export interface OrderDetail {
@@ -490,14 +519,35 @@ export async function getOrderDetail(entityKey: string): Promise<OrderDetail | n
);
const transactions = await queryRaw<OrderLinkedTxn>(
`SELECT t.id::int AS transaction_id, t.transaction_date, t.description,
t.amount, em.source_message_id
FROM expense_metadata em
JOIN transactions t
ON t.id = COALESCE(em.matched_transaction_id, em.transaction_id)
WHERE em.source = 'order-bridge'
AND (em.source_message_id = $1 OR em.source_message_id LIKE $1 || '#f%')
ORDER BY t.transaction_date`,
// Links first, expense_metadata as fallback, deduped on transaction_id —
// see LINK_LATERAL above for why both stores are read. DISTINCT ON keeps
// the LINK row when a transaction is in both, because only that row knows
// whether the payment was one charge or leg 3 of 4.
// The DISTINCT ON must be ordered by t.id, so the chronological sort the
// page needs goes on the OUTER query — legs read as "1, 2, 3, 4" only if
// they come back by date.
`SELECT * FROM (
SELECT DISTINCT ON (t.id)
t.id::int AS transaction_id, t.transaction_date, t.description,
t.amount, both_stores.source_message_id,
both_stores.leg_kind, both_stores.leg_index, both_stores.leg_count
FROM (
SELECT l.transaction_id, l.leg_kind, l.leg_index::int, l.leg_count::int,
l.entity_key AS source_message_id, 0 AS pref
FROM order_transaction_links l
WHERE l.entity_key = $1
UNION ALL
SELECT COALESCE(em.matched_transaction_id, em.transaction_id),
'charge', NULL::int, NULL::int, em.source_message_id, 1
FROM expense_metadata em
WHERE em.source = 'order-bridge'
AND (em.source_message_id = $1 OR em.source_message_id LIKE $1 || '#f%')
AND COALESCE(em.matched_transaction_id, em.transaction_id) IS NOT NULL
) both_stores
JOIN transactions t ON t.id = both_stores.transaction_id
ORDER BY t.id, both_stores.pref
) deduped
ORDER BY transaction_date, leg_index NULLS FIRST`,
[entityKey]
);
+35 -1
View File
@@ -37,7 +37,16 @@ export interface TransactionRow {
payment_method: string | null;
/** Uber pick-up/drop-off, when this row came from an order receipt. */
order_route: RoutePointRow[] | null;
/** Receipt platform ONLY — it gates the disclosure arrow, so it must stay
* true to "there is a receipt behind this row". */
order_platform: "doordash" | "ubereats" | "uber" | null;
/** Phase 2 (board 205) — set when this transaction is linked to an order.
* 'instalment' means it is one leg of a plan; leg_index/leg_count carry
* "2 of 4", which is what stops one purchase reading as four. */
order_leg_kind: string | null;
order_leg_index: number | null;
order_leg_count: number | null;
order_name: string | null;
// override fields
category_override: string | null;
merchant_override: string | null;
@@ -314,7 +323,17 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
txn_tags.tags,
txn_splits.splits,
order_ctx.route as order_route,
order_ctx.platform as order_platform
-- Deliberately NOT COALESCEd with the link's platform. order_platform
-- gates the receipt disclosure arrow on /transactions, and a BNPL leg
-- has no receipt behind it — filling this in put an arrow on four
-- Afterpay rows that expand to nothing, which is the exact promise the
-- arrow exists to avoid making. The leg fields below carry the sub-line
-- instead.
order_ctx.platform as order_platform,
order_link.leg_kind as order_leg_kind,
order_link.leg_index as order_leg_index,
order_link.leg_count as order_leg_count,
order_link.canonical_name as order_name
FROM transactions t
LEFT JOIN transaction_overrides o ON o.transaction_id = t.id
LEFT JOIN statements s ON s.id = t.statement_id
@@ -323,12 +342,27 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
-- only thing that tells them apart, and it was already stored.
-- Both directions, because a card-settled order has no transaction of its
-- own and points at the statement line instead (I5).
-- Phase 2 (board 205): a BNPL leg has no expense_metadata row of its own,
-- so COALESCE in the link's platform. Without it the four Afterpay debits
-- behind the A$1,599 drone keep an empty sub-line while their order sits
-- one join away. The route column only ever exists on the receipt side.
-- (No backticks in here: this SQL lives in a TS template literal and a
-- backtick ends the string — TS1005 on a line that looks like a comment.)
LEFT JOIN LATERAL (
SELECT em.route, em.platform
FROM expense_metadata em
WHERE em.transaction_id = t.id OR em.matched_transaction_id = t.id
LIMIT 1
) order_ctx ON true
LEFT JOIN LATERAL (
SELECT o.platform, l.leg_kind, l.leg_index::int AS leg_index,
l.leg_count::int AS leg_count, e.canonical_name
FROM order_transaction_links l
LEFT JOIN entities e ON e.entity_key = l.entity_key
LEFT JOIN entity_orders o ON o.entity_id = e.id
WHERE l.transaction_id = t.id
LIMIT 1
) order_link ON true
LEFT JOIN participants p ON p.id = COALESCE(t.owner_id, s.owner_id)
LEFT JOIN transactions src ON src.reconciled_with_id = t.id AND src.statement_id IS NULL
LEFT JOIN trips tr ON tr.id = o.trip_id