siddharthd
df4b875b82
ci / lint-test (push) Failing after 43s
Four things the view could not tell you, all from reading the rows (user,
2026-07-27).
**Which platform.** The parser has always known — it has to, to read the
template — and then discarded it. "Order - Burger Corner" gives no way to know
whether to open DoorDash or Uber Eats for the detail, and restaurants exist on
both. Now stored on expense_metadata and named in the description:
"Order - Burger Corner (Uber Eats)". Migration 0021 recovers it for the 101
backfilled rows from the order_reference shape — DoorDash receipts carry no id
of their own so ingestion synthesises `msg:<message-id>`, Uber carries a real
trip UUID, which makes the discriminator exact.
**Bank said "Manual".** That label is derived, not stored, and "Manual" reads
as "hand-entered, still awaiting a card line to match". A gift-card order has
no card line coming, ever. It now reads "Gift Card", and — the part that
actually mattered — credits joins cash in needsCardMatch(), so these stop
sitting in the pending-reconciliation queue. All 81 were queued against a match
that could not exist.
**Uber line items were never parsed.** 67 of 101 orders had none. Uber itemises
groceries but not restaurant orders, so some of that is genuine; the rest was
simply unread. Its markup is better than DoorDash's — every cell carries a
data-testid with the item's uuid, so qty/title/amount bind by id rather than by
column position. Sold-out items (0.00) are kept: they are why a total is lower
than what was ordered.
**Uber prints pick-up and delivery addresses on every receipt** and they were
thrown away. Captured as `route` [{label, time, address}], de-duplicated
because the template renders the whole block twice for narrow screens. Wording
is kept as printed ("Pick-up" on some receipts, "Pickup" on others) rather than
normalised, so a template change stays visible. This is the same block a *trip*
receipt uses for start and destination — rides are not ingested today, but the
reader will not need changing when they are.
Also stores source_email_subject/from, which order ingestion had left null on
columns that already existed.
Verified against the captured corpus: route on all 6 Uber fixtures, 5/5 items
on the GLOMARK grocery receipt including the sold-out one. Production data
updated by smarthome:docker/scripts/order-presentation-2026-07-27.sql
(81 descriptions, `backfill` tag, re-run clean). `route` and Uber line items
are parsed from here on only — recovering them for already-ingested orders
means re-reading the mail, which I7 idempotency refuses by design.
40 lines
1.9 KiB
SQL
40 lines
1.9 KiB
SQL
-- Order provenance: which platform the receipt came from, and the message it
|
|
-- came from.
|
|
--
|
|
-- The parser has always known the platform (it has to, to read the template)
|
|
-- and then threw it away. Without it a transaction reads "Order - Burger
|
|
-- Corner" with no way to tell whether to look in DoorDash or Uber Eats for the
|
|
-- detail, and no way to answer "how much of this is DoorDash?" at all.
|
|
--
|
|
-- `source_email_subject` / `source_email_from` already existed for the
|
|
-- Paperless expense path and were simply never populated by order ingestion.
|
|
|
|
ALTER TABLE expense_metadata ADD COLUMN IF NOT EXISTS platform text;
|
|
|
|
COMMENT ON COLUMN expense_metadata.platform IS
|
|
'doordash | ubereats | uber — the receipt template the order was read from.';
|
|
|
|
-- Backfill the 101 rows written by the 2026-07-27 backfill. DoorDash receipts
|
|
-- carry no order id of their own, so ingestion synthesises `msg:<message-id>`;
|
|
-- Uber receipts carry a real trip UUID. That is the only surviving
|
|
-- discriminator, and it is exact.
|
|
UPDATE expense_metadata
|
|
SET platform = CASE WHEN order_reference LIKE 'msg:%' THEN 'doordash' ELSE 'ubereats' END
|
|
WHERE platform IS NULL
|
|
AND source = 'email'
|
|
AND paperless_doc_id IS NULL -- exclude the Paperless expense path
|
|
AND order_reference IS NOT NULL;
|
|
|
|
-- Pick-up / delivery stops, as the receipt prints them. Uber puts these on
|
|
-- every order under `Order details`; DoorDash prints no addresses at all, so
|
|
-- this stays '[]' there. Same block a *trip* receipt uses for start and
|
|
-- destination, so this column already fits rides when they come into scope.
|
|
ALTER TABLE expense_metadata ADD COLUMN IF NOT EXISTS route jsonb NOT NULL DEFAULT '[]'::jsonb;
|
|
|
|
COMMENT ON COLUMN expense_metadata.route IS
|
|
'Uber only: [{label, time, address}] — pick-up and delivery stops as printed.';
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_expense_metadata_platform
|
|
ON expense_metadata (platform)
|
|
WHERE platform IS NOT NULL;
|