Ingestion now runs on the rebuilt parser. Three substantive changes.
Deferred card reconciliation. A 'MasterCard 8032 and/or credits' receipt never
states the split, but the card leg lands on the statement — Subway's $29.08
order shows $13.06 on 8032, so $16.02 was credits. For a live order that
statement is weeks away, so the split cannot be settled at ingest time. Such
orders are now parked with provenance and no transaction, and
reconcilePendingOrders() resolves them once the statement arrives. Backfill
takes the same path and resolves immediately. Migration 0019 adds the columns
that make an order resumable; applied to personal_test only, prod untouched.
Payment detection bug, found by the new tests: the old regex delimited the
'Paid with' line on a double space, which whitespace collapsing removes. Every
card and mixed receipt fell through to the credits branch — the Woolworths
receipt booked $60.93 of credits spend that never happened.
Category resolution reversed deliberately. Correction 1 said never default to
dining; the implementation of that sent everything unrecognised to 'other', and
knowing six merchants meant Carl's Jr, Taco Bell, Chilli India, Oporto, Schnitz
and Souvlaki GR all landed there. Grocers are an enumerable set and restaurants
are not, so match groceries explicitly and let the residual be dining.
Tests rebuilt on real captured receipts; the synthetic fixtures are deleted.
60 unit + 41 integration green on three consecutive runs.
The previous parser was written against synthetic fixtures shaped to match the
code. It invented a table layout DoorDash does not send, generated
order_reference from Math.random(), read the order date from a 'Date:' string
present in no real message, and detected [Family] by searching the body for the
substring 'family'. Its tests passed because the fixtures were built to satisfy
it. Against 36 real DoorDash and 29 real Uber Eats receipts it does not work.
Rebuilt from the real corpus. 63 of 65 now parse and validate; the 2 rejected
are correctly rejected — one is an order-adjustment notice and one a refund,
neither of which is a receipt.
Corrects an inherited diagnosis: the Mad Mex 'Discounts -$24.09' was recorded
as an HTML-flattening artefact masking a 'true discount of $9.45'. Parsing the
table cells structurally returns the same figures and no $9.45 exists anywhere
in the message — DoorDash genuinely prints a Discounts line that equals subtotal
plus service fee, and the components fail to reconcile on 32 of 36 receipts. So
the breakdown is stored as provenance and never gated on; validation instead
cross-checks the two independently stated totals and the payment line, which is
the number that becomes money.
Real-world cases the corpus forced, none of which were in the spec: [Family]
orders are LKR purchases for family in Sri Lanka (reading them as dollars
inflates ~200x), Swiss orders arrive in CHF, grocery 'Final receipt' mails carry
no Total Charged row, and a declined payment is printed alongside the successful
retry and must be skipped or it records money that never moved.
order_reference now comes from the Uber order UUID embedded in the body, or the
provider message id where DoorDash supplies no order id at all — never random,
so re-ingestion is genuinely idempotent.
siddharthd