feat(orders): make an ingested order legible in the transactions view
ci / lint-test (push) Failing after 43s
ci / lint-test (push) Failing after 43s
Four things the view could not tell you, all from reading the rows (user,
2026-07-27).
**Which platform.** The parser has always known — it has to, to read the
template — and then discarded it. "Order - Burger Corner" gives no way to know
whether to open DoorDash or Uber Eats for the detail, and restaurants exist on
both. Now stored on expense_metadata and named in the description:
"Order - Burger Corner (Uber Eats)". Migration 0021 recovers it for the 101
backfilled rows from the order_reference shape — DoorDash receipts carry no id
of their own so ingestion synthesises `msg:<message-id>`, Uber carries a real
trip UUID, which makes the discriminator exact.
**Bank said "Manual".** That label is derived, not stored, and "Manual" reads
as "hand-entered, still awaiting a card line to match". A gift-card order has
no card line coming, ever. It now reads "Gift Card", and — the part that
actually mattered — credits joins cash in needsCardMatch(), so these stop
sitting in the pending-reconciliation queue. All 81 were queued against a match
that could not exist.
**Uber line items were never parsed.** 67 of 101 orders had none. Uber itemises
groceries but not restaurant orders, so some of that is genuine; the rest was
simply unread. Its markup is better than DoorDash's — every cell carries a
data-testid with the item's uuid, so qty/title/amount bind by id rather than by
column position. Sold-out items (0.00) are kept: they are why a total is lower
than what was ordered.
**Uber prints pick-up and delivery addresses on every receipt** and they were
thrown away. Captured as `route` [{label, time, address}], de-duplicated
because the template renders the whole block twice for narrow screens. Wording
is kept as printed ("Pick-up" on some receipts, "Pickup" on others) rather than
normalised, so a template change stays visible. This is the same block a *trip*
receipt uses for start and destination — rides are not ingested today, but the
reader will not need changing when they are.
Also stores source_email_subject/from, which order ingestion had left null on
columns that already existed.
Verified against the captured corpus: route on all 6 Uber fixtures, 5/5 items
on the GLOMARK grocery receipt including the sold-out one. Production data
updated by smarthome:docker/scripts/order-presentation-2026-07-27.sql
(81 descriptions, `backfill` tag, re-run clean). `route` and Uber line items
are parsed from here on only — recovering them for already-ingested orders
means re-reading the mail, which I7 idempotency refuses by design.
This commit is contained in:
@@ -0,0 +1,39 @@
|
||||
-- Order provenance: which platform the receipt came from, and the message it
|
||||
-- came from.
|
||||
--
|
||||
-- The parser has always known the platform (it has to, to read the template)
|
||||
-- and then threw it away. Without it a transaction reads "Order - Burger
|
||||
-- Corner" with no way to tell whether to look in DoorDash or Uber Eats for the
|
||||
-- detail, and no way to answer "how much of this is DoorDash?" at all.
|
||||
--
|
||||
-- `source_email_subject` / `source_email_from` already existed for the
|
||||
-- Paperless expense path and were simply never populated by order ingestion.
|
||||
|
||||
ALTER TABLE expense_metadata ADD COLUMN IF NOT EXISTS platform text;
|
||||
|
||||
COMMENT ON COLUMN expense_metadata.platform IS
|
||||
'doordash | ubereats | uber — the receipt template the order was read from.';
|
||||
|
||||
-- Backfill the 101 rows written by the 2026-07-27 backfill. DoorDash receipts
|
||||
-- carry no order id of their own, so ingestion synthesises `msg:<message-id>`;
|
||||
-- Uber receipts carry a real trip UUID. That is the only surviving
|
||||
-- discriminator, and it is exact.
|
||||
UPDATE expense_metadata
|
||||
SET platform = CASE WHEN order_reference LIKE 'msg:%' THEN 'doordash' ELSE 'ubereats' END
|
||||
WHERE platform IS NULL
|
||||
AND source = 'email'
|
||||
AND paperless_doc_id IS NULL -- exclude the Paperless expense path
|
||||
AND order_reference IS NOT NULL;
|
||||
|
||||
-- Pick-up / delivery stops, as the receipt prints them. Uber puts these on
|
||||
-- every order under `Order details`; DoorDash prints no addresses at all, so
|
||||
-- this stays '[]' there. Same block a *trip* receipt uses for start and
|
||||
-- destination, so this column already fits rides when they come into scope.
|
||||
ALTER TABLE expense_metadata ADD COLUMN IF NOT EXISTS route jsonb NOT NULL DEFAULT '[]'::jsonb;
|
||||
|
||||
COMMENT ON COLUMN expense_metadata.route IS
|
||||
'Uber only: [{label, time, address}] — pick-up and delivery stops as printed.';
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_expense_metadata_platform
|
||||
ON expense_metadata (platform)
|
||||
WHERE platform IS NOT NULL;
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
type MessageMeta,
|
||||
} from "../../lib/order-ingestion";
|
||||
import { EXCLUDE_NON_SPEND } from "../../lib/analytics-sql";
|
||||
import { bankLabel, needsCardMatch } from "../../lib/queries";
|
||||
|
||||
/**
|
||||
* These run against REAL captured receipts, not synthetic fixtures. The earlier
|
||||
@@ -163,7 +164,7 @@ describe("Order ingestion — invariants", () => {
|
||||
expect(b.skipped).toBe("already_ingested");
|
||||
expect(b.metadataId).toBe(a.metadataId);
|
||||
const n = await queryRow<{ c: string }>(
|
||||
`SELECT count(*)::text c FROM transactions WHERE description = 'Order - Mad Mex'`
|
||||
`SELECT count(*)::text c FROM transactions WHERE description = 'Order - Mad Mex (DoorDash)'`
|
||||
);
|
||||
expect(Number(n!.c)).toBe(1);
|
||||
});
|
||||
@@ -379,3 +380,60 @@ describe("owner scoping", () => {
|
||||
expect(visible).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("how an ingested order presents in the app", () => {
|
||||
it("names the platform in the description", async () => {
|
||||
// "Order - Burger Corner" gives no way to know where to look for the
|
||||
// detail, and the same restaurant can be on both platforms.
|
||||
const p = parseOrderHTML(html("dd-01"), meta({ messageId: `desc-${Date.now()}` }));
|
||||
const res = await processOrderIngestion(p);
|
||||
const row = await queryRow<{ description: string }>(
|
||||
`SELECT description FROM transactions WHERE id = $1`, [res.transactionId]
|
||||
);
|
||||
expect(row!.description).toMatch(/\(DoorDash\)$/);
|
||||
});
|
||||
|
||||
it("reads as 'Gift Card', not 'Manual', and stays out of the reconcile queue", async () => {
|
||||
// bank_name is derived — no statement means "Manual", which reads as
|
||||
// "hand-entered, awaiting a card line". A credits order has no card line
|
||||
// coming, ever; 81 of them sat in the queue waiting for one.
|
||||
const p = parseOrderHTML(html("dd-01"), meta({ messageId: `bank-${Date.now()}` }));
|
||||
const res = await processOrderIngestion(p);
|
||||
|
||||
const row = await queryRow<{ bank_name: string }>(
|
||||
`SELECT ${bankLabel()} as bank_name
|
||||
FROM transactions t LEFT JOIN statements s ON s.id = t.statement_id
|
||||
WHERE t.id = $1`,
|
||||
[res.transactionId]
|
||||
);
|
||||
expect(row!.bank_name).toBe("Gift Card");
|
||||
|
||||
const queued = await queryRaw(
|
||||
`SELECT t.id FROM transactions t
|
||||
WHERE t.id = $1 AND t.statement_id IS NULL AND ${needsCardMatch("t")}`,
|
||||
[res.transactionId]
|
||||
);
|
||||
expect(queued).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("records the platform and the message it came from", async () => {
|
||||
const p = parseOrderHTML(
|
||||
html("ue-00"),
|
||||
meta({ messageId: `prov-${Date.now()}`, subject: "Your Wednesday order with Uber Eats", sender: "uber.com" })
|
||||
);
|
||||
const res = await processOrderIngestion(p, {
|
||||
messageId: `prov-${Date.now()}`,
|
||||
subject: "Your Wednesday order with Uber Eats",
|
||||
sender: "uber.com",
|
||||
});
|
||||
const row = await queryRow<{
|
||||
platform: string; source_email_from: string; route: { label: string }[];
|
||||
}>(
|
||||
`SELECT platform, source_email_from, route FROM expense_metadata WHERE id = $1`,
|
||||
[res.metadataId]
|
||||
);
|
||||
expect(row!.platform).toBe("ubereats");
|
||||
expect(row!.source_email_from).toBe("uber.com");
|
||||
expect(row!.route.map((r) => r.label)).toEqual(["Pick-up", "Delivery"]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -70,6 +70,7 @@ describe("validateOrderTotals", () => {
|
||||
service_fee: null, tip: null, discounts: null, total_charged: 10,
|
||||
},
|
||||
line_items: [],
|
||||
route: [],
|
||||
is_family: false,
|
||||
flags: [],
|
||||
...over,
|
||||
@@ -180,3 +181,69 @@ describe("mixed Uber payment (issuer-named card leg)", () => {
|
||||
expect(validateOrderTotals(p).ok).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Uber route (pick-up / delivery)", () => {
|
||||
const uber = (f: string, subject = "Your Wednesday order with Uber Eats") =>
|
||||
parseOrderHTML(html(f), meta({ subject, sender: "uber.com" }));
|
||||
|
||||
it("reads both stops with their times, as printed", () => {
|
||||
const p = uber("ue-00");
|
||||
expect(p.route).toEqual([
|
||||
{ label: "Pick-up", time: "1:20 pm", address: "197 Watton St, Werribee VIC 3030, Australia" },
|
||||
{ label: "Delivery", time: "1:40 pm", address: "19 Lady Penrhyn Dr, Wyndham Vale VIC 3024, Australia" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("de-duplicates the block Uber renders twice", () => {
|
||||
// The receipt emits the whole address section a second time for narrow
|
||||
// screens. Without de-duplication every trip has four stops.
|
||||
expect(uber("ue-00").route).toHaveLength(2);
|
||||
expect(uber("ue-26").route).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("keeps the receipt's own wording rather than normalising it", () => {
|
||||
// Uber is not internally consistent: "Pick-up" on some receipts,
|
||||
// "Pickup" on others. Inventing a canonical spelling would hide that a
|
||||
// template changed.
|
||||
expect(uber("ue-mixed", "Your Friday morning order with Uber Eats").route[0].label).toBe("Pickup");
|
||||
});
|
||||
|
||||
it("works on an international receipt", () => {
|
||||
const p = uber("ue-26");
|
||||
expect(p.route[1].address).toContain("Luzern, Switzerland");
|
||||
});
|
||||
|
||||
it("DoorDash has no route — its receipts carry no addresses", () => {
|
||||
expect(parseOrderHTML(html("dd-01"), meta()).route).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Uber line items", () => {
|
||||
it("itemises a grocery order, binding qty/title/amount by item id", () => {
|
||||
const p = parseOrderHTML(
|
||||
html("ue-09"),
|
||||
meta({ subject: "Your Sunday evening order with Uber Eats", sender: "uber.com" })
|
||||
);
|
||||
expect(p.line_items).toHaveLength(5);
|
||||
expect(p.line_items[0]).toMatchObject({
|
||||
qty: 1,
|
||||
description: "Highland Brewing MILK FULL CREAM U H T 900ML",
|
||||
amount: 440,
|
||||
});
|
||||
// A sold-out item prints 0.00 and is kept: it is why the total is lower
|
||||
// than what was ordered, and dropping it makes the receipt unexplainable.
|
||||
expect(p.line_items.map((i) => i.amount)).toContain(0);
|
||||
});
|
||||
|
||||
it("a restaurant order legitimately has none", () => {
|
||||
// Uber itemises groceries only; a restaurant receipt states a total and
|
||||
// nothing else. Empty here is the receipt, not a parse failure — so it
|
||||
// must not raise no_line_items_parsed either.
|
||||
const p = parseOrderHTML(
|
||||
html("ue-00"),
|
||||
meta({ subject: "Your Wednesday order with Uber Eats", sender: "uber.com" })
|
||||
);
|
||||
expect(p.line_items).toEqual([]);
|
||||
expect(p.flags).not.toContain("no_line_items_parsed");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -72,7 +72,11 @@ export async function POST(req: NextRequest) {
|
||||
|
||||
if (dryRun) return NextResponse.json({ kind: "order", order });
|
||||
|
||||
const result = await processOrderIngestion(order, { messageId: meta.messageId });
|
||||
const result = await processOrderIngestion(order, {
|
||||
messageId: meta.messageId,
|
||||
subject: meta.subject,
|
||||
sender: meta.sender,
|
||||
});
|
||||
return NextResponse.json({
|
||||
kind: "order",
|
||||
order_reference: order.order_reference,
|
||||
|
||||
@@ -12,6 +12,24 @@ export const CUTOVER_DATE = "2026-01-09";
|
||||
*/
|
||||
export const DEFAULT_OWNER_ID = 1;
|
||||
|
||||
/** Human labels for the platform a receipt came from. */
|
||||
export const PLATFORM_LABEL: Record<ParsedOrder["platform"], string> = {
|
||||
doordash: "DoorDash",
|
||||
ubereats: "Uber Eats",
|
||||
uber: "Uber",
|
||||
};
|
||||
|
||||
/**
|
||||
* Transaction description.
|
||||
*
|
||||
* The merchant alone ("Order - Burger Corner") does not say where to go and
|
||||
* look for the detail, and there are restaurants on both platforms. The
|
||||
* platform is the one thing the parser always knows and used to discard.
|
||||
*/
|
||||
export function orderDescription(platform: ParsedOrder["platform"], merchant: string): string {
|
||||
return `Order - ${merchant} (${PLATFORM_LABEL[platform]})`;
|
||||
}
|
||||
|
||||
export interface IngestResult {
|
||||
transactionId: number | null;
|
||||
metadataId: number | null;
|
||||
@@ -93,7 +111,13 @@ async function ensureTag(name: string): Promise<number> {
|
||||
*/
|
||||
export async function processOrderIngestion(
|
||||
order: ParsedOrder,
|
||||
options: { messageId?: string; backfillMode?: boolean; ownerId?: number } = {}
|
||||
options: {
|
||||
messageId?: string;
|
||||
backfillMode?: boolean;
|
||||
ownerId?: number;
|
||||
subject?: string;
|
||||
sender?: string;
|
||||
} = {}
|
||||
): Promise<IngestResult> {
|
||||
const flags = [...order.flags];
|
||||
const day = order.order_datetime.slice(0, 10);
|
||||
@@ -159,7 +183,7 @@ export async function processOrderIngestion(
|
||||
RETURNING id`,
|
||||
[
|
||||
day,
|
||||
`Order - ${order.merchant_name}`,
|
||||
orderDescription(order.platform, order.merchant_name),
|
||||
creditsAmount,
|
||||
// No FX rate is available at ingest, so amount_aud is left NULL for
|
||||
// foreign orders rather than asserting a conversion we cannot make.
|
||||
@@ -194,8 +218,9 @@ export async function processOrderIngestion(
|
||||
`INSERT INTO expense_metadata (
|
||||
transaction_id, source, source_message_id, order_reference, line_items,
|
||||
subtotal, amount, merchant_normalized, transaction_date,
|
||||
card_last4, currency, flags, reconciled_at
|
||||
) VALUES ($1,'email',$2,$3,$4::jsonb,$5,$6,$7,$8,$9,$10,$11::jsonb,$12)
|
||||
card_last4, currency, flags, reconciled_at,
|
||||
platform, source_email_subject, source_email_from, route
|
||||
) VALUES ($1,'email',$2,$3,$4::jsonb,$5,$6,$7,$8,$9,$10,$11::jsonb,$12,$13,$14,$15,$16::jsonb)
|
||||
RETURNING id`,
|
||||
[
|
||||
transactionId,
|
||||
@@ -210,6 +235,10 @@ export async function processOrderIngestion(
|
||||
order.currency,
|
||||
JSON.stringify(flags),
|
||||
pending ? null : new Date().toISOString(),
|
||||
order.platform,
|
||||
options.subject ?? null,
|
||||
options.sender ?? null,
|
||||
JSON.stringify(order.route ?? []),
|
||||
]
|
||||
);
|
||||
|
||||
@@ -239,9 +268,10 @@ export async function reconcilePendingOrders(): Promise<{
|
||||
merchant_normalized: string;
|
||||
card_last4: string | null;
|
||||
currency: string | null;
|
||||
platform: ParsedOrder["platform"] | null;
|
||||
}>(
|
||||
`SELECT id, order_reference, amount::text, transaction_date::text,
|
||||
merchant_normalized, card_last4, currency
|
||||
merchant_normalized, card_last4, currency, platform
|
||||
FROM expense_metadata
|
||||
WHERE transaction_id IS NULL
|
||||
AND reconciled_at IS NULL
|
||||
@@ -255,7 +285,7 @@ export async function reconcilePendingOrders(): Promise<{
|
||||
const total = Number(row.amount);
|
||||
const probe: ParsedOrder = {
|
||||
order_reference: row.order_reference,
|
||||
platform: "doordash",
|
||||
platform: row.platform ?? "doordash",
|
||||
merchant_name: row.merchant_normalized,
|
||||
order_datetime: `${row.transaction_date}T00:00:00Z`,
|
||||
currency: row.currency || "AUD",
|
||||
@@ -265,6 +295,7 @@ export async function reconcilePendingOrders(): Promise<{
|
||||
service_fee: null, tip: null, discounts: null, total_charged: total,
|
||||
},
|
||||
line_items: [],
|
||||
route: [],
|
||||
is_family: false,
|
||||
flags: [],
|
||||
};
|
||||
@@ -291,7 +322,7 @@ export async function reconcilePendingOrders(): Promise<{
|
||||
payment_method, merchant_name, merchant_normalized, transaction_type, owner_id
|
||||
) VALUES ($1,$2,$3,$3,$5,'credits',$4,$4,'debit',$6)
|
||||
RETURNING id`,
|
||||
[row.transaction_date, `Order - ${row.merchant_normalized}`, remainder, row.merchant_normalized, category, DEFAULT_OWNER_ID]
|
||||
[row.transaction_date, orderDescription(row.platform ?? "doordash", row.merchant_normalized), remainder, row.merchant_normalized, category, DEFAULT_OWNER_ID]
|
||||
);
|
||||
txnId = txn!.id;
|
||||
created++;
|
||||
|
||||
+95
-1
@@ -20,6 +20,18 @@ export interface LineItem {
|
||||
options?: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* A stop on the receipt's map: pick-up, delivery, or (for a trip) the ride's
|
||||
* start and end. Uber prints these for every order under `Order details`.
|
||||
*/
|
||||
export interface RoutePoint {
|
||||
/** "Pick-up" / "Delivery" — whatever the receipt itself calls it. */
|
||||
label: string;
|
||||
/** Local time as printed, e.g. "1:20 pm". No date; the receipt gives none. */
|
||||
time: string | null;
|
||||
address: string;
|
||||
}
|
||||
|
||||
export interface PaymentBreakdown {
|
||||
credits_amount: number | null;
|
||||
card_amount: number | null;
|
||||
@@ -47,6 +59,8 @@ export interface ParsedOrder {
|
||||
payment: PaymentBreakdown;
|
||||
totals: OrderTotals;
|
||||
line_items: LineItem[];
|
||||
/** Uber only. Empty for DoorDash, whose receipts carry no addresses. */
|
||||
route: RoutePoint[];
|
||||
is_family: boolean;
|
||||
flags: string[];
|
||||
}
|
||||
@@ -116,6 +130,81 @@ function tdPairValue(html: string, label: string): number | null {
|
||||
return m ? money(m[1]) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Uber itemises only *grocery* orders — a restaurant receipt states a total and
|
||||
* nothing else, which is why 67 of the 101 backfilled orders have no items.
|
||||
* When it does itemise, the markup is far better than DoorDash's: every cell
|
||||
* carries a `data-testid` naming its role and the item's own uuid, so quantity,
|
||||
* title and amount can be bound to each other by id rather than by position.
|
||||
*/
|
||||
function parseUberLineItems(html: string): LineItem[] {
|
||||
const items: LineItem[] = [];
|
||||
const titleRe =
|
||||
/data-testid="shoppingCart_item_title_([0-9a-f-]+)"[^>]*>([\s\S]*?)<\/td>/gi;
|
||||
|
||||
for (const m of html.matchAll(titleRe)) {
|
||||
const [, id, rawTitle] = m;
|
||||
const description = collapse(decodeEntities(stripTags(rawTitle)));
|
||||
if (!description) continue;
|
||||
|
||||
const qtyM = html.match(
|
||||
new RegExp(`data-testid="shoppingCart_item_quantity_${id}"[^>]*>\\s*(\\d+)\\s*<`, "i")
|
||||
);
|
||||
const amtM = html.match(
|
||||
new RegExp(
|
||||
`data-testid="shoppingCart_item_amount_${id}"[^>]*>([\\s\\S]*?)<\\/td>`,
|
||||
"i"
|
||||
)
|
||||
);
|
||||
const amtText = amtM ? collapse(decodeEntities(stripTags(amtM[1]))) : "";
|
||||
const amtNum = amtText.match(/(-?[\d,]+\.\d{2})/);
|
||||
|
||||
items.push({
|
||||
qty: qtyM ? parseInt(qtyM[1], 10) : 1,
|
||||
description,
|
||||
// A sold-out item prints 0.00 and is genuinely part of the order — it
|
||||
// explains a total that does not match what was asked for. Keep it.
|
||||
amount: amtNum ? money(amtNum[1]) : 0,
|
||||
});
|
||||
}
|
||||
return items;
|
||||
}
|
||||
|
||||
/**
|
||||
* Uber's `Order details` block, anchored on `data-testid="address_point_N_*"`.
|
||||
*
|
||||
* The template repeats the whole block twice (once hidden for narrow screens),
|
||||
* so the same stop appears more than once and has to be de-duplicated. This is
|
||||
* the same markup a *trip* receipt uses for its start and destination — rides
|
||||
* are not ingested today, but the reader will not need changing when they are.
|
||||
*/
|
||||
function parseUberRoute(html: string): RoutePoint[] {
|
||||
const seen = new Set<string>();
|
||||
const points: RoutePoint[] = [];
|
||||
|
||||
const labelRe = /data-testid="address_point_(\d+)_time"[^>]*>([\s\S]*?)<\/td>/gi;
|
||||
for (const m of html.matchAll(labelRe)) {
|
||||
const [, idx, rawLabel] = m;
|
||||
const addrM = html.match(
|
||||
new RegExp(`data-testid="address_point_${idx}_address"[^>]*>([\\s\\S]*?)<\\/td>`, "i")
|
||||
);
|
||||
if (!addrM) continue;
|
||||
|
||||
const address = collapse(decodeEntities(stripTags(addrM[1])));
|
||||
// "1:20 pm - Pick-up" — time and label share one cell.
|
||||
const combined = collapse(decodeEntities(stripTags(rawLabel)));
|
||||
const split = combined.match(/^(.*?)\s+-\s+(.*)$/);
|
||||
const time = split ? split[1] : null;
|
||||
const label = split ? split[2] : combined;
|
||||
|
||||
const key = `${label}|${time}|${address}`;
|
||||
if (!address || seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
points.push({ label, time, address });
|
||||
}
|
||||
return points;
|
||||
}
|
||||
|
||||
function parseDoorDashLineItems(html: string): LineItem[] {
|
||||
// <td width="10%">1x</td><td width="75%"><b>Name</b> (Cat)<br><font>• Opt</font>…</td><td width="15%">$22.10</td>
|
||||
const re =
|
||||
@@ -397,11 +486,15 @@ export function parseOrderHTML(html: string, meta: MessageMeta): ParsedOrder {
|
||||
// ---- line items ----------------------------------------------------------
|
||||
// Uber Eats receipts carry no itemisation (verified across 29 real mails).
|
||||
const line_items =
|
||||
platform === "doordash" ? parseDoorDashLineItems(clean) : [];
|
||||
platform === "doordash" ? parseDoorDashLineItems(clean) : parseUberLineItems(clean);
|
||||
if (platform === "doordash" && line_items.length === 0) {
|
||||
flags.push("no_line_items_parsed");
|
||||
}
|
||||
|
||||
// Uber prints addresses on every receipt; DoorDash prints none at all, so an
|
||||
// empty route there is expected rather than a parse failure.
|
||||
const route = platform === "doordash" ? [] : parseUberRoute(clean);
|
||||
|
||||
const currency =
|
||||
explicitCurrency ||
|
||||
(/\b(NZD|USD|LKR|CHF|EUR|GBP|SGD|INR)\b/.test(text)
|
||||
@@ -417,6 +510,7 @@ export function parseOrderHTML(html: string, meta: MessageMeta): ParsedOrder {
|
||||
payment,
|
||||
totals,
|
||||
line_items,
|
||||
route,
|
||||
is_family,
|
||||
flags,
|
||||
};
|
||||
|
||||
+51
-19
@@ -26,7 +26,7 @@ export interface TransactionRow {
|
||||
principal_amount: number | null;
|
||||
interest_amount: number | null;
|
||||
// How it was paid (migration 0016). NULL = unknown, treated as reconcilable.
|
||||
// 'cash' is excluded from reconciliation — see notCash().
|
||||
// 'cash' and 'credits' are excluded from reconciliation — see needsCardMatch().
|
||||
payment_method: string | null;
|
||||
// override fields
|
||||
category_override: string | null;
|
||||
@@ -133,17 +133,24 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
|
||||
params.push(filters.categories);
|
||||
}
|
||||
if (filters.bank_names?.length) {
|
||||
// "Manual" and "Gift Card" are not banks — they are the two shapes a
|
||||
// statement-less row can take, and bankLabel() decides which. The filter
|
||||
// has to split on the same condition or the chip selects nothing.
|
||||
const hasManual = filters.bank_names.includes("Manual");
|
||||
const bankList = filters.bank_names.filter((b) => b !== "Manual");
|
||||
if (hasManual && bankList.length > 0) {
|
||||
conditions.push(`(t.statement_id IS NULL OR s.bank_name = ANY($${paramIdx++}::text[]))`);
|
||||
params.push(bankList);
|
||||
} else if (hasManual) {
|
||||
conditions.push(`t.statement_id IS NULL`);
|
||||
} else {
|
||||
conditions.push(`s.bank_name = ANY($${paramIdx++}::text[])`);
|
||||
const hasGiftCard = filters.bank_names.includes("Gift Card");
|
||||
const bankList = filters.bank_names.filter((b) => b !== "Manual" && b !== "Gift Card");
|
||||
const alternatives: string[] = [];
|
||||
if (hasManual) {
|
||||
alternatives.push(`(t.statement_id IS NULL AND t.payment_method IS DISTINCT FROM 'credits')`);
|
||||
}
|
||||
if (hasGiftCard) {
|
||||
alternatives.push(`(t.statement_id IS NULL AND t.payment_method = 'credits')`);
|
||||
}
|
||||
if (bankList.length > 0) {
|
||||
alternatives.push(`s.bank_name = ANY($${paramIdx++}::text[])`);
|
||||
params.push(bankList);
|
||||
}
|
||||
conditions.push(`(${alternatives.join(" OR ")})`);
|
||||
}
|
||||
if (filters.tag_ids?.length) {
|
||||
const noTags = filters.tag_ids.includes("untagged");
|
||||
@@ -210,7 +217,7 @@ export async function getTransactions(ownerId: number, filters: TransactionFilte
|
||||
o.category_override, o.merchant_normalized as merchant_override, o.notes, o.my_share_percent,
|
||||
COALESCE(o.category_override, t.category) as effective_category,
|
||||
COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) as effective_merchant,
|
||||
COALESCE(s.bank_name, 'Manual') as bank_name,
|
||||
${bankLabel()} as bank_name,
|
||||
COALESCE(s.currency, 'AUD') as currency,
|
||||
-- My share, resolved the same way analytics does it (see myShare in
|
||||
-- analytics-sql.ts): explicit split row, then override, then whatever is
|
||||
@@ -294,7 +301,7 @@ export async function getTransactionById(id: number) {
|
||||
o.category_override, o.merchant_normalized as merchant_override, o.notes, o.my_share_percent,
|
||||
COALESCE(o.category_override, t.category) as effective_category,
|
||||
COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) as effective_merchant,
|
||||
COALESCE(s.bank_name, 'Manual') as bank_name,
|
||||
${bankLabel()} as bank_name,
|
||||
COALESCE(t.owner_id, s.owner_id) as owner_id,
|
||||
p.name as owner_name
|
||||
FROM transactions t
|
||||
@@ -376,12 +383,20 @@ export async function getMerchantSuggestions(search: string) {
|
||||
}
|
||||
|
||||
export async function getBankNames() {
|
||||
const [bankRows, manualCount] = await Promise.all([
|
||||
const [bankRows, statementless] = await Promise.all([
|
||||
queryRaw<{ bank_name: string }>(`SELECT DISTINCT bank_name FROM statements ORDER BY bank_name`),
|
||||
queryRaw<{ count: number }>(`SELECT COUNT(*)::int as count FROM transactions WHERE statement_id IS NULL`),
|
||||
queryRaw<{ label: string }>(
|
||||
`SELECT DISTINCT ${bankLabel("t", "s")} as label
|
||||
FROM transactions t LEFT JOIN statements s ON s.id = t.statement_id
|
||||
WHERE t.statement_id IS NULL`
|
||||
),
|
||||
]);
|
||||
const banks = bankRows.map((r) => r.bank_name);
|
||||
if (manualCount[0]?.count > 0) banks.push("Manual");
|
||||
// Order matters for the filter chips: real banks first, then the
|
||||
// statement-less kinds, in a stable order rather than whatever the DB returns.
|
||||
for (const label of ["Manual", "Gift Card"]) {
|
||||
if (statementless.some((r) => r.label === label)) banks.push(label);
|
||||
}
|
||||
return banks;
|
||||
}
|
||||
|
||||
@@ -533,8 +548,25 @@ export async function batchInsertCSVTransactions(
|
||||
* transaction accounts are imported, and NULL means unknown — both stay
|
||||
* candidates, which preserves the behaviour of every pre-existing row.
|
||||
*/
|
||||
export const notCash = (alias = "t") =>
|
||||
`(${alias}.payment_method IS NULL OR ${alias}.payment_method <> 'cash')`;
|
||||
/**
|
||||
* Payment methods that can still be matched against a card statement line.
|
||||
*
|
||||
* Cash never appears on one. Neither does a credits-funded delivery order: the
|
||||
* gift card already paid it, so there is no card leg coming, ever. Leaving
|
||||
* those in the queue meant 81 orders sat in "pending reconciliation" waiting
|
||||
* for a match that could not exist (user, 2026-07-27).
|
||||
*/
|
||||
export const needsCardMatch = (alias = "t") =>
|
||||
`(${alias}.payment_method IS NULL OR ${alias}.payment_method NOT IN ('cash', 'credits'))`;
|
||||
|
||||
/**
|
||||
* Bank label for a transaction. A row with no statement was not imported from
|
||||
* one, and the label has to say *why*: "Manual" reads as "hand-entered, still
|
||||
* awaiting a card line", which is wrong for a gift-card order — nothing is
|
||||
* awaited. `s` must be the statements alias in scope.
|
||||
*/
|
||||
export const bankLabel = (t = "t", s = "s") =>
|
||||
`COALESCE(${s}.bank_name, CASE WHEN ${t}.payment_method = 'credits' THEN 'Gift Card' ELSE 'Manual' END)`;
|
||||
|
||||
export interface PotentialMatch {
|
||||
id: number;
|
||||
@@ -578,7 +610,7 @@ export async function getPendingReconciliations(ownerId: number): Promise<Manual
|
||||
WHERE ts.transaction_id = t.id
|
||||
) txn_splits ON true
|
||||
WHERE t.statement_id IS NULL AND t.owner_id = $1 AND t.reconciled_with_id IS NULL
|
||||
AND ${notCash("t")}
|
||||
AND ${needsCardMatch("t")}
|
||||
ORDER BY t.transaction_date DESC, t.row_index ASC`,
|
||||
[ownerId]
|
||||
);
|
||||
@@ -620,7 +652,7 @@ export async function getPendingReconciliations(ownerId: number): Promise<Manual
|
||||
WHERE m.statement_id IS NULL
|
||||
AND m.owner_id = $1
|
||||
AND m.reconciled_with_id IS NULL
|
||||
AND ${notCash("m")}
|
||||
AND ${needsCardMatch("m")}
|
||||
AND COALESCE(t.owner_id, s.owner_id) = $1
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM transactions mt WHERE mt.reconciled_with_id = t.id
|
||||
@@ -687,7 +719,7 @@ export async function getSharedTransactions(ownerId: number, tagIds?: number[],
|
||||
o.category_override, o.merchant_normalized as merchant_override, o.notes,
|
||||
COALESCE(o.category_override, t.category) as effective_category,
|
||||
COALESCE(o.merchant_normalized, t.merchant_normalized, t.merchant_name) as effective_merchant,
|
||||
COALESCE(s.bank_name, 'Manual') as bank_name,
|
||||
${bankLabel()} as bank_name,
|
||||
COALESCE(t.owner_id, s.owner_id) as owner_id,
|
||||
p_owner.name as owner_name,
|
||||
COALESCE(src.created_at, t.created_at) as created_at,
|
||||
|
||||
Reference in New Issue
Block a user