intake: the word "Image" is not a receipt, and the prompt must not be data
ci / lint-test (push) Successful in 52s

Batch 3d2ffbe8 arrived as one text part whose entire content was the five
characters `Image` — an iOS Shortcut coerced the shared photo to text. The
model was asked anyway and answered out of its prompt: Burgertory
(Braybrook), two "The Pride" lines at $22.80, a $41.15 total. Nothing on it
existed. Burgertory was in ORDER_ITEM_PROMPT as the example of how to print
a branch in parentheses. Only the missing date kept it out of the matcher;
the invented header and items were stored on the batch regardless.

Three layers, none of them a model:

1. textCouldBeReceipt() — deterministic, before any model call. A TEXT part
   with fewer than two words, no digit, or under 40 characters and no money
   figure cannot be a receipt. Length is NOT the test on its own: `TOTAL
   $17.00` is twelve characters and is exactly what this door accepts. A
   rejected part is dropped like an unreadable file type, so one junk part
   beside a good screenshot still goes through; if nothing usable is left
   the card quotes what the phone actually sent. PDFs and images untouched.

2. The prompt names no real shop — merchant examples are
   PROMPT_PLACEHOLDER_MERCHANTS, exported from intake-rules so the prompt
   and the guard cannot drift — and it is told what to return when the input
   is not a receipt at all.

3. confabulationReason() — the belt. Reachable where the pre-check cannot
   look: a PDF that renders to a scrap of junk still reaches the model. Also
   rejects any answer whose merchant IS a placeholder.

All three file `unsupported`, never `needs_header`. A confabulated header is
not a partial one, and the needs_header path stores what it is given.

Same shape as the ingestion engine's 2026-08-05 empty-body defect: a stage
that can be handed nothing has to say so, because the model downstream will
always answer.

Tests: 16 unit cases over the two pure functions and the prompt; three
integration cases — `Image` is declined with zero model calls and nothing
but `unsupported` on the batch row, a junk part beside a real receipt still
enriches, and a junk PDF is unsupported rather than needs_header. The first
and third fail on the tree before this commit.
This commit is contained in:
2026-08-22 11:46:05 +10:00
parent d88ca2a24c
commit 0823244e4a
6 changed files with 335 additions and 5 deletions
+106 -1
View File
@@ -3,17 +3,21 @@ import { readFileSync } from "fs";
import { resolve } from "path";
import {
classifyInput,
confabulationReason,
GROCERY_PATTERN,
merchantKey,
merchantMatches,
platformForSource,
PROMPT_PLACEHOLDER_MERCHANTS,
quoteShared,
reconcileItems,
resolveMerchantKind,
textCouldBeReceipt,
toLineItems,
validateHeader,
type ExtractedOrder,
} from "../../lib/intake-rules";
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor } from "../../lib/intake-extract";
import { buildMessageContent, extractPdfText, parseExtraction, EXTRACT_RESPONSE_FORMAT, ExtractionError, liteLLMExtractor, ORDER_ITEM_PROMPT } from "../../lib/intake-extract";
import { askCard, candidatesCard, enrichedCard, type Block } from "../../lib/intake-blocks";
/**
@@ -298,3 +302,104 @@ describe("liteLLMExtractor — a finish_reason that is present must be `stop`",
await expect(extractor("tool_calls")([{ kind: "text", text: "x", name: "x" }])).rejects.toThrow(/tool_calls/);
});
});
describe("textCouldBeReceipt — nothing that cannot be a receipt reaches the model", () => {
const why = (t: string | null | undefined) => {
const r = textCouldBeReceipt(t);
return r.ok ? null : r.reason;
};
it("the live defect: the phone sent the WORD `Image`, not the image", () => {
expect(textCouldBeReceipt("Image").ok).toBe(false);
expect(why("Image")).toMatch(/single word/);
expect(why("Image\n")).toMatch(/single word/);
});
it("a filename is not a receipt, digits in it notwithstanding", () => {
expect(why("IMG_0042.jpeg")).toMatch(/single word/);
expect(why("Receipt_19Aug2026.pdf")).toMatch(/single word/);
});
it("empty, blank and whitespace-only", () => {
expect(why("")).toMatch(/empty/);
expect(why(" \n\t ")).toMatch(/empty/);
expect(why(null)).toMatch(/empty/);
expect(why(undefined)).toMatch(/empty/);
});
it("prose with no numbers cannot be a receipt", () => {
expect(why("Shared from Photos")).toMatch(/no numbers/);
expect(why("here is the receipt for dinner last night, sorry it took so long")).toMatch(/no numbers/);
});
it("short but with a printed amount goes through — this door accepts a pasted line", () => {
expect(textCouldBeReceipt("TOTAL $17.00").ok).toBe(true);
expect(textCouldBeReceipt("Total 17.00").ok).toBe(true);
expect(textCouldBeReceipt("Gesamt 12,50 EUR").ok).toBe(true);
});
it("short with digits but no money figure does not — a screenshot name is not a purchase", () => {
expect(why("Screenshot 2026 08 22")).toMatch(/too short/);
expect(why("2 items")).toMatch(/too short/);
});
it("a real receipt body passes", () => {
expect(
textCouldBeReceipt("Burgertory (Werribee)\n19 Aug 2026\n1x Michael 14.00\n1x Joel 14.00\nSubtotal 28.00\nTotal 22.40").ok
).toBe(true);
});
it("quoteShared flattens and truncates what goes on the card", () => {
expect(quoteShared("Image")).toBe("Image");
expect(quoteShared(" a \n b ")).toBe("a b");
expect(quoteShared("x".repeat(200))).toHaveLength(80);
expect(quoteShared("x".repeat(200)).endsWith("…")).toBe(true);
});
});
describe("confabulationReason — the belt to that pre-check", () => {
const nothing = { textParts: ["Image"], hasImages: false };
const readable = { textParts: ["Total $22.40 Burgertory (Werribee) 2026-08-19"], hasImages: false };
const order = { merchant: "Burgertory (Braybrook)", items: [{ description: "The Pride" }] };
it("flags an order extracted from nothing readable", () => {
expect(confabulationReason(nothing, order)).toMatch(/nothing shared could be read/);
});
it("does not flag one extracted from something readable", () => {
expect(confabulationReason(readable, order)).toBeNull();
});
it("does not flag an image share — the pixels are what we asked for", () => {
expect(confabulationReason({ textParts: [], hasImages: true }, order)).toBeNull();
});
it("an empty answer to an unreadable share is honest, not a confabulation", () => {
expect(confabulationReason(nothing, { merchant: null, items: [] })).toBeNull();
});
it("the prompt's own placeholder coming back as a merchant is always a confabulation", () => {
for (const p of PROMPT_PLACEHOLDER_MERCHANTS) {
expect(confabulationReason(readable, { merchant: p, items: [] })).toMatch(/prompt's own example/);
expect(confabulationReason({ textParts: [], hasImages: true }, { merchant: p, items: [] })).toMatch(/prompt's own example/);
}
});
it("a real merchant that happens to be named in the prompt's platform list is NOT flagged", () => {
expect(confabulationReason(readable, { merchant: "Coles (Wyndham Vale)", items: [] })).toBeNull();
expect(confabulationReason(readable, { merchant: "Woolworths", items: [] })).toBeNull();
});
});
describe("the prompt carries no real merchant to echo", () => {
it("names no shop it could hand back as data", () => {
expect(ORDER_ITEM_PROMPT).not.toMatch(/Burgertory/);
expect(ORDER_ITEM_PROMPT).not.toMatch(/Wyndham Vale/);
expect(ORDER_ITEM_PROMPT).toContain(PROMPT_PLACEHOLDER_MERCHANTS[0]);
});
it("tells the model what to return when the input is not a receipt", () => {
expect(ORDER_ITEM_PROMPT).toMatch(/NOT a receipt/);
expect(ORDER_ITEM_PROMPT).toMatch(/Do not construct a plausible order/);
});
});