test clips where an AI model stated a wrong total. Here code adds up every cent, and the AI never states one. How it was counted
$600
a real roofer's job, sent through PayPal's sandbox with what was fixed on every line, as the customer's PayPal page shows it on the home page.
Why not Housecall Pro or Jobber? Both have a price list of their own. We couldn't find one built from your past PayPal invoices, or one that checks a price you said against them. Housecall Pro price book · Jobber Products & Services list
The roofer's job is real, said out loud by the roofer who did it. The dryer job is a made-up sample. Both were read by ElevenLabs Scribe and Claude Sonnet and sent as PayPal sandbox invoices on 7 Nov 2026. Customer names on the invoices are made up.
Roofer
$600.00
Roof leak repair, valley replacement · PayPal sandbox invoice #0134, re-sent as #0207 on 8 Nov 2026
2 of 2 lines right first time: Labor $400.00 flat, Material $200.00.
Checked before it went out
Asked whether $400 was for the job or by the hour, since three and a half hours was said too. By the hour, the total would have been $1,600.
Pointed out that the $200 material was one lump sum, not itemized.
Appliance tech (sample job)
$472.03
Dryer repair: new part and two hours' labor · PayPal sandbox invoice #0135
2 of 2 lines right first time: the part $329.00 taxed, labor 2 h at $60.00.
Checked before it went out
Tax was said with no rate. Speech spelled Iowa with a lowercase L (“lowa”); it still found Iowa's rates from the state tables, and 7% was picked.
The street address came through split by speech and was read back joined, for a check.
The part wasn't named, so it was listed as “Dryer part” with a note to check.
How “6 of 20” was counted
10 made-up jobs from five trades, each spoken by a text-to-speech voice twice (clean, and with background noise), make 20 clips. Chrome's speech recognition wrote them down and Claude Sonnet returned the line items and a total. On 6 of the 20 the stated total was wrong, though the model's own line items added up right. With code doing the sum, 0 of 20 were wrong. A small, synthetic sample, run on 2026-11-01: it shows this happens, not how often.