I run a small self-hosted tool that prices second-hand items for eBay. You photograph the thing, it works out what it is, gathers evidence about what it is worth, and proposes an asking price. It has been running for a fortnight against my own stock, and it keeps teaching me the same lesson in new clothes.
Here is Friday's version of it, with the numbers as they happened.
The item
A Raspberry Pi case. Aluminium, fan, power button, the kind that sells new on Amazon for £26.99 and has done for a year.
| When | What the desk asked | Why |
|---|---|---|
| Tuesday | £79.32 | it thought the case was an expansion card |
| Friday morning | £35.39 | identity fixed, priced from nine live asking prices |
| Friday afternoon | £22.94 | see below |
Tuesday's number was an identification bug and a story for another day. Friday morning's number is the interesting one, because nothing was broken. Every stage did what it was written to do, and the result was a used case asked at 31% above the price of a new one.
Why £35.39 was "correct"
The desk prices from evidence in a fixed order. Completed sales of the same product first, because a sale is the only reading where money changed hands. If there are none, live asking prices, with a warning that nobody has paid them. Alongside both sits a retail anchor: what the thing costs new today, scaled down by a used-goods factor. That anchor is the one signal independent of every resale price, so it can catch the others when they drift.
On Friday it had no completed sales for this case. The nine live asks ran from £35 to £240, and the desk took 15% under the cheaper half, which is where £35.39 came from. And the retail anchor?
retail-anchor discovery unavailable: no results and these engines were unresponsive: google cse
The anchor's only source was a price-comparison site, and its only route to that site was a site: search through a self-hosted metasearch instance. The engine behind that search had been rate-limited since breakfast. So the one signal built to catch exactly this failure sat out every research run of the day, and it said so, politely, in a log line nobody reads before a listing goes up.
I had the Amazon page open in another tab. There was no way to hand the number over.
Fix one: let the person type the number
The arithmetic half of the anchor is reliable. The discovery half is not. So the fragile half can be done by a person: put new on amazon £26.99 in the item's notes and the anchor uses it directly and skips the search.
Two traps in something that small.
The word "new" already meant something. The notes parser reads condition words, and new tells the comp filter to discard every used sale, on the theory that a new item should not be priced off used ones. "New on amazon £26.99" is a fact about the shop price, not a claim about this item. The price phrase has to be read and removed before the condition words are read, or typing the retail price silently changes what the item is.
Anchor on the keyword nearest the pound sign. "brand new, sealed, rrp £30" must attach the price to rrp, not to new. My first pattern matched from the first keyword and swallowed "new, sealed, rrp £30" whole, so the item stopped being new. A negative lookahead in the gap between keyword and £ fixed it, and a test now says so.
Fix two: a used one cannot cost more than a new one
With the typed price in, the anchor produced its number, and the crosscheck did what it was built to do:
crosscheck the price sources disagree with each other (live asks £42.29, retail anchor £12.62)
Then it marked the price "thin" and left it at £35.39.
A warning does not stop a listing. The anchor was a signal, one vote among several, and the design was careful never to let a single source dictate. Fine. But "a used item cannot ask more than a new one" is not a signal. It is a fact about how buyers behave, and nothing enforced it.
So now there is one hard line: once the new price is known, the ask is capped at that price times the condition multiplier. 85% for "good", 92% for "like new", 70% for "fair". Not the category factor, which is an estimate of what used goods realise and belongs to the anchor. Just the line above which the listing is worse than buying new.
£35.39 became £22.94.
Fix three: read Amazon, since that is where the price was
The comparison site indexes electronics well and everything else badly. My stock includes a tin of malt drink powder, a newborn bottle set and a graphing calculator, none of which it had ever anchored. Amazon sells all of it.
Before writing a scraper I measured three things.
- Amazon's
robots.txtdoes not disallow the search page or product pages forUser-agent: *. It disallows sub-paths I never touch. - The search page serves title and price to a client that identifies itself as
listing-desk/2.31with a link to the source. No browser string, no headers pretending to be Chrome. If a page would not serve an honest client, it is not mine to take. - My own metasearch instance found zero Amazon product pages for four real products, so discovery had to be Amazon's own search.
One search per item researched, paced, and only when the person typed nothing and the comparison site had nothing. Sponsored placements, renewed and refurbished listings, and unpriced hits dropped deterministically. Then every remaining hit through the same relevance gate the live eBay listings pass, a language model asked "is this the same product?", because a search for "malt drink powder 800g" returns 2×400g packs and imports, and token overlap cannot tell a variant from the thing.
I tested it on a discontinued smart speaker. Amazon returns only mounts, stands and chargers for it, and the gate dropped all ten as accessories. No anchor. Correct.
Then I ran it for real.
The charger
retail-anchor 55% of the £14.99 cheapest new price on Amazon across 2 matching listing(s)
retail-ceiling capped at £12.74: it costs £14.99 new and a 'good' used one cannot ask more than 85% of that
The same model, the same query, minutes later, kept two of the ten. Both were 15W chargers that fit the speaker. The anchor read £14.99 as the price of a new speaker. Fix two, the rule I had just been pleased with, took a £23.71 ask and cut it to £12.74.
That is the sentence I want to leave you with. A model's vote must never become a price with authority. The relevance gate was always a vote, and it was designed as one: fail open, warn, let the arithmetic weigh it. The ceiling was authority. I had connected a vote to authority and the first non-deterministic answer went straight through.
The two boring rules
Neither involves a model.
An accessory names what it fits after "for". "15W AC Charger Fit for Google Nest Hub, WiFi, Nest Mini 2nd Gen". "Wall Mount for Google Home Mini". "For Google Nest Mini (2nd Generation), 3D-printed holder". In every one, none of the product's words appear before the "for". The product itself puts its own name first: "Argon ONE V2 Aluminium Case for Raspberry Pi 4" has "Argon" and "ONE" ahead of the clause, so it is the case, not an accessory for a Pi. That rule vetoed ten of the twelve hits before the model saw them. The two survivors were stands whose titles happened to start with the brand, and the model dropped those.
A discovered new price may not cap the ask when the used market asks more than double it. If eight people are asking £35 for a used one and Amazon's "new" one is £14.99, the "new" one is a different product, or the real one is discontinued and the clearance price is not its value any more. Either way, no ceiling. The price a person typed is exempt, because they looked at it.
Three layers now stand between an Amazon result and a price: a rule, a model, a sanity check against the market. The model is still the weakest of the three. It accepted two 400g packs as the same product as an 800g tin, which is defensible on quantity and wrong on product. The difference is that its mistakes can no longer set a price on their own.
Two smaller things from the same day
The verdict sentence quoted the wrong number. The "challenge" button re-gathers evidence and writes one line for a human: "2 fresh sources agree around £36.98, which is 15% below the £35.39 ask." £36.98 is 4.5% above £35.39. The 15% was a different quantity, the desk's undercut suggestion measured against the ask, reused inside a sentence about the consensus. Wrong number, wrong direction, on the one line the button exists to produce. The percentage in a sentence has to be computed from the two numbers the sentence names, and there is now a test that says so with those exact figures.
A check that cannot fail is theatre. Earlier the same day I "verified" that a deleted cloud API key was dead by calling an old helper that listed a bucket with it. "Still works, listed 0 objects", three times over ninety seconds. The helper swallowed every exception and returned an empty list, so a rejected request and an empty bucket printed the same words. Running the same call with a deliberately wrong secret would have shown that in ten seconds. Now I run the wrong-secret twin first, every time, and the same rule went into the desk's own search-index canary that morning: it had degraded to first-token matching and a brand-led test query passed anyway.
What it cost
Nothing new. The desk's only paid dependency is a language model API at about three pence a month at measured volume. I looked at the paid Amazon-data services and their MCP servers and they were the wrong order of magnitude for a retail anchor on used goods. One honest search per item does the job.
Where the code is
The pricing desk itself is private. The part of it worth sharing, the corpus of completed eBay sales it builds for itself and the harness that measures whether that corpus can replace a paid source yet, is public under MIT at github.com/casareanderson/sold-shadow. The README carries the real coverage numbers, which are currently 9%. It is not close, and it says so.
This research is also available as a PDF, with the source list in one place: The Price Data Playbook (£19).
Top comments (16)
"Where the boundary sits: at the action, not the number." This is the cleanest formulation of mutation gating I've read all week.
In our multi-agent pipelines, we arrived at the exact same architectural boundary after watching models happily produce high-confidence hallucinations:
Inference is an untrusted proposal, not an authority: A model's output (whether a pricing estimate, a classified SKU, or an API call payload) must be treated strictly as an unverified proposal. It has zero intrinsic permission to execute an external side effect.
The gate must live in the code path that spends money: Prompt admonitions ("be careful not to price below X"), warning logs, and LLM self-reflection are fine heuristics, but they are terrible security gates. Real business invariants (source corroboration, rate-of-change ceilings, hard margin floors) belong as deterministic assertions inside the leaf executor immediately before the external mutation is committed.
Attested overrides over ad-hoc UI bypasses: Your point about tracking whose number went through is critical. When a reviewer overrides a flagged "thin" price, bypassing the system via a separate web UI breaks traceability. Binding the human reviewer's approval directly to the execution receipt (
approved_by,override_reason,target_hash) preserves the audit trail while keeping the pipeline moving.Treating models as untrusted proposal generators and enforcing deterministic invariants at the mutation boundary is the only sustainable way to run unattended systems.
Glad that line landed. The override part is where I got it wrong first time round: the block refused a person's own typed price too, so the only way past it was eBay's own site, which left no trail at all. Now a price someone types goes through with their name on the approval, and there's a strict switch for when the person approving isn't the one whose money it is. I hadn't thought about hashing the target like you describe, that's a nice touch.
Target hashing is what stops the classic race where a human approves price X for item A, but the worker re-queries and commits price X against item B (or a mutated version of A). Binding the approval hash directly to
(canonical_item_id, target_sku, proposed_price)means the executor can verify the target hasn't drifted under the hood before it spends money.Keeping the override inside the audit trail rather than forcing operators to use the vendor UI is huge. Once they bypass the tool to fix a false positive, you lose visibility into where your bounds were actually too rigid.
This is a good example of why model output should be treated as evidence, not authority. Once a number can cross directly into a pricing decision, the boundary needs to validate its type, provenance, and permitted range before the suggestion becomes business state.
Yeah, that's pretty much where I ended up. The ceiling checked the range but never asked where the £14.99 came from, so a charger "for Nest Mini" got treated as the speaker itself. What fixed it in the end wasn't a smarter model, it was two boring rules: if the product's name only shows up after "for", it's an accessory, and a discovered new price isn't allowed to cap anything when used ones are asking more than double it.
This is a strong example of why model output needs a typed boundary before it can become business data. A model can suggest a candidate value, but prices, IDs, balances, dates, and other source-of-truth fields need deterministic validation against the system that actually owns them.
Agreed, and the awkward bit here is that nothing actually owns this number. There's no source of truth for what a used speaker is worth, just asking prices and whatever sold recently. So the best I could do was make every price say where it came from, and have the publish step refuse to list one the tool couldn't back up.
This is a good example of why AI outputs need to stay proposals until the business rules have had their turn. The model can be perfectly reasonable about bad evidence. A ceiling against the current new price, provenance on every comparable, and rules that keep accessory prices from becoming parent-item evidence would catch a lot of this before it reaches the listing.
Funny thing is, the ceiling against the new price is exactly what did the damage here. I added it to catch a used item priced above new, and then the "new price" it trusted turned out to be a £14.99 charger. So I'd put provenance at the top of that list. A ceiling's only as good as the number you anchor it to.
The failure mode you describe is the one that gets me every time in agent pipelines too: the degraded signal is logged, just not anywhere a human reads before acting on the output. "retail-anchor discovery unavailable: no results" is a perfectly good log line and a terrible production gate.
So the design question I'd take from this is where the "unconfirmed price" boundary should sit. You chose to make the missing anchor visible in the price itself rather than block the listing, which is the right call for a tool you run yourself, but the moment anyone else uses it the £35.39 becomes the machine's opinion and the log line stops existing. Did you consider a hard stop when the anchor's only source was rate-limited, or is the manual
new on amazon £26.99override deliberately cheaper to build than the guard?Direct answers, because a first version of this reply blurred them.
Where the boundary sits: at the action, not the number. A price the desk cannot corroborate is marked "thin", and the publish step refuses to list the desk's own thin number. That refusal lives in the code path that spends money, not in a log, so it survives a second user: the review page shows £35.39 with its warning, the machine cannot list it. Friday's £35.39 was thin from the moment it was priced off live asks.
The hard stop on a rate-limited source: considered and rejected, deliberately. The source was down for every run that day, so a stop keyed to it would have halted the desk, not the one price it could not justify. The typed "new £26.99" was cheaper than a guard on the source and I'd defend that, because the guard that matters already existed at publish.
What your question exposed, and what changed since I first answered: the block ignored whose number the price was. A person who read the warning and typed their own price on the review page was refused too, so the only override was the marketplace's own UI, outside every audit trail the desk keeps. Now the engine's thin number never goes live, a person's typed price goes through with their name on the approval, and a strict switch withdraws that override for installs where the person approving is not the person whose money it is.
"Every stage did what it was written to do and the result was still wrong" is the failure mode that makes agent pipelines humbling. The evidence hierarchy is the right instinct, and the retail anchor is doing the real work: asking prices are fiction nobody has paid for, so an anchor independent of every resale signal is the only thing that can catch a drift the whole market agrees on. A used item priced above new is such a clean canary for "your evidence is circular." Curious how the charger entered the story - cross-category evidence leaking in?
Not cross-category leakage from the resale side. It came in through the fix. The Pi case and the charger are two different items. The case was caught by the new ceiling once I typed Amazon's £26.99. The charger turned up later the same day, when I added an Amazon search so the anchor didn't depend on me typing a price. For a discontinued smart speaker, Amazon returns only accessories. The model gate that asks "is this the same product?" dropped all ten on one run. Minutes later it kept two 15W chargers "fit for Nest Mini". The anchor read £14.99 as the new speaker's price, and the ceiling I'd just added cut a £23.71 ask to £12.74. So the "used above new" check you're praising was the part that did the damage: a model's vote got wired straight into a hard cap. Two deterministic rules fixed it. A title that names the product only after "for" is an accessory. And a discovered new price can't cap anything when used asks are more than double it.
Nice approach! I've found that keeping API wrappers simple and well-tested is key. What testing framework do you prefer for Python APIs?
Honestly nothing fancy, just plain unittest from the standard library. No pytest. It's past 700 tests now and I haven't once wished for pytest's extras. The tests that actually earned their keep weren't the wrapper ones though, they were the ones pinning dumb little things, like the price regex grabbing "new" instead of "rrp". What are you using?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.