Proof of concept · Building-materials retail
Every sentence a shop's assistant says is an offer.
A messaging assistant over a builders' merchant catalogue of 13,312 products, run as a proof of concept with the owner. The point was never the demo. It was to find out which sentences the thing should not be allowed to say.
- Role
- Product lead & builder
- Stack
- Python · in-memory inverted index · tool-calling agent
- Status
- Proof of concept, validated · client anonymised at their request
A chatbot that is wrong on a website is embarrassing. In a shop it is binding.
Brazilian consumer law treats sufficiently precise information conveyed by any means as binding on the supplier and part of the contract. When an assistant speaking for the shop states a price, a voltage or a capacity, the shop answers for it. I am not a lawyer and this deserves one, but the direction is clear enough to design against.
There is a second kind of exposure that has nothing to do with law. Recommending a unit by room size is technical advice: oversize a heater and it overheats the room, undersize a pump and the customer paid for pressure they will not get. Getting that wrong costs more than a bad answer.
The model never sees the catalogue.
The obvious build in 2026 is a vector store and a retrieval chain. The previous attempt went that way through a workflow tool and died on timeouts, because putting thirteen thousand rows anywhere near a prompt is not a retrieval strategy.
So the catalogue stays out of the model entirely. Thirteen thousand products load into memory behind an inverted index and answer in one to five milliseconds. No database — thirteen thousand rows fit in RAM, and Postgres only earns its place when there is real concurrency. The model receives eight rows and writes a reply around them.
The hard part was never the index, it was the vocabulary. The catalogue is written in ERP shorthand and the customer speaks counter vernacular: rebar is filed under a product code nobody says out loud, and a bricklayer asking for a "ballerina" means a hole saw. A synonym layer carries that gap, and the right side of every mapping has to exist in the actual catalogue — otherwise the mapping is a lie that looks like a fix.
25mm never returns 32mm. Better to say we don't have it.
Every measurement is parsed into a value and a dimension, so 18 litres of paint is never compared against 18mm of pipe. Inches are kept as a separate dimension from millimetres on purpose: a quarter inch of pipe is 20mm, a quarter inch of rebar is 6.3mm, and collapsing the two put the wrong bar first in the results.
The unit field in the ERP has 136 distinct values, so it is repeated verbatim and never interpreted. Confusing a 0.9 litre tin with a 3.6 litre one is not a rounding error, it is an argument at the counter.
Retrieval stopped lying. The writer did not.
Taking the model out of retrieval worked: across the sessions, the search did not return the wrong product. What it exposed is that the remaining surface — turning eight rows into a sentence — invents precisely where the catalogue is silent. Four behaviours, all observed:
It stated specifications that do not exist
Asked for detail on a pump, it described the motor type, the intended use and a supply voltage. None of that is in the catalogue, which holds a description, a price and a stock figure. The following day, asked the voltage directly, the same assistant correctly answered that it is not in the system. Same question, opposite answers, and the invented one came first.
It read a description instead of a field
A product named with four stages and two horsepower became, in the reply, a four-horsepower pump — and that invented figure was then used to compare models for the customer. The number was never a field; it was a string the model interpreted.
It agreed first and checked second
A customer mentioned a product line they had heard about. The assistant explained how it works, endorsed it as a good fit, and only afterwards searched and found the shop does not carry it. The endorsement was already said.
When a tool failed, it improvised
The sizing tool errored on two attempts. Instead of stopping, the assistant produced a recommendation of its own. That is the exact failure the tool exists to prevent.
Five guardrails, written from four failures.
This is the deliverable. Not a demo that impressed anyone — a specification for what the assistant must be prevented from doing, each line traceable to something it actually did.
- 01
Grounding, checked deterministically
The reply may only assert attributes present in the rows retrieved. Enforced twice: as a contract in the prompt, and as a check after generation that extracts the claims and compares them against the payload. The check is deterministic on purpose — I have built a model-graded evaluation panel before and watched it approve almost everything, including the bad output. A model asking a model whether it hallucinated inherits the problem it is meant to catch.
- 02
Typed attributes, never an interpreted string
Stages, power and voltage become typed fields at index time, the way measurements already are, and the writer receives fields rather than a description to read. The pattern exists in the system already — it is why inches and millimetres are separate dimensions — and it simply had not been extended to the attributes buried in product names.
- 03
Search before claiming
No statement about what the shop carries, or about how a named product works, before a retrieval has returned in that same turn. Ordering enforced in the agent loop, not requested in the prompt.
- 04
A failed tool stops
If the sizing tool errors, the assistant says it cannot check right now and offers a person. It does not substitute its own judgement for the tool whose whole purpose is that its judgement is not good enough.
- 05
Sized recommendations stay inside validated ranges
A recommendation driven by room area or number of outlets is advice with a physical consequence, so it is bounded: return the smallest unit that covers the requirement, refuse outside the validated range, and escalate rather than extrapolate.
And one that is not a guardrail
Keep what the customer was shown.
Price, availability and specification as they were presented, with a timestamp. Not for analytics — because when a customer arrives at the counter holding a different number, that record is the only thing that settles it.
An honest reading
This was a proof of concept, run with the owner rather than with the public, and it is anonymised at their request. There is no revenue figure and no deployment to point at. The guardrails above are a specification produced by the exercise; not all of them are built.
What it does show is a decision most people would not make — keeping the fashionable tool out of the part of the system everyone currently puts it in — and then the discipline of going looking for where the decision did not go far enough, instead of stopping at a demo that worked.
Thinking about putting an assistant in front of customers?
The useful conversation is usually about which sentences it must never be allowed to say.