Notes
Working notes from live builds. What we measured, what we expected to find, and what turned out to be true instead.
Most of these come out of testing that was done for a client project rather than for an article, so the findings are narrow and specific. That is deliberate. The general advice already exists everywhere and it has not helped anyone.
Reversible or not
An agent that can act can be wrong in ways an assistant cannot. One classification, applied before anything is built, removes most of the risk.
Reading a page instead of its characters
Traditional OCR reads characters and throws the layout away. That is the wrong shape for a form, a table or anything handwritten.
Resolution, not prompt engineering
We were convinced the prompt controlled stability in reference-to-video generation. Six controlled tests said otherwise, and the real lever was the setting nobody was looking at.
When self-hosting is the wrong answer
Two things send people towards private deployment, and only one of them is a real decision. The other is arithmetic that usually gets done wrong.
The hard part of RAG is teaching it to say “I don’t know”
Retrieval systems rarely fail loudly. They fail by answering a question their documents never covered, fluently, and being believed.