TAG

LLM

Everything filed under this tag.

6 entries

01Writing

The Synthetic Set Picked C. Forty-Four Real Questions Picked B.2026-10-27A synthetic evaluation set gave a clean answer to what granularity memory should be stored at. Real questions from real chats gave a different one — and along the way turned up three measurement problems.#Memory#Evaluation#Measurement#LLM Where the Prefix-Cache Hits Went2026-10-27The hit rate dropped, and the total alone couldn’t say why. Looked at in 128-token blocks, the losses came from one section placed in the wrong spot, and from two providers with completely different caching rules.#LLM#Cost#Observability Why the More You Write a Persona, the Less Human It Sounds2026-10-21The persona grew from 800 to 8,000 characters, and replies started sounding like the character reciting its own spec. Most of the problems turned out to be in the code that assembles the prompt, not in the persona text itself.#Prompt#Persona#LLM Fully Configured, Silently Ignored2026-10-21A field sits in the config, looks like it’s active, and no code ever reads it. This kind of failure raises no error — it just makes people say “something feels off”.#Engineering#Observability#LLM Treat the LLM as an Untrusted External Service2026-10-18It behaves like a remote service: it times out, it returns nothing, it mixes its reasoning into the body, and it fills in a number by feel.#LLM#Engineering

02Notes

The First Time a Fallback Chain Runs Is Usually the Day the Model Is Retired2026-10-16A specific free model was retired and good-night generation failed outright, because the fallback branch had never executed.#LLM

← All tags Search the site