It behaves like a remote service: it times out, it returns nothing, it mixes its reasoning into the body, and it fills in a number by feel.
Table of Contents
When I integrated the first model I treated it like a function: pass messages, get content back. It turned out to behave more like a remote service. It times out, it returns empty, it leaks reasoning into the body, and it hands back half a sentence when it runs out of budget.
I have logged over twenty incidents involving model output format. Reading back through them, most came down to checks I hadn’t put in place.
01Reasoning leaks into the body
Reasoning models usually have two channels: reasoning and content. Some providers push reasoning through in streaming mode as well, so if your code just concatenates every delta, the user ends up reading an English inner monologue.
Worse, it looks like body text, so it slips past health-reminder exemptions based on length. One record is exactly that: English reasoning passed the length check and went straight into the chat.
The protocol gives you two fields, so the code has to handle them separately.
02max_tokens too small means it thinks but never answers
I tested one model: below 1,500 max_tokens the entire budget burns on the thinking chain, content comes back as an empty string, and finish_reason is empty too.
Looking at the return value alone, it looks like “the character is refusing to answer”. Before judging whether the output is valid, check whether completion_tokens hit the cap.
This became a standing check: empty content plus high completion_tokens means truncation, not refusal.
There is a harder kind to defend against, too: it gives you a very specific-looking number without having called a single tool. One morning at 6:47 I asked for a sleep score, and the first reply came back as a score of just 56 against a true value of 85. It had looked nothing up and supplied the number by feel. It took three rounds to get right.
03Structured output is not optional
Another class: the model writes a tool call as raw text, or sends a numbered list straight out as body content. The user sees markup that was supposed to be parsed away.
So the output side needs at least four gates:
- Structure check: where JSON is expected, retry on parse failure; never treat half a JSON document as body text.
- Length and format gate: over the threshold, contains a code block, contains a numbered list — let the channel decide whether to sanitize.
- Content allowlist: only known voice tags and emotion tags are permitted. A blocklist always leaks.
- Degradation chain: fall back to an alternative when the primary is unavailable, and drill that chain regularly: a branch that has never run is usually broken by the time you need it.
04Streaming and sync are two different paths
With the same model, streaming requests may not return usage by default while sync requests do. So only half your calls have token counts in the cost dashboard.
For the same model, streaming and non-streaming are effectively two implementations, so metrics, rate limits, and retries all need verifying on both.
Validation has weights too. Tool calls, state changes, and structured data get strict validation; natural-language text only gets content-safety and obvious-anomaly gates, and the rest is left to the prompt. Chat text needs to sound human — chop one sentence into schema-compliant fragments and it reads like a robot.