OpenRouter tests three routes to structured LLM output: 209 of 216 replies met a strict schema, and all 7 misses came from one provider that ignores it
OpenRouter compared three ways to get reliable JSON out of models, using 24 messy support messages across nine models that support structured outputs: send a strict JSON Schema, force a single tool call, or switch to a decision model that only returns labels. Across the nine models, 209 of 216 replies matched a strict schema, and all 7 misses came from one provider that ignores the schema; asking for JSON only in the prompt produced 18 failures, each from a code fence. For GLM 5.3 FlashX, which only supports tools, forcing the tool call gave 22 valid outputs out of 24, against 0 when the schema was sent directly.