Opus 4.8 and Sonnet 5 apparently do something older siblings did not: they invent extra fields inside a tool call and break a perfectly valid custom edit schema. That matters because better models are not automatically better at your tools. If you build coding agents, MCP servers, or any custom harness around tool use, this is the kind of regression that hurts in practice.
This is a short post from Simon Willison pointing to Armin Ronacher's experience with Pi. The concrete failure mode is simple: the model makes the right edit, but wraps it in arguments that do not match the schema, so the harness rejects the call and asks it to try again. The surprising part is not that malformed tool calls happen. It is that newer flagship models can be worse than older ones at a third-party tool they were not implicitly trained around.
What is actually going on?
The theory is plausible: vendors increasingly train models around their own preferred tool contracts. Anthropic's editing flow appears to bias toward a search-and-replace style editor; OpenAI has also pushed models toward specific patch-style editing patterns. That is rational from the vendor side. It makes their first-party coding product feel more reliable.
But for everyone else, it creates a compatibility layer problem. If your product exposes one custom edit tool, and the model has a strong prior for a different shape, you may see more retries, more validation failures, and a worse user experience even when the model itself is smarter.
In practice, this pushes teams toward one of two choices:
- adapt your tool surface to match the model's habits
- support multiple tool contracts and route by model family
I would not be surprised if the second option becomes normal for serious agent builders. That is especially relevant for teams building custom MCP servers or broader Microsoft Copilot and AI agents that need to work across providers.
Why this matters beyond one edit tool
There is a second-order effect here. Once model vendors optimize for their own harnesses, third-party ecosystems get a little more fragile. Not because the APIs are closed, but because the behavior around those APIs becomes vendor-shaped.
That is different from a normal schema mismatch. A normal mismatch is a bug. This is closer to behavioral lock-in.
The operational consequence is straightforward: if you run agents in production, schema validation alone is not enough. You also need compatibility testing by model version, and you need to re-run it when the model changes. For most teams, this is not a research problem. It is a release engineering problem. If you are already doing AI workflow automation, this belongs in your regression suite next to auth, latency, and retry handling.
My take
I think the core observation is right: coherent toolkits beat sprawling ones, and models perform better when there is one obvious way to do a task. But the more important lesson is for platform teams, not model vendors. Design your tools like adapters, not like sacred abstractions.
If a model family clearly prefers search-replace or patch semantics, meet it there. Keep your internal action model stable, but expose a compatibility layer the model can actually use. Otherwise you end up blaming the model for a systems integration problem.
The practical takeaway: test tool use by model family, expect regressions from newer releases, and do not assume "best model" means "best fit" for your agent stack.
