Catching agent failures

Why this conversation

An agent in production does not fail like a web service. It does not return a 500 and stop. It takes a wrong turn, calls a tool it should not, or goes round in circles, and the first anyone hears of it is a strange answer. Teams that run agents already collect traces. The open question is what to do with them once there are too many to read.

On the second day of AGNTCon + MCPCon Europe in Amsterdam, the host spoke with Vladimir Vučković, Founding AI Engineer at moyai. He had joined the company six to nine months earlier. moyai describes itself as an AI-native agent company: it catches anomalous behaviour in production agents, classifies it, and gives the team a report of what went wrong in the agentic flow, together with a proposed fix. The rest of the week is in AGNTCon + MCPCon Europe 2026: who we met.

What counts as a failure

An agent with tools moves from point A to point B, calling those tools to find the information it needs on the way. Along that path it can hallucinate, it can get stuck in a loop, and it can make orphan tool calls. Those are the behaviours moyai looks for. Some are generic and show up in almost any agent. Others are specific to the use case, and depend on what the team has built.

Statistics first, then agents

The detection works in two stages, both on traces. moyai does not replace the observability platform a team already runs; in Vladimir’s words, “we just kind of stack on top of it”. The first stage is deterministic and statistical analysis of the traces. The second is non-deterministic: moyai’s own multi-agent system takes the statistical results and digs into the detail. Its agents look for evidence, widen the sample, and search for patterns. They then put forward hypotheses and test them, confirming or refuting each one. Everything is gathered into a report on the platform, with what moyai believes the problem is and the steps to solve it.

The method, he says, came out of many experiments and a lot of iteration, testing on moyai’s own agent and on those of its design partners.

Any agent, any traces

Asked which agent SDKs it supports, Vladimir says the platform is agnostic. Any agent, internal or external, works as long as it has some complex behaviour, such as tool calls, handling context or summarising. On the observability side, any platform built for observing LLMs can be used, and traces are exported into moyai. The preference is for OpenTelemetry-native traces.

Evals on every tool call

The host raised the shift from a deterministic world to a probabilistic one, and the weight now put on evaluations. moyai’s own agents run a layered eval system: code quality evals, and tool evals for every tool call on every agent in the system. The platform as a whole works as a more nuanced, online eval system. moyai also observes itself with it, so the platform feeds back into its own development.

The architecture borrows from the ReAct loop, in which a model alternates between reasoning and acting. Some of moyai’s agents have a single tool, others have complex sets and choose between them based on context. Each agent acts as one step of reasoning and acting, and together they form what he calls “a macro ReAct loop”: the same pattern a chat assistant runs inside a single answer, built out of whole agents.

How to try it

The starting point is moyai.ai. From there, people can book a meeting with the founder or register on the platform. Registration comes with a limited number of scans; anyone who needs more talks to the founder.