DEV Community: Ravi Verma

Why your multi-agent system fails silently (and the gate pattern that fixes it)

Ravi Verma — Wed, 28 Oct 2026 11:32:42 +0000

Multi-agent systems rarely crash. They fail worse: they succeed.

The pipeline completes, the report generates, the answer ships, and it is wrong. Nobody notices because nothing between the agents and production ever said no.

I have built three multi-agent systems, and all three failed the same way before I rebuilt the orchestration layer. This post is about the pattern that fixed it: enforceable quality gates. At the end I will mention the starter kit I packaged it into, but the pattern is the point and you can implement it in an afternoon.

Here is the dashboard the kit ships with, so you can see what "watching gates fire" looks like before reading further:

The three silent failures

1. Low-confidence output ships anyway.

Agents generate output with some internal confidence. In every system I built before, that confidence was ignored. An agent at 40% certainty produced the same artifact as one at 95%, and both flowed downstream. The fix is not better prompts. It is a threshold with consequences.

2. Specialist agents disagree with no tie-breaker.

Run two specialist agents over the same input and they will eventually conflict. My early systems resolved this by accident: whichever agent wrote last won. That is not a resolution strategy, it is a race condition.

3. Nobody owns the fallback.

When a phase fails, what happens? In my first systems: an infinite retry loop, or a crash, or worse, a default value that looked plausible. Every phase needs a named fallback behavior and a budget for how hard it tries before using it.

The gate pattern

A quality gate is a check between phases that can block the cycle:

Each phase declares its acceptance criteria.
Output that fails the criteria does not advance. The cycle stops, or the phase retries within budget, or the system escalates to a safe fallback.
Gates are enforceable, not advisory. An advisory warning is a suggestion; production ignores suggestions.

The counterintuitive part: strict gates feel dangerous. What if they block good output? In practice, strict gates surfaced bugs that my "successful" runs had been hiding for months. Completion rate dropped 5-10%. Correctness went up more than that.

Confidence thresholds

Give agents a self-reported confidence score and enforce a threshold. Below it, the agent falls back to conservative behavior: narrower claims, explicit uncertainty, or refusal to produce the artifact.

The exact number matters less than the mechanism. I use 85% as a default because it worked across my runs; tune it per deployment. What matters is that low confidence has a consequence other than "ship it and see."

Mediation for disagreement

When specialists conflict, route through an explicit mediation step: re-state both positions, apply resolution rules (scope authority, evidence weight, recency), and log the decision. The losing position is not deleted; it is recorded. Disagreement is information.

Retry budgets and escalation

Every agent gets a retry budget. Exhaust it and the system escalates to a pre-declared safe fallback: a degraded-but-honest output, a human queue, or a hard stop. The budget converts "infinite silent retries" into a bounded, observable event.

What this looks like in practice

I packaged all of this into AETHER, a runnable Python starter kit: a 6-phase pipeline with enforceable gates, confidence-gated audits, retry budgets with escalation, a mediation protocol, and a FastAPI simulation dashboard with live telemetry and cycle replay so you can watch gates fire. There is a CLI for headless CI runs. It is pure Python, runs on a laptop, no GPU, no containers.

The agent roster view, where each specialist's state, retry budget, and confidence are visible at a glance:

And the cycle history view, for replaying a run after the fact and seeing exactly which gate blocked what:

It is $49 one-time (full disclosure: my product, AETHER on Gumroad), because I would rather sell a working implementation than maintain an open-source project on goodwill.

But you do not need my kit to use the pattern. The gate pattern is three things: declared criteria, blocking enforcement, and a named fallback. Implement it this week and your pipeline will complete less often and be right more often. That trade is the whole game.

What has failed silently in your agent systems? I am collecting failure modes for the next version of the gates library.

I Built an AI Mock Interview Coach That Doesn't Fake Praise (Next.js + NVIDIA NIM)

Ravi Verma — Sun, 09 Aug 2026 10:03:12 +0000

Most AI interview prep tools are polite. They tell you "great answer!" and move on — which means you walk into the real interview with a false sense of confidence.

I built the opposite.

InterviewIQ is an AI mock interview coach that scores you like a strict hiring manager. No fake praise. If your answer is vague, it says so. If you'd get a "No Hire" in real life, it tells you — and tells you exactly why.

Live demo: https://interviewiq-hazel.vercel.app
Repo: https://github.com/ravikumarve/interviewiq

What it does

You pick a role (Frontend, Backend, Data Science, PM, DevOps, AI/ML — or type any custom role), optionally paste a real job description, and answer 5 questions from a live AI interviewer with a 90-second timer.

Then you get a dossier-style report:

Score out of 100 and a hiring verdict: Hire / Lean Hire / Lean No Hire / No Hire
Strengths and weaknesses — specific, not generic
A targeted practice plan for your weakest skill
Peer benchmarking — "You beat 66% of Frontend candidates" with a live leaderboard

Why the "strict evaluator" matters

This is the differentiator. I tested it: answer "Yes, I know React" to a React question, and the evaluator scores you below 30 with a No Hire verdict. Answer with a detailed, example-backed story, and you get 85+ with a Hire.

The evaluation prompt is deliberately aggressive:

const EVALUATOR_SYSTEM = (role, jd) =>
  `You are a strict hiring manager for "${role}". Evaluate the candidate's answers. ` +
  `Penalize one-word/vague answers HARD: if most answers are under 20 words, score MUST be below 30. ` +
  `Never pass (>50) without concrete technical knowledge. Do not invent strengths.`;

Most AI wrappers optimize for "user feels good." I optimized for "user is ready for the real thing."

Adaptive follow-ups

If you give a thin answer (under 14 words), the interviewer doesn't just move on. It fires a follow-up probe — "Can you give a concrete example?" — up to twice per question, without consuming your question count.

Verified in a real flow: a 10-word vague answer triggered "Can you give an example of a React component…", and the counter stayed at "Question 2 of 5."

JD targeting

Paste a real job posting and every question adapts to its exact stack, tools, and responsibilities. The evaluator scores you against those requirements and calls out missed core skills.

Verified: pasted a "Senior React Engineer — Redux/GraphQL/WCAG" JD → Q1 was React-specific, Q3 was accessibility-specific, and the final evaluation penalized a candidate who never touched GraphQL or WCAG.

Tech stack

Next.js 16 (App Router, TypeScript, Tailwind CSS)
NVIDIA NIM API — meta/llama-3.1-8b-instruct, ~450ms responses
Ollama fallback — local CPU inference if the API key is missing (perfect for demos)
jsonblob — zero-signup shared store for the leaderboard (swap-in ready for Upstash Redis)
Vercel — one-command deploy

What I learned building it in 8 hours

Strict prompts beat big models. llama-3.1-8b with a hard "never pass without substance" prompt outperformed a larger model with a soft one. Evaluation quality is prompt engineering, not model size.
JSON extraction needs a fallback. Models wrap JSON in prose sometimes. I built an extractJson() that does balanced-brace recovery — it saved the eval flow twice during testing.
A local fallback is a superpower. When the NVIDIA key rate-limits, Ollama on my laptop keeps the demo alive. Users see "evaluating…" instead of an error page.
The viral loop is the differentiator. The peer percentile + share button ("I beat 66% of Frontend candidates") is what makes users share the report. Building that in early matters more than polish.

Try it

The live demo takes about 8 minutes for a full interview. Paste a real job description you're targeting and see how specific it gets.

Live: https://interviewiq-hazel.vercel.app
Repo: https://github.com/ravikumarve/interviewiq

Built solo in 8 hours for Hack Devengers 1.0. Questions, feedback, and ruthless criticism welcome — I built an app that doesn't fake praise, so I can take it. 😄