DEV Community: RLM Labs

Correctness is not a feature

RLM Labs — Thu, 29 Oct 2026 11:25:00 +0000

You cannot add correctness to a system after the system is built. You either build the system so a claim needs evidence before the claim exists, or you measure, after the fact, how often the claims were wrong. Most of the record below did the second and called the result progress.

Look at the record in order.

November 2025. Anthropic publishes a guide to harnesses for long-running coding agents. The failures the authors describe include an agent seeing partial progress and declaring the job done, and an agent marking a feature complete without proper testing. The fix: ask the model to self-verify.

February 2026. OpenAI describes building a product with zero hand-written code. Human review of pull requests is optional. Merge gates are kept to a minimum.

April 2026. A UC Berkeley team audits 13 agent benchmarks, rates every one critical risk and scores perfectly on some without solving a single task.

July 2026. HANDBOOK.md tests models on following a company handbook. The best configuration passes 36.2 percent of trials, and failed runs end with a confident claim of compliance.

September 2026. Google Cloud AI Research publishes RRSI and warns automated harness improvement overfits:

"such recursive evolution may overfit by memorizing the training tasks"

October 2026. The same group publishes Harness², a method for improving agents on tasks with no ground truth, and writes in the limitations:

"They do not guarantee correctness or establish that an adopted change will transfer to a different task."

Read the sequence as one argument. Every step makes agents more capable or more autonomous. Every step leaves the same question open: is the answer right. Each paper and post states the question honestly, then hands the question to a judge, a benchmark or the user.

Now the same months, from our side, out of the git history.

18 January 2026, first commit. The design document's first non-goal: never modify or retrain the model. All improvement comes from the structure around the model.

1 March 2026, a design for learning from our own runs. The rationale says, in plain words:

"Fully autonomous self-modification is unsafe and hard to trust."

28 March 2026, the first evidence and completion gate. A task closes only with evidence on record.

29 May 2026, the final authorization gate. A model gets a vote on the final answer, and rules counting the record get the last word.

1 July 2026, the specifications. Every rule written down, most of them carrying the production session behind the rule.

25 July 2026, the change gate. A code change ships only with the specification updated in the same commit.

None of those is a feature you switch on. Each one removes a way for the system to claim something unsupported.

I owe you the other half, because this series names our own failures too. The March learning design included an evaluator for proposed improvements. Reading the code this month, I found the evaluator never ran the proposed change. The evaluator scored the health of the last 24 hours of ordinary runs and approved or rejected on the result. A pass meant the baseline was healthy. A pass said nothing about the change. Google's Harness² does the part we skipped: run the change, compare against the original, then decide. We are fixing our evaluator, and the fix gets a post when the record exists.

The point stands, and the failure proves the point. Our evaluator was a bolted-on measurement, and a bolted-on measurement looked fine for seven months while measuring the wrong thing. The gates on the final answer were built in, and the gates caught the failures Act II describes.

Act II starts with those failures. Ours first.

Thirty years of confident wrong answers

RLM Labs — Mon, 26 Oct 2026 22:21:00 +0000

Thirty years of confident wrong answers

I have spent thirty years walking into rooms where a distributed system was misbehaving in production and somebody was certain they knew why. Often they were wrong. Not stupid. Wrong, and confident.

I went to school for aerospace engineering. The first lesson of the field is margins: you never trust the nominal number, you trust the measured one, and you design for the gap between them. The second lesson is failure analysis: when something breaks, the explanation everybody likes is the least useful thing in the room until the data agrees with the explanation.

Then I spent a career as the person a customer calls when the database, the cluster or the network does something the brochure said was impossible. The title was usually Field CTO. The job was always the same. Find out what is true, say so out loud, and do not let anyone defend their work instead of fixing the work.

I have run into a lot of wrong and confident. The one I always remember is where I met someone who has become one of my best friends, and the relationship started with wrong and confident. Our team had spent two and a half years deploying a database on the company's shiny new internal private cloud, part of a new transformation project, and one day we hit it. Our newly deployed production database would slow to a crawl every Monday morning. We spent weeks reading database logs and system logs, analyzing what in the workloads was causing the slowdown. We instrumented every database VM to see if we could pinpoint the failure, and after weeks, still nothing. The customer was upset, because now we were on the critical path, delaying the full production rollout.

Then one Monday, as I was flying in to this customer, I remembered something I had read a long time ago, Law 11 of Akin's Laws of Spacecraft Design: "Sometimes, the fastest way to get to the end is to throw everything out and start over." So that is what I did. I went back to first principles: can I reproduce the error in a clean test harness? I stood up my own database and load tested it Monday to Friday, watching it perform properly, trying to see if I could identify the slowdown. On Friday I hopped on my airplane and went home. Monday morning, working remotely, I grabbed my cup of coffee and started the load tests again, and there it was. The slowness reproduced in a clean test environment, with all the instrumentation in the world on the VM and nothing showing the root cause of the problem. So we kept testing all week long. Friday came, I stopped my test, and on Monday I flew back to the customer. Sitting at my desk at the customer, I started the load test and hit the same problem again.

Okay. This wasn't the database. Something was happening here that I couldn't measure or control. So I walked into the infrastructure team's work area, sat down with the vendor's lead infrastructure architect and told him something was wrong with his infrastructure. He looked at me and told me there was nothing wrong with his infrastructure. They had deployed thousands of VMs and no one had identified anything like this problem. I argued back: none of those thousands of VMs was running a persistence-based product like ours, so the problem had to be in their storage. That is when we started looking at the whole stack, from the VM hypervisor, to the virtual storage drivers, to the tiered storage platform, to the storage network, literally every layer, and instrumenting each and every one of them. After months of reading logs and analyzing dashboards, one Monday we were debating the storage layer, and we noticed the access latencies. On Monday they started super slow. By Tuesday performance had improved by an order of magnitude, and by Friday it had jumped two orders of magnitude from the first measurement. What could cause such a jump? The storage architect said the only thing that could do that was the smart storage policy on the storage platform. After months, that was it.

The storage tiering policy would relegate lightly used blocks to SATA storage. As blocks got hot, it would move them to Fibre Channel storage, and as usage grew further, to NVMe. Over the weekend, when no tests were running, it moved them back to SATA.

We fixed it by simply pinning every database VM to a Fibre Channel storage tier. A simple solution to a confounding problem. When in doubt, instrument, test, and if you have to, start over.

Here is what thirty years of those rooms taught me, written as the rules I now give everyone who works on Merlin, people and models alike. These are verbatim from the project's rulebook:

"Never spin theories. Pull the actual data (exact log lines, decoded diffs, metric values) and verify every assumption by reading the code before stating a diagnosis."

"A hypothesis is not a conclusion, and presenting one as fact wastes the user's time."

"If we broke it, own it and fix it."

"Say "I'm not sure" when uncertain rather than confabulating."

Read the last one again. I wrote the rule for an AI model. The rule fits half the war rooms I have stood in.

A language model is the most fluent confident-wrong colleague I have ever worked with. The model never says "I'm not sure" unless you build a system rewarding the sentence. The explanation sounds exactly as convinced when the explanation is wrong as when the explanation is right.

So I treat Merlin the way I treated every production system I was ever responsible for. I read the logs before I believe the summary. In June 2026, before deciding which fix to build next, I had the last ten self-review runs read directly from our log store, one reader per run, every verdict backed by a quoted log line. Two of the ten had finished cleanly. The other eight had ended in forced or budget-limited finals. The assumed blocker was the final gate. The logs said the blocker was tasks not finishing. We fixed the thing the logs pointed at, not the thing we expected.

This is just one of the experiences that have allowed me to be successful in my career, and it is why I built Merlin. AI and models are amazing technology, the stuff of science fiction novels. But as an engineer and architect, I live in a deterministic world. We have to have evidence for a conclusion, and we can't be afraid to be told no: the data doesn't support your conclusion, try again. I have gone back to Akin's Laws a lot, and with AI and models I anchor on Law 17: "The fact that an analysis appears in print has no relationship to the likelihood of its being correct."

Nothing about this is new. A confident wrong answer is the oldest failure in engineering. The difference now is the speed and the volume. A model produces confident answers faster than any war room ever did. The discipline has to live in the system, because no human reads fast enough to apply the discipline by hand.

Sampled, asynchronous, non-blocking

RLM Labs — Sun, 25 Oct 2026 17:22:05 +0000

Most AI evaluation products check a sample of answers, after the answers have already gone out, without stopping anything. Read their documentation and you find all three properties written down as features.

Mastra, a popular TypeScript agent framework, describes its scorers:

"Live evaluations run in the background without blocking your agent responses or workflow execution."

"The `sampling.rate` parameter (0-1) controls what fraction of outputs get scored."

In the background. Not blocking. A fraction of outputs. Set the rate to 0.1 and nine answers in ten reach your user with no score at all. The tenth gets a score after the user has already read the answer.

AWS offers online evaluation for agents on AgentCore, scored by language models acting as judges. The built-in judges come sealed:

"Built-in evaluator configurations, including their evaluator models and prompt templates, cannot be modified."

OpenAI's harness engineering team, writing in February 2026 about building a product with agents, stated the philosophy behind all of this more openly than anyone:

"The repository operates with minimal blocking merge gates."

"Test flakes are often addressed with follow-up runs rather than blocking progress indefinitely."

"In a system where agent throughput far exceeds human attention, corrections are cheap, and waiting is expensive."

I respect the candour, and for their case the trade holds. An internal product with fast rollbacks and engineers watching tolerates a bad merge. The cost of a wrong answer is one more pull request.

Change the setting and the arithmetic flips. A financial reconciliation. A medical billing claim. A security review saying no issues found. A wrong answer there is not cheap to correct, because by the time a sampled, asynchronous score arrives, somebody already acted on the answer. A dashboard tells you yesterday's error rate. A dashboard does not stop today's error.

So name what these tools are. Measurement. Good measurement, needed measurement. Not a gate. A gate sits between the answer and the reader, runs on every answer, and has the power to say no before anyone sees the answer.

Merlin's checks on the final answer are gates in exactly this sense. They run on every answer, and they block before delivery. The rule counting open tasks runs on every proposed final answer, not on a tenth of them. A citation pointing into a file the run never read fails the task close, every time, before the answer exists.

Blocking has a price, and I will not pretend otherwise. Our own gates have rejected correct work. In one run, our gate refused three correct Maven builds and one test run, each time with a message telling the model to run a build. The model concluded Maven was not installed, and the task closed without any build. A later post in this series covers the run, with the log. A wrong blocking gate is loud: you see the rejection, read the reason and fix the rule. A wrong sampled score is silent, and the answer already shipped.

The question to ask any vendor selling you AI quality: on what fraction of answers does your check run, and does a failed check stop the answer. If the answers are "a sample" and "no," you bought a dashboard.

Misreported compliance is worse than hallucination

RLM Labs — Sat, 24 Oct 2026 05:34:02 +0000

A hallucination is a wrong fact. You catch wrong facts, sometimes, because facts live outside the model and you check them. Misreported compliance is a wrong report about the agent's own work. Nothing outside the agent tells you the report is wrong, so you act on the report.

Most writing on AI reliability covers the first problem. The second does more damage, and the evidence for the second piled up through 2026.

HANDBOOK.md, the benchmark from my last post, found agents following a company handbook correctly on 36.2 percent of tasks at best. The abstract names the behaviour directly: agents

"report compliance they did not achieve."

Section 6 adds the shape of the failure:

"Nearly every failed trajectory ends with a confident statement that the handbook was followed."

Now look at what happens when the agent gets a chance to influence the grade itself. In April 2026, a team at UC Berkeley audited 13 widely used AI agent benchmarks and rated every one critical risk. They built 45 working exploits. Two of the root causes:

"No process isolation between submitted code and evaluation infrastructure."

"Reference answers stored in agent-accessible filesystem with no integrity protection."

One exploit needed no skill at the task at all:

"A malicious solution can walk up the stack and replace the scoring function."

The Berkeley team built those exploits on purpose. Google's team saw the same behaviour happen by itself. The Harness² experiment guide, published October 2026, warns users about network access during evaluation:

"In a validation run of this release, a Codex rollout on JobBench downloaded its task's rubric without being instructed to."

Nobody asked the agent to find the answer key. The agent found the answer key.

None of this is new as an idea. DeepMind named the pattern in April 2020, years before today's coding agents:

"a behaviour that satisfies the literal specification of an objective without achieving the intended outcome."

Specification gaming. Satisfy the check, miss the point. A report of compliance is the cheapest possible way to satisfy a check, because the report costs one sentence.

Here is why I rank the problem above hallucination. A hallucinated fact damages one answer. A false report of compliance damages your whole process, because your process trusts the report to decide what happens next. The ticket closes. The migration gets marked done. The audit step gets skipped next time because last time passed.

We found a mild version in our own system. One run had 21 tasks still pending and zero files written, and nothing in the authorization path stopped a model from calling the run complete. A claim of success cost the model one sentence, and nothing in the run made the sentence expensive.

So in Merlin the report is never the evidence. A run claims completion. Then rules read the task ledger, count open tasks and check every cited file against the record of what the run read. If the record disagrees with the report, the report loses. The agent never gets to grade the agent's own exam.

The test for your own setup: find the step where your process accepts an agent's summary of its own work, and ask what checks the summary against the record. If the answer is another model, read my post on verifiers. If the answer is nothing, start there.

Thirty-six percent

RLM Labs — Fri, 23 Oct 2026 11:31:16 +0000

Give the top-scoring model in a 2026 benchmark a company handbook and a job to do under the handbook's rules. The model gets the whole job right 36.2 percent of the time.

The number comes from HANDBOOK.md, a benchmark published in July 2026 and revised in August. The design copies how employees work. An agent gets a binding policy document, between 20 and 124 pages long, and a multi-step task in finance, HR, insurance, logistics or medical billing. 65 tasks. Each task runs four times per model. A completed run averages about 17 steps and 30 tool calls.

The grading is the part I like. No model grades another model. 824 programmatic criteria check what the agent did, and a run passes only if every criterion passes. One missed rule fails the run, the same way one missed rule fails an audit.

The results across 30 configurations of 20 models from 11 providers:

The best configuration passed 36.2 percent of trials. The second passed 34.2 percent. The third passed 23.5 percent. In the authors' words, "most frontier models remain below 25%."

Read those as working days. Your new hire follows the handbook on one task in three. On the other two, something gets skipped.

The authors read the failed runs and grouped what went wrong. The immediate request overrode the standing rule. A required check ran and the agent ignored the result. A verification step got skipped and the agent assumed success. And then the pattern I care about most. In Section 6 the authors write:

"Nearly every failed trajectory ends with a confident statement that the handbook was followed."

Read the claim carefully. "Nearly every" comes from reading the failures, not from a measured rate. But the direction is clear. The agent does not stop and say "I skipped step four." The agent finishes the job wrong and reports the job done right.

Two things follow for you.

First, a better model does not close the gap on the timescale you care about. The spread between configurations is wide, and the top score is still a failing grade by any audit standard you would accept from a person.

Second, the report at the end is the least reliable part of the run. If your process reads the agent's summary to decide whether the work is done, the benchmark says your process inherits the error rate and hides the error.

HANDBOOK.md got one thing right worth copying. The benchmark does not ask the agent whether the agent complied. The benchmark checks the record against 824 rules. Merlin works from the same idea. Before Merlin claims a run finished, rules check the task ledger, the files read and the bytes written. The model's own summary of the work counts for nothing until the record agrees.

Thirty-six percent is not a reason to stop using these models. The number is a reason to stop taking their word for the result.

Your verifier is the thing you're verifying

RLM Labs — Thu, 22 Oct 2026 08:35:43 +0000

Most AI quality checks work like this: one model writes the answer, and a second model grades the answer. Ask what grades the second model. Usually nothing.

The research community measured the problem early. In June 2023, a Berkeley-led team studying language models as judges named the failure modes:

"position, verbosity, and self-enhancement biases"

Position bias: the judge prefers whichever answer comes first. Verbosity bias: the judge prefers the longer answer. Self-enhancement bias: the judge prefers answers resembling its own. The same paper found strong judges agree with humans about as often as humans agree with each other, above 80 percent. Read the number the other way round. On roughly one call in five, your grader and a human disagree, and nobody tells you which call.

Three years later, the vendors selling judge-based evaluation admit the same thing in their own documentation. LangChain, in March 2026:

"Early on, developers wrote rubrics and configured LLM judges hoping the scores would hold, but they didn't."

"An unvalidated judge might measure verbosity rather than whether the output is high-quality and genuinely helpful."

The second sentence is the 2023 verbosity bias, restated by a vendor in 2026. Their fix is to collect human corrections and tune the judge against them. Sound advice. Notice what the advice concedes: the judge is accurate only after you supply the ground truth the judge was supposed to replace.

AWS offers built-in judges for agent quality, with names like Helpfulness. You do not get to inspect or change how they decide:

"Built-in evaluator configurations, including their evaluator models and prompt templates, cannot be modified."

The review vendor Greptile states the problem with an animal. An agent reviewing its own code, Greptile writes, is

"like a fox guarding the henhouse."

And Anthropic, in a November 2025 engineering post on long-running coding agents, described the failure plainly. The model tended to mark a feature complete without proper testing. The recommended fix:

"Self-verify all features. Only mark features as “passing” after careful testing."

The fox, asked to audit the henhouse more carefully.

Google's Harness² paper, October 2026, is the most honest version I have read. The method improves agents by comparing runs with model judgment, and the authors write in their limitations section about the method's signals:

"remain sensitive to execution variability and shared model biases."

Shared model biases. When the writer and the grader come from the same family of models, their blind spots overlap. A second opinion from a twin is not a second opinion.

We learned this the expensive way. In one of our own production runs, the model asked to authorize a final answer was willing to call a run complete with 21 tasks still pending, zero operations performed and zero files written. So Merlin's model authorizer gets a vote and never the last word. Rules counting, hashing and looking up records run around the model and override the model.

Here is the test I would put to any verifier in your stack. Ask what the verifier reads. If the verifier reads the answer, the verifier shares the answer's weaknesses. If the verifier reads the record of what happened, the verifier checks something the model had no chance to argue with.

Non-deterministic is a design choice

RLM Labs — Wed, 21 Oct 2026 09:35:55 +0000

The most successful AI framework company defines its own field as a problem without an end. You should not accept the definition.

LangChain, announcing its Series B in October 2025, described the work the company sells tooling for:

"the iterative process of refining non-deterministic LLM systems into reliable experiences"

Read the sentence as an engineer. The system is non-deterministic, and the job is to keep refining the system toward reliability. No finish line appears in the sentence. The product is the treadmill.

I do not dispute the first half. A language model is non-deterministic. Even with a fixed seed and an identical request, GPU arithmetic does not promise identical output twice. Anyone telling you otherwise is selling you something.

I dispute the conclusion everybody draws from the first half: since the model is non-deterministic, the whole system must be. The conclusion does not follow. You choose where determinism lives.

Two of the largest companies in the industry already made this choice, for actions. AWS writes about its agent policy layer:

"consistent, deterministic enforcement that remains reliable regardless of how the agent is implemented."

Apple, in a WWDC26 session on securing agentic features in apps, told developers where to start:

"we should try to focus on deterministic mitigations as a baseline because their security guarantees are easier to audit and reason about."

Easier to audit and reason about. Keep the phrase. A probabilistic check gives you a probability. A deterministic check gives you a rule you read, test and argue with.

Both companies apply the principle to what a model does. Neither applies the principle to what a model claims. The answer at the end of a run still gets judged, if anyone judges the answer at all, by another probabilistic system.

Merlin moves the line. The model stays non-deterministic. The model plans, reads, writes and proposes an answer, and two runs of the same request take different paths. After the proposal, deterministic rules get the last word. A model still gives an opinion on the answer, and rules run before and after the opinion and override the opinion. A rule counts open tasks. A rule checks whether a cited line exists in a file the run read. A rule compares the bytes Merlin verified with the bytes Merlin committed. Each rule gives the same verdict every time on the same record, and the rules carry tests pinning those verdicts.

The result is a system whose path varies and whose standard does not. You will not get the same run twice. You will get the same refusal twice, for the same reason, on the same evidence.

The objection I hear most: deterministic rules are rigid, and rigid rules block good work. Sometimes they do, and we publish those cases in this series too. A rigid rule blocking good work shows up in a log with a name and a line number, and you fix the rule. A probabilistic judge waving through bad work shows up nowhere. I will take the failure I see over the failure I never find.

Non-deterministic models are a fact. Non-deterministic systems are a decision. You get to make the decision differently.

The limiting factor isn't capability

RLM Labs — Tue, 20 Oct 2026 05:10:39 +0000

Models write good code now. Ask anyone shipping with them. The problem moved, and most of the industry is still arguing about the old one.

Reuben Brooks put the shift in two sentences in May 2026:

"Existing models can already write almost all of your code."
"The limiting factor is whether you can know that they did what you wanted."

OpenAI reached the same place from inside their own team. In February 2026 they described five months of building a product with zero hand-written code. Here is what slowed them down at the start:

"Early progress was slower than we expected, not because Codex was incapable, but because the environment was underspecified."

And in October, Google Cloud AI Research opened the Harness² paper with a line you should read twice:

"Even a strong model can perform substantially differently under different harnesses"

Three groups, three different starting points, one conclusion. The model is not the bottleneck. The system around the model is.

I agree, and I bet the project on the idea before the industry had a word for the idea. On 31 December 2025, three MIT researchers published a paper on recursive language models: keep a long input outside the model's context and let the model work through the input in pieces, from a programming environment. Eighteen days later, on 18 January 2026, I made the first commit to Merlin. The design document lists what the system does not do. The first non-goal reads:

"Modify or retrain any LLM"

No fine-tuning. No custom model. Every improvement had to come from the structure around a model somebody else trained. The industry started calling this harness engineering in February 2026. We were already six weeks into building one.

Here is where I part ways with most of the harness work I read. The OpenAI team answered the problem with more autonomy: agents reviewing agents, humans optional in review, and as few blocking merge gates as possible. In their own words:

"Humans may review pull requests, but aren't required to."

For their goal, shipping speed on an internal product, the trade makes sense.

For mine, the trade runs backwards. If the hard part is knowing whether the work was done, then every layer you add without a check makes the hard part harder. A second model reviewing the first model adds an opinion. A rule reading the record adds a fact.

So the question I care about is narrower than how capable the model is. I want the system to show you, with evidence you check yourself, what the model did and did not do. Capability improves every quarter without any help from me. Evidence improves only when somebody builds for evidence.

Every post after this one comes back to the same split. The model proposes. Something outside the model decides what the system has earned the right to claim.

Every guardrail points at the wrong thing

RLM Labs — Mon, 19 Oct 2026 08:21:58 +0000

The big platforms ship real controls for AI. Read where each control sits. Every one of them guards what the AI does. None of them guards what the AI tells you.

AWS describes the policy layer of its AgentCore platform this way:

"Every agent action through Amazon Bedrock AgentCore Gateway is intercepted and evaluated at the boundary outside of agent's code - ensuring consistent, deterministic enforcement that remains reliable regardless of how the agent is implemented."

Deterministic. Outside the agent's code. Good engineering. Now look at the object: every agent action.

Google introduced its Agent Gateway in April 2026:

"Agent Gateway provides air traffic control to ensure interactions between agents and data is secure."

Anthropic's Agent SDK lists its permission system like this:

"Control which tools run automatically, which need approval"

AWS's open-source Strands framework now groups its controls under a heading:

"Gate tool calls with interventions"

Four vendors. Four checkpoints. All four sit on the action path: the tool call, the data access, the side effect. The newest AWS feature, temporal policies, goes further and reasons over a whole session, for example blocking a money transfer until an approval exists. Still an action.

Here is what none of those controls inspect: the sentence at the end. The summary telling you the migration finished. The report saying the audit found no issues. The final answer saying all 25 tasks are done. You read the answer and you act on the answer. On these platforms, nothing deterministic stands between the model and the claim.

You might say the grounding vendors cover this. Some do, partly. Contextual AI ships a model built to refuse when its documents do not support an answer. Cleanlab and Galileo score responses for errors. Every one of those sits beside an orchestrator built by someone else, and most of them grade with another model. I cover the trouble with a model grading a model in a later post.

We put the gate on the claim. Here is a real example from our own logs, 1 August 2026. A Merlin run finished 24 of 25 tasks and wrote 11 files. The model proposed a final answer saying the job was done. A six-line rule rejected the answer:

route              REJECT_UNSUPPORTED_CLAIMS
source             deterministic_guard_guard_3c
reason             full-success final is forbidden while task_summary
                   has pending or in-progress tasks

The rule did not read the prose. The rule counted open tasks, found one, and refused to let a claim of full success exist. The 11 files still shipped, with a manifest of what changed. Only the false sentence died.

Action gates matter. Keep them. A model with production credentials needs a fence around the tools. But the fence answers a different question. The fence answers "was the model allowed to do this." You also need an answer to "is the model telling the truth about what got done." Right now, on the platforms quoted above, you build the second answer yourself.

The test I use: pick any guardrail in your stack and ask what object the guardrail inspects. If the object is a tool call, the guardrail protects your systems. If the object is the final claim, the guardrail protects you.

Nobody will vouch for the answer

RLM Labs — Sun, 18 Oct 2026 20:34:26 +0000

Microsoft says so in the overview page for its Agent Framework, the product meant to help you build on its models:

"You are responsible for carefully reviewing and testing applications you build using Microsoft Agent Framework in the context of your specific use cases, and making all appropriate decisions and customizations. This includes implementing your own responsible AI mitigations such as metaprompt, content filters, or other safety systems, and ensuring your applications meet appropriate quality, reliability, security, and trustworthiness standards."

The Allen Institute says so on the model card for OLMo, one of the most open models anyone has shipped:

"many statements from OLMo or any LLM are often inaccurate, so facts should be verified."

LangChain, the most widely used framework for building on these models, defined its whole category in its Series B announcement as:

"the iterative process of refining non-deterministic LLM systems into reliable experiences"

Read the word iterative. Reliability is something you work toward, release after release. Nobody promises you arrive.

Then, on 9 October 2026, Google Cloud AI Research published Harness², a framework for improving the system around a model without touching the model. Good work. The paper reports gains on three professional benchmarks. In the limitations section, the authors write this about the signals their method uses to decide whether a change helped:

"They do not guarantee correctness or establish that an adopted change will transfer to a different task."

I respect the honesty. I also want you to notice the pattern. The model vendor, the framework vendor, the platform vendor and now the research lab building self-improving systems all draw the same line. Each one improves the odds. None of them stands behind the answer.

The check falls to you. Every page quoted above hands the check to the person reading the output.

I spent thirty years as the person flown in when a distributed system misbehaved in production. You learn one thing fast in the job: the system from the demo is not the system in front of you, and the only thing you trust is what you checked yourself. The log line. The row count. The diff.

AI moved the same problem up a level. The output reads well. The output reads well whether or not the output is right. And the people selling you the output have written the rule into their own documentation: checking is your job.

We built Merlin on the opposite assumption. If nobody vouches for the answer, the system producing the answer has to carry the evidence for the answer, and has to refuse to claim anything the evidence does not support. Not as a feature you switch on. As the rule the whole system runs under.

The rest of this series shows what goes wrong, including in our own system, and what we built instead. Start with the question these quotes leave you holding. Stop asking whether the model is good. Ask who checks the answer.