DEV Community: Conor Bronsdon

Let your LLM say I don't know instead of guessing

Conor Bronsdon — Sun, 01 Nov 2026 20:30:07 +0000

Your user asks a factual question. The model returns a polished paragraph. Nothing in the stack threw an error. That is exactly when you are most exposed. A language model generates the most plausible next words, not the most true ones, so a wrong answer can look identical to a right one. That mechanism is what people call AI hallucination: same token prediction, missing or weak facts, fluent fiction at the end.

Dan Klein, co-founder and CTO of Scaled Cognition, put the internals this way on Chain of Thought: we label outputs after the fact, but the model never splits “truth” from “guess” on its own.

If they're right, we call them truth. And if they're wrong, we call them hallucination. But the model does not internally distinguish between those.

Dan Klein, co-founder and CTO of Scaled Cognition

I’d stop treating confidence in the UI as a proxy for correctness. Treat every fluent sentence as a claim you still owe evidence for.

Why does fluency hide the errors you care about?

Obvious mistakes get caught in QA. Klein’s audit story from enterprise work is worse: when his team audits enterprise systems, he says, the real hallucination rate is often about five times what the customer has noticed, because the rest are fluent enough to pass. His examples are a refund policy that is real and plausible but not yours, or yours but for a customer with a different status.

Looks right. It looks right and is right are not the same. It is very hard to detect hallucinations when they are packaged up sufficiently fluently.

Dan Klein, co-founder and CTO of Scaled Cognition

Chip Huyen made the data angle plain on Chain of Thought ep 5: missing or wrong context pushes the model toward invention.

Models are more likely to hallucinate when it does not have access to the necessary and correct information to answer questions.

Chip Huyen, author of AI Engineering

So the design job is to give the model the facts at answer time, allow silence when facts are missing, and verify what still matters. A smarter model does not remove that work.

What breaks when you only punish wrong guesses?

Pre-training shows the model fluent text, not labeled false statements. Rare facts (like a specific birthday) do not repeat as patterns, so the model still produces something that sounds like an answer. Benchmarks make the incentive worse: “I don’t know” often scores like a wrong answer, while a random guess sometimes hits. OpenAI’s 2025 write-up on why language models hallucinate argues for penalizing confident errors more than honest uncertainty. Anthropic’s interpretability work on Claude describes a default to decline on unrecognized entities, with a “known entity” signal that can flip that off and invite a guess anyway.

For your app, that means instructions that forbid uncertainty fight the same scoreboards that trained the model to guess. If your eval sheet marks abstain as a miss, you are training your team to ship guessers.

How do you ground answers and still leave an out?

Grounding means building the reply from material you put in context at question time, ideally with citations per claim. That cuts invented detail because the model has less room to freestyle. It does not guarantee truth: sources can be stale, retrieval can miss the doc, and the model can still stretch past what the sources say. Many production issues that look like “model bugs” are retrieval or document problems first.

General models also carry a knowledge cutoff: training frozen at a date, then released later. Ask about anything newer and you get “I don’t know” or a plausible guess. The standard fix is the same as grounding: fetch current or private text into the context window before generation (RAG, search, or both).

My default pattern for factual features:

Retrieve candidate passages for the question.
If nothing passes a relevance threshold, return abstain (or escalate to a human). Do not ask the model to “try anyway.”
If you have passages, instruct answer-only-from-these, cite span or doc id per claim.
Run a faithfulness check on answers that can move money, policy, or compliance.

Step 4 is where AI evaluation and reliability work pays off: separate counts for answered, abstained, answered_supported and answered_unsupported.

What does an abstain-and-ground sketch look like?

Label this as illustration, not production code from any guest or vendor.

# Sketch: abstain if retrieval is weak; answer only from chunks

MIN_SCORE = 0.42  # tune on your corpus

def answer(user_question: str, retrieve, llm, faithfulness_check):
    chunks = retrieve(user_question, top_k=8)
    best = max((c.score for c in chunks), default=0.0)
    if best < MIN_SCORE:
        return {"status": "abstain", "message": "No trusted source found."}

    prompt = {
        "instruction": (
            "Answer using ONLY the sources. "
            "If they do not support an answer, reply ABSTAIN."
        ),
        "sources": [c.text for c in chunks],
        "question": user_question,
    }
    draft = llm.complete(prompt)
    if "ABSTAIN" in draft:
        return {"status": "abstain", "message": "Model declined."}

    if not faithfulness_check(draft, chunks):
        return {"status": "abstain", "message": "Failed source check."}

    return {"status": "ok", "text": draft, "sources": chunks}

The abstain path is a first-class HTTP response, not a thrown exception buried in logs. Users should see “we don’t know” when you mean it.

Checklist

Decide where a wrong fluent answer is costly; tighten grounding and checks there first.
Retrieve before generate; abstain when retrieval is empty or weak instead of prompting harder.
Tell the model abstaining is allowed; score abstain better than a confident wrong answer in evals.
Check high-stakes outputs against supplied sources (faithfulness), not vibes.
Remember fluent output is plausible text until your sources and checks say otherwise.

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Five lessons for getting enterprise AI past the pilot phase

Conor Bronsdon — Sun, 01 Nov 2026 15:30:06 +0000

Demoing AI is the easy part. Putting it in front of thousands of people, tying it to outcomes, and keeping it safe when models and data sprawl is the work. These five episodes from the Chain of Thought collection for enterprise AI leaders each point at one decision developers and product engineers control, even when leadership wants speed.

How do you deploy agents company-wide without a playbook?

How Block Deployed AI Agents to 12,000 Employees in 8 Weeks w/ MCP is the clearest story of breadth plus speed: Block put its open-source agent Goose in front of roughly 12,000 employees in about eight weeks, with Angie Jones describing deployment with no playbook already written.

For builders, the practical split is adoption versus control. Block leaned on security guardrails so wide rollout stayed defensible, and supported letting employees choose their own models (Claude, GPT, Gemini, plus open and local options in the discussion) because that choice drove uptake. MCP shows up as the connective tissue for hooking agents to real systems, with human-in-the-loop controls, OAuth, and tool annotations called out as part of the guardrail stack.

If you are past a pilot, I would treat model lock-in and missing guardrails as ship blockers before you chase more use cases. Ship one agent path with explicit approval steps for destructive tools, then expand connectors.

Why does so much enterprise AI spend fail to show ROI?

Why Most Enterprise AI Projects Fail to Show ROI still works as the straight ROI entry point. HP, ServiceNow, and Accenture walk through why heavy investment often ends in a vague "what did we get?" question instead of a number anyone trusts.

The builder takeaway is to pick use cases you can prioritize, implement, and measure, and to treat trust and explainability as part of delivery from the start. Alex Klug at HP frames risk in customer-facing terms: nobody wants to deploy something that steers customers wrong. That is a useful filter when product asks for AI in every surface.

Before you add another copilot button, write down who gets hurt if the model is wrong and whether you can detect and roll back bad outputs. If you cannot, you are still in pilot clothing.

How do you find where AI pays off instead of token maxing?

Stop Token Maxxing: Find Where AI Actually Pays Off with Jiaona Zhang (CPO at Laurel) targets a common failure mode: top-down "use AI everywhere" mandates that push teams to burn tokens without learning what returned time or revenue.

Zhang's through-line is measurement over mandate. Laurel runs its own product on itself and uses time data to see where AI helps versus where teams only assume it helps. The episode also ties efficient use to org design (lean teams, non-engineers shipping with agents, bottom-up plus top-down re-architecture), but the decision rule for developers is simpler: instrument time before you celebrate usage stats.

Sketch (illustration only, not Laurel's production code):

# Sketch: log task time before/after AI assist for one workflow
events = []  # {"workflow": str, "minutes": float, "used_ai": bool}

def median_minutes(workflow: str, used_ai: bool) -> float:
    times = [e["minutes"] for e in events if e["workflow"] == workflow and e["used_ai"] == used_ai]
    times.sort()
    return times[len(times) // 2] if times else float("nan")

Compare medians per workflow with and without AI. Kill mandates that raise token charts but flatline time saved.

What if the agent calls the model fine but cannot reach the data?

Data Federation, Not Centralization, Is What Enterprise AI Needs with Jitender Aswani (Starburst) argues that stalled enterprise AI often loses a hidden fight: queries to the model work, access to data spread across dozens or hundreds of sources does not. The collection summary puts typical enterprises at 52 to 200 data sources; centralize-everything strategies pile ETL debt and brittle pipelines when SaaS schemas shift.

Aswani treats fragmentation as something you design around. Federation and query layers are his architectural direction, and MCP shows up in the episode as a driver of query volume. For application developers, the lesson is to stop blocking pilots on a mythical single lake and start exposing governed, federated access paths agents can call.

When an agent "hallucinates finance numbers," check data reachability before you fine-tune the prompt.

What separates enterprise AI that ships from AI that demos?

Inside IBM's watsonx: Building Enterprise AI That Ships with Dr. Maryam Ashoori, Head of Product for watsonx AI at IBM, focuses on trustworthy, scalable delivery across tools and regulations. Ashoori discusses simplifying the AI stack, automating pipelines, data management and lineage, scaling agentic systems, and effective agent evaluation, backed by survey findings from 1,000 developers on how enterprise AI implementation is going in the wild.

I'd take "ships" to mean repeatable pipelines, eval systems you trust enough to gate releases, and cost awareness as you scale agents. Demos skip lineage and eval; production cannot.

Checklist

Pair wide agent rollout with explicit security guardrails and model choice where that choice drives adoption (Block/Goose pattern).
Prioritize ROI-ready use cases only where trust, explainability, and customer harm if wrong are spelled out (HP/ServiceNow/Accenture panel themes).
Replace "AI everywhere" mandates with time-based measurement per workflow so you stop token maxing (Jiaona Zhang, Laurel).
Assume 52 to 200 data sources and plan federation and governed access instead of waiting for a finished centralization project (Jitender Aswani, Starburst).
Treat simplified stacks, automated pipelines, lineage, and agent evals as shipping requirements (Maryam Ashoori, IBM).

More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Price a coding call, then pick the model tier

Conor Bronsdon — Sat, 31 Oct 2026 20:30:08 +0000

For a short coding task, price the fast tier and the flagship on the same prompt, then keep the cheapest run that still returns code you would ship. On the pricing Paige Bailey read off the screen, Gemini 3 Flash input was about 25% of Gemini 3 Pro, a bit lower with a large input context, and output was about 25% as well. With thinking set to minimal, Flash wrote a lat/long helper in about one tenth the time, and the call came in at less than a penny.

Paige Bailey is the AI developer relations lead at Google DeepMind. The episode aired in January 2026. She opened compare mode in AI Studio and ran one Python prompt on Gemini 3 Pro and Gemini 3 Flash.

How do you choose a model tier for a short coding task?

Run the task on the fast tier with thinking turned down, run it on the flagship, and keep the cheaper result that matches the code you wanted. Bailey framed Gemini 3 Pro as the flagship, with thinking built in, a larger model, and a much higher price. Gemini 3 Flash can turn thinking on, at what she called a significant discount next to Pro.

In that compare Pro's thinking level was high by default, and she set Flash to minimal. The prompt asked for a Python function that returns lat/long for a city. Flash returned while Pro was still thinking. Sources were cited, both answers used GeoPy, and she called the Python more or less the same. That output had no example usage.

For folks who can't see, the input is about 25% of what you would expect to pay for Gemini three Pro, even a little bit lower if you have significant number of tokens in your context window for inputs. And then the output is also significantly less expensive, just coming in at about 25%.

Paige Bailey, AI Developer Relations Lead at Google DeepMind, on Chain of Thought ep 47

Gemini 3 & Robot Dogs: Inside … - Chain of Thought | AI Agents, Infrastructure & Engineering - Apple Podcasts

Podcast Episode · Chain of Thought | AI Agents, Infrastructure & Engineering · January 14 · 51m

podcasts.apple.com

Read the current per-million rates, then apply them to token counts from a run of your own.

When do you turn thinking on?

Leave thinking at minimal when the fast tier already returns the function you needed. Turn it up when you want a fuller answer and you can afford the extra tokens and the wait.

Gemini three Pro, obviously, our flagship model. It has thinking baked in, which means that the answers you're going to get are probably going to be much more fully expressed, but they'll probably take a a lot longer to get you the answer that you're hoping for

Paige Bailey, AI Developer Relations Lead at Google DeepMind, on Chain of Thought ep 47

On the city lookup, she tied the savings to not using all of the intermediate thinking tokens on that run. Measure the thinking tokens you were billed. Flash still has a switch if a later task needs thinking. Minimal thinking fits a narrow function with an obvious library. Raise it above minimal when the prompt asks for a design, several files, or a justification you will review. One demo where the Python matched does not settle the next prompt.

How do you estimate the cost of one coding call?

Multiply input tokens and output tokens by the per-million prices you supply for that tier, then pick the cheapest tier that passed a quality check you recorded. On hover, the compare view showed input cost, output cost, and total. The Flash total for that function was less than a penny.

The sketch below does the same arithmetic. It takes prices as parameters, so you can pass the card you are billed on.

A minimal sketch to adapt, not a drop-in library.

def call_cost_usd(
    input_tokens,
    output_tokens,
    thinking_tokens,
    usd_per_million_in,
    usd_per_million_out,
):
    billed_out = output_tokens + thinking_tokens
    return (
        input_tokens * usd_per_million_in
        + billed_out * usd_per_million_out
    ) / 1_000_000


def cheapest_tier_that_passed(runs, prices):
    passed = [run for run in runs if run["passed"]]
    if not passed:
        return None

    def cost(run):
        rate = prices[run["name"]]
        return call_cost_usd(
            run["input_tokens"],
            run["output_tokens"],
            run["thinking_tokens"],
            rate["usd_per_million_in"],
            rate["usd_per_million_out"],
        )

    return min(passed, key=cost)["name"]

Feed it token counts from the two runs, including the thinking tokens you were billed, and the per-million input and output prices from the card you are billed on. Check the billing documentation for how thinking tokens are charged and whether reported output counts already include them. On that compare she tied Flash's lower token use to not using all of the intermediate thinking tokens, and she put Pro's slower answer down to those tokens and its larger size. If nothing passed, the function returns no tier. Change the prompt or the check before you accept the cheap failure.

Mark a run as passed only from checks you can rerun. The function should do the job, verified by a tiny fixture you wrote. Dependencies should be ones you will ship. Both of her samples pulled in GeoPy; if that is not a dependency you will ship, both runs fail. Latency should fit the product: about one tenth of the wait matters in an inline editor and matters less in a nightly job. Cost at your volume should still look like that penny-scale call once you multiply by how often the path runs. Those figures come from one prompt, so copy the shape of the test.

FAQ

Is Gemini 3 Flash about a quarter of Pro on both input and output?
On the comparison Bailey showed, input was about 25% of Pro, and lower than that with a significant number of input tokens. Output was about 25%. Confirm the card for the context length you send.

Does the fast tier have thinking at all?
Pro had thinking baked in, and in that UI the thinking level defaulted to high. Flash let her turn thinking on. She set thinking to minimal for the lat/long prompt. Measure the thinking tokens that call billed.

Will both tiers emit the same code?
On that one prompt, she said the Python was more or less the same and both used GeoPy. Treat that as a single data point. Keep a fixture so a later prompt can fail the bar even when it is cheap.

Takeaway

Run one real coding prompt on the fast tier with thinking minimal and on the flagship with thinking at its default.
Record input tokens, output tokens, wall time, and total cost from the run.
Use about 25% of Pro on input and on output only as the ratio Bailey showed, and pass today's per-million prices into your own estimator.
Raise thinking above minimal when the short answer is too thin, and count the thinking tokens you were billed.
Ship the cheapest tier that passed your fixture, dependency check, latency limit, and budget.

The full conversation, with the transcript, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Count transcript tags in a podcast RSS feed

Conor Bronsdon — Sat, 31 Oct 2026 15:30:08 +0000

Five of eight AI podcasts in a June 11, 2026 feed audit declared zero transcripts in RSS. Across the 2,148 items those feeds served, 190 carried at least one podcast:transcript tag (8.8%). An absent tag does not establish whether a transcript exists on a website, video platform, or elsewhere. If you ship a feed, or you parse one, count the tag and keep the count inside what a declaration can mean.

What does a podcast:transcript tag declare?

The Podcasting 2.0 transcript tag links to a transcript or captions file. It tells apps, directories, and crawlers where the text of an episode lives. The figures below come from a benchmark of public RSS feeds and Apple's iTunes lookup API, fetched 2026-06-11. That write-up sits with the other original-data reports on the reports page. The audit counts declarations. It does not test which answer engines consume them, whether the linked files work, or whether adding tags changes citations or search traffic.

A September 5 clarification narrows how to read the original June data. It adds no new feed observations and makes no claim about current coverage or measured search impact. Use these percentages as a snapshot.

Chain of Thought publishes a full transcript on its site for every episode, which is also why this report exists. The feed tag is a separate check from the page on the site.

How do you count tagged items in a feed?

Transcript availability in the audit is the percent of in-feed episodes carrying at least one podcast:transcript tag. Catalog depth came from Apple trackCount. In this run, every show's feed served its full archive, so feed item counts matched those catalog counts, and feed-derived metrics cover whole catalogs. Some hosts cap feeds to recent items. This run was not truncated. A re-run on other shows should check that before you treat the item count as the whole catalog.

The sketch only counts tags in XML you already have. It does not fetch a feed, and it does not open the URL on the tag. It is an illustration, not the command-line tool that produced the benchmark.

# Sketch only. Counts item-level transcript tags in RSS XML you already have.
import xml.etree.ElementTree as ET

def _is_transcript(tag: str) -> bool:
    name = tag.rsplit("}", 1)[-1]
    return name == "transcript" or name.endswith(":transcript")

def count_transcript_tags(xml_text: str) -> tuple[int, int]:
    root = ET.fromstring(xml_text)
    channel = root.find("channel")
    items = [] if channel is None else channel.findall("item")
    tagged = 0
    for item in items:
        if any(_is_transcript(child.tag) for child in list(item)):
            tagged += 1
    return tagged, len(items)

I'd start with that pair: tagged items, then items the feed served. Percent is tagged divided by items. That is how 190 of 2,148 became 8.8%. The pooled percent weights every feed item equally, so a large catalog moves the total more than a small one.

What did the eight-show snapshot report?

The benchmark sets Chain of Thought against seven AI engineering podcasts: Latent Space, Practical AI, The TWIML AI Podcast, Gradient Dissent, AI Engineering Podcast, No Priors, and The Cognitive Revolution. Every number comes from those public feeds and the Apple lookup, collected 2026-06-11 (07:26 UTC).

Five shows declared zero transcripts: Latent Space (0 of 207), The TWIML AI Podcast (0 of 787), Gradient Dissent (0 of 137), No Priors (0 of 166), and The Cognitive Revolution (0 of 349). Chain of Thought led at 98.4% (61 of 62). Its untagged item was the trailer. AI Engineering Podcast followed at 93.7% (74 of 79). Practical AI was at 15.2% (55 of 361).

The 8.8% aggregate weights items equally. The TWIML AI Podcast accounts for 787 of the 2,148 sampled items and had no transcript declarations. Read the pooled count beside the per-show table. Neither the aggregate nor this peer set estimates adoption across all AI podcasts.

Chain of Thought and Practical AI both used Transistor feed URLs and still differed sharply. A hosting provider's name alone cannot explain that. Hosting support, configuration, and catalog history are possible explanations. They were not measured.

A feed score is also silent on episode subject. For a separate list of episodes on how AI agents get built, deployed, and evaluated, see the best AI agent podcast episodes. If you want five Latent Space episodes paired with a Chain of Thought episode on the question next door, that page is podcasts like Latent Space.

What does the percentage fail to prove?

Here, 0% means zero transcript declarations in the sampled feed. A show can publish transcripts on its website without emitting the tag, and then it reads as 0% here. The metric measures feed-declared transcripts, not every transcript that exists.

Declared transcript URLs were not validated for availability, format correctness, or transcript accuracy. Declaration coverage does not establish usable transcript coverage. A tag exposes a link to applications that support it. The audit does not establish which applications use the link or how they rank the episode.

High declaration coverage does not establish an advantage in audience reach or AI citations. Testing that relationship would require separate outcome measurements and a design that accounts for other differences between shows. The report says nothing about audience size. Download numbers stay private to each show, and the audit left them out because estimating them would make the report worthless.

This is a point-in-time snapshot of eight shows chosen as Chain of Thought's peer set. It is not a census of AI podcasts, and a different peer set would shift the rankings. Catalog counts can differ by a few episodes between sources depending on trailers and crawl timing.

Checklist

Count items with at least one podcast:transcript tag, then divide by items that feed served.
Date the fetch. These figures are from 2026-06-11, and the September 5 note adds no new feed observations.
Read 0% as no declaration in that feed. A website or video transcript can still exist.
Put the pooled 8.8% next to per-show counts. Large feeds, including TWIML's 787 items, move an equal-weighted total.
Stop before claiming the linked file works, that any app consumes it, or that the tag changes citations or search traffic.

More on this topic, with the related episodes, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Four agent metrics that a final accuracy score hides

Conor Bronsdon — Fri, 30 Oct 2026 20:30:10 +0000

Score every recorded agent run on four process checks: constraint satisfaction, tool-call correctness, loop termination, and escalation to a human. Compute each one from the trace: which declared constraints passed, whether every tool call had a valid tool and valid parameters, whether the run reached a stop state, and whether a human took over.

Chip Huyen, then a storyteller at Tép Studio, and Vivienne Zhang of NVIDIA, spelled out those failure points on Chain of Thought in December 2024. Alex Ratner, co-founder and CEO of Snorkel AI, described the measurement shortfall from the other side in April 2026.

How do you check that an agent stayed inside the constraints?

Write the constraints down before the run, then score the trace against that list. Chip Huyen's planning example is a two-week trip with a budget of 5,000. The plan has to cover two weeks and stay inside that budget. A fluent itinerary that breaks the budget fails.

Long outputs need the same item-by-item check. Alex Ratner described legal deep-research tasks that might take a human five, 10, or 15 hours, with multiple tools and multiple citations. Compare that with a math item whose output is a single number you can check against an answer key. For a 30-page PDF with 50 citations, you need a grading rubric, because you cannot settle the output by asking whether A equals B. Store each rubric item as a boolean on the run. Constraint satisfaction is true only when every item you declared has passed.

whenever I try to evaluate some use case, I would probably try to break down, like, how is this application going to fail. It tries to put in a lot of evaluation metrics around the points of failures.

Chip Huyen, then Storyteller at Tép Studio, on Chain of Thought ep 5

Practical Lessons for GenAI Ev… - Chain of Thought | AI Agents, Infrastructure & Engineering - Apple Podcasts

Podcast Episode · Chain of Thought | AI Agents, Infrastructure & Engineering · December 4, 2024 · 48m

podcasts.apple.com

How do you measure tool-call correctness?

Count a tool call as correct only when the tool is valid and the parameters are valid. Chip Huyen named both misses. The agent calls a tool it does not have, or it calls the right tool with the wrong parameters. In the weather case, the model has to extract a location and a time, and those values have to be valid. The right tool name with a bad location still fails the call.

Log each call as that pair. The run passes tool-call correctness when every call passes both halves. When the task requires a specific tool, put that requirement on the constraint list, so a run that never calls anything does not pass on an empty call list.

How do you tell a finished loop from a run that does not stop?

Mark the run terminated only when it reached a stop state, and store the step count beside that flag. Chip Huyen described a very long sequence of actions that might go on and on and fail to stop in the time available. Alex Ratner treats length as its own axis. Autonomy horizon is the number of steps a task needs, including goals that change while the task is running. A coding challenge with one unit-test answer is the short end. The long end is a multi-turn task with shifting specifications. Measuring out to the length of a real-world task is still at the frontier. Very few public benchmarks test it, and most of those are unsaturated.

Pick a step cap for the task before you score a suite. Hitting the cap without a stop state fails loop termination. Report the termination rate and the step counts together so a prompt or model change that lengthens the path is visible.

A minimal sketch to adapt, not a drop-in library.

def score_agent_runs(runs):
    if not runs:
        return {}
    n = len(runs)

    def tool_calls_ok(run):
        return all(
            c["valid_tool"] and c["valid_params"]
            for c in run["tool_calls"]
        )

    return {
        "constraint_satisfaction": sum(r["constraints_met"] for r in runs) / n,
        "tool_call_correctness": sum(tool_calls_ok(r) for r in runs) / n,
        "loop_termination": sum(r["terminated"] for r in runs) / n,
        "mean_steps": sum(r["steps"] for r in runs) / n,
        "human_escalation": sum(r["escalated"] for r in runs) / n,
    }

Fill each run from your own logs. constraints_met is true when every constraint you declared passed. Each entry in tool_calls carries valid_tool and valid_params. An empty call list makes tool_calls_ok true, so a required tool belongs on the constraint list. terminated is true only when the run reached a stop state within your step cap, and steps is how many steps it took. escalated is true when a human took the task over.

And yet, you can ask a coding agent to do a much more mundane-seeming task, but one that has messier inputs and messier outputs and a longer set of steps required, and it might fall down.

Alex Ratner, Co-founder and CEO at Snorkel AI, on Chain of Thought ep 57

When should escalation to a human be its own metric?

Count a handoff when the agent stops and sends the task to a person. Vivienne Zhang's familiar case is a customer-service chatbot. It keeps handling queries until it can no longer deal with the complexity, then sends the query to a human. Getting that handoff to a good user experience takes iteration on those same failure points. In the split she described, the human defines the goals and the parameters, and the agent carries out the means. Track how often a person reclaimed the task, and open the traces when that rate jumps after a prompt change.

And you will keep handling your queries until it decides it can no longer deal with the complexity of the query. And then you will send the query to a human and that will be the human in the loop.

Vivienne Zhang of NVIDIA, on Chain of Thought ep 5

FAQ

What is a constraint check on an agent plan?
A pass or fail against limits you set before the run. Chip Huyen used a two-week trip with a budget of 5,000: the plan has to cover the two weeks and stay inside the budget. For a long document, Alex Ratner said a rubric replaces a single answer-key comparison, as with a 30-page PDF that carries 50 citations.

What makes a tool call incorrect?
An invalid tool, or the right tool with invalid parameters. A weather call that does not extract a valid location and time fails the parameter half even when the tool name was right.

What handoff rate should I target?
Set it per task. Vivienne Zhang described a chatbot that hands off when the query is too complex, and that handoff should be evaluated on the same failure points as the rest of the agent. The target is a product decision for your own task.

Takeaway

Declare every constraint before the run, and store a boolean for each one.
Pass a tool call only when the tool and the parameters are both valid.
If the task requires a tool, put that requirement on the constraint list.
Store a step count and a terminated flag, and set the step cap before you score the suite.
Store whether a human took the task over, and read that rate with the other three.

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

What to change when agents write most of the code

Conor Bronsdon — Fri, 30 Oct 2026 15:30:07 +0000

When agents write most of the code, the scarce work moves downstream. Generation gets cheap, and the bottleneck moves to review and merge. In my reading, onboarding and choosing what is worth building get more expensive too. Five episodes on Chain of Thought each point at one change I would make in that world.

What should you review when you stop reading every line?

Anush Elangovan, Corporate VP of AI Software at AMD, lays this out on 250,000 Lines of Code a Week. He runs 10 to 12 Claude Code agents across three machines and burns 6.5 billion tokens a week. The claim I keep from that episode is that testing is the new code review. The chapter list frames the habit as 80% tests first. He argues that normal SDLC is dead in an agentic world, and that software comes down to tokens. In his account, speed is the only durable moat. The same conversation covers a 25-year-old Slurm replacement, rewritten as Spur in Rust in a single night, without opening the editor. An earlier chapter asks when AI-generated code becomes a liability. He describes agents as sneaky and dumb, and he runs them with dangerously-skip-permissions. I would leave that flag as his risk setting. I would start by requiring tests before an agent is allowed to widen the change.

# Sketch: a local check before merging agent output.
# Not a workflow used by any guest or company on these episodes.

def merge_allowed(change: dict) -> bool:
    if not change.get("tests_written_first"):
        return False
    if not change.get("tests_passed"):
        return False
    return True

How do you merge when agents open pull requests in the background?

Greg Foster, co-founder and CTO of Graphite, describes the review bottleneck on After Code Gen. He argues that the bottleneck has moved from writing code to reviewing it, once agents open pull requests in the background faster than humans can merge them. He outlines three waves of AI that lead to autonomous agents spawning those pull requests. He argues that as AI automates the inner loop of writing code, the outer loop of reviewing, merging, and deploying comes under pressure. The tactic I would copy is stacking. Split a large change into units small enough for a human or an agent to review. He also identifies a hiring gap. Experienced engineers with strong architectural context become lethal with AI tools. If you hold that context, spend it on how changes are split and how they land.

How do you onboard an agent that will own most of a day?

Sterling Chin, Applied AI Engineer and Senior Developer Advocate at Postman, treats staffing as the method on I Built an AI Coworker That Runs 90% of My Day. He stopped thinking of AI as a tool and started treating it like a junior employee. He onboarded it with context, corrected its mistakes, and gave it writing rules. Forty days later, MARVIN, built on Claude Code, was handling 90% of his workday. The episode is built around live demos. MARVIN bookends the day from first login to the end, using personality, sub-agents, and writing rules. The demos run from meeting transcripts to Jira tickets, drafted posts, and calendar management. Show notes call MARVIN a full-time AI chief of staff. Sterling has onboarded 12 or more colleagues at Postman, including non-technical knowledge workers, and he argues that DIY assistants outperform big tech alternatives. I would start with one assistant, written rules, and a correction habit. At a fixed checkpoint, count how much of the day it actually runs. Forty days and 90% are his reported outcome.

What should a coding model be scored on?

Jason Warner and Eiso Kant, CEO and CTO of Poolside, make the training case on First Code, Then AGI. They explain why Poolside trains for code specifically instead of general capability, and what reinforcement learning against real execution feedback buys a model. They believe synthetic data will help expand the surface area of software by 1000x. Their focus includes high-consequence systems. On the human side, they argue that agency, taste, and judgment become central for engineers working with AI coworkers. The buying decision I would make is narrow. Prefer a coding agent whose feedback loop includes real execution, and keep a person with taste on whether the passing behavior is the one you wanted.

Where should you point agents if you want a measurable payback?

Rachelle Palmer, Director of Product Management at MongoDB, covers the least glamorous case on Using AI to Modernize Your Legacy Applications. The collection page describes pointing AI at legacy codebases as one of the few coding-agent uses with an obvious payback measured in years off a timeline. Her forward-deployed AI engineering team works on complex, outdated codebases. She details LLMs automating documentation improvements, test generation, and business logic conversion. In her account, modernization timelines drop from years to months, and work that once demanded teams of dozens can be done by a small team. I would start on a legacy system already stuck on a multi-year plan. Constrain the agent to documentation, tests, or a bounded logic conversion, then compare the calendar you had with the calendar you have.

Checklist

Make testing the review gate. Anush Elangovan's episode frames this as tests first, with agents he describes as sneaky and dumb.
Stack changes into reviewable units, the tactic Greg Foster gives once background pull requests are the bottleneck.
Onboard one assistant like a junior employee, with context, corrections, and writing rules, then check results on a set day, as Sterling Chin reported after forty days.
Prefer coding models trained with reinforcement learning on real execution feedback, the bet Jason Warner and Eiso Kant describe, and keep taste with a human.
Point the first large agent effort at legacy modernization, where Rachelle Palmer describes timelines moving from years to months.

More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

The Minimum Agent Loop You Can Ship (and When to Skip It)

Conor Bronsdon — Thu, 29 Oct 2026 20:30:25 +0000

Most "agents" in production are a loop: the model picks a step, calls a tool, reads the result, and decides whether to stop. Name that loop and its stop condition before you call the thing an agent.

An AI agent is given a goal and runs until the goal is met or it gives up. A chatbot takes a prompt and returns a response. A bigger model does not turn one into the other. The system around the model does: tools, state, and something that decides "one more step" versus "done." Vikram Chatterji described that 2025 shift as generative AI moving from chat completion toward actual task completion.

What belongs in the smallest useful loop?

I'd start with four pieces only:

Model: produces the next intent (tool name and arguments, or a final answer).
Tools: functions that fetch data or have real side effects (search, SQL, creating a ticket).
Loop: append tool results to state, call the model again.
Stop condition: max steps, an explicit finish tool, or a terminal field such as done: true in the model's JSON.

Memory, multiple models and evals come next, and if production is close, evals belong on day one. Aishwarya Srinivasan frames the shift well: production agents are software systems to architect, and a benchmark winner alone will not get you there.

The following is a sketch (not production code, no real SDK):

# Sketch: minimum agent loop (illustration only)
from typing import Any, Callable

MAX_STEPS = 8
tools: dict[str, Callable[..., Any]] = {
    "search_docs": lambda q: f"results for {q}",
    "finish": lambda answer: {"answer": answer},
}

def run_agent(goal: str) -> str:
    messages = [{"role": "user", "content": goal}]
    for step in range(MAX_STEPS):
        plan = model_next(messages)  # your LLM wrapper
        if plan.get("type") == "final":
            return plan["text"]
        name, args = plan["tool"], plan.get("args", {})
        if name not in tools:
            messages.append({"role": "tool", "content": f"unknown tool: {name}"})
            continue
        result = tools[name](**args)
        messages.append({"role": "tool", "name": name, "content": str(result)})
        if name == "finish":
            return str(result)
    return "stopped: max steps"

That is the operational shape of an agentic workflow: chained decisions toward a goal, with the model choosing the next move inside bounds you set.

How does this relate to ReAct?

ReAct (Reason + Act) interleaves reasoning with one action at a time: reason, act, observe, repeat. Your sketch does not need the model to print long reasoning traces, but the control flow is the same. The model does not plan the full path up front; it reacts to what came back from the last tool call.

That adaptability has a cost. In a reason-act loop, an agent with incomplete context can keep firing queries against data it cannot access, without ever reasoning its way to "I can't answer this." Before you widen tool access, tighten the stop rules and cap steps. Autonomy without bounds is how you get runaway tool and query volume.

When should I use a loop instead of a fixed workflow?

Use a fixed workflow (scripted steps, deterministic branches) when:

The path is known before runtime (same steps, same order, every time).
Mistakes are cheap and easy to undo.
You can test every branch without the model inventing new ones.

Use the minimum loop when:

The number of steps depends on the user's goal or on tool output you cannot predict.
The model must choose among several tools based on intermediate results.
You are okay trading predictability for flexibility, and you will add checks later.

Do not reach for a loop because "agents" are the product story. Reach for it when a single prompt or a rigid pipeline cannot reach the goal without repeated tool use. Most shipping teams let the model decide within bounded steps, with guardrails and evaluation at each stage, rather than running a free agent. That middle ground is where a lot of practical building on AI agents actually lives.

What do I add after the loop runs once?

The canonical "chatbot to agent" stack names four production layers beyond the loop itself: orchestration across models, memory and context, tools, and evals with guardrails. The loop is the engine; those layers are how it stays trustworthy.

For builders stepping from demos to systems, Aishwarya Srinivasan's line is the guardrail I keep in mind:

It is not a model that you are surfacing to your user. It is a software. It is a system that you're surfacing to your user

Aishwarya Srinivasan, Fireworks AI, on Chain of Thought ep 42

Her production rule from the same conversation theme: quantify each step so when something breaks, you can see where and why. Your first loop can log step index, tool name, and latency before you add fancy evals.

So the short answer is trying to quantify anything and everything in that system that you're building.

Aishwarya Srinivasan, Fireworks AI, on Chain of Thought ep 42

Checklist

Can I write the happy path as a fixed DAG without a loop? If yes, start there.
If I need a loop, do I have a named stop condition (max steps, finish tool, or explicit terminal state)?
Are tool schemas small and bounded so the model cannot call arbitrary endpoints?
Does each iteration append observations to state so the model can recover from a bad tool call?
Before scaling users, do I log every step so I can trace failures (even if evals come later)?

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Measure an AI agent before you call it ready to ship

Conor Bronsdon — Thu, 29 Oct 2026 15:30:11 +0000

The Chain of Thought collection on AI evaluation has five episodes. Read in order, they make a ship gate: define the measurement, engineer the eval, check the trust bar, wire the eval into release, and seed it with the failures that matter.

How do I know the agent is still a demo?

An agent that looks finished in a demo is often unmeasured. On Every AI Agent Has an Evaluation Gap, Alex Ratner of Snorkel AI frames the evaluation gap as a hypothesis: AI capabilities have advanced faster than our ability to define and measure them precisely. The collection page ties that gap to the moment an agent leaves the demo, when capability has outpaced measurement.

Ratner maps the gap as what blocks agents from real enterprise work. In the same episode he walks through Snorkel's $3M Open Benchmarks Grant and explains why pure environment vendors don't actually understand how AI works. I would not treat an environment as a substitute for a measurement. If I cannot write down what better means on the task, the agent stays a demo.

Before I widen a rollout, I write a short definition of the task and a score I can recompute. If I cannot define the measurement, I cannot tell whether a change helped, and I cannot defend a deployment.

When does an eval deserve to run on the request path?

Explaining Eval Engineering with Vikram Chatterji treats evaluation as an engineering discipline. The episode summary describes the common habit: teams write evals like unit tests, check a box, and move on. That habit fails for agents that make real decisions for customers and spend real money. The marker I would follow is to author, test, and measure the evals.

The episode page adds that once an eval is accurate enough, it can run as a runtime control. I would author the eval, test it on cases I already understand, and measure how often it matches a reviewer. Only then would I put it on the request path. One green run does not justify that step.

What latency bar should I write down?

Trust is the bar for shipping agents. That is how the collection page frames The AI Agent Trust Gap with Philipp Krenn of Elastic. He recalls a 200-millisecond Elasticsearch query that once felt unacceptably slow, while a five-second LLM answer is now treated as perfectly fine. I would record both numbers for any system under review. The latency older search work rejected, and the latency the model is allowed to use, are one view of the trust gap.

If a five-second answer is being treated as fine, I want that decision written down next to the 200-millisecond bar Krenn remembers, with a guardrail on the same path as the agent.

Where should the eval sit when we ship?

How Engineering Leaders Can Future-Proof Their AI Strategy is a re-shared conversation with Andrew Zigler about evaluation-driven development, and the collection page states its case plainly: build the eval loop into how you ship.

I would make that loop a rule of the pipeline. A change to the agent runs its eval before release, and a failed eval has a named owner. I would delay a release until that run is part of the ship.

Which failure belongs in the first eval set?

The collection page calls Practical Lessons for GenAI Evals the most-cited starting point for evals on the show. Chip Huyen and Vivienne Zhang discuss practical GenAI evaluation as agents and multimodal models become more common. Generative AI is harder to assess than traditional software.

The first case I would save comes from Chip Huyen's account of LinkedIn's job-fit bot. Asked if someone was a good fit for a role, it answered that they were a terrible fit. That reply finishes the question and still fails a user-facing eval. I want a set that includes judgment failures of that kind.

The episode also covers objective criteria for open-ended tasks, hallucinations, and human-in-the-loop systems. I would score a small set with criteria a second person can apply, and I would keep a person in the loop until the eval is accurate enough to act as a runtime control.

# Sketch only: the order of checks above. Not a guest's system or a vendor SDK.
def ship_gate(measurement, eval_report, trust, ship_path, cases):
    if not measurement:
        return "demo"
    authored = eval_report.authored and eval_report.tested and eval_report.measured
    if not authored:
        return "prototype"
    if eval_report.accurate_enough:
        eval_report.use_as_runtime_control = True
    if trust.five_second_answer_treated_as_fine and trust.old_bar_was_stricter:
        return "trust_review"
    if not ship_path.eval_runs_before_release:
        return "eval_outside_ship_path"
    if any(case.harmful_judgment for case in cases):
        return "hold"
    return "ship"

Checklist

Write the measurement before the agent leaves the demo. If capability is ahead of that definition, you are still in the evaluation gap.
Author, test, and measure the eval. Use it as a runtime control only after it is accurate enough.
Record the latency you used to reject and the latency you now accept. Krenn's contrast is about 200 milliseconds versus a five-second answer.
Run the eval before release, with a named owner for failures, so the loop is part of how you ship.
Seed the set with judgment failures, such as a job-fit bot that tells someone they are a terrible fit, and keep a person in the loop until the eval earns the runtime role.

More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Lint your technical drafts for AI writing tells before you publish

Conor Bronsdon — Wed, 28 Oct 2026 20:30:07 +0000

A draft about your retrieval layer can read like a launch announcement and still say nothing you could test. Checking the prose is cheap. Checking the engineering claim is a different job. I treat AI-writing lint the way I treat a style checker on code: fast feedback on patterns, and zero proof that the logic is right.

Why does fluent technical writing fool you first?

A large language model is trained to produce likely text, not verified text, so a polished explanation can be wrong and still read like documentation. Dan Klein, co-founder and CTO of Scaled Cognition, described the problem in his episode: older language technology often showed its mistakes, because a translation that read badly also tended to be unfaithful. A technology that always sounds right but often is not makes it hard to know what to trust.

Like I said, one of my biggest fears is that, as a field, we are building systems which produce outputs indistinguishable from the truth.

Dan Klein, on Chain of Thought ep 54

The Chain of Thought writing page makes the same point with a made-up paragraph about a retrieval layer. It marks four patterns from the open-source avoid-ai-writing catalog, and then notes that fixing them makes the paragraph read better while telling you nothing about whether the retrieval layer reduces hallucinations.

Which four patterns should you mark?

The page labels these:

Significance inflation: the sentence claims historic weight without saying what changed. Ask what the before and after are, in a number or a behavior.
Vague attribution: a crowd of unnamed experts agrees. Ask which experts. With no source, nobody can check it.
A contrast frame that rebuts a claim nobody made, the "it is not merely X, it is Y" shape. Ask who said it was merely X. Usually nobody did, and the sentence is adding drama instead of a mechanism.
A generic conclusion that could end any paragraph about anything, usually an upbeat line about the future. Ask what forecast it commits to.

When you edit, highlight every instance. If you cannot attach something verifiable on the next pass, cut the sentence or rewrite it into a claim a reviewer could prove wrong.

How do you run a quick pass yourself?

Two passes on a technical post, in this order:

Pattern pass: search for inflation openers, unnamed authorities, contrast frames, and closing lines that praise the future without a number or a boundary condition.
Mechanism pass: for every claim left, write down what you would measure and what result would prove you wrong. If the answer is nothing, the sentence is marketing.

On a long post I would rather let a script find the pattern families first. This sketch is mine, and deliberately small; a maintained detector is the better tool for real use.

# Sketch: flag a few AI-writing tells in a markdown draft
import re
from pathlib import Path

text = Path("draft.md").read_text(encoding="utf-8")
rules = [
    (r"\b(experts agree|studies show|researchers say)\b", "vague attribution"),
    (r"\bnot (just|only|merely)\b[^.]*\bit'?s\b", "contrast frame"),
    (r"\bthe future (looks|is)\b", "generic conclusion"),
    (r"\b(watershed|sea change|in the evolution of)\b", "significance inflation"),
]
for pattern, label in rules:
    for m in re.finditer(pattern, text, re.I):
        line = text.count("\n", 0, m.start()) + 1
        print(f"{label} ~ line {line}: {m.group()[:60]}")

Treat the hits like linter warnings: review each one, because some will be false positives, and fix the real ones.

Why can't a clean result mean the post is true?

A detector only sees shape. Removing an unnamed expert does not add evidence. Replacing a claim of historic weight with a concrete before and after does, but only if the before and after are real.

That is why the writing page sits next to three episodes on the gap between plausible and true: Dan Klein on plausibility engines (episode 54), Sudhir Hasbe on hallucinations as a data architecture problem (episode 55), and Alex Ratner on the evaluation gap, where everyone says their system works and almost nobody can say what "works" means in a number (episode 57). Lint gets you to honest prose. Those conversations are about what honest prose is still allowed to claim.

If you want to see the patterns in a real draft, the essays include a piece where an AI agent's first draft runs with formatting fixes only, so you can read it for tells yourself. More about the show and how it is made is on the about page.

Checklist

Highlight all four pattern families before you fact-check anything.
Replace every unnamed authority with a named source or a metric, or cut the line.
Run a quick regex or detector pass, then review every hit by hand.
Keep "reads well" and "is true" as separate checks, and schedule the second one for every claim that matters.
If the post describes AI product behavior, define what "works" means in a number before you publish.

More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Ship AI features inside a hard latency budget

Conor Bronsdon — Wed, 28 Oct 2026 15:30:08 +0000

Superhuman's full drafting call will not finish inside a 100 millisecond interaction budget. That budget is the immediate UI response, and completed generation is separate. Prefetch context that does not depend on this turn, such as how this person writes to this recipient and what the thread already says, before they speak. Send only the new transcription when they stop. Prove the work on the most capable model you can call. Once people keep using it and the job is narrow, move that task to a smaller or fine-tuned model. Leave send on a human click.

Loïc Houssier, CTO of Superhuman Mail, described that constraint on Chain of Thought in May 2026. He had joined Superhuman a year and a half earlier. At 100 milliseconds the product cannot depend on the network for every step, so it works offline by default, with logic on the desktop and the phone and async jobs that sync when the device is online. Email fits that pattern because it is asynchronous. He was plain that inference under 100 milliseconds does not work, so the team changed the shape of the call.

How do you keep an AI feature inside a 100ms feel?

You pull prepared context out of the waiting path and send a short payload when the user finishes.

Superhuman had just released voice on mobile. The team was not satisfied with the speed. If transcription drags and a model then rewrites for voice and tone, people will not use it, because it feels unlike the rest of the product.

Speech to text was the easier part. On-device transcription is one way to do it. The hard part is a draft that frames intent after people correct themselves, and that matches the thread and the tone used with that person. Answering your mom and answering your boss are different.

Much of that context can sit ready before the dictation. They cached so the live step was a transcription payload, not one call with a huge context. That caching has a cost: open voice and then type anyway, and the context was sent for nothing.

A minimal sketch to adapt, not a drop-in library.

CONTEXT_CACHE = {}

# proving: capable model. specialized: one task, after the feature sticks.
FEATURE_ROUTE = {
    "voice_draft": "proving",
    "auto_label": "specialized",
}

def model_for(feature):
    stage = FEATURE_ROUTE[feature]
    return frontier_model() if stage == "proving" else specialized_model(feature)

def warm_context(user_id, thread_id, feature):
    context = {
        "tone": load_voice_and_tone(user_id, thread_id),
        "thread": load_thread_summary(thread_id),
    }
    # Preload on the model. Keep only a session reference on the client.
    session_id = call_model(model_for(feature), context=context, cache_only=True)
    CONTEXT_CACHE[(user_id, thread_id, feature)] = session_id
    return session_id

def run_feature(feature, user_id, thread_id, payload):
    session_id = CONTEXT_CACHE[(user_id, thread_id, feature)]
    # Live call is the transcription payload plus that session, not the full context.
    return call_model(model_for(feature), session_id=session_id, payload=payload)

So instead of having one call with a huge context, we pre- I would say load the context as much as we can and boom, then we send just like a payload of the transcription.

Loïc Houssier, CTO at Superhuman Mail, on Chain of Thought ep 59

How Superhuman Built AI Into a… - Chain of Thought | AI Agents, Infrastructure & Engineering - Apple Podcasts

Podcast Episode · Chain of Thought | AI Agents, Infrastructure & Engineering · May 22 · 50m

podcasts.apple.com

When should a feature leave the frontier model?

When people still want the feature, and when you can name the narrow job a smaller model will own.

They start with one expensive model, the best they have, to see if the feature can work and how it feels. Then each use case gets its own model. The agentic framework runs on the latest version of Opus. The sequence is to try the expensive model, make it useful, then make it efficient.

Auto-label shows the handoff. The job is classification: FYI, pitch, partner, newsletter, and the rest. Auto-label started on an expensive frontier model, run on every email, which was expensive. Once they knew which 10 or 12 labels would be useful, he said, you can fine-tune dedicated BERT models and maybe run inference on dedicated infrastructure built for cheap inference at scale. With the labels known, he described it as a typical BERT classifier job: an open model, fine-tuned a bit, on cheaper dedicated infrastructure rather than a general LLM provider, and good enough for those use cases.

Why did repeating the prompt at both ends stop helping?

That pattern was a fix for an earlier model generation, and it helps less on newer ones.

On those earlier models, repeating the request at the start and at the end worked well. The assumption was that those models weighted the beginning and the end of the message more than the middle, so copying the request into both ends lifted the middle, and they saw better results. On newer models, repeating the message does not help as much. Where you place the instruction still matters. Sometimes it goes at the end.

Repeating the message was surprisingly good for 4 models. It doesn't work as much on newer models.

Loïc Houssier, CTO at Superhuman Mail, on Chain of Thought ep 59

Why auto-draft the email and leave send to the user?

Send carries more risk than the draft, so the draft waits for the person.

They do not send email in the user's name, because it is too risky. The product auto-drafts, and the person hits send. Users were not ready to let that go.

If the agent can send mail or move money, the safeguards around it are probabilistic, and at some point it will mess up. His rule: if you want a real gate, you don't expose that action to the agent. He already had scars from giving an agent too much room in his own notes.

We do not send, we auto-draft. It's here for you to just hit send, but we won't send any mail for you, for now.

Loïc Houssier, CTO at Superhuman Mail, on Chain of Thought ep 59

FAQ

Can you make the model itself return in under 100 milliseconds?

Loïc said that does not work. Prefetch voice, tone, and thread context so the live request is mostly the new transcription.

When is a fine-tuned classifier enough?

After a capable model has shown the task is useful and the label set is known. Loïc described auto-label that way: an expensive model on every email first, then, once they knew which 10 or 12 labels would be useful, a fine-tuned open BERT-style classifier on cheaper dedicated infrastructure. The agentic framework stayed on the latest Opus.

Should the agent send once drafts look reliable?

Draft quality and permission to send are separate. This team auto-drafts and does not send on the user's behalf for now. Any safeguard around an agent that can send is probabilistic. Loïc's real gate is not exposing send to the agent at all, so leave send with the person.

Takeaway

Write the interaction budget down (here, 100 milliseconds) before you design the model call.
Prefetch recipient tone and thread context before the user speaks. Send the transcription as the live payload.
Start on the most capable model. Move to a smaller or fine-tuned model only after the task and the labels are stable.
Re-measure prompt tricks after a model change. A pattern that helped on an earlier model may not help on the next one.
Auto-draft. Keep send, and any other hard-to-undo action, on a human confirmation.

The full conversation, with the transcript, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

When a support assistant should hand off to a person

Conor Bronsdon — Tue, 27 Oct 2026 20:30:07 +0000

In December 2024, Vinnie Giarrusso, principal software engineer at Twilio, described how to add AI to customer communication without making it colder. The worry he named is that assistants take work away and the experience gets less personal. He wants the assistant on the boring technical work and the mundane tasks, so a person can stay with the human part of the conversation.

I think that there's a lot of worry about agents taking work away and resulting in like a less personal experience. But actually, I think it's, I think it's the opposite, I think that there's a huge opportunity here to use these assistants in taking away the boring technical work and the sort of like mundane tasks,that would ordinarily be done by a human.

Vinnie Giarrusso, principal software engineer at Twilio, on Chain of Thought episode 7

Twilio calls them AI assistants on purpose. The industry word is agents. Early on, Vinnie said, the system sits side by side with a person, adding what he called superhuman powers, and depending on the setup a person may accept or reject certain actions. I would keep that accept or reject step on anything the customer cannot undo.

What should the assistant take, and when should a person step in?

I would let the assistant finish checks that are factual once the needed fact is known: order status, the customer's plan, and a single price. Vinnie described the support call where the person is reading a script and still misses the question. If those checks are already done, the person who joins can focus on the human problem, with time to be empathetic. He hopes the customer is not left waiting an hour.

He also drew a line through what an assistant may do alone. He said he had not tested this pattern and was spitballing:

Like there are things that an assistant can be trusted with to do. just kind of fully on their own. And then there are things that at least today, assistants should not be trusted to do fully on their own. especially in the early phase.

Vinnie Giarrusso, principal software engineer at Twilio, on Chain of Thought episode 7

He named the mechanism a stop mode or a request help mode. The assistant works inside a scope, then asks for supervisor approval or checks in on the steps it took. He left the customer-facing wording open. I would send one message that a person has the next step, and I would give that person a report of what was asked, what was answered, and which source was used. Complaints and account changes sit on the human side of the line. The assistant may prepare them. It may not commit them.

Enterprises, he said, will be more measured than the demo cycle. Some teams will treat a new capability as about a year out. If you are building a company on this, read the episodes gathered for AI founders before you promise full autonomy. They focus on defensibility, go to market, and market reality.

How do you stop a confident wrong answer?

Accuracy in this setting is specific. Vinnie's example is the price of a text message in Spain versus another country, when the user never said where they are. The assistant can hold every price row and still not know which one applies. He said accuracy is the assistant asking a follow-up, across one answer or a series of messages, instead of picking a row at random.

The follow-up itself is generative AI: the model produces new text. Sending it on a channel, or changing an account, is a further action. I would not let the function that drafts a reply also commit the account change.

Retrieval is one of the biggest failure modes Vinnie has seen once a system scales. A pipeline from the LangChain docs comes up quickly. After 10, 15, or 20 knowledge sources, the assistant may choose the wrong source, or return top chunks that cannot support the answer. He said context-aware retrieval, informed by Anthropic's contextual research, raised their retrieval accuracy, and he declined to give hard numbers.

On a bad trace, check two things. Which knowledge source did it pick? If that source was right, did the top chunks contain the answer? If you cannot tell, do not add another source.

What should you measure before you widen autonomy?

Vinnie said Twilio put an observability layer in place at day minus one. The team constantly checks context adherence, complexity, accuracy, and tone. Investigating low context or chunk adherence could reveal a mistaken assumption about the case, or an unexpected outcome in a small share of cases. I would narrow what the assistant may do alone until that signal moves.

A platform cannot know every future use. Twilio runs evals on cases that generalize across some customers, but cannot predict every customer use or knowledge source. They open-sourced an evals package in the Twilio Alpha GitHub and NPM namespace. It began as security evals and grew into evals more broadly, on the view that it is more convincing when people try their own questions and find gaps in sources and prompting. I would ship a small shared set plus a way for the domain owner to add cases.

Vinnie said few tools worked equally well for engineers, PMs, and business partners, and that shared view mattered. For a question like which handoffs had weak context adherence, use conversational analytics: someone asks in plain language and gets an answer from the data, without knowing the report or writing a query.

Sketch of the route only. This is not a Twilio system, and it calls no vendor SDK.

type Incoming = {
  intent: "status" | "pricing" | "complaint" | "account_change" | "unknown";
  missingFacts: string[];
  tone: "calm" | "strained" | "angry";
  action: "reply" | "lookup" | "change_account";
};

function route(turn: Incoming): "auto" | "clarify" | "human" {
  if (turn.action === "change_account" || turn.intent === "complaint") return "human";
  if (turn.tone === "angry") return "human";
  if (turn.missingFacts.length > 0) return "clarify";
  if (turn.intent === "status" || turn.intent === "pricing") return "auto";
  return "human";
}

Checklist

Write two lists before you ship: tasks the assistant may finish alone, and actions that always wait for a person.
If a required fact is missing, force a clarifying question and block a random choice among price rows or sources.
On every handoff, pass a short report of what was asked, what was answered, and which knowledge source was used.
For each failure, check the chosen source and the top chunks, then watch context adherence, complexity, accuracy, and tone.
Let domain owners add their own eval cases before you expand what the assistant is allowed to do alone.

The full conversation, with the transcript, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.

Count tokens before you ship so agent bills do not surprise you

Conor Bronsdon — Tue, 27 Oct 2026 15:30:14 +0000

Your API dashboard shows a spike, but the feature "felt small." Look at the tokens: what the model reads, writes, and re-sends on every step. Tokenization is not academic. Context limits, per-token pricing, and latency all use the same unit, and dense code or odd formatting can cost more than the word count suggests.

Why does billing use tokens instead of words?

Language models do not read whole words or raw characters. They read tokens, sub-word chunks from a fixed vocabulary. In English, a common word is often one token; rare words and code can split into several, and punctuation and spaces count too. OpenAI's rule of thumb is about four characters per token, or about three-quarters of a word, so 100 tokens is roughly 75 words.

That ratio drifts. Non-English text, code, and weird formatting usually burn more tokens per visible line. Before you argue about model tier, paste representative prompts into a tokenizer and note input tokens per request. Guessing from word count is how teams under-budget chat history and tool output.

What counts toward the context window each turn?

The context window is a token budget for one response, and the latest user message is only one part of it. System instructions, prior turns, tool definitions, and tool results all sit in that budget. Output tokens need room too. In chat, unless your app trims or summarizes, the full history goes back on the next turn, so input tokens climb every step.

Exceeding the limit is not one universal behavior. Depending on the API, the request may be rejected or generation may stop, and many applications truncate or compact the input before sending it. Compaction changes what the model sees next; it does not make the window bigger. Anthropic's documentation also warns that accuracy and recall can degrade as context grows, what it calls context rot. For cost planning, whether a prompt fits in the window and whether it is cheap to keep sending are different questions.

Where does the money go on each inference call?

For most teams on hosted models, inference is the bill that repeats: every message and agent step is another run. Training is someone else's capex; your product pays per call.

Providers charge input and output separately. On Anthropic's price list as of October 2026, output costs five times as much as input for each Claude model. Reasoning models can bill hidden reasoning as output, so a short visible answer can still be expensive. Repeated static instructions may qualify for prompt caching: Anthropic charges about a tenth of normal input to read a cached block, with a higher cost the first time you write it.

One successful task is rarely one call. Agents plan, call tools, retry, and often carry a growing context each time. Cost per task is roughly the sum over every call of (input tokens × input price) plus (output tokens × output price), including failures and retries.

Jeetu Patel, Cisco's president and chief product officer, argued in episode 67 that price per token could turn into a vanity metric: the price per token falls while long-running agents use more tokens. What matters to him is the price of the outcome. Your estimator should still start with tokens, because that is what you control in design.

How do I sketch a monthly token bill?

Use your provider's current input and output rates. If you only remember the Anthropic ratio above, set OUTPUT_PRICE = 5 * INPUT_PRICE as a placeholder until you paste real numbers from the price list.

Label this a sketch, not production accounting:

# Sketch: monthly inference cost from volume.
# Prices are $ per 1M tokens; replace the placeholders with your provider's list.
def monthly_cost(tasks_per_month, calls_per_task, avg_input_tokens,
                 avg_output_tokens, input_price, output_price):
    calls = tasks_per_month * calls_per_task
    per_call = (avg_input_tokens * input_price
                + avg_output_tokens * output_price) / 1_000_000
    return calls * per_call

INPUT_PRICE = 1.0                 # placeholder
OUTPUT_PRICE = 5 * INPUT_PRICE    # the ratio on Anthropic's list
# 50k tasks, 8 model calls each, 12k tokens in and 800 out per call
bill = monthly_cost(50_000, 8, 12_000, 800, INPUT_PRICE, OUTPUT_PRICE)

Run three scenarios on the same workflow: today's averages, a heavy month (longer history, more retries), and a trimmed prompt (fewer input tokens or fewer calls per task). The line that moves the total most is where you spend engineering time.

Set calls_per_task realistically for agents. A single user action might be dozens of inferences. Divide total cost by completed tasks, not by API calls, when you compare features.

I think that today a lot of organizations are not thinking about their token spend regularly and they're getting hit with their bills.

Jiaona Zhang, Laurel

Jiaona Zhang ties that to "token maxxing": mandating AI use without asking whether the task earned the tokens, like using a model to redo slide fonts when one click would do. The estimator is how you catch that before finance does.

Which knob should I turn first?

If output tokens dominate (long answers, reasoning traces, verbose tool JSON), tighten schemas, cap completion length, or route easy steps to a smaller model. If input tokens dominate (system prompt, full chat log, fat tool results), trim history, summarize older turns, cache stable instructions, and stop pasting whole documents when a chunk will do.

Jeetu Patel describes rationing tokens like headcount: a fixed pool and an expected output. That matches measuring cost per completed task and watching context on every turn, themes that show up across AI infrastructure conversations on the show.

Checklist

Tokenize real prompts and tool payloads; do not budget from word count alone.
Budget context as instructions plus history plus tools plus reserved output tokens.
Estimate with (input × input price + output × output price) × calls per task × tasks per month.
Compare scenarios: trim input, trim output, and reduce calls per successful task.
Report cost per completed task, retries included, not per isolated API call.

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.