DEV Community: Madhesh Vivekanandan

Five Labelled Examples Match Jev. Then I Tried Tanglish.

Madhesh Vivekanandan — Fri, 30 Oct 2026 14:35:50 +0000

"Online la oru item order panni romba naal aachu, aana innum varave illa. Enakku podhum! En money-a thirupi kudunga, indha aalungaluku naan pay panna maaten."

(I ordered something online ages ago and it still hasn't come. I've had enough! Give me my money back, I'm not paying these people.)

That's a Banking77 refund request, rewritten in Tanglish (Tamil and English in one message) by a Claude agent, the way a lot of us actually type to a bank. Cosine similarity filed it under supported_cards_and_currencies. A classifier trained on 35 English examples per intent got it wrong in 2 of 3 runs. TypeSafe's Jev, a model that picks one option from a list you give it, filed it under request_refund with confidence 0.98.

I expected Jev to win everywhere. It didn't. On clean English, five labelled examples per category let a free classifier match Jev. On Hinglish, Tanglish and trick questions, 35 English examples didn't close the gap, even with a multilingual embedder.

I couldn't find a write-up that tests Jev on how Indian users actually type, so this is one: the numbers, a router you can copy, and a rule for when five examples are enough. Every answer is in the repo.

The contest in 30 seconds

The job: route a bank customer's message to 1 of 77 intents from Banking77 (card_arrival, declined_card_payment, request_refund, ‚Ä¶). Five contenders:

Method	Compares the message with	Labelled examples
Cosine	each intent's one-line description, both as embeddings (vectors where similar meanings sit close)	0
Jev	the same 77 descriptions, as a list of options	0
Cross-encoder	each description, read together with the message	0
Cosine voting	past labelled messages; the 5 nearest vote	1 to 35 per intent
Classifier	logistic regression on the same past messages	1 to 35 per intent

Think of two clerks: a fast, free one (cosine, voting, the classifier) and a careful one, Jev, who reads every option. When is the fast one enough?

Jev returns one option and a probability for every option, not text, so there's nothing to parse. The whole call:

from typesafe_sdk import Choice, RetryPolicy, TypeSafeClient

client = TypeSafeClient(api_key="public", base_url="https://opencode.ai/zen",  # free route
                        retry=RetryPolicy(max_retries=0))  # retries count against its daily cap
question = Choice(
    instructions="Which banking support intent does the customer_message express? "
                 "Pick the single best match.",
    criteria=descriptions,  # {"card_arrival": "The customer ordered a card and it has not arrived yet, ...", ...} x 77
)
response = client.system_one({"customer_message": text}, {"intent": question}, model="jev-1.13-free")
answer = response.choices["intent"]
answer.choice, answer.confidence  # ("request_refund", 0.98)

Every call sends all 77 descriptions, about 2,400 input tokens.

Round 1 ¬∑ Clean English: five labelled examples match Jev

With zero examples, Jev beats cosine by 10 points across 1,001 test messages.

A logistic-regression classifier on the embeddings matches Jev at 5 examples per category: 385 labelled messages in total. Its 0.9-point edge is inside Jev's margin of error, so call it a tie. Cosine voting needs 10.

One example is worse than one sentence. A single labelled message per category scores 63%. A one-line description scores 73%.

The cross-encoder, which reads message and description together, did no better than cosine. Pasting the same 5 examples into Jev's descriptions reached 90.4% on 249 messages: a borderline lead over the classifier, at 3.7 times the tokens.

On clean English, the fast clerk catches up with five examples. Then I changed the language.

Round 2 ¬∑ Hinglish and Tanglish: the embeddings fall apart

Your users won't type like your test set. Hinglish mixes Hindi and English, Tanglish mixes Tamil and English, both in Latin letters. Claude agents rewrote 144 test messages (all 77 intents) as natural chat in each, keeping words like card, top up and PIN in English, the way people type them. Other agents checked every rewrite; no native speaker has yet.

	Message	Cosine	Classifier, 35 examples (3 random draws)	Jev
English	"I ordered something ages ago online but it's simply not arriving‚Ä¶ Give me back my money‚Ä¶"	‚úó `card_arrival`	‚úì	‚úì 0.99
Hinglish	"Maine kaafi time pehle online kuch order kiya tha‚Ä¶ Mere paise wapas do‚Ä¶"	‚úó `getting_virtual_card`	‚úó in all 3 draws	‚úì 0.99
Tanglish	"Online la oru item order panni romba naal aachu‚Ä¶ En money-a thirupi kudunga‚Ä¶"	‚úó `supported_cards_and_currencies`	‚úó in 2 of 3 draws	‚úì 0.98

One message, picked to show the pattern. All 144:

"Isn't that just your English-only embedding model?" Fair: bge-small-en is English-only. So I re-ran cosine voting and the classifier with bge-m3, a multilingual model about 17 times bigger (the dashed line, the better of the two). It helps, by 3 to 6 points. Jev stays clearly ahead on Hinglish; on Tanglish, 144 messages make the gap borderline. What I haven't tested is training on Hinglish or Tanglish examples. That's next.

Round 3 ¬∑ Trick questions: more examples don't help

Next, 105 new messages built to fool keyword matching. Three agents wrote them and three more labelled each one blind; all 105 passed, which may mean the check was lenient.

Message (and the right answer)	Cosine	Classifier, 35 examples	Jev
Negation: "It's not contactless that's the problem, tapping my card works fine. It's Google Pay that refuses to add my card." ‚Üí `apple_pay_or_google_pay`	‚úó `contactless_not_working`	‚úó same, 86‚Äì91% sure	‚úì
Decoy: "An ATM in Lisbon swallowed my card last week, but that's been dealt with and a replacement is ordered. How long does delivery usually take?" ‚Üí `card_delivery_estimate`	‚úó `card_swallowed`	‚úó same, all 3 draws	‚úì
Indirect: "The four numbers I tap in at shops are my old house number and I'd like something different. How do I swap them?" ‚Üí `change_pin`	‚úó `edit_personal_details`	‚úó same, all 3 draws	‚úì

Jev got all 105 right. These were written to fool surface matching, so read 100% as "Jev doesn't fall for this", not "Jev is perfect".

The classifier stayed at 64% with 5 examples per intent or 35, and reached 76.5% with the multilingual bge-m3. Cosine took the planted bait 23 times.

My favourite detail: the bug was in my description

Jev's worst category on English was get_physical_card: 0 out of 13. Cosine's too.

As a fairness rule, I wrote every description from the label name alone. Mine said: "The customer asks how to get their physical card." Then I read the messages:

"Where is my PIN number located?" ¬∑ "Is PIN delivered separately?" ¬∑ "I just got my new card but am not sure how to check its PIN."

In Banking77, get_physical_card really means "where's my PIN?" (105 of its 106 training messages mention one). Jev sent all 13 to change_pin, 9 of them at confidence 0.95 or higher. I haven't re-run it fixed: the cosine, cross-encoder and Jev numbers carry the bug, while voting and the classifier never read descriptions.

Your descriptions are your prompt. Read five training messages per category before you write one.

When to trust an answer, and when to ask Jev

Confidence works as a filter. On English, Jev's answers at 1.0 (half of them) are 98% right, and the classifier's own probability works just as well. But on Hinglish and Tanglish only about 1 in 10 classifier answers clear 0.9: it knows it's lost. So the fast clerk answers, and passes the odd letter to the careful one:

Let the classifier answer. When its confidence is below 0.7, ask Jev.

from sentence_transformers import SentenceTransformer
from sklearn.linear_model import LogisticRegression

model = SentenceTransformer("BAAI/bge-small-en-v1.5")
embed = lambda texts: model.encode(texts, normalize_embeddings=True)
clf = LogisticRegression(C=100, max_iter=5000).fit(embed(examples), labels)  # 35 per intent; C=1 underfits

def route(text: str) -> str:
    probs = clf.predict_proba(embed([text]))[0]
    if probs.max() >= 0.7:
        return clf.classes_[probs.argmax()]
    answer = client.system_one({"customer_message": text}, {"intent": question},
                               model="jev-1.13-free").choices["intent"]
    return answer.choice  # client and question: from the Jev call above

Measured on Round 2's 144 messages and Round 3's 105, with 35 examples per intent. The English column is the 144 originals, so Jev scores 86.1% there, not 83.3%:

	English (144)	Hinglish (144)	Tanglish (144)	Trick (105)
Jev alone	86.1%	85.4%	77.1%	100%
Classifier alone	93.1%	56.2%	52.8%	64.1%
Classifier first, Jev below 0.7	92.6%	84.5%	78.7%	91.7%
Messages sent to Jev	14%	73%	75%	54%

On English and code-mixed text it stays within about a point of the better single method, or above it on Tanglish. That saves 86% of Jev calls on English but only about a quarter on Hinglish and Tanglish, where the classifier is rarely sure. Trick questions are the exception: the classifier was 86‚Äì91% sure of its wrong Google Pay answer, so the 0.7 gate let it through.

I didn't tune 0.7; it's the middle of a fixed 0.9 / 0.7 / 0.5 grid. Your classifier's probabilities will differ, so sweep 0.5 to 0.9 on a few hundred held-out messages, as TRIAGE.md does. It also shows a simpler rule (trust answers when voting and the classifier agree) that wins on clean English and breaks on Hinglish.

So which one should you use?

flowchart TD
  Q1{"Labelled messages<br/>per intent?"} -- "fewer than 5" --> J["Jev<br/>(cosine on descriptions if it must be free)"]
  Q1 -- "5 or more" --> Q2{"Only clean English?"}
  Q2 -- yes --> E["Classifier on embeddings"]
  Q2 -- "no, or not sure" --> H["Classifier first;<br/>ask Jev below 0.7<br/>(measured with 35 per intent)"]
  J --> P["Jev's answer below 0.95?<br/>Send it to a person<br/>(Jev alone: 3 in 10 English, 6 in 10 Tanglish)"]
  H --> P

For scale: the classifier answers in 7 ms on a laptop CPU (43 ms with bge-m3), for $0. Jev takes 0.76 s a message, median round trip from India, and about $0.10 per 1,000 messages at TypeSafe's list price of $0.042 per million input tokens.

What's missing, honestly

The classifier only saw English examples; training on code-mixed examples is untested.
The probe sets are small: 105 to 144 messages, roughly ¬±6 to ¬±9 points.
Banking77 is public, so Jev may have seen it in training.
One cross-encoder, trained for search; larger rerankers may do much better.
Jev went through a free third-party route: checked against TypeSafe's published answers but not proven identical, capped near 250 calls a day, and down for about four days in late October. TypeSafe's own signups were paused when I checked. Latency is from one laptop in India.

Steal this, then run it

Needs Python 3.12 or newer; the install pulls in PyTorch.

git clone https://github.com/Madheshvivekanandan/jev-vs-cosine && cd jev-vs-cosine
python3.12 -m venv .venv && .venv/bin/pip install -e .
.venv/bin/python -m app.main triage && cat results/TRIAGE.md   # no key, no API calls

Every Jev answer is committed, so the reports rebuild for free. To call Jev yourself, .env.example points at OpenCode Zen's free route, a third party that verify-route checks against TypeSafe's published answers before you spend quota. Install only the official typesafe-sdk; lookalike packages exist.

If you take three things from this post:

Write descriptions from real messages, not label names. One wrong sentence zeroed a whole category.
Test on how your users actually type. English said five examples were enough. Tanglish disagreed.
Route on confidence, and pick the threshold on your own data. Let the cheap model answer what it's sure of; escalate the rest to Jev or to a person.

Run it on your own users' messages and tell me where Jev breaks. If you speak Hindi or Tamil and a rewrite reads wrong, I genuinely want the issue.

Previously: The Coding Agent That Can't Grade Its Own Homework, on why the agent that writes the code shouldn't be the one that grades it.

All illustrations in this post were generated with ChatGPT; the charts were drawn from the repo's result files.

Banking77 is by PolyAI (CC BY 4.0). Not affiliated with TypeSafe AI.

The Coding Agent That Can't Grade Its Own Homework

Madhesh Vivekanandan — Wed, 09 Sep 2026 15:05:27 +0000

"All tests pass. ✅"

You've seen that message from a coding agent. And at least once, you've opened the file afterwards and found the tests were never run — or the test file was quietly edited until it agreed.

The problem isn't that the model writes bad code. It's that the same context window writes the code, reviews the code, and declares the code correct. One brain, grading its own homework — and it grades generously. That cycle — understand → change → check → repeat — is the agent loop, and every coding tool ships one that runs inside a single context window.

agent-loop is a small open-source project that restructures it: six explicit stages, file-based handoffs, an independent verifier, hard iteration caps. No framework, no daemon, no SDK — markdown plus one shell script, and it runs unchanged in Claude Code, Codex, Cursor, Gemini CLI, Copilot, and 30+ other tools.

The whole loop in 30 seconds

Each stage starts with a fresh context, reads only its declared inputs, and hands off through a markdown file in a gitignored .agent-loop/ directory:

.agent-loop/
  profile.md                    # cached project profile (commands, conventions)
  runs/2026-09-09-rate-limit/
    task.md
    analysis.md
    plan.md
    implementation.md
    test-report.md              # ends in PASS or FAIL — never "looks good"

Those files aren't logs. They're the interface between stages — and an audit trail you can read afterwards to see exactly why the agent did what it did.

Not every task deserves a pipeline

Anthropic measured multi-agent systems at roughly 15× the tokens of a regular chat. For a typo fix, that's a five-course tasting menu because you wanted a glass of water.

So the first move is triage — one cheap classification, never delegated:

Tier	The task	What runs
S	Diff fits in one sentence	implement → verify
M	Localized change	analyze → implement → verify
L	Multi-file or uncertain	the full pipeline

One rule does more work than it looks like: torn between two tiers, pick the smaller one. Left alone, agents will write a design document for a null check — the bias has to be encoded.

Artifacts, not transcripts

Why fresh contexts? Because context poisoning is real. Exploring a codebase fills the window with dead ends — irrelevant files, abandoned hypotheses, stale test logs — and in one long session, all that noise quietly influences every later decision.

File handoff cuts the cord. The analyzer can burn its whole budget exploring, but only analysis.md survives into the next stage. What crosses the boundary is a document, not a transcript.

One rule decides whether handoffs work at all:

The plan must carry rationale, not just conclusions. An implementer that gets only "use middleware, put it in upload.py" re-derives the reasoning, reaches a different answer, and contradicts the plan while believing it follows it. (Cognition's "Don't Build Multi-Agents" circles the same failure.)

The templates enforce it structurally: plan.md has required sections, and a plan without a named runnable check — the command that will decide pass or fail — is returned once, then the loop stops. No check, no run.

The verifier never sees the reasoning

This is the part of the design I'd defend in a fight. Verification starts in a completely fresh context, and its inputs are a closed list:

The verifier gets	It never gets
The plan	The implementer's summary
The diff	The implementer's reasoning
A shell, to run the check	`implementation.md`

Why so strict? An agent that reads "here's why my implementation is correct…" doesn't verify that claim — it confirms it. Sycophancy isn't a personality flaw you can prompt away; the fix is structural. The verifier learns what the code was supposed to do, sees what actually changed, runs the check named in the plan, and writes PASS or FAIL with command output as evidence. "Looks good to me" is not a verdict.

And my favorite detail in the whole project: you might think spawn the verifier without write tools and it can't cheat. But it must have a shell to run the tests — and a shell can edit files (sed, echo >, git apply…). So the real guarantee is a diff fingerprint: the working tree is fingerprinted before and after verification, and if it moved, the verdict is void. Tool restriction is defence in depth; detection is the guarantee — on every host, including those that can't restrict tools at all.

(Reviewers fail in the other direction too, so the contract counts only correctness-affecting gaps — no invented nitpicks.)

Three strikes, then a human

On FAIL, a debugger spins up — fresh context again, so failed attempts don't pile up as noise that poisons the next one. It must reproduce the failure before fixing it, and if it concludes the plan is wrong, that's an escalation, not a code change.

The debug loop caps at 3 iterations — with oscillation detection, and a full re-verification after every fix. On cap, you get a distilled summary of what was tried and what the evidence says, not a 400-line transcript.

An agent loop without a termination condition isn't autonomous. It's just unsupervised.

One folder, thirty-plus tools

The loop is a single Agent Skills folder — an open standard 30+ coding agents read — and the stage contracts name no vendor, tool, or product. Hosts differ wildly in what they offer, so every capability declares an explicit fallback:

Capability	With it	Without it
Subagents	Mode A — one isolated context per stage	Mode B — sequential stages, still reading only declared inputs
Tool restriction	Verifier spawned without write tools	The diff fingerprint alone — the real guarantee anyway
Turn caps	Debugger bounded mechanically	Iteration counting; the 3-cap holds either way
Per-stage models	Strong models on plan/verify/debug	One model throughout — costs more, quality holds

Every run announces which mode it got. One requirement is non-negotiable: shell execution. Verification that can't run real commands gives the loop no termination condition — so it refuses to run rather than emit confident, unverified output. That refusal is the product.

And because everything is markdown plus one POSIX installer — no daemon, no database — there's nothing racing the features agent hosts ship natively every month, and the skill costs zero context until you explicitly invoke it.

So what does this actually improve?

Each failure you've met on real work maps to a specific structural counter — that mapping is the design:

Failure you've seen	What the loop does about it
Bad early output contaminates everything after it	Fresh context per stage; artifacts cross the boundary, transcripts don't
"All tests pass" that nobody ran	Independent verifier, diff fingerprint, evidence required in the verdict
The agent grinds on a fix forever	Cap of 3, oscillation detection, escalation with a failure summary
It rebuilds a helper that already exists	The analyzer's mandatory Found / Exemplars / Missing / Reuse-plan gate
A design doc for a one-line fix	Triage tiers, biased toward the smaller tier
The implementer "follows the plan" into a different design	Plans carry decisions and rationale, enforced by template sections

None of this makes the model smarter. It makes the process resistant to the specific ways coding agents fail.

Steal this, then run it

git clone https://github.com/Madheshvivekanandan/agent-loop && cd agent-loop
./install.sh claude-code     # or: codex · cursor · agents
# Claude Code, zero-install alternative:  claude --plugin-dir path/to/agent-loop

Then, in any project:

/agent-loop add rate limiting to the upload endpoint

First run profiles your project — it discovers your test/build/lint commands and executes them once before trusting them. Every run after that states its tier and mode up front, works the stages, and returns a verdict, a diff summary, and a folder of evidence you can actually read.

If you take nothing else from this post, take the two rules that travel to any agent workflow:

Never let the agent that wrote the code declare it correct. The separation must be structural — fresh context, plan + diff only — not a prompt asking it to "be critical."
Every loop needs a number where it stops and calls a human. Caps aren't a lack of faith in the model; they're what makes it safe to stop watching.

The repo is github.com/Madheshvivekanandan/agent-loop — MIT-licensed, one folder, readable in an evening. If you run it on a host I haven't tested, or find a hole in the verifier's guarantees, I genuinely want the issue.

Previously: The Dashboard That Builds Itself, on generative UI and letting a model order from a menu without getting into the kitchen.

All images in this post were generated with ChatGPT.

The dashboard that builds itself

Madhesh Vivekanandan — Mon, 07 Sep 2026 04:41:51 +0000

Every dashboard you've ever used was designed long before you opened it. Someone guessed which charts you would need, arranged them on a grid, and shipped that guess to everyone. This post is about a dashboard that skips the guess: it doesn't exist until you ask it a question. You type "how did Q3 go?" and the answer comes back as a screen, not a paragraph — stat cards and a chart, building on screen while the AI is still thinking.

I built it with three plain parts: OpenAI's structured outputs (a mode where the reply is guaranteed to match a shape you define), a small Python backend (FastAPI), and Google's open A2UI protocol (a standard way for an AI to describe a UI as plain data), rendered by @a2ui/react, the A2UI project's official React renderer. This post explains how the pieces fit and the five safety nets that make "an AI draws my UI" boring instead of scary. The whole app — backend, frontend, tests and config — is about 4,300 lines, and all of it is on GitHub.

Two questions and a button press, recorded from the running app — every answer re-forms the same canvas.

Three ways to do generative UI (and where A2UI sits)

Nielsen Norman Group defines generative UI as "a user interface that is dynamically generated in real time by artificial intelligence to provide an experience customized to fit the user's needs and context." In this app, that context is the question you just asked. There are three ways to build it, and they differ in one thing: how much you let the model do.

Let the AI write real code — HTML, CSS, JavaScript. Google ships this in Gemini and the results are impressive. But in your own product it means running AI-written code in your users' browsers — a malicious input can trick the model into writing a script that then runs on someone else's screen: the classic cross-site scripting (XSS) attack, delivered by the AI itself.
Let the AI pick from a menu. You build the components; the model only sends data saying which ones, with what values. The model orders from a menu — it never gets into the kitchen.
The same menu idea, with a shared protocol — so any client that speaks the protocol can draw the output of any agent that speaks it. That's A2UI, announced by Google in December 2025. An agent describes an interface as a stream of small JSON messages, and a renderer draws them using only components from a catalog the client defines. UI travels as data, never as code.

The official docs cover the protocol well, and I won't repeat them. Most A2UI tutorials run on Google's own tools — their Agent Development Kit (ADK) and Gemini models — and the exceptions go through large agent frameworks (Microsoft's Agent Framework pairs A2UI with FastAPI and an OpenAI client; A2UI's own docs show LangGraph and Strands setups). What I couldn't find is a write-up that wires A2UI directly to the plain OpenAI API and FastAPI, with no agent framework in between. That's this post.

The whole system in 30 seconds

flowchart LR
    U(["You"]) -->|"how did Q3 go?"| B["Browser · @a2ui/react"]
    B -->|"your question"| F["FastAPI"]
    F -->|"schema + data"| L["OpenAI"]
    L -->|"typed blocks, streaming"| F
    F -->|"A2UI messages, card by card"| B
    B -->|"button press"| F

The browser sends your question to the backend.
The backend asks OpenAI (gpt-4o-mini in this build) for a plan, not code: one to four "blocks" chosen from six types (stat card, line chart, bar chart, donut, table, text note). The plan is forced through a schema, so it always has the right shape. The numbers come from a fixed demo dataset that the backend includes in the prompt, so the model copies and combines real figures rather than inventing them.
A small compiler turns each finished block into A2UI messages.
Those messages stream to the browser using server-sent events, or SSE (one long-lived reply that the server keeps adding to), and @a2ui/react draws them. Cards appear one by one, while the model is still writing.
Cards carry buttons the model invented. Pressing one sends the request back, and the loop runs again.

And one deliberate rule ties it together: every answer is drawn into the same single canvas. Before a new answer streams in, the old one is cleared — so the dashboard is regenerated in place, never stacked. Ask a different question, get a different dashboard, same spot. The ✕ button brings back the starting view.

The menu, not the kitchen

The most important design decision sounds like a technicality: the model never speaks A2UI.

The tutorials I found — including the official A2UI agent guide — do the opposite: paste the protocol schema into the prompt, ask the model to write A2UI JSON directly, then check it afterwards. The format A2UI actually sends over the network is a flat list of components that refer to each other by ID. Great shape for a renderer; terrible shape for a model to write reliably — nothing stops it from referencing an ID that doesn't exist or forgetting a field.

So there are two languages, on purpose:

	Who reads it	What it is
Typed blocks	the model	six block types with required, typed fields
A2UI messages	the browser	the real protocol, produced by a compiler

The typed blocks are the schema I hand to OpenAI. Give the API a shape and it cannot produce JSON in any other shape, because the rule is applied while the text is being written rather than checked afterwards. The model cannot invent a seventh component type or skip a required field. Not "usually doesn't" — cannot. (One caveat: a guaranteed shape can still hold wrong content — the schema stops invalid structure, not bad answers. And this app keeps its count limits, like "at most four blocks," in one line of ordinary code rather than in the schema.)

The cost is real: the agent can only compose layouts my compiler knows how to build. For a dashboard, that's the right trade. For a general-purpose agent canvas, it wouldn't be.

It paid off in a way you can see in the code. Version 1 drew the UI with a hand-rolled renderer — three hundred lines that looked up each card type in a table and drew it by hand. Migrating to the protocol deleted that file, and the frontend no longer contains a single line that asks what kind of card something is; the only place a card type is named is one small lookup table in the backend compiler.

Watching an answer build itself

The reply streams in as it's written, which raises a fun question: when is a card inside a half-written answer safe to show?

The rule in this codebase: a card can only be trusted once the model has moved on to the next one. So the server sends each finished card immediately, and always holds back the last one it can see:

# A block is only provably finished once the next one has started,
# so the last one in the snapshot is always left to the caller.
for index in range(emitted, len(raw_blocks) - 1):

Each card shows the moment it's provably finished. Then, when the whole answer is done, the server sends the complete, fully-checked version once more, and it quietly replaces what is on screen. That works because an update overwrites whatever already sits at the same address — same data path, same component IDs — instead of adding a copy next to it. Any rushed mistake is corrected the moment the answer finishes.

sequenceDiagram
    participant B as Browser
    participant F as FastAPI
    participant O as OpenAI
    B->>F: your question
    F-->>B: open the canvas
    F->>O: ask for typed blocks (schema attached)
    O-->>F: …card 1 finished
    F-->>B: card 1 data + layout
    Note over B: first card renders
    O-->>F: …card 2 finished
    F-->>B: card 2 data + layout
    O-->>F: model finishes
    F-->>B: final complete answer (verified)
    F-->>B: chat explanation + suggestions · done

Here is what one message looks like on its way to the browser. This is all the "protocol" really is:

{ "version": "v0.9",
  "updateDataModel": {
    "surfaceId": "dashboard",   // the one canvas
    "path": "/blocks/0",
    "value": { "title": "Q3 Revenue", "value": 128400,
               "unit": "USD", "delta_pct": 12.4 } } }

Cards don't hold values — they hold pointers into this data tree, so updating the data is all it takes to update the screen. (The full capture is here — a real turn, straight off the wire.)

Buttons that talk back

Without buttons, every answer is a dead end: you read the chart, and then you are on your own to work out what to ask next. The model already knows what is worth asking. So every card carries up to two buttons the model wrote itself, and pressing one talks back to the agent.

All buttons share one action name, refine. The interesting part travels inside the button: as it writes the card, the model puts a complete follow-up request inside the button — label "Split by region", request "Break Q3 revenue down by region as a bar chart." The backend just pulls that request out and runs the same loop again. One handler covers every button the model will ever dream up — and pressing one feels like the dashboard changing its mind.

After a button press — the same canvas, re-formed. The chat keeps the conversation; the dashboard keeps only the current answer.

"Why would you trust an AI to draw your screen?"

When A2UI hit Hacker News, one blunt reaction captured a recurring worry in the thread: "Why on earth would you trust an LLM to output a UI?" It deserves an architectural answer. Mine is five nets, each catching what the previous one can't:

Structured outputs — the schema is enforced during generation. Wrong shapes can't exist.
A sanitizer — a valid shape can still be nonsense. This layer trims oversized data (at most four chart series, 25 table rows, and so on) and rejects anything that cannot be drawn at all — a donut with one slice is not a chart, it is a circle. A bad donut next to a good chart costs you the donut, not the whole answer.
The catalog whitelist — the renderer refuses any component name not in the catalog, enforced by the library itself.
Forgiving readers — every field a view renders passes through a tiny reader that turns bad input into a safe fallback instead of a crash: a missing number becomes a dash, a chart with no usable data becomes an empty state. Agent-written data is the expected input.
An error boundary per card — if a chart does break while being drawn, it takes down its own card and nothing else.

On top of all that sits one promise: the generate endpoint never returns an error. A missing key, a refusal from the model, a network failure — each one comes back as an ordinary text card, drawn through the same path as real content. That leaves the renderer with a single path to follow and no error branch at all.

That promise has a story behind it. A code review on day one caught the fallback card printing raw exceptions — and an OpenAI auth error contains part of your API key. The app was painting key material into a dashboard card. The fix: stable error codes on the wire, generic copy on screen, details only in the server log. If you draw errors from anywhere near the model onto the screen, you will eventually draw a secret onto it too. Decide the boundary on day one.

A real generated answer — every element on screen was ordered from the menu, including the buttons the model wrote itself.

Steal this, then run it

What I'd tell you before you build one:

Don't make the model speak the protocol. Let it fill in a strict, friendly schema; compile to the protocol in code.
Show partial results fast, then re-send the complete checked answer so it replaces anything the quick version got wrong. That final replace is what makes streaming safe to attempt.
Layer your defenses and expect each one to fire. Schema, sanitizer, whitelist, forgiving readers, boundary.
Treat the failure card as normal output. One render path for success and failure means the error path is tested on every fallback.
Test that streaming streams. The worst thing a broken streaming system can do is keep working. Assert the first update arrives before the stream closes.
Build the starting view the same way. The dashboard you see on load is compiled by the same code the agent uses, so bugs show up before you type anything.

What's missing, honestly: no auth, no database, mock data, and A2UI v1.0 will eventually mean some rework.

The whole thing runs with docker compose up and an OpenAI key — without a key it still boots and shows the baseline. The repo is here — clone it, type something odd into it, and tell me what breaks.