DEV Community: Ishank Choudhary

NVIDIA Nemotron 3.5 Bolt E4B: Running an Offline Voice Agent With Tool Calling on a CPU

Ishank Choudhary — Wed, 28 Oct 2026 04:46:48 +0000

NVIDIA released Nemotron 3.5 Bolt E4B on October 23. The pitch is a small model tuned for agents: it calls tools, follows instructions, and is small enough for laptops, Jetsons and other edge hardware. The model card's benchmarks are tested on H100s, RTX 5090s and DGX Spark. I wanted to see what happens without any of those.

So I took the official 4-bit GGUF, ran it on a plain 8-vCPU Xeon VM with 16 GB of RAM and no GPU, and built a small voice assistant around it. It listens to a WAV file, picks a tool, calls it, and speaks the answer back. Everything below is from my own runs. I've kept the raw CSVs for every number.

What Bolt E4B actually is

A few facts from the model card and the files themselves, because the name hides some of them:

Architecture: Gemma 4 E4B, post-trained by NVIDIA with reinforcement learning (GRPO via NeMo RL) for tool use, instruction following and multi-turn chat. About 8B total parameters and about 4B "effective" ones.
License: OpenMDW-1.1 on NVIDIA's release. The base Gemma 4 weights are Apache 2.0.
Context: 64K tokens. Text in, text out. There's no audio in the model, so a voice agent still needs separate speech-to-text and text-to-speech.
Weights: the official repo is nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB, with a single Q4_K_M GGUF. NVIDIA also ships a BF16 checkpoint for vLLM and a DFlash speculative-decoding drafter aimed at GPUs.
Recommended serving: llama.cpp with --jinja, sampling at temperature 1.0 and top_p 0.95, and a chat-template switch for "thinking" on or off.

The first surprise: the "4GB" file is 6.18 GB on disk. I opened the GGUF and summed the tensors:

Tensor group	Size
Per-layer embedding tables (mostly Q8_0)	3.05 GB
Transformer blocks (Q4_K + Q6_K)	2.40 GB
Token embeddings	0.71 GB

The per-layer embeddings are lookup tables: each token only reads a few rows. That's what makes the "4GB" label fair at run time, as you'll see in the memory section. You still need about 6.2 GB of disk space and download time, though.

The setup

Hardware: 8 vCPU Intel Xeon (AVX-512, VNNI), 16 GB RAM, no swap, no GPU.
Runtime: llama.cpp CPU build b11913, using llama-server as an OpenAI-compatible endpoint.
Speech-to-text: faster-whisper, base.en and small.en, int8 on CPU.
Text-to-speech: Piper with the en_US-lessac-medium voice.

The server command was nothing special:

llama-server -m Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf --alias nemotron-bolt \
  --host 127.0.0.1 --port 8000 -c 8192 -np 1 -t 8 --jinja --metrics

The model loaded in 2.5 to 6 seconds. My first sanity check was 17 Ã— 23, and it answered 391.

Raw speed on 8 CPU cores

llama-bench, 8 threads, 3 repetitions:

Test	Tokens/s
Prompt processing (512 tokens)	194.6 Â± 32.8
Generation (128 tokens)	10.9 Â± 2.8

About 11 tokens per second of generation is slow next to a GPU, but it's enough for an assistant whose replies are one or two sentences. Prefill speed matters more than I expected. My system prompt plus five tool schemas came to about 500 tokens, and the very first request took 6.4 seconds before anything came back. After that, llama.cpp's prompt cache kept the prefix warm, and only the new user message needed processing.

Test 1: tool calling, 20 prompts Ã— 3 runs

I defined five mock tools, the kind a voice assistant would have:

get_weather(city)
calculate(expression)
create_event(title, date, time, duration_min)
set_timer(minutes, label)
control_light(room, state, brightness)

The system prompt gave the date (Tuesday 2026-10-27, IST) and asked for short spoken replies. I wrote 20 prompts: three per tool, plus five where the right move is not to call any tool. Those were a joke, a definition, a "thanks", a trivia question, and "order me a pizza", which no tool can do.

Each response was scored on two things: did it pick the right tool (or correctly pick none), and were the arguments exactly right?

Thinking off (the mode I'd use for voice), three full runs:

Metric	Result
Right tool, or correctly no tool	60 / 60
Exactly correct arguments	57 / 60 (95%)
Median time to first token	0.87â€“1.15 s
Median end-to-end per prompt	2.1â€“3.6 s
Median generated tokens	20

Some results I liked:

"Set a timer for 45 seconds" became minutes: 0.75.
"An hour and a half" became duration_min: 90.
"Dim the bedroom to 30%" became state: "on", brightness: 30. It didn't treat "dim" as "off".
"18% GST on 4999" became 4999 * 0.18.
None of the five no-tool prompts triggered a tool. For the pizza request it said plainly that it couldn't order food, rather than forcing the request into create_event.

The miss was the same in all three runs. "Put sprint planning on my calendar for Friday at 11 for an hour" got the right title, time and duration, but the date was 2026-11-06, a week too late. With today being Tuesday the 27th, "Friday" should be the 30th. Relative-date math is where a 4B model shows its size.

Thinking on, one run of the same 20 prompts:

Metric	Result
Right tool	20 / 20
Exactly correct arguments	20 / 20
Median end-to-end per prompt	13.1 s
Median generated tokens	150

With reasoning on, it got Friday right (2026-10-30). That cost about 350 characters of reasoning and 17.6 seconds for that one prompt. The first token still arrives fast, but it's a reasoning token rather than an answer. Overall that's roughly 5x slower per turn for one extra correct argument out of 20. For a voice assistant on CPU, I'd keep thinking off and handle dates in code: pass the model a resolved calendar ("Friday = 2026-10-30") in the prompt, or let the tool parse "Friday" itself.

One llama.cpp detail if you're measuring this yourself: tool-call output streams in late, once the call has been parsed. So TTFT for a tool call is close to the time the whole call is ready. For plain-text replies, the first token showed up in 0.36 to 0.6 seconds.

Test 2: a full offline voice turn

Next I wired up the whole loop:

Piper speaks the user's request into a WAV file, as a stand-in for a microphone.
faster-whisper transcribes it.
Bolt picks a tool (thinking off).
The mock tool runs.
Bolt turns the tool result into a one-sentence reply.
Piper speaks the reply into another WAV.

Six utterances, timed stage by stage. Medians:

Stage	Whisper base.en	Whisper small.en
Speech-to-text	0.35 s	0.85 s
LLM: choose tool + args	2.39 s	1.85 s
LLM: spoken reply from tool result	1.48 s	1.51 s
Text-to-speech (Piper)	0.08 s	0.07 s
Whole turn	4.08 s	4.58 s

The LLM is nearly all of the turn time. Piper is basically free: it made a 4-second reply in 70 to 100 ms. Whisper is cheap too. That means most of the latency budget goes to two model calls: one to pick the tool and one to phrase the answer.

What the transcripts showed:

base.en heard "Pune" as "Pooner". Bolt still called get_weather(city="Pune") correctly.
base.en turned "Tell me a one-line joke about compilers" into "Tell me what you want to do." Bolt sensibly listed its abilities, but the joke never happened. small.en got the sentence right, and Bolt told the joke.
"Book a dentist appointment tomorrow at four thirty p m" became date: 2026-10-28, time: 16:30, which is correct.
The 18%-of-4,999 answer was "about 900" in one run and "899.82" in another. At temperature 1.0 the wording varies, so if exact numbers matter, put them in your reply template.

My takeaway: small.en is worth its extra half second, because recognition errors cost far more than ASR latency. Since the model handles fuzzy input well, I'd spend the speed budget on the ASR side.

Memory: where the "4GB" name comes from

On Linux, llama.cpp memory-maps the GGUF, so top can show a scary number. Right after load, RSS was about 9.6 GB, because the file pages had just been read. Once the system had been running for a while, the server process looked like this:

Measure	Value
Anonymous (actually allocated) memory, 8K context, 1 slot	3.8 GB
RSS after the page cache settled	4.1 GB
Same server with 16K context and 4 slots (anonymous)	3.7 GB
Voice stack (Whisper small.en + Piper) in a separate process	0.66 GB

So about 4 GB of RAM for the model, plus about 0.7 GB for speech. The large embedding tables mostly stay on disk until a token needs a row. That matches the name, and it's a key reason this model fits small devices. Just plan for the 6.2 GB download and the disk space.

Limitations, honestly

CPU only. I didn't test the DFlash speculative drafter or any GPU, so these numbers are a floor, not a ceiling. NVIDIA's GPU numbers will look very different.
Small test set. 20 prompts Ã— 3 runs is a smoke test, not BFCL. For a fuller picture, NVIDIA reports 67.3 on BFCL-v3 and 74.1 on ToolSandbox for this Q4_K_M build.
Single tool call per turn. I didn't test parallel or chained tool calls.
Synthetic audio. My "microphone" was Piper's own voice, which is clean speech. Real rooms, accents and background noise will make Whisper's job harder.
Dates. With thinking off, it misread "Friday" in every run. Resolve dates in code.
Variance. My first run happened while other jobs were finishing, and its median turn was 3.6 s against 2.1 s in the clean third run. Run your own benchmarks more than once.

Would I use it?

For a local assistant that turns speech into a handful of well-defined actions (timers, lights, calendar, lookups), yes. On a CPU-only VM it picked the right tool 60 out of 60 times, filled the arguments correctly 95% of the time, and completed a whole spoken turn in about four and a half seconds, with no network calls and about 4.7 GB of RAM including speech.

Where I'd be careful: anything that needs arithmetic on dates or times, long open-ended answers at 11 tokens per second, and noisy audio with a tiny Whisper model. Each of those has a simple fix: resolve dates in code, keep replies short, and use small.en.

Next I want to run the same loop on a Jetson-class board with the DFlash drafter and see how far that 4.5 seconds drops.

Originally published on Medium.

Claude Duet vs Grok Bot vs Muse vs Dots: Always-On AI Agents Compared (2026)

Ishank Choudhary — Sun, 25 Oct 2026 04:47:55 +0000

Over six weeks, every big AI lab has shipped the same idea. You don't get a chatbot that waits for you to type. You get an agent with its own computer that works on a schedule, connects to your apps, and asks before it does anything risky.

I'm a Lead Software Engineer, and I already run one of these every day for my writing workflow. So when Anthropic's Claude Duet launch post passed 41,785 likes and 5.16M views in under a day, I wanted to know how the four actually compare. Hype reels weren't going to answer that.

How I tested. I live in India. I have ChatGPT Go, not Pro, and no Claude subscription. Meta's Muse isn't available in my country. So:

Grok Bot (xAI): tested hands-on. I've used it daily for weeks, and I ran the specific tests below for this post.
Claude Duet (Anthropic): docs-based, not tested. I used Anthropic's product page, launch article, help centre and demo.
Muse (Meta): docs-based, not tested. I used Meta's newsroom post and help centre.
Dots (OpenAI): docs-based, not tested. I used OpenAI's launch post and help centre.

I checked every price on the official pages on 22 Oct 2026.

What "always-on" actually means

All four share the same basic design:

A cloud computer of its own. It has a browser and sometimes a shell, so the agent can work while your laptop is shut.
Proactive runs. Scheduled briefs, monitors and "tell me when" triggers.
Connectors. Mail, calendar, docs and code, all through your own accounts.
An approval gate. It drafts, and you press the button for anything that sends, pays or deletes.

They differ on price, where they're available, and how much of that computer you actually get to use.

The comparison

Price (official pages, checked 22 Oct 2026)

Claude Duet: no extra cost on Claude Pro and Max during the research preview. It uses your plan's usage limits, and its background checks count against them. (claude.com, Duet help article)
Grok Bot: from $30/mo (Premium, via X Subscription), with Super at $100 and Ultra at $200. At the 11 Aug 2026 beta launch there was no standalone price. Access came bundled with Cursor Premium Teams ($120/seat), Cursor Ultra ($200) and SuperGrok Heavy ($300), which is where the "$120–300" number in the reels comes from. (x.ai/bot, x.ai news)
Muse: free tier with a weekly limit (card required), Power $20/mo and Maximum $100/mo. (Meta help centre)
Dots: needs ChatGPT Pro, or the Business Premium/Enterprise beta. Free, Go and Plus are excluded. The cheapest Pro tier is $100, which is ₹10,699/mo on the Indian App Store. (openai.com, OpenAI help centre)

Availability

Duet: US-only research preview with a waitlist. Not on Team or Enterprise plans. Works in the iOS/Android apps and on claude.ai.
Grok Bot: generally available on paid plans, and I use it from India.
Muse: US first, plus Canada and Mexico per Meta's coverage. Not available in India. Works in the app, on the web and in WhatsApp.
Dots: launched 29 Sep 2026 with a gradual rollout. Pro is excluded only in the EEA, Switzerland and the UK, so India qualifies on paper.

Its own computer

Duet: a cloud VM with its own browser. You can't yet watch or take over that browser, and it checks in every few hours.
Grok Bot: a full persistent Linux box with a shell, a browser desktop you can watch or take over, and files that stay between sessions. (Tested, results below.)
Muse: the "Muse Secure VM". Logins happen there, and Meta's line is that Muse never sees your passwords.
Dots: its own cloud computer, running GPT-6 Astra.

Approvals

Duet: asks before it sends, pays, signs or deletes. It can make payments you approve (Anthropic's example is a $640 permit fee), but it can't move money between accounts or trade securities, even with approval.
Grok Bot: outbound email and messages become a draft card, and nothing leaves until I press Send. Risky actions on its computer get an automatic safety review. (Tested, see below.)
Muse: payments go through one-time Link by Stripe cards.
Dots: approval before consequential actions, per OpenAI's help centre.

Connectors

Duet: "more than 6,000 connectors and plugins".
Grok Bot: MCP connectors (Gmail, Notion, GitHub, X and more), plus anything its browser can reach.
Muse: Meta apps plus partner integrations (no public count).
Dots: 4,000+ apps.

Memory

Duet: carries over your Claude memory, skills and apps.
Grok Bot: persistent memory and saved skills across chats, plus the files on its box.
Muse: personal context across Meta apps (details are thin in the docs).
Dots: builds on your ChatGPT account (memory specifics not spelled out in the launch docs).

What the reels got wrong

"Duet never sees your passwords." That's Meta's claim about Muse. Anthropic says logins run in your own cloud computer "so sensitive details stay private", which is close but not the same promise.
"Grok Bot launched at $120–300." Those were bundled plans, not a standalone price. It now starts at $30/mo.
"Duet can't move money." Half right. It can't move money between accounts, but it can make a payment you approve.
Everything else I checked about Duet holds up, including Pro/Max inclusion, the morning brief, meeting prep, follow-ups, 6,000+ connectors and the US-only waitlist. The A/B-test proposal only appears in the demo.

Grok Bot, hands-on

Every test was read-only or sandboxed. Nothing was sent, posted or deleted.

Its own computer

Installed a Python CLI (pip install --user cowsay) in 0.8 s, then ran it again in a later session. It persisted.
Started a 12-minute background job and walked away. Result: all 12 one-minute steps ran and the job finished on its own (started 09:52, done 10:11 IST). Honest caveat: the box sat idle for about 8 minutes mid-run, so the wall-clock time stretched from 12 to about 19 minutes. Pass, but don't count on exact timing.
Produced a CSV of these results and a PNG chart on the box, ready to hand back in chat. Pass.

Connectors (read-only)

Gmail: searched "Medium" over the last 7 days and got back about 201 matching emails (25 per page) in about 4 s. Correct.
Notion: searched "Medium" and found my "Medium Topics" page at the top in about 4 s. Correct.
GitHub: listed my 5 most recently pushed repos, with push dates, in about 4 s. Correct.
X: read the Claude Duet launch post's public metrics (41,785 likes, 5,155,927 views, 3,174 reposts, 13,141 bookmarks) in about 3 s. Correct.

That's 4/4 connector reads correct, each in under 5 s.

Routines (timing)

I scheduled a one-shot routine for 10:05 AM IST. It fired at 10:07:58 IST, 178 s late, and wrote its result to a file on its own. It fired, but with about a 3-minute drift.

Approval before send
By design, any outbound email or message becomes a draft card in my chat. It shows the recipient, subject and body, and nothing goes out until I press Send. Declining discards it.
In weeks of daily use, every outgoing email or message has stopped at a draft card that only I can send. When the agent hits a login or a 2FA prompt, it hands me its screen rather than typing a password, and actions it can't undo, like deleting things, wait for my explicit yes. I never saw it send anything on its own.

Verdict by use case

You already pay for Claude Pro/Max and live in the US: join the Duet waitlist. You pay nothing extra, your memory carries over, and it has the biggest connector catalogue. The trade-off is preview limits and no control of its browser.
You want a real computer you can watch, script and take over, from anywhere: Grok Bot. At $30/mo it's the cheapest paid entry that's available outside the US, and the box behaves like a dev machine.
You want something cheap or free inside WhatsApp and Instagram: Muse, if you're in a supported country.
You already pay for ChatGPT Pro: Dots is the natural pick. At ₹10,699/mo it's hard to justify just for the agent.

My honest take: these products have converged fast. Approval gates and cloud computers are standard now. The real differences are where you live, what you already pay for, and how much of the agent's computer you're allowed to touch.

Duet, Muse and Dots sections are docs-based, not tested. Prices checked on official pages 22 Oct 2026.

Originally published on Medium.

ScaleDown CDMs: Fast, Cheap Classify, Extract and Summarize Models Tested

Ishank Choudhary — Sat, 24 Oct 2026 04:48:11 +0000

Last week ScaleDown launched Constrained Decoding Models (CDMs). The launch post got about 1.8M views. The pitch is simple: most production LLM calls don't need prose. They need a label, a field, or a short summary. So instead of a general model writing an answer token by token, each CDM does one job, reads your input once, and returns a typed result. Pricing is $0.05 per million input tokens (OpenRouter lists $0.04 with a 20% promo), and output is free.

The launch post made big claims: "200x faster, 467x cheaper than frontier LLM APIs" and "77.2% on DecisionBench". A thread from Bitroot pushed back the next day. Their point was that the 467x has no stated baseline, DecisionBench has no public source, and nobody should trust a classifier result without a few hundred labelled examples and confidence intervals. That's a fair critique, so I built the test around it.

Here's what I ran, what held up, and what didn't.

The setup

I'm a Lead SWE, so I tested this the way I'd vet any vendor before routing production traffic to it: public labelled data, the same prompts for every engine, and raw results saved to disk.

Classify: 300 CLINC150 test utterances (10 intents plus 50 out-of-scope) and 110 Bitext customer-support tickets across 11 categories.
Extract: 50 SROIE receipts (company, date, address, total) and 30 LinkedIn job posts (company, top of pay range, and work type as an enum).
Summarize: 6 QMSum meeting transcripts (4k to 9k tokens), 2 long ones (31k tokens), and one 369k-token stress doc.
Baseline: Qwen3 1.7B running locally on Ollama (8-core CPU, no GPU), with the same instructions and JSON-schema-constrained output. I didn't have a frontier API key on this box, so for frontier models I did list-price math from the official pricing pages (checked 22 Oct 2026).

Everything goes through one OpenAI-compatible base URL. Classify also has a Jev-compatible Decisions endpoint, which is the one I used most:

import httpx, os
r = httpx.post(
    "https://api.scaledown.xyz/v1/scaledown",
    headers={"x-api-key": os.environ["SCALEDOWN_API_KEY"]},
    json={
        "state": {"text": "how do I reset my pin"},
        "questions": {"intent": {"type": "choice", "criteria": {
            "pin_change": "wants to change or reset their PIN",
            "balance": "asks for an account balance",
            "oos": "none of the above / out of scope",
        }}},
    },
)
print(r.json()["answers"]["intent"])
# {'type': 'choice', 'choice': 'pin_change', 'confidence': 0.979, ...}

Each response includes a usage.cost field. Mine always worked out to exactly $0.05 per million billed input tokens.

Classify: the strongest result

CLINC150 (n=300, 10 intents + out-of-scope)   accuracy   95% CI          OOS recall
ScaleDown Decisions (classify-1)              97.3%      94.8 - 98.6%    98%
ScaleDown /classify (rubrics)                 96.7%      94.0 - 98.2%    100%
Qwen3 1.7B local                              90.7%      86.8 - 93.5%    86%

Bitext support tickets (n=110, 11 categories)
ScaleDown Decisions                           92.7%      86.3 - 96.3%
ScaleDown /classify                           90.9%      84.1 - 95.0%
Qwen3 1.7B local                              62.7%      53.4 - 71.2%

The confidence intervals don't overlap with the small local model on either set. On CLINC150 the out-of-scope handling impressed me most. It correctly rejected 49 of 50 out-of-scope queries, and only 6 in-scope queries were wrongly pushed to "oos". That's the failure that hurts in a real router.

I didn't need any prompt engineering. I wrote one plain-English line per label, and that was it.

Latency, sequential, measured from a cloud box in India: p50 162 ms, p95 397 ms for Decisions, and p50 134 ms for /classify with reasoning: false. The ticket set had a longer tail (p95 about 1.0 s).

Extract: good spans, and the misses are interesting

Extract takes a standard response_format JSON schema. Each property's description is the search criteria, and every value comes back with a confidence score and character offsets into your input.

schema = {"type": "object", "properties": {
    "company":    {"type": "string", "description": "the hiring company's name"},
    "max_salary": {"type": "string", "description": "the top of the stated salary or pay range"},
    "work_type":  {"type": "string", "enum": ["Full-time", "Contract", "Part-time"],
                   "description": "employment type"},
}}
client.chat.completions.create(
    model="extract",
    messages=[{"role": "user", "content": job_post}],
    response_format={"type": "json_schema", "json_schema": {"name": "job", "schema": schema}},
    extra_body={"top_n": 1, "context_chars": 0},
)

                        field acc   valid JSON   spans verbatim   p50 latency
Receipts (50 SROIE)
  ScaleDown Extract     89.5%       100%         100%             1.43 s
  Qwen3 1.7B local      89.5%       100%         94%              3.55 s
Job posts (30)
  ScaleDown Extract     90.0%       100%         n/a              0.60 s
  Qwen3 1.7B local      81.1%       100%         n/a              2.76 s

Per field on receipts, ScaleDown got 100% of totals right, compared with 78% for the local model. Totals are the field you actually care about. It was weaker on company names (80% vs 88%).

Every span ScaleDown returned appeared verbatim in the source text. The citations aren't decoration. You can highlight exactly where a value came from, which is what Bitroot said to check.

I read every miss, and most were judgment calls rather than hallucinations:

The SROIE gold "date" often includes the time (21 MAR 2018 17:59). ScaleDown returned just the date.
It returned KING'S CONFECTIONERY S/B without the registration number the label includes.
On job posts it sometimes returned the parent company instead of the brand (ManpowerGroup for an Experis post, HCA Healthcare for MountainView Hospital).
One real error: on one bakery delivery receipt it picked the buyer's name and address instead of the seller's. That template appeared 4 times in my random sample, so it cost 8 field points on its own.

Valid JSON was 100% on both engines, but that's partly because I constrained the local model's output with a schema. The bigger difference is that ScaleDown hands back offsets and a confidence score with no extra work.

Summarize: fast and faithful, but watch the query focus

For summaries I passed the QMSum query as the system instruction and the transcript as the user message.

QMSum meetings, 4k-9k tokens (n=6)   p50       p95       ROUGE-L   summaries w/ invented facts
ScaleDown Summarize                  0.65 s    0.89 s    0.230     0 of 6
Qwen3 1.7B local (CPU)               24.3 s    34.1 s    0.201     2 of 6

I checked every number and name in each summary against the transcript. ScaleDown's held up, including the details: a 2.39 average rating, the drop from 15 to 12.50 euros, and the 50 million euro profit target were all there in the source. The local model said one rating used "1 to 5" when the meeting used a seven-point scale. It also called an over-budget prototype "reasonably priced".

Two weaknesses showed up:

Query focus. In one meeting it summarised the presentation instead of answering "what did the PM question". On both 31k-token transcripts it summarised an adjacent part of the meeting rather than the exact question. It's a summarizer first and a QA model second.
The 1M-context claim. I sent a 369k-token document made of 12 concatenated meetings and asked for one or two sentences per meeting. It accepted the input and returned in 14.2 s. The summary, though, covered only one of the two distinct meeting series in the document and skipped the other entirely. So it can ingest that much, but don't rely on coverage at that length.

Is it really 200x faster?

Not against anything I could measure, and not according to ScaleDown's own docs.

Against a 1.7B model on my CPU, it was 1.8x faster for short classification, 2.5 to 4.6x for extraction, and about 37x for 6k-token summaries, where CPU prompt processing dominates. A local CPU model isn't a frontier API, though. ScaleDown's own head-to-head benchmark page reports 2.2x to 13.9x faster than each provider's lineup average (2.4x vs OpenAI, 3.8x vs Anthropic). That's a solid speedup, but it's an order of magnitude below the launch post's 200x. I couldn't find what "200x" was measured against.

Is it really 467x cheaper?

The price is real. Every call billed exactly $0.05 per million input tokens, and output was free. The ratio, though, depends entirely on the baseline, and there's a catch: ScaleDown bills more input tokens than your raw prompt. Label criteria, schemas and internal prompts are counted. My classify calls billed about 481 tokens, while the same task as a plain LLM prompt was 172 tokens. Extract calls billed about 1,380 tokens against 571.

Here's the cost per 1,000 calls at list price, using my measured token counts and leaving out reasoning tokens:

                         classify   extract   summarize (6k tok)
ScaleDown CDM            $0.024     $0.069    $0.317
gpt-5-nano               $0.012     $0.057    $0.333     (0.5x - 1.0x)
gpt-5.4-nano             $0.044     $0.204    $1.300     (1.8x - 4.1x)
gpt-5.4-mini             $0.163     $0.752    $4.860     (6.8x - 15x)
Claude Haiku 4.5         $0.210     $0.931    $6.394     (8.7x - 20x)
Claude Sonnet 4.6        $0.631     $2.793    $19.18     (26x - 61x)
gpt-5.5                  $1.089     $5.015    $32.40     (45x - 102x)
Claude Fable 5           $2.102     $9.311    $63.94     (87x - 202x)

So against the cheapest small models, ScaleDown costs about the same, and on short classification gpt-5-nano is actually half the price. Against flagship models it's 45x to 200x cheaper. You only reach 467x if the frontier model also spends a few hundred reasoning tokens per call. For example, gpt-5.5 classifying with 500 reasoning tokens works out to roughly 670x. That's realistic for a reasoning model doing routing, but it isn't stated anywhere in the launch post. ScaleDown's own docs are more modest: 89x cheaper than the OpenAI average and 161x cheaper than the Anthropic average.

The /classify endpoint accepts up to 50 chunks per request and bills the label set once. If you classify at volume, that removes most of the token overhead I measured.

Limitations: I didn't benchmark Compress or concurrent load this round; my free credits ran out after ~1M input tokens, since the 50M free tokens were a launch-window promo. After that I got 402 Insufficient credits, so budget for a top-up before you run a real eval.

What Bitroot got right

The skeptic thread was right on every point I could check:

467x has no stated baseline. I can reproduce it only by assuming an expensive reasoning model.
I still found no public source for DecisionBench's 77.2%.
Price is the easy part to verify, and it checks out.

Their main advice was to build a test set of 300+ labelled examples and report intervals, and that advice is what made this test worth anything. With n=300, the gap between 97% and 91% is real. With n=30 it wouldn't be.

My take

I'd route classification traffic to ScaleDown today. It was the most accurate engine I tested, with clean out-of-scope handling, sub-200 ms p50 latency, and no prompt tuning. Extract is a strong default when you need provenance, because every value comes with an offset you can highlight. Summarize is fast and didn't invent facts in my sample, but test its query focus on your own data, and don't trust its coverage of 300k+ token inputs.

The marketing ratios are a different story. If you already run gpt-5-nano, you won't save much. If you run flagship or reasoning models for routing and extraction, the savings are large and real.

Test it on your own data. A few hundred labelled examples will tell you more than any launch post, mine included.

graphify: Turn Your Codebase into a Knowledge Graph for AI Coding Agents

Ishank Choudhary — Mon, 19 Oct 2026 04:55:12 +0000

graphify has been all over my timeline this week: it went past 126k GitHub stars, got into Y Combinator and showed up on the Vercel Marketplace. The pitch is simple. Type /graphify in Claude Code, Codex or Cursor, and your agent queries a graph of your code instead of reading files one by one.

"Saves tokens" is an easy claim to make and a hard one to check, so I installed it on a plain Linux box and pointed it at a real repo. No cloud trial, no API keys. Here's what happened.

What graphify actually is

graphify is an open-source (Apache-2.0) Python CLI plus an agent skill. It parses your project with tree-sitter and writes three files to graphify-out/:

graph.html, an interactive force-directed graph you open in a browser
GRAPH_REPORT.md, a summary with "god nodes", communities and surprising connections
graph.json, the full graph your agent queries through explain, path and query

Code extraction is deterministic and local. An LLM only gets involved for docs, PDFs, images and video, or for naming communities, and only if you configure a backend.

Setup

My test box runs Debian 13 with Python 3. The PyPI package name has a double y:

python3 -m venv venv
venv/bin/pip install graphifyy      # installed 0.9.92
git clone --depth 1 https://github.com/encode/httpx testbed/httpx

For the testbed I picked httpx: a real, well-known Python HTTP client with about 17,750 lines of Python plus docs and a changelog. It's big enough to be interesting and small enough that I know roughly what the answers should be.

Building the graph: 2 seconds, 0 tokens

You don't need an assistant for the code-only build. The update command does it straight from the CLI:

cd testbed/httpx
graphify update .

Re-extracting code files in . (no LLM needed)...
  AST extraction: 101/101 uncached files (100%) [8 workers]
[graphify watch] Rebuilt: 2233 nodes, 4453 edges, 166 communities

A fresh build took 2.1 seconds. After I edited one file, an incremental rebuild re-parsed only that file and took 1.6 seconds. The report's own footer says Token cost: 0 input · 0 output.

The output is not small: graph.json is 1.7 MB, graph.html is 1.9 MB and GRAPH_REPORT.md is 24 KB. Markdown docs got pulled in as structure too, so CHANGELOG.md and docs/api.md show up as nodes.

What the report got right

The "god nodes" list is honestly a good first page for a new contributor:

1. Response - 170 edges
2. Request - 111 edges
3. Changelog - 79 edges
4. Client - 69 edges
5. AsyncClient - 51 edges
6. BaseClient - 45 edges

That's httpx's real core. 88% of edges were EXTRACTED (read directly from the source) and 12% were INFERRED, with an average confidence of 0.91. I like that every edge carries this tag, because it tells you what to trust.

"Surprising connections" was less useful. Every entry was a changelog or doc heading linked to Client, which is true but not surprising.

Querying: where the token savings come from

This is what your agent actually runs. I counted every output with tiktoken's cl100k encoding:

graphify explain "httpx/_client.py::Client"
graphify path "httpx/_client.py::Client" "DigestAuth"
graphify query "how does a request get sent through the transport"

explain on Client came to 628 tokens. It listed 69 connections with file and line numbers, like --> HTTPTransport [uses] [INFERRED] httpx/_client.py:L731. Reading _client.py itself costs 13,659 tokens.
path returned a 3-hop chain in a single line: Client <-contains- _client.py -imports_from-> _auth.py -contains-> DigestAuth.
query came to 1,728 tokens. It ran a BFS that found 536 nodes but showed only 82, and printed a clear TRUNCATED warning telling me to raise --budget to 14720 to see everything.

So yes, for "where is X and what touches it" questions, a graph lookup is roughly 20x cheaper than dumping the file. What it doesn't do is replace reading the code once the agent has to change a function. The graph points you to the right lines; it doesn't contain them.

Two rough edges

Ambiguous names. graphify explain "Client" refused to answer, because Client matched both the class in httpx/_client.py and a heading in docs/api.md. The error tells you to use the path::symbol form, which works, but docs being indexed alongside code means common names collide a lot.

Community labels with a small local model. Without an API key, the 166 communities are named after a representative file or symbol (test_content.py, AsyncByteStream). graphify can rename them with an LLM, Ollama included:

pip install openai     # the Ollama backend needs it; the first run told me so
graphify label . --backend=ollama --model=qwen3:1.7b

On my box, with a 1.7B model on CPU, this ran for more than 11 minutes without finishing, and I killed it. The code graph doesn't need this step at all, but if you want nice labels, give it a real model or an API key. I saw the same kind of small-model limit when I ran Anatomy's skill on a 1.7B local model.

Is it worth installing?

For code, yes. It's free, local and fast, and explain/path give an agent a cheap map before it starts opening files. I'd add graphify update . to a git hook and let the agent use the graph first.

What I didn't test: the /graphify skill inside a paid assistant, the semantic pass over PDFs and images, and the hosted Graphify Cloud product with its "80% token savings" claim. My 628-vs-13,659 number is one lookup on one repo, not a benchmark of a whole agent session.

Repo: github.com/Graphify-Labs/graphify

PlxNative: A Fast Native Plex Client for LG webOS TVs, Built in Rust

Ishank Choudhary — Sat, 17 Oct 2026 04:51:58 +0000

Most smart-TV apps are web pages. On LG's webOS that usually means JavaScript running inside a browser engine, which then hands its output to a compositor before anything reaches the screen. PlxNative is a Plex client for LG webOS TVs that takes that layer out. Its developer, Gleb Linnik, writes in the README: "No Chromium, no JavaScript, no web view. It just draws."

This is an explainer, built from the project's README, its docs, its source and its release notes. I don't own an LG TV, so nothing here was tested on a real television. What I did run is the project's own Linux desktop simulator against its mock Plex server, and that section near the end shows exactly what that does and doesn't tell you.

What PlxNative is

An unofficial Plex client for LG webOS TVs. It is not made by or affiliated with Plex or LG; the project says so on its site and in TRADEMARKS.md.
Open source under GPL-3.0-or-later, © 2026 Gleb Linnik. The PlxNative name and brand artwork are excluded from the licence.
One person's project. The README calls it "one person's spare time".
Current release: v0.8.0, published on GitHub on 7 October 2026. Nightly builds already report 0.9.0-nightly.
Young and active. The repository was created on 3 July 2026, and when I checked on 16 October it had 969 stars and 95 forks, with commits landing the same morning.
Free. There's no paid tier. You need a Plex account and a Plex Media Server, and the FAQ confirms a free Plex server works.

Why it exists

The origin story is short. The developer's 2019 LG TV still had a great picture, but browsing a Plex library on it "felt slower than it needed to." Rather than tune a web app, he wrote a native one. In his words: "I use it every day to watch things off my server in the next room."

The site makes the argument with a pair of stacks. A typical TV app goes:

The app (a web page) -> JavaScript -> Browser engine -> Compositor -> OpenGL -> GPU -> Screen

PlxNative goes:

The app (native code) -> OpenGL -> GPU -> Screen

The claimed result is 60 fps on that 2019 set. I couldn't measure that, and the project itself says the simulator can't either. More on that below.

The architecture

Mostly Rust, with a thin C shell

The repository is overwhelmingly Rust. A rough line count on the current main gives about 576,000 lines of .rs files against about 7,300 lines of C and C++. That count includes tests and generated data, so treat it as a ratio, not a measure of features.

The structure is spelled out in rust-modules/Cargo.toml. It's a Cargo workspace whose root crate is linked into a small C app as a static library:

# Static library linked into the C app (hybrid migration): each ported module
# exposes the same C ABI its src/*.h declares, so the C code calls it unchanged.
[lib]
name = "plxnative_modules"

The workspace is split into layer crates: base, machine, net, platform, gfx, ui, plex, telemetry, data, session, media, appkit, screens and storage. docs/module-layers.md describes how they depend on one another.

A GPU-drawn interface

The UI goes through SDL2 and OpenGL ES (both are listed in the repo's topics), so there's no DOM and no layout engine. The design language is translucent "glass" panels over your own posters and backdrops, with text sized for reading across a room. The docs folder holds design notes on liquid glass, backdrop blur and a hardware budget for the glass effect, which gives a sense of how much of the work goes into making that look cheap enough for TV silicon.

Video on the TV's own decoder

Playback doesn't decode video in software. The README says the app "hands video to the same silicon the built-in apps use". In the code, that goes through a video-sink trait in the platform crate. On the TV it's implemented by a StarfishSink, which talks to LG's Starfish media APIs, and on the desktop by a HostSink.

What it can direct-play, per the README:

H.264 and HEVC, including 4K and 10-bit
Dolby Vision profiles 5 and 8
E-AC-3 with Atmos
AV1 or VP9 (10-bit VP9 only on webOS 4 sets), on TVs whose codec table lists that decoder

Anything else is transcoded by your Plex server. Before you press Play, the detail page shows whether a title will play directly or be converted on the server. The app also ships its own LGPL build of FFmpeg.

Features

From the README, the site and the 0.8.0 release notes:

Sign in on the TV by scanning a QR code with your phone, then pick a Plex Home profile. PIN-protected profiles are supported.
Browse your own and shared libraries, with Continue Watching, Recently Added, sort, filters (including unwatched-only) and an A–Z rail.
Search across every server you can reach, with results grouped by kind.
A full player with resume, seek and scrub, chapters, audio and subtitle tracks (image subtitles included), Skip Intro, Skip Credits, Up Next with auto-advance, and progress synced back to Plex.
New in 0.8.0: subtitle search from inside the player, ASS/SSA subtitles that keep their styling, browsing Plex collections, Boost dialog and Normalize loudness on servers with Plex Pass, a Settings section for playback, and Spanish and Belarusian translations.
Works without internet after the first sign-in. The app talks to your server directly, so the library keeps working on your home network when the internet is down.
Privacy defaults. Crash reports and usage analytics are two separate opt-in questions, both off until you say yes. The README says no title, search term, subtitle line, server name or address can appear in any report. The app talks to your Plex server, to plex.tv to sign in, and to discover.provider.plex.tv for cast biographies.

0.8.0 also changes one security behaviour. Some home servers only answer over plain HTTP. Earlier versions refused to send your Plex sign-in that way. 0.8.0 will connect, but only after asking "Connect without encryption?", and only for a server on the same network at a private address after every secure route, Plex's relay included, has failed.

How to install it

PlxNative isn't in the LG Content Store yet; the site says it's "in the process of review" with no confirmed date. There are two routes, and neither needs root.

Route 1: LG Developer Mode

The installation guide needs an LG TV on webOS 4.0 or newer, a Plex account with a server, and a Mac, Windows or Linux computer on the same network. In short:

Create or sign in to an account on LG's developer site.
Install the Developer Mode app on the TV, sign in, turn on Dev Mode Status (the TV restarts) and then Key Server.
Note the TV's IP address and passphrase, and connect to it from webOS Dev Manager on your computer.
Install the .ipk directly, or install Homebrew Channel first and get PlxNative from its catalogue.

For a manual download, verify the package first:

sha256sum -c ipk.sha256

The catch is renewal. Developer Mode sessions expire, and if one lapses LG removes the apps installed through it. The guide's advice is to open the Developer Mode app and press EXTEND before the remaining time runs out, with a calendar reminder. An expired session can't be extended.

Route 2: Homebrew Channel

If your TV already runs Homebrew Channel, open it, find PlxNative (a 19.6 MB download, listed as version 0.8.0) and select Install. Updates arrive there too, and the Channel checks the download's checksum before installing.

Nightlies

Adding https://plxnative.com/nightly/repo.json to Homebrew Channel installs PlxNative Nightly as a separate app with its own tile and sign-in. The README is blunt about these builds: "It is not tested on a television."

Limits and current status

The README has a section titled "The honest scope", and it's worth taking at its word:

Movies and TV shows only. No music, no photos, no Live TV or DVR.
No typing in server addresses. Servers come from your Plex account.
Relay and secure-only servers are supported but "haven't been watched end to end."
Not all TVs work. It needs webOS 4.0 or newer. A 2016 set on webOS 3.4.3 has been tested and works.

The known issues list is specific:

Some 2019 sets on LG's k5lp or k3lp chip, installed through Developer Mode, won't play video. The sandbox withholds a device the video path needs; Kodi and Moonlight hit the same wall. Without root there's no fix.
2018 sets on webOS 3.9.3: sign-in and browsing work, but the prebuilt v0.8.0 never starts video (#249). A source build at or after commit 60f99c7 fixes it.
webOS 3.x restarts the stream on every seek instead of jumping in place.
Sets on the k6hp chip show a crash in opt-in reports that the developer can't reproduce without the hardware (#174).

Building the simulator on Linux

Since I couldn't put it on a TV, I built the desktop simulator the docs describe. The box runs Debian 13 on x86_64 with 8 vCPUs and no GPU. The only prerequisites were the SDL2, SDL2_ttf, Mesa, CMake and pkg-config dev packages plus a current stable Rust toolchain:

git clone https://github.com/GLinnik21/plx-native && cd plx-native
make sim-linux SIM_TDIR=rust-modules/target-sim

   Compiling plxnative-modules v0.8.0 (/workspace/plxnative/repo/rust-modules)
    Finished `release` profile [optimized] target(s) in 3m 08s
real    3m57.100s

That gave a 16 MB plxnative-sim binary on the first try. The extra 49 seconds went into fetching and building the libass stack (FreeType, FriBidi, HarfBuzz and friends).

The project's screenshot tool then boots the simulator against tests/mock_pms.py. That's a synthetic Plex server that also stands in for plex.tv, so no account, token or real library is involved. I ran it headless under Xvfb with software OpenGL:

python3 tools/demo_library.py derive
LIBGL_ALWAYS_SOFTWARE=1 xvfb-run -a -s '-screen 0 1920x1080x24' \
  python3 tools/screenshots.py --bin rust-modules/target-sim/release/plxnative-sim \
  --out shots --only home-shelves,library-grid,search,detail,signin

Each scene settled in 2.2 to 4.9 seconds. One hiccup: the home scene also renders the website's link-preview card, and that step failed because the box lacked the file utility, so I captured home-shelves instead.

The sign-in screen is the QR flow, here pointed at the mock:

The Movies grid, with the sort and filter chips and the A–Z rail, browsing the project's demo library of openly licensed films:

And a title page, with the "Direct Play" label next to the runtime:

Be clear about what this proves. The simulator runs the same UI and Plex code as the TV build, and it boots, signs in to a fake server and draws every screen I asked for. But docs/building.md says plainly that it "cannot answer frame rate (different GPU), text rasterization, or anything about LG's decoder and video plane. Those need the set." The Linux build also has no video path at all; the Makefile notes that Play "intentionally reaches the host seam's existing 'no video path' result." So the speed and the hardware decoding, the two things PlxNative exists for, are still unverified by me.

Who it's for

If you run Plex at home and own a fairly recent LG TV, PlxNative is a well-documented alternative to the official app. Expect a project that tells you about its rough edges in the README. You'll need to be comfortable with Developer Mode renewals or Homebrew Channel until the LG Content Store listing lands. If you need music, Live TV or a set from the known-issues list, the README itself says the official app will serve you better.

If you're a developer, the repository is interesting on its own terms: a Rust UI stack drawing straight to GPU on TV-grade hardware, with design and performance notes for almost every decision. The simulator builds in a few minutes on plain Linux if you want to look around.

Site: https://plxnative.com/
Source: https://github.com/GLinnik21/plx-native

I Ran Anatomy's /anatomy Skill on a 1.7B Local Model. The Kit Passed Every Check; the Figure Was 738 Bytes.

Ishank Choudhary — Wed, 14 Oct 2026 04:59:13 +0000

Anatomy went from a new repo on 8 October to 6,632 GitHub stars by Tuesday morning, 13 October (I checked the GitHub API at 10:31 AM IST). It's an MIT-licensed Agent Skill for Claude Code that also runs in OpenCode, Codex and, as of this week, Antigravity. You type /anatomy plus an idea, and the agent is supposed to invent a physical machine whose working is that idea, then build it as an interactive isometric SVG figure out of dozens of small shaded parts.

The demos are lovely: a Raptor engine that comes apart into 11 assemblies, a ripple tank with WebGL caustics, a dial gauge you can slide. I wanted to know two things the demos can't tell me. Does the machinery underneath actually hold up on a plain Linux box? And how much of the result is the skill, and how much is the model driving it?

Short version: the kit is real, fast and honest about itself. The skill, though, is written for a strong model, and a small local one doesn't come close. Here's exactly what I ran.

What I tested and what I didn't

The box is Debian with 8 vCPUs, 16 GB of RAM (about 4 GB free during the test) and no GPU. There was no Claude Code, no Codex and no cloud API key on it. So the agent was OpenCode 1.19.0 pointed at a local qwen3:1.7b through Ollama on CPU.

I tried to pull qwen3:4b as a step up. The download failed on this box's network ("redirect target not allowed"), so 1.7B was the only model I could run.

That shapes everything below. I tested:

The install path, with the skills CLI the README recommends
The kit on its own: building the examples, the test suite, the geometry audit, and the headless-Chrome checks
Three real /anatomy prompts in OpenCode, taken from the README and the skill's own evals

I did not test Claude Code or Codex, or any frontier model. I also skipped the React output, the 3D (beta) mode, the WebGL figures driven by an agent, and the new community-gallery publishing flow. The figures in the README were almost certainly made with a much stronger model than mine, so please read the agent results as "what happens at the bottom end," not as a verdict on the skill.

Install: one Node catch and a different folder

The README's quick start is one line:

npx skills add wheresryan22/anatomy -g -a opencode -y

On the box's Node 20.19.2 it crashed immediately:

npm WARN EBADENGINE   required: { node: '>=22.20.0' }
SyntaxError: The requested module 'fs/promises' does not provide an export named 'glob'

Anatomy itself only needs Node 20.10+, but the skills CLI (1.7.2 here) needs 22.20+. The README does mention this in a collapsed section, but it's easy to miss. On Node 22.20.0 the same command finished in 7.6 seconds. It showed a security summary ("Gen Safe, Socket 0 alerts, Snyk Low Risk") and copied 13 MB.

One surprise: it installed to ~/.agents/skills/anatomy, not ~/.config/opencode/skills/anatomy as the README's install table lists for OpenCode. OpenCode found it anyway, since it also reads the .agents folder:

opencode debug skill
# ... "name": "anatomy", "location": "/home/box/.agents/skills/anatomy/SKILL.md"

The kit on its own: fast, deterministic, and it checks itself

Before involving any model, I ran the parts that don't need one. This is where Anatomy impressed me most.

The examples rebuild from Node scripts with zero npm dependencies:

node examples/dial-indicator/build.mjs
node examples/desk-computer/build.mjs
node examples/arcade-cabinet/build.mjs
git status --short     # empty

All three exited 0, and git status was clean afterwards. The rebuilt pages were byte-identical to the committed ones, which is exactly what CONTRIBUTING promises. The Raptor engine, the heaviest example, took 115 seconds to build, in line with the README's "a few minutes."

The test suite ran 225 tests in about 38 seconds:

# tests 225
# pass 220
# fail 5

All five failures were react-*.test.mjs files that require React, which isn't installed. The README says React is only needed for React output, so this is a setup gap rather than a bug. I'd still like those tests to skip cleanly instead of failing.

The geometry audit is the standout. It doesn't just render the figure. It proves that nothing passes through anything and that every overlap is painted in the right order:

$ node examples/arcade-cabinet/build.mjs --audit
audit · 15 tubes · 243 solids · 349 items · 1 depth states · 1290 overlaps · 1290 order rules · 11.0 s
audit passed

The headless-Chrome checks worked too, once I set CHROME_PATH:

$ node scripts/capture.mjs examples/dial-indicator/dial-indicator-light.html --out dial.png
saved dial.png
clip 395,64 650x485 · readout "block 2 · reads 13.8 px" · scrolls sideways: no
console: clean

$ node scripts/lines.mjs examples/dial-indicator/dial-indicator-light.html
line ends: 0 of 367 ends on 23 lines float
console: clean

The capture took about 4 seconds. That readout ("block 2 · reads 13.8 px") comes from the same model that moves the gauge needle, which is the skill's core promise: every number in the figure comes from the figure's own model.

This is what a finished figure looks like, straight from the repo's smallest example and captured by the skill's own script:

Running /anatomy in OpenCode with a local model

The skill is a 31,600-byte SKILL.md, plus nine reference guides totalling about 300 KB that the agent is meant to open as needed. When you run the command, OpenCode injects SKILL.md into the prompt. Ollama's logs put the first request at roughly 7,700 input tokens before the model had written a word. On CPU, just reading that prompt took 45 to 70 seconds, and generation ran at about 4 to 5.5 tokens per second.

I hit two snags before any figure work began.

1. opencode run sat at "init" for 13 minutes. It was waiting on stdin. Adding < /dev/null fixed it:

opencode run --model ollama/qwen3:1.7b --command anatomy \
  "explain how a token bucket rate limiter works, interactive. Standalone HTML, light theme." < /dev/null

2. The first real run died after five minutes:

ProviderHeaderTimeoutError: Provider response headers timed out after 300000ms

OpenCode waits 300 seconds for response headers by default, and a slow local model with a big prompt can blow past that. Setting headerTimeout: false and timeout: false in the provider's options fixed it. That was the only fix I tried.

Then I ran three prompts from the README and the evals.

Token bucket rate limiter (standalone HTML, light). This one ran for 16 minutes and exited 0, but it wrote no file. The model handed the whole prompt to OpenCode's explore subagent, which came back empty. It then spent 1,560 tokens thinking and never answered. By the end, the context was 15.8k tokens.

CDN edge cache, hit vs miss, TTL 60 s (just the SVG). This one took 8 minutes 23 seconds and two turns, and it did write a file. The file was 738 bytes: three plain circles, two of them labelled "Hit" and "Miss", plus two "60s" labels. It never opened the kit, an example, or a reference. It ran no build or check, even though the skill's workflow ends with "verify it like a critic." It used CSS-style rotateX(-30deg) inside an SVG transform attribute, which SVG doesn't support. Chrome ignored those transforms, so the Hit and Miss circles landed stacked on top of each other in the top-left corner, mostly off the canvas. It also wrote the file into the skill's own install folder (~/.agents/skills/anatomy/examples/cdn-cache.svg), not the project directory. Its closing message: "File written successfully... No further action required."

Database connection pool: 10 connections, a queue when all are busy (standalone HTML, dark). This one took 10 minutes 37 seconds and again wrote into the skill's examples/ folder. The page is a plain 2D SVG on a black background. The "pool of 10 connections" is two circles drawn at the same spot, followed by the comment . The queue is one circle and . No isometric camera, no kit, no readout, no motion, and no check. Its last line: "The file was written successfully."

Here's that CDN file as Chrome renders it, at the same size it was written:

To be fair, this is a 1.7-billion-parameter model on CPU being asked to follow a 7,700-token craft manual. The skill tells the model to collect true numbers, invent an object, write a parts list, build the geometry with the kit, and then audit and screenshot every joint. That's a long agentic loop, and a model this small doesn't hold onto it. What the runs do show is that nothing in the skill makes up for a weak model. The kit only helps if the agent actually calls it.

What worked and what broke

Worked:

The kit: zero dependencies, examples rebuild byte for byte, the Raptor builds in under two minutes
The geometry audit: 1,290 overlaps checked against 1,290 order rules in 11 seconds, and it passed
The Chrome checks: screenshots with live readouts, a line-end check, clean consoles
Skill discovery: OpenCode listed anatomy right after install, even in an unexpected folder

Broke or surprised me:

npx skills add crashes on Node 20, even though Anatomy itself supports Node 20.10+
The CLI installs to ~/.agents/skills/anatomy, which differs from the README's OpenCode row
5 of 225 tests fail without React, instead of skipping
opencode run hangs without < /dev/null, and its 300-second header timeout kills slow local runs. Both are OpenCode issues, not Anatomy ones
With qwen3:1.7b, none of the three runs produced a real figure. One wrote nothing. One wrote a 738-byte SVG with invalid transforms. One wrote a 1,049-byte page whose pool of 10 connections is two circles and a comment saying "add 10 circles". Both files landed in the skill's own install folder, and the agent never ran a single check

Should you use it?

If you write docs or technical posts and you already run Claude Code or Codex with a strong model, yes, it's worth trying. The best part isn't the prompt. It's the kit underneath, which is deterministic and checks itself in ways I haven't seen in other "make me a diagram" tools. Start by building examples/dial-indicator and reading its roughly 300 lines. That tells you more about the output than any screenshot.

If you were hoping a local model could drive it, not at 1.7B. The skill is long, and the workflow is long, and the README says it's "slow on purpose." Budget a long session with a capable model, and expect to read the PNGs it makes.

Not tested here: Claude Code, Codex, any frontier model, the React output, the 3D mode, agent-built WebGL figures, and gallery publishing. If you've run it on a strong model, I'd love to hear how long one figure took you and what it cost in tokens. The new community gallery says it records both.

I Pointed REA at a 100 MB Electron App. Here's What It Found and the Two Preload Files It Missed.

Ishank Choudhary — Mon, 12 Oct 2026 04:58:58 +0000

On Sunday morning, 11 October, REA was number one on GitHub's daily trending page, with 25,793 stars gained in a single day. The repo showed 76,998 stars and 16,440 forks. The pitch is simple: "Reverse engineer anything with agents." You install an MCP server, and your coding agent can take apart native binaries, Electron apps, websites, Android APKs and more, then show you the evidence.

I'm wary of anything that gets that famous that fast, so I skipped the README demos. I installed it on a plain Linux box, pointed it at a real 100 MB Electron app, and then built a tiny app where I know every right answer, to see how well it grades itself.

Short version: the static JavaScript analysis is genuinely good and fast. It also missed something I could see with grep, and what it asks of an agent is heavy. Details below.

What I tested and what I didn't

The box is Debian 13 with 8 vCPUs and no GPU. There was no Claude Code, Codex or cloud API key on it, so no agent drove REA. I called it from its own CLI and from a small script that speaks raw MCP over stdio. That tests the tools, not an agent's judgment about when to use them.

REA lists a lot of targets: native binaries, .NET, APKs, firmware, websites, process capture. Native work needs Hopper, Ghidra or IDA, and I had none of the three on this Linux box (the Hopper features are Mac-only). So I stuck to the one target that needs only Node: JavaScript and Electron apps, analyzed statically.

The real target was PI-Desktop v0.16.1, the Electron app from my previous article. Its app.asar is 100 MB and holds 459 relevant files. I also built a five-file toy Electron app, because I can't grade a tool on an app whose source I haven't read.

Install: fast, with one Node catch

REA wants Node 22.19+, 24.11+ or 26+. The box shipped Node 20, so I grabbed 22.20 from nodejs.org. The box has no xz, so I unpacked the tarball with Python. Then:

npm install rea-agents@latest
# added 144 packages, and audited 145 packages in 3s   (node_modules: 96 MB)
npx rea --version
# 6.4.0

The version matters. npm lists 31 versions since the package appeared in July, and GitHub shows releases 6.1, 6.2 and 6.3 all on 9 October. 6.4.0 landed on 11 October, the day I tested. The README itself says it "changes quickly", so pin a version.

Then rea doctor:

healthy: false
  node     ok    22.20.0
  host     FAIL  unsupported_host   debian 13
  hopper   FAIL  missing_analysis_engine
  ghidra   FAIL  GHIDRA_INSTALL_DIR is not set

The supported hosts are macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara, Arch, CachyOS and Omarchy, so Debian 13 gets a red mark. It didn't stop anything I ran. Static JavaScript analysis worked fine on the "unsupported" host. I also ran it once under Node 20, where doctor flagged the version, and the analysis still ran. I wouldn't rely on that, but it's good to know the check is stricter than the code.

I didn't want setup touching the box's real config, so I ran it against a throwaway home directory:

HOME=/tmp/fakehome npx rea setup --client claude_code --dry-run   # prints the plan
HOME=/tmp/fakehome npx rea setup --client claude_code --yes
# appliedActions[2]: configured_claude_code, installed_skill

The dry run listed exactly two changes before anything was written. The result was a small mcpServers.rea entry (npx -y rea-agents@6.4.0 mcp) and a skill folder with a SKILL.md and references. Small, clear and reversible. That's how I like setup to behave.

The scan: 52 seconds, 3.1 million AST nodes

time npx rea analyze-javascript-application app.asar --format json > out.json
# real 0m52.2s     out.json: 59.2 MB

REA read 459 files (24.2 MB of text), parsed 421 JavaScript files, walked 3,114,427 AST nodes and had zero parse failures. For a 100 MB bundle, 52 seconds is fine.

Then I checked its Electron summary against the unpacked source, using plain grep:

Here is where it matched:

Browser windows: REA said 5. There are five new BrowserWindow( calls (four in one chunk, one in index.js).
contextBridge APIs: REA said 2. The preload scripts call exposeInMainWorld for piDesktop and pluginBridge.
IPC: REA said 4 main-process handlers, 6 renderer transmissions and 5 renderer listeners. My grep counts for ipcMain.handle/on, ipcRenderer.invoke/send/sendSync and ipcRenderer.on come to the same numbers: 4, 6 and 5.
Sender checks: it reported 4 places where the main process looks at the sender of an IPC message. I didn't verify each of those by hand, so I'm only passing the number along.

And here is what it got wrong:

Preload entrypoints: REA said 0. The app has two preload files, preload/index.cjs and preload/plugin-panel.js. The windows load them through code like join(getModuleDirectory(import.meta.url), "../preload/index.cjs"). My guess is that this helper call is the reason REA couldn't resolve the path statically, but I haven't confirmed that.
Exposed API members: REA said 0. Both bridges obviously expose members, so this zero follows from the missing preload link.

REA is also candid about what it dropped. The result carries 744 limitations. For 365 of them, the text says: "Expanded application relationships for [file] exceed the parent's result budget; complete static facts were retained on the file node." In plain terms, on a big app it keeps counts and per-file facts but throws away some of the connections between them.

That cost me the most useful feature on the real app. I asked it to trace anything touching the channel pi-plugin-panel:

seed_matches: []
coverage.status: no-match
"No graph entity matched the literal seed; this is not evidence that the feature is absent."

The string pi-plugin-panel-drop is right there in the source. The graph just didn't have channel nodes for it. The tool says so, which I respect, but you'd still be stuck.

The toy app: where it shines

So I wrote a five-file Electron app (main.js, preload.js, index.html, renderer.js, package.json) with known contents:

one window with a preload script
one contextBridge API with four members
three literal IPC channels: notes:read, notes:write, open-external
one loop that registers job:sync and job:export from a variable, and a preload call that builds "job:" + topic
two handlers that check the sender, through a small helper function

I analyzed it as written, then again after esbuild --minify, which squashes main.js into one 732-byte line with one-letter variable names.

On both versions, REA's counts were identical, and nearly all of them were right: 1 window, 1 preload, 1 bridge API with 4 members, 3 literal channels, 2 dynamic channel operations, 4 main handlers, 4 renderer transmissions, 3 of them paired with their handlers (the fourth is the dynamic one). Each run took about two seconds.

The tracer is the highlight. I fed the evidence back in with trace-application-feature and a seed of channel: "notes:read". It returned the chain I wanted, on the minified bundle: preload.js invokes the channel notes:read, and the channel is handled by the notes:read handler. It even kept the dynamic channel honest. In the minified code the label is "job:"+e, the actual expression with the renamed variable, rather than a made-up string.

The one miss on the toy: sender validation. I wrote two handlers that call a checkSender(e) helper, which looks at e.senderFrame.url. REA reported 0 sender-validation observations. That's the kind of check you'd want a security-minded tool to notice. A check hidden behind a helper function got past it.

The part nobody puts in the README: what it costs an agent

REA exposes 170 MCP tools. I opened a stdio session, sent initialize and tools/list, and measured the reply:

init: 1.4 s    serverInfo: {"name":"rea","version":"6.4.0"}
tools/list: 170 tools in 0.3 s
payload: 1,954,704 bytes

That's about 1.95 MB of JSON before the agent does anything. By the usual rule of thumb of four characters per token, that's roughly half a million tokens, though that's my estimate and clients may defer or trim tool lists. For scale, I run the local model I use for tests with a 16k context. The project's own docs mention a related problem: GitHub Copilot CLI with gpt-4.1 can block chat with REA's full schema profile, and they suggest a "compact" profile. I tried it with REA_MCP_INPUT_SCHEMA_PROFILE=compact. The list shrank to 1,772,775 bytes, which is 9% smaller. Helpful, not a cure.

This is why I didn't try to drive it with a small local model. The tool list alone wouldn't fit in the context window, so any result would say more about my setup than about REA.

The tool call itself is heavy too. I called analyze_javascript_application on the same app.asar over MCP with detail: "summary":

call: 60.5 s    response: 1,350,868 bytes (text part: 672 KB)

A "summary" that is 672 KB of text is a lot for an agent to read. If you use REA through an agent, I'd expect the agent to spend a lot of its budget on REA's output. I'd test it with your exact client before you commit.

Native binaries: stopped at the door

I did try the native path, to see how it fails. Pointing rea analyze at PI-Desktop's 20 MB Rust host binary:

code: provider_unavailable
candidate_ids: ghidra, hopper, ida

That's a clear message with a clear fix, and I'd have been disappointed by anything vaguer. rea inspect-artifact on the same file still worked without a provider. In 1.7 seconds it returned the format (ELF) and a SHA-256 that matches what sha256sum printed, plus a suggested next step: open_binary, which needs one of the three engines. So you get identification, not understanding, until you install a disassembler. I didn't, so I can't tell you how good the native side is.

What worked and what didn't

What worked:

Install in 3 seconds, and a setup that shows its plan before writing anything
Static analysis of a 100 MB Electron app in about 52 seconds, with zero parse failures
Counts for windows, bridges and IPC that matched my grep on the real app
Identical, correct results on plain and minified code in my toy app
An honest evidence trail: limitations listed, "no match" is not claimed as "absent"

What didn't:

Missed both preload files on the real app, which zeroed out the exposed-API count
Dropped connections on 365 files because of its own result budget, so the trace on a real channel found nothing
Missed a sender check hidden behind a helper function
A 1.95 MB tool list and a 672 KB "summary", which an agent has to carry
doctor goes red on Debian 13 even though the JavaScript path ran fine

What I didn't test: Hopper, Ghidra or IDA, websites and browser capture, APKs, firmware, runtime capture, and above all an agent actually using the tools to "build a similar feature", which is the headline pitch. That claim needs a frontier model and a real session, and I didn't have one.

Should you try it?

If you work with Electron or bundled JavaScript, yes. The pairing of fast static analysis and a graph you can trace is useful, and the toy-app result on minified code impressed me more than any star count. Run it on an app you already understand first, the way I did, so you learn where it's blind before you trust it on one you don't.

If you plan to hand it to an agent, go in with care. Count the tokens in the tool list, try the compact profile, and keep an eye on how much an analysis result takes from the context window.

As for the trending page: stars tell you people are curious, not whether the tool matches a given job. I couldn't check the claim that it jumped from about 9k stars in four days, because GitHub only gives star timestamps to signed-in API calls. What I could check is that on Sunday morning it was real, MIT-licensed, released four times in three days (6.1 to 6.4), and good at most of the job I gave it. It also missed two files that grep found in a second.

I'd like to see this on a Mac with Hopper and a frontier model driving it. If you've run REA that way, tell me what held up.

I Ran PI-Desktop's Plan, Goal and Subagent Modes on a 1.7B Local Model. Here's What Broke.

Ishank Choudhary — Sat, 10 Oct 2026 04:59:13 +0000

Pi, the open-source coding agent engine, reached 1.0 on 1 October. Three days later, on 4 October, PI-Desktop shipped v0.16.1, the first release built on Pi 1.0.0. PI-Desktop is a desktop GUI for that engine: Electron on the front, a Rust "host core" underneath, and the Pi agent packages running in a Node sidecar.

One thing to clear up first. PI-Desktop is a community project by vastsa, not something from the Pi team. It's LGPL-3.0, has about 6.4k GitHub stars, and its README calls the 0.16.x line an "Early Preview". Several other projects use the "pi-desktop" name, so if you go looking, start from the repo and the official docs.

I didn't want to write a feature list. I wanted to know what actually happens when you install it on a plain Linux box and run the three things that set it apart from a terminal agent: Plan mode, Goal mode and subagents.

The honest caveat up front

There was no cloud API key on my test machine. So I ran everything against qwen3:1.7b through Ollama, on CPU only: Debian 13, 8 vCPUs, no GPU. That model managed about 250 tokens/s on prompt processing and about 8 tokens/s on generation.

That is a stress test, not a fair benchmark. A 1.7B model is the weakest setup the app would ever see. Some of what broke is the model's fault, and I'll say which parts. Other things broke regardless of the model, and those are the more useful findings.

The test project was lukeed/clsx v2.1.1: 21 files and 32 passing tests. Small enough for a tiny model to have a chance.

Install: fast and uneventful

Linux gets an AppImage, a .deb and an .rpm (glibc 2.35+, so Ubuntu 22.04+, Debian 12+ or Fedora 36+). I used the AppImage, extracted it instead of mounting it so libfuse2 wasn't needed:

wget https://github.com/vastsa/PI-Desktop/releases/download/v0.16.1/PI-Desktop-0.16.1-linux-x86_64.AppImage
chmod +x PI-Desktop-0.16.1-linux-x86_64.AppImage
./PI-Desktop-0.16.1-linux-x86_64.AppImage --appimage-extract
./squashfs-root/AppRun --no-sandbox

What I measured:

Download: 164.8 MB, and the SHA-512 matched latest-linux.yml.
Extract: about 3 seconds, 414 MB on disk. No missing shared libraries according to ldd.
First launch: the window appeared in 1.56 s cold and 0.54 s warm.
Memory: about 630 MB RSS across the Electron, host-core and sidecar processes, before the agent did anything.
--no-sandbox was needed only because the extracted tree's chrome-sandbox isn't setuid. The installed packages shouldn't need it.

Inside the bundle you get Chromium 150, the Rust pi-desktop-host-core binary, the Pi sidecar, a bundled models.dev catalog of 8,344 models, and two built-in plugins (a browser and a file manager).

The first screen is a calm "What can I help you build?" with a Get started checklist. The composer has three chips that matter for this test: the mode (Agent / Plan / Goal), the permission (Ask every time / accept edits / Auto) and the model.

Hooking up a local model

There's no dedicated Ollama tile in 0.16.1. Local models go in through Settings → Agent → Models → Add provider → Custom endpoint, which takes "any OpenAI- or Anthropic-compatible URL".

Base URL:   http://127.0.0.1:11434/v1
API key:    ollama        (dummy)
API format: OpenAI Chat Completions

Fetch list found the model and the provider showed "Connected · 1 model found". The provider list is long, too: subscription sign-ins (Claude Pro/Max, ChatGPT, GitHub Copilot, SuperGrok/X Premium and others) plus around 30 API-key providers.

A smoke test ("Reply with exactly: PI-Desktop smoke test OK") worked, but it took 1 minute 54 seconds, and it showed me the first real number:

The app's system prompt plus tool schemas came to about 4.1k tokens (Ollama cached 4,098 of 4,717 prompt tokens).
The context ring in the composer read 96% after one turn, because I'd set Ollama to a 16k context and the custom model's context window showed as "— —" in the UI.
The reasoning setting appended /think to my prompt (Qwen's reasoning switch), and the model echoed it back in its answer.

Lesson one: when you add a local model, open the model's Advanced settings and set the Context Window yourself. With 16k of context, the harness takes a quarter of it before you type anything.

Plan mode: it worked, but the plan was fiction

I opened the clsx folder as a project, set the chip to Plan with "Ask every time", and asked it to plan a test for nested arrays and falsy values.

The mechanics were solid:

The plan was saved as a real file, .pi/plan/add-nested-array-falsy-values-tests-for-clsx-20261007-1108.md (315 bytes).
A plan card appeared with Open plan, Reject and Approve (Ask), and the File Manager pane opened showing the markdown.
The chat got an automatic, sensible title.

The content was not. The plan said to add test files in /work/clsx/tests, a path that doesn't exist. The run log showed "3 issues · 4 tools": a search failed, a read failed, the first SubmitPlan failed and the second one succeeded. The app reported "Processed for 5m 19s", but wall-clock time from sending the prompt to seeing the card was about 17 minutes. The context meter sat at 95%.

That's mostly the model. A 1.7B model with a quarter of its context taken by instructions invents paths. But the workflow around it (a separate artifact, an explicit approval gate) is exactly what I'd want to put in front of a stronger model.

Goal mode: the permission chip changed on its own

Goal mode is meant to write a .pi/goal/*.md file with an outcome and acceptance criteria, wait for approval, then switch to Agent and work until it can report which criteria it verified.

I clicked the mode chip twice to get to Goal. Then I noticed the permission chip next to it had silently switched from "Ask every time" to "Auto". I hadn't touched it. For a mode whose whole job is to run autonomously toward a target, that's the one setting I'd expect it never to loosen on its own.

The run didn't go well:

The first model call took 4 min 26 s.
No goal file and no acceptance criteria were ever written.
The model went looking in /work/tests again.

The safety net did work: even under Auto, a Grep on /work/tests stopped with "Accesses a path outside the session workspace", rated LOW RISK. I denied it. The model then tried to Edit /work/tests/new-test.js, which failed. I stopped it at about 12 minutes with nothing verified, and the chat had been auto-renamed in Chinese ("新增嵌套假值测试用例", roughly "add nested falsy value test cases").

The failed run is the model's fault. The permission flip and the Chinese title happen in the app, and they're worth reporting to the maintainers. Fairly, the docs do say Plan and Goal are "contract modes, not strict read-only security profiles". I'd still check that chip every time you switch modes.

Subagents: the one clean win

PI-Desktop ships five built-in subagents, each with a defined tool set:

Explorer: Read, Glob, Grep, Bash
Code reviewer: Read, Glob, Grep (read-only)
Test runner: Read, Glob, Grep, Bash
Fixer: adds Edit and Write
UI designer: adds BrowserPreview, Edit and Write

You can add your own as markdown files in ~/.agents/subagents/ (up to 16). Delegation uses a Task tool that runs in the background and returns a delegation ID, with TaskWait, TaskList and TaskStop alongside it, and up to 10 at once per session. Subagents only work in Agent mode.

In Agent mode I asked: "Use the test-runner subagent to run npm test and report the result."

This went well:

The UI drew a small delegation graph: Main agent → test-runner, "Coordinating 1 delegated task".
The permission prompt said clearly that it was "Asked by the test-runner subagent", showed the exact command (npm test) and labelled it HIGH RISK. Running a shell command is the right thing to flag.
After Allow once, the subagent finished in 2 min 44 s (3 steps). The whole turn took 5 min 55 s.
The result was exit 0, 32/32 passed, which matches what I got running npm test in a shell myself.

The subagent's own transcript opened in a side pane marked read-only ("Subagents are driven by the main agent"). And the chat was renamed again, to "npm测试成功" ("npm test succeeded").

The lesson: give a small model a narrow, well-defined job and the harness carries it. Ask it to plan open-ended work and it falls apart.

Two findings that have nothing to do with the model

1. "OS keychain" isn't quite true yet. The landing page and the README say credentials live in the OS keychain. I ran gnome-keyring with Secret Service available, saved the Ollama provider, and zero items went into the keyring. Instead, two files appeared:

~/.pi-desktop/secrets/<sha>.bin    48 B   mode 644
~/.pi-desktop/secrets/.machine-key 32 B   mode 600

The project's own storage spec explains it: the shipped backend is an AES-256-GCM file store keyed by a machine key that "host-core generates once and keeps beside them", and the OS keychain backend is one "that neither host-core nor Electron main implements today, so a same-user process that can read the data directory can also decrypt the secrets." The spec is honest about this. The marketing isn't. If you put real cloud keys in it, treat the ~/.pi-desktop folder as sensitive.

2. The window only redraws when the mouse moves. On my Xvfb/xfwm4 desktop, timers and streaming output froze until I moved the cursor. That could be specific to my headless setup, but it made the long local runs feel even longer.

The rest, briefly

MCP Market: Settings → Agent → MCP has a Market tab that merges a built-in catalog with the official MCP registry. It listed Memory, Sequential Thinking, Filesystem, Playwright, Context7, Fetch, Git, Time, Serena and others, most marked Verified, each with its npx/uvx command. Some descriptions showed up in Chinese. As of v0.16.1, tools from user-added MCP servers need approval.
Session import: it can import sessions from Claude Code, Codex, OpenCode and Pi.
Scheduled tasks: hourly, daily or weekly, but they only run while the app is open.
No telemetry and no required account, according to the README.

What a bigger cloud model would likely change

I didn't test this, so treat it as an expectation, not a result. With a capable cloud model and a large context window, I'd expect:

The 4.1k-token system prompt to stop mattering. At 128k+ context it's a rounding error, not 25%.
Plan mode to produce real paths, since a stronger model would actually read the repo before planning.
Goal mode to write its acceptance criteria and run through them in minutes, not stall after 12.
Total time per turn to fall from minutes to seconds.

What I don't expect to change: the permission chip flipping to Auto, the keychain wording, the Chinese auto-titles and the redraw bug. Those are in the app.

Verdict

PI-Desktop v0.16.1 installs in seconds, starts in under two, and its Plan, Goal and subagent workflows are thought through: separate artifacts, approval gates, clear risk labels, a visible delegation graph. Subagents did their job even on a 1.7B model on CPU.

It's also clearly an early preview. Check the permission chip when you switch to Goal, don't trust the "OS keychain" line yet, and set your local model's context window by hand.

If you run it with a frontier model, I'd like to know whether Goal mode holds up. That's the test I want to see next.

Originally published on Medium.

I Pointed Matryoshka's MCP Server at War and Peace and a 10k-Line Log, With No LLM Key

Ishank Choudhary — Wed, 07 Oct 2026 04:54:30 +0000

When a coding agent needs something from a file that is too big for its context window, it usually does one of two things. It reads the file in slices and hopes the answer is in one of them, or someone has already built a RAG index with chunks and embeddings. Both lose information at the seams.

Matryoshka takes a third route. It is an open-source TypeScript project by Dmitri Sotnikov (yogthos, the author of the Luminus Clojure framework). It keeps the whole document on the server and gives the agent a small query language to work on it. Results stay server-side, and the agent gets back short "handles" instead of the raw text. The README says it is based on the Recursive Language Models paper by Alex L. Zhang, Tim Kraska and Omar Khattab of MIT CSAIL. Matryoshka is an independent implementation of that idea, not the authors' own code.

When I checked on 4 October 2026, the repo had 149 stars, 20 forks and no open issues. It is Apache-2.0 licensed, and the latest npm release was matryoshka-rlm 0.2.40 from 17 May 2026. It's a small project, so I wanted to see how it holds up on real files.

One constraint shaped the whole test: there was no LLM API key on my machine and no local Ollama. Matryoshka ships an MCP server, lattice-mcp, that needs no LLM at all. So I played the agent myself and wrote every query by hand. This tests the engine and the tool interface. It says nothing about how well a model would drive them.

How it works, in one paragraph

The model doesn't write JavaScript or Python. It writes commands in Nucleus, a small S-expression language: (grep "ERROR"), (filter RESULTS ...), (count RESULTS), (sum RESULTS). The Lattice engine parses and type-checks each command and runs it against the document. Results go into an in-memory SQLite store, and the client gets a stub like $grep_error: Array(1000) [preview...]. It only pulls real lines with lattice_expand when it needs them. The README claims this gives "97%+ token savings", and "80%+ token savings compared to reading files directly" for the MCP server. Those are the project's numbers, and I didn't benchmark them.

Setup

The box was Linux x86_64 with 8 vCPUs, about 15 GB of RAM, Node 20.19.2 and pnpm 10.33.4. The README suggests pnpm add -g matryoshka-rlm. I installed it locally in a small Node/TS project instead, next to the official MCP SDK:

pnpm add matryoshka-rlm@0.2.40 @modelcontextprotocol/sdk tsx typescript @types/node

It finished in 2.5 seconds and took 220 MB of node_modules. pnpm 10 skipped the native build scripts for the tree-sitter grammars. I left them unbuilt, and symbol listing still worked.

Gotcha 1: don't run plain npx lattice-mcp. From a folder where Matryoshka isn't installed, npx lattice-mcp downloaded an unrelated npm package called lattice-mcp (1.6.2), which asked for LATTICE_API_URL and LATTICE_API_TOKEN. Use the binary that comes with matryoshka-rlm (node_modules/.bin/lattice-mcp, or npx -p matryoshka-rlm lattice-mcp). The real one prints lattice-mcp v0.2.40.

The client is a few lines of TypeScript with the MCP SDK's stdio transport:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";

const transport = new StdioClientTransport({
  command: "/workspace/matryoshka-test/client/node_modules/.bin/lattice-mcp",
  cwd: "/workspace/matryoshka-test", // lattice_load only accepts files under this folder
});
const client = new Client({ name: "ishank-test", version: "0.0.1" });
await client.connect(transport);

await client.callTool({ name: "lattice_load", arguments: { filePath: "docs/war-and-peace.txt" } });
const r = await client.callTool({ name: "lattice_query", arguments: { command: '(grep "NatÃ¡sha")' } });

It connected in 409 ms and reported the server as lattice 0.2.40, with 12 tools: lattice_load, lattice_query, lattice_expand, lattice_close, lattice_status, lattice_bindings, lattice_reset, lattice_memo, lattice_memo_delete, lattice_help, lattice_llm_respond and lattice_llm_batch_respond.

The test documents

War and Peace from Project Gutenberg (Maude translation): 3.36 MB, 66,041 lines, 566,333 words, about 770,000 tokens (769,825 with tiktoken's o200k_base). That is far more than most models' context windows.
A real Apache access log from Elastic's examples repo: 10,000 lines (2.37 MB) of traffic from May 2015.
Matryoshka's own source files, for the code features.

Before asking Matryoshka anything, I worked out the true answers separately with grep and a short Python script.

War and Peace: facts, counts and quotes

Loading the novel took about 0.75â€“0.8 seconds. Every query after that took between 1 and 430 ms.

What I asked	Nucleus command	Matryoshka	Ground truth
Chapter headings	`(grep "^CHAPTER [IVXLC]+$")` â†’ `(count RESULTS)`	365	365 âœ…
Book headings	`(grep "^BOOK [A-Z]+")` â†’ count	17	15 âŒ
Mentions of NatÃ¡sha	`(grep "NatÃ¡sha")` â†’ count	1,213	1,213 matches âœ… (on 1,197 lines)
Pierre lines that also mention NatÃ¡sha	`(filter RESULTS (lambda x (match x "NatÃ¡sha" 0)))`	44	44 âœ…
Where is "Genoa"?	`(grep "Genoa")` â†’ expand	lines 840, 1573, 9153	same âœ…
Quote the opening	`(lines 840 846)` â†’ expand	exact 7-line "Well, Prince, so Genoa and Luccaâ€¦"	âœ…
Mentions of BorodinÃ³	`(grep "BorodinÃ³")` â†’ count	108	108 âœ…

Gotcha 2: grep is case-insensitive and counts matches, not lines. The 17 books threw me until I expanded the handle. Two of the hits were ordinary sentences starting with a lowercase "book" ("book from the high desk."). The source confirms that grep runs with the gmi flags. It also returns one item per match, so 1,213 is the number of times "NatÃ¡sha" appears, not the 1,197 lines she appears on. Once I knew that, every count matched grep -oi exactly. A tighter pattern, (grep "^BOOK [A-Z]+: "), returned the correct 15.

The ranked searches showed where the "no RAG" approach has its limits:

(bm25 "battle of Borodino" 5) returned five lines that just said "battle." The translation spells it BorodinÃ³. With the accent, (bm25 "BorodinÃ³ battle" 5) returned five lines that all contain "battle of BorodinÃ³".
(fuzzy_search "Natasha Rostova" 3) found nothing. (fuzzy_search "NatÃ¡sha RostÃ³va" 3) found her straight away.
(semantic "wounded at Austerlitz looking at the sky" 5) didn't return the famous lines where Prince Andrew lies wounded under "the lofty sky". Its closest hit was "been looking at." on line 15862, seven lines before. Plain (grep "lofty sky") found the passage at line 15869. The README describes semantic as TF-IDF cosine similarity, not embeddings, so this isn't surprising.

Gotcha 3: spell it the way the document does. Lexical ranking does not fold accents, so with translated or non-English text an agent should check the document's spelling before trusting a ranked search.

Gotcha 4: lattice_expand needs a named handle. RESULTS works inside queries, but lattice_expand with RESULTS returns "Invalid handle: RESULTS". lattice_bindings lists the real names, such as $grep_genoa. Patterns full of quotes get generic names ($grep, $grep_2), so lattice_bindings is worth calling often.

Two smaller things worked as documented. Loading /etc/hostname was refused with "Path outside working directory is not allowed." And a memo I saved with lattice_memo while the novel was loaded could still be expanded after I switched to the log file.

How much text actually reached my "agent"? My first session made 27 tool calls against the 3.36 MB novel, and the tool responses added up to 5,625 characters. That's my own count, not a benchmark. A real agent wouldn't read the whole book either, but it shows the design: the document stays on the server and only stubs and the lines you ask for come back.

The access log: counting and summing

What I asked	Matryoshka	Ground truth
404 responses, `(grep "\" 404 ")` â†’ count	213	213 âœ…
Googlebot requests	543	543 âœ…
200 responses	9,126	9,126 âœ…
Bytes served with 200, `(sum RESULTS)` on the grep result	1,067,060.464	âŒ meaningless
Same, after `(map RESULTS (lambda x (match x "\" 200 (\\d+)" 1)))`	2,735,455,845	2,735,455,845 âœ…

Gotcha 5: sum on raw lines adds up the wrong number. When you call sum on grep results, it takes the first number on each line. In an access log, that's the start of the client IP (83.149â€¦), so you get a confident, meaningless total. Pull out the field first with map and match, and sum returns the exact byte count.

I also tried the sub-LLM path without an LLM. My client doesn't support MCP sampling, so (llm_batch RESULTS (lambda x (llm_query "Is this request from a bot or a human? {item}" (item x) (one_of "bot" "human")))) over the three 500-error lines returned a single [LLM_BATCH_REQUEST id=â€¦ count=3] message. It held three ready-made prompts, each with its log line and an "answer with exactly one" instruction. I typed the answers myself (["bot","bot","human"]) and sent them back with lattice_llm_batch_respond. Matryoshka stored them as a new handle, and filtering on "bot" counted 2. So the protocol works with any MCP client. To be clear, no model judged those lines. I did.

Code files: symbols yes, graph no

On Matryoshka's own lc-solver.ts (2,658 lines), (list_symbols "function") returned 20 functions, the same count grep gives for top-level function declarations. Each came with its line range and signature. (get_symbol_body "evaluate") returned the function's source, but inline and 64,311 characters long. Unlike search results, it isn't returned as a handle, so a big function goes straight into the agent's context.

The knowledge graph didn't work for me. (callers ...) and (god_nodes ...) returned "No symbol graph available" on all four source files I tried. Each time the server's stderr showed Graph.addEdge: source & target are the same (...), with names like evaluate, init and run. The graph library rejects self-loops, so a recursive function or a same-name call seems to stop the graph from being built. That's on 0.2.40 with these four files. I can't say how widespread it is.

The REPL and the full RLM CLI

lattice-repl docs/war-and-peace.txt is useful for trying commands out: (grep "Austerlitz") then (count RESULTS) gave 51, matching grep -oi. Piping :load and a query in together failed, because the load hadn't finished when the query arrived. Passing the file as an argument worked.

The rlm CLI is the part where the LLM writes the Nucleus itself. With no config, it defaults to Ollama on localhost, retried three times, and stopped after about 9 seconds with [aborted: llm 3 consecutive LLM call failures: fetch failed]. That's expected. It needs a model.

What I didn't test

A real agent choosing Nucleus commands. That is the main promise, and I wrote every command by hand.
rlm / rlm-mcp with a model, recursive rlm_query / rlm_batch, MCP sampling, compaction and resource limits.
lattice-setup (Claude Code wiring), the HTTP and pipe adapters, program synthesis, fuse / dampen / rerank, and multi-document loading.
Any of the README's token-savings figures or the paper's benchmark results.

Takeaways

The engine was exact on literal tasks. Every count, lookup and quote matched ground truth once I understood the semantics. Most of the work happens on the server, and only stubs come back.
The semantics matter more than the syntax. Case-insensitive, per-match grep and first-number sum are reasonable defaults, but an agent that doesn't know them will report 17 books and a nonsense byte total with full confidence.
The ranked searches are lexical. BM25, fuzzy and "semantic" (TF-IDF) depend on exact spelling. Don't expect embedding-style recall.
The code graph is the weak spot in 0.2.40. Symbol listing worked, but the call graph failed on every file I tried.
It's worth a try if your agent works over big logs, transcripts or books and you want answers you can check, not chunk retrieval. Next I want to put a real model in the loop and see whether it avoids these gotchas on its own.

If you've wired lattice-mcp into Claude Code or another agent, I'd like to hear whether your model figured out the case-insensitive grep by itself.

Originally published on Medium.

I Gave Two TypeScript Agent Sessions One Shared Memory With Hindsight (No LLM Key)

Ishank Choudhary — Mon, 05 Oct 2026 04:50:59 +0000

Most of my coding-agent sessions start the same way. The agent rediscovers that the repo uses pnpm, gets told again not to touch the legacy payment file, and trips over the test that only passes in UTC. None of that is in the code. It lives in someone's head, or in a previous session that is gone.

Hindsight by Vectorize is built for that gap. It is an open-source agent memory server (MIT) that agents write to and read from across sessions. When I checked on 3 October 2026 its GitHub repo had 44,886 stars, sat second on GitHub's weekly trending page with 16,183 stars that week, and its latest release was v0.10.2 from 29 September.

I wanted to see what it does in practice, so I installed it on a Linux box and called it from a small Node/TypeScript project. One constraint shaped the whole test: I didn't have an LLM API key on that machine. Hindsight documents a mode for exactly that, so this is a test of its no-LLM floor, not of everything it can do. I've marked clearly which parts I ran and which parts I only read about.

The setup

The box was Linux x86_64 with 8 vCPUs, about 15 GB of RAM and Node 20. There was no Docker, so I used the bare-metal path from the README (pip install hindsight-api), through uv:

uv venv .venv-hs --python 3.12
uv pip install hindsight-api==0.10.2

It resolved 227 packages and finished in 11.6 seconds, including the PyTorch download. The catch is size: the virtual environment came to 6.3 GB, mostly torch and CUDA wheels. Plan disk space accordingly.

The configuration docs describe HINDSIGHT_API_LLM_PROVIDER=none as "chunk storage + semantic search only, no API key needed." In this mode retain stores your text as chunks without LLM fact extraction, recall still works, reflect returns HTTP 400, and observations are switched off.

export HINDSIGHT_API_LLM_PROVIDER=none
hindsight-api

The startup banner showed v0.10.2 with an embedded PostgreSQL (pg0, started automatically), LLM none / none, local embeddings (BAAI/bge-small-en-v1.5), a local reranker (cross-encoder/ms-marco-MiniLM-L-6-v2), and MCP enabled at /mcp.

The first start took about 38 seconds to reach "Uvicorn running", including model downloads from Hugging Face.
A warm restart took about 19 seconds to a healthy /health.
The worker process used roughly 1.35 GB of resident memory.

Session 1: one agent writes down what it learned

For the Node/TypeScript side I installed the official client (version 0.10.2) and ran the scripts with tsx:

npm install @vectorize-io/hindsight-client typescript tsx @types/node

The scenario was a fictional shop-api repo. The first script plays "Agent A" at the end of a session. It creates a bank, which is Hindsight's isolated memory store, and retains five project facts that aren't obvious from the code.

// src/agentA.ts
import { HindsightClient } from '@vectorize-io/hindsight-client';

const BANK = 'shop-api-repo';
const client = new HindsightClient({ baseUrl: process.env.HINDSIGHT_URL ?? 'http://localhost:8888' });

const learnings = [
  { content: 'The shop-api repo uses pnpm, not npm. Run `pnpm test` with Vitest; Jest was removed in March.', context: 'tooling' },
  { content: 'All money values are stored as integer cents in Postgres (column type bigint). Never use floats for prices.', context: 'data model convention' },
  { content: 'Route handlers live in src/routes/*.ts and must validate request bodies with zod schemas from src/schemas.', context: 'code convention' },
  { content: 'The flaky test checkout.spec.ts fails when TZ is not UTC; CI sets TZ=UTC, so set it locally too.', context: 'debugging note' },
  { content: 'Do not touch src/legacy/paypal.ts; it is scheduled for deletion and has no tests.', context: 'team decision' },
];

async function main() {
  await client.createBank(BANK, {
    reflectMission: 'Long-term memory for coding agents working on the shop-api TypeScript repo.',
  });
  for (const l of learnings) {
    const t0 = performance.now();
    await client.retain(BANK, l.content, { context: l.context, tags: ['repo:shop-api'] });
    console.log(`retain: ${(performance.now() - t0).toFixed(0)} ms`);
  }
}
main();

The five retains took 184, 42, 46, 56 and 41 ms. Every response reported zero LLM tokens, which is expected with no model. listMemories showed five memories, all with fact_type: "world". In chunks mode that means the text is stored as written: no extracted entities and no restructuring.

Session 2: a fresh process asks what it should know

The second script is a separate process with a new client and no shared state apart from the bank name. It asks four questions an agent would plausibly ask on day one, then tries reflect.

// src/agentB.ts
import { HindsightClient } from '@vectorize-io/hindsight-client';

const BANK = 'shop-api-repo';
const client = new HindsightClient({ baseUrl: process.env.HINDSIGHT_URL ?? 'http://localhost:8888' });

const questions = [
  'How do I run the tests in this repo?',
  'How should I store a product price?',
  'Why does checkout.spec.ts fail on my machine?',
  'Can I refactor the PayPal integration?',
];

async function main() {
  for (const q of questions) {
    const t0 = performance.now();
    const r: any = await client.recall(BANK, q, { tags: ['repo:shop-api'], maxTokens: 512 });
    console.log(`Q: ${q} (${(performance.now() - t0).toFixed(0)} ms)`);
    for (const m of r.results.slice(0, 2)) console.log('  ->', m.text);
  }
  try {
    await client.reflect(BANK, 'What should a new agent know before editing this repo?');
  } catch (e: any) {
    console.log('reflect error:', e.message);
  }
}
main();

Here is what came back:

Query	Latency	Top result
How do I run the tests in this repo?	227 ms	the pnpm / Vitest note ✅
How should I store a product price?	150 ms	the integer-cents note ✅
Why does checkout.spec.ts fail on my machine?	62 ms	the TZ=UTC note ✅
Can I refactor the PayPal integration?	68 ms	the legacy/paypal.ts note ✅

The right memory came first on all four questions. Two caveats keep that in proportion. The bank had five entries, so every query returned all five, and what I was really seeing was ranking, not filtering. The second result was sometimes unrelated: the price question had the TZ note in second place. That's fine for a toy, but on a real bank you would want the maxTokens budget or the minScores floors the client exposes.

reflect failed exactly as documented:

Reflect requires an LLM provider. Current provider is set to 'none'.
Set HINDSIGHT_API_LLM_PROVIDER to a real provider (e.g., openai, anthropic, gemini).

Next I stopped the server, started it again and re-ran Agent B. The top results didn't change (201 ms and 91 ms on the first two queries), so the memories live on disk in the embedded Postgres, not just in the process.

A third client over MCP

Every Hindsight server exposes a Model Context Protocol endpoint per bank at /mcp/{bank_id}/. I talked to it with plain curl. You send an initialize request, read back the mcp-session-id header, and send that header with each later call:

curl -s -D hdr.txt -X POST http://localhost:8888/mcp/shop-api-repo/ \
  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'

SID=$(grep -i mcp-session-id hdr.txt | awk '{print $2}' | tr -d '\r')

curl -s -X POST http://localhost:8888/mcp/shop-api-repo/ \
  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
  -H "mcp-session-id: $SID" \
  -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"recall","arguments":{"query":"which package manager and test runner?"}}}'

The server identified itself as hindsight-mcp-server 0.10.2, and tools/list returned 36 tools. They cover retain and recall, reflect, mental models, directives, memory and document management, operations, tags, bank settings and knowledge-page CRUD. The recall call returned the pnpm/Vitest memory that Agent A had written through the TypeScript SDK. The SDK and MCP both work against the same bank, so a TypeScript service and any MCP-capable agent can share one memory.

What the coding-agent installer actually wires up

Coding agents are the headline use case. Vectorize ships @vectorize-io/hindsight-coding-agents, and the version I tried (0.8.0) lists 20 harnesses in its installer: Claude Code, Codex, Cursor CLI, GitHub Copilot CLI, opencode, Grok Build, Cline CLI, Devin CLI, Antigravity CLI, Qwen Code, Kimi Code, DeepSeek Harness and more.

I did not run a real coding-agent session. The only agent the installer detected on the box was a Cursor CLI install that other tools share, and I didn't want to change its config. I installed the integration into a throwaway home directory instead and inspected what it writes:

HOME=./fakehome npx -y @vectorize-io/hindsight-coding-agents install cursor-cli \
  --server self-hosted --api-url http://localhost:8888

It reported hooks merged into ~/.cursor/hooks.json, an MCP server added to ~/.cursor/mcp.json, and a skill installed under ~/.cursor/skills/hindsight-coding-agent. Here are the hooks it wrote (paths shortened):

{
  "version": 1,
  "hooks": {
    "sessionStart":       [{ "command": "node \".../cursor-sessionstart-hook.js\"", "timeout": 30 }],
    "beforeSubmitPrompt": [{ "command": "node \".../cursor-hook.js\"" }],
    "stop":               [{ "command": "node \".../cursor-stop-hook.js\"", "timeout": 30 }]
  }
}

The MCP entry runs a local stdio server, node ~/.hindsight/coding-agents/dist/mcp-server.js ${workspaceFolder}. I started that server myself with my Node project as the workspace and listed its tools. It has eight: hindsight_sync_status, hindsight_diagnose, hindsight_search_knowledge_pages, hindsight_list_knowledge_pages, hindsight_read_knowledge_page, hindsight_reflect, hindsight_capture_initiative and hindsight_ingest_document.

hindsight_diagnose resolved the project to a bank called coding-agent::node-agents on my local server. hindsight_sync_status reported synced: false, no git log and zero knowledge pages. That fits the docs, which say ingestion of git history and past sessions starts from the session-start hook, and I never fired that hook with a real agent. So I can show you the wiring, but not how it behaves inside an agent.

What I could not test

All of the following needs an LLM, so I can only describe it from Vectorize's docs:

Fact extraction on retain. With a model, retain pulls out facts, entities, relationships and dates instead of storing raw chunks.
Observations. Related facts are consolidated in the background into deduplicated beliefs that keep their supporting evidence.
Reflect. A slower, reasoning pass over the bank, as opposed to a lookup.
Mental models and knowledge pages. Standing answers and wiki-like pages that the bank rewrites as it learns. The coding-agent integration builds on these.
The LLM wrapper (hindsight-litellm), which recalls and retains automatically around each model call.
Benchmarks. The README says Hindsight is state of the art on LongMemEval and that Virginia Tech's Sanghani Center and The Washington Post independently reproduced its results. I didn't benchmark anything, and five memories prove nothing about accuracy at scale.

Should you try it?

Here is what I can say from what I ran. The install is quick but large. The server starts with no external services and no API key. The TypeScript client is simple. Memories persist across restarts. One bank serves both SDK calls and MCP tool calls, and on my small, deliberately easy set of questions recall put the right note first every time.

Without a model, though, Hindsight is a well-packaged local vector-plus-keyword store with an MCP front end. The "learning" in its tagline (extraction, observations, reflect, knowledge pages) is the part that needs an LLM, and that is the part I haven't verified. If you have a key, or a local model through Ollama or LM Studio (both are supported providers), that is where I'd point the next test. If you don't, the none mode is still an honest, quick way to see whether the API and the bank model fit your agent setup before you spend tokens.

Have you put Hindsight or another memory layer behind your coding agents? I'd like to hear in the comments what actually stuck between sessions.

Originally published on Medium.

ChatGPT Dots: What OpenAI's Always-On Agents Mean for How I Work as a Lead SWE

Ishank Choudhary — Wed, 30 Sep 2026 04:04:38 +0000

Have you ever closed your laptop at night knowing that three things will quietly rot while you sleep? A flaky test nobody owns. A customer complaint that will sit in a Slack thread until someone has "a minute". A proposal doc that is already out of date because the requirements changed at 6 PM. As a Lead Software Engineer, most of my job isn't writing code. It's keeping a dozen half-finished threads from falling on the floor.

So when OpenAI published "Introducing dots" on September 29, 2026, I didn't skim it. I read it the way I read an RFC: slowly, looking for the parts that are concrete and the parts that are just vibes. This post is my breakdown. Everything here about the product comes straight from OpenAI's announcement. Where I'm speculating or giving my own opinion, I'll say so clearly.

So What Exactly Are Dots?

In OpenAI's words, dots are "remarkably capable, always-on agents" that live in ChatGPT. The pitch is simple: an assistant that gets to know what matters to you, keeps working on your behalf, and takes work off your plate so you get your time and attention back.

The concrete details are what caught my eye as an engineer:

Model: Dots are powered by GPT-6 Astra.
Compute: Each dot has its own cloud computer and its own browser.
Always on: Dots can work toward your goals 24/7, and they learn from your feedback over time.
Integrations: Through OpenAI's plugin ecosystem, dots can connect to over 4,000 apps.

You start with a single primary dot. You give it a name and make it your own. OpenAI says that over time they envision teams of dots working together for you, but today the starting point is one dot per person.

That's the part that matters to me. This isn't a chat window that forgets you when you close the tab. It's a persistent worker with its own machine.

Dot as a Coworker, Not a Chatbot

The mental model shift here is bigger than it looks. With a normal chat assistant, I'm the scheduler. I decide when it runs, I paste in context, and I babysit every step.

OpenAI describes dots differently. Your dot can take a project and run with it, even while it's working on several others. You can keep handing it new tasks and ideas without juggling separate threads or directing every step. The more you work together, the more it learns your preferences, how you think, and, in OpenAI's phrase, "what good looks like to you."

There's also a proactive side. OpenAI says the magic is when a dot brings you work done the way you would do it, "sometimes before you even think to ask." They share a few internal examples:

A bug shows up in Slack, and dots immediately start investigating.
A new design arrives, and dots turn it into a working app while the team focuses on customer feedback.
A planning cycle starts, and dots keep everyone moving in sync toward the deadline.

My favourite example is the least flashy one. An early tester's dot noticed he had forgotten to invoice a publication. It prepared the invoice and sent it after his approval. That last bit matters, and I'll come back to it.

How It Actually Works Under the Hood

Here's what the announcement says about mechanics. No speculation.

Its own computer. Dots can do nearly anything using their own cloud computer, their own browser, and the apps you've connected. You can open your dot's computer at any time to inspect its work. As someone who has debugged plenty of "the agent said it did it" situations, being able to look at the actual machine is the feature I care about most.

Your devices, if you allow it. Dots can connect to other devices besides their own. You can also give your dot permission to connect to and use your laptop, so it can work directly alongside you. That's opt-in. Your computer stays separate unless you choose to connect it.

Proactive research. When you aren't actively working with it, your dot looks for ways to help in the background. OpenAI calls this "proactive research." The important constraint is that this runs on the apps you've already connected, using tools that are restricted to be read-only. In background mode it can't send messages, change app content, or control your browser or computer.

Where you talk to it. You can message or call your dot in ChatGPT on desktop, web, and mobile. You can also message it in Slack and Teams, and OpenAI says texting is coming soon. Dots carry context across every channel. So I could start something in ChatGPT, drop context in a team Slack channel, and the dot follows through with the full picture. It can also message you with progress, questions, or decisions that need you.

The Scenario That Made Me Sit Up

OpenAI lists several "extension of you" scenarios: a launch lead, a scientist, a sales lead, a content creator. The first one describes my world almost exactly:

You're a developer working on an app. Your dot watches customer feedback for recurring requests, scopes smaller improvements and bugfixes, builds and tests them, and brings you complete PRs to review with attached videos showing the changes.

Read that again as a tech lead. The long tail of small fixes, like the copy tweak, the edge-case crash, or the "can we add a sort option" request, is exactly the work that never makes it into a sprint. It isn't hard. It just never beats the big feature for priority.

If a dot can really turn that backlog into reviewable PRs with videos attached, my role moves further toward reviewer and editor. I'd keep building the next major feature, and the dot would work the queue. The key word in OpenAI's description is review. The dot brings me PRs. It doesn't merge them.

To be clear, I haven't benchmarked this, and OpenAI doesn't publish benchmarks or success rates for it in the announcement. I'm treating it as a stated design goal, not a proven throughput number.

You're Always in Control (and This Is the Part I Read Twice)

Any agent with its own computer and access to 4,000+ apps should make an engineer nervous. OpenAI spends a big chunk of the announcement on safeguards, and they're more specific than I expected:

Isolation by default. Each dot works on its own cloud computer. Your computer and its contents stay separate unless you connect it.
Passwords stay hidden from the model. For signing into supported websites, dots can use saved passwords without exposing them to the model.
Monitoring. Built-in security safeguards help defend against malicious instructions and watch for potentially harmful behaviour. If monitoring detects a safety concern, it can pause or stop the dot's work.
Permissions you already know. You choose which apps your dot can access and manage those permissions through your existing ChatGPT app controls.
Custom Rules. Dots start with built-in rules for when to act on their own and when to ask for approval. Custom Rules let you allow specific actions, require approval, or block them. Built-in safety requirements always apply.
Activity View. You can follow progress, including background work, and redirect your dot as needed.
Auto-review. Actions that could affect your accounts or share information are checked against your instructions, Custom Rules, and safety requirements. This decides what can proceed, what needs your approval, and what you have to do yourself.
Some things never leave your hands. Certain sensitive tasks, such as changing a password, always stay with you.

On data, OpenAI says it doesn't use content from ChatGPT Business, Enterprise, or Edu workspaces to improve its models by default. On personal plans, you can control whether your dot's conversations and work are used for training. They also say they don't train directly on proactive research or your dot's notes to itself.

And there's one line I want every engineer to tattoo somewhere: "Dots can still make mistakes, so always review consequential work." OpenAI points to a dedicated dots safety blog, its Help Center, and a system card for more detail.

My take: this is the right shape. Default-deny for risky actions, read-only background mode, human approval for anything consequential, and a visible audit trail. It's basically how I'd want a new junior engineer onboarded: broad read access, narrow write access, and a reviewer on every merge.

Specialist Dots: The Enterprise Angle

Beyond your personal dot, OpenAI is previewing specialist dots. These take on dedicated responsibilities inside an organization rather than working on behalf of one person.

Per the announcement:

Your company sets up each specialist dot with its own identity, credentials, and access to the systems it needs.
The preview includes identity for access management, IT-provisioned hardware, and deep integrations with a company's systems of record.
The dot adapts to new information and improves with feedback from your team.
OpenAI is building on internal testing across procurement, invoice processing, email marketing, customer support, and commercial contracting.

This is starting with focused enterprise pilots, where OpenAI's engineering teams work directly with organizations to define each dot's responsibilities, the tools it can use, and how people review and approve its work.

OpenAI is also working with Microsoft to integrate specialist dots with Microsoft's enterprise governance and security controls in Agent 365, so businesses can manage dots with the Microsoft tools they already use.

For those of us who've fought with service accounts and shared credentials, giving an agent its own identity is a big deal. It means the audit logs can say "the procurement dot did this" instead of "someone using Dave's token did this."

Availability, Pricing, and Limits

Here's the practical part, straight from the announcement:

Who gets it now: Dots are rolling out in ChatGPT to Pro and Business Premium users in eligible markets.
Enterprise: Enterprise users (including Edu and Healthcare) can try the beta once their workspace admin enables it.
More users: OpenAI says it plans to expand dots to more users soon.
Cost: Your first dot is included in Pro or Business Premium at no extra cost. It's available 24/7 to talk and help you think.
Deeper work: Your plan includes an allowance for deeper work, with extended limits for the first month after launch. OpenAI doesn't publish specific numbers for that allowance in the announcement.
Usage limits: Conversations with your dot don't count toward your ChatGPT usage limits. But when you ask your dot to start or manage tasks in Codex or ChatGPT Work, those tasks count toward your limits as usual.
Later: OpenAI says you'll eventually be able to add more dots and scale each one's output by increasing its speed or the total amount of work it can take on per month.
Setup: Create your first dot in the ChatGPT desktop app or a desktop browser, connect your apps, and let it introduce itself. After that initial setup, you can message it from the ChatGPT mobile app.

That usage-limit detail is easy to miss and worth planning around. Chatting with your dot is "free" against limits, but delegating real Codex work isn't.

How I'm Planning My First Week With a Dot

This is my own plan, not an OpenAI recommendation. If you lead a team, it might be a useful starting template:

Week 1 — dot onboarding plan (my notes)
Day 1: Create the dot on desktop, name it, connect only Slack + GitHub + calendar
Day 2: Set Custom Rules — require approval for anything that posts, merges, or emails
Day 3: Let proactive research run; check Activity View at end of day
Day 4: Hand it one small, well-scoped bugfix from the feedback backlog
Day 5: Review the PR like a junior's PR; write feedback, not just approve/reject
Weekend: Decide what to widen (or narrow) next week

The principle is the same one I use with any new teammate: start narrow, watch closely, widen access based on evidence. OpenAI has given us the controls (Custom Rules, Activity View, app permissions, auto-review). It's on us to actually use them instead of clicking "allow all."

My Honest Reflection as a Lead SWE

I've been burned by agent hype before, so here's where I land.

What I'm genuinely excited about: persistence and isolation. An agent with its own inspectable computer, context that carries across ChatGPT, Slack, and Teams, and a read-only proactive mode is a real architectural step. It's not just a smarter prompt box.

What I'm cautious about: everything that depends on the dot learning "what good looks like" for me. That's the hardest part to evaluate from an announcement. It will only prove itself through weeks of real review cycles. OpenAI's own advice to always review consequential work tells me they know that too.

What I'm not claiming: I don't have numbers on speed, accuracy, or cost per task, and OpenAI didn't publish any in the launch post. Anyone quoting benchmarks for dots right now should show their source.

The shift I expect is this: my job gets less about doing the long tail and more about defining standards and reviewing output. That was always the job of a good lead. Dots just make it hard to pretend otherwise.

Key Takeaways for Fellow Developers

Dots are persistent agents, not chat sessions. GPT-6 Astra, their own cloud computer and browser, 24/7 operation, 4,000+ app connections.
Control is built in. Use it. Custom Rules, Activity View, auto-review, and ChatGPT app permissions exist so you can start narrow.
Background work is read-only. Proactive research can look, but it can't send, edit, or control your machine.
Mind the limits. Dot conversations don't count against ChatGPT limits, but Codex and ChatGPT Work tasks it kicks off do.
Availability is staged. Pro and Business Premium first, Enterprise beta via admins, specialist dots via pilots, and Microsoft Agent 365 integration in the works.

What would you hand your dot first: the bug backlog, the inbox, or the planning doc nobody updates? Tell me in the comments. I'm genuinely curious how other teams plan to onboard their first always-on teammate.

Originally published on Medium.

Supercharging Your Security Audits: My Deep Dive into a Game-Changing AI Skill

Ishank Choudhary — Mon, 28 Sep 2026 05:00:45 +0000

Have you ever stared at a complex codebase, knowing there are hidden security vulnerabilities lurking, but feeling overwhelmed by the sheer scale of manual auditing? Or perhaps you've wished for a way to scale your security efforts without scaling your human team linearly? If so, you're not alone. As a Lead SWE, I constantly grapple with the challenge of building secure applications at speed. This past week, something truly remarkable caught my attention: a particular security-audit-skill from Cloudflare that absolutely exploded on GitHub Trending, racking up roughly +15.4k stars in just 7 days. While the repository itself has been around since June, this sudden surge in interest is the real story, and it immediately sent me down a rabbit hole.

My initial thought was, "Another security tool? What makes this one different?" But as I dug deeper, I realized this isn't just another scanner. This is a paradigm shift in how we approach security audits, leveraging the power of AI agents to perform structured, multi-phase vulnerability discovery. Imagine turning a coding agent like Claude Code or Codex into a structured, six-phase security auditor with one npx install and a one-line prompt, with a separate verifier re-checking every finding. That's precisely what this skill promises, and the design behind it holds up to scrutiny.

The "Why" Behind the Explosive Growth: A Developer's Perspective

The recent surge in popularity isn't just hype; it reflects a genuine need in our industry. We're all under pressure to deliver features faster, but security can't be an afterthought. Traditional security audits are often slow, expensive, and require specialized expertise that isn't always readily available. Automated scanners exist, of course, but they often struggle with context, complex logic, and the nuanced understanding required to uncover truly subtle vulnerabilities.

What I've found so compelling about this security-audit-skill is its intelligent orchestration. It doesn't just run a static analysis; it emulates a structured audit process, much like a human auditor would, but at machine speed and with a tireless attention to detail. The idea that a single npx install can set up such a comprehensive, AI-driven process for my codebase is incredibly appealing. It’s like having a team of specialized security agents working around the clock, systematically dissecting every layer of my application.

My Deep Dive: Unpacking the Six Phases of an AI-Powered Audit

What truly sets this skill apart is its structured, multi-phase approach. It’s not a black box; it's a transparent, methodical process that mirrors best practices in security auditing. I spent some time dissecting each phase, and here’s what I learned:

Reconnaissance: Mapping the Digital Landscape
This is where it all begins. Just like a human auditor, the AI agent first maps out the architecture of the target system. It identifies trust boundaries, input surfaces, and any prior evidence or known issues. It even attempts to determine deterministic coverage, creating an architecture.md and coverage-ledger.json. This initial mapping is crucial because it provides the foundational context for the entire audit. Without a clear understanding of the system's layout, any subsequent hunting would be akin to stumbling in the dark. I appreciate that it generates a coverage ledger from the start, setting a baseline for what's been explored.
Coverage-led Hunting: Intelligent Exploration
Once the reconnaissance is complete, the skill assigns "isolated hunters" based on units defined in the coverage ledger. These hunters are individual AI agents, each tasked with exploring specific parts of the codebase. They record their checks, and critically, "coverage critics" are used to find gaps in their exploration. This isn't just random fuzzing; it's a directed, intelligent exploration designed to maximize the chances of finding vulnerabilities by ensuring comprehensive coverage. It's like having multiple specialists each focusing on their domain, but with an overarching supervisor making sure no stone is left unturned.
Candidate Validation: The Art of Disproving
This phase is brilliant. Every unique potential finding – or "candidate" – is handed off to a fresh, independent verifier agent. This verifier's job isn't to confirm the finding, but to try and disprove it. This adversarial validation process is a cornerstone of robust security testing. It minimizes false positives and ensures that only genuinely problematic candidates proceed. As a developer, few things are more frustrating than chasing down a "vulnerability" that turns out to be a false alarm. This built-in skepticism is a huge win for efficiency and accuracy.
Structured Output: Machine-Readable Insights
After validation, the skill doesn't just spew out raw text. It writes confirmed, needs_validation, and rejected records into a findings.json file. What's more, these findings are validated against a report-schema.json. This structured, machine-readable output is invaluable. It means the results aren't just for human consumption; they can be easily integrated into other tools, dashboards, or CI/CD pipelines. For me, this is a critical feature, as it allows for automation downstream and better tracking of security posture over time. A needs_validation verdict, for instance, means there's an exact unresolved fact that needs human attention, without assigning a premature severity.
Independent Record Verification: Trust, But Verify (Again)
Even after structured output, the process isn't over. Fresh, independent agents are brought in to verify the final source claims. If any material replacements occur during this phase (e.g., a finding's details are updated), another independent verifier is dispatched. This double-checking mechanism is a testament to the skill's commitment to accuracy and reliability. It reinforces the principle of "adversarial validation" and ensures that the final reports are as trustworthy as possible. This is where the "separate verifier re-checks every finding" promise truly shines, adding an extra layer of confidence.
Target-Neutral Reporting: Actionable Intelligence
Finally, all the verified records and the coverage ledger are used to derive comprehensive reports: REPORT.md, FINDINGS-DETAIL.md, and NEEDS-VALIDATION.md. These reports are "target-neutral," meaning they focus on the vulnerabilities themselves rather than being tied to a specific tool or framework. This makes the findings universally understandable and actionable. Having distinct reports for confirmed issues, detailed findings, and items requiring further human validation is incredibly helpful for prioritizing and allocating resources.

Beyond the Phases: Core Principles That Resonate

My deep dive also revealed some fundamental design principles that I believe are crucial for any effective security tool:

Only confirm established boundary failures: This means it focuses on genuine security flaws, not just "could be bad" scenarios.
Adversarial validation: As mentioned, the agent that checks a finding is never the agent that found it. This dramatically reduces bias.
Severity requires impact: A deviation from a checklist isn't a vulnerability unless it has a measurable impact. This helps prioritize real risks.
Defense-in-depth gaps are not vulnerabilities: If one layer of security prevents an attack, the absence of another layer is a hardening note, not a critical vulnerability. This is a nuanced but important distinction for practical security.

These principles show a maturity in design that goes beyond simple pattern matching, aiming for a more intelligent and context-aware assessment.

Getting Started: Bringing AI to Your Codebase

What truly excites me as a Lead SWE is how accessible this technology is. Getting started with the security-audit-skill is surprisingly straightforward. It integrates with existing coding agents that support tool use and parallel sub-agents (like the aforementioned Claude Code or Codex).

First, you install the skill using npx:

npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit

For a user-level installation, you can add --global:

npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit \
  --global

Once installed, you simply point your coding agent at your codebase and ask it to perform an audit. The skill activates automatically when your request matches its triggers (e.g., "security audit," "find vulnerabilities," "pen-test the code").

For instance, if you're working on a Node.js or TypeScript repository, you might instruct your agent:

security audit this codebase

Or, to focus on a specific directory:

find security vulnerabilities in ./src

Or even specify an output directory:

do a security review, output to ~/audits/my-project

The tool requires Node.js for its zero-dependency validators and, critically, an OS-enforced sandbox for executing target code safely. This sandbox is essential for preventing any malicious code within the audited project from affecting your system. Without it, the workflow wisely keeps findings as needs_validation rather than executing potentially unsafe target code, which is a fantastic safety measure.

One detail I liked for Node/TS monorepos: in full audit mode the output defaults to ~/security-audit-skill/<repo-name>/run-<N>, outside your repository. It only writes inside the target repo when you explicitly pick a directory that version control ignores, so your working tree stays clean.

The Power of Repetition: Continuous Improvement

One insight from my deep dive that really stuck with me is the concept of additive runs. The documentation suggests that multiple runs against the same repository are not redundant; they actually improve coverage. The skill uses prior ledgers and findings to target gaps, revalidate changed source, and carry forward current-source evidence. Cloudflare is candid about why: in their own test runs, a single run found roughly half of the vulnerabilities that repeated runs found in total. My experience tells me that security isn't a one-time check; it's a continuous process. A tool that learns and improves its coverage with each iteration is a significant advantage in a rapidly evolving threat landscape.

Honest Ship/Skip: Is This For You?

So, is this the silver bullet for all security woes? Not necessarily. It's a powerful tool, but like any technology, it has its best use cases.

Ship it if:

You're working on complex codebases where manual audits are time-consuming and prone to human error.
You need to scale your security efforts and provide systematic, structured vulnerability discovery.
You're looking to integrate advanced AI capabilities into your CI/CD pipeline for continuous security assessment.
You appreciate a methodical approach with independent verification, aiming for high-quality, low-false-positive findings.
You have access to powerful AI agents (like Claude Code or Codex) and are comfortable setting up a secure sandboxed environment for execution.

Skip it (or use with caution) if:

You're working on extremely small, simple projects where a quick manual review or a basic linter might suffice.
You don't have the infrastructure or expertise to set up an OS-enforced sandbox, which is a critical requirement for safely executing target code.
You expect it to magically fix all your security problems without any human oversight or understanding of its findings. It's an assistant, not a replacement for human intelligence.

For a Lead SWE like myself, navigating enterprise-grade applications with intricate logic and numerous dependencies, this security-audit-skill feels like a genuine game-changer. It packages audit methodology that was once the domain of specialized security teams, and it seeded the much larger fleet-wide harness Cloudflare describes in its Build your own vulnerability harness post.

Conclusion: A New Horizon for Developer Security

The explosion of interest in Cloudflare's security-audit-skill on GitHub Trending is a clear signal: developers are hungry for smarter, more efficient ways to secure their code. My deep dive into its architecture and methodology has convinced me that this isn't just a fleeting trend; it's a significant step forward in making advanced security auditing accessible and actionable for every engineering team.

Here are my key takeaways:

Structured & Systematic: The six-phase audit process provides a level of methodical rigor often missing in automated tools.
AI-Powered Precision: Leveraging AI agents for reconnaissance, hunting, and especially adversarial validation leads to higher accuracy and fewer false positives.
Independent Verification: The repeated, independent checks on findings instill a high degree of confidence in the results.
Actionable Output: Machine-readable findings and clear reports (confirmed, needs_validation, rejected) streamline the remediation process.
Accessible Power: With a simple npx command, developers can tap into a sophisticated auditing harness that was once an internal Cloudflare innovation.

I encourage you to explore this skill for your own projects. Start with one repo, read the NEEDS-VALIDATION.md file as carefully as the confirmed findings, and run it more than once. What are your thoughts on AI-driven security auditing? Have you experimented with similar tools, or are you ready to take the plunge? Share your experiences in the comments below! The future of secure software development is here, and it's worth getting hands-on with now.

Connect with me:

https://ishankdev.github.io/

If this deep-dive helped, you can buy me a coffee — totally optional.