DEV Community: Qingfeng

GAL: a single-binary AI agent runtime that is kernel-sandboxed by default

Qingfeng — Mon, 26 Oct 2026 14:08:14 +0000

Most "AI agent" projects you can try today want you to pip install a tree of dependencies, hand over an API key, and then trust the thing with rm -rf-level access to your shell. That is a lot of trust to give a model that is, famously, confidently wrong sometimes.

GAL (General Agent Loop & Executor) takes a different bet: an LLM agent you can actually let loose on your own machine, because it ships as one binary and runs inside a real OS sandbox by default.

No Python. No Node. No Docker. Download a file, run it.

What it is

GAL is a self-contained agent runtime with three front ends built into the same binary family:

gal — a terminal/SSH agent (great for servers, CI, and "just get it done" work)
galw — a desktop GUI you double-click
gal serve — an IM / HTTP server so the same agent is reachable from chat

It runs on Linux, macOS, and Windows, and it is local-first: your files, your machine, your tokens. Zero install means there is nothing to break and nothing to keep in sync.

The part I actually care about: safe by default

The reason most people won't let an agent touch their real machine is that one bad turn can cost hours. GAL's answer is a kernel-level sandbox that behaves the same way on every OS:

Linux — private mount namespace + OverlayFS
macOS — Seatbelt
Windows — AppContainer

Inside your project the agent gets full productivity. Outside it — corrupting the machine, exfiltrating secrets — it is blocked by the kernel, not by a polite prompt.

And when it genuinely needs elevated access? You type

/sudo restart the nginx service and tail the log

which grants exactly one turn of unrestricted access and then automatically snaps back to the sandbox. No permanent "yes, allow everything" checkbox.

Long sessions that don't lose the plot

Model-driven memory means the agent decides what to summarize and what to keep, so you can run tens of thousands of agent rounds in a single session with the current turn always protected. No more "sorry, I forgot what we were doing 40 steps ago."

Self-extending

Skills — drop a SKILL.md plus scripts into a folder; the LLM discovers it and turns the scripts into tools.
MCP — connect any MCP server and its tools/resources show up automatically.

So the runtime stays small; you grow its capability with plain files and standard MCP servers.

Install (60 seconds)

Grab the archive for your platform from the download page or the GitHub releases:

Platform	CLI (`gal`)	GUI (`galw`)
Linux amd64	`gal-linux-amd64.tar.gz`	`galw-linux-amd64.tar.gz`
macOS Intel	`gal-darwin-amd64.tar.gz`	`galw-darwin-amd64.zip`
macOS Apple Silicon	`gal-darwin-arm64.tar.gz`	`galw-darwin-arm64.zip`
Windows amd64	`gal-windows-amd64.zip`	`galw-windows-amd64.zip`

Linux / macOS:

tar xzf gal-linux-amd64.tar.gz   # or gal-darwin-arm64.tar.gz, etc.
chmod +x gal
./gal init && ./gal chat

Windows: unzip gal-windows-amd64.zip, double-click gal.exe (CLI) or galw.exe (GUI).

Verify what you downloaded:

sha256sum -c checksums.txt --ignore-missing

Why another agent runtime?

Because the hard part of putting an agent on a real machine was never the model call — it's the blast radius. GAL's wager is that capability is becoming cheap, and the thing worth engineering carefully is the boundary: what the agent may touch, when it must ask, and how it gets put back in its box afterwards. A single binary with a kernel sandbox is a much smaller thing to trust than a dependency tree with sudo.

Site + docs: https://agent-home.ai
Releases: https://github.com/gal-agent/gal-release
Feedback group (Feishu): see the README

If you try it, I'd genuinely like to hear where it fights you — that's the useful feedback.

Give your agent permissions, not promises

Qingfeng — Wed, 21 Oct 2026 16:21:41 +0000

Most agent demos look great until they touch a real repo. The moment an agent can run shell commands, edit files, hit APIs, and push commits, the interesting question stops being "how smart is the model" and becomes "what is it allowed to do, and can I prove what it did."

That shift is the whole game. A confident agent that says "done" is a claim. An agent that can be bounded, audited, and rolled back is a product.

The failure mode everyone hits

The classic incident is not the model being evil. It is a fuzzy instruction executed too literally:

"don't notify anyone yet" -> the agent disables its own alerting.
"clean up the temp files" -> it removes something that looked temporary.
"make the tests pass" -> it edits the test.

None of this requires a rogue AI. It just requires an agent that can operate on code, files, and settings with the same authority you gave it for the happy path. The fix is not a better prompt. It is a permission model.

Three things that actually bound an agent

1. Sandbox by default. Run the agent where the blast radius is small: a container, a scratch directory, a staging copy. Anything that writes to real state is a deliberate escalation, not the default.

2. Gate on reversibility, not on action count. Approving every tool call does not scale -- humans rubber-stamp by the tenth prompt. Instead classify actions:

reversible local edits -> let them run, log them
writes that touch shared state -> stage and promote in a batch
destructive or outward-facing actions (deletes, payments, sends, pushes) -> require an explicit human ok

3. Log every write with a diff. "The agent did something" is useless. What you want is: what changed, why (the decision), and what it touched. If you cannot diff it, you cannot review it, and if you cannot review it, you are trusting the most confident thing in the room.

The pattern that scales: review the batch, not the call

Per-action approval breaks at volume. What works in practice is a staging layer:

writes land in a copy you can diff
the agent records a one-line decision for each change (changed X because Y)
you review the batch with diffs and promote it

This keeps a human in the loop where it matters (irreversible steps) while letting the agent move fast where the downside is a cheap revert.

Where this shows up in tooling

I run agents this way locally on purpose. The setup I keep coming back to is one where the sandbox and the permission tiers are defaults, not an afterthought -- every risky action needs a human OK, and every step is logged so a bad batch is a rollback, not a postmortem. That is the model GAL is built around (agent-home.ai): local-first, bounded by default, auditable.

The point is not the specific tool. The point is the order of operations:

clever is cheap now. control is the product.

GAL is a single static binary, zero dependencies, and you can bound every action it takes. Download it for Windows, macOS, and Linux: https://agent-home.ai/download.html

If you are shipping agents against real codebases, spend your effort on the permission model, the diff log, and the rollback path -- before you spend another hour on the prompt. The prompt keeps improving for free. The blast radius you did not design for will not.

What is your rule for "the agent may do this without asking me"? Drops and shells, kill switches, staging copy, approval tiers -- curious what holds up under real traffic.

The Two Hour Audit I Do Before Letting an Agent Touch Real Files

Qingfeng — Wed, 21 Oct 2026 09:13:13 +0000

Last spring I let an agent run unsupervised in a real project directory for the first time. It finished the task I gave it, then kept making changes it thought were helpful, and one of those changes overwrote a config file I had spent a week tuning. Nothing was deleted. Nothing looked broken until the next morning. I lost two days rebuilding numbers I had never written down. That was the cheap version of the story.

So now there is a two hour audit before any agent's first real run. Not because I assume it is hostile or incompetent. Because the first unattended run is the only run where I genuinely do not know the shape of the damage, and the price of learning it by accident is a config file, a database index, or a directory of half-written documents. The audit is deliberately boring. Four lists, one switch, one copy, and a condition I can check with a command.

Hour one: map the blast radius

I open a scratch note and write four headings: reads, writes, executes, sends. Then I walk the configuration, the tool definitions, the permission flags, the prompt, and the directory it will live in, and I put every capability I can find under one of the four.

Reads is always the longest list and the one I used to skip. Everything in the working directory, obviously, but also the home directory if the process runs as me, environment variables, SSH keys, browser profiles, a shared mount with client documents on it. I check with a file tree listing taken from inside the sandbox the agent will actually run in, not from my own shell, because those are frequently different worlds. If the tree holds paths I would not paste into a chat window, the agent should not have read access either.

Writes is where the audit earns its keep. A directory, a log file, a temp folder, a package cache, the stdout of a build. For each, I ask what a wrong write costs me. Overwritten notes I can restore. An overwritten migration file can take a whole evening. A write into a directory that syncs to a remote gets copied everywhere before I notice.

Executes covers shell access, package managers, anything that can spawn a subprocess. The question is not whether it can run commands but which commands, and I try to make that list finite. A single easy escape hatch like unrestricted shell turns the other three lists into decoration, because anything the agent can write it can then run.

Sends is the list people forget, and it is the one that has surprised me every time. Domains it can reach, a webhook, a git remote, an email address. I write every destination as a hostname, then turn the list into an allowlist in the actual configuration, so anything not written down fails closed instead of succeeding quietly at three in the morning.

Hour two: build a dry run

A dry run is not a log level or a preview screen. It is a mode where the agent goes through every step except the last one and prints what it would have done. The one I build for a new agent takes about forty minutes and looks like this.

$ agent run --dry-run --task "rebuild the docs index"
[dry-run] would exec : python tools/reindex.py --root ./docs
[dry-run] would write: ./docs/index.json          (2.1 MB, 1 file)
[dry-run] would delete: ./docs/index.json.bak     (regenerate instead)
[dry-run] would send : https://hooks.example.net  -> blocked, not in allowlist
[dry-run] result: 0 files changed, 0 bytes sent, 1 blocked action

The flag is worth having only because it names the exact command and the exact paths. If the output says "would update the index" I go back and make it print the file list, because a summary cannot be checked and a list can. It also has to say what it would have deleted, since deletion is what I am least able to undo.

Then I run the three tasks I actually care about in dry-run mode, in the order I would run them for real, including one I expect to fail. That third one matters most. A dry run that only exercises the happy path tells me nothing about how the agent behaves when it hits a permission error or an empty directory, which is exactly when it improvises something I did not authorize.

The snapshot

Before the first real run, I copy the one directory the agent will write to. Just the directory that appeared in the writes list from hour one, not a full image. I timestamp the copy with the date and the task name so that in six weeks I still know what it is without opening it. Somewhere in the same note I write the restore command in full, and I run it once against a throwaway file so I know the syntax is right when I am tired. A backup I have never restored from is a hope, not a backup.

The finish condition

For each task I intend to run, I write one line that a machine could check. Not "the docs are updated" but "docs/index.json exists, parses as JSON, and contains at least as many entries as the source directory". Not "the tests pass" as an aspiration but as the literal command and its exit code. For anything message-shaped, the condition is a line appearing in an outbox file, so I can verify without relying on the agent's own report.

If I cannot write that line, the task is not ready. That rule has killed more tasks than any other part of this audit, and it has been right nearly every time. A task without a checkable finish condition is a task where the agent's own summary becomes the ground truth, which means I am reading a description of the work instead of the work.

The first real run

When the real run happens, I watch it. In the room, with the log streaming next to me. I keep that log, dated, beside the snapshot, because in an hour it will be the only record of what happened and the diff alone will not explain why.

Afterwards I read the diff line by line. The summary at the top I ignore until the end, and sometimes I read the diff a second time backwards, starting from the files I care most about. Summaries are written by the same process I am auditing. They are useful for orientation and useless as evidence. Reading the diff takes fifteen minutes on a small task, and those fifteen minutes are what catch the change nobody asked for.

What this actually buys you

I want to be honest about the size of the thing. Two hours of audit does not make an agent safe. It does not prevent the confident wrong edit, the plausible file with wrong contents, the permission that turns out broader than the documentation said. A checklist fixes none of that.

What it buys is a different distribution of failures. Without the audit, the first failure is large, silent, and discovered by someone else. With it, the failure tends to be small, loud, and mine. A blocked domain in a dry run, a diff line I did not expect, a restore that takes four minutes instead of two days. The difference is who finds out, when, and whether the record of what happened is sitting in a log file or in my memory at nine in the morning.

That is the whole audit. It fits on one page, it takes two hours, and it is the cheapest insurance I have found for the specific moment when a very confident process gets write access to files I care about.

The Two Lists I Keep for an Agent That Runs Unattended

Qingfeng — Wed, 21 Oct 2026 07:47:36 +0000

Anything that runs while I am not watching needs two lists, and they are not the same list.

The first is what it may touch. Directories, accounts, endpoints. Boring, and the only thing standing between a bug and a deleted folder. Mine is shorter than I expected: two working directories, one scratch directory, read-only everywhere else. Everything I have ever regretted came from a task that quietly needed one more permission and I gave it without thinking.

The second list is what it may do when something goes wrong. Retry twice. Stop. Report the state it was in. Never, under any circumstance, start over from the beginning, because starting over is how a half-finished change becomes a half-finished change plus a duplicate.

That second list is the one nobody writes.

Why the failure policy is harder

Permission lists are static, so you can reason about them once and forget them. Failure policy is a decision tree that only gets exercised on the days everything is already going badly, which means you write it in a hurry, if at all.

The practical version I use is three lines per task: on error, retry N times; then either abort or degrade; and always emit the last known good state so a human can pick up from it. Degrade means the task publishes what it has with the gaps marked, instead of pretending everything succeeded.

The test that matters

I now deliberately break something before trusting a scheduled task. Kill the network mid-run. Make one input file unreadable. Point one endpoint at a 500. If the task retries forever, or claims success anyway, it is not ready, no matter how good the normal-path output looks.

The awkward truth is that the failure path is the product for anything unattended. The happy path is a demo; the failure path is whether you can sleep.

The Hardest Part of a Local Agent Is Deciding When to Stop

Qingfeng — Wed, 21 Oct 2026 07:12:33 +0000

Every agent demo I have seen ends the same way: a task, a burst of activity, a confident summary. Nobody ever shows the part where it should have stopped three steps ago.

Stopping is the actual product.

Three ways to stop, only one of which I trust

Step budget. Stop after N actions. Cheap, predictable, and completely blind to whether progress was being made. My step budgets are now emergency brakes, not the main mechanism.

Time budget. Same problem plus a clock dependency. Useful only as a hard ceiling.

Goal verification. Ask: is the thing I was asked to produce actually present? A file that parses. A test that passes. A message that appears in the outbox. This is the only one that tracks reality, and it is also the only one that is hard to write.

Why verification is hard

Because it requires you to say, in advance, what finished looks like. That sentence is the whole design. If I cannot write it, I do not actually know what I am asking for, and neither does the agent.

I keep a rule from this: no task enters my queue without a finish condition I can check mechanically. Not "the report is good". "The file exists, has more than 200 characters, and its last line is a date". Yes it is crude. Crude and checkable beats elegant and unverifiable.

The failure mode that taught me this

I had an agent tidying a notes directory. It ran for eleven minutes, reported success, and had moved everything into a folder called misc - including the source files for a post I was halfway through. The task had no finish condition, so any state counted as done.

The fix was not a better model. It was one line: the task may not end while any file it touched has a modification time newer than the start of the run and no recorded destination.

The uncomfortable conclusion

Most of the work of a reliable agent is not in the agent. It is in the sentences you write about what counts as finished, and in the small piece of code that checks them.

That work is boring, it does not demo well, and it is the difference between a toy and a tool you leave running overnight.

A Script and a Task Are Not the Same Thing

Qingfeng — Wed, 21 Oct 2026 07:07:45 +0000

The word "task" gets used for two very different objects, and the confusion costs people weeks.

A script is deterministic. Same input, same output, no judgement, no surprises. A task is a small delegation: something has to look at the world, decide, and act. Scripts fail loudly and immediately. Tasks fail quietly, three steps later.

Most of the disappointment I see with agents comes from treating one as the other.

When it should be a script

If you can write down the steps without using the word "check" or "figure out", it is a script. Renaming files, running the test suite, deploying a known branch - all scripts. Do not put an agent in the loop. It is slower, it costs money, and it introduces a failure mode that did not exist.

My rule: if I have to explain the task to a colleague twice, and both times I use the same words, it is a script that nobody has written yet.

When it is genuinely a task

When the steps depend on what the world looks like at the moment. "Find the failing test and fix it" is a task. So is "check whether the deploy actually took effect" and "figure out which of these three dependency bumps is the breaking one".

Notice that all of these contain an implicit decision. That is the signal. A task has a fork in it, and which way to go is not known until someone looks.

The mixed case that gets people

The dangerous middle is a script with one unknown step. It looks like a task, so people hand the whole thing to an agent, and now a deterministic sequence of ten steps is being improvised by something with judgement, at ten times the cost, with a new failure mode per step.

The fix is to keep the script and delegate only the unknown step. The script calls the task, that task returns a decision, the script continues. Most of my reliable automations have exactly this shape: a boring deterministic spine with one intelligent node in the middle.

Why the distinction holds up

Scripts are auditable. I can read the file and know what will happen. Tasks have to be verified by result, which is why the finish condition matters so much.

When the two get mixed, you lose both properties: you cannot audit it, and you cannot fully verify it either. That is the worst of both worlds, and it is a surprisingly common place to end up.

If you have found a task that turned out to be a script in disguise, I would like to hear it. Mine was a weekly report I had been "delegating" for a month before realising the numbers came from the same three queries every time.

I Read My Agent's Log Backwards, and the Bug Was in Line Two

Qingfeng — Wed, 21 Oct 2026 06:42:29 +0000

Normal debugging reads the log from the top: what did it try, what happened, where did it stop. That works when the failure is at the end, which is not where most of mine are.

Now I read the last line first, then the line that produced it, then walk backwards to the first decision that was wrong. The run usually failed at the top.

Why backwards is faster here

An agent that goes wrong at step two keeps going. It spends the rest of the run working on a false premise, and the last thing it does is the least informative thing in the file - an error on an already-wrong task.

Reading forwards, I walked through twenty correct-looking steps before reaching the failure. Reading backwards, I ask "what produced this state" at each step, and I arrive at the bad decision in three or four hops.

What I look for on the walk back

The first step whose input does not match what I expected. This is almost always the one, and it is usually a file read, a listing, or a state check.
The first step that silently changed something. Agents write files and start services; the log should say so. If a step has a side effect that is not in the log, that is a logging bug as well as a real one.
The last plausible command. Not the last command. The last one whose output would be different if the world were what I thought it was.

What this changed in the setup

Once I was debugging this way, I stopped logging so much of the middle and started logging more at the boundaries: every read of external state, every side effect, the exact input that produced each decision. The middle steps are mostly derivable. The boundaries are not.

That is a small change with a large effect: my logs got shorter, and my time to find the wrong step went down, because the wrong step is almost always a boundary read that returned something I did not expect.

If you have done this long enough to have a rhythm, I am curious whether you walk forward or backwards. Mine was forwards for two months and it was costing me more than I noticed.

What I Automate First in a New Project, and Why It Is Never the Important Part

Qingfeng — Wed, 21 Oct 2026 06:17:30 +0000

When I start working in a new repo or a new environment, there is a temptation to automate the interesting thing first. I have learned to resist it, and the order I use instead looks almost backwards.

The order

The thing I run to find out where I am. Branch, dirty files, running services, last commit, what changed yesterday. This is not work, it is orientation, and I run it five times a day.
The thing I run after every change. The build, the tests, the linter, whatever is fast enough to run constantly.
The thing I run before I ask a human. The check that tells me whether my change does what I think.
Only then the actual task that made me open the project.

The first three are not the interesting part of the project. They are also the three that run most often, which is exactly why automating them pays back within days rather than weeks.

Why this order is better than it looks

Automating the interesting task first feels productive and produces one saved task. Automating the orientation and the checks produces something else: it makes every later task cheaper, including the interesting one, including the ones the agent runs and the ones I run myself.

There is a second effect that I did not expect. Once the checks are automated, the agent can be given tasks I would never have trusted it with, because the check catches the failures I was worried about. The tooling is what made the delegation safe, not the model.

The trap

The trap is that steps one to three feel like overhead. They are not part of the deliverable, nobody thanks you for them, and they are invisible in the demo.

They are also the only part of the setup that keeps working after the demo, which is the entire point.

A concrete test

If a new person joined the project tomorrow, what would you tell them to run first? Not what to read - what to run. That command, or that script, is step one. If the answer is "ask someone", that is the work.

I am aware this is unglamorous advice. It is also the difference between a setup that survives three months and one that gets abandoned in week two, which is where most of the abandoned ones sit.

Letting the Agent Touch the Browser Changed My Task List

Qingfeng — Wed, 21 Oct 2026 05:47:59 +0000

For the first month I kept the agent inside the terminal. Files, git, commands - all fine. The browser was off limits because I assumed it would be flaky, and every GUI automation story I had heard ended in a pile of brittle coordinates.

Then a task forced the issue: a weekly report that only exists behind a login, with no export button. Either the agent learns the browser or I keep doing it by hand forever.

What was actually hard

Not clicking. Clicking was fine. The hard parts were, in order:

Knowing when the page was ready. Sleeps are a lie. I ended up waiting on a specific element that only exists after the data loads, and failing loudly after a timeout rather than continuing into an empty table.
Telling a real failure from a slow one. A page that did not load looks identical to a page that is still loading. The fix was the same element wait, plus a screenshot saved on every timeout, so the log tells me which it was.
Finding things that do not have stable coordinates. Layout changes, banners appear, a cookie notice shifts everything down. Reading the accessibility tree and clicking by role and name was more robust than pixels, and it also produces a log I can read.
Anything that needs me. Two factor prompts, captchas, "confirm your identity". These are not automation problems, they are stop conditions, and the agent should stop rather than guess.

What changed about the task list

The interesting effect was not that old tasks got faster. It was that new tasks became possible.

Things I had classified as "has to be done by hand" turned out to be "has no API". Reading a dashboard, pulling numbers from a supplier portal, checking whether a listing is live - none of these are hard, they were just behind a screen.

That is a bigger deal than shaving minutes off something. The list of what is automatable was wrong, and it was wrong because I had been treating the interface as a wall.

The rules I keep

Never cache a screenshot between steps. Read the current state every time.
If a click produces no visible change, treat it as an error, not as done.
Stop and ask on anything that looks like a security prompt. Do not try to click through it.
Save a screenshot on every failure, named after the step. Half my debugging is looking at those.

I am still cautious about this. A browser agent that half-works is more dangerous than one that refuses, because a half-working flow produces plausible wrong data. The element waits and the loud failures are what make it trustworthy enough to use.

The Twenty Minute Review That Keeps My Agent Honest

Qingfeng — Wed, 21 Oct 2026 05:24:05 +0000

I do a short review every Friday. Twenty minutes, sometimes less. It is the only maintenance ritual I have kept for more than two months, and it is the reason the setup has not quietly rotted.

What I look at

Five things, in this order:

The refusals. Every task the agent declined or escalated. Not to fix them, just to check whether the reason still holds. Half of them were right last week and wrong now.
Tasks that took more than twice my estimate. Either the estimate was wrong or the task is not automatable yet. Both are worth knowing.
The retries. Any task run more than once this week. The reason for the retry is the bug, not the retry.
One log, read end to end. Not skimmed. One, chosen at random. Skimming ten tells you less than reading one.
Things I approved without reading. If the list is long, my approval gate has become a rubber stamp, and the gate is now decoration.

What I do with the output

Usually two or three lines of change. A pre-flight check gets added, a task file gets a better finish condition, a note gets written to the notes file so the same thing does not happen next month.

Most weeks there is nothing dramatic. That is the point - the review is not for finding incidents, it is for catching drift before it becomes one.

Why twenty minutes and not an hour

An hour-long review becomes a project, and projects get postponed. Twenty minutes fits between other things on a Friday, so it actually happens. The value is in the repetition, not the depth.

The failure mode of an agent setup is not a dramatic break. It is a slow accumulation of exceptions that everyone remembers exists but nobody wrote down, until one day the automation is a fragile museum piece that only its author can operate.

I have run that setup before, in other forms. The review is cheap insurance against building it again.

If you do something similar, I would like to know what you look at that I do not. Mine has stayed the same for two months, which probably means it is missing something.

What I Got Wrong About "Just Give It a Better Model"

Qingfeng — Wed, 21 Oct 2026 04:59:23 +0000

For the first few weeks I assumed the failures were the model's fault. It was the easy explanation and it also meant the fix was someone else's job.

Then I counted. About 18% of the failures I logged were actually model errors. The rest were state, permissions, and tasks whose finish condition was never written down.

The three model swaps I did anyway

To be fair to the instinct, I did swap models three times. Each swap did something real:

The first one fixed a class of malformed tool arguments. Genuine improvement, visible in a week.
The second fixed tone on writing tasks so the drafts stopped needing a rewrite. Worth it, but it also made me notice the writing tasks were the ones I should not have delegated in the first place.
The third fixed nothing observable and cost more. I kept it for a month out of sunk-cost stubbornness before going back.

That is the shape of it: better models fix the failures that are actually the model's. They do not touch a wrong branch, a held port, or a task that ends whenever the agent feels it is done.

The trap in the framing

"Just use a better model" sounds like advice about tooling. In practice it is often advice about avoiding the boring part, which is describing the task and the state well enough that a cheap model could do it.

I have watched myself buy tokens to avoid writing five lines. It works, briefly, and then the same failure comes back at a higher price point.

What the better model is genuinely for

I am not arguing against it. When the remaining failures are concentrated in tool selection and multi-step reasoning, a better model moves the needle immediately, and the money is well spent.

The point is order. Pin the state, write the finish condition, define the permission edges, then upgrade the model if there is anything left to fix. Doing it in the other order means paying to run the same failures faster.

If you have done a swap recently, did it fix something you could point at afterwards? Mine did, twice out of three times, and the two that worked were the two where I already knew exactly which failure I was trying to kill.

The Second Run Is Where the Lies Show Up

Qingfeng — Wed, 21 Oct 2026 04:36:01 +0000

Every agent demo I have ever seen works on the first run. Mine did too. The first run has a clean state, a fresh context, and a demo-friendly task that was probably chosen after a few attempts.

The second run is different, and it is the only run I trust.

What changes between run one and run two

The state is no longer clean. Files were created, ports were opened, a lock file exists. Half the failures I logged live in this gap.
The context is no longer empty. Yesterday's task file may still be around. A note in the notes file may contradict what I am asking today.
The task is no longer chosen for the demo. If it works twice on a task I did not specially pick, it probably works.

There is a fourth difference that took me longer to see: on the second run, nobody is watching. On the first run I am looking at the screen, ready to intervene. On the second I am doing something else. That is the real test, and it is also when I find out how good the failure messages are.

The rules I use now

A task is not automated until it has run twice, unattended, on a dirty state.
I keep the state dirty on purpose for the second run. Cleaning up before testing cleanup tests nothing.
If the second run fails, the bug is almost always in what the first run left behind, not in the task.
I read the log of the second run, not the first. The first log is a story, the second log is evidence.

Why this matters for how I judge tools

When I look at a tool now, the question is not "does the demo work". It is "what happens when I run it again tomorrow, with the mess from today still on disk". That question filters out a surprising amount, including things I would otherwise have installed.

The same standard applies to my own work, which is the uncomfortable part. Half the things I call automated have never had a second unattended run. I know which ones they are, and I keep not fixing them.

If you test this way, what does your second run look like? I would especially like to hear from anyone whose second run is genuinely clean, because mine never is.