DEV Community: Riley Li

A Burst-Timing Test: Does Free Inference Hold Up at Your Busiest Hour?

Riley Li — Tue, 27 Oct 2026 13:14:11 +0000

Free model access rarely collapses on average latency. It collapses inside your busiest ninety minutes, when everyone on the shared path wants the same GPU at the same time. So the only comparison worth running is a burst test scheduled at the hours you actually work, not a single quiet-afternoon benchmark. Below is a decision guide I use to choose between a free hosted lane, a free server running your own runtime, a paid hosted lane, and hardware you own.

The core claim is simple: pick your inference lane by how it behaves in your peak window, then by ops load, and only then by sticker price. Price per thousand tokens is a tiebreaker, not a starting point. If a lane cannot clear your floor at 16:00 on a Tuesday, its cheapness is a rounding error on the cost of a missed deadline.

1. Define the floor before you measure anything

A floor is the slowest acceptable round trip for one unit of work in your peak window, and it must be a number you can defend. For an inline coding assistant that a developer waits on, my default floor is 25 seconds to first complete answer for a small, fixed task. For batch jobs, the floor is throughput per hour instead of per-request latency. Write the number and the window into the harness config so it cannot drift.

Why does the window matter more than the number? Because free and shared capacity is a queue, and queues are shaped by other people's schedules, not yours. A p50 measured at 09:00 tells you almost nothing about 16:30. If you only have time for one measurement, measure the busy hour.

2. Build one harness, run it against every lane

The harness below sends the same prompt in fixed-size bursts at four local times, records wall-clock latency and retries per attempt, and appends to CSV. I keep the prompt small and deterministic so differences come from the lane rather than the task. The script is a template: I have not executed it against any provider for this article, and the numbers you get will be your own.

# burst_timing.py -- template, unexecuted here
import csv, json, os, time, urllib.request

BASE_URL = os.environ["LLM_BASE_URL"].rstrip("/")   # e.g. https://host/v1
API_KEY  = os.environ.get("LLM_API_KEY", "")
MODEL    = os.environ["LLM_MODEL"]                  # whatever the lane lists today
PROMPT   = "Write a Python function that reverses a linked list, plus one test."
BURSTS   = ["09:00", "13:00", "16:00", "22:00"]
REQS     = 8          # attempts per burst
FLOOR_S  = 25.0       # your floor from step 1
TIMEOUT  = 180
OUT      = "bursts.csv"

def call_once():
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": PROMPT}],
        "temperature": 0,
    }).encode()
    headers = {"Content-Type": "application/json"}
    if API_KEY:
        headers["Authorization"] = f"Bearer {API_KEY}"
    req = urllib.request.Request(f"{BASE_URL}/chat/completions",
                                 data=body, headers=headers)
    t0 = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
            json.loads(r.read())
        return round(time.monotonic() - t0, 4), True, ""
    except Exception as e:
        return round(time.monotonic() - t0, 4), False, type(e).__name__

with open(OUT, "a", newline="") as f:
    w = csv.writer(f)
    if f.tell() == 0:
        w.writerow(["ts", "burst", "attempt", "latency_s", "ok", "err"])
    for burst in BURSTS:
        for i in range(REQS):          # sequential on purpose: no self-inflicted queue
            lat, ok, err = call_once()
            w.writerow([time.strftime("%Y-%m-%dT%H:%M:%S"), burst, i, lat, ok, err])
            f.flush()
        time.sleep(5)

print("done")

Run it for three working days per lane. Swap lanes only through environment variables so the measurement code never changes:

export LLM_BASE_URL="https://lane-a/v1" LLM_MODEL="<listed-model>" LLM_API_KEY="..."
python burst_timing.py
# repeat with lane-b, lane-c, lane-d; one CSV per lane

3. Reduce the CSV to three numbers per burst

Latency alone is misleading, so reduce each burst to a pass rate against your floor, a p90 latency, and a retry rate. Those three numbers capture waiting, tail behavior, and flakiness in one line each.

# summarize.py -- template, unexecuted here
import csv, statistics, sys

floor = float(sys.argv[2])
rows = list(csv.DictReader(open(sys.argv[1])))
for burst in sorted({r["burst"] for r in rows}):
    b = [r for r in rows if r["burst"] == burst]
    ok = [float(r["latency_s"]) for r in b if r["ok"] == "true"]
    if not ok:
        print(burst, "all failed", len(b)); continue
    ok.sort()
    p90 = ok[min(len(ok) - 1, int(round(0.9 * (len(ok) - 1))))]
    passes = sum(1 for x in ok if x <= floor) / len(b)
    print(f"{burst}  pass={passes:.0%}  p90={p90:.1f}s  "
          f"retries={1 - len(ok)/len(b):.0%}  n={len(b)}")

The output line I care about most is not p90; it is the pass rate inside the busy burst. A lane that passes 95 percent at 09:00 and 50 percent at 16:00 is not a 72 percent lane, it is a lane with a hard scheduling constraint, and you should treat it that way.

4. Place each lane on the tradeoff table

Once you have three days of numbers, the decision usually stops being about speed and becomes about the shape of the failure. This table is the framework I apply after the measurements land.

Signal at your peak window	Free hosted model access	Free server + your runtime	Paid hosted	Self-hosted on owned hardware
Tail latency	Moves with shared load	Predictable once warm, but cold start is yours	Usually smoothed by capacity you pay for	Stable, bounded by your box
Cost shape	Zero spend, unknown queue	Zero spend, machine-limited	Linear with volume	Fixed, plus your maintenance hours
Ops load	Near zero	You own updates, restarts, disk	Low	Highest
Failure mode	Slow or rate-limited during peaks	Dies with the box	Bill grows quietly	Dies with the box, no support path
Data path	Leaves your network	Stays on the server you control	Leaves your network	Stays local
Exit cost	Low; switch endpoints	Medium; rebuild runtime	Low to medium	High; hardware and setup

Fit criteria, stated as rules rather than vibes. Choose free hosted access when your pass rate at your busiest burst stays at or above 90 percent and retries stay under 5 percent for three days. Choose the free server option when the same harness on that machine clears the floor and you have a named person who will patch it. Go paid when the busy-hour pass rate is fine but you need a contractual latency or higher concurrency. Go self-hosted when the data path, not the latency, decides the question.

MonkeyCode is relevant here because it advertises both of those free lanes, model access and a server option, which means one harness can compare them without a procurement step. I treat the free allowance as operator-stated rather than permanent, so I check the current terms and limits on the day I run the test instead of trusting a number from a screenshot.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

5. Set a graduation trigger instead of arguing later

Decision guides fail when they are one-time. Write the trigger down before you start: if the busy-burst pass rate drops below 80 percent on two of three working days, or retries exceed 10 percent, move the affected task class one lane over. That trigger makes the choice reversible and keeps the debate about data rather than preference.

Then re-run the counter-test. Change one thing, wait two days, and rerun the same bursts at the same hours. If the pass rate does not move, your original lane was never the problem, and you just saved yourself a migration.

Where this test is the wrong tool

Small samples lie, and this design uses eight attempts per burst, which is enough to catch a gross scheduling problem and not enough to prove a p99. If you have a hard latency SLO, regulated data that cannot leave your network, or a workload of one forty-minute generation per day, burst statistics will not help you and a contract or a local box will.

The free server lane also assumes someone owns maintenance; if nobody does, that machine becomes an unpatched mystery within a quarter. And no free tier is a commitment, so treat every measurement as a snapshot with an expiry date, not a capacity plan.

If you want to run the harness before you spend anything, the two free lanes are a reasonable place to start, and the useful thing you can bring back is your own burst table rather than a verdict.

Task Bar, Then Queue: Two Trials That Decide Free Tier vs Paid vs Self-Hosted

Riley Li — Mon, 26 Oct 2026 11:46:01 +0000

Task Bar, Then Queue: Two Trials That Decide Free Tier vs Paid vs Self-Hosted

You cannot compare a free model tier with a self-hosted box until you have two numbers, and neither of them is a price. The first is the highest task difficulty the free tier actually clears on your own repository. The second is the latency that tier holds when four requests are in flight at once, because agents rarely run alone. Everything else — token allowances, hourly rates, GPU rentals — is arithmetic you do afterwards, and arithmetic is the easy part.

So why do so many comparisons start with the invoice? Because a price list is public and a task bar is not, and public numbers feel like evidence. They are not evidence about your code, and they never were.

Why the cost table answers the wrong question first

Most "free vs paid vs local" posts I read benchmark the same task: write a small function, explain it, maybe fix a typo. That task clears on nearly every tier, so the comparison collapses into price per million tokens. Meanwhile the work that decides your month is the cross-file rename, the flaky test, the migration that touches eleven call sites. A tier that fails there does not save you 100% of anything; it saves you the tokens you spend discovering that it failed. Is that a discount you would sign up for?

The order I use now is deliberate: capability first, concurrency second, cost third. Capability decides whether the tier is in the race at all. Concurrency decides whether it can run beside your other work. Cost, once both of those pass, is a spreadsheet you finish in ten minutes and never argue about again.

Trial 1 — Build the task bar from your own accepted diffs

Stop inventing test prompts. Your repository already contains a labelled dataset: every pull request you merged last quarter, and every one you rejected. Pull twelve of them, three per difficulty tier, and turn each into a task that checks itself.

Pick the tiers. T1 is a single-function edit with tests already present; T2 is a single-file change with a hidden edge case; T3 is a cross-file change; T4 is "write the failing test for this bug". Keep three tasks per tier so one lucky pass cannot move the average.
Freeze the input. Copy the repository at the parent commit into repo/, and store the prompt you actually wrote as prompt.txt. Do not edit that prompt later to help the model along; that is how benchmarks become flattering.
Write the check, not the answer. Each task folder gets a check.sh that runs one narrow test and exits non-zero on failure. The model never sees it, and you never grade by eyeballing a diff.
Record the ceiling. The tier where the pass rate drops below half is your task bar, and it is the only number here you should trust — yours, not mine.

The harness (a template, not a benchmark result)

This is dependency-light Python against any OpenAI-compatible endpoint; endpoint, key, and model come from environment variables, so nothing assumes a particular provider. I am deliberately not publishing pass rates, because a number from someone else's repository tells you nothing about yours.

# taskbar.py — run: python taskbar.py tasks/ | tee taskbar.json
import json, os, pathlib, subprocess, sys, tempfile
from openai import OpenAI

REPO = pathlib.Path("./repo").resolve()
client = OpenAI(base_url=os.environ["AGENT_BASE_URL"], api_key=os.environ["AGENT_API_KEY"])
MODEL = os.environ["AGENT_MODEL"]

def ask(prompt):
    r = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.2,
    )
    return r.choices[0].message.content, r.usage.total_tokens

def run_task(task_dir, attempts=3):
    target = (task_dir / "target.txt").read_text().strip()  # file the model may rewrite
    tier = (task_dir / "tier.txt").read_text().strip()
    base = (task_dir / "prompt.txt").read_text()
    prompt, spent = base, 0
    for n in range(1, attempts + 1):
        code, tokens = ask(prompt)
        spent += tokens
        with tempfile.TemporaryDirectory() as tmp:
            work = pathlib.Path(tmp) / "repo"
            subprocess.run(["cp", "-r", str(REPO), str(work)], check=True)
            (work / target).write_text(code)
            res = subprocess.run(["bash", str(task_dir / "check.sh")],
                                 cwd=work, capture_output=True, text=True)
        if res.returncode == 0:
            return {"task": task_dir.name, "tier": tier, "pass_at": n, "tokens": spent}
        prompt = f"{base}\n\nYour previous patch failed. Test output:\n{res.stdout[-1500:]}{res.stderr[-1500:]}"
    return {"task": task_dir.name, "tier": tier, "pass_at": None, "tokens": spent}

if __name__ == "__main__":
    tasks = [p for p in sorted(pathlib.Path(sys.argv[1]).iterdir()) if p.is_dir()]
    rows = [run_task(p) for p in tasks]
    for r in rows:
        print(json.dumps(r))
    for tier in sorted({r["tier"] for r in rows}):
        sel = [r for r in rows if r["tier"] == tier]
        wins = sum(1 for r in sel if r["pass_at"])
        print(f"{tier}: {wins}/{len(sel)} cleared, {sum(r['tokens'] for r in sel)} tokens", file=sys.stderr)

Reading the output honestly

Two columns matter and you should print both. pass_at tells you whether the tier is usable at all for that class of task. The token sum tells you what the retries actually cost, which is the only fair way to price a free allowance. If most tasks need three attempts, your effective budget is a third of the headline figure, and your review load is three patches instead of one.

Trial 2 — The queue trial

Capability testing happens with a single request in flight, which is not how you work. Send the same short prompt at concurrency 1, 4, and 8, then watch the 95th percentile instead of the mean — the mean hides exactly the stalls that break an agent loop.

# queue_probe.py — compare p95 at your real concurrency
import os, statistics, time
from concurrent.futures import ThreadPoolExecutor
from openai import OpenAI

client = OpenAI(base_url=os.environ["AGENT_BASE_URL"], api_key=os.environ["AGENT_API_KEY"])
MODEL = os.environ["AGENT_MODEL"]
PROMPT = "Return a 30-line Python function that deduplicates a list while preserving order, plus one paragraph of explanation."

def one(_):
    t0 = time.perf_counter()
    client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": PROMPT}])
    return time.perf_counter() - t0

def probe(k):
    with ThreadPoolExecutor(max_workers=k) as pool:
        lat = sorted(pool.map(one, range(k)))
    p95 = lat[max(0, int(round(0.95 * (len(lat) - 1))))]
    return {"concurrency": k, "p50_s": round(statistics.median(lat), 2), "p95_s": round(p95, 2)}

for k in (1, 4, 8):
    print(probe(k))

Now convert latency into a budget. If your loop makes three calls per task and your deadline slack is 90 seconds, each call gets 30 seconds. A p95 above that number means the tier cannot carry parallel work, no matter how free the tokens are.

The decision table

Combine both trials and the choice usually falls out on its own.

Task bar	Queue at your concurrency	Verdict
T1–T2 clear, T3 fails	p95 within slack	Free tier for T1–T2, paid or self-hosted for T3+
All tiers clear	p95 within slack	Free tier as default, keep a local model for offline and private work
All tiers clear	p95 above slack	Free tier for batch work, paid or local for anything interactive
Only T1 clears	any	Not a coding tier for you; fine for boilerplate and docstrings
Nothing clears	any	Stop — the allowance is not the constraint, capability is

Notice what the table does not contain: a recommendation to move everything. The most common honest answer is a split, because your task mix is already a split. Self-hosting earns its keep on the tier your free access cannot reach, not on the whole backlog.

Where MonkeyCode fits in this workflow

MonkeyCode is an open-source coding agent that offers free model access and a free server option, which makes it a low-cost place to run both trials before you spend anything on the alternatives. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server option matters more than it first sounds, because a queue probe run from your laptop measures your home network rather than the endpoint. The operator quotes a 10-million-token free allowance at the time of writing; treat that as a moving number, verify the current terms yourself, and never plan a quarter around a figure you have not re-read this month. Free access changes what a failed trial costs you — nothing — but it does not change what a failed task bar means for your deadline.

Who should not use this approach

Be honest about the boundary before you invest an afternoon in it.

Teams that cannot send source outside a controlled network. Both trials assume you may post code to an external endpoint; if that is not allowed, measure a local model instead.
Anyone without a merged-PR history or a person to write the checks. The harness is only as sharp as its check.sh files, and vague checks produce flattering results.
Sub-second interactive workflows. Any queue at all violates the requirement, so concurrency testing is beside the point.
Tasks that exceed the tier's context window. The harness will fail them for the wrong reason, and you will misread the task bar.
People who read free as a service-level agreement. These trials measure today; they do not promise next quarter.

What I would do differently next time

I would write the checks before I write the prompts, because the first version of every check I have drafted was too generous. I would also keep the trial folder in the repository, so the next person who asks "can we just use the free tier?" gets a table instead of an opinion. And I would re-run both trials after any tier change, since the number you measured in October is a historical fact by November.

If you run this week, run it against two tiers rather than one — the comparison is where the lesson lives. I would genuinely like to know what your task bar turned out to be, because that single number is more useful to the rest of us than any published price list.

The Spend-Curve Test: Free Token Allowance or Self-Hosted, Decided With a Replay Harness

Riley Li — Tue, 20 Oct 2026 09:52:11 +0000

Most developers compare a free token allowance with a self-hosted box by lining up monthly prices, and that comparison quietly answers the wrong question. The number you actually need is the shape of your cumulative spend, because a fixed allowance survives a steady workload and evaporates under a bursty one. So here is my conclusion up front: replay your own prompt set, plot tokens against the allowance, and let the crossover day make the hosting decision for you. Everything below is a decision guide around one small, runnable harness — roughly twenty minutes of work, and no benchmark folklore required.

1. The question a price tag cannot answer

A free allowance is a fixed budget of tokens, and budgets do not fail linearly. One afternoon of agent retries can eat a week of your planned headroom, and then you are debugging a 429 instead of shipping. Self-hosting has the opposite failure mode: a fixed hardware bill that is generous on idle days and punishing when your queue is empty. Neither failure is visible in a per-token price, which is why so many "free vs paid" comparisons collapse the moment real traffic arrives. What we want is a curve, and curves are cheap to measure.

2. Build the spend-curve harness in four steps

The harness replays a JSONL file of prompts against any OpenAI-compatible chat endpoint and records the usage block from each response. It uses only the Python standard library, so there is nothing to install and nothing to pin.

Step 1 — describe your workload as prompts, with a simulated day per line. The day field lets you model bursty traffic instead of pretending every day looks the same.

{"day": 1, "id": "summarize-pr-1", "prompt": "Summarize this diff in five bullets: ..."}
{"day": 1, "id": "summarize-pr-2", "prompt": "Summarize this diff in five bullets: ..."}
{"day": 4, "id": "refactor-call", "prompt": "Rewrite this function to accept a cancellation token: ..."}

Step 2 — write the replay script. It stops on the first 429 rather than retrying around a rate limit, because a retry loop turns a measurement into a traffic incident.

#!/usr/bin/env python3
"""spend_curve.py - replay prompts, record tokens, report the crossover day."""
import csv, json, os, sys, urllib.error, urllib.request

BASE = os.environ["BASE_URL"].rstrip("/")
KEY = os.environ.get("API_KEY", "")
MODEL = os.environ["MODEL"]
ALLOWANCE = int(os.environ.get("ALLOWANCE", "0"))


def call(prompt: str, timeout: int = 90) -> dict:
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": 512,
    }).encode()
    req = urllib.request.Request(
        f"{BASE}/chat/completions",
        data=body,
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {KEY}"},
    )
    with urllib.request.urlopen(req, timeout=timeout) as resp:
        return json.load(resp)


def tokens(payload: dict):
    usage = payload.get("usage")
    if not usage:
        return None  # fail loudly; never guess a metered number
    return usage["prompt_tokens"] + usage["completion_tokens"]


cumulative, rows, crossover = 0, [], None
with open(sys.argv[1]) as fh:
    for line in fh:
        item = json.loads(line)
        try:
            payload = call(item["prompt"])
        except urllib.error.HTTPError as err:
            if err.code == 429:
                print(f"rate limited at {item['id']}; stopping", file=sys.stderr)
                break
            raise
        spent = tokens(payload)
        if spent is None:
            sys.exit("endpoint returned no usage block; use a metered route")
        cumulative += spent
        rows.append([item["day"], item["id"], spent, cumulative])
        if crossover is None and ALLOWANCE and cumulative > ALLOWANCE:
            crossover = item["day"]

with open("spend_curve.csv", "w", newline="") as fh:
    csv.writer(fh).writerows([["day", "request", "tokens", "cumulative"], *rows])
print(f"total={cumulative} allowance={ALLOWANCE} crossover_day={crossover}")

Step 3 — run it against each candidate route with identical inputs. Same prompt file, same output ceiling, so the only variable left is the endpoint itself.

BASE_URL=https://your-endpoint.example/v1 API_KEY=... MODEL=... \
ALLOWANCE=10000000 python3 spend_curve.py prompts.jsonl

Step 4 — sanity-check the meter before you trust the curve. Send three identical prompts and confirm the token counts are stable; if they wobble, your template is injecting something you did not intend. A second caveat worth knowing: some newer models want max_completion_tokens instead of max_tokens, and some gateways silently ignore both. Read the endpoint's documentation and reconcile the usage block against your own estimate before drawing conclusions.

3. Read the curve: three shapes, three decisions

Once you have spend_curve.csv, aggregate by day and look at the slope rather than the total. I care about three shapes, and each one points somewhere different.

Steady slope, low daily volume: the allowance lasts for months, and self-hosting is pure overhead you would be paying for out of pride.
Steady slope, high volume: crossover arrives inside your planning window, so you need a costed second route designed before day one.
Spiky slope with long idle gaps: the allowance is either plenty or gone by lunchtime, and a capped queue is safer than either extreme.

Turning that into a fit decision gives a table I would genuinely paste into a design document, because it forces the tradeoffs into the open.

Signal	Free allowance fits when	Self-hosted fits when	Paid hosted fits when
Volume vs budget	measured crossover lands past your planning horizon	you keep the box busy most days	demand is bursty and ops appetite is low
Data residency	prompts carry no regulated data	data cannot leave your network	vendor terms are already approved
Latency tail	p95 is acceptable for interactive use	you can tune and own the tail	you need a contracted SLA
Ops capacity	nobody on the team wants to patch drivers	someone owns the machine and the on-call	you would rather buy uptime
Iteration speed	you are still rewriting prompts daily	the workload is stable and understood	the workload changes but budget exists
Exit cost	the API is OpenAI-compatible and swappable	hardware holds resale value	the contract term is short

4. Where a free tier actually earns its place

The operator of the project I used for this workflow describes two free-tier options: free model access with a stated allowance of ten million tokens, and a free server option for running work off your laptop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I treat both as availability claims to verify on the project's own page rather than permanent guarantees, because free tiers move and allowances get adjusted. What makes them useful for this harness is narrower and more honest: the free route is the cheapest place to collect your first spend curve, and the free server is a reasonable place to park the nightly replay job.

Does that mean you should build a production path on a free allowance? No, and I would push back on anyone who says so without a measured crossover day. The free route is a measuring instrument and a prototyping surface; the moment your curve crosses the allowance in week one, you have learned something more durable than a discount.

5. Limitations and who should skip this

This harness measures tokens, not output quality, so pair it with a small acceptance check before you trust any route with real work. It also assumes your endpoint returns a usage block, which some proxies strip; without that block you are estimating, and estimates should not drive a hosting decision. Finally, the curve is only valid for the prompt set you replayed — change the template and the number changes with it, which is why the JSONL file belongs in the repository.

Skip this approach if a compliance rule forbids third-party inference entirely, because then the decision is already made and the curve is trivia. Skip it too if your workload is one predictable daily batch that fits on a napkin, or if you cannot spare half a day to write realistic prompts, since a curve built from toy inputs is worse than no curve at all.

If you do run it, commit the CSV next to the decision it justified. Six weeks from now, when someone asks why the agent runs where it runs, that file is the answer — and if you want a zero-cost place to generate the first version of it, the free route is a sensible starting line.

Run an Endpoint Portability Drill Before You Commit to a Free Hosted Model

Riley Li — Mon, 19 Oct 2026 08:32:50 +0000

Free hosted model access looks like the cheapest decision you will make this quarter, and that is exactly why it deserves a test. The cost you should be afraid of is not the invoice, it is the afternoon you lose when the endpoint you built around stops fitting. So here is my rule: measure your exit cost before you measure anything else. If you cannot move the workflow to another backend in one sitting, you have not saved money, you have bought a lease.

This post is a decision guide, not a product review. It gives you a small reproducible drill that scores a hosted endpoint, a self-hosted endpoint, and a paid endpoint on the same frozen task suite, using the same code path. Run it once, keep the numbers, and re-run it when the terms or your workload change. Fair warning: the scripts below are a template I have not executed against any live endpoint, so treat them as scaffolding you fill in with your own credentials and your own tasks.

The decision is a seam, not a vendor

Most teams evaluate endpoints by asking which one is cheaper per call, and that question hides the expensive part of the answer. The expensive part is how many files you touch when the cheap option changes its limits, its model list, or its availability. So before comparing anything, put one seam between your agent logic and the network call, and let an environment variable choose the backend behind that seam. Then the comparison becomes a configuration change instead of a refactor.

Step 1: Write the seam once

Keep it boring, keep it dependency-free, and keep it in one file that nothing else imports around. The point is that every backend speaks the same OpenAI-compatible request shape, so the drill can swap base URLs without touching prompts.

# seam.py - the only file that knows a backend exists
import json, os, time, urllib.request

BASE = os.environ["AGENT_BASE_URL"].rstrip("/")  # e.g. https://host/v1
KEY = os.environ["AGENT_API_KEY"]
MODEL = os.environ["AGENT_MODEL"]

def complete(prompt: str, schema: dict, timeout: int = 90) -> dict:
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "response_format": {"type": "json_schema",
                            "json_schema": {"name": "task", "schema": schema}},
        "temperature": 0,
    }).encode()
    req = urllib.request.Request(
        f"{BASE}/chat/completions",
        data=body,
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {KEY}"},
    )
    t0 = time.monotonic()
    with urllib.request.urlopen(req, timeout=timeout) as resp:
        payload = json.load(resp)
    return {"latency_s": round(time.monotonic() - t0, 2),
            "raw": payload["choices"][0]["message"]["content"]}

Notice what the seam refuses to do. It does not retry, it does not fall back to a second provider, and it does not cache, because those behaviors would blur the comparison you are trying to make. Retries and fallbacks belong in the layer above the drill, not inside it.

Step 2: Freeze a suite you can replay

A comparison is only reproducible if the inputs never move, so serialize twenty real tasks to JSONL and commit them next to a strict JSON schema. Pick tasks that already appear in your review queue, because synthetic prompts will happily pass a schema your production prompts fail. Keep the schema tight enough that a malformed answer is obviously malformed.

{"id": "rename-01", "prompt": "Return a JSON patch that renames parse_rows to parse_records across src/parser.py"}
{"id": "test-02", "prompt": "Return a JSON patch that adds a regression test for empty input in tests/test_parser.py"}
{"id": "doc-03", "prompt": "Return a JSON patch that documents the retry parameter added in the last commit"}

Step 3: Score three numbers, not one

Most comparisons stop at latency and quality, which is two numbers, and the missing third number is the one that decides the lease. Write the runner so it records schema adherence, wall-clock latency, and a manual acceptance flag you set during review. Then compute per-task agreement between two runs to see whether the cheaper backend is merely slower or genuinely different.

# drill.py - replay the frozen suite, one JSON row per task
import json, sys
from pathlib import Path
from seam import complete

SCHEMA = json.loads(Path("task.schema.json").read_text())
rows = []
for line in Path("suite.jsonl").read_text().splitlines():
    task = json.loads(line)
    score, out = 0, {"latency_s": None, "raw": ""}
    try:
        out = complete(task["prompt"], SCHEMA)
        score = 1 if json.loads(out["raw"]).get("patch") else 0
    except Exception as exc:
        print(f"{task['id']}: FAIL {type(exc).__name__}", file=sys.stderr)
    rows.append({"id": task["id"], "score": score, **out})
Path(sys.argv[1]).write_text("\n".join(json.dumps(r) for r in rows))

Run it twice against different backends and diff the results, which takes about as long as making coffee.

AGENT_BASE_URL="$HOSTED_URL" AGENT_API_KEY="$HOSTED_KEY" \
AGENT_MODEL="$HOSTED_MODEL" python drill.py run-hosted.jsonl

AGENT_BASE_URL="http://127.0.0.1:8080/v1" AGENT_API_KEY="local" \
AGENT_MODEL="$LOCAL_MODEL" python drill.py run-local.jsonl

python compare.py run-hosted.jsonl run-local.jsonl

The three numbers you take away are schema pass rate, p50 and p90 latency, and accepted-patch rate after your own review. None of those numbers is portable across teams, which is the whole reason you measure them yourself instead of reading a leaderboard.

Read the numbers against a fit table

Once the drill has run, the decision is mostly mechanical, and a table keeps the argument honest. Fill in your own thresholds, but do not change the row labels, because those are the constraints that actually force a choice.

Constraint	Free hosted access fits when	Self-hosted fits when	Paid hosted fits when
Data class	Prompts are already public or synthetic	Code cannot leave your network at all	Contract and region terms cover the data
Burst shape	Low concurrency, tolerant of queueing	You own spare hardware already	Traffic is spiky and deadline-bound
Model pinning	You can accept the current model moving	You need a specific weight hash	Vendor offers version pinning
Review capacity	A human reads every patch anyway	You have ops time, not review time	You need an SLA on time-to-first-token
Exit cost	Under 200 changed lines to switch	You can rebuild the image in a day	Contract exit clause is short

Two rows deserve emphasis. The data-class row is a hard gate rather than a tradeoff, so a free tier never wins it by being cheap. The exit-cost row is the one people skip, and it is measurable: run git diff --stat against the seam directory after the drill and count the lines you would have to rewrite to leave.

Where a free tier earns a seat

MonkeyCode offers free model access and a free server option, and both claims matter for different rows of that table. The hosted access shortens the time between "we need an agent here" and "we have one running", which is the burst-shape row. The server option covers the self-hosted row for teams that want the code path to stay inside their own boundary without buying hardware first. I would still point the drill at both before trusting either one with a release path, and I would check the current allowance and hardware in the console, because free-tier terms move faster than blog posts do.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

If your suite passes schema on the hosted endpoint and your seam is thin, you have a cheap place to start. If your suite fails schema on every backend, the problem is your prompt contract, and no price list will fix it.

Limits, and who should skip this

This drill tells you about portability and schema behavior, and it tells you nothing about throughput under concurrency, long-context behavior, or cost at ten times your current volume. Re-run it with a concurrency parameter before you scale, because a backend that is fine at one request is not automatically fine at forty.

Skip this approach entirely if you handle regulated data with no approved free-tier path, if you need a pinned weight hash for reproducibility, or if your team has no capacity to review outputs at all. In those cases the drill will produce numbers you are not allowed to act on, which is worse than no numbers. Also skip it if you need a contractual SLA today, because a measurement cannot manufacture one.

So before you pick the free option or the self-hosted one, ask yourself a blunt question. If the endpoint disappeared tomorrow morning, would your diff be small?

Cost per Accepted Diff: The Number to Compare Free, Paid, and Self-Hosted Inference

Riley Li — Sun, 18 Oct 2026 07:13:32 +0000

The cheapest inference seat is almost never the one with the lowest sticker price; it is the one with the lowest cost per accepted diff, measured on your own task set before you commit. If you have spent the last month watching your team juggle three AI tools and still argue about which one pays for itself, this post is for you. Two of the most-discussed DEV posts in the past week circled the same fatigue from a cultural angle, and I want to answer it with arithmetic instead of vibes. So let me ask the uncomfortable question first: when did you last measure what an AI seat actually produces per dollar?

Below is the workflow I use: a fixed task set, a small harness you can run against any OpenAI-compatible endpoint, a review log, and a decision table that tells you when self-hosting or a paid seat is genuinely worth it. The code is a template, not a benchmark result — I am not going to hand you numbers from someone else's repo and pretend they transfer to yours.

Why the free-versus-self-hosted argument goes in circles

Most comparisons die because each side measures a different thing: one team quotes a monthly price, another quotes GPU-hours, a third quotes latency. None of those numbers survive contact with the only thing that matters, which is how many changes you accept after review. The result is a debate that never converges, and a pile of subscriptions nobody can defend at budget time. Isn't that exactly the fatigue everyone is writing about?

The fix is to make the unit of comparison the accepted diff, with review time folded in as a real cost. Review time is the part people forget, and it is usually the largest cost in the loop once a model is competent enough to be useful. A seat that produces twice as many usable patches but triples your review minutes is a worse seat, not a better one.

The metric, written down

Here is the definition I use, and it is deliberately boring:

cost_per_accepted_diff =
  (prompt_tokens * in_price + completion_tokens * out_price)  # metered spend
  + (review_minutes / 60 * loaded_hourly_rate)                # human spend
  + (setup_hours / amortization_months)                       # fixed spend
  / accepted_diffs                                            # denominator

Every term is an input you can defend in a budget meeting, and every term is something you can log mechanically except accepted. That one needs a human, which is why the workflow below spends most of its effort on a review step rather than on prompt tricks. Yes, it takes an afternoon — and no, you cannot get this from a leaderboard.

Step 1: Freeze a task set of 20 real changes

Pull twenty tasks from your own backlog, not from a public eval: six small (single function or config edit), ten medium (one file plus a test), and four deliberately nasty cases your current tool gets wrong. Store each as a JSON object with an id, the prompt you would actually type, and the files it may touch. Keep the set frozen for at least a quarter so that later runs are comparable, because a task set you keep editing measures nothing.

The four nasty cases matter more than the sixteen easy ones, because they are what separates a tool you keep from a tool you cancel. If you have no nasty cases handy, use the last four PRs that needed a second review round. Write them down before you run anything, or you will unconsciously grade in favor of whichever seat you already paid for.

Step 2: Run one harness against every seat

Any seat that speaks the OpenAI chat-completions shape can be driven by the same script, which is the whole reason this comparison is cheap to run. Save each output to disk, because your reviewer needs to read the diff, not your terminal scrollback.

# seatbench.py - one fixed task set, any OpenAI-compatible endpoint
import csv, json, time, urllib.request
from pathlib import Path

def ask(base, key, model, prompt, timeout=120):
    req = urllib.request.Request(
        base.rstrip("/") + "/chat/completions",
        data=json.dumps({"model": model, "temperature": 0,
                         "messages": [{"role": "user", "content": prompt}]}).encode(),
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {key}"})
    t0 = time.monotonic()
    with urllib.request.urlopen(req, timeout=timeout) as r:
        body = json.loads(r.read())
    usage = body.get("usage", {})
    return body["choices"][0]["message"]["content"], usage, time.monotonic() - t0

tasks = json.loads(Path("tasks.json").read_text())
seats = json.loads(Path("seats.json").read_text())  # label, base_url, model, key_env
writer = csv.writer(open("results.csv", "w", newline=""))
writer.writerow(["seat", "task", "tok_in", "tok_out", "latency_s",
                 "accepted", "review_min", "outfile"])

for seat in seats:
    import os
    key = os.environ[seat["key_env"]]
    for t in tasks:
        out, usage, dt = ask(seat["base_url"], key, seat["model"], t["prompt"])
        path = f"out/{seat['label']}-{t['id']}.md"
        Path(path).parent.mkdir(exist_ok=True)
        Path(path).write_text(out)
        writer.writerow([seat["label"], t["id"], usage.get("prompt_tokens", ""),
                         usage.get("completion_tokens", ""), round(dt, 1),
                         "", "", path])

seats.json is just a list of endpoints, so the same loop covers a hosted free tier, a paid API, and your own served model. Set the key through an environment variable and never inline it, and keep the whole directory out of your repo's commit history.

Step 3: Make a human score every output

A reviewer opens each output file, applies it to a scratch branch if it looks plausible, and fills two columns: accepted (y/n) and review_min (minutes spent). Reject anything that does not apply cleanly, because a diff you had to rewrite by hand is not a win for the tool. Time your reviews with a stopwatch for the first run; estimations drift toward round numbers and flatter your favorite.

This step is where most comparisons become honest, and it is also where free tiers often lose ground they looked like they were winning. Cheap tokens that generate almost-good patches convert into expensive review minutes. Conversely, a slow, metered seat that returns one correct diff may beat everyone on the only number that survives the quarter.

Step 4: Compute and compare

Read the CSV back, fill in your own price per million tokens for each seat (zero is a legitimate value), and print the composite.

import csv
from collections import defaultdict

PRICE_PER_MTOK = {"free-tier": 0.0, "paid-api": 3.0, "self-host": 0.0}
LOADED_RATE = 85.0  # your fully loaded hourly cost, not salary

rows = list(csv.DictReader(open("results.csv")))
by_seat = defaultdict(list)
for r in rows:
    by_seat[r["seat"]].append(r)

for seat, rs in by_seat.items():
    acc = [r for r in rs if r["accepted"] == "y"]
    mtok = sum(int(r["tok_in"] or 0) + int(r["tok_out"] or 0) for r in rs) / 1e6
    review = sum(float(r["review_min"] or 0) for r in rs) / 60
    spend = mtok * PRICE_PER_MTOK[seat] + review * LOADED_RATE
    print(f"{seat:12} accepted={len(acc):2} spend={spend:7.2f} "
          f"per_accepted={spend / max(len(acc), 1):7.2f}")

For self-hosting, add a break-even line instead of a token price: monthly_host_cost / cost_per_accepted_elsewhere gives the number of accepted diffs you must produce per month before the fixed cost is justified. If that number exceeds your realistic monthly diff volume, self-hosting is a hobby, which is fine as long as you call it one.

Step 5: Decide with fit criteria, not with the ranking

The lowest cost per accepted diff is necessary but not sufficient, so put the candidates through a second filter before you sign anything.

Option	Pick it when	Refuse it when
Free tier / free server	You need a zero-commitment seat for a 20-task trial, or your volume is spiky and low	Your data cannot leave your boundary, or your latency budget is tight
Paid API	Accepted-diff cost is stable and your volume is predictable	Your tokens per accepted diff drift upward without a re-measure
Self-hosted	You already run capacity, and break-even diff volume is clearly inside your monthly reality	Nobody owns the serving stack after launch

This is where a free option earns its place in a serious workflow rather than as a novelty: it lets you run the whole five-step measurement with no procurement cycle. MonkeyCode is a project in that space, offering free model access and a free server option; the operator states a free allowance of ten million tokens at the time of writing. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I have not independently verified the current quota or server terms, and quotas of that kind move, so read the pricing page before you design around a number.

Step 6: Set a re-measure trigger

A decision made in October should not silently govern January, so write down the conditions that force a re-run. Re-measure when a model behind any seat changes, when your task mix shifts by more than a fifth, or when review minutes per accepted diff move by 25 percent in either direction. Add one calendar checkpoint per quarter and put the CSV in the same place each time, or the comparison quietly becomes folklore.

Limitations and who should skip this

Twenty tasks produce a direction, not a confidence interval, and the four nasty cases dominate the denominator more than they should. Free tiers and hosted models change under you, so any number here has a shelf life measured in weeks. Human review timing is noisy, and a single slow afternoon can move a seat's ranking. If your weekly diff volume is under ten, or your data cannot touch a hosted endpoint at all, skip the framework and just pick the option your compliance team will actually approve.

What would change your mind about the seat you are paying for right now — and could you prove it with a CSV by Friday?

Decide Where the Agent Runs With a Four-Axis Fit Ledger

Riley Li — Fri, 16 Oct 2026 05:16:25 +0000

Every week brings another round of confident model announcements, and every week my review queue looks roughly the same as it did a month earlier. That gap is not a scandal and it is not a mystery to me; it is a routing problem that most teams never name out loud. We keep asking which model is best when the operationally useful question is which compute tier this specific unit of work should run on. I have been answering that question with a four-axis ledger instead of a price tag, and what follows is the entire workflow with runnable code.

Three tiers show up in that ledger: a free hosted tier, a paid hosted tier, and self-hosted compute. A concrete example of the first is MonkeyCode's free model access, which the operator describes as including free model access plus a free server option, with a free allowance reported at ten million tokens. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I have not audited those numbers, so treat them as claims to verify inside your own account rather than constants to plan around.

Conclusion First: Price Is the Last Question, Not the First

When a free tier appears, the instinct is to route everything to it and celebrate the savings on Monday. That instinct fails for a boring reason: the expensive part of agent work was never the token bill, it was the human time spent catching bad output. A free generation that costs me forty minutes of review is more expensive than a paid generation I can trust on the first pass.

So I stopped optimizing for price and started optimizing for fit. Fit means the tier matches the data boundary, the reversibility of the output, the patience of the waiting human, and the blast of a missed error. Price is a derived quantity that falls out of those four answers, not an input you start from.

So why do so many teams still open the pricing page first? Because price is the only number that arrives pre-formatted on a marketing site. The other three axes require you to measure your own work, which is slower and far more honest.

The Four Axes That Actually Decide the Route

Egress class. Can the prompt and its context leave your boundary, or does it contain internal and regulated material? This axis has veto power over every other consideration.
Output reversibility. If the result is wrong, can you delete it with no cleanup, or does it land in a branch, a ticket, or a production path? Throwaway output tolerates a weaker tier.
Latency budget. How long will a human actually wait before they switch windows and lose the thread? This is a measurement, not a feeling.
Cost of a wrong answer. What happens if the error ships and nobody catches it before Friday? High cost buys review, not a fancier model.

Notice that nothing on that list mentions price per million tokens. That is deliberate, because price only becomes meaningful after the four axes have already narrowed the field.

Step 1: Write the Policy Before You Write the Router

I keep the policy in a file so the arguments happen in review rather than inside my head at midnight. The thresholds below are placeholders, and you should replace each one with your own measured numbers.

# policy.yaml
axes:
  egress_class: [public, internal, regulated]
  reversible: [true, false]
  latency_budget_s: [interactive_10, batch_600]
  wrong_answer_cost: [low, medium, high]

routes:
  - when: {egress_class: regulated}
    tier: self_hosted
    note: 'boundary rule, no hosted call at any price'
  - when: {egress_class: internal}
    tier: self_hosted
  - when: {egress_class: public, reversible: true, wrong_answer_cost: low}
    tier: free_hosted
    note: 'output gets discarded, nothing downstream depends on it'
  - when: {egress_class: public, latency_budget_s: interactive_10}
    tier: paid_hosted
    note: 'measure your own p95 before you trust this line'
  - default: true
    tier: paid_hosted

Step 2: Implement the Router as a Pure Function

The router should be boring, deterministic, and easy to test, which means no network calls and no hidden state. Here it is as a single file with no dependencies beyond the standard library.

# route.py
from dataclasses import dataclass
from enum import Enum
from typing import Literal

class Tier(str, Enum):
    FREE_HOSTED = 'free_hosted'
    PAID_HOSTED = 'paid_hosted'
    SELF_HOSTED = 'self_hosted'

Egress = Literal['public', 'internal', 'regulated']
Cost = Literal['low', 'medium', 'high']

@dataclass(frozen=True)
class Task:
    name: str
    egress_class: Egress
    reversible: bool
    latency_budget_s: int
    wrong_answer_cost: Cost

def route(task: Task) -> tuple[Tier, str]:
    if task.egress_class in ('regulated', 'internal'):
        return Tier.SELF_HOSTED, 'rule 1: the data boundary beats every other axis'
    if task.reversible and task.wrong_answer_cost == 'low':
        return Tier.FREE_HOSTED, 'rule 2: throwaway output over public data'
    if task.latency_budget_s <= 10:
        return Tier.PAID_HOSTED, 'rule 3: interactive budget, check your own p95'
    return Tier.PAID_HOSTED, 'rule 4: default until you have evidence to move'

Two things deserve emphasis here. First, the first matching rule wins, and I ordered them so that the unglamorous boundary rule can never be overridden by a clever saving. Second, every return carries a reason string, because a router that cannot explain itself is impossible to debug when it misroutes.

Step 3: Split Authoring From Verification

Here is the part that changed my workflow the most: authoring and verification do not need the same tier. Authoring is probabilistic and touches your context, while verification is deterministic and touches your repository.

That split means I can run generation on whichever tier the axes allow and send the test suite somewhere cheaper, as long as the repository itself is clean. Verification on a disposable workspace is exactly the kind of job a free server option fits, because the output is a pass or fail rather than a mergeable artifact.

def route_verification(task: Task, repo_has_secrets: bool) -> tuple[Tier, str]:
    if task.egress_class != 'public':
        return Tier.SELF_HOSTED, 'verification still crosses the same boundary'
    if repo_has_secrets:
        return Tier.SELF_HOSTED, 'build logs can echo environment state'
    return Tier.FREE_HOSTED, 'deterministic job, disposable workspace, no secrets'

That guard clause is not paranoia; build logs leak more credentials than anyone expects. Run a coarse preflight before you hand a repository to any hosted runner.

# coarse preflight, not a replacement for a real secret scanner
git grep -nIE '(AKIA[0-9A-Z]{16}|-----BEGIN [A-Z ]*PRIVATE KEY-----)' -- . \
  && echo 'key-like material in tracked files: stop and self-host' \
  || echo 'no obvious key material found in tracked files'

Step 4: Prove the Router With Labeled Tasks

A policy file that only lives in your head drifts within two weeks, so I pin expectations in a test. These cases represent my labeled decisions, not universal truths, and the suite protects consistency rather than correctness.

# test_route.py
import pytest
from route import Task, Tier, route, route_verification

CASES = [
    (Task('summarize internal incident', 'internal', True, 5, 'low'), Tier.SELF_HOSTED),
    (Task('rename vars in sample repo', 'public', True, 5, 'low'), Tier.FREE_HOSTED),
    (Task('rewrite prod migration', 'public', False, 900, 'high'), Tier.PAID_HOSTED),
    (Task('generate throwaway fixtures', 'public', True, 600, 'low'), Tier.FREE_HOSTED),
]

@pytest.mark.parametrize('task,expected', CASES)
def test_route(task, expected):
    tier, reason = route(task)
    assert tier is expected, reason

def test_verification_refuses_dirty_repo():
    task = Task('run unit tests', 'public', True, 600, 'low')
    assert route_verification(task, repo_has_secrets=True)[0] is Tier.SELF_HOSTED

Run it with pytest -q. When a case fails after a policy edit, you get to argue with your past self instead of guessing, which is the entire point of writing the policy down in the first place.

Step 5: Change Rules, Not Cases

Every misroute gets one line in a log, because patterns only appear after you stop explaining incidents away. The log is plain JSON Lines so the Friday review ritual stays cheap and boring.

printf '%s\n' '{"task":"fixture rewrite","chosen":"free_hosted","problem":"output was merged, not discarded","fix":"make reversible an explicit input"}' >> misfits.jsonl
jq -r '.fix' misfits.jsonl | sort | uniq -c | sort -nr | head

The rule I follow is simple: if the same fix shows up twice in a month, I amend the policy instead of adding a third exception. Exceptions rot quietly, while a changed rule is visible to everyone who reads the file. My review hour is short because most of the questions were already answered on Monday.

Where Free Access and a Free Server Fit

The free tier earns its place in route two: public data, throwaway output, low wrong-answer cost. Fixture generation, sample-data munging, docstring drafts, and "explain this stack trace" all land there comfortably, because a bad answer costs me nothing but a rerun.

The free server option fits the verification half rather than the authoring half, which surprised me at first. Deterministic test runs, lint passes, and build checks produce no mergeable artifact, so there is nothing to review and nothing to regret. Keep credentials out of the workspace and the boundary question mostly answers itself.

What I do not do is pretend the free tier is an availability guarantee. Operator-supplied allowances, quotas, and server terms change, and any workflow that breaks when they change was never a workflow; it was a bet.

The Decision Table I Use

Situation	Route	Reasoning
Regulated or internal data	Self-hosted	The boundary rule outranks every price argument
Public data, throwaway output, low error cost	Free hosted	Nothing downstream depends on the result
Public data, output gets merged, medium or high error cost	Paid hosted plus human review	You are buying accountability, not tokens
Interactive loop under ten seconds	Paid hosted	Replace this row with your own measured p95
Deterministic verification, no secrets in the repo	Free server	Disposable workspace, repeatable result
Credentials in tracked files	Self-hosted	Build logs can echo environment state

Limitations and Who Should Skip This

The thresholds in my policy are placeholders calibrated against my own task mix, so yours will differ and probably by a wide margin. The router also cannot see your repository, which means the secret preflight stays a separate step that a human has to remember.

Free-tier availability, quotas, and free server terms are operator-supplied claims here rather than promises, so I keep a manual override for the days when a tier misbehaves. Self-hosting is not free either; it trades a bill for maintenance time, patching, and someone who gets paged at an inconvenient hour.

Who should skip this? If you run one task type against one repository, the ledger is ceremony, and a single documented default will serve you better. If your team has no review process, routing will not save you, because the fourth axis is unknowable until someone is accountable for misses. And if a compliance review of hosted egress has not finished, stay self-hosted and revisit the table later.

What I Would Do Next

Start by labeling ten real tasks from last week against the four axes, then route them by hand before you write any code. You will probably find that two rules cover most of your work and that the free tier is bigger than you expected. If you want a hosted free tier to pressure-test that third route against your own task list, that is a cheap first stop before committing budget anywhere else.

Write the Fallback Path Before You Choose Free Model Access

Riley Li — Thu, 15 Oct 2026 11:25:35 +0000

Here is my conclusion before any comparison table: the question "free, paid, or self-hosted?" is really the question "what does my code do during the thirty seconds when the answer is no?" I have watched teams argue over per-million-token pricing for a week while their retry logic was a bare while True loop with no ceiling and no fallback. Pick the failure behaviour first, and the seat choice becomes almost mechanical.

Free model access is attractive for exactly the reason it is awkward: the capacity is not yours, so you cannot promise anything downstream about it. That is not a defect in any particular vendor, it is a property of anything you do not pay to reserve. The engineering response is not to distrust free tiers, but to make a throttled primary a normal branch in your code instead of an incident report.

I am using MonkeyCode here as the free end of this comparison, since its free model access and its free server option are the two things I can point at without guessing.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Quotas, hardware and terms change over time, so verify the current offer yourself before you build a plan on it; this post relies only on the operator-supplied claims that free model access and a free server option exist.

Three questions that beat any price table

Before you open a pricing page, write down answers to three questions and put them in the repo, because they decide this faster than any spreadsheet. Who is on the hook at 2:00 a.m. when the endpoint refuses you? What does the user see when a request is throttled — a spinner, a degraded answer, or a refund request? How long can the queued work survive a cold stretch of capacity before the answer stops being useful?

If your honest answers are "me, a degraded answer, and about thirty seconds", you are describing a workload that tolerates free capacity as long as a fallback exists. If they are "a customer, an outage, and never", you are describing a workload that needs a reserved commitment, and no free tier will rescue that. The comparison table is downstream of these answers, not a substitute for them.

Step 1 — Profile the workload in numbers you can defend

Write a small file before you compare anything, so that every later claim has something to be checked against. Mine usually looks like this workload.yaml.

workload: support-triage
calls_per_minute_peak: 40
median_prompt_tokens: 1800
max_prompt_tokens: 9000
latency_budget_ms: 2500
retries_allowed: 2
fallback: deterministic-rules        # must already exist before the primary dies
data_sensitivity: internal
must_run_offline: false

The peak rate sets the ceiling for your limiter, so measure it rather than naming a round number you like. The fallback line is the important one: if it names something you have not built yet, that is your first task, not your last. And data_sensitivity quietly decides whether self-hosting is a lifestyle preference or a hard constraint.

Step 2 — Put an admission controller in front of every call

This wrapper is the real deliverable, and it is small enough to write in an afternoon. It shapes your own traffic with a token bucket, retries with jitter, and returns which path served the request so you can count it later.

# admission.py
from __future__ import annotations

import random
import threading
import time
from dataclasses import dataclass, field


class Retryable(Exception):
    """Raised when the primary refuses us; safe to retry elsewhere."""


@dataclass
class TokenBucket:
    rate: float                     # permits per second, kept below your observed limit
    burst: float                    # how large a spike you are willing to queue
    tokens: float = field(init=False)
    lock: threading.Lock = field(default_factory=threading.Lock)

    def __post_init__(self) -> None:
        self.tokens = self.burst
        self._last = time.monotonic()

    def take(self, n: float = 1.0, timeout: float = 5.0) -> bool:
        deadline = time.monotonic() + timeout
        while True:
            with self.lock:
                now = time.monotonic()
                self.tokens = min(self.burst, self.tokens + (now - self._last) * self.rate)
                self._last = now
                if self.tokens >= n:
                    self.tokens -= n
                    return True
            if time.monotonic() >= deadline:
                return False
            time.sleep(random.uniform(0.05, 0.15))


def complete(prompt, *, primary, fallback, bucket, attempts=3):
    for attempt in range(attempts):
        if not bucket.take():
            break
        try:
            return primary(prompt), "primary"
        except Retryable:
            time.sleep((2 ** attempt) + random.random())
    return fallback(prompt), "fallback"

Notice what the caller never does: it never raises to the user, and it never waits forever on one endpoint. The bucket protects the primary from your own traffic spike, and its rate should be set conservatively below whatever limit you actually observe, because a guess is not a contract. The tuple return is deliberate — label every response with "primary" or "fallback" and you get a real signal about how much of your product is running on the free path.

Step 3 — Force the fallback with a chaos test

A fallback path that has never executed is a comment, so test it before you trust it in production. This test is short and it catches the exact bug that hurts: a fallback that is technically wired up but unreachable because an exception escapes first.

# tests/test_admission.py
from admission import Retryable, TokenBucket, complete


def test_fallback_is_reachable():
    calls = {"primary": 0}

    def primary(prompt):
        calls["primary"] += 1
        raise Retryable("throttled")

    def fallback(prompt):
        return "rules-only answer"

    bucket = TokenBucket(rate=100, burst=10)
    text, path = complete("hi", primary=primary, fallback=fallback, bucket=bucket)

    assert path == "fallback"
    assert text == "rules-only answer"
    assert calls["primary"] == 3

Run it with python -m pytest -q tests/test_admission.py and add one more assertion afterwards: that the bucket itself returns False after you drain it, so a runaway loop cannot hammer anything. Then, in staging, emit a counter with the path label and alarm on a fallback rate you would not accept; five percent might be normal, forty percent means you mis-sized the bucket.

Step 4 — Score the three options against your own profile

Now the comparison is boring, which is the point. Map each line of your workload.yaml onto one of three columns and keep the cells honest.

Signal from your profile	Free model access (e.g. MonkeyCode)	Paid managed API	Self-hosted
Who fixes a bad hour	You, through the fallback you wrote	Vendor on-call plus you	You, on your own box
Cost shape	Zero to start, wrapper is the investment	Metered and forecastable	Fixed hardware, spiky attention
Peak-rate ceiling	Whatever your own bucket allows	Raised by buying quota	Whatever your hardware allows
Sensitive data	Check current terms before sending anything	Contract terms	Stays inside your boundary
What you must build	Retries, fallback, path telemetry	Retries and telemetry	All of the above plus serving
Fits well when	Prototypes, internal tools, batch jobs that can wait	Customer-facing paths with a latency promise	Hard data-boundary or offline needs

The column that decides most real cases is the last lazy one: what you must build. Choosing free access is not choosing zero work, it is choosing to spend the work on resilience instead of on invoices, and that trade is worth it precisely when the fallback answer is acceptable to the user. Choosing self-hosting moves the same work to capacity planning and upgrades, which is a different job with a longer horizon.

Limitations and who should skip this

I would not run this pattern on a path where a degraded answer breaks a promise, such as billing actions or anything a customer paid for a specific latency on. I also would not recommend it to a team with no one available to own the retry budget, because an unowned limiter becomes a silent throttle that nobody notices until a launch. And if your data cannot leave a boundary you control, the free and paid columns are simply off the table, regardless of how good the wrapper is.

None of this depends on a number I cannot verify today, which is deliberate: capacity offers, quotas and machine specifications change, and a plan built on a stale figure is worse than no plan. What does not change is the shape of the code that survives a refusal.

If you want to try the free end of this comparison, MonkeyCode's project is open source and its free model access plus free server option are the parts worth testing first — bring the wrapper, and let the fallback path tell you whether the fit is real.

Pick a Free, Paid, or Self-Hosted Agent Seat by the Lease You Can End

Riley Li — Sun, 11 Oct 2026 21:34:48 +0000

The right agent seat is the one you can close without losing the only copy of the work. I would rather take a shorter paid lease, or run a box I operate, than keep a free chat I cannot audit. A seat is a temporary grant of model access, context, and side effects, not a teammate you quietly adopt for the week. If you cannot name the exit evidence before the first prompt, you have not chosen a tool yet.

Recent discussion keeps returning to a plain tension, even when the headlines change from week to week. Model quality can move in a week, while the release path does not automatically move with it. I am not treating any weekly roundup as evidence, because a trending title is only a topic signal. A topic signal is not a measurement I can cite, and it is not a reason to skip the lease card.

Start from the exit, not the invoice

Have you ever left a session open because closing it felt like throwing the reasoning away? That small hesitation is how a free hour becomes an unofficial path into the release branch. I now write the ending before I write the prompt, and I keep that ending in the repository rather than in a vendor tab. The invoice matters only after the ending is a file that a reviewer can open without me.

I have already used other gates for blast radius and review time, so this note does not rebuild those checks. The question here is narrower, and it is easy to dodge when the model feels helpful. Which seat can I actually end today, and which committed file proves that I ended it?

Ask four questions before the session exists

I use the same four questions for a bugfix, a documentation edit, and a spike that might be deleted. Each answer is a word or a small integer, because a paragraph of enthusiasm is not a decision. If any answer is mushy, I shrink the task until the answer fits on one line.

What data class may enter the prompt? I allow a public sample, internal code that contains no secrets, or a resident class that must stay on hardware I control. When the class is unclear, I stop, and I do not try a redacted version inside the same unbounded chat.
How many retries is the review time worth? I write an integer, usually one to three, and I stop when the counter reaches zero. Another retry without a new failing test is just a slower guess, even when the seat did not charge me.
Which artifact leaves with me? I want a diff, a failing test name, or a rejected hypothesis written on the lease card. A transcript that exists only in a browser tab is not an artifact I can hand to someone else.
Who can say no before the merge? If the honest answer is nobody, the seat is wrong for that change, including the seat that costs nothing. A free run can still ship a confident mistake, and the price of the mistake is not on the pricing page.

Would you still sign the diff if the model name were missing from the commit message? If the answer is no, the seat did not finish the job, and another hour will not finish it either.

Keep a lease card the branch can diff

I store the card beside the task notes, and I commit it when the choice itself needs a trail. The script below is a proposal, not a benchmark I executed for this article, so the printed labels are decisions rather than measurements. It refuses a recommendation when the exit fields are empty, which is the behavior I want on a tired afternoon.

#!/usr/bin/env python3
"""Proposal: label a one-task agent lease. Not a measured benchmark."""

from __future__ import annotations

import json
import sys
from dataclasses import dataclass


@dataclass(frozen=True)
class Lease:
    data_class: str  # public | internal | resident
    retries: int
    artifact: str  # diff | test_name | hypothesis
    reviewer: str  # self | teammate | none
    needs_long_context: bool
    can_operate_a_server: bool


def choose(lease: Lease) -> str:
    if lease.reviewer == "none" or not lease.artifact:
        return "stop: no exit evidence"
    if lease.data_class == "resident":
        if lease.can_operate_a_server:
            return "self-hosted: you accepted the restart duty"
        return "stop: residency without an operator"
    if lease.needs_long_context or lease.retries > 3:
        return "paid: continuity is the product you are buying"
    if lease.data_class == "public" and lease.retries <= 2:
        return "free: short lease, keep the artifact, drop the transcript"
    if lease.data_class == "internal" and lease.reviewer == "teammate":
        return "paid or free-with-policy: only if current terms allow this data class"
    return "stop: shrink the task or rewrite the card"


def main() -> int:
    raw = json.load(sys.stdin)
    lease = Lease(
        data_class=raw["data_class"],
        retries=int(raw["retries"]),
        artifact=raw.get("artifact", ""),
        reviewer=raw.get("reviewer", "none"),
        needs_long_context=bool(raw.get("needs_long_context", False)),
        can_operate_a_server=bool(raw.get("can_operate_a_server", False)),
    )
    print(choose(lease))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

A public fixture card can live in the issue, and it should be boring enough to review in a minute. I keep secrets out of the sample on purpose, because a teaching card that leaks a token is already a failed lease. If the real task cannot be reduced to this shape, the free row is the wrong row. The JSON is an example of the input, not a log from a session I ran.

{
  "data_class": "public",
  "retries": 2,
  "artifact": "diff",
  "reviewer": "self",
  "needs_long_context": false,
  "can_operate_a_server": false
}

Run the label before the first prompt

I hash the prompt and the card together, so a later edit cannot pretend it was the authorized task. These commands stay on my own machine, and they do not call a model provider at all. If the hash changes after I start, I treat that as a new lease rather than as a continuation.

python3 lease_seat.py < lease.json
sha256sum prompt.md lease.json | tee lease.sha256
git diff --stat

If the script prints stop, I do not open a session to see what the model might say. Curiosity without an artifact is how the retry counter becomes a suggestion instead of a rule. If the script prints free, I still copy the diff into the branch, and I keep whatever the card promised to keep. Does a larger context window change the hash I already wrote down beside the lease card?

The hash does not move when the window grows, and that mismatch is the useful part. The hash records the task I authorized, not the confidence the model expressed after it saw more files. I would rather fail the check in ten seconds than discover the scope creep during review. Have you watched a session helpfully open files that you never placed on the lease card?

Work the lease in five steps

I walk these five steps in order, and I do not skip to the model because the card feels obvious. The script is only a reminder, not a boss, but I still run it when I am tempted to improvise. Have you noticed that the improvised path is usually the one that touches one extra file?

I write the four answers on the card before I open any model tab, including a tab that is free.
I run the proposal script against that card, and I treat a printed stop line as a finished decision.
I hash the prompt beside the card, and I start only when the label matches the data class.
I copy the artifact into the branch, and I stop when the retry integer on the card hits zero.
I add the closing line under the card, and I do not reopen the session to polish a rejected hypothesis.

Read the three seats as fit criteria

Free model access fits a short public or synthetic task when the artifact is a diff or a named test and the retry count stays small. I do not put secrets, production credentials, or an irreversible migration on that seat, even for a minute. What is missing is not a slogan about quality, but a retention and uptime promise I have checked on the current terms page.

Paid access fits when I am buying continuity, a team audit trail, or a support path I can actually open. The same card still applies, because a larger invoice does not invent a reviewer who was absent at the start. If the current terms forbid my data class, the paid column is a stop, not a quieter upgrade. Why would I pay to repeat a data-class mistake I could have caught on paper?

Self-hosting fits when the data class is resident, or when prompts and logs must stay on hardware I operate. That fit includes patches, disk, and the hour when the process dies while I am offline. If I cannot name the person who restarts it, I am choosing a hope with extra steps, not self-hosting. The box can be cheaper on paper and still be the most expensive seat if nobody owns the restart.

I keep these tradeoffs in one table so the argument does not hide inside a paragraph. The cells are fit notes, not scores, and they assume you already answered the four questions. A blank cell means I would stop, rather than pretend that the seat is secretly fine. Can you place your real task into one row without quietly inventing a reviewer for it?

Signal	Free seat	Paid seat	Self-hosted
Data class	Public or synthetic sample	Internal code only if current terms allow it	Resident data on hardware you operate
Continuity	One short session, then exit	Multi-step context you are willing to pay for	As long as you can keep the process healthy
Exit evidence	Diff or test name you copy out	Same evidence, plus a log you are allowed to export	Logs on disk you already control
Retry budget	One or two, then stop	The integer on the card, not an open tab	The same integer, paid in your own time
Skip when	Secrets, migrations, or no reviewer	Terms conflict, or nobody can say no	No named operator for patches and restarts

Place a free server only where the card already allowed it

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode shows up in this method only as a candidate for the free row on the card. For this draft I am relying on two availability claims only: free model access, and a free server option. I am not stating a token quota, a hardware shape, a duration, or a comparison score. Those details were not verified for this draft, and product figures like that go stale quickly.

Before I rely on either claim, I read the current product terms and confirm that my data class is permitted. If the terms are silent or confusing, I leave the card on stop and edit in a local tool I already trust. I would not move a hosted seat into the resident row unless I control the deployment and have checked that path myself. A free server is still a lease with an owner, a log policy, and an end time you should assume can change.

If you want to rehearse the exit on a throwaway public fixture, a free model seat is enough for that drill. Save the diff, discard the transcript, and ignore the seat when the script told you to stop. I would not use that rehearsal as proof that the seat is safe for internal code. The rehearsal only proves that you can end a session you were willing to throw away.

Who should leave this card on the shelf

Skip this approach when you need a formal threat model, a compliance signature, or a measured comparison of model quality. The script does not score accuracy, latency, or money, and I did not execute it against a live provider while writing this note. Skip it when the change touches credentials, customer data, or a migration you cannot roll back before the day ends. Also skip it if you came for a vendor ranking, because I am not publishing one.

The limitation I watch most carefully is the false comfort that comes from a tidy label. A printed free line can still hide a sloppy prompt, a flaky test, or a terms page I skimmed too quickly. The card governs my behavior, and it is not a certificate that the model was right. If the label and the diff disagree, which one are you actually planning to trust in review?

Close the lease before the day ends

I end the session when the artifact is on the branch, or when the retry counter hits zero, whichever arrives first. Then I add one line under the card saying I kept the diff, or I rejected the hypothesis. That closing line remains useful even if every product name is later deleted from the page. The seat you can end cleanly is the only seat you were really allowed to start.

A Review-Hour Gate for Free, Paid, and Local Agent Work

Riley Li — Sat, 10 Oct 2026 17:57:52 +0000

I do not choose a free, paid, or self-hosted agent seat by the invoice that arrives later. I choose it by the review hours I can still spend on the diffs that seat will emit. If I cannot read the patch, the cheaper seat is not a bargain, and the expensive seat is not safer. The question I keep asking is simple: whose eyes are actually on this change before it lands?

A seat is only a place where an agent edits files, runs commands, and talks back to me. The free option, the paid box, and the machine under my desk all fail in different ways when my attention runs out. I want a worksheet I can rerun on a laptop, not a slogan about which vendor feels modern this month. Would I sign the resulting commit after a tired afternoon, or would I rubber-stamp it because the queue already moved on?

What this decision is, and what it is not

I am deciding where a bounded agent task may run, and what evidence must come back with it. I am not deciding which model is smartest, because I have not measured that on this repository. I am not deciding a permanent architecture, because availability, quotas, and hardware change without asking me. Can I name the files the agent may touch, and can I name the files it must never open?

Three seats stay on the table until the sheet says otherwise, and I do not drop one early. A free hosted seat means free model access plus a free server option, when those are currently offered, and nothing more specific. A paid hosted seat means I am buying priority, retention, or a support path I can name in a contract. A self-hosted seat means I already own the runner, the logs, and the failure, including the night it falls over.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I mention MonkeyCode here because its free model access and free server option are one concrete way to fill the free hosted column. I do not treat that mention as a measurement of quality, uptime, token allowance, or model identity. If a quota, a model name, or a hardware shape matters to you, read the current docs before you plan a sprint around them.

Fit criteria I refuse to skip

I score each seat against the questions below, and I write the answers in plain text next to the task. What data class sits in the workspace: a public fixture, internal code without secrets, or anything regulated? How many review minutes can I honestly give this task before the week ends and the context fades? Can I export the diff, the test log, and the prompt, and who reruns the job if the seat vanishes?

A free hosted seat fits a public fixture or a synthetic bug, when losing the session would only cost a rerun. It does not fit when secrets, customer data, or a release branch sit in the same tree as the task. A paid hosted seat fits when I need a named retention window or a support contact, and when review time is already funded. A self-hosted seat fits when the data cannot leave, and when I can actually patch the runner myself.

Would I still pick the free seat if the docs changed the allowance tomorrow morning without notice? Only if the task is disposable and the evidence already lives on my disk, where a vanished tab cannot take it. Would I pick self-hosted for a one-file typo in a public sample that nobody is waiting on? Probably not, because I would spend the afternoon nursing a runner instead of reading the diff itself.

A local gate you can reread

The script below is a proposal, not a benchmark I have executed against a live vendor. It reads a small JSON description of the task and prints which seats remain eligible today. You can change the thresholds, but do not pretend those numbers came from a measured lab. I keep the inputs boring on purpose, so a teammate can argue with a field instead of arguing with a vibe.

#!/usr/bin/env python3
"""Proposal only: gate agent seats by data class and review budget.

Unexecuted example. It does not call a vendor API and it does not
measure model quality. Replace the sample task before you trust it.
"""

from __future__ import annotations

import json
import sys
from dataclasses import dataclass


@dataclass(frozen=True)
class Task:
    data_class: str  # public_fixture | internal_code | regulated
    review_minutes: int
    expected_diff_lines: int
    needs_named_support: bool
    can_export_locally: bool
    secrets_in_tree: bool


def eligible(task: Task) -> dict[str, str]:
    reasons: dict[str, str] = {}
    leaves_blocked = task.secrets_in_tree or task.data_class == "regulated"

    if leaves_blocked:
        reasons["free_hosted"] = "blocked: data must not leave the machine"
        reasons["paid_hosted"] = "blocked: data must not leave the machine"
    elif task.data_class == "public_fixture" and task.can_export_locally:
        reasons["free_hosted"] = "eligible: disposable work and local evidence"
    elif task.data_class == "internal_code" and task.can_export_locally:
        reasons["free_hosted"] = "hold: confirm current data terms before use"
    else:
        reasons["free_hosted"] = "blocked: no local export path"

    if not leaves_blocked and (task.needs_named_support or task.review_minutes >= 90):
        reasons["paid_hosted"] = "eligible: support or a funded review window"
    elif not leaves_blocked:
        reasons["paid_hosted"] = "optional: only if the contract names retention"

    if leaves_blocked or task.data_class == "internal_code":
        reasons["self_hosted"] = "eligible: you keep the runner and the logs"
    elif task.expected_diff_lines <= 80 and task.review_minutes <= 45:
        reasons["self_hosted"] = "skip: runner care exceeds the diff"
    else:
        reasons["self_hosted"] = "optional: use if you already operate the box"

    return reasons


def main() -> int:
    raw = json.load(sys.stdin)
    task = Task(
        data_class=raw["data_class"],
        review_minutes=int(raw["review_minutes"]),
        expected_diff_lines=int(raw["expected_diff_lines"]),
        needs_named_support=bool(raw["needs_named_support"]),
        can_export_locally=bool(raw["can_export_locally"]),
        secrets_in_tree=bool(raw["secrets_in_tree"]),
    )
    print(json.dumps(eligible(task), indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The command below only runs that local script, and it does not call a remote agent or upload the fixture. Swap the JSON for your own notes, and keep secrets out of the file even on your laptop. If you cannot describe the task without pasting a credential, stop and shrink the workspace first. A worksheet that requires a secret in order to run is already telling you the free hosted column is closed.

python3 seat_gate.py <<'JSON'
{
  "data_class": "public_fixture",
  "review_minutes": 40,
  "expected_diff_lines": 60,
  "needs_named_support": false,
  "can_export_locally": true,
  "secrets_in_tree": false
}
JSON

I feed it a fixture that looks like the work, not like a marketing page I saw this week. A public parser bug with a short guess and half an hour of review should not drag a regulated workspace into the same decision. If the printout says hold, I stop and read the terms, because a script cannot grant a permission that policy never gave. Would I be comfortable pasting that JSON into the ticket where my reviewer can see every field?

Steps I follow before I trust a column

I copy the task into a scratch repository that contains no secrets, no customer dumps, and no production environment files. If I cannot make that copy, the free hosted column is already closed, and I do not negotiate with myself. A missing fixture is a data decision, not a tooling inconvenience I can wave away later.
I write the expected diff size in lines, then I reserve review minutes at about one minute per five changed lines. I add fifteen minutes to read the test log, and I raise that guess when the patch touches concurrency or authentication. That ratio is a personal planning habit, not a measured law you should quote as science.
I run the worksheet and I keep every hold or blocked line in the pull request description. A green eligible mark is permission to try a seat, not permission to merge the result. Would I still merge if the reviewer asked me to explain the prompt from memory alone?
I capture evidence on my own disk before I close the session: the diff, the command, and the short prompt. A free server option is useful only when that capture still works after the browser tab dies. I use a local archive so the seat cannot become the only copy of the reasoning.

mkdir -p handoff && git diff > handoff/agent.diff
# Replace the echo with the real prompt. Do not paste secrets.
{ echo "prompt: reproduce the public fixture only"; git status --short; } > handoff/notes.txt
sha256sum handoff/agent.diff handoff/notes.txt > handoff/SHA256SUMS

I compare the three columns only after that archive already exists on my own laptop disk. Free hosted stays in the running when the fixture is public and the archive is complete. Paid hosted stays when I need a support path or a retention promise I can point to in writing. Self-hosted stays when the data class forbids leaving, and when I can operate the box without stealing review time.
I rerun the same fixture once on the seat I picked, and once locally from the saved diff, before I talk about speed. If the local rerun fails, the seat did not save me time, because I still cannot explain the change. How many of last week's agent patches could I still rebuild from the notes I actually kept?

Tradeoffs I want written down

I keep a short table so the argument does not collapse into a price screenshot from a vendor page. The cells are fit notes, not scores I measured on identical hardware during some quiet week. I will not fill those cells with token counts, GPU names, or uptime percentages I cannot re-check next week. A number I cannot point to in a current primary document is not a decision input for this sheet.

Question	Free hosted	Paid hosted	Self-hosted
May the workspace leave your network?	Treat as yes until current terms say otherwise	Treat as yes unless the contract says otherwise	No, if the runner stays inside your boundary
What you might be paying with	Review time, plus a rerun if the session dies	Money, contract reading, and the same review time	Review time plus runner maintenance
What a bad day looks like	Allowance or session changes under you	The bill grows while your review hours stay fixed	Box care steals the afternoon you reserved
What you must already hold	A local archive of diff, command, and prompt	That archive, plus the clause you relied on	That archive, plus notes a teammate can boot

If MonkeyCode's free model access or free server option is the free hosted candidate, confirm the live limits before you start. Write those limits into the ticket yourself, with the date you read them, so next week's reader is not guessing. I am not claiming a token grant, a duration, or a hardware shape, because no primary source for those figures was attached here. Would you trust a sprint plan that depended on a number nobody could open in a browser today?

Who should walk away

This sheet does not tell you which model writes better patches, and it will not catch a subtle bug because the fixture was public. It assumes you can make a secret-free copy, which is false for plenty of real services that mix config into the tree. It also assumes a human still merges, which is false if your pipeline applies agent output without a reviewer in the loop. Do not use this approach for production credentials, regulated records, or any path where an agent can push alone.

Do not use a free hosted seat as a silent default for the whole monorepo just because that column looked cheap. Do not use self-hosted as a moral upgrade if nobody on the team can restore the runner after a bad image. Do not use a paid seat to skip the archive step, because a receipt is not a diff you can explain later. I would rather throw away a free exploration than explain a missing prompt three days after the session died.

If the worksheet marks a public fixture as eligible for a free hosted seat, that is when I open the current MonkeyCode docs. I check whether free model access and the free server option still match the task I wrote down. I keep the same fixture, the same archive commands, and the same refusal rules if I later move that task. The seat can change next week, but the evidence I am willing to merge should not change with the invoice.

A Parser Contract Before Free Agent Compute Joins the Release Path

Riley Li — Fri, 09 Oct 2026 14:31:17 +0000

I would not keep free agent compute on a release path until a schema swap drill still passes on a fallback seat. A low invoice does not prove that your parser will survive a seat change next Tuesday. If the saved output drifts, the discount was never the cost that your on-call rotation actually paid. Would you sign the merge when a week-old screenshot is the only evidence you can show?

Start from the contract, not the invoice

Hosted agent seats feel easy because the first prompt returns quickly and the setup screen looks generous. The fragile part arrives later, when a review script and a human habit both assume one response shape. A model string can drift, a timeout can move, or a teammate can paste a real ticket into a toy seat. I treat each of those shifts as a contract problem, not as a surprise the vendor owes me to explain.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator supplied two availability claims only: free model access, and a free server option you can try. I am not stating a token quota, a hardware shape, a duration, or a benchmark, because I have not verified those figures. Read the current product documents on the day you decide, and treat this page as a method rather than a price sheet.

What a passing swap actually proves

This drill does not crown a winner among free compute, a paid hosted plan, and a self-hosted stack. It asks whether saved outputs still satisfy one schema when you swap the seat and keep the fixture fixed. A pass means the release parser can survive the swap, while a fail keeps the free seat in a scratch lane. Why would you let a parser exception become the opening slide of your next incident review?

I am describing a local check on files you already saved, not a live call against a vendor endpoint. I have not executed this checker against a live vendor server, and I am not publishing timings or scores. You should label your own run the same way until a teammate can open the log and repeat it. Would a reviewer trust a pass that exists only in your memory and not in the repository?

Fit criteria before any seat touches a branch

The table below is a fit guide, and it is not a ranking that I measured on shared hardware. Use it to decide which questions must be true before a seat may draft, review, or release. The free column stays attractive when the fixture is fake and a second seat already passes the same schema. It stops being attractive when the prompt holds customer text, secrets, or an unattended release gate.

Fit question	Free hosted option	Paid hosted plan	Self-hosted stack
Who runs the process?	The vendor, on terms you re-read	The vendor, if the plan names a support path	Your team
What may enter the prompt under this method?	Synthetic fixtures only	Only classes your contract allows	Only classes your own policy allows
What does a schema pass support?	A scratch or draft lane	A review-automation candidate	A release-path candidate
What does a schema fail imply?	Stop the experiment	Change the prompt or the plan	Patch the stack you operate
When is this the wrong home?	Customer data, secrets, or a merge gate	You need a pin the current plan does not document	You cannot staff the host

Paid hosting earns attention when you need a support path you can name in the incident channel. Self-hosting earns attention when you must control the image or keep prompts inside a network you administer. Free hosting earns a scratch lane when both of those needs are absent and the schema still passes. Would you paste a live production token into a seat that you do not administer yourself?

Step 1: Freeze a synthetic fixture

Start with a task you can show in a public review without asking anyone for permission. I use a tiny changelog note because it has fields a parser can check and no customer data. If your real job needs private context, do not try the free seat while you are still deciding. Write the forbidden fields down before you write the prompt, or the later table will lie to you.

{
  "ticket": "CHG-1042",
  "summary": "Add a retry cap to the export worker",
  "constraints": ["no secrets", "synthetic only"]
}

Keep the expected shape small enough that a reviewer can defend every key without a meeting. A short object is easier to compare across seats than a glowing paragraph with no edges. If you cannot name the required keys, you are not ready to compare free and paid compute yet. What would the parser do tomorrow if a friendly sentence replaced the object you promised?

Step 2: Capture one file per seat

Run the same fixture on the free seat, and on a paid or self-hosted seat if you operate one. Save the raw model text, then save the parsed object under a file name that names the seat. Do not mix a hand-edited file into the set and then describe that file as a capture. If a seat cannot return JSON, that seat already failed, even when the surrounding prose sounds helpful.

I am not giving you an endpoint, a model id, or a shell command for any hosted seat. Those values go stale quickly, and a copied command is how tokens leak into shell history. Use the client your operator already approved, and store only the output files next to the fixture. Can you point to the file a second person should open, or only to a chat you cannot share?

Step 3: Run the local contract checker

The checker below is a proposal you can run with Python 3.9 or newer and the standard library only. It does not call a network, and it does not pretend to score writing quality or model taste. It asks whether each saved object has the keys, the risk enum, and the note length you promised. Treat a failure as a seat decision, not as a prompt to quietly edit the file until it passes.

#!/usr/bin/env python3
"""Local schema swap check. Proposal: not a live product benchmark."""

import json
import sys
from pathlib import Path

REQUIRED = ("ticket", "risk", "note", "seat")
RISKS = {"low", "medium", "high"}
FIXTURE_TICKET = "CHG-1042"

def load_obj(path: Path) -> dict:
    data = json.loads(path.read_text(encoding="utf-8"))
    if not isinstance(data, dict):
        raise SystemExit(f"{path}: top-level JSON must be an object")
    return data

def violations(name: str, obj: dict) -> list[str]:
    found = []
    for key in REQUIRED:
        if key not in obj:
            found.append(f"{name}: missing {key}")
    if obj.get("risk") not in RISKS:
        found.append(f"{name}: risk must be one of {sorted(RISKS)}")
    note = obj.get("note", "")
    if not isinstance(note, str) or not 12 <= len(note) <= 240:
        found.append(f"{name}: note length out of range")
    if obj.get("ticket") != FIXTURE_TICKET:
        found.append(f"{name}: ticket drifted from the fixture")
    return found

def main(paths: list[str]) -> int:
    if len(paths) < 2:
        print("FAIL")
        print("pass at least two JSON files")
        return 1
    problems = []
    seats = []
    for raw in paths:
        path = Path(raw)
        obj = load_obj(path)
        problems.extend(violations(path.name, obj))
        seats.append(obj.get("seat"))
    if len(set(seats)) < 2:
        problems.append("need outputs from at least two distinct seats")
    if problems:
        print("FAIL")
        print("\n".join(problems))
        return 1
    print("PASS")
    print("seats=" + ",".join(str(item) for item in seats))
    return 0

if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1:]))

A sample object that should pass is short, boring, and tied to the same ticket as the fixture. Run the checker only after you have captures from at least two distinct seats on disk. If you only have the free seat today, compare its file with a golden object your parser already accepts. Say that limitation in the pull request, because a golden file is not a second vendor.

{
  "ticket": "CHG-1042",
  "risk": "medium",
  "note": "Retry cap stays in the worker, with no customer payload attached.",
  "seat": "free"
}

python3 schema_swap.py free.json fallback.json

Step 4: Apply the fit rules in order

I apply four rules in order, and I stop at the first hard no instead of averaging them away. If the fixture is not synthetic, the free seat is out until a written policy allows that data class. If the checker fails on the free output, keep that seat away from any job that merges or pages. If free and one fallback both pass, the free seat may draft and the fallback remains the seat that may merge.

If you cannot name a person who can repeat the swap from this page, you do not have a strategy yet. A channel name is not an owner, and a shared login is not a handoff you can trust at midnight. Does your current seat meet the rule you actually have, or the rule you wish the vendor had written? I would rather delay the experiment than discover the missing owner during a release freeze.

Step 5: Leave a decision record in the repo

Leave a short record beside the fixture so the next person does not rerun the argument from memory. A chat transcript is not a decision, and a pricing page is not a runbook your team can execute. If the free schema later fails, the record already says which seat still owns the release path. Would the next reviewer know what you decided if your laptop disappeared after standup today?

date: 2026-10-09
fixture: CHG-1042 synthetic
free_schema: pass|fail|not_run
fallback_schema: pass|fail|not_run
data_class: synthetic
release_seat: fallback until both passes exist
owner: name a person, not a channel
status: example record, not a completed run

Limitations you should not talk past

This method will not tell you which model writes better prose, and it will not estimate a token bill. I did not measure latency, uptime, or quota, so do not quote this article as evidence of those numbers. A schema pass can still hide a wrong risk label when your enum is too coarse for the change. The checker trusts the files you hand it, so an edited capture can launder a real failure.

Who should not use this drill

Do not use this approach for health data, payment data, credentials, or prompts a customer did not approve. Do not use it when your release process requires a signed service level that a free option does not offer. Skip it if you cannot keep a second seat, because a single seat has nothing to swap against. Also skip it if you wanted a public leaderboard, because this drill is a gate and not a contest.

A scratch lane until the fallback exists

I would keep free model access and the free server option in the scratch lane until the checker passes. Promotion to a release path should wait for a fallback you can actually operate without the free seat. If you already have a free MonkeyCode seat, run this drill on a public fixture before the next release branch. The useful question is not whether the seat is free, but whether you can leave it without breaking the parser.

Score a Free Agent Seat Against Paid and Self-Hosted With a Replay Card

Riley Li — Thu, 08 Oct 2026 10:17:53 +0000

I do not choose a free agent seat, a paid box, or a self-hosted runner from a price tag. I freeze one small task on a replay card, then I let that card reject seats. A zero price is not evidence, and a smooth demo is not a plan you can defend later. Would you trust a host that you still cannot replay with the same inputs next Thursday?

People compare seats by the monthly cost, and only later discover that the real cost was custody. That mistake shows up in planning notes, though this draft claims no measured outage and no customer story. The question is simple, and it is also easy to skip when a banner says free. Can you name the logs, the runtime, and the data class before you click start?

Start from the constraint, not the banner

This week's public developer posts keep celebrating how quickly someone can assemble a visible side project alone. I am using that mood only as a warning, not as a source of facts or rankings. Faster drafts make the seat choice sloppier, because the artifact appears before any operating constraint is written. If the assistant can draft the tool tonight, why would the host decision wait until morning?

I compare three seats, and I refuse to add a fourth until these three are honest. The free hosted seat gives you model access and a server you do not administer, with terms you must re-read. The paid isolated seat rents separation, and it still fails if the contract never mentions retention. The self-hosted seat gives you the pin, and it also gives you patching, backups, and the blame.

MonkeyCode is one example of that free hosted seat, because free model access and a free server were supplied for this draft. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am not listing model names, token quotas, hardware, duration, or a promise that the offer stays open. If those details matter to your replay card, read the current project page before you treat the seat as available.

I also will not label the project open source in this note, because I was not given a repository URL or a license text to cite. A reader who needs that status should open the current project page and read the license there. Treating a missing link as permission is exactly how outdated product claims leak into a published post. Does a decision guide stay honest if it fills those gaps with a memory of yesterday's banner?

Write the replay card before you touch a host

A replay card is a tiny record of the task, not a manifesto about agents in general. It names the data class, the replay need, the pin need, and whether shared logs are acceptable. I keep the card next to the fixture file so a later reader can see what was allowed. Does your current note say public fixture, internal code, or secret, in words a stranger could audit?

Run the seat trial in six steps

Write a local fixture that contains no customer data, no access tokens, and no private repository contents at all.
Fill the replay card with the strictest data class you can defend, not the class you hope is true.
Run the scorer below and save the JSON beside the fixture hash before you open any hosted session.
Use the free hosted seat only when the scorer says stay, and only for the card you just saved.
Move to a paid isolated seat when the card needs separation but you cannot staff the host this week.
Choose self-hosting only when the pin is required and the weekly ops time is already on a calendar.

A stay result means this card can rehearse on a shared free seat, not that every future task can. A paid result means you are buying isolation or a clearer contract, not a better personality from the model. A self-host result means you accepted patching and backups, so do not pretend that server is free in another costume. Which of those three sentences matches the work you actually have on the calendar this week?

Read the tradeoffs in one table

I put the same questions in a table so a review can happen without rereading the whole narrative. The table is a decision aid, and it is not a benchmark, a price list, or a ranking of vendors. Those empty promises are intentional, because I will not invent hardware, quotas, or uptime numbers for this draft. Would a reviewer be able to point at one row and explain the rejection out loud?

Question on the card	Free hosted	Paid isolated	Self-hosted
Fixture is public and shared logs are acceptable	Stay, if you also store a receipt	Works, often wasteful for a rehearsal	Works only if ops time already exists
Context may include a secret or production credential	Reject	Reject until retention is written and quoted	Eligible if you control disk, logs, and access
Runtime must still match next month	Do not assume a pin	Buy only after the pin is in the offer	You can pin it, then you must patch it
Ops time is under two hours each week	Reasonable for a public card	Often the calmer path when isolation matters	Poor fit, because drift becomes the outage
You want a model-quality ranking	This method will not produce one	A vendor chart is still not your fixture	Your result is only as honest as the card

Keep the rules in a file you can diff

The following Python file is an unexecuted example, and it never calls a model or a remote server. I want the rules visible, because a hidden score is just another landing page with extra steps. Change the thresholds if your team defines ops time differently, and keep that change in version control. Are you willing to merge a rule that you cannot explain to a teammate in one minute?

#!/usr/bin/env python3
'''Unexecuted example: score one replay card against three agent seats.

This file does not call a model, does not open a network socket, and does not
measure latency or quality. Edit the rules in version control before you trust them.
'''

from __future__ import annotations

import json
from dataclasses import dataclass


@dataclass(frozen=True)
class ReplayCard:
    task_id: str
    data_class: str  # public_fixture | internal_code | secret
    needs_replay: bool
    needs_pinned_runtime: bool
    allows_shared_logs: bool
    ops_hours_per_week: int


def notes_for(card: ReplayCard) -> dict[str, list[str]]:
    seats = {'free_hosted': [], 'paid_isolated': [], 'self_hosted': []}

    if card.data_class == 'secret' or not card.allows_shared_logs:
        seats['free_hosted'].append('reject: secrets or private logs are out of scope')
    elif card.data_class == 'internal_code':
        seats['free_hosted'].append('reject: internal code needs a written retention promise')
    else:
        seats['free_hosted'].append('fit: public rehearsal only')

    if card.needs_pinned_runtime:
        seats['free_hosted'].append('weak: a free seat does not prove next month pin')
        seats['paid_isolated'].append('check: quote the pin from the current offer')
        seats['self_hosted'].append('fit: you can pin the runtime you operate')
    else:
        seats['paid_isolated'].append('fit: isolation without owning the host')
        seats['self_hosted'].append('optional: skip unless ops time is already scheduled')

    if card.ops_hours_per_week < 2 and card.needs_pinned_runtime:
        seats['self_hosted'].append('reject: a pin without ops time becomes drift')

    if card.data_class == 'secret' and card.ops_hours_per_week < 2:
        seats['paid_isolated'].append('hold: do not send secrets until retention is quoted')
        seats['self_hosted'].append('reject: secret custody needs named ops time')

    if card.needs_replay:
        for name in seats:
            seats[name].append('require: store inputs, prompt hash, and raw output')
    return seats


def choose(seats: dict[str, list[str]]) -> str:
    def rejected(name: str) -> bool:
        return any(item.startswith('reject') or item.startswith('hold') for item in seats[name])

    if not rejected('free_hosted'):
        return 'stay_on_free_hosted_for_this_card'
    if not rejected('self_hosted') and any('you can pin' in item for item in seats['self_hosted']):
        return 'self_host_if_ops_time_is_real'
    if not rejected('paid_isolated'):
        return 'paid_isolated_until_custody_is_clear'
    return 'narrow_the_card_before_any_seat'


def score(card: ReplayCard) -> dict:
    seats = notes_for(card)
    return {'task_id': card.task_id, 'choice': choose(seats), 'notes': seats}


if __name__ == '__main__':
    sample = ReplayCard(
        task_id='fixture-readme-summary',
        data_class='public_fixture',
        needs_replay=True,
        needs_pinned_runtime=False,
        allows_shared_logs=True,
        ops_hours_per_week=1,
    )
    print(json.dumps(score(sample), indent=2))

I also keep a tiny shell sequence so the receipt has a time and a hash, even when the score is boring. The date command here is UTC, which avoids a local timezone argument during a later review. Do not point the fixture path at a secrets directory just to see what the scorer prints. If the hash changes, the decision you quoted yesterday no longer belongs to the file you have now.

# Unexecuted example. Keep fixtures free of secrets before you run this.
mkdir -p receipts
stamp=$(date -u +%Y%m%dT%H%M%SZ)
python3 seat_trial.py | tee receipts/$stamp.json
sha256sum seat_trial.py fixtures/readme-summary.md | tee -a receipts/manifest.txt

Apply two cards before you generalize

Suppose the fixture is a short public readme, and the task is to list setup commands without inventing flags. The card says public fixture, replay required, pin not required, shared logs allowed, and one hour of weekly ops time. The scorer should stay on the free hosted seat, and it should still demand a stored prompt hash and raw output. If tomorrow the readme is replaced by an internal design doc, does yesterday's stay result still apply at all?

Now change only the data class to secret, and leave every other field on the card untouched. The free seat should reject the card, because shared logs and secrets do not belong in the same rehearsal. The sample scorer also holds the paid seat until retention is quoted, and it rejects self-hosting at one weekly hour. Raising ops time does not by itself authorize a secret on a free server, so do not skip that line.

Fit criteria I will not soften

Stay on the free option when the fixture is public, shared logs are acceptable, and you do not need a pin. Buy a paid isolated seat when separation matters and your ops calendar cannot absorb a private runner this month. Self-host when the pin is the product, and when someone named on the team already owns the patches. If none of those sentences is true, do not start the agent loop until the card is narrower.

Free hosted compute trades custody for a faster entry, and that trade is sometimes the right one. Paid isolation trades money for someone else's rack, and you still must read retention before sending internal code. Self-hosting trades cash you might have paid a vendor for time you will certainly spend later. Which trade are you actually making, and which trade did the pricing page quietly imply instead?

Limitations, and who should walk away

This approach will not tell you which model writes better prose, because the script never sees a completion. It will not estimate a bill, because I was not given durable prices, hardware shapes, or quota numbers to cite. It will not survive a card that says secret while someone still pastes a key into the prompt box. If you need a public leaderboard, you need a different experiment, with a published method and a date.

You should not use the free seat, or this rehearsal as permission, when the prompt can touch production credentials. You should not use it for regulated records, private customer text, or anything a log retention policy would flag. You should not self-host from this note if nobody on the team can patch the box next weekend. Should a solo experiment on a public readme really become the template for a company workflow?

I ask four questions in a pull request, and I do not accept a screenshot of a chat as an answer. Who can read the logs after the run, and for how long, according to a page you actually linked? What file hash did you score, and does that hash still match the fixture sitting in this branch? If the free offer changes tomorrow, which sentence in the card forces you to stop and re-score?

Keep the receipt, then pick the seat

I would run the sample card, read the rejection lines, and only then open a hosted session on a public fixture. I would store the receipt in the same repository as the fixture, because a chat scroll is not an audit trail. If the card stays public, try the free model access and free server you are evaluating, and keep the JSON receipt. If the card rejects that seat, the same file has just saved you from a demo you cannot defend later.

I will revisit this note if a primary page states a quota, a duration, or a hardware shape I can cite without guessing. Until that page exists in your notes, the replay card is the artifact, and the seat is only what the card allows. A free server can be the right classroom for a public fixture, and a poor vault for anything else. Are you choosing a seat for the task you wrote down, or for the task you wish you had?

Pick the Agent Seat by the Commit You Would Sign

Riley Li — Thu, 08 Oct 2026 07:47:17 +0000

Pick the Agent Seat by the Commit You Would Sign

I do not start with the price tag when I place an agent on a machine today. I start with a harder question: would I sign the commit this run is about to create? If the answer is no, a free shared seat can still be a fair draft bench. If the answer is yes, I want a host whose git identity, license files, and secret boundary I can name before the push.

That split sounds fussy until a generated patch lands in a public repository with the wrong author line. Have you ever opened a pull request and then noticed the committer email belonged to a shared runner? I treat that moment as a hosting decision, not as a cosmetic git fix after the fact. The seat should match the signature you are actually willing to defend in a public review.

What this guide actually decides

This note is a fit guide between seats, not a benchmark and not a vendor price shootout. I am comparing three seats you can actually reason about before you spend a weekend on setup. Those seats are a free shared agent server, a self-hosted box you administer, and a paid isolated runner. The real question is which seat may draft, which seat may commit, and which seat may push.

You might ask why a commit signature should outrank raw speed on a quiet Tuesday afternoon at all. A fast draft you cannot attribute becomes a real liability the moment it leaves your laptop. A slower draft you can sign, license, and later revert is usually the safer promotion path. Would you rather explain a missing signature in review, or explain a token that rode along in the diff?

I am not claiming timings, token quotas, hardware sizes, or how long any free offer lasts. Those product facts change quickly, and I have not measured any of them for this particular note. If a dashboard shows a number, read that dashboard, and do not borrow my silence as a measurement. Can you point to a primary page for the exact limit you are about to rely on today?

Fit criteria I refuse to skip

I keep the criteria short so I can apply them on a tired afternoon without a spreadsheet. Each one is a yes or no I can check in the repository, not a feeling about a logo. If I cannot answer a criterion, I do not promote that run past a local patch. Would you really promote a branch that you cannot even describe clearly to a waiting teammate today?

Author identity. Can I set the git name and email to an identity I control, and can a shared runner not overwrite them later?
Signature path. Can I sign with a key that never sits on the shared seat, or am I only pretending the commit is mine?
License and notice. Does the tree already contain the license I must preserve, and can the agent edit be stopped from stripping it?
Secret boundary. Can I prove the diff does not add keys, tokens, or environment files before anything is pushed upstream?
Rollback story. If the patch is wrong, can I drop the branch without negotiating with a host I do not administer?

A free seat can still win the draft half of this list when I copy the patch home before I commit. It usually loses the signature path, because I should not upload a private key to a machine I do not control. Self-hosted and paid isolated seats win when the push itself must happen on that same machine. Which half of this list is your current agent run actually asking the host to satisfy today?

A worksheet you can run locally

The following script is a proposed check, not a log from a run I executed on any host. It inspects the repository you are standing in and prints local git facts you can read as a gate. It does not call the network, and it does not print secret file contents, only matching names. Run it only inside a repository you already trust enough to open on your own machine.

#!/usr/bin/env bash
# Proposed, unexecuted example: seat fit from local git facts.
set -euo pipefail

echo "1. Identity"
git config --get user.name || echo "MISSING user.name"
git config --get user.email || echo "MISSING user.email"

echo "2. Signing key configured on THIS machine? (presence only)"
if git config --get user.signingkey >/dev/null; then
  echo "signing key is configured here"
else
  echo "no signing key on this machine"
fi

echo "3. License file"
ls -1 LICENSE LICENSE.md COPYING 2>/dev/null || echo "NO license file found"

echo "4. Suspicious paths in the unstaged diff (names only)"
git diff --name-only -- . ':(exclude)*.lock' \
  | grep -E '(^|/)\.env|id_rsa|\.pem|credentials|secret' \
  || echo "no obvious secret filenames in diff"

echo "5. Branch state"
git status -sb

I read that output as a gate, not as a score I can average into a winner. Missing identity means I am not ready to commit the patch on any seat at all. A signing key that exists only on my laptop means the shared server may draft, and the laptop must sign. A secret filename sitting in the diff means I stop, even when the seat looks inexpensive.

Numbered path from draft to signed push

I follow the same sequence every single time so that the host choice stays boring and repeatable. You can skip a step only when the answer is already written down beside the branch. If you skip the secret scan because a model usually behaves, you are guessing with a public history. Does your team actually want that kind of guess attached to your name in the public log?

Create the change on a throwaway branch, and keep the default remote unset until the diff is clean.
Run the worksheet in the repository root, and save the printed text next to the patch file.
If identity or a license file is missing, fix that on your own machine before any agent continues.
If the diff names a secret file, remove the file and rotate the credential before you discuss hosting.
Choose the seat from the table below, then move the patch only, and leave the signing key at home.
Sign and push from the machine that holds the key, even when a cheaper seat wrote the draft.

The next block is a proposed local sequence for step 6, not a transcript from a hosted run. I keep the push commented until the signature line looks like a commit I would defend. Would you paste a private key into a chat box just to avoid these six lines?

# Proposed local sequence on the signing machine. Do not paste a key into a prompt.
git switch -c draft/agent-seat-check
git diff --name-only
git add -p
git commit -S -m "Apply the reviewed agent draft"
git log -1 --show-signature
# Push only after the signature line looks right.
# git push -u origin HEAD

How the three seats trade off

Fit signal	Free shared agent server	Self-hosted box	Paid isolated runner
Draft a patch you will rewrite	Fits if you copy the patch home	Fits, often more setup than the draft deserves	Fits, though you may pay for exploration
Commit with your identity	Only if that identity cannot be overwritten	Fits when you administer the git config	Fits when the contract lets you set identity
Sign with your private key	Poor fit; do not upload the key	Fits if the key stays on a machine you control	Fits only if the key never has to be pasted into a prompt
Touch production secrets or release tokens	Do not use for this run	Fits only with an explicit secret store	Fits only under the same secret-store rules
Need a named rollback owner	Weak unless you can export the branch	Strong, because you own the disk	Strong if snapshots are actually in the contract

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free model access and the free server option are the two availability claims I was given to discuss here. I am not naming models, token quotas, hardware, duration, or benchmark figures, because those were not verified for this draft. If you use that free seat, treat it as the left column: draft there, then carry the patch home before you sign.

I would read the current MonkeyCode terms only after this table says the free column matches the run in front of me. That quiet reading step is the only invitation I am willing to make anywhere in this note. Confirm the live limits on the product page yourself, because an outreach draft is not a contract and it is not a measurement. Are you choosing a seat you can leave, or a seat you will have to explain later?

Tradeoffs I want written down

Free shared compute lowers the cost of a bad draft, and that lower cost is the actual point of it. The tradeoff is weaker custody over base images, log retention, and a workspace you might forget to wipe. Self-hosting flips that tradeoff toward maintenance hours, and toward git config you can actually pin down. A paid isolated runner sits between them when you want separation without becoming the person on call.

Have you noticed how often a temporary free workspace becomes the place a release script quietly starts to live? I assume that drift will happen unless I schedule the move back home before the branch grows. The worksheet is how I notice the drift before a push, not after a reviewer does. If I cannot export the branch, I was never sitting in a seat I should sign from.

None of these seats will repair a vague prompt or a repository that still has no tests. They only change who can see the mess, who can retain the logs, and who can put a name on the commit. I still want a failing test on my own machine before I call the patch done. Would a green agent transcript satisfy you if the local test suite had never actually been run?

Who should not use this approach

Do not use this guide when the change already needs production credentials, customer data, or a signed release on the first pass. That work does not belong on a free shared seat, and a filename scan is not a security review. Do not use it when your organization already mandates a specific runner, a hardware security module, or a reviewed signing service. In that case the seat is chosen, and this table would only invent a shadow path around policy.

Also skip this method when you cannot read the diff yourself before you push the branch. An agent draft you do not understand is not ready for a free seat or a paid one. This check is local and name-level, so it misses secrets pasted into a source file under an innocent name. Pair it with your real secret scanner and with the host's current retention note before you promote the branch.

Where I land on the next run

I keep the conclusion in the same place I started, because the price tag will try to sneak back in. If I would not sign the commit, I may draft on the free seat and I leave the key at home. If I would sign it, I move to a self-hosted or paid seat whose identity and rollback I can name out loud. The script is the artifact I re-run, and the table is the argument I am willing to keep.