DEV Community: Riley Xu

Migration Diary: Your Golden Fixtures Expire the Day You Switch AI Providers

Riley Xu — Tue, 27 Oct 2026 13:23:43 +0000

The fixtures in your test suite are a recording of the provider you are leaving, and they will keep passing long after cutover makes them meaningless. That is the failure mode that makes this class of migration dangerous: the suite goes green, the team relaxes, and nobody notices that every assertion still describes a response shape only the old endpoint produces. The fix is small and boring — one provenance field on each fixture, one split in your assertions, and one command you can run on cutover night.

Earlier entries in this diary covered the streaming parser, the cost ledger, and the rollback rehearsal. This entry covers the thing that tells you whether those three are still working after the switch.

1. Why fixtures rot silently instead of failing loudly

A recorded fixture is a snapshot of a request and a response captured from a specific provider, model, and tool schema at a specific moment. When you change any of those three, the fixture does not become wrong in a way your test runner can detect — it becomes a test of the past that still exits zero. Your CI keeps rewarding you for not touching code that no longer reflects production.

The damage compounds because fixtures are usually the cheapest thing to keep and the most annoying thing to regenerate. Someone records them once during the original integration, they encode a few quirks of that provider, and every later refactor is validated against those quirks. By the time you plan a cutover, the suite is measuring fidelity to a memory rather than fidelity to a contract.

Provenance is the cheapest countermeasure available. If every fixture file states which provider and model produced it, a cutover checklist can ask a mechanical question — how many fixtures are still stamped with the old provider — instead of relying on somebody remembering.

2. Step 1: stamp every fixture with where it came from

The example below is illustrative rather than tested, so adapt the field names to whatever your harness already writes. The point is that the metadata travels inside the fixture file, not in a spreadsheet that drifts away from it.

# tools/fixture_provenance.py — illustrative skeleton
from dataclasses import dataclass, asdict
from pathlib import Path
import hashlib, json

@dataclass
class Provenance:
    fixture: str
    provider: str
    model: str
    recorded_at: str
    prompt_hash: str
    tool_schema_hash: str

def _hash(obj) -> str:
    blob = json.dumps(obj, sort_keys=True, ensure_ascii=False).encode()
    return hashlib.sha256(blob).hexdigest()[:12]

def stamp(path: Path, provider: str, model: str, recorded_at: str) -> Provenance:
    case = json.loads(path.read_text())
    request = case["request"]
    return Provenance(
        fixture=path.name,
        provider=provider,
        model=model,
        recorded_at=recorded_at,
        prompt_hash=_hash(request.get("messages", [])),
        tool_schema_hash=_hash(request.get("tools", [])),
    )

def write_manifest(suite_dir: Path, entries: list[Provenance]) -> None:
    manifest = {e.fixture: asdict(e) for e in sorted(entries, key=lambda e: e.fixture)}
    (suite_dir / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")

Two hashes matter more than the timestamp. The prompt hash catches fixtures that were edited by hand after recording, and the tool schema hash catches the quiet breakage where you renamed a tool parameter and every recorded tool-call fixture silently stopped matching.

3. Step 2: split each assertion into contract and content

Re-recording everything on cutover night is too risky, so you need to know which assertions must survive a provider change and which are allowed to differ. Sort them into two buckets before you touch the endpoint configuration.

Assertion	Bucket	What a failure means
Response parses as your internal message type	Contract	Your adapter is broken, stop the cutover
`role` present, `content` is a string or a block list	Contract	Schema mismatch between providers
Tool call arguments validate against your JSON Schema	Contract	Tool wiring is wrong, not the model
Exact token counts match the recording	Content	Expected; update the budget numbers
Exact assistant wording matches	Content	Noise; delete it unless wording is a product requirement
Exact finish/stop reason matches	Content	Usually fine, but alert on unknown values

Contract assertions should live in a live test that hits the real endpoint, because a recorded contract test can only prove you recorded it correctly. Content assertions belong with the fixtures and should be regenerated rather than argued over.

4. Step 3: make re-recording one command with a drift report

Re-recording by script is not optional at this size, and the script must print what changed in structure before it overwrites anything. A shape diff that only tracks key paths, not values, is enough to spot a real adapter break while ignoring harmless wording changes.

# tools/rerecord.py — illustrative skeleton, not a finished tool
def shape(obj, prefix=""):
    if isinstance(obj, dict):
        out = set()
        for key, value in obj.items():
            out |= shape(value, f"{prefix}.{key}")
        return out
    if isinstance(obj, list):
        return shape(obj[0], f"{prefix}[]") if obj else {f"{prefix}[]"}
    return {prefix}

def rerecord(suite_dir, client, write=False):
    drift = []
    for path in sorted(suite_dir.glob("*.json")):
        case = json.loads(path.read_text())
        fresh = client.call(case["request"])
        added = shape(fresh) - shape(case["response"])
        removed = shape(case["response"]) - shape(fresh)
        if added or removed:
            drift.append({"fixture": path.name, "added": sorted(added), "removed": sorted(removed)})
        if write:
            case["response"] = fresh
            case["meta"]["provider"] = client.provider
            path.write_text(json.dumps(case, indent=2) + "\n")
    return drift

Run it twice. The first run, without --write, tells you where the new endpoint's shape diverges from the old recordings; the second run commits the fixtures and refreshes the manifest. If the drift list contains a contract key, stop and fix the adapter instead of re-recording around it.

5. Step 4: keep exactly one live contract lane running nightly

Recorded fixtures cannot tell you that a free endpoint changed something on its side, so you need a small live suite that runs on a schedule and skips gracefully when credentials are absent. Keep it deliberately tiny: a chat round-trip, a tool-call request, and a streaming chunk sequence.

# tests/live/test_contract.py — illustrative
import os
import pytest

pytestmark = pytest.mark.skipif(
    not os.environ.get("LIVE_CONTRACT"),
    reason="live contract lane runs nightly only",
)

def test_chat_returns_message(live_client, chat_request):
    resp = live_client.call(chat_request)
    message = resp["choices"][0]["message"]
    assert message.get("role") == "assistant"
    assert isinstance(message.get("content"), (str, list))


def test_tool_arguments_validate(live_client, tool_request, validate_tool_args):
    resp = live_client.call(tool_request)
    call = resp["choices"][0]["message"]["tool_calls"][0]
    validate_tool_args(call["function"]["name"], call["function"]["arguments"])

Three tests, one nightly job, one alert. Anything more and the lane becomes flaky enough that people mute it, which is the same as not having it.

6. Step 5: sequence the cutover and write down the leftovers

Cutover order matters more than cutover speed, so run it in five moves. Freeze fixture writes, re-record against the new endpoint, review the drift report, flip the default provider behind a flag, then keep the old path warm for one rollback window.

The leftovers from this step are specific and easy to forget: scheduled jobs still pinned to the old model identifier, cached responses keyed by the old provider name, dashboards grouped by a field the new endpoint omits, retry budgets tuned for the previous latency profile, and a manifest full of stale provenance entries you never regenerated. Write each of these into the cutover ticket as a named line item with an owner, because "check the cron jobs" is not a task anybody completes.

7. Where a free endpoint and a free server actually earn their place

This is the part of the workflow where free model access and a free server option are genuinely useful rather than decorative. A second endpoint is what makes the nightly contract lane affordable, and a small host is enough to run the fixture re-record job without borrowing production capacity.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option, per the project's own materials — verify the current limits, regions, and terms on the project page before you design around them, because quotas and hardware change without notice. What I will not claim is anything about throughput, uptime, or model breadth, because none of that is verifiable from here and your own contract lane is the only measurement that matters.

If you want a cheap second target for the drift and contract steps above, MonkeyCode's free model access and free server are a reasonable place to start, provided you read the limits first.

8. Who should not take this route

Skip this approach if you need contractual capacity guarantees, if your data cannot leave your current jurisdiction or processor list, or if you have no way to run a scheduled live test — in that last case a free endpoint becomes an unmonitored dependency very quickly. Teams with hard latency SLOs should also keep production on a paid tier and use the free endpoint only for the fixture and contract lanes described here.

9. Pre-cutover checklist

Every fixture file carries provider, model, prompt hash, and tool schema hash.
Contract assertions are separated from content assertions in the test tree.
rerecord runs without --write and you have read the drift report line by line.
The nightly live lane exists, is small, and has a named owner for its alerts.
Leftover items are written into the ticket with owners, not left as verbal reminders.
The manifest is regenerated after the final re-record, not before it.

None of these steps are clever, and that is the point — the fixtures were never the interesting part of the migration, but they are the part that decides whether you find out about a break at 2am or at 2pm.

Migration Diary: The Streaming Parser Is What Actually Breaks at Cutover

Riley Xu — Mon, 26 Oct 2026 11:56:58 +0000

Conclusion first: your JSON will diff clean on the first day of the migration, and the server-sent stream underneath it will still hand you truncated answers, half-assembled tool calls, and a finish_reason that never fires. Transport is the layer that fails quietly, because nothing raises and nothing looks wrong until an agent loop calls a function with {} as its arguments.

This entry in the migration diary is about that layer: recording real streams as byte fixtures, reducing every provider's chunk shape into one event model, and gating the cutover on structural equality rather than a spot check you ran once in a notebook.

Why the body passes while the stream fails

A non-streaming request returns a document you can validate, diff, and archive; a streaming request returns a sequence of deltas whose contract lives in details that most API references never fully specify. Two endpoints can both claim OpenAI compatibility and still disagree about where usage appears, whether [DONE] is sent at all, and how tool-call arguments are split across chunks. Your integration test compares the final message, so it goes green, and your UI still breaks for the second user who tries a tool call.

Step 1: Record the stream as bytes, not as a transcript

Copying a response out of a terminal gives you a cleaned-up paraphrase with keep-alive comments stripped and line endings normalized, which is exactly the information you need to keep. Record the raw bytes instead, one file per shape you care about.

mkdir -p fixtures
curl -sN "$OLD_HOST/v1/chat/completions" \
  -H "Authorization: Bearer $OLD_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  --data @requests/tool-call.json \
  > fixtures/old-toolcall.sse

Capture at least four shapes before you touch the client: a plain text answer, a multi-chunk tool call, a stop-sequence hit, and an error object returned after a 200 status line. Those four cover the failure modes that survive a happy-path demo, and each one costs you a few kilobytes of disk.

Step 2: Reduce every chunk shape to one event model

The normalizer below treats the stream as a state machine rather than a string split, which is what lets you key tool-call fragments by index instead of by arrival order. It ignores comments and unknown fields on purpose, because a parser that throws on an unrecognized line is a parser that breaks the first time a provider adds one.

import json
from dataclasses import dataclass, field


@dataclass
class Event:
    kind: str                    # text | tool_call | usage | end
    text: str = ""
    index: int = -1
    name: str = ""
    args: str = ""
    usage: dict = field(default_factory=dict)


def events_from_lines(lines):
    tool_name, tool_args = {}, {}
    for raw in lines:
        line = raw.decode("utf-8", "replace").rstrip("\r\n")
        if not line or line.startswith(":"):
            continue                              # keep-alive comment
        if not line.startswith("data:"):
            continue                              # event:, id:, retry:
        payload = line[5:].strip()
        if payload == "[DONE]":
            yield Event("end", text="[DONE]")
            return
        chunk = json.loads(payload)
        if chunk.get("usage"):
            yield Event("usage", usage=chunk["usage"])
        for choice in chunk.get("choices") or []:
            delta = choice.get("delta") or {}
            if delta.get("content"):
                yield Event("text", text=delta["content"])
            for call in delta.get("tool_calls") or []:
                i = call.get("index", 0)
                fn = call.get("function") or {}
                if fn.get("name"):
                    tool_name[i] = fn["name"]
                frag = fn.get("arguments") or ""
                tool_args[i] = tool_args.get(i, "") + frag
                yield Event("tool_call", index=i, name=tool_name.get(i, ""), args=frag)
            if choice.get("finish_reason"):
                yield Event("end", text=choice["finish_reason"])

Two properties matter more than the parsing itself. The reducer is incremental, so a fragment that arrives in a later chunk still lands on the right tool call, and it never blocks on the socket, so a stalled stream shows up as a timeout you control rather than a hung request handler. That is the difference between a cutover you can debug and one you can only restart.

Step 3: Prove the reducer against a fixture, then against a live stream

Write the fixture test first, because it fails for reasons you can read in a traceback instead of reasons buried in a timeline. The version below splits the argument JSON at an arbitrary offset, which is the failure you are actually guarding against.

import json, pytest

ARGS = '{"q":"x"}'

def sse(obj):
    return ("data: " + json.dumps(obj) + "\n").encode()

FIXTURE = [
    b": keep-alive\n",
    sse({"choices": [{"index": 0, "delta": {"role": "assistant"}}]}),
    sse({"choices": [{"index": 0, "delta": {"tool_calls": [
        {"index": 0, "function": {"name": "search", "arguments": ARGS[:4]}}]}}]}),
    sse({"choices": [{"index": 0, "delta": {"tool_calls": [
        {"index": 0, "function": {"arguments": ARGS[4:]}}]}}]}),
    sse({"choices": [{"index": 0, "delta": {}, "finish_reason": "tool_calls"}]}),
    b"data: [DONE]\n",
]

def test_arguments_survive_reassembly():
    log = list(events_from_lines(FIXTURE))
    args = "".join(e.args for e in log if e.kind == "tool_call")
    assert json.loads(args) == {"q": "x"}
    assert log[-1].kind == "end" and log[-1].text == "tool_calls"

Now point the same reducer at the candidate endpoint and compare shapes rather than text, since the model's wording will legitimately differ while the protocol should not.

import urllib.request

def http_lines(url, key, body):
    req = urllib.request.Request(
        url,
        data=json.dumps(body).encode("utf-8"),
        headers={"Authorization": "Bearer " + key,
                 "Content-Type": "application/json",
                 "Accept": "text/event-stream"},
    )
    with urllib.request.urlopen(req, timeout=120) as resp:
        for line in resp:
            yield line

@pytest.mark.parametrize("fixture", sorted(Path("fixtures").glob("*.sse")))
def test_candidate_stream_shape(fixture):
    recorded = list(events_from_lines(fixture.read_bytes().splitlines(keepends=True)))
    live = list(events_from_lines(http_lines(CANDIDATE_URL, CANDIDATE_KEY, payload_for(fixture))))
    assert [e.kind for e in recorded] == [e.kind for e in live]

This is where a free endpoint is genuinely useful rather than decorative: you need a second stream to replay against, and you do not want to pay twice for a rehearsal that may run fifty times in an afternoon.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option, which is enough to stand up that second endpoint for the harness above; treat both as availability claims you verify on the project page rather than as guarantees, and measure the limits yourself before you plan around them.

Step 4: Classify each difference before you argue about it

Not every mismatch is a blocker, and treating all of them as fatal is how a migration stalls for a week. Sort each one into cosmetic, negotiable, or fatal, then fix only the last two.

Difference	Typical symptom	Verdict
Keep-alive comment lines	parser raises on `: ping`	cosmetic, but fix the parser
No `[DONE]` marker	spinner never stops	fatal for any UI that waits on it
`usage` missing or late	cost ledger reads zero	negotiated: count tokens locally
Tool arguments split differently	`{}` passed to a function	cosmetic if you key by index
`finish_reason` null on the last chunk	agent loop never exits	fatal
HTTP 200 then error object	silent empty answer	fatal, needs its own branch

The leftovers you will forget to delete

Every migration leaves wiring that still points at the old provider, and the stream layer hides several of them. Delete the retry wrapper keyed on the old error envelope, or it will retry a 400 that the new endpoint means literally. Retire the hardcoded assumption that only one tool call can be in flight, because the second one will arrive later as a duplicate action. Re-point the timeout tuned to the old first-token latency, and remove the dashboard-shaped usage read before you close the account you were reading it from. None of these announce themselves; they surface as odd behavior three weeks after the cutover, when nobody remembers which endpoint is answering.

Who should not do this

Do not run this migration if you depend on provider-specific features such as logprobs or enforced structured output, because a normalizer cannot invent them. Skip it if you cannot record fixtures, since replaying unrecorded traffic means testing against a moving target. And do not put a free tier underneath a paid SLA without a fallback path you have actually exercised, because availability claims and availability guarantees are different documents.

The harness above is deliberately small: one reducer, one fixture per shape, one structural assertion per endpoint. Run it before the cutover rather than after, and keep the fixtures in the repository next to the tests, because they are the only surviving record of what the old stream actually did.

Migration Diary: Rebuild the Cost Ledger You Lose When the Meter Disappears

Riley Xu — Tue, 20 Oct 2026 10:01:05 +0000

The short version: a paid AI vendor ships a billing dashboard, and a free endpoint ships a rate-limit error and very little else. So the first artifact you build before a cutover is not a prompt suite or a load test, but your own per-call usage ledger, because the meter you lose is the exact instrument you need to decide which workloads stay on free capacity and which ones earn a paid fallback.

MonkeyCode is an open-source project that, per its operator, offers free model access and a free server option; the free token allowance was advertised as 10 million tokens at the time of writing. Treat every quota number as perishable and confirm the current figure on the project page before you size a workload around it. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

1. Start with an inventory of what the meter was giving you

Before you cancel anything, open the old billing page one last time and write down the four things it actually provided. You got per-call cost attribution, which told you that one feature burned most of the budget. You got a monthly trend line, which was your only early warning that a retry loop had started doubling traffic. You got quota headroom signals, and you got an invoice you could hand to someone for audit purposes.

Three of those four are reconstructible from your own request logs, and the fourth exists on the free side too. Nothing here requires getting clever, but it does require deciding the record shape before your first free request lands, because retrofitting identifiers into a running pipeline is miserable.

2. Define a ledger record you can actually reconcile

Keep the record flat, append-only, and free of prompt text. A single JSON line per model call is enough to answer the questions that matter at the end of a week, and the omission of prompt bodies keeps the file safe to copy between machines.

{"ts":"2026-10-20T09:12:03Z","run_id":"cutover-7","route":"free","feature":"summarize","prompt_tokens":812,"completion_tokens":144,"latency_ms":2310,"status":200,"fallback":false}

The feature field is the one people skip and then regret, because without it you can only report totals. The fallback boolean lets you count the calls that quietly escaped to a paid route, and that count is the real cost of the migration. The run_id header lets you join the ledger to your existing traces whenever something looks wrong.

3. Put a recording proxy in front of every model call

You want one hop between your application and any model endpoint, and that hop writes the ledger. The reference sketch below is a Node/Express proxy you should adapt and test against your own stack; it is not a hardened gateway, and it deliberately never inspects or stores message content.

// ledger-proxy.mjs — reference sketch, adapt and test before relying on it
import express from "express";
import { appendFile } from "node:fs/promises";

const app = express();
app.use(express.json({ limit: "2mb" }));

const UPSTREAM = process.env.UPSTREAM_BASE_URL;
const KEY = process.env.UPSTREAM_API_KEY;
const LEDGER = process.env.LEDGER_PATH ?? "./ledger.jsonl";

app.post("/v1/chat", async (req, res) => {
  const started = Date.now();
  const route = req.get("x-route") ?? "free";
  const feature = req.get("x-feature") ?? "unknown";
  let status = 200, usage = {}, fallback = false;

  try {
    const r = await fetch(`${UPSTREAM}/chat/completions`, {
      method: "POST",
      headers: {
        "content-type": "application/json",
        authorization: `Bearer ${KEY}`,
      },
      body: JSON.stringify(req.body),
    });
    status = r.status;
    const data = await r.json();
    usage = data.usage ?? {};
    res.status(r.status).json(data);
  } catch {
    status = 502;
    fallback = true;
    res.status(502).json({ error: "upstream_unavailable" });
  } finally {
    await appendFile(LEDGER, JSON.stringify({
      ts: new Date().toISOString(),
      run_id: req.get("x-run-id") ?? "adhoc",
      route,
      feature,
      prompt_tokens: usage.prompt_tokens ?? null,
      completion_tokens: usage.completion_tokens ?? null,
      latency_ms: Date.now() - started,
      status,
      fallback,
    }) + "\n");
  }
});

app.get("/healthz", (_req, res) => res.json({ ok: true }));
app.listen(process.env.PORT ?? 8787);

Note the ?? null on the token fields. Some endpoints return a usage object and some do not, so a null count is honest information rather than a bug. When usage is missing you can estimate locally with a tokenizer and store the estimate in a separate field, but never mix estimates and reported counts in one column.

4. Run the proxy on the free server option

This is where free capacity pays for itself twice, because the proxy is small, boring, and does not need a GPU. Start it, point your application's base URL at the proxy, and verify with one request before you touch any production configuration.

npm init -y && npm i express
PORT=8787 UPSTREAM_BASE_URL="$MONKEYCODE_BASE_URL" UPSTREAM_API_KEY="$MONKEYCODE_KEY" node ledger-proxy.mjs

curl -s localhost:8787/v1/chat \
  -H 'content-type: application/json' \
  -H 'x-feature: summarize' \
  -H 'x-run-id: smoke-1' \
  -d '{"messages":[{"role":"user","content":"ping"}]}' | head -c 300

Then handle the boring failure modes: run it under a supervisor so a restart is automatic, ship the JSONL file off the host on a timer so a wiped disk cannot erase your history, and keep an eye on whether the free server's disk and memory fit both the proxy and whatever tooling you colocate.

5. Reconcile weekly and route on evidence

Once you have a week of records, the ledger stops being overhead and starts being a decision tool. This jq one-liner gives you per-feature call volume, token totals, and a rough p95 latency that is good enough for triage.

jq -s 'group_by(.feature) | map({
  feature: .[0].feature,
  calls: length,
  prompt: (map(.prompt_tokens // 0) | add),
  completion: (map(.completion_tokens // 0) | add),
  errors: (map(select(.status >= 500)) | length),
  p95_ms: (map(.latency_ms) | sort | .[((length * 0.95) | floor)]) 
})' ledger.jsonl

Feed the output into a routing table like this one, and revisit it monthly rather than per incident.

Workload	Default route	Escalate when	Why
Batch summarization	free	error rate over 5% for an hour	latency is irrelevant, throughput matters
Code review comments	free, then paid on failure	two consecutive upstream errors	developer already waits on CI
Multi-step tool calling	paid until proven	never, until replay tests pass	partial tool state is expensive to unwind
Latency-critical chat	paid	never	user-visible p95 is a product constraint
Regulated or PII payloads	self-hosted	never	third-party routing needs a review

6. Write down the leftovers before you cancel the old plan

The leftovers are the parts of the old workflow that were never about the model in the first place. Prompt caches keyed to a provider's caching semantics usually do not survive a cutover. Evaluation baselines pinned to a specific model version stop being comparable the moment the model behind the endpoint changes without notice. Billing alerts that referenced a spend threshold have no meaning when spend reads zero.

Also inventory the human leftovers: IDE plugins tied to a seat license, shared dashboards bookmarked by teammates, and the retry policy you tuned against a provider's specific error codes. Each of those needs an owner and a replacement date, or it silently becomes tribal knowledge that leaves when the last person who configured it does.

Limitations and who should not do this

A proxy is a single point of failure, so you need it supervised, and you need a documented switch that bypasses it during an incident. Local token counts are unreliable whenever the upstream omits usage, which means your reconciliation is approximate rather than exact. Free capacity changes shape over time, so any routing table built on today's numbers needs a scheduled review.

Skip this approach entirely if you carry regulated data that cannot transit an intermediary, or if you owe users a latency SLO with contractual teeth. Teams that need a vendor SLA and a support escalation path should stay on a paid contract and use the ledger as a cost report rather than a cutover instrument.

If you want a low-stakes place to run step 3, the free model access and the free server option are enough to host the proxy and absorb the calls while you collect that first week of records. Build the ledger first, read it honestly, and let the numbers — not the price tag — decide what stays free.

Corrections welcome, especially if your endpoint returns usage in a shape the sketch above mishandles.

Migration Diary: Rehearse the Rollback Before the Free Endpoint Becomes Primary

Riley Xu — Mon, 19 Oct 2026 08:44:41 +0000

The cutover is the easy half of a migration. The half that actually bites is proving you can go back inside your support window, with the same request IDs, the same stored history, and the same tool-call ordering you had an hour before the switch.

Most migration plans I have seen spend their detail budget on the forward path: who flips the flag, when DNS moves, which dashboard to watch. Almost none of them contain a scripted, repeatable rollback, so when the free endpoint starts behaving differently the team improvises under pressure. This entry is the artifact I now build before any cutover: a rollback drill you run twice, once while you still have a comfortable fallback.

1. Write down what "rolled back" means before you touch anything

A rollback is not a config change. It is a claim about state, and claims need definitions you can test. Fill in this inventory for your own system first, because the blanks are where migrations fail quietly.

State	Where it lives	Re-derivable from the fallback?
Trace / request IDs	Your gateway, provider logs	No — you must own the ID
Conversation history + tool calls	Your database	Partly — depends on your schema
Idempotency keys	Your ledger	Yes, if you froze the ledger
Prompt / rule pack version	Your repo, maybe a sidecar	Yes, if it is versioned
Evaluation baseline scores	Your eval store	No — snapshot before cutover
Rate-limit and quota budgets	Provider dashboards	No — record the numbers

Only the middle two rows are usually safe. Everything else needs an owner and a copy that lives with you, not with the provider you are leaving.

2. Put the switch in configuration, and make fallback loud

A rollback you cannot trigger from an environment variable is not a rollback, it is a meeting. Route every model call through one dispatch point, and make the default behaviour fail loudly instead of silently drifting to a different backend.

# backend.py - template, adapters omitted
import os

BACKENDS = {"primary": PrimaryAdapter(), "fallback": FallbackAdapter()}
ACTIVE = os.environ.get("AI_BACKEND", "primary")

if ACTIVE not in BACKENDS:
    raise RuntimeError(f"unknown AI_BACKEND={ACTIVE!r}")

def complete(request, *, backend: str | None = None) -> dict:
    name = backend or ACTIVE
    adapter = BACKENDS[name]
    try:
        return adapter.complete(request)
    except adapter.RetryableError:
        raise  # never fall back implicitly during a drill

The raise on retryable errors is deliberate. Automatic fallback turns your rollback into an untested code path that only executes during an incident, which is exactly when you cannot afford to discover it is broken.

The same rule applies on the server side: keep the old base URL, credentials, and worker image alive and addressable until the drill passes. Deleting the paid workspace on cutover day removes the only thing your rollback was going to talk to.

3. Build the drill harness in four steps

Step 1: Freeze a fixture set

Capture 30 to 50 real requests that cover your awkward cases: long context, tool calls, retries, malformed model output, and one deliberately invalid request. Strip user data, keep the structure, and commit the fixtures. These are the same inputs you will replay against both backends.

Step 2: Record artifacts, not just answers

For each fixture, capture the response text, the ordered list of tool names invoked, the error code if any, and the token counts you were billed. Answer quality is a human judgement; the artifacts above are machine-comparable.

# drill.py - template; normalization rules must be written by you
import json, pathlib

def normalize(resp: dict) -> dict:
    return {
        "tool_order": [c["name"] for c in resp.get("tool_calls", [])],
        "error": resp.get("error_code"),
        "shape": sorted(resp.get("content", {}).keys()),
        "prompt_tokens": resp.get("usage", {}).get("prompt_tokens"),
    }

def run(fixtures: pathlib.Path, backend: str) -> dict:
    out = {}
    for f in sorted(fixtures.glob("*.json")):
        req = json.loads(f.read_text())
        out[f.stem] = normalize(complete(req, backend=backend))
    return out

def diff(a: dict, b: dict) -> list[str]:
    return sorted(k for k in a if a[k] != b.get(k))

Step 3: Flip the switch and replay

Run the fixtures against the fallback first, then against the candidate. Run the drill three times, because a shared free tier will not give you identical latency or identical sampling twice.

Step 4: Assert, then time the flip

Compare the normalized artifacts and record how long the actual environment-variable change takes to reach every worker. That number is your real recovery time, and it is usually larger than the plan assumed because of cached config, container restarts, or a queue that drains slowly.

4. Score the result with a table, not a feeling

Check	Pass condition	If it fails
Tool-call order	Identical for every fixture	Block cutover
Error taxonomy	Every new code mapped to your retry policy	Block cutover
Response shape	Schema-valid after your parser	Block cutover
Token accounting	Counts present and attributable to a request ID	Warn, fix within a week
Text drift	Reviewed by a human on 10 fixtures	Warn, canary eligible

Note that latency is not in the pass column. Latency belongs in monitoring after cutover, not in a go/no-go vote, because a shared free tier is not a latency contract.

5. Inventory the leftovers before you celebrate

Every migration leaves residue, and naming it in advance stops it from becoming folklore six weeks later.

Credentials used during the drill. Rotate keys you created for testing, including on the throwaway server.
Scheduled jobs. Cron entries and queue consumers still pointing at the old base URL are the classic silent leftover.
Log and trace sinks. Assume the target keeps less than you are used to and export to your own storage.
Billing alarms. Keep them until the old workspace is genuinely deleted, then delete the alarms too.
Prompt cache warmup. Cold starts cost more tokens and produce different timing on day one.

6. Where the free tier fits in this drill

This is the part where the tooling matters. MonkeyCode offers free model access and a free server option, and the useful role for both in this workflow is a second environment: you can run the rollback drill against a scratch server without borrowing production capacity or paying for another workspace. The operator states that free access comes with a 10M-token allowance and that the server side is available at no cost; treat those numbers as current-at-time-of-writing and confirm the live terms before you plan a budget around them. The project is described as open source, so check the repository and license directly rather than trusting a diary entry.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I would use it here specifically because a rollback drill is disposable work: it needs an environment for an afternoon, produces artifacts you keep, and should never touch real traffic.

Limitations and who should skip this

This drill proves state parity and switch mechanics. It does not prove quality parity, and it does not tell you whether a different model family will pass your evaluations. If your workload has a contractual latency SLO, a data-residency requirement in a specific region, or a regulated audit trail, a free shared tier is the wrong primary and you should stay on a paid agreement you can point at in a review.

If your migration is a single internal script with no stored history, the whole table above is overkill and a one-page checklist will do. And if nobody owns the fixture set, the drill will rot within a month, so assign that person before you assign the cutover date.

Start with the rollback, not the rollout. If you need a spare environment to rehearse in before you commit, the free server is a low-risk place to run the first three drill cycles.

Migration Diary: Prove the Free Endpoint Can Hold Your Token Load Before Cutover

Riley Xu — Sun, 18 Oct 2026 07:22:33 +0000

You should not cancel the paid key until an offline projection shows your real traffic fits the free allowance with slack left over. This entry is the fifth step in my migration diary, and it covers the part everyone skips: proving the token math before the cutover window opens. The artifact below is a small, dependency-free Python gate that reads your existing usage logs, projects a full month, and fails loudly when the headroom is too thin. If it passes, you move on to the leftovers inventory; if it fails, you keep paying and you know exactly why.

Most teams measure tokens by reading a dashboard once, dividing by days, and multiplying by thirty. That estimate quietly assumes the new endpoint tokenizes text the same way the old one did, which is rarely true. A five to fifteen percent drift on the input side is common, and it is enough to push a tight budget over the edge mid-month. Measure the drift instead of assuming it, then feed the measured number into the projection.

Step 1 — Freeze a usage corpus from logs you already have

Start with request-level records, not monthly totals, because you need a distribution rather than an average. Any structured log line that carries prompt and completion token counts will do, and you only need fourteen to thirty days of history. Export to JSONL so the gate can stream records without loading everything into memory.

# From an NDJSON log stream that carries a usage object per request
jq -c 'select(.usage != null)
  | {day: .timestamp[0:10],
     input_tokens: .usage.prompt_tokens,
     output_tokens: .usage.completion_tokens}' \
  usage.ndjson > corpus.jsonl

Two details matter here. First, keep the day field, because the gate needs the number of distinct observed days to compute a daily rate; second, drop records where usage is null instead of substituting zeros. Zero-filled rows will drag your daily rate down and hand you a falsely optimistic projection.

Step 2 — Measure tokenizer drift with a small probe

Before projecting, send a fixed sample of real prompts to the target endpoint and compare its reported counts against your local estimate. Thirty to fifty prompts is enough to get a usable multiplier, and you should sample across your longest prompts rather than the average ones. Record the ratio per request and take the high percentile, not the median, because your tail is what breaks the budget.

# probe_drift.py (sketch — wire in your own client call)
from statistics import quantiles

ratios = []
for prompt in sample_prompts[:50]:
    local_tokens = len(prompt.split())          # placeholder local estimate
    reported = call_target(prompt)               # returns input token count
    ratios.append(reported / local_tokens)

ratios.sort()
drift = quantiles(ratios, n=20)[-1]              # p95 multiplier
print(f"apply drift multiplier: {drift:.3f}")

Use the p95 multiplier, not the mean, when you feed --drift into the gate. If your local estimate is good, the multiplier lands near 1.0 and the whole step costs you a few cents and ten minutes. If it lands at 1.2, that is not noise; that is the reason your first week on a free tier would have ended early.

Step 3 — Run the budget gate

This script is deliberately offline: it makes no network calls, needs no SDK, and can run in CI against a committed corpus. It exits 0 when the projection fits with your required headroom, 1 when it does not, and 2 when the corpus is empty so you never get a silent pass. Keep the allowance as a flag rather than a constant so a pricing change is a one-line diff.

#!/usr/bin/env python3
"""budget_gate.py - offline token-budget gate for an AI endpoint cutover."""
from __future__ import annotations

import argparse
import json
import statistics
import sys


def load_corpus(path: str) -> list[dict]:
    records = []
    with open(path, encoding="utf-8") as handle:
        for line_number, line in enumerate(handle, start=1):
            line = line.strip()
            if not line:
                continue
            try:
                raw = json.loads(line)
                records.append({
                    "day": str(raw["day"]),
                    "tokens": int(raw["input_tokens"]) + int(raw["output_tokens"]),
                })
            except (KeyError, TypeError, ValueError) as exc:
                sys.exit(f"{path}:{line_number}: bad record ({exc})")
    return records


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("corpus")
    parser.add_argument("--allowance", type=int, default=10_000_000)
    parser.add_argument("--days", type=int, default=30)
    parser.add_argument("--headroom", type=float, default=0.20)
    parser.add_argument("--drift", type=float, default=1.0)
    args = parser.parse_args()

    records = load_corpus(args.corpus)
    if not records:
        print("empty corpus: refusing to project a budget from nothing")
        return 2

    observed_days = len({record["day"] for record in records})
    total = sum(record["tokens"] for record in records)
    daily = total / observed_days * args.drift
    projected = daily * args.days
    per_request = sorted(record["tokens"] for record in records)
    p95 = statistics.quantiles(per_request, n=20)[-1] if len(per_request) >= 20 else per_request[-1]
    ceiling = args.allowance * (1 - args.headroom)

    print(f"observed days      : {observed_days}")
    print(f"requests           : {len(per_request)}")
    print(f"tokens (recorded)  : {total:,}")
    print(f"drift multiplier   : {args.drift:.3f}")
    print(f"projected {args.days}d     : {projected:,.0f}")
    print(f"budget ceiling     : {ceiling:,.0f}")
    print(f"p95 request tokens : {p95:,.0f}")

    if projected > ceiling:
        print("FAIL: projected load does not fit with the requested headroom")
        return 1
    print("PASS: projected load fits")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run it three times with different headroom values to see how brittle the fit is: twenty percent, ten percent, and zero. A workload that passes at zero but fails at ten is not ready, because one retry storm or one batch backfill will consume the slack you convinced yourself you had.

python3 budget_gate.py corpus.jsonl --drift 1.14 --headroom 0.20

Step 4 — Inventory the leftovers before you delete anything

Cutover is not the moment you stop calling one URL and start calling another; it is the moment you find out which side systems were quietly pinned to the old provider. Walk this table with your own repository open, and mark each row honestly rather than aspirationally.

Leftover	Symptom if ignored	Action
Prompt-shim layer	Prompts tuned to one model degrade on another	Keep the shim, rewrite its rules
Retry and backoff config	Backoff tuned to paid latency hammers a shared server	Re-derive from new error mix
Eval baseline	Scores are pinned to the old model's output	Re-record with labelled provenance
Cost and usage dashboard	Alerts fire on a key you no longer hold	Repoint at the new usage source
Prompt cache warmer	Budget spent warming a cache nobody reads	Delete it
Fallback route	Silent failover resurrects the cancelled key	Remove the credential, not just the call

The last row is the one that bites hardest in my experience, because a dormant fallback path can sit unreferenced for weeks and then reactivate during an incident. Remove the credential itself rather than the code path that uses it, so a stale branch cannot spend money you thought you had stopped spending.

Where a free tier fits into this cutover

This is the point in the diary where the tooling matters, so here is the honest version. MonkeyCode offers free model access and a free server option, and the operator of this account states the free tier includes 10 million tokens; treat that figure and the server option as claims to re-verify on the vendor's pricing page on the day you plan the cutover, not as constants you can bake into a three-month roadmap. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The reason the gate above exists is that a fixed allowance converts a soft cost problem into a hard capacity problem. With metered billing, a traffic spike costs more money; with a fixed allowance, a traffic spike costs you availability, and availability failures are much harder to explain to users than a larger invoice. The gate turns that risk into a build failure you can see on a Tuesday afternoon instead of an outage on a Saturday night.

Limitations and who should skip this

This workflow needs request-level logs with token counts, and if your observability pipeline only stores aggregates you should fix that first rather than reconstructing a corpus from memory. It also assumes a relatively stable traffic shape; a product launch, a seasonal spike, or a new batch job invalidates the projection and requires a fresh corpus. The projection is an estimate with an explicit confidence boundary, not a guarantee, and you should re-run it monthly while the workload is young.

Skip this approach if you carry hard latency or availability obligations that a shared free endpoint cannot contractually meet, or if your data cannot legally leave its current processor. Skip it as well if you have no way to remove the old credential, because a migration you cannot reverse is a migration you should not start on a Friday. None of the code here makes a free tier suitable for regulated workloads; it only tells you whether the arithmetic works.

If you run the gate on your own corpus this week, the useful output is not the pass or fail line but the p95 request size, because that number tells you which single prompt in your product is quietly eating the budget. Start there, and let the cutover wait until the projection agrees with you.

Migration Diary: Rehydrate Stored Chat History Without Orphaned Tool Calls

Riley Xu — Fri, 16 Oct 2026 05:28:02 +0000

Migration Diary: Rehydrate Stored Chat History Without Orphaned Tool Calls

The first thing that breaks in a model migration is rarely the prompt; it is the stored transcript you replay into the new endpoint. Tolerant endpoints hide structural damage for months: tool results whose matching call vanished during last month's context trimming, call ids reused across turns, extra fields nobody ever validated. A stricter endpoint evaluates the same array on arrival, so your cutover dies on message seven with a schema error that describes the provider instead of your actual bug. You can prevent most of that with a small guard that validates and repairs history before rehydration, and this diary entry is the cutover plan for building it.

1. Write the invariants down before you compare vendors

You will not debug a migration by reading two documentation pages side by side, because the failure usually lives in your stored data rather than in their prose. List the structural invariants your transcript store must satisfy instead, then treat every endpoint difference as a test case to run rather than an argument to win.

Every tool result references a call id that appears in an earlier assistant message.
Every assistant tool call has exactly one result, except a call that is still pending at the tail.
Call ids are unique across the whole session, not merely unique inside a single turn.
Role ordering satisfies the strictest endpoint you might ever send stored history to.
Unknown fields are stripped at the boundary, so a rehydrated payload contains only fields you deliberately keep.

Invariants one and two fail most often in practice, and the cause is usually your own trimming logic. A tail-window trimmer keeps the newest messages and cuts the older assistant call that owned a tool result, leaving a result with no parent. One endpoint ignores that orphan silently, another rejects the request, and your logs end up blaming the new provider for a bug you shipped weeks earlier.

2. Decide what to keep, drop, and rewrite

Before writing any code, agree on a repair policy, because "normalize the history" means nothing to a teammate reading a diff at 2 a.m.

Leftover in the stored history	Verdict before rehydrate	Why
Tool result with no earlier call	Drop	It has no parent turn, so no endpoint can attach it to anything
Assistant tool call with no stored result	Drop the call, keep the text	The call is unresolvable; leaving it invites a strict validator to fail
Duplicate call id inside one session	Reassign new ids	Uniqueness lets you match results to calls after a repair
Unknown or provider-specific fields	Strip at the boundary	Strict validators reject payloads they never documented
Duplicated system prompt	Keep the first one	Two system turns change precedence in ways you cannot predict
Very long middle turns	Trim the middle, keep head and recent tail	Preserves the original task statement and the current state

Rewriting is not a synonym for dropping, and this is the part teams get wrong. When you strip an assistant's tool call, you must leave behind something a model can read, or the following turn looks as if the user answered themselves. Keep the stored transcript immutable and repair a copy, because someone will eventually ask why one session behaved differently on either side of the cutover.

3. The guard: validate first, repair second

The example below uses a plain list of message dictionaries, which maps cleanly onto most chat APIs. It has no third-party dependencies, so you can drop it into an existing service or run it as a standalone CLI over an export file.

# transcript_guard.py -- keep stored chat history structurally valid
# before you rehydrate it against a different model endpoint.

from __future__ import annotations

from dataclasses import dataclass
from typing import Any, Iterable

Message = dict[str, Any]
KEEP_AS_IS = {"role", "content", "tool_calls", "tool_call_id", "name"}


@dataclass(frozen=True)
class Problem:
    code: str
    index: int
    detail: str

    def __str__(self) -> str:
        return f"[{self.code}] message #{self.index}: {self.detail}"


def check_transcript(
    messages: Iterable[Message], *, allow_trailing_call: bool = True
) -> list[Problem]:
    """Return structural problems; an empty list means the history is rehydratable."""
    msgs = list(messages)
    problems: list[Problem] = []
    pending: dict[str, int] = {}
    answered: set[str] = set()

    for i, msg in enumerate(msgs):
        role = msg.get("role")
        if role == "assistant":
            for call in msg.get("tool_calls") or []:
                cid = call.get("id")
                if not cid:
                    problems.append(Problem("call_without_id", i, "tool call emitted with no id"))
                elif cid in pending or cid in answered:
                    problems.append(Problem("duplicate_call_id", i, f"call id {cid!r} reused"))
                else:
                    pending[cid] = i
        elif role == "tool":
            cid = msg.get("tool_call_id")
            if cid not in pending:
                problems.append(Problem("orphan_result", i, f"result {cid!r} has no earlier call"))
            else:
                answered.add(cid)
                pending.pop(cid)

    unresolved = sorted(pending.items(), key=lambda kv: kv[1])
    if allow_trailing_call and unresolved and unresolved[-1][1] == len(msgs) - 1:
        unresolved = unresolved[:-1]  # a call at the tail is a request for work, not damage
    for cid, idx in unresolved:
        problems.append(Problem("unanswered_call", idx, f"call {cid!r} never received a result"))
    return problems


def purge_orphans(messages: Iterable[Message]) -> list[Message]:
    """Drop both halves of every incomplete call/result pair. Returns a new list."""
    msgs = [dict(m) for m in messages]
    call_ids = {
        c.get("id")
        for m in msgs
        if m.get("role") == "assistant"
        for c in (m.get("tool_calls") or [])
    }
    result_ids = {m.get("tool_call_id") for m in msgs if m.get("role") == "tool"}

    repaired: list[Message] = []
    for m in msgs:
        role = m.get("role")
        if role == "tool":
            if m.get("tool_call_id") in call_ids:
                repaired.append(m)
            continue
        if role == "assistant" and m.get("tool_calls"):
            kept = [c for c in m["tool_calls"] if c.get("id") in result_ids]
            if kept:
                m["tool_calls"] = kept
            else:
                m.pop("tool_calls", None)
                if not str(m.get("content") or "").strip():
                    m["content"] = "[tool call dropped during migration: no stored result]"
        repaired.append(m)
    return repaired


def rehydrate(
    messages: Iterable[Message],
    *,
    keep_last: int = 8,
    max_chars: int = 20_000,
    strict: bool = True,
) -> list[Message]:
    """Produce a transcript that is safe to send to a stricter endpoint."""
    msgs = purge_orphans(messages)
    problems = check_transcript(msgs, allow_trailing_call=False)
    if problems and strict:
        raise ValueError("transcript is not rehydratable: " + "; ".join(map(str, problems)))

    system = [m for m in msgs if m.get("role") == "system"]
    rest = [m for m in msgs if m.get("role") != "system"]
    head, tail = rest[:1], rest[1:][-keep_last:]
    while tail and tail[0].get("role") == "tool":
        tail = tail[1:]  # never open the window on a tool result

    window = system + head + tail
    while len(window) > len(system) + 1 and sum(len(str(m)) for m in window) > max_chars:
        window.pop(len(system) + 1)
    return purge_orphans(window)


def strip_unknown_fields(
    messages: Iterable[Message], allowed: set[str] = KEEP_AS_IS
) -> list[Message]:
    """Run this last: it is the boundary that hides provider-specific fields."""
    return [{k: v for k, v in m.items() if k in allowed} for m in messages]


if __name__ == "__main__":
    import json
    import sys

    history = json.load(open(sys.argv[1], encoding="utf-8"))
    found = check_transcript(history)
    for p in found:
        print(p)
    print(f"{len(found)} problem(s) in {len(history)} messages")
    raise SystemExit(1 if found else 0)

Run it as a read-only gate over an export before you touch anything:

python transcript_guard.py transcripts/session-8890.json && echo "clean"

4. Tests that pin the leftovers

A validator without tests is a comment. These three cases encode the failures that actually reached production in migrations I would rather not repeat.

# test_transcript_guard.py
from transcript_guard import check_transcript, purge_orphans, rehydrate

CALL = {"id": "c1", "type": "function", "function": {"name": "get_rate", "arguments": "{}"}}


def test_orphan_result_is_reported():
    history = [
        {"role": "user", "content": "convert 40 EUR"},
        {"role": "tool", "tool_call_id": "c1", "content": "42.10"},
    ]
    assert [p.code for p in check_transcript(history)] == ["orphan_result"]


def test_purge_drops_both_halves_of_an_incomplete_pair():
    history = [
        {"role": "user", "content": "convert 40 EUR"},
        {"role": "assistant", "content": "", "tool_calls": [CALL]},
    ]
    repaired = purge_orphans(history)
    assert "tool_calls" not in repaired[1]
    assert repaired[1]["content"].startswith("[tool call dropped")


def test_trim_never_opens_the_window_on_a_tool_result():
    history = [
        {"role": "system", "content": "be terse"},
        {"role": "user", "content": "turn 1"},
        {"role": "assistant", "content": "", "tool_calls": [CALL]},
        {"role": "tool", "tool_call_id": "c1", "content": "42.10"},
        {"role": "assistant", "content": "about 42 EUR"},
    ]
    window = rehydrate(history, keep_last=2, max_chars=10_000)
    assert [m["role"] for m in window] == ["system", "user", "assistant"]

python -m pytest test_transcript_guard.py -q

The third test is the one that earns its keep, because it fails the moment someone "optimizes" the trimmer and reintroduces the orphan result bug.

5. The cutover sequence

Freeze writes to the transcript store for the length of the rehearsal window, or snapshot it so the export stays consistent.
Export a sample of real sessions, including at least one long agentic session with several tool calls per turn.
Run the checker in read-only mode and record problem counts grouped by code; that histogram tells you which invariant your own code has been violating.
Repair copies with rehydrate, then re-run the checker on the repaired copies and assert zero problems.
Send each repaired session once to the new endpoint and capture only structural outcomes first: rejected, accepted, or accepted with warnings.
Strip unknown fields at the boundary, enable the write path for a small percentage of sessions, and log every structural rejection with its problem codes.
Keep the previous endpoint reachable for one rollback window, then delete the old keys deliberately rather than letting them expire by accident.

Step three deserves emphasis: the histogram is your evidence. If every rejected session shares the same code, you have a bug in your own trimming rather than an incompatibility with a vendor, and you can fix it without touching the migration plan.

6. Using a second endpoint for the structural check

A migration diary is only useful if the replacement path is actually exercised, so the repaired copies need somewhere to go. MonkeyCode offers free model access and a free server option, which I used for the structural pass rather than for quality comparison. Disclosure: This article was prepared as part of MonkeyCode's product outreach. That let me point the validator at a second endpoint without opening a second invoice, and the free server option hosted the scheduled checker as a small job that reports problem counts per day. Availability, limits, and terms are set by MonkeyCode and can change, so read the current terms for the project and confirm what your workload needs before you route real traffic there. If you want to reproduce this on your own transcripts, running the guard against a free endpoint first is the cheapest way to learn which invariants your history has been quietly violating.

7. Limitations and who should not use this

Structural repair has a narrow, honest scope, and it is worth naming where it stops being useful.

Stateless applications that store no history gain nothing; the guard only pays off when you persist and rehydrate conversations.
Regulated or evidentiary transcripts must never be mutated in place. Repair into a separate store with its own retention policy, and keep the original byte-for-byte.
Sessions that exceed the new context budget need summarization, not trimming. Cutting the middle changes what the model can answer, and that is a product decision rather than a migration chore.
Quality regressions are invisible to this guard. A perfectly valid transcript sent to a weaker endpoint still produces worse answers, so keep an output-level check separate.
Free endpoints and free servers can change limits or availability, so avoid making them the only path for a workflow with an SLA.
Silent automatic rewriting on every request hides the problem you are trying to measure; run the guard once at the migration boundary, with logging.

Your transcript store is the one artifact that survives every provider change, every pricing decision, and every rewrite of your prompt templates. Ship the guard in continuous integration, fail the build when orphan results appear, and keep your migration energy for the failures that only show up in output quality.

Migration Diary: Shadow Replay Your Prompts Before the Free Endpoint Takes Over

Riley Xu — Thu, 15 Oct 2026 11:30:05 +0000

The cheapest cutover test is to replay your own recorded prompts through both endpoints and read the diff yourself. You can plan retry classes and response contracts for weeks and still lose on the long tail of inputs nobody remembers writing. Shadow replay converts "it looked fine in the demo" into a divergence count you can defend in a review. This entry covers that step end to end: freeze a corpus, normalize outputs, classify divergence, then decide what you still own after the flip.

There is a familiar shape to the current wave of posts about cheaper or free model access. One happy transcript, one benchmark number, and an implied promise that the swap is boring. That framing hides the expensive part, because the first twenty requests are easy and requests two thousand through twenty thousand are not. Your real job during cutover is finding quiet failures before users do.

Why a model swap fails quietly

Most substitutions do not crash; they degrade in ways that pass a smoke test and fail a review. The candidate model returns the same answer in a slightly different shape, and your downstream parser reads a string where it expected an object. Or it produces a shorter answer, which your UI happily renders as a blank-looking card. These failures are invisible until someone reads the output, which is exactly what replay is for.

The five drift classes worth naming are schema drift, refusal drift, truncation drift, formatting drift, and semantic drift. Schema and truncation are mechanical, so you can catch them with code. Refusal drift usually appears only on your ugliest inputs, such as billing disputes or medical wording. Semantic drift is the residue that needs a human, and you should treat it as a queue rather than an alarm.

Step 1 — Freeze a replay corpus before you touch anything

Record a sample of real production traffic and store it as JSONL, one request per line. Include the parameters you actually sent, because temperature and token caps change the comparison more than the model does. Redact identifiers at capture time rather than at review time, so the corpus stays usable for future migrations.

{"id": "sum-001", "source": "support-ticket", "messages": [{"role": "system", "content": "Summarize the ticket in two bullets."}, {"role": "user", "content": "..."}], "params": {"temperature": 0, "max_tokens": 300}, "baseline": {"model": "paid-flagship", "text": "..."}}

Store the baseline output alongside the request, not only in your logs. If you only keep a pointer, the first thing you will do during a review is re-run the old endpoint, and that reintroduces the cost you are trying to remove. A few hundred well-chosen fixtures beat ten thousand random ones, provided you sample the ugly cases deliberately.

Step 2 — Normalize before you diff

The naive comparison is string equality, and it will drown you in noise from bullets, whitespace, and trailing punctuation. Normalize first, then fingerprint, then compare fingerprints to separate cosmetic churn from real change. The helper below is a skeleton that was not executed against any live provider; treat it as a template you adapt.

# shadow_replay.py — skeleton, not executed against any live provider.
import hashlib, json, pathlib, re

BULLET = re.compile(r"^\s*[-*•]\s+", re.M)
WS = re.compile(r"[ \t]+")


def normalize(text: str) -> str:
    text = BULLET.sub("- ", text)
    text = WS.sub(" ", text)
    return "\n".join(line.strip() for line in text.strip().splitlines())


def fingerprint(text: str) -> str:
    return hashlib.sha256(normalize(text).encode()).hexdigest()[:12]


def json_object(text: str):
    try:
        value = json.loads(text)
    except json.JSONDecodeError:
        return None
    return value if isinstance(value, dict) else None

Normalization rules are an editorial decision, so write them down. If your product sells summaries, bullet markers probably do not matter, but numbers and negation absolutely do. A normalizer that lowercases everything is convenient and dangerous, because it erases "not approved" into "approved" for the diff.

Step 3 — Classify divergence with a decision table

Categories turn a pile of diffs into a work queue with owners and severities. Keep the taxonomy small enough to memorize, and give every class a routing rule rather than a vibe. The table below is the one worth copying into your migration doc.

REFUSALS = ("i can't help", "i cannot assist", "as an ai")


def classify(baseline: str, candidate: str) -> tuple[str, int]:
    if baseline == candidate:
        return "same", 0
    if fingerprint(baseline) == fingerprint(candidate):
        return "cosmetic", 0
    if any(p in candidate.lower() for p in REFUSALS):
        return "refusal", 3
    if len(candidate) < 0.6 * len(baseline):
        return "truncation", 2
    a, b = json_object(baseline), json_object(candidate)
    if a and b and set(a) != set(b):
        return "schema", 3
    return "semantic", 1

Class	Example symptom	Severity	Action before cutover
same / cosmetic	whitespace, bullet glyphs	none	record the pass count, move on
truncation	answer cut at the token cap	high	raise the cap or shorten the prompt
schema	key renamed or nested differently	high	pin the contract, add a validator
refusal	candidate declines a valid task	high	reword or route those inputs
semantic	different but plausible answer	medium	sample for human review

Severity is about blast radius, not about how strange the output looks. A renamed JSON key breaks your parser for every user, while one odd paraphrase breaks nothing. Route high-severity classes to code fixes and medium-severity classes to a review queue you actually staff.

Step 4 — Run the shadow window on a bounded budget

The replay run should be boring: fixed concurrency, a hard cap on requests, and a checkpoint per fixture. Checkpointing matters more than speed, because a half-finished run that you cannot resume tempts you into skipping fixtures. The parallel version below keeps the failure surface small and writes each verdict immediately.

import httpx
from concurrent.futures import ThreadPoolExecutor

OUT = pathlib.Path("replay-out")
OUT.mkdir(exist_ok=True)


def judge(base_url, api_key, model, item, timeout=60):
    payload = {"model": model, "messages": item["messages"], **item.get("params", {})}
    response = httpx.post(
        f"{base_url}/chat/completions",
        headers={"Authorization": f"Bearer {api_key}"},
        json=payload,
        timeout=timeout,
    )
    response.raise_for_status()
    text = response.json()["choices"][0]["message"]["content"]
    verdict, severity = classify(item["baseline"]["text"], text)
    return {"id": item["id"], "verdict": verdict, "severity": severity, "text": text}


def run(items, workers=4):
    with ThreadPoolExecutor(max_workers=workers) as pool:
        futures = [pool.submit(judge, BASE_URL, API_KEY, MODEL, item) for item in items]
        for future in futures:
            row = future.result()
            with (OUT / "verdicts.jsonl").open("a") as fh:
                fh.write(json.dumps(row) + "\n")

Keep concurrency low, because free capacity is shared and a burst looks like an attack rather than a benchmark. Cap the run at a number you can afford to repeat, and record the endpoint and model identifier in every verdict row. When the results look good, re-run a small slice after a day to check whether the endpoint is behaving consistently.

Step 5 — Where MonkeyCode fits in this plan

Shadow replay needs a second endpoint, and that is the practical place a free option helps. According to its operator, MonkeyCode is an open-source project that provides free model access and a free server option, which lets you run the candidate side of the comparison without opening a second paid account.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use it where the replay harness needs a target: point the candidate client at the free endpoint, keep your paid baseline frozen in the fixtures, and let the harness grade both. I have not independently verified the license terms, quota, or server specifications, so confirm the current numbers and limits in the project's own documentation before you quote them in a plan. Availability claims in this space change quickly, and a figure copied from an older post is worse than no figure at all. If you do try it, run the harness on your own fixtures first and let the divergence report decide whether the swap is safe.

The leftovers you still own after cutover

Endpoints change; the surrounding scaffolding does not disappear with them. Expect to keep vendor-specific parameters, prompt caching prefixes, streaming behavior differences, tool schemas, log fields, and the alert thresholds you tuned on the old latency profile. Each one is a small piece of the paid workflow that quietly outlives the account you closed.

The corpus is the most valuable leftover, and it is also the one teams forget to assign an owner to. Name a maintainer, version the fixtures, and add a case every time production surprises you. A replay corpus that nobody updates decays into a museum of prompts from the week of the migration.

Limitations and who should not use this

Replay measures output quality on recorded inputs, and it says nothing about latency under real concurrency. It also cannot grade inputs you never captured, which is usually the tail that hurts. If your prompts contain regulated data you cannot retain, this workflow is not for you without a redaction pipeline that you trust.

Skip shadow replay if a wrong answer is legally or financially catastrophic and every response needs a human sign-off anyway. Skip it too if you cannot freeze a baseline, because a comparison without a fixed reference is just two opinions. For everyone else mid-migration, it is the cheapest evidence you will get before the flip.

Migration Diary: Classify Every Retry Before the Free Server Inherits Paid Backoff

Riley Xu — Tue, 13 Oct 2026 02:27:47 +0000

You should classify every retry before a free server inherits the backoff your paid agent runtime still uses. A policy tuned for a fast metered endpoint can stampede a shared free process or replay a write you already committed. This diary walks you through a cutover plan, a decision table, and a small policy module you can test locally. Nothing here is a benchmark, a quota, or a claim that a free tier will stay available.

Name the leftover before you cancel the paid workspace

You are not only swapping a model name when you leave a paid agent workspace for a cheaper host. You are also leaving timeout defaults, retry headers, and a habit of treating errors as cheap because the invoice made them visible. The leftover sits in the client, the worker, and the CI job that still sleeps and tries again. If you close the paid workspace first, that schedule keeps firing at whatever URL you forgot to rebind.

A free model tier and a free server can host the canary only after you shrink what a retry may do. Operator notes for this draft say MonkeyCode is open source and offers free model access plus a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. You still read current product documentation for limits, because this note states no quotas, hardware, or duration.

The article stays useful if you drop that product and keep the same plan on any free model and free server. You should not copy numbers from a marketing page into a runbook without a primary document dated for your cutover week. Retry classification is an evergreen migration problem, not a claim about this week's model leaderboard or a vendor score. You verify every time-sensitive limit yourself before you publish a runbook or flip any traffic onto the new host.

Decide which failures may try again

You write the decision table before you touch code, and you keep it next to the workflow config. A timeout on the model call is not the same event as a tool that already charged a card or opened a ticket. Transport failures may retry inside a small budget, while committed side effects must stop and wait for a human. Use the table below as a starting contract, then replace the signals with the ones your logs actually emit.

Class	Signal you can observe	On the free server	Next action
Transport	Connection reset, HTTP 502, HTTP 503	One retry, delay set by you	Stop if the budget is spent
Deadline	Client timeout before any tool result	Retry only if no tool started	Record the attempt id
Client fault	HTTP 400 or schema mismatch	No retry	Fix the request, do not loop
Tool committed	Tool returned a receipt or external id	No retry	Reconcile, do not call again
Unknown	Empty body or unparsed error	No retry	Send to the dead-letter path

You treat unknown as stop, because a free server will not tell you whether the paid side effect already landed. You also refuse to retry a client fault, because a tighter free-model parser will keep rejecting the same bad payload. The table is a proposal you adapt to your signals, not a universal standard published by any vendor. If your domain has a safer class split, you write that split down and test it.

Cutover plan

1. Inventory every inherited retry

You search the repository before you argue about policy, because forgotten sleeps cause more damage than a wrong constant. Run the search from the repository root and review every hit that is not a vendored dependency. You are looking for max attempts, backoff helpers, and framework defaults that retry on any exception. Write the file path and the current attempt count into the cutover note before you edit anything.

rg -n -i "retry|backoff|max_attempts|tenacity" --glob '!vendor/**' --glob '!node_modules/**'
git grep -n -E "sleep|wait_exponential|stop_after_attempt" -- "*.py" "*.yml" "*.yaml"

You expect this search to find client code, worker code, and at least one workflow file. If the search returns nothing, you still check framework defaults, because silence does not mean the runtime will try only once. You paste the file list into the migration note so the next person can see what you froze. You do not delete those call sites until the new classifier is imported and the old helper is unused.

2. Separate transport retries from tool retries

You give the free server a policy that stops when it sees a tool receipt instead of trying the call again. The module below is an unexecuted proposal you copy into your repository and review like any other patch. It refuses to retry once a tool reports a receipt, and it caps transport retries at one. It also stamps each decision so your logs can show why a second call did or did not leave the worker.

# Unexecuted proposal for Python 3.10+. This draft did not run it.
from dataclasses import dataclass

TRANSPORT = {"connection_reset", "http_502", "http_503"}
DEADLINE = {"client_timeout"}

@dataclass(frozen=True)
class RetryDecision:
    allow: bool
    reason: str
    attempt_budget: int

def classify(signal: str, tool_receipt: str | None, attempt: int) -> RetryDecision:
    if tool_receipt:
        return RetryDecision(False, "tool_already_committed", 0)
    if signal in {"http_400", "schema_mismatch"}:
        return RetryDecision(False, "client_fault", 0)
    if signal not in TRANSPORT | DEADLINE:
        return RetryDecision(False, "unknown_stop", 0)
    if attempt >= 1:
        return RetryDecision(False, "budget_spent", 1)
    return RetryDecision(True, "transport_or_deadline_once", 1)

def delay_seconds(decision: RetryDecision, measured_delay: float) -> float:
    # Caller supplies measured_delay after its own timing run. Do not invent one here.
    if not decision.allow:
        return 0.0
    return measured_delay

You keep the attempt budget at one until you have measured your own latency and error mix on the free server. This draft does not invent that measurement, and you should not paste a blog backoff curve into production. If your idempotency ledger is not already sealed, you finish that work before you enable even this single retry. A classifier cannot protect a tool that applies a payment twice under two different idempotency keys.

3. Shadow the classifier on recorded failures

You run the classifier against saved failure records before it is allowed to change live behavior. A small fixture file is enough, as long as you include one committed tool receipt and one bare timeout. You assert the decision, not a latency number, because this test does not call a model and does not spend a quota. The commands below are the check you run in CI on the branch that still points at the paid endpoint.

# Unexecuted proposal. Add these assertions beside the module, then run them yourself.
def test_timeout_may_retry_once():
    decision = classify("client_timeout", tool_receipt=None, attempt=0)
    assert decision.allow is True
    assert decision.reason == "transport_or_deadline_once"

def test_receipt_never_retries():
    decision = classify("http_503", tool_receipt="rcpt_1001", attempt=0)
    assert decision.allow is False
    assert decision.reason == "tool_already_committed"

def test_second_attempt_stops():
    decision = classify("http_502", tool_receipt=None, attempt=1)
    assert decision.allow is False
    assert decision.reason == "budget_spent"

def test_unknown_stops():
    decision = classify("", tool_receipt=None, attempt=0)
    assert decision.allow is False
    assert decision.reason == "unknown_stop"

python -m pytest tests/test_retry_budget.py -q

You treat a red test as a block on the cutover, not as a warning you can waive in chat. You also add one fixture from a real redacted log if your policy allows that copy, and you keep secrets out of the fixture. If you cannot redact the log, you synthesize the signal names and you do not paste production payloads into the branch. The goal is a repeatable yes or no, not a story about how the last outage felt.

4. Flip the canary only after the paid path remains a rollback

You change configuration in two steps, and you never cancel the paid workspace in the same change window. First you point one workspace at the free model and free server, with the new classifier imported on that path. Then you watch attempt counts, dead-letter volume, and tool receipt duplicates for a period your own error budget defines. This note does not name that period, because a safe window depends on your traffic and is not a universal constant.

# Proposed canary fragment. Replace hosts from your own current documentation.
agent:
  runtime: free_server
  model_source: free_tier
  retry_policy: classify_once
  rollback_runtime: paid_workspace
  cancel_paid_workspace: false

You leave the cancel flag false until the canary has a written rollback drill and a named owner. You rehearse the drill by sending one synthetic timeout and confirming the second attempt does not fire when a receipt exists. You do not rehearse the drill with a live payment tool or any other irreversible external write. After the drill, you schedule the paid workspace closure as a separate change, with the classifier still in place.

Leftovers that still steer the new path

The retry helper is not the only leftover that survives after you cancel the paid workspace. Your alert still pages on a threshold that assumed a paid error rate, and that page will tire on-call during a cold start. Your dashboard may still divide failures by spend, which hides a retry storm that costs tokens you no longer itemize the same way. You update the alert to count classified retries and dead letters, and you keep the old chart until the canary window ends.

You also check the queue consumer, because a worker outside the agent can still apply the paid backoff you thought you deleted. A second process with its own sleep loop will ignore the Python classifier and hammer the free server anyway. You add the same decision function to that consumer or you disable its retry until the port is done. Two policies in one workflow are how a cutover looks green in agent logs and red in the ticket system.

Limits and who should skip this plan

You should not use this plan when you cannot tell a transport error from a committed tool result. You should also skip it for payment capture, credential rotation, or any write that a second call would duplicate without an idempotency key. A free server is a poor place to discover that your domain needs a human approval queue you have not built yet. If your service level needs a latency bound you have not measured on the free path, you keep the paid runtime.

This proposal does not prove that one retry is safe for your traffic, and it does not compare vendors. It fails open if you catch a bare exception and pass an empty signal into the classifier function. An empty signal counts as stop only when your wrapper actually calls the function with the name unknown. You should wrap the client so unknown exceptions become the signal unknown and then hit the stop branch.

You review that wrapper in the same pull request as the policy module, before any canary config merges. If the golden tests and the shadow run agree, point one canary workspace at the free model and the free server. Keep the paid endpoint as rollback until your next release review, and do not close that workspace in the same change. That pairing is the only invitation this cutover needs, and you still confirm limits in the current product docs first.

Migration Diary: Separate Propose from Apply Before You Cancel the Workspace

Riley Xu — Sun, 11 Oct 2026 21:58:34 +0000

You should detach the merge hook from any coding agent before you cancel the paid workspace, because that hook can still apply a diff after the vendor relationship ends. The paid product often treated proposal, review, and apply as one trusted gesture, and a free replacement does not inherit that trust. A free model can still write a useful patch, yet the runner must not receive the credential that pushes it. Rebind approval to a place you control, and leave the old key in place until that bind is tested.

The leftover is the merge, not the prompt

The dangerous leftover is rarely the prompt, and it is usually a script, workflow, or webhook that still knows how to merge. That automation trusted a vendor identity, a stable workspace, and a review click that lived in the old interface. After the cutover, the same automation trusts any process that can drop a diff into the watched path. If your free server can write that path, you have rebuilt the old privilege without the old review.

You are migrating three different things, and only one of them should move onto the cheaper host. The proposal channel can move, because a draft diff is not yet a change to the default branch. The approval record should stay with reviewers, in a pull request or a protected file that the model cannot edit. The apply credential should stay on protected CI, or with a person, until you have compared real tasks.

Write those three owners into the diary before you change a single environment variable or webhook. A cutover without named owners becomes a chat log, and that chat log vanishes when the paid workspace closes. You will want those names during the first bad patch, not during the cheerful launch note.

Step 1: Inventory every path that can apply a diff

Start from a clean checkout, and search for the verbs that turn generated text into a commit or a deploy. You are not judging model quality yet, and you are only finding the side effect that can outlive the vendor. Include local scripts and editor tasks, because a laptop hook survives cancellation as easily as a hosted workflow does.

Search workflows and scripts for apply, merge, and push behavior, and save the file list into the diary. Paste the matching paths under a diary heading called apply paths, so the later table has evidence rather than memory. Ignore documentation hits unless the command in that file can change your branch or your deploy target.

git grep -nE 'git apply|gh pr merge|git push|patch -p' -- .github scripts || true

List every secret name that an agent process could read, and mark any name that can push, deploy, or open a firewall change. A secret that only reads logs can stay for now, but a secret that pushes must be called out in bold in the diary. Do not copy those values into the diary, and record only names, owners, and the workflow files that reference them.

git grep -nE 'GITHUB_TOKEN|DEPLOY_KEY|SSH_KEY|API_KEY' -- .github scripts .env.example || true

Open recent agent runs and check whether a human review identifier appears before the merge event in the log. A jump from "diff written" to "pushed" means the gate is already missing on the old path. Do not assume the paid interface still sits in front of that jump after you remove the integration.
Name the rollback owner in the same note as the secret owner, including the channel where that person can be reached. A free server with no rollback owner is not a substitute for a paid agent your on-call already understood. If either name is blank, stop the migration and repair the runbook before you repoint anything.

Step 2: Freeze a proposal contract the new host must honor

A valid proposal carries a task id, a base revision, a unified diff, and a risk label from a list your team already uses. A valid apply cites the same task id, a hash of that diff, and a human actor tied to review. The model may write the proposal, and the model may not write the approval, even when the patch looks small or obvious.

Keep the checker boring, and fail closed when any required field is empty or when the actor is the agent itself. The following program is an example you should run locally and adapt, not a policy engine this diary claims to have executed. It reads two JSON files, rejects a model-written approval, and allows apply only when the hashes match.

# Example checker. Label: unexecuted until you run it on your own files.
import hashlib
import json
import pathlib
import sys

REQUIRED = ("task_id", "base_rev", "diff", "risk")

def load(path: str) -> dict:
    return json.loads(pathlib.Path(path).read_text())

def proposal_digest(proposal: dict) -> str:
    missing = [key for key in REQUIRED if not proposal.get(key)]
    if missing:
        raise SystemExit(f"proposal missing {missing}")
    if proposal.get("approved_by"):
        raise SystemExit("model must not set approved_by")
    payload = proposal["diff"].encode()
    return hashlib.sha256(payload).hexdigest()

def main() -> None:
    proposal = load(sys.argv[1])
    approval = load(sys.argv[2])
    digest = proposal_digest(proposal)
    if approval.get("task_id") != proposal["task_id"]:
        raise SystemExit("task_id mismatch")
    if approval.get("diff_sha256") != digest:
        raise SystemExit("diff hash mismatch")
    actor = approval.get("actor", "")
    if not actor or actor == "agent":
        raise SystemExit("human actor required")
    print("apply allowed")

if __name__ == "__main__":
    main()

Create two sample files so the command is reproducible on your laptop before any network call exists. The proposal file must not contain an approval field, and the approval file must carry the hash you compute yourself. A deliberate mismatch is a useful first test, because you want to see the gate fail before you trust a pass.

{
  "task_id": "task-184",
  "base_rev": "a1b2c3d4",
  "risk": "low",
  "diff": "diff --git a/README.md b/README.md\n"
}

python -c 'import hashlib,json,pathlib,sys; p=json.loads(pathlib.Path(sys.argv[1]).read_text()); print(hashlib.sha256(p["diff"].encode()).hexdigest())' proposals/task-184.json
python check_apply.py proposals/task-184.json approvals/task-184.json

Store proposals where the runner can write, and store approvals on a path that runner cannot push. A single shared directory with loose permissions collapses the split, even when the checker function looks strict on your laptop. Use code owners or a separate approvals repository, and confirm the agent token is rejected when it tries to write there.

proposals/<task_id>.json
approvals/<task_id>.json

Step 3: Point only the proposal writer at the free option

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator describes MonkeyCode as an open-source project with free model access and a free server option for a proposal writer. That writer should hold no push credential, and you should confirm current terms in the project documentation before you rely on them. If the project page lists a free token allowance, copy that live figure into the diary yourself. Do not reuse a token number from an old chat, because allowances change and this draft did not verify one.

An availability note is not a quality result, and it is not a promise about how long logs will remain. Treat the free option as a place to run the proposal writer, not as proof that review can be skipped. Your repository rules stay in force on the free host, even when that host stops sending you an invoice.

Keep the paid workspace readable while you move the writer, and do not move the apply job in the same change. The free server should emit a proposal file and then exit, without a token that can update the default branch. Protected CI should run the checker, and only a separate job should apply the diff after apply allowed is printed. If the free server image still contains the old deploy key, remove that key before the first remote task.

Choose three tasks you have already accepted: one low-risk edit, one dependency bump, and one change that touches auth, billing, or data migrations.
Save the paid run's file list, base revision, and diff hash, and do not invent a fresh task during cutover week.
Run the same task through the free model, and allow the process to write a proposal file only.
Compare paths and the pinned base revision, not the surrounding explanation, because wording can drift while scope must not.
Apply nothing from a laptop while you wait, and let CI refuse the job until a reviewer writes the matching approval.

A file outside the saved list is a stop, even when tests on the expected files are green. Scope drift is how an unreviewed edit passes a badge that never looked at the extra path. Record the stop in the diary, and do not apply the extra file from a personal checkout just this once.

Also pin the base revision before you compare, or a moving default branch will make both hosts look wrong. This command records the tip you intend both runs to start from, and you should paste that value into the proposal. If the free run reports a different parent, reject the proposal and rerun it against the pinned revision.

git rev-parse origin/main

Do not rebase the generated diff by hand during the comparison week, because that hides whether the new host respected the contract. Hand cleanup can return after the gate is proven, and it should not be part of the evidence you use to cut over. A hand-edited diff can look cleaner while hiding a host that ignored the base revision you pinned.

Step 4: Read the decision table before you delete the paid key

Treat the table as a gate rather than a summary you skim after the migration is already done. An unknown cell means apply traffic stays on the old path, even when proposal generation already looks cheaper or more convenient. Judge each row from the inventory in step 1, not from a feature list you have not rechecked this week.

Check	Paid side today	Free side requirement	Move apply?
Diff writer	Vendor agent identity	Proposal path only	Propose only
Approval writer	Human in the old UI	Path the agent cannot push	No, until enforced
Base revision	Workspace snapshot	`base_rev` pinned and checked	No, if tip can drift
Push secret	Vendor scope or protected CI	Absent from the free server	No, if the key was copied
Rollback owner	Named in the old runbook	Same owner, updated link	No, if the name is blank
Three-task hash compare	Often skipped historically	Recorded before cutover	No, until the diary has it

Move the proposal writer only when the diff-writer row and the push-secret row are both satisfied. Move the apply job only when every other row is satisfied on the three tasks you saved. Another week with the paid key still present is cheaper than reversing a merge that no reviewer saw.

Limitations that remain after a green check

A matching hash proves the approved bytes are the proposed bytes, and it does not prove those bytes are safe to ship. The free model can edit the expected files and still break a caller, a migration, or a permission check you forgot to name. Keep executable tests on the task, and keep a trusted path available until those tests are green on the new writer.

The sample checker trusts your repository permissions to protect the approvals path from the agent token. If that token can push the path, the gate is theater, and you should not migrate the apply job at all. Free model access and a free server option can also change latency, context, and retention without a new contract. Copy proposal files into storage you control before the paid workspace closes and takes the old run history with it.

This diary does not rank products and does not report a benchmark, a token price, or a hardware shape. Your three saved tasks are the only evidence that should decide whether the apply job moves. If those tasks never touch production risk, do not generalize from a documentation edit to an auth change.

Who should not use this cutover yet

Do not use this plan when the current agent can deploy to production without a pull request or an equivalent change record. You need a stronger control than a JSON file, and you should install that control before you change hosts. Do not use the plan when no named reviewer will read high-risk diffs, because an empty queue recreates the leftover.

Skip the free server when the task includes secrets, customer data, or details of an unreleased vulnerability. A smaller bill does not relax a data rule you already accepted for the paid agent. If the team still applies patches from laptops and has no protected CI, inventory those machines before you add another writer.

Leave the apply key until the table is true

After the checker prints apply allowed on three real tasks, remove the push credential from the agent environment and leave proposal writing in place. Keep the paid workspace readable until you copy the retention you need, including older diffs and the review notes beside them. If a free server should host that writer, read MonkeyCode's current project notes and keep only what the table marks safe.

Migration Diary: Seal the Idempotency Ledger Before a Free Server Replays Paid Runs

Riley Xu — Sat, 10 Oct 2026 18:33:23 +0000

You should seal an idempotency ledger before a free server inherits runs from a paid coding agent. A cheaper host can retry the same tool call and mutate production twice if that key never left the old vendor. This diary covers the cutover plan, the leftover files, and a small checker you can run on your own machine. It does not assume a particular model name, a fixed token quota, or a permanent free-tier promise from any host.

Paid coding agents often hide retry behavior inside a session store that you do not own or export by default. When you close that account, the session rows, the trace identifiers, and the dedupe keys can disappear together. A free server then sees a fresh run and may execute a write that the paid agent already committed successfully. You prevent that leftover by exporting the ledger first and refusing cutover until a local checker passes cleanly.

What fails if you skip the seal

You usually notice the failure only after a duplicate ticket, a second comment, or another pull request appears. The free server logs still look healthy, because each individual call returned success and never reported a conflict. Your parser can still match a response schema you froze earlier, so this bug is not a shape mismatch at all. The missing piece is a stable run identity that both hosts must honor before any side effect is allowed.

An in-memory retry map often dies with the paid agent, and your new host cannot reconstruct it from logs alone. A vendor trace identifier becomes useless after closure, because the free server has no authority to resolve that id. A tool result cached only in the old interface is another leftover, since your scripts may never have downloaded it. You should treat each of those traces as data you must replace, not as history you can query after the account closes.

Export the ledger before the paid agent closes

You want a narrow file rather than a full transcript dump, because transcripts drift and often contain secrets you should not move. Keep one row for every side-effecting tool call, and ignore pure reads that cannot mutate an external system. Store the tool name, a hash of the arguments, the idempotency key, and the terminal status, plus a finish timestamp. Leave raw prompts, token counts, and customer text out of this file unless your retention policy already allows that export.

mkdir -p cutover/ledger
# Proposed export. Run this only against a store you are allowed to read.
jq -c 'select(.side_effect == true) | {key, tool, args_sha256, status, finished_at}' \
  paid-agent-runs.ndjson > cutover/ledger/side_effects.ndjson
sha256sum cutover/ledger/side_effects.ndjson | tee cutover/ledger/SHA256SUMS
wc -l cutover/ledger/side_effects.ndjson

That shell snippet is a proposed workflow, not a transcript from a migration this account has already executed. You should rename the fields so they match the export your current vendor actually provides to operators. If that vendor cannot emit an idempotency key, you stop the cutover and add keys in your own tool wrapper first. A free server cannot invent a trustworthy key from a missing column, no matter which model you attach later.

Numbered cutover

You should follow the four steps below in order, and you should not enable live writes until the last step passes. Keep the paid agent readable until the checker is green, because a failed export is cheaper than a double merge. Rotate any credential that lived only in the paid agent's secret store before the new host is allowed to boot. This diary does not include vault-specific rotation commands, because those steps belong in your internal runbook instead.

Freeze new writes on the paid agent, export the side-effect ledger, and record the checksum next to that file.
Pin the key format in a small schema file that both hosts can load without making a network call at all.
Run the local checker against a fixture replay, and store the exit code in the same directory as the checksum.
Start the free server in dry-run mode, reject unknown or terminal keys, and enable writes only after that gate holds.

The schema file is the contract both hosts load, and it should stay boring enough that a model is not required to parse it. You pin the required fields, the allowed statuses, and the hash algorithm so a later host cannot quietly drop a column. The example below is a proposal, and you should reject exports that fail it before you copy anything onto the free server. A schema check will not prove business correctness, but it will stop a truncated file from looking like a finished seal.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "required": ["key", "tool", "args_sha256", "status", "finished_at"],
  "properties": {
    "key": {"type": "string", "minLength": 8},
    "tool": {"type": "string"},
    "args_sha256": {"type": "string", "pattern": "^[a-f0-9]{64}$"},
    "status": {"enum": ["succeeded", "failed_permanent", "running"]},
    "finished_at": {"type": "string"}
  },
  "additionalProperties": false
}

# proposal: validate each exported row with a checker you already trust
jq -c . cutover/ledger/side_effects.ndjson | while read -r row; do
  printf '%s\n' "$row" | check-jsonschema --schemafile cutover/ledger/side_effect.schema.json
done

Treat a failed schema row as a blocked cutover, not as a warning you can override from the free server console. You fix the export on the paid side, recompute the checksum, and only then copy the directory to the new host. Do not hand-edit a single row on the new host, because that edit will not exist in the checksum you just sealed. If you must correct a key, you do it before sealing, and you record why in the commit message next to the ledger.

Checker you can run locally

The module below is a proposal you should copy into a branch and execute in your own test environment. It has not been run against a production agent for this article, and it deliberately does not call any model. The function only decides whether a replay is safe when a sealed ledger and a new tool call are compared. You should change the status names if your vendor uses different terminal labels than the two strings shown here.

# proposal: cutover/ledger_check.py — unexecuted example
import hashlib
import json
from pathlib import Path

TERMINAL = {"succeeded", "failed_permanent"}

def args_hash(payload: dict) -> str:
    blob = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
    return hashlib.sha256(blob).hexdigest()

def load_ledger(path: Path) -> dict:
    rows = {}
    for line in path.read_text().splitlines():
        if not line.strip():
            continue
        row = json.loads(line)
        key = row["key"]
        if key in rows:
            raise SystemExit(f"duplicate key in ledger: {key}")
        rows[key] = row
    return rows

def admit(ledger: dict, call: dict) -> str:
    key = call["key"]
    prior = ledger.get(key)
    digest = args_hash(call["args"])
    if prior is None:
        return "allow_new"
    if prior["args_sha256"] != digest or prior["tool"] != call["tool"]:
        return "reject_conflict"
    if prior["status"] in TERMINAL:
        return "skip_replay"
    return "allow_resume"

You can pair that module with fixtures that encode the three decisions your cutover actually depends on. A brand-new key should be allowed, a finished key should be skipped, and a conflicting hash should be rejected. The sample test is also a proposal, so you should run it locally before you trust any exit code it prints. A checker that only covers the happy path will bless a cutover that can still double-write under replay.

# proposal: tests/test_ledger_check.py — unexecuted example
from pathlib import Path
import json
from cutover.ledger_check import admit, load_ledger

def test_new_key_is_not_enough(tmp_path: Path):
    path = tmp_path / "side_effects.ndjson"
    row = {
        "key": "job-1",
        "tool": "open_pr",
        "args_sha256": "pending",
        "status": "succeeded",
    }
    path.write_text(json.dumps(row) + "\n")
    ledger = load_ledger(path)
    call = {"key": "job-2", "tool": "open_pr", "args": {"title": "new"}}
    assert admit(ledger, call) == "allow_new"
    # Add skip_replay and reject_conflict cases before you trust this file.

The sample only asserts allow_new, and that branch returns before the hash is compared, so a green run proves little. You should add skip_replay and reject_conflict cases that use args_hash, or the checker can still miss a double write. Compute those hashes with the same helper the server will import, then store the fixture beside the sealed ledger. If the helper output changes, update the fixture in the same commit, or the ledger and the server will quietly diverge.

Decision table

Use the table below when a replay arrives on the free server during the first week after cutover. You should keep this rule in ordinary code, because a later prompt edit must not be able to widen it. Do not add an outcome that asks a model what to do, since a guess is not an idempotency decision. Model wording will drift when you change hosts, while a coded table stays stable across both environments.

Prior status	Args hash	Decision	Why
missing	any	allow_new	No sealed side effect exists for this key
succeeded	same	skip_replay	The paid agent already committed the write
failed_permanent	same	skip_replay	Retrying would ignore an explicit terminal failure
running	same	allow_resume	Resume only if the tool cannot emit a second write
any	different	reject_conflict	The key was reused for a different mutation

Where a free host fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Operator notes say MonkeyCode offers free model access and a free server option for this kind of dry-run host. You should read the current project documentation for limits before you rely on either option in a real cutover. This diary does not state a token quota, a machine size, a duration, or an expiry date for that offer.

The checker still matters if you remove that host, because any replacement server can replay a committed write. You can let a free model draft explanations for rejected lines, but you should not let it invent idempotency keys. Generate the ledger from the paid agent's export, then ask the model only to describe a conflict in plain language. If the free model rewrites a terminal status, you discard that draft and reload the sealed file from the checksum.

Leftovers you still own

After the free server admits traffic, you still own three leftovers that the paid account will not clean up for you. Archive or delete the old session store so a teammate cannot resume a run you have already sealed in the ledger. Revoke the old tool credentials even when the interface says the workspace is frozen and no longer accepting jobs. Keep the checksum beside the checker so a later host can prove it evaluated the same ledger you exported.

You should also list every tool that cannot accept a client-supplied idempotency key, because the table cannot protect those calls. A send-email tool without a provider dedupe key will double-send even when your checker returns a skip decision. Disable those tools until the provider gives you a real key, rather than hoping a prompt will stop retries. Write that disablement into the cutover checklist so a later operator does not re-enable the tool during a quiet afternoon.

Who should not use this approach

You should skip this cutover when your tools cannot accept a client-supplied idempotency key in any form. You should also skip it when contracts require the paid vendor to retain the only legal copy of run history. A solo weekend project with no external side effects does not need this ledger, and adding one will only slow you down. Teams that need a contractual uptime promise should not treat a free server as production without a separate capacity review.

The approach also fails if you treat the sample code as a finished migration report from a measured production move. Those snippets are proposals for you to adapt, execute, and discard when your export shape differs from the example. They do not prove that any free tier is large enough for your traffic, and they do not rank one host above another. Confirm the live free-server terms, then place this ledger next to the schemas and fixtures you already froze.

Migration Diary: Lock the Response Schema Before a Free Model Inherits Your Parser

Riley Xu — Fri, 09 Oct 2026 15:01:56 +0000

Liquid syntax error: Unknown tag 'endraw'

Migration Diary: Freeze the Mount and Egress Contract Before the Paid Agent Closes

Riley Xu — Thu, 08 Oct 2026 10:36:07 +0000

You should freeze a mount-and-egress contract before you cancel a paid coding agent, because the replacement host will not inherit it. The paid product often remembered which directories were writable, which trees were context only, and which hosts the sandbox could reach. A free server starts from its own defaults, so yesterday's silent allowances become today's missing checks or unexpected network calls. This diary walks you through a cutover plan, a small checker, and the leftovers you should not drag along.

Why this leftover is not another prompt file

A rule file tells the agent how to speak, while a mount contract names writable paths and permitted hosts. Those two leftovers fail in different ways, and mixing them during cutover hides the boundary you actually need. You can copy every instruction into the new tool and still lose the limit that kept generated patches inside the repository. Treat that boundary as data you can diff in review, not as a memory you hope the next host will share.

What you leave behind and what you keep

You are leaving the paid agent's private sandbox, its hidden allowlist, and any scheduler that lived only in that vendor console. You keep the repository, your tests, and a written contract that names writable roots, read-only context, and permitted egress hosts. You do not keep session transcripts, browser cookies, or tokens that were stored in the old agent's settings panel. If a path existed only inside the vendor container, record it as a leftover rather than a path the new host must recreate.

Step 1: Inventory the boundary while the old account still opens

Open the paid agent's settings while the account still works, and list every directory it could edit without an extra prompt. Note read-only trees such as vendored docs, generated clients, and lockfiles you never wanted the agent to rewrite. Record egress as hostnames only, not as credentials, secret query strings, or copied authorization headers from old logs. If the product never showed a boundary, write unknown in the contract rather than guessing a permissive default.

Step 2: Write a contract the next host can check

Put the inventory in a YAML file inside the repository, and keep every value boring enough for a normal pull request review. Use repository-relative paths, explicit deny notes, and a short owner field so the next person knows who must update it. Do not store API keys, personal access tokens, or customer data in that file, even as examples that only look fake. Label the file as a cutover aid for humans, not as a certification that the new environment is secure.

# unexecuted example — review before use
version: 1
writable:
  - src
  - tests
read_only:
  - docs/vendor
  - third_party
egress_hosts:
  - registry.npmjs.org
  - pypi.org
deny_notes:
  - do not mount the home directory
  - do not call hosts that are not listed
owner: team-platform

Step 3: Add a checker you can run without the old agent

The checker below is a proposal, and it has not been executed against a live paid account or a live free server. It reads the contract, confirms that listed paths exist, and prints hosts so you can compare them with firewall notes. It does not open network connections, and it does not try to prove that any listed host is safe to call. Run it in a scratch clone first, then decide whether a failing path is a real leftover or a simple typo.

# proposal: not executed in this article
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    sys.exit("install pyyaml in a local venv before running")

contract = yaml.safe_load(Path("agent-boundary.yaml").read_text())
root = Path(".").resolve()
missing = []
for key in ("writable", "read_only"):
    for rel in contract.get(key, []):
        if not (root / rel).exists():
            missing.append(f"{key}:{rel}")
if missing:
    print("missing paths:")
    print("\n".join(missing))
    sys.exit(1)
print("paths ok")
print("egress hosts to re-approve:")
for host in contract.get("egress_hosts", []):
    print(f"- {host}")

Step 4: Decide what moves, what stays, and what you delete

Use a small decision table during the cutover meeting so leftovers do not sneak into the new scheduled job. Each row should name the artifact, the owner, and the action, instead of a vague promise that someone will clean up later. Delete vendor-only interface state, because a free server cannot import a screenshot of a settings page from the old product. Keep the contract in version control, and keep the old account available until the checker passes on the replacement host.

Artifact	Move?	Why
Writable and read-only path lists	Yes	The new host needs an explicit boundary
Egress hostnames without secrets	Yes	Firewall notes fail if the names stay trapped in a closed UI
Vendor session memory and chat archive	No	That drain is a separate task, and it is not a boundary
Tokens stored in the old agent panel	No	Rotate them, then delete the stored copies
Hidden container paths that exist only in the paid sandbox	No	Recreate needed files in the repo, or drop the task

Step 5: Rehearse on a free server before you cancel

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can support this rehearsal when current docs still list them. Use the free server as a place to run the checker and one dry task, not as a silent copy of the old sandbox. Use free model access only to draft the human notes beside the contract, and reject any broader allowlist that nobody reviewed.

Do not treat either option as a quota, a named model, or a hardware promise this article never verified. Confirm duration, limits, and terms in the product's own documentation before you let a critical job depend on them. The commands below assume a local virtual environment, and you should skip them if your policy forbids installing a YAML parser. Cancel the paid account only after the rehearsal matches the decision table, not when a calendar reminder starts to feel urgent.

python3 -m venv .venv
. .venv/bin/activate
python -m pip install pyyaml
python check_boundary.py
git diff -- agent-boundary.yaml

Then run one narrow task, such as editing a file under a writable root, and confirm that a read-only path stayed untouched. If the free model suggests extra hosts or extra writable roots, reject that draft unless a person updates the contract in review. Save the checker output next to the decision table so the cutover record shows what you compared, not only what you intended. Repeat the narrow task once on the free server after you move the job, because a local pass does not prove the remote mount.

Step 6: Debug a failed check before you blame the new host

When the checker reports a missing path, do not widen the contract until you confirm the path existed before. Search the repository for that relative path, and also search your notes for a vendor container path that never lived in git. If the path was only a container mount, mark it as a leftover and either recreate the file or drop the task. If the path exists but writes still land outside it, the bug is the process mount, so fix the host next.

A short test plan you can repeat

Run the checker on a clean clone, then temporarily rename one writable directory and confirm the command exits with a missing path. Restore the directory, add a fake hostname to the egress list, and confirm the script prints that name without opening a socket. Ask the free model to summarize the contract, and fail the test if that summary adds hosts or roots. Record the three results in the cutover note, because a passing local run without these checks is not evidence the boundary moved.

Leftovers you should expect after cutover

You will still find editor plugins, local caches, and shell aliases that point at the paid agent's old command name. Remove those pointers in the same week, or the next person will launch a tool you already stopped paying for. You may also find generated files whose headers mention the old product, and those headers are documentation debt rather than a restore reason. Leave a one-line note in the deny section if a workflow truly cannot move, so the gap stays visible to the next reviewer.

Limitations of this approach

This contract records human intent, and it does not enforce a kernel sandbox, a container profile, or a corporate egress proxy. A checker that only looks for paths can pass while the process still runs with a wider mount than the file describes. Hostnames without ports, protocols, or identity checks are not an allowlist a security team should accept as complete. The example YAML and Python are unexecuted proposals, so you must adapt them to your layout and review every failure yourself.

Who should not use this cutover

Do not use this path if production deploys still run inside the paid agent, because a diary checklist is not a release plan. Skip it when the workspace holds regulated data and you need a formal control, since this file will not satisfy an auditor alone. Avoid it when you cannot inspect the old settings before cancellation, because a boundary rebuilt from memory creates false confidence. Teams that require a contracted uptime promise should not treat a free server option as the home for a critical job.

A quiet next step

Read the current MonkeyCode docs, then try the checker on a scratch repository if the free server option still fits this rehearsal. Keep the paid account until every move row has a passing note, and keep secrets out of model drafts. If the docs no longer match the availability assumed here, stop and rewrite the rehearsal around whatever the current page actually offers.