How I check that my Rust port still does what the Go original does

Vayun Godara — Mon, 26 Oct 2026 19:36:33 +0000

cliproxy-rs is my Rust port of CLIProxyAPI, a Go proxy that puts Claude, Codex, Gemini and OpenAI-compatible APIs behind one local endpoint. The promise is that a Go user can point the Rust binary at their existing config.yaml and credential files and get the same routes and the same answers.

Trying requests by hand only tells you the requests you tried work. I wanted a coverage number I could trace back to evidence, item by item.

The checklist comes from the Go tree

A list I wrote by hand would only contain the things I remembered, so the checklist is extracted from the Go source at a pinned commit. Each item is one contract: a Go test file, a config key, a public or management route, a plugin RPC method, a CLI flag. At Go v8.0.35 that's 1,951 items. The biggest groups:

Kind	Items
Go test-suite files	989
Config leaves	436
Management routes	186
Public routes	77
Plugin RPC methods	62
CLI flags	34

Every item gets one of three statuses:

covered: implemented, and a Rust test or a fixture recorded from Go exercises it
partial: implemented in part, or implemented with no test that pins Go's behaviour
missing: not there

A Python script (audit.py) fills in most of it. Fixtures recorded from Go carry the Go test name and line, so it can match each Go test against the names cited in Rust tests and fixtures. A second script starts the server and records which method and path pairs actually route. When the scan gets something wrong I override it in a TSV, one row per item, with the evidence and a note.

The first audit was too generous

When I redid the audit for v0.2.6, the generated numbers were 854 covered, 722 partial, 111 missing. The README said 835 / 707 / 145, so it was already stale.

I went through all 833 items that weren't covered and 164 changed status. Most moved up (73 partial to covered, 36 missing to covered, 42 missing to partial). 13 went from covered down to partial, and those were the useful ones.

The scanner had counted a route as covered when a fixture hit the same path, even with a different HTTP method, so a GET test was vouching for a POST handler. Now the evidence has to match the method and the complete route.

Two more rules came out of that pass. A subsystem that works doesn't make a whole Go test suite covered: Go has 23 Claude alias-restoration cases, I replay 20, so that item is partial. And a Go test name in a Rust comment is a search hint. It doesn't prove the assertions were ported.

Moving the target

Go keeps shipping. I'd been comparing against v8.0.10, and moving to v8.0.35 meant going through 288 upstream commits and adding 264 checklist items. I kept all 1,687 existing IDs and appended the new ones at the end of each section, so branches I had in flight still point at the right item. Tests Go deleted are marked historical instead of disappearing.

The last audit (20 Oct, against v8.0.35) reads 1,053 covered, 872 partial and 26 missing.

Partial is the biggest bucket. It holds everything implemented in part or implemented without a test that pins Go's behaviour, and the audit won't round any of it up to covered.

If you're porting something

Generate the checklist from the original's source. Make "covered" need evidence a script can find, and make every manual override say why. My first honest pass moved 13 items backwards.

The checklist, the scripts and the per-item notes are in the repo under docs/parity-audit: https://github.com/vayungodara/cliproxy-rs

Two 32 MB tokenizers: hunting a memory floor in a Rust proxy

Vayun Godara — Wed, 07 Oct 2026 18:21:46 +0000

cliproxy-rs is a Rust rewrite of CLIProxyAPI. It's one local server that Codex CLI, Claude Code and other tools talk to, and it forwards to whatever you've configured: your own API keys, your own accounts. I run it on a small Linux box under real coding-agent traffic.

On 0.2.0 that box showed something I didn't like. The resting memory, meaning what the process holds between requests, rose from 26 MB to 89 MB over about seven hours. Peaks were 300 to 400 MB. Nothing was failing, but a floor that only goes up is how a leak usually starts.

Reproducing it

My soak test only replayed Claude streams of 100 to 500 KB. The real traffic was different: Claude and Codex requests of 1 to 2 MB from long agent sessions, plus count_tokens calls. So I wrote a "field mix" replay with exactly that shape (four sessions, bodies averaging 1 MB) and ran 0.2.0 and the fix branch side by side on separate GitHub runners for an hour.

0.2.0 rested at 128 to 145 MB. That was clearly too much for a proxy that does no caching of its own.

What heaptrack found

After two batches of 100 field-mix turns, heaptrack showed 67.1 MB still allocated at exit. 63.8 MB of that was two o200k_base tokenizer encoders, 31.9 MB each. They were identical. Claude count_tokens had built one, and Codex count_tokens had built the other.

Five modules each kept their own lazily built tiktoken encoder in a static. Each was built the first time something needed a token count and never freed. On a server that counts tokens for both Claude and Codex models and streams Claude-format requests to Codex models, up to seven copies could exist, 164 MB of tables in total.

One encoder is bigger than its 31.9 MB of live data suggests. It holds about 600,000 short token allocations, and at malloc's 32-byte minimum chunk that comes to 47 MB resident.

I also checked whether glibc was just holding freed memory. It wasn't: after a trim, calling malloc_trim(0) again changed resident memory by 40 kB. The floor was live data.

The fix

One shared encoder per encoding, still built only when a count first needs it. The same mix now holds 71 MB of tables instead of 164.

Same hour of traffic, same runners:

	0.2.0	0.2.2
Resting RSS	128.2 to 145.4 MB	80.6 to 97.6 MB
Peak RSS (VmHWM)	220.1 MB	169.0 MB

CPU per request didn't change (43.1 ms against 42.9 and 44.1 ms on one 2-vCPU machine).

What I haven't solved

The replay steps up once and then stays flat. It never shows the slow climb from the field. My best guess is that the field server built its encoders hours apart, which would look like a slow rise, but I don't know that yet. A 24-hour trace of 0.2.2 is running now.
Peaks are still big. A Codex request peaks at 8.8 times its body size, because request shaping rewrites the body pass by pass and each pass is a new copy. Claude requests peak at 6.0x.
A fixed MALLOC_MMAP_THRESHOLD_ took 26.6 to 33.8 MB off the peak, but cost 16 to 31% more CPU per request, so I didn't ship it.
The Go original is still faster on small requests. In my 0.1.0 benchmark, it served 1,568 non-streamed chat requests a second to cliproxy-rs's 1,168.

Keeping it fixed

Every release tag now runs an hour of the field mix, and a 5-hour run happens every Saturday. A run fails if the resting floor rises, if the peak goes over its bound or if any request fails. Heap budgets in the tests now cover the Codex and count_tokens routes as well as Claude.

The method and the raw rows are in docs/BENCHMARKS.md, and the code is at github.com/vayungodara/cliproxy-rs.

Built with some help from AI.

DEV Community: Vayun Godara