DEV Community: RanThis

Claude Fable 5.1 Agent Test: Sonnet Tied It at 1/8 the Cost

RanThis — Fri, 30 Oct 2026 04:25:07 +0000

Originally published on RanThis. Cover: AI-generated illustration.

Claude Fable 5.1 is not worth its price for a month-end close with tools. We gave Fable, Opus 5.5 and Sonnet 5.5 three exports and ten traps; all three closed the books correctly, and Fable cost the most and took the longest.

Anthropic sells Fable "for demanding reasoning and long-horizon agentic work", at $10/$50 per million tokens: 2.5 times the list price of Opus 5.5 and five times Sonnet 5.5. So we tested exactly that kind of job: paging through an invoice list, a bank statement and a card report, and matching every payment to an invoice. Finance teams do this at every month end, and AI agents are now pitched for it. We measured it on 28 Oct 2026 through the Anthropic API, from a Python script. The data is invented. The prompt, the answer key and the scoring code were committed before any model was called, and the raw logs of every run are kept with this article's source files.

Key results

All three models found all ten traps. In round 1 (140 invoices) Fable 5.1, Opus 5.5 and Sonnet 5.5 gave the right answer to every question in every run. The one exception was a formatting slip: one Opus run sent two JSON objects. In round 2 (700 invoices, about 58 page reads), every run of every model again scored 10/10.
Cost per month-end close, round 1 (median): Sonnet 5.5 $0.087, Opus 5.5 $0.20, Fable 5.1 $0.70. Same answers, an 8× spread.
Round 2 (median): Sonnet 5.5 $0.56, Opus 5.5 $1.02, Fable 5.1 $3.74 (one run). Sonnet was also the fastest in both rounds: 30 s and 2.6 minutes, against 85 s and 6.2 minutes for Fable.
Fable worked one page at a time. In round 2 it made 80 tool calls over 81 model turns, one file page per turn. Sonnet and Opus asked for 8–10 pages at once and finished in 11–22 turns. Each turn re-sends the whole conversation, so Fable's run re-read 3.8 million cached tokens; the median Sonnet run re-read 0.6 million and the median Opus run 0.8 million.
Prompt caching is what made Fable affordable here. At the full input price, those 3.8 million re-read tokens alone would have cost about $38 instead of $0.94.
What we did not test: Fable at a lower effort setting, Opus 5.5 at high effort, the Claude apps (claude.ai, Claude for Excel), real bank formats, or files the model has to open itself. This is one invented month, ten questions with fixed answers, and three runs per model per round (one Fable run in round 2, see Setup).

Setup

The job: close October for a small agency from three CSV exports: the invoice list (September and October invoices), the October bank statement, and the October card-processor report. Ten questions: how many invoices are still open, the total customers owe, the overpaid invoice and by how much, the card payout still in transit and its amount, total card fees, the biggest debtor and what they owe, and the bank receipt that matches no invoice.
Ten traps, none hinted in the prompt: a partial credit note written in the notes column; a cancelled invoice; three international transfers short by a wire fee (the rest is still owed); a payment with a typo reference (letter O for zero); one transfer paying two invoices; a half payment "on account"; an invoice paid twice and refunded; an invoice paid twice and not refunded; a card payout created on Friday 30 October that reaches the bank on Monday; and a deposit from a company with no invoice.
Tools, not a pasted file: the models never saw the files whole. They got three tools: list_files, read_file (at most 25 lines per call) and calculate (exact decimal arithmetic). The agent loop ran until the model stopped asking for tools.
Two sizes: round 1 had 140 invoices and about 300 rows across the three files (13 reads at minimum). Round 2 was sealed after round 1 came back perfect. It hid the same traps in 700 invoices and about 1,430 rows (58 reads at minimum), with the trap invoices moved deep into the file.
Scoring (code, not AI): one point per question, numbers to the cent, and the reply must parse as one JSON object.
Models: claude-fable-5-1, claude-opus-5-5 and claude-sonnet-5-5 at their defaults (no effort or thinking setting), three runs each per round. We stopped Fable after one run in round 2. That run cost $3.74 and scored 10/10, and two more runs would have added about $7.50 without changing the conclusion.
Cost: tokens × list price, with prompt caching on, because that is how an agent loop is run for real: each turn re-sends the whole conversation, and cached tokens are billed at a fraction. Prices per million tokens: Fable 5.1 $10 in, $50 out, $0.25 cache read; Opus 5.5 $4, $20, $0.20; Sonnet 5.5 $2, $10, $0.10. Cache writes are billed at 1.25× input, and output includes thinking tokens. The English rounds cost $11.58 in total (round 1 $2.93, round 2 $8.65). One more Opus run in round 2 was cut off by a shell timeout before it saved a result and was rerun; its partial cost is not in that total.

The results

Model (default settings)	Round 1 scores (3 runs)	Round 1 median time / cost	Round 2 scores	Round 2 median time / cost	Round 2 turns
Claude Fable 5.1	10, 10, 10	85 s / $0.70	10 (1 run)	370 s / $3.74	81
Claude Opus 5.5	10, 0*, 10	52 s / $0.20	10, 10, 10	189 s / $1.02	19–22
Claude Sonnet 5.5	10, 10, 10	30 s / $0.087	10, 10, 10	157 s / $0.56	11–17

* All ten answers were right, but the reply contained two JSON objects (see below), and the sealed scoring code reads only one.

Where the models differed

None of the ten traps separated the models. Every model excluded the cancelled invoice, applied the credit note, matched the typo reference by customer and amount, saw that one double payment had been refunded and the other had not, and found the payout that had not reached the bank. Differences showed up in how they worked and in how they reported.

How they used the tools. Sonnet and Opus planned the reading: the first turn listed the files, and the next few turns asked for 8–10 pages at a time. Fable read sequentially in two of its four logged runs, one page per turn: 18 turns in round 1 and 81 turns in round 2, against 11–22 turns for the others on the same files. It reached the same answers, but an agent loop pays for every turn, and a 1,430-row job took Fable over six minutes against under three for Sonnet.

How they reported. The prompt asked for "only a JSON object". Fable and Opus often added notes anyway, and the notes are the useful part for a finance team. Opus in round 2 listed each judgement call with its effect on the totals: "If finance writes off the three small shortfalls, the count drops to 111 and the total to $527,322.32." Fable listed every adjustment under "Notes on how I got there, so you can check my judgement calls". Sonnet gave plain JSON in round 1. In round 2 it added a warning that it had "added up the figures by hand from the three exports, so a missed or misread line is possible". It had in fact used the calculator tool 21 times.

The one failed run. In Opus run 2 of round 1, the model sent a JSON object with nine keys, then wrote "Sorry, that's only nine keys. Here is the full object with all ten" and sent the full object. All ten numbers in the second object were right. The sealed scoring code expects one object and scored the run 0. If a script reads your agent's output, check that you get exactly one object back.

What worked

Every model read every line of all three files in every run. The logs record which lines each model read, and none skipped a page.
The ten traps cover what month-end reconciliation usually throws up, and every model handled all ten.
Prompt caching kept the long loops cheap. Fable's round 2 run would have cost roughly ten times as much without it.

What didn't work

The test did not separate the models on accuracy. Making the files five times larger did not either. A harder job might separate them: messier formats, missing data, or a task where the model has to choose its own method. We have not measured that. Our messy sales sheet test of GPT-6.1 Terra, Astra and Fable 5.1 is one where Fable did drop a point, on a single ambiguous cell.
The wire-fee shortfalls are a judgement call (chase the customer, or write the fee off), and our key counts them as still owed. All three models agreed in English. In our Vietnamese version of the same test, which we ran for our sister site, two of three Sonnet runs wrote them off and lost two points each, and said so in their notes.
Fable in round 2 is one run, not three, so its round 2 numbers show what one run cost, not a typical range.

FAQ

How much does Claude Fable 5.1 cost?

Anthropic lists Claude Fable 5.1 at $10 per million input tokens and $50 per million output tokens, with cache reads at $0.25 per million. In our test, one agent run closing a small month (about 300 rows) cost a median $0.70. A run five times the size cost $3.74, with prompt caching on.

Is Claude Fable 5.1 better than Opus 5.5 for agents?

Not on this job. Fable 5.1, Opus 5.5 and Sonnet 5.5 all found the same ten traps in every scored run. Fable cost 3.4 times as much as Opus in round 1 and 3.7 times in round 2, and it was the slowest, partly because it read one page per turn.

When should I pick Fable 5.1?

Anthropic's own guidance is to use Opus 5.5 for most workloads and to move to Fable 5.1 "when your evals on Claude Opus 5.5 at higher effort still fall short". Our results fit that advice. Run your own task on Sonnet or Opus first, and pay for Fable only if they fail it.

Our verdict

For a month-end close with tools, we would not use Claude Fable 5.1. Anthropic describes Fable as built for long-horizon agentic work, and this was such a job: 58 page reads, 10 traps, numbers to the cent. Sonnet 5.5 gave the same answers for about an eighth of the cost in round 1 and a seventh in round 2, in less than half the time. If you work in Claude, start agent jobs like this on Sonnet 5.5. Opus 5.5 explained its judgement calls most clearly in our runs, so it is the step up if you want that, and keep Fable for jobs where both of them fail. Whatever model you use, ask it to list the calls it made (written-off fees, overpayments) next to the numbers, because those notes are what a person checks before closing the month.

How this was made: an AI (Claude) helped write this article; the RanThis editor ran the test script, checked every number against the raw results and checked facts against the linked sources.

GPT-6.1 Terra vs GPT-6 Astra vs Claude Fable 5.1: Messy Sales Sheet Test

RanThis — Wed, 28 Oct 2026 04:12:25 +0000

Originally published on RanThis.

Cover: AI-generated illustration.

OpenAI released GPT-6.1 Terra on 26 October as its cheaper, faster model for everyday work: $1 per million input tokens and $5 per million output, a tenth of GPT-6 Astra's list price. Cheap only helps if the answers hold up, so the day after launch we gave Terra, GPT-6 Astra and Claude Fable 5.1 the same messy sales sheet: 40 rows, six planted problems, three branch totals that have to be right to the dollar. On "high" effort Terra matched Astra's perfect score in all three runs at about a tenth of the cost. The more useful finding was elsewhere: one ambiguous cell changed a branch total by $10,800 from one run to the next, on the same model with the same settings.

Measured on 27 Oct 2026 through the OpenAI and Anthropic APIs, from a Python script. The data is invented; the task, answer key and scoring code were committed before any model was called, and the raw answers are kept with this article's source files.

Key results

Terra on "high" scored 10/10 in all three runs, the same as GPT-6 Astra, in a median 12.9 s vs 16.3 s, for $0.0062 vs $0.061 per run.
Terra on its default setting ("medium") was faster and cheaper still ($0.0053, 11 s) but scored 10, 9, 9. In two of three runs it dropped one row instead of counting it.
Claude Fable 5.1 scored 9, 10, 10 at $0.10 per run, the most expensive of the five setups.
"Max" effort scored 10/10 three times but took 39–106 s and 3,100–9,400 output tokens per run, 4× the cost of "high" for the same score.
Every model found all six planted problems in every run, and none flagged a clean row. Every lost point came from the same cell: row 18, a price written "1.2k".
What we did not test: Terra's headline feature, computer use (operating apps on screen), or the ChatGPT apps. This is one structured task through the API, three runs per setup.

Setup

Task: a CSV of 40 invented Q3 sales rows for an office furniture company with branches in Boston, Chicago and San Francisco, prices in US dollars. Every model got the same prompt: total revenue per branch, the top branch, and every abnormal row by number, returned as JSON.
Rules in the prompt: exclude rows that look like bad data; normalize rows that are only written differently (an abbreviation, another name for the same branch or price) and count them; count duplicates once.
Planted problems: a duplicated order (rows 13 and 39, both must be listed), the date 2026-09-31, a unit price written "1.2k", the branch written "SF", and a quantity of −2.
Scoring (code, not AI): 1 point per branch total correct to the dollar, 1 for the top branch, 1 per planted problem found, minus 1 for each clean row flagged. Maximum 10.
Models: gpt-6.1-terra at reasoning effort medium (the model's default), high and max; gpt-6-astra at high; claude-fable-5-1 at its default. Three runs each, interleaved, no streaming.
Cost: tokens × list price ($1/$5 per million for Terra, $10/$50 for Astra and Fable 5.1), output including reasoning tokens. We did not subtract OpenAI's cached-input discount, so the OpenAI figures are slightly high. All 15 runs together cost $0.61.

The results

Model (effort)	Score, 3 runs	Median time	Median output tokens	Median cost per run
GPT-6.1 Terra (medium, default)	10, 9, 9	11.1 s	824	$0.0053
GPT-6.1 Terra (high)	10, 10, 10	12.9 s	995	$0.0062
GPT-6.1 Terra (max)	10, 10, 10	55.5 s	4,836	$0.025
GPT-6 Astra (high)	10, 10, 10	16.3 s	973	$0.061
Claude Fable 5.1 (default)	9, 10, 10	19.1 s	1,699	$0.101

One cell, two different totals

Row 18 sold 9 filing cabinets at "1.2k", that is $1,200. Every other filing cabinet in the sheet sold at $3,200. Our answer key treats "1.2k" as a price written differently, so it should be normalized and counted. But a price that low can also look like a typo, and the sheet invites that reading: the shorthand and the odd price sit in the same cell.

Astra and Terra on high and max counted it in every run. Terra on default counted it once and dropped it twice; Fable 5.1 dropped it once and counted it twice. Same prompt, same data, same settings.

What the models wrote about the row is the part to watch. When Terra (default) counted it, its note said: "The unit price was normalized from 1.2k to 1200." When it dropped it, the note said only: "The price of 1.2k means $1,200, which differs from the $3,200 price of the other filing cabinets." That note never says the row was left out of the total; you find out only by checking the Chicago number. Fable's note in the run where it dropped the row was explicit: the price "does not match the Filing cabinet price of 3200 used everywhere else, so the row was excluded as bad data."

Earlier that day we had run the same 40 rows and six traps as a Vietnamese-language sheet (Vietnamese labels, prices and shorthand); the English sheet above was derived from it. The pattern flipped: there, Terra on default counted row 18 in all three runs and Terra on high dropped it in all three. So the effort setting is not what decides this row; it varies run to run. Only GPT-6 Astra and Terra on max counted it in all six runs across the two sheets.

What this means if you clean sheets with AI

Ask what happened to each flagged row, not only why it is flagged. "Counted after fixing" and "excluded" should be separate answers. In our runs, a note about a price difference sometimes meant the row was dropped and sometimes did not.
Write the rule for your edge case into the prompt. "A price written like 1.2k means 1,200; count it" removes the coin flip. The models followed the rules they were given; the gap was in a rule we left open.
Run it twice, or check one total by hand. A single run can't show you the variance; two runs that disagree on a total tell you where to look.
Don't jump to maximum effort. Terra on high matched max on accuracy here at a quarter of the cost and under a quarter of the time.

What worked

JSON output parsed on every run of every model; no run needed a retry for format.
All five setups caught the duplicate, the impossible date, the "SF" branch and the negative quantity in every run.
Pinning the task before calling any model kept the comparison fair: each model got byte-identical input.

What didn't work

Three runs per setup is enough to show that row 18 flips, not to measure how often. Terra on default dropped it in 2 of 3 runs here and 0 of 3 on the Vietnamese-language sheet.
One API request (Terra on max, run 3) hung for over nine minutes on our side; we stopped it and re-ran that run. The timing in the table is from the completed re-run.
One task cannot rank these models in general. OpenAI's launch claims for Terra (computer use, scores on Agents' Last Exam, a benchmark of agents using professional software) are theirs; we did not measure them.

Our verdict

For structured office work like checking and totalling a sales sheet, GPT-6.1 Terra on "high" did everything GPT-6 Astra did in our test, a little faster and at about a tenth of the price. The default setting is cheaper again but less steady on judgment calls. Whichever model you use, make it say whether each flagged row was counted or excluded, and spell out how to treat shorthand like "1.2k": on our sheet, that one cell was the difference between two Chicago totals $10,800 apart.

How this was made: an AI (Claude) helped write this article; the RanThis editor ran the test script, checked every number against the raw results and checked facts against the linked sources.

ElevenLabs API key: free tier tested on 4 models, incl. v4 and Vietnamese

RanThis — Mon, 26 Oct 2026 03:56:02 +0000

An ElevenLabs API key works on the free plan, and it is enough to try every current text-to-speech model, including Eleven v4. We created one with a 2,000-credit cap and ran the same sentence through four models in English and Vietnamese, then hit the free plan's walls on purpose to see the exact errors. Below: how the key behaves, what each call cost in credits, how fast it was, and what the free tier blocks.

Measured on 24 Oct 2026 with one key on an ElevenLabs Free account, from a Python script on a Windows 11 PC (WSL, Ubuntu 24.04) in Vietnam. Prices are from ElevenLabs' pricing pages, read the same day.

Key results

The free key ran all four models we tried: Flash v2.5, Multilingual v2, v3 and the new v4. Each call returned a 128 kbps MP3 of about 8–9 seconds for a 140-character sentence.
Flash cost half. A 140-character sentence cost 70 credits on Flash v2.5 and 140 on the other three. The API says so in a character-cost response header on every call.
The usage counter lags. Right after eight calls, /v1/user/subscription still showed 0 credits used; a few minutes later it showed 1,050, exactly the sum of the headers.
Pick the model by language. Played back through ElevenLabs' own speech-to-text, Vietnamese from Flash v2.5, v3 and v4 came back word for word; Multilingual v2, which does not list Vietnamese, came back garbled (61% word error rate).
Streaming is fast. Flash v2.5 over the streaming endpoint sent its first audio bytes in 0.19 s and finished in 0.40 s.
The free tier blocks three things we tried: 192 kbps MP3 (Creator and up), 44.1 kHz PCM (Pro and up) and voices from the community library (Creator and up). The 21 default voices work.

Setup

Account: ElevenLabs Free, 10,000 credits a month, no payment method.
Key: created on 24 Oct 2026 in the ElevenLabs web app under Developers → API Keys, with a 2,000-credit limit and no expiry. It sits in an .env file on the server that runs the script, never in a browser.
Client: Python 3.13 with httpx, calling https://api.elevenlabs.io directly with the xi-api-key header (no SDK).
Voice: "Bella", the first of the 21 default voices the key could list.
Text: one sentence of 140 characters in English and one of 140 characters in Vietnamese, the same meaning in both.
Checks: credits from the character-cost header and from /v1/user/subscription; timing from the start of the request to the last byte; Vietnamese accuracy by sending each file to /v1/speech-to-text (scribe_v1) and comparing the transcript with the original sentence. Audio length is estimated from file size at 128 kbps.

How to get an ElevenLabs API key

Sign in to ElevenLabs (a free account is enough).
Open Developers → API Keys and create a key.
Give it a name per app, and set the limits ElevenLabs offers: which endpoints the key can call, a credit quota, and an IP allowlist. We set a 2,000-credit quota so a bug in the script could not drain the month's 10,000 credits.
Store it as an environment variable (ELEVENLABS_API_KEY) on your server.
Send it in an xi-api-key header on every request. A first test call is GET /v1/models, which lists the models your key can use.

ElevenLabs' docs say plainly that the key is a secret and should not go into client-side code such as browsers or apps. For calls that must come from the client, the docs point to single-use tokens instead.

What the API told us when the key was wrong:

Request	Status	Message
No key	401	"Neither authorization header nor xi-api-key received, please provide one."
Wrong key	401	"Invalid API key"
Our key	200	Subscription details: tier `free`, 10,000-credit limit

What each model cost and how fast it was

Same voice, same 140-character sentence, one call each, full MP3 returned (not streamed):

Model	Credits (EN / VI)	Time to full file, EN	Time to full file, VI	Audio length
Flash v2.5	70 / 70	2.04 s*	0.36 s	~8.7 s / ~7.5 s
Multilingual v2	140 / 140	1.43 s	1.48 s	~8.9 s / ~9.1 s
v3	140 / 140	2.67 s	3.17 s	~8.4 s / ~8.4 s
v4	140 / 140	2.51 s	2.35 s	~9.2 s / ~8.9 s

*The first call of the run, which included opening the connection; the Vietnamese Flash call right after took 0.36 s.

On the streaming endpoint (/v1/text-to-speech/{voice_id}/stream), Flash v2.5 delivered the first bytes of the English sentence in 0.19 s and the whole file in 0.40 s. That is the number that matters for anything that talks back to a user.

The /v1/models list explains the credits: Flash, Turbo, v4 Turbo and v3 Conversational carry a 0.5 character-cost multiplier, the rest 1.0. It also shows the per-request ceiling on the free plan: 40,000 characters for Flash v2.5, 10,000 for v4 and Multilingual v2, 5,000 for v3.

Which models speak Vietnamese

We fed each Vietnamese file to ElevenLabs' speech-to-text and compared what came back with what we sent.

Model	Vietnamese in its language list	Word error rate	What came back
Flash v2.5	yes	6.5%	Identical except "khoá" spelled "khóa" (both are correct spellings)
v3	yes	6.5%	Same as Flash
v4	yes	6.5%	Same, but the first word heard as "Khoa"
Multilingual v2	no	61.3%	Mostly garbled: "giống như mật khẩu" became "Gyeonghu mắt kháu"

So the 6.5% for the first three is spelling, not pronunciation. Multilingual v2 still returns audio and charges full credits for a language it does not support, without an error. Check a model's languages in /v1/models before you choose it.

A machine transcript is not a native listener. It catches words the model got wrong, not how natural the voice sounds.

What the free key cannot do

What we tried	Result
`output_format=mp3_44100_192`	403: "Output format 'mp3_44100_192' is only available on the Creator tier and above."
`output_format=pcm_44100`	403: "Output format 'pcm_44100' is only available on the Pro tier and above."
A voice from the community voice library	400: "You need to be on the creator tier or above to use this voice."
Instant voice cloning	Not tried; the subscription response reports `can_use_instant_voice_cloning: false`

Failed requests did not use credits. The 21 default voices and 128 kbps MP3 all worked. The subscription response also lists a limit of 3 custom voices on our free account.

The other limit is the licence. On ElevenLabs' pricing page the Commercial License is listed from the Starter plan up, not on Free, so a free key is for building and testing.

What it costs after the free credits

Plan or billing	Price	Credits or characters
Free	$0	10,000 credits a month, no rollover
Starter	$6 a month (a $1 first month was on offer until 1 Nov when we checked)	30,000 credits a month, commercial licence
Creator	$22 a month ($11 the first month)	121,000 credits a month
API, pay as you go	$0.08 per 1,000 characters on v4, v3 and Multilingual v2; $0.04 on Flash/Turbo	billed in dollars, not credits

Prices exclude tax, according to the pricing page. At the Flash rate our 140-character sentence costs well under a cent: $0.04 per 1,000 characters is about $0.0056 for 140.

What worked

One environment variable and a header: the first request worked without an SDK.
The character-cost header gave an exact cost per call immediately, which the subscription endpoint did not.
The per-key credit cap is a cheap safety net. Ours capped the test at 2,000 of the month's 10,000 credits.
Every limit we hit came back with a readable message that names the plan you need.

What didn't work

The usage counter was not real-time. If you meter your own users from /v1/user/subscription, you will undercount; read the response header instead.
Multilingual v2 accepted Vietnamese text and charged full price for unusable audio.
One sentence per model is enough to spot a broken language, not to rank voice quality. We did not run long texts, many voices or a listening test.

Our verdict

If you want to try ElevenLabs from code, a free API key is enough: it runs the newest model, v4, and the cheap Flash model, returns usable 128 kbps audio, and its limits are clear. Put a credit cap on the key, keep it server-side, and log the character-cost header rather than trusting the usage counter. For a language other than English, check that it is in the model's list first: Flash v2.5 was the cheapest and fastest model that read our Vietnamese correctly. Pay when you need commercial use, higher-quality formats or library voices.

Frequently asked questions

Is the ElevenLabs API key free?

Yes. A free account can create API keys and uses the plan's 10,000 monthly credits. On Flash v2.5 that was 0.5 credits per character in our test.

Where do I find my ElevenLabs API key?

In the ElevenLabs web app under Developers → API Keys. You can create several keys and limit each one by endpoints, credits and IP addresses.

How do I use the ElevenLabs API key?

Send it in an xi-api-key header, for example on GET https://api.elevenlabs.io/v1/models. Keep it on a server; ElevenLabs says not to put it in browser or app code.

Can I use ElevenLabs v4 with a free API key?

In our test, yes: Eleven v4 returned audio on a free key and cost 1 credit per character.

Can I use the free ElevenLabs API commercially?

ElevenLabs' pricing page lists the Commercial License from the Starter plan up, not on Free.

Original article with the full tables and sources: https://ranthis.com/guides/elevenlabs-api-key

Claude Haiku vs Sonnet 5.5: 18x Cheaper and Just as Accurate in a Three-Task API Test

RanThis — Sat, 24 Oct 2026 01:18:53 +0000

Originally published at ranthis.com. The scripts and raw answers are in the public repo.

We gave Claude Haiku 5.5 and Sonnet 5.5 the same three office tasks through the API and graded them with code. Both scored 15/15, but Haiku was faster and cost 18 times less.

By the RanThis team, written with AI assistance · Updated October 17, 2026 · 6 min read

Why trust this: every result, number and screenshot comes from a real run on our own accounts, not from the AI. How we test

In our Claude Haiku vs Sonnet test, Claude Haiku 5.5 matched Claude Sonnet 5.5 on three short office tasks: both scored 15/15 on code-graded checks. Haiku also finished faster (11.7 s vs 15.7 s in total) and cost 18 times less ($0.00129 vs $0.02357). Sonnet's edge never showed up on these tasks; long, multi-step work and coding were not tested.

Tested on 13 October 2026 by calling the Claude API directly with claude-haiku-5-5 and claude-sonnet-5-5, each at its default effort (Haiku: medium, Sonnet: high). Each task ran once, so this is a real-world observation, not a statistic. The tasks use made-up data and were written in Vietnamese, so this is also a test with non-English content. Prices come from Anthropic's official pricing page on the same day.

Watch the real run (Vietnamese narration): https://youtu.be/AqLwBZqVld8

Key results

Accuracy: Haiku 5.5 and Sonnet 5.5 both scored 15/15 items graded by code (5 invoice fields, 6 commission figures, 4 email conditions).
Speed: 11.7 seconds for all three tasks on Haiku vs 15.7 seconds on Sonnet.
Cost: $0.00129 on Haiku vs $0.02357 on Sonnet, an 18× gap. Scaled to 1,000 runs of these tasks, that is about $1.30 vs about $23.60.
Instruction following: each model slipped once on formatting. Sonnet ignored "No explanation" on the commission task; Haiku added a "[Company name]" signature after the hotline that was supposed to end the email.

Claude API pricing: Haiku 5.5 vs Sonnet 5.5

On the price list, Claude Haiku 5.5 costs a twentieth of Claude Sonnet 5.5.

	Claude Haiku 5.5	Claude Sonnet 5.5
Input price (per million tokens)	$0.10 (prompts up to 100,000 tokens)	$2
Output price (per million tokens)	$0.50 (prompts up to 100,000 tokens)	$10
Anthropic's description	Released 7 October 2026. For high-volume, latency-sensitive tasks such as classification, extraction and routing	The best combination of speed and intelligence

One Haiku 5.5 caveat: prompts over 100,000 tokens cost $0.50 input and $2.50 output per million tokens. Sonnet 5.5 keeps one price across its full 1M-token window. Sources: Anthropic's pricing page and the Claude Haiku 5.5 overview.

How we tested Claude Haiku vs Sonnet

Three office tasks, with answers worked out by hand before running: Read a messy email and extract invoice details as JSON. Traps: a mistyped amount that was later corrected, an old company name, and a due date written in words. Work out tiered sales commission for 5 staff: 2% / 4% / 6% by tier, plus a 1 million VND bonus for sales of 150 million VND or more with no returns. Traps: one person at exactly 200 million, one at 149.5 million (500,000 short of the bonus), and one with enough sales but a returned order. Write a reply to a customer whose order arrived late: at most 120 words, a 10% discount code, no promised delivery date, and end with the hotline.
The same prompt sent to each model through the API.
Code grades each item: 5 invoice fields, 6 commission figures, 4 email conditions. That is 15 items per model.
Real measurements: time from request to response, token counts reported by the API, and cost calculated from the price list.

Results task by task

Task 1: invoice extraction (JSON)

Both models returned the same JSON with all 5 fields right: tax code, pre-tax amount of 12,450,000 VND (not the mistyped figure), tax of 996,000, total of 13,446,000, and a due date of 24 October 2026. Haiku took 3.9 seconds, Sonnet 5.2.

Terminal output: the invoice JSON Claude Haiku 5.5 returned in 3.9 seconds, with all 5 fields graded correct by code

Task 2: tiered sales commission

Both got all 6 figures right, and neither paid the bonus to the person who was 500,000 short. The difference was in following the instruction "No explanation": Haiku returned only the table, while Sonnet wrote out its full working before the table.

Commission answers side by side: Haiku 5.5 gives only the table, Sonnet 5.5 writes out its working before the table

Task 3: customer apology email

The two drafts are very similar: 110 and 113 words. Both apologize, give the warehouse move as the reason, offer the 10% code and promise no date. Sonnet added a subject line and put the hotline on the last line as asked. Haiku put a "[Company name]" signature after the hotline.

The two customer emails side by side: Haiku 5.5 at 110 words, Sonnet 5.5 at 113 words, both with the 10% code and the hotline

Haiku 5.5 vs Sonnet 5.5: equally correct, 18× the cost

Across all three tasks, the Claude Haiku vs Sonnet totals look like this:

	Haiku 5.5	Sonnet 5.5
Correct (graded by code)	15/15	15/15
Total time, 3 tasks	11.7 s	15.7 s
Total cost, 3 tasks	$0.00129	$0.02357
These 3 tasks × 1,000 (scaled from the measurement)	about $1.30	about $23.60

Why 18× and not the 20× on the price list? Haiku used slightly more output tokens (thinking included): 674, 800 and 909 tokens, versus Sonnet's 640, 626 and 899.

Terminal summary: Haiku 5.5 and Sonnet 5.5 both score 15/15; Haiku took 11.7 s and cost $0.00129, Sonnet took 15.7 s and cost $0.02357

The comparison code (Python)

Below is the API call and cost calculation from the script we used (Python, httpx). Each task sends the same prompt to both models. A separate function compares the reply with the expected answer: it parses the JSON and checks each field (task 1), takes the last number on each table row (task 2), and counts words and looks for "10%", the hotline and any dates (task 3).

import time, httpx

PRICES = {"claude-haiku-5-5": (0.10, 0.50), "claude-sonnet-5-5": (2.00, 10.00)}  # USD per million tokens

def ask(model: str, prompt: str, api_key: str):
    started = time.monotonic()
    reply = httpx.post(
        "https://api.anthropic.com/v1/messages",
        headers={"x-api-key": api_key, "anthropic-version": "2023-06-01"},
        json={"model": model, "max_tokens": 8000, "messages": [{"role": "user", "content": prompt}]},
        timeout=300,
    ).json()
    text = "".join(b.get("text", "") for b in reply["content"] if b["type"] == "text")
    used = reply["usage"]
    price_in, price_out = PRICES[model]
    cost = used["input_tokens"] * price_in / 1e6 + used["output_tokens"] * price_out / 1e6
    return text, time.monotonic() - started, cost

We ran the same three-task method on Google's models too: see Gemini 4 Argon vs Gemini 3.8 Flash.

Limits of this test

Each task ran once. A rerun may word things differently.
All tasks were short with clear answers. Long multi-step work, coding and large-document analysis were not tested. That is where a larger model usually has an edge, and it needs its own test.
Very long prompts (over 100,000 tokens) make Haiku five times more expensive, narrowing the gap with Sonnet.

Is Claude Haiku 5.5 good enough? Picking a model for repeated work

Short, well-defined tasks (extraction, rule-based calculation, templated writing): try Haiku first. In this test it matched Sonnet, ran faster and cost 18 times less, which makes it the cheapest Claude model worth starting with for this kind of work.
Write a few lines of code to check the output, as this test does, so you know the cheaper model is good enough for your work.
For harder work, compare Claude Haiku vs Sonnet on your own tasks. With the API, switching models is a one-line change to model.

If you have a Claude Max plan, you can run tests like this with its included API credit: see how to claim Claude Max API credits.

Frequently asked questions

How much does Claude Haiku 5.5 cost?

$0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens; $0.50 and $2.50 above that. The Batch API is 50% off. Prices are from Anthropic's official pricing page on 13 October 2026.

Is Haiku 5.5 worse than Sonnet 5.5?

Not on the three short office tasks in this test: both scored 15/15 items graded by code, and each slipped once on formatting. Long, multi-step work and coding were not tested.

How much more does Sonnet cost than Haiku?

20 times by the price list, for prompts up to 100,000 tokens. In this test the actual total differed by 18 times ($0.02357 versus $0.00129), because Haiku used slightly more output tokens.

Which was faster, Claude Haiku or Sonnet?

Haiku 5.5. It finished all three tasks in 11.7 seconds versus 15.7 seconds for Sonnet 5.5; on the invoice task alone it took 3.9 seconds versus 5.2.

How can I compare two Claude models myself?

Prepare a few tasks with known answers, send the same prompt to each model through the API, and let code compare the replies with the answers. Record the time and the usage token counts from the API response to work out cost from the price list.

Sources

Claude API — PricingVendor page
Claude Haiku 5.5 — model overviewVendor document
Claude models overviewVendor document