Claude Sonnet 5 is a Disappointment... (Fable is Back Though!)

Englishالعربية Deutsch Español Français हिन्दी Bahasa Indonesia 日本語 한국어 Português Русский 中文

Computing/SoftwareBusiness NewsInternet Technology

Transcript

00:00:00Anthropic dropped Claude Sonnet 5 last week and it was disappointing, but at least Fable 5 is back,

00:00:04just maybe nerfed, and oh, Claude Code might have some spyware-like code in it to detect Chinese

00:00:09usage. Let's jump in. I'll start with Sonnet 5. This is actually the first Sonnet update in four

00:00:18and a half months, and Anthropic says this is built to be their most agentic Sonnet model yet,

00:00:22with performance close to Opus 4.8, but at lower prices. Pricing is a really interesting one with

00:00:28this model, so we'll be coming back to that. First though, let's take a look at the benchmarks

00:00:31that Anthropic provides. It beats its predecessor on all of these, and it nears Opus on a few of them

00:00:35like Terminal Bench and Computer Use, even beating Opus slightly in knowledge work. I prefer to look

00:00:40at third-party benchmarks though, and the ones I trust are from Artificial Analysis, and on that

00:00:44coding index, it comes out a few points behind Opus, but very slightly ahead of GPT 5.4. It's made a very

00:00:50nice leap from its predecessor, Sonnet 4.6. It's the same story on the Intelligence Index as well, slotted

00:00:56between GPT 5.5 and GPT 5.4, but interestingly, on the agentic index, it actually outperformed GPT 5.5.

00:01:03So when they said in that blog that they put a lot of work into its agentic capabilities,

00:01:06it seems to have paid off here. From the benchmarks then, it looks like a pretty competent model,

00:01:10and it's definitely a step up from Sonnet 4.6, but now let's talk about that pricing element.

00:01:14On the API pricing, they're actually doing a discounted price at $2 for a million input tokens,

00:01:18and $10 for a million output tokens, that is until August 31st. Then it's going to go back to the same

00:01:23price as the other Sonnet models, which was $3 for a million input tokens and $15 for a million output

00:01:28tokens. This places it around Gemini 3.5 Flash and GPT 5.6 Terra, if we ever get access to that,

00:01:34and as they said, it's cheaper than Opus 4.8. The trouble is though, in practice, this is just not

00:01:39true at all. This model is extremely token hungry. It uses around 69,000 tokens to run one task in the

00:01:45Artificial Analysis Intelligence Index, making it more than Sonic 4.6, Opus 4.8, Fable 5, GPT 5.4,

00:01:52and 5.5. If you look at it on this quadrant here, where green is the place to be, it is as far away

00:01:57as it can be out of these models. This then obviously has a knock-on effect on the actual cost of a task,

00:02:02and Artificial Analysis found that Sonic 5 came out to be more expensive than Opus 4.8 per task,

00:02:07only beaten by Fable. Plus, if you take a model like GPT 5.5, which actually does better on most of the

00:02:12benchmarks, it is half the price of Sonic 5. It is worth noting here that Artificial Analysis are

00:02:16actually using the standard price of the model and not the discounted one, so I would be super

00:02:20curious to see if Anthropic try and fix this by maybe extending that discount or just leaving that

00:02:25as the permanent pricing. Even worse than cost per task though, Artificial Analysis actually found to

00:02:29run their entire test suite that Sonic 5 cost more than Fable 5. Like, what happened here? Even on

00:02:34Cursor Bench, if you look at Sonic 5 High, it's about the same price as Opus 4.8 for a similar

00:02:39result, and as you go up the effort levels, Sonic 5 Max scores worse than Opus 4.8 extra high while

00:02:44costing more. It just really feels like they need to update or fix something here, or I just don't see

00:02:48where this model fits in. It is by no means a bad model capability-wise, just economically, and I've

00:02:54been a big fan of the Sonic models. I think they were the first ones that many of us daily drove,

00:02:58but over the past few months I've always used Opus 4.8, and none of this has made me change my mind.

00:03:02Especially with Fable being back, it's going to be Fable for the hard tasks,

00:03:05and Opus 4.8 for daily driving, and Sonic for nothing. Talking of Fable 5, the export controls

00:03:11got lifted, and it's now back for everyone, regardless of your citizenship, but sadly you

00:03:15had a week to use this within your plan limits, or at least 50% of your plan limits, and after July 7th

00:03:20it will only be available via usage credits. There were some interesting points in that blog about

00:03:24the initial ban. It did actually come from a supposed jailbreak from an Amazon researcher who managed to

00:03:29prompt it to find security vulnerabilities, and in one case demonstrated how to exploit one,

00:03:33but when looking into this, Anthropic found that loads of models like Claude Opus 4.8,

00:03:37GPT 5.5, and Kimi K 2.7 could identify the exact same vulnerability as Fable 5 did in that report,

00:03:43and when it came to a demonstration of how to exploit the single vulnerability,

00:03:47every single model tested could do it, including Claude Haiku 4.5, Sonic 4.6, Opus 4.6,

00:03:524.7, 4.8, GPT 5.4, 5.5, and Kimi K 2.7. So it seems to me that the ban was pretty pointless

00:03:58then, and all it's really achieved is that Anthropic have now made that safeguards even

00:04:02stricter on Fable, making loads more normal requests get blocked, and in their announcement

00:04:06tweet they actually said that routine tasks like debugging and coding would fall back to Opus 4.8,

00:04:11but an Anthropic employee has later corrected this saying it should only be a small number of them.

00:04:14But it has seemingly impacted some benchmarks more than others, with claims that Fable has now been

00:04:18nerfed. Basically it's just falling back to Opus 4.8 more, so it's performing a lot worse on these

00:04:23benchmarks. On a similar topic to the export controls and the bans, some people noticed that Claude

00:04:27code now has some attempts at fingerprinting in a very sneaky way, by modifying its system

00:04:32prompts date string. This is a great blog post that goes into all of the details, and I'll leave this

00:04:35linked below, but the TLDR is there's a function in Claude code that checks if your time zone is in

00:04:40China. If it is, your date string will change from using hyphens to using slashes. Then it will also

00:04:44check if your API-based URL matches this list, or if the hostname contains specific AI lab keywords

00:04:49like DeepSeq, Minimax, or ZAI, and if it does, it will change the apostrophe in today's to a different

00:04:55Unicode character that basically looks the exact same. It's a very clever technique called steganography,

00:05:00and most of you won't be impacted by this. It's basically meant to try and detect those API

00:05:03resellers, unauthorized Claude code gateways, and model distillation attacks. I don't really have an

00:05:08issue with them trying to do that, I just prefer that they're a bit more honest about the things that

00:05:12are in Claude code, and don't try to hide it in this way. Do you think that's an issue, and what do you

00:05:16think of Sonic 5 and Fable's Return? Let me know in the comments down below, or are you there subscribed,

00:05:20and as always, see you in the next one.

Key Takeaway

While Sonnet 5 shows improved agentic performance, its high token consumption and actual cost per task make it less economical than Opus 4.8 or GPT 5.5 for practical use cases.

Highlights

Claude Sonnet 5 consumes approximately 69,000 tokens per task, exceeding the token usage of Sonnet 4.6, Opus 4.8, Fable 5, and GPT models.
Artificial Analysis benchmarks show Sonnet 5 costs more per task than Opus 4.8, despite Anthropic's marketing claims of lower pricing.
Fable 5 is available to all users again following the lift of export controls, but is now subject to stricter safeguard triggers.
Claude Code utilizes steganography via system prompt modification to identify API resellers and unauthorized model distillation attempts by checking for specific time zones and hostname keywords.
Every model tested, including Haiku 4.5 and GPT 5.5, could replicate the exploit used as justification for the initial Fable 5 export ban.

Timeline

Claude Sonnet 5 Performance and Economic Viability

Sonnet 5 surpasses Sonnet 4.6 in benchmarks and outperforms GPT 5.5 on the agentic index.
The model is significantly more token-intensive than previous iterations, often exceeding 69,000 tokens per task.
Operational costs for Sonnet 5 exceed those of Opus 4.8 per task when using standard pricing.
API discounts for Sonnet 5 are scheduled to end on August 31st, returning pricing to $3 per million input tokens and $15 per million output tokens.

Anthropic released Sonnet 5 as its most agentic model to date, with benchmark scores nearing Opus 4.8. However, third-party analysis reveals that the model is extremely token-hungry. This high consumption leads to a higher total cost per task compared to both Opus 4.8 and GPT 5.5. Because of this economic inefficiency, Opus 4.8 remains the preferred choice for daily tasks.

Fable 5 Return and Security Safeguards

Export controls on Fable 5 are lifted, making the model accessible regardless of user citizenship.
Initial export bans were based on security exploit concerns that proved to be universal across all major tested AI models.
Stricter safety measures on Fable 5 now trigger more frequent fallbacks to Opus 4.8 for routine coding tasks.
Access to Fable 5 will be limited to usage credits after July 7th.

The return of Fable 5 follows a controversial ban initiated after a researcher demonstrated a security exploit. Testing revealed that this vulnerability and the ability to exploit it were present in every major model, including various versions of Claude and GPT. Consequently, the ban appeared ineffective, leading to current restrictions where Fable 5 is now more prone to blocking normal requests and defaulting to Opus 4.8.

Claude Code Steganographic Fingerprinting

Claude Code modifies system prompt date strings based on user time zones and API hostnames.
Unicode character manipulation serves as a steganographic method to identify unauthorized model distillation.
The identification mechanism targets API resellers and specific AI labs like DeepSeek, Minimax, and ZAI.

Claude Code incorporates a fingerprinting technique designed to detect unauthorized usage. The system silently changes character formats, such as date separators or apostrophes, when it detects specific environmental conditions like China-based time zones or requests routed through specific AI hostnames. This practice aims to combat model distillation and unauthorized gateway services.

Community Posts

Write about this video