Why should you re-run token counting before migrating to Claude Sonnet 5?

Because the new tokenizer changes how the same text maps to tokens, any cost estimate, context-budget calculation, or max_tokens setting derived from Sonnet 4.6 token counts is no longer a reliable guide for Sonnet 5 - the underlying numbers have shifted independent of anything else about the migration.

This matters even for teams whose code requires zero other changes, since a request with no sampling parameters, no manual thinking configuration, and no prefilling can still be affected purely by the tokenizer difference, making it easy to overlook if migration testing focuses only on functional correctness rather than cost and budget behavior.

The concrete action is to use the token counting API against representative prompts on Sonnet 5 specifically, rather than reusing figures measured against Sonnet 4.6 or estimating the difference informally, since the actual multiplier can vary somewhat depending on the type of content being counted.