How can you optimize Claude Sonnet 5's context window usage given the tokenizer change?

Recognize that the practical, usable capacity of the 1M-token window has effectively shrunk in text terms, since the same amount of actual content now consumes more tokens than it did on Sonnet 4.6 - budgeting context usage by document count or character count rather than by token count can significantly misjudge how much room is actually left.

Re-measure your typical context payloads - system prompts, retrieved documents, conversation history - against Sonnet 5's tokenizer specifically, rather than reusing a rough percentage-of-window estimate that was calibrated against Sonnet 4.6's token counts.

For workloads that were already operating close to the context ceiling on Sonnet 4.6, consider whether some previously-included content can be trimmed, summarized, or retrieved more selectively, since the same content mix may now push closer to, or past, the effective capacity than it did before, purely from the tokenizer shift.

Combine this with awareness of the shared thinking-and-response max_tokens budget on the output side, since a request that's both context-heavy on the input side and thinking-heavy on the output side compounds the tokenizer's effect from two different directions within the same request.