API errors

context_length_exceeded: what it means and how to fix it

This error means the request needs more tokens than the model’s context window allows. The window includes the system prompt, tool schemas, conversation history, retrieved documents, the user message and max_tokens. If input plus reserve exceeds the window, the API rejects the request. Retrying the same payload will fail again.

P
Plugsky Editorial Team
Verified 19 November 2026

Short answer: compute input_tokens + max_tokens against the model’s context window. If it is over, reduce the prompt, reduce the reserve, or move to a larger-window model. Do not retry the same request; it is a request-size error, not a temporary rate limit.

The fit formula

usable_input = context_window
             − max_tokens
             − system_prompt
             − tool_schemas
             − safety_margin

fits = input_tokens ≤ usable_input

A 200K model with an 8K output reserve, 2K system prompt and 2K margin has 188K usable input. A 199K-token document does not fit even though it is under 200K. This is the most common reason the error appears “below the limit.”

Causes and fixes

CauseFix
Prompt is genuinely too largeSummarise, retrieve fewer chunks, or split the request
max_tokens is set to the model maximumLower it to the p95 completion length plus headroom
Conversation history keeps growingKeep the last few turns plus a rolling summary
Tool schemas are largeRemove unused tools; shorten descriptions and nested parameters
RAG sends too many chunksRetrieve 5–10 relevant chunks, not the whole corpus
Tokenizer mismatchUse the provider’s tokenizer or token-count endpoint
Workload is genuinely long-contextUse a larger-window model and still reserve output

Why your token count looked fine

Three things are easy to miss:

  • Tool schemas. Ten tools with nested JSON can consume thousands of tokens before the user says anything.
  • History. A chat that resends the full transcript grows every turn. A 20-turn conversation can be larger than the document.
  • Tokenizer differences. A character-based estimate can be 10–20% off on prose and further off on code or non-Latin text. A prompt that looks like 3,900 tokens can be 4,100 at admission.

Use the tokens-to-pages converter for a first estimate, then a native tokenizer or provider endpoint for the exact count.

How to fix it in code

context = 200_000
max_tokens = 8_000
system_tokens = 2_000
tool_tokens = 0
margin = 2_000

usable = context - max_tokens - system_tokens - tool_tokens - margin
input_tokens = count_tokens(prompt)  # use the provider tokenizer

if input_tokens > usable:
    raise ValueError(
        f"prompt {input_tokens} exceeds usable input {usable}; "
        f"trim {input_tokens - usable} tokens or use a larger model"
    )

Log the computed budget with the request ID. When the error appears in production, you want the input count, reserve and model window in one place, not three dashboards.

Chunk, summarise or switch models?

SituationBest move
Over by less than 5%Trim history or lower max_tokens; do not switch models
Document is 2–5× the windowSummarise or retrieve the relevant sections
Document is 10×+ the windowRAG or hierarchical summarisation; a bigger window is not a retrieval strategy
Repeated long-document workloadUse a 256K–1M model and measure the usable input after reserve
Needle-in-haystack recallRetrieve the section; long context alone can lose the middle

The context-window calculator shows the usable input per model and the page equivalent of the overage. If the document does not fit, it also reports how many chunks you would need.

Prevent it before the request

  1. Count the full prompt: system, tools, history, retrieved documents and user message.
  2. Reserve the p95 output length, not the model maximum.
  3. Keep a 2–5% safety margin for tokenizer error.
  4. Set a per-request budget in the client and reject oversized requests before calling the API.
  5. Log the input count and the rejected overage so the next fix is data-driven.

If the error is actually a rate limit, see the OpenAI 429 guide. A 429 is temporary; context_length_exceeded is not.

FAQ

What causes context_length_exceeded?

The request needs more tokens than the model’s context window allows. The window includes the system prompt, tool schemas, conversation history, retrieved documents, the user message and max_tokens. If input plus reserve exceeds the window, the API rejects the request.

Is context_length_exceeded a rate limit?

No. It is a request-size error, usually HTTP 400. Retrying the same request will fail again. Reduce the prompt, lower max_tokens, or use a model with a larger window.

Should I chunk, summarize or switch models?

If the document is much larger than the window, retrieve or summarize first. If you are slightly over, lower max_tokens or trim history. If the workload is genuinely long-context and repeated, use a larger-window model. Measure the usable input after the output reserve.

How do I calculate usable input?

usable_input = context − max_tokens − system_prompt − tool_schemas − safety_margin. The context-window calculator does this per model and shows whether the document fits.

Check the fit before the request fails

The context-window calculator subtracts output, prompt, tools and margin, then shows which models fit and how many chunks you need.

Open the calculator → See models