Prev Next

AI / Core OpenAI Codex Application Fundamentals Interview Questions

How do you handle context window management in long-running OpenAI applications?

Every OpenAI model has a finite context window - the maximum tokens (input + output) it can process in one call. In long-running applications (chat sessions, agentic workflows, document analysis), managing this window efficiently is critical for both functionality and cost.

from openai import OpenAI
import tiktoken

client = OpenAI()
enc = tiktoken.encoding_for_model("gpt-5.5")

# Context window strategy 1: Sliding window
# Keep the most recent N turns, discarding the oldest
def sliding_window(messages: list, max_tokens: int = 100_000) -> list:
    total = sum(len(enc.encode(str(m["content"]))) for m in messages)
    while total > max_tokens and len(messages) > 2:
        # Remove oldest user+assistant pair (keep system message)
        removed = messages.pop(1)
        total -= len(enc.encode(str(removed["content"])))
    return messages

# Context window strategy 2: Summarisation
# Summarise old conversation instead of dropping it entirely
async def summarise_history(messages: list[dict]) -> str:
    summary = await client.responses.create(
        model="gpt-5.4-mini",  # cheap model for summarisation
        input=f"Summarise this conversation concisely:\n{messages}",
    )
    return summary.output_text

# Strategy 3: Responses API state management (avoids manual history)
# Use previous_response_id - OpenAI manages the context server-side
first = client.responses.create(model="gpt-5.5", input="Hello!", store=True)
second = client.responses.create(
    model="gpt-5.5",
    input="What did I just say?",
    previous_response_id=first.id,  # server retrieves context automatically
)

# Strategy 4: use file_search for large static documents
# Instead of pasting 100k-token documents into every prompt:
# Upload once to a vector store, retrieve only relevant chunks
vector_store_id = "vs_abc123"
response = client.responses.create(
    model="gpt-5.5",
    tools=[{"type": "file_search", "vector_store_ids": [vector_store_id]}],
    input="What does section 4.2 of our API spec say about authentication?",
    # Only the relevant section is retrieved, not the entire doc
)

Context management strategies
StrategyBest forTrade-off
Sliding windowConversational UIsMay lose important early context
SummarisationLong research sessionsAdds latency and cost for summariser call
previous_response_idResponses API multi-turnRequires store:true; server manages context
RAG / file_searchLarge static documentsRetrieval may miss nuanced context
What does using 'previous_response_id' in the Responses API provide for context management?
Why is RAG (via file_search) preferred over pasting entire documents into the context window?

Invest now in Acorns!!! 🚀 Join Acorns and get your $5 bonus!

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Earn passively and while sleeping

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.


Webull! Receive free stock by signing up using the link: Webull signup.

More Related questions...

What is OpenAI Codex and how has it evolved from its original form? What are the current recommended models for OpenAI Codex and the API, and when do you choose each? What is the OpenAI Responses API and how does it differ from the Chat Completions API? What are the built-in tools available in the OpenAI Responses API? What is the OpenAI Agents SDK and what are its four core primitives? What is the Codex CLI and what are its key features? What is the Chat Completions API and when should you still use it? What is function calling (tool use) in the OpenAI API and how does it work? What are structured outputs in the OpenAI API and how do you use them? What is the OpenAI Assistants API and what is its deprecation timeline? What is prompt caching in the OpenAI API and how does it reduce costs? What are reasoning models in the OpenAI API and what is the 'effort' parameter? What are OpenAI's key API authentication and security concepts? What are rate limits in the OpenAI API and how do you handle them in production? What is the Batch API and when should you use it? What is streaming in the OpenAI API and how do you implement it? What is OpenAI's Codex Skills feature and what are Automations? What is Model Context Protocol (MCP) and how does it integrate with OpenAI tools? What is fine-tuning in the OpenAI API and when should you use it? What are embeddings in the OpenAI API and what are they used for? What is the OpenAI moderation API and why is it important for application safety? What are OpenAI's image generation models and how do you use them in applications? What is the OpenAI Realtime API and what use cases does it enable? What is retrieval-augmented generation (RAG) and how do you implement it with OpenAI? What is OpenAI's approach to responsible use and safety in the API? What is the OpenAI token system and how do you count and optimise token usage? What are guardrails in the context of OpenAI application development? What is the OpenAI Files API and how is it used for document management? How do you implement multi-agent systems using the OpenAI Agents SDK? What is the OpenAI Evals framework and why is evaluation critical for production applications? How do you implement error handling in OpenAI API applications? What is the OpenAI Codex App and what are its main features for software teams? What are the key differences between OpenAI's o-series reasoning models and the GPT series? How does the Codex IDE extension integrate with development environments? What are the key considerations for building production-grade OpenAI applications? What is the OpenAI Codex and API pricing model and how do you estimate costs? What is the OpenAI computer use capability and what does it enable? How do you use the OpenAI API for code generation, review, and debugging tasks? What is the role of system prompts (instructions) in OpenAI applications and how do you design them effectively? How do you handle context window management in long-running OpenAI applications?
Show more question and Answers...

Database

Comments & Discussions