AI Context Windows

In the last lesson, you learned AI Transformers and Attention. Now we need context windows. This simply means: how much information the model can see at one time.

Here is what fits inside context:

🧠 What Is Context?

In an AI request, context means the information available to the model right now.

In normal English, context means background information.

Example:

• If I say “it is too expensive”, you need context to know what “it” means.
• Am I talking about a phone, a course, a car, or a server?
• Without context, the sentence is unclear: The listener cannot know what “it” refers to.

AI works in a similar way. The model needs enough background information to answer properly.

Context can include:

The model uses this information to generate the next answer.

📌 Context Is the Model’s Working Area

The model can only work with what is in front of it.

• If the refund policy is in context, the model can use it.
• If the refund policy is not in context, the model may guess.
• If old chat history is removed, the model may not know what was discussed.
• If the context contains wrong data, the model may repeat wrong data.

So context quality matters as much as model quality.

🌍 Context in ChatGPT, Claude, and Gemini

🪟 What Is a Context Window?

A context window is the maximum amount of text the model can handle in one request.

Window does not mean a browser window here. It means the limited space available for information.

+--------------------------------+
|        Context window          |
|                                |
| System instructions            |
| Conversation                   |
| Retrieved documents            |
| User question                  |
| Space for output               |
+--------------------------------+

The size is measured in tokens.

So a context window is like a fixed-size notebook page. You can write only so much on it.

So if the request is too large, the app must remove, summarize, or retrieve less information.

🔄 What Happens When Context Is Too Large?

The application must make a decision.

• Reject and ask for smaller input: If the user pastes a huge document, the app can ask them to upload a smaller part.
• Summarize older messages: If the chat is long, the app can keep a short summary instead of sending every old message.
• Remove irrelevant chunks: If some document parts do not help the question, the app should not send them.
• Retrieve fewer documents: In RAG, the app can fetch only the top useful sections instead of many weak sections.
• Split the task: If one request is too big, the app can handle it in smaller steps.

Bad apps hide this problem. Good apps manage it clearly.

User asks: What is our refund policy?

Good context:
Refund policy document included or retrieved

Bad context:
No policy document included

🧠 Context Window Is Not Memory

This distinction is very important.

• Context window is the information sent in the current request.
• Memory is information saved somewhere for later use.

An LLM does not automatically remember everything forever.

This point is very important for beginners:

• Chat history shown on screen is not always fully sent to the model.
• Old details may be summarized: The app can keep a short version instead of every old message.
• Some details may be removed: Less useful messages may be left out to save context space.
• Long conversations can exceed the limit: If the chat becomes too long, it cannot all fit into one request.

If an application has memory, that memory is usually implemented by software around the model:

• Database: The app stores facts or user data in tables and fetches them when needed.
• User profile: The app stores preferences like language, role, or learning level.
• Vector store: The app stores searchable document chunks for RAG.
• Conversation summary: The app saves a short version of old chat so it can fit into context later.
• Settings file: The app stores configuration like preferred model, tone, or output format.

🧩 Types of Context in an AI App

Not all context has the same job.

• Instructions: These tell the model how to behave, like “answer simply” or “use only the provided document”.
• Conversation tells the model what has already happened, so it does not answer like every message is brand new.
• Documents provide factual grounding, so the model can answer from trusted text instead of guessing.
• Tool results provide live or private information, like account status or current price.
• Examples show the desired format, like “return answer in three bullet points”.

When debugging an AI answer, check which type of context was missing or weak.

🤔 Why Context Windows Matter

Context affects:

More context is useful only when it is relevant. Too much unrelated context can confuse the model.

🧩 What You’ve Learned

• ✅ Context is the information the model can see in the current request.
• ✅ Context window and memory are different: context is sent now, memory is stored for later.

🚀 What’s Next?

When we build a support bot, we want steady answers. When we build a story bot, we may want creative answers. Next, let us learn the settings that control this.