LLM Context Windows

In the previous tutorials, we learned about tokens, vectors, embeddings, transformers, attention, and parameters.

Now let’s understand another important concept in Large Language Models:

Context

Before we understand a context window, let’s first understand what context means.

📌 What is Context?

Context is the information that helps us understand something correctly.

Consider this sentence:

The boy went to the bank.

What does bank mean?

We don’t know yet.

Now add another sentence:

The boy went to the bank to deposit money.

Now we understand that the bank is a financial institution.

The additional information helped us understand the first sentence.

That additional information is context.

Humans use context naturally when communicating.

LLMs also need context to process language effectively.

🧠 Context in an LLM

Consider:

Ravi bought a new phone yesterday. He likes it very much.

To understand “He” and “it”, we need information from the earlier part of the sentence.

The earlier words provide context for the later words.

This is one of the things an LLM processes when it receives text.

🪟 What is a Context Window?

An LLM cannot process an unlimited amount of information in one request.

It has a maximum amount of information that it can consider.

This is called the context window.

A context window is the maximum amount of context an LLM can handle for a particular request.

Context windows are measured in tokens.

For example, a model might support:

8K tokens
32K tokens
128K tokens

The actual limit depends on the model.

Remember:

📋 What Goes Into the Context Window?

The context isn’t necessarily just your latest question.

Depending on the application, it can include:

For example, during a conversation:

User: My name is Ravi.

User: I am learning Python.

User: What am I learning?

The earlier messages can be included in the context when processing the last question.

🛑 Context Has a Limit

Suppose a model has a context window of:

100,000 tokens

You cannot keep adding information indefinitely.

Eventually, the context reaches its limit.

A long conversation might look like:

Message 1
Message 2
Message 3
...
Message 500

If the total information becomes too large, the application has to manage the context.

For example, it might:

• Remove older messages
• Summarise older information
• Keep only relevant information

The exact approach depends on the application.

The important point is:

A context window has a finite size.

⚠️ Context Is Not Memory

This is an important distinction.

Suppose you tell an AI:

My favourite colour is blue.

That information can be available in the current conversation context.

But that doesn’t mean the model has permanently learned:

“This person’s favourite colour is blue.”

Context is information available during processing.

Memory is a broader concept where information can be stored and retrieved later.

So:

while:

They are related concepts, but they are not the same thing.

⚠️ Context Is Not Training

Context should also not be confused with training.

During training:

During normal use:

Giving an LLM information in a conversation does not normally retrain the model or change its parameters.

🤔 Why Does Context Window Size Matter?

Consider two situations.

Small amount of context

Question
+
A few previous messages

This is easy to fit into a context window.

Large amount of context

Question
+
Hundreds of previous messages
+
Large amounts of additional information

This requires much more context capacity.

So a larger context window allows an application to provide more information to the model at once.

This can be useful for things such as:

• Long conversations
• Large documents
• Long pieces of code
• Multiple instructions

💡 A Simple Example

Imagine a model has a context window of 10 tokens.

Suppose the current context contains:

6 tokens

There is much less room remaining than if the context contained:

2 tokens

The numbers here are just an analogy.

Real models work with much larger token counts.

The important idea is:

The information provided to the model consumes part of its available context.

📤 Context Window and Output

An LLM doesn’t only receive information.

It also generates information.

For example:

The model’s context rules determine how much input and generated output can fit within the available context.

This is why the amount of input information can affect how much output can be generated.

The exact limits and accounting depend on the model and API.

⚖️ Context Window vs Parameters

These are two completely different things.

Parameters

The learned numerical values inside the model.

Context window

The amount of information the model can process for a particular request.

So:

Parameters describe the learned model; the context window describes how much information it can work with at one time.

📚 Context Window Is Not the Model’s Total Knowledge

An LLM may have learned from a huge amount of information during training.

But that doesn’t mean all of that information is sitting inside the context window when you ask a question.

Think of it this way:

Then, when you use the model:

The context is the information currently available to the model for that request.

🧩 Key Takeaways

Let’s keep the important points simple.

• Context

  Information that helps the model understand the current input.

• Context Window

  The maximum amount of context an LLM can handle for a particular request.

• Context is measured in

  Tokens.

• Context is not

  • Training
  • Parameters
  • Permanent memory

• Why does it matter?

  Because an LLM can only work with a finite amount of context at one time.

🧩 Our Journey So Far

We have now covered:

The next question is very natural:

Once we give information to a trained LLM, how does it produce an answer?

That process is called Inference.