Context Length

What is Context Length?

Context length refers to the amount of information a model can "remember" and process in a conversation, similar to a model's short-term memory. It enables the model to understand the conversation history and generate more coherent responses.

In short, the longer the context length, the more conversation content the model can refer to when answering questions.

How it Works

The following content occupies context space:

• All historical conversations (your questions and the model's answers)
• System instructions
• Current questions and attachments (such as images, web page content, etc.)

When the conversation content exceeds the context length limit, the system automatically removes content starting from the earliest messages to accommodate new input. At this time, you may see a prompt "Some messages have been removed to fit the model context length."

Configuration Method

Configure the context length in the model editor:

1. Open the model editor and find the "Context Length" setting.
2. Select the desired context length.
3. The configuration will be saved automatically.

How to Choose the Appropriate Context Length

Consider the following factors:

• Conversation Complexity: Long conversations or complex tasks require a longer context length.
• Cost Effectiveness: The longer the context length, the higher the computational cost. Choose the minimum length that meets your needs.
• Model Limitations: Ensure that the selected length is within the model's supported range. Different models support different maximum lengths.
• Hardware Resources: For local models, it is recommended to choose a shorter context length (such as 4k, 8k). Larger contexts consume more system resources and may cause the program to crash.

Handling Exceeded Context Length

When the conversation exceeds the context length limit, the system will:

1. Delete content starting from the oldest messages.
2. Display a prompt indicating that some messages have been removed.
3. Retain the latest user input and necessary system instructions.

This may cause the model to "forget" information from earlier conversations, affecting the coherence and accuracy of the answers.

If the context length of the current conversation still cannot be met after deleting all historical messages, the reasoning will be terminated. Because the text of the current conversation and system prompts cannot be deleted.

For the configuration of system prompts, please refer to this article.

Summary

Properly configuring the context length is crucial for model performance and usage costs. Please select the most suitable context length setting according to your actual needs.