Skip to main content
Large language models have a physical constraint: the context window. No matter how smart the model is, the amount of information it can “see” at once is limited. Profy’s Context Engine solves this contradiction — enabling Experts to maintain comprehension of infinitely long conversations within a finite window.

​
Two Core Mechanisms

Context Compaction

When conversations exceed the model window, older messages are intelligently summarized to free space for new content — without losing critical information.

Output Recovery

When a model’s single output exceeds the token limit, truncation is automatically detected and seamlessly continued — completely transparent to the user.

​
Context Compaction: Five-Layer Pipeline

Context compaction is not simply “deleting old messages” — it is a fine-grained five-layer pipeline, each layer handling information compression at a different granularity:

​
Layer 1: Tool Result Budget

When a single turn has multiple tool calls (e.g., the Expert searches 10 web pages in parallel), the total volume of all tool results is governed by an aggregate budget. Results exceeding the budget are truncated by priority.

​
Layer 2: Historical Tool Result Shrinking

Tool results from earlier messages have decreasing reference value for current decisions. This layer automatically compresses old-turn tool outputs into summaries — preserving “what was done and what the conclusion was” while discarding raw details.

​
Layer 3: Cold Zone Message Folding

The earliest batch of messages in the conversation (the “cold zone”) is summarized by the AI into a structured summary injected as a system message. The summary preserves:
  • User’s key decisions and preferences
  • Important tool execution results
  • Conventions and context established during the conversation

​
Layer 4: Memory Flush

Before compaction occurs, information identified in the conversation with long-term value (user preferences, project context, key decisions) is written into the Expert’s long-term memory. Even when conversation messages are folded, this information is not lost.

​
Layer 5: Real-Time Tool Output Compression

When tool execution returns excessively long results (e.g., a bash command outputting thousands of log lines), they are intelligently truncated before entering the conversation state:
The design philosophy behind compression strategies is “better to let the AI re-fetch than to give it incorrect half-truncated information.” After file reads are compressed, the AI can always call the read tool again to get the full content.

​
Output Recovery: Transparent Continuation

Sometimes an Expert needs to generate very long content — a complete article, a complex code file, a detailed analysis report. When the output length exceeds the model’s single maximum token limit, the output recovery mechanism automatically takes over.

​
How It Works

1

Truncation detection

The Runtime monitors the model’s stop signal — different providers have different truncation markers (max_tokens, length), all unified under a single detection mechanism.
2

Tail anchor compression

Truncated output may be tens of thousands of tokens long. It is compressed to the last ~2,000 characters as a “tail anchor” — the AI only needs to know “where it left off” to continue. This step saves 91% of repeated input costs.
3

Continuation instruction injection

A special instruction is injected: “Your output was truncated. Continue directly from where you stopped — don’t apologize, don’t repeat, don’t summarize.”
4

Partial tool call handling

If truncation happens in the middle of tool call arguments, incomplete tool calls are automatically stripped, and the AI is instructed to re-issue the complete call.
5

Recovery limit

Recovery is attempted up to 3 times. If still incomplete after 3 recoveries, an “output truncated” notice is displayed with a “Continue generating” button.

​
User Experience

The entire recovery process is completely transparent to users:
Output recovery is not a “hack to make AI write more” — it’s an engineering solution to a physical model limitation. It’s as natural as packet segmentation and reassembly in network transmission.

​
Reactive Compaction

Beyond the preventive layered compaction, the system has a reactive safety net: When a model call returns a context overflow error (some providers directly reject oversized inputs), the Runtime doesn’t simply report an error — instead, it:
  1. Identifies the overflow signal
  2. Triggers emergency compaction (more aggressive summarization strategy)
  3. Re-issues the request with the compacted context
The user sees only a brief “Thinking…” — they may not even realize an overflow and recovery occurred in the background.

​
Quality Assurance for Smart Summarization

The core challenge of context compaction is not technical implementation, but “not losing critical information.” Profy’s summarization strategies follow several principles:
Explicit decisions made by the user (“use React,” “no TypeScript,” “budget of 10K”) have the highest priority in summaries and are never compressed away.
Tool execution conclusions (“file created successfully,” “42/42 tests passed”) are more important than the execution process (complete command output).
File contents can be re-read by the AI, web pages can be re-searched — this type of information is prioritized for compression when space is tight. But user-described requirements cannot be re-fetched and must be preserved.
The most recent 2-3 turns of conversation are kept in full original text; older messages are progressively replaced by higher-level summaries. This mimics human memory decay — recent events are detailed, distant ones only remembered by key points.

​
Collaboration with the Memory System

The Context Engine doesn’t work in isolation — it deeply collaborates with the Expert’s long-term memory system: Each time context is built, the system first retrieves relevant information from memory, then adds the current conversation history (after compaction), together forming the AI’s “working memory.” This lets the Expert remember key points from last week’s conversation while maintaining fluency in the current one.
The ultimate goal of the Context Engine is to make users feel like they’re talking to an AI with “unlimited memory” — while in reality, it’s carefully managing information trade-offs within a finite window.