How Dream Prompt Compression Keeps Long AI Sessions Fast and Focused

Long AI sessions have a predictable failure mode.

They start out sharp, fast, and cheap. Then the session grows. More user messages arrive. More assistant responses accumulate. Tool calls pile up. Function wrappers, function results, context lookups, and other runtime chatter quietly expand the working set until the model is carrying far more history than it actually needs.

That is where performance starts to drift.

Latency goes up. Cost rises. Focus softens. And in tool-heavy systems, the problem gets worse much faster than most people expect.

This is especially relevant in systems like Magic Cloud, where the AI is not just chatting, but actively generating APIs, invoking tools, inspecting context, and coordinating backend work. In those environments, long-session behavior matters a lot. It is one reason we also made token growth visible in the UI through real-time telemetry, so context bloat stops being guesswork and becomes observable. That same theme led to another improvement: dream prompt compression.

The problem with rolling session history

Magic keeps cached conversation history per session.

Over time, that cache can become large because it stores much more than plain user-assistant dialogue. It includes things like:

• user messages
• assistant replies
• function invocation wrappers
• function results
• tool lookup payloads
• other system and runtime messages produced during execution

That is useful for continuity, but expensive to carry forever.

In a tool-heavy conversation, much of the raw transcript is not durable knowledge. It is execution residue. The model does not necessarily need every old wrapper, every old tool payload, or every old intermediate step to stay coherent in the next turn. But if all of that remains in session memory, it still consumes tokens and still competes for attention.

This is the same practical problem behind our work on real-time token telemetry: once context growth becomes visible, it becomes obvious how much performance and cost are tied to session design rather than just model choice.

What dream prompt compression is

To address this, we implemented a mechanism in Magic we call a dream prompt.

The dream prompt is a post-processing compression step that takes rolling session history and compacts it into durable working memory before storing it back into cache.

Instead of replaying the entire older transcript forever, Magic now converts older conversation history into a compact summary and rebuilds the session in a smaller, more useful form.

The goal is simple:

keep what matters, discard what does not, and preserve continuity for the next turn.

How it works

The flow is now:

1. Load the cached session history.
2. Use that session for the current LLM request.
3. Let the model finish the full task, including all tool and function work.
4. After the task is complete, run dream compaction for the next turn.

The compaction itself works like this:

1. Take the current session-history messages.
2. Send them to OpenAI with a system instruction asking it to compress the conversation into durable working memory.
3. Receive back a compact summary.
4. Rebuild the cached session as:
   • one synthetic system message containing the compressed memory
   • plus the last user task and everything that happened after it

This means the next invocation still has two important things:

• durable memory of what happened before
• the full details of the most recent completed task

That combination matters a lot.

A summary alone is often not enough. But dragging the whole transcript forward is wasteful. Dream prompt compression sits between those extremes.

Why we moved it after the main LLM call

Originally, dream compaction was considered before the main LLM invocation.

That approach worked, but it had two practical problems.

First, it added latency to the critical path. If you compact before the main request, then every active user interaction pays that extra cost immediately.

Second, and more importantly, it could compact context while the current task was still unfolding. In multi-step function-invocation loops, that is risky. A task may still be gathering data, calling tools, or depending on intermediate state. Compressing too early can interfere with continuity right when continuity matters most.

So we moved the mechanism to after the main task is complete.

That makes the design simpler and safer. The current request runs on the existing session. The model completes its work. Only then do we compact the history for the next turn.

This reduces user-visible friction and avoids compressing an in-flight workflow.

The key design insight

One of the most important lessons was that keeping only a summary plus the last one or two messages was too lossy.

That sounds compact in theory. In practice, it breaks continuity.

If the last user prompt triggered one or more function calls, then the next user message may depend on the full result of those calls. The user might say:

• do the same for the next table
• use the output from the previous step
• now turn that into a widget
• compare this to what you just found

In those cases, preserving only a high-level summary is not enough. The recent task tail may contain the exact information needed to make sense of the next instruction.

So instead of keeping only a tiny trailing slice, Magic preserves:

• the compressed summary
• the last user message
• everything after that user message

That preserves one complete finished task cycle.

This is the real architectural point. The session is no longer a raw transcript, but it is also not a naïve summary. It becomes a hybrid memory model: compact long-term memory plus intact recent operational detail.

What we intentionally do not compress

Some messages do not belong in rolling session memory in the first place.

So the dream prompt intentionally does not compress:

• the main system instruction
• questionnaire or context prefix messages

Those are reattached on each invocation anyway, so there is no reason to keep them inside the rolling cache.

This keeps session memory focused on conversation continuity rather than static framing that is already supplied elsewhere.

That design is consistent with how Hyperlambda works: keep runtime structure explicit, avoid duplicated baggage, and make the execution model easier to reason about.

Why this matters in real agent workflows

This is not just a summarization trick.

It is a session architecture choice for long-running agents.

In systems where the model is generating tools, invoking functions, reading outputs, and coordinating backend execution, continuity is not just conversational. It is operational. The next user turn often depends on what the system actually did, not just what it talked about.

That is especially true in environments built for agentic work, such as AI agent builders and database-connected AI agents, where sessions often include tool chatter, data transformations, and execution-heavy workflows.

If you compress too aggressively, the agent loses grounding.

If you compress too weakly, the session becomes bloated and expensive.

Dream prompt compression aims for the practical middle: compress older history into durable memory, but keep the most recent task intact.

What we have seen so far

The token reduction has been substantial.

Observed examples include:

• reductions from around 20,000 tokens to 10,000
• reductions from around 50,000 tokens to 1,500

Those are not cosmetic improvements.

In some cases, the mechanism cuts context roughly in half. In others, it removes more than 95 percent of the rolling session footprint. The biggest gains tend to appear when older history is dominated by tool chatter, execution wrappers, and intermediate system output rather than truly essential conversational memory.

This lines up with a broader pattern we have seen elsewhere too: when unnecessary token flow is reduced, systems get cheaper, faster, and more focused. That same principle also shows up in a different form in our article on how Hyperlambda can cut AI agent costs by 75 to 90 percent, where the core idea is again to stop making the model carry work it should not be carrying.

Practical benefits

The benefits are straightforward:

• lower token usage
• lower latency
• lower cost
• better long-session stability
• less irrelevant historical baggage
• more focused model behavior

Just as important, the system becomes more predictable over time.

Without compaction, long sessions often degrade unevenly. Some stay usable. Others become sluggish and noisy. With dream prompt compression, the session has a better chance of staying operationally clean even after many rounds of tool use.

The remaining tradeoff

The current heuristic still preserves the entire last task tail, including large tool outputs when they matter.

That is intentional.

Yes, it means the final retained chunk can still be large. But the alternative is often worse. If the next user message depends on those outputs, removing them would damage continuity right where the system is most likely to need precision.

So the current design favors:

correctness and continuity over maximum compression

That is the right default for real agent workflows.

There may be future refinements. But as a practical baseline, keeping one full completed task cycle intact gives the system a strong balance between memory efficiency and operational reliability.

Conclusion

Dream prompt compression is not just about making sessions smaller.

It is about turning raw transcript history into something more useful: durable working memory plus intact recent context.

Older conversation state is compacted. The latest completed task remains fully available. Token growth drops dramatically. And long-running tool-heavy sessions behave much better as a result.

That makes dream prompt compression more than a summarization feature. It is part of a larger design philosophy: make token usage visible, reduce unnecessary context, preserve the parts that matter, and build AI systems that stay fast and focused even as sessions grow.

If you want to understand the bigger architecture behind that philosophy, it also connects naturally to what Hyperlambda is and how it works, why controlled runtime execution matters for AI agent builders, and why Magic Cloud now supporting GPT-5.5 becomes more useful when stronger models are paired with better session design rather than just bigger prompts.