Prompt InsightsSubscribe free
All terms

GlossaryUpdated

Context compaction

Context compaction is when an AI tool replaces the older part of a long chat with a short summary, so the conversation can keep going past its length limit.


Context compaction is what an AI tool does when a chat gets too long: it replaces the older messages with a short summary and carries on from that summary. The chat keeps going. But anything the summary left out is gone, and the AI does not know what it lost.

What it means for you at work

Have you had a long chat where the AI suddenly ignored a rule you gave at the start? Compaction may be the reason. You said "keep every email under 120 words" in your third message. Fifty messages later, the summary says "drafting customer emails", and the word limit is gone.

You cannot always see when it happens. Some tools tell you. Others do it quietly. Either way, the fix is the same: keep the important things somewhere a summary cannot lose them.

Why AI needs to do it

Every AI model can only read a fixed amount of text at once. This limit is called the context window. Think of it as the AI's desk. Everything in the chat has to fit on the desk: your messages, its answers, the files you pasted, and the results of any searches it ran.

The limit is measured in tokens, which are small pieces of words. In English, a token is about three quarters of a word. A long chat with a few large documents can fill the desk. Then the tool has three options:

  1. Stop, and ask you to start a new chat.
  2. Drop the oldest messages.
  3. Summarize the oldest messages and keep the summary.

Option 3 is compaction. Many tools that handle long tasks choose it. Dropping the oldest messages throws away the start of the chat, which is usually where your instructions were.

What the summary tends to drop

A summary is a guess about what will matter later. These things lose that guess most often:

  1. Rules you said once. "Do not mention pricing", said early in the chat.
  2. What already failed. If the summary drops "the client rejected the short version", the AI may suggest it again.
  3. Exact details. Names, numbers, dates, file names and quotes get rounded off or reworded.
  4. The reason behind a choice. The summary keeps "we chose option B" and drops why option A was ruled out.
  5. Your style notes. Corrections like "less formal" or "no bullet points" are often the first to go.

How to keep important details safe

  • Put standing rules in instructions, not in the chat. Custom instructions, project instructions or a saved brief load every time. They do not depend on a summary.
  • Keep a short notes block. For a long piece of work, keep a few lines with the goal, the decisions and the rules. Paste them back in when the AI starts to drift.
  • Make your own handover summary. Before a chat gets very long, ask the AI to summarize it with the prompt below. Then start a fresh chat with that summary.
  • Split big jobs. A chat that runs for hours is usually two or three jobs. Finish one, save what matters, and start the next one clean.
  • Check the next answer after a long stretch. If the AI repeats an old idea or breaks a rule, the summary probably lost it. Say the rule again.

A prompt for a handover summary

Use this when a chat is getting long. Then paste the result into a new chat:

Summarize this chat so I can continue the work in a new chat.
Use exactly these headings:
GOAL: the task in one or two sentences.
RULES: every rule or preference I gave, word for word, even if I said it only once.
DECISIONS: each decision we made, with the reason in one line.
DID NOT WORK: what we tried and why it failed.
WHERE WE ARE: what is done and what is still open.
NEXT: the next three steps.
Do not drop a rule to save space. Shorten everything else first.

The last line sets the priority. Rules and dead ends take little space, and they cost a lot when they are lost.

Compaction, memory and trust

Compaction happens inside one chat. Memory is what an AI keeps from one chat to the next. They meet at the summary: whatever the summary says becomes the AI's version of what happened.

That has one risk worth knowing. Suppose the AI read an email or a web page earlier in the chat, and it contained hidden instructions. Those instructions can slip into the summary as if you had asked for them. This is a form of prompt injection. If a long chat included outside content, read the summary before you rely on it.

Context compaction in the news

Questions

What does context compaction mean?

It means the AI summarizes the earlier part of a long chat so it takes up less space, then carries on from that summary instead of the full messages. The chat keeps going, but from a shorter account of what happened.

Why does my AI chat get worse after a while?

Often because the chat got too long. The AI may have summarized the early messages and lost a detail the work depended on: a rule you gave once, a decision and its reason, or something you already tried. Restate the rule, or start a new chat with a short summary of what matters.

What is /compact in Claude Code?

Claude Code is Anthropic's AI tool for coding work. /compact is the command that compacts the conversation by hand. Claude Code also compacts on its own when the conversation gets full. You can add a note after the command, such as which decisions to keep, and the summary will focus on them.

Is compaction the same as memory?

No. Compaction happens inside one chat and can lose details. Memory is information saved outside the chat on purpose, so the AI can read it back later, even in a new chat.

Should I keep going or start a new chat?

Start a new chat when the next task is different from the last one, and bring over a short summary of what matters. Keep going when you are in the middle of one task, and if the tool lets you, tell it what to keep when it summarizes.

Related terms

Prompt packs for this

27,287 readers and counting

One new term, explained, in every issue