How does context compaction work?

dev
14d1e20Revert "fix(app): support anthropic models on azure cognitive services" (#8966)
This post might have stale content, as dev is 7685 commits ahead.

how does context compaction work?

Avatar of julianbenegas
Julian Benegas
commented

what model is used for the summary generation?

Avatar of julianbenegas
Julian Benegas
commented

how is it used with the AI sdk? generateText? or streamText?

Avatar of julianbenegas
Julian Benegas
commented

nice. any reason why they don't let the stream fail and THEN compact, vs preemtively doing it?

Avatar of julianbenegas
Julian Benegas
commented

are u sure they have a gap here? maybe they're handling it in another way?

Avatar of julianbenegas
Julian Benegas
commented

do they compact with some threshhold? meaning idk, at 90% full?

Avatar of julianbenegas
Julian Benegas
commented

what gets presented to the compaction agent? the full conversation in its context? or a path for it to read the convo in the filesystem?

Avatar of julianbenegas
Julian Benegas
commented

cool. do they include recent messages after compaction? or just the summary and then straight to work?

Avatar of julianbenegas
Julian Benegas
commented

doesn't the compaction agent get confused if they pass the full conversation as if they have been part of it? then ask it to summarize?

Avatar of julianbenegas
Julian Benegas
commented

sounds fine. so they ignore the "system" role message? doesn't that kill prompt caching?

Avatar of julianbenegas
Julian Benegas
commented

i meant the original system prompt, not the compaction agent's system prompt

Avatar of julianbenegas
Julian Benegas
commented

gotcha, makes sense. do the compaction messages include all the tool calls and results? doesn't that make it super expensive?


END OF POST

How does context compaction work? — anomalyco/opencode