How does prompt caching work across different models?

how does prompt caching work across different models? say, fable/claude model? checkpoint every step of the way?

Prompt caching is provider-side prefix caching; OpenCode does not keep a local cached copy of the model state. Cache entries are generally scoped to the provider/model, so switching from one Claude model to another should be treated as a cache miss.

###Claude / Anthropic

On the default AI SDK runtime, OpenCode adds ephemeral cache markers to:

• The first two system messages
• The last two non-system messages

That produces at most four breakpoints, matching Anthropic’s limit. See packages/opencode/src/provider/transform.ts:359-405.

Every agent/tool step sends the conversation again, so the markers are recalculated each step. Consequently:

• The stable system/tool/user prefix can be reused.
• The newest trailing breakpoint advances as tool calls and results are appended.
• The provider may write a new cached prefix for that newer boundary.
• This is not a durable model checkpoint or KV-state snapshot after every step.

Conceptually:

request 1: [system ... user] [cache marker]
request 2: [same prefix] [cache hit] assistant/tool result [new marker]
request 3: [same longer prefix] [cache hit] more tool results [new marker]

The markers use type: "ephemeral", normally the provider’s short-lived cache window. The cache only hits when the prefix—including tools, system prompt, messages, and their ordering—is identical.

If you literally mean a future/aliased model such as claude-fable-5, it follows the Claude/Anthropic behavior when routed through the Anthropic SDK. The cache logic recognizes Anthropic/Claude through provider ID, model/API ID, or SDK package; it is not hard-coded per Claude family (transform.ts:466-484).

###Experimental native runtime

With OPENCODE_EXPERIMENTAL_NATIVE_LLM enabled, the newer native path uses a slightly cleaner automatic policy:

1. Last tool definition
2. Last system part
3. Latest user message

The latest-user boundary is intentional: it remains stable throughout all assistant/tool round-trips within that user turn, allowing each internal step to reuse the prefix. See:

• packages/llm/src/cache-policy.ts
• packages/llm/src/schema/options.ts:250-274

###Other providers

The session cache key handling is in packages/opencode/src/provider/transform.ts:1261-1274.

So the short answer is: each inference step is cache-aware and can establish a newer cached prefix, but OpenCode is not checkpointing the model after every step. It is repeatedly marking/routing matching prompt prefixes and letting the selected provider manage the cache.

The first two system messages
The last two non-system messages

interesting. so

The last two non-system messages

do you mean last two steps inside an assistant turn?

Not exactly. It means the last two ModelMessage entries in the request array whose role is not system, not two abstract execution steps.

For an agentic turn, the history might look like:

user       "Fix the bug"
assistant  tool_call(read_file)
tool       tool_result(...)
assistant  tool_call(edit_file)
tool       tool_result(...)

On the next model request, the last two non-system messages are:

assistant  tool_call(edit_file)
tool       tool_result(...)

Those receive cache markers. After another tool round-trip, the trailing pair moves forward.

At the beginning of a user turn, it may instead be:

assistant  previous final response
user       new question

A message can also contain multiple parts—for example, several parallel tool calls or results. That still counts as one message, not several steps.

So: roughly the latest tool/model round-trip inside the overall user turn, but implemented structurally as two messages rather than semantically as “two steps.” The exact selection is:

msgs.filter((msg) => msg.role !== "system").slice(-2)

in packages/opencode/src/provider/transform.ts:361.

so they're essentially always moving the marker to the end... why? can you do a timetable of the calculation?

Yes—the two non-system markers form a rolling cache frontier near the end of the prompt.

The reason is that a cache marker means:

“Try to reuse the longest cached prefix ending at or before this point; cache the remaining prefix through this point.”

It does not mean “only cache these two messages.” Everything before a marker is part of the cacheable prefix.

The exact OpenCode calculation on every request is:

systemMarkers = messages.filter(m => m.role === "system").slice(0, 2)
tailMarkers   = messages.filter(m => m.role !== "system").slice(-2)

markers = unique([...systemMarkers, ...tailMarkers])

##Example timeline

Assume:

• S = system prompt
• U1 = user request
• A1 = assistant tool call
• T1 = tool result
• A2 = another assistant tool call
• T2 = another tool result
• * = cache marker

###Request 1: initial user prompt

S* → U1*

Selected markers:

system:     S
non-system: U1

Likely result:

cache read:  none, assuming a cold cache
cache write: prefix through S and/or U1

The cache entry through U1 represents:

[S, U1]

###Request 2: after the first tool execution

S* → U1 → A1* → T1*

Selected tail messages:

A1, T1

The marker moved forward, but the earlier cache is still useful. Anthropic searches backward from a breakpoint for a matching cached prefix. If the earlier S → U1 prefix is within its lookback window:

cache read:  [S, U1]
new input:   [A1, T1]
cache write: extended prefix through T1

Afterward, the provider can have a cache entry equivalent to:

[S, U1, A1, T1]

###Request 3: after the second tool execution

S* → U1 → A1 → T1 → A2* → T2*

Selected tail messages:

A2, T2

Likely calculation:

cache read:  [S, U1, A1, T1]
new input:   [A2, T2]
cache write: extended prefix through T2

###Request 4

S* → U1 → A1 → T1 → A2 → T2 → A3* → T3*

Likely calculation:

cache read:  [S, U1, A1, T1, A2, T2]
new input:   [A3, T3]
cache write: extended prefix through T3

##Why move the frontier?

If the only marker remained fixed at U1, each subsequent request could reuse:

[S, U1]

but would repeatedly process the entire growing tool transcript:

[A1, T1, A2, T2, A3, T3, ...]

Moving the marker allows the provider to incrementally cache that transcript:

call 1: cache through U1
call 2: read through U1, cache through T1
call 3: read through T1, cache through T2
call 4: read through T2, cache through T3

So it behaves somewhat like incremental checkpointing, but technically it is a series of provider-side cached prompt prefixes, not model-state checkpoints.

##Why two trailing messages?

Two gives multiple nearby breakpoint opportunities and usually covers one complete agent round-trip:

assistant tool call
tool result

It also helps with provider lookback rules and message structures where one message expands into several Anthropic content blocks. If the newest boundary cannot match, the provider can search backward for an earlier cached prefix.

The permanent system marker acts as the fallback:

S*

Even if the rolling transcript no longer matches or exceeds the provider’s lookback window, the static system/tool prefix may still produce a cache hit.

One caveat: this describes OpenCode’s default AI SDK path in packages/opencode/src/provider/transform.ts:359-405. The experimental native runtime uses a different policy—last tool definition, last system part, and latest user message—so its message boundary intentionally stays fixed during the tool loop rather than advancing on every step.

put numbers into the calculation please. and at every point, the whole conversation is being written into the cache?

Yes, the newest cache marker makes the entire input prefix up to that marker cacheable. Because OpenCode marks the last non-system message, that usually means the whole conversation available before the next generation.

But after the first request, the whole conversation is not necessarily rewritten and billed again. The provider can read the old prefix and write only the newly appended suffix.

##Numerical example

Assume:

Therefore, each tool round-trip adds:

100 assistant tokens + 900 tool-result tokens = 1,000 tokens

For simplicity, assume all prefixes meet Claude’s minimum cacheable size and each request arrives before the ephemeral cache expires.

###Request 1: initial user request

S  = 10,000 tokens
U1 =    500 tokens
------------------
Total = 10,500 tokens

Prompt:

S* → U1*

Cache calculation:

Conceptually, the resulting cached prefix is:

[S, U1] = 10,500 tokens

Claude then generates A1, but that newly generated output is not yet part of the input cache.

###Request 2: after the first tool result

Suppose:

A1 = 100 tokens
T1 = 900 tokens

The next prompt contains:

S → U1 → A1* → T1*

Total:

10,000 + 500 + 100 + 900 = 11,500 tokens

Assuming the previous prefix hits:

Calculation:

old cached prefix = 10,500
new suffix        =  1,000
new total prefix  = 11,500

The resulting cache can represent:

[S, U1, A1, T1] = 11,500 tokens

So yes, the newest cached prefix contains the whole conversation—but only the new A1 + T1 suffix needed to be written on this request.

###Request 3: after the second tool result

Add:

A2 = 100 tokens
T2 = 900 tokens

Prompt:

S → U1 → A1 → T1 → A2* → T2*

Total:

11,500 previous tokens + 1,000 new tokens = 12,500

Calculation:

Resulting cached prefix:

[S, U1, A1, T1, A2, T2] = 12,500 tokens

###Request 4

Add another 1,000-token tool round-trip:

S → U1 → A1 → T1 → A2 → T2 → A3* → T3*

Resulting prefix:

13,500 tokens

##Timeline summary

Without caching, Claude would process at the normal input rate:

10,500 + 11,500 + 12,500 + 13,500 = 48,000 input tokens

With perfect rolling cache reuse:

cache writes = 10,500 + 1,000 + 1,000 + 1,000 = 13,500
cache reads  = 10,500 + 11,500 + 12,500      = 34,500

The same 48,000 tokens are logically present across the requests, but most are charged at the cheaper cache-read rate.

##Example cost

Using an illustrative base input rate of $3 / million tokens, and Anthropic’s documented relative rates used by OpenCode’s cache rationale:

• 5-minute cache write: 1.25 × input = $3.75/M
• Cache read: 0.1 × input = $0.30/M

Request 3 would cost approximately:

11,500 cache-read tokens × $0.30/M = $0.00345
 1,000 cache-write tokens × $3.75/M = $0.00375
                                          -----
                                         $0.00720

Without caching:

12,500 normal input tokens × $3.00/M = $0.03750

Output-token charges are separate.

##Important distinction

At request 3, the provider may logically hold a cached prefix containing all 12,500 input tokens. That does not mean OpenCode transmitted only 1,000 tokens—it still sends the complete 12,500-token conversation. It means the provider recognizes:

first 11,500 tokens: already cached
last   1,000 tokens: new cache write

Also, the response being generated right now is not included yet. It only becomes cacheable when OpenCode sends it back as part of a subsequent request.

So the concise answer is:

The newest cache frontier generally covers the whole conversation so far, but on a successful hit the provider reads the old prefix and writes only the newly appended portion—not the entire conversation from scratch.