how does the modelRuntime thing work?

I have enough context now to explain how ModelRuntime works.

#How ModelRuntime works

ModelRuntime is the coding-agent's canonical "hub" for models + auth + streaming. It lives at packages/coding-agent/src/core/model-runtime.ts and implements the Models interface from @earendil-works/pi-ai. Every piece of the SDK/agent that needs to know "what models exist," "does this provider have a key?" or "stream a completion" goes through this object.

##What it composes together

The class holds several layered sources of provider information and merges them:

// model-runtime.ts:96-113
export class ModelRuntime implements Models {
  private readonly models: MutableModels;              // the pi-ai Models collection
  private readonly credentials: RuntimeCredentials;    // auth.json + runtime overrides
  private readonly defaultBuiltins: ReadonlyMap<...>;  // pristine builtin providers
  private readonly builtins = new Map<...>();          // effective builtins (+ Radius overlays)
  private readonly nativeExtensionProviders = new Map();  // SDK-registered Provider objects
  private readonly extensionProviders = new Map();        // SDK-registered ProviderConfigInput
  private readonly compositionErrors = new Map();
  private readonly modelsPath: string | undefined;     // ~/.pi/agent/models.json (usually)
  private config: ModelConfig;                         // parsed models.json
  private snapshot: ModelRuntimeSnapshot = { ... };    // cached "what's available now"
}

The four provider sources it merges (in providerIds() at ~L191):

1. Builtin providers from @earendil-works/pi-ai/providers/all (Anthropic, OpenAI, etc.), each wrapped via withRemoteCatalog so they can fetch remote model listings.
2. Radius overlays created from models.json oauth: "radius" entries (see configureRadiusProviders).
3. User models.json overlays (custom models, base URLs, headers, api keys) via ModelConfig.
4. Extension providers registered at runtime via registerProvider / registerNativeProvider.

recomposeProvider(providerId) uses composeModelProvider(...) from provider-composer.ts to blend base+config+extension into one effective Provider, then hands it to the underlying pi-ai Models via this.models.setProvider(...).

##Construction

// model-runtime.ts:134
static async create(options: CreateModelRuntimeOptions = {}): Promise<ModelRuntime>

create():

• Wraps a CredentialStore (defaults to AuthStorage reading authPath, typically <agentDir>/auth.json) in RuntimeCredentials.
• Loads models.json from modelsPath (default <agentDir>/models.json) via ModelConfig.load.
• Picks a ModelsStore (file-backed at models-store.json, or in-memory).
• Instantiates builtin providers (wrapped with a remote catalog fetcher, except radius).
• Runs configureRadiusProviders() then rebuildProviders().
• Optionally does a network refresh of catalogs (gated by allowModelNetwork and env PI_OFFLINE), with a default 15s timeout.

PI_OFFLINE=1 disables all network model refreshes.

##The snapshot cache

Because auth checks and availability are async but a lot of UI/CLI code needs synchronous answers, ModelRuntime keeps a snapshot:

// model-runtime.ts:49
interface ModelRuntimeSnapshot {
  all: readonly Model<Api>[];
  available: readonly Model<Api>[];            // models whose provider has auth configured
  configuredProviders: ReadonlySet<string>;
  storedProviders: ReadonlySet<string>;        // provider ids present in credential store
  auth: ReadonlyMap<string, AuthCheck | undefined>;
}

runAvailabilityRefresh() (L246) rebuilds it by calling getAvailable, checkAuth on every provider, and credentials.list() in parallel. queueAvailabilityRefresh coalesces concurrent callers onto a single in-flight refresh (availabilityRefresh promise). forceRefreshAvailability chains a fresh refresh after any pending one so mutations don't see stale results.

That's why you see both sync APIs (hasConfiguredAuth, isUsingOAuth, getAvailableSnapshot) and async ones (getAvailable, checkAuth).

##Public API surface (grouped)

Discovery — mostly sync, backed by snapshot or the underlying Models:

• getProviders(), getProvider(id)
• getModels(providerId?), getModel(providerId, modelId)
• getAvailable(providerId?) — async; kicks a refresh
• getAvailableSnapshot() — sync, last known available list
• hasConfiguredAuth(providerId), isUsingOAuth(providerId)
• getError() — aggregates config + composition + availability errors

Auth:

• checkAuth(providerId), getAuth(providerOrModel, overrides?) — overrides can supply an apiKey, env, and minOAuthValidityMs. When given a Model, it also merges provider-configured request headers (from models.json and extension config) on top of the pi-ai auth result.
• setRuntimeApiKey(providerId, apiKey) / removeRuntimeApiKey(providerId) — in-memory overrides that are not persisted to auth.json.
• listCredentials(), getProviderAuthStatus(providerId) — status is one of runtime, stored, configured (from models.json/extension), or environment (via pi-ai check).
• login(providerId, type, interaction) / logout(providerId) — persists via credentials.

Streaming — the thin wrapper around each Provider:

// model-runtime.ts:466
stream<TApi>(model, context, options?): AssistantMessageEventStream
complete<TApi>(model, context, options?): Promise<AssistantMessage>
streamSimple(model, context, options?): AssistantMessageEventStream
completeSimple(model, context, options?): Promise<AssistantMessage>

They all go through the private prepareRequest (L440):

1. Look up the effective Provider.
2. Call getAuth(model, { apiKey, env }) to resolve credentials.
3. Merge auth headers with caller headers; optionally run transformHeaders (used e.g. for OpenRouter attribution).
4. If auth supplied a baseUrl, override model.baseUrl.
5. Call provider.stream / streamSimple.
6. Wrapped in lazyStream so errors during preparation surface through the event stream instead of being thrown synchronously.

Registration & refresh (used by extensions and settings changes):

• registerProvider(id, ProviderConfigInput) — validated via validateExtensionProvider, then re-composed; also does an optimistic snapshot update if auth is already configured, then schedules a background refresh({ allowNetwork: false }).
• registerNativeProvider(Provider) — same idea, but you supply a full pi-ai Provider object rather than a config.
• unregisterProvider(id).
• refresh(options) — reloads models.json, rebuilds providers, calls models.refresh (which is the network catalog refresh in pi-ai), then forces an availability refresh.

##How the agent-session wires it in

Callers rarely construct these primitives themselves — createAgentSession does it:

// packages/coding-agent/src/core/sdk.ts:174-176
const authPath = options.agentDir ? join(agentDir, "auth.json") : undefined;
const modelsPath = options.agentDir ? join(agentDir, "models.json") : undefined;
const modelRuntime = options.modelRuntime ?? (await ModelRuntime.create({ authPath, modelsPath }));

And in agent-session-services.ts:139-180, after creating the runtime it applies any pending extension provider registrations from the ResourceLoader, then calls modelRuntime.refresh({ allowNetwork: false }) before returning the services bundle.

The AgentSession itself stores it and uses it throughout (agent-session.ts:403):

• Resolving auth for the current model when prompting (_getRequiredRequestAuth, _maybeGetRequestAuth).
• Detecting OAuth vs API key for error messages.
• Enforcing setModel (rejects if checkAuth fails).
• Cycling available models (_cycleAvailableModel).
• Auto-refreshing the model reference after config changes (_refreshCurrentModelFromRegistry).
• Exposing registerProvider/unregisterProvider to extensions.

Model-resolution helpers (resolveCliModel, resolveInitialModel, resolveModelScope) all take a modelRuntime and use it as the sole source of truth for "does this provider exist," "does it have auth," and "what are its models."

##Typical usage patterns

Default (agent-dir based):

const modelRuntime = await ModelRuntime.create();
const { session } = await createAgentSession({ modelRuntime, sessionManager: SessionManager.inMemory() });

Custom paths / in-memory credentials:

const modelRuntime = await ModelRuntime.create({
  authPath: "/my/app/auth.json",
  modelsPath: "/my/app/models.json",
});
// or:
const inMem = await ModelRuntime.create({ credentials: new InMemoryCredentialStore() });

Ephemeral key override (not written to disk):

modelRuntime.setRuntimeApiKey("anthropic", process.env.MY_KEY!);

The full example is at packages/coding-agent/examples/sdk/09-api-keys-and-oauth.ts, and the docs section is under "API Keys and OAuth" and "Model" in packages/coding-agent/docs/sdk.md.

###TL;DR mental model

Think of ModelRuntime as a stateful facade over pi-ai's Models:

• In: builtin catalog + models.json + extension registrations + credentials.
• Cached: a synchronous snapshot of models/availability/auth.
• Out: model lookup, provider auth status, and stream/complete calls that already have credentials + headers + baseUrl resolved.

Anywhere in the coding-agent that touches "which model, which key, how do I call it," it does so through a single ModelRuntime instance.