How to implement message interruptions with stop()?

HEAD
c123363Version Packages (#12611)
This post might have stale content, as HEAD is 1931 commits ahead.

how do i implement message interruptions? there's a "stop" from the @ai-sdk/react, but that doesn't auto-stop the stream server side, no? i mean it aborts the signal maybe, but the server might get "one more part" done, and the stop() is instant client side. do we accept that "one more part might come", or does the ai sdk provide a recommendation for this?

Avatar of julianbenegas
Julian Benegas
commented

The more advanced pattern from examples/next/app/api/chat/route.ts shows an approach where server-side cancellation is driven by a database flag rather than the HTTP connection:

good approach. but the timestamp might have issues/races as well, right? is a better metric, a "cancelledAtPartIndex" or something like that? something that the client can know "where did i stop" and send to the server for it to slice parts as needed...

Avatar of julianbenegas
Julian Benegas
commented

what i was thinking was the client sends its message.parts.length, which corresponds to UIMessage parts. i do my persistence in onFinish, and i think i have ui message parts at that point, right? toUIMessageStream, then onFinish.

but still, i may be storing parts that are not yet "done", right? (like tool calls). ah, but still, the index might match...

this seems like the way to go

Avatar of julianbenegas
Julian Benegas
commented

ah, but that wouldn't work well for text parts. we'd want chunk/token granularity... can it be done in that case?

Avatar of julianbenegas
Julian Benegas
commented

This is all doable, but entirely userland

how could this be done robustly? imagine i'm using useChat in the client, and the toUiMessageStream + onFinish in the backend.

In practice, the question is whether it's worth it

that's not something for you to judge. do your job.

Avatar of julianbenegas
Julian Benegas
commented

what if the client sent everything? like the full last part? so like lastPart: { index, part } or something. fully. so the server just fully slices + replaces


END OF POST

How to implement message interruptions with stop()? — vercel/ai