const stream = createDataStream({ execute: (dataStream) => { const result = streamText({…
- User
const stream = createDataStream({
execute: (dataStream) => {
const result = streamText({
model: myProvider.languageModel(selectedChatModel),
system: systemPrompt({ selectedChatModel, requestHints }),
messages,
maxSteps: 5,
experimental_activeTools:
selectedChatModel === 'chat-model-reasoning'
? []
: [
'getWeather',
'createDocument',
'updateDocument',
'requestSuggestions',
],
experimental_transform: smoothStream({ chunking: 'word' }),
experimental_generateMessageId: generateUUID,
tools: {
getWeather,
createDocument: createDocument({ session, dataStream }),
updateDocument: updateDocument({ session, dataStream }),
requestSuggestions: requestSuggestions({
session,
dataStream,
}),
},
onFinish: async ({ response }) => {
if (session.user?.id) {
try {
const assistantId = getTrailingMessageId({
messages: response.messages.filter(
(message) => message.role === 'assistant',
),
});
if (!assistantId) {
throw new Error('No assistant message found!');
}
const [, assistantMessage] = appendResponseMessages({
messages: [message],
responseMessages: response.messages,
});
await saveMessages({
messages: [
{
id: assistantId,
chatId: id,
role: assistantMessage.role,
parts: assistantMessage.parts,
attachments:
assistantMessage.experimental_attachments ?? [],
createdAt: new Date(),
},
],
});
} catch (_) {
console.error('Failed to save chat');
}
}
},
experimental_telemetry: {
isEnabled: isProductionEnvironment,
functionId: 'stream-text',
},
});
result.consumeStream();
result.mergeIntoDataStream(dataStream, {
sendReasoning: true,
});
},
onError: () => {
return 'Oops, an error occurred!';
},
});
const streamContext = getStreamContext();
if (streamContext) {
return new Response(
await streamContext.resumableStream(streamId, () => stream),
);
} else {
return new Response(stream);
}
} catch (error) {
if (error instanceof ChatSDKError) {
return error.toResponse();
}
}
}
I want to replace the function above by a call for python backend, but wihout changing the behavior of the application. this is a repo provided as is from vercel ai, ai chatbot sdk
- Assistant
To integrate a Python backend (e.g., FastAPI, Flask, etc.) and replace the
streamTextfunction call in this Vercel AI Chat SDK-style JavaScript code — without breaking the app's behavior or structure — you need to ensure that:- The Python backend mimics the behavior of
streamText() - Streaming remains intact (i.e., server-sent events / response chunking)
- Tool handling and responses continue working normally
- Telemetry and ID generation still happen as needed
- You abstract the execution using
fetchor other stream-capable client call
✅ Plan Overview
Your goals:
- Replace this line:
→ with:const result = streamText({ ... });const result = await callPythonBackendWithStreaming({ ... });
🧠 Step-by-Step Breakdown
1. Setup Python backend (FastAPI example)
Here’s an example of Python endpoint that streams output word-by-word like
streamText({ experimental_transform: smoothStream({ chunking: 'word' }) }).# app.py (FastAPI) from fastapi import FastAPI from fastapi.responses import StreamingResponse from pydantic import BaseModel import asyncio app = FastAPI() class Message(BaseModel): role: str content: str class ChatRequest(BaseModel): model: str messages: list[Message] system_prompt: str def fake_llm_response(messages): response = "This is a simulated LLM response streamed word by word." for word in response.split(): yield word yield " " asyncio.sleep(0.1) @app.post("/chat-stream") async def chat_stream(data: ChatRequest): async def word_stream(): async for word in fake_llm_response(data.messages): yield word return StreamingResponse(word_stream(), media_type="text/plain")
2. From your JS code, replace
streamText(...)withfetch+ stream logicimport { createDataStream } from 'ai'; import { streamToAsyncIterable } from './utils/streamToAsyncIterable'; const stream = createDataStream({ execute: async (dataStream) => { const controller = new AbortController(); const response = await fetch('http://<your-python-api>/chat-stream', { method: 'POST', body: JSON.stringify({ model: selectedChatModel, system_prompt: systemPrompt({ selectedChatModel, requestHints }), messages, }), headers: { 'Content-Type': 'application/json', }, signal: controller.signal, }); if (!response.ok || !response.body) { throw new Error('Failed to connect to Python backend'); } // transform ReadableStream to AsyncIterable const streamIterator = streamToAsyncIterable(response.body); for await (const chunk of streamIterator) { // Send chunk to UI dataStream.append({ type: 'text-delta', value: chunk }); } dataStream.done(); // Perform onFinish logic // Here you'd handle assistant message, save messages, telemetry, etc. }, onError: () => 'Oops, an error occurred!', });
3.
streamToAsyncIterable.jshelperexport async function* streamToAsyncIterable(readableStream) { const reader = readableStream.getReader(); const decoder = new TextDecoder(); while (true) { const { value, done } = await reader.read(); if (done) break; yield decoder.decode(value); } }
🔁 Emulating Tooling (Optional)
If you need tools like:
tools: { getWeather, createDocument, updateDocument, requestSuggestions }You can:
- Either send them to Python via a JSON RPC-like request.
- Or handle on the JS side, and Python just responds with instructions (
callTool), which JS then executes.
🧪 Test Stream Compatibility
Ensure: ✅ Your Python returns a stream (
StreamingResponse) ✅ JS reads fromresponse.bodyand pushes todataStream.append(...)
✅ AllonFinish,telemetryand message saving logic remains intact
Final Thoughts
- ✅ You don't need to change the rest of the app
- ✅ You abstract just
streamText()but simulate same behavior - ✅ You can incrementally implement tools in Python or JS depending on complexity
- ✅ Don’t forget to handle
onFinish(message saving) in thedataStream.done()block
If you want a copy-paste ready version of the full updated
executefunction with Python integration, let me know the tools your model uses (or if you want to skip tool calling for now). - The Python backend mimics the behavior of
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.