Skip to content
TheChatArchiveThe Chat Archive
chatgpt-4o-latest-20250326· OpenAIHugging Face LMArena Human Preference 140K

const stream = createDataStream({ execute: (dataStream) => { const result = streamText({…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Amazon Novaamazon.nova-pro-v1:0’s answer.
  1. User

    const stream = createDataStream({

    execute: (dataStream) => {

    const result = streamText({

    model: myProvider.languageModel(selectedChatModel),

    system: systemPrompt({ selectedChatModel, requestHints }),

    messages,

    maxSteps: 5,

    experimental_activeTools:

    selectedChatModel === 'chat-model-reasoning'

    ? []

    : [

    'getWeather',

    'createDocument',

    'updateDocument',

    'requestSuggestions',

    ],

    experimental_transform: smoothStream({ chunking: 'word' }),

    experimental_generateMessageId: generateUUID,

    tools: {

    getWeather,

    createDocument: createDocument({ session, dataStream }),

    updateDocument: updateDocument({ session, dataStream }),

    requestSuggestions: requestSuggestions({

    session,

    dataStream,

    }),

    },

    onFinish: async ({ response }) => {

    if (session.user?.id) {

    try {

    const assistantId = getTrailingMessageId({

    messages: response.messages.filter(

    (message) => message.role === 'assistant',

    ),

    });

    if (!assistantId) {

    throw new Error('No assistant message found!');

    }

    const [, assistantMessage] = appendResponseMessages({

    messages: [message],

    responseMessages: response.messages,

    });

    await saveMessages({

    messages: [

    {

    id: assistantId,

    chatId: id,

    role: assistantMessage.role,

    parts: assistantMessage.parts,

    attachments:

    assistantMessage.experimental_attachments ?? [],

    createdAt: new Date(),

    },

    ],

    });

    } catch (_) {

    console.error('Failed to save chat');

    }

    }

    },

    experimental_telemetry: {

    isEnabled: isProductionEnvironment,

    functionId: 'stream-text',

    },

    });

    result.consumeStream();

    result.mergeIntoDataStream(dataStream, {

    sendReasoning: true,

    });

    },

    onError: () => {

    return 'Oops, an error occurred!';

    },

    });

    const streamContext = getStreamContext();

    if (streamContext) {

    return new Response(

    await streamContext.resumableStream(streamId, () => stream),

    );

    } else {

    return new Response(stream);

    }

    } catch (error) {

    if (error instanceof ChatSDKError) {

    return error.toResponse();

    }

    }

    }

    I want to replace the function above by a call for python backend, but wihout changing the behavior of the application. this is a repo provided as is from vercel ai, ai chatbot sdk

  2. Assistant

    To integrate a Python backend (e.g., FastAPI, Flask, etc.) and replace the streamText function call in this Vercel AI Chat SDK-style JavaScript code — without breaking the app's behavior or structure — you need to ensure that:

    1. The Python backend mimics the behavior of streamText()
    2. Streaming remains intact (i.e., server-sent events / response chunking)
    3. Tool handling and responses continue working normally
    4. Telemetry and ID generation still happen as needed
    5. You abstract the execution using fetch or other stream-capable client call

    ✅ Plan Overview

    Your goals:

    • Replace this line:
      const result = streamText({ ... });
      
      → with:
      const result = await callPythonBackendWithStreaming({ ... });
      

    🧠 Step-by-Step Breakdown

    1. Setup Python backend (FastAPI example)

    Here’s an example of Python endpoint that streams output word-by-word like streamText({ experimental_transform: smoothStream({ chunking: 'word' }) }).

    # app.py (FastAPI)
    from fastapi import FastAPI
    from fastapi.responses import StreamingResponse
    from pydantic import BaseModel
    import asyncio
    
    app = FastAPI()
    
    class Message(BaseModel):
        role: str
        content: str
    
    class ChatRequest(BaseModel):
        model: str
        messages: list[Message]
        system_prompt: str
    
    def fake_llm_response(messages):
        response = "This is a simulated LLM response streamed word by word."
        for word in response.split():
            yield word
            yield " "
            asyncio.sleep(0.1)
    
    @app.post("/chat-stream")
    async def chat_stream(data: ChatRequest):
        async def word_stream():
            async for word in fake_llm_response(data.messages):
                yield word
        return StreamingResponse(word_stream(), media_type="text/plain")
    

    2. From your JS code, replace streamText(...) with fetch + stream logic

    import { createDataStream } from 'ai';
    import { streamToAsyncIterable } from './utils/streamToAsyncIterable';
    
    const stream = createDataStream({
      execute: async (dataStream) => {
        const controller = new AbortController();
        const response = await fetch('http://<your-python-api>/chat-stream', {
          method: 'POST',
          body: JSON.stringify({
            model: selectedChatModel,
            system_prompt: systemPrompt({ selectedChatModel, requestHints }),
            messages,
          }),
          headers: {
            'Content-Type': 'application/json',
          },
          signal: controller.signal,
        });
    
        if (!response.ok || !response.body) {
          throw new Error('Failed to connect to Python backend');
        }
    
        // transform ReadableStream to AsyncIterable
        const streamIterator = streamToAsyncIterable(response.body);
    
        for await (const chunk of streamIterator) {
          // Send chunk to UI
          dataStream.append({ type: 'text-delta', value: chunk });
        }
    
        dataStream.done();
    
        // Perform onFinish logic
        // Here you'd handle assistant message, save messages, telemetry, etc.
      },
      onError: () => 'Oops, an error occurred!',
    });
    
    

    3. streamToAsyncIterable.js helper

    export async function* streamToAsyncIterable(readableStream) {
      const reader = readableStream.getReader();
      const decoder = new TextDecoder();
    
      while (true) {
        const { value, done } = await reader.read();
        if (done) break;
        yield decoder.decode(value);
      }
    }
    

    🔁 Emulating Tooling (Optional)

    If you need tools like:

    tools: {
      getWeather,
      createDocument,
      updateDocument,
      requestSuggestions
    }
    

    You can:

    1. Either send them to Python via a JSON RPC-like request.
    2. Or handle on the JS side, and Python just responds with instructions (callTool), which JS then executes.

    🧪 Test Stream Compatibility

    Ensure: ✅ Your Python returns a stream (StreamingResponse) ✅ JS reads from response.body and pushes to dataStream.append(...)
    ✅ All onFinish, telemetry and message saving logic remains intact


    Final Thoughts

    • ✅ You don't need to change the rest of the app
    • ✅ You abstract just streamText() but simulate same behavior
    • ✅ You can incrementally implement tools in Python or JS depending on complexity
    • ✅ Don’t forget to handle onFinish (message saving) in the dataStream.done() block

    If you want a copy-paste ready version of the full updated execute function with Python integration, let me know the tools your model uses (or if you want to skip tool calling for now).

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.