Skip to content
TheChatArchiveThe Chat Archive
Perplexitysonar-pro-high· PerplexityHugging Face LMArena Search Arena 24K

How to make ollama stream the response to the terminal if Im using python?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-4o-search-preview-high’s answer.
  1. User

    How to make ollama stream the response to the terminal if Im using python?

  2. Perplexity Assistant

    To stream responses from Ollama to the terminal using Python, you can enable the stream parameter in the ollama.chat function. This allows the function to return a Python generator, which yields parts of the response as they are generated. Here's how you can implement it:

    Example Code

    import ollama
    
    # Define the user message
    message = {'role': 'user', 'content': 'Why is the sky blue?'}
    
    # Use the chat function with streaming enabled
    for part in ollama.chat(model='llama2', messages=[message], stream=True):
        # Print each part of the response as it streams
        print(part['message']['content'], end='', flush=True)
    

    Key Points

    1. Enable Streaming: Set stream=True in the ollama.chat function. This modifies the function to return a generator.
    2. Iterate Over the Generator: Use a for loop to process each part of the response as it streams.
    3. Real-Time Output: Use print() with end='' and flush=True to display the response in real time without buffering.

    This approach ensures that the response is streamed token-by-token, providing a more interactive and responsive experience in the terminal[1][7].

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.