Skip to content
TheChatArchiveThe Chat Archive
o1-preview-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

The code: import streamlit as st from gradioclient import Client import os Set page…

  1. User

    The code:

    import streamlit as st from gradio_client import Client import os

    Set page configuration to 'wide' and restore default menu and footer st.set_page_config( page_title="LLM Chat Application", layout="wide", # Sets the interface to wide mode initial_sidebar_state="auto" )

    Constants HISTORY_FILE = 'history.txt' MODEL_CLIENT_NAME = 'yuntian-deng/ChatGPT4' # Replace with your actual model identifier

    Function to load history from the file def load_history(): if os.path.exists(HISTORY_FILE): with open(HISTORY_FILE, 'r', encoding='utf-8') as f: return f.read() return ""

    Function to save history to the file def save_history(history): with open(HISTORY_FILE, 'w', encoding='utf-8') as f: f.write(history)

    Initialize session state for history if 'history' not in st.session_state: st.session_state.history = load_history()

    Optionally, you can hide the footer if desired, but we'll keep the settings wheel visible If you wish to hide the footer, uncomment the following CSS hide_footer = """ """ st.markdown(hide_footer, unsafe_allow_html=True) Create a form for input with st.form("chat_form", clear_on_submit=False):

    Editable text area for chat history and input

    history_input = st.text_area( "Chat History & Input", value=st.session_state.history, height=400, key='history_input', placeholder="Start chatting...", )

    Generate Response button

    generate = st.form_submit_button("Generate Response")

    Sliders for Temperature and Top-P sampling

    col1, col2 = st.columns(2) with col1: temperature = st.slider( "Temperature", min_value=0.0, max_value=5.0, value=1.0, step=0.1, help="Controls the randomness of the model's output. Higher values like 1.0 make the output more random, while lower values like 0.2 make it more focused and deterministic." ) with col2: top_p = st.slider( "Top-P Sampling", min_value=0.0, max_value=1.0, value=1.0, step=0.05, help="Nucleus sampling: the model considers the smallest set of tokens with a cumulative probability above this value." ) Handle manual edits to the chat history if history_input != st.session_state.history: st.session_state.history = history_input save_history(history_input)

    Handle form submission if generate: try:

    Initialize the client

    client = Client(MODEL_CLIENT_NAME)

    # Prepare inputs - assuming the model expects the full history as a single string
    prompt = st.session_state.history
    
    # Call the model to get a response
    result = client.predict(
        inputs=prompt,
        top_p=top_p,
        temperature=temperature,
        chat_counter=0,
        chatbot=[],
        api_name='/predict'
    )
    # Adjust based on actual API response structure
    output = result[0][0][1].strip()
    
    # Append model response to history
    # Ensure there's a newline before appending
    if not st.session_state.history.endswith('\n'):
        st.session_state.history += f"\n{output}"
    else:
        st.session_state.history += f"{output}"
    
    # Save the updated history to the file
    save_history(st.session_state.history)
    
    # Update the text area to reflect the new history
    # This ensures that the UI displays the latest conversation
    st.session_state.history_input = st.session_state.history
    
    st.success("LLM response generated and appended to history.")
    

    except Exception as e: st.error(f"Error communicating with the model: {e}") Yields the error:

    Error communicating with the model: st.session_state.history_input cannot be modified after the widget with key history_input is instantiated. I want you to add a new "send message" box, and refresh (or recreate) the history box after every generation. In fact, you should reload dynamically the whole page every time, storing the top-p and temperature in a config file.

  2. Assistant

    Certainly! Here's the updated code that implements your requirements:

    1. Added a new "send message" box: There's now a separate input box for sending messages.
    2. Refreshes the history box after every generation: The conversation history is refreshed and displayed after each message.
    3. Reloads dynamically the whole page every time: The page reloads using st.experimental_rerun() to ensure the history is updated.
    4. Stores the top-p and temperature in a config file: The sliders for top_p and temperature now read from and save to a config.json file.

    Here's the complete code:

    import streamlit as st
    from gradio_client import Client
    import os
    import json
    
    # Set page configuration
    st.set_page_config(
        page_title="LLM Chat Application",
        layout="wide",
        initial_sidebar_state="auto"
    )
    
    # Constants
    HISTORY_FILE = 'history.txt'  # This file stores the conversation history
    MODEL_CLIENT_NAME = 'yuntian-deng/ChatGPT4'  # Replace with your actual model identifier
    CONFIG_FILE = 'config.json'  # Config file to store top_p and temperature
    
    # Function to load history from the file
    def load_history():
        if os.path.exists(HISTORY_FILE):
            with open(HISTORY_FILE, 'r', encoding='utf-8') as f:
                return f.read()
        return ""
    
    # Function to save history to the file
    def save_history(history):
        with open(HISTORY_FILE, 'w', encoding='utf-8') as f:
            f.write(history)
    
    # Function to load config from the file
    def load_config():
        if os.path.exists(CONFIG_FILE):
            with open(CONFIG_FILE, 'r', encoding='utf-8') as f:
                return json.load(f)
        else:
            return {"top_p": 1.0, "temperature": 1.0}
    
    # Function to save config to the file
    def save_config(config):
        with open(CONFIG_FILE, 'w', encoding='utf-8') as f:
            json.dump(config, f)
    
    # Initialize session state for history
    if 'history' not in st.session_state:
        st.session_state.history = load_history()
    
    # Initialize session state for config
    if 'config' not in st.session_state:
        st.session_state.config = load_config()
    
    # Display the conversation history
    st.header("Conversation History")
    st.text_area("Conversation", value=st.session_state.history, height=400, key='history_display', disabled=True)
    
    # Create a form for new input
    with st.form("chat_form", clear_on_submit=True):
        user_input = st.text_input("Your Message", key='user_input', placeholder="Type your message here...")
    
        # Sliders for Temperature and Top-P sampling
        col1, col2 = st.columns(2)
        with col1:
            temperature = st.slider(
                "Temperature",
                min_value=0.0,
                max_value=5.0,
                value=st.session_state.config.get('temperature', 1.0),
                step=0.1,
                help="Controls the randomness of the model's output."
            )
        with col2:
            top_p = st.slider(
                "Top-P Sampling",
                min_value=0.0,
                max_value=1.0,
                value=st.session_state.config.get('top_p', 1.0),
                step=0.05,
                help="Nucleus sampling."
            )
    
        # Generate Response button
        generate = st.form_submit_button("Send Message")
    
        # Update the config in session_state and save to file when sliders change
        if temperature != st.session_state.config.get('temperature', 1.0) or top_p != st.session_state.config.get('top_p', 1.0):
            st.session_state.config['temperature'] = temperature
            st.session_state.config['top_p'] = top_p
            save_config(st.session_state.config)
    
    # Handle form submission
    if generate:
        if user_input:
            st.session_state.history += f"\nUser: {user_input}"
            save_history(st.session_state.history)
            try:
                # Initialize the client
                client = Client(MODEL_CLIENT_NAME)
    
                # Prepare the prompt
                prompt = st.session_state.history
    
                # Call the model to get a response
                result = client.predict(
                    inputs=prompt,
                    top_p=st.session_state.config['top_p'],
                    temperature=st.session_state.config['temperature'],
                    chat_counter=0,
                    chatbot=[],
                    api_name='/predict'
                )
                # Adjust based on actual API response structure
                # Assuming 'result' is a list of lists where result[0][0][1] is the response
                output = result[0][0][1].strip()
    
                # Append model response to history
                st.session_state.history += f"\nBot: {output}"
                # Save updated history
                save_history(st.session_state.history)
                # Re-render the page to show updated history
                st.experimental_rerun()
    
            except Exception as e:
                st.error(f"Error communicating with the model: {e}")
        else:
            st.warning("Please enter a message.")
    

    Explanation of Changes:

    • Separated Message Input: A new message input box (st.text_input) is added for the user to type their message.

    • Conversation History Display: The conversation history is displayed in a read-only text area using st.text_area with disabled=True.

    • Dynamic Page Reload: After each message is sent and the response is received, st.experimental_rerun() is called to reload the page and refresh the conversation history display.

    • Configuration File Handling:

      • Loading Config: The load_config() function reads top_p and temperature from config.json. If the file doesn't exist, default values are used.
      • Saving Config: When the sliders for top_p and temperature are changed, the new values are saved to config.json using the save_config() function.
    • Session State Management: The conversation history and configuration settings are stored in st.session_state to persist across user interactions.

    Notes:

    • Avoiding Widget Modification After Instantiation: The previous error was caused by attempting to modify a widget's value after it was created. In this updated code, we avoid modifying widget values after they are instantiated by using separate variables and managing state appropriately.

    • Model Response Handling: Ensure that the way you extract the model's response from result matches the actual structure returned by your model.

    • Error Handling: Added checks to prompt the user if they attempt to send an empty message and catch exceptions when communicating with the model.

    Config File (config.json) Structure:

    {
        "top_p": 1.0,
        "temperature": 1.0
    }
    

    This file will be updated automatically whenever you adjust the sliders for top_p and temperature.

    Usage:

    • Run the Streamlit App: Start your Streamlit app as usual using streamlit run your_script_name.py.

    • Interact with the App:

      • View Conversation: The conversation history will be displayed at the top.
      • Send a Message: Type your message in the "Your Message" input box and click "Send Message".
      • Adjust Parameters: Use the sliders to adjust temperature and top_p. These settings will be saved and persist across sessions.

    Reminder: Replace 'yuntian-deng/ChatGPT4' with the actual identifier for your model if different.