Skip to content
TheChatArchiveThe Chat Archive
o1-preview-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

Hi. I want you to design and implement a streamlit GUI based LLM chatting app. The…

  1. User

    Hi. I want you to design and implement a streamlit GUI based LLM chatting app. The features would be as follows:

    • A context history, allowing for having a list of messages from the user and the LLM
    • Automatically scrolling to the last message sent
    • Message boxes should be automatically resized to the size of the content inside
    • Messages from the user are aligned left while messages from the LLM are aligned right (or at least the box is aligned in such a way)
    • Markdown formatted inputs and outputs per message, using st.markdown
    • Parameters for controlling:
      • The temperature
      • Top-P sampling
      • User and LLM prefix, like: "User: " and "Assistant: ", inserted before messages.
    • Exporting the whole history to json format
    • Loading the history from json format

    Context information surrounding the API:

    api_name: /predict
    copy
    from gradio_client import Client
    
    client = Client("yuntian-deng/ChatGPT4")
    result = client.predict(
    		inputs="Hello!!",
    		top_p=1,
    		temperature=1,
    		chat_counter=0,
    		chatbot=[],
    		api_name="/predict"
    )
    print(result)
    Accepts 5 parameters:
    inputs str Required
    
    The input value that is provided in the "Type an input and press Enter" Textbox component.
    
    top_p float Default: 1
    
    The input value that is provided in the "Top-p (nucleus sampling)" Slider component.
    
    temperature float Default: 1
    
    The input value that is provided in the "Temperature" Slider component.
    
    chat_counter float Default: 0
    
    The input value that is provided in the "parameter_16" Number component.
    
    chatbot Tuple[str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None, str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None] Default: []
    
    The input value that is provided in the "parameter_4" Chatbot component.
    
    Returns tuple of 4 elements
    [0] Tuple[str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None, str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None]
    
    The output value that appears in the "value_4" Chatbot component.
    
    [1] float
    
    The output value that appears in the "value_16" Number component.
    
    [2] str
    
    The output value that appears in the "Status code from OpenAI server" Textbox component.
    
    [3] str
    
    The output value that appears in the "Type an input and press Enter" Textbox component.
    
  2. Assistant

    Here is the code for your Streamlit GUI-based LLM chatting app, incorporating all the features you've requested:

    import streamlit as st
    from gradio_client import Client
    import json
    
    # Initialize session_state variables
    if 'chat_history' not in st.session_state:
        st.session_state['chat_history'] = []
    
    if 'chat_counter' not in st.session_state:
        st.session_state['chat_counter'] = 0
    
    if 'user_prefix' not in st.session_state:
        st.session_state['user_prefix'] = "User: "
    
    if 'assistant_prefix' not in st.session_state:
        st.session_state['assistant_prefix'] = "Assistant: "
    
    if 'temperature' not in st.session_state:
        st.session_state['temperature'] = 1.0
    
    if 'top_p' not in st.session_state:
        st.session_state['top_p'] = 1.0
    
    # Sidebar for parameters
    with st.sidebar:
        st.header("Parameters")
        st.session_state['temperature'] = st.slider('Temperature', 0.0, 1.0, st.session_state['temperature'])
        st.session_state['top_p'] = st.slider('Top-p (nucleus sampling)', 0.0, 1.0, st.session_state['top_p'])
        st.session_state['user_prefix'] = st.text_input('User prefix', st.session_state['user_prefix'])
        st.session_state['assistant_prefix'] = st.text_input('Assistant prefix', st.session_state['assistant_prefix'])
        st.write("## Export / Import Chat History")
        # Export chat history
        json_data = json.dumps(st.session_state['chat_history'], indent=4)
        st.download_button(label='Download Chat History', data=json_data, file_name='chat_history.json', mime='application/json')
    
        # Upload chat history
        uploaded_file = st.file_uploader("Upload Chat History (JSON)", type='json')
        if uploaded_file is not None:
            file_content = uploaded_file.read()
            st.session_state['chat_history'] = json.loads(file_content)
            st.experimental_rerun()
    
    # Function to process user input
    def process_input():
        user_input = st.session_state['input']
        if user_input:
            # Append user's message to chat history
            user_message = {'role': 'user', 'content': st.session_state['user_prefix'] + user_input}
            st.session_state['chat_history'].append(user_message)
    
            # Prepare chatbot history for the API
            # The 'chatbot' parameter expects a list of tuples (user message, assistant message)
            # We'll rebuild this from the session chat history
            chatbot_history = []
            for i in range(0, len(st.session_state['chat_history']), 2):
                user_msg = st.session_state['chat_history'][i]['content'] if i < len(st.session_state['chat_history']) else ''
                assistant_msg = st.session_state['chat_history'][i+1]['content'] if i+1 < len(st.session_state['chat_history']) else ''
                chatbot_history.append((user_msg, assistant_msg))
    
            # Use Gradio Client to send input and get response
            client = Client("yuntian-deng/ChatGPT4")
            response = client.predict(
                inputs=user_input,
                top_p=st.session_state['top_p'],
                temperature=st.session_state['temperature'],
                chat_counter=st.session_state['chat_counter'],
                chatbot=chatbot_history,
                api_name="/predict"
            )
    
            # Extract assistant's response
            assistant_response = response[0][-1][1]  # Get the last assistant message from the chatbot output
    
            # Append assistant's message to chat history
            assistant_message = {'role': 'assistant', 'content': st.session_state['assistant_prefix'] + assistant_response}
            st.session_state['chat_history'].append(assistant_message)
            # Update chat counter
            st.session_state['chat_counter'] += 1
            # Clear input box
            st.session_state['input'] = ''
            # Scroll to the bottom
            # st.experimental_rerun()
    
    # Main interface
    st.title("LLM Chatbot")
    
    # Container to hold chat history
    message_container = st.container()
    
    # Display chat messages
    with message_container:
        for chat in st.session_state['chat_history']:
            if chat['role'] == 'user':
                st.markdown(
                    f"<div style='text-align: left; background-color:#DCF8C6; padding:10px; border-radius:10px; margin:5px 0; max-width:70%; word-wrap:break-word;'>{chat['content']}</div>",
                    unsafe_allow_html=True)
            else:
                st.markdown(
                    f"<div style='text-align: right; background-color:#E0E0E0; padding:10px; border-radius:10px; margin:5px 0 5px auto; max-width:70%; word-wrap:break-word;'>{chat['content']}</div>",
                    unsafe_allow_html=True)
    
    # Text input for user message
    st.text_input("You:", key='input', on_change=process_input)
    
    # Auto-scroll to the bottom
    st.markdown("""
        <script>
        var chat_box = parent.document.getElementsByClassName('main')[0];
        chat_box.scrollTop = chat_box.scrollHeight;
        </script>
        """, unsafe_allow_html=True)
    

    Explanation of Features:

    • Context History: The chat history is stored in st.session_state['chat_history'] and displayed in the main interface.

    • Automatically Scrolling to the Last Message: JavaScript code is injected at the bottom of the page to auto-scroll to the latest message.

    • Message Boxes Resizing: CSS styles are applied to message containers to ensure they resize based on content length.

    • Message Alignment: User messages are left-aligned, and assistant messages are right-aligned using CSS in the st.markdown calls.

    • Markdown Formatting: Messages are displayed using st.markdown, which supports Markdown formatting.

    • Parameter Controls: Temperature and Top-p sampling sliders, as well as User and Assistant prefixes, are available in the sidebar.

    • Exporting and Loading History: The sidebar includes a download button for exporting chat history to JSON and a file uploader for loading history from a JSON file.

    Instructions to Run the App:

    1. Install Dependencies:

      • Ensure you have Streamlit and Gradio Client installed:
        pip install streamlit gradio_client
        
    2. Save the Script:

      • Save the code above to a file named app.py.
    3. Run the App:

      • Run the Streamlit app using the command:
        streamlit run app.py
        
    4. Interact with the App:

      • Open the provided local URL in your browser.
      • Use the text input at the bottom to send messages.
      • Adjust parameters and prefixes in the sidebar as needed.
      • Export and import chat histories using the options in the sidebar.

    Notes:

    • Gradio Client: The app uses the Gradio Client to communicate with the LLM model hosted at yuntian-deng/ChatGPT4.

    • Session State Management: st.session_state is utilized to maintain the chat history and parameters across user interactions.

    • Auto-Scroll Limitation: Auto-scrolling via JavaScript may have limitations due to Streamlit's rendering mechanisms. If it doesn't work as expected, you might need to scroll manually.

    • Customization: Feel free to modify the CSS styles in the st.markdown calls to adjust the appearance of the chat messages.

    @assistant