Hi. I want you to design and implement a streamlit GUI based LLM chatting app. The…
- User
Hi. I want you to design and implement a streamlit GUI based LLM chatting app. The features would be as follows:
- A context history, allowing for having a list of messages from the user and the LLM
- Automatically scrolling to the last message sent
- Message boxes should be automatically resized to the size of the content inside
- Messages from the user are aligned left while messages from the LLM are aligned right (or at least the box is aligned in such a way)
- Markdown formatted inputs and outputs per message, using
st.markdown - Parameters for controlling:
- The temperature
- Top-P sampling
- User and LLM prefix, like: "User: " and "Assistant: ", inserted before messages.
- Exporting the whole history to json format
- Loading the history from json format
Context information surrounding the API:
api_name: /predict copy from gradio_client import Client client = Client("yuntian-deng/ChatGPT4") result = client.predict( inputs="Hello!!", top_p=1, temperature=1, chat_counter=0, chatbot=[], api_name="/predict" ) print(result) Accepts 5 parameters: inputs str Required The input value that is provided in the "Type an input and press Enter" Textbox component. top_p float Default: 1 The input value that is provided in the "Top-p (nucleus sampling)" Slider component. temperature float Default: 1 The input value that is provided in the "Temperature" Slider component. chat_counter float Default: 0 The input value that is provided in the "parameter_16" Number component. chatbot Tuple[str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None, str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None] Default: [] The input value that is provided in the "parameter_4" Chatbot component. Returns tuple of 4 elements [0] Tuple[str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None, str | Dict(file: filepath, alt_text: str | None) | Dict(component: str, value: Any, constructor_args: Dict(), props: Dict()) | None] The output value that appears in the "value_4" Chatbot component. [1] float The output value that appears in the "value_16" Number component. [2] str The output value that appears in the "Status code from OpenAI server" Textbox component. [3] str The output value that appears in the "Type an input and press Enter" Textbox component. - Assistant
Here is the code for your Streamlit GUI-based LLM chatting app, incorporating all the features you've requested:
import streamlit as st from gradio_client import Client import json # Initialize session_state variables if 'chat_history' not in st.session_state: st.session_state['chat_history'] = [] if 'chat_counter' not in st.session_state: st.session_state['chat_counter'] = 0 if 'user_prefix' not in st.session_state: st.session_state['user_prefix'] = "User: " if 'assistant_prefix' not in st.session_state: st.session_state['assistant_prefix'] = "Assistant: " if 'temperature' not in st.session_state: st.session_state['temperature'] = 1.0 if 'top_p' not in st.session_state: st.session_state['top_p'] = 1.0 # Sidebar for parameters with st.sidebar: st.header("Parameters") st.session_state['temperature'] = st.slider('Temperature', 0.0, 1.0, st.session_state['temperature']) st.session_state['top_p'] = st.slider('Top-p (nucleus sampling)', 0.0, 1.0, st.session_state['top_p']) st.session_state['user_prefix'] = st.text_input('User prefix', st.session_state['user_prefix']) st.session_state['assistant_prefix'] = st.text_input('Assistant prefix', st.session_state['assistant_prefix']) st.write("## Export / Import Chat History") # Export chat history json_data = json.dumps(st.session_state['chat_history'], indent=4) st.download_button(label='Download Chat History', data=json_data, file_name='chat_history.json', mime='application/json') # Upload chat history uploaded_file = st.file_uploader("Upload Chat History (JSON)", type='json') if uploaded_file is not None: file_content = uploaded_file.read() st.session_state['chat_history'] = json.loads(file_content) st.experimental_rerun() # Function to process user input def process_input(): user_input = st.session_state['input'] if user_input: # Append user's message to chat history user_message = {'role': 'user', 'content': st.session_state['user_prefix'] + user_input} st.session_state['chat_history'].append(user_message) # Prepare chatbot history for the API # The 'chatbot' parameter expects a list of tuples (user message, assistant message) # We'll rebuild this from the session chat history chatbot_history = [] for i in range(0, len(st.session_state['chat_history']), 2): user_msg = st.session_state['chat_history'][i]['content'] if i < len(st.session_state['chat_history']) else '' assistant_msg = st.session_state['chat_history'][i+1]['content'] if i+1 < len(st.session_state['chat_history']) else '' chatbot_history.append((user_msg, assistant_msg)) # Use Gradio Client to send input and get response client = Client("yuntian-deng/ChatGPT4") response = client.predict( inputs=user_input, top_p=st.session_state['top_p'], temperature=st.session_state['temperature'], chat_counter=st.session_state['chat_counter'], chatbot=chatbot_history, api_name="/predict" ) # Extract assistant's response assistant_response = response[0][-1][1] # Get the last assistant message from the chatbot output # Append assistant's message to chat history assistant_message = {'role': 'assistant', 'content': st.session_state['assistant_prefix'] + assistant_response} st.session_state['chat_history'].append(assistant_message) # Update chat counter st.session_state['chat_counter'] += 1 # Clear input box st.session_state['input'] = '' # Scroll to the bottom # st.experimental_rerun() # Main interface st.title("LLM Chatbot") # Container to hold chat history message_container = st.container() # Display chat messages with message_container: for chat in st.session_state['chat_history']: if chat['role'] == 'user': st.markdown( f"<div style='text-align: left; background-color:#DCF8C6; padding:10px; border-radius:10px; margin:5px 0; max-width:70%; word-wrap:break-word;'>{chat['content']}</div>", unsafe_allow_html=True) else: st.markdown( f"<div style='text-align: right; background-color:#E0E0E0; padding:10px; border-radius:10px; margin:5px 0 5px auto; max-width:70%; word-wrap:break-word;'>{chat['content']}</div>", unsafe_allow_html=True) # Text input for user message st.text_input("You:", key='input', on_change=process_input) # Auto-scroll to the bottom st.markdown(""" <script> var chat_box = parent.document.getElementsByClassName('main')[0]; chat_box.scrollTop = chat_box.scrollHeight; </script> """, unsafe_allow_html=True)Explanation of Features:
-
Context History: The chat history is stored in
st.session_state['chat_history']and displayed in the main interface. -
Automatically Scrolling to the Last Message: JavaScript code is injected at the bottom of the page to auto-scroll to the latest message.
-
Message Boxes Resizing: CSS styles are applied to message containers to ensure they resize based on content length.
-
Message Alignment: User messages are left-aligned, and assistant messages are right-aligned using CSS in the
st.markdowncalls. -
Markdown Formatting: Messages are displayed using
st.markdown, which supports Markdown formatting. -
Parameter Controls: Temperature and Top-p sampling sliders, as well as User and Assistant prefixes, are available in the sidebar.
-
Exporting and Loading History: The sidebar includes a download button for exporting chat history to JSON and a file uploader for loading history from a JSON file.
Instructions to Run the App:
-
Install Dependencies:
- Ensure you have Streamlit and Gradio Client installed:
pip install streamlit gradio_client
- Ensure you have Streamlit and Gradio Client installed:
-
Save the Script:
- Save the code above to a file named
app.py.
- Save the code above to a file named
-
Run the App:
- Run the Streamlit app using the command:
streamlit run app.py
- Run the Streamlit app using the command:
-
Interact with the App:
- Open the provided local URL in your browser.
- Use the text input at the bottom to send messages.
- Adjust parameters and prefixes in the sidebar as needed.
- Export and import chat histories using the options in the sidebar.
Notes:
-
Gradio Client: The app uses the Gradio Client to communicate with the LLM model hosted at
yuntian-deng/ChatGPT4. -
Session State Management:
st.session_stateis utilized to maintain the chat history and parameters across user interactions. -
Auto-Scroll Limitation: Auto-scrolling via JavaScript may have limitations due to Streamlit's rendering mechanisms. If it doesn't work as expected, you might need to scroll manually.
-
Customization: Feel free to modify the CSS styles in the
st.markdowncalls to adjust the appearance of the chat messages.
@assistant
-