The code: import streamlit as st from gradioclient import Client import os Set page…
- User
The code:
import streamlit as st from gradio_client import Client import os
Set page configuration to 'wide' and restore default menu and footer st.set_page_config( page_title="LLM Chat Application", layout="wide", # Sets the interface to wide mode initial_sidebar_state="auto" )
Constants HISTORY_FILE = 'history.txt' MODEL_CLIENT_NAME = 'yuntian-deng/ChatGPT4' # Replace with your actual model identifier
Function to load history from the file def load_history(): if os.path.exists(HISTORY_FILE): with open(HISTORY_FILE, 'r', encoding='utf-8') as f: return f.read() return ""
Function to save history to the file def save_history(history): with open(HISTORY_FILE, 'w', encoding='utf-8') as f: f.write(history)
Initialize session state for history if 'history' not in st.session_state: st.session_state.history = load_history()
Optionally, you can hide the footer if desired, but we'll keep the settings wheel visible If you wish to hide the footer, uncomment the following CSS hide_footer = """ """ st.markdown(hide_footer, unsafe_allow_html=True) Create a form for input with st.form("chat_form", clear_on_submit=False):
Editable text area for chat history and input
history_input = st.text_area( "Chat History & Input", value=st.session_state.history, height=400, key='history_input', placeholder="Start chatting...", )
Generate Response button
generate = st.form_submit_button("Generate Response")
Sliders for Temperature and Top-P sampling
col1, col2 = st.columns(2) with col1: temperature = st.slider( "Temperature", min_value=0.0, max_value=5.0, value=1.0, step=0.1, help="Controls the randomness of the model's output. Higher values like 1.0 make the output more random, while lower values like 0.2 make it more focused and deterministic." ) with col2: top_p = st.slider( "Top-P Sampling", min_value=0.0, max_value=1.0, value=1.0, step=0.05, help="Nucleus sampling: the model considers the smallest set of tokens with a cumulative probability above this value." ) Handle manual edits to the chat history if history_input != st.session_state.history: st.session_state.history = history_input save_history(history_input)
Handle form submission if generate: try:
Initialize the client
client = Client(MODEL_CLIENT_NAME)
# Prepare inputs - assuming the model expects the full history as a single string prompt = st.session_state.history # Call the model to get a response result = client.predict( inputs=prompt, top_p=top_p, temperature=temperature, chat_counter=0, chatbot=[], api_name='/predict' ) # Adjust based on actual API response structure output = result[0][0][1].strip() # Append model response to history # Ensure there's a newline before appending if not st.session_state.history.endswith('\n'): st.session_state.history += f"\n{output}" else: st.session_state.history += f"{output}" # Save the updated history to the file save_history(st.session_state.history) # Update the text area to reflect the new history # This ensures that the UI displays the latest conversation st.session_state.history_input = st.session_state.history st.success("LLM response generated and appended to history.")except Exception as e: st.error(f"Error communicating with the model: {e}") Yields the error:
Error communicating with the model: st.session_state.history_input cannot be modified after the widget with key history_input is instantiated. I want you to add a new "send message" box, and refresh (or recreate) the history box after every generation. In fact, you should reload dynamically the whole page every time, storing the top-p and temperature in a config file.
- Assistant
Certainly! Here's the updated code that implements your requirements:
- Added a new "send message" box: There's now a separate input box for sending messages.
- Refreshes the history box after every generation: The conversation history is refreshed and displayed after each message.
- Reloads dynamically the whole page every time: The page reloads using
st.experimental_rerun()to ensure the history is updated. - Stores the top-p and temperature in a config file: The sliders for
top_pandtemperaturenow read from and save to aconfig.jsonfile.
Here's the complete code:
import streamlit as st from gradio_client import Client import os import json # Set page configuration st.set_page_config( page_title="LLM Chat Application", layout="wide", initial_sidebar_state="auto" ) # Constants HISTORY_FILE = 'history.txt' # This file stores the conversation history MODEL_CLIENT_NAME = 'yuntian-deng/ChatGPT4' # Replace with your actual model identifier CONFIG_FILE = 'config.json' # Config file to store top_p and temperature # Function to load history from the file def load_history(): if os.path.exists(HISTORY_FILE): with open(HISTORY_FILE, 'r', encoding='utf-8') as f: return f.read() return "" # Function to save history to the file def save_history(history): with open(HISTORY_FILE, 'w', encoding='utf-8') as f: f.write(history) # Function to load config from the file def load_config(): if os.path.exists(CONFIG_FILE): with open(CONFIG_FILE, 'r', encoding='utf-8') as f: return json.load(f) else: return {"top_p": 1.0, "temperature": 1.0} # Function to save config to the file def save_config(config): with open(CONFIG_FILE, 'w', encoding='utf-8') as f: json.dump(config, f) # Initialize session state for history if 'history' not in st.session_state: st.session_state.history = load_history() # Initialize session state for config if 'config' not in st.session_state: st.session_state.config = load_config() # Display the conversation history st.header("Conversation History") st.text_area("Conversation", value=st.session_state.history, height=400, key='history_display', disabled=True) # Create a form for new input with st.form("chat_form", clear_on_submit=True): user_input = st.text_input("Your Message", key='user_input', placeholder="Type your message here...") # Sliders for Temperature and Top-P sampling col1, col2 = st.columns(2) with col1: temperature = st.slider( "Temperature", min_value=0.0, max_value=5.0, value=st.session_state.config.get('temperature', 1.0), step=0.1, help="Controls the randomness of the model's output." ) with col2: top_p = st.slider( "Top-P Sampling", min_value=0.0, max_value=1.0, value=st.session_state.config.get('top_p', 1.0), step=0.05, help="Nucleus sampling." ) # Generate Response button generate = st.form_submit_button("Send Message") # Update the config in session_state and save to file when sliders change if temperature != st.session_state.config.get('temperature', 1.0) or top_p != st.session_state.config.get('top_p', 1.0): st.session_state.config['temperature'] = temperature st.session_state.config['top_p'] = top_p save_config(st.session_state.config) # Handle form submission if generate: if user_input: st.session_state.history += f"\nUser: {user_input}" save_history(st.session_state.history) try: # Initialize the client client = Client(MODEL_CLIENT_NAME) # Prepare the prompt prompt = st.session_state.history # Call the model to get a response result = client.predict( inputs=prompt, top_p=st.session_state.config['top_p'], temperature=st.session_state.config['temperature'], chat_counter=0, chatbot=[], api_name='/predict' ) # Adjust based on actual API response structure # Assuming 'result' is a list of lists where result[0][0][1] is the response output = result[0][0][1].strip() # Append model response to history st.session_state.history += f"\nBot: {output}" # Save updated history save_history(st.session_state.history) # Re-render the page to show updated history st.experimental_rerun() except Exception as e: st.error(f"Error communicating with the model: {e}") else: st.warning("Please enter a message.")Explanation of Changes:
-
Separated Message Input: A new message input box (
st.text_input) is added for the user to type their message. -
Conversation History Display: The conversation history is displayed in a read-only text area using
st.text_areawithdisabled=True. -
Dynamic Page Reload: After each message is sent and the response is received,
st.experimental_rerun()is called to reload the page and refresh the conversation history display. -
Configuration File Handling:
- Loading Config: The
load_config()function readstop_pandtemperaturefromconfig.json. If the file doesn't exist, default values are used. - Saving Config: When the sliders for
top_pandtemperatureare changed, the new values are saved toconfig.jsonusing thesave_config()function.
- Loading Config: The
-
Session State Management: The conversation history and configuration settings are stored in
st.session_stateto persist across user interactions.
Notes:
-
Avoiding Widget Modification After Instantiation: The previous error was caused by attempting to modify a widget's value after it was created. In this updated code, we avoid modifying widget values after they are instantiated by using separate variables and managing state appropriately.
-
Model Response Handling: Ensure that the way you extract the model's response from
resultmatches the actual structure returned by your model. -
Error Handling: Added checks to prompt the user if they attempt to send an empty message and catch exceptions when communicating with the model.
Config File (
config.json) Structure:{ "top_p": 1.0, "temperature": 1.0 }This file will be updated automatically whenever you adjust the sliders for
top_pandtemperature.Usage:
-
Run the Streamlit App: Start your Streamlit app as usual using
streamlit run your_script_name.py. -
Interact with the App:
- View Conversation: The conversation history will be displayed at the top.
- Send a Message: Type your message in the "Your Message" input box and click "Send Message".
- Adjust Parameters: Use the sliders to adjust
temperatureandtop_p. These settings will be saved and persist across sessions.
Reminder: Replace
'yuntian-deng/ChatGPT4'with the actual identifier for your model if different.