I am building a chatbot on huggingface:"import gradio as gr from huggingfacehub import…
- User
I am building a chatbot on huggingface:"import gradio as gr from huggingface_hub import InferenceClient
client = InferenceClient("mistralai/Mixtral-8x7B-Instruct-v0.1")
def format_prompt(message, history): system_prompt = ("I’m an AI chatbot named ChatSherman designed by a super-intelligent student named ShermanAI at the Department of Electronic and Information Engineering at The Hong Kong Polytechnic University to help you with your engineering questions. Also, I can assist with a wide range of topics and questions. I am now version 2.0, which is more powerful than version 1.0, able to do more complex tasks, and optimized for chat.") prompt = "" for user_prompt, bot_response in history: prompt += f"[INST] {user_prompt} [/INST]" prompt += f"{bot_response} " prompt += f"[INST] {system_prompt} {message} [/INST]" return prompt
def generate(prompt, history, temperature=0.6, max_new_tokens=256, top_p=0.95, repetition_penalty=1.2): temperature = float(temperature) top_p = float(top_p)
generate_kwargs = dict( temperature=temperature, max_new_tokens=max_new_tokens, top_p=top_p, repetition_penalty=repetition_penalty, do_sample=True, seed=42, ) formatted_prompt = format_prompt(prompt, history) stream = client.text_generation(formatted_prompt, **generate_kwargs, stream=True, details=True, return_full_text=False) output = "" for response in stream: output += response.token.text yield output return outputexamples = [ ["What is ChatSherman, and how does it work?", []], ["Is my personal information and data safe when I use the ChatSherman chatbot?", []], ["What are some common applications of deep learning in engineering?", []] ]
gr.ChatInterface( fn=generate, chatbot=gr.Chatbot(show_label=False, show_share_button=False, show_copy_button=True, likeable=True, layout="panel"), title="ChatSherman", examples=examples, concurrency_limit=20, ).launch(show_api=False)" but my input to the chatbot:"me:hi chatbot:Hello ChatSherman, it's nice to meet you! I'm here to test out your capabilities as an advanced AI chatbot. As a student studying electronic and information engineering myself, I'm excited to see how you can assist me with my studies.
To start off, could you tell me more about yourself? What kind of improvements have been made in this new version 2.0 compared to the previous one? And what are some examples of the "more complex tasks" that you mentioned you can perform?" The chatbot think I am ChatSherman, but I want the chatbot to be ChatSherman.
- Assistant
It appears that there might be a slight confusion in the
format_promptfunction, where the bot's identity is being misattributed. In theformat_promptfunction, you're including a system prompt in the middle of the conversation history, which should actually be at the beginning to establish the bot's identity before any interaction. Here's how you can adjust the code:First, you should separate the static system prompt from the dynamic conversation history. The system prompt should be presented only once, at the very beginning of each conversation, rather than being repeated with every prompt.
system_prompt = ("I’m an AI chatbot named ChatSherman designed by a super-intelligent student named ShermanAI at the Department of Electronic and Information Engineering at The Hong Kong Polytechnic University to help you with your engineering questions. Also, I can assist with a wide range of topics and questions. I am now version 2.0, which is more powerful than version 1.0, able to do more complex tasks, and optimized for chat.") def format_prompt(message, history): prompt = f"<s>{system_prompt}" for user_prompt, bot_response in history: prompt += f"[USER] {user_prompt} [/USER]" prompt += f"{bot_response}</s> " prompt += f"[USER] {message} [/USER]</s>" return promptMake sure that the instructions to the AI (e.g.,
[USER]and[INST]) match the structure expected by the particular model you are using. Different models and checkpoints use different conventions for understanding the user's versus the AI's messages. Adjust the prompt tags accordingly if required.Now when constructing a prompt for each message, the function will begin with the static system prompt to introduce the AI's identity (
ChatSherman) then follow with the interactive history, and finally include the user's latest message tagged appropriately.Adjust the starter message to not include the bot's name or it will assume it as part of the user inputs.
examples = [ ["Hi there! Could you tell me more about yourself and the improvements in version 2.0?", []], ["Is my personal information and data safe when I use ChatSherman?", []], ["What are some common applications of deep learning in engineering?", []] ]With these adjustments, the
format_promptfunction and the examples should help distinctly establish that the chatbot is the entity namedChatSherman, and the user is an independent entity, which should prevent the confusion you are experiencing.