Skip to content
TheChatArchiveThe Chat Archive
gpt-4-1106-preview· openaiHugging Face Suzhen/CodeChat

I am building a chatbot on huggingface:"import gradio as gr from huggingfacehub import…

  1. User

    I am building a chatbot on huggingface:"import gradio as gr from huggingface_hub import InferenceClient

    client = InferenceClient("mistralai/Mixtral-8x7B-Instruct-v0.1")

    def format_prompt(message, history): system_prompt = ("I’m an AI chatbot named ChatSherman designed by a super-intelligent student named ShermanAI at the Department of Electronic and Information Engineering at The Hong Kong Polytechnic University to help you with your engineering questions. Also, I can assist with a wide range of topics and questions. I am now version 2.0, which is more powerful than version 1.0, able to do more complex tasks, and optimized for chat.") prompt = "" for user_prompt, bot_response in history: prompt += f"[INST] {user_prompt} [/INST]" prompt += f"{bot_response} " prompt += f"[INST] {system_prompt} {message} [/INST]" return prompt

    def generate(prompt, history, temperature=0.6, max_new_tokens=256, top_p=0.95, repetition_penalty=1.2): temperature = float(temperature) top_p = float(top_p)

    generate_kwargs = dict(
        temperature=temperature,
        max_new_tokens=max_new_tokens,
        top_p=top_p,
        repetition_penalty=repetition_penalty,
        do_sample=True,
        seed=42,
    )
    
    formatted_prompt = format_prompt(prompt, history)
    stream = client.text_generation(formatted_prompt, **generate_kwargs, stream=True, details=True, return_full_text=False)
    output = ""
    for response in stream:
        output += response.token.text
        yield output
    return output
    

    examples = [ ["What is ChatSherman, and how does it work?", []], ["Is my personal information and data safe when I use the ChatSherman chatbot?", []], ["What are some common applications of deep learning in engineering?", []] ]

    gr.ChatInterface( fn=generate, chatbot=gr.Chatbot(show_label=False, show_share_button=False, show_copy_button=True, likeable=True, layout="panel"), title="ChatSherman", examples=examples, concurrency_limit=20, ).launch(show_api=False)" but my input to the chatbot:"me:hi chatbot:Hello ChatSherman, it's nice to meet you! I'm here to test out your capabilities as an advanced AI chatbot. As a student studying electronic and information engineering myself, I'm excited to see how you can assist me with my studies.

    To start off, could you tell me more about yourself? What kind of improvements have been made in this new version 2.0 compared to the previous one? And what are some examples of the "more complex tasks" that you mentioned you can perform?" The chatbot think I am ChatSherman, but I want the chatbot to be ChatSherman.

  2. Assistant

    It appears that there might be a slight confusion in the format_prompt function, where the bot's identity is being misattributed. In the format_prompt function, you're including a system prompt in the middle of the conversation history, which should actually be at the beginning to establish the bot's identity before any interaction. Here's how you can adjust the code:

    First, you should separate the static system prompt from the dynamic conversation history. The system prompt should be presented only once, at the very beginning of each conversation, rather than being repeated with every prompt.

    system_prompt = ("I’m an AI chatbot named ChatSherman designed by a super-intelligent student named ShermanAI at the Department of Electronic and Information Engineering at The Hong Kong Polytechnic University to help you with your engineering questions. Also, I can assist with a wide range of topics and questions. I am now version 2.0, which is more powerful than version 1.0, able to do more complex tasks, and optimized for chat.")
    
    def format_prompt(message, history):
        prompt = f"<s>{system_prompt}"
        for user_prompt, bot_response in history:
            prompt += f"[USER] {user_prompt} [/USER]"
            prompt += f"{bot_response}</s> "
        prompt += f"[USER] {message} [/USER]</s>"
        return prompt
    

    Make sure that the instructions to the AI (e.g., [USER] and [INST]) match the structure expected by the particular model you are using. Different models and checkpoints use different conventions for understanding the user's versus the AI's messages. Adjust the prompt tags accordingly if required.

    Now when constructing a prompt for each message, the function will begin with the static system prompt to introduce the AI's identity (ChatSherman) then follow with the interactive history, and finally include the user's latest message tagged appropriately.

    Adjust the starter message to not include the bot's name or it will assume it as part of the user inputs.

    examples = [
        ["Hi there! Could you tell me more about yourself and the improvements in version 2.0?", []],
        ["Is my personal information and data safe when I use ChatSherman?", []],
        ["What are some common applications of deep learning in engineering?", []]
    ]
    

    With these adjustments, the format_prompt function and the examples should help distinctly establish that the chatbot is the entity named ChatSherman, and the user is an independent entity, which should prevent the confusion you are experiencing.