Skip to content
TheChatArchiveThe Chat Archive
gpt-4-1106-preview· OpenAIAi2 WildChat-4.8M (AI2)

import gradio as gr import os import sys import json import requests MODEL = "gpt-4"…

  1. User

    import gradio as gr import os import sys import json import requests

    MODEL = "gpt-4" API_URL = os.getenv("API_URL") DISABLED = os.getenv("DISABLED") == 'True' OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") NUM_THREADS = int(os.getenv("NUM_THREADS", "1"))

    print (NUM_THREADS)

    def exception_handler(exception_type, exception, traceback): print("%s: %s" % (exception_type.name, exception)) sys.excepthook = exception_handler sys.tracebacklimit = 0

    #https://github.com/gradio-app/gradio/issues/3531#issuecomment-1484029099 def parse_codeblock(text): lines = text.split("\n") for i, line in enumerate(lines): if "" in line: if line != "": lines[i] = f'' else: lines[i] = '' else: if i > 0: lines[i] = "" + line.replace("<", "<").replace(">", ">") return "".join(lines)

    def predict(inputs, top_p, temperature, chat_counter, chatbot, history, request:gr.Request): payload = { "model": MODEL, "messages": [{"role": "user", "content": f"{inputs}"}], "temperature" : 1.0, "top_p":1.0, "n" : 1, "stream": True, "presence_penalty":0, "frequency_penalty":0, }

    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {OPENAI_API_KEY}",
        "Headers": f"{request.kwargs['headers']}"
    }
    
    # print(f"chat_counter - {chat_counter}")
    if chat_counter != 0 :
        messages = []
        for i, data in enumerate(history):
            if i % 2 == 0:
                role = 'user'
            else:
                role = 'assistant'
            message = {}
            message["role"] = role
            message["content"] = data
            messages.append(message)
        
        message = {}
        message["role"] = "user" 
        message["content"] = inputs
        messages.append(message)
        payload = {
            "model": MODEL,
            "messages": messages,
            "temperature" : temperature,
            "top_p": top_p,
            "n" : 1,
            "stream": True,
            "presence_penalty":0,
            "frequency_penalty":0,
        }
    
    chat_counter += 1
    
    history.append(inputs)
    token_counter = 0 
    partial_words = "" 
    counter = 0
    
    try:
        # make a POST request to the API endpoint using the requests.post method, passing in stream=True
        response = requests.post(API_URL, headers=headers, json=payload, stream=True)
        response_code = f"{response}"
        #if response_code.strip() != "<Response [200]>":
        #    #print(f"response code - {response}")
        #    raise Exception(f"Sorry, hitting rate limit. Please try again later. {response}")
        
        for chunk in response.iter_lines():
            #Skipping first chunk
            if counter == 0:
                counter += 1
                continue
                #counter+=1
            # check whether each line is non-empty
            if chunk.decode() :
                chunk = chunk.decode()
                # decode each line as response data is in bytes
                if len(chunk) > 12 and "content" in json.loads(chunk[6:])['choices'][0]['delta']:
                    partial_words = partial_words + json.loads(chunk[6:])['choices'][0]["delta"]["content"]
                    if token_counter == 0:
                        history.append(" " + partial_words)
                    else:
                        history[-1] = partial_words
                    token_counter += 1
                    yield [(parse_codeblock(history[i]), parse_codeblock(history[i + 1])) for i in range(0, len(history) - 1, 2) ], history, chat_counter, response, gr.update(interactive=False), gr.update(interactive=False)  # resembles {chatbot: chat, state: history}  
    except Exception as e:
        print (f'error found: {e}')
    yield [(parse_codeblock(history[i]), parse_codeblock(history[i + 1])) for i in range(0, len(history) - 1, 2) ], history, chat_counter, response, gr.update(interactive=True), gr.update(interactive=True)
    print(json.dumps({"chat_counter": chat_counter, "payload": payload, "partial_words": partial_words, "token_counter": token_counter, "counter": counter}))
                   
    

    def reset_textbox(): return gr.update(value='', interactive=False), gr.update(interactive=False)

    title = """GPT4 Chatbot""" if DISABLED: title = """This app has reached OpenAI's usage limit. We are currently requesting an increase in our quota. Please check back in a few days.""" description = """Language models can be conditioned to act like dialogue agents through a conversational prompt that typically takes the form:

    User: <utterance>
    Assistant: <utterance>
    User: <utterance>
    Assistant: <utterance>
    ...
    

    In this app, you can explore the outputs of a gpt-4 LLM. """

    theme = gr.themes.Default(primary_hue="green")

    with gr.Blocks( css="""#col_container { margin-left: auto; margin-right: auto;} #chatbot {height: 520px; overflow: auto;}""", theme=theme) as demo: gr.HTML(title) # 这里省略了部分原有HTML内容和注释 with gr.Column(elem_id="col_container") as main_block: # 将visible设为True或完全移除该参数 chatbot = gr.Chatbot(elem_id='chatbot') inputs = gr.Textbox(placeholder="Hi there!", label="Type an input and press Enter") state = gr.State([]) with gr.Row(): with gr.Column(scale=7): b1 = gr.Button("Send Message").style(full_width=True) with gr.Column(scale=3): server_status_code = gr.Textbox(label="Status code from OpenAI server", )

        # 这里省略了配置参数部分
    
    # 移除用户同意逻辑相关的所有代码
    
    # 连接前端事件与后端函数
    inputs.submit(predict, [inputs, chatbot, state], [chatbot, state],)  # 根据实际情况调整predict函数的参数和返回值
    b1.click(predict, [inputs, chatbot, state], [chatbot, state],)  # 根据实际情况调整predict函数的参数和返回值
    
    demo.launch(share=False)  # 根据实际需要设置launch函数的参数
    
  2. Assistant

    It seems you've pasted a large block of code aiming to create a conversational interface using Gradio and a GPT-4 model from OpenAI. There's a lot to unpack here, but there are essentially a few key aspects of this setup:

    1. Environment Variables: At the start of the code, there are several environment variables being read. These are typically stored in the operating system or a .env file and are used to pass confidential data or configurations into your program.

    2. Exception Handling: A custom exception handler is defined for logging errors without showing stack traces to the end-user.

    3. HTML Handling: A function, parse_codeblock, is provided to format the text response that might include HTML or markdown-like code blocks into HTML that can be displayed on the web page.

    4. API Interaction: The predict function handles the interaction with the OpenAI GPT-4 model. It constructs a payload based on the inputs from the user and makes a POST request. It includes functionalities such as maintaining the conversation history (state) and streaming the response.

    5. Gradio Interface: A Gradio Blocks interface is being set up with an HTML title, a text input box, a chatbot widget, and some buttons.

    6. Launch Application: Finally, the Gradio application is launched at the bottom of the code with demo.launch().

    Here are some points to consider:

    • Check Environment Variables: Ensure that all required environment variables like API_URL, OPENAI_API_KEY, etc., are correctly set in your execution environment.

    • Error Handling: The predict function streams responses from the API and appends the output to the history. Errors are logged but also printed directly. You might consider providing cleaner user feedback in case of API failure or rate-limiting.

    • Gradio Elements: The Gradio interface appears to include a Chatbot widget and text inputs. Ensure they are correctly wired to the corresponding front-end events.

    • API Threading: The NUM_THREADS variable is defined but not actually used in the current implementation. If you plan to use multi-threading for handling requests, consider implementing a thread pool executor.

    • API Call Configuration: Variables like temperature and top_p are hardcoded into the predict function. You may want to provide these as configurable inputs for the user.

    • Gradio Event Handlers: The event handlers (inputs.submit and b1.click) are connected to the predict function but with different arguments than are defined in the predict function itself. You need to ensure that the arguments list and the event binding are consistent.

    • Shared Deployment: You've set share=False in demo.launch(), which means this app won't be publicly shareable via a Gradio link. If this is for local or private use, that's fine, but set share=True if you want to generate a shareable link.

    Remember to sanitize the user input properly to avoid any potential security issues, such as code injection or execution of unwanted scripts. Additionally, check OpenAI's usage terms and conditions and ensure compliance with their API guidelines.

  3. User

    目前出现了问题,这个app不会回应我的提问

  4. Assistant

    从您提供的代码段来看,整个程序设计得相对复杂,是一个与OpenAI GPT-4 API对接的Gradio界面。如果应用没有响应您的问题,可能有几个问题需要检查:

    1. API密钥和URL:确保OPENAI_API_KEY和API_URL环境变量都已正确设置。如果其中一个是空的或者不正确的,程序将无法成功与OpenAI的API进行交互。

    2. 网络问题:确认您的设备可以访问OpenAI API的服务器地址,可能需要检查是否有网络问题或者防火墙设置。

    3. 环境变量DISABLED:如果DISABLED被设置为True,则应用的前端会显示一个消息说明无法使用,并且后端的predict函数将不会执行。

    4. API请求状态:您可以在Gradio界面中增加一个组件来显示API响应的HTTP状态码,这样可以帮您确定请求是否成功。您代码中已经有一个名为server_status_code的文本框,您可以在异常处理中使用它来显示错误代码。

    5. API流响应处理:predict函数使用了流处理(stream=True)来处理API的响应。这部分代码可能需要仔细检查,因为流响应的解析需要特别注意。

    6. 错误处理:程序中的异常处理似乎是将错误打印到标准输出,并尝试继续产生输出。在Gradio界面上,用户不会看到后端的标准输出,所以如果发生错误(如API调用失败),用户可能没有得到适当的反馈。

    7. Javascript控制台:查看浏览器的Javascript控制台是否有任何错误。控制台中的错误或警告通常可以提供关于可能导致应用程序不响应的问题的线索。

    8. Python控制台:检查启动 Gradio 应用程序的 Python 控制台,看是否有抛出任何异常或错误。

    9. API限制:OpenAI 的 API 可能有频率限制以及用量限制。如果超出了这些限制,API 可能会返回错误信息,导致应用无法正常工作。

    10. 前端事件绑定:确认Gradio前端的事件像按钮点击或提交行为已经正确绑定到相应的Python函数。这通常通过.submit和.click等方法完成。

    如果在排查这些问题后仍然无法解决问题,建议进一步简化您的代码测试基础功能,例如直接在Python脚本中调用API而不使用Gradio界面,以确认API调用本身是否有问题。然后逐步增加其他组件和功能,直到找到原因为止。