Skip to content
TheChatArchiveThe Chat Archive
Qwenqwen3-30b-a3b· Alibaba (Qwen)Hugging Face LMArena Human Preference 140K

You may call one or more functions to assist with the user query. You are provided with…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Hunyuanhunyuan-turbos-20250416’s answer.
  1. User

    You may call one or more functions to assist with the user query.

    You are provided with function signatures within XML tags: {"type": "function", "function": {"name_for_human": "mobile_use", "name": "mobile_use", "description": "Use a touchscreen to interact with a mobile device, and take screenshots.

    • This is an interface to a mobile device with touchscreen. You can perform actions like clicking, typing, swiping, etc.
    • Some applications may take time to start or process actions, so you may need to wait and take successive screenshots to see the results of your actions.
    • The screen's resolution is 672x1428.", "parameters": {"properties": {"action": {"description": "The action to perform. The available actions are:
    • key: Perform a key event on the mobile device.
      • This supports adb's keyevent syntax.
      • Examples: "volume_up", "volume_down", "power", "camera", "clear".
    • click: Click the point on the screen with coordinate (x, y).
    • long_press: Press the point on the screen with coordinate (x, y) for specified seconds.
    • swipe: Swipe from the starting point with coordinate (x, y) to the end point with coordinates2 (x2, y2).
    • type: Input the specified text into the activated input box.
    • system_button: Press the system button.
    • open: Open an app on the device.
    • wait: Wait specified seconds for the change to happen.
    • terminate: Terminate the current task and report its completion status.", "enum": ["key", "click", "long_press", "swipe", "type", "system_button", "open", "wait", "terminate"], "type": "string"}, "coordinate": {"description": "(x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by action=click, action=long_press, and action=swipe.", "type": "array"}, "coordinate2": {"description": "(x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by action=swipe.", "type": "array"}, "text": {"description": "Required only by action=key, action=type, and action=open.", "type": "string"}, "time": {"description": "The seconds to wait. Required only by action=long_press and action=wait.", "type": "number"}, "button": {"description": "Back means returning to the previous interface, Home means returning to the desktop, Menu means opening the application background menu, and Enter means pressing the enter. Required only by action=system_button", "enum": ["Back", "Home", "Menu", "Enter"], "type": "string"}, "status": {"description": "The status of the task. Required only by action=terminate.", "type": "string", "enum": ["success", "failure"]}}, "required": ["action"], "type": "object"}, "args_format": "Format the arguments as a JSON object."}}

    For each function call, return a json object with function name and arguments: {"name": , "arguments": }

    以上是我定义的用于GUI操控的action space,在实际使用过程中,有时候会遇到广告、弹窗等遮挡,需要先通过click处理到遮挡,才可以继续完成原有的用户指令。在这种情况下,除了click的坐标点,还需要某种方式能够通知框架侧当前的click操作实际是在处理广告等遮挡,而不是真正的用户指令。请帮我修改上述action space定义,从而能够达到预期效果。

  2. Qwen Assistant

    为了区分正常用户指令与处理广告/弹窗的点击操作,可以在 mobile_use 函数的参数中新增一个 ad_context 字段,用于标记点击的意图。修改后的函数定义如下:


    修改后的 mobile_use 函数定义

    {
      "type": "function",
      "function": {
        "name_for_human": "mobile_use",
        "name": "mobile_use",
        "description": "Use a touchscreen to interact with a mobile device, and take screenshots.\n* This is an interface to a mobile device with touchscreen. You can perform actions like clicking, typing, swiping, etc.\n* Some applications may take time to start or process actions, so you may need to wait and take successive screenshots to see the results of your actions.\n* The screen's resolution is 672x1428.",
        "parameters": {
          "properties": {
            "action": {
              "description": "The action to perform. The available actions are: key, click, long_press, swipe, type, system_button, open, wait, terminate.",
              "enum": ["key", "click", "long_press", "swipe", "type", "system_button", "open", "wait", "terminate"],
              "type": "string"
            },
            "coordinate": {
              "description": "(x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by `action=click`, `action=long_press`, and `action=swipe`.",
              "type": "array"
            },
            "coordinate2": {
              "description": "(x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by `action=swipe`.",
              "type": "array"
            },
            "text": {
              "description": "Required only by `action=key`, `action=type`, and `action=open`.",
              "type": "string"
            },
            "time": {
              "description": "The seconds to wait. Required only by `action=long_press` and `action=wait`.",
              "type": "number"
            },
            "button": {
              "description": "Back means returning to the previous interface, Home means returning to the desktop, Menu means opening the application background menu, and Enter means pressing the enter. Required only by `action=system_button`",
              "enum": ["Back", "Home", "Menu", "Enter"],
              "type": "string"
            },
            "status": {
              "description": "The status of the task. Required only by `action=terminate`.",
              "type": "string",
              "enum": ["success", "failure"]
            },
            "ad_context": {
              "description": "Indicates if the action is for handling an ad/popup. 'ad' for ad dismissal, 'normal' for main task.",
              "type": "string",
              "enum": ["normal", "ad"]
            }
          },
          "required": ["action"],
          "type": "object"
        },
        "args_format": "Format the arguments as a JSON object."
      }
    }
    

    使用示例

    1. 处理广告的点击

      {
        "name": "mobile_use",
        "arguments": {
          "action": "click",
          "coordinate": [300, 700],
          "ad_context": "ad"
        }
      }
      
    2. 正常用户指令的点击

      {
        "name": "mobile_use",
        "arguments": {
          "action": "click",
          "coordinate": [200, 500],
          "ad_context": "normal"
        }
      }
      

    框架侧逻辑说明

    • 当 ad_context 设置为 "ad" 时,框架可触发以下行为:
      • 优先处理广告遮挡(如自动重试或记录日志)。
      • 在截图或后续操作中额外标记该点击为“广告处理”。
      • 若广告未被正确处理,可触发错误上报或重新尝试。
    • 当 ad_context 为 "normal" 或未提供时,按原始逻辑执行。

    此修改通过新增参数明确区分操作意图,同时保持原有功能兼容性。

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.