Skip to content
TheChatArchiveThe Chat Archive
GLMz-ai/glm-5.2· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION Deferred capabilities throw error 400 with Deepseek models Initial…

  1. User

    ISSUE DESCRIPTION

    Deferred capabilities throw error 400 with Deepseek models

    Initial Checks

    Description

    When running an agent using a DeepSeek reasoning model, using Capability(..., defer_loading=True) causes a ModelHTTPError: status_code: 400 during execution loop turns.

    openai.BadRequestError: Error code: 400 - {'error': {'message': 'The `reasoning_content` in the thinking mode must be passed back to the API.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
    

    This is triggered because DeepSeek requires the original reasoning_content produced during the thinking phase to be passed back exactly as-is in the subsequent conversation message history array. When defer_loading=True, the graph loop intercepted by PydanticAI's capability runner seems to drop or fail to serialize/propagate the reasoning_content token block back to DeepSeek's chat completions endpoint.

    Setting defer_loading=False solves the issue, and the agent executes successfully.

    Minimal, Reproducible Example

    # Remember to export the DEEPSEEK_API_KEY environment variable first, then run this code
    
    import random
    from pydantic_ai import Agent, RunContext
    from pydantic_ai.capabilities import Capability
    
    
    def roll_dice() -> str:
      """Roll a six-sided die and return the result."""
      return str(random.randint(1, 6))
    
    def get_player_name(ctx: RunContext[str]) -> str:
      """Get the player's name."""
      return ctx.deps
    
    
    agent = Agent(
      'deepseek:deepseek-v4-flash',  
      deps_type=str,  
      instructions=(
          "You're a dice game, you should roll the die and see if the number "
          "you get back matches the user's guess. If so, tell them they're a winner. "
          "Use the player's name in the response."
      ),
      capabilities=[Capability(id='DICE_ROLL', tools=[get_player_name, roll_dice], defer_loading=True)] # Change this to False and the agent will run successfully.
    )
    
    dice_result = agent.run_sync('My guess is 4', deps='Anne')  
    print(dice_result.output)
    

    Logfire Trace

    No response

    Python, Pydantic AI & LLM client version

    • Python: 3.14
    • Pydantic AI: 1.106.0
    • LLM provider SDK: deepseek

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /pydantic-ai, current file: none) bash-$

  2. GLM Assistant

    I'll start by exploring the repository structure to understand the codebase and then investigate the issue.

  3. GLM Assistant

    Let me explore the capabilities directory, especially the deferred capability loader and thread executor, since the issue is about deferred loading.

  4. GLM Assistant

    Let me look at the deferred_tool_handler.py since that's likely the key file handling deferred loading.

  5. GLM Assistant

    Let me look at the deferred capability loader toolset to understand how loading works.

  6. GLM Assistant

    Now let me look at the process_history capability and the thinking part handling, since the issue is about reasoning_content being dropped.

  7. GLM Assistant

    Let me search for how ThinkingPart / reasoning_content is handled in the models, particularly OpenAI/DeepSeek.

  8. GLM Assistant

    Let me look at how ThinkingParts are mapped in the chat completions request, especially around line 2735-3050.

  9. GLM Assistant

    This is the Responses API. Let me find the chat completions request mapping for ThinkingParts.

  10. GLM Assistant

    Now let me look at the chat completions request mapping (DeepSeek uses chat completions).

  11. GLM Assistant

    This is the Responses API. DeepSeek uses chat completions. Let me find the chat completions model and its message mapping.

  12. GLM Assistant

    Let me look at the abstract capability to understand the hooks and how the run is wrapped.

  13. GLM Assistant

    Let me look at the wrap_node_run, wrap_model_request, before/after_model_request hooks in the abstract capability.

  14. GLM Assistant

    Now let me look at the _agent_graph.py to understand the model request node and graph loop.

  15. GLM Assistant

    Let me look at the streamed response handling for chat completions, around line 3300-3400.

  16. GLM Assistant

    Let me look at _clean_message_history and how messages are cleaned.

  17. GLM Assistant

    Let me look at the tests for deferred capabilities and DeepSeek to understand the test infrastructure.

  18. GLM Assistant

    Let me look at the DeepSeek model profile and the openai chat thinking field configuration.

  19. GLM Assistant

    Let me look at _narrow_tool_call_parts and the _finish_handling / _append_response methods.

  20. GLM Assistant

    Let me look at the CallToolsNode.run to understand how tool results (including load_capability) are processed.

  21. GLM Assistant

    Let me search for places where ThinkingParts might be filtered or removed, especially related to capabilities/loading.

  22. GLM Assistant

    Let me look at how tests mock the OpenAI client to reproduce the issue.

  23. GLM Assistant

    Let me check the deepseek_model_profile.

  24. GLM Assistant

    Now let me write a reproduction script using the MockOpenAI to simulate DeepSeek behavior with deferred capabilities.

  25. GLM Assistant

    The pydantic-graph version is a dev version. Let me check how the tests are run. Let me look at the pyproject.toml and how the test environment is set up. Maybe there's a virtual environment already.

  26. GLM Assistant

    There's a venv. Let me run my reproduction script with the venv Python.

  27. GLM Assistant

    Let me rewrite the reproduction script to construct the mock client inline to avoid import issues.

  28. GLM Assistant

    My reproduction shows reasoning_content IS preserved. Let me dig deeper - let me print the actual ModelResponse parts and the full message structure to understand the discrepancy.

  29. GLM Assistant

    The result is defined inside main(). Let me fix the script.

  30. GLM Assistant

    Interesting! I see a ToolSearchCallPart and ToolSearchReturnPart being synthesized automatically. Let me investigate where this comes from - it has NO reasoning_content. Let me search for where ToolSearchCallPart is synthesized.

  31. GLM Assistant

    I found the root cause area. The ToolSearch.before_model_request synthesizes a ModelResponse with a ToolSearchCallPart but NO ThinkingPart. Let me look at how ToolSearchCallPart is mapped to chat completions for DeepSeek.

  32. GLM Assistant

    Let me look at how the OpenAI chat model maps ToolSearchCallPart and ToolSearchReturnPart.

  33. GLM Assistant

    Let me look at how the chat completions model decides whether to use native tool search, and check the profile flags.

  34. GLM Assistant

    Let me look at the Model base class prepare_messages to understand how the synthetic tool search exchange is handled.

  35. GLM Assistant

    Let me look at synthesize_local_tool_search_messages.

  36. GLM Assistant

    Let me look at the tests for the synthetic tool-search exchange on capability load.

  37. GLM Assistant

    Let me look at tool_defs_for_loaded_capabilities and how tools become available after a capability loads.

  38. GLM Assistant

    Let me search for tests exercising deferred capabilities with the OpenAI chat model (non-native tool search).

  39. GLM Assistant

    Let me look at the git history for the tool search capability to understand the intent.

  40. GLM Assistant

    Let me look at how the synthetic exchange is serialized in a real chat completions request. Let me check the cross-provider replay cassettes.

  41. GLM Assistant

    Let me search for where discovered_tool_names is updated.

  42. GLM Assistant

    OK so the tools become visible based on loaded_capability_tool_names (which comes from available_capability_ids). So after loading a capability, the tools ARE visible in the tools list. The synthetic exchange is for recording the discovery in history (so discovered_tool_names is updated and the discovery is visible across turns/providers).

    Now I need to determine the right fix. Let me reconsider.

    The synthetic exchange creates an assistant message with a search_tools tool call but no reasoning_content. For DeepSeek in thinking mode, this is invalid.

    Let me look at the actual DeepSeek requirement more carefully. DeepSeek's error: "The reasoning_content in the thinking mode must be passed back to the API."

    So the fix needs to ensure that the synthetic assistant message either:

    1. Has reasoning_content (but it can't, since it's synthesized), OR
    2. Is not sent as an assistant tool_call message to DeepSeek.

    Now, let me think about option 2 more concretely. For DeepSeek (local fallback), the synthetic search_tools call is sent as an assistant message. But DeepSeek never made this call. So it's a "fake" assistant message.

    Actually, hold on. Let me reconsider whether the synthetic exchange should be sent to DeepSeek at all. For native providers (Anthropic, OpenAI Responses), the synthetic exchange is replayed as native tool search items, which the provider understands. For non-native providers (DeepSeek), the synthetic exchange is replayed as a search_tools function call.

    But wait - for non-native providers, does the model even know about the search_tools tool? Yes, it's registered (with unless_native='tool_search', but since DeepSeek doesn't support native tool search, the search_tools function is kept on the wire). So the model has search_tools available.

    So the synthetic exchange shows the model "called search_tools and got these results". This is meant to inform the model. But it's a fake call.

    Hmm. Actually, I wonder if the real fix is to NOT synthesize the exchange for the local fallback case, since the tools are already visible in the tools list. But that might break the cross-provider replay semantics.

    Actually, let me reconsider. Let me look at the test that checks the synthetic exchange for a non-native provider. Let me look at test_capabilities.py around line 3600 (the test about keeping the synthetic call/return pair singular).

    Let me read that test.

  43. GLM Assistant

    Let me look at how the OpenAI chat model handles the synthetic search_tools call in tests. Let me check the openai-chat tool search eval.

  44. GLM Assistant

    The synthetic exchange IS persisted to history. Let me check if any cassettes record the synthetic exchange for chat completions providers.

  45. GLM Assistant

    Let me look at the cross-provider capability replay test to understand the expected behavior.

  46. GLM Assistant

    Let me confirm the bug by simulating DeepSeek's 400 rejection when an assistant message with tool_calls lacks reasoning_content.

  47. GLM Assistant

    Confirmed! The synthetic search_tools assistant message (index 5) has tool_calls but no reasoning_content, causing the 400. Let me verify the discovered tools are also present in the tools list (so the synthetic exchange is redundant for chat completions).

  48. GLM Assistant

    The append didn't run main again. The main() was already called. Let me restructure. Let me just make a fresh repro that prints the tools.

  49. GLM Assistant

    The discovered tools ARE in the tools list (request 1 has get_player_name, roll_dice), so the synthetic exchange is redundant for chat completions. Let me look at the chat completions _map_messages to implement the fix.

  50. GLM Assistant

    Let me check which providers use OpenAIChatModel and might have the same reasoning_content issue.

  51. GLM Assistant

    Let me check if OpenAIChatModel overrides prepare_messages, and look at the model class hierarchy.

  52. GLM Assistant

    Let me re-read the _prepare_request flow carefully to understand what's persisted vs. what's sent to the wire.

  53. GLM Assistant

    Let me check if any tests use OpenAIChatModel/MockOpenAI with deferred capabilities and check the wire format.

  54. GLM Assistant

    No OpenAIChatModel/MockOpenAI usage in test_capabilities.py. So the capability tests use FunctionModel or custom models, not the chat completions adapter. So my fix in OpenAIChatModel won't break test_capabilities.py.

    Let me check test_tool_search.py for OpenAIChatModel/MockOpenAI usage with deferred capabilities.

  55. GLM Assistant

    The defer_loading=True in test_tool_search.py are for individual tools, not capabilities. Let me check the test around line 1049 which mentions load_capability.

  56. GLM Assistant

    These tests use custom models, not the chat completions adapter. Let me check NoNativeToolSearchModel to confirm.

  57. GLM Assistant

    Now let me verify that real model responses from OpenAIChatModel always have provider_name set (so I can distinguish synthetic ones).

  58. GLM Assistant

    Real responses always have provider_name set. Now let me check count_tokens for OpenAIChatModel to decide between prepare_messages vs _map_messages.

  59. GLM Assistant

    The count_tokens at line 1945 is on OpenAIResponsesModel, not OpenAIChatModel. So count_tokens isn't a concern for chat models. Let me check OpenAIResponsesModel's prepare_messages to ensure my override on OpenAIChatModel won't affect it.

  60. GLM Assistant

    OpenAIResponsesModel is a separate class with native tool search, so it won't be affected. OpenAIModel is a deprecated alias for OpenAIChatModel. Let me look at the end of _tool_search.py to see existing helpers and where to add mine.

  61. GLM Assistant

    Now let me look at the ModelResponse and ModelRequest fields to make sure my detection is correct, then implement the helper.

  62. GLM Assistant

    Let me check the imports in _tool_search.py to write the helper correctly.

  63. GLM Assistant

    The ModelMessage, ModelRequest, ModelResponse are accessed via _messages.ModelResponse etc. at runtime (since they're only imported under TYPE_CHECKING). Let me look at how synthesize_local_tool_search_messages accesses them. It uses _messages.ModelResponse, _messages.ModelRequest. And it imports from . import messages as _messages? Let me check the top imports.

  64. GLM Assistant

    _messages is imported. Now let me find the end of synthesize_local_tool_search_messages to add my helper after it.

  65. GLM Assistant

    Let me view lines 560-565 to add the helper after the function.

  66. GLM Assistant

    Now let me add the prepare_messages override to OpenAIChatModel. First, let me check the imports in openai.py for the tool search module.

  67. GLM Assistant

    No direct import of _tool_search in openai.py. The ToolSearchCallPart etc. are imported from ..messages. Let me check how openai.py imports tool search related things.

    Let me check the imports.

  68. GLM Assistant

    Let me add the import of strip_synthetic_tool_search_exchange. Let me add it after the native_tools import block.

  69. GLM Assistant

    Now let me add the prepare_messages override to OpenAIChatModel. Let me view the area after system_prompt_role.

  70. GLM Assistant

    The issue is that ModelMessage might not be imported in openai.py. Let me check. Also, the search text needs exact match. Let me view lines 897-901.

  71. GLM Assistant

    ModelMessage is imported (line 51). The search failed due to whitespace. Let me use the edit command with line numbers instead.

  72. GLM Assistant

    I need to add a blank line between my new method and async def request. Let me fix that.

  73. GLM Assistant

    Now let me run my reproduction scripts to verify the fix.

  74. GLM Assistant

    The fix works - the simulated DeepSeek 400 error is gone. Let me verify the wire messages no longer have the synthetic assistant message without reasoning_content.

  75. GLM Assistant

    The wire no longer has the synthetic assistant message without reasoning_content. Now let me verify the message HISTORY still has the synthetic exchange (preserving persistence/cross-provider replay).

  76. GLM Assistant

    The message HISTORY still has the synthetic exchange (msg 3, msg 4), so persistence/cross-provider replay is preserved. The fix only strips from the WIRE. Now let me verify the import works and run the existing tests.

  77. GLM Assistant

    All passing. Let me run the deepseek tests (VCR cassettes) and the tool tests.

  78. GLM Assistant

    Let me run the agent tests and the openrouter tests (which also use chat completions and reasoning).

  79. GLM Assistant

    The 3 failures are due to ModuleNotFoundError: No module named 'mcp' (unrelated environment issue). Let me confirm they're pre-existing by stashing my change and re-running those specific tests.

  80. GLM Assistant

    With my change stashed, the MCP tests passed. Let me re-run the same selection WITH my change to check.

  81. GLM Assistant

    Let me re-run the full test_agent.py to check if the MCP failures are consistent or flaky/order-dependent.

  82. GLM Assistant

    Let me check if mcp is installed and investigate the order-dependence.

  83. GLM Assistant

    The 3 MCP failures are pre-existing (occur without my change too) - they're subprocess environment issues, not caused by my fix. Let me check whether OpenRouter and other reasoning providers subclass OpenAIChatModel (to confirm my fix's coverage).

  84. GLM Assistant

    OpenRouter imports OpenAIChatModel. Let me check if OpenRouterModel subclasses it.

  85. GLM Assistant

    OpenRouterModel subclasses OpenAIChatModel (covered by my fix). But xAI, Groq, and Mistral have their own adapters. Let me make the fix more general by putting the stripping in the base Model.prepare_messages (non-native branch) so all non-native adapters benefit. First, let me check if xAI/Groq/Mistral override prepare_messages.

  86. GLM Assistant

    None override prepare_messages, so they all use the base. Let me check which providers support native tool search (so my non-native-branch fix is correctly scoped).

  87. GLM Assistant

    Anthropic and OpenAI Responses have native tool search (no stripping needed). Bedrock doesn't. Let me check the base Model.supported_native_tools default and Gemini's.

  88. GLM Assistant

    The base default supported_native_tools is frozenset(). Let me check the Google profile's supported_native_tools to understand Gemini's handling.

  89. GLM Assistant

    Let me check whether the google cassettes record the synthetic exchange as a function call (which would mean Gemini is non-native and a base-level fix could break them).

  90. GLM Assistant

    Gemini records the synthetic exchange as a search_tools function call. This means a base-level fix would break the google cassette. Let me verify the cassette structure to understand whether the synthetic exchange is in the request or response.

  91. GLM Assistant

    The google cassette sends the synthetic exchange as a function call, so a base-level fix would break it. The OpenAIChatModel override is the correct, safe approach. Let me double-check no test_openai.py or eval cassette tests the synthetic exchange for chat completions.

  92. GLM Assistant

    No OpenAIChatModel test/cassette uses the synthetic exchange, so my fix is safe. Let me refine the helper to use direct condition-based filtering (avoiding the not in dropped edge case). Let me view the current helper.

  93. GLM Assistant

    Let me rewrite the helper body with cleaner condition-based filtering.

  94. GLM Assistant

    Now let me replace the old helper body with the new one. Let me find the exact line range.

  95. GLM Assistant

    Let me replace the helper body (lines 585-629) with the cleanerversion.

  96. GLM Assistant

    The helper is now cleaner. Let me re-run the reproduction to confirm it still works.

  97. GLM Assistant

    The fix works. Let me verify the streaming path is also covered (since prepare_messages is called in _prepare_request, shared by both paths) and run a broader set of tests.

  98. GLM Assistant

    Let me run the relevant test suites once more, including model tests and streaming.

  99. GLM Assistant

    All 1131 pass. Let me run the deepseek and openai streaming tests too.

  100. GLM Assistant

    Let me write a streaming repro to confirm the streaming path is also fixed end-to-end.

  101. GLM Assistant

    Streaming path is also fixed. Let me confirm the streaming repro would have failed WITHOUT my fix (by stashing).

  102. GLM Assistant

    Confirmed: without my fix, streaming throws the 400; with my fix, it works. Now let me verify my fix does NOT strip REAL search_tools calls (made by the model, which have reasoning_content). Let me write a repro with a deferred tool (not capability) where the model calls search_tools directly.

  103. GLM Assistant

    My fix correctly preserves REAL search_tools calls (request 2 msg[1] has reasoning_content) while stripping synthetic ones. Let me review the final diff and clean up temp files.

  104. GLM Assistant

    The diff looks clean. Let me check the import ordering against the project's linter (ruff).

  105. GLM Assistant

    Ruff is available and reports the import block is un-sorted. Let me let ruff fix it.

  106. GLM Assistant

    Let me see where ruff placed the import.

  107. GLM Assistant

    Ruff placed the import in the correct sorted position. Let me verify ruff passes and re-run the tests.

  108. GLM Assistant

    Ruff added a blank line after the docstring (format style). Let me re-run the relevant tests to confirm everything still works.

  109. GLM Assistant

    All 1133 pass. Let me run the openrouter tests and a broader model test sweep to be safe.

  110. GLM Assistant

    All passing. Let me re-verify the fix is still functional after the ruff reformatting, and review the final diff.

  111. GLM Assistant

    The fix is functional. Let me review the final diff once more.

  112. GLM Assistant

    Let me do a final review of the complete helper function.

  113. GLM Assistant

    Everything looks clean. Let me verify no temp files are left in the repo and do a final status check.

  114. GLM Assistant

    No untracked files. Let me do a final broad test run to be thorough.

  115. GLM Assistant

    All passing. Let me verify the annotations work at runtime in _tool_search.py (the list[ModelMessage] annotation uses a TYPE_CHECKING-only import).

  116. GLM Assistant

    Let me confirm from __future__ import annotations is present in _tool_search.py (so the ModelMessage annotation doesn't fail at runtime).

  117. GLM Assistant

    The fix is complete and consistent with existing patterns. Let me do one final confirmation with deepseek-v4-flash (the exact model from the issue, which has thinking_always_enabled=False).

  118. GLM Assistant

    The fix works for deepseek-v4-flash (the exact model from the issue). Let me do a final review of the complete diff before submitting.

  119. GLM Assistant

    The diff is clean and complete. Let me run the deepseek tests one final time to confirm cassettes replay correctly, then submit.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.