ISSUE DESCRIPTION Deferred capabilities throw error 400 with Deepseek models Initial…
- User
ISSUE DESCRIPTION
Deferred capabilities throw error 400 with Deepseek models
Initial Checks
- I'm using the latest version of Pydantic AI
- I've searched for my issue in the issue tracker before opening this issue
Description
When running an agent using a DeepSeek reasoning model, using
Capability(..., defer_loading=True)causes aModelHTTPError: status_code: 400during execution loop turns.openai.BadRequestError: Error code: 400 - {'error': {'message': 'The `reasoning_content` in the thinking mode must be passed back to the API.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}This is triggered because DeepSeek requires the original
reasoning_contentproduced during the thinking phase to be passed back exactly as-is in the subsequent conversation message history array. Whendefer_loading=True, the graph loop intercepted by PydanticAI's capability runner seems to drop or fail to serialize/propagate thereasoning_contenttoken block back to DeepSeek's chat completions endpoint.Setting
defer_loading=Falsesolves the issue, and the agent executes successfully.Minimal, Reproducible Example
# Remember to export the DEEPSEEK_API_KEY environment variable first, then run this code import random from pydantic_ai import Agent, RunContext from pydantic_ai.capabilities import Capability def roll_dice() -> str: """Roll a six-sided die and return the result.""" return str(random.randint(1, 6)) def get_player_name(ctx: RunContext[str]) -> str: """Get the player's name.""" return ctx.deps agent = Agent( 'deepseek:deepseek-v4-flash', deps_type=str, instructions=( "You're a dice game, you should roll the die and see if the number " "you get back matches the user's guess. If so, tell them they're a winner. " "Use the player's name in the response." ), capabilities=[Capability(id='DICE_ROLL', tools=[get_player_name, roll_dice], defer_loading=True)] # Change this to False and the agent will run successfully. ) dice_result = agent.run_sync('My guess is 4', deps='Anne') print(dice_result.output)Logfire Trace
No response
Python, Pydantic AI & LLM client version
- Python:
3.14 - Pydantic AI:
1.106.0 - LLM provider SDK: deepseek
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /pydantic-ai, current file: none) bash-$
Assistant
I'll start by exploring the repository structure to understand the codebase and then investigate the issue.
Assistant
Let me explore the capabilities directory, especially the deferred capability loader and thread executor, since the issue is about deferred loading.
Assistant
Let me look at the deferred_tool_handler.py since that's likely the key file handling deferred loading.
Assistant
Let me look at the deferred capability loader toolset to understand how loading works.
Assistant
Now let me look at the process_history capability and the thinking part handling, since the issue is about
reasoning_contentbeing dropped.Assistant
Let me search for how ThinkingPart / reasoning_content is handled in the models, particularly OpenAI/DeepSeek.
Assistant
Let me look at how ThinkingParts are mapped in the chat completions request, especially around line 2735-3050.
Assistant
This is the Responses API. Let me find the chat completions request mapping for ThinkingParts.
Assistant
Now let me look at the chat completions request mapping (DeepSeek uses chat completions).
Assistant
This is the Responses API. DeepSeek uses chat completions. Let me find the chat completions model and its message mapping.
Assistant
Let me look at the abstract capability to understand the hooks and how the run is wrapped.
Assistant
Let me look at the wrap_node_run, wrap_model_request, before/after_model_request hooks in the abstract capability.
Assistant
Now let me look at the
_agent_graph.pyto understand the model request node and graph loop.Assistant
Let me look at the streamed response handling for chat completions, around line 3300-3400.
Assistant
Let me look at
_clean_message_historyand how messages are cleaned.Assistant
Let me look at the tests for deferred capabilities and DeepSeek to understand the test infrastructure.
Assistant
Let me look at the DeepSeek model profile and the openai chat thinking field configuration.
Assistant
Let me look at
_narrow_tool_call_partsand the_finish_handling/_append_responsemethods.Assistant
Let me look at the CallToolsNode.run to understand how tool results (including load_capability) are processed.
Assistant
Let me search for places where ThinkingParts might be filtered or removed, especially related to capabilities/loading.
Assistant
Let me look at how tests mock the OpenAI client to reproduce the issue.
Assistant
Let me check the deepseek_model_profile.
Assistant
Now let me write a reproduction script using the MockOpenAI to simulate DeepSeek behavior with deferred capabilities.
Assistant
The pydantic-graph version is a dev version. Let me check how the tests are run. Let me look at the pyproject.toml and how the test environment is set up. Maybe there's a virtual environment already.
Assistant
There's a venv. Let me run my reproduction script with the venv Python.
Assistant
Let me rewrite the reproduction script to construct the mock client inline to avoid import issues.
Assistant
My reproduction shows reasoning_content IS preserved. Let me dig deeper - let me print the actual ModelResponse parts and the full message structure to understand the discrepancy.
Assistant
The
resultis defined insidemain(). Let me fix the script.Assistant
Interesting! I see a
ToolSearchCallPartandToolSearchReturnPartbeing synthesized automatically. Let me investigate where this comes from - it has NO reasoning_content. Let me search for where ToolSearchCallPart is synthesized.Assistant
I found the root cause area. The
ToolSearch.before_model_requestsynthesizes aModelResponsewith aToolSearchCallPartbut NOThinkingPart. Let me look at howToolSearchCallPartis mapped to chat completions for DeepSeek.Assistant
Let me look at how the OpenAI chat model maps
ToolSearchCallPartandToolSearchReturnPart.Assistant
Let me look at how the chat completions model decides whether to use native tool search, and check the profile flags.
Assistant
Let me look at the Model base class
prepare_messagesto understand how the synthetic tool search exchange is handled.Assistant
Let me look at
synthesize_local_tool_search_messages.Assistant
Let me look at the tests for the synthetic tool-search exchange on capability load.
Assistant
Let me look at
tool_defs_for_loaded_capabilitiesand how tools become available after a capability loads.Assistant
Let me search for tests exercising deferred capabilities with the OpenAI chat model (non-native tool search).
Assistant
Let me look at the git history for the tool search capability to understand the intent.
Assistant
Let me look at how the synthetic exchange is serialized in a real chat completions request. Let me check the cross-provider replay cassettes.
Assistant
Let me search for where
discovered_tool_namesis updated.Assistant
OK so the tools become visible based on
loaded_capability_tool_names(which comes fromavailable_capability_ids). So after loading a capability, the tools ARE visible in the tools list. The synthetic exchange is for recording the discovery in history (sodiscovered_tool_namesis updated and the discovery is visible across turns/providers).Now I need to determine the right fix. Let me reconsider.
The synthetic exchange creates an assistant message with a
search_toolstool call but no reasoning_content. For DeepSeek in thinking mode, this is invalid.Let me look at the actual DeepSeek requirement more carefully. DeepSeek's error: "The
reasoning_contentin the thinking mode must be passed back to the API."So the fix needs to ensure that the synthetic assistant message either:
- Has reasoning_content (but it can't, since it's synthesized), OR
- Is not sent as an assistant tool_call message to DeepSeek.
Now, let me think about option 2 more concretely. For DeepSeek (local fallback), the synthetic
search_toolscall is sent as an assistant message. But DeepSeek never made this call. So it's a "fake" assistant message.Actually, hold on. Let me reconsider whether the synthetic exchange should be sent to DeepSeek at all. For native providers (Anthropic, OpenAI Responses), the synthetic exchange is replayed as native tool search items, which the provider understands. For non-native providers (DeepSeek), the synthetic exchange is replayed as a
search_toolsfunction call.But wait - for non-native providers, does the model even know about the
search_toolstool? Yes, it's registered (withunless_native='tool_search', but since DeepSeek doesn't support native tool search, thesearch_toolsfunction is kept on the wire). So the model hassearch_toolsavailable.So the synthetic exchange shows the model "called search_tools and got these results". This is meant to inform the model. But it's a fake call.
Hmm. Actually, I wonder if the real fix is to NOT synthesize the exchange for the local fallback case, since the tools are already visible in the tools list. But that might break the cross-provider replay semantics.
Actually, let me reconsider. Let me look at the test that checks the synthetic exchange for a non-native provider. Let me look at test_capabilities.py around line 3600 (the test about keeping the synthetic call/return pair singular).
Let me read that test.
Assistant
Let me look at how the OpenAI chat model handles the synthetic search_tools call in tests. Let me check the openai-chat tool search eval.
Assistant
The synthetic exchange IS persisted to history. Let me check if any cassettes record the synthetic exchange for chat completions providers.
Assistant
Let me look at the cross-provider capability replay test to understand the expected behavior.
Assistant
Let me confirm the bug by simulating DeepSeek's 400 rejection when an assistant message with tool_calls lacks reasoning_content.
Assistant
Confirmed! The synthetic
search_toolsassistant message (index 5) has tool_calls but no reasoning_content, causing the 400. Let me verify the discovered tools are also present in the tools list (so the synthetic exchange is redundant for chat completions).Assistant
The append didn't run main again. The
main()was already called. Let me restructure. Let me just make a fresh repro that prints the tools.Assistant
The discovered tools ARE in the tools list (request 1 has
get_player_name,roll_dice), so the synthetic exchange is redundant for chat completions. Let me look at the chat completions_map_messagesto implement the fix.Assistant
Let me check which providers use
OpenAIChatModeland might have the same reasoning_content issue.Assistant
Let me check if OpenAIChatModel overrides
prepare_messages, and look at the model class hierarchy.Assistant
Let me re-read the
_prepare_requestflow carefully to understand what's persisted vs. what's sent to the wire.Assistant
Let me check if any tests use OpenAIChatModel/MockOpenAI with deferred capabilities and check the wire format.
Assistant
No OpenAIChatModel/MockOpenAI usage in test_capabilities.py. So the capability tests use FunctionModel or custom models, not the chat completions adapter. So my fix in OpenAIChatModel won't break test_capabilities.py.
Let me check test_tool_search.py for OpenAIChatModel/MockOpenAI usage with deferred capabilities.
Assistant
The defer_loading=True in test_tool_search.py are for individual tools, not capabilities. Let me check the test around line 1049 which mentions load_capability.
Assistant
These tests use custom models, not the chat completions adapter. Let me check
NoNativeToolSearchModelto confirm.Assistant
Now let me verify that real model responses from OpenAIChatModel always have
provider_nameset (so I can distinguish synthetic ones).Assistant
Real responses always have
provider_nameset. Now let me checkcount_tokensfor OpenAIChatModel to decide betweenprepare_messagesvs_map_messages.Assistant
The
count_tokensat line 1945 is onOpenAIResponsesModel, notOpenAIChatModel. Socount_tokensisn't a concern for chat models. Let me checkOpenAIResponsesModel'sprepare_messagesto ensure my override onOpenAIChatModelwon't affect it.Assistant
OpenAIResponsesModelis a separate class with native tool search, so it won't be affected.OpenAIModelis a deprecated alias forOpenAIChatModel. Let me look at the end of_tool_search.pyto see existing helpers and where to add mine.Assistant
Now let me look at the
ModelResponseandModelRequestfields to make sure my detection is correct, then implement the helper.Assistant
Let me check the imports in
_tool_search.pyto write the helper correctly.Assistant
The
ModelMessage,ModelRequest,ModelResponseare accessed via_messages.ModelResponseetc. at runtime (since they're only imported under TYPE_CHECKING). Let me look at howsynthesize_local_tool_search_messagesaccesses them. It uses_messages.ModelResponse,_messages.ModelRequest. And it importsfrom . import messages as _messages? Let me check the top imports.Assistant
_messagesis imported. Now let me find the end ofsynthesize_local_tool_search_messagesto add my helper after it.Assistant
Let me view lines 560-565 to add the helper after the function.
Assistant
Now let me add the
prepare_messagesoverride toOpenAIChatModel. First, let me check the imports in openai.py for the tool search module.Assistant
No direct import of
_tool_searchin openai.py. TheToolSearchCallPartetc. are imported from..messages. Let me check how openai.py imports tool search related things.Let me check the imports.
Assistant
Let me add the import of
strip_synthetic_tool_search_exchange. Let me add it after the native_tools import block.Assistant
Now let me add the
prepare_messagesoverride toOpenAIChatModel. Let me view the area aftersystem_prompt_role.Assistant
The issue is that
ModelMessagemight not be imported in openai.py. Let me check. Also, the search text needs exact match. Let me view lines 897-901.Assistant
ModelMessageis imported (line 51). The search failed due to whitespace. Let me use theeditcommand with line numbers instead.Assistant
I need to add a blank line between my new method and
async def request. Let me fix that.Assistant
Now let me run my reproduction scripts to verify the fix.
Assistant
The fix works - the simulated DeepSeek 400 error is gone. Let me verify the wire messages no longer have the synthetic assistant message without reasoning_content.
Assistant
The wire no longer has the synthetic assistant message without reasoning_content. Now let me verify the message HISTORY still has the synthetic exchange (preserving persistence/cross-provider replay).
Assistant
The message HISTORY still has the synthetic exchange (msg 3, msg 4), so persistence/cross-provider replay is preserved. The fix only strips from the WIRE. Now let me verify the import works and run the existing tests.
Assistant
All passing. Let me run the deepseek tests (VCR cassettes) and the tool tests.
Assistant
Let me run the agent tests and the openrouter tests (which also use chat completions and reasoning).
Assistant
The 3 failures are due to
ModuleNotFoundError: No module named 'mcp'(unrelated environment issue). Let me confirm they're pre-existing by stashing my change and re-running those specific tests.Assistant
With my change stashed, the MCP tests passed. Let me re-run the same selection WITH my change to check.
Assistant
Let me re-run the full test_agent.py to check if the MCP failures are consistent or flaky/order-dependent.
Assistant
Let me check if
mcpis installed and investigate the order-dependence.Assistant
The 3 MCP failures are pre-existing (occur without my change too) - they're subprocess environment issues, not caused by my fix. Let me check whether OpenRouter and other reasoning providers subclass
OpenAIChatModel(to confirm my fix's coverage).Assistant
OpenRouter imports
OpenAIChatModel. Let me check ifOpenRouterModelsubclasses it.Assistant
OpenRouterModelsubclassesOpenAIChatModel(covered by my fix). But xAI, Groq, and Mistral have their own adapters. Let me make the fix more general by putting the stripping in the baseModel.prepare_messages(non-native branch) so all non-native adapters benefit. First, let me check if xAI/Groq/Mistral overrideprepare_messages.Assistant
None override
prepare_messages, so they all use the base. Let me check which providers support native tool search (so my non-native-branch fix is correctly scoped).Assistant
Anthropic and OpenAI Responses have native tool search (no stripping needed). Bedrock doesn't. Let me check the base
Model.supported_native_toolsdefault and Gemini's.Assistant
The base default
supported_native_toolsisfrozenset(). Let me check the Google profile'ssupported_native_toolsto understand Gemini's handling.Assistant
Let me check whether the google cassettes record the synthetic exchange as a function call (which would mean Gemini is non-native and a base-level fix could break them).
Assistant
Gemini records the synthetic exchange as a
search_toolsfunction call. This means a base-level fix would break the google cassette. Let me verify the cassette structure to understand whether the synthetic exchange is in the request or response.Assistant
The google cassette sends the synthetic exchange as a function call, so a base-level fix would break it. The OpenAIChatModel override is the correct, safe approach. Let me double-check no test_openai.py or eval cassette tests the synthetic exchange for chat completions.
Assistant
No OpenAIChatModel test/cassette uses the synthetic exchange, so my fix is safe. Let me refine the helper to use direct condition-based filtering (avoiding the
not in droppededge case). Let me view the current helper.Assistant
Let me rewrite the helper body with cleaner condition-based filtering.
Assistant
Now let me replace the old helper body with the new one. Let me find the exact line range.
Assistant
Let me replace the helper body (lines 585-629) with the cleanerversion.
Assistant
The helper is now cleaner. Let me re-run the reproduction to confirm it still works.
Assistant
The fix works. Let me verify the streaming path is also covered (since
prepare_messagesis called in_prepare_request, shared by both paths) and run a broader set of tests.Assistant
Let me run the relevant test suites once more, including model tests and streaming.
Assistant
All 1131 pass. Let me run the deepseek and openai streaming tests too.
Assistant
Let me write a streaming repro to confirm the streaming path is also fixed end-to-end.
Assistant
Streaming path is also fixed. Let me confirm the streaming repro would have failed WITHOUT my fix (by stashing).
Assistant
Confirmed: without my fix, streaming throws the 400; with my fix, it works. Now let me verify my fix does NOT strip REAL
search_toolscalls (made by the model, which have reasoning_content). Let me write a repro with a deferred tool (not capability) where the model callssearch_toolsdirectly.Assistant
My fix correctly preserves REAL
search_toolscalls (request 2 msg[1] has reasoning_content) while stripping synthetic ones. Let me review the final diff and clean up temp files.Assistant
The diff looks clean. Let me check the import ordering against the project's linter (ruff).
Assistant
Ruff is available and reports the import block is un-sorted. Let me let ruff fix it.
Assistant
Let me see where ruff placed the import.
Assistant
Ruff placed the import in the correct sorted position. Let me verify ruff passes and re-run the tests.
Assistant
Ruff added a blank line after the docstring (format style). Let me re-run the relevant tests to confirm everything still works.
Assistant
All 1133 pass. Let me run the openrouter tests and a broader model test sweep to be safe.
Assistant
All passing. Let me re-verify the fix is still functional after the ruff reformatting, and review the final diff.
Assistant
The fix is functional. Let me review the final diff once more.
Assistant
Let me do a final review of the complete helper function.
Assistant
Everything looks clean. Let me verify no temp files are left in the repo and do a final status check.
Assistant
No untracked files. Let me do a final broad test run to be thorough.
Assistant
All passing. Let me verify the annotations work at runtime in
_tool_search.py(thelist[ModelMessage]annotation uses a TYPE_CHECKING-only import).Assistant
Let me confirm
from __future__ import annotationsis present in_tool_search.py(so theModelMessageannotation doesn't fail at runtime).Assistant
The fix is complete and consistent with existing patterns. Let me do one final confirmation with
deepseek-v4-flash(the exact model from the issue, which hasthinking_always_enabled=False).Assistant
The fix works for
deepseek-v4-flash(the exact model from the issue). Let me do a final review of the complete diff before submitting.Assistant
The diff is clean and complete. Let me run the deepseek tests one final time to confirm cassettes replay correctly, then submit.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →