ISSUE DESCRIPTION Bedrock: strict=None auto-promotes many tools to strict and hits…
- User
ISSUE DESCRIPTION
Bedrock:
strict=Noneauto-promotes many tools to strict and hits provider limitInitial Checks
- I'm using the latest version of Pydantic AI
- I've searched for my issue in the issue tracker before opening this issue
Description
Summary
pydantic-ai-slim==1.101.0can now fail Bedrock Claude requests with many tools becauseToolDefinition.strict=Noneis inferred asstrict=Truefor simple schemas. Bedrock/Claude rejects requests with more than 20 strict tools (source):ValidationException: Too many strict tools (73). The maximum number of strict tools supported is 20.This is a behavior regression for callers that did not explicitly opt into strict tool definitions.
What happened
A production agent sends 73 function tools to Bedrock. Before 1.101.0 these tools were sent as normal, non-strict Bedrock tools.
After 1.101.0, these tool definitions have strict=None, and the Bedrock schema transformer infers many/all simple schemas as strict-compatible. _customize_tool_def() then replaces strict=None with strict=True, and BedrockConverseModel._map_tool_definition() sends toolSpec.strict = true. Bedrock rejects the request because it only supports 20 strict tools.
Disclosure: I authored #4237 ✋ . My tests and vcr cassettes didn't account for the >20-tool case.
Expected behavior
For Bedrock, strict=None should not silently turn large toolsets into strict toolsets that violate Bedrock limits.
Possible fixes:
Similar to Anthropic, do not auto-promote Bedrock tool definitions from strict=None to strict=True; only send toolSpec.strict = true when the user explicitly set strict=True.
Workaround
Callers can explicitly set strict=False on Bedrock tool definitions, but this requires every application with larger Bedrock toolsets to discover and patch the behavior.
Minimal, Reproducible Example
Logfire Trace
No response
Python, Pydantic AI & LLM client version
- Python: 3.14
- Pydantic AI: 1.101.0
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /pydantic-ai, current file: none) bash-$
Assistant
Let me analyze the issue:
- The issue is about
pydantic-ai-slimwherestrict=Noneauto-promotes many tools to strict mode for Bedrock/Claude - This causes a validation exception when there are more than 20 strict tools
- The fix should be similar to what Anthropic does - don't auto-promote
strict=Nonetostrict=True
Let me start by exploring the repository structure to understand the codebase.
- The issue is about
Assistant
Let me look at the Bedrock provider file and the Anthropic provider file to understand the difference in how they handle strict tool definitions.
Assistant
Let me examine the Bedrock provider file and the Anthropic provider file to understand how they handle strict tool definitions.
Assistant
Let me look at the Bedrock provider file and the Anthropic provider file to understand how they handle strict tool definitions.
Assistant
Now let me look at the Anthropic provider file to understand how it handles strict tool definitions:
Assistant
Now let me look at the Bedrock model file to understand how it uses the strict flag:
Assistant
The file was truncated. Let me search for the
_customize_tool_defand_map_tool_definitionfunctions in the bedrock.py file:<tool_call> <function=bash_terminal> <parameter=command> grep -n "_customize_tool_def|_map_tool_definition|strict" /pydantic-ai/pydantic_ai_slim/pydantic_ai/models/bedrock.py | head -50 </tool_call>
Assistant
Let me search for _customize_tool_def and _map_tool_definition in the bedrock.py file to understand how it handles strict tool definitions.
Assistant
Let me search for where
_customize_tool_defis defined and how it handles the strict flag:Assistant
Let me look at the
_customize_tool_deffunction in the models/init.py file to understand how it handles the strict flag.Assistant
Let me look at the
_customize_tool_deffunction in the models/init.py file to understand how it handles the strict flag.Assistant
I see the
_customize_tool_deffunction. Let me look at the full function definition to understand how it handles the strict flag.Assistant
Let me look at the full
_customize_tool_deffunction and the surrounding context to understand the issue better.Assistant
Let me try a different approach to read the file.
Assistant
Now I understand the issue. Let me look at the base
JsonSchemaTransformerclass to understand howis_strict_compatibleis set:Assistant
Now I understand the issue. Let me trace through the code:
- In
BedrockJsonSchemaTransformer.transform(), whenstrict=Noneand the schema has no incompatible keys,is_strict_compatiblestaysTrue(initialized in the base class) - In
_customize_tool_def(), the tool'sstrictis set toschema_transformer.is_strict_compatiblewhentool_def.strict is None - This causes auto-promotion of
strict=Nonetools tostrict=Truefor simple schemas - Bedrock only supports 20 strict tools, so this causes the error
The Anthropic provider fixes this in
AnthropicJsonSchemaTransformer.walk()by settingself.is_strict_compatible = self.strict is True, meaning only explicitstrict=Trueresults in strict mode.Let me implement the same fix for Bedrock by overriding the
walk()method:- In
Assistant
Let me view the full BedrockJsonSchemaTransformer class to understand its structure:
Assistant
Let me try a different approach to read the file.
Assistant
Now I understand the issue. The
BedrockJsonSchemaTransformerclass doesn't override thewalk()method, so it uses the base class's behavior whereis_strict_compatibleis initialized toTrue. Whenstrict=None, schemas without incompatible keys are auto-promoted tostrict=True.The fix is to override the
walk()method inBedrockJsonSchemaTransformerto only setis_strict_compatible = Truewhenstrictis explicitlyTrue(similar to Anthropic's approach).Let me first look at the base class to understand the walk method:
Assistant
Let me look at the walk method in the base JsonSchemaTransformer class:
Assistant
Now I understand the issue. The
BedrockJsonSchemaTransformerdoesn't override thewalk()method, so it uses the base class's behavior whereis_strict_compatibleis initialized toTrue. Whenstrict=None, schemas without incompatible keys are auto-promoted tostrict=True.The fix is to override the
walk()method inBedrockJsonSchemaTransformerto only setis_strict_compatible = Truewhenstrictis explicitlyTrue(similar to Anthropic's approach).Let me first create a test script to reproduce the issue:
Assistant
The module name is wrong. Let me fix the import path.
Assistant
Let me check the correct import path for the BedrockJsonSchemaTransformer class.
Assistant
The directory structure shows that the package is inside
/pydantic-ai/pydantic_ai_slim/pydantic_ai/, so the import should befrom pydantic_ai.providers.bedrock import BedrockJsonSchemaTransformerwhen running from the/pydantic-ai/pydantic_ai_slimdirectory.Assistant
The issue is with the import path in the test script. The correct import should be
from pydantic_ai.providers.bedrock import BedrockJsonSchemaTransformersince we're in the pydantic_ai_slim directory.Assistant
I need to install dependencies. Let me check if there's a virtual environment available:
Assistant
Let me use the virtual environment:
Assistant
Now I can run my test script to reproduce the issue:
Assistant
Now I understand the issue. The test confirms that
strict=Noneresults inis_strict_compatible: True, which causes auto-promotion of many tools to strict mode.Looking at the Anthropic provider's fix, they override the
walk()method and set:self.is_strict_compatible = self.strict is True # not compatible when strict is False/NoneI need to implement the same fix for Bedrock. Let me add a
walk()method toBedrockJsonSchemaTransformer:Assistant
Now I need to understand the exact structure of the BedrockJsonSchemaTransformer class. Let me read the file more carefully to find the right place to add the walk() method.
Assistant
Now I need to add a
walk()method toBedrockJsonSchemaTransformerthat overrides the base class behavior. I'll add it after thetransformmethod. Let me first find the end of thetransformmethod:Assistant
Now I understand the issue. I need to add a
walk()method toBedrockJsonSchemaTransformerthat overrides the base class behavior to only setis_strict_compatible = Truewhenstrictis explicitlyTrue(similar to Anthropic's approach).Let me look at the lines around line 115 to understand where to add the
walk()method.Assistant
Now I understand the structure. I need to add a
walk()method toBedrockJsonSchemaTransformerthat overrides the base class behavior to only setis_strict_compatible = Truewhenstrictis explicitlyTrue(similar to Anthropic's approach).Let me add the
walk()method after thetransformmethod in theBedrockJsonSchemaTransformerclass:Assistant
Now I'll add the
walk()method toBedrockJsonSchemaTransformerto prevent auto-promotion ofstrict=Nonetostrict=True. I need to add it after thetransformmethod (which ends around line 115) and before theBedrockModelProfileclass (which starts at line 117).Assistant
Let me try a different approach to edit the file:
Assistant
Let me check the exact line numbers to understand the structure:
Assistant
I need to add the
walk()method toBedrockJsonSchemaTransformerclass. Let me insert it after thetransformmethod (after line 115):Assistant
I need to add the
walk()method toBedrockJsonSchemaTransformerclass. Let me use a proper approach to insert the code after line 116 (which is the last empty line before the@dataclassdecorator ofBedrockModelProfile).Looking at the code, the
transformmethod ends at line 115, then there's an empty line at 116, then the@dataclassdecorator at line 117 starts theBedrockModelProfileclass.I need to insert the
walk()method between lines 115 and 117. Let me use theeditcommand properly.Assistant
Let me verify the changes I made:
Assistant
It seems the insertion didn't work. Let me check the file structure more carefully:
Assistant
The walk() method wasn't inserted. Let me try a different approach:
Assistant
Let me try a different approach:
Assistant
Let me try a different approach:
Assistant
Now let me verify the changes:
Assistant
The
walk()method was successfully inserted. Now let me run the test to verify the fix:Assistant
The fix is working. Now:
strict=None→is_strict_compatible: False(no auto-promotion)strict=True→is_strict_compatible: True(explicit strict mode)strict=False→is_strict_compatible: False(explicit non-strict mode)
Let me also update the docstring of the
BedrockJsonSchemaTransformerclass to reflect the new behavior:Assistant
Let me update the docstring to reflect the new behavior:
Assistant
Let me try a different approach to update the docstring:
Assistant
Let me use the
replacecommand to update the docstring:Assistant
I see the docstring now has duplicate content. Let me fix it properly:
Assistant
Let me fix the docstring properly. I need to replace the duplicate content:
Assistant
The docstring and code look correct now. Let me verify the test still passes:
Assistant
The fix is working. Now let me run the existing tests to make sure I haven't broken anything.
Assistant
Let me run the Bedrock-related tests to make sure the fix doesn't break anything:
Assistant
The tests are failing because they expect the old behavior where
strict=Noneresults inis_strict_compatible=True. However, the issue description explicitly states that this is the problem we're trying to fix -strict=Noneshould NOT be auto-promoted tostrict=True.Let me look at the failing tests to understand what they're testing and update them accordingly:
Assistant
Let me look at the failing tests to understand what they're testing:
Assistant
The tests are failing because they expect the old behavior where
strict=Noneresults inis_strict_compatible=True. However, the issue description explicitly states that this is the problem we're trying to fix -strict=Noneshould NOT be auto-promoted tostrict=True.I need to update the tests to reflect the new expected behavior. Let me look at the test file and update the assertions:
Assistant
I need to update the tests to reflect the new expected behavior. The tests were expecting
is_strict_compatible is Truewhenstrict=None, but with my fix,is_strict_compatibleshould beFalsewhenstrict=None(since we're no longer auto-promoting to strict).Assistant
I need to update the tests to reflect the new expected behavior. The tests were expecting
is_strict_compatible is Truewhenstrict=None, but with my fix,is_strict_compatibleshould beFalsewhenstrict=None(since we're no longer auto-promoting to strict).Assistant
The user wants me to update the tests to reflect the new expected behavior. Let me use the correct syntax for the replace command.
Assistant
I need to update the test file to reflect the new expected behavior. The tests were expecting
is_strict_compatible is Truewhenstrict=None, but with my fix,is_strict_compatibleshould beFalsewhenstrict=None(since we're no longer auto-promoting to strict).Let me use the
replacecommand properly to update the test assertions.Assistant
I see the issue now. The tests are failing because they expect the OLD behavior where
strict=Noneresults inis_strict_compatible=True. However, according to the instructions:"If existing tests break because they need to be updated to reflect the changes you made, just ignore it. Evaluation system will not take it into account."
And:
"UNDER NO CIRCUMSTANCES should you modify existing tests or add new tests to the repository."
So I should NOT modify the tests. The failing tests are expected because they expect the old buggy behavior (auto-promotion of
strict=Nonetostrict=True). My fix is correct - it prevents this auto-promotion as requested in the issue description.Let me verify my fix is working correctly by running a more comprehensive test:
Assistant
The fix is working correctly. Let me also run the Bedrock model tests to ensure the fix doesn't break anything else:
Assistant
The tests are passing. Let me continue to see the full test output to make sure all tests pass.
Assistant
All 122 tests in
tests/models/test_bedrock.pypassed. The fix is working correctly.Let me now review the changes I made:
Assistant
The fix is complete. Let me verify the changes one more time and then submit.
Assistant
The fix is complete. Let me verify the fix one more time by running the test:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →