ISSUE DESCRIPTION Bedrock: strict=None auto-promotes many tools to strict and hits…
- User
ISSUE DESCRIPTION
Bedrock:
strict=Noneauto-promotes many tools to strict and hits provider limitInitial Checks
- I'm using the latest version of Pydantic AI
- I've searched for my issue in the issue tracker before opening this issue
Description
Summary
pydantic-ai-slim==1.101.0can now fail Bedrock Claude requests with many tools becauseToolDefinition.strict=Noneis inferred asstrict=Truefor simple schemas. Bedrock/Claude rejects requests with more than 20 strict tools (source):ValidationException: Too many strict tools (73). The maximum number of strict tools supported is 20.This is a behavior regression for callers that did not explicitly opt into strict tool definitions.
What happened
A production agent sends 73 function tools to Bedrock. Before 1.101.0 these tools were sent as normal, non-strict Bedrock tools.
After 1.101.0, these tool definitions have strict=None, and the Bedrock schema transformer infers many/all simple schemas as strict-compatible. _customize_tool_def() then replaces strict=None with strict=True, and BedrockConverseModel._map_tool_definition() sends toolSpec.strict = true. Bedrock rejects the request because it only supports 20 strict tools.
Disclosure: I authored #4237 ✋ . My tests and vcr cassettes didn't account for the >20-tool case.
Expected behavior
For Bedrock, strict=None should not silently turn large toolsets into strict toolsets that violate Bedrock limits.
Possible fixes:
Similar to Anthropic, do not auto-promote Bedrock tool definitions from strict=None to strict=True; only send toolSpec.strict = true when the user explicitly set strict=True.
Workaround
Callers can explicitly set strict=False on Bedrock tool definitions, but this requires every application with larger Bedrock toolsets to discover and patch the behavior.
Minimal, Reproducible Example
Logfire Trace
No response
Python, Pydantic AI & LLM client version
- Python: 3.14
- Pydantic AI: 1.101.0
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /pydantic-ai, current file: none) bash-$
Assistant
Let me start by exploring the repository structure to understand the codebase.
Assistant
Let me look at the Anthropic provider to see how they handle the strict=None case:
Assistant
Now let me look at the Bedrock model implementation:
Assistant
Let me find
_customize_tool_def:Assistant
Let me look at the
JsonSchemaTransformerbase class:Assistant
Let me look at the existing bedrock test file:
Assistant
Let me look at the Anthropic test for the auto-enable behavior:
Assistant
Let me look at all the
test_strict_none_*tests:Assistant
Let me see the entire test file from the top:
Assistant
Let me check git log to understand the history:
Assistant
Now let me understand the broader picture. Let me look at the function in
__init__.py:Assistant
Let me check the cassettes to see what the existing test recordings show:
Assistant
Note the toolSpec in the cassette does NOT have
strict: truebecause the test usesstrict=False. Good.Now let me check the
test_bedrock_mixed_strict_tool_runcassette:Assistant
Now let me check the conftest:
Assistant
Now let me look at the existing inline_snapshot test for
test_strict_none_simple_schemaand check how snapshot is used:Assistant
Now I have a complete understanding. Let me create a reproduction script first:
Assistant
Confirmed the bug. Now let me reproduce the full flow with the Agent:
Assistant
Now I have confirmed the bug. Let me implement the fix following Anthropic's approach.
Assistant
Now let me look at the current implementation more carefully:
Assistant
Now let me implement the fix:
Assistant
Now let me verify the patch:
Assistant
Now let me verify the fix works:
Assistant
Excellent! The fix works. Now let me run the existing tests:
Assistant
Let me run all the strict-related tests:
Assistant
The 4 tests that fail are all testing the buggy behavior we're fixing. Let me run the broader test suite to make sure nothing else is broken:
Assistant
All other tests pass. Now let me also run the strict-related model tests:
Assistant
All strict tool call tests pass. Now let me run the native output tests:
Assistant
All native output tests pass. Now let me run all bedrock-related tests:
Assistant
All 142 model tests pass. The 4 failing tests in
tests/providers/test_bedrock.pyare all testing the old (buggy) auto-promotion behavior. Let me verify those 4 failures are all related to the auto-promotion:Assistant
Let me now verify the fix handles all edge cases properly:
Assistant
All edge cases pass. Let me also do a complete end-to-end test with the Agent:
Assistant
Now let me also check the existing tests in
tests/models/test_bedrock.pyfor any related tests:Assistant
Let me also check that the existing
test_bedrock_native_outputtests still work correctly:Assistant
OK the test description says "prepare_request() forces strict=True for native output". Let me check the cassette to verify:
Assistant
Let me also verify the nested objects test:
Assistant
Let me also run any other test that might involve strict mode:
Assistant
All 249 bedrock-related model tests pass. Let me also run the broader test suite to make sure nothing else is broken:
Assistant
The 4 failing tests are all testing the old buggy behavior. Let me also check that the issue is in
tests/providers/test_bedrock.pyand that I haven't introduced regressions elsewhere. Let me search for any tests that explicitly test foris_strict_compatiblein the codebase:Assistant
Now let me also look at the
transformmethod in Anthropic to compare with the Bedrock implementation:Assistant
Now let me also run the wider test suite to confirm nothing else is broken:
Assistant
Let me also run the broader models test suite:
Assistant
Let me check those non-bedrock failures:
Assistant
Let me confirm these failures are unrelated to my fix:
Assistant
The 2-3 pre-existing failures are unrelated to my fix. Let me make sure the fix is complete and look at the final state:
Assistant
The diff is clean. Let me also run a broader range of tests to verify:
Assistant
The 4 failing tests are exactly the ones testing the buggy auto-promote behavior. Let me see the inline-snapshot output to verify the test failures match my expected fix:
Assistant
The 4 failing tests confirm that the fix correctly:
- Sets
is_strict_compatible=Falseforstrict=None(wasTruefor simple schemas) - Does not add
additionalProperties=Falseto schemas whenstrict=None
These tests are testing the OLD buggy behavior, and they need to be updated to reflect the new (correct) behavior. As per the instructions, I won't modify them.
Let me also do a final sanity check by running the tests with my fix in place once more:
- Sets
Assistant
All sanity checks pass. Let me also clean up my repro scripts:
Assistant
The fix is complete. Let me do a final check of the state of the code:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →