Skip to content
TheChatArchiveThe Chat Archive
QwenQwen/Qwen3.6-27B· alibabaHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION Bedrock: strict=None auto-promotes many tools to strict and hits…

  1. User

    ISSUE DESCRIPTION

    Bedrock: strict=None auto-promotes many tools to strict and hits provider limit

    Initial Checks

    Description

    Summary

    pydantic-ai-slim==1.101.0 can now fail Bedrock Claude requests with many tools because ToolDefinition.strict=None is inferred as strict=True for simple schemas. Bedrock/Claude rejects requests with more than 20 strict tools (source):

     ValidationException: Too many strict tools (73). The maximum number of strict tools supported is 20.
    

    This is a behavior regression for callers that did not explicitly opt into strict tool definitions.

    What happened

    A production agent sends 73 function tools to Bedrock. Before 1.101.0 these tools were sent as normal, non-strict Bedrock tools.

    After 1.101.0, these tool definitions have strict=None, and the Bedrock schema transformer infers many/all simple schemas as strict-compatible. _customize_tool_def() then replaces strict=None with strict=True, and BedrockConverseModel._map_tool_definition() sends toolSpec.strict = true. Bedrock rejects the request because it only supports 20 strict tools.

    Disclosure: I authored #4237 ✋ . My tests and vcr cassettes didn't account for the >20-tool case.

    Expected behavior

    For Bedrock, strict=None should not silently turn large toolsets into strict toolsets that violate Bedrock limits.

    Possible fixes:

    Similar to Anthropic, do not auto-promote Bedrock tool definitions from strict=None to strict=True; only send toolSpec.strict = true when the user explicitly set strict=True.

    Workaround

    Callers can explicitly set strict=False on Bedrock tool definitions, but this requires every application with larger Bedrock toolsets to discover and patch the behavior.

    Minimal, Reproducible Example

    Logfire Trace

    No response

    Python, Pydantic AI & LLM client version

    • Python: 3.14
    • Pydantic AI: 1.101.0

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /pydantic-ai, current file: none) bash-$

  2. Qwen Assistant

    Let me analyze the issue:

    1. The problem is that in pydantic-ai-slim==1.101.0, when ToolDefinition.strict=None, the Bedrock schema transformer auto-promotes it to strict=True for simple schemas.
    2. Bedrock/Claude only supports 20 strict tools, so when there are many tools (e.g., 73), the request gets rejected.
    3. The fix should be similar to how Anthropic handles it - do not auto-promote Bedrock tool definitions from strict=None to strict=True; only send toolSpec.strict = true when the user explicitly set strict=True.

    Let me start by exploring the repository structure to understand the codebase.

  3. Qwen Assistant

    Let me explore the Bedrock provider files to understand the current implementation.

  4. Qwen Assistant

    I need to use the correct syntax for the open command without the file parameter.

  5. Qwen Assistant

    Let me also look at the models/bedrock.py file where the BedrockConverseModel is defined:

  6. Qwen Assistant

    Let me search for the _map_tool_definition method in the Bedrock model:

  7. Qwen Assistant

    Now let me look at the _map_tool_definition method and the surrounding context more closely:

  8. Qwen Assistant

    I see that _map_tool_definition only sets tool_spec['strict'] = f.strict when f.strict is truthy (i.e., True). But the issue is that the BedrockJsonSchemaTransformer is auto-promoting strict=None to strict=True for simple schemas. Let me look at how the _customize_tool_def method works in the codebase:

  9. Qwen Assistant

    Let me look at the _customize_tool_def function:

  10. Qwen Assistant

    Now I see the issue! In _customize_tool_def, when tool_def.strict is None, the result is:

    strict=schema_transformer.is_strict_compatible if tool_def.strict is None else tool_def.strict
    

    So if is_strict_compatible is True (which is the default for simple schemas), the tool definition gets auto-promoted to strict=True.

    Let me now look at how the Anthropic provider handles this to understand the expected fix:

  11. Qwen Assistant

    I can see the key difference now. In the AnthropicJsonSchemaTransformer.walk() method, it sets:

    self.is_strict_compatible = self.strict is True  # not compatible when strict is False/None
    

    This means when strict=None, is_strict_compatible is False, so _customize_tool_def won't auto-promote it to strict=True.

    But looking at the BedrockJsonSchemaTransformer, it doesn't override walk() and doesn't set is_strict_compatible = self.strict is True. The base JsonSchemaTransformer class defaults is_strict_compatible = True.

    Let me check the base class:

  12. Qwen Assistant

    Let me look at the JsonSchemaTransformer class:

  13. Qwen Assistant

    Now I understand the issue clearly. The base JsonSchemaTransformer class sets self.is_strict_compatible = True by default. When the BedrockJsonSchemaTransformer processes a schema with strict=None, it may set is_strict_compatible = False if it finds incompatible keys. But for simple schemas (without those incompatible keys), is_strict_compatible remains True, which causes _customize_tool_def to auto-promote strict=None to strict=True.

    The fix should be similar to the Anthropic approach: in the BedrockJsonSchemaTransformer, we need to override the walk() method (or set is_strict_compatible = False when strict is None) so that simple schemas with strict=None are not auto-promoted to strict=True.

    Let me look at the Anthropic implementation more closely:

  14. Qwen Assistant

    Now I understand. The fix is clear: the BedrockJsonSchemaTransformer needs to override walk() and set self.is_strict_compatible = self.strict is True to prevent auto-promotion of strict=None to strict=True.

    But wait, there's another consideration. Looking at the BedrockJsonSchemaTransformer.transform() method:

    elif self.strict is None:
        if schema.get('additionalProperties', None) not in (None, False):
            self.is_strict_compatible = False
        else:
            schema['additionalProperties'] = False
    

    And:

    elif self.strict is None:
        self.is_strict_compatible = False
    

    So currently, the BedrockJsonSchemaTransformer only sets is_strict_compatible = False when there are incompatible keys. For simple schemas, is_strict_compatible stays True, which causes the auto-promotion issue.

    The fix should be: after the walk() method from the base class runs, we should check if strict is None and, if so, set is_strict_compatible = False (similar to how Anthropic does it).

    Let me now check the tests for Bedrock to understand the expected behavior better:

  15. Qwen Assistant

    I see that the existing tests expect is_strict_compatible is True when strict=None. This is the behavior that needs to change. Let me look at more tests:

  16. Qwen Assistant

    Now I can see that the tests have mixed expectations. Some tests (like test_strict_none_preserves_schema, test_strict_none_simple_schema) expect is_strict_compatible is True when strict=None, while others (like test_strict_none_incompatible_schema_disables_auto_strict, test_strict_none_with_additional_properties_true, test_strict_none_preserves_numeric_constraints) expect is_strict_compatible is False.

    Wait, looking more carefully at the tests, I notice that test_strict_none_preserves_numeric_constraints expects is_strict_compatible is False for schemas with numeric constraints. But test_strict_none_simple_schema expects is_strict_compatible is True for simple schemas.

    The issue is that for simple schemas (without incompatible keys), is_strict_compatible stays True, which causes the auto-promotion problem reported in the issue.

    The fix should be: for Bedrock, when strict=None, is_strict_compatible should always be False, regardless of whether the schema is "simple" or has incompatible keys. This is consistent with the Anthropic approach.

    Let me verify my understanding by looking at how the Anthropic provider tests are structured:

  17. Qwen Assistant

    I see that the Anthropic profile tests confirm the expected behavior:

    • strict=True → is_strict_compatible=True
    • strict=False/None → is_strict_compatible=False

    Now let me implement the fix. I need to modify the BedrockJsonSchemaTransformer to override the walk() method and set is_strict_compatible = False when strict is None.

    Let me first look at the current BedrockJsonSchemaTransformer implementation more closely:

  18. Qwen Assistant

    Now I understand the full picture. The fix is to override the walk() method in BedrockJsonSchemaTransformer and, after calling super().walk(), set self.is_strict_compatible = self.strict is True when strict is None or False.

    Let me implement the fix:

  19. Qwen Assistant

    Now I'll create the fix. I need to add a walk() method to BedrockJsonSchemaTransformer that overrides the parent's walk() and sets is_strict_compatible = False when strict is not True.

  20. Qwen Assistant

    Now I need to add the walk() method to BedrockJsonSchemaTransformer. The fix should:

    1. Override walk() to call super().walk() first
    2. After walking, if strict is not True, set is_strict_compatible = False to prevent auto-promotion

    Let me also update the docstring to reflect the new behavior:

  21. Qwen Assistant

    Let me verify the fix was applied correctly:

  22. Qwen Assistant

    I see that the walk() method was not added. Let me check what happened:

  23. Qwen Assistant

    The walk() method wasn't added. Let me check the exact content around the transform method:

  24. Qwen Assistant

    I see. The pattern replacement didn't work because there were only two newlines between return schema and @dataclass, not three. Let me fix this:

  25. Qwen Assistant

    command </tool_call>

  26. Qwen Assistant

    The previous fix was applied, let me verify it now by checking the file content.

  27. Qwen Assistant

    The walk method was successfully added to the BedrockJsonSchemaTransformer class. Now let me verify the implementation by looking at the full method.

  28. Qwen Assistant

    Let me verify the complete class looks correct:

  29. Qwen Assistant

    Now let me create a test script to verify the fix works correctly:

  30. Qwen Assistant

    The module is not installed. Let me check how to run tests in this repository.

  31. Qwen Assistant

    The package can't be installed via pip because of a local dependency. Let me try running Python with the proper PYTHONPATH set.

  32. Qwen Assistant

    Let me install the missing dependencies or try a different approach to test the fix.

  33. Qwen Assistant

    Let me try installing more dependencies or check if there's a simpler way to test this. Let me look for how tests are run in this repository.

  34. Qwen Assistant

    Let me try to install the dependencies needed to run the tests. The repo uses uv, let me check if uv is available and use it to install the project.

  35. Qwen Assistant

    Good, uv is available. Let me try to install the dependencies using uv.

  36. Qwen Assistant

    The installation seems to have succeeded. Let me now run the test script.

  37. Qwen Assistant

    The fix is working. Now let me run the existing Bedrock provider tests to verify the change and see which tests might be affected:

  38. Qwen Assistant

    head -50

  39. Qwen Assistant

    The two failing tests (test_strict_none_preserves_schema and test_strict_none_simple_schema) are testing the old behavior where strict=None auto-promoted to is_strict_compatible=True. My fix intentionally changes this behavior - these tests reflect the bug that the issue is reporting.

    Let me also run some related tests to make sure the fix doesn't break other functionality:

  40. Qwen Assistant

    All 18 tool-related Bedrock model tests pass. Let me also verify that the integration between _customize_tool_def and BedrockJsonSchemaTransformer works correctly by creating a more comprehensive test:

  41. Qwen Assistant

    The fix is working correctly. Now let me verify that the _map_tool_definition method in the Bedrock model also works correctly with this change:

  42. Qwen Assistant

    Let me run all bedrock-related tests to see the full picture:

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.