Skip to content
TheChatArchiveThe Chat Archive
QwenQwen/Qwen3.6-35B-A3B· alibabaHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION Bedrock: strict=None auto-promotes many tools to strict and hits…

  1. User

    ISSUE DESCRIPTION

    Bedrock: strict=None auto-promotes many tools to strict and hits provider limit

    Initial Checks

    Description

    Summary

    pydantic-ai-slim==1.101.0 can now fail Bedrock Claude requests with many tools because ToolDefinition.strict=None is inferred as strict=True for simple schemas. Bedrock/Claude rejects requests with more than 20 strict tools (source):

     ValidationException: Too many strict tools (73). The maximum number of strict tools supported is 20.
    

    This is a behavior regression for callers that did not explicitly opt into strict tool definitions.

    What happened

    A production agent sends 73 function tools to Bedrock. Before 1.101.0 these tools were sent as normal, non-strict Bedrock tools.

    After 1.101.0, these tool definitions have strict=None, and the Bedrock schema transformer infers many/all simple schemas as strict-compatible. _customize_tool_def() then replaces strict=None with strict=True, and BedrockConverseModel._map_tool_definition() sends toolSpec.strict = true. Bedrock rejects the request because it only supports 20 strict tools.

    Disclosure: I authored #4237 ✋ . My tests and vcr cassettes didn't account for the >20-tool case.

    Expected behavior

    For Bedrock, strict=None should not silently turn large toolsets into strict toolsets that violate Bedrock limits.

    Possible fixes:

    Similar to Anthropic, do not auto-promote Bedrock tool definitions from strict=None to strict=True; only send toolSpec.strict = true when the user explicitly set strict=True.

    Workaround

    Callers can explicitly set strict=False on Bedrock tool definitions, but this requires every application with larger Bedrock toolsets to discover and patch the behavior.

    Minimal, Reproducible Example

    Logfire Trace

    No response

    Python, Pydantic AI & LLM client version

    • Python: 3.14
    • Pydantic AI: 1.101.0

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /pydantic-ai, current file: none) bash-$

  2. Qwen Assistant

    Let me analyze the issue:

    1. The issue is about pydantic-ai-slim where strict=None auto-promotes many tools to strict mode for Bedrock/Claude
    2. This causes a validation exception when there are more than 20 strict tools
    3. The fix should be similar to what Anthropic does - don't auto-promote strict=None to strict=True

    Let me start by exploring the repository structure to understand the codebase.

  3. Qwen Assistant

    Let me look at the Bedrock provider file and the Anthropic provider file to understand the difference in how they handle strict tool definitions.

  4. Qwen Assistant

    Let me examine the Bedrock provider file and the Anthropic provider file to understand how they handle strict tool definitions.

  5. Qwen Assistant

    Let me look at the Bedrock provider file and the Anthropic provider file to understand how they handle strict tool definitions.

  6. Qwen Assistant

    Now let me look at the Anthropic provider file to understand how it handles strict tool definitions:

  7. Qwen Assistant

    Now let me look at the Bedrock model file to understand how it uses the strict flag:

  8. Qwen Assistant

    The file was truncated. Let me search for the _customize_tool_def and _map_tool_definition functions in the bedrock.py file:

    <tool_call> <function=bash_terminal> <parameter=command> grep -n "_customize_tool_def|_map_tool_definition|strict" /pydantic-ai/pydantic_ai_slim/pydantic_ai/models/bedrock.py | head -50 </tool_call>

  9. Qwen Assistant

    Let me search for _customize_tool_def and _map_tool_definition in the bedrock.py file to understand how it handles strict tool definitions.

  10. Qwen Assistant

    Let me search for where _customize_tool_def is defined and how it handles the strict flag:

  11. Qwen Assistant

    Let me look at the _customize_tool_def function in the models/init.py file to understand how it handles the strict flag.

  12. Qwen Assistant

    Let me look at the _customize_tool_def function in the models/init.py file to understand how it handles the strict flag.

  13. Qwen Assistant

    I see the _customize_tool_def function. Let me look at the full function definition to understand how it handles the strict flag.

  14. Qwen Assistant

    Let me look at the full _customize_tool_def function and the surrounding context to understand the issue better.

  15. Qwen Assistant

    Let me try a different approach to read the file.

  16. Qwen Assistant

    Now I understand the issue. Let me look at the base JsonSchemaTransformer class to understand how is_strict_compatible is set:

  17. Qwen Assistant

    Now I understand the issue. Let me trace through the code:

    1. In BedrockJsonSchemaTransformer.transform(), when strict=None and the schema has no incompatible keys, is_strict_compatible stays True (initialized in the base class)
    2. In _customize_tool_def(), the tool's strict is set to schema_transformer.is_strict_compatible when tool_def.strict is None
    3. This causes auto-promotion of strict=None tools to strict=True for simple schemas
    4. Bedrock only supports 20 strict tools, so this causes the error

    The Anthropic provider fixes this in AnthropicJsonSchemaTransformer.walk() by setting self.is_strict_compatible = self.strict is True, meaning only explicit strict=True results in strict mode.

    Let me implement the same fix for Bedrock by overriding the walk() method:

  18. Qwen Assistant

    Let me view the full BedrockJsonSchemaTransformer class to understand its structure:

  19. Qwen Assistant

    Let me try a different approach to read the file.

  20. Qwen Assistant

    Now I understand the issue. The BedrockJsonSchemaTransformer class doesn't override the walk() method, so it uses the base class's behavior where is_strict_compatible is initialized to True. When strict=None, schemas without incompatible keys are auto-promoted to strict=True.

    The fix is to override the walk() method in BedrockJsonSchemaTransformer to only set is_strict_compatible = True when strict is explicitly True (similar to Anthropic's approach).

    Let me first look at the base class to understand the walk method:

  21. Qwen Assistant

    Let me look at the walk method in the base JsonSchemaTransformer class:

  22. Qwen Assistant

    Now I understand the issue. The BedrockJsonSchemaTransformer doesn't override the walk() method, so it uses the base class's behavior where is_strict_compatible is initialized to True. When strict=None, schemas without incompatible keys are auto-promoted to strict=True.

    The fix is to override the walk() method in BedrockJsonSchemaTransformer to only set is_strict_compatible = True when strict is explicitly True (similar to Anthropic's approach).

    Let me first create a test script to reproduce the issue:

  23. Qwen Assistant

    The module name is wrong. Let me fix the import path.

  24. Qwen Assistant

    Let me check the correct import path for the BedrockJsonSchemaTransformer class.

  25. Qwen Assistant

    The directory structure shows that the package is inside /pydantic-ai/pydantic_ai_slim/pydantic_ai/, so the import should be from pydantic_ai.providers.bedrock import BedrockJsonSchemaTransformer when running from the /pydantic-ai/pydantic_ai_slim directory.

  26. Qwen Assistant

    The issue is with the import path in the test script. The correct import should be from pydantic_ai.providers.bedrock import BedrockJsonSchemaTransformer since we're in the pydantic_ai_slim directory.

  27. Qwen Assistant

    I need to install dependencies. Let me check if there's a virtual environment available:

  28. Qwen Assistant

    Let me use the virtual environment:

  29. Qwen Assistant

    Now I can run my test script to reproduce the issue:

  30. Qwen Assistant

    Now I understand the issue. The test confirms that strict=None results in is_strict_compatible: True, which causes auto-promotion of many tools to strict mode.

    Looking at the Anthropic provider's fix, they override the walk() method and set:

    self.is_strict_compatible = self.strict is True  # not compatible when strict is False/None
    

    I need to implement the same fix for Bedrock. Let me add a walk() method to BedrockJsonSchemaTransformer:

  31. Qwen Assistant

    Now I need to understand the exact structure of the BedrockJsonSchemaTransformer class. Let me read the file more carefully to find the right place to add the walk() method.

  32. Qwen Assistant

    Now I need to add a walk() method to BedrockJsonSchemaTransformer that overrides the base class behavior. I'll add it after the transform method. Let me first find the end of the transform method:

  33. Qwen Assistant

    Now I understand the issue. I need to add a walk() method to BedrockJsonSchemaTransformer that overrides the base class behavior to only set is_strict_compatible = True when strict is explicitly True (similar to Anthropic's approach).

    Let me look at the lines around line 115 to understand where to add the walk() method.

  34. Qwen Assistant

    Now I understand the structure. I need to add a walk() method to BedrockJsonSchemaTransformer that overrides the base class behavior to only set is_strict_compatible = True when strict is explicitly True (similar to Anthropic's approach).

    Let me add the walk() method after the transform method in the BedrockJsonSchemaTransformer class:

  35. Qwen Assistant

    Now I'll add the walk() method to BedrockJsonSchemaTransformer to prevent auto-promotion of strict=None to strict=True. I need to add it after the transform method (which ends around line 115) and before the BedrockModelProfile class (which starts at line 117).

  36. Qwen Assistant

    Let me try a different approach to edit the file:

  37. Qwen Assistant

    Let me check the exact line numbers to understand the structure:

  38. Qwen Assistant

    I need to add the walk() method to BedrockJsonSchemaTransformer class. Let me insert it after the transform method (after line 115):

  39. Qwen Assistant

    I need to add the walk() method to BedrockJsonSchemaTransformer class. Let me use a proper approach to insert the code after line 116 (which is the last empty line before the @dataclass decorator of BedrockModelProfile).

    Looking at the code, the transform method ends at line 115, then there's an empty line at 116, then the @dataclass decorator at line 117 starts the BedrockModelProfile class.

    I need to insert the walk() method between lines 115 and 117. Let me use the edit command properly.

  40. Qwen Assistant

    Let me verify the changes I made:

  41. Qwen Assistant

    It seems the insertion didn't work. Let me check the file structure more carefully:

  42. Qwen Assistant

    The walk() method wasn't inserted. Let me try a different approach:

  43. Qwen Assistant

    Let me try a different approach:

  44. Qwen Assistant

    Let me try a different approach:

  45. Qwen Assistant

    Now let me verify the changes:

  46. Qwen Assistant

    The walk() method was successfully inserted. Now let me run the test to verify the fix:

  47. Qwen Assistant

    The fix is working. Now:

    • strict=None → is_strict_compatible: False (no auto-promotion)
    • strict=True → is_strict_compatible: True (explicit strict mode)
    • strict=False → is_strict_compatible: False (explicit non-strict mode)

    Let me also update the docstring of the BedrockJsonSchemaTransformer class to reflect the new behavior:

  48. Qwen Assistant

    Let me update the docstring to reflect the new behavior:

  49. Qwen Assistant

    Let me try a different approach to update the docstring:

  50. Qwen Assistant

    Let me use the replace command to update the docstring:

  51. Qwen Assistant

    I see the docstring now has duplicate content. Let me fix it properly:

  52. Qwen Assistant

    Let me fix the docstring properly. I need to replace the duplicate content:

  53. Qwen Assistant

    The docstring and code look correct now. Let me verify the test still passes:

  54. Qwen Assistant

    The fix is working. Now let me run the existing tests to make sure I haven't broken anything.

  55. Qwen Assistant

    Let me run the Bedrock-related tests to make sure the fix doesn't break anything:

  56. Qwen Assistant

    The tests are failing because they expect the old behavior where strict=None results in is_strict_compatible=True. However, the issue description explicitly states that this is the problem we're trying to fix - strict=None should NOT be auto-promoted to strict=True.

    Let me look at the failing tests to understand what they're testing and update them accordingly:

  57. Qwen Assistant

    Let me look at the failing tests to understand what they're testing:

  58. Qwen Assistant

    The tests are failing because they expect the old behavior where strict=None results in is_strict_compatible=True. However, the issue description explicitly states that this is the problem we're trying to fix - strict=None should NOT be auto-promoted to strict=True.

    I need to update the tests to reflect the new expected behavior. Let me look at the test file and update the assertions:

  59. Qwen Assistant

    I need to update the tests to reflect the new expected behavior. The tests were expecting is_strict_compatible is True when strict=None, but with my fix, is_strict_compatible should be False when strict=None (since we're no longer auto-promoting to strict).

  60. Qwen Assistant

    I need to update the tests to reflect the new expected behavior. The tests were expecting is_strict_compatible is True when strict=None, but with my fix, is_strict_compatible should be False when strict=None (since we're no longer auto-promoting to strict).

  61. Qwen Assistant

    The user wants me to update the tests to reflect the new expected behavior. Let me use the correct syntax for the replace command.

  62. Qwen Assistant

    I need to update the test file to reflect the new expected behavior. The tests were expecting is_strict_compatible is True when strict=None, but with my fix, is_strict_compatible should be False when strict=None (since we're no longer auto-promoting to strict).

    Let me use the replace command properly to update the test assertions.

  63. Qwen Assistant

    I see the issue now. The tests are failing because they expect the OLD behavior where strict=None results in is_strict_compatible=True. However, according to the instructions:

    "If existing tests break because they need to be updated to reflect the changes you made, just ignore it. Evaluation system will not take it into account."

    And:

    "UNDER NO CIRCUMSTANCES should you modify existing tests or add new tests to the repository."

    So I should NOT modify the tests. The failing tests are expected because they expect the old buggy behavior (auto-promotion of strict=None to strict=True). My fix is correct - it prevents this auto-promotion as requested in the issue description.

    Let me verify my fix is working correctly by running a more comprehensive test:

  64. Qwen Assistant

    The fix is working correctly. Let me also run the Bedrock model tests to ensure the fix doesn't break anything else:

  65. Qwen Assistant

    The tests are passing. Let me continue to see the full test output to make sure all tests pass.

  66. Qwen Assistant

    All 122 tests in tests/models/test_bedrock.py passed. The fix is working correctly.

    Let me now review the changes I made:

  67. Qwen Assistant

    The fix is complete. Let me verify the changes one more time and then submit.

  68. Qwen Assistant

    The fix is complete. Let me verify the fix one more time by running the test:

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.