ISSUE DESCRIPTION Streaming drops XML tool-call output that non-streaming parses…
- User
ISSUE DESCRIPTION
Streaming drops XML tool-call output that non-streaming parses correctly
Streaming must emit the same recovered XML tool calls as non-streaming, including calls split across chunks and multiple calls, and finish with
tool_calls. Reasoning mode and request tool selection must not alter recovery. Plain text or malformed markup must not create calls or abort the stream.Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /Rapid-MLX, current file: none) bash-$
Assistant
Let me start by understanding the repository structure and the issue. The issue is about "Streaming drops XML tool-call output that non-streaming parses correctly". This seems to be related to a language model framework (MLX). Let me explore the repository first.
Let me understand the structure and find relevant files related to XML tool calls, streaming, and parsing.
Assistant
Let me explore more to find the relevant files related to XML tool calls and streaming.
Assistant
Let me explore the streaming-related files more closely, and look at the tests for streaming tool calls
Assistant
Let me look at the key files - the postprocessor (which seems to handle XML tool call parsing) and the streaming tool filter test.
Assistant
Let me look at the tool parsers and the streaming-related tests.
Assistant
I need to make just one command at a time. Let me do them sequentially.
Assistant
Let me look at the streaming pipeline integration test and the tool parsers that deal with XML:
Assistant
Let me look at the StreamingToolCallFilter and StreamingThinkRouter in the API utils
Assistant
Let me see the rest of the api/utils.py file, and also look at test files related to XML tool calls
Assistant
Let me look at the StreamingToolCallFilter and StreamingThinkRouter classes:
Assistant
Let me read those classes.
Assistant
Let me find where
_TOOL_CALL_TAGSis defined:Assistant
Let me look at the
_TOOL_CALL_TAGSlistAssistant
Let me look at the test files related to this
Assistant
Let me look at the minimax tool parser and the test for it
Assistant
Let me look at the test for tool calling to find XML-related tests
Assistant
Let me look at the tool_parsers init.py and the tool_calling.py file:
Assistant
Let me search for existing failing tests or the specific issue about XML tool calls being dropped:
Assistant
Let me look at recent changes and search for where the issue might be:
Assistant
Let me run the existing tests related to streaming and tool calling to see if any fail:
Assistant
Let me run the postprocessor tests and the streaming pipeline integration tests
Assistant
Let me run the minimax tool parser test
Assistant
Let me look at the minimax tool parser test file to understand the streaming patterns better
Assistant
Let me look at the Mistral tool parser to see its format:
Assistant
Let me look at several tool parsers to see all the formats:
Assistant
Let me look at the server.py and chat routes to understand the streaming
Assistant
Let me look at the anthropic route to understand how
_emit_content_piecesandStreamingToolCallFilterare usedAssistant
Let me look at the chat route to understand the streaming flow
Assistant
Let me look at the chat route streaming code around line 822-1000
Assistant
Let me look at the git diff for the changes related to streaming tool calls
Assistant
Let me look at commit #424 which seems related to the streaming tool fallback
Assistant
Let me look at the test_chat_streaming_spec.py file
Assistant
Let me look at the seed_oss_tool_parser to see its XML format:
Assistant
Let me see where StreamingToolCallFilter is used:
Assistant
Let me look at the anthropic route more carefully around line 425:
Assistant
Let me see the rest of the anthropic route to understand how tool calls are recovered after
Assistant
Let me look at
_parse_tool_calls_with_parser:Assistant
Let me look at where
_parse_tool_calls_with_parseris defined:Assistant
Let me look at
_parse_tool_calls_with_parserin helpers.pyAssistant
Let me run all tests to see if there are any failures:
Assistant
Let me check what libraries are available and try to install mlx
Assistant
Let me check the mlx installation more carefully
Assistant
Let me look at the Llama tool parser to see its format:
Assistant
Let me search for
[/TOOL_CALLS]in the codeAssistant
Let me search for any test that might be failing related to this
Assistant
Let me try running just the tool filter tests (which don't depend on mlx)
Assistant
Let me look at the agents testing module and adapter to understand the full picture:
Assistant
Let me look at how the agent adapter registers tags
Assistant
Let me look at the agent adapter tag registration
Assistant
Let me search for TODO or FIXME comments related to tool calls or XML
Assistant
Let me create a reproduction script to test the StreamingToolCallFilter behavior:
Assistant
Wait, the output shows
[TOOL_CALL]not[TOOL_CALLS]. Let me re-check the actual codeAssistant
Let me look at the actual code around line 308:
Assistant
Let me look at the Qwen tool parser's streaming method
Assistant
Let me check what format uses
[TOOL_CALL](Assistant
I think I found a bug! Let me verify it:
Assistant
Let me check the Nemotron and other parsers for
[TOOL_CALL]format:Assistant
I found at least one clear bug. Let me fix the
_scan_for_openmethod inStreamingToolCallFilterwhich finds the first tag in list order rather than the earliest in the text. Let me also check if[TOOL_CALL]should be[TOOL_CALLS]Assistant
Let me open the file for
Assistant
Let me navigate to the _scan_for_open method:
Assistant
Let me look at the _scan_for_open method:
Assistant
Now I can see the bug clearly. Let me fix the
_scan_for_openmethod to find the earliest match across all tags, not the first tag in listAssistant
Let me use the heredoc syntax for the edit:
Assistant
Let me use a Python script to make the edit:
Assistant
Now let me replace the method with the fix:
Assistant
Let me verify the fix works:
Assistant
Let me check the xLAM parser to see if it uses
[TOOL_CALL]:Assistant
Now let me also fix the
[TOOL_CALL]tag - it should be[TOOL_CALLS]to match the Mistral/xLAM format. The close tag[/TOOL_CALLS]doesn't actually exist in those formats, but that's OK becauseflush()will discard the unterminated block:Assistant
Let me verify the changes with comprehensive tests:
Assistant
Now let me run the existing tests for
StreamingToolCallFilter:Assistant
Let me also run the streaming pipeline integration test and other related tests that don't require mlx
Assistant
Let me also add the Seed-OSS format to the tool call tags, as it uses
<seed:tool_call>which wasn't coveredAssistant
Let me add Seed-OSS to the default
_TOOL_CALL_TAGSlist permanentlyAssistant
Let me verify the final state of the
_TOOL_CALL_TAGSand run all tests:Assistant
Let me run the full test suite for all unaffected tests:
Assistant
Let me also check if the
_has_plausible_markupcheck infinalizemisses the[TOOL_CALLS]format (without braces):Assistant
Let me add
[TOOL_CALLSto the_has_plausible_markupcheck in the postprocessor to cover the Mistral format edge caseAssistant
Let me look at the postprocessor file for the
_has_plausible_markupcheckAssistant
Let me update the
_has_plausible_markupcheck to also include[TOOL_CALLSAssistant
Let me run the postprocessor tests:
Assistant
Let me also check if
_consume_blockhas any similar issues (looking for correct close tag handlingAssistant
Let me verify all the changes I made
Assistant
Let me run all the relevant tests one final time to confirm everything passes
Assistant
All 470 tests pass. Let me now do a final review of the diff:
Assistant
The changes look good. Let me verify one more edge case and then submit.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →