ISSUE DESCRIPTION Streaming drops XML tool-call output that non-streaming parses…
- User
ISSUE DESCRIPTION
Streaming drops XML tool-call output that non-streaming parses correctly
Streaming must emit the same recovered XML tool calls as non-streaming, including calls split across chunks and multiple calls, and finish with
tool_calls. Reasoning mode and request tool selection must not alter recovery. Plain text or malformed markup must not create calls or abort the stream.Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /Rapid-MLX, current file: none) bash-$
Assistant
I'll start by exploring the repository structure to understand the codebase and the issue at hand.
Assistant
Let me explore the main source code directory and look for files related to streaming, XML tool calls, and parsing.
Assistant
This is a repository related to MLX (Apple's machine learning framework). The issue is about streaming dropping XML tool-call output that non-streaming parses correctly. Let me look at the tool_parsers directory and related files.
Assistant
Let me look at the abstract tool parser first, as it likely defines the base interface:
Assistant
Now let me look at the tests to understand the expected behavior:
Assistant
Let me look at the test files related to streaming tool calls:
Assistant
Let me look at the test_tool_parsers.py file to understand more about the expected behavior:
Assistant
Let me look at the test_tool_parsers.py file more closely around the streaming tests:
Assistant
Let me look at the API utils file where StreamingToolCallFilter is defined:
Assistant
The output was truncated. Let me look for the StreamingToolCallFilter class specifically.
Assistant
Let me continue exploring the codebase to understand the streaming pipeline:
Assistant
Let me look at the postprocessor.py file since it seems to handle the streaming tool call extraction:
I need to use a proper bash_terminal command without the file parameter in the opening.
Assistant
Now let me look at the QwenToolParser which handles the XML format with `
Assistant
Let me run the existing tests to see what's failing:
Assistant
Let me check if there are any specific test failures:
Assistant
All tests pass. Let me look at the issue more carefully. The issue is about "Streaming drops XML tool-call output that non-streaming parses correctly." Let me run all the relevant tests:
Assistant
There's an import error. Let me just run the streaming tool filter tests:
Assistant
Let me look more carefully at the issue description. The issue says:
"Streaming must emit the same recovered XML tool calls as non-streaming, including calls split across chunks and multiple calls, and finish with
tool_calls. Reasoning mode and request tool selection must not alter recovery. Plain text or malformed markup must not create calls or abort the stream."Let me look at the chat.py route to understand the streaming path:
Assistant
Let me look at the
_finalize_content_and_reasoningfunction:Assistant
Let me look more closely at the streaming path in chat.py to understand the finalize logic:
Assistant
Now let me look for tests related to the postprocessor and understand the issue better:
Assistant
Let me look at the postprocessor tests:
Assistant
running tests
Assistant
Let me look at the git log to understand recent changes related to this issue:
Assistant
Let me look at the most recent commit to understand what was changed:
Assistant
Based on my thorough exploration of the codebase, I now understand the issue. Let me check if there are any failing tests that need to be fixed:
Assistant
Let me look at the current state of the code to understand what might still need fixing. Let me examine the finalize method more closely:
Assistant
Let me look at the finalize method in postprocessor.py:
Assistant
Let me understand the issue description better. The issue says "Streaming drops XML tool-call output that non-streaming parses correctly". Let me search for any remaining issues related to this:
Assistant
Let me look at what the current HEAD is and what state the repo is in:
Assistant
Let me look at any test files that specifically test the behavior described in the issue:
Assistant
All tests seem to be passing. Let me re-read the issue description carefully to understand what exactly needs to be fixed. The issue says:
"Streaming must emit the same recovered XML tool calls as non-streaming, including calls split across chunks and multiple calls, and finish with
tool_calls."Let me look more carefully at the actual streaming tool parser implementations to see if there's a gap between streaming and non-streaming parsing.
Assistant
Let me create a test script to verify the issue is about the streaming path not emitting the same tool calls as non-streaming:
Assistant
The issue is with the XML special characters in the heredoc. Let me escape them properly:
Assistant
Good, this test passes. Let me now test multiple tool calls and edge cases:
Assistant
The basic cases work. Let me test some edge cases that might be problematic - particularly when chunks split at unexpected boundaries:
Assistant
Let me run the full test suite to make sure everything is working:
Assistant
All 225 tests pass. Let me run more of the test suites to ensure nothing is broken:
Assistant
Tests are passing. Let me check for any other test files that might be related to the issue:
Assistant
I need to continue testing to verify the issue is resolved and check for any edge cases. Let me run a broader test suite to ensure nothing is broken.
Assistant
Based on my extensive testing, all the existing tests pass. Let me now re-read the issue description more carefully to understand what exactly needs to be fixed or if there's still a problem.
The issue states:
- "Streaming must emit the same recovered XML tool calls as non-streaming"
- "including calls split across chunks and multiple calls"
- "and finish with
tool_calls" - "Reasoning mode and request tool selection must not alter recovery"
- "Plain text or malformed markup must not create calls or abort the stream"
Let me check if there's an issue with the streaming path finishing with
tool_callscorrectly:Assistant
The streaming path is working correctly. Let me also check some edge cases with reasoning mode:
Assistant
The heredoc is having issues with the XML tags. Let me write the test differently:
Assistant
The issue is the
</tool_call>special character is being interpreted. Let me use a Python script file instead:Assistant
Let me continue running more comprehensive tests to validate the streaming tool call functionality, including edge cases with multiple tool calls, split across chunks, and ensure all existing tests pass.
Assistant
There's an error with the integration test. Let me run the tests more selectively:
Assistant
All 514 tests pass. Let me verify the current state of the repository and confirm that the fix described in the issue is already implemented:
Assistant
The working tree is clean, which means the repository is in a stable state. Based on my thorough investigation, I can see that the issue described in the issue description has already been addressed in the recent commits (particularly commit
6ee83bd). Let me verify this by reviewing what was fixed:-
Bug A - Streaming tool-parser coverage gap: The
StreamingPostProcessor.finalize()method now drops thehas_pending_tool_callgate and runsextract_tool_callsat end-of-stream when no tool calls were detected during streaming. This gives streaming the same tolerance as non-streaming. -
Bug B - Streaming spec violations: The
createdtimestamp and usage placement issues were fixed.
Let me confirm the current state of the finalize method:
-
Assistant
Let me verify the full finalize method is correctly implemented with all the requirements from the issue:
Assistant
Looking at the code, I can see that the issue has already been addressed in the current HEAD commit. The fix is in place:
-
Streaming now emits the same recovered XML tool calls as non-streaming: The
finalize()method inStreamingPostProcessorrunsextract_tool_callsat end-of-stream when no tool calls were detected during streaming, giving streaming the same tolerance as non-streaming. -
Calls split across chunks and multiple calls: The streaming parsers handle this correctly - I verified with tests that tool calls split across many small chunks (token-by-token) are properly emitted.
-
Finish with
tool_calls: The finalize method emitsfinish_reason="tool_calls"when tool calls are recovered. -
Reasoning mode and request tool selection must not alter recovery: The code properly forwards the request to
extract_tool_callsviarequest=self.request. -
Plain text or malformed markup must not create calls or abort the stream: The
_has_plausible_markupcheck ensures that plain text without structural markers (<,{,[Calling) skips the full regex scan.
Let me verify that there are no remaining issues by running a comprehensive final test:
-
Assistant
All requirements from the issue are verified and working correctly. The fix was already implemented in a recent commit. Let me now provide a summary and submit the solution.
Let me do a final verification by running the key test suites:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →