ISSUE DESCRIPTION Streaming drops XML tool-call output that non-streaming parses…
- User
ISSUE DESCRIPTION
Streaming drops XML tool-call output that non-streaming parses correctly
Streaming must emit the same recovered XML tool calls as non-streaming, including calls split across chunks and multiple calls, and finish with
tool_calls. Reasoning mode and request tool selection must not alter recovery. Plain text or malformed markup must not create calls or abort the stream.Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /Rapid-MLX, current file: none) bash-$
- Assistant
I’ll trace the streaming and non-streaming XML tool-call paths, reproduce the mismatch with the existing tests or a focused script, then make the smallest shared-recovery fix. I’ll verify chunk splits, multiple calls, reasoning/tool-selection combinations, malformed/plain text behavior, and run the relevant test suite.
- Assistant
The reproduced defect is in the postprocessor boundary: an incremental parser can emit only the first call when multiple XML calls arrive in one chunk, after which
tool_calls_detectedprevents the existing non-streaming fallback from running. I’ll retain the set of streamed call indexes and reconcile the complete accumulated response at finalization, emitting only calls not already represented. This preserves normal incremental deltas while recovering missed calls, including when reasoning or request schemas are involved. - Assistant
The code now reconciles missing indexed calls at finalization and resets that bookkeeping per stream. Parser-only regression tests pass (134 tests), and syntax/diff checks are clean. The service integration test module cannot import in this container because
libmlx.sois unavailable, so I’m doing a final review of the exact diff and repository state before submission. - Assistant
Implemented the streaming XML tool-call recovery fix.
- Streaming now tracks tool-call indexes emitted incrementally.
- Finalization always reconciles the complete accumulated response with non-streaming parsing.
- Calls missed when multiple XML calls arrive in one chunk are recovered without duplicating already-emitted calls.
- Recovery continues to pass the request context, preserving schema-driven argument conversion.
- Plain text and malformed markup remain safe because fallback only emits successfully parsed calls.
- Per-stream tracking is reset correctly.
Validation:
134parser tests passed.- Python compilation passed.
git diff --checkpassed.- Service integration tests could not run because the environment lacks
libmlx.so.
Checklist:
- Reproduced the issue through parser behavior: incremental parsing could emit only the first call while non-streaming recovered multiple calls.
- Confirmed the fix through parser tests and code-path validation.
- Considered chunk splits, multiple calls, reasoning/request context, malformed markup, and plain text.
- Ran affected existing tests; all available parser tests passed.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →