ISSUE DESCRIPTION [FEATURE] support non Result return types for AiService when tool…
- User
ISSUE DESCRIPTION
[FEATURE] support non Result return types for AiService when tool execution is IMMEDIATE Is your feature request related to a problem? Please describe. If using ReturnBehavior.IMMEDIATE, your AiService method has to return Result even if all tools called return
voidor all returnnullor there's one tool method that returns same type as AiService return type.Describe the solution you'd like I'd like to support non-Result return types for AiService methods when tool execution has IMMEDIATE results based on these rules:
- If ai service method return type is void, just return null
- If any tool executions are TO_LLM throw exception (IMMEDIATE_IF_LAST case)
- If all tool executions have a null result and return type is nullable (not a primitive), then return null
- If there is one non-null tool execution result, and the result can be resolved to the return type, then return that object
I have a PR for this open in Quarkus too:
https://github.com/quarkiverse/quarkus-langchain4j/pull/2529
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /langchain4j, current file: none) bash-$
Assistant
I'll start by exploring the repository structure to understand the codebase and find relevant files.
Assistant
Let me make one command at a time.
Assistant
Let me find the relevant files for AiService and IMMEDIATE return behavior.
Assistant
Let me look at how the immediate return is handled in AiServices.java.
Assistant
Let me find the tool execution loop that handles immediate returns.
Assistant
Let me find where ToolServiceResult is built with immediateToolReturn.
Assistant
Let me look at the relevant part of DefaultAiServices.java around line 338 where immediateToolReturn is handled.
Assistant
Let me look at the executeInferenceAndToolsLoop method in ToolService.java.
Assistant
Let me look at the ToolExecutionResult and ToolExecution classes.
Assistant
Let me look at the existing tests for immediate return behavior.
Assistant
Let me check how DefaultToolExecutor populates the raw result object.
Assistant
Let me look at the streaming handler's immediate return logic.
Assistant
The streaming case is more complex. For streaming, the return type isn't directly resolved to a value—it uses handlers. The streaming case is harder to handle for non-Result return types because there's no direct return value. The issue focuses on the immediate return scenario.
For streaming, currently when shouldReturnImmediately is true, it fires completeResponseHandler with the finalChatResponse. There's no special "Result" handling. The streaming case probably just returns the final response through the completeResponseHandler. For non-Result streaming return, the immediate return just short-circuits. I'm not sure the feature applies to streaming in the same way. Let me focus on the non-streaming case for now, but be careful not to break streaming.
Actually wait, the streaming case doesn't have the
!isReturnTypeResultthrow. Let me check the streaming handler more fully to see if it has a throw for non-Result. Let me search.Assistant
Let me look at the broader DefaultAiServices context to understand returnType and related helpers.
Assistant
Let me look at the helpers in DefaultAiServices for type handling.
Assistant
Let me look at the full ToolServiceResult class.
Assistant
Let me look at the existing tests to understand the immediate return behavior expectations.
Assistant
Let me look at the test that expects an exception for non-Result return type.
Assistant
Let me look at the ImmediateToolWithPrimitiveParameters and StringAssistant to understand the scenario.
Assistant
Let me search for tests that use non-Result with IMMEDIATE and expect exceptions.
Assistant
Only defined in the abstract class. Let me check if it's called/overridden anywhere (it's
protected, so likely called by subclasses via a different mechanism, or maybe it's not invoked). Let me check.Assistant
Let me check if that test method has @Test annotation.
Assistant
It's a
@ParameterizedTestin the abstract class. It would be inherited by subclasses. Let me find subclasses of AbstractAiServiceWithToolsIT.Assistant
These are integration tests that need real models (OpenAI). They won't run in this environment without API keys. So they likely won't run. Good.
Now, let me also check the LocalAiServiceWithToolsIT or similar that might run without external models. Let me search more broadly for subclasses.
Assistant
Let me look at how programmatic ToolExecutor lambdas produce results and whether raw result is set.
Assistant
Let me view the exact loop code section to edit precisely.
Assistant
Now let me make the edits. First, the ToolService.java loop to track current round start and handle non-Result case.
Assistant
No python3. Let me use the edit command. Let me open the file.
Assistant
Let me use the edit command with the proper LINE_RANGE syntax.
Assistant
Let me try the edit command with a simpler single-line replacement form.
Assistant
The special edit command isn't working. Let me use perl/sed via bash instead. Let me check availability.
Assistant
Let me use perl to make the edits in ToolService.java.
Assistant
Let me just insert the line using sed after line 447.
Assistant
Now let me replace the shouldReturnImmediately block. Let me view it again with updated line numbers.
Assistant
I'll replace lines 473-486 with the new block using sed.
Assistant
Now let me add the
immediateReturnToolExecutionsfield to ToolServiceResult.Assistant
Let me make the edits to ToolServiceResult.java using sed. First the field declaration.
Assistant
Now let me add the accessor method after
immediateToolReturn().Assistant
Now update equals, hashCode, toString, and the Builder. Let me view those sections.
Assistant
The equals line got mangled. Let me fix it.
Assistant
Now let me add the builder field and method. Let me view the Builder section.
Assistant
Now let me edit DefaultAiServices.java to handle the non-Result immediate return. Let me view the relevant section.
Assistant
Let me add a blank line after the closing brace for readability.
Assistant
Now let me add the
resolveImmediateToolReturnhelper method. Let me find a good place to add it, nearfireEventAndReturn.Assistant
Let me insert the helper methods after
fireEventAndReturn(after line 411).Assistant
Now let me check the imports needed in DefaultAiServices.
Assistant
I need to check if
ToolExecutionandToolExecutionResultare imported.Assistant
Now let me check ToolService.java imports for
ReturnBehaviorandList,ArrayList.Assistant
Now let me compile the affected modules to verify the changes. Let me build with maven.
Assistant
Let me try building without offline mode (network may be available).
Assistant
There's a name conflict. Let me look at line 426.
Assistant
Let me check what
ToolExecution.result()returns.Assistant
ToolExecution.result()returns String. There'sresultObject()for the raw object. Let me check it.Assistant
I should use
execution.resultObject()for the raw object. Let me fix the helper method.Assistant
Now remove the unused
ToolExecutionResultimport.Assistant
Compilation succeeds. Now let me write a reproduction/test script to verify the feature works. Let me look at existing test patterns using ChatModelMock.
Assistant
Let me look at ReturnBehaviorCombinationsTest to understand the test structure and the @Tool-based approach.
Assistant
Let me create a temporary test to verify the new feature works.
Assistant
The feature works (it returned the tool's String result). The "hello null" is just an argument-parsing issue in my test data. Let me fix the test tool to use no arguments.
Assistant
The arguments replacement didn't happen because the escaping. Let me fix the arguments line directly.
Assistant
All 6 tests pass. Now let me run the existing related tests to ensure I didn't break anything.
Assistant
Let me search the whole repo for any other tests that use IMMEDIATE with non-Result return types expecting exceptions.
Assistant
Let me check AbstractStreamingAiServiceIT for immediate return tests.
Assistant
This streaming test is an IT requiring real streaming models; the streaming handler is unchanged. Now let me remove my temporary test file since I shouldn't add tests to the repo.
Assistant
Let me update the
ReturnBehaviorJavadoc to reflect the new behavior. Let me view the relevant parts.Assistant
The perl replacement worked. Let me replace XX with the new Javadoc text.
Assistant
Now let me update the two "Only allowed on AI services returning Result" Javadoc lines for IMMEDIATE and IMMEDIATE_IF_LAST.
Assistant
Let me check AiServices.java for any docs about IMMEDIATE requiring Result.
Assistant
Let me recompile both modules to verify everything compiles.
Assistant
Let me compile the test classes to make sure my changes don't break test compilation.
Assistant
Let me run the broader service tool package tests once more plus check for any other affected unit tests.
Assistant
Let me verify the IT's
modelssource requires real models (so it won't run here).Assistant
Let me review the final state of the DefaultAiServices changes.
Assistant
Let me re-create the temp test with an additional multi-round scenario to verify the final-round isolation, then run it.
Assistant
All 7 tests pass including the multi-round scenario. Now let me remove the temp test file.
Assistant
Let me run the relevant existing unit tests one final time to confirm nothing is broken.
Assistant
All tests pass. Let me review the complete diff of my changes.
Assistant
Let me check the project version to use the correct
@sinceannotation.Assistant
The version is 1.16.0-beta26-SNAPSHOT. Let me check the most recent
@sincevalues used in the codebase to pick the right one.Assistant
There are existing
@since 1.16.0entries. Let me update my new method's@sinceto 1.16.0.Assistant
Let me directly replace the @since on line 105.
Assistant
Let me search the whole repo for any other places that enforce Result for IMMEDIATE returns.
Assistant
Let me run the langchain4j-core tests to be safe (I modified ReturnBehavior Javadoc).
Assistant
Let me check if any other modules use IMMEDIATE return behavior that might be affected.
Assistant
Let me verify no other modules reference the changed APIs.
Assistant
Let me verify the final git status and that the temp test is removed.
Assistant
Let me do a final compile check of the langchain4j module to confirm everything is good.
Assistant
Let me run a broader set of service unit tests (using mocks) to ensure the restructured flow didn't break the normal (non-immediate) path.
Assistant
Let me check the AiServices class-level Javadoc about return types to see if it mentions IMMEDIATE requiring Result.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →