ISSUE DESCRIPTION Add reasoning.effort support for OpenAI gpt-realtime-2 Feature Type…
- User
ISSUE DESCRIPTION
Add reasoning.effort support for OpenAI gpt-realtime-2
Feature Type
Would make my life easier
Feature Description
OpenAI
gpt-realtime-2supports configurable reasoning viareasoning.effort, but the Python OpenAI Realtime plugin does not seem to expose this option onopenai.realtime.RealtimeModel.It would be useful to support something like:
from livekit.plugins import openai llm = openai.realtime.RealtimeModel( model="gpt-realtime-2", reasoning={"effort": "low"}, )Workarounds / Alternatives
I don't think there is a clean workaround through the current Python
RealtimeModelAPI.The only workaround I can see is patching the plugin locally to add
reasoningto the session update payload.Additional Context
OpenAI docs list these effort values:
minimal,low,medium,high,xhigh.References:
- https://developers.openai.com/api/docs/models/gpt-realtime-2
- https://developers.openai.com/api/docs/guides/realtime-models-prompting#set-reasoning-effort
- Related JS issue: https://github.com/livekit/agents-js/issues/1433
- Related JS PR: https://github.com/livekit/agents-js/pull/1575
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /agents, current file: none) bash-$
Assistant
Let me start by exploring the repository structure to understand what we're working with.
Assistant
Let me look at the realtime_model.py file to understand its structure.
Assistant
Let me look at the
_RealtimeOptionsclass:Assistant
Now let me look at the
_create_session_update_eventmethod to understand how session parameters are constructed:Assistant
Let me check the related JS PR and the OpenAI Realtime Models documentation for the structure:
Assistant
Let me check the RealtimeSessionCreateRequest class from the OpenAI SDK and what fields it has:
Assistant
Let me look for the newer (v2) realtime API:
Assistant
Now I have the complete picture. The RealtimeSessionCreateRequest has a
reasoningfield of typeRealtimeReasoning. Let me look at what RealtimeReasoning is:Assistant
Let me check the models.py to see how
_supports_reasoning_effortis implemented:Assistant
Now let me look at the related JS PR to understand exactly how it was implemented there:
Assistant
Now let me look at the existing imports in realtime_model.py to understand which imports we already have:
Assistant
Now I have all the info I need. Let me check what tests exist:
Assistant
Now let me look at the JS PR to understand the exact naming convention. Let me check the git log:
Assistant
Let me look at the complete Azure conversion:
Assistant
Now I understand the structure. Let me check the
__init__.pyfile:Assistant
Now let me also check the existing tests for the realtime model to see how tests are structured:
Assistant
Now let me look at the full update_options method:
Assistant
Now let me look at the
update_optionsmethod onRealtimeSession(within the class):Assistant
Let me look at the Azure
AzureSessionto see if it supports reasoning:Assistant
Let me see how the existing _supports_reasoning_effort is being used to determine if the model needs default reasoning effort:
Assistant
Now let me first write a small reproduction script to confirm that the feature doesn't currently work:
Assistant
Let me see the conversion helpers in
utils.py:Assistant
Now I have a clear understanding of the patterns. Let me make the changes to add
reasoningsupport. First, let me update the imports:Assistant
Now let me make my changes step by step:
Step 1: Update imports to add
RealtimeReasoningandRealtimeReasoningEffortAssistant
Let me start making the changes to add the
reasoningsupport. First, update the imports:Assistant
Now let me start with the actual changes. First, update the
realtime_model.pyfile:Assistant
Now let me make the edits. First, I'll add the import for
RealtimeReasoning:Assistant
Now let me make the edits.
Step 1: Add imports
Assistant
Step 2: Add the conversion helper in
utils.pyAssistant
Let me add the conversion helper and update the imports in utils.py:
Assistant
Now add the
to_realtime_reasoninghelper function in utils.py:Assistant
Let me verify the inserted code:
Assistant
Step 3: Update
_RealtimeOptionsto addreasoningfieldAssistant
Now let me add the
reasoningfield to_RealtimeOptions:Assistant
Step 4: Add
reasoningparameter to all three__init__overloadsAssistant
Now I'll add
reasoningparameter to all three__init__overloads, plus the docstring. Let me carefully do this with one big edit:Assistant
Step 5: Pass
reasoningto_RealtimeOptionsin__init__Assistant
Now I need to import
to_realtime_reasoningin realtime_model.py:Assistant
Step 6: Use
reasoningin_create_session_update_eventAssistant
Step 7: Update
_oai_session_to_azureto skip reasoning (Azure doesn't support it)Let me check the AzureSession model:
Assistant
Let me verify the Azure conversion function doesn't need updates:
Assistant
Step 8: Add
reasoningparameter toupdate_optionsonRealtimeModelAssistant
Step 9: Add
reasoningtoRealtimeSession.update_optionsAssistant
Now let me run my reproduction script to see if the changes work:
Assistant
The basic case works. Now let me also test the other cases:
Assistant
The first 5 tests pass. The 6th test failed because we can't run the session creation in a sync context. Let me modify the test to be more direct:
Assistant
Now let me test the actual session update event is created correctly:
Assistant
The reasoning field is correctly set in the session update event. Let me also test the
update_optionsfunctionality:Assistant
Let me also test the Azure path:
Assistant
Let me also verify the exact example from the issue:
Assistant
All tests pass. Now let me also run any existing tests to make sure I haven't broken anything:
Assistant
Let me run the existing tests to make sure my changes don't break anything (but with a sanity check on imports first):
Assistant
Let me also check that the existing tests can be collected:
Assistant
Tests collect correctly. Now let me check if any unit tests don't need API keys:
Assistant
Now let me also look at how the project handles mypy / pyright / linting:
Assistant
Let me run mypy on the openai plugin to see if my changes are type-safe:
Assistant
Now let me also check the responses module to make sure my changes don't impact it:
Assistant
These errors are not from my changes - they're from running mypy without the proper environment (it needs the SDK stubs). Let me look at how the official type-check script works:
Assistant
Type checking passes. Now let me also test the
update_optionson the model:Assistant
Excellent! All my tests pass. Now let me also clean up the diff and make sure the changes are well organized. Let me look at the complete diff:
Assistant
Now let me check if there are any other places where the realtime model is created or exposed that need to be updated:
Assistant
Let me also check for any tests in the test directory that may exercise my code:
Assistant
Let me also verify the
livekit-plugins-openaipyproject.toml:Assistant
The OpenAI SDK version is
openai[realtime]>=2which should supportRealtimeReasoning. Let me verify this is the case:Assistant
Now let me check if there's a similar reasoning feature in other realtime plugins (like xAI):
Assistant
Let me verify the linter (ruff) doesn't complain:
Assistant
Let me verify this is a pre-existing issue:
Assistant
Hmm, the lint passes on the original. Let me see what's wrong:
Assistant
The import order was fixed by the ruff fix. Let me also check the utils.py:
Assistant
Let me check the new realtime_model.py file at the imports to see if the import was sorted by ruff:
Assistant
Now let me also run lint on the entire openai plugin:
Assistant
Let me also verify mypy:
Assistant
Now let me also run my reproduction test again to confirm everything still works:
Assistant
All my tests pass. Let me also test edge cases:
Assistant
All edge cases pass. Now let me also verify the Azure path more thoroughly:
Assistant
Now let me also test that the issue example works exactly as the user requested:
Assistant
The exact example from the issue now works. Let me also do a quick sanity check on the
__init__.pyto make sure everything is exported correctly:Assistant
Let me clean up the temporary test files:
Assistant
Let me also check that the final diff looks clean:
Assistant
Let me make a final comprehensive test to confirm everything is working:
Assistant
All tests pass! Let me also clean up the test file and verify final state:
Assistant
Let me also verify the final state of the files:
Assistant
Now let me run the mypy and ruff checks one more time to confirm everything is clean:
Assistant
Let me also check that the tests in
tests/test_realtime/test_realtime.pydon't break by trying to collect them:Assistant
Tests collect successfully. Let me also run the test_config.py to make sure those basic tests still pass:
Assistant
Now let me also check the
responsesLLM to see if I missed anything:Assistant
All changes are in the right place. Let me also verify that everything compiles in the
__init__.py:Assistant
Let me do one final check to verify all my changes are complete and working:
Assistant
Now let me go through the checklist to verify everything is done:
-
If an error was reported in the issue description, I have successfully reproduced it. ✓ The issue was that
reasoningwas not supported inRealtimeModel.__init__. I reproduced this with a test script that calledRealtimeModel(model="gpt-realtime-2", reasoning={"effort": "low"})and got anAttributeError. -
If an error was reported in the issue description, I have confirmed that my fix resolves the error. ✓ After my changes, the exact example from the issue works:
llm = openai.realtime.RealtimeModel(model="gpt-realtime-2", reasoning={"effort": "low"})creates a model withllm._opts.reasoningproperly set toRealtimeReasoning(effort="low"). -
I have thought about edge cases and made sure my fix handles them as well. ✓ Tested with:
- dict form
{"effort": "low"}- works RealtimeReasoninginstance form - works- All valid effort values (
minimal,low,medium,high,xhigh) - work Noneand no reasoning (default) - work- Invalid effort value - raises
ValidationError - Azure path - reasoning is silently dropped (since Azure doesn't support it)
update_optionson bothRealtimeModelandRealtimeSession- work- Session update event contains the reasoning field - works
- dict form
-
I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. ✓
tests/test_config.pypasses ✓tests/test_realtime/test_realtime.py- 49 tests collect successfully (couldn't run because they need API keys) ✓mypypasses with "no issues found in 15 source files" ✓ruffpasses with "All checks passed!"
Let me also verify the final state of the diff one more time:
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →