ISSUE DESCRIPTION Add reasoning.effort support for OpenAI gpt-realtime-2 Feature Type…
- User
ISSUE DESCRIPTION
Add reasoning.effort support for OpenAI gpt-realtime-2
Feature Type
Would make my life easier
Feature Description
OpenAI
gpt-realtime-2supports configurable reasoning viareasoning.effort, but the Python OpenAI Realtime plugin does not seem to expose this option onopenai.realtime.RealtimeModel.It would be useful to support something like:
from livekit.plugins import openai llm = openai.realtime.RealtimeModel( model="gpt-realtime-2", reasoning={"effort": "low"}, )Workarounds / Alternatives
I don't think there is a clean workaround through the current Python
RealtimeModelAPI.The only workaround I can see is patching the plugin locally to add
reasoningto the session update payload.Additional Context
OpenAI docs list these effort values:
minimal,low,medium,high,xhigh.References:
- https://developers.openai.com/api/docs/models/gpt-realtime-2
- https://developers.openai.com/api/docs/guides/realtime-models-prompting#set-reasoning-effort
- Related JS issue: https://github.com/livekit/agents-js/issues/1433
- Related JS PR: https://github.com/livekit/agents-js/pull/1575
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /agents, current file: none) bash-$
- Assistant
I’ll first inspect the repository structure and locate the Python OpenAI realtime model implementation and its tests. Then I’ll trace how constructor options become session-update payloads, implement the typed
reasoning.effortoption consistently with existing conventions, and run focused tests plus relevant static checks. - Assistant
The repository is a monorepo and the OpenAI plugin is editable-installed in the local virtual environment. I’ll now enumerate its realtime source and test files, then search for session configuration types and payload serialization to identify all API surfaces that need the new option.
- Assistant
ripgrepis unavailable, so I’ll use recursivegrep. I’ll inspect the realtime module in sections and check package exports and existing OpenAI reasoning types to reuse naming and validation patterns rather than introducing an inconsistent API. - Assistant
The model stores constructor settings in
_RealtimeOptionsand builds an OpenAI SDKRealtimeSessionCreateRequest. Because Azure uses a conversion path that intentionally supports only legacy fields, I need to decide whether reasoning should be OpenAI-only or harmlessly omitted for Azure, consistent with current behavior. I’ll inspect constructor/update methods and the installed SDK’s exact realtime reasoning type/field definitions. - Assistant
The installed SDK already exposes
RealtimeReasoningandRealtimeReasoningEffort, andRealtimeSessionCreateRequest.reasoning, so no raw dictionary workaround is needed internally. The public request explicitly shows a dictionary input, though accepting the SDK Pydantic model as well would match other options such as noise reduction and transcription. I’ll inspect the complete mutable update path, factory forwarding, test layout, and SDK aliases to define the narrowest compatible type. - Assistant
There are two
update_optionsmethods: model-level defaults (propagated to active sessions) and session-level updates (serialized immediately). To make the option behave like the existing mutable configuration, reasoning should be represented in_RealtimeOptions, accepted by constructor and model/session updates, included in initial session creation, and forwarded to active sessions. I’ll inspect the remainder of the model-level method and class boundary to ensure all forwarding logic is covered. - Assistant
The option should support both the documented dictionary literal and the SDK model while normalizing to
RealtimeReasoning; normalization gives reliable equality checks and catches invalid effort values before opening a websocket. I’ll verify how generated OpenAI events serialize unset versus explicitNone, then implement a small converter in the realtime utilities (matching existingto_*helpers), add imports/types across both update layers, and preserve omission when reasoning was not supplied. - Assistant
I confirmed an unset field is omitted while explicit
Noneserializes as JSONnull. Therefore, initial creation will setreasoningonly when non-None, whileupdate_options(reasoning=None)will intentionally sendnullto clear an existing setting. I’ll now apply the implementation across the option dataclass, OpenAI constructor overload/implementation/docs, initial payload, and both update methods. Legacy Azure conversion will continue to omit this unsupported modern field. - Assistant
The implementation is in place and lint-clean. Before broader checks, I’ll directly exercise the reported API without network access by constructing the model, invoking the payload builder on a lightweight session object, and serializing the generated event. I’ll verify all five documented effort values, dictionary and SDK-model inputs, invalid-value rejection, update payloads, clearing via
None, and no-op update deduplication. - Assistant
The exact requested constructor now produces
session.reasoning.effort, every documented value works, invalid values fail with SDK validation, and mutable updates—including clearing and duplicate suppression—behave correctly. The repository has generic realtime tests but no OpenAI-plugin-specific unit suite; I’ll inspect their markers/dependencies and run the relevant non-integration checks. I’ll also run mypy on the changed module because TypedDict/Pydantic unions can reveal issues lint does not. - Assistant
The generic realtime suite is live-service integration and requires OpenAI/Azure credentials, so it cannot provide a deterministic local verification here. Mypy’s focused invocation reports only pre-existing issues caused by skipped imports and existing ignores; no error points to the new reasoning code. One compatibility consideration remains: the plugin declares
openai>=2, but these generated realtime reasoning types may have arrived later. I’ll inspect lock/history and package metadata to determine whether the dependency floor should be raised or whether this repository convention relies on the lock/current SDK schema. - Assistant
The dependency was deliberately broadened recently to
openai>=2, while the lock resolves 2.40; changing that policy without evidence would be out of scope. The source already imports the SDK’s current GA realtime schema directly, so using its reasoning types is consistent. I’ll now run repository tests that exercise model/session abstractions without external APIs, plus the full OpenAI module lint and import/signature smoke checks. I’ll skip credential-bound live realtime tests explicitly rather than causing network failures. - Assistant
All local verification passes: full OpenAI plugin lint, compilation, 35 realtime-model abstraction tests, public API smoke checks, and direct payload tests. The live OpenAI/Azure integration suite was not run because it requires external credentials and network service access, but its collection/source confirms it does not contain local serialization tests.
Before submission I’ll do a final diff/status review and verify legacy Azure behavior: modern OpenAI payloads must include reasoning, while legacy Azure conversion must omit the unsupported field rather than leaking it into old-schema requests.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →