Skip to content
The Chat Archive
ChatGPT (model unspecified)· OpenAI ShareLM (browser-extension contributions)

This code throws an error: """ from together import Together import os…

  1. User

    This code throws an error: """ from together import Together import os os.environ["TOGETHER_API_KEY"] = "tgp_v1_gvDcBb9V-suRtciB2EBmY9pT1uBjcbLN9emimHfZXYg" client = Together() response = client.chat.completions.create( model="Qwen/Qwen3-235B-A22B-fp8-tput", messages=[ { "role": "system", "content": PARSING_DIALOGUE_V2 }, { "role": "user", "content": sample } ], response_format={ "type": "json_schema" }, temperature=0., max_output_tokens=8192, top_p=1, store=True ) print(response.choices[0].message.content) """ While this doesn't: """ from together import Together import os os.environ["TOGETHER_API_KEY"] = "tgp_v1_gvDcBb9V-suRtciB2EBmY9pT1uBjcbLN9emimHfZXYg" client = Together() response = client.chat.completions.create( model="Qwen/Qwen3-235B-A22B-fp8-tput", messages=[ { "role": "system", "content": PARSING_DIALOGUE_V2 }, { "role": "user", "content": sample } ], text={ "format": { "type": "json_object" } }, temperature=0., max_output_tokens=32000, top_p=1, store=True ) print(response.choices[0].message.content)"""

    Can you explain why?

  2. Assistant

    Yes — the key issue lies in the difference between the response_format parameter (which throws an error) and the text parameter (which works). These are not interchangeable, and their expected structure differs based on the Together API's client library and how it wraps LLM endpoints.

    🔴 The Failing Version pythonCopyEditresponse = client.chat.completions.create( ... response_format={ "type": "json_schema" }, ... )

    This throws an error because response_format is not a valid or supported parameter for the chat.completions.create method in the together Python SDK (as of its current or recent versions). You are likely mixing up parameter names or formats used in OpenAI's SDK or in the raw OpenAI-compatible API spec with those supported by the Together SDK.

    ❗ response_format is used in OpenAI's SDK (e.g., response_format="json" or "text") for certain models like GPT-4o or GPT-4 Turbo, but Together's client does not currently support that keyword — especially not in that dict format.

    ✅ The Working Version pythonCopyEdittext={ "format": { "type": "json_object" } }

    This works because text is the correct way to pass formatting instructions to Together's version of the model. For example, Qwen3-235B may have special support for structured generation, and this format: pythonCopyEdittext={ "format": { "type": "json_object" } }

    …is part of Together's extended functionality, especially for newer models with structured output support like Qwen, Mixtral, or Command R.

    🧠 Summary ParameterWorks?Whyresponse_format={...}❌Not a valid parameter in Together SDK. Possibly confused with OpenAI SDK syntax.text={ "format": { "type": "json_object" }}✅Valid usage in Together's SDK for structured outputs.

    ✅ Fix Suggestion If you're aiming for structured output (like a JSON object), use the text.format structure, not response_format. pythonCopyEditresponse = client.chat.completions.create( model="Qwen/Qwen3-235B-A22B-fp8-tput", messages=[ {"role": "system", "content": PARSING_DIALOGUE_V2}, {"role": "user", "content": sample} ], text={ "format": { "type": "json_object" } }, temperature=0., max_output_tokens=8192, top_p=1, store=True )

    Let me know if you want to validate that the model is outputting JSON properly or need help parsing the results.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.