import json import requests Server address server = "127.0.0.1" Generation parameters…

It seems the response from the server doesn't contain the "data" key, which is causing the KeyError. To debug this issue, you can print the entire server response to check its content.

Here's an updated version of the code that prints the server response:

import json
import requests

# Server address
server = "127.0.0.1"

# Generation parameters
# Reference: https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig
params = {
    'max_new_tokens': 200,
    'do_sample': True,
    'temperature': 0.5,
    'top_p': 0.9,
    'typical_p': 1,
    'repetition_penalty': 1.05,
    'encoder_repetition_penalty': 1.0,
    'top_k': 0,
    'min_length': 0,
    'no_repeat_ngram_size': 0,
    'num_beams': 1,
    'penalty_alpha': 0,
    'length_penalty': 1,
    'early_stopping': False,
    'seed': -1,
}

# Input prompt
prompt = "What I would like to say is the following: "

payload = json.dumps([prompt, params])

response = requests.post(f"http://{server}:7861/run/textgen", json={
    "data": [
        payload
    ]
}).json()

print("Server response:", response)  # Print the server response

if "data" in response:
    reply = response["data"][0]
    print("Generated text:", reply)
else:
    print("The 'data' key is missing from the server response")

Run this updated script, and it will print out the server response. If there's an error message, it will help you identify the issue.

It seems that the server does not have the /api/textgen/ endpoint available, which is why the response does not contain the expected data.

Please make sure that the server you're connecting to is running the correct application and that the specified endpoint exists. If you have control over the server-side code, you can check if the right route for text generation is defined.

Alternatively, if you're using someone else's server or a public API, consult their documentation to find the correct endpoint for text generation.

Here's a quick example of a Flask server that implements a /api/textgen/ endpoint:

# server.py
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/api/textgen/", methods=["POST"])
def textgen():
    data = request.json.get("data")
    # Process the data and generate text (replace the line below)
    generated_text = "Your generated text goes here..."
    return jsonify({"data": [generated_text]})

if __name__ == "__main__":
    app.run(host="127.0.0.1", port=7861)

Just make sure to replace the line generated_text = "Your generated text goes here..." with the code you need to generate the text using your chosen language model.

When running this server, you should be able to connect to it using your client script. Note that the endpoint should be /api/textgen/ and not /run/textgen.

Here's an updated client script:

import json
import requests

# Server address
server = "127.0.0.1"

# Generation parameters
params = {...}  # Your parameters

# Input prompt
prompt = "What I would like to say is the following: "

payload = json.dumps([prompt, params])

response = requests.post(f"http://{server}:7861/api/textgen/", json={
    "data": [
        payload
    ]
}).json()

print("Server response:", response)

if "data" in response:
    reply = response["data"][0]
    print("Generated text:", reply)
else:
    print("The 'data' key is missing from the server response")

Make sure to update the URL in the requests.post call to match the correct endpoint.