Skip to content
TheChatArchiveThe Chat Archive
gpt-3.5-turbo-0301· OpenAIAi2 WildChat-4.8M (AI2)

Run this code import shutil import torch from transformers import AutoTokenizer,…

  1. User

    Run this code import shutil import torch from transformers import AutoTokenizer, GPT2LMHeadModel

    Set the maximum GPU memory limit

    gpu_memory_limit = 0.5 # Set the fraction as needed device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    if device.type == "cuda": torch.cuda.set_device(0) torch.cuda.empty_cache() torch.cuda.max_memory_allocated = gpu_memory_limit * torch.cuda.get_device_properties(device).total_memory

    Load the downloaded files into a chatbot

    model_dir = "HuggingFaceH4/starchat-alpha" output_dir = "./starchat-alpha"

    Clear GPU memory before loading model and tokenizer

    torch.cuda.empty_cache()

    tokenizer = AutoTokenizer.from_pretrained(model_dir) model = GPT2LMHeadModel.from_pretrained(model_dir) model = model.to(device)

    Save the model files to the output directory

    shutil.copytree(tokenizer.config_dir, output_dir)

    Interact with the chatbot

    while True: user_input = input("You: ") encoded_input = tokenizer.encode(user_input, return_tensors="pt").to(device) response = model.generate(encoded_input, max_length=100) generated_text = tokenizer.decode(response[0], skip_special_tokens=True) print("Bot:", generated_text)

  2. Assistant

    This code loads a pre-trained GPT-2 model and tokenizer from a specified directory, saves the tokenizer files to an output directory, and then interacts with the chatbot by generating responses to user input using the GPT-2 model. The code also sets a maximum GPU memory limit and clears GPU memory before loading the model and tokenizer to improve performance. The user is prompted to input text, which is encoded using the tokenizer and fed to the model to generate a response. The response is then decoded using the tokenizer and printed to the console as the chatbot's response. The loop continues until the user stops the program.