Skip to content
TheChatArchiveThe Chat Archive
gpt-4-0125-preview· openaiHugging Face Suzhen/CodeChat

write code to solve this Traceback (most recent call last): File…

  1. User

    write code to solve this Traceback (most recent call last): File "C:\Users\admin\Mm\generateText.py", line 136, in fire.Fire(main) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "C:\Users\admin\Mm\generateText.py", line 52, in main model = PeftModel.from_pretrained( File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 353, in from_pretrained model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 694, in load_adapter adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\utils\save_and_load.py", line 326, in load_peft_weights adapters_weights = safe_load_file(filename, device=device) File "C:\Users\admin\Mm\venv\lib\site-packages\safetensors\torch.py", line 308, in load_file with safe_open(filename, framework="pt", device=device) as f: safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization

  2. Assistant

    The error you're encountering seems to pertain to a failure in deserializing the header of a saved model or adapter with the safetensors package, which is used by the peft package for loading model weights. This can result from various issues such as a corrupted file, compatibility issues, or an incorrect file path.

    Below are steps and example code to troubleshoot and potentially solve this issue:

    1. Verify File Integrity: Ensure that the model or adapter file you're trying to load is not corrupted. If you downloaded this file, try re-downloading it to ensure it's not a partial or corrupted download.

    2. Compatibility Check: Make sure that the versions of the peft, safetensors, and related packages are compatible with each other. Sometimes, updates or breaking changes can cause functionality to fail.

    3. Specify Correct Path: Double-check the path to the model or adapter to ensure it's correct and accessible from your script.

    4. Update Packages: Ensure that your environment's packages are up-to-date, particularly peft and safetensors. Outdated packages might contain bugs that have been fixed in newer versions.

    pip install --upgrade peft safetensors
    
    1. Catch and Debug: Modify your script to catch exceptions around the failing call, which might give you additional insights or allow you to handle the error gracefully.
    import torch
    from peft.peft_model import PeftModel
    
    def main():
        try:
            # Assuming 'model_id' and 'adapter_name' are correctly defined
            model = PeftModel.from_pretrained(model_id, adapter_name)
            # Add your code logic here
        except Exception as e:
            print(f"Failed to load model: {e}")
    
    if __name__ == "__main__":
        main()
    
    1. Manual Loading: If automated loading continues to fail, consider manually loading the weights into your model if you have access to them and the failure is specifically with the load_adapter process.

    2. Check for Updates or Issues: Look at the official documentation or GitHub issues for peft or safetensors for any known issues or updates related to your problem.

    If these troubleshooting steps do not resolve the issue, consider reaching out for support on the relevant GitHub repositories or forums with detailed information about your setup, versions of the libraries you're using, and the steps you've already tried.

  3. User

    generate_text py

    import os import sys import fire import json import gradio as gr import torch import transformers from peft import PeftModel from transformers import GenerationConfig, LlamaForCausalLM, LlamaTokenizer from transformers import AutoTokenizer, AutoModelForCausalLM from tqdm import tqdm from prompter import Prompter

    if torch.cuda.is_available(): device = "cuda" else: device = "cpu"

    try: if torch.backends.mps.is_available(): device = "mps" except: # noqa: E722 pass

    def main( load_4bit: bool = True, base_model: str = "C:/Users/admin/MjM/mistral7bv2", lora_weights: str = "C:/Users/admin/Mm/check/checkpoint/checkpoint-35000", prompt_template: str = "alpaca", # The prompt template to use, will default to alpaca. input_dir: str = 'C:/Users/admin/MjM/moldata/Molecule-oriented_Instructions/description_guided_molecule_design.json', output_dir: str = 'C:/Users/admin/MjM/moldata/', batch_size: int = 5, max_new_tokens: int = 128, ): base_model = base_model or os.environ.get("BASE_MODEL", "") assert ( base_model ), "Please specify a --base_model, e.g. --base_model='decapoda-research/llama-7b-hf'"

    prompter = Prompter(prompt_template)
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    if device == "cuda":
        model = AutoModelForCausalLM.from_pretrained(
            base_model,
            load_in_8bit=True,
            torch_dtype=torch.float16,
            device_map="auto",
        )
        model = PeftModel.from_pretrained(
            model,
            lora_weights,
            torch_dtype=torch.float16,
        )
    elif device == "mps":
        model = AutoModelForCausalLM.from_pretrained(
            base_model,
            device_map={"": device},
            torch_dtype=torch.float16,
        )
        model = PeftModel.from_pretrained(
            model,
            lora_weights,
            device_map={"": device},
            torch_dtype=torch.float16,
        )
    else:
        model = AutoModelForCausalLM.from_pretrained(
            base_model, device_map={"": device}, low_cpu_mem_usage=True
        )
        model = PeftModel.from_pretrained(
            model,
            lora_weights,
            device_map={"": device},
        )
    
    # unwind broken decapoda-research config
    model.config.pad_token_id = tokenizer.pad_token_id = 0  # unk
    model.config.bos_token_id = 1
    model.config.eos_token_id = 2
    
    if not load_8bit:
        model.half()  # seems to fix bugs for some users.
    
    model.eval()
    if torch.__version__ >= "2" and sys.platform != "win32":
        model = torch.compile(model)
    
    def evaluate(
            instruction,
            input=None,
            temperature=0.1,
            top_p=0.75,
            top_k=40,
            num_beams=4,
            **kwargs,
    ):
        prompt = prompter.generate_prompt(instruction, input)
        inputs = tokenizer(prompt, return_tensors="pt")
        input_ids = inputs["input_ids"].to(device)
        generation_config = GenerationConfig(
            temperature=temperature,
            top_p=top_p,
            top_k=top_k,
            num_beams=num_beams,
            **kwargs,
        )
        with torch.no_grad():
            generation_output = model.generate(
                input_ids=input_ids,
                generation_config=generation_config,
                return_dict_in_generate=True,
                output_scores=True,
                max_new_tokens=max_new_tokens,
            )
        s = generation_output.sequences[0]
        output = tokenizer.decode(s)
        return prompter.get_response(output)
    
    with open(input_dir, 'r') as f:
        data = json.load(f)
    
    with open(output_dir, 'w') as f:
        f.write("description\tground_truth\toutput\n")
        for item in tqdm(data):
            description = item['input']
            ground_truth = item['output']
            output = evaluate(instruction=item['instruction'], input=item['input'])
    
            f.write(f"{description}\t{ground_truth}\t{output}\n")
    

    if name == "main": fire.Fire(main) write code to solve this Traceback (most recent call last): File "C:\Users\admin\Mm\generateText.py", line 136, in fire.Fire(main) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "C:\Users\admin\Mm\generateText.py", line 52, in main model = PeftModel.from_pretrained( File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 353, in from_pretrained model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 694, in load_adapter adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\utils\save_and_load.py", line 326, in load_peft_weights adapters_weights = safe_load_file(filename, device=device) File "C:\Users\admin\Mm\venv\lib\site-packages\safetensors\torch.py", line 308, in load_file with safe_open(filename, framework="pt", device=device) as f: safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization

  4. Assistant

    The error you're encountering, safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization, points to a problem with loading a file, possibly due to a corrupt file, incompatibility, or incorrect usage of the safetensors library within the peft package. Your traceback suggests that the error happens when PeftModel.from_pretrained tries to load adapter weights via load_peft_weights.

    Let's focus on potential solutions or workarounds for this issue:

    1. Verify and Validate File Paths

    Ensure that the lora_weights path points to a valid and uncorrupted file. Incorrect paths or corrupted files can lead to deserialization errors.

    2. Update Dependencies

    Ensure that all related packages (peft, safetensors, and any dependencies they might have) are up to date. Older or incompatible versions may cause issues.

    pip install --upgrade peft safetensors transformers
    

    3. Try Loading Manually (If Applicable)

    If the automatic loading mechanism fails, consider loading the weights manually, if possible. This might require diving into the peft documentation or source code to understand the expected format and process.

    4. Error Handling

    While not a solution, adding error handling around the problematic code can help you bypass or log errors more gracefully, allowing the rest of your script to run or providing more insight into the error.

    try:
        model = PeftModel.from_pretrained(
            model,
            lora_weights,
            torch_dtype=torch.float16,
        )
    except safetensors_rust.SafetensorError as e:
        print(f"Failed to load PEFT weights: {e}")
        # Fallback or alternative code here
    

    5. Check File Integrity

    Try verifying the integrity of the .pt file. This can be done by loading it with basic PyTorch utilities to see if the issue is specific to safetensors or a more general file corruption problem.

    try:
        checkpoint = torch.load(lora_weights, map_location=device)
        # If this succeeds without errors, the file is not corrupt at a basic level.
    except Exception as e:
        print(f"Error loading file directly with PyTorch: {e}")
    

    6. Consult Documentation and Community

    If these steps don't resolve the issue, consult the documentation and community forums or GitHub issues for the peft, safetensors, and transformers libraries. This error might be known, and there could be specific advice or patches available.

    7. Ensure Compatibility

    Ensure that your environment and library versions are all compatible. Sometimes, new versions of libraries like transformers can introduce changes that affect how models and weights need to be loaded, including peft.

    Going Forward

    Since the scripts and libraries involved are complex and depend on external files, a process of elimination can help isolate the issue. If none of these steps resolve the problem, consider creating a minimal reproducible example and seeking support from the library maintainers or the community.

  5. User

    model = PeftModel.from_pretrained( model, lora_weights, torch_dtype=torch.float16, ) PyDev console: starting. Traceback (most recent call last): File "C:\Program Files\JetBrains\PyCharm 2023.1.3\plugins\python\helpers\pydev_pydevd_bundle\pydevd_exec2.py", line 3, in Exec exec(exp, global_vars, local_vars) File "", line 1, in File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 353, in from_pretrained model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 694, in load_adapter adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\utils\save_and_load.py", line 326, in load_peft_weights adapters_weights = safe_load_file(filename, device=device) File "C:\Users\admin\Mm\venv\lib\site-packages\safetensors\torch.py", line 308, in load_file with safe_open(filename, framework="pt", device=device) as f: safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization

  6. Assistant

    The error you’re facing, safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization, suggests a problem with loading a file using the safetensors package, likely due to incorrect file format, corruption, or incompatibility.

    Given the context and provided details, let's address potential solutions more specifically. The error occurs when attempting to load adapter weights in the PEFT model, which could mean that the specified file does not match the expected format or there might be version mismatches between the libraries.

    Verify the File and Path

    Ensure the file specified by lora_weights exists and is accessible. Double-check the path to ensure no typos or incorrect directories.

    Ensure File Integrity and Format

    • File is not corrupted: If the file was downloaded from the internet, try re-downloading it in case the download was incomplete or corrupted.
    • Correct file format: Ensure that the file is indeed in the format expected by safetensors. If this is a file you obtained from a repository or created yourself, verify it was saved correctly and matches expected specifications.

    Compatibility Checks

    Incompatibilities between library versions can lead to unexpected behaviors:

    • Library Versions: Ensure you are using compatible versions of peft, safetensors, transformers, and any other related package. Check the package's documentation or GitHub repository for compatibility notes.
    • Python Version: Make sure your Python version is compatible with the libraries you are using. While less likely to be the issue, it’s still worth verifying.
    pip install --upgrade peft safetensors transformers
    

    Manual Debugging

    Try manually loading the file using tools provided by safetensors or even PyTorch directly (if applicable) to see if the issue might be with the file itself:

    from safetensors.torch import load_file
    try:
        data = load_file("path_to_your_file.pt")
        print("File loaded successfully")
    except Exception as e:
        print(f"Error loading file: {e}")
    

    If this step also results in an error, it's more evidence that the issue lies with the file or its compatibility with the current version of safetensors.

    Seek Support

    If after these troubleshooting steps the issue persists:

    • Consult the documentation: Sometimes, specific versions or configurations are required for things to work as expected.
    • Community and GitHub Issues: Check the peft, safetensors, or related GitHub repositories for similar issues. Someone else might have experienced the same problem, or you could open a new issue detailing your problem.
    • File Format and Library Support: Ensure that safetensors and other libraries you are using support the format of your adapters and model weights correctly. If the format has changed or there's an update, you may need to adjust accordingly.

    Final Notes

    The error seems to indicate a fundamental incompatibility or issue with the file being loaded. Ensure that all paths are correct, files are not corrupted, and that all libraries are up-to-date and compatible. If these checks do not lead to a resolution, reaching out for more targeted support with comprehensive details of what you've tried and the specific configurations you're using will be essential.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.