write code to solve this Traceback (most recent call last): File…
- User
write code to solve this Traceback (most recent call last): File "C:\Users\admin\Mm\generateText.py", line 136, in fire.Fire(main) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "C:\Users\admin\Mm\generateText.py", line 52, in main model = PeftModel.from_pretrained( File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 353, in from_pretrained model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 694, in load_adapter adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\utils\save_and_load.py", line 326, in load_peft_weights adapters_weights = safe_load_file(filename, device=device) File "C:\Users\admin\Mm\venv\lib\site-packages\safetensors\torch.py", line 308, in load_file with safe_open(filename, framework="pt", device=device) as f: safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization
- Assistant
The error you're encountering seems to pertain to a failure in deserializing the header of a saved model or adapter with the
safetensorspackage, which is used by thepeftpackage for loading model weights. This can result from various issues such as a corrupted file, compatibility issues, or an incorrect file path.Below are steps and example code to troubleshoot and potentially solve this issue:
-
Verify File Integrity: Ensure that the model or adapter file you're trying to load is not corrupted. If you downloaded this file, try re-downloading it to ensure it's not a partial or corrupted download.
-
Compatibility Check: Make sure that the versions of the
peft,safetensors, and related packages are compatible with each other. Sometimes, updates or breaking changes can cause functionality to fail. -
Specify Correct Path: Double-check the path to the model or adapter to ensure it's correct and accessible from your script.
-
Update Packages: Ensure that your environment's packages are up-to-date, particularly
peftandsafetensors. Outdated packages might contain bugs that have been fixed in newer versions.
pip install --upgrade peft safetensors- Catch and Debug: Modify your script to catch exceptions around the failing call, which might give you additional insights or allow you to handle the error gracefully.
import torch from peft.peft_model import PeftModel def main(): try: # Assuming 'model_id' and 'adapter_name' are correctly defined model = PeftModel.from_pretrained(model_id, adapter_name) # Add your code logic here except Exception as e: print(f"Failed to load model: {e}") if __name__ == "__main__": main()-
Manual Loading: If automated loading continues to fail, consider manually loading the weights into your model if you have access to them and the failure is specifically with the
load_adapterprocess. -
Check for Updates or Issues: Look at the official documentation or GitHub issues for
peftorsafetensorsfor any known issues or updates related to your problem.
If these troubleshooting steps do not resolve the issue, consider reaching out for support on the relevant GitHub repositories or forums with detailed information about your setup, versions of the libraries you're using, and the steps you've already tried.
-
- User
generate_text py
import os import sys import fire import json import gradio as gr import torch import transformers from peft import PeftModel from transformers import GenerationConfig, LlamaForCausalLM, LlamaTokenizer from transformers import AutoTokenizer, AutoModelForCausalLM from tqdm import tqdm from prompter import Prompter
if torch.cuda.is_available(): device = "cuda" else: device = "cpu"
try: if torch.backends.mps.is_available(): device = "mps" except: # noqa: E722 pass
def main( load_4bit: bool = True, base_model: str = "C:/Users/admin/MjM/mistral7bv2", lora_weights: str = "C:/Users/admin/Mm/check/checkpoint/checkpoint-35000", prompt_template: str = "alpaca", # The prompt template to use, will default to alpaca. input_dir: str = 'C:/Users/admin/MjM/moldata/Molecule-oriented_Instructions/description_guided_molecule_design.json', output_dir: str = 'C:/Users/admin/MjM/moldata/', batch_size: int = 5, max_new_tokens: int = 128, ): base_model = base_model or os.environ.get("BASE_MODEL", "") assert ( base_model ), "Please specify a --base_model, e.g. --base_model='decapoda-research/llama-7b-hf'"
prompter = Prompter(prompt_template) tokenizer = AutoTokenizer.from_pretrained(base_model) if device == "cuda": model = AutoModelForCausalLM.from_pretrained( base_model, load_in_8bit=True, torch_dtype=torch.float16, device_map="auto", ) model = PeftModel.from_pretrained( model, lora_weights, torch_dtype=torch.float16, ) elif device == "mps": model = AutoModelForCausalLM.from_pretrained( base_model, device_map={"": device}, torch_dtype=torch.float16, ) model = PeftModel.from_pretrained( model, lora_weights, device_map={"": device}, torch_dtype=torch.float16, ) else: model = AutoModelForCausalLM.from_pretrained( base_model, device_map={"": device}, low_cpu_mem_usage=True ) model = PeftModel.from_pretrained( model, lora_weights, device_map={"": device}, ) # unwind broken decapoda-research config model.config.pad_token_id = tokenizer.pad_token_id = 0 # unk model.config.bos_token_id = 1 model.config.eos_token_id = 2 if not load_8bit: model.half() # seems to fix bugs for some users. model.eval() if torch.__version__ >= "2" and sys.platform != "win32": model = torch.compile(model) def evaluate( instruction, input=None, temperature=0.1, top_p=0.75, top_k=40, num_beams=4, **kwargs, ): prompt = prompter.generate_prompt(instruction, input) inputs = tokenizer(prompt, return_tensors="pt") input_ids = inputs["input_ids"].to(device) generation_config = GenerationConfig( temperature=temperature, top_p=top_p, top_k=top_k, num_beams=num_beams, **kwargs, ) with torch.no_grad(): generation_output = model.generate( input_ids=input_ids, generation_config=generation_config, return_dict_in_generate=True, output_scores=True, max_new_tokens=max_new_tokens, ) s = generation_output.sequences[0] output = tokenizer.decode(s) return prompter.get_response(output) with open(input_dir, 'r') as f: data = json.load(f) with open(output_dir, 'w') as f: f.write("description\tground_truth\toutput\n") for item in tqdm(data): description = item['input'] ground_truth = item['output'] output = evaluate(instruction=item['instruction'], input=item['input']) f.write(f"{description}\t{ground_truth}\t{output}\n")if name == "main": fire.Fire(main) write code to solve this Traceback (most recent call last): File "C:\Users\admin\Mm\generateText.py", line 136, in fire.Fire(main) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "C:\Users\admin\Mm\venv\lib\site-packages\fire\core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "C:\Users\admin\Mm\generateText.py", line 52, in main model = PeftModel.from_pretrained( File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 353, in from_pretrained model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 694, in load_adapter adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\utils\save_and_load.py", line 326, in load_peft_weights adapters_weights = safe_load_file(filename, device=device) File "C:\Users\admin\Mm\venv\lib\site-packages\safetensors\torch.py", line 308, in load_file with safe_open(filename, framework="pt", device=device) as f: safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization
- Assistant
The error you're encountering,
safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization, points to a problem with loading a file, possibly due to a corrupt file, incompatibility, or incorrect usage of thesafetensorslibrary within thepeftpackage. Your traceback suggests that the error happens whenPeftModel.from_pretrainedtries to load adapter weights viaload_peft_weights.Let's focus on potential solutions or workarounds for this issue:
1. Verify and Validate File Paths
Ensure that the
lora_weightspath points to a valid and uncorrupted file. Incorrect paths or corrupted files can lead to deserialization errors.2. Update Dependencies
Ensure that all related packages (
peft,safetensors, and any dependencies they might have) are up to date. Older or incompatible versions may cause issues.pip install --upgrade peft safetensors transformers3. Try Loading Manually (If Applicable)
If the automatic loading mechanism fails, consider loading the weights manually, if possible. This might require diving into the
peftdocumentation or source code to understand the expected format and process.4. Error Handling
While not a solution, adding error handling around the problematic code can help you bypass or log errors more gracefully, allowing the rest of your script to run or providing more insight into the error.
try: model = PeftModel.from_pretrained( model, lora_weights, torch_dtype=torch.float16, ) except safetensors_rust.SafetensorError as e: print(f"Failed to load PEFT weights: {e}") # Fallback or alternative code here5. Check File Integrity
Try verifying the integrity of the
.ptfile. This can be done by loading it with basic PyTorch utilities to see if the issue is specific tosafetensorsor a more general file corruption problem.try: checkpoint = torch.load(lora_weights, map_location=device) # If this succeeds without errors, the file is not corrupt at a basic level. except Exception as e: print(f"Error loading file directly with PyTorch: {e}")6. Consult Documentation and Community
If these steps don't resolve the issue, consult the documentation and community forums or GitHub issues for the
peft,safetensors, andtransformerslibraries. This error might be known, and there could be specific advice or patches available.7. Ensure Compatibility
Ensure that your environment and library versions are all compatible. Sometimes, new versions of libraries like
transformerscan introduce changes that affect how models and weights need to be loaded, includingpeft.Going Forward
Since the scripts and libraries involved are complex and depend on external files, a process of elimination can help isolate the issue. If none of these steps resolve the problem, consider creating a minimal reproducible example and seeking support from the library maintainers or the community.
- User
model = PeftModel.from_pretrained( model, lora_weights, torch_dtype=torch.float16, ) PyDev console: starting. Traceback (most recent call last): File "C:\Program Files\JetBrains\PyCharm 2023.1.3\plugins\python\helpers\pydev_pydevd_bundle\pydevd_exec2.py", line 3, in Exec exec(exp, global_vars, local_vars) File "", line 1, in File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 353, in from_pretrained model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\peft_model.py", line 694, in load_adapter adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs) File "C:\Users\admin\Mm\venv\lib\site-packages\peft\utils\save_and_load.py", line 326, in load_peft_weights adapters_weights = safe_load_file(filename, device=device) File "C:\Users\admin\Mm\venv\lib\site-packages\safetensors\torch.py", line 308, in load_file with safe_open(filename, framework="pt", device=device) as f: safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization
- Assistant
The error you’re facing,
safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization, suggests a problem with loading a file using thesafetensorspackage, likely due to incorrect file format, corruption, or incompatibility.Given the context and provided details, let's address potential solutions more specifically. The error occurs when attempting to load adapter weights in the PEFT model, which could mean that the specified file does not match the expected format or there might be version mismatches between the libraries.
Verify the File and Path
Ensure the file specified by
lora_weightsexists and is accessible. Double-check the path to ensure no typos or incorrect directories.Ensure File Integrity and Format
- File is not corrupted: If the file was downloaded from the internet, try re-downloading it in case the download was incomplete or corrupted.
- Correct file format: Ensure that the file is indeed in the format expected by
safetensors. If this is a file you obtained from a repository or created yourself, verify it was saved correctly and matches expected specifications.
Compatibility Checks
Incompatibilities between library versions can lead to unexpected behaviors:
- Library Versions: Ensure you are using compatible versions of
peft,safetensors,transformers, and any other related package. Check the package's documentation or GitHub repository for compatibility notes. - Python Version: Make sure your Python version is compatible with the libraries you are using. While less likely to be the issue, it’s still worth verifying.
pip install --upgrade peft safetensors transformersManual Debugging
Try manually loading the file using tools provided by
safetensorsor even PyTorch directly (if applicable) to see if the issue might be with the file itself:from safetensors.torch import load_file try: data = load_file("path_to_your_file.pt") print("File loaded successfully") except Exception as e: print(f"Error loading file: {e}")If this step also results in an error, it's more evidence that the issue lies with the file or its compatibility with the current version of
safetensors.Seek Support
If after these troubleshooting steps the issue persists:
- Consult the documentation: Sometimes, specific versions or configurations are required for things to work as expected.
- Community and GitHub Issues: Check the
peft,safetensors, or related GitHub repositories for similar issues. Someone else might have experienced the same problem, or you could open a new issue detailing your problem. - File Format and Library Support: Ensure that
safetensorsand other libraries you are using support the format of your adapters and model weights correctly. If the format has changed or there's an update, you may need to adjust accordingly.
Final Notes
The error seems to indicate a fundamental incompatibility or issue with the file being loaded. Ensure that all paths are correct, files are not corrupted, and that all libraries are up-to-date and compatible. If these checks do not lead to a resolution, reaching out for more targeted support with comprehensive details of what you've tried and the specific configurations you're using will be essential.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.