Skip to content
TheChatArchiveThe Chat Archive
gpt-4o-mini-2024-07-18· OpenAIAi2 WildChat-4.8M (AI2)

I have a "TrOCR" model converted to "onnx", in Python how can I change the script below…

  1. User

    I have a "TrOCR" model converted to "onnx", in Python how can I change the script below to interfere with the "onnx" model, the code is for the original "TrOCR" model. My "onnx" model is hosted on huggingface.co as "gagan3012/Qalam_onnx"

    `` from PIL import Image from transformers import TrOCRProcessor, VisionEncoderDecoderModel, DonutProcessor, NougatProcessor import torch

    def predict_arabic(img, model_name="UBC-NLP/Qalam"): processor = TrOCRProcessor.from_pretrained(model_name) model = VisionEncoderDecoderModel.from_pretrained(model_name) images = img.convert("RGB") pixel_values = processor(images, return_tensors="pt").pixel_values generated_ids = model.generate(pixel_values, max_length=256) generated_text = processor.batch_decode( generated_ids, skip_special_tokens=True)[0] return generated_text ``

  2. Assistant

    To modify your script to work with the ONNX model hosted on Hugging Face, you will need to change the way you process the image and run inference. The transformers library doesn't directly support running ONNX models like it does for PyTorch models, so you'll need to use onnxruntime to run the inference.

    Here's how you can adjust your script to work with the ONNX model:

    1. Install the onnxruntime library if you haven't already:
    pip install onnxruntime
    
    1. Modify the predict_arabic function to utilize the onnxruntime for inference. Here's the modified code:
    from PIL import Image
    from transformers import TrOCRProcessor
    import numpy as np
    import onnxruntime as ort
    
    def predict_arabic(img, model_path="gagan3012/Qalam_onnx"):
        # Load the processor
        processor = TrOCRProcessor.from_pretrained(model_path)
    
        # Prepare the image
        images = img.convert("RGB")
        pixel_values = processor(images, return_tensors="pt", padding=True).pixel_values
    
        # Convert PyTorch tensor to numpy array for ONNX
        pixel_values = pixel_values.detach().cpu().numpy()
    
        # Load the ONNX model
        ort_session = ort.InferenceSession(f"{model_path}.onnx")
    
        # Prepare input for ONNX model
        inputs = {ort_session.get_inputs()[0].name: pixel_values}
    
        # Run inference
        generated_ids = ort_session.run(None, inputs)[0]
    
        # Decode the generated ids
        generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
    
        return generated_text
    

    Key Changes Explained:

    1. Processor Loading: The processor is still loaded from the Hugging Face model path.
    2. Image Processing: The image is converted and processed similarly. However, the result is transformed to a NumPy array for compatibility with onnxruntime.
    3. Model Loading: The ONNX model is loaded using onnxruntime.
    4. Inference: The run method of the onnxruntime session is called to perform inference.
    5. Text Decoding: Finally, the generated IDs are decoded back to text as before.

    Note:

    • Ensure that the ONNX model file exists at the specified path and that you've replaced "gagan3012/Qalam_onnx" with the appropriate path to your ONNX model if needed.
    • Depending on your model dimensions and input formats, you may need to adjust how the input tensor is structured or how outputs are handled. Ensure that the input format matches what the ONNX model expects.
  3. User

    I got this error for your script: Unused or unrecognized kwargs: padding. Traceback (most recent call last): File "/home/incognito/huggingface-source-py3.10/a.py", line 36, in <module> ocr_text = predict_arabic(cropped_img) File "/home/incognito/huggingface-source-py3.10/a.py", line 19, in predict_arabic ort_session = ort.InferenceSession(f"{model_path}.onnx") File "/home/incognito/huggingface-source-py3.10/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__ self._create_inference_session(providers, provider_options, disabled_optimizers) File "/home/incognito/huggingface-source-py3.10/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 480, in _create_inference_session sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model) onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from gagan3012/Qalam_onnx.onnx failed:Load model gagan3012/Qalam_onnx.onnx failed. File doesn't exist

  4. Assistant

    The error message indicates two separate issues:

    1. "Unused or unrecognized kwargs: padding." - The processor method does not recognize the padding argument, which suggests that the version of the transformers library you are using does not support this option for TrOCRProcessor. Instead, you can ensure that the input is already the appropriate size or handle padding manually.

    2. "Load model from gagan3012/Qalam_onnx.onnx failed: File doesn't exist." - This indicates that the script is not able to find the ONNX model at the specified path. When using Hugging Face models through the repository API, the model file is not directly accessible via a file path as you would with local files.

    Correcting the Issues

    Let's address both problems:

    1. Remove the Padding Argument: Adjust the image processing step to remove the padding argument.

    2. Loading ONNX Model from Hugging Face Hub: If you want to load the model from the Hugging Face Hub, first download the ONNX model locally and then load it. You can do this with the transformers library, but you'll avoid using direct paths like you did earlier.

    Here's the corrected code:

    from PIL import Image
    from transformers import TrOCRProcessor
    import numpy as np
    import onnxruntime as ort
    import requests
    import os
    
    def download_model(model_name):
        """Download the ONNX model from Hugging Face Hub if not already present."""
        model_path = f"{model_name}.onnx"
        if not os.path.exists(model_path):
            # Only download if it's not present
            url = f"https://huggingface.co/{model_name}/resolve/main/model.onnx"
            response = requests.get(url)
            if response.status_code == 200:
                with open(model_path, 'wb') as f:
                    f.write(response.content)
            else:
                raise Exception(f"Failed to download model from {url}, status code: {response.status_code}")
        return model_path
    
    def predict_arabic(img, model_name="gagan3012/Qalam_onnx"):
        # Load the processor
        processor = TrOCRProcessor.from_pretrained(model_name)
    
        # Prepare the image
        images = img.convert("RGB")
        pixel_values = processor(images, return_tensors="pt").pixel_values
    
        # Convert PyTorch tensor to numpy array for ONNX
        pixel_values = pixel_values.detach().cpu().numpy()
    
        # Download the ONNX model
        model_path = download_model(model_name)
    
        # Load the ONNX model
        ort_session = ort.InferenceSession(model_path)
    
        # Prepare input for ONNX model
        inputs = {ort_session.get_inputs()[0].name: pixel_values}
    
        # Run inference
        generated_ids = ort_session.run(None, inputs)[0]
    
        # Decode the generated ids
        generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
    
        return generated_text
    

    Key Changes:

    1. Download Model: A function (download_model) is added to download the model if it isn't already on the filesystem. It requests the model from the Hugging Face repository.
    2. Removed Padding Parameter: The code no longer includes the padding parameter when processing images.
    3. Updated Model Loading: The model is fetched using the defined download function instead of assuming it exists locally.

    Important Notes:

    • Model Name: Ensure the model name passed to download_model is correct and matches the path structure in your Hugging Face repository.
    • Internet Access: The script requires internet access to download the ONNX model from Hugging Face.
    • Model URI: Ensure the model URI is formatted correctly. Adjust the URL if your ONNX model is stored under a different name in your Hugging Face space.

    Final Check:

    Make sure your environment has the appropriate permissions to write files when downloading the model. If you encounter further errors, verify the specified model path and the capabilities of your current library versions.