I have a "TrOCR" model converted to "onnx", in Python how can I change the script below…
- User
I have a "TrOCR" model converted to "onnx", in Python how can I change the script below to interfere with the "onnx" model, the code is for the original "TrOCR" model. My "onnx" model is hosted on huggingface.co as "gagan3012/Qalam_onnx"
`` from PIL import Image from transformers import TrOCRProcessor, VisionEncoderDecoderModel, DonutProcessor, NougatProcessor import torch
def predict_arabic(img, model_name="UBC-NLP/Qalam"): processor = TrOCRProcessor.from_pretrained(model_name) model = VisionEncoderDecoderModel.from_pretrained(model_name) images = img.convert("RGB") pixel_values = processor(images, return_tensors="pt").pixel_values generated_ids = model.generate(pixel_values, max_length=256) generated_text = processor.batch_decode( generated_ids, skip_special_tokens=True)[0] return generated_text ``
- Assistant
To modify your script to work with the ONNX model hosted on Hugging Face, you will need to change the way you process the image and run inference. The
transformerslibrary doesn't directly support running ONNX models like it does for PyTorch models, so you'll need to useonnxruntimeto run the inference.Here's how you can adjust your script to work with the ONNX model:
- Install the
onnxruntimelibrary if you haven't already:
pip install onnxruntime- Modify the
predict_arabicfunction to utilize theonnxruntimefor inference. Here's the modified code:
from PIL import Image from transformers import TrOCRProcessor import numpy as np import onnxruntime as ort def predict_arabic(img, model_path="gagan3012/Qalam_onnx"): # Load the processor processor = TrOCRProcessor.from_pretrained(model_path) # Prepare the image images = img.convert("RGB") pixel_values = processor(images, return_tensors="pt", padding=True).pixel_values # Convert PyTorch tensor to numpy array for ONNX pixel_values = pixel_values.detach().cpu().numpy() # Load the ONNX model ort_session = ort.InferenceSession(f"{model_path}.onnx") # Prepare input for ONNX model inputs = {ort_session.get_inputs()[0].name: pixel_values} # Run inference generated_ids = ort_session.run(None, inputs)[0] # Decode the generated ids generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0] return generated_textKey Changes Explained:
- Processor Loading: The processor is still loaded from the Hugging Face model path.
- Image Processing: The image is converted and processed similarly. However, the result is transformed to a NumPy array for compatibility with
onnxruntime. - Model Loading: The ONNX model is loaded using
onnxruntime. - Inference: The
runmethod of theonnxruntimesession is called to perform inference. - Text Decoding: Finally, the generated IDs are decoded back to text as before.
Note:
- Ensure that the ONNX model file exists at the specified path and that you've replaced
"gagan3012/Qalam_onnx"with the appropriate path to your ONNX model if needed. - Depending on your model dimensions and input formats, you may need to adjust how the input tensor is structured or how outputs are handled. Ensure that the input format matches what the ONNX model expects.
- Install the
- User
I got this error for your script:
Unused or unrecognized kwargs: padding. Traceback (most recent call last): File "/home/incognito/huggingface-source-py3.10/a.py", line 36, in <module> ocr_text = predict_arabic(cropped_img) File "/home/incognito/huggingface-source-py3.10/a.py", line 19, in predict_arabic ort_session = ort.InferenceSession(f"{model_path}.onnx") File "/home/incognito/huggingface-source-py3.10/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__ self._create_inference_session(providers, provider_options, disabled_optimizers) File "/home/incognito/huggingface-source-py3.10/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 480, in _create_inference_session sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model) onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from gagan3012/Qalam_onnx.onnx failed:Load model gagan3012/Qalam_onnx.onnx failed. File doesn't exist - Assistant
The error message indicates two separate issues:
-
"Unused or unrecognized kwargs: padding." - The
processormethod does not recognize thepaddingargument, which suggests that the version of thetransformerslibrary you are using does not support this option forTrOCRProcessor. Instead, you can ensure that the input is already the appropriate size or handle padding manually. -
"Load model from gagan3012/Qalam_onnx.onnx failed: File doesn't exist." - This indicates that the script is not able to find the ONNX model at the specified path. When using Hugging Face models through the repository API, the model file is not directly accessible via a file path as you would with local files.
Correcting the Issues
Let's address both problems:
-
Remove the Padding Argument: Adjust the image processing step to remove the
paddingargument. -
Loading ONNX Model from Hugging Face Hub: If you want to load the model from the Hugging Face Hub, first download the ONNX model locally and then load it. You can do this with the
transformerslibrary, but you'll avoid using direct paths like you did earlier.
Here's the corrected code:
from PIL import Image from transformers import TrOCRProcessor import numpy as np import onnxruntime as ort import requests import os def download_model(model_name): """Download the ONNX model from Hugging Face Hub if not already present.""" model_path = f"{model_name}.onnx" if not os.path.exists(model_path): # Only download if it's not present url = f"https://huggingface.co/{model_name}/resolve/main/model.onnx" response = requests.get(url) if response.status_code == 200: with open(model_path, 'wb') as f: f.write(response.content) else: raise Exception(f"Failed to download model from {url}, status code: {response.status_code}") return model_path def predict_arabic(img, model_name="gagan3012/Qalam_onnx"): # Load the processor processor = TrOCRProcessor.from_pretrained(model_name) # Prepare the image images = img.convert("RGB") pixel_values = processor(images, return_tensors="pt").pixel_values # Convert PyTorch tensor to numpy array for ONNX pixel_values = pixel_values.detach().cpu().numpy() # Download the ONNX model model_path = download_model(model_name) # Load the ONNX model ort_session = ort.InferenceSession(model_path) # Prepare input for ONNX model inputs = {ort_session.get_inputs()[0].name: pixel_values} # Run inference generated_ids = ort_session.run(None, inputs)[0] # Decode the generated ids generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0] return generated_textKey Changes:
- Download Model: A function (
download_model) is added to download the model if it isn't already on the filesystem. It requests the model from the Hugging Face repository. - Removed Padding Parameter: The code no longer includes the
paddingparameter when processing images. - Updated Model Loading: The model is fetched using the defined download function instead of assuming it exists locally.
Important Notes:
- Model Name: Ensure the model name passed to
download_modelis correct and matches the path structure in your Hugging Face repository. - Internet Access: The script requires internet access to download the ONNX model from Hugging Face.
- Model URI: Ensure the model URI is formatted correctly. Adjust the URL if your ONNX model is stored under a different name in your Hugging Face space.
Final Check:
Make sure your environment has the appropriate permissions to write files when downloading the model. If you encounter further errors, verify the specified model path and the capabilities of your current library versions.
-