import cv2 import numpy as np import torch from transformers import YolosImageProcessor,…
- User
import cv2 import numpy as np import torch from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image
Tải mô hình (nhớ kiểm tra hỗ trợ GPU)
device = 'cuda' if torch.cuda.is_available() else 'cpu' model = YolosForObjectDetection.from_pretrained('yolos-tiny').to(device) image_processor = YolosImageProcessor.from_pretrained("yolos-tiny")
Mở video
cap = cv2.VideoCapture(0)
Xử lý từng khung hình
while True: ret, frame = cap.read() if not ret: break # Kết thúc khi video hết khung hình
# Chuẩn bị dữ liệu images = [] for i in range(4): ret, frame = cap.read() if not ret: break image = Image.fromarray(frame) inputs = image_processor(images=image, return_tensors="pt", size={"height": frame.shape[0], "width": frame.shape[1]}) images.append(inputs["pixel_values"]) # Chuyển đổi sang tensor batch inputs = torch.cat(images, dim=0).to(device) # Phát hiện đối tượng with torch.no_grad(): outputs = model(images=inputs.pixel_values, sizes=inputs.image_sizes) results = image_processor.post_process_object_detection(outputs, threshold=0.9, target_sizes=torch.tensor([image.size[::-1] for image in images])) # Vẽ các khung chứa for i, result in enumerate(results): frame = cv2.cvtColor(np.array(images[i].cpu().image), cv2.COLOR_RGB2BGR) for score, label, box in zip(result["scores"], result["labels"], result["boxes"]): box = [round(i, 2) for i in box.tolist()] cv2.rectangle(frame, (int(box[0]), int(box[1])), (int(box[2]), int(box[3])), (0, 255, 0), 2) cv2.imshow("Video", frame) # Nhấn 'q' để thoát if cv2.waitKey(1) & 0xFF == ord('q'): breakcap.release() cv2.destroyAllWindows()
- Assistant
Your code snippet seems to attempt to read video frames from a webcam and pass them through a YOLOs model for object detection, using the Hugging Face Transformers library. However, there are a few issues that need to be fixed for the code to function properly.
Here's a corrected and streamlined version of your code:
import cv2 import numpy as np import torch from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image # Tải mô hình (nhớ kiểm tra hỗ trợ GPU) device = 'cuda' if torch.cuda.is_available() else 'cpu' model = YolosForObjectDetection.from_pretrained('hustvl/yolos-tiny').to(device) image_processor = YolosImageProcessor.from_pretrained("hustvl/yolos-tiny") # Mở video cap = cv2.VideoCapture(0) # Xử lý từng khung hình while True: ret, frame = cap.read() if not ret: break # Kết thúc khi video hết khung hình # Chuẩn bị dữ liệu image = Image.fromarray(frame[...,::-1]) # Convert BGR (OpenCV) to RGB (PIL) inputs = image_processor(images=image, return_tensors="pt").to(device) pixel_values = inputs["pixel_values"].unsqueeze(0) # Add batch dimension # Phát hiện đối tượng with torch.no_grad(): outputs = model(pixel_values=pixel_values) # Xử lý kết quả đầu ra results = image_processor.post_process_object_detection( outputs, threshold=0.9, target_sizes=inputs["pixel_values"].shape[-2:] )[0] # We take only the first batch item for display # Vẽ các khung chứa for score, label, box in zip(results["scores"], results["labels"], results["boxes"]): box = box.cpu().numpy().astype(np.int32) # Draw rectangle (ensure OpenCV uses (x1, y1), (x2, y2) format) cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2) cv2.imshow("Video", frame) # Nhấn 'q' để thoát if cv2.waitKey(1) & 0xFF == ord('q'): break cap.release() cv2.destroyAllWindows()Some key points to consider:
- Ensure that you're using the correct model name when loading the YOLOs model (e.g.,
'hustvl/yolos-tiny'if that is the correct model repo on Hugging Face). - You only need to read one frame at a time within the while loop, not an extra four frames as was done originally.
- The PIL image must be in RGB format because
YolosImageProcessorexpects it. OpenCV uses BGR by default, hence theframe[...,::-1]to reverse the channels. - You don't need a separate loop to accumulate the images; you're dealing with one frame at a time.
- Remove the
.pixel_valuesattribute when passing inputs to the model. You should directly pass the tensor inpixel_valuesvariable. - The shape for
target_sizesshould match the original image size, but in terms of the tensor shape not the PIL image. - The post-processing should be done for the outputs of the model, with appropriate thresholding.
- When drawing the boxes, they should be converted to NumPy arrays and rounded to integers since OpenCV requires this.
- The box coordinates should be passed in (x1, y1, x2, y2) format to
cv2.rectangle. - Finally, the escape key handling is kept for exiting the loop.
- Ensure that you're using the correct model name when loading the YOLOs model (e.g.,
- User
The
max_sizeparameter is deprecated and will be removed in v4.26. Please specify insize['longest_edge'] instead. Traceback (most recent call last): File "D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien\rrunv.py", line 34, in outputs = model(pixel_values=pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 784, in forward outputs = self.vit( ^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 651, in forward embedding_output = self.embeddings(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 139, in forward batch_size, num_channels, height, width = pixel_values.shape ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: too many values to unpack (expected 4) - Assistant
The error you're receiving indicates that the shape of
pixel_valuesbeing passed to the model does not have the expected four dimensions (batch_size, num_channels, height, width). The error you're seeing occurs when thepixel_valuestensor has more or fewer dimensions than the model expects.Based on the error message, it seems like the model is expecting a 4D tensor but is receiving a tensor with a different number of dimensions. Let's make sure the
pixel_valuestensor has the right dimensions.Here's a revised section of your code to ensure that
pixel_valuesis a 4D tensor when passed to the model and also addresses the warning message onmax_size:# Phát hiện đối tượng with torch.no_grad(): # Ensure inputs is 4D by adding unsqueeze if necessary pixel_values = inputs["pixel_values"].unsqueeze(0) if inputs["pixel_values"].ndim == 3 else inputs["pixel_values"] outputs = model(pixel_values=pixel_values)Additionally, to address the
max_sizedeprecation warning and set the resize configuration, we can modify the image processor call to use thesizeparameter with thelongest_edgekey instead. However, since PIL images don't natively provide a longest_edge method, it may be simpler to define a target size (width, height) by ourselves or use the original image size:# Define a target size target_size = 640 # You may choose an appropriate size for your application # Resize the image and convert color simultaneously frame_rgb_resized = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) frame_rgb_resized = cv2.resize(frame_rgb_resized, (target_size, target_size)) image = Image.fromarray(frame_rgb_resized) # Since size information is no longer required explicitly, we can omit the size parameter now inputs = image_processor(images=image, return_tensors="pt").to(device)Make sure you've removed any deprecated parameters like
max_sizefrom your code, if they were used initially.If you need to use the original frame dimensions sent to the model, make sure those dimensions are compatible with the model input requirements. Some models expect the input size to be a multiple of a certain number or to fit within a specific range of dimensions.
It is important that you ensure that the resizing maintains the aspect ratio or is acceptable to the model you are using; otherwise, your detection results may not be accurate.
- User
import cv2 import numpy as np import torch from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image
Thiết lập GPU
device = 'cuda' if torch.cuda.is_available() else 'cpu'
Tải mô hình
model = YolosForObjectDetection.from_pretrained('yolos-tiny').to(device) image_processor = YolosImageProcessor.from_pretrained("yolos-tiny")
Mở video
cap = cv2.VideoCapture(0)
Kiểm tra kết nối của camera
if not cap.isOpened(): print("Error: Could not open camera.") exit()
while True: ret, frame = cap.read() if not ret: break
# Chuẩn bị dữ liệu image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)) inputs = image_processor(images=image, return_tensors="pt").to(device) pixel_values = inputs["pixel_values"].unsqueeze(0) with torch.no_grad(): outputs = model(pixel_values) # Xử lý kết quả đầu ra results = image_processor.post_process_object_detection( outputs, threshold=0.9, target_sizes=pixel_values.shape[-2:] )[0] for score, label, box in zip(results["scores"], results["labels"], results["boxes"]): box = box.cpu().numpy().astype(np.int32) cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2) label_text = f"Label: {label}, Score: {score:.2f}" cv2.putText(frame, label_text, (box[0], box[1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2) cv2.imshow("Video", frame) if cv2.waitKey(1) & 0xFF == ord('q'): breakcap.release() cv2.destroyAllWindows() Traceback (most recent call last): File "D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien\rrunv.py", line 34, in outputs = model(pixel_values) ^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 784, in forward outputs = self.vit( ^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 651, in forward embedding_output = self.embeddings(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 139, in forward batch_size, num_channels, height, width = pixel_values.shape ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: too many values to unpack (expected 4) - Assistant
Your code is very close to being correct, but it is not handling the dimensions of the tensor correctly when it is passed to the model for inference. Based on your code, you are dealing with one frame at a time, so the
pixel_valuesshould be a 4D tensor with dimensions [1, num_channels, height, width], but the error indicates it is not in that format.Here are the potential issues and how to fix them:
- If the
inputs["pixel_values"]already has the batch dimension as the first dimension, you should not useunsqueeze(0). It may already be a 4D tensor. - Ensure that your
model(pixel_values)call passes the correct input tensor shape.
Given the error, make sure that when you pass
pixel_valuesto the model, it's a 4D tensor. You added an extra dimension withunsqueeze(0), which might have made it a 5D tensor ifinputs["pixel_values"]was already 4D.Here's how you should modify the relevant part of your code:
# Chuẩn bị dữ liệu image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)) inputs = image_processor(images=image, return_tensors="pt").to(device) pixel_values = inputs["pixel_values"] # Ensure pixel_values is 4D (batch size, channels, height, width) if pixel_values.ndim == 3: pixel_values = pixel_values.unsqueeze(0) # Add batch dimension if not present # Phát hiện đối tượng with torch.no_grad(): # If pixel_values is already in the correct shape, this should work outputs = model(pixel_values=pixel_values)Also, ensure you have the latest version of the Transformers library installed. It is crucial to have a compatible version because the newer versions may have fixes and changes to how the model expects input.
To ensure you have the latest version, you can update the library using
pip:pip install --upgrade transformersRemember, proper tensor dimensions are critical when passing data through neural networks. Make sure to always check the expected input size of the pre-trained model from its documentation or an error message.
- If the
- User
import cv2 import numpy as np import torch from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image
Thiết lập GPU
device = 'cuda' if torch.cuda.is_available() else 'cpu'
Tải mô hình
model = YolosForObjectDetection.from_pretrained('yolos-tiny').to(device) image_processor = YolosImageProcessor.from_pretrained("yolos-tiny")
Mở video
cap = cv2.VideoCapture(0)
Kiểm tra kết nối của camera
if not cap.isOpened(): print("Error: Could not open camera.") exit()
while True: ret, frame = cap.read() if not ret: break
# Chuẩn bị dữ liệu image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)) inputs = image_processor(images=image, return_tensors="pt").to(device) pixel_values = inputs["pixel_values"].unsqueeze(0) with torch.no_grad(): outputs = model(pixel_values) # Xử lý kết quả đầu ra results = image_processor.post_process_object_detection( outputs, threshold=0.9, target_sizes=pixel_values.shape[-2:] )[0] for score, label, box in zip(results["scores"], results["labels"], results["boxes"]): box = box.cpu().numpy().astype(np.int32) cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2) label_text = f"Label: {label}, Score: {score:.2f}" cv2.putText(frame, label_text, (box[0], box[1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2) cv2.imshow("Video", frame) if cv2.waitKey(1) & 0xFF == ord('q'): breakcap.release() cv2.destroyAllWindows() ValueError: too many values to unpack (expected 4)
- Assistant
The error you're getting is due to the
pixel_valuestensor not having the expected shape. This is because in the following line:pixel_values = inputs["pixel_values"].unsqueeze(0)you are adding an additional dimension to
pixel_valuespossibly making it a 5D tensor when it should be a 4D tensor. The model expects the input in the format of[batch size, channels, height, width], and theinputsfrom theYolosImageProcessorshould already be in this format.To resolve this issue, you should ensure that
pixel_valuesis a 4D tensor before passing it to the model. Ifinputs["pixel_values"]is already 4D, you should not callunsqueeze(0)on it.Additionally, in your code, you are using the target size of the original image to process the output. The
post_process_object_detectionmethod expects thetarget_sizesparameter to be a list of(height, width)tuples for all images that were processed. If you are only processing one frame at a time, this becomes a single tuple. Let's correct both of these.Here's the corrected code:
# Rest of your setup code... while True: ret, frame = cap.read() if not ret: break # Convert frame to PIL Image and preprocess image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)) inputs = image_processor(images=image, return_tensors="pt").to(device) pixel_values = inputs["pixel_values"] # Remove the unnecessary unsqueeze operation as the processor should handle it # pixel_values = pixel_values.unsqueeze(0) # This line should be removed or commented out with torch.no_grad(): outputs = model(pixel_values=pixel_values) # Process the output results = image_processor.post_process_object_detection( outputs=outputs, threshold=0.9, # When processing one image at a time, extract the single tuple like `(height, width)` target_sizes=inputs["pixel_values"].shape[-2:][::-1] # Reverse from (channels, height, width) to (height, width) )[0] # Get the first batch of results # Your code to display results... cap.release() cv2.destroyAllWindows()With these modifications, the code should work correctly, assuming that you are processing one image at a time (as is typical with webcam input). Please try this updated code and let me know if you encounter any additional errors.
- User
ValueError: Make sure that you pass in as many target sizes as the batch dimension of the logits
- Assistant
The post-processing step where the error is occurring is expecting target sizes corresponding to each image in the batch of the processed images. If you're only processing one frame at a time, the target sizes should still be passed as a list (or converted to a tuple), even though there's only one image.
Given that you're working with batch size 1 (a single image at a time from the camera feed), you still have to make sure that you provide the resized dimensions in the appropriate format.
We have multiple ways to resolve the issue. Here is one solution:
# Rest of your setup code... while True: ret, frame = cap.read() if not ret: break # Convert frame to PIL Image and preprocess image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)) inputs = image_processor(images=image, return_tensors="pt").to(device) pixel_values = inputs["pixel_values"] with torch.no_grad(): outputs = model(pixel_values=pixel_values) # Process the output results = image_processor.post_process_object_detection( outputs=outputs, threshold=0.9, # Ensure target_sizes is a list of one tuple, even for a single image target_sizes=[inputs["image_sizes"][0].tolist()] # The processor returns image sizes as a list of (height, width) )[0] # Get the first batch of results # Your code to display results... cap.release() cv2.destroyAllWindows()By using
inputs["image_sizes"][0].tolist(), we convert the processed image size into a list containing a single tuple of(height, width). This should align the number of target sizes with the number of images in the batch (batch dimension) as expected by the method. Theimage_sizesfield should be a 2D tensor with shape[batch_size, 2]where2corresponds to[height, width].Please try the updated code snippet, and it should resolve the
ValueErrorregardingtarget sizes. - User
PS D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien> py .\rrunv.py The
max_sizeparameter is deprecated and will be removed in v4.26. Please specify insize['longest_edge'] instead. Traceback (most recent call last): File "D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien\rrunv.py", line 40, in target_sizes=[inputs["image_sizes"][0].tolist()] # The processor returns image sizes as a list of (height, width) ~~~~~~^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\feature_extraction_utils.py", line 86, in getitem return self.data[item] ~~~~~~~~~^^^^^^ KeyError: 'image_sizes' - Assistant
The error message suggests that the
inputsdictionary does not have animage_sizeskey, which is why attempting to accessinputs["image_sizes"]is raising aKeyError. It appears that the way we are trying to extract the image size information is not right according to the API of theYolosImageProcessor.Based on the warning message you received about the
max_sizeparameter, we should adjust the size as mentioned in the warning and ignore theimage_sizespart.The corrected code for image preprocessing and post-processing object detection should look something like this:
# Rest of your setup code... while True: ret, frame = cap.read() if not ret: break # Convert frame to PIL Image and preprocess with the longest edge specified image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)) # Use size parameter with longest_edge instead inputs = image_processor(images=image, return_tensors="pt", size={"longest_edge": 384}).to(device) pixel_values = inputs["pixel_values"] target_size = pixel_values.shape[-2:] # Width and Height of processed image tensor with torch.no_grad(): outputs = model(pixel_values=pixel_values) # Process the output (specify target size directly since size info is not returned) results = image_processor.post_process_object_detection( outputs=outputs, threshold=0.9, target_sizes=np.array([target_size]) # The model expects an np.array of target_sizes )[0] # Get the first batch of results # Your code to display results... cap.release() cv2.destroyAllWindows()In this updated code, you will notice that I specified the size with
{"longest_edge": 384}in theimage_processorcall. You can adjust the"longest_edge"value to suit the model's input size requirement.Also, I'm now using
target_sizes=np.array([target_size])which is expected to be annp.arrayof sizes. Thetarget_sizeis retrieved directly from thepixel_valuestensor shape of the processed image.Please try this updated code and check if it resolves your error. Remember that the
longest_edgeyou specify should be appropriate for the model you're using. Adjust it to the recommended input dimensions to ensure the model operates effectively.