Skip to content
TheChatArchiveThe Chat Archive
gpt-4-1106-preview· openaiHugging Face Suzhen/CodeChat

import cv2 import numpy as np import torch from transformers import YolosImageProcessor,…

  1. User

    import cv2 import numpy as np import torch from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image

    Tải mô hình (nhớ kiểm tra hỗ trợ GPU)

    device = 'cuda' if torch.cuda.is_available() else 'cpu' model = YolosForObjectDetection.from_pretrained('yolos-tiny').to(device) image_processor = YolosImageProcessor.from_pretrained("yolos-tiny")

    Mở video

    cap = cv2.VideoCapture(0)

    Xử lý từng khung hình

    while True: ret, frame = cap.read() if not ret: break # Kết thúc khi video hết khung hình

    # Chuẩn bị dữ liệu
    images = []
    for i in range(4):
        ret, frame = cap.read()
        if not ret:
            break
        image = Image.fromarray(frame)
        inputs = image_processor(images=image, return_tensors="pt", size={"height": frame.shape[0], "width": frame.shape[1]})
    
    
    
        images.append(inputs["pixel_values"])
    
    # Chuyển đổi sang tensor batch
    inputs = torch.cat(images, dim=0).to(device)
    
    # Phát hiện đối tượng
    with torch.no_grad():
        outputs = model(images=inputs.pixel_values, sizes=inputs.image_sizes)
    
        results = image_processor.post_process_object_detection(outputs, threshold=0.9, target_sizes=torch.tensor([image.size[::-1] for image in images]))
    
    # Vẽ các khung chứa
    for i, result in enumerate(results):
        frame = cv2.cvtColor(np.array(images[i].cpu().image), cv2.COLOR_RGB2BGR)
        for score, label, box in zip(result["scores"], result["labels"], result["boxes"]):
            box = [round(i, 2) for i in box.tolist()]
            cv2.rectangle(frame, (int(box[0]), int(box[1])), (int(box[2]), int(box[3])), (0, 255, 0), 2)
        cv2.imshow("Video", frame)
    
    
    
    # Nhấn 'q' để thoát
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    

    cap.release() cv2.destroyAllWindows()

  2. Assistant

    Your code snippet seems to attempt to read video frames from a webcam and pass them through a YOLOs model for object detection, using the Hugging Face Transformers library. However, there are a few issues that need to be fixed for the code to function properly.

    Here's a corrected and streamlined version of your code:

    import cv2
    import numpy as np
    import torch
    from transformers import YolosImageProcessor, YolosForObjectDetection
    from PIL import Image
    
    # Tải mô hình (nhớ kiểm tra hỗ trợ GPU)
    device = 'cuda' if torch.cuda.is_available() else 'cpu' 
    model = YolosForObjectDetection.from_pretrained('hustvl/yolos-tiny').to(device) 
    image_processor = YolosImageProcessor.from_pretrained("hustvl/yolos-tiny")
    
    # Mở video
    cap = cv2.VideoCapture(0)
    
    # Xử lý từng khung hình
    while True:
        ret, frame = cap.read()
        if not ret:
            break  # Kết thúc khi video hết khung hình
    
        # Chuẩn bị dữ liệu
        image = Image.fromarray(frame[...,::-1])  # Convert BGR (OpenCV) to RGB (PIL)
        inputs = image_processor(images=image, return_tensors="pt").to(device)
        pixel_values = inputs["pixel_values"].unsqueeze(0)  # Add batch dimension
    
        # Phát hiện đối tượng
        with torch.no_grad():
            outputs = model(pixel_values=pixel_values)
    
        # Xử lý kết quả đầu ra
        results = image_processor.post_process_object_detection(
            outputs, 
            threshold=0.9, 
            target_sizes=inputs["pixel_values"].shape[-2:]
        )[0]  # We take only the first batch item for display
    
        # Vẽ các khung chứa
        for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
            box = box.cpu().numpy().astype(np.int32)
            # Draw rectangle (ensure OpenCV uses (x1, y1), (x2, y2) format)
            cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2)
        cv2.imshow("Video", frame)
    
        # Nhấn 'q' để thoát
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    
    cap.release()
    cv2.destroyAllWindows()
    

    Some key points to consider:

    1. Ensure that you're using the correct model name when loading the YOLOs model (e.g., 'hustvl/yolos-tiny' if that is the correct model repo on Hugging Face).
    2. You only need to read one frame at a time within the while loop, not an extra four frames as was done originally.
    3. The PIL image must be in RGB format because YolosImageProcessor expects it. OpenCV uses BGR by default, hence the frame[...,::-1] to reverse the channels.
    4. You don't need a separate loop to accumulate the images; you're dealing with one frame at a time.
    5. Remove the .pixel_values attribute when passing inputs to the model. You should directly pass the tensor in pixel_values variable.
    6. The shape for target_sizes should match the original image size, but in terms of the tensor shape not the PIL image.
    7. The post-processing should be done for the outputs of the model, with appropriate thresholding.
    8. When drawing the boxes, they should be converted to NumPy arrays and rounded to integers since OpenCV requires this.
    9. The box coordinates should be passed in (x1, y1, x2, y2) format to cv2.rectangle.
    10. Finally, the escape key handling is kept for exiting the loop.
  3. User

    The max_size parameter is deprecated and will be removed in v4.26. Please specify in size['longest_edge'] instead. Traceback (most recent call last): File "D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien\rrunv.py", line 34, in outputs = model(pixel_values=pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 784, in forward outputs = self.vit( ^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 651, in forward embedding_output = self.embeddings(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 139, in forward batch_size, num_channels, height, width = pixel_values.shape ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: too many values to unpack (expected 4)

  4. Assistant

    The error you're receiving indicates that the shape of pixel_values being passed to the model does not have the expected four dimensions (batch_size, num_channels, height, width). The error you're seeing occurs when the pixel_values tensor has more or fewer dimensions than the model expects.

    Based on the error message, it seems like the model is expecting a 4D tensor but is receiving a tensor with a different number of dimensions. Let's make sure the pixel_values tensor has the right dimensions.

    Here's a revised section of your code to ensure that pixel_values is a 4D tensor when passed to the model and also addresses the warning message on max_size:

    # Phát hiện đối tượng
    with torch.no_grad():
        # Ensure inputs is 4D by adding unsqueeze if necessary
        pixel_values = inputs["pixel_values"].unsqueeze(0) if inputs["pixel_values"].ndim == 3 else inputs["pixel_values"]
        outputs = model(pixel_values=pixel_values)
    

    Additionally, to address the max_size deprecation warning and set the resize configuration, we can modify the image processor call to use the size parameter with the longest_edge key instead. However, since PIL images don't natively provide a longest_edge method, it may be simpler to define a target size (width, height) by ourselves or use the original image size:

    # Define a target size
    target_size = 640  # You may choose an appropriate size for your application
    
    # Resize the image and convert color simultaneously
    frame_rgb_resized = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    frame_rgb_resized = cv2.resize(frame_rgb_resized, (target_size, target_size))
    
    image = Image.fromarray(frame_rgb_resized)
    # Since size information is no longer required explicitly, we can omit the size parameter now
    inputs = image_processor(images=image, return_tensors="pt").to(device)
    

    Make sure you've removed any deprecated parameters like max_size from your code, if they were used initially.

    If you need to use the original frame dimensions sent to the model, make sure those dimensions are compatible with the model input requirements. Some models expect the input size to be a multiple of a certain number or to fit within a specific range of dimensions.

    It is important that you ensure that the resizing maintains the aspect ratio or is acceptable to the model you are using; otherwise, your detection results may not be accurate.

  5. User

    import cv2 import numpy as np import torch from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image

    Thiết lập GPU

    device = 'cuda' if torch.cuda.is_available() else 'cpu'

    Tải mô hình

    model = YolosForObjectDetection.from_pretrained('yolos-tiny').to(device) image_processor = YolosImageProcessor.from_pretrained("yolos-tiny")

    Mở video

    cap = cv2.VideoCapture(0)

    Kiểm tra kết nối của camera

    if not cap.isOpened(): print("Error: Could not open camera.") exit()

    while True: ret, frame = cap.read() if not ret: break

    # Chuẩn bị dữ liệu
    image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))  
    inputs = image_processor(images=image, return_tensors="pt").to(device)
    pixel_values = inputs["pixel_values"].unsqueeze(0) 
    
    
    with torch.no_grad():
        outputs = model(pixel_values)
    
    # Xử lý kết quả đầu ra
    results = image_processor.post_process_object_detection(
        outputs,
        threshold=0.9,
        target_sizes=pixel_values.shape[-2:]
    )[0] 
    
    for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
        box = box.cpu().numpy().astype(np.int32)
        cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2)
        
        label_text = f"Label: {label}, Score: {score:.2f}"
        cv2.putText(frame, label_text, (box[0], box[1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
    
    cv2.imshow("Video", frame)
    
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    

    cap.release() cv2.destroyAllWindows() Traceback (most recent call last): File "D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien\rrunv.py", line 34, in outputs = model(pixel_values) ^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 784, in forward outputs = self.vit( ^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 651, in forward embedding_output = self.embeddings(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\models\yolos\modeling_yolos.py", line 139, in forward batch_size, num_channels, height, width = pixel_values.shape ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: too many values to unpack (expected 4)

  6. Assistant

    Your code is very close to being correct, but it is not handling the dimensions of the tensor correctly when it is passed to the model for inference. Based on your code, you are dealing with one frame at a time, so the pixel_values should be a 4D tensor with dimensions [1, num_channels, height, width], but the error indicates it is not in that format.

    Here are the potential issues and how to fix them:

    1. If the inputs["pixel_values"] already has the batch dimension as the first dimension, you should not use unsqueeze(0). It may already be a 4D tensor.
    2. Ensure that your model(pixel_values) call passes the correct input tensor shape.

    Given the error, make sure that when you pass pixel_values to the model, it's a 4D tensor. You added an extra dimension with unsqueeze(0), which might have made it a 5D tensor if inputs["pixel_values"] was already 4D.

    Here's how you should modify the relevant part of your code:

    # Chuẩn bị dữ liệu
    image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))  
    inputs = image_processor(images=image, return_tensors="pt").to(device)
    pixel_values = inputs["pixel_values"]
    
    # Ensure pixel_values is 4D (batch size, channels, height, width)
    if pixel_values.ndim == 3:
        pixel_values = pixel_values.unsqueeze(0)  # Add batch dimension if not present
    
    # Phát hiện đối tượng
    with torch.no_grad():
        # If pixel_values is already in the correct shape, this should work
        outputs = model(pixel_values=pixel_values)
    

    Also, ensure you have the latest version of the Transformers library installed. It is crucial to have a compatible version because the newer versions may have fixes and changes to how the model expects input.

    To ensure you have the latest version, you can update the library using pip:

    pip install --upgrade transformers
    

    Remember, proper tensor dimensions are critical when passing data through neural networks. Make sure to always check the expected input size of the pre-trained model from its documentation or an error message.

  7. User

    import cv2 import numpy as np import torch from transformers import YolosImageProcessor, YolosForObjectDetection from PIL import Image

    Thiết lập GPU

    device = 'cuda' if torch.cuda.is_available() else 'cpu'

    Tải mô hình

    model = YolosForObjectDetection.from_pretrained('yolos-tiny').to(device) image_processor = YolosImageProcessor.from_pretrained("yolos-tiny")

    Mở video

    cap = cv2.VideoCapture(0)

    Kiểm tra kết nối của camera

    if not cap.isOpened(): print("Error: Could not open camera.") exit()

    while True: ret, frame = cap.read() if not ret: break

    # Chuẩn bị dữ liệu
    image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))  
    inputs = image_processor(images=image, return_tensors="pt").to(device)
    pixel_values = inputs["pixel_values"].unsqueeze(0) 
    
    
    with torch.no_grad():
        outputs = model(pixel_values)
    
    # Xử lý kết quả đầu ra
    results = image_processor.post_process_object_detection(
        outputs,
        threshold=0.9,
        target_sizes=pixel_values.shape[-2:]
    )[0] 
    
    for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
        box = box.cpu().numpy().astype(np.int32)
        cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2)
        
        label_text = f"Label: {label}, Score: {score:.2f}"
        cv2.putText(frame, label_text, (box[0], box[1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
    
    cv2.imshow("Video", frame)
    
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    

    cap.release() cv2.destroyAllWindows() ValueError: too many values to unpack (expected 4)

  8. Assistant

    The error you're getting is due to the pixel_values tensor not having the expected shape. This is because in the following line:

    pixel_values = inputs["pixel_values"].unsqueeze(0)
    

    you are adding an additional dimension to pixel_values possibly making it a 5D tensor when it should be a 4D tensor. The model expects the input in the format of [batch size, channels, height, width], and the inputs from the YolosImageProcessor should already be in this format.

    To resolve this issue, you should ensure that pixel_values is a 4D tensor before passing it to the model. If inputs["pixel_values"] is already 4D, you should not call unsqueeze(0) on it.

    Additionally, in your code, you are using the target size of the original image to process the output. The post_process_object_detection method expects the target_sizes parameter to be a list of (height, width) tuples for all images that were processed. If you are only processing one frame at a time, this becomes a single tuple. Let's correct both of these.

    Here's the corrected code:

    # Rest of your setup code...
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
        
        # Convert frame to PIL Image and preprocess
        image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
        inputs = image_processor(images=image, return_tensors="pt").to(device)
        pixel_values = inputs["pixel_values"]
        
        # Remove the unnecessary unsqueeze operation as the processor should handle it
        # pixel_values = pixel_values.unsqueeze(0) # This line should be removed or commented out
    
        with torch.no_grad():
            outputs = model(pixel_values=pixel_values)
    
        # Process the output
        results = image_processor.post_process_object_detection(
            outputs=outputs,
            threshold=0.9,
            # When processing one image at a time, extract the single tuple like `(height, width)`
            target_sizes=inputs["pixel_values"].shape[-2:][::-1]  # Reverse from (channels, height, width) to (height, width)
        )[0]  # Get the first batch of results
    
        # Your code to display results...
    
    cap.release()
    cv2.destroyAllWindows()
    

    With these modifications, the code should work correctly, assuming that you are processing one image at a time (as is typical with webcam input). Please try this updated code and let me know if you encounter any additional errors.

  9. User

    ValueError: Make sure that you pass in as many target sizes as the batch dimension of the logits

  10. Assistant

    The post-processing step where the error is occurring is expecting target sizes corresponding to each image in the batch of the processed images. If you're only processing one frame at a time, the target sizes should still be passed as a list (or converted to a tuple), even though there's only one image.

    Given that you're working with batch size 1 (a single image at a time from the camera feed), you still have to make sure that you provide the resized dimensions in the appropriate format.

    We have multiple ways to resolve the issue. Here is one solution:

    # Rest of your setup code...
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
        
        # Convert frame to PIL Image and preprocess
        image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
        inputs = image_processor(images=image, return_tensors="pt").to(device)
        pixel_values = inputs["pixel_values"]
        
        with torch.no_grad():
            outputs = model(pixel_values=pixel_values)
    
        # Process the output
        results = image_processor.post_process_object_detection(
            outputs=outputs,
            threshold=0.9,
            # Ensure target_sizes is a list of one tuple, even for a single image
            target_sizes=[inputs["image_sizes"][0].tolist()]  # The processor returns image sizes as a list of (height, width)
        )[0]  # Get the first batch of results
    
        # Your code to display results...
    
    cap.release()
    cv2.destroyAllWindows()
    

    By using inputs["image_sizes"][0].tolist(), we convert the processed image size into a list containing a single tuple of (height, width). This should align the number of target sizes with the number of images in the batch (batch dimension) as expected by the method. The image_sizes field should be a 2D tensor with shape [batch_size, 2] where 2 corresponds to [height, width].

    Please try the updated code snippet, and it should resolve the ValueError regarding target sizes.

  11. User

    PS D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien> py .\rrunv.py The max_size parameter is deprecated and will be removed in v4.26. Please specify in size['longest_edge'] instead. Traceback (most recent call last): File "D:\Tailieuhoctap\thi giac may tinh\phanloaiphathien\rrunv.py", line 40, in target_sizes=[inputs["image_sizes"][0].tolist()] # The processor returns image sizes as a list of (height, width) ~~~~~~^^^^^^^^^^^^^^^ File "D:\Application\python\Lib\site-packages\transformers\feature_extraction_utils.py", line 86, in getitem return self.data[item] ~~~~~~~~~^^^^^^ KeyError: 'image_sizes'

  12. Assistant

    The error message suggests that the inputs dictionary does not have an image_sizes key, which is why attempting to access inputs["image_sizes"] is raising a KeyError. It appears that the way we are trying to extract the image size information is not right according to the API of the YolosImageProcessor.

    Based on the warning message you received about the max_size parameter, we should adjust the size as mentioned in the warning and ignore the image_sizes part.

    The corrected code for image preprocessing and post-processing object detection should look something like this:

    # Rest of your setup code...
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
        
        # Convert frame to PIL Image and preprocess with the longest edge specified
        image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
        
        # Use size parameter with longest_edge instead
        inputs = image_processor(images=image, return_tensors="pt", size={"longest_edge": 384}).to(device)
        pixel_values = inputs["pixel_values"]
        target_size = pixel_values.shape[-2:]  # Width and Height of processed image tensor
        
        with torch.no_grad():
            outputs = model(pixel_values=pixel_values)
    
        # Process the output (specify target size directly since size info is not returned)
        results = image_processor.post_process_object_detection(
            outputs=outputs,
            threshold=0.9,
            target_sizes=np.array([target_size])  # The model expects an np.array of target_sizes
        )[0]  # Get the first batch of results
    
        # Your code to display results...
    
    cap.release()
    cv2.destroyAllWindows()
    

    In this updated code, you will notice that I specified the size with {"longest_edge": 384} in the image_processor call. You can adjust the "longest_edge" value to suit the model's input size requirement.

    Also, I'm now using target_sizes=np.array([target_size]) which is expected to be an np.array of sizes. The target_size is retrieved directly from the pixel_values tensor shape of the processed image.

    Please try this updated code and check if it resolves your error. Remember that the longest_edge you specify should be appropriate for the model you're using. Adjust it to the recommended input dimensions to ensure the model operates effectively.