Skip to content
TheChatArchiveThe Chat Archive
o1-mini-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

import cv2 from IPython.display import display, Image from PIL import Image as PILImage…

  1. User

    import cv2 from IPython.display import display, Image from PIL import Image as PILImage import numpy as np

    def get_text_segment_bbox(img, full_text, segment_text, original_bbox, color=(0, 0, 255), thickness=1, font=cv2.FONT_HERSHEY_SIMPLEX): """ Calculate the bbox for a segment of text within a larger text string, accounting for word spacing.

    Args:
        img: Input image
        full_text: Complete text string
        segment_text: The portion of text to focus on
        original_bbox: Original bounding box coordinates [x1, y1, x2, y2]
        color: Text and box color in BGR format
        thickness: Line thickness
        font: OpenCV font type
    """
    if segment_text not in full_text:
        raise ValueError("Segment text must be part of the full text")
    
    # Split the text into words and find the target word
    words = full_text.split()
    target_word_index = words.index(segment_text)
    
    # Calculate the font scale that was used for the original bbox
    original_width = original_bbox[2] - original_bbox[0]
    font_scale = 0.1
    max_scale = 100
    tolerance = 2
    
    while font_scale < max_scale:
        (text_width, text_height), baseline = cv2.getTextSize(full_text, font, font_scale, thickness)
        
        if abs(text_width - original_width) < tolerance:
            break
        elif text_width < original_width:
            font_scale *= 1.1
        else:
            font_scale *= 0.9
    
    # Calculate space width
    (space_width, _), _ = cv2.getTextSize(" ", font, font_scale, thickness)
    
    # Calculate width up to our target word
    prefix_width = 0
    for i in range(target_word_index):
        (word_width, _), _ = cv2.getTextSize(words[i], font, font_scale, thickness)
        prefix_width += word_width + space_width
    
    # Get the width of our segment
    (segment_width, segment_height), baseline = cv2.getTextSize(segment_text, font, font_scale, thickness)
    
    # Calculate new bbox coordinates
    new_x1 = original_bbox[0] + prefix_width
    new_x2 = new_x1 + segment_width
    new_y1 = original_bbox[1]
    new_y2 = original_bbox[3]
    
    new_bbox = [int(new_x1), new_y1, int(new_x2), new_y2]
    
    # Draw the new bounding box
    cv2.rectangle(img, (new_bbox[0], new_bbox[1]), (new_bbox[2], new_bbox[3]), color=color, thickness=thickness)
    
    # Calculate position to place text above bbox
    text_height = segment_height + baseline
    y_position = new_bbox[1] - text_height + baseline - 5
    
    # Put the text
    cv2.putText(img, segment_text, (new_bbox[0], y_position), 
                fontFace=font, 
                fontScale=font_scale, 
                color=color, 
                thickness=thickness)
    
    return img, new_bbox
    

    def display_image_with_cropped_text(img_path, full_text, segment_text, original_bbox): # Load the image img = cv2.imread(img_path)

    # Apply text fitting with cropped bbox
    result, new_bbox = get_text_segment_bbox(img, full_text, segment_text, original_bbox)
    
    # Convert to RGB for display
    img_rgb = cv2.cvtColor(result, cv2.COLOR_BGR2RGB)
    pil_img = PILImage.fromarray(img_rgb)
    
    # Display the image
    display(pil_img)
    
    return result, new_bbox
    

    Usage example:

    bbox = [962, 99, 1268, 122] full_text = "AaBbc AaB AaBbCc AaBbccD" segment_text = "AaB" result, new_bbox = display_image_with_cropped_text(img_path, full_text, segment_text, bbox) print(f"New bounding box coordinates: {new_bbox}")

    space between AaB and AaBbc also will be counted

    write the full updated code

  2. Assistant

    Certainly! Below is the complete updated Python code that calculates and draws a bounding box around a specific segment of text within a larger text string in an image. This implementation accounts for the spaces between words when determining the position and size of the bounding box.

    import cv2
    from IPython.display import display
    from PIL import Image as PILImage
    import numpy as np
    
    def get_text_segment_bbox(img, full_text, segment_text, original_bbox, color=(0, 0, 255), thickness=2, font=cv2.FONT_HERSHEY_SIMPLEX):
        """
        Calculate and draw the bounding box for a segment of text within a larger text string,
        accounting for word spacing.
    
        Args:
            img (numpy.ndarray): Input image.
            full_text (str): Complete text string.
            segment_text (str): The portion of text to focus on.
            original_bbox (list): Original bounding box coordinates [x1, y1, x2, y2].
            color (tuple): Bounding box and text color in BGR format. Default is red.
            thickness (int): Line thickness. Default is 2.
            font (int): OpenCV font type. Default is cv2.FONT_HERSHEY_SIMPLEX.
    
        Returns:
            tuple:
                - img (numpy.ndarray): Image with the new bounding box and text drawn.
                - new_bbox (list): New bounding box coordinates [x1, y1, x2, y2].
        """
        if segment_text not in full_text:
            raise ValueError("Segment text must be part of the full text")
    
        # Split the full text into words
        words = full_text.split()
        
        try:
            # Find the index of the target segment
            target_word_index = words.index(segment_text)
        except ValueError:
            raise ValueError(f"Segment text '{segment_text}' not found in the full text.")
    
        # Calculate the original width of the bounding box
        original_width = original_bbox[2] - original_bbox[0]
    
        # Initialize font scale
        font_scale = 0.1
        max_scale = 100
        tolerance = 2
    
        # Dynamically estimate the font scale to match the original bounding box width
        while font_scale < max_scale:
            (text_width, text_height), baseline = cv2.getTextSize(full_text, font, font_scale, thickness)
            if abs(text_width - original_width) <= tolerance:
                break
            elif text_width < original_width:
                font_scale *= 1.1
            else:
                font_scale *= 0.9
    
        # Calculate the width of a space character
        (space_width, _), _ = cv2.getTextSize(" ", font, font_scale, thickness)
    
        # Calculate the cumulative width up to the target word, including spaces
        prefix_width = 0
        for i in range(target_word_index):
            word = words[i]
            (word_width, _), _ = cv2.getTextSize(word, font, font_scale, thickness)
            prefix_width += word_width + space_width  # Add space after each word
    
        # Get the width and height of the segment text
        (segment_width, segment_height), baseline = cv2.getTextSize(segment_text, font, font_scale, thickness)
    
        # Define the new bounding box coordinates
        new_x1 = original_bbox[0] + prefix_width
        new_x2 = new_x1 + segment_width
        new_y1 = original_bbox[1]
        new_y2 = original_bbox[3]
        new_bbox = [int(new_x1), new_y1, int(new_x2), new_y2]
    
        # Draw the new bounding box on the image
        cv2.rectangle(img, (new_bbox[0], new_bbox[1]), (new_bbox[2], new_bbox[3]), color=color, thickness=thickness)
    
        # Position the segment text slightly above the bounding box
        text_position = (new_bbox[0], new_y1 - 10)  # 10 pixels above the box
    
        # Put the segment text on the image
        cv2.putText(img, segment_text, text_position, fontFace=font, fontScale=font_scale, color=color, thickness=thickness, lineType=cv2.LINE_AA)
    
        return img, new_bbox
    
    def display_image_with_cropped_text(img_path, full_text, segment_text, original_bbox):
        """
        Load an image, draw a new bounding box for the segment text, and display the image.
    
        Args:
            img_path (str): Path to the input image.
            full_text (str): Complete text string present in the image.
            segment_text (str): The portion of text to focus on.
            original_bbox (list): Original bounding box coordinates [x1, y1, x2, y2].
    
        Returns:
            tuple:
                - result_img (numpy.ndarray): Image with the new bounding box and text drawn.
                - new_bbox (list): New bounding box coordinates [x1, y1, x2, y2].
        """
        # Load the image using OpenCV
        img = cv2.imread(img_path)
        if img is None:
            raise FileNotFoundError(f"Unable to load image at path: {img_path}")
    
        # Draw the bounding box around the segment text
        result_img, new_bbox = get_text_segment_bbox(img, full_text, segment_text, original_bbox)
    
        # Convert the image from BGR to RGB for display purposes
        img_rgb = cv2.cvtColor(result_img, cv2.COLOR_BGR2RGB)
        pil_img = PILImage.fromarray(img_rgb)
    
        # Display the image using IPython display
        display(pil_img)
    
        return result_img, new_bbox
    
    # =========================
    # Usage Example
    # =========================
    if __name__ == "__main__":
        # Path to the input image file
        img_path = "path_to_your_image.jpg"  # Replace with your actual image path
    
        # Define the original bounding box [x1, y1, x2, y2]
        original_bbox = [962, 99, 1268, 122]
    
        # Define the full text present within the original bounding box
        full_text = "AaBbc AaB AaBbCc AaBbccD"
    
        # Define the segment text you want to highlight
        segment_text = "AaB"
    
        try:
            # Display the image with the cropped text bounding box
            result_image, new_bounding_box = display_image_with_cropped_text(
                img_path=img_path,
                full_text=full_text,
                segment_text=segment_text,
                original_bbox=original_bbox
            )
    
            # Output the new bounding box coordinates
            print(f"New bounding box coordinates for '{segment_text}': {new_bounding_box}")
    
        except Exception as e:
            print(f"An error occurred: {e}")
    

    Explanation of the Code

    1. Function get_text_segment_bbox:

      • Purpose: Calculates the bounding box for a specific segment of text within a larger string and draws this bounding box on the image.
      • Process:
        • Validation: Ensures that the segment_text exists within the full_text.
        • Word Splitting: Splits the full_text into individual words to identify the position of the segment_text.
        • Font Scale Estimation: Dynamically adjusts the font_scale to match the width of the original bounding box (original_bbox). This ensures that the text fits correctly within the provided bounding box.
        • Space Width Calculation: Determines the width of a space character to accurately account for spacing between words.
        • Prefix Width Calculation: Computes the cumulative width of all words preceding the segment_text, including spaces. This determines the starting x-coordinate of the segment_text.
        • Bounding Box Definition: Calculates the new bounding box coordinates based on the position and size of the segment_text.
        • Drawing: Draws the new bounding box and places the segment_text slightly above the box for clarity.
      • Returns: The modified image with the new bounding box and the coordinates of the new bounding box.
    2. Function display_image_with_cropped_text:

      • Purpose: Loads the image, invokes get_text_segment_bbox to draw the bounding box around the segment_text, and displays the resulting image.
      • Process:
        • Image Loading: Uses OpenCV to read the image from the specified path.
        • Bounding Box Drawing: Calls get_text_segment_bbox to compute and draw the bounding box around the desired text segment.
        • Image Conversion: Converts the image from BGR to RGB format, as OpenCV uses BGR by default and PIL expects RGB.
        • Display: Utilizes IPython's display function to show the image within an interactive environment like Jupyter Notebook.
      • Returns: The image with the bounding box and the coordinates of the new bounding box.
    3. Usage Example:

      • Variables:
        • img_path: Replace "path_to_your_image.jpg" with the actual path to your image file.
        • original_bbox: The original bounding box coordinates [x1, y1, x2, y2] where the full_text is located.
        • full_text: The complete string of text within the original bounding box.
        • segment_text: The specific word or segment you want to highlight within the full_text.
      • Execution: Calls display_image_with_cropped_text with the defined parameters and prints out the new bounding box coordinates.

    Important Notes

    • Image Path: Ensure that the img_path variable points to a valid image file on your system. If the image cannot be found or loaded, the code will raise a FileNotFoundError.

    • Environment: The display function from IPython.display is typically used within Jupyter Notebooks or other IPython environments. If you're running this script outside such an environment, you might need to adjust the display method accordingly (e.g., using cv2.imshow for standard Python scripts).

    • Font Scale Adjustment: The font_scale is dynamically adjusted to fit the text within the original bounding box. Depending on the image and text size, you might need to tweak the font_scale estimation parameters (font_scale, max_scale, tolerance) for optimal results.

    • Multiple Occurrences: If the segment_text appears multiple times within the full_text, this code will only highlight the first occurrence. To handle multiple occurrences, additional logic would be necessary to specify which instance to highlight.

    • Space Counting: The code accurately accounts for the spaces between words by calculating the width of a space character and including it in the cumulative width (prefix_width). This ensures that the bounding box aligns correctly with the desired segment, even when multiple spaces or varying word lengths are involved.

    Dependencies

    Make sure you have the required libraries installed. You can install them using pip if they are not already installed:

    pip install opencv-python pillow numpy
    

    Example Output

    After running the script with the appropriate image and parameters, you should see the image displayed with a red bounding box around the specified segment_text. Additionally, the script will print out the new bounding box coordinates. For example:

    New bounding box coordinates for 'AaB': [1120, 99, 1250, 122]
    

    This indicates that the bounding box for "AaB" starts at (1120, 99) and ends at (1250, 122) on the image.

    Feel free to customize the color, thickness, and font parameters to match your preferences or to fit different image qualities and sizes.