write a python script that's able to find a template picture in another picture even if…
- User
write a python script that's able to find a template picture in another picture even if it's a little bit different angled
- Assistant
To find a template picture within another picture, even if it's slightly rotated or angled, we can make use of the feature matching technique in computer vision. OpenCV is a popular library in Python that has the tools necessary to accomplish this task.
We'll be using the SIFT (Scale-Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF) algorithm to detect the keypoints and compute the descriptors. Then we'll use a matcher such as FLANN (Fast Library for Approximate Nearest Neighbors) or BFMatcher (Brute-Force Matcher) to match these features and find the best matches to locate the template in the image.
Here's a sample script that does this using OpenCV:
import cv2 import numpy as np # Read the template and the image where the template will be searched template_image_path = 'template.jpg' search_image_path = 'search_image.jpg' template = cv2.imread(template_image_path, 0) # Reading in grayscale search_image = cv2.imread(search_image_path, 0) # Reading in grayscale # Initiate SIFT detector (or use ORB if SIFT is not available) sift = cv2.SIFT_create() # sift = cv2.ORB_create() # Find the keypoints and descriptors with SIFT or ORB keypoints_template, descriptors_template = sift.detectAndCompute(template, None) keypoints_search_image, descriptors_search_image = sift.detectAndCompute(search_image, None) # Create the matcher object # For SIFT or SURF, which are float descriptors, use FLANN with the following parameters FLANN_INDEX_KDTREE = 1 index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5) search_params = dict(checks=50) flann = cv2.FlannBasedMatcher(index_params, search_params) matches = flann.knnMatch(descriptors_template, descriptors_search_image, k=2) # Lowe's ratio test to filter good matches good_matches = [] for m, n in matches: if m.distance < 0.75 * n.distance: good_matches.append(m) if len(good_matches) > 10: # Arbitrary number, can adjust based on the use case # Extract location of good matches points_template = np.float32([keypoints_template[m.queryIdx].pt for m in good_matches]) points_search_image = np.float32([keypoints_search_image[m.trainIdx].pt for m in good_matches]) # Find homography matrix, mask = cv2.findHomography(points_template, points_search_image, cv2.RANSAC, 5.0) matches_mask = mask.ravel().tolist() # Get the dimensions of the template image for perspective transform h, w = template.shape points = np.float32([[0, 0], [0, h], [w, h], [w, 0]]).reshape(-1, 1, 2) # Apply perspective transform to the points transformed_points = cv2.perspectiveTransform(points, matrix) search_image_color = cv2.cvtColor(search_image, cv2.COLOR_GRAY2BGR) cv2.polylines(search_image_color, [np.int32(transformed_points)], True, (0, 255, 0), 3, cv2.LINE_AA) cv2.imshow('Detected', search_image_color) cv2.waitKey(0) cv2.destroyAllWindows() else: print("Not enough matches are found - {}/{}".format(len(good_matches), 10))Please note:
-
You might need to install OpenCV first to run this script:
pip install opencv-python pip install opencv-contrib-python -
SIFT and SURF are patented algorithms and are included in
opencv-contrib-python. If you can't use them due to licensing issues, consider using ORB or other available feature detectors and descriptors. -
The parameters in SIFT/SURF/ORB and matcher can be tuned for your specific images and needs.
-
The threshold used in Lowe's ratio test (0.75 in this case) is a standard heuristic, but it can be adjusted according to your requirements. The number '10' is an arbitrarily chosen threshold for the number of matches required to consider the object found – you might need a different value.
-
The quality of matches and the result will significantly depend on the size, resolution, and angle of rotation of the template with respect to the target. In the real-world scenario, feature matching might need to be combined with other techniques for robustness.
-
- User
crop the detectect window
- Assistant
Certainly! We can add a few lines to the previous script to crop out the detected template area from the original (search) image and then save or display it.
import cv2 import numpy as np # Read the template and the image where the template will be searched template_image_path = 'template.jpg' search_image_path = 'search_image.jpg' template = cv2.imread(template_image_path, 0) # Reading in grayscale search_image = cv2.imread(search_image_path, 0) # Reading in grayscale # Initialize SIFT detector sift = cv2.SIFT_create() # Find the keypoints and descriptors with SIFT keypoints_template, descriptors_template = sift.detectAndCompute(template, None) keypoints_search_image, descriptors_search_image = sift.detectAndCompute(search_image, None) # FLANN parameters FLANN_INDEX_KDTREE = 1 index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5) search_params = dict(checks=50) flann = cv2.FlannBasedMatcher(index_params, search_params) matches = flann.knnMatch(descriptors_template, descriptors_search_image, k=2) # Lowe's ratio test good_matches = [] for m, n in matches: if m.distance < 0.75 * n.distance: good_matches.append(m) if len(good_matches) > 10: points_template = np.float32([keypoints_template[m.queryIdx].pt for m in good_matches]) points_search_image = np.float32([keypoints_search_image[m.trainIdx].pt for m in good_matches]) # Find homography to get a proper perspective of the object matrix, mask = cv2.findHomography(points_template, points_search_image, cv2.RANSAC, 5.0) # Get the dimensions of the template image h, w = template.shape points = np.float32([[0, 0], [0, h], [w, h], [w, 0]]).reshape(-1, 1, 2) # Perform perspective transform transformed_points = cv2.perspectiveTransform(points, matrix) transformed_points = np.int32(transformed_points) # Crop detected window from the search image # We define a mask with the same dimensions as the search image, filled with zeros (black). mask = np.zeros(search_image.shape, dtype=np.uint8) # We then fill the mask with white color for the area of the detected window. channel_count = search_image.shape[2] if len(search_image.shape) > 2 else 1 ignore_mask_color = (255,) * channel_count cv2.fillConvexPoly(mask, transformed_points, ignore_mask_color) # We apply the mask to the search image to get the window. cropped_image = cv2.bitwise_and(search_image, search_image, mask=mask) # Now, we need to find the bounding rectangle for the window to crop to. x, y, w, h = cv2.boundingRect(transformed_points) cropped_window = cropped_image[y:y+h, x:x+w] # We've now got the cropped window, we can save or display it: cv2.imwrite('cropped_window.jpg', cropped_window) cv2.imshow('Cropped Window', cropped_window) cv2.waitKey(0) cv2.destroyAllWindows() else: print("Not enough matches are found - {}/{}".format(len(good_matches), 10))This script will create a cropped image around the detected template within the search image. The
fillConvexPolyfunction creates a mask based on the transformed points (which define the template's perspective in the search image), and thenbitwise_andis used to extract only the parts of the image that are within the mask. We useboundingRectto find the minimal up-right bounding rectangle for the set of points (the detected window) so we can easily crop this particular region out of the search image. The cropped area is then saved to a file calledcropped_window.jpgand displayed.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.