My merge image function replaces pixels from the latest image. However I am thinking it…
- User
My merge image function replaces pixels from the latest image. However I am thinking it is better to interpolate alpha value so that the previous images are visible. My merge image function aims to create panorama images so I am not sure if this is good idea. Can you help me. Here is my code: "def image_merging(images, homographies): if len(images) - 1 != len(homographies): raise ValueError("The number of homography matrices must be one less than the number of images.")
# Warp all images except the first one using their corresponding homography matrices warped_images = [images[0]] + [warp_perspective(img, H, True) for img, H in zip(images[1:], homographies)] # Calculate the bounds of each warped image warped_image_bounds = np.array([(0, 0, *images[0].shape[:2])] + [get_warped_image_bounds(img, H) for img, H in zip(images[1:], homographies)]) # Compute the size of the merged image min_x = min(-warped_image_bounds[:,0].max(), warped_image_bounds[:,0].min()) min_y = min(-warped_image_bounds[:,1].max(), warped_image_bounds[:,1].min()) max_x = warped_image_bounds[:,2].max() max_y = warped_image_bounds[:,3].max() print("new: ", min_x, min_y, max_x, max_y) print(warped_image_bounds) merged_width = int(np.ceil(max_y - min_y - min_y)) merged_height = int(np.ceil(max_x - min_x - min_x)) print("---") print(merged_width) print(max_x) print(min_x) print(max_x - min_x) print(np.ceil(max_x - min_x)) print(int(np.ceil(max_x - min_x))) print("---") # Initialize the merged image with zeros merged_image = np.zeros((merged_height, merged_width, 3), dtype=np.uint8) print("Merged Shape: ", merged_image.shape) # Merge the warped images by overlaying them on the merged image for img, bounds in zip(warped_images, warped_image_bounds): yy, xx= bounds[:2] print("yx: ", yy ,xx) mask = np.all(img == 0, axis=-1) y_offset, x_offset = int(max(0, -min_x+yy)), int(max(0, -min_y+xx)) merged_image_slice = merged_image[y_offset:y_offset + img.shape[0], x_offset:x_offset + img.shape[1]] print(img.shape, merged_image_slice.shape, mask.shape) merged_image_slice[~mask] = img[~mask] print("Merged Shape 2: ", merged_image.shape) return merged_image" - Assistant
To implement interpolation of alpha values between images, you can adjust the blending of the images using their alpha channels. In this way, you can give more priority to the visible pixels of the previous images while creating the panorama. Here's a modified version of your code that uses alpha channels for blending:
import numpy as np def image_merging(images, homographies): if len(images) - 1 != len(homographies): raise ValueError("The number of homography matrices must be one less than the number of images.") # Warp all images except the first one using their corresponding homography matrices warped_images = [images[0]] + [warp_perspective(img, H, True) for img, H in zip(images[1:], homographies)] # Calculate the bounds of each warped image warped_image_bounds = np.array([(0, 0, *images[0].shape[:2])] + [get_warped_image_bounds(img, H) for img, H in zip(images[1:], homographies)]) # Compute the size of the merged image min_x = min(-warped_image_bounds[:,0].max(), warped_image_bounds[:,0].min()) min_y = min(-warped_image_bounds[:,1].max(), warped_image_bounds[:,1].min()) max_x = warped_image_bounds[:,2].max() max_y = warped_image_bounds[:,3].max() merged_width = int(np.ceil(max_y - min_y - min_y)) merged_height = int(np.ceil(max_x - min_x - min_x)) # Initialize the merged image with zeros and an alpha channel merged_image = np.zeros((merged_height, merged_width, 4), dtype=np.uint8) # Merge the warped images by overlaying them on the merged image for img, bounds in zip(warped_images, warped_image_bounds): yy, xx = bounds[:2] # Convert the image to have an alpha channel img_rgba = np.concatenate((img, np.ones((*img.shape[:2], 1), dtype=np.uint8) * 255), axis=-1) # Create the mask based on the alpha channel mask = img_rgba[:, :, 3] > 0 y_offset, x_offset = int(max(0, -min_x + yy)), int(max(0, -min_y + xx)) merged_image_slice = merged_image[y_offset:y_offset + img_rgba.shape[0], x_offset:x_offset + img_rgba.shape[1]] # Blend the current image with the merged image using the alpha channel alpha = img_rgba[:, :, 3] / 255.0 merged_image_slice[:, :, :3] = (1 - alpha)[:, :, np.newaxis] * merged_image_slice[:, :, :3] + alpha[:, :, np.newaxis] * img_rgba[:, :, :3] merged_image_slice[:, :, 3] = np.maximum(merged_image_slice[:, :, 3], img_rgba[:, :, 3]) return merged_image[..., :3]This code converts each image to have an alpha channel and then blends them together using the alpha channel. It also takes care of updating the alpha channel in the merged image. Note that the returned
merged_imageno longer contains the alpha channel. - User
Instead of this solution you gave, I was thinking more like instead of "merged_image_slice[~mask] = img[~mask]" which replaces the pixel you create a solution where the previously assigned value is also considered so that new value doesn't replace.
- Assistant
I see, you want to blend the images without using an alpha channel. In that case, you can simply average the pixel values of the overlapping regions. Here's a modified version of your code that does this:
import numpy as np def image_merging(images, homographies): if len(images) - 1 != len(homographies): raise ValueError("The number of homography matrices must be one less than the number of images.") # Warp all images except the first one using their corresponding homography matrices warped_images = [images[0]] + [warp_perspective(img, H, True) for img, H in zip(images[1:], homographies)] # Calculate the bounds of each warped image warped_image_bounds = np.array([(0, 0, *images[0].shape[:2])] + [get_warped_image_bounds(img, H) for img, H in zip(images[1:], homographies)]) # Compute the size of the merged image min_x = min(-warped_image_bounds[:,0].max(), warped_image_bounds[:,0].min()) min_y = min(-warped_image_bounds[:,1].max(), warped_image_bounds[:,1].min()) max_x = warped_image_bounds[:,2].max() max_y = warped_image_bounds[:,3].max() merged_width = int(np.ceil(max_y - min_y - min_y)) merged_height = int(np.ceil(max_x - min_x - min_x)) # Initialize the merged image with zeros and a count matrix to keep track of the number of images contributing to each pixel merged_image = np.zeros((merged_height, merged_width, 3), dtype=np.float32) count_matrix = np.zeros((merged_height, merged_width), dtype=np.uint8) # Merge the warped images by overlaying them on the merged image for img, bounds in zip(warped_images, warped_image_bounds): yy, xx = bounds[:2] mask = np.all(img == 0, axis=-1) y_offset, x_offset = int(max(0, -min_x + yy)), int(max(0, -min_y + xx)) merged_image_slice = merged_image[y_offset:y_offset + img.shape[0], x_offset:x_offset + img.shape[1]] count_matrix_slice = count_matrix[y_offset:y_offset + img.shape[0], x_offset:x_offset + img.shape[1]] # Update the merged image and count matrix merged_image_slice[~mask] += img[~mask] count_matrix_slice[~mask] += 1 # Normalize the merged image by dividing by the count matrix merged_image /= np.maximum(count_matrix[..., np.newaxis], 1) return merged_image.astype(np.uint8)This code initializes a
count_matrixto keep track of the number of images contributing to each pixel. It then updates themerged_imageandcount_matrixfor each warped image. Finally, it normalizes themerged_imageby dividing by thecount_matrix. Note that the returnedmerged_imageis of typenp.uint8.