Skip to content
TheChatArchiveThe Chat Archive
gpt-4o-2024-05-13· OpenAIAi2 WildChat-4.8M (AI2)

I am building a 3d renderer in C++ using Vulkan. I am using the C++ implementation of…

  1. User

    I am building a 3d renderer in C++ using Vulkan. I am using the C++ implementation of Vulkan. I've been trying to implement shadow mapping to my renderer. I've set up a separate render pass and command buffer to handle the shadow mapping and I have another render pass and command buffer to render the scene. I am no longer getting any validation errors when running the renderer. I am now trying to validate that the shadow mapping is working as intended by rendering the outputs to a texture within the scene. The transitions between the depth stencil and the output image are being handled using subpass dependencies in the shadow render pass. While testing the shadow mapping output, the resulting texture is now entirely red. Do you know what might be causing this problem?

    Here is some of the code:

    Shadow Render Pass:

    vk::AttachmentDescription depthAttachment{}; depthAttachment.format = FindDepthFormat(*GetPhysicalDevice()); // High precision format depthAttachment.samples = vk::SampleCountFlagBits::e1; depthAttachment.loadOp = vk::AttachmentLoadOp::eClear; depthAttachment.storeOp = vk::AttachmentStoreOp::eStore; depthAttachment.stencilLoadOp = vk::AttachmentLoadOp::eDontCare; depthAttachment.stencilStoreOp = vk::AttachmentStoreOp::eDontCare; depthAttachment.initialLayout = vk::ImageLayout::eUndefined; depthAttachment.finalLayout = vk::ImageLayout::eDepthStencilAttachmentOptimal;

    vk::AttachmentReference depthAttachmentRef{};
    depthAttachmentRef.attachment = 0;
    depthAttachmentRef.layout = vk::ImageLayout::eDepthStencilAttachmentOptimal;
    
    vk::SubpassDescription subpass{};
    subpass.pipelineBindPoint = vk::PipelineBindPoint::eGraphics;
    subpass.colorAttachmentCount = 0;
    subpass.pDepthStencilAttachment = &depthAttachmentRef;
    
    // Use subpass dependencies for layout transitions
    std::array<vk::SubpassDependency, 2> dependencies;
    
    dependencies[0].srcSubpass = VK_SUBPASS_EXTERNAL;
    dependencies[0].dstSubpass = 0;
    dependencies[0].srcStageMask = vk::PipelineStageFlagBits::eFragmentShader;// VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
    dependencies[0].dstStageMask = vk::PipelineStageFlagBits::eEarlyFragmentTests;// VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT;
    dependencies[0].srcAccessMask = vk::AccessFlagBits::eShaderRead; //VK_ACCESS_SHADER_READ_BIT;
    dependencies[0].dstAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentWrite; //VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT;
    dependencies[0].dependencyFlags = vk::DependencyFlagBits::eByRegion; //VK_DEPENDENCY_BY_REGION_BIT;
    
    dependencies[1].srcSubpass = 0;
    dependencies[1].dstSubpass = VK_SUBPASS_EXTERNAL;
    dependencies[1].srcStageMask = vk::PipelineStageFlagBits::eLateFragmentTests;// VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT;
    dependencies[1].dstStageMask = vk::PipelineStageFlagBits::eFragmentShader; //VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
    dependencies[1].srcAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentWrite; //VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT;
    dependencies[1].dstAccessMask = vk::AccessFlagBits::eShaderRead;// VK_ACCESS_SHADER_READ_BIT;
    dependencies[1].dependencyFlags = vk::DependencyFlagBits::eByRegion;// VK_DEPENDENCY_BY_REGION_BIT;
    
    
    vk::RenderPassCreateInfo renderPassInfo{};
    renderPassInfo.attachmentCount = 1;
    renderPassInfo.pAttachments = &depthAttachment;
    renderPassInfo.subpassCount = 1;
    renderPassInfo.pSubpasses = &subpass;
    renderPassInfo.dependencyCount = static_cast<uint32_t>(dependencies.size());
    renderPassInfo.pDependencies = dependencies.data();
    
    if (device.createRenderPass(&renderPassInfo, nullptr, &shadowRenderPass) != vk::Result::eSuccess)
    {
        throw std::runtime_error("Failed to create render pass.");
    }
    

    The Begin Shadow Frame:

    // Wait for any previous work on this swapchain image to complete device.waitForFences(1, &inFlightFences[currentFrame], VK_TRUE, UINT64_MAX); device.resetFences(1, &inFlightFences[currentFrame]);

    // Acquire an image from the swapchain, then begin recording commands for the current frame.
    vk::Result acquireResult = device.acquireNextImageKHR(swapchain, UINT64_MAX, imageAvailableSemaphores[currentFrame], vk::Fence(), &imageIndex);
    
    if (acquireResult != vk::Result::eSuccess && acquireResult != vk::Result::eSuboptimalKHR) {
        throw std::runtime_error("Failed to acquire next swapchain image.");
    }
    
    vk::CommandBufferBeginInfo beginInfo{};
    beginInfo.flags = vk::CommandBufferUsageFlagBits::eOneTimeSubmit;
    
    // Shadow Pass
    shadowCommandBuffers[0].begin(beginInfo);
    
    vk::RenderPassBeginInfo shadowPassInfo{};
    shadowPassInfo.renderPass = shadowRenderPass;
    shadowPassInfo.framebuffer = shadowFramebuffers[0];
    shadowPassInfo.renderArea.offset = vk::Offset2D{ 0, 0 };
    shadowPassInfo.renderArea.extent = vk::Extent2D(SHADOW_WIDTH, SHADOW_HEIGHT);
    
    vk::ClearValue shadowClearValue = { vk::ClearDepthStencilValue(1.0f, 0) };
    shadowPassInfo.clearValueCount = 1;
    shadowPassInfo.pClearValues = &shadowClearValue;
    
    shadowCommandBuffers[0].beginRenderPass(shadowPassInfo, vk::SubpassContents::eInline);
    // (Render scene from light perspective)
    

    The End Shadow Frame:

    shadowCommandBuffers[0].endRenderPass();
    
    shadowCommandBuffers[0].end();
    
    vk::SubmitInfo shadowSubmitInfo{};
    shadowSubmitInfo.commandBufferCount = 1;
    shadowSubmitInfo.pCommandBuffers = &shadowCommandBuffers[0];
    
    vk::Semaphore waitSemaphores[] = { imageAvailableSemaphores[currentFrame] };
    vk::PipelineStageFlags waitStages[] = { vk::PipelineStageFlagBits::eColorAttachmentOutput };
    shadowSubmitInfo.waitSemaphoreCount = 1;
    shadowSubmitInfo.pWaitSemaphores = waitSemaphores;
    shadowSubmitInfo.pWaitDstStageMask = waitStages;
    
    vk::Semaphore signalSemaphores[] = { shadowMapCompletedSemaphore };
    shadowSubmitInfo.signalSemaphoreCount = 1;
    shadowSubmitInfo.pSignalSemaphores = signalSemaphores;
    
    graphicsQueue.submit(1, &shadowSubmitInfo, nullptr);
    

    The Image creation:

        imageInfo.imageType = vk::ImageType::e2D;
        imageInfo.format = FindDepthFormat(*GetPhysicalDevice();
        imageInfo.extent = vk::Extent3D(SHADOW_WIDTH, SHADOW_HEIGHT, 1);
        imageInfo.mipLevels = 1;
        imageInfo.arrayLayers = 1;
        imageInfo.samples = vk::SampleCountFlagBits::e1;
        imageInfo.tiling = vk::ImageTiling::eOptimal;
        imageInfo.usage = vk::ImageUsageFlagBits::eDepthStencilAttachment | vk::ImageUsageFlagBits::eSampled;
        imageInfo.sharingMode = vk::SharingMode::eExclusive;
    
  2. Assistant

    Rendering shadows correctly in Vulkan can be tricky. There are several areas to look at when debugging issues like your textures rendering entirely red. Let's go through some potential causes:

    1. Depth Attachment Configuration: Ensure the depth attachment configuration is correct. It seems like your render pass and framebuffer setup is correct, but double-check the format and the way you've created the depth image.

    2. Memory Barriers: The subpass dependencies and the synchronization primitives must be correctly set up to ensure proper transitions and access control of the depth image.

    3. Rendering the Shadow Map: The content rendered to the shadow map should be correctly represented. Ensure that your shadow map generation includes proper transformations and projection from light space.

    4. Reading the Shadow Map: When you sample the shadow map in the fragment shader, verify that you're correctly reading depth values. If you see pure red, it might indicate incorrect sampling or texture binding issues.

    5. Pipeline Configuration: Ensure that the graphics pipeline configurations for both shadow pass and scene pass are correct. Misconfigurations can lead to unexpected outputs.

    Steps to Investigate

    1. Check Depth Attachment Format

    Ensure that FindDepthFormat function returns a valid depth format (like vk::Format::eD32Sfloat for high precision).

    vk::Format FindDepthFormat(vk::PhysicalDevice physicalDevice) {
        std::vector<vk::Format> candidates = {
            vk::Format::eD32Sfloat,
            vk::Format::eD32SfloatS8Uint,
            vk::Format::eD24UnormS8Uint
        };
    
        for (vk::Format format : candidates) {
            vk::FormatProperties props = physicalDevice.getFormatProperties(format);
    
            if (props.optimalTilingFeatures & vk::FormatFeatureFlagBits::eDepthStencilAttachment) {
                return format;
            }
        }
    
        throw std::runtime_error("Failed to find supported depth format.");
    }
    

    2. Validate Subpass Dependencies

    The dependencies look correct, but ensure they are necessary. The two dependencies manage external stages correctly.

    3. Sampling in the Shader

    Ensure you have correctly bound the shadow map texture in your fragment shader and are reading the depth properly:

    // Fragment Shader
    layout (set = 1, binding = 0) uniform sampler2D shadowMap;
    
    void main() {
        vec4 shadowColor = texture(shadowMap, shadowCoord.xy);
        // Do something with shadowColor
    }
    

    When binding the texture to the descriptor set, ensure it's done right. If the texture is entirely red, perhaps what you fetch from the shadow map is incorrect.

    4. Transition between Render Passes

    Make sure memory barriers are correct. If you switch from depth write to depth read, proper synchronization needs to be there.

    5. Debugging

    • Frame Capture Tools: Use tools like RenderDoc to capture frames and inspect textures, render passes, framebuffers, and memory barriers.
    • Shader Outputs: Debug draw shader outputs as color to inspect the depth values read from the shadow map.

    Example Code for Submission and Shader Binding:

    // Shadow Pass Submission
    vk::SubmitInfo shadowSubmitInfo{};
    shadowSubmitInfo.commandBufferCount = 1;
    shadowSubmitInfo.pCommandBuffers = &shadowCommandBuffers[0];
    
    vk::Semaphore signalSemaphores[] = { shadowMapCompletedSemaphore };
    shadowSubmitInfo.signalSemaphoreCount = 1;
    shadowSubmitInfo.pSignalSemaphores = signalSemaphores;
    
    graphicsQueue.submit(1, &shadowSubmitInfo, nullptr);
    
    // Scene Pass for rendering
    vk::PipelineStageFlags waitStages[] = { vk::PipelineStageFlagBits::eFragmentShader };
    vk::Semaphore waitSemaphores[] = { shadowMapCompletedSemaphore };
    
    sceneSubmitInfo.waitSemaphoreCount = 1;
    sceneSubmitInfo.pWaitSemaphores = waitSemaphores;
    sceneSubmitInfo.pWaitDstStageMask = waitStages;
    // Other scene submit setup
    graphicsQueue.submit(1, &sceneSubmitInfo, nullptr);
    
    // Binding the shadow map
    vk::DescriptorImageInfo shadowMapInfo{};
    shadowMapInfo.imageLayout = vk::ImageLayout::eShaderReadOnlyOptimal;
    shadowMapInfo.imageView = shadowImageView;
    shadowMapInfo.sampler = shadowSampler;
    
    vk::WriteDescriptorSet descriptorWrite{};
    descriptorWrite.dstSet = descriptorSet;
    descriptorWrite.dstBinding = 0;
    descriptorWrite.dstArrayElement = 0;
    descriptorWrite.descriptorType = vk::DescriptorType::eCombinedImageSampler;
    descriptorWrite.descriptorCount = 1;
    descriptorWrite.pImageInfo = &shadowMapInfo;
    
    device.updateDescriptorSets(1, &descriptorWrite, 0, nullptr);
    

    Carefully validate each setup, ensuring no synchronization issues or descriptor binding mishaps.

    Combining these checks should help diagnose what is going wrong with your shadow mapping.

  3. User

    Could there be an issue with the sharingMode in the image creation? It is currently set to Exclusive. Does it need to be changed to Concurrent since the image is being used across different buffers, or is it still Exclusive since they are part of the same queue?

  4. Assistant

    The sharingMode parameter in Vulkan image creation specifies how the image will be accessed by multiple queues. Specifically, it dictates whether the image can be accessed exclusively by a single queue family at a time (vk::SharingMode::eExclusive) or concurrently by multiple queue families (vk::SharingMode::eConcurrent).

    In your case, the shadow map image is used across different phases of rendering but within the same queue family (the graphics queue). Hence, you could indeed use vk::SharingMode::eExclusive since it is more efficient and avoids potential overhead associated with concurrent sharing mode.

    However, issues could arise if there are synchronization problems or if you're missing appropriate image layout transitions. Here are a few specific steps and debugging tips you can follow:

    Image Layout Transitions

    Ensure that you handle image layout transitions properly. When you use an image (depth texture in this case) for multiple purposes, you need appropriate barriers or transitions between uses.

    For example:

    1. Shadow Map Generation: Transition the depth image to vk::ImageLayout::eDepthStencilAttachmentOptimal before rendering the shadow map.
    2. Shadow Map Sampling: Transition the depth image to vk::ImageLayout::eShaderReadOnlyOptimal before sampling in the fragment shader.

    Ensure Proper Synchronization

    It's critically important that you properly synchronize the usage of the image across different stages of the pipeline. This includes ensuring that writes to the depth image in the shadow pass have completed before the image is read in the scene pass.

    Example of Layout Transitions

    Here's an example showing how to perform the image layout transitions:

    void transitionImageLayout(vk::CommandBuffer commandBuffer, vk::Image image, vk::Format format, vk::ImageLayout oldLayout, vk::ImageLayout newLayout) {
        vk::ImageMemoryBarrier barrier{};
        barrier.oldLayout = oldLayout;
        barrier.newLayout = newLayout;
        barrier.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
        barrier.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
        barrier.image = image;
        barrier.subresourceRange.aspectMask = vk::ImageAspectFlagBits::eDepth;
        barrier.subresourceRange.baseMipLevel = 0;
        barrier.subresourceRange.levelCount = 1;
        barrier.subresourceRange.baseArrayLayer = 0;
        barrier.subresourceRange.layerCount = 1;
    
        vk::PipelineStageFlags sourceStage;
        vk::PipelineStageFlags destinationStage;
    
        if (oldLayout == vk::ImageLayout::eUndefined && newLayout == vk::ImageLayout::eDepthStencilAttachmentOptimal) {
            barrier.srcAccessMask = {};
            barrier.dstAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentRead | vk::AccessFlagBits::eDepthStencilAttachmentWrite;
    
            sourceStage = vk::PipelineStageFlagBits::eTopOfPipe;
            destinationStage = vk::PipelineStageFlagBits::eEarlyFragmentTests;
        } else if (oldLayout == vk::ImageLayout::eDepthStencilAttachmentOptimal && newLayout == vk::ImageLayout::eShaderReadOnlyOptimal) {
            barrier.srcAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentWrite;
            barrier.dstAccessMask = vk::AccessFlagBits::eShaderRead;
    
            sourceStage = vk::PipelineStageFlagBits::eLateFragmentTests;
            destinationStage = vk::PipelineStageFlagBits::eFragmentShader;
        } else {
            throw std::invalid_argument("unsupported layout transition!");
        }
    
        commandBuffer.pipelineBarrier(
            sourceStage, destinationStage,
            {},
            0, nullptr,
            0, nullptr,
            1, &barrier
        );
    }
    

    Ensure you call this function at the appropriate points:

    • Before starting the shadow pass to transition the depth image to vk::ImageLayout::eDepthStencilAttachmentOptimal.
    • After the shadow pass completes and before using the shadow map in the scene pass to transition the depth image to vk::ImageLayout::eShaderReadOnlyOptimal.

    Descriptor Binding

    Also ensure your descriptor sets and pipelines are correctly configured to use the shadow map as a texture:

    vk::DescriptorImageInfo shadowMapInfo{};
    shadowMapInfo.imageLayout = vk::ImageLayout::eShaderReadOnlyOptimal;
    shadowMapInfo.imageView = shadowImageView;
    shadowMapInfo.sampler = shadowSampler;
    
    vk::WriteDescriptorSet descriptorWrite{};
    descriptorWrite.dstSet = descriptorSet;
    descriptorWrite.dstBinding = 0;
    descriptorWrite.dstArrayElement = 0;
    descriptorWrite.descriptorType = vk::DescriptorType::eCombinedImageSampler;
    descriptorWrite.descriptorCount = 1;
    descriptorWrite.pImageInfo = &shadowMapInfo;
    
    device.updateDescriptorSets(1, &descriptorWrite, 0, nullptr);
    

    Debugging

    Use tools like RenderDoc or Vulkan's validation layers to capture frames and check the status of the image at different points. RenderDoc can give you a detailed view of the image layout, format, contents, and the sequence of operations applied to it.

    Also, consider rendering debug output directly from the shader, color-coding different aspects of the depth buffer to visually inspect if depth values are being written and read correctly.

    Combining these steps should help identify and fix the issue with your shadow mapping.

  5. User

    Please note that I am handling the format transitions using subpass dependencies. The imageLayout is also set as eDepthAttachmentStencilReadOnlyOptimal. If I use ShaderReadOnlyOptimal I will get mismatch errors.

  6. Assistant

    Given that you're handling layout transitions using subpass dependencies and have imageLayout set to vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal, let's focus on ensuring these transitions and other configurations are correctly managed.

    Subpass Dependencies and Layouts

    Since you mentioned using vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal, the key points are verifying the subpass dependencies and ensuring proper usage in shader descriptors. The issue might not be with sharing the image but rather with how and when the image's layout is transitioned and sampled from.

    Please double-check that vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal is used correctly where depth reading is required, and vk::ImageLayout::eDepthStencilAttachmentOptimal during depth writing.

    Triple-checking Subpass Dependencies

    Ensure the subpass dependencies correctly handle layout transitions and access masks:

    • Image Used as Depth Attachment: Image layout should be vk::ImageLayout::eDepthStencilAttachmentOptimal.
    • Image Used for Sampling: Transition to vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal.

    Check how dependencies are set:

    // Shadow Render Pass
    std::array<vk::SubpassDependency, 2> dependencies{};
    
    dependencies[0].srcSubpass = VK_SUBPASS_EXTERNAL;
    dependencies[0].dstSubpass = 0;
    dependencies[0].srcStageMask = vk::PipelineStageFlagBits::eFragmentShader;
    dependencies[0].dstStageMask = vk::PipelineStageFlagBits::eEarlyFragmentTests;
    dependencies[0].srcAccessMask = vk::AccessFlagBits::eShaderRead;
    dependencies[0].dstAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentWrite;
    dependencies[0].dependencyFlags = vk::DependencyFlagBits::eByRegion;
    
    dependencies[1].srcSubpass = 0;
    dependencies[1].dstSubpass = VK_SUBPASS_EXTERNAL;
    dependencies[1].srcStageMask = vk::PipelineStageFlagBits::eLateFragmentTests;
    dependencies[1].dstStageMask = vk::PipelineStageFlagBits::eFragmentShader;
    dependencies[1].srcAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentWrite;
    dependencies[1].dstAccessMask = vk::AccessFlagBits::eShaderRead;
    dependencies[1].dependencyFlags = vk::DependencyFlagBits::eByRegion;
    
    vk::RenderPassCreateInfo renderPassInfo{};
    renderPassInfo.attachmentCount = 1;
    renderPassInfo.pAttachments = &depthAttachment;
    renderPassInfo.subpassCount = 1;
    renderPassInfo.pSubpasses = &subpass;
    renderPassInfo.dependencyCount = static_cast<uint32_t>(dependencies.size());
    renderPassInfo.pDependencies = dependencies.data();
    

    Shader Descriptor Configuration

    Ensure that your shader descriptors correctly sample from the depth texture with the expected layout:

    In the fragment shader's descriptor:

    layout (set = 1, binding = 0) uniform sampler2DShadow shadowMap;
    
    void main() {
        float shadow = texture(shadowMap, shadowCoord.xy);
        // Use shadow value to blend shadow
    }
    

    Ensure shadowCoord is correctly transformed into [0, 1] range for depth sampling.

    Descriptor and Pipeline Configuration

    Verify that the descriptor set correctly binds to a layout compatible with vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal:

    vk::DescriptorImageInfo shadowMapInfo{};
    shadowMapInfo.imageLayout = vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal;
    shadowMapInfo.imageView = shadowImageView;
    shadowMapInfo.sampler = shadowSampler;
    
    vk::WriteDescriptorSet descriptorWrite{};
    descriptorWrite.dstSet = descriptorSet;
    descriptorWrite.dstBinding = 0;
    descriptorWrite.dstArrayElement = 0;
    descriptorWrite.descriptorType = vk::DescriptorType::eCombinedImageSampler;
    descriptorWrite.descriptorCount = 1;
    descriptorWrite.pImageInfo = &shadowMapInfo;
    
    device.updateDescriptorSets(1, &descriptorWrite, 0, nullptr);
    

    Debugging with RenderDoc

    1. Capture Frame: Use RenderDoc to capture a frame where shadow mapping is performed.
    2. Inspect Attachments: Check the shadow map texture contents after the shadow pass.
    3. Verify Transitions: Ensure layout transitions align with expectations.
    4. Check Shader Reads: Verify that depth values are correctly sampled in fragment shader.

    Additional Considerations

    • Verify Clear Values: Ensure the depth clear value is correct (typically 1.0f for depth).
    • Viewport and Scissor: Ensure the viewport and scissor correctly match the shadow map extents.

    Final Code Example of Frame Setup

    Ensure the shadow rendering frame and scene rendering frame are correctly synchronized and ordered:

    Begin Shadow Frame

    vk::RenderPassBeginInfo shadowPassInfo{};
    shadowPassInfo.renderPass = shadowRenderPass;
    shadowPassInfo.framebuffer = shadowFramebuffers[0];
    shadowPassInfo.renderArea.offset = vk::Offset2D{0, 0};
    shadowPassInfo.renderArea.extent = vk::Extent2D(SHADOW_WIDTH, SHADOW_HEIGHT);
    
    vk::ClearValue shadowClearValue = {vk::ClearDepthStencilValue{1.0f, 0}};
    shadowPassInfo.clearValueCount = 1;
    shadowPassInfo.pClearValues = &shadowClearValue;
    
    shadowCommandBuffers[0].beginRenderPass(shadowPassInfo, vk::SubpassContents::eInline);
    // Render shadow scene
    shadowCommandBuffers[0].endRenderPass();
    

    End Shadow Frame

    shadowCommandBuffers[0].end();
    
    vk::SubmitInfo shadowSubmitInfo{};
    shadowSubmitInfo.commandBufferCount = 1;
    shadowSubmitInfo.pCommandBuffers = &shadowCommandBuffers[0];
    
    vk::Semaphore signalSemaphores[] = {shadowMapCompletedSemaphore};
    shadowSubmitInfo.signalSemaphoreCount = 1;
    shadowSubmitInfo.pSignalSemaphores = signalSemaphores;
    
    graphicsQueue.submit(1, &shadowSubmitInfo, nullptr);
    

    Scene Rendering (Post-Shadow Mapping)

    Bind the shadow map before rendering your final scene:

    // Update descriptor set and pipeline to use shadow map
    vk::DescriptorImageInfo shadowMapInfo{};
    shadowMapInfo.imageLayout = vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal;
    shadowMapInfo.imageView = shadowImageView;
    shadowMapInfo.sampler = shadowSampler;
    
    // Write descriptor sets
    vk::WriteDescriptorSet descriptorWrite{};
    descriptorWrite.dstSet = descriptorSet;
    descriptorWrite.dstBinding = 0;
    descriptorWrite.dstArrayElement = 0;
    descriptorWrite.descriptorType = vk::DescriptorType::eCombinedImageSampler;
    descriptorWrite.descriptorCount = 1;
    descriptorWrite.pImageInfo = &shadowMapInfo;
    
    device.updateDescriptorSets(1, &descriptorWrite, 0, nullptr);
    
    // Bind descriptor set in pipeline before rendering scene
    commandBuffer.bindDescriptorSets(vk::PipelineBindPoint::eGraphics, pipelineLayout, 1, 1, &descriptorSet, 0, nullptr);
    // Render the scene
    

    Conclusion

    By verifying each step meticulously, especially how subpass dependencies and layout transitions are handled, you can pinpoint if any part of the pipeline needs correction. Use the diagnostic tools and ensure synchronization barriers are in place to avoid access conflicts.

  7. User

    I should mention that the lighting that I have in my scene is dynamic, so the shadow mapping is constantly being updated. Do I somehow need to rebind or update the image each frame during the scene render pass or does the update to the image attachment from the previous pass handle this?

  8. Assistant

    Given that your scene uses dynamic lighting and shadow maps are updated every frame, ensuring proper synchronization and bindings is essential to ensure the updated shadow map is correctly used in subsequent render passes.

    Continuous Update for Dynamic Lighting

    When dealing with dynamic lighting, the depth image for the shadow map is updated each frame. Here's a checklist and further detailed steps to ensure everything is in place:

    1. Render the Shadow Map: Ensure the depth image is correctly updated during the shadow pass.
    2. Barrier for Depth Image: Ensure proper memory barriers using subpass dependencies to transition the image layout correctly.
    3. Update Descriptor Set: Make sure the descriptor set using the shadow map is properly updated and bound before rendering the scene each frame.
    4. Synchronization Primitives: Ensure proper use of semaphores and fences to sync rendering operations.

    Key Steps and Points

    Barrier and Dependencies

    Ensure the subpass dependencies are correctly facilitating the layout transition and memory barrier for the depth image between the shadow pass and the scene pass.

    dependencies[0].srcSubpass = VK_SUBPASS_EXTERNAL;
    dependencies[0].dstSubpass = 0;
    dependencies[0].srcStageMask = vk::PipelineStageFlagBits::eFragmentShader;
    dependencies[0].dstStageMask = vk::PipelineStageFlagBits::eEarlyFragmentTests;
    dependencies[0].srcAccessMask = vk::AccessFlagBits::eShaderRead;
    dependencies[0].dstAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentWrite;
    dependencies[0].dependencyFlags = vk::DependencyFlagBits::eByRegion;
    
    dependencies[1].srcSubpass = 0;
    dependencies[1].dstSubpass = VK_SUBPASS_EXTERNAL;
    dependencies[1].srcStageMask = vk::PipelineStageFlagBits::eLateFragmentTests;
    dependencies[1].dstStageMask = vk::PipelineStageFlagBits::eFragmentShader;
    dependencies[1].srcAccessMask = vk::AccessFlagBits::eDepthStencilAttachmentWrite;
    dependencies[1].dstAccessMask = vk::AccessFlagBits::eShaderRead;
    dependencies[1].dependencyFlags = vk::DependencyFlagBits::eByRegion;
    

    Updating Descriptor Set If Needed

    Typically, if the shadow map image remains the same, you don't need to rebind or update the descriptor set every frame unless the layout or actual image view changes. If you modify the image or layout dynamically, ensure the descriptor set is updated accordingly.

    Handling Command Buffers and Synchronization

    Command buffers should record the necessary commands each frame, including transitions and render passes. Semaphores and fences should ensure synchronization between shadow pass and scene pass.

    Example Workflow

    1. Begin Shadow Pass:

      • Record commands to render shadow map, ensuring the layout transition to vk::ImageLayout::eDepthStencilAttachmentOptimal.
    2. End Shadow Pass:

      • Transition depth image layout to vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal for use in the scene pass.
    3. Bind Shadow Map Descriptor:

      • Ensure the descriptor using the shadow map is bound correctly before rendering the actual scene.
    4. Submit Command Buffers:

      • Use semaphores between shadow pass command submission and scene pass command submission to handle dependencies.
    // Record shadow command buffer
    shadowCommandBuffer.begin(beginInfo);
    shadowCommandBuffer.beginRenderPass(shadowPassInfo, vk::SubpassContents::eInline);
    // Render shadow map
    shadowCommandBuffer.endRenderPass();
    shadowCommandBuffer.end();
    
    // Submit shadow command buffer
    vk::SubmitInfo shadowSubmitInfo{};
    shadowSubmitInfo.commandBufferCount = 1;
    shadowSubmitInfo.pCommandBuffers = &shadowCommandBuffer;
    vk::Semaphore signalSemaphores[] = { shadowMapCompletedSemaphore };
    shadowSubmitInfo.signalSemaphoreCount = 1;
    shadowSubmitInfo.pSignalSemaphores = signalSemaphores;
    
    graphicsQueue.submit(1, &shadowSubmitInfo, nullptr);
    
    // Ensure the shadow map layout is transitioned correctly for reading in the scene pass
    
    // Record scene command buffer
    sceneCommandBuffer.begin(beginInfo);
    sceneCommandBuffer.bindDescriptorSets(vk::PipelineBindPoint::eGraphics, pipelineLayout, 1, 1, &descriptorSet, 0, nullptr);
    sceneCommandBuffer.beginRenderPass(scenePassInfo, vk::SubpassContents::eInline);
    // Render scene with shadows
    sceneCommandBuffer.endRenderPass();
    sceneCommandBuffer.end();
    
    // Submit scene command buffer
    vk::PipelineStageFlags waitStages[] = { vk::PipelineStageFlagBits::eFragmentShader };
    vk::Semaphore waitSemaphores[] = { shadowMapCompletedSemaphore };
    
    vk::SubmitInfo sceneSubmitInfo{};
    sceneSubmitInfo.pWaitDstStageMask = waitStages;
    sceneSubmitInfo.waitSemaphoreCount = 1;
    sceneSubmitInfo.pWaitSemaphores = waitSemaphores;
    sceneSubmitInfo.commandBufferCount = 1;
    sceneSubmitInfo.pCommandBuffers = &sceneCommandBuffer;
    
    graphicsQueue.submit(1, &sceneSubmitInfo, nullptr);
    

    Using RenderDoc

    Using RenderDoc to inspect the transitions and image contents can be invaluable. It helps to verify that the depth buffer contains the expected values and transitions correctly.

    • Capture the frame: Ensure you capture the frame during dynamic light updates.
    • Inspect Depth Attachment: Make sure the shadow map shows reasonable depth values after rendering from the light's perspective.
    • Check Transitions: Verify that layout transitions happen as expected between the shadow pass and the scene pass.

    Conclusion

    To summarize:

    1. Ensure subpass dependencies and layout transitions are correct.
    2. Only update descriptor sets if the image view or layout changes dynamically.
    3. Use synchronization primitives (semaphores, fences) to manage rendering order and dependencies between passes.
    4. Use debugging tools like RenderDoc to visualize and verify each step.

    By following these steps, you should be able to handle dynamic shadow mapping effectively and ensure correct rendering in your Vulkan-based application.