I am building a 3d renderer in C++ using Vulkan.hpp. I am currently running into an…
- User
I am building a 3d renderer in C++ using Vulkan.hpp. I am currently running into an issue with the lighting stage in my deferred rendering pipeline. It uses the Blinn-Phong method. The stage seems to render fine, but when I look downward or upward I get a dark shadow affecting either the top or bottom half of the screen. I think the issue might be in my lighting fragment shader. I have used RenderDoc to confirm that all the image inputs from the g-buffer are correct. Can you take a look at the shader code and see if there is anything that might be causing this issue?
#version 450
layout(location = 0) out vec4 outColor;
layout(input_attachment_index = 0, set = 0, binding = 0) uniform subpassInputMS gPosition; layout(input_attachment_index = 1, set = 0, binding = 1) uniform subpassInputMS gNormal; layout(input_attachment_index = 2, set = 0, binding = 2) uniform subpassInputMS gAlbedo; layout(input_attachment_index = 3, set = 0, binding = 3) uniform subpassInputMS gSpecular; layout(input_attachment_index = 4, set = 0, binding = 4) uniform subpassInputMS gShadow;
layout(location = 0) in vec2 fragTexCoord;
layout(set = 0, binding = 5) uniform Ambient_Data { vec3 color; float intensity; } ambient;
layout(set = 0, binding = 6) uniform Directional_Data { vec3 position; vec3 color; float intensity; } directional;
layout(set = 0, binding = 7) uniform Camera_Data { vec3 position; } camera;
layout(set = 0, binding = 8) uniform sampler2D ssao;
const float kPi = 3.14159265;
void main() { vec3 fragPosition = subpassLoad(gPosition,gl_SampleID).xyz; vec3 fragNormal = normalize(subpassLoad(gNormal,gl_SampleID).xyz); vec3 fragAlbedo = subpassLoad(gAlbedo,gl_SampleID).rgb; float fragShadow = subpassLoad(gShadow,gl_SampleID).r; float AmbientOcclusion = texture(ssao, fragTexCoord).r;
float specular_intensity = subpassLoad(gSpecular,gl_SampleID).x; float specular_shininess = subpassLoad(gSpecular,gl_SampleID).y; vec3 view_dir = -normalize(camera.position - fragPosition); vec3 ambientLight = ambient.intensity * ambient.color * AmbientOcclusion; vec3 lightDir = normalize(directional.position - fragPosition); const float kEnergyConservation = ( 8.0 + specular_shininess ) / ( 8.0 * kPi ); vec3 reflect_dir = normalize(lightDir + view_dir); float spec = pow(max(dot(view_dir, reflect_dir), 0.0), kEnergyConservation); vec3 specular = specular_intensity * spec * directional.color; float diff = max(dot(fragNormal, lightDir), 0.0); vec3 diffuse = diff * directional.color; vec3 hdrColor = vec3((ambientLight + (fragShadow) * (specular + diffuse)) * fragAlbedo); outColor = vec4(hdrColor, 1.0);}
- Assistant
It looks like you're experiencing an issue where the lighting calculations become inconsistent when looking upward or downward. This can often be attributed to the way normals and lighting are calculated, particularly if there are assumptions or mistakes related to vector directionality.
Here are a few suggestions and potential issues to investigate based on your shader code and problem description:
-
Light Direction Calculation:
You're calculatinglightDiras the difference between the light's position and the fragment's position. For directional lights,lightDirshould generally be a fixed direction rather than varying with the fragment's world position. Ifdirectional.positionis actually meant to be a direction, then calculatelightDiras:vec3 lightDir = normalize(directional.position);If
directional.positionis intended as a direction in view space, ensure it stays consistent across all fragments. -
Reflection Vector Calculation:
The calculation ofreflect_dirusing the sum oflightDirandview_dirdoesn't follow the standard Blinn-Phong model, which uses the halfway vector. The reflection calculation should be changed to use the halfway vectorH:vec3 halfwayDir = normalize(lightDir + view_dir); float spec = pow(max(dot(fragNormal, halfwayDir), 0.0), specular_shininess); -
Specular Shininess Calculation:
Ensure thatkEnergyConservationis not used to calculate the power for the specular reflection. The original Blinn-Phong model uses the shininess value directly:float spec = pow(max(dot(fragNormal, halfwayDir), 0.0), specular_shininess); -
Normal Direction Consistency:
If looking up or down affects the shading, it's worth checking that your normals are consistently oriented. Negatively affected halves of the screen can be a sign that the normals are sometimes pointing in the wrong direction (e.g., into the surface). Ensure the normals from the G-buffer are being loaded and normalized correctly. -
Lighting Calculations:
Ensure that your dot products use vectors correctly. Specifically, the dot product in the diffuse calculation:float diff = max(dot(fragNormal, lightDir), 0.0);Make sure
fragNormalis pointing outwards from the surface. -
Directional Light Source:
Verify the directional light attributes. Ifdirectional.positionis dynamic or not set correctly for pointing consistently to or from the entire scene, it might lead to inconsistent light direction across the screen. -
Ambient Occlusion Influence:
TheAmbientOcclusionterm modifies the ambient light. Double-check that the SSAO texture appears correct across the entire screen. A miscalculation here could lead to shadows appearing when they shouldn't.
By investigating and applying these changes, you should be able to resolve the issue affecting the lighting consistency when changing the camera direction. If problems persist, it may help to visualize individual components of the lighting equation to further diagnose the issue.
-
- User
-
Light Direction Calculation - The lighting direction will change direction periodically. This was done as a means to validate the implementation of shadow mapping and will be set to a more static position once point lighting has been implemented. As the light direction is changing, but the lighting issue I'm seeing is constant, it's unlikely the be the cause.
-
Reflection Vector Calculation - The reflect_dir is actually calculated the same way as the halfwayDir in your example. The value is just labelled as reflect_dir.
-
Specular Shininess Calcuation - The use of kEnergyConservation was an enhancement to the Blinn-Phong model that I had implemented. I have now disabled it and reverted to using specular_shininess. Unfortunately it has not addressed the issue.
-
Normal Direction Consistency - I believe all of the normals are oriented correctly. They are normalised when loaded into the shader. The only possible issue I can see is that the normal are bounded between -1 and 1. I'm not sure if this would have any impact though.
-
Lighting Calculations - Same as above.
-
Directional Light Source - As mentioned above, the directional light source is inconsistent by design. However, the issues I'm seeing are consistent. Therefore I don't believe it's the issue.
-
Ambient Occlusion Influence - I have verified using RenderDoc that the SSAO texture does not contain any artifacts that might be causing this issue.
-
- Assistant
Given your update that some factors have been verified or addressed, and the issue persists with a consistent darkening when looking upward or downward, let's delve a bit deeper into areas that might be affecting the lighting across the view direction:
-
Normal Mapping Range: Even though normal values are typically in the range
[-1, 1], if there are any discrepancies in how these normals are used across your shaders or processed during rendering, it could cause lighting to react incorrectly. To rule out issues with normal mapping, you can output the normals directly to the screen as a color map to visually verify consistent normal directions. -
Camera Position/Direction: Since the issue is dependent on the view direction, ensure that the calculation involving the camera position, such as
view_dir, is accurately computed. Check thatview_dirindeed reflects the correct normalized direction from the fragment to the camera. Consistent camera position across your rendering passes is crucial since inaccurate transformations could affect shading. -
Shadow Bias and Precision: Although shadows might visually look correct in RenderDoc, shading issues can occur at boundary angles due to imprecise shadow computations. If shadow mapping is involved, subtle biases or precision errors might manifest more noticeably when looking at certain angles. Consider testing with a hardcoded
fragShadowvalue to see if it mitigates the issue, thus isolating the problem. -
Normal and View Dir Calculation: Normalize both the normal and view direction vectors after loading them from the G-buffer. They can become denormalized due to precision errors or transformation discrepancies.
vec3 fragNormal = normalize(subpassLoad(gNormal, gl_SampleID).xyz); vec3 view_dir = normalize(camera.position - fragPosition); -
Lighting Model Considerations: Reexamine the lighting model in the context of varying view angles. If the implementation of the lighting affects high angles differently (e.g., due to clamping or thresholds), try varying parameters to observe changes in pattern.
-
Debugging Steps:
- Use Fragment Shader Outputs: Output intermediate calculations (ambient, diffuse, specular) as colors to make sure each step behaves as expected. This helps you visually verify the integrity of lighting calculations at extreme angles.
- Isolate External Influences: Temporarily remove the SSAO or use constant albedo/specular values to verify these don't contribute to the disparity.
-
Vulkan and Coordinate Transformations: If further transformations are involved in between the graph elements (vertex processing vs. fragment shading), ensure the coordinate systems are aligned throughout rendering processes (world space vs. view space discrepancies).
If after these checks the problem persists, further investigation might involve examining coordinate system transformations or potential issues in your matrix math, ensuring normals are correctly oriented to the view/camera space. Through incrementally testing each component in isolation can usually pinpoint more elusive bugs.
-
- User
I have further isolated the issue. For context, the scene I am rendering consists of an object in a cube-shaped room with the directional light rotating around the object. The issue I am having does not impact the object, only the inverted cube that forms the room, and then only the floor and ceiling of the room. I therefore suspect it might be related to the normals of those two surfaces. Using RenderDoc, I have found that the normal value for the floor is 0,-1,0 and the normal value for the ceiling is 0,1,0. Are these normals correct for the surfaces they are meant to represent, or is there possibly something in the shader that is causing issues with correctly interpreting the y-axis normal?
- Assistant
The normals you described for the floor and ceiling surfaces appear to be logically correct in terms of their direction:
- Floor Normal: A normal of
(0, -1, 0)for the floor is standard, pointing downward, assuming that the positive Y-axis is upward in your coordinate system. - Ceiling Normal: Similarly, a normal of
(0, 1, 0)for the ceiling is correct, pointing upward.
If these normals aren't behaving as expected during lighting calculations, particularly showing shading issues while appearing correct in isolation, there are several points to consider that may affect normal interpretation:
-
Double-Sided Normals: If the normals are intended for single-sided rendering and you're noticing issues in the appearance of the surfaces, ensure that there isn't a mismatch in face culling or lighting calculations which might expect inverted normals. Ensure your shader handles two-sided normals if that's necessary.
-
Normal Format and Packing: Verify how these normals are packed into the G-buffer. Normals should precisely encode the expected range and orientation without being truncated or manipulated incorrectly in precision conversion.
-
Transformation Consistency: Ensure that any transformations (model, view, or projection) applied to these surfaces preserve the orientation of the normals correctly. Often, transformed normals require scaling adjustments if non-uniform transformations are applied to the geometry.
-
Lighting Model Constraints: The Blinn-Phong reflection model you use involves a dot product with light direction and view direction, potentially affecting rendering based on their alignment with the normal. If the floor and ceiling receive minimal illumination due to light direction or view position often approaching parallel, the results might seem unexpectedly dark.
-
Other Shader Influences: Check other components affecting the pixel color, such as ambient lighting or shadowing factors, which might behave differently on horizontal surfaces where the angle with the light source becomes more extreme. Differences in
fragShadowvalues, especially if around angles or boundaries, might warrant sufficient tweaks in shadow filtering techniques or bias. -
Debugging Techniques:
- Norm Visual Debugging: Temporarily visualize the normals by outputting them as colors directly onto the screen. This allows visual validation of correct normal usage by presenting them in RGB format where each channel represents an axis.
- Lighting Isolation: Test the shader by removing/cancelling specific lighting elements (e.g., shadows, specular highlights) to see when the artifacts persist.
-
Bias in Shading Calculations: When dealing with surfaces aligned with or against the dominant axis (like vertical light direction onto horizontal surfaces), numerical stability can sometimes need manual adjustments. Slightly altering techniques in rendering such as bias in shadow mapping or raising diffuse/specular floor could be required.
Given your specific problem context, focus on how lighting angles interact with horizontal surfaces. Adjustments in interaction between shading and normal usage should aid in resolving the issue for primary surfaces distinct from object geometry within the rendered environment.
- Floor Normal: A normal of
- User
I believe I have further narrowed down the cause of the issue. I believe it has something to do with the gPosition texture. The input for this texture is the fragment position information from the previous geometry stage. It therefore extends well beyond the 0 to 1 range. However, when viewed only using the 0 to 1 range, it shows four distinct quadrants. The line separating the top two quadrants from the bottom two quadrants aligns perfectly with the artifact I am seeing. The key difference between the values in the top half from the bottom half are that all the bottom values have positive y values and all the top values have negative y values. I'm not sure how this is causing the issue I'm seeing, or how to fix it, but I believe it is the underlying cause.
- Assistant
Given the additional information, it's likely that the issue arises from the way the position information encoded in
gPositionis being interpreted within your deferred rendering pipeline, particularly regarding how negative and positive Y values are handled and how they interact with the lighting calculations.Here's how to address this specific issue:
-
Understanding
gPositionEncoding:- Ensure that the position data stored in the
gPositionbuffer matches the coordinate space expected by subsequent shader stages. If you're interpreting positions in different spaces (e.g., model space vs. world space vs. view space), inconsistencies may arise, particularly in lighting calculations. - The appearance of distinct quadrants may be caused by mismatched interpretation of these coordinates.
- Ensure that the position data stored in the
-
Coordinate Space Transformation:
- Check the transformations applied to the positions before storing them into
gPosition. Positions should typically be encoded in view space if lighting calculations are performed in view space. - If positions are in world space, ensure that all other calculations, like light direction and view vector computations, are transformed to match the coordinate space accurately.
- Check the transformations applied to the positions before storing them into
-
Encoding Range and Precision:
- Verify how position values are scaled and packed into the
gPositiontexture. Since positions extend beyond a [0, 1] range, using floating-point textures (vk::Format::eR16G16B16A16Sfloator higher precision) is vital to avoid clipping or precision loss. - Ensure that any shader outputs take value ranges into account correctly when interpreting these position values.
- Verify how position values are scaled and packed into the
-
Handling Negative/Positive Y Values:
- If there’s a mismatch in shading computations based on Y values (positive vs. negative), inspect the lighting logic within the fragment shader. Ensuring normalization of light directions and view vectors, for instance, could prevent the issue if they depend on position values.
- Be aware of how the floor and ceiling normals interact with the light direction and the view vector, as both influence lighting with respect to the camera.
-
Visual Diagnostics:
- To visually debug the spatial layout of
gPosition, output it directly to verify consistency in position encoding. Use colors that map X, Y, and Z coordinates to RGB, respectively.
- To visually debug the spatial layout of
-
Shading Logic Debugging:
- Insert debug checks or comments in the shader code to observe intermediate variables and operations at runtime. Shader debuggers can simulate different inputs and reveal discrepancies.
- Since the Y values appear to segment the shading differently, examine the shader branches or calculations specific to Y coordinate dependencies.
-
Potential Fixes:
- Double-check that calculations involving positions, such as obtaining the view direction (
view_dir), use the full range of position data and are properly normalized and handled. - Reassess any specific clamping, thresholding, or biasing behaviors directly connected to height that might contribute to the quadrants.
- Double-check that calculations involving positions, such as obtaining the view direction (
Once you align coordinate spaces properly between stages and ensure consistent interpretation of the
gPositiondata, the artifacts should diminish accordingly. Leveraging these strategies should help eliminate the artifact introduced by Y boundary conditions. -
- User
I should mention again that when viewed at a higher range (e.g. 0 to 10) the gPosition image looks perfectly normal. I'm concerned the normalization process in the shader might not be handling this information correctly. Here is the shader code again for reference:
#version 450
layout(location = 0) out vec4 outColor;
layout(input_attachment_index = 0, set = 0, binding = 0) uniform subpassInputMS gPosition; layout(input_attachment_index = 1, set = 0, binding = 1) uniform subpassInputMS gNormal; layout(input_attachment_index = 2, set = 0, binding = 2) uniform subpassInputMS gAlbedo; layout(input_attachment_index = 3, set = 0, binding = 3) uniform subpassInputMS gSpecular; layout(input_attachment_index = 4, set = 0, binding = 4) uniform subpassInputMS gShadow;
layout(location = 0) in vec2 fragTexCoord;
layout(set = 0, binding = 5) uniform Ambient_Data { vec3 color; float intensity; } ambient;
layout(set = 0, binding = 6) uniform Directional_Data { vec3 position; vec3 color; float intensity; } directional;
layout(set = 0, binding = 7) uniform Camera_Data { vec3 position; } camera;
layout(set = 0, binding = 8) uniform sampler2D ssao;
const float kPi = 3.14159265;
void main() { vec3 fragPosition = subpassLoad(gPosition,gl_SampleID).xyz; vec3 fragNormal = normalize(subpassLoad(gNormal,gl_SampleID).xyz); vec3 fragAlbedo = subpassLoad(gAlbedo,gl_SampleID).rgb; float fragShadow = subpassLoad(gShadow,gl_SampleID).r; float AmbientOcclusion = texture(ssao, fragTexCoord).r;
float specular_intensity = subpassLoad(gSpecular,gl_SampleID).x; float specular_shininess = subpassLoad(gSpecular,gl_SampleID).y; vec3 view_dir = -normalize(camera.position - fragPosition); vec3 ambientLight = ambient.intensity * ambient.color * AmbientOcclusion; vec3 lightDir = normalize(directional.position - fragPosition); const float kEnergyConservation = ( 8.0 + specular_shininess ) / ( 8.0 * kPi ); vec3 reflect_dir = normalize(lightDir + view_dir); //float spec = pow(max(dot(view_dir, reflect_dir), 0.0), kEnergyConservation); float spec = pow(max(dot(view_dir, reflect_dir), 0.0), specular_shininess); vec3 specular = specular_intensity * spec * directional.color; float diff = max(dot(fragNormal, lightDir), 0.0); vec3 diffuse = diff * directional.color; vec3 hdrColor = vec3((ambientLight + (fragShadow) * (specular + diffuse)) * fragAlbedo); outColor = vec4(hdrColor, 1.0);}
- Assistant
The description of the
gPositiontexture appearing normal at a higher range suggests that the position values are correctly stored in world or view space but might be misinterpreted at certain phases of lighting/shading computation. The key aspect here seems to be how these positions are utilized in your lighting calculations, especially considering the apparent split based on the sign of the Y coordinate.Let's delve into potential issues and solutions:
-
Normalization and Vector Math:
-
Ensure that the
view_dirandlightDirvectors are correctly calculated. Pay close attention to their normalization and computation withfragPosition.vec3 view_dir = normalize(camera.position - fragPosition); vec3 lightDir = normalize(directional.position - fragPosition); -
Sanity Check: Try outputting
view_dirandlightDirto colors to verify they're calculated as expected. Sometimes, visual inconsistency here can lead to unexpected results.
-
-
Coordinate System Consistency:
- Double-check that all calculations involving positions (like light position, camera position, and
fragPosition) are in the expected coordinate space. If lighting direction or view direction mismatches the expected space, it could affect rendering.
- Double-check that all calculations involving positions (like light position, camera position, and
-
Handling of
gPositionValues:- As you're dealing with potentially large and negative values, confirm the calculations involving
fragPositionhave appropriate handling of these ranges. If scaling or normalization operations unintentionally disregard negative values, it can cause issues.
- As you're dealing with potentially large and negative values, confirm the calculations involving
-
Debugging
fragPosition:- Test calculations by clamping or simplifying
fragPositiontemporarily to check how these affect the shading result. This can highlight potential discrepancies in how the position impacts the remainder of the computations.
- Test calculations by clamping or simplifying
-
Lighting Calculations:
- Directional Light: If
lightDircalculations rely heavily onfragPositionand you see quadrants aligning specifically with sign changes, examine attentively how these signs are effectively used in dot products or reflection.
- Directional Light: If
-
Potential Issue with Y-value Handling:
- Y-dependencies potentially subtlety alter lighting interactions, particularly with horizontal surfaces like floors and ceilings. Investigate if the shader logic has implicit or explicit dependencies that could be affected by these sign flips, possibly influencing how diffuse or specular terms are computed.
-
Debug Visualization:
-
Output intermediate results (e.g., position, normal, specular, and diffuse values) to various framebuffer channels to visualize and diagnose any incorrect calculations or assumptions:
// Example: Debugging visual output for view_dir outColor = vec4(normalize(0.5 * (view_dir + 1.0)), 1.0);
-
Given these considerations, revisiting the way lighting variables are computed and making sure vector math properly accounts for directional consistencies should help resolve the shading artifact issue. If positions are misapplied during transformations or within your lighting model, correcting and verifying assumptions about coordinate spaces can rectify unexpected behaviors.
-
- User
It seems the conversion from HDR to LDR in a subsequent stage exacerbates the issue. However, I believe I have narrowed it down specifically to the lightDir value. When I render that value to the output the issue is most severe. When rendering the view_dir, it is somewhat noticeable after the conversion to LDR. Do you have any suggestions on how to fix the issue?
- Assistant
The issue seems to be primarily linked to the
lightDircalculations and their effect on HDR to LDR conversion. Such conversions can amplify even subtle errors in lighting calculations, leading to more pronounced artifacts.Potential Solutions to Address
lightDirIssues:-
Verify Light Direction Calculation:
-
Ensure that
lightDiris being calculated with the correct vector subtraction and normalization:vec3 lightDir = normalize(directional.position - fragPosition); -
Consider if
directional.positionis meant to be a directional vector rather than a position. For directional light sources, clarify if you should subtract fragment positions in this manner or directly use a direction (e.g.,normalize(vec3(0, 0, -1))for a fixed light direction).
-
-
Vector Normalization:
- Problems with
lightDiroften arise from incorrect normalization. Ensure all vectors (lightDir,view_dir) are persisted as unit vectors throughout calculations. - Besides checking normalization, ensure there aren’t accidental denormalized values or NaNs appearing due to division by near-zero or zero-length vectors.
- Problems with
-
Value Range and Precision:
- Since certain HDR values can exacerbate LDR conversion, careful consideration of the range of values before tone mapping is needed. A linear tone mapping operator or Reinhard tone mapping might help avoid sudden shifts in brightness.
-
HDR to LDR Conversion Adjustments:
-
Use a robust tone mapping algorithm that effectively maps the HDR colors into the visible LDR space without causing severe banding or excessive clamping:
vec3 toneMapping(vec3 color) { return color / (color + vec3(1.0)); } -
Apply gamma correction after mapping to LDR:
vec3 applyGammaCorrection(vec3 color, float gamma) { return pow(color, vec3(1.0 / gamma)); }
-
-
Debugging Steps:
- Output normalized values for
lightDirand see their entirety when sampled accurately across multiple frames. Visualize intermediary results before HDR finalization to discern particular artifacts. - Test with various light positions and directions to measure how perspective and fragment structures influence the resulting shading.
- Output normalized values for
-
Testing with Simplified Scenarios:
- Temporarily simplify the shading model to see how
lightDirandview_dirchanges affect artifacts. Minimal models help illustrate foundational issues like alignment or intensity scaling exaggerations.
- Temporarily simplify the shading model to see how
-
Projection and Clipping:
- Look for transitions or clipping at extreme position values. If the light direction calculation exhibits sudden swings (due to precision errors or shader logic), the movements might seem pronounced in LDR.
From these structured approaches, ensuring the
lightDiris calculated correctly along with appropriate transformations and conversions should significantly reduce artifacts during HDR to LDR translation. Debugging intermediary visual outputs while ensuring consistent coordinate systems/models often highlights misunderstood assumptions within rendering pipelines. -