Could you write a code in GML that automatically detects enclosed/tight spaces and…
- User
Could you write a code in GML that automatically detects enclosed/tight spaces and paints the surface (not the entire object) darker or lighter in a gradient based on that?
- Assistant
Certainly! Creating an effect that detects enclosed or tight spaces within a sprite and applies a gradient shading based on those detections can add depth and visual interest to your game objects. While this task involves complex image processing, it can be efficiently achieved in GameMaker Studio using a combination of GML (GameMaker Language) and GLSL shaders.
Below is a comprehensive guide to implementing this effect:
- Understanding the Approach
- Setting Up Your Project
- Creating the Vertex and Fragment Shaders
- Implementing the GML Code
- Final Touches and Optimization
1. Understanding the Approach
Objective: Automatically detect enclosed/tight spaces within a sprite and apply a dynamic gradient shading to the surface based on the proximity to these spaces.
Methodology:
-
Edge Detection: Identify the edges within the sprite where tight spaces occur. This can be achieved using edge detection algorithms like the Sobel operator.
-
Gradient Shading: Once edges are detected, apply a gradient that darkens or lightens areas based on their distance from these edges, enhancing the perception of depth and enclosure.
Why Use Shaders? While it's possible to perform pixel-level manipulation in GML, it's computationally intensive and not suitable for real-time applications. Shaders utilize the GPU, allowing for efficient parallel processing of pixels, making them ideal for tasks like edge detection and gradient shading.
2. Setting Up Your Project
Before diving into the code, ensure you have the following setup:
-
Sprite Preparation: Ensure your sprite has sufficient contrast and clear edges for effective edge detection.
-
Shader Resources: You'll need to create two shaders—a vertex shader and a fragment shader.
-
Object to Apply Shader: Create or select an object in your game to which you want to apply this effect.
3. Creating the Vertex and Fragment Shaders
a. Vertex Shader
The vertex shader processes each vertex of your sprite. For this task, a basic vertex shader suffices.
File:
vertex_shader.glsl// Vertex Shader attribute vec3 in_Position; attribute vec4 in_Colour; attribute vec2 in_TextureCoord; varying vec2 v_vTexcoord; varying vec4 v_vColour; void main() { v_vTexcoord = in_TextureCoord; v_vColour = in_Colour; gl_Position = vec4(in_Position.xy, 0.0, 1.0); }b. Fragment Shader
The fragment shader handles the pixel-level operations, including edge detection and gradient shading.
File:
fragment_shader.glsl// Fragment Shader varying vec2 v_vTexcoord; varying vec4 v_vColour; uniform sampler2D u_sampler; // Parameters for edge detection uniform float u_resolution_x; uniform float u_resolution_y; void main() { // Texture coordinates vec2 texCoord = v_vTexcoord; // Define pixel size based on resolution vec2 pixel = vec2(1.0 / u_resolution_x, 1.0 / u_resolution_y); // Sobel operator kernels float Gx = 0.0; float Gy = 0.0; // Sample surrounding pixels for(int i = -1; i <= 1; i++) { for(int j = -1; j <=1; j++) { vec2 offset = vec2(float(i), float(j)) * pixel; float gray = texture2D(u_sampler, texCoord + offset).r; // Assuming grayscale Gx += gray * float(j) * (i == 0 ? 1.0 : 2.0); Gy += gray * float(i) * (j == 0 ? 1.0 : 2.0); } } // Calculate gradient magnitude float edge = length(vec2(Gx, Gy)); // Normalize edge value edge = clamp(edge * 4.0, 0.0, 1.0); // Adjust multiplier as needed // Apply gradient shading based on edge proximity // The closer to the edge, the darker the color vec3 originalColor = texture2D(u_sampler, texCoord).rgb; vec3 shadedColor = originalColor * (1.0 - edge * 0.5); // Adjust 0.5 for shading intensity gl_FragColor = vec4(shadedColor, texture2D(u_sampler, texCoord).a); }Explanation:
-
Sobel Operator: The fragment shader uses the Sobel operator to detect edges by calculating the gradient magnitude (
GxandGy) at each pixel. -
Gradient Shading: After detecting edges, it darkens the original color based on the proximity to these edges. The
edgevalue determines how much to darken the pixel, creating a gradient effect. -
Parameters:
u_resolution_xandu_resolution_yhelp calculate the size of each pixel based on the sprite's resolution.
4. Implementing the GML Code
Now, integrate the shaders with your GameMaker object using GML.
a. Loading the Shaders
Create Event of Your Object:
// Create Event // Load shaders vertex_shader = shader_get_builtin_vertex(shader_flat); // Using built-in flat vertex shader fragment_shader = shader_create(" // Fragment Shader Code goes here \n varying vec2 v_vTexcoord; varying vec4 v_vColour; uniform sampler2D u_sampler; // Parameters for edge detection uniform float u_resolution_x; uniform float u_resolution_y; void main() { // Texture coordinates vec2 texCoord = v_vTexcoord; // Define pixel size based on resolution vec2 pixel = vec2(1.0 / u_resolution_x, 1.0 / u_resolution_y); // Sobel operator kernels float Gx = 0.0; float Gy = 0.0; // Sample surrounding pixels for(int i = -1; i <= 1; i++) { for(int j = -1; j <=1; j++) { vec2 offset = vec2(float(i), float(j)) * pixel; float gray = texture2D(u_sampler, texCoord + offset).r; // Assuming grayscale Gx += gray * float(j) * (i == 0 ? 1.0 : 2.0); Gy += gray * float(i) * (j == 0 ? 1.0 : 2.0); } } // Calculate gradient magnitude float edge = length(vec2(Gx, Gy)); // Normalize edge value edge = clamp(edge * 4.0, 0.0, 1.0); // Adjust multiplier as needed // Apply gradient shading based on edge proximity // The closer to the edge, the darker the color vec3 originalColor = texture2D(u_sampler, texCoord).rgb; vec3 shadedColor = originalColor * (1.0 - edge * 0.5); // Adjust 0.5 for shading intensity gl_FragColor = vec4(shadedColor, texture2D(u_sampler, texCoord).a); } "); // Check for shader compilation errors if (fragment_shader == -1) { show_error("Failed to compile fragment shader.", true); } // Set shader parameters shader_set(fragment_shader); shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_x"), sprite_width); shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_y"), sprite_height); shader_reset();Explanation:
-
Shader Loading: The vertex shader uses GameMaker’s built-in flat vertex shader. The fragment shader is created using
shader_createwith the GLSL code provided earlier. -
Error Handling: It’s crucial to check if the shader compiled successfully to avoid runtime errors.
-
Uniforms: The shader requires the resolution of the sprite to calculate pixel offsets accurately. These are set using
shader_set_uniform_f.
b. Drawing the Sprite with Shader
Draw Event of Your Object:
// Draw Event // Set the shader shader_set(fragment_shader); // Set shader parameters (in case sprite size changes) shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_x"), sprite_width); shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_y"), sprite_height); // Draw the sprite draw_self(); // Reset the shader to default shader_reset();Explanation:
-
Applying the Shader: Before drawing the sprite, the shader is set, and the required uniforms are updated to match the current sprite dimensions.
-
Drawing the Sprite:
draw_self()draws the sprite with the shader effect applied. -
Resetting the Shader: It’s good practice to reset the shader after drawing to prevent affecting other objects.
c. Cleaning Up
Cleanup Event of Your Object:
// Cleanup Event // Free shaders to prevent memory leaks if (shader_exists(fragment_shader)) { shader_delete(fragment_shader); } if (shader_exists(vertex_shader)) { shader_delete(vertex_shader); }Explanation:
- Memory Management: Shaders consume GPU resources. Always ensure they are deleted when no longer needed to prevent memory leaks.
5. Final Touches and Optimization
a. Handling Colored Sprites
The provided shader assumes the sprite is in grayscale for edge detection. If your sprite is colored, you might need to convert it to grayscale within the shader before applying the Sobel operator.
Modify Fragment Shader for Colored Sprites:
// Convert to grayscale float gray = dot(originalColor, vec3(0.299, 0.587, 0.114));Replace the grayscale sampling in the Sobel operator with the above conversion.
b. Adjusting Sensitivity
-
Edge Sensitivity: Tweak the multiplier in
edge = clamp(edge * 4.0, 0.0, 1.0);to make edge detection more or less sensitive. -
Shading Intensity: Adjust the value in
shadedColor = originalColor * (1.0 - edge * 0.5);to increase or decrease the shading effect.
c. Performance Considerations
-
Shader Optimization: Complex shaders can impact performance, especially on lower-end devices. Optimize by minimizing calculations within loops and using precision qualifiers where appropriate.
-
Sprite Size: Larger sprites require more processing in shaders. Ensure sprites are optimized in size and resolution based on game requirements.
Complete Example
Here's a consolidated example incorporating all the elements discussed above.
Vertex Shader (
vertex_shader.glsl):// Vertex Shader attribute vec3 in_Position; attribute vec4 in_Colour; attribute vec2 in_TextureCoord; varying vec2 v_vTexcoord; varying vec4 v_vColour; void main() { v_vTexcoord = in_TextureCoord; v_vColour = in_Colour; gl_Position = vec4(in_Position.xy, 0.0, 1.0); }Fragment Shader (
fragment_shader.glsl):// Fragment Shader varying vec2 v_vTexcoord; varying vec4 v_vColour; uniform sampler2D u_sampler; // Parameters for edge detection uniform float u_resolution_x; uniform float u_resolution_y; void main() { // Texture coordinates vec2 texCoord = v_vTexcoord; // Define pixel size based on resolution vec2 pixel = vec2(1.0 / u_resolution_x, 1.0 / u_resolution_y); // Sobel operator kernels float Gx = 0.0; float Gy = 0.0; // Sample surrounding pixels for(int i = -1; i <= 1; i++) { for(int j = -1; j <=1; j++) { vec2 offset = vec2(float(i), float(j)) * pixel; vec4 sampleColor = texture2D(u_sampler, texCoord + offset); float gray = dot(sampleColor.rgb, vec3(0.299, 0.587, 0.114)); // Grayscale conversion // Sobel Gx if (i == -1 && j == -1) { Gx += gray * -1.0; } if (i == -1 && j == 0) { Gx += gray * 0.0; } if (i == -1 && j == 1) { Gx += gray * 1.0; } if (i == 0 && j == -1) { Gx += gray * -2.0; } if (i == 0 && j == 0) { Gx += gray * 0.0; } if (i == 0 && j == 1) { Gx += gray * 2.0; } if (i == 1 && j == -1) { Gx += gray * -1.0; } if (i == 1 && j == 0) { Gx += gray * 0.0; } if (i == 1 && j == 1) { Gx += gray * 1.0; } // Sobel Gy if (i == -1 && j == -1) { Gy += gray * -1.0; } if (i == -1 && j == 0) { Gy += gray * -2.0; } if (i == -1 && j == 1) { Gy += gray * -1.0; } if (i == 0 && j == -1) { Gy += gray * 0.0; } if (i == 0 && j == 0) { Gy += gray * 0.0; } if (i == 0 && j == 1) { Gy += gray * 0.0; } if (i == 1 && j == -1) { Gy += gray * 1.0; } if (i == 1 && j == 0) { Gy += gray * 2.0; } if (i == 1 && j == 1) { Gy += gray * 1.0; } } } // Calculate gradient magnitude float edge = length(vec2(Gx, Gy)); // Normalize edge value edge = clamp(edge * 0.2, 0.0, 1.0); // Adjust multiplier as needed // Apply gradient shading based on edge proximity vec3 originalColor = texture2D(u_sampler, texCoord).rgb; vec3 shadedColor = originalColor * (1.0 - edge * 0.5); // Adjust 0.5 for shading intensity gl_FragColor = vec4(shadedColor, texture2D(u_sampler, texCoord).a); }GML Code for the Object:
// Create Event // Load shaders vertex_shader = shader_get_builtin_vertex(shader_flat); // Using GameMaker's built-in flat vertex shader // Read the fragment shader code from file or define it as a string // For illustration, we're defining it as a string. In practice, load it from a file. var fragment_code = " varying vec2 v_vTexcoord; varying vec4 v_vColour; uniform sampler2D u_sampler; // Parameters for edge detection uniform float u_resolution_x; uniform float u_resolution_y; void main() { // Texture coordinates vec2 texCoord = v_vTexcoord; // Define pixel size based on resolution vec2 pixel = vec2(1.0 / u_resolution_x, 1.0 / u_resolution_y); // Sobel operator kernels float Gx = 0.0; float Gy = 0.0; // Sample surrounding pixels for(int i = -1; i <= 1; i++) { for(int j = -1; j <=1; j++) { vec2 offset = vec2(float(i), float(j)) * pixel; vec4 sampleColor = texture2D(u_sampler, texCoord + offset); float gray = dot(sampleColor.rgb, vec3(0.299, 0.587, 0.114)); // Grayscale conversion // Sobel Gx if (i == -1 && j == -1) { Gx += gray * -1.0; } if (i == -1 && j == 0) { Gx += gray * 0.0; } if (i == -1 && j == 1) { Gx += gray * 1.0; } if (i == 0 && j == -1) { Gx += gray * -2.0; } if (i == 0 && j == 0) { Gx += gray * 0.0; } if (i == 0 && j == 1) { Gx += gray * 2.0; } if (i == 1 && j == -1) { Gx += gray * -1.0; } if (i == 1 && j == 0) { Gx += gray * 0.0; } if (i == 1 && j == 1) { Gx += gray * 1.0; } // Sobel Gy if (i == -1 && j == -1) { Gy += gray * -1.0; } if (i == -1 && j == 0) { Gy += gray * -2.0; } if (i == -1 && j == 1) { Gy += gray * -1.0; } if (i == 0 && j == -1) { Gy += gray * 0.0; } if (i == 0 && j == 0) { Gy += gray * 0.0; } if (i == 0 && j == 1) { Gy += gray * 0.0; } if (i == 1 && j == -1) { Gy += gray * 1.0; } if (i == 1 && j == 0) { Gy += gray * 2.0; } if (i == 1 && j == 1) { Gy += gray * 1.0; } } } // Calculate gradient magnitude float edge = length(vec2(Gx, Gy)); // Normalize edge value edge = clamp(edge * 0.2, 0.0, 1.0); // Adjust multiplier as needed // Apply gradient shading based on edge proximity vec3 originalColor = texture2D(u_sampler, texCoord).rgb; vec3 shadedColor = originalColor * (1.0 - edge * 0.5); // Adjust 0.5 for shading intensity gl_FragColor = vec4(shadedColor, texture2D(u_sampler, texCoord).a); } "; // Create the fragment shader fragment_shader = shader_create(vertex_shader, fragment_code); // Check if shader compiled successfully if (fragment_shader == -1) { show_error("Failed to compile fragment shader.", true); } // Set initial shader parameters shader_set(fragment_shader); shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_x"), sprite_width); shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_y"), sprite_height); shader_reset();Draw Event:
// Draw Event // Set the shader shader_set(fragment_shader); // Update shader uniforms in case of sprite scaling or resizing shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_x"), sprite_width); shader_set_uniform_f(shader_get_uniform(fragment_shader, "u_resolution_y"), sprite_height); // Draw the sprite with shader applied draw_self(); // Reset shader to default to avoid affecting other draw calls shader_reset();Cleanup Event:
// Cleanup Event // Delete shaders to free resources if (shader_exists(fragment_shader)) { shader_delete(fragment_shader); } if (shader_exists(vertex_shader)) { shader_delete(vertex_shader); }
Conclusion
By leveraging GLSL shaders within GameMaker Studio, you can efficiently detect enclosed or tight spaces within sprites and apply dynamic gradient shading based on these detections. This method ensures real-time performance and allows for flexible adjustments to the shading effect.
Key Takeaways:
-
Shaders Over GML for Pixel Operations: For tasks involving per-pixel manipulation like edge detection and gradient shading, shaders are the optimal choice due to their performance benefits.
-
Flexibility: Shaders can be adjusted to tweak the sensitivity of edge detection and the intensity of shading, allowing for customizable visual effects.
-
Resource Management: Always manage shader resources carefully to prevent memory leaks, especially when dynamically creating and deleting shaders.
Feel free to customize the shaders further to match the specific aesthetic requirements of your game. Happy coding!