Pixel perfect 3D outline

For a stylized or toon shaded effect, the outline we have built is pretty good. It’s also decent if the camera is at a fixed distance from the object and the object doesn’t change size. But there’s a problem if you want to have an object outline be the same size from far away, or with an object that changes scale. Because we’re outlining the object using the distance from its vertices, the outline width and the distance will vary depending on distance and scale.

It’s possible to specify the width of the outline using pixels, and it’s always going to be the same regardless of size and distance. That’s good for UI highlights or x-ray effects.

Rat with a constant outline regardless of angle or distance

Object, view, clip and ndc space coordinates

Five coordinate systems are often relevant in shaders:

Matrices are what lets you go from one space to another. They are 4x4 tables of numbers that represent the translation, rotation, scale and skew of each of the views.

In the outline shader, we calculated the outline width in Object space by using the raw VERTEX and NORMAL built-ins before Godot transformed them into view space.

What if, instead, we transformed everything we need into clip space while we have control? Then we could specify the outline size using pixels.

The POSITION built-in

Godot’s shaders work by giving you a space to do some math inside of the actual GLSL shader. It’s what allows you to write a blank shader but still have it work. In the Vertex shader, Godot sets up the preliminary work; your code runs, then Godot applies matrices for you. OpenGL transfers the final vertex position in clip space using the gl_Position constant.

To give the user the option of controlling that outcome, Godot provides the POSITION built-in. When written to, no matrix transformation occurs. Godot expects you to output the final position yourself.

Getting vertex and normal in clip space

With all that theory out of the way, we can write some code. The beginning of the shader is the same as the previous outline. Spatial shader, cull front, unshaded, thickness, and color.

shader_type spatial;
render_mode cull_front, unshaded;

uniform vec4 outline_color : hint_color;
uniform float outline_width = 1.0;

void vertex() {

}

Remember that the VERTEX built-in we have is in Object space. To get to Clip space, we have to go into View space, then into projection space.

To go from Object to View, Godot provides the built-in matrix MODELVIEW_MATRIX. It combines the camera’s view matrix and the model’s transform. To go from View to Clip, Godot provides the built-in matrix PROJECTION_MATRIX.

In Matrix multiplication, the order is vital. Multiplying by a translation matrix and then a rotation matrix gives a very different result than the other way around.

We start by transforming the vertex position by the model-view matrix, then multiply it by the projection matrix, enforcing the order using parenthesis.

We wrap the vertex inside of a vec4 because the fourth w parameter contains the perspective correcting skew we can use.

vec4 clip_position = PROJECTION_MATRIX * (MODELVIEW_MATRIX * vec4(VERTEX, 1.0));

We do the same with the normal. We don’t care about the fourth parameter and transform the matrices into 3x3 with a cast into mat3.

vec3 clip_normal = mat3(PROJECTION_MATRIX) * (mat3(MODELVIEW_MATRIX) * NORMAL);

Now we have a position and a normal in clip space. We can extrude it in 2D by extending the normal in the X and Y direction.

Applying pixel size

The offset is along the X and Y axis and is the size of the outline in pixels.

vec2 offset = normalize(clip_normal.xy) * outline_width;

But there’s a minor issue. Most screens have one axis larger than the other. Often the horizontal is larger than the vertical, so the pixels we’re dealing with are not square. The X normal is not going to be the same size as the Y normal.

To correct for the aspect ratio, we can divide by the size of the screen. In Clip space, we have a range of 0 to 1, but when the vertex position projects onto NDC space after the vertex shader, it will have double the range. We account for that by doubling the offset.

vec2 offset = normalize(clip_normal.xy) / VIEWPORT_SIZE * outline_width * 2.0;

Now the pixels are square, and a value of 1 equates to one pixel. We can add this offset to the clip_position’s xy parameters, then output the final calculated position into the POSITION built-in.

clip_position.xy += offset;

POSITION = clip_position;

Correcting for perspective

But things aren’t quite right yet.

Perspective issue due to distance

Vertices that are further away from the camera are thinner, and those that are closer are thicker. This is because the camera we are using has a Perspective effect to it. Objects that are farther away are smaller.

When the GPU projects the Clip space vertex into NDC space, it divides its xyz by the w component of the vertex to get the final projection. We can cancel this shift in perspective by multiplying the offset by the w component.

vec2 offset = normalize(clip_normal.xy) / VIEWPORT_SIZE * clip_position.w * outline_width * 2.0;
Fixed perspective issue

Now all lines are the same size, regardless of distance.

Color

The fragment shader is the same as in the regular outline. We output the color in ALBEDO and the alpha in ALPHA.

void fragment() {
    ALBEDO = outline_color.rgb;
    ALPHA = outline_color.a;
}

The final shader code

shader_type spatial;
render_mode cull_front, unshaded;

uniform vec4 outline_color : hint_color;
uniform float outline_width = 1.0;

void vertex() {
    vec4 clip_position = PROJECTION_MATRIX * (MODELVIEW_MATRIX * vec4(VERTEX, 1.0));
    vec3 clip_normal = mat3(PROJECTION_MATRIX) * (mat3(MODELVIEW_MATRIX) * NORMAL);

    vec2 offset = normalize(clip_normal.xy) / VIEWPORT_SIZE * clip_position.w * outline_width * 2.0;

    clip_position.xy += offset;

    POSITION = clip_position;
}

void fragment() {
    ALBEDO = outline_color.rgb;
    if(outline_color.a < 1.0) {
        ALPHA = outline_color.a;
    }
}