Drawing simple outlines around 3D objects

The outlining of an object emphasizes it. You may do this to tell the player they can select or interact with an item. You can also use outlines as an artistic choice for cel-shaded models.

In this tutorial, you will learn a simple and efficient shader technique to put a colored outline around a 3D model.

We will use a multi-pass material for 3D objects, manipulate vertex positions with vertex normals, and use culling not to have the outline draw on top of the object.

In short

The shader’s algorithm is simple. We draw the object twice: once with a regular spatial material, then a second time with the mesh expanded along the model’s vertex normals, placed behind the first drawing and unshaded.

Setting up the scene

To get started, we need to create a mesh and its material to apply our shader to:

1. Create a new MeshInstance node.
2. Assign a new Sphere mesh to the Mesh property.
3. Assign a new SpatialMaterial to the Material property.
4. Click on the material you just created to expand the resource and edit it to look to your liking.

That is the first drawing pass on the object. To create an outline, we need to draw a second copy of it. To do so, we’re going to use a second drawing pass with our shader.

To add the second drawing pass:

1. At the bottom of the Inspector, click the box next to the Next Pass property and create a new ShaderMaterial.
2. Expand that newly produced material and assign a new Shader resource to the Shader property.
3. Right-click on the new Shader and select Save to store the shader on your disk. I will name the file outline3D.shader.

Coding the outline shader

Double-click on outline3D.shader to open it, and let’s get coding.

Our shader is going to draw the outline behind the sphere. With this second drawing pass:

• We want a stroke with a flat color. As such, we do not need any lighting on our object. So we will use the unshaded render mode.
• We want to draw behind our sphere. We can use the cull_front render mode for that.

A 3D object is made of triangles that go all the way around, but there is a cost to drawing geometry. The default render mode, cull_back, only draws triangles that the camera is able to see. Because we’re drawing the object a second time, but bigger, this mode would just draw the object on top of the original! cull_front makes it so only the triangles facing away from the camera are drawn instead by culling those facing the camera; in other words, “culling the front” of the object.

shader_type spatial;
render_mode unshaded, cull_front;

Note: To cull means to remove or reject something; in this case, geometry to draw in the viewport.

We will need to expose two parameters to control the size and the color of the outline easily.

To do so, create two uniforms: thickness in world units, a floating-point value, and outline_color, a vec4.

uniform float thickness = 0.1;
uniform vec4 outline_color : hint_color = vec4(1.0);

The hint_color makes a convenient color picker appear next to the property in the Inspector.

The goal of the vertex shader is to draw the object bigger than normal, so it looks like an outline. You accomplish this by moving the vertices along their normal - the direction in space the triangle is pointing to. The NORMAL built-in variable gives you access to that:

void vertex() {
    VERTEX += (NORMAL * thickness);
}

The fragment shader’s job is just to set the color of the outline object. You also want to set the ALPHA, even if you don’t intend on using transparency: it prevents the outline from casting a shadow!

void fragment() {
    ALBEDO = outline_color.rgb;
    ALPHA = outline_color.a;
}

You can now assign this shader to any object to turn it into an outline. You can also save your entire material with its two passes to easily assign it to another object. You will need to duplicate it by right clicking on it and using the Make Unique shortcut; shared materials share properties!

Tweak the thickness and outline_color parameters in the Inspector to modify the outline to your liking.

Here is the complete shader code:

shader_type spatial;
render_mode unshaded, cull_front;

uniform float thickness = 0.1;
uniform vec4 outline_color : hint_color = vec4(1.0);

void vertex() {
    VERTEX += (NORMAL * thickness);
}

void fragment() {
    ALBEDO = outline_color.rgb;
    ALPHA = outline_color.a;
}

Troubleshooting

While you can already use the outline shader at this point, you might encounter some issues.

The outline clips into other objects

As our outline is a 3D mesh, it may go into the floor or intersect with other 3D objects, clipping as a result.

One possible fix is to draw all objects that may cause clipping with the transparent pipeline. To do so, set the Transparent flag on the object’s SpatialMaterial. Then, add the depth_draw_always render mode on the 3D outline shader.

An important caveat is that the transparent pipeline is more expensive to draw and does not cast shadows without the opaque alpha pre-pass depth draw mode parameter (under Parameters of SpatialMaterial.)

A more complex, edge-detection post-process shader or a post-process shader that draws the outline inside a separate viewport would also solve this issue.

Broken outlines

Objects with sharp faces, like a cube without normal smoothing, will have broken outlines:

It’s important to remember that a 3D mesh is made of separate triangular panels. Where you see two faces meet with a sharp edge, they are two independent panels, each with their vertices and vertex normals. In this case, the vertices’ normals point in different directions, which causes that that split in our outline.

There is no simple fix, but there are solutions:

• Instead of offsetting vertices along their normal, scale them up using a multiplication, replacing the use of NORMAL with VERTEX *= (1.0 + thickness). This calculation can work well for simple geometric shapes, like cubes. But this will break with more complex shapes.
• You could write a post-process shader that uses edge detection on the entire display. An image filter like this is more complex and may not produce outlines with the same look.
• Smooth the normals of your object in your 3D editor, and use a normal map for baking the version with hard edges and keeping control of the final shading.
• Smooth the normals of your object in code before you render it, and put those normals in the vertex color channel.

There is an example of the last option in the Godot demo project: SmoothNormalsMeshInstance.gd, and outline3D_smooth_normals_color.shader.