[UE4]Shader Development Notes

Keywords: UE4, Shader Development, Material Optimization

Common Case

Environment Setup

1, Set LoadingPhase as PostConfigInit, default is Default.
e.g. TestProj.uproject:

{
    "FileVersion": 3,
    "EngineAssociation": "4.25",
    "Category": "",
    "Description": "",
    "Modules": [
        {
            "Name": "TestProj",
            "Type": "Runtime",
            "LoadingPhase": "PostConfigInit"
        }
    ]
}

Otherwise there’s error on building:

Shader type was loaded after engine init, use ELoadingPhase::PostConfigInit on your module to cause it to load earlier.

2, Turn on r.ShaderDevelopmentMode in \Engine\Config\ConsoleVariables.ini.
ConsoleVariables.ini:

[Startup]
; Uncomment to get detailed logs on shader compiles and the opportunity to retry on errors
r.ShaderDevelopmentMode=1

Otherwise there’s error on building:

Failed to compile global shader XXXX. Enable `r.ShaderDevelopmentMode` in ConsoleVariables.ini for retries.

3, Add RenderCore, Renderer in your .Build.cs. Maybe RHI is also need to be added.

PublicDependencyModuleNames.AddRange(new string[] { "RenderCore", "Renderer", "RHI" });

4, Now you can setup your shader source in your game source, and these shader would be compiled on editor starting.
MyActor.cpp (quoted from Engine\Source\Runtime\Renderer\Private\PostProcess\PostProcessVisualizeBuffer.cpp):

namespace
{
    class FVisualizeBufferPS : public FGlobalShader
    {
    public:
        DECLARE_GLOBAL_SHADER(FVisualizeBufferPS);
        SHADER_USE_PARAMETER_STRUCT(FVisualizeBufferPS, FGlobalShader);

        BEGIN_SHADER_PARAMETER_STRUCT(FParameters, )
            SHADER_PARAMETER_STRUCT(FScreenPassTextureViewportParameters, Output)
            SHADER_PARAMETER_RDG_TEXTURE(Texture2D, InputTexture)
            SHADER_PARAMETER_SAMPLER(SamplerState, InputSampler)
            SHADER_PARAMETER(FLinearColor, SelectionColor)
            RENDER_TARGET_BINDING_SLOTS()
        END_SHADER_PARAMETER_STRUCT()

        static bool ShouldCompilePermutation(const FGlobalShaderPermutationParameters& Parameters)
        {
            return IsFeatureLevelSupported(Parameters.Platform, ERHIFeatureLevel::ES3_1);
        }
    };

    IMPLEMENT_GLOBAL_SHADER(FVisualizeBufferPS, "/Engine/Private/PostProcessVisualizeBuffer.usf", "MainPS", SF_Pixel);

    FScreenPassTexture AddVisualizeBufferPass(FRDGBuilder& GraphBuilder, const FViewInfo& View, const FVisualizeBufferInputs& Inputs)
    {
        ...
        FVisualizeBufferPS::FParameters* PassParameters = GraphBuilder.AllocParameters<FVisualizeBufferPS::FParameters>();
        PassParameters->Output = GetScreenPassTextureViewportParameters(OutputViewport);
        PassParameters->RenderTargets[0] = Output.GetRenderTargetBinding();
        PassParameters->InputTexture = Tile.Input.Texture;
        PassParameters->InputSampler = BilinearClampSampler;
        PassParameters->SelectionColor = SelectionColor;
        
        TShaderMapRef<FVisualizeBufferPS> PixelShader(View.ShaderMap);
        ...
    }
    
    void Test_RenderThread(FRHICommandListImmediate& RHICmdList)
    {
        ...
        TShaderMapRef<FDownsamplePS> PixelShader(ShaderMap, PermutationVector);
        ...
    }
}

AMyActor::TestFun()
{
    ENQUEUE_RENDER_COMMAND(FDownsampleTestCommand)(
    [](FRHICommandListImmediate& RHICmdList)
    {
        Test_RenderThread(RHICmdList);
    });
}

5, If want to specify the Shader Type (Global, Material, MeshMaterial, Niagara etc.), use IMPLEMENT_SHADER_TYPE instead of IMPLEMENT_GLOBAL_SHADER, see the example of PostProcessMaterial.cpp in engine.
quoted from Engine\Source\Runtime\Renderer\Private\PostProcess\PostProcessMaterial.cpp:

IMPLEMENT_SHADER_TYPE(,FPostProcessMaterialVS, TEXT("/Engine/Private/PostProcessMaterialShaders.usf"), TEXT("MainVS"), SF_Vertex);
IMPLEMENT_SHADER_TYPE(,FPostProcessMaterialPS, TEXT("/Engine/Private/PostProcessMaterialShaders.usf"), TEXT("MainPS"), SF_Pixel);

How to dump shader statistics of all materials

UE5 solution:

"D:/UE_5.2/Engine/Binaries/Win64/UnrealEditor-Cmd.exe" "D:/TestProj/TestProj.uproject" -run=DumpMaterialShaderTypes  -targetplatform=Windows

Then results output to [MyProj]/Saved/MaterialStats/.txt.

DumpMaterialShaderTypes only works for UE5.

UE4 solution:
Turn on follows switch in Engine\Config\ConsoleVariables.ini and restart editor (if want to build package from editor):

r.DumpShaderDebugInfo=1
r.DumpShaderDebugShortNames=1

Then execute cook command:

D:/UE4/Engine/Binaries/Win64/UE4Editor-Cmd.exe D:/TestTP/TestTP.uproject -run=Cook -TargetPlatform=WindowsNoEditor -fileopenlog -ddc=DerivedDataBackendGraph -unversioned -abslog=D:/UE4/Engine/Programs/AutomationTool/Saved/Cook-2023.06.09-17.05.16.txt -stdout -CrashForUAT -unattended -NoLogTimes -UTF8Output

This cook command also will be executed automatically while building Development package.

Then statistics data output to directory: [MyProj]/Saved/MaterialStats/*.csv.
UE4’s cook command only output sum of shader count, UE5’s command output shader count and shader type.

References

Official Documents

Shader Development
Adding Global Shaders to Unreal Engine
Shaders In Plugins

Blogs

Extend the UE4 Shading Model
https://qiita.com/dgtanaka/items/41f96ef2090820035609

Unreal Engine 4 Rendering Part 1: Introduction
Unreal Engine 4 Rendering Part 2: Shaders and Vertex Data
Unreal Engine 4 Rendering Part 3: Drawing Policies
Unreal Engine 4 Rendering Part 4: The Deferred Shading Pipeline
Unreal Engine 4 Rendering Part 5: Shader Permutations
Unreal Engine 4 Rendering Part 6: Adding a new Shading Model

剖析虚幻渲染体系（08）- Shader体系
https://www.cnblogs.com/timlly/p/15092257.html

Material Optimization

Material Optimization
https://forums.unrealengine.com/t/material-optimization/59111

How the Unreal Engine Translates a Material Graph to HLSL
https://forums.unrealengine.com/docs?topic=498490

Shader Permutation

Understanding Shader Permutations
https://forums.unrealengine.com/t/understanding-shader-permutations/264928

The Shader Permutation Problem - Part 1: How Did We Get Here?
https://therealmjp.github.io/posts/shader-permutations-part1/
The Shader Permutation Problem - Part 2: How Do We Fix It?
https://therealmjp.github.io/posts/shader-permutations-part2/

Vertex Shader

Rotating meshes using Vertex Shaders
https://www.tomlooman.com/unreal-engine-material-vertex-shaders/

The “Normal”-pin runs on the pixel-shader while the “World Position Offset”-pin runs on the vertex shader.

Customized UVs: Feature that allows running calculations in the vertex shader to increase performance over running them per-pixel.
https://docs.unrealengine.com/4.27/en-US/RenderingAndGraphics/Materials/CustomizedUVs/

Compiling

Compiling Shaders Manually
http://mpolaczyk.pl/compiling-shaders-manually/

Examples

Unreal Official Shaders
https://github.com/EpicGames/UnrealEngine/tree/release/Engine/Shaders/Private

A tutorial project that shows how to implement HLSL Pixel and Compute shaders in UE4
https://github.com/Temaran/UE4ShaderPluginDemo

A compute shader plugin that is capable of sorting positional data in parallel directly on the GPU.
https://github.com/ValentinKraft/UE4_SortingComputeShader

The minimal source code for adding and using a custom compute shader in Unreal Engine 4
https://github.com/AyoubKhammassi/CustomComputeShaders

Business has only two functions — marketing and innovation. ― Peter Drucker