Blendshape Target (Morph) Normals Issues

August 2022

On This Page

TLDR;

Due to it's highly efficient runtime performance, Unreal Engine's posedriver technology offers an unprecedented opportunity to use JCMs to achieve near photo-realistic body deformations in realtime.

However, due to the lack of blendshape target vertex normal information in fbx files, the application of many JCMs causes shading artefacts and so severely diminishes the tech's potential:


The omission of blendshape target normal information causes shading errors

Epic's proposed solution is to use the 'Recompute Tangents' functionality. Whilst this does fix the shading errors, it introduces seam matching issues instead, as can be seen under the jaw and around the armpit in the images below:



Using 'Recompute Tangents' causes UV seams to be visible

As can be seen, using 'Recompute Tangents' in Unreal Engine is the lesser of two evils - it is not a perfect solution.

Summary Of The Problem

By design, JCMs and other morphs (aka blendshape targets or shapekeys) exported inside fbx files do not contain vertex normal information. This leads to incorrect normals being displayed when JCMs and other morphs are applied. This increasingly becomes problematic as the amount of vertex offset increases and is made exponentially worse if the mesh is twisted by the morph. Corrections that are considered essential for the shoulder, hip and knee joints are the worst affected. By using 'Recompute Tangents' in Unreal Engine the shading errors can be overcome. However, this solution causes seam matching issues.

Historically, both Autodesk and Unreal Engine's makers Epic have said they do not plan to address the issue.

Autodesk's Response

The issue stems from Autodesk's decision to not export vertex normals information with blendshape targets. The topic is discussed in a troubleshooting article from June 2021 where a representative from Autodesk says the following:

'This is a purposeful design choice intended to keep files small.'

I feel it would be better if this choice was left to users - if file size is an issue then the number of morphs / blendshape targets exported can be reduced accordingly. Or even better, it would be great to have the option of exporting normals on a target-by-target basis, as not all blendshape targets deform the mesh sufficiently to warrant the inclusion of normals information. It is also worth bearing in mind that for most users these days, neither bandwidth or hard drive space is really much of an issue; so it is a challenge to understand Autodesk's reasoning here.

In their response, Autodesk also suggest a number of workarounds:

'Ideally, adding more geometry to the model rather than altering the normals would avoid this issue entirely'
I have found that the issue is still a problem with the higher poly count Daz figures (see image below). This is because the issue is only indirectly related to vertex density - the amount of vertex offset and any twisting of the mesh performed by the morph are far greater factors. With this in mind, I have to disagree with the statement that adding more geometry would avoid the issue entirely.

  
Test results from using higher resolution models (Maya 2020 fbx export to UE5)
Note the artefacts on the knees here are actually slightly worse in the case of the higher resolution models

'Export Targets Individually'
If UE were to have morph loader functionality this would work - morphs could be exported in the form of obj files (complete with normal information on the vertices) for each character model, batch imported into UE and applied to the skeletal mesh. However, UE does not currently have this functionality. This would be a viable solution otherwise.

'Use the "Send to [Unreal]" feature to export [from Maya]'
This has been tested. Using the "Send to [Unreal]" export path does not solve the issue.

Unreal Engine / Epic's Response

The issue has been reported as a bug in Unreal Engine a number of times:

  • Morph Target Normals behaving strangely when deforming UE-19861 (August 2015)
  • Skeletal Mesh Imported Normals do not update correctly UE-26894 (February 2016)
  • Unexpected vertex normal orientation when manipulating some morph targets UE-33456 (July 2016)

Epic has marked the bug as either Won't Fix or By Design each time it has been reported. On UE-33456 there is a developer note saying the following:

'We don't plan to work on this,
users can use the GPU recompute tangent option for accurate normals with morph targets.'

Whilst using 'Recompute Tangents' (as described here) method works quite well, it does make the UV seams visible as shown above.

Community Response

There are forums discussions about this issue going back several years (note this is not an exhaustive list)

Solution

The ideal situation would be whereby Autodesk update the fbx file specification to include normals information for blendshape targets and Epic modify their engine code to make use of the new functionality. However, at the time of writing this seems unlikely.

A second solution would be for Epic to implement morph loading functionality. Morphs could be exported in the form of obj files (complete with normal information on the vertices) from the DCC application and loaded into the Engine, complete with normals information.

↑