[UE4][Animation]Runtime Procedural Programming

Keywords: UE4, Animation, Runtime Procedural Programming

Usages

How to get bone name list of SkeletalMesh

void AMyCharacter::BeginPlay()
{
    Super::BeginPlay();
    
    if(USkeletalMeshComponent* SkMeshComp = GetMesh())
    {
        TArray<FName> OutBoneNames;
        SkMeshComp->GetBoneNames(OutBoneNames);
    }
}

How to extract transform of bones from animation sequence

.Build.cs

PublicDependencyModuleNames.AddRange(new string[] { "AnimationModifiers" });

cpp source

void AMyCharacter::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);
    
    if (USkeletalMeshComponent* SkMeshComp = GetMesh())
    {
        TArray<FName> OutBoneNames;
        SkMeshComp->GetBoneNames(OutBoneNames);

        if (UAnimSequence* AnimSeq = LoadObject<UAnimSequence>(nullptr, TEXT("AnimSequence'/Game/Mannequin/Animations/ThirdPersonWalk.ThirdPersonWalk'")))
        {
            const float FrameCost = 1.f / 30;

            AnimTime += FrameCost;
            
            if(AnimTime < AnimSeq->SequenceLength)
            {
                TArray<FTransform> OutPoses;
                UAnimationBlueprintLibrary::GetBonePosesForTime(AnimSeq, OutBoneNames, AnimTime, true, OutPoses);

                int TestIndex = 54;

                FName BoneName = OutBoneNames[TestIndex];

                FTransform Trans = OutPoses[TestIndex];
                FVector Translation = Trans.GetTranslation();
                FRotator Rotation = Trans.Rotator();
                FVector Scale = Trans.GetScale3D();

                FString Text = FString::Printf(TEXT("Bone: %s Time: %f Loc: %s Rot: %s"), *BoneName.ToString(), AnimTime, *Translation.ToString(), *Rotation.ToString());
                GEngine->AddOnScreenDebugMessage(0, 3.f, FColor::Yellow, Text, false);
            }
        }
    }
}

How to get user data which animation asset was stored

Engine\Source\Runtime\Engine\Classes\Engine\SkeletalMesh.h

UPROPERTY(EditAnywhere, AdvancedDisplay, Instanced, Category=SkeletalMesh)
TArray<UAssetUserData*> AssetUserData;

How to get bone index from bone name

Engine\Source\Runtime\Engine\Classes\Engine\SkeletalMesh.h

RefSkeleton.FindBoneIndex(BoneName);

Engine\Source\Runtime\Engine\Classes\Components\SkinnedMeshComponent.h

int32 USkinnedMeshComponent::GetNumBones()const;

int32 USkinnedMeshComponent::GetBoneIndex( FName BoneName) const;

FName USkinnedMeshComponent::GetBoneName(int32 BoneIndex) const;

Get Animation CompactPose (Transform) from AnimSequence by Current Time

Quoted from void FAnimInstanceProxy::SlotEvaluatePose()

FSlotEvaluationPose NewPose(EvalState.MontageWeight, AdditiveAnimType);

// Bone array has to be allocated prior to calling GetPoseFromAnimTrack
NewPose.Pose.SetBoneContainer(&RequiredBones);
NewPose.Curve.InitFrom(RequiredBones);

// Extract pose from Track
FAnimExtractContext ExtractionContext(EvalState.MontagePosition, Montage->HasRootMotion() && RootMotionMode != ERootMotionMode::NoRootMotionExtraction);
AnimTrack->GetAnimationPose(NewPose.Pose, NewPose.Curve, ExtractionContext);

API

Common API in AnimationBlueprintLibrary

Engine\Source\Editor\AnimationModifiers\Public\AnimationBlueprintLibrary.h

How to get bone pose:

/** Retrieves Bone Pose data for the given Bone Names at the specified Time from the given Animation Sequence */
UFUNCTION(BlueprintPure, Category = "AnimationBlueprintLibrary|Pose")
static void GetBonePosesForTime(const UAnimSequence* AnimationSequence, TArray<FName> BoneNames, float Time, bool bExtractRootMotion, TArray<FTransform>& Poses, const USkeletalMesh* PreviewMesh = nullptr);

/** Retrieves Bone Pose data for the given Bone Names at the specified Frame from the given Animation Sequence */
UFUNCTION(BlueprintPure, Category = "AnimationBlueprintLibrary|Pose")
static void GetBonePosesForFrame(const UAnimSequence* AnimationSequence, TArray<FName> BoneNames, int32 Frame, bool bExtractRootMotion, TArray<FTransform>& Poses, const USkeletalMesh* PreviewMesh = nullptr);

How to add or remove notify to animation at run-time:

/** Adds an Animation Notify Event to Notify track in the given Animation with the given Notify creation data */
UFUNCTION(BlueprintCallable, Category = "AnimationBlueprintLibrary|NotifyEvents")
static UAnimNotify* AddAnimationNotifyEvent(UAnimSequence* AnimationSequence, FName NotifyTrackName, float StartTime, TSubclassOf<UAnimNotify> NotifyClass);

/** Adds an Animation Notify State Event to Notify track in the given Animation with the given Notify State creation data */
UFUNCTION(BlueprintCallable, Category = "AnimationBlueprintLibrary|NotifyEvents")
static UAnimNotifyState* AddAnimationNotifyStateEvent(UAnimSequence* AnimationSequence, FName NotifyTrackName, float StartTime, float Duration, TSubclassOf<UAnimNotifyState> NotifyStateClass);

/** Adds an the specific Animation Notify to the Animation Sequence (requires Notify's outer to be the Animation Sequence) */
UFUNCTION(BlueprintCallable, Category = "AnimationBlueprintLibrary|NotifyEvents")
static void AddAnimationNotifyEventObject(UAnimSequence* AnimationSequence, float StartTime, UAnimNotify* Notify, FName NotifyTrackName);

/** Adds an the specific Animation Notify State to the Animation Sequence (requires Notify State's outer to be the Animation Sequence) */
UFUNCTION(BlueprintCallable, Category = "AnimationBlueprintLibrary|NotifyEvents")
static void AddAnimationNotifyStateEventObject(UAnimSequence* AnimationSequence, float StartTime, float Duration, UAnimNotifyState* NotifyState, FName NotifyTrackName);

/** Removes Animation Notify Events found by Name within the Animation Sequence, and returns the number of removed name instances */
UFUNCTION(BlueprintCallable, Category = "AnimationBlueprintLibrary|NotifyEvents")
static int32 RemoveAnimationNotifyEventsByName(UAnimSequence* AnimationSequence, FName NotifyName);

/** Removes Animation Notify Events found by Track within the Animation Sequence, and returns the number of removed name instances */
UFUNCTION(BlueprintCallable, Category = "AnimationBlueprintLibrary|NotifyEvents")
static int32 RemoveAnimationNotifyEventsByTrack(UAnimSequence* AnimationSequence, FName NotifyTrackName);

Common API in BonePose

Engine\Source\Runtime\Engine\Public\BonePose.h

// Populate InOutPose based on raw animation data. 
extern void BuildPoseFromRawData(const TArray<FRawAnimSequenceTrack>& InAnimationData, const TArray<struct FTrackToSkeletonMap>& TrackToSkeletonMapTable, FCompactPose& InOutPose, float InTime, EAnimInterpolationType Interpolation, int32 NumFrames, float SequenceLength, FName RetargetSource);

Common API in AnimInstance

Engine\Source\Runtime\Engine\Classes\Animation\AnimInstance.h

// the below functions are the native overrides for each phase
// Native initialization override point
virtual void NativeInitializeAnimation();

// Native update override point. It is usually a good idea to simply gather data in this step and 
// for the bulk of the work to be done in NativeUpdateAnimation.
virtual void NativeUpdateAnimation(float DeltaSeconds);

// Native Post Evaluate override point
virtual void NativePostEvaluateAnimation();

// Native Uninitialize override point
virtual void NativeUninitializeAnimation();

// Executed when begin play is called on the owning component
virtual void NativeBeginPlay();

Common API in AnimMontage

Tick function on animation simulatiing:

/** Simulate is same as Advance, but without calling any events or touching any of the instance data. So it performs a simulation of advancing the timeline. */
bool SimulateAdvance(float DeltaTime, float& InOutPosition, struct FRootMotionMovementParams & OutRootMotionParams) const;
void Advance(float DeltaTime, struct FRootMotionMovementParams * OutRootMotionParams, bool bBlendRootMotion);

AnimationEditorPreviewActor

Path:
Engine\Source\Editor\Persona\Public\AnimationEditorPreviewActor.h

AnimationEditorPreviewActor is an Actor used to preview AnimSequence.
Example:

void UMyAnimNotify::Notify(USkeletalMeshComponent* MeshComp, UAnimSequenceBase* Animation)
{
    if(AActor* Actor = MeshComp->GetOwner())
    {
        if(GIsEditor)
        {
            AAnimationEditorPreviewActor* PrevActor = Cast<AAnimationEditorPreviewActor>(Actor);
        }
        else
        {
            AMyCharacter* Character = Cast<AMyCharacter>(Actor);
        }
    }
}

Build.cs:

PublicDependencyModuleNames.Add("Persona");

Engine Source Analytics

Callstack on `FAnimNode_SkeletalControlBase::EvaluateSkeletalControl_AnyThread()`

UE4Editor-CustomAnimNode-Win64-DebugGame.dll!FAnimNode_SpeedWarping::EvaluateSkeletalControl_AnyThread(FComponentSpacePoseContext & Output, TArray<FBoneTransform,TSizedDefaultAllocator<32>> & OutBoneTransforms) Line 32    C++
UE4Editor-AnimGraphRuntime.dll!FAnimNode_SkeletalControlBase::EvaluateComponentSpace_AnyThread(FComponentSpacePoseContext & Output) Line 165    C++
UE4Editor-Engine.dll!FComponentSpacePoseLink::EvaluateComponentSpace(FComponentSpacePoseContext & Output) Line 443    C++
UE4Editor-Engine.dll!FAnimNode_ConvertComponentToLocalSpace::Evaluate_AnyThread(FPoseContext & Output) Line 34    C++
UE4Editor-Engine.dll!FPoseLink::Evaluate(FPoseContext & Output) Line 381    C++
UE4Editor-Engine.dll!FAnimInstanceProxy::EvaluateAnimationNode_WithRoot(FPoseContext & Output, FAnimNode_Base * InRootNode) Line 1365    C++
UE4Editor-Engine.dll!FAnimInstanceProxy::EvaluateAnimation_WithRoot(FPoseContext & Output, FAnimNode_Base * InRootNode) Line 1298    C++
UE4Editor-Engine.dll!UAnimInstance::ParallelEvaluateAnimation(bool bForceRefPose, const USkeletalMesh * InSkeletalMesh, FBlendedHeapCurve & OutCurve, FCompactPose & OutPose) Line 658    C++
UE4Editor-Engine.dll!USkeletalMeshComponent::EvaluateAnimation(const USkeletalMesh * InSkeletalMesh, UAnimInstance * InAnimInstance, FVector & OutRootBoneTranslation, FBlendedHeapCurve & OutCurve, FCompactPose & OutPose) Line 1755    C++
UE4Editor-Engine.dll!USkeletalMeshComponent::PerformAnimationProcessing(const USkeletalMesh * InSkeletalMesh, UAnimInstance * InAnimInstance, bool bInDoEvaluation, TArray<FTransform,TSizedDefaultAllocator<32>> & OutSpaceBases, TArray<FTransform,TSizedDefaultAllocator<32>> & OutBoneSpaceTransforms, FVector & OutRootBoneTranslation, FBlendedHeapCurve & OutCurve) Line 1838    C++
UE4Editor-Engine.dll!USkeletalMeshComponent::ParallelAnimationEvaluation() Line 3528    C++
UE4Editor-Engine.dll!FParallelAnimationEvaluationTask::DoTask(ENamedThreads::Type CurrentThread, const TRefCountPtr<FGraphEvent> & MyCompletionGraphEvent) Line 110    C++
UE4Editor-Engine.dll!TGraphTask<FParallelAnimationEvaluationTask>::ExecuteTask(TArray<FBaseGraphTask *,TSizedDefaultAllocator<32>> & NewTasks, ENamedThreads::Type CurrentThread) Line 847    C++
[Inline Frame] UE4Editor-Core.dll!FBaseGraphTask::Execute(TArray<FBaseGraphTask *,TSizedDefaultAllocator<32>> & CurrentThread, ENamedThreads::Type) Line 514    C++
UE4Editor-Core.dll!FTaskThreadAnyThread::ProcessTasks() Line 1029    C++
UE4Editor-Core.dll!FTaskThreadAnyThread::ProcessTasksUntilQuit(int QueueIndex) Line 855    C++
[Inline Frame] UE4Editor-Core.dll!FTaskThreadBase::Run() Line 524    C++
UE4Editor-Core.dll!FTaskThreadAnyThread::Run() Line 931    C++
UE4Editor-Core.dll!FRunnableThreadWin::Run() Line 96    C++
UE4Editor-Core.dll!FRunnableThreadWin::GuardedRun() Line 45    C++

Issues

Animation parallel simulation: poses of character could not match if play two animation separately

Case:
If character was composed by multiple skeletal mesh (e.g. one skeletal mesh is head, another skeletal mesh is arm), and thess skeletal meshes have dedicated AnimSequence asset, and these AnimSequence are in synchronized pose transform, when you play these apart animation at the same time, arm’s pose and head’s pose don’t match seamlessly.

Caused by:
Multi-thread animation simulating is on by default, and if there’re some overhead under other work threads, then animations would not match with each other seamlessly.

Solution:
Override CanRunParallelWork of UAnimInstance, and return false;

// Can this animation instance run Update or Evaluation work in parallel
virtual bool CanRunParallelWork() const override { return false; }

Or set bUseMultiThreadedAnimationUpdate as false, and uncheck Allow Multi Threaded Animation Update in project settings.

Refresh bone transform for a certain frame on dedicated server

If set VisibilityBasedAnimTickOption as OnlyTickPoseWhenRendered on dedicated server:

if(GetMesh())
{
    GetMesh()->VisibilityBasedAnimTickOption = EVisibilityBasedAnimTickOption::OnlyTickPoseWhenRendered;
}

bone transform can’t be update, so we can’t get correct socket transform.
Solution:

if (SkelMeshComponent->AnimScriptInstance)
{
    // Tick the animation
    SkelMeshComponent->AnimScriptInstance->UpdateAnimation(0.f, false);
}
SkelMeshComponent->RefreshBoneTransforms();

Origin: 【UE4】在 Dedicate Server 上刷新一帧骨骼 Mesh Pose
https://zhuanlan.zhihu.com/p/352407350

Docs

Offical Docs

Demystifying Bone Indices
https://www.unrealengine.com/en-US/tech-blog/demystifying-bone-indices

Working with Modular Characters (Skeletal Mesh Merge at runtime)
https://docs.unrealengine.com/4.27/en-US/AnimatingObjects/SkeletalMeshAnimation/WorkingwithModularCharacters/

Cloth: Making the Most of Animation Blueprints | Unreal Fest Europe 2019 (Recommended)
https://www.youtube.com/watch?v=Oe7fYS9qxmk&t=1272s

Motion Warping is a feature where you can dynamically adjust a character’s root motion to align to targets.
https://docs.unrealengine.com/5.0/en-US/motion-warping-in-unreal-engine/

Skeletal Mesh Rendering Paths, different paths of Skeletal Meshes rendering (UE5)
https://docs.unrealengine.com/5.3/en-US/skeletal-mesh-rendering-paths-in-unreal-engine/

Runtime Retargeting

Runtime Animation Retargeting
https://docs.metahuman.unrealengine.com/en-US/retargeting-animations-to-a-metahuman-at-runtime/

UE5 Runtime Retargeting Tutorial
https://www.youtube.com/watch?v=xbFOF8TY_D4

Examples

Examples - Mocap

This plugin enables live-streaming of character animation from the Shadow motion capture system directly into UE4.
https://www.unrealengine.com/marketplace/en-US/product/shadow-mocap-plugin

Tools & Plugins

Skeletal Mesh Merge

Plugin tha merges skeletal meshes at runtime, good for procedural skeletal meshes
https://github.com/Luis24989/SkeletalMeshMerger

Modular Character Plugin, A plugin that can merge skeletalmesh and duplicate morphtarget cloth assets. (Recommended)
https://www.unrealengine.com/marketplace/en-US/product/modularcharacterplugin

Control Rig (UE5)

Documents - Control Rig

Using Control Rig in Unreal Engine
https://www.youtube.com/watch?v=y2WzNvJZk0E

Introduction To Creating And Modifying Control Rig - Official Docs

Control Rig Editor
https://docs.unrealengine.com/5.3/en-US/control-rig-editor-in-unreal-engine/

How to keep the pose from the current frame of animation

Check Set Initial Transform From Mesh in Control Rig details.

Otherwise the skeleton was distroted when mesh play default animation sequence.

I distrust official charity.All charity should be done by stealth. ― Romain Rolland