[UE4]Procedural Audio Notes

Keywords: UE4, Procedural Audio Notes

USoundWaveProcedural

Decoding in USoundWaveProcedural

Origin:
Audio decoding to be used for USoundWaveProcedural
https://github.com/BlueMountainsIO/UnrealVlcAudioPlayer

SoundWave = NewObject<USoundWaveProcedural>(USoundWaveProcedural::StaticClass());
SoundWave->SetSampleRate(VLC_SAMPLE_RATE);
SoundWave->NumChannels = VLC_CHANNELS;
SoundWave->Duration = INDEFINITELY_LOOPING_DURATION;
SoundWave->bLooping = false;
SoundWave->bCanProcessAsync = true;
#if ENGINE_MINOR_VERSION <= 22
SoundWave->bVirtualizeWhenSilent = true; // Play sound even when out of range
#else
SoundWave->VirtualizationMode = EVirtualizationMode::PlayWhenSilent;
#endif

auto AudioPlayer = VlcInstance::Get()->CreateAudioPlayer(File);

if (!AudioPlayer->IsValid())
{
    UE_LOG(LogTemp, Error, TEXT("AudioPlayer->IsValid() failed"));
    return false;
}

AudioPlayer->SetEvents(
    [this]() {
        UE_LOG(LogTemp, Warning, TEXT("VLC reports an error %i"), FPlatformTLS::GetCurrentThreadId());
    },
    []() {
        UE_LOG(LogTemp, Warning, TEXT("VLC starts playing! %i"), FPlatformTLS::GetCurrentThreadId());
    },
    [this]() {
        UE_LOG(LogTemp, Warning, TEXT("VLC stopped. %i"), FPlatformTLS::GetCurrentThreadId());
    });

if (!AudioPlayer->Play())
{
    UE_LOG(LogTemp, Error, TEXT("AudioPlayer->Play() failed"));
    return false;
}

weak_ptr<VlcAudioPlayer> WeakPlayer = AudioPlayer;
SoundWave->OnSoundWaveProceduralUnderflow = FOnSoundWaveProceduralUnderflow::CreateLambda([WeakPlayer](USoundWaveProcedural* InProceduralWave, int32 SamplesRequired) {
    if (!WeakPlayer.expired())
    {
        auto AudioPlayer = WeakPlayer.lock();

        if (AudioPlayer)
        {
            FVlcAudioSamples AudioData;
            if (AudioPlayer->GetAudio(&AudioData))
            {
                UE_LOG(LogTemp, VeryVerbose, TEXT("SoundID Underflow requires %i samples, got %i"), SamplesRequired, AudioData.BufferSize);

                InProceduralWave->QueueAudio(AudioData.SamplesBuffer.GetData(), AudioData.BufferSize);
            }
        }
    }
});

Issues

Audio hitched on CPU overheading

Case:
There’re some random hitch (lagging) of voice on decoding using USoundWaveProcedural while CPU working on overhead.

Caused by:
Samples count would be decreased while there’s no available bytes, so audio would be hitched easily when there’s overheading.

Solution:
Increase NumBufferUnderrunSamples and NumSamplesToGeneratePerCallback under constructor of USoundWaveProcedural:

NumBufferUnderrunSamples = 512 * 32;
NumSamplesToGeneratePerCallback = DEFAULT_PROCEDURAL_SOUNDWAVE_BUFFER_SIZE * 16;

Reference

Low Latency Audio

Low latency audio
https://inetic.github.io/2016-05-19-android-unreal-and-lla/

The most important thing in communication is to hear what isn't being said. ― Peter F. Drucker