Diligent Engine
 
Loading...
Searching...
No Matches
Classes | Public Types | Public Member Functions | Static Public Member Functions | List of all members
Diligent::TemporalAntiAliasing Class Reference

Implements temporal anti-aliasing. More...

#include <TemporalAntiAliasing.hpp>

Classes

struct  CreateInfo
 Create info. More...
 
struct  RenderAttributes
 Render attributes that are passed to the effect. More...
 

Public Types

enum  FEATURE_FLAGS : Uint32
 Feature flags that control the behavior of the effect.
 

Public Member Functions

 TemporalAntiAliasing (IRenderDevice *pDevice, const CreateInfo &CI)
 Creates a new instance of the effect.
 
float2 GetJitterOffset (Uint32 AccumulationBufferIdx=0) const
 Returns the jitter offset for the specified accumulation buffer index.
 
void PrepareResources (IRenderDevice *pDevice, IDeviceContext *pDeviceContext, PostFXContext *pPostFXContext, FEATURE_FLAGS FeatureFlag, Uint32 AccumulationBufferIdx=0)
 Prepares the effect for rendering.
 
void Execute (const RenderAttributes &RenderAttribs)
 Executes the effect.
 
ITextureView * GetAccumulatedFrameSRV (bool IsPrevFrame=false, Uint32 AccumulationBufferIdx=0) const
 Returns the shader resource view of the accumulated frame.
 

Static Public Member Functions

static bool UpdateUI (HLSL::TemporalAntiAliasingAttribs &TAAAttribs, FEATURE_FLAGS &FeatureFlags)
 Adds the ImGui controls to the UI.
 
static float4x4 GetJitteredProjMatrix (float4x4 Proj, const float2 &Jitter)
 Computes the jittered projection matrix.
 

Detailed Description

Implements temporal anti-aliasing.

Temporal Anti Aliasing

Table of contents

  • Introduction
  • Integration guidelines
    • Input resources
    • Host API
  • Implementation details
  • References

Introduction

We needed to add temporal antialiasing to our project that could run in WebGL. We based our implementation on the method described in [Emilio López, 2022]

Integration guidelines

Input resources

The following table enumerates all external inputs required by TAA.

Name Format Notes
Color buffer APPLICATION SPECIFIED (3x FLOAT) The color buffer of the current frame provided by the application, in linear space.

The effect uses a number of parameters to control the quality of the effect organized into the HLSL::TemporalAntiAliasingAttribs structure. The following table lists the parameters and their descriptions.

Name Notes
Temporal stability factor The current implementation of TAA uses linear weighting based on the number of accumulated samples for each pixel. However, we can still determine a minimum alpha when blending the current and previous frames.
Reset accumulation If this parameter is set to true, the current frame will be written to the current history buffer without interpolation with the previous history buffer.
Skip rejection This parameter allows skipping the disocclusion check stage. Disocclusion check can cause flickering on a high-frequency signal. Use this parameter for static images to achieve honest supersampling.

The effect can be configured using the TemporalAntiAliasing::FEATURE_FLAGS enumeration. The following table lists the flags and their descriptions.

Name Notes
FEATURE_FLAG_GAUSSIAN_WEIGHTING Use Gaussian weighting in the variance clipping step
FEATURE_FLAG_BICUBIC_FILTER Use Catmull-Rom filter to sample from the history buffer

Host API

To integrate TAA into your project, include the following necessary header files:

#include "PostFXContext.hpp"
#include "TemporalAntiAliasing.hpp"
namespace HLSL
{
#include "Shaders/Common/public/BasicStructures.fxh"
#include "Shaders/PostProcess/TemporalAntiAliasing/public/TemporalAntiAliasingStructures.fxh"
} // namespace HLSL

Now, create the necessary objects:

m_PostFXContext = std::make_unique<PostFXContext>(m_pDevice);
m_TAA = std::make_unique<TemporalAntiAliasing>(m_pDevice);

Next, call the methods to prepare resources for the PostFXContext and TemporalAntiAliasing objects. This needs to be done every frame before starting the rendering process.

{
PostFXContext::FrameDesc FrameDesc;
FrameDesc.Index = m_CurrentFrameNumber; // Current frame number.
FrameDesc.Width = SCDesc.Width; // Current screen width.
FrameDesc.Height = SCDesc.Height; // Current screen height.
m_PostFXContext->PrepareResources(m_pDevice, FrameDesc, PostFXContext::FEATURE_FLAG_NONE);
TemporalAntiAliasing::FEATURE_FLAGS ActiveFeatures = ...;
m_TAA->PrepareResources(m_pDevice, m_pImmediateContext, m_PostFXContext.get(), ActiveFeatures);
}
Diligent::TemporalAntiAliasing::FEATURE_FLAGS
FEATURE_FLAGS
Feature flags that control the behavior of the effect.
Definition TemporalAntiAliasing.hpp:223

After that, invoke the PostFXContext::Execute method. At this stage, some intermediate resources necessary for all post-processing objects dependent on PostFXContext are calculated. This method can take a constant buffer directly containing an array from the current and previous cameras (for this method, you can refer to this section of the code [0] and [1]). Alternatively, you can pass the corresponding pointers const HLSL::CameraAttribs* pCurrCamera and const HLSL::CameraAttribs* pPrevCamera for the current and previous cameras, respectively. You also need to pass the depth of the current and previous frames (the depth buffers should not contain transparent objects), and a buffer with motion vectors in NDC space, into the corresponding ITextureView* pCurrDepthBufferSRV, ITextureView* pPrevDepthBufferSRV, ITextureView* pMotionVectorsSRV pointers.

{
PostFXContext::RenderAttributes PostFXAttibs;
PostFXAttibs.pDevice = m_pDevice;
PostFXAttibs.pDeviceContext = m_pImmediateContext;
PostFXAttibs.pCameraAttribsCB = m_FrameAttribsCB; // m_Resources[RESOURCE_IDENTIFIER_CAMERA_CONSTANT_BUFFER].AsBuffer();
PostFXAttibs.pCurrDepthBufferSRV = m_CurrDepthBuffer; // m_Resources[RESOURCE_IDENTIFIER_DEPTH0 + CurrFrameIdx].GetTextureSRV();
PostFXAttibs.pPrevDepthBufferSRV = m_PrevDepthBuffer; // m_Resources[RESOURCE_IDENTIFIER_DEPTH0 + PrevFrameIdx].GetTextureSRV();
PostFXAttibs.pMotionVectorsSRV = m_MotionBuffer; // m_GBuffer->GetBuffer(GBUFFER_RT_MOTION_VECTORS)->GetDefaultView(TEXTURE_VIEW_SHADER_RESOURCE);
m_TAA->Execute(PostFXAttibs);
}

TAA requires that each frame be rendered with some offset relative to the pixel. Therefore, you need to pass a modified projection matrix and jitter when updating the camera buffer HLSL::CameraAttribs for the current frame, as in the code below (you can refer to this section of the code [2])

auto ComputeProjJitterMatrix = [&](const float4x4& ProjMatrix, float2 Jitter) -> float4x4 {
float4x4 Result = ProjMatrix;
Result[2][0] = Jitter.x;
Result[2][1] = Jitter.y;
return Result;
};
const float2 Jitter = m_TAA->GetJitterOffset();
const float4x4 CameraView = m_Camera.GetViewMatrix();
const float4x4 CameraProj = ComputeProjJitterMatrix(GetAdjustedProjectionMatrix(YFov, ZNear, ZFar), Jitter);
const float4x4 CameraViewProj = CameraView * CameraProj;
auto& CurrCamAttribs = m_CameraAttribs[CurrFrameIdx];
// ...
CurrCamAttribs.mViewT = CameraView.Transpose();
CurrCamAttribs.mProjT = CameraProj.Transpose();
CurrCamAttribs.mViewProjT = CameraViewProj.Transpose();
CurrCamAttribs.mViewInvT = CameraView.Inverse().Transpose();
CurrCamAttribs.mProjInvT = CameraProj.Inverse().Transpose();
CurrCamAttribs.mViewProjInvT = CameraViewProj.Inverse().Transpose();
CurrCamAttribs.f2Jitter.x = Jitter.x;
CurrCamAttribs.f2Jitter.y = Jitter.y;
// ...

After rendering the entire frame, you need to invoke the TAA computation. Note that the frame passed in the color buffer argument must be in linear space, that is, before the tone mapping, as well as before effects such as Bloom and Depth Of Field. To do this, we call the TemporalAntiAliasing::Execute method. Before this, we need to fill the passed structures TemporalAntiAliasingAttribs and TemporalAntiAliasing::RenderAttributes with the necessary data. Refer to the Input resources section for parameter description.

{
HLSL::TemporalAntiAliasingAttribs TAASettings{};
TemporalAntiAliasing::RenderAttributes TAARenderAttribs{};
TAARenderAttribs.pDevice = m_pDevice;
TAARenderAttribs.pDeviceContext = m_pImmediateContext;
TAARenderAttribs.pPostFXContext = m_PostFXContext.get();
TAARenderAttribs.pColorBufferSRV = m_ColorBuffer; // m_Resources[RESOURCE_IDENTIFIER_RADIANCE0 + CurrFrameIdx].GetTextureSRV();
TAARenderAttribs.pTAAAttribs = &TAASettings;
m_TAA->Execute(TAARenderAttribs);
}
Diligent::TemporalAntiAliasing::RenderAttributes
Render attributes that are passed to the effect.
Definition TemporalAntiAliasing.hpp:238
Diligent::TemporalAntiAliasing::RenderAttributes::pDevice
IRenderDevice * pDevice
Render device that may be used to create new objects needed for this frame, if any.
Definition TemporalAntiAliasing.hpp:240

Now, you can directly obtain a ITextureView on the texture containing the TAA result using the method TemporalAntiAliasing::GetAccumulatedFrameSRV. After this, you can pass the TAA result to the tone mapping stage.

Implementation details

In general, all TAA implementations use similar core concepts. For a brief introduction to how the initial implementation of temporal anti-aliasing works, refer to [Emilio López, 2022]. We based our implementation on the one described in the article, but we made some modifications:

  • We use linear weighting instead of exponential weighting, meaning we keep track of the number of accumulated samples for each pixel. This allows us to increase the rate of convergence and reduce ghosting. You can read more about it in [ReBLUR, 2020] page 25
  • We use variance clipping instead of AABB clamping. This results in slightly better quality, more detailed description can be found here [Marco Salvi, 2016]. We perform variance clipping in the YCoCg space, as it helps to eliminate the artifact of a purple hue appearing on some surfaces [Brian Karis, 2014] page 35.
  • We use dynamic variance gamma based on pixel velocity, which helps to reduce flickering on static frames.
  • During cubic filtering from the history buffer, we use the approach from [Jorge Jimenez, 2016] slide 77. This allows a slight performance improvement.

References