c++ - Why does DirectXToolkit ruin my depth testing

Question

I'm sure I'm just missing some simple step that I've been too blind to notice so far, but I cannot seem to get depth testing to work at all. This is with DirectX 11.

The code that should set it all up:

    DXGI_SWAP_CHAIN_DESC swapDesc = { };
    swapDesc.BufferDesc.Width = 0;
    swapDesc.BufferDesc.Height = 0;
    swapDesc.BufferDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
    swapDesc.BufferDesc.RefreshRate.Numerator = 0;
    swapDesc.BufferDesc.RefreshRate.Denominator = 1;
    swapDesc.BufferDesc.Scaling = DXGI_MODE_SCALING_UNSPECIFIED;
    swapDesc.BufferDesc.ScanlineOrdering = DXGI_MODE_SCANLINE_ORDER_UNSPECIFIED;

    swapDesc.SampleDesc.Count = 1;
    swapDesc.SampleDesc.Quality = 0;

    swapDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
    swapDesc.BufferCount = 1;
    swapDesc.OutputWindow = hwnd;
    swapDesc.Windowed = TRUE;
    swapDesc.SwapEffect = DXGI_SWAP_EFFECT_DISCARD;
    swapDesc.Flags = 0;

    UINT flg = 0;
#if MAGE_DEBUG
    flg |= D3D11_CREATE_DEVICE_DEBUG;
#endif
    GFX_THROW_INFO(D3D11CreateDeviceAndSwapChain(nullptr, D3D_DRIVER_TYPE_HARDWARE, nullptr,
        flg,
        nullptr, 0,
        D3D11_SDK_VERSION, &swapDesc, &mSwap, &mDevice, nullptr,
        &mContext));

    COMptr<ID3D11Resource> backBuffer;
    GFX_THROW_INFO(mSwap->GetBuffer(0, __uuidof(ID3D11Resource), &backBuffer));
    GFX_THROW_INFO(mDevice->CreateRenderTargetView(backBuffer.Get(), nullptr, &mTarget));


    LOG_INFO("Setting depth stencil dimensions ({}, {})", width, height);
    COMptr<ID3D11Texture2D> depthStencil;
    D3D11_TEXTURE2D_DESC texDesc = { };
    texDesc.Width = width;
    texDesc.Height = height;
    texDesc.MipLevels = 1;
    texDesc.ArraySize = 1;
    texDesc.Format = DXGI_FORMAT_D32_FLOAT;
    texDesc.SampleDesc.Count = 1;
    texDesc.SampleDesc.Quality = 0;
    texDesc.Usage = D3D11_USAGE_DEFAULT;
    texDesc.BindFlags = D3D11_BIND_DEPTH_STENCIL;
    GFX_THROW_INFO(mDevice->CreateTexture2D(&texDesc, nullptr, &depthStencil));

    D3D11_DEPTH_STENCIL_DESC depth = { };
    depth.DepthEnable = TRUE;
    depth.DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ALL;
    depth.DepthFunc = D3D11_COMPARISON_LESS;


    COMptr<ID3D11DepthStencilState> depthState;
    GFX_THROW_INFO(mDevice->CreateDepthStencilState(&depth, &depthState));


    D3D11_DEPTH_STENCIL_VIEW_DESC dsvDesc = { };
    dsvDesc.Format = DXGI_FORMAT_D32_FLOAT;
    dsvDesc.ViewDimension = D3D11_DSV_DIMENSION_TEXTURE2D;
    dsvDesc.Texture2D.MipSlice = 0;
    GFX_THROW_INFO(mDevice->CreateDepthStencilView(depthStencil.Get(), &dsvDesc, &mDepthStencilView));

    mContext->OMSetDepthStencilState(depthState.Get(), 1);
    mContext->OMSetRenderTargets(1, mTarget.GetAddressOf(), mDepthStencilView.Get());

    LOG_INFO("Setting viewport dimensions ({}, {})", width, height);
    D3D11_VIEWPORT vp;
    vp.Width = (float) width;
    vp.Height = (float) height;
    vp.MinDepth = 0.0f;
    vp.MaxDepth = 1.0f;
    vp.TopLeftX = 0.0f;
    vp.TopLeftY = 0.0f;
    mContext->RSSetViewports(1, &vp);

And of course, before every frame I call the following:

    mContext->ClearRenderTargetView(mTarget.Get(), color);
    mContext->ClearDepthStencilView(mDepthStencilView.Get(), D3D11_CLEAR_DEPTH, 1.0f, 0);

But unfortunately, the result ends up being this (note that the crysis nanosuit model is behind the goblin head) I believe this could potentially also be why the goblin model is rendering incorrectly even when alone but haven't figured that one out yet.

Example 1

And with just the goblin, looking from an angle

Example 2

If anyone can help me figure out why its not working, I'd greatly appreciate it!

EDIT

After some more frustrating testing I discovered the depth testing was broken because of some test text rendering I was doing with DirectX ToolKit's SpriteBatch and SpriteFont classes. Has anyone come across this issue before? I don't really want/need the toolkit for anything other than text rendering and perhaps loading dds textures so I'm hoping if I want to use those classes I don't need to drastically change my existing code?

question from:https://stackoverflow.com/questions/65893542/why-does-directxtoolkit-ruin-my-depth-testing

Answer 1

DirectX Tool Kit does not 'capture/restore' state like the legacy D3DX9/D3DX10 sprite did. This was inefficient and relied on some hacky back-door functionality to capture the 'state block' for Direct3D 10+. In most cases, you are already going to set the bulk of the commonly used state to set up for the next draw call anyhow.

Instead, I have fully documented all state impacted by each class. You are expected to change all required state after the DirectX Tool Kit object renders. For example, SpriteBatch docs state:

SpriteBatch makes use of the following states:

• BlendState
• Constant buffer (Vertex Shader stage, slot 0)
• DepthStencilState
• Index buffer
• Input layout
• Pixel shader
• Primitive topology
• RasterizerState
• SamplerState (Pixel Shader stage, slot 0)
• Shader resources (Pixel Shader stage, slot 0)
• Vertex buffer (slot 0)
• Vertex shader

So in short, you just need to set the DepthStencilState to what you want to use after you call SpriteBatch::End.

As a general habit for state management, you should set all state you rely on every frame. While in Direct3D 11 the 'last state' at the time you call Present is still there at the start of the next frame, this isn't true of DirectX 12. As such, you should make a habit of at the start of a new frame setting everything like current render target, viewport, render states you expect to be present for your whole scene, etc.

For example, most "HUD" rendering is done last, so the state changes by SpriteBatch would normally be reset on the next frame's start--again, assuming you set up the required state at the start of the frame rather than assuming it remains unchanged over many frames.

TL;DR: Move this code to just after you clear the render target each frame:

    mContext->OMSetDepthStencilState(depthState.Get(), 1);
    mContext->OMSetRenderTargets(1, mTarget.GetAddressOf(), mDepthStencilView.Get());

    D3D11_VIEWPORT vp = { 0.f, 0.f, float(width), float(height), D3D11_MIN_DEPTH, D3D11_MAX_DEPTH };
    mContext->RSSetViewports(1, &vp);