We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 4832 - Loss of queued commands on SDL_SetRenderTarget with direct3d11 renderer.
Summary: Loss of queued commands on SDL_SetRenderTarget with direct3d11 renderer.
Status: RESOLVED FIXED
Alias: None
Product: SDL
Classification: Unclassified
Component: render (show other bugs)
Version: 2.0.10
Hardware: x86_64 Windows 10
: P2 critical
Assignee: Ryan C. Gordon
QA Contact: Sam Lantinga
URL:
Keywords: target-2.0.12
Depends on:
Blocks:
 
Reported: 2019-10-18 14:09 UTC by Martin Vee
Modified: 2019-10-26 16:57 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Vee 2019-10-18 14:09:35 UTC
Using the Direct3D11 renderer, it's possible that a call to SDL_SetRenderTarget() returns an error and discrads the command queue, thus putting the renderer in an uncertain state. When it happens, the console will display the following error : 
> __FUNCTION__, ID3D11Device1::CreateBuffer [vertex buffer]: Invalid parameter.

As I reported in Discourse (https://discourse.libsdl.org/t/bug-buffered-commands-lost-with-possible-fix-also-fixes-id3d11device1-createbuffer-vertex-buffer/26662), here's what happens :

*SDL_render.c*
- Line 1833 : SDL_SetRenderTarget().
- Line 1843 : FlushRenderCommands(), success : command queue now empty.
- Line 1900 : QueueCmdSetViewport(), one command queued.
- Line 1903 : QueueCmdSetClipRect(), two commands queued.
- Line 1908 : FlushRenderCommandsIfNotBatching()
- Line  257 : FlushRenderCommands()
- Line  218 : RunCommandQueue(), renderer->vertex_data_used equal 0 (there's no vertex data ; the only two queued commands are SetViewport and SetClipRect).
*SDL_render_d3d11.c*
- Line 2189 : D3D11_UpdateVertexBuffer(), vertsize equal 0.
- Line 1814 : ID3D11Device_CreateBuffer(), vertexBufferDesc.ByteWidth equal 0. The call fails. The function returns -1.
- Line 2190 : D3D11_RunCommandQueue() returns -1 immediately, command queue still contains SetViewport and SetClipRect.
*SDL_render.c*
- Line  228 : Command queue emptied, SetViewport and SetClipRect never executed. FlushRenderCommands() returns -1.
- Line 1908 : SDL_SetRenderTarget() returns a failure code.

Result : Since the target was valid, the renderer will render to the target, but the viewport and the clipping are invalid (SetViewport and SetCliprect command did not execute).

*PROPOSED PATCH*
Add that code at the beginning of D3D11_UpdateVertexBuffer()

  if(dataSizeInBytes == 0)
    return 0;
Comment 1 Ryan C. Gordon 2019-10-26 16:57:03 UTC
Good catch, this fix is now https://hg.libsdl.org/SDL/rev/0d6a584ace27, thanks!

--ryan.