We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 5430 - Add NV12/21 Update Texture
Summary: Add NV12/21 Update Texture
Status: RESOLVED FIXED
Alias: None
Product: SDL
Classification: Unclassified
Component: render (show other bugs)
Version: 2.0.14
Hardware: x86_64 Windows 10
: P2 normal
Assignee: Sylvain
QA Contact: Sam Lantinga
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-31 17:00 UTC by cmediaplayer
Modified: 2021-01-11 09:08 UTC (History)
1 user (show)

See Also:


Attachments
test-case (1.99 KB, text/x-csrc)
2021-01-05 10:58 UTC, Sylvain
Details
test_direct3d11_nv12 (350.82 KB, image/png)
2021-01-05 18:39 UTC, cmediaplayer
Details
test-case (3.27 KB, text/x-csrc)
2021-01-09 20:28 UTC, Sylvain
Details

Note You need to log in before you can comment on or make changes to this bug.
Description cmediaplayer 2020-12-31 17:00:46 UTC
Hi, I need a function something like SDL_UpdateYUVTexture. When i use FFmpeg with SDL, FFmpeg hardware acceleration decoding uses NV12 texture by default. direct3d11 supports NV12/21 pixel format. So i need a function named SDL_UpdateNVTexture something like this:

For example:

SDL_UpdateNVTexture(texture, frame->data[0], frame->linesize[0], frame[1]->data[1], frame->linesize[1]);

Is it possible to add a function like that?

Thanks in advance.
Comment 1 Sylvain 2021-01-05 10:57:53 UTC
I've added SDL_UpdateNVTexture()
for software/opengl/opengles2 only
https://hg.libsdl.org/SDL/rev/d3c71331483d
Comment 2 Sylvain 2021-01-05 10:58:23 UTC
Created attachment 4639 [details]
test-case

Test-case
Comment 3 Sylvain 2021-01-05 10:59:17 UTC
When using the software renderer, there is an invalid read:

==75622== Invalid read of size 16
==75622==    at 0x4A70357: yuvnv12_argb_sseu (SDL_hg/src/video/yuv2rgb/yuv_rgb_sse_func.h:432)
==75622==    by 0x49BCFBB: yuv_rgb_sse (SDL_hg/src/video/SDL_yuv.c:279)
==75622==    by 0x49BC173: SDL_ConvertPixels_YUV_to_RGB (SDL_hg/src/video/SDL_yuv.c:416)
==75622==    by 0x49AD462: SDL_ConvertPixels_REAL (SDL_hg/src/video/SDL_surface.c:1396)
==75622==    by 0x48ED210: SDL_SW_CopyYUVToRGB (SDL_hg/src/render/SDL_yuv_sw.c:426)
==75622==    by 0x48E48D0: SDL_UpdateTextureNVPlanar (SDL_hg/src/render/SDL_render.c:1674)
==75622==    by 0x48E44FB: SDL_UpdateNVTexture (SDL_hg/src/render/SDL_render.c:1794)
==75622==    by 0x4014A9: main (test_bug_5430_yuv.c:90)
Comment 4 Sylvain 2021-01-05 11:10:37 UTC
fixed the read access by using align memory: https://hg.libsdl.org/SDL/rev/c124e24fcf74
Not absolutly sure if there isn't something wrong
Comment 5 Sylvain 2021-01-05 11:11:21 UTC
Add the D3D11 UpdateNVTexture: https://hg.libsdl.org/SDL/rev/7f3f477d9101
but this isn't tested
Comment 6 Sylvain 2021-01-05 11:14:03 UTC
@cmediaplayer:

can you give a try to use the SDL_UpdateNVTexture():

first using the software renderer.

SDL_SetHint(SDL_HINT_RENDER_DRIVER, "software");


when it works, use the D3D11 renderer. It should work immediately.
it not, maybe some offset to play with:
https://hg.libsdl.org/SDL/rev/7f3f477d9101#l1.24
(UVpitch vs UVpitch/2 ...)
Comment 8 Sylvain 2021-01-05 11:30:53 UTC
Also added for METAL: https://hg.libsdl.org/SDL/rev/724e9980492e
(not tested)
Comment 9 Sylvain 2021-01-05 11:37:35 UTC
Fix warning on METAL: https://hg.libsdl.org/SDL/rev/6bdd3c5638e1
Comment 10 cmediaplayer 2021-01-05 18:39:03 UTC
Created attachment 4640 [details]
test_direct3d11_nv12
Comment 11 cmediaplayer 2021-01-05 18:42:01 UTC
Hi, thanks for the patches. When i testing the new functions for direct3d11 renderer, half of screen becomes green. You can see it at the attachment.
Comment 12 cmediaplayer 2021-01-05 19:01:34 UTC
By the way, I have tested opengl and opengles2 and works properly.
Comment 13 Sylvain 2021-01-05 19:20:01 UTC
what if you change: 
https://hg.libsdl.org/SDL/rev/7f3f477d9101#l1.24

UVpitch / 2 by UVpitch ?
Comment 14 cmediaplayer 2021-01-05 19:30:58 UTC
The correct parameters have to be like this:

if (D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV, SDL_BYTESPERPIXEL(texture->format), rect->x, rect->y, rect->w, rect->h / 2, UVplane, UVpitch) < 0) {
        return -1;
}

After this change, it works perfectly :)
Comment 15 Sam Lantinga 2021-01-05 19:41:13 UTC
(In reply to cmediaplayer from comment #14)
> The correct parameters have to be like this:
> 
> if (D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV,
> SDL_BYTESPERPIXEL(texture->format), rect->x, rect->y, rect->w, rect->h / 2,
> UVplane, UVpitch) < 0) {
>         return -1;
> }
> 
> After this change, it works perfectly :)

I'm pretty sure it should be this:
if (D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV, SDL_BYTESPERPIXEL(texture->format), rect->x, rect->y / 2, rect->w, rect->h / 2, UVplane, UVpitch) < 0) {
    return -1;
}

The NV plane is full width, half height.
Comment 16 Sylvain 2021-01-05 19:47:46 UTC
I commited cmediaplayer tested https://hg.libsdl.org/SDL/rev/84ac170523cc

But I thought it should be the same as:
SDL_UpdateTexture path
https://hg.libsdl.org/SDL/file/7f3f477d9101/src/render/direct3d11/SDL_render_d3d11.c#l1368

D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV, 2, rect->x / 2, rect->y / 2, rect->w / 2, rect->h / 2, UVplane, UVpitch) 


 
If you have the possibility there is METAL renderer to double-check
Comment 17 cmediaplayer 2021-01-05 20:25:53 UTC
(In reply to Sam Lantinga from comment #15)
> (In reply to cmediaplayer from comment #14)
> > The correct parameters have to be like this:
> > 
> > if (D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV,
> > SDL_BYTESPERPIXEL(texture->format), rect->x, rect->y, rect->w, rect->h / 2,
> > UVplane, UVpitch) < 0) {
> >         return -1;
> > }
> > 
> > After this change, it works perfectly :)
> 
> I'm pretty sure it should be this:
> if (D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV,
> SDL_BYTESPERPIXEL(texture->format), rect->x, rect->y / 2, rect->w, rect->h /
> 2, UVplane, UVpitch) < 0) {
>     return -1;
> }
> 
> The NV plane is full width, half height.

Yep, Sam is right. it should be like this:

if (D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV, SDL_BYTESPERPIXEL(texture->format), rect->x, rect->y / 2, rect->w, rect->h / 2, UVplane, UVpitch) < 0) {
    return -1;
}

I tested and it works.
Comment 18 Sam Lantinga 2021-01-05 20:53:06 UTC
(In reply to Sylvain from comment #16)
> I commited cmediaplayer tested https://hg.libsdl.org/SDL/rev/84ac170523cc
> 
> But I thought it should be the same as:
> SDL_UpdateTexture path
> https://hg.libsdl.org/SDL/file/7f3f477d9101/src/render/direct3d11/
> SDL_render_d3d11.c#l1368
> 
> D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV, 2,
> rect->x / 2, rect->y / 2, rect->w / 2, rect->h / 2, UVplane, UVpitch) 

That might be a bug in the SDL_UpdateTeture path. Can someone verify?
Comment 19 Sam Lantinga 2021-01-05 20:54:51 UTC
@cmediaplayer, are the Y and NV planes separately allocated chunks of memory? If they were contiguous, you could just use SDL_UpdateTexture(), as Sylvain pointed out.
Comment 20 Sylvain 2021-01-05 21:10:21 UTC
I test with a non fullscreen rect, 
https://hg.libsdl.org/SDL/rev/38bf617e797a

D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV, SDL_BYTESPERPIXEL(texture->format), rect->x / 2, rect->y / 2, rect->w, rect->h / 2, UVplane, UVpitch)



Also, fix software renderer:
https://hg.libsdl.org/SDL/rev/b836d9333e26
Comment 21 cmediaplayer 2021-01-05 21:17:06 UTC
(In reply to Sam Lantinga from comment #19)
> @cmediaplayer, are the Y and NV planes separately allocated chunks of
> memory? If they were contiguous, you could just use SDL_UpdateTexture(), as
> Sylvain pointed out.

FFmpeg library separately allocates chunks of memory for the Y and UV(U and V together) planes. I use the function implemented like this:

int ret = SDL_UpdateNVTexture(pSDLTexture,NULL, frame->data[0], frame->linesize[0], frame->data[1], frame->linesize[1];

That's why I asked for kind of this function.
Comment 22 Sylvain 2021-01-05 21:22:14 UTC
I've also tried METAL renderer which is ok with the test
Comment 23 cmediaplayer 2021-01-05 21:42:28 UTC
(In reply to Sylvain from comment #20)
> I test with a non fullscreen rect, 
> https://hg.libsdl.org/SDL/rev/38bf617e797a
> 
> D3D11_UpdateTextureInternal(rendererData, textureData->mainTextureNV,
> SDL_BYTESPERPIXEL(texture->format), rect->x / 2, rect->y / 2, rect->w,
> rect->h / 2, UVplane, UVpitch)
> 
> 
> 
> Also, fix software renderer:
> https://hg.libsdl.org/SDL/rev/b836d9333e26

My media player automatically detects which renderers support which pixel formats. Software and opengles renderers don't support nv12/21 pixel formats in my app. Because those renderers do not return nv texture format. Others do. Should be something wrong software and opengles implementation. Can you check for it?
Comment 24 Sylvain 2021-01-06 10:11:50 UTC
software render somehow supports all pixel format as an output.
(but blitting in-between differe YUV/NV surfaces I think).

maybe we could increase the list here:

https://hg.libsdl.org/SDL/file/38bf617e797a/src/render/software/SDL_render_sw.c#l871
Comment 25 Sylvain 2021-01-09 20:28:56 UTC
Created attachment 4648 [details]
test-case

Update the test-case so it can  use SDL_UpdateTexture.
Comment 26 Sylvain 2021-01-09 20:29:38 UTC
I've double-checked D3D11 syntax for SDL_UpdateTexture(), and it's ok.
The difference is with the 3rd paramterer BPP : 
2 in SDL_UpdateTexture vs SDL_BYTESPERPIXEL(texture->format) (eg 1).

I updated the code to match SDL_UpdateTexture notation:
https://hg.libsdl.org/SDL/rev/a89a79f8ac30


Marking the issue as resolved
Comment 27 Sylvain 2021-01-11 09:08:29 UTC
fix for opengles2: 
https://hg.libsdl.org/SDL/rev/e10fb6ab0d77

when updating a non fullscreen rect