We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 3788

Summary: software renderer crashes in SDL_RenderCopyEx with rotation and dstrect w or h is 0
Product: SDL Reporter: Anthony @ POW Games <ant>
Component: renderAssignee: Sam Lantinga <slouken>
Status: RESOLVED FIXED QA Contact: Sam Lantinga <slouken>
Severity: normal    
Priority: P2 CC: sylvain.becker
Version: 2.0.5   
Hardware: x86   
OS: Windows 10   
Attachments: screen shot of comprehensive graphics options in PowBall Renaissance

Description Anthony @ POW Games 2017-08-30 13:36:18 UTC
Scaling a rotated texture in software mode down to nothing (0) will crash SDL_RenderCopyEx. It should draw nothing, but there are lots of cases where scaling results in a 0 width or height. I've added a condition to prevent it crashing, but should SDL crash in this case?
Comment 1 Sam Lantinga 2017-08-30 16:04:39 UTC
I wasn't able to reproduce this by modifying test/testrendercopyex.c to set the width and height to zero in the SDL_RenderCopyEx() call.

What renderer are you using, and can you attach a quick test to show the crash?

Thanks!
Comment 2 Anthony @ POW Games 2017-08-30 16:15:09 UTC
That was quick Sam, hi :)

Only happens in the software renderer.

SDL_CreateRenderer(window, 3, 0); // 3 = software renderer on my system

If that doesn't trigger it, I will dig deeper and put a test together.
Comment 3 Sam Lantinga 2017-08-30 16:17:59 UTC
Yeah, that didn't trigger it for me. Are you using the latest SDL snapshot?
http://www.libsdl.org/tmp/SDL-2.0.zip

I remember that something like this was fixed a while back.
Comment 4 Anthony @ POW Games 2017-08-30 16:48:03 UTC
(In reply to Sam Lantinga from comment #3)
> Yeah, that didn't trigger it for me. Are you using the latest SDL snapshot?
> http://www.libsdl.org/tmp/SDL-2.0.zip
> 
> I remember that something like this was fixed a while back.

I'm using the 2.0.5 .lib files.

I see there have been fixes to the software SDL_RenderCopyEx recently this month, which has probably fixed it.

I had problems linking the snapshots before, but I will try again and see if that solves it. Thanks Sam. Please bare with...
Comment 5 Sylvain 2017-09-20 10:09:46 UTC
I have tried to reproduce the case also, with latest source, 2.0.4 and 2.0.5 and I have seen nothing. So is there a real issue with this ?
Comment 6 Anthony @ POW Games 2017-09-20 11:21:17 UTC
Can you confirm you were using the software renderer and drawing a rotated texture?

Set either the w or h of the destination rectangle to 0, and rotation to a value higher than 0.

I've tested again and it's still crashing. I dropped the 2.0.6 DLL in, and it still crashes.
Comment 7 Sylvain 2017-09-20 11:35:26 UTC
Yes angle = 30, sprite->w = 3, renderer = software.

maybe, make sure you are using the software renderer :

SDL_RendererInfo info;
SDL_GetRendererInfo(renderer, &info);
SDL_Log("Renderer %s", info.name);

I tried also with testrendercopyex.c
Comment 8 Anthony @ POW Games 2017-09-20 11:57:00 UTC
Created attachment 2953 [details]
screen shot of comprehensive graphics options in PowBall Renaissance
Comment 9 Anthony @ POW Games 2017-09-20 11:58:55 UTC
You mean sprite->w = 0? The crash only happens when width or height is zero.

I have comprehensive graphics settings in the game I made which lets you cycle through all the avaliable renderers. This only happens in the software renderer. opengl, opengles, direct3D all work fine.
Comment 10 Sylvain 2017-09-20 15:57:42 UTC
Sorry, I mean't "s->sprite_rect.w = 0;" of testrendercopyex.c

this is intriguing. you should definitively build a testcase.
Comment 11 Sam Lantinga 2017-09-20 16:27:09 UTC
I think this is fixed in the 2.0.6 pre-release:
http://www.libsdl.org/tmp/download-2.0.php
Comment 12 Sylvain 2017-09-20 17:45:50 UTC
yep, but Anthony says it still happens with 2.0.6 DLL.
And It won't get reproduced with an older SDL2. 

Maybe a bug that testrendercopyex is not covering.
Comment 13 Anthony @ POW Games 2017-09-20 20:57:25 UTC
I've been digging deeper into this problem. I think it's something to do with the format of my SDL_Texture. It could be the pixel format or dimensions of the texture. If I create an empty texture and use that in place of the texture I've created from loading a PNG (with LodePNG), it doesn't crash. I'll continue to dig deeper.

I've never been able to compile SDL from source; I don't know how to set it up, so I can't step through where CopyEx crashes.
Comment 14 Anthony @ POW Games 2017-09-20 22:30:52 UTC
This is what's making the software renderer crash with rotated destination rectangles of w or h = 0:

SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "2");
Comment 15 Sam Lantinga 2017-09-21 00:38:25 UTC
Great. Can you attach a minimal example that we can use to reproduce this?

Thanks!
Comment 16 Sylvain 2017-09-21 08:09:14 UTC
Great, this can be reproduced now. This happens when renderCopyEx take the "smooth" path. it's a regression from bug 1550 (https://hg.libsdl.org/SDL/rev/5c4a85c5b648)

and putting back the correct condition fix it:


diff -r 8ab64b5850ae src/render/software/SDL_rotate.c
--- a/src/render/software/SDL_rotate.c	Fri Sep 15 17:27:32 2017 -0700
+++ b/src/render/software/SDL_rotate.c	Thu Sep 21 10:06:16 2017 +0200
@@ -257,7 +257,7 @@
                 dy = (sdy >> 16);
                 if (flipx) dx = sw - dx;
                 if (flipy) dy = sh - dy;
-                if ((unsigned)dx < (unsigned)sw && (unsigned)dy < (unsigned)sh) {
+                if ((dx > -1) && (dy > -1) && (dx < (src->w-1)) && (dy < (src->h-1))) {
                     sp = (tColorRGBA *) ((Uint8 *) src->pixels + src->pitch * dy) + dx;
                     c00 = *sp;
                     sp += 1;
Comment 17 Sam Lantinga 2017-09-21 08:22:45 UTC
Fixed, thanks!
https://hg.libsdl.org/SDL/rev/833d4fbb3d76