We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 3994

Summary: SIGSEGV on SDL_stack_alloc
Product: SDL Reporter: olivier <olivier-dion>
Component: renderAssignee: Ryan C. Gordon <icculus>
Status: RESOLVED FIXED QA Contact: Sam Lantinga <slouken>
Severity: trivial    
Priority: P2 CC: icculus
Version: HG 2.0   
Hardware: x86_64   
OS: Linux   

Description olivier 2017-12-06 18:16:21 UTC 
In many Render functions, such as SDL_RenderDrawLines, there's a stack memory
allocation using the macro SDL_stack_alloc which is define as
(type*)__builtin_alloca(sizeof(type)*(count)). For every of those stack
allocation, there's a condition to check if the pointer is NULL. However, the
linux man of alloca stipulates that "If the allocation causes stack overflow,
program behavior is undefined". This is not a big problem, because for small
values of count, alloca works fine. However, for big values, the behavior is
undefined and result in SIGSEGV, SDL_OutOfMemory is never call like it should be.
Comment 1 Sam Lantinga 2017-12-06 19:09:36 UTC 
The behavior is undefined and that check is only helpful on platforms that return NULL from alloca().

Are you proposing rewriting alloca() for Linux so that it returns NULL? If so, feel free to attach a proposed implementation.
Comment 2 olivier 2017-12-07 01:32:36 UTC 
I see. Wouldn't be possible to check the count before using alloca, and if it's too high call malloc?

I could simply make multiple call to SDL_RenderDrawLines with differents pointers I guess.
Comment 3 Sam Lantinga 2017-12-07 16:20:48 UTC 
Yes, that would be possible, though the number of lines that are safe would vary from system to system and application to application, depending on how much stack space is left.

We could use some general rule of thumb though... Out of curiosity, how many lines does it take to crash in your case?
Comment 4 olivier 2017-12-07 17:49:10 UTC 
In the order of 2^20. I know it's a huge number of lines. And
like I said, I should be able to do so by passing different
intervals at a time.

One option would be to define a macro, system dependent, with
the -D option, or something similar.
Comment 5 Ryan C. Gordon 2019-05-18 18:39:42 UTC 
In the latest in revision control, these are heap allocated instead of stack allocated, resolving this problem.

--ryan.