We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 4590

Summary: Integer overflow generates Illegal instruction under sanitizers
Product: SDL Reporter: mail
Component: renderAssignee: Sam Lantinga <slouken>
Status: NEW --- QA Contact: Sam Lantinga <slouken>
Severity: normal    
Priority: P2    
Version: 2.0.9   
Hardware: x86_64   
OS: Linux   
Attachments: Proposed patch

Description mail 2019-04-11 15:53:12 UTC
Created attachment 3741 [details]
Proposed patch

Hello,
I tried to use SDL2 with sanitizers enabled and found a few lines of code with the same bug: left-shifting Uint8 by 24 bits; integer promotion rule promotes Uint8 to int and therefore shifting yields signed integer overflow.

Steps to reproduce: add -fsanitize=address,undefined when building SDL2.
Compile the sample program:

#include <SDL.h>

int main(int argc, char** argv) {
    SDL_Init(SDL_INIT_EVERYTHING);
    SDL_Window* win = SDL_CreateWindow("window", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 800, 600, 0);
    SDL_Renderer* ren = SDL_CreateRenderer(win, -1, 0);

    SDL_SetRenderDrawColor(ren, 0xff, 0xff, 0xff, 0xff);
    SDL_RenderClear(ren);
    SDL_RenderPresent(ren);

    SDL_DestroyRenderer(ren);
    SDL_DestroyWindow(win);
    SDL_Quit();
}

Expected Results: The application exits normally.

Actual results: The pplication crashes with SIGILL:
[1]    15110 illegal hardware instruction (core dumped)  ./sdl2-il
#0  0x0000555555822355 in GL_RunCommandQueue (renderer=0x6160000e6180, cmd=0x6040000566d0, vertices=0x0, vertsize=0)
    at /home/sergey/dev/test/sdl2-il/build/_deps/sdl2-src/src/render/opengl/SDL_render_gl.c:1145
#1  0x000055555579c90e in FlushRenderCommands (renderer=0x6160000e6180) at /home/sergey/dev/test/sdl2-il/build/_deps/sdl2-src/src/render/SDL_render.c:215
#2  0x00005555557ed136 in SDL_RenderPresent_REAL (renderer=0x6160000e6180) at /home/sergey/dev/test/sdl2-il/build/_deps/sdl2-src/src/render/SDL_render.c:3078
#3  0x000055555572e501 in SDL_RenderPresent (a=0x6160000e6180) at /home/sergey/dev/test/sdl2-il/build/_deps/sdl2-src/src/dynapi/SDL_dynapi_procs.h:377
#4  0x0000555555726f30 in main(int, char**) (argc=1, argv=0x7fffffffe188) at /home/sergey/dev/test/sdl2-il/main.cpp:9

Found problems (rev 0ff5bbe35bf5):
src/render/SDL_render.c
src/render/opengl/SDL_render_gl.c
src/render/opengles/SDL_render_gles.c
src/render/opengles2/SDL_render_gles2.c
src/render/psp/SDL_render_psp.c
src/video/x11/edid-parse.c (?)

Proposed patch attached.