We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 3546

Summary: SDL_EVDEV_is_console() uses type of wrong size when calling ioctl
Product: SDL Reporter: Rob <rgubler>
Component: eventsAssignee: Sam Lantinga <slouken>
Status: RESOLVED FIXED QA Contact: Sam Lantinga <slouken>
Severity: normal    
Priority: P2    
Version: 2.0.5   
Hardware: ARM   
OS: Linux   
See Also: https://bugzilla.libsdl.org/show_bug.cgi?id=3545
Attachments: KDGKBTYPE type patch

Description Rob 2017-01-05 22:55:52 UTC 
Created attachment 2659 [details]
KDGKBTYPE type patch

When calling ioctl(fd, KDGKBTYPE, &type) in SDL_EVDEV_is_console(), we declare type as an 'int'.  This should be a 'char'.  The subsequent syscall, and kernel code, only writes the lower byte of the word. 

See: http://lxr.free-electrons.com/source/drivers/tty/vt/vt_ioctl.c?v=4.4#L399

ucval = KB_101;
ret = put_user(ucval, (char __user *)arg);

I've observed intermittent behavior related to this, and I can force an error condition by using an int initialized to 0xFFFFFFFF.  The resulting ioctl will set type to 0XFFFFFF02, and the conditional return in SDL_EVDEV_is_console() will fail.

Recommend changing to char, or masking off unused bits.

See attached patch for suggested fix.

-Rob
Comment 1 Sam Lantinga 2017-01-06 07:26:55 UTC 
This is in, thanks!
https://hg.libsdl.org/SDL/rev/6cfccf993c5d