We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 3546 - SDL_EVDEV_is_console() uses type of wrong size when calling ioctl
Summary: SDL_EVDEV_is_console() uses type of wrong size when calling ioctl
Status: RESOLVED FIXED
Alias: None
Product: SDL
Classification: Unclassified
Component: events (show other bugs)
Version: 2.0.5
Hardware: ARM Linux
: P2 normal
Assignee: Sam Lantinga
QA Contact: Sam Lantinga
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-01-05 22:55 UTC by Rob
Modified: 2017-01-06 07:26 UTC (History)
0 users

See Also:

Attachments
KDGKBTYPE type patch (463 bytes, patch)
2017-01-05 22:55 UTC, Rob 		Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Rob 2017-01-05 22:55:52 UTC 
Created attachment 2659 [details]
KDGKBTYPE type patch

When calling ioctl(fd, KDGKBTYPE, &type) in SDL_EVDEV_is_console(), we declare type as an 'int'.  This should be a 'char'.  The subsequent syscall, and kernel code, only writes the lower byte of the word. 

See: http://lxr.free-electrons.com/source/drivers/tty/vt/vt_ioctl.c?v=4.4#L399

ucval = KB_101;
ret = put_user(ucval, (char __user *)arg);

I've observed intermittent behavior related to this, and I can force an error condition by using an int initialized to 0xFFFFFFFF.  The resulting ioctl will set type to 0XFFFFFF02, and the conditional return in SDL_EVDEV_is_console() will fail.

Recommend changing to char, or masking off unused bits.

See attached patch for suggested fix.

-Rob
Comment 1 Sam Lantinga 2017-01-06 07:26:55 UTC 
This is in, thanks!
https://hg.libsdl.org/SDL/rev/6cfccf993c5d