We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 5165 - Undefined left shift in blit functions
Summary: Undefined left shift in blit functions
Status: RESOLVED FIXED
Alias: None
Product: SDL
Classification: Unclassified
Component: video (show other bugs)
Version: HG 2.1
Hardware: x86_64 Linux
: P2 normal
Assignee: Sam Lantinga
QA Contact: Sam Lantinga
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-29 12:37 UTC by James Legg
Modified: 2020-05-29 20:29 UTC (History)
0 users

See Also:

Attachments
Patch (2.28 KB, text/plain)
2020-05-29 12:37 UTC, James Legg 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description James Legg 2020-05-29 12:37:14 UTC 
Created attachment 4359 [details]
Patch

With the right inputs, undefined behaviour sanitizer reports that some left shifts in some blit functions such as Blit_3or4_to_3or4__inversed_rgb are undefined.

This occurs when an input pixel with an alpha value > 127 is present and dstfmt->Ashift is 24. The Uint8 alpha value is promoted to a (signed) int because of the bit shift, however values of 0x80 or more shifted left 24 places are not representable by an int, assuming a platform with 32 bit ints.

Explicitly casting the Uint8 alpha value to a Uint32 before the shift, as in the attach patch, allows the result type of the shift expression to hold the correct result without undefined behaviour.
Comment 1 Sam Lantinga 2020-05-29 20:29:00 UTC 
Patch added, thanks!
https://hg.libsdl.org/SDL/rev/e58323b0cf8e