We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 3158

Summary: SDL display window scrambled over VNC
Product: SDL Reporter: Witek Jachimczyk <witek>
Component: videoAssignee: Sam Lantinga <slouken>
Status: RESOLVED FIXED QA Contact: Sam Lantinga <slouken>
Severity: major    
Priority: P2 CC: sezeroz, witek
Version: HG 2.1   
Hardware: x86_64   
OS: Linux   

Description Witek Jachimczyk 2015-10-26 17:01:42 UTC 
Hi,

I'm using SDL to develop a video viewer for MATLAB.  The window is scrambled while using thightVNC with its default mode of RGB656.

SDL does not correctly recognize the pixel mode.  


I found a solution for this problem.  The solution involves modifying 
SDL/src/video/SDL_pixels.c

Adding the following "if statement" under case 16: of SDL_MasksToPixelFormatEnum resolves the issue:

        if (Rmask == 0x003F &&
            Gmask == 0x07C0 &&
            Bmask == 0xF800 &&
            Amask == 0x0000) {
            return SDL_PIXELFORMAT_RGB565;
        }

I hope that this helps someone.  I took me a while to figure it out.

Witek
Comment 1 Sam Lantinga 2016-10-01 20:45:49 UTC 
That's really weird. The mask you're describing would be BGR556, which is a new format. Is that actually the pixel format being used? If you feed RGB565 into other SDL functions, does it expand to the correct colors for that format?
Comment 2 Sam Lantinga 2016-10-01 20:47:44 UTC 
It looks like this was changed in TightVNC 1.2.0:
Xvnc: Rules to set default pixel formats have been changed: now they are RGB565 instead of BGR556 for color depth 16, and RGB888 instead of BGR888 for depth 24. This makes Xvnc compatible with Imlib renderer used in Gnome and also helps to avoid unnecessary pixel format translations in many cases.

I'm going to close this for now, since I've never seen anyone else use that format, and TightVNC is no longer using it either.
Comment 3 Witek Jachimczyk 2016-10-02 11:19:04 UTC 
Hi Sam,

Please don't dismiss the suggested fix. I've had it released to several thousand customers successfully. We use it at the company for automated testing on a daily basis. The tests run for linux, Mac and Windows although I recall only linux being affected.

Dismissing this fix will cause me to reapply this patch on my own to newer versions of SDl. Yes, we do use tight vnc.

I've seen several posts on the net for this issue but no one offered a solution like I did.

Thank you,

Witek
Comment 4 Sam Lantinga 2017-08-12 23:02:47 UTC 
This is in, thanks!
https://hg.libsdl.org/SDL/rev/c9b15b9a04e3