We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 2921

Summary: Deprecation/Removal or Global Mouse Function APIs
Product: SDL Reporter: x414e54
Component: eventsAssignee: Ryan C. Gordon <icculus>
Status: ASSIGNED --- QA Contact: Sam Lantinga <slouken>
Severity: normal    
Priority: P2 CC: dmitry.rekman, icculus
Version: HG 2.0   
Hardware: x86_64   
OS: Linux   

Description x414e54 2015-03-22 08:10:32 UTC 
I notice with the upcoming SDL 2.0.4 there has been a plague of global mouse based APIs.

I have worked on porting a few windows tool-kits to other platforms before and global/absolute mouse functions are one of the mouse subjective and non cross platform APIs out there. 

They require calculating window borders, desktop areas, highdpi/scaling and mouse acceleration. 
There is no guarantee that the way window borders and desktop areas are calculated is constant.
They may change based on taskbars/docks or scaling or even between OS api releases.

The following functions really should not make it into SDL 2.0.4:
SDL_GetGlobalMouseState.
SDL_WarpMouseGlobal.
SDL_CaptureMouse.

Issues are:
Some platforms/wms do not support mouse events outside of a window.
Some platforms/wms do not support warping or controlling the mouse outside of a window.
Some platforms/wms may support mirrored desktops at different resolutions.
Some platforms/wms may support virtual desktops where two windows may be in the same place but not both visible.
Some platforms/wms may dynamically scale the usable desktop area based on pop-outs, docks or even expose style tiling
Some platforms/wms may not support global window positioning (tiling window manages etc.) making global mouse pointless.

I can see SDL_CaptureMouse as possibly a useful extension to SDL_SetRelativeMouse in the sense that it allows SDL_GetMouseState to produce x and y co-ordinates outside the window.
Which may be useful for a child window trying to work out if it is over any parent windows. But as this would require relative mouse to implement on some platforms it should not be a separate function and just a hint for SDL_SetRelativeMouse.

The original reason for SDL_GetGlobalMouseState was added in the SDL-gui-backend for Slate. But the Linux developers decided that it was too chatty to X, so it now caches the result of SDL_GetGlobalMouseState and invalidates it based on mouse motion events (which only occur when inside a window). 
This means for the original case SDL_GetGlobalMouseState is completely redundant and could theoretically be calculated from SDL_GetMouseState.

It would be setting a precedence to developers that it is cross platform to use these implementations when it is not. Obviously there are other parts of SDL which are not cross-platform such as window positioning (e.g. fullscreen only platforms) but SDL is not just about implementing things on Window and Mac.

You should look at the reason why people are using these APIs and offer them a cross platform API rather than hacking in a Window based solution and hoping it will work for other platforms.

A suggestion is something such as SDL_MoveWindowBasedOnMouse (might not be completely cross platform and need parent/child window support) or extending the SDL_SetWindowHitTest draggable. Then internally this can use a global mouse pos, global window pos set loop etc.
Comment 1 x414e54 2015-03-22 14:50:24 UTC 
It should also be noted that SDL_GetGlobalMouseState interacts really badly with SDL_SetRelativeMouseMode. 

Relative mouse mode does an initial warp to the window center and on non support platforms constantly warps to center so the coordinates from SDL_GetMouseState and SDL_GetGlobalMouseState differ. 

An application which warps the cursor back to the start before any relative motion or uses a software cursor will have issues.
Comment 2 Ryan C. Gordon 2015-03-24 06:33:57 UTC 
(In reply to x414e54 from comment #1)
> It should also be noted that SDL_GetGlobalMouseState interacts really badly
> with SDL_SetRelativeMouseMode.

We should probably change it to report failure from SDL_GetGlobalMouseState() when the mouse is in relative mode (since there is no concept of a global position at that point).

I sent you an email about the rest of this bug report, and I don't think we're going to ditch these APIs, but we _did_ build them with the expectation that they can refuse to work at all on certain platforms, like smartphones.

I think desktop Wayland should add an optional protocol to support some of these things, honestly, and both SDL and Wayland will likely improve in the coming months as we get closer to abandoning X11 as our primary target.

Marking this bug as ASSIGNED until I deal with the global vs relative issue.

--ryan.
Comment 3 Dmitry 'RCL' Rekman 2015-03-24 14:56:55 UTC 
>The original reason for SDL_GetGlobalMouseState was added in the SDL-gui-
>backend for Slate. But the Linux developers decided that it was too chatty to 
>X, so it now caches the result of SDL_GetGlobalMouseState and invalidates it 
>based on mouse motion events (which only occur when inside a window). 
>This means for the original case SDL_GetGlobalMouseState is completely
>redundant and could theoretically be calculated from SDL_GetMouseState.

This is too early to say. I cached the function in UE4 main branch for a test, and this caching may or may not work in all cases. To truly work around the lack of SDL_GetGlobalMouseState() UE4 might need to create a transparent window covering all the desktops (and resize it appropriately when new displays are being added, etc) to always receive mouse notifications - just like UE4 does on Mac. This might be the solution for Wayland, but until we investigate this I propose not to remove this function from SDL.
Comment 4 Ryan C. Gordon 2015-03-25 15:14:28 UTC 
> notifications - just like UE4 does on Mac. This might be the solution for
> Wayland, but until we investigate this I propose not to remove this function
> from SDL.

I don't think we will remove it, but we _do_ need to investigate if we are doing things inefficiently on X11.

For example, perhaps instead of XSync()'ing and polling the server every time we call this function, we can try hooking up to XInput and only talking to the X server when it reports a mouse has moved. I haven't tried this, it's just a hypothetical way to reduce X chatter.

That will likely happen after 2.0.4 ships in any case, and I think the API itself is reasonable as-is, even if we later improve its implementation in a different bug report.

--ryan.
Comment 5 x414e54 2015-03-27 06:14:47 UTC 
(In reply to Dmitry 'RCL' Rekman from comment #3)
> >The original reason for SDL_GetGlobalMouseState was added in the SDL-gui-
> >backend for Slate. But the Linux developers decided that it was too chatty to 
> >X, so it now caches the result of SDL_GetGlobalMouseState and invalidates it 
> >based on mouse motion events (which only occur when inside a window). 
> >This means for the original case SDL_GetGlobalMouseState is completely
> >redundant and could theoretically be calculated from SDL_GetMouseState.
> 
> This is too early to say. I cached the function in UE4 main branch for a
> test, and this caching may or may not work in all cases. To truly work
> around the lack of SDL_GetGlobalMouseState() UE4 might need to create a
> transparent window covering all the desktops (and resize it appropriately
> when new displays are being added, etc) to always receive mouse
> notifications - just like UE4 does on Mac. This might be the solution for
> Wayland, but until we investigate this I propose not to remove this function
> from SDL.

As an example against global mouse co-ordinates (even on X11) open SlateViewer and them move one of the windows onto a different virtual desktop/workspace. 

Then take one of tabs and move it to where the other window is on the original workspace. 
The tab disappears and acts strangely because Slate thinks the window is there when it is not. SlateViewer also crashed for me in this case.

Currently my Wayland code just works by taking the SDL_GetMouseState co-ords and offsetting based on where Slate thinks the window is.
In all my tests of SDL_GetGlobalMouseState vs SDL_GetMouseState the co-ordinates are always the same. 

The only benefit of SDL_GetGlobalMouseState is that you do not loose the mouse events for doing the tab dragging but I am pretty sure you could do this with some kind of implicit grab drag and drop protocols.
Comment 6 x414e54 2015-03-27 06:46:54 UTC 
(In reply to Ryan C. Gordon from comment #2)
> (In reply to x414e54 from comment #1)
> > It should also be noted that SDL_GetGlobalMouseState interacts really badly
> > with SDL_SetRelativeMouseMode.
> 
> We should probably change it to report failure from
> SDL_GetGlobalMouseState() when the mouse is in relative mode (since there is
> no concept of a global position at that point).
> 
> I sent you an email about the rest of this bug report, and I don't think
> we're going to ditch these APIs, but we _did_ build them with the
> expectation that they can refuse to work at all on certain platforms, like
> smartphones.
> 
> I think desktop Wayland should add an optional protocol to support some of
> these things, honestly, and both SDL and Wayland will likely improve in the
> coming months as we get closer to abandoning X11 as our primary target.
> 
> Marking this bug as ASSIGNED until I deal with the global vs relative issue.
> 
> --ryan.

Fair enough, but I do think it needs to be thought about a bit more.

What happens if you are in "expose mode" e.g. GNOME Shell. What is supposed to happen in this case?
It would still be getting the global mouse position from X11 but would be inaccurate to where your window was.

Also SDL does not report any information about virtual desktops/workspaces. 
So there is no way to work out if the global co-ordinates are over a window or not as per my previous comment. 

Both of these cases could have some seriously weird issues if it is trying to globally warp the mouse cursor without checking if the window has focus.


I kind of agree with the direction Wayland have at the moment. If you cannot guarantee something then it should not be in the core protocol.

Obviously any compositor is free to implement its own extension protocols which SDL can use but as an developer I have had enough of calculating window borders and task bar sizes. I prefer just to let the OS take care of it.
Comment 7 x414e54 2015-03-28 01:22:03 UTC 
I have also thought of a few other problems:


What about if the window is scaled/zoomed? 

A user may scale the window separately from the desktop. SDL will not get a resize event because the user is not requesting that the back-buffer size be changed. 

The application need know nothing about the scale change as the compositor will scale the mouse event positions automatically. But global mouse co-ordinates will be wrong.


What about multiple "pointer" input devices?

It is okay for a game to assume one input device but for a GUI application it cannot hit test off the non-primary pointer and then start moving a window around based on the location of the primary pointer.