Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New High DPI Support API is needed #2119

Closed
SDLBugzilla opened this issue Feb 11, 2021 · 25 comments
Closed

New High DPI Support API is needed #2119

SDLBugzilla opened this issue Feb 11, 2021 · 25 comments
Milestone

Comments

@SDLBugzilla
Copy link
Collaborator

SDLBugzilla commented Feb 11, 2021

This bug report was migrated from our old Bugzilla tracker.

These attachments are available in the static archive:

Reported in version: HG 2.1
Reported for operating system, platform: All, All

Comments on the original bug report:

On 2016-03-01 19:08:34 +0000, Yuya Kumagai wrote:

SDL's High DPI support is already available for Mac OS X and iOS (marked as DRAFT). But there is no Windows support. So, windows are scaled by Windows.

Win32 API has functions to specify a process's dpi awareness they named SetProcessDPIAware and SetProcessDpiAwareness. If a process is specified as dpi aware, windows are not scaled.(SFML and GLFW calls them).

Currently, SDL can create a high dpi window (by SDL_WINDOW_ALLOW_HIGHDPI when SDL_HINT_VIDEO_HIGHDPI_DISABLED is set to 0). But, Windows has no function to specify dpi awareness per-window.

And, SDL_HINT_VIDEO_HIGHDPI_DISABLED is set to 0 (so, high dpi window is allowed) by default. This cause breaking change if High DPI support on Windows is added.

So, I think we should create a new API for High DPI support.

The same opinion is commented on here by Eric Wasylishen: https://bugzilla.libsdl.org/show_bug.cgi?id=2713#c5

On 2017-08-12 04:34:18 +0000, Sam Lantinga wrote:

Hey Eric, what's the current state of your Windows high DPI patch? Have you and Alex sorted out the semantic differences between Windows and Mac implementations?

Ryan and I discussed it a bit last week and agreed that having the API work in virtual screen points and then having rendering contexts have a separate physical pixel size made sense for SDL.

On 2017-08-12 04:40:58 +0000, Alex Szpakowski wrote:

Eric's changes seem good to me, but I haven't actually built and tested them on a real Windows system yet – I definitely plan on doing so soon because I want to use the changes in an upcoming version of my own project that uses SDL, though.

On 2017-08-12 20:00:57 +0000, Eric Wasylishen wrote:

I think it's in pretty good shape.

Ryan and I discussed it a bit last week and agreed that having the API work in
virtual screen points and then having rendering contexts have a separate
physical pixel size made sense for SDL.

Yeah, agreed. The patch emulates a virtual screen coordinate system in points on top of Windows' virtual screen coordinates, which are in pixels when the app is highdpi aware.

The outstanding things to do for my patch:

  • rebase it to to account for https://hg.libsdl.org/SDL/rev/0060bcf673e8 (should be easy / doable today)

  • check that it sends a window resize event when the dpi changes

  • decide on the API changes. What I currently do is deprecate the SDL_WINDOW_ALLOW_HIGHDPI window flag and add a SDL_HINT_VIDEO_HIGHDPI_ENABLED hint, so if your SDL app is high-dpi aware, you would set the hint in your app's startup code before initializing SDL. Maybe "SDL_HINT_VIDEO_HIGHDPI_ALLOWED" would be a clearer name? Alex also suggested we could merge the enable/disable hints into one hint.

The other option is, if we limit highdpi support to Windows 10 Anniversary Update and above, I think we could avoid this API change and stick with the window flag, since MS added the ability to enable highdpi on a per-window basis in that update. (otherwise, with my patch as-is, we get highdpi support on Vista and up).

A disadvantage of going with a hint is, you lose the ability to change DPI awareness while the app is running.

Also I'm not sure about the situation on x11, wayland, etc., whether they allow DPI awareness to be set per-window.

  • regarding SDL_DisplayMode, Alex mentioned he was working on adding an API to get the pixel dimensions of a mode. I'm still not totally sure I understand having the w/h fields of display modes in points, at least, for games using SDL I'm pretty sure they will only care about pixels (e.g. in the game's settings menu).

BTW, here is the link where Alex and I were discussing the patch:
ericwa/SDL-mirror@8870be8

On 2017-08-27 23:14:23 +0000, Alex Szpakowski wrote:

I wonder if it's worth it to have both the global app-startup DPI hint and the per-window DPI flag, and a new API to query whether the latter is supported? That way macOS/iOS/Win10-anniversary could enjoy the benefits of per-window high DPI support, and stock Win10 and older can still have global high DPI support.

On 2017-08-27 23:16:57 +0000, Alex Szpakowski wrote:

Side note: I haven't looked into what kind of high DPI / retina support is available on Linux, so I don't know if it's capable of per-window high DPI or not (although I do know SDL's linux backends don't have any code for highdpi support at all, currently).

On 2017-09-04 07:25:37 +0000, Alex Szpakowski wrote:

I tested the latest code from the branch for a few minutes today, some things I noticed:

  • SDL_GetDesktopDisplayMode seemed to return the size in raw pixels rather than DPI-scaled points of the desktop (which doesn't play well with the window position, which is in points). It behaved like that regardless of whether I had the high DPI hint enabled.

  • Similarly, enumerating available display modes seemed to give back sizes in pixels.

  • Creating an exclusive-fullscreen window and then calling SDL_SetWindowFullscreen caused weird issues with the window not sizing correctly / getting stuck in limbo.

On 2017-09-05 07:27:45 +0000, Eric Wasylishen wrote:

Thanks for testing it out!

  • SDL_GetDesktopDisplayMode seemed to return the size in raw pixels rather than DPI-
    scaled points of the desktop (which doesn't play well with the window position, which
    is in points). It behaved like that regardless of whether I had the high DPI hint
    enabled.

I just pushed a commit that should make SDL_GetDesktopDisplayMode return points, consistent with macOS:
https://github.com/ericwa/SDL-mirror/commits/windows-highdpi

  • Similarly, enumerating available display modes seemed to give back sizes in
    pixels.

The API's for enumerating and changing display modes seem to be exclusively pixels -
EnumDisplaySettings ( https://msdn.microsoft.com/en-us/library/windows/desktop/dd162611(v=vs.85).aspx ) and ChangeDisplaySettingsEx ( https://msdn.microsoft.com/en-us/library/windows/desktop/dd183413(v=vs.85).aspx ).

Did a bit of searching, and it looks like the scale factor is stored in the registry on a per-monitor basis: https://stackoverflow.com/questions/35233182/how-can-i-change-windows-10-display-scaling-programmatically-using-c-sharp

So, it's not like macOS where there the OS's mode list also includes scale factors (unless I've overlooked something).

  • Creating an exclusive-fullscreen window and then calling SDL_SetWindowFullscreen
    caused weird issues with the window not sizing correctly / getting stuck in limbo.

I'm not seeing any issues with Ctrl+Enter in testgl2, which toggles between these:
SDL_SetWindowFullscreen(window, SDL_WINDOW_FULLSCREEN);
SDL_SetWindowFullscreen(window, SDL_FALSE);

Are those the SDL_SetWindowFullscreen calls that were causing issues for you?

I wonder if it's worth it to have both the global app-startup DPI hint and the per-
window DPI flag, and a new API to query whether the latter is supported? That way
macOS/iOS/Win10-anniversary could enjoy the benefits of per-window high DPI support,
and stock Win10 and older can still have global high DPI support.

Yeah - I'm thinking the same thing, and going to have a shot at implementing that.

One thing that might be possible is getting rid of the messiest / most fragile part of my patch, the WIN_PhysicalToVirtual_ScreenPoint function, and getting Windows to do the conversion - see the "Windows Placement" section of this blog post:

https://blogs.windows.com/buildingapps/2016/10/24/high-dpi-scaling-improvements-for-desktop-applications-and-mixed-mode-dpi-scaling-in-the-windows-10-anniversary-update/#SYriggHty8TDC6jJ.97

On 2017-10-26 23:04:06 +0000, Eric Wasylishen wrote:

Created attachment 3045
windows highdpi patch

Here's an update to my patch.

I tried, and gave up on, doing per-window DPI awareness on Windows 10 Anniversary (what Windows docs call "mixed-mode" DPI Awareness; see: https://msdn.microsoft.com/en-us/library/windows/desktop/mt744321(v=vs.85).aspx ). It creates a confusing situation where the units returned by API's (e.g. GetClientRect) depend on when you call them (whether inside a window's WindowProc or outside). The bigger problem is it doesn't seem to interact well with OpenGL, at least with Nvidia drivers: what happens is, if you have the process set as DPI unaware, but create a DPI-aware window, OpenGL can only render in 1/4 of the window (with the monitor using 200% scaling). I'm guessing that something in the WGL internals is calling GetClientRect to determine the framebuffer size, but it's not set up to handle the case when the thread's DPI awareness differs from the window's.

So, I'm back to suggesting deprecating the window flag and adding the hint. I renamed the hint to SDL_HINT_VIDEO_ALLOW_HIGHDPI.

Other changes:

  • standardized the terminology in the code to "Windows" and "SDL" coordinates, it was previously all over the place (points/pixels/unscaled/scaled). I went with that instead of "points" and "pixels" because Windows coordinates are only pixels if the process is DPI aware, and also the "SDL" coordinate system is a weird hybrid of points and pixels (the monitor top-left corners are in pixels, but everything else is in points).

  • implement WM_GETDPISCALEDSIZE

  • add documentation for the SDL_HINT_VIDEO_ALLOW_HIGHDPI hint and lots of comments on the confusing bits.

  • figured out how Windows handles display mode changes and high-dpi. I added some explanation in WIN_SetDisplayMode. Short version: changing the screen resolution resets the monitor DPI to 96... unless you change to whatever the current desktop resolution is (in pixels), which changes the monitor DPI to the value saved in Windows's Display Settings.

The main thing I am unsure about in the patch is the change to CreateWindow, now it creates every window as 0x0 initially, then moves/sizes it to the desired location in SetupWindowData. I haven't seen an problems from this, but I can probably modify this to use the WIN_ScreenRectFromSDL function if this looks like a bad idea.

On 2018-10-24 11:13:08 +0000, Ellie wrote:

Ok, so I haven't looked at the patch yet, but:

But can you PLEASE for the love of all things, can you PLEASE CHANGE that SDL_GetWindowPos(), SDL_GetWindowSize() and others do sometimes NOT return native hardware pixels?? (which seems to be the case according to the libSDL wiki)

This is kind of a coordinate nightmare, because 1.) the wiki is absolutely unclear which functions are truly affected, 2.) some parts like the renderer always expect native coordinates anyway, 3.) the application needs to be high dpi aware in any case for things to work correctly, so why do this weird half-baked sometimes-coordinates-are-scaled-up-oddly thing?? It just doesn't make any sense.

Maybe for the original underlying OS API this made sense for some cases, but SDL should really abstract this away. The current behavior is just a mess, hard to understand and doesn't help anyone. Can you please remove this?

I propose adding new flag to SDL_Init to switch to the new behavior. I can also make a separate ticket for this if required.

On 2018-10-24 11:16:26 +0000, Ellie wrote:

I am referring to this paragraph by the way:

"The window size in screen coordinates may differ from the size in pixels, if the window was created with SDL_WINDOW_ALLOW_HIGHDPI on a platform with high-dpi support (e.g. iOS or OS X). Use SDL_GL_GetDrawableSize() or SDL_GetRendererOutputSize() to get the real client area size in pixels. "

For me quite a nightmare, because on Windows with HighDPI as it is now, I can't test this, so how am I even supposed to make sure my application works with it? I tested HighDPI, but with sane pixel coordinates, not whatever that paragraph suggests might be going on sometimes. So I propose that this simply shouldn't be happening, I can't even see why that would ever be useful...

On 2018-10-24 11:27:43 +0000, Ellie wrote:

I just read more in the comments above, the "points" coordinate system is what I am referring to. I would LOVE a way to just get rid of it, some "I am properly DPI aware, just give me the pixels and stop scaling coordinate systems around" flag.

On 2019-10-15 04:19:51 +0000, Anthony Pesch wrote:

Hi everyone,

Is there any interest in getting Eric's patch into master?

I've just recently started using SDL_WINDOW_ALLOW_HIGHDPI for https://redream.io on Mac, but Windows is still problematic.

I'm game for rebasing Eric's patch, but I'm having a hard time figuring out what other work is required / what blocked it from getting in ~2 years ago.

On 2019-10-15 23:45:20 +0000, Alex Szpakowski wrote:

I haven't had time to look at this in a while, but in general I think I'd prefer if per monitor v2 DPI awareness is supported first (since it's more flexible, more modern, and theoretically matches up with SDL's existing APIs a lot better than the older versions), and then the older versions of Windows DPI awareness can be added after if there's a high demand.

On 2019-10-15 23:47:34 +0000, Alex Szpakowski wrote:

(In reply to Alex Szpakowski from comment # 13)

in general I think I'd prefer if per monitor v2 DPI awareness is supported first

Also for selfish reasons - my current monitor setup at my job has two monitors at 150% DPI scaling, and one at 100%, and most Windows apps are horrible at supporting scaling in that situation. I don't want SDL to be added to that pile.

On 2019-10-19 20:06:40 +0000, Sam Lantinga wrote:

Agreed.

Does anyone have a design that we can point to and say "let's do that?" If not, let's coordinate an actual meeting to talk through the issues and figure out what design makes sense for SDL.

On 2019-10-20 11:47:54 +0000, Ellie wrote:

Is this the right time and place to note my suggestion that I think using points instead of just the true pixels anywhere does more harm than good? https://bugzilla.libsdl.org/show_bug.cgi?id=4423 It's ok if this is eventually decided against, I am just suggesting it would be useful to at least consider changing this inconsistency

On 2019-10-20 20:47:06 +0000, Alex Szpakowski wrote:

(In reply to Jonas Thiem from comment # 16)

Is this the right time and place to note my suggestion that I think using
points instead of just the true pixels anywhere does more harm than good?

This is your fourth post about it in this bug report, plus the three other posts you made in the other bug report you linked. I think we understood your opinion in your first post.

I get that it's simpler as a developer to only care about raw pixels and nothing else, but operating systems, display hardware, and user expectations don't work like that anymore. They will continue to work less and less like that going into 2020 and the future. Going down the route of "literally everything is in pixels" results in apps that end up looking broken on various display/OS configurations. I explained a bit in my post in your other thread.

SDL currently has consistent high-dpi support on macOS, iOS, and Wayland, where all windowing and mouse coordinates are in DPI-scaled units, and SDL_GL_GetDrawableSize and SDL_GetRendererOutputSize are in pixels. SDL doesn't have any knowledge about DPI scaling on Windows, X11, and Android (I'm not sure about other platforms such as emscripten), so using those as examples of what to do doesn't make sense. Your other thread talked about consistency and I agree that's important - and SDL on macOS etc. behaves consistently with respect to DPI scales.


In terms of API/design changes going forward, IMO the current system works pretty well on the operating systems it's implemented in (except I'd like a way to determine the DPI scale of a particular display or display mode before creating a window with it), I dunno if I like the idea of drastic changes to the way SDL exposes DPI scale.

If Windows' pmv2 can work with OpenGL drivers (does it work to enable DPI awareness globally and then selectively disable it per window, instead of the opposite?) then the only big question in my mind is whether to expose windowing/monitor/input coordinates on Windows as DPI-scaled points (consistent with SDL on other platforms, and consistent with other platforms in general) or as raw pixels (sort of consistent with Windows right now I think, but who knows in the future).

If windowing/monitor/input coordinates are exposed as pixels instead of DPI-scaled points on Windows, then there also needs to be new APIs to get the DPI scale factor for a given window and a given screen index (and maybe display mode), because dividing the drawable size by the window size won't work anymore.

I suppose a third option is to add new APIs for everything that returns window/monitor/input coordinates, to allow SDL users to choose whether to get the coordinates in DPI-scaled units or pixels, but it also sounds easy for users to use the wrong function accidentally, and it would be a lot of new APIs to add.

On 2019-10-21 07:52:30 +0000, Eric Wasylishen wrote:

Hey, sorry for dropping this. I'm game to help get the patch updated and work out the details.

The last time I updated my patch was Feb 2018: https://github.com/ericwa/SDL-mirror/tree/windows-highdpi

If Windows' pmv2 can work with OpenGL drivers (does it work to enable DPI
awareness globally and then selectively disable it per window, instead of the
opposite?)

Agreed, this is the key issue, because if it works we can implement the current SDL API on Windows (SDL_WINDOW_ALLOW_HIGHDPI window flag), otherwise it's impossible and we would need to go the hint route instead. Good idea to try enabling it and then selectively disabling.

Other API concerns that come to mind:

  • It's unfortunate the SDL mouse events are ints. So if these are in DPI-scaled points on macOS/iOS/Wayland, you'd get mouse movement in 2px steps if you have a 2x scale factor.

  • SDL should report relative mouse movement (xrel/yrel) as raw USB data with no scaling, not points, right?

  • Having SDL_DisplayMode use width/height in points only, with no mention of the pixel size, has some issues.

e.g. consider the retina MBP returning these modes: (hypothetical, not sure what macOS actually returns)
a) 2880x1800 pixels @1x (2880x1800 points), so the "Everything is tiny" mode
b) 2880x1800 pixels @2x (1440x900 points)
c) 1440x900 pixels @1x (1440x900 points), things are visually the same size as b but the game is rendering 1/4 the pixels.

If the only value you can see in the SDL_DisplayMode is points, i.e. you're given the scale factor and pixel size multiplied together, you can't distinguish between b) and c), which are very different display modes.

(On Windows, last I checked, there is no notion of scale factors for display modes other than the desktop one, and changing the display mode to something other than the what the Windows desktop is configured to use resets the scale factor to 1x (96DPI))

  • Lastly there's SDL_GetDisplayDPI. There's the issue where 1x scaling is called 96 DPI on Windows, whereas 1x is called 72DPI on macOS (+wayland?). We could always rescale the Windows values by 72/96 and just document that in SDL 1x = 72DPI on all platforms. Also I remember something about one of the x/y/diagonal DPI components reported by SDL are meant to be the actual physical "pixels / inches" measurement, and others are the scale factor * 72DPI.

On 2019-10-21 14:46:46 +0000, Alex Szpakowski wrote:

I agree with pretty much everything you said. :)

(In reply to Eric Wasylishen from comment # 18)

  • It's unfortunate the SDL mouse events are ints. So if these are in
    DPI-scaled points on macOS/iOS/Wayland, you'd get mouse movement in 2px
    steps if you have a 2x scale factor.

Ah, thanks for bringing this up, I forgot about that. I definitely think non-integer mouse coordinate APIs are needed in the future, if we continue with DPI-scaled units.

(In reply to Eric Wasylishen from comment # 18)

  • SDL should report relative mouse movement (xrel/yrel) as raw USB data with
    no scaling, not points, right?

That's an interesting question.. intuitively I'd say it should be points to be consistent with other mouse movements, but I'm not too familiar with developer and user expectations around relative mouse mode.

@DanielGibson
Copy link
Contributor

As far as I can tell, SDL2 on Windows still has no HighDPI support - does anyone (@ericwa or @slouken ?) know what happened to this?

Is there any chance projects using SDL will ever be able to get rid of their custom workarounds?

@ericwa
Copy link
Contributor

ericwa commented May 19, 2021

I'll take a look at dusting off my PR this weekend, bringing it up to date with current sdl2 etc.

@DanielGibson
Copy link
Contributor

Awesome, thanks a lot!

@PJB3005
Copy link
Contributor

PJB3005 commented Feb 27, 2022

@ericwa Have any advancements been made on this front yet? I'm willing to try dusting the patch off too if necessary.

@ericwa
Copy link
Contributor

ericwa commented Mar 6, 2022

@PJB3005 Sorry for the delay, here's the current status:

My previous work is in a branch called windows-highdpi2, I'm keeping notes on its status in a PR in my SDL repo: ericwa#1

Overall, it's working OK, I think I've ironed out most of the issues. My plan for a while has been to split it into 2 separate patches which I'm hoping to make PR's for soon:

  1. the first PR will introduce a new SDL hint (tentatively calling it SDL_HINT_WINDOWS_DPI_AWARENESS) which makes SDL call the Windows API's to declare the process DPI aware. Alongside this, I'll make SDL work properly if the process is "per-monitor V2" DPI aware - which could either be done through that hint, or in external code. This should fix Windows bug: window size continues to increase or shrink across monitors of different magnifications #4712 (which I think is due to external code enabling "per-monitor V2" DPI awareness while SDL doesn't (yet) support it).

    Current status of this patch is here: https://github.com/ericwa/SDL/commits/windows-dpi-awareness2

    This is just about ready to make a PR for, just needs a last bit of cleanup.

    For SDL apps that just want "1 SDL unit = 1 pixel", this hint would be all they need.

  2. the second PR will be the rest of Windows highdpi2 ericwa/SDL#1 rebased onto the above patch. This will be another SDL hint which change the SDL coordinate system to DPI-scaled points (matching SDL's highdpi behaviour on macOS/Wayland). This one requires more invasive changes to SDL, but basically all of that work is done in Windows highdpi2 ericwa/SDL#1 and I'm hoping splitting it from the first patch (SDL_HINT_WINDOWS_DPI_AWARENESS) will make it more manageable.

ericwa added a commit to ericwa/SDL that referenced this issue Mar 10, 2022
The hint allows setting a specific DPI awareness ("unaware", "system", "permonitor", "permonitorv2").

This is the first part of High-DPI support on Windows ( libsdl-org#2119 ).
It doesn't implement a virtualized SDL coordinate system, which will be
addressed in a later commit. (This hint could be useful for SDL apps
that want 1 SDL unit = 1 pixel, though.)

Detecting and behaving correctly under per-monitor V2
(calling AdjustWindowRectExForDpi where needed) should fix the
following issues:

libsdl-org#3286
libsdl-org#4712
@slouken slouken removed the bug label May 11, 2022
slouken pushed a commit that referenced this issue Jun 11, 2022
The hint allows setting a specific DPI awareness ("unaware", "system", "permonitor", "permonitorv2").

This is the first part of High-DPI support on Windows ( #2119 ).
It doesn't implement a virtualized SDL coordinate system, which will be
addressed in a later commit. (This hint could be useful for SDL apps
that want 1 SDL unit = 1 pixel, though.)

Detecting and behaving correctly under per-monitor V2
(calling AdjustWindowRectExForDpi where needed) should fix the
following issues:

#3286
#4712
@icculus icculus added this to the 3.0 milestone Aug 23, 2022
@slime73
Copy link
Contributor

slime73 commented Dec 28, 2022

I had some thoughts last night about display/window API changes to better support high dpi systems. There are a few other github issues scattered around (#6038, #5290, and mouse integer vs float talk in #2999) so I'm not sure if this is the best place to discuss, but here they are.

With this design I'm also assuming that there aren't any systems which have unique DPI scales for horizontal versus vertical resolution. I think that's true but someone correct me if I'm wrong - some stuff here would have to change.


SDL_DisplayMode: width and height are in DPI-scaled points. have an extra DPI scale factor float field.

typedef struct
{
    Uint32 format;              /**< pixel format */
    int w;                      /**< width, in DPI-scaled points */
    int h;                      /**< height, in DPI-scaled points */
    float dpi_scale;            /**< DPI scale factor used to convert from points to pixels */
    int refresh_rate;           /**< refresh rate (or zero for unspecified) */
    void *driverdata;           /**< driver-specific data, initialize to 0 */
} SDL_DisplayMode;

Rationale: This allows SDL users to know the DPI-scaled size and the pixel size of a display / display mode. Not all platforms have the concept of per-displaymode DPI scales, but I think it's important to have that info available for those that do. All platforms do have at least per-display DPI scales, so the dpi scale field of a display mode can be filled in with that on platforms where there's no real per-mode scale.

I'm not entirely sold on having a dpi scale field here instead of pixel width/height fields (along with the existing dpi-scaled width/height fields). Having extra pixel width/height fields is arguably more immediately useful, but maybe less flexible than the dpi scale field (although there are pitfalls to a dpi scale value, see below). I suppose we could have both even if it's a bit redundant.

edit: after re-reading Eric's description of Windows display mode behaviour (#5778 (comment)) I'm now more in favour of having all 3 of dpiscaledsize, pixelsize, and dpiscale in the display mode struct. (If we end up not using DPI scale values at all anywhere else then that one could be removed in display modes too, but otherwise I still feel like it's handy.)


Add new functions to convert between pixels and DPI-scaled points using the same rounding method as the active platform backend uses:

// These names could use a lot of workshopping, it's just a demo.
// I'm also not sure if we should have both the vector and single component versions or just pick one.
int SDL_GetPointComponentFromPixel(int pixel, float dpiscale);
int SDL_GetPixelComponentFromPoint(int point, float dpiscale);

void SDL_GetPointFromPixel(int pixelx, int pixely, float dpiscale, int * pointx, int * pointy);
void SDL_GetPixelFromPoint(int pointx, int pointy, float dpiscale, int * pixelx, int * pixely);

Rationale: each platform has a specific way of rounding when converting between pixels and DPI-scaled units (#5827 (comment)). These functions would call into per-backend code to guarantee that they round in the same manner as the platform would. This makes sure there aren't 1 pixel/point differences between user code which uses this API, versus the actual backbuffer size and other OS-computed sizes.

When users just want high precision floating point conversions between DPI-scaled points and pixels, they can do floating point multiplications and divisions with the DPI scale instead of using these functions.


Keep SDL_GetWindowSizeInPoints, also add SDL_GetWindowDPIScale:

void SDL_GetWindowSize(SDL_Window * window, int *w, int *h);
void SDL_GetWindowSizeInPixels(SDL_Window * window, int *w, int *h);
float SDL_GetWindowDPIScale(SDL_Window * window);

Rationale: SDL_GetWindowSizeInPixels is a nice convenience function for when you just care about what the backbuffer size should be. A separate DPI scale function is also useful to use together with the above conversion functions, and to make sure users aren't getting a subtly wrong system/monitor DPI scale by dividing window pixel size by window size when the window size is very small (see discussion in #5778).


Also, SDL_GetDisplayDPI has caused a lot of confusion in the past. IMO it should be renamed to be more clear that it doesn't have to do with DPI scale factors (maybe SDL_GetDisplayPhysicalDPI or something), if it isn't removed outright.

@icculus icculus modified the milestones: 3.x, 3.2.0 Jan 9, 2023
@Kontrabant
Copy link
Contributor

Kontrabant commented Jan 9, 2023

Add new functions to convert between pixels and DPI-scaled points using the same rounding method as the active platform backend uses:

// These names could use a lot of workshopping, it's just a demo.
// I'm also not sure if we should have both the vector and single component versions or just pick one.
int SDL_GetPointComponentFromPixel(int pixel, float dpiscale);
int SDL_GetPixelComponentFromPoint(int point, float dpiscale);

void SDL_GetPointFromPixel(int pixelx, int pixely, float dpiscale, int * pointx, int * pointy);
void SDL_GetPixelFromPoint(int pointx, int pointy, float dpiscale, int * pixelx, int * pixely);

Would it be better for these functions to take a window handle instead of the scale float value? These are going to be used to convert coordinates related to the current state of a window, and using the window handle would take the onus of querying the DPI scale off of the user and prevent potential bad behavior like caching the scale value, which could cause issues on mixed DPI setups if an application doesn't update the value in every case where it's required and performs calculations with a stale value.

@slime73
Copy link
Contributor

slime73 commented Jan 9, 2023

Yeah using it with a window definitely seems like the most common use case and it makes sense to design it to avoid mistakes while using the API.

I was thinking with the more generic versions they could also be used when a window hasn't been created yet, and for screens or display modes a window isn't currently active in (since the other proposed changes would allow you to query the DPI scale in those situations). But I don't know if that would actually be useful in practice. We could have a SDL_Window* and a float dpiscale version, but that might bloat the API a bit too much if one of them doesn't have a lot of practical uses.

@DanielGibson
Copy link
Contributor

I would appreciate if there was a mode (maybe set via flag when creating a window) that just lets me use physical pixels for everything, so the backbuffer size matches the real window size in pixels on the screen and the mouse coordinates etc also match.

Not sure what SDL_GetWindowDPIScale() would return then - probably 1.0?
But it would be nice to have a function that gives me the scale that would otherwise have been used, so I can implement my own scaling if I want to.

@slime73
Copy link
Contributor

slime73 commented Jan 9, 2023

I think a mode like that would have to be set globally (probably via a hint before video initialization).

Various platforms don't natively work like that, so there would have to be a lot of conversion code inside SDL to turn units into pixel coordinates automatically for you.

I think it could also lead to a lot of trouble and buggy user code in multi-display setups and such. Personally I'm against the idea, but I won't stop anyone from implementing it as an opt-in feature of course. With SDL2's existing APIs plus my proposals above, you could already use pixel units for everything if you call the appropriate conversion functions.

@DanielGibson
Copy link
Contributor

so there would have to be a lot of conversion code inside SDL

Better than having this in user code.

Remember, SDL is for games, and while I'm not sure how modern games handle High DPI, a big usecase are sourceports of old games, and those definitely expect all sizes to match.

And having such a mode would also help sdl12-compat (and possibly sdl2-compat), because SDL1.2 had no concept of HighDPI at all, and I wouldn't expect all existing SDL2 applications to handle it correctly either (and apart from that, SDL2 doesn't even handle it correctly itself, see https://discourse.libsdl.org/t/how-do-i-bugfix-surface-size-in-windows-high-dpi/41647/18)

@DanielGibson
Copy link
Contributor

DanielGibson commented Jan 9, 2023

I think it could also lead to a lot of trouble and buggy user code in multi-display setups and such.

In the contrary: When moving a Window from one screen to another, (in this mode) SDL should keep its physical size, so if the SDL application has the window size cached somewhere or doesn't handle some event to resize the OpenGL framebuffer or whatever (not unreasonable if the application didn't set the window as resizable), it still works

@slime73
Copy link
Contributor

slime73 commented Jan 9, 2023

Again I think there are big problems with user display setups in practice when you try to do this, especially for code that doesn't know about DPI scales at all (which you're suggesting is one of the main reasons for this change). The way legacy apps have been handled across pretty much every operating system in the past when running on a high dpi screen is for the OS itself to pretend to the app that it's running at 1x DPI scale, and then the OS scales up the window's contents automatically to be in DPI-scaled units. This is what happens in SDL already when you don't enable high dpi support programmatically, and it's sort of the inverse of what you suggest.

I don't think there are plans to change app-handled DPI scaling from being opt-in right now (but it hasn't been brought up either as far as I know, so maybe people have opinions.)

I have some bad memories of certain AAA games just telling the OS they support high DPI, without doing any actual scaling. I could barely read any UI text and every UI element was tiny, definitely not a fun experience.

When moving a Window from one screen to another, (in this mode) SDL should keep its physical size, so if the SDL application has the window size cached somewhere or doesn't handle some event to resize the OpenGL framebuffer or whatever (not unreasonable if the application didn't set the window as resizable), it still works

Keep in mind the physical size to the user is not actually the size in pixels, it's the DPI-scaled size. So just keeping the size in pixels consistent will make the window extremely big or extremely small (from the user's perspective) as soon as it's moved to a different display with a different DPI scale.

@DanielGibson
Copy link
Contributor

The way legacy apps have been handled across pretty much every operating system in the past when running on a high dpi screen is for the OS itself to pretend to the app that it's running at 1x DPI scale, and then the OS scales up the window's contents automatically to be in DPI-scaled units.

My main experience with this is on Windows, and there that breaks many applications, and even more so games ("why does it render only in one corner of the window?!").

I can imagine that in some games there are problems with UI elements being too small on HighDPI, but I don't think you can rely on some kind of scaling magic to work, you'll likely just get different problems instead. If they didn't have HighDPI in mind for scaling the UI, they most likely won't like it when the OpenGL (or DirectX or whatever) framebuffer has a different size than what they think the window has.

This is what happens in SDL already when you don't enable high dpi support programmatically, and it's sort of the inverse of what you suggest.

If this were true, Yamagi Quake II and dhewm3 wouldn't need this kind of code: https://github.com/dhewm/dhewm3/blob/477252308d1ea8660e2d040902d58a599122047f/neo/sys/win32/win_main.cpp#L891-L939

(Ok, maybe it is true now, but when we added that code it wasn't and I haven't tried if it's still needed in latest SDL2)

Keep in mind the physical size to the user is not actually the size in pixels, it's the DPI-scaled size. So just keeping the size in pixels consistent will make the window extremely big or extremely small (from the user's perspective) as soon as it's moved to a different display with a different DPI scale.

That's the users problem then, they can resize it before or afterwards (and apart from that, who moves a game window from one screen to another, esp. when they have vastly different resolutions)

@slime73
Copy link
Contributor

slime73 commented Jan 9, 2023

I agree Windows has had many problems with proper high DPI support. That's why I don't want to replicate its mistakes and I'd rather learn from them instead. The way it dealt with coordinates in its own APIs is a big reason why so many apps on Windows have had poor user experience in high DPI situations. Contrast that with macOS where I don't think I've seen an app that has broken layout or sizing issues as a result of DPI scaling.

SDL's current approach, and changes we've discussed in the past to try to make things more consistent and useful, have kept closer to the macOS model (points-first) than the Windows model (pixels-first) and I think it's been the right choice.

@DanielGibson
Copy link
Contributor

I also had confusing bugreports from Mac users regarding High DPI, but couldn't debug them in detail because I don't have a Mac.
IIRC it included fun stuff like "if the window is DPI scaled or not depends on the requested resolution"

But note that I'm not arguing against your approach, I just want an option for it to behave like many games expect, to minimize breakage (and required changes to the games code)

@slime73
Copy link
Contributor

slime73 commented Jan 9, 2023

if the window is DPI scaled or not depends on the requested resolution

Yeah, macOS exposes both low-DPI and high-DPI exclusive fullscreen modes which is a little different than most other desktop operating systems I think. The trouble there has been that SDL doesn't provide enough information to tell what's what. #2119 (comment) should help with that at least.

@Kontrabant
Copy link
Contributor

Kontrabant commented Jan 9, 2023

I was thinking with the more generic versions they could also be used when a window hasn't been created yet, and for screens or display modes a window isn't currently active in (since the other proposed changes would allow you to query the DPI scale in those situations). But I don't know if that would actually be useful in practice. We could have a SDL_Window* and a float dpiscale version, but that might bloat the API a bit too much if one of them doesn't have a lot of practical uses.

The issue I can see here is that in a mixed-DPI setup, the actual, final scale value isn't known until the window is shown and in its final position. Even on platforms that let you specify where the created window will be, there's no guarantee that the OS or something else won't override your request and put the window somewhere else.

SDL's current approach, and changes we've discussed in the past to try to make things more consistent and useful, have kept closer to the macOS model (points-first) than the Windows model (pixels-first) and I think it's been the right choice.

Not only Mac, but basically every platform besides Windows uses logical points for scaled desktops.

If someone really wants to get a window with a physical backbuffer of a certain size, they can use the proposed void SDL_GetPointFromPixel(int pixelx, int pixely, float dpiscale, int * pointx, int * pointy); to get the required logical window size and resize the window to that. Not a huge fan of building an 'official' path for windows that work in physical pixels though, as if you are building a modern DPI-aware application, you have to deal with scaling if you want your application to behave in a way that the user expects. There's really no way around it. With displays becoming increasingly pixel-dense, naively using physical pixels on something like an 8k display can result in microscopic windows and UI elements.

Having one opinionated way of doing things that gives identical results across platforms ultimately means fewer bugs and badly-behaved applications in the long-term.

That's the users problem then, they can resize it before or afterwards (and apart from that, who moves a game window from one screen to another, esp. when they have vastly different resolutions)

Laptop users for one. If it can be done, someone will do it, and it should be handled.

@Frenzie
Copy link

Frenzie commented Jan 9, 2023

Not only Mac, but basically every platform besides Windows uses logical points for scaled desktops.

Maybe I'm misunderstanding, but that sounds like everything other than Windows is always guaranteed to be blurry? You need to know both the desired "scaling" factor and the actual number of pixels or it can never be sharp.

@slime73
Copy link
Contributor

slime73 commented Jan 9, 2023

Logical points is the coordinate system (which can have fractional values), not the resolution of things. Maybe this will help illustrate it: https://developer.apple.com/library/archive/documentation/GraphicsAnimation/Conceptual/HighResolutionOSX/Explained/Explained.html

Apple devices in particular are pretty well known at this point for having high pixel density displays with sharp / non-blurry content. :)

In SDL2, when you opt into high dpi, you can get the pixel dimensions of a window's backbuffer via SDL_GetWindowSizeInPixels, and the dimensions in points via SDL_GetWindowSize. The ratio of the two is the scaling factor but you don't always need that ratio directly if you have those two other values.

@Frenzie
Copy link

Frenzie commented Jan 10, 2023

Reading those docs it sounds like it's still describing blurry.

When the framework detects that the controls need to be drawn on a high-resolution display, it adjusts the backing store to accommodate scaling each dimension by 2x. Whether standard or high resolution, the user sees high-quality rendering of the controls. The controls appear the same size to the user regardless of the backing store size, but on a high-resolution display, the controls look sharper due to the density of the backing pixels.

If you render everything at "2x" (or 3x, etc.) followed by downscaling then you're not only rendering much more than needed but you're making it blurry by downscaling. A better type of blurry is still blurry.

But that seems to be a Cocoa-specific implementation oddity, orthogonal to the logical coordinates and real pixels you described, which sound like they should be good.

@ericwa
Copy link
Contributor

ericwa commented Jan 23, 2023

SDL_DisplayMode

edit: after re-reading Eric's description of Windows display mode behaviour (#5778 (comment)) I'm now more in favour of having all 3 of dpiscaledsize, pixelsize, and dpiscale in the display mode struct. (If we end up not using DPI scale values at all anywhere else then that one could be removed in display modes too, but otherwise I still feel like it's handy.)

Sounds good to me, so something like this?

typedef struct
{
    Uint32 format;              /**< pixel format */
    int w_points;               /**< width, in DPI-scaled points. */
    int h_points;               /**< height, in DPI-scaled points */
    int w_pixels;               /**< width, in pixels */
    int h_pixels;               /**< height, in pixels */
    float dpi_scale;            /**< DPI scale factor used to convert from points to pixels */
    float refresh_rate;         /**< refresh rate (or zero for unspecified) */
    void *driverdata;           /**< driver-specific data, initialize to 0 */
} SDL_DisplayMode;

Observation: I believe we will have the invariant that, once a SDL window successfully enters exclusive fullscreen, the current display mode's w_points and h_points should match the SDL_GetWindowSize as well as SDL_GetDisplayBounds, and w_pixels/h_pixels should match the window's SDL_GetWindowSizeInPixels. This should apply even if the window isn't SDL_WINDOW_ALLOW_HIGHDPI, I think? Anyways, just something to check for if we go with this design.

All platforms do have at least per-display DPI scales, so the dpi scale field of a display mode can be filled in with that on platforms where there's no real per-mode scale.

Small comment here - on Windows I would suggest the non-desktop modes to be filled in with a scale factor of 1.0.

Reasons:

  1. Windows itself returns 100% if you change the mode to something other than the desktop mode, and then query the scale factor
  2. From personal experience with a 4K/32" monitor, I use either 125% or 150% scaling with the monitor's native resolution, but if I switch to a lower resolution (usually 1440p) for gaming, I wouldn't want any DPI scaling. The useful scale factor is dependent on the mode in pixels that I'm using.

Also, one Windows-specific question: if the process is DPI aware, but SDL's highdpi mode isn't enabled (SDL_HINT_WINDOWS_DPI_SCALING is 0), would dpi_scale be 1.0 for all SDL_DisplayModes? I'm thinking probably 1.0 for all (but see below, I'm thinking we would want SDL_GetDisplayDPIScale to return the scale factor in this case.) The same question might also apply on SDL backends that don't currently support SDL_WINDOW_ALLOW_HIGHDPI e.g. Android.


Keep SDL_GetWindowSizeInPoints, also add SDL_GetWindowDPIScale:

void SDL_GetWindowSize(SDL_Window * window, int *w, int *h);
void SDL_GetWindowSizeInPixels(SDL_Window * window, int *w, int *h);
float SDL_GetWindowDPIScale(SDL_Window * window);

Rationale: SDL_GetWindowSizeInPixels is a nice convenience function for when you just care about what the backbuffer size should be. A separate DPI scale function is also useful to use together with the above conversion functions, and to make sure users aren't getting a subtly wrong system/monitor DPI scale by dividing window pixel size by window size when the window size is very small (see discussion in #5778).

Sounds good, we should also document the official way to convert e.g. mouse coodinates back to pixels is:

float x_pixels = event.motion.x * SDL_GetWindowDPIScale(window);

Precision question: say the SDL game requests a (103x100) window, and the OS scale factor is 1.5x.
This gives 154.5 x 150 pixels, which would probably get rounded to 155x150.

This introduces a possible inconsistency of two different interpretations:

  1. if you trust the scale factor of 1.5x and the backbuffer size of 155x150 pixels, the true window size in points is (103.333.. x 100)
  2. if you trust the (103x100) points and (155x150) pixels, then the scale factor is actually non-uniform in X and Y: (1.5048 x 1.5)

I don't know how much this matters.. but doing a float x_pixels = event.motion.x * SDL_GetWindowDPIScale(window); is implicitly taking interpretation 1.


Also, SDL_GetDisplayDPI has caused a lot of confusion in the past. IMO it should be renamed to be more clear that it doesn't
have to do with DPI scale factors (maybe SDL_GetDisplayPhysicalDPI or something), if it isn't removed outright.

Agreed. To add to the confusion about SDL_GetDisplayDPI:

  • it returns the user's DPI scale on Windows, versus physical DPI on macOS
  • the units: on Windows, it uses the Windows internal units where 96 = 100%, etc.
  • the meaning of ddpi is inconsistent - in WIN_GetDisplayDPI it just returns the same thing as hdpi and vdpi, in other places (wherever SDL_ComputeDiagonalDPI is called), ddpi returns sqrt(hdpi*hdpi + vdpi*vdpi)

For physical DPI, if we want to keep this, I would drop the ddpi param and rename it to SDL_GetDisplayPhysicalDPI:

/* returns an error code */
int SDL_GetDisplayPhysicalDPI(int displayIndex, float *xdpi, float *ydpi);

We'll probably also want a way to get the monitor's DPI scale factor - this would replace current uses of SDL_GetDisplayDPI on Windows:

float SDL_GetDisplayDPIScale(int displayIndex);

Not sure if it needs to return an int error code. One special attribute of this (matching current behaviour of SDL_GetDisplayDPI on Windows) is, I would have it return the scale factor even if SDL's DPI awareness isn't enabled (SDL_HINT_WINDOWS_DPI_SCALING is 0, but SDL_HINT_WINDOWS_DPI_AWARENESS is permonitorv2 or Windows DPI awareness is enabled externally).

This would give some ability for games to bypass SDL's hihghdpi support but maybe still adjust the initial window size.


Other thoughts:

Should we unify the ways of activating highdpi mode in SDL3? Currently we have:

  • SDL_HINT_WINDOWS_DPI_SCALING hint on Windows
  • SDL_WINDOW_ALLOW_HIGHDPI window flag on macOS/Wayland/emscripten

We could replace these with a single hint, SDL_HINT_ALLOW_HIGHDPI or SDL_HINT_DPI_SCALING, and delete the window flag?

@Kontrabant
Copy link
Contributor

Small comment here - on Windows I would suggest the non-desktop modes to be filled in with a scale factor of 1.0.

I don't think there are any platforms where it shouldn't be 1.0, as exclusive fullscreen in general implies that the application owns the display (even if every modern desktop just fakes it now, iirc even Windows doesn't actually change modes anymore) and any scaling will be handled further down the chain.

Precision question: say the SDL game requests a (103x100) window, and the OS scale factor is 1.5x. This gives 154.5 x 150 pixels, which would probably get rounded to 155x150.

This introduces a possible inconsistency of two different interpretations:

1. if you trust the scale factor of 1.5x and the backbuffer size of 155x150 pixels, the true window size in points is (103.333.. x 100)

2. if you trust the (103x100) points and (155x150) pixels, then the scale factor is actually non-uniform in X and Y: (1.5048 x 1.5)

I don't know how much this matters.. but doing a float x_pixels = event.motion.x * SDL_GetWindowDPIScale(window); is implicitly taking interpretation 1.

This is a situation where the aforementioned functions such as SDL_GetPixelFromPoint() would be useful, since it means that applications don't have to worry about this. Just call the function, which can be passed through all the way to the video backend if necessary, and let it handle any rounding issues. Maybe even return both window-relative logical and backbuffer-relative physical coordinates in mouse events so applications generally don't have to resort to doing extra calculations to figure this out.

Something else that might be useful would be a setting to lock window resize increments to whole pixels so that if someone resizes a window by dragging, they won't wind up with a case where the backbuffer size has to be rounded off.

Other thoughts:

Should we unify the ways of activating highdpi mode in SDL3? Currently we have:

* `SDL_HINT_WINDOWS_DPI_SCALING` hint on Windows

* `SDL_WINDOW_ALLOW_HIGHDPI` window flag on macOS/Wayland/emscripten

We could replace these with a single hint, SDL_HINT_ALLOW_HIGHDPI or SDL_HINT_DPI_SCALING, and delete the window flag?

A hint is all-or-nothing though, so you couldn't have mixed high-dpi/non-high-dpi windows in the same application. Might be a problem in SDL2-compat, as SDL2 allows mixing window types.

It might be also be worth thinking about having dpi-awareness be opt-out instead of opt-in going forward, as anything new really should be thinking about dpi-awareness from the start and legacy apps can opt out.

@slouken
Copy link
Collaborator

slouken commented Jan 23, 2023

This patch is in (thanks @ericwa!) and I'm moving discussion for the final SDL 3.0 approach to #7134.

@slouken slouken closed this as completed Jan 23, 2023
@slime73
Copy link
Contributor

slime73 commented Jan 23, 2023

I don't think there are any platforms where it shouldn't be 1.0

macOS has display modes with different DPI scales (#5290 (comment)). Each display mode is associated with a different intended user experience in that case: 2880x1800@1x is different from 1440x900@2x in terms of expected scaling. e.g. the former mode doesn't allow a fullscreen app to use the operating system's scaling when the app doesn't support its own scaling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants