This bug report was migrated from our old Bugzilla tracker.

Reported in version: HG 2.1
Reported for operating system, platform: Windows (All), All

Comments on the original bug report:

On 2013-08-29 12:35:55 +0000, Nathaniel J Fries wrote:

The Windows implementation of SDL should always just use the Unicode version of the Win32 functions. Not only is this what is recommended by Microsoft, but usually the ANSI versions are simply wrappers around the Unicode version anyway, so it's just extra overhead.

The only reason Microsoft even keeps the ANSI functions around is to support old applications originally created for Windows 95/98/ME. SDL 2.0 couldn't be built for Windows 95 without significant changes anyway, so even if someone wanted to use SDL 2.0 for a Windows 95 application, it would basically require its own video backend.

See:
Introduction to Windows Programming: Working with Strings
http://msdn.microsoft.com/en-us/library/windows/desktop/ff381407%28v=vs.85%29.aspx

On 2013-08-29 13:15:38 +0000, Nathaniel J Fries wrote:

Additionally, you're treating wchar_t/WCHAR/WSTR as UCS-2-INTERNAL. It's actually UTF-16LE, even if Windows support for it isn't perfectly consistent yet. Windows does, in fact, handle Window input outside the BMP (codepoint > U+FFFF), as it is necessary to represent a character set required by law to be used in Chinese government documents.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/dd374081%28v=vs.85%29.aspx

Also, there are two related errors in src/video/windows/SDL_windowsevents.c
1: Handling WM_KEYDOWN
ToUnicode is used improperly. ToUnicode fills a buffer of type WCHAR with the UTF-16 representation of the current keyboard input, if it is in fact character input. Not only is this buffer wrongly treated as UTF-32, but the use of the function ignores the possibility that the input may require more than one codepoint to encode the current input character (in fact, it may require several, depending on the input language). Best to use a buffer that can hold at least 8 code units (i.e., WCHAR buffer[8]).
http://msdn.microsoft.com/en-us/library/windows/desktop/ms646320%28v=vs.85%29.aspx
2: Handling WM_CHAR
Similarly to WM_KEYDOWN, WM_CHAR improperly treats the character as UTF-32. In Unicode-enabled windows (Windows created with CreateWindowW), wParam would be UTF-16. In ANSI windows, wParam would have the associated ANSI encoding.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms646276%28v=vs.85%29.aspx

On 2013-09-05 13:16:09 +0000, Sam Lantinga wrote:

I agree, this would be a good change. Do you already have a proposed patch?

On 2013-10-18 07:16:22 +0000, Sam Lantinga wrote:

FYI, I committed fixes which make SDL use UNICODE by default, and UTF-16LE for string conversion.

The windows message handling isn't fixed yet, so I'm leaving this bug open.

Thanks for the good info!