| Summary: | Message boxes on X don't support UTF-8 | ||
|---|---|---|---|
| Product: | SDL | Reporter: | Sik <sik.the.hedgehog> |
| Component: | *don't know* | Assignee: | Ryan C. Gordon <icculus> |
| Status: | RESOLVED FIXED | QA Contact: | Sam Lantinga <slouken> |
| Severity: | normal | ||
| Priority: | P2 | CC: | jwyatt, meklu |
| Version: | HG 2.0 | Keywords: | triage-2.0.4 |
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Attachments: | Broken message box screenshot | ||
Upon further investigation, SDL seems to be using XDrawString, which takes in 8-bit characters. There's XDrawString16 which takes 16-bit characters, so that may be of some help though I have no idea how it handles the encoding. Does anybody know how does X handle text encoding?
Yeah, we're using XFontStruct and XDrawString, we should be using XFontSet and XmbDrawString.
http://www.debian.org/doc/manuals/intro-i18n/ch-output.en.html
--ryan.
Should be fixed in hg changeset 7c464a9ecf48. Please note that you'll probably need to export LANG=en_US.UTF-8 (or something like that, as most Linux distros do by default now), and you _may_ need to install fonts that have non-English glyphs. --ryan. Looks like this bug is back again? What happens when I use all ASCII: http://i.imgur.com/zU3qKiY.png What happens when the value inside parenthesis is changed to "SÉGA MÉGA DRÍVE¡¡": http://i.imgur.com/XNk3CHc.png What happens when the value inside parenthesis is changed to "セガメガドライブ！！": http://i.imgur.com/F37uR2q.png SDL version 2.0.3-9008. Marking a large number of bugs with the "triage-2.0.4" keyword at once. Sorry if you got a lot of email from this. This is to help me sort through some bugs in regards to a 2.0.4 release. We may or may not fix this bug for 2.0.4, though! I did some research on this a few months(?) ago, and the culprit is that XCreateFontSet() wants a UTF-8 locale to be set before being called. It just so happens that when the application itself doesn't set the program-wide locale to something_or_other.UTF-8, things explode. A mere thread-specific uselocale() (and a locale restoration) in the message box functions didn't work in resolving the issue, so the fix might be a tad tricky to implement; a full-blown setlocale() was needed. Hypothesis: this might maybe almost be fixable if from the very beginning we spawn an X thread that does all X communication, and set that thread's locale to something_or_other.UTF-8. Just to recap: X wants you to set a UTF-8 locale yourself, and doesn't really let you do it in a thread-safe way.
Pretty sure this is the specific culprit:
https://hg.libsdl.org/SDL/rev/c259f0817583
--ryan.
(In reply to Ryan C. Gordon from comment #7) > Pretty sure this is the specific culprit: > > https://hg.libsdl.org/SDL/rev/c259f0817583 That is to say, if you flip SDL_SET_LOCALE to 1 in that file, the bug vanishes. The downside is that setlocale() isn't thread safe, so while the message box is up, we can be screwing up other threads, and if one also set another locale while the message box is up, we'll reset it to the wrong value. We already have code to fork() off and do the message box in a child process, so we can do whatever we want with setlocale(), but that got turned off too. I think it was considered too heavyweight? I don't know. --ryan. I turned this back on in https://hg.libsdl.org/SDL/rev/3df83030dab9, we'll see what happens. --ryan. |
Created attachment 996 [details] Broken message box screenshot So, I updated to the latest snapshot (2.0.6713 as of today) and suddenly my game breaks - it can't initialize the video mode apparently. While I'm trying to figure out what's wrong with that, I also noticed another issue: the game was set to Japanese for whatever reason, and when the game tried to show the message box the output was completely mangled (see attached image). The game passes the error string as UTF-8, but it seems the message box is using a different encoding. This is on Ubuntu 11.10 x86-64. PS: the message should read "グラフィックモードをイニシャライズすることができませんでした。" if displayed properly.