This bug report was migrated from our old Bugzilla tracker.

These attachments are available in the static archive:

• Xlib test program for Input Method (xlib-im-test.c, text/plain, 2006-02-01 18:25:20 +0000, 11170 bytes)
• Xlib test program, extended (xlib-im-test.c, text/plain, 2006-02-03 03:33:22 +0000, 11338 bytes)
• First pass at making composite characters work (x11-composite-char-patch.txt, text/plain, 2006-02-03 08:37:13 +0000, 11831 bytes)

Reported in version: HG 1.2
Reported for operating system, platform: Linux, x86

Comments on the original bug report:

On 2006-02-01 18:09:45 +0000, John Popplewell wrote:

Using a non-English, e.g. Spanish, keyboard mapping on Xorg or XFree86 with a slightly tweaked 'checkkeys' test program (see the mailing list post), various dead-key-followed-by-e.g.-vowel sequences don't produce the correct Unicode character in the unicode field of SDL_keysym.

The correct characters are displayed in non-SDL programs e.g. xterm, konsole. I've tried various versions of Xorg and SDL 1.2.8, 1.2.9 and latest CVS with no luck.

The mailing list post has some specific but not very clear examples :-) I'm hampered by only having a UK and US(laptop) keyboard. Here's a picture of a spanish keyboard I found useful: http://www.forlang.wsu.edu/images/kspanish.gif

On 2006-02-01 18:25:20 +0000, John Popplewell wrote:

Created attachment 62
Xlib test program for Input Method

This test program is based on an Xlib tutorial I found on the web, the existing IM code in SDL, and some changes that make dead-key compose sequences work. The output is a bit like that from the SDL test program checkkeys.

The key changes are the use of XFilterEvent() - the sequences aren't decoded correctly without this, and the removal of a call to XUnsetICFocus().

With a bit of fiddling, I believe this could be made to work in 'SDL_x11events.c', and preserve existing behavior.

'Typical' use of XFilterEvent() isn't possible because it filters out the raw key pressed/release events that we need.

On 2006-02-03 03:33:22 +0000, Sam Lantinga wrote:

Created attachment 66
Xlib test program, extended

Thanks John, this is very helpful!
I

On 2006-02-03 03:40:58 +0000, Sam Lantinga wrote:

(In reply to comment # 2)

With a little more testing, it looks like we can process events with a keycode normally, and then deliver the composed input as a separate message.

For 1.2, we can deliver SDLK_UNKNOWN down with the unicode field set, and then SDLK_UNKNOWN up, for each character.

For 1.3, maybe we should create a separate message for text input? Perhaps even a message for partially composed input and another message for completed input? The completed input can be a phrase of characters (chinese), so maybe the data for the input message should be a sequence of utf-8 or ucs-32 characters?

Thoughts?

On 2006-02-03 03:44:23 +0000, Sam Lantinga wrote:

Does anyone have an asian IME that they can try with these test programs?

On 2006-02-03 07:57:07 +0000, John Popplewell wrote:

(In reply to comment # 3)

With a little more testing, it looks like we can process events with a keycode
normally, and then deliver the composed input as a separate message.
Yes, that seems to be possible. I've nearly got a patch together that covers this.

For 1.2, we can deliver SDLK_UNKNOWN down with the unicode field set, and then
SDLK_UNKNOWN up, for each character.
I had a problem doing this, related to SDL_PrivateKeyboard() which needs SDL_keysym to have a valid 'sym' field in press/release event pairs or it starts rejecting events leaving 'stuck' keys.

I've had some success 'remembering' sym and scancode values from previous X key events which can then be used to fill-in-the-blanks in later messages.

For 1.3, maybe we should create a separate message for text input? Perhaps
even a message for partially composed input and another message for completed
input? The completed input can be a phrase of characters (chinese), so maybe
the data for the input message should be a sequence of utf-8 or ucs-32
characters?

Thoughts?
I don't know enough about the IM/IME stuff yet to have anything to add, but from my recent experiences, initially, ucs-16 seems to be the worst possible choice: a single character can't hold the full character-set, it isn't compact and it isn't C-string safe.

However, Windows uses ucs-16 for wchar, with wobbly support on Win98 for anything else and there's this 'surrogate' scheme allowing the full character-set to be represented using ucs-16 characters, but no (official) support on Win98, and registry settings, as described here: http://www.i18nguy.com/surrogates.html, required for 2K/XP etc. Yuck!

Everyone else punts, and uses utf-8, except where they use ucs-16 (TT fonts?), or ucs-32 (line editing?).

In Tux Paint we want to be able to do case-conversion and others might need ordering. The current (strictly broken) ucs-16 scheme allows a stunt to be pulled where the same wide-character functions can be called on Windows/Linux/OSX in the application (I sneakily cast ucs-16 into wchar_t as they are received, and then back to ucs-16 (redundantly on Windows) when calling SDL_ttf).

Annoyingly, fixing the current scheme will reveal design defects in Windows (No! say it aint so! :-)

Perhaps SDL (>=1.3) could provide a very basic set of support functions and conversions (simple stubs on capable systems) and (yet another!) string type (ucs-32) to alleviate this?

Just some thoughts, I'm not a Unicode wizard.

On 2006-02-03 08:37:13 +0000, John Popplewell wrote:

Created attachment 67
First pass at making composite characters work

This seems to work, but not extensively tested.
Not sure about the multiple ODD_keymap[] entries set to SDL_COMPOSE (prevents sticking keys).
Tried with keyrepeat on and off - seemed OK.
Watch out for the evil 'SDL_dgaevents.c' declarations!
I think the minor refactoring of X11_TranslateKey() is an improvement.
I haven't tested the #ifdef X_HAVE_UTF8_STRING stuff - probably errors in there.

On 2006-02-04 03:37:05 +0000, Sam Lantinga wrote:

(In reply to comment # 6)

Created an attachment (id=67) [edit]
First pass at making composite characters work

I saw your patch after I had almost completed mine... I win!
Can you try out the current CVS and see how it works for you?
Thanks for the bit about mapping the dead keys to SDLK_COMPOSE