We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 523

Summary: X11 keyboard handling for 1.3 - WIP & call for help
Product: SDL Reporter: Christian Walther <cwalther>
Component: eventsAssignee: Christian Walther <cwalther>
Status: RESOLVED FIXED QA Contact: Sam Lantinga <slouken>
Severity: enhancement    
Priority: P2    
Version: HG 2.0   
Hardware: All   
OS: All   
Attachments: Patch against http://svn.libsdl.org/trunk/SDL r3452
New patch against http://svn.libsdl.org/trunk/SDL r3501

Description Christian Walther 2007-12-20 12:35:24 UTC 
Some time ago, I set out to implement the new SDL 1.3 keyboard API for X11, based on my previous work in this area (Bug 319). At some point I got stuck on a problem that I could not find a satisfactory solution for. In the hope that someone else can help me out, here is a patch that contains what I arrived at.

In its present state, the code mostly works with the western keyboard layouts I have tested, but most likely fails miserably as soon as input methods are involved. Specifically, GetLayoutKey() turned out to be less straightforward to implement than I had hoped. What I need there, for character keys, is basically a static mapping from X11 KeySyms to one-character (unicode) key names (static because KeySyms correspond 1:1 to our concept of a "layout key code"). But I have been unable to find an elegant way of doing this. Xutf8LookupString(), which I'm using now, is inappropriate because it takes into account dead keys and input methods (and probably messes up the XIC that I'm currently borrowing from the first window). XLookupString() works, but only for Latin-1 characters. XKeysymToString() produces ASCII strings, not single characters. The only correct way I can imagine is to have a giant look-up table containing all thousands of KeySyms. I'm hoping some X11 expert (which I am not) has a better idea here... or perhaps the API we designed is just unimplementable and needs to be revised.

Other unfinished or questionable points:
- I haven't implemented generation of text events because I don't understand what's going on in the commented-out SDL 1.2 code with saved_keysym.
- I've resurrected the X11_KeyRepeat() function from SDL 1.2. Was there a particular reason why it was removed?
- Due to what I wrote in <http://article.gmane.org/gmane.comp.lib.sdl/34610>, I can't use SDL_iconv() for UTF-8/SDLKey conversions (it's probably a bit heavy-weight anyway). Instead I've taken fragments of the integrated SDL_iconv() implementation and put them into the functions encodeUtf8() in SDL_keyboard.c and decodeUtf8() in SDL_x11keyboard.c. This should probably be centralized somewhere.
Comment 1 Christian Walther 2007-12-20 12:36:31 UTC 
Created attachment 238 [details]
Patch against http://svn.libsdl.org/trunk/SDL r3452
Comment 2 Sam Lantinga 2007-12-29 14:04:00 UTC 
Christian, can you get in touch with Bob to help address your main point with input methods?

Key repeat is now defined as handled by the OS, since the SDL implementation worked by saving the last key down event and re-sending it at specified intervals, which doesn't make sense in SDL 1.3.
Comment 3 Christian Walther 2007-12-30 08:23:34 UTC 
Created attachment 242 [details]
New patch against http://svn.libsdl.org/trunk/SDL r3501
Comment 4 Christian Walther 2007-12-30 08:27:26 UTC 
Thanks to Bob Pendleton who pointed me to "imKStoUCS", I found out that:

- we're not the first ones to have this problem, and
- everyone has solved it using the "giant look-up table" approach.

I have found two basic varieties of such tables out in the wild:

- http://webcvs.freedesktop.org/xorg/xc/lib/X11/imKStoUCS.c?view=markup
  (libX11.6.dylib here on Mac OS X 10.5.1 even exports the KeySymToUcs4() function defined there, but as it doesn't seem to be part of any official API, I don't think we can rely on it.)
- http://www.cl.cam.ac.uk/~mgk25/ucs/keysym2ucs.c

They have 819 entries in common, 377 are in the former only, 46 in the latter only, and 84 conflict. Unable to judge the quality of the mappings (particularly with respect to the conflicting entries, most of which appear in the Korean range (0x0Exx)), I decided to go with the numbers and choose the former. In the hope that this helps avoid a mess, I integrated that file as-is, except for the addition of the license header (which lives in a central file in the X.org source) and modification of #include statements.

So, here is an updated patch that should now work properly in all cases (after some minor tweaking, I expect). Bob, what do you think about this?

Some of the caveats mentioned above still stand:
- No text inputs are generated yet.
- encodeUtf8() still needs to be reconciled with SDL_iconv() (but decodeUtf8() is gone).
Comment 5 Sam Lantinga 2009-02-16 20:53:07 UTC 
I think this is resolved in the current SDL 1.3 code.  Christian, can you confirm?
Comment 6 Christian Walther 2009-02-22 12:17:33 UTC 
I haven't followed SDL development for a while (as you may have noticed), but from a cursory glance at the current source, it looks like it. imKStoUCS.c, which was the solution for the original problem, is still used.