This bug report was migrated from our old Bugzilla tracker.

Reported in version: 2.0.9
Reported for operating system, platform: Android (All), All

Comments on the original bug report:

On 2019-03-29 15:15:21 +0000, Ellie wrote:

SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH on Android set to 1 leads to mouse events only being reported for TRUE mouses, and SDL_TOUCH_MOUSEID is NEVER used.

I propose that this is bad, and should be changed. Here is why:

1. The current modes of operations available render SDL_TOUCH_MOUSEID USELESS:

With SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=0 everything is SDL_TOUCH_MOUSEID (even a hw mouse), and with SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=1 NOTHING ever is SDL_TOUCH_MOUSEID. There is literally no point to the entire constant on Android, so why does it even exist?

2. The current modes of operation are INCONSISTENT:

On desktop, I believe the current behavior is exactly what I am suggesting, which is SDL_TOUCH_MOUSEID events for touch, and which=0,1,... for regular mouses. This mode is not even not the default for Android, but apparently not even AVAILABLE. That is not good

3. Working around this limitation is NOT TRIVIAL:

My current code assumes the desktop operations mode, and now my only choices are either not being able to deal properly with hw mouses at all (=no scrolling wheel, no proper hover effects of any kind, ...) or supporting hw mouses ONLY (why would I want that?!) or completely rewrite the code to not rely on SDL_TOUCH_MOUSEID. These are somewhat nasty choices

4. This mechanism would be the ONLY CROSS-PLATFORM approach for single finger touch with no caveats

SDL_TOUCH_MOUSEID is not really usable on Android due to this issue here, because it makes proper hw mouse support impossible. (Unless you accept that trade-off, of course) However, SDL_TouchFingerEvent is almost useless for multi-windowed applications on desktop because due to the oversight of the missing windowID, meaning it can't actually be mapped to any useful coordinates like buttons, ...: https://bugzilla.libsdl.org/show_bug.cgi?id=4331 So do I need to branch out for the different platforms now just to get touch to work?

5. It causes almost ZERO TROUBLE if people don't want this new behavior:

To get back to the old behavior, you just need to drop events with which=SDL_TOUCH_MOUSEID and done. Magically making these missing events reappear is a lot more work, especially since again, the true touch events are somewhat unusably broken for desktop for multi window apps due to https://bugzilla.libsdl.org/show_bug.cgi?id=4331

Minor nitpick: this was also simply UNEXPECTED for me based on the hint name:

The hint is named SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH, which to me just says I can properly tell apart touch & mouse. My first thought was I'd get the desktop behavior, not that this would actually prevent SDL_TOUCH_MOUSEID events entirely. (since these can still be properly told apart/separated, can't they?) After all, it's not named SDL_HINT_ANDROID_ONLY_DEDICATED_TOUCH_EVENTS or something, which would have made this more clear.

Note on backwards compatibility / practicality of this change:

If you're too worried about backwards compatibility, I suggest you introduce SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=2 with this new sane behavior, which again would be SDL_TOUCH_MOUSEID mouse events AND SDL_FingerTouchEvents for touch, and NOT SDL_TOUCH_MOUSEID mouse events for hw mouse with actual mouse motion if not pressed down etc properly reported - same as on desktop

On 2019-04-02 07:59:14 +0000, Sylvain wrote:

https://hg.libsdl.org/SDL/file/d5b4d374a312/include/SDL_hints.h#l860

With SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH on Android set to 1:
hw mouse events only come as SDL_MouseEvents
touch events only come as SDL_TouchEvents
They are separated (and there is no SDL_TOUCH_MOUSEID involved)

With SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH on Android set to 0:
touch events are duplicated to come as SDL_MouseEvents (and they use SDL_TOUCH_MOUSEID)
hw mouse events are duplicated to come as SDL_TouchEvents (not sure what is the FingerId ?)

(I see a "< 2.0.3", maybe that has changed a little bit)

With SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=0 everything is SDL_TOUCH_MOUSEID (even a hw mouse),

Are you sure ? that sounds to be a bug.
Whatever the value of SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH, hw mouse should have their own id (field .which) and always different from SDL_TOUCH_MOUSEID.

can you add some log and see how your mouse is recognized ?

maybe you can log in isDeviceSDLJoystick(), file SDLControllerManager.java ?
int sources = device.getSources();

On 2019-04-02 09:16:16 +0000, Ellie wrote:

Are you sure ? that sounds to be a bug. Whatever the value of SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH, hw mouse should have their own id (field .which) and always different from SDL_TOUCH_MOUSEID.

Yes, and this behavior is even documented on the wiki for this hint! The idea appears to be that the default SDL android app will only be aware of touch and no hw mouses, so all mouses are sent as fingers with the following results:

• no motion events at all unless button is pressed down (since a finger can't do that)
• no mouse wheel events at all
• all events are always SDL_TOUCDH_MOUSEID

and these three things essentially make proper hw mouse support in this mode completely impossible, which is precisely why I filed this ticket

On 2019-04-02 09:45:26 +0000, Ellie wrote:

By the way if it behaves like you say there is still no mode that matches desktop, which as far as I know is

• touch is duplicated as SDL_TOUCH_MOUSEID mouse events and touch events
• mouse is just sent as regular mouse events

Is there a reason this behavior is not available? Because that is the main reason why cross-platform touch handling with SDL2 really is kind of annoying to work with. All the behavior differences just mean more testing and branching for me with no real gain

On 2019-04-02 10:17:01 +0000, Ellie wrote:

can you add some log and see how your mouse is recognized ?

I can, but is this really relevant if it is properly reported as mouse during SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=1? I really got the feeling this is intentional and it's recognized as a mouse, and SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=0 just forces it to be reported as fake mouse finger anyway (intentionally, wiki says "mouse events will be handled as touch events and touch will raise fake mouse events" which sounds a lot like "mouse events will be SDL_TOUCH_MOUSEID & touch, and touch will be SDL_TOUCH_MOUSEID" which would match what I'm seeing)

On 2019-04-02 11:21:33 +0000, Sylvain wrote:

By the way if it behaves like you say there is still no mode that matches desktop,
which as far as I know is

• touch is duplicated as SDL_TOUCH_MOUSEID mouse events and touch events
• mouse is just sent as regular mouse events

Is there a reason this behavior is not available? Because that is the main reason why
cross-platform touch handling with SDL2 really is kind of annoying to work with. All
the behavior differences just mean more testing and branching for me with no real gain

On linux/x11, I don't have this behavior.
touch only produces touch events, and they don't produce mouse events.

-> maybe this another issue. Which platform are you using ?
(maybe open a different ticket if this ends up to be an issue).

Which SEPARATE_MOUSE_AND_TOUCH=0 on Android, I confirm the behavior:

1/My mouse HW has always which == -1
2/And no whell event!

With SEPARATE_MOUSE_AND_TOUCH=1
mouse HW has which = 0, and I do have wheel (eg SCROLL) events.

1/ has been always there (even if the code somehow changed) ... from the initial commit :

https://hg.libsdl.org/SDL/annotate/fff5af5de6dd/android-project/src/org/libsdl/app/SDLActivity.java#l1134

https://hg.libsdl.org/SDL/rev/fff5af5de6dd

=> Maybe this is wrong.
with this check for SEPARATE_MOUSE_AND_TOUCH (or mSeparateMouseAndTouch in latest src code),
the mouse get anonymized as touchid.

2/ I think this is added this is added in GameController mSeparateMouseAndTouch:
https://hg.libsdl.org/SDL/rev/10e7bb76a62a

On 2019-04-02 11:24:10 +0000, Sylvain wrote:

What I understand from SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=0
is that:
mouse hw should be duplicated as TouchEvents
touch hw should be duplicated as MouseEvents with SDL_TOUCH_MOUSEID

I don't think mouse hw MouseEvent should also have SDL_TOUCH_MOUSEID.

On 2019-04-02 11:29:12 +0000, Ellie wrote:

I don't think mouse hw MouseEvent should also have SDL_TOUCH_MOUSEID.

It would certainly help me A LOT if SDL2 didn't do that, ever.
I just want proper mouse events including 1. proper id, 2. proper mouse motion even when button is not pressed down and 3. proper mouse wheel support in ADDITION to fake touch.

So changing this would certainly make things a lot easier for me

On 2019-04-02 13:01:11 +0000, Sylvain wrote:

the fact I don't observe the same behavior on desktop is because it's not implement for x11, nor cocoa. but it's the case for waylend, window, etc.
created bug 4576.

On 2019-04-02 15:33:04 +0000, Sylvain wrote:

with bug 4576 / https://hg.libsdl.org/SDL/rev/0dd4bae437a5
it should match a little bit better other os behaviour.

On 2019-04-02 15:36:33 +0000, Sylvain wrote:

https://hg.libsdl.org/SDL/rev/9fdfbf246f09

https://hg.libsdl.org/SDL/rev/994f7540d7f3

On 2019-04-02 15:41:58 +0000, Ellie wrote:

Ok, very neat! Unification is super nice, thanks so much!

Will that make touch always issue SDL_TOUCH_MOUSEID fake events even with SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=1? Because that would totally solve my problem!

(Alternatively, SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=0 should not have the mouse emulate a finger for touch events and also emulate a finger via mouse events/SDL_TOUCH_MOUSEID as it does now, but emulate a finger for touch events but report proper not-SDL_TOUCH_MOUSEID events for the mouse events. Any of the two solutions would work for me really)

On 2019-04-02 16:50:01 +0000, Sylvain wrote:

Yes, it always duplicates the first finger touch events as MouseEvents with which==SDL_TOUCH_MOUSEID.
It makes various targets more similar.

Not sure how we should deal with the other mSeparateMouseAndTouch issue of this ticket.
Maybe we should just remove SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH, and set it to 1.

It still remains, with SEPARATE_MOUSE_AND_TOUCH=0 on Android, the behavior:

1/My mouse HW has always which == -1
2/And no whell event!

On 2019-04-02 16:59:30 +0000, Ellie wrote:

Maybe we should just remove SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH, and set it to 1.

For what it's worth I'll just be setting it to 1 and use this new version and I'll be fine :-) I have a suggestion though for the general API:

SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=0 currently does two things I think:

1. the mouse events are now duplicated as actual touch events

2. all mouse events are turned into weird SDL_TOUCH_MOUSEID fake-touch mouse events with no proper mouse wheel support etc

So my suggestion would be: remove 2. entirely, but keep 1. and have SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=0 toggle it (with =1 disabling it as well). That seems like a useful remaining functionality of the toggle to me

On 2019-04-04 00:54:35 +0000, Ellie wrote:

Ok, I just tried this with my real world app using https://hg.libsdl.org/SDL/rev/b5ff77fd39f2 / https://hg.libsdl.org/SDL/archive/b5ff77fd39f2.zip & SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH=1.

What I am seeing is that I'm getting SDL_MOUSEMOTION with which=SDL_TOUCH_MOUSIED, but I appear to be NOT getting any SDL_MOUSEBUTTONUP / SDL_MOUSEBUTTONDOWN for any touch taps, so my code just never recognizes any actual clicks and this is still somewhat ineffective in actually allowing me to use the app.

So it seems to almost work, but not quite fully yet

On 2019-04-04 06:54:02 +0000, Sylvain wrote:

I tried and this is working, so far I tried: on windows / android / linux (x11,evdev).
Maybe you didn't pull the latest fix:
https://hg.libsdl.org/SDL/rev/a34ab2e5ddcf

Try to put some debug log when you receive events.

( Also there is a hint SDL_HINT_TOUCH_MOUSE_EVENTS that you shouldn't set. )

On 2019-04-04 10:43:54 +0000, Ellie wrote:

I indeed haven't pulled the latest fix (I linked the commit I used which is one before) I'll retry and see if that fixes anything. The coordinates however seemed to be ok if that is related at all, I just really didn't get any mouse up/downs for any finger taps. SDL_HINT_TOUCH_MOUSE_EVENTS I haven't ever changed, I just grepped my entire source code just to be sure

On 2019-04-04 12:41:09 +0000, Ellie wrote:

Nice, that latest fix commit actually did it! Works now! Thanks so much <3 <3

On 2019-04-04 13:24:08 +0000, Sylvain wrote:

I've actually changed the code a little bit.

• some cleanup
• and mainly because we need to track both FingerId and TrackId to record the first finger that will produce the mouse-event.
  (Maybe we should also filter by SDL_TouchDeviceType direct/indirect/relative)

Please pull the latest change again.

On 2019-04-04 13:28:48 +0000, Ellie wrote:

Oh that's so cool, I think you may have accidentally fixed https://bugzilla.libsdl.org/show_bug.cgi?id=4509 as well! :-D :-D

(I'm still on a34ab2e5ddcf but even there, the single finger touch emulation now always follows the first finger, that's so cool!)

I'll try the newest 712d84dd14c9 right now and report back, really cool to see these improvements!

On 2019-04-04 13:57:08 +0000, Sylvain wrote:

Ok great, let me know and mark bug 4509 resolved if needed !

On 2019-04-04 15:16:54 +0000, Ellie wrote:

Ok, I tested 712d84dd14c9 now with the following awesome findings:

1. SDL_TOUCH_MOUSEID works with down/up and motion as expected

2. When putting on more than one finger the SDL_TOUCH_MOUSEID always stay with the first finger (which means https://bugzilla.libsdl.org/show_bug.cgi?id=4509 is fixed!)

3. Unlike the one revision before, when removing all fingers except one again, the SDL_TOUCH_MOUSEID events continue without issues (good!)

So it seems to work perfectly except for one corner case, but I'm not sure what the ideal behavior should even be in that case:

• If you put a finger onto the screen, then a 2nd one, then release the 1st one, SDL_TOUCH_MOUSEID will just register a mouse-up (of the 1st finger) and then ignore everything until all fingers are released. It would seem more ideal to me if in case the first finger is lifted, it would pick any other finger to follow from that point, but that really is just a minor nitpick

In overall, thanks a lot for this massive improvement of SDL_TOUCH_MOUSEID behavior!

On 2019-04-04 15:19:21 +0000, Ellie wrote:

I just tested further, and when I put the 1st finger back on it follows that one again, which actually is kind of nice. And I guess choosing an alternative "1st finger" would lead to jumpiness again, so maybe the current behavior is actually the most reasonable one. I'm really undecided but just wanted to let you know that this was one minor thing that may or may not be considered suboptimal by some people

On 2019-04-04 15:24:49 +0000, Sylvain wrote:

• If you put a finger onto the screen, then a 2nd one, then release the 1st
  one, SDL_TOUCH_MOUSEID will just register a mouse-up (of the 1st finger) and
  then ignore everything until all fingers are released. It would seem more ideal to me if in case the
  first finger is lifted, it would pick any other finger to follow from that point, but that really is just a minor nitpick

Yep, I understand but then this is ambiguous because there is not new finger_down. Even if the 2nd finger could become the tracked one.
if you press a 3rd finger (will not releasing the 2nd) you should get the mouse again.
It seems while responding, you got it!

.. There was some new commit:
removing SDL_HINT_ANDROID_SEPARATE_MOUSE_AND_TOUCH
https://hg.libsdl.org/SDL/rev/d581caf613cd

and adding SDL_HINT_MOUSE_TOUCH_EVENTS (reverse of SDL_HINT_TOUCH_MOUSE_EVENTS)
https://hg.libsdl.org/SDL/rev/cc45bcb16ef2

On 2019-04-04 15:32:52 +0000, Ellie wrote:

Neat, that's cool! I really like this much more coherent cross-platform unification, that's all SUPER AWESOME!! :-)

Just to clarify that tracking remark though, bug # 4509 was about:

1. putting on finger one,
2. adding a finger two,
3. keeping BOTH ON and moving them differently
   -> SDL_TOUCH_MOUSEID used to jump to & track finger two while finger one was also still on which was rather confusing & jumpy in practice, which is now FIXED (nice!)

However, there is also the case of:

1. putting on finger one,
2. adding a finger two,
3. releasing finger one,
4. now moving single finger two
   -> currently the app feels a bit stuck in that state (no SDL_TOUCH_MOUSEID happens). That made me wonder if it's good SDL2 refuses to change the "finger focus" even there it does now, or if maybe it would be more intuitive here to change to follow the remaining finger? That is all I was saying

However, I am very happy with this behavior as far as my app is concerned, and I suppose people who want this behavior differently can use the touch events instead. It was just something that I found interesting to bring up

On 2019-04-04 15:43:25 +0000, Sylvain wrote:

1. putting on finger one,
2. adding a finger two,
3. releasing finger one,
4. now moving single finger two
   -> currently the app feels a bit stuck in that state (no SDL_TOUCH_MOUSEID happens).
   That made me wonder if it's good SDL2 refuses to change the "finger focus" even there it does now,
   or if maybe it would be more intuitive here to change to follow the remaining finger? That is all I was saying

Ok this is what I understood, and this what is expected now. But that might be changed in the future ...

On 2019-04-04 15:52:15 +0000, Ellie wrote:

Thanks again for all the nice revamping! I think it makes sense if I close this as fixed right now?

I also think the hints are so much better now, makes much more sense to control mouse fakes touch, and touch fakes mouse in this clean & symmetric manner without android-only special stuff!

I think with a default SDL_MOUSE_TOUCH_EVENTS=0 the android default will notably change though (for people who used touch events only) so that might be worth putting in big bold letters in the release notes just to be safe

On 2019-04-04 18:12:28 +0000, Sylvain wrote:

Ok, thanks ! I've update WhatsNew.txt
https://hg.libsdl.org/SDL/rev/1e0b433edcbb

On 2019-04-04 18:18:41 +0000, Ellie wrote:

Might be worth adding an additional line for android similar to:

"* Changed old default behavior from hardware mouse triggering touch events to now not doing so (SDL_MOUSE_TOUCH_EVENTS=0) to match the default behavior of other platforms"

Of course that's kind of obvious when looking at the hints in details, but people might miss it

On 2019-04-04 18:25:36 +0000, Sylvain wrote:

yep done! https://hg.libsdl.org/SDL/rev/3125dc339d8d
thanks

On 2019-04-05 06:39:49 +0000, Sylvain wrote:

In the ends, the behaviour on Android should be kept:
Android: default SDL_HINT_MOUSE_TOUCH_EVENTS to 1 as previous behaviour
https://hg.libsdl.org/SDL/rev/69f24f4ef99e

On 2019-04-05 09:35:09 +0000, Ellie wrote:

I understand. it is indeed a little bit more foolproof as a default and after all it is now trivial to configure SDL2 to behave as on desktop if desired, thanks to your work :-)