We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 4576 - Duplicate touch event to mouse event
Summary: Duplicate touch event to mouse event
Status: RESOLVED FIXED
Alias: None
Product: SDL
Classification: Unclassified
Component: events (show other bugs)
Version: don't know
Hardware: All All
: P2 blocker
Assignee: Ryan C. Gordon
QA Contact: Sam Lantinga
URL:
Keywords: target-2.0.10
Depends on:
Blocks:
 
Reported: 2019-04-02 12:57 UTC by Sylvain
Modified: 2019-06-15 00:17 UTC (History)
4 users (show)

See Also:


Attachments
patch (2.08 KB, patch)
2019-04-02 12:59 UTC, Sylvain
Details | Diff
patch cocoa (4.20 KB, patch)
2019-06-11 11:22 UTC, Sylvain
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sylvain 2019-04-02 12:57:22 UTC
SDL duplicates first finger touch events to mouse events.

but it's missing for x11 and cocoa.
and it would be better to be done in generic layer so here's a patch to add it.
Comment 1 Sylvain 2019-04-02 12:59:50 UTC
Created attachment 3727 [details]
patch

Here's a patch.
(behaviour of https://hg.libsdl.org/SDL/rev/bc838d7a29dc)

It's also needed to remove each platform code that do the same thing
Comment 2 Sylvain 2019-04-02 13:07:36 UTC
Was an idea from Ryan https://hg.libsdl.org/SDL/rev/c09f06c4e8c8 (and easier than touching the x11 code !)
Comment 3 Ryan C. Gordon 2019-04-02 13:53:18 UTC
Win32 generates fake mouse events for touches at the lower level, which is likely why we ended up implementing it ourselves in several backends, to match that behavior.

(Grep for MOUSEEVENTF_FROMTOUCH to see where this happens.)

I'm 100% okay with moving this to a higher level in SDL, so long as we deal with Windows properly.

--ryan.
Comment 4 Sylvain 2019-04-02 14:43:18 UTC
Ok, I see that part of code ! I'll just prevent the duplication there !
So I'll start to merge this !
Comment 6 Sylvain 2019-04-02 15:31:02 UTC
linux/EVDEV:
https://hg.libsdl.org/SDL/rev/9fdfbf246f09

Wayland:
https://hg.libsdl.org/SDL/rev/108f8fab6175

Emscripten:
https://hg.libsdl.org/SDL/rev/798c467b33c9

WinRT:
https://hg.libsdl.org/SDL/rev/450b42f2f24c

IOS:
https://hg.libsdl.org/SDL/rev/aec854f934db

Android:
https://hg.libsdl.org/SDL/rev/0dd4bae437a5


This is clearly not well tested, but I plan to update my builds in the upcoming days/weeks, so I'll try this in a better way.

Android may appear a little different since it has the hint SEPARATE_MOUSE_AND_TOUCH, but now it looks more than other platform.

x11 and cocoa, will now generate the duplication event.

IOS: we may miss the TapCount ...
Comment 7 Sylvain 2019-04-02 16:16:14 UTC
About MOUSEEVENTF_FROMTOUCH:

I thought my Window10 was using the VisualC project but it uses the Visual-WinRT project. Even if it's about Win10 inside.

I am not sure how I can build and test this win32 / VisualC / SDL_windowsevents.c ?
Comment 8 Dominik Reichardt 2019-05-12 12:08:24 UTC
This broke touchevents in our iOS app completely. Well nearly completely. It shows there is a touch event but it no longer maps it to the screen coordinates or something similar.
I'm not good enough to embed hooks in our codes to see where it breaks. Just enough that I was able to bisect and find changeset 12676 to be the culprit.

https://github.com/litchie/exult-ios - our xcode project file is in the iOS subfolder and you need to replace the contents of the SDL2 folder with mercurial of SDL2. Due to another bug opened by me (bug #4603) you will need to edit SDL_render.c and place #if SDL_VIDEO_RENDER_METAL below the OGL_ES2)
Comment 9 Dominik Reichardt 2019-05-12 15:56:21 UTC
> https://github.com/litchie/exult-ios - our xcode project file is in the iOS
> subfolder and you need to replace the contents of the SDL2 folder with
> mercurial of SDL2. Due to another bug opened by me (bug #4603) you will need
> to edit SDL_render.c and place #if SDL_VIDEO_RENDER_METAL below the OGL_ES2)

Sorry, bug #4629
Comment 10 Dominik Reichardt 2019-05-12 18:28:16 UTC
and one more comment, after adding the hint SDL_HINT_TOUCH_MOUSE_EVENTS, it's clear that changeset 12682 broke iOS for me. After that no touch event is working anymore in our app.
Comment 11 Caleb Cornett 2019-05-16 17:38:02 UTC
@Dominik: I ran into that too. If I remember correctly, touch device 0 is now exclusively the mouse by default. (Bad default behavior imo...) Touch device 1 is the *actual* touch device. Try setting both SDL_HINT_MOUSE_TOUCH_EVENTS and SDL_HINT_TOUCH_MOUSE_EVENTS to "0" and see if it works then?
Comment 12 Ryan C. Gordon 2019-05-18 18:48:55 UTC
Tagging a bunch of bugs with "target-2.0.10" so we have a clear list of things to address before a 2.0.10 release.

Please note that "addressing" one of these bugs might mean deciding to defer on it until after 2.0.10, or resolving it as WONTFIX, etc. This is just here to tell us we should look at it carefully, and soon.

If you have new information or feedback on this issue, this is a good time to add it to the conversation, as we're likely to be paying attention to this specific report in the next few days/weeks.

Thanks!

--ryan.
Comment 13 Ryan C. Gordon 2019-05-20 04:40:08 UTC
Fun problem introduced in https://hg.libsdl.org/SDL/rev/994f7540d7f3 ...

On macOS, the trackpad built into Macbooks will report touch events to SDL while also reporting mouse input, so using the primary "mouse" on these computers now makes you think you're holding the mouse button down as you're moving the cursor by touching the trackpad.  :/

I can look into the macOS part if you're busy, Sylvain. Let me know!

Thanks,
--ryan.
Comment 14 Sylvain 2019-05-20 07:15:01 UTC
Hi Ryan,
Please have a look, my macOS is a Mac Mini and I have not trackpad on it!
Thanks,
Sylvain
Comment 15 Dominik Reichardt 2019-05-21 06:44:38 UTC
Changeset 12744 fixed my initial iOS touch problem. I'm having problems with the onscreen keyboard now but need to investigate further whether it's us or SDL
Comment 16 Dominik Reichardt 2019-05-21 12:52:05 UTC
An UIAlertController Textfield does not get "focus" and the softkeyboard does not get triggered.
Not get focus means that you cannot put the "cursor int he text field, nr can you hit ok or dismiss. Every touch is registered in the gameview below the UIAlertController.
Comment 17 Alex Szpakowski 2019-05-26 21:27:54 UTC
Should this only apply to direct (touch-screen) touches? Indirect touch events include ones generated by trackpads.
Comment 18 Sam Lantinga 2019-06-08 17:35:40 UTC
It seems reasonable that trackpad touches which also generate mouse input should not be included in the code path that simulates mouse input from touch, but this case should be documented in the code so we don't scratch our heads about it later.

Ryan, can you try that and see if it fixes your issue?

@Dominik, can you report a separate bug for any issue you're still having, so we can fix it for 2.0.10?

Thanks!
Comment 19 Dominik Reichardt 2019-06-08 19:35:06 UTC
Thanks, bug report committed, bug #4659
Comment 20 Ryan C. Gordon 2019-06-09 02:56:35 UTC
Been going around and around with this one.

So the problem in Cocoa is (afaik)...

- There’s no distinction between touch events and trackpad events...at least, you can’t tell if a mouse event came from a trackpad vs some other touch device.

- You can’t ignore mouse events outright because you’ll need them when a real mouse is plugged in.

- You can’t ignore touch events, because multitouch is useful and not reproducible in mouse events.

- If you just synthesize mouse events from touch inside SDL, I bet people will start complaining that trackpad input feels different and/or wrong, because there is likely a lot of subtle adjustment macOS’s driver does when deciding how to generate mouse input from touches.

- we can’t leave it alone because touching the trackpad to move the “mouse” now looks like a mouse button is being dragged.

I’m not actually sure what the best plan is here. It might be to ignore touch events in Cocoa outright.  :/  Does anything use touch on a Mac? This was more for mobile devices, right?

It’s possible that multitouch on a trackpad could be useful, but this system obviously isn’t working as-is. I also suspect no one has tried this on Windows with hardware that would act the same, too, which suggests we should back out the touch/mouse unification work for now.

Opinions, anyone?
Comment 21 Sylvain 2019-06-09 14:42:51 UTC
I am confused a little bit ..

I think the unification didn't modify anything for macosx.
It's failing by lack of providing the feature.

Maybe the synthetic hints should be restricted to mobile only ?

https://hg.libsdl.org/SDL/annotate/8a4b1beb4f6e/src/events/SDL_mouse.c#l98
SDL_TouchMouseEventsChanged() and SDL_MouseTouchEventsChanged()



or not sure, but I found this link:

https://stackoverflow.com/questions/29459735/how-to-find-out-that-the-nsevent-click-event-is-from-a-mouseclick-or-trackpad-cl

"You should be able to check "event.subtype" to see from which device it came.

If it's a mouse, the "subtype" value should be NSMouseEventSubtype. If it's not a mouse, the value will be something else (e.g.: NSApplicationActivatedEventType or NSTouchEventSubtype)."
Comment 22 Ryan C. Gordon 2019-06-09 23:30:36 UTC
(In reply to Sylvain from comment #21)
> I am confused a little bit ..
> 

You're confused because I didn't read closely enough before opening my mouth.  :)

You higher-level code is robust and handles this case already!

> https://stackoverflow.com/questions/29459735/how-to-find-out-that-the-
> nsevent-click-event-is-from-a-mouseclick-or-trackpad-cl
> 
> "You should be able to check "event.subtype" to see from which device it
> came.

There were other docs from Apple that says this doesn't work, but I just tried it and it obviously does.

That being said: I bet there are multitouch trackpads on Windows that do the same thing, and right now we throw away FROMTOUCH events. I'll fix that up if you make sure the Mac patch matches your intentions.

That Mac patch is here: https://hg.libsdl.org/SDL/rev/4f015bc7cbb1

--ryan.
Comment 23 Alex Szpakowski 2019-06-09 23:44:48 UTC
All these changes (the latest patch included) relates a lot to my comments in this old issue: https://bugzilla.libsdl.org/show_bug.cgi?id=3542#c1

Since I wrote those comments we've added APIs to query whether a touch device generates direct or indirect touches, but even so I believe it's worth considering the pros and cons of potentially breaking apps that rely on the old behaviour.

An example of a situation that will break is if the app code ignores all mouse events that have SDL_TOUCH_MOUSEID set (in order to handle touch events in a dedicated codepath), and thus will ignore trackpad-generated mouse events even though the mouse position/acceleration/etc is wildly different from the touch press events.
Comment 24 Ryan C. Gordon 2019-06-10 02:08:39 UTC
(In reply to Alex Szpakowski from comment #23)
> An example of a situation that will break is if the app code ignores all
> mouse events that have SDL_TOUCH_MOUSEID set (in order to handle touch
> events in a dedicated codepath), and thus will ignore trackpad-generated
> mouse events even though the mouse position/acceleration/etc is wildly
> different from the touch press events.

So what you're saying is we should have a way to say "this touch event came from a mousey device like a MacBook trackpad instead of a non-mousey thing like a touchscreen, so don't generate a fake mouse event from this, but also don't treat it as SDL_TOUCH_MOUSEID," right?

That wouldn't be a hard change over what's already there. I will note that we already have SDL_TOUCH_MOUSEID and SDL_MOUSE_TOUCHID and I can't correctly tell them apart, so can we add a flag or something instead?  :)

Also: do drawing tablets count as mousey things?

--ryan.
Comment 25 Sylvain 2019-06-10 10:20:46 UTC
ok so if https://hg.libsdl.org/SDL/rev/4f015bc7cbb1 is working,

Then, I believe cocoa should not generate itself the synthesized events.
from this commit, when cocoa uses SDL_TOUCH_MOUSEID or SDL_MOUSE_TOUCHID, it should instead emit nothing.
so this is SDL layer that will genertes the synthesize, isn't it ?
(which can be disable/enable with the hints also)
Comment 26 Ryan C. Gordon 2019-06-10 16:46:22 UTC
(In reply to Sylvain from comment #25)
> ok so if https://hg.libsdl.org/SDL/rev/4f015bc7cbb1 is working,
> 
> Then, I believe cocoa should not generate itself the synthesized events.
> from this commit, when cocoa uses SDL_TOUCH_MOUSEID or SDL_MOUSE_TOUCHID, it
> should instead emit nothing.
> so this is SDL layer that will genertes the synthesize, isn't it ?
> (which can be disable/enable with the hints also)

Yeah, that's right. When I slide my finger across the trackpad, I get appropriate touch and mouse events from the OS to my app and no synthetic events from SDL, which is correct as far as I can tell.

I need to fix up the Windows code to do the same in case there are similar cases with multitouch trackpads there, and then all that's left is Alex's concern and we can finally close this bug.   :)

--ryan.
Comment 27 Ryan C. Gordon 2019-06-11 02:54:50 UTC
(In reply to Ryan C. Gordon from comment #26)
> I need to fix up the Windows code to do the same in case there are similar
> cases with multitouch trackpads there

Having spent some time with this, I think for the WM_TOUCH codepath, this is probably good enough as-is. Apparently starting with Windows 8, we should be using the better WM_POINTER* API, so for 2.0.11 I might rework the Windows bits for it.

But that's a different bug for a different release.

--ryan.
Comment 28 Sylvain 2019-06-11 11:22:17 UTC
Created attachment 3815 [details]
patch cocoa

I mean this modification for macOS : so that only mouse event produce SDL_MouseEvent, and only touch produce SDL_TouchEvent.
Then synthetics events are produce in the SDL_mouse.c and SDL_touch.c

Un-tested patch (maybe it dont event compile, but might also be ok :))
Comment 29 Sam Lantinga 2019-06-11 13:35:59 UTC
Sylvain, Ryan and I discussed it, and we think the OS generated synthetic events will give better "feel" than the SDL ones, so we want to prefer them if possible on platforms that support it.

Ryan, can you confirm that the behavior with the various hints is correct with your trackpad? If so, we can close this bug.
Comment 30 Sylvain 2019-06-11 14:21:40 UTC
Ok!

So, I think: the hint still needs to be checked to produce synthetic events or not. (eg mouse->touch_mouse_events and mouse->mouse_touch_events)

And also, we need to make sure that they are not generated twice: by platform layer and SDL layer.
Comment 31 Alex Szpakowski 2019-06-11 23:48:11 UTC
(In reply to Ryan C. Gordon from comment #24)
> So what you're saying is we should have a way to say "this touch event came
> from a mousey device like a MacBook trackpad instead of a non-mousey thing
> like a touchscreen, so don't generate a fake mouse event from this, but also
> don't treat it as SDL_TOUCH_MOUSEID," right?
> 
> That wouldn't be a hard change over what's already there. I will note that
> we already have SDL_TOUCH_MOUSEID and SDL_MOUSE_TOUCHID and I can't
> correctly tell them apart, so can we add a flag or something instead?  :)

That sounds reasonable.. I think? I haven't spent much time thinking about this so the amount of things to consider feels too big for my head. :)


> Also: do drawing tablets count as mousey things?

I believe they generate mouse events with a sub-type of NSEventSubtypeTabletPoint, but I think they generate low level tablet events as well (instead of low level touch events). I'm not 100% positive though since I've never tested with a tablet.
Comment 32 Sam Lantinga 2019-06-12 03:09:54 UTC
Is this bug fixed for what we want to ship in 2.0.10?
Comment 33 Ryan C. Gordon 2019-06-12 05:28:48 UTC
Just to make sure we're all on the same page:

- On the Mac, we touch the trackpad.
- The OS sends a touch event with a NSEventSubtypeMouseEvent subtype.
- SDL sends this to SDL_SendTouch() with a SDL_MOUSE_TOUCHID, to say "this is a touch event from a mousey device"
- SDL_SendTouch() sees the hint that says "we want SDL to generate fake mouse events from real touch events," but also sees that this touch was from a SDL_MOUSE_TOUCHID device, so it doesn't generate a mouse event.
- SDL_SendTouch() _does_ generate an SDL touch event for this and sends it to the app.
- NEXT, the OS sends a mouse event _for this same touch_ with a NSEventSubtypeTouch subtype.
- SDL sends this to SDL_SendMouseWhatever() with a SDL_TOUCH_MOUSEID, to say "this is a mouse event from a touch device"
- SDL_SendMouseWhatever() sees the hint that says "we want SDL to generate fake touch events from real mouse events," but also sees that this mouse input was from a SDL_TOUCH_MOUSEID device, so it doesn't generate a touch event.
- SDL_SendMouseWhatever() _does_ generate an SDL mouse event for this and sends it to the app.


Is this how it's intended to work? It's completely possible that I misunderstood the higher level, but this is why I have the Cocoa code send them with SDL_TOUCH_MOUSEID instead of dropping them: my assumption is Apple's drivers makes subtle efforts to make their own fake mouse events "feel" better rather than just mapping touches directly to mouse motion (to say nothing of mapping the scrollwheel to sliding two fingers across the trackpad, etc).


Now, on Windows: WM_TOUCH works in different ways, and is apparently obsoleted by WM_POINTER in Windows 8 and later, so it doesn't do all this tapdancing but instead just drops the OS level fake mouse events. In the presence of a Macbook-style trackpad that looks like a multitouch device, instead of a trackpad that looks like a hardware mouse to the OS, I assume any mouse input the app sees will be fake events generated by SDL from WM_TOUCH messages, but I'm okay with that until we can get WM_POINTER support in place for 2.0.11 or whatever. I predict this scenario is somewhere between uncommon and doesn't-actually-work-like-that-anyhow.


On iOS and Android: touch is touch, mice aren't (generally) present, and a phone's screen doesn't simulate mouse input at the OS level.

Anyhow, that's my thinking, please correct me if I got any of this wrong, so we can resolve any confusion.

Thanks!

--ryan.
Comment 34 Sylvain 2019-06-12 08:13:47 UTC
I think there is a little confusion!

SDL_MOUSE_TOUCHID (-1) and SDL_TOUCH_MOUSEID (-1) are meant to be used only and only for synthetic events. 
Eg, *Touch* Event that would be generated from a *Mouse* Device. or *Mouse* events from a *Touch* Device.

For genuine events (Mouse events from a Mouse Device), it has to use the real MouseId.
And for Touch events from a Touch Device, it has to use the real TouchId.


On Android for instance:
- you touch the screen.
- OS call nativeSendTouch, 
- it call SDL_SendTouch()
  - if the mouse hint is set, it pushes a SDL Mouse synthetic event.
  - always, it pushes a genuine SDL Touch Event.


On cocoa, 
with the platform layer sending itself the synthetics events.
the issue I think:

- when we touch the trackpad once, we should *not* send *both* a SDL_MOUSE_TOUCHID and a SDL_TOUCH_MOUSEID.
one has to be genuine (real Id), the other one has to be the synthetic id and is generated upon the hint.

So for cocoa, (with the platform layer sending itself the synthetics events), 
the generic layer blocks for synthetics have to be compiled out by an ifdef:
https://hg.libsdl.org/SDL/file/8a0e446a4cf9/src/events/SDL_mouse.c#l331
https://hg.libsdl.org/SDL/file/8a0e446a4cf9/src/events/SDL_mouse.c#l487
https://hg.libsdl.org/SDL/file/8a0e446a4cf9/src/events/SDL_touch.c#l251
https://hg.libsdl.org/SDL/file/8a0e446a4cf9/src/events/SDL_touch.c#l352


NB:
There was some confusion with the lines in those blocks (4 times):
  "if (id != SDL_MOUSE_TOUCHID) {"
https://hg.libsdl.org/SDL/file/8a0e446a4cf9/src/events/SDL_touch.c#l253

the purpose is for SDL not to end in an infinite generation loop of synthetics events.
Not to genereate synthetics mouse events from synthetics touch events, and 
Not to genereate synthetics touch events from synthetics mouse events, 
otherwise, we would have something like: 
SendMouseEvent(), so send SyntheticTouchEvent, so send SyntheticMouseEvent, so send SyntheticTouchEvent, etc..


NB2:

"Apple's drivers makes subtle efforts to make their own fake mouse events "feel" better rather than just mapping touches directly to mouse motion",
maybe this could be checked by counting the events ? and/or comparing the positions also ? 

if this is true, we should have more synthetic events in the end that real events.
(or same number of events, but smoother).



** 
I try quote the previous steps.
You said "mousey" device for the trackpad, so this is like a mouse I guess.
So we expects SDL_MouseEvent from trackpad and synthetic SDL_TouchEvent from trackpad
**


- On the Mac, we touch the trackpad.
[SB] ok
- The OS sends a touch event with a NSEventSubtypeMouseEvent subtype.
[SB] ok, we foresee a synthetic event.
- SDL sends this to SDL_SendTouch() with a SDL_MOUSE_TOUCHID, to say "this is a touch event from a mousey device"
[SB] ok, use the id for synthetic event.
[SB] but it should be sent only if the hint is set.
[SB] /* SDL_HINT_MOUSE_TOUCH_EVENTS: controlling whether mouse events should generate synthetic touch events */
[SB] if (mouse->mouse_touch_events) {  
- SDL_SendTouch() sees the hint that says "we want SDL to generate fake mouse events from real touch events," but also sees that this touch was from a SDL_MOUSE_TOUCHID device, so it doesn't generate a mouse event.
[SB] this should be compiled out !
- SDL_SendTouch() _does_ generate an SDL touch event for this and sends it to the app.
[SB] yes, the synthetic touch event is sent
- NEXT, the OS sends a mouse event _for this same touch_ with a NSEventSubtypeTouch subtype.
[SB] yes, but this should be a mouse event with NSEventSubtypeMouse subtype ???
- SDL sends this to SDL_SendMouseWhatever() with a SDL_TOUCH_MOUSEID, to say "this is a mouse event from a touch device"
[SB] it should use the real mouseID. and this is a mouse event from a mouse device !
- SDL_SendMouseWhatever() sees the hint that says "we want SDL to generate fake touch events from real mouse events," but also sees that this mouse input was from a SDL_TOUCH_MOUSEID device, so it doesn't generate a touch event.
[SB] this should be compiled out
- SDL_SendMouseWhatever() _does_ generate an SDL mouse event for this and sends it to the app.
[SB] yes, the genuine mouse event is sent



NB3:
just wondering ...
if we move a real mouse device, cocoa is also sending both mouse and touch events ?
(so that still can make synthetics touch event, real mouse)
Comment 35 Ryan C. Gordon 2019-06-13 05:59:31 UTC
(In reply to Sylvain from comment #34)
> just wondering ...
> if we move a real mouse device, cocoa is also sending both mouse and touch
> events ?
> (so that still can make synthetics touch event, real mouse)

At the OS level, it doesn't send a touch event for a real USB mouse; it's sending them for the trackpad because it _is_ a multitouch device, but since almost every app on the planet needs this device to look like a mouse, and because Apple can dedicate engineering to getting the "feel" of mouse input right on the trackpad, the OS synthesizes mouse events for it too.

It isn't universal compatibility thing: iPhone touch screens don't generate mouse events and a USB mouse plugged into a Mac won't generate touch events. But in the USB case: if you really want fake touch events from for some reason, turn on the hint.   :)

Anyhow, I _think_ I'm understanding you now. Can you see if this resolves the concerns? Tested here and I get touch and mouse events as appropriate. As an added bonus: it resolves Alex's concern too.

So if you like this: we can mark this bug resolved.  :)

https://hg.libsdl.org/SDL/rev/d7d64c1a3969

--ryan.
Comment 36 Sylvain 2019-06-13 07:21:55 UTC
I think we are not fully sync'ed but this is getting also clearer for me :)

About the USB mouse.
So the USB mouse only generates Mouse NSEvent, macOSX doesn't generate any synthetic event. 
So this is the SDL generic layer that can provide them with the hint enabled. (as you said).

So, the macOS port is using half of the generic synthetic feature.
- to generate synthetic TouchEvent from MouseEvent, this relies on the generic layer. (USB Mouse)
- to generate synthetic MouseEvent from TouchEvent, we use the ones that comes from the OS. (Trackpad).


From https://hg.libsdl.org/SDL/rev/d7d64c1a3969
There are no more SDL_TOUCH_MOUSEID generated from the SDL cocoa layer.
Since we use the fact that "macOS synthesizes its own events for this", at some point you have to use the virtual ids for the synthetized events.
- when a synthesized event one is detected, it has to be flagged either SDL_TOUCH_MOUSEID
- when it's a genuine one, it has to be flagged with the real mouse->mouseId or touchId.

Since we used half of the generic synthetic feature.
in the cocoa layer, you have to flag sometimes the MouseEvent with SDL_TOUCH_MOUSEID.


( Maybe the previous revisions has part of what it should be. but in the previous comment "Just to make sure we're all on the same page:" you were flagging at the same times both events with SDL_MOUSE_TOUCHID and SDL_TOUCH_MOUSEID. Which disturbs me.)



Then, the synthesized events should be generated only if the hint is set. Which means, that for cocoa, when the hint is not set, we should in fact drop (or not send) the synthesized events.

here you disabled the SDL generic part for generating the synthesize mouse events.
https://hg.libsdl.org/SDL/rev/d7d64c1a3969#l1.7
but the "mouse->touch_mouse_events" variable is useful to know when the hint is set or not, and so to drop it or not.

I think this is more convenient to revert this ifdef and use "mouse->touch_mouse_events" in the cocoa code.
And to disable the SDL generic part there:
https://hg.libsdl.org/SDL/file/8a0e446a4cf9/src/events/SDL_touch.c#l251
https://hg.libsdl.org/SDL/file/8a0e446a4cf9/src/events/SDL_touch.c#l352

---
I think in cocoamouse.m

there should something like:

if ([event subtype] == NSEventSubtypeTouch) {  // this is a synthetic from the OS
  if (mouse->touch_mouse_events) { // Hint is set
    SDL_SendMouseMotion(mouse->focus, SDL_TOUCH_MOUSEID, 1, (int)deltaX, (int)deltaY);
  }
} else {
  // genuine event
  SDL_SendMouseMotion(mouse->focus, mouse->mouseID, 1, (int)deltaX, (int)deltaY);
}
Comment 37 Sam Lantinga 2019-06-14 00:14:14 UTC
Sylvain's suggestion seems like a good one, where we revert the ifdef (so the hint default is the same on all platforms) and then check the hints in the cocoa code before dispatching the events.
Comment 38 Ryan C. Gordon 2019-06-14 01:35:13 UTC
Another shot at this is here: https://hg.libsdl.org/SDL/rev/ac0c65535817

Note that I had to add this, otherwise there's no relative mouse motion when you aren't holding down the trackpad "button" ...

https://hg.libsdl.org/SDL/file/ac0c65535817/src/events/SDL_mouse.c#l386

We are also back to Alex's concern that all mouse input is now going to report itself as SDL_TOUCH_MOUSEID in the most common macOS case.   *shrug*

If anyone wants any other changes, let me know.

--ryan.
Comment 39 Sylvain 2019-06-14 09:08:24 UTC
This seems ok for the Mouse. so the hint really enable/disable touch for the mouse.
I'll test it next week.

Yes, now the trackpad is producing genuine touch but appears as a virtual mouse.
The hint is by default enabled so we get the mouse event by default, but there are identified with the virtual id.

This indeed may be problematic or unsync'ed with other platforms. 
For instance, on my linux, touchpad is producing genuine MouseEvent (but it's a few years old and a not multi-touch trackpad one).

Just wondering ... how do single-touch trackpads behave currently on macosx?
And what about multi-touch trackpad on linux or windows ?

Maybe we add another case to identify and inverse that. (so trackpads are always mouse devices, and virtual touch devices).
But that means, it should use the real mouseId to generate mouseEvent and always be enable,
And for the touchId, it should use the virtualId and be enabled/disabled by the hint.
If so, maybe that should discriminated by the  SDL_TOUCH_DEVICE_INDIRECT_{RELATIVE,ABSOLUTE} or by SubTypeTablet ?
Comment 40 Ryan C. Gordon 2019-06-14 19:02:28 UTC
(In reply to Sylvain from comment #39)
> Just wondering ... how do single-touch trackpads behave currently on macosx?
> And what about multi-touch trackpad on linux or windows ?

I don't know. Apple doesn't ship a single-touch trackpad; I assume USB ones will work but I don't know if they give touch events or just mouse.

I'm willing to call this Good Enough for 2.0.10, though. I don't think Alex's concern about the mouse ID is wrong, but I suspect most things don't check if the device is "virtual" at all, and things that do because they want to ignore it in favor of touch devices are still going to get what they want anyhow (...probably).

--ryan.
Comment 41 Sam Lantinga 2019-06-14 20:59:02 UTC
I agree, I think this is fine.

Thanks everyone!
Comment 42 Alex Szpakowski 2019-06-14 21:25:33 UTC
[The below comment might warrant a separate issue, but it relates to these changes so I'll leave it here for now and maybe repost it if people think that'd be a good idea].

In my codebase, I want to ignore touch events that come from a trackpad but keep other (direct) touches. I also want to ignore mouse events which are synthetically generated from direct touches (since I handle them with touch events), but I want to keep/use mouse events which are generated from trackpad touches.

For macOS as a special case I could add #ifdefs in my code to ignore all touch events and to pretend the mouse instance id is never SDL_TOUCH_MOUSEID, and I think that'd work for current Mac hardware.

If I wanted to make that more robust, right now I believe I have *almost* enough information provided by SDL to have runtime checks for that: For touch events, I can query the touch device type to see whether it's an indirect (e.g. trackpad) or direct (e.g. touchscreen) touch. But for mouse events generated by touches, I don't know of a way to relate the mouse event back to a touch device type so I can ignore direct-touch mouse events but not indirect-touch mouse events.

Maybe it makes sense to add a field to the mouse event structs containing the touch device id, which would be filled in when the event's mouse instance id is set to SDL_TOUCH_MOUSEID (and invalid otherwise)? I'm not positive macOS provides enough information in its synthetic mouse events generated from trackpad touches to do that, but if it does, that approach seems logical to me.
Comment 43 Sam Lantinga 2019-06-15 00:17:39 UTC
I see what you're saying, and I think that makes sense. Let's break that out into a separate feature enhancement request.