My theory is that SDL2 somehow doesn't get a finger up or loses it in some way internally, and therefore it thinks the one finger that left the screen is still pressed down forever - and this finger is treated as the "first" finger that controls SDL_TOUCH_MOUSEID, so all "real" fingers that follow later are 2nd+ and ignored

This is possible but I would be suprised if that appeared.
What device are you using ? and which android version ? 

Can you add some log after your poll event loop, like:


   if (evt.type == SDL_MOUSEWHEEL)
   {
      const int normal  = (evt.wheel.direction == SDL_MOUSEWHEEL_NORMAL);
      const int y       = evt.wheel.y;
      SDL_Log("SDL Mouse Wheel y=%d normal=%d", y, normal);
   }

   if (evt.type == SDL_MOUSEBUTTONDOWN)
   {
       const int which = evt.button.which;
       int x = evt.button.x;
       int y = evt.button.y;
       SDL_Log("SDL Mouse Down which=%d (%d, %d)", which, x, y);
   }

   if (evt.type == SDL_MOUSEMOTION) {
       int x = evt.motion.x;
       int y = evt.motion.y;
       const int which = evt.motion.which;
      SDL_Log("SDL Mouse Motion which=%d (%d, %d)", which, x, y);
   }

   if (evt.type == SDL_MOUSEBUTTONUP)
   {
       const int which = evt.button.which;
       int x = evt.button.x;
       int y = evt.button.y;
       SDL_Log("SDL Mouse Up which=%d (%d, %d)", which, x, y);
   }

   if (evt.type == SDL_FINGERDOWN)
   {
      const SDL_TouchID touchId     = evt.tfinger.touchId;
      const SDL_FingerID fingerId   = evt.tfinger.fingerId;
      const int num_fingers         = SDL_GetNumTouchFingers(touchId);
      float x = evt.tfinger.x;
      float y = evt.tfinger.y;
 
      SDL_Log("Finger Down [%lld,%lld]! num_fingers=%d  (%f, %f)", 
            touchId, fingerId, num_fingers, x, y);
   }

   if (evt.type == SDL_FINGERUP)
   {
      const SDL_TouchID touchId     = evt.tfinger.touchId;
      const SDL_FingerID fingerId   = evt.tfinger.fingerId;
      const int num_fingers         = SDL_GetNumTouchFingers(touchId);
      float x = evt.tfinger.x;
      float y = evt.tfinger.y;
 
      SDL_Log("Finger Up [%lld,%lld]! num_fingers=%d (%f, %f)", 
            touchId, fingerId, num_fingers, x, y);
   }

   if (evt.type == SDL_FINGERMOTION)
   {
      const SDL_TouchID touchId     = evt.tfinger.touchId;
      const SDL_FingerID fingerId   = evt.tfinger.fingerId;
      const int num_fingers         = SDL_GetNumTouchFingers(touchId);
      float x = evt.tfinger.x;
      float y = evt.tfinger.y;
        
      SDL_Log("Finger Motion [%lld,%lld]! num_fingers=%d (%f, %f)", 
            touchId, fingerId, num_fingers, x, y);
    }

Created attachment 3732 [details]
SDL event log

I attached the requested event log of the bug happening

windowunknownevent-11 is an SDL_WindowEvent with type=11, which according to SDL2/SDL_video.h appears to be SDL_WINDOWEVENT_LEAVE. That is also right where the finger leaves the screen, and after which SDL_TOUCH_MOUSEID just breaks down forever. I can't remember ever seeing that before so while I haven't explicitly retested stable, I'm pretty sure this is a recent regression

The device is Android 7.1.1 / Gemini PDA

(sorry that the output is slightly different from the one you suggested, I just gave you the event log as wobblui already provides it. If you need some detail it doesn't have I'd be happy to add it)

Even if SDL was missing a Finger Up, it will catch the next one in Log. Because the FingerId is always 0.

Maybe log also TouchId to see ...

You really you copy paste the previous log code, after your SDL_PollEvent method or enable the Event log, I think there is an hint:
see SDL_HINT_EVENT_LOGGING

also it logs a mousemotion with which = 0

04-06 20:26:52.548 11594 11627 I python  : DEBU 2019-4-6-18:26:52+0 wobblui: wobblui.__init__.py: DEBUG: sdl event: mousemotion(which:0,x:0,y:644)

whereas there should be -1 ?

Oh, fascinating find! The reason I haven't just pasted your code is because I'm actually doing everything in Python through PySDL2 lol, making this a bit more unusually complicated...

For what it's worth, SDL_TOUCH_MOUSEID is logged as 4294967295 (since the struct member is defined as unsigned if I recall) so which:0 should definitely mean it's sent as mouse event 0 and not SDL_TOUCH_MOUSEID. Really fascinating!

I'm just adding the touchId and numFingers info, will attach the new log with the info added

I think when it goes out of the windows you have this WINDOW_LEAVE 
with set mouse focus == NULL
( SDL_SetMouseFocus(NULL); ) 

and then you don't receive any the other mouse events

Try to call: SDL_SetMouseFocus(window);

I've done an update: 
https://hg.libsdl.org/SDL/rev/eabdcf59246a

So that the tracking is done even if there is no focus window.
It's a little bit more robust.
Can you try to see if it helps?

Sorry that my responses are taking so long, I had to fix issues with my debug terminal widget and my build time is a bit lengthy :-O

Nevertheless, regarding SDL_WINDOWEVENT_LEAVE, I'm just wondering:
No matter when I get SDL_TOUCH_MOUSEID or not, shouldn't I get SDL_WINDOWEVENT_ENTER if the window is tapped and no hw mouse is attached? (I guess it makes sense if I don't get any if there is a hw mouse with the mouse cursor still truly outside the window) That seems a bit of a weird behavior of the focus itself that may be worth changing/fixing

About to test the newer revision & a more detailed debug log when the build has run through

I suspect that SDL_UpdateMouseFocus is having issue with the focus.
What about the patch below ?
if that works, it would be nice to see what happens in this SDL_UpdateMouseFocus function


diff -r 0cc2592ed0f4 src/events/SDL_mouse.c
--- a/src/events/SDL_mouse.c	Sat Apr 06 21:52:51 2019 +0200
+++ b/src/events/SDL_mouse.c	Sat Apr 06 23:12:49 2019 +0200
@@ -294,7 +294,7 @@
 int
 SDL_SendMouseMotion(SDL_Window * window, SDL_MouseID mouseID, int relative, int x, int y)
 {
-    if (window && !relative) {
+    if (window && !relative && mouseID != SDL_TOUCH_MOUSEID) {
         SDL_Mouse *mouse = SDL_GetMouse();
         if (!SDL_UpdateMouseFocus(window, x, y, mouse->buttonstate)) {
             return 0;

Created attachment 3733 [details]
SDL Event log of bug with revision d311fcb8066a (including touchId info)

Log now attached for still the old revision d311fcb8066a without patch, but now with better debug output that includes touchId (sorry I'm way too slow with my testing XD)

I will now test 0cc2592ed0f4 with that focus patch on top, unless you want me to test something else

Created attachment 3735 [details]
SDL Event log of bug with revision 0cc2592ed0f4 with patch in comment 11

Okay, ran the test now with latest revision + patch. The result is I no longer get any mouse leave or enter anymore, but it also works. Not sure if that is the intended behavior since after all a mouse leave doesn't seem so unreasonable, as long as there is a mouse enter once the finger/mouse returns

Anyway, I hope that helps you figure out if there is still something that needs changing!

(sorry, I was unclear: I do still get one initial SDL_WINDOWEVENT_ENTER when the app starts, but what I meant to say is that when the finger leaves the app area then I no longer get any more leave/enter events, not even a single leave as before. This was tested With revision d311fcb8066a combined with your patch in comment 11)

Meant 0cc2592ed0f4 (the newer one) with patch, regarding the previous comment 14, whoops! I'll stop now before I add any more conflicting info, sorry about that! lol

So the patch seems to work right. We get closer. But I still can't reproduce it.

A few things to investigate : 
Do you use SDL_WINDOW_FULLSCREEN or not ? 
Do it happens for all borders ? bottom/top/left/right ?

It would be nice if you could patch SDL to add some debug:

in 'src/events/SDL_mouse.c' (line 256), where it sets: 
inWindow = SDL_FALSE;

add (line 254), just after SDL_GetWindowSize(), something like
SDL_Log("inWindow==%d x=%d y=%d w=%d h=%d", inWindow, x, y, w, h);

also, before SDL_SetMouseFocus(NULL);
SDL_Log("call SDL_SetMouseFocus(NULL);");

> Do you use SDL_WINDOW_FULLSCREEN or not ? 

No it runs in windowed, with the action bar and status bar visible

> Do it happens for all borders ? bottom/top/left/right ?

I tested side for which it happens, and also just moving the finger up onto the status bar (doesn't even need to leave the screen) where it also happens

> It would be nice if you could patch SDL to add some debug:

Sure, can do. Should I test on the latest revision, or go back to the broken one to test this? With or without the patch in Comment 11?

> I tested side for which it happens, and also just moving the finger up onto the status bar 
> (doesn't even need to leave the screen) where it also happens

Not sure I still even understand the issue.
You lose the mouse events, while keeping the finger in the screen ?

I thought you have to move the finger outside the screen. This is why I asked if that happens, 
crossing the left egde, right edge, bottom edge or top edge ?


> Sure, can do. Should I test on the latest revision, or go back to the 
> broken one to test this? With or without the patch in Comment 11? 

Please try with the head (so 0cc2592ed0f4)  which is still broken, right ?

Don't try with the patch from comment 11. Just to confirm, the patch would solve the issue ?

With older revision d311fcb8066a and NO PATCH, the issue is:

1. I get SDL_WINDOWEVENT_LEAVE once the finger leaves the screen
2. I DON'T get SDL_WINDOWEVENT_ENTER once it returns
3. From then on, SDL_TOUCH_MOUSEID just isn't sent anymore

With newer revision 0cc2592ed0f4 with patch in comment 11 I get:

1. Once the finger leaves the screen I see no event in indicating it, and SDL_TOUCH_MOUSEID is consistently sent
2. Otherwise it works fine and SDL_TOUCH_MOUSEID works fine, but no SDL_WINDOWEVENT_LEAVE/SDL_WINDOWEVENT_ENTER anymore which probably are useful to have

That's all the combinations I tested so far. So yes, the newest revision with patch kinda solves it but at the cost of leave/enter events

Ok, I would like to know with newer revision 0cc2592ed0f4 *without* patch in comment 11.
(and the log to see why updatefocus is broken or if it does a SetFocus(NULL) ... )

I think we can't receive the  enter/leave window event because the mouse event are duplicated from the touch event.
so when your finger reach the border, it just becomes up. and so the simulated mouse becomes also up at this very instant.

Created attachment 3737 [details]
SDL Event log of bug with revision 0cc2592ed0f4 WITHOUT patch, but WITH debug info

Ok, attached SDL Event log of bug with revision 0cc2592ed0f4 WITHOUT patch, but WITH debug info in the following form:

--- a/src/events/SDL_mouse.c
+++ b/src/events/SDL_mouse.c
@@ -252,6 +252,7 @@ SDL_UpdateMouseFocus(SDL_Window * window, int x, int y, Uint32 buttonstate)
     if (window && ((window->flags & SDL_WINDOW_MOUSE_CAPTURE) == 0)) {
         int w, h;
         SDL_GetWindowSize(window, &w, &h);
+        SDL_Log("inWindow==%d x=%d y=%d w=%d h=%d", inWindow, x, y, w, h);
         if (x < 0 || y < 0 || x >= w || y >= h) {
             inWindow = SDL_FALSE;
         }
@@ -276,6 +277,7 @@ SDL_UpdateMouseFocus(SDL_Window * window, int x, int y, Uint32 buttonstate)
             printf("Mouse left window, synthesizing move & focus lost event\n");
 #endif
             SDL_PrivateSendMouseMotion(window, mouse->mouseID, 0, x, y);
+            SDL_Log("call SDL_SetMouseFocus(NULL);");
             SDL_SetMouseFocus(NULL);
         }
         return SDL_FALSE;
@@ -285,6 +287,7 @@ SDL_UpdateMouseFocus(SDL_Window * window, int x, int y, Uint32 buttonstate)
 #ifdef DEBUG_MOUSE
         printf("Mouse entered window, synthesizing focus gain & move event\n");
 #endif
+        SDL_Log("call SDL_SetMouseFocus(%p);", window);
         SDL_SetMouseFocus(window);
         SDL_PrivateSendMouseMotion(window, mouse->mouseID, 0, x, y);
     }


(the result of that test being also that the bug occurs, that is when the finger leaves the screen you can also see in the event log that no SDL_TOUCH_MOUSEID mouse events no longer occur)

So it seems there are negative values for the touch events:

04-07 17:58:52.611 29676 29716 I python  : DEBU 2019-4-7-15:58:52+0 wobblui: wobblui.__init__.py: DEBUG: sdl event: fingermotion(fingerId:0,touchId:1,numFingers:1,x:-0.0016459295293316245,y: 0.6993396878242493)

So it leads to negative value for synthetic mouse events: 

04-07 17:58:52.592 29676 29676 I SDL/APP : inWindow==1 x=-1 y=712 w=1006 h=1019

and it calls: 
04-07 17:58:52.592 29676 29676 I SDL/APP : call SDL_SetMouseFocus(NULL);

Hence, the WINDOW_LEAVE event ..


Not sure why there are negative touch motion  (That would be in SDLActivity, onTouch() -> ACTION_MOVE )

And when you touch in the right direction, maybe you go greater than "width window" and there is also a window leave event.
maybe this answer will help :

"Try using getRawX() and getRawY() instead of getX() and getY()."
( https://stackoverflow.com/questions/6237200/motionevent-gety-and-getx-return-incorrect-values )

I achieved to get also negative value by touching under the status bar, and I got the same issue.

The issue wasn't appearing with the former version, because we always gave the unique Android window. Now we use a generic code and it uses the focused window.
The focus gets out when the synthetic mouse leave the window.

And there is no focus-window information coming from the touch event.

Maybe the best way to fix this is to filter out the synthetic mouse event, to prevent from removing the focus window ? (eg previous patch).

Sam ? Ryan ?

But then what if an actual hw mouse connected (with SDL_TOUCH_MOUSEID) leaves the window? Wouldn't that still break touch as a consequence with your patch above for the duration the hardware mouse cursor is outside of the window?

In fact, it sounds a little like for any application with 2+ windows, touch currently only works in the one window where the actual hardware mouse is (= which has mouse focus) which doesn't seem like a reasonable filter criteria. I don't know how to test that on Android though where you can only have one

I could be wrong of course, I'm just extrapolating/guessing so ignore me if that makes no sense :-D

> But then what if an actual hw mouse connected (with SDL_TOUCH_MOUSEID)

Sorry, I meant: NOT with SDL_TOUCH_MOUSEID

it shouldn't disturb hw  mouse. 

I'm not sure sure how we should fix the issue:
- maybe we should discard focus handling for virtual mouse.
- but we may lose enter/leave event.

> it shouldn't disturb hw  mouse. 
I was really thinking of hw mouse disturbing the SDL_TOUCH_MOUSEID handling rather than the other way around

Anyway, isn't this a general problem with mouse focus & multiple mouses?

If all mouses including the SDL_TOUCH_MOUSEID "virtual mouse" had their focus separately tracked by SDL2, and SDL_WINDOWEVENT_ENTER was sent as soon as ANY mouse is inside the window, and SLD_WINDOWEVENT_LEAVE was sent as soon as ALL mouses left the window, thten this should work fine right? Of course that would mean multiple windows could get SDL_WINDOWEVENT_ENTER (without leave in between) as soon as 2+ mouses are involved

I am right guessing that on most platforms, multiple mouses will use multiple mouse cursors, right? If not then what I just said makes no sense and please ignore it :-D

A configuration with several mice isn't really usual. And all this is a little bit theoretical.
The simulated mouse is somehow a degraded/fallback mode, so that app that were designed for mouse, could also work when using a touchscreen.

Okay, this is the most simple suggestion I can come up with:

What if

- no mouse events are never filtered for mouse focus (= essentially what you suggested as in just ignoring the entire concept when producing the mouse events themselves)

- and any mouse event ever on a window will simply cause SDL_WINDOWEVENT_ENTER if that window hasn't had that yet, and SDL_WINDOWEVENT_LEAVE on all others

That would work with an infinite amount of independent mouse cursors, while being pretty simple to implement (all you need is a flag on each window if it was sent SDL_WINDOWEVENT_ENTER without a leave afterwards) while never possibly leading to mouse events being lost ever

>  - no mouse events are never filtered 

the double negation was unintentional, I'm really brilliant at typoing my comments apparently

Virtual mouse events should never leave the window or change focus for single window applications. 

Fixed https://hg.libsdl.org/SDL/rev/1223f50edde3

Created attachment 3739 [details]
SDL Event log with revision 1223f50edde3 WITHOUT patches

Ok I tested the latest revision 1223f50edde3 with zero patches, and it works BETTER but there is still a problem:

The attached log is me dragging the finger out at the top (over the status bar) and afterwards I tap somewhere onto the lower half of the screen

- As you can see, the latest SDL_TOUCH_MOUSEID mouse MOTION event is at x:447,y:0 when my drag reaches the upper window border
- Then there are a lot of touch only events, followed by a touch finger UP (but no mouse up!) as I let go somewhere over the statusbar
- Then there is another SDL_TOUCH_MOUSEID mouse MOTION event at x:160,y:712 caused by my the single tap

There is no mouse up/down in between! Therefore, an application may reasonably perceive this as a sudden super fast drag from x:447,y:0 to x:160,y:712 in a single event jump. It would be good if there was an actual mouse up in between, at the very latest when there is the coinciding finger up event!

Created attachment 3740 [details]
patch

Then, here's a patch to try. So that simulated mouse events are contained in the window. 
And there would be mouse up/down even if the finger is outside.

Okay, finally got around to test the patch in Comment 34 on top of revision 1223f50edde3 and from all I can see it appears to be working perfectly! No more issues, touch input continues working, and when letting the finger to onto the status bar I still get the SDL_TOUCH_MOUSEID mouse up as expected and it's all working nicely. Thanks for all the fixes!

Ok ! so hopefully the bug is resolved !
https://hg.libsdl.org/SDL/rev/3477a301a5a8
Thanks a lot for the report, feedback and helping with the various patch!