We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 4553 - touchscreen coordinates don't match screen size
Summary: touchscreen coordinates don't match screen size
Status: NEW
Alias: None
Product: SDL
Classification: Unclassified
Component: events (show other bugs)
Version: 2.0.9
Hardware: ARM Linux
: P2 normal
Assignee: Sam Lantinga
QA Contact: Sam Lantinga
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-14 09:29 UTC by Alain
Modified: 2019-04-04 08:44 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alain 2019-03-14 09:29:02 UTC
Hello,

I made a test with https://hg.libsdl.org/SDL/rev/f100ca4dd31e changeset to be able to use eGalax touchscreen.

Unfortunately, I think there is a problem as the coordinates are transmited according to the touchscreen resolution and not the screen size.

Maybe I'm wrong but I guess it's the responsability of SDL to process the touchscreen coordinates regarding the screen size.

Here is my test case:

	#include <stdbool.h>
	#include <stdlib.h>

 	#include <SDL2/SDL.h>
	#include <SDL2/SDL_events.h>

	int main(void) {
		SDL_Window *WID_main_window;
		SDL_Renderer *WID_main_renderer;
	    SDL_Event sdlevt;

	    if ( SDL_Init( SDL_INIT_EVERYTHING) < 0 ) {
	        exit(-1);
	    }

		int video_driver_index = 1;
		int renderer_index = 1;

	    SDL_RendererInfo info;

	    setenv("SDL_VIDEODRIVER",SDL_GetVideoDriver(video_driver_index),1);

		SDL_GetRenderDriverInfo(renderer_index,&info);

		if(SDL_SetHint(SDL_HINT_RENDER_DRIVER, info.name) != SDL_TRUE)
		{
			printf("Fail to execute SDL_SetHint %s\n", SDL_GetError());
			exit(-1);
		}

		if ((WID_main_window = SDL_CreateWindow("tests", 0, 0, 1024, 768, SDL_WINDOW_SHOWN)) == NULL) {
			exit(-1);
		}

		if ((WID_main_renderer = SDL_CreateRenderer(WID_main_window, renderer_index, info.flags)) == NULL) {
			exit(-1);
		}

		if (SDL_SetRenderTarget(WID_main_renderer, NULL) != 0) {
			exit(-1);
		}
		SDL_SetRenderDrawColor(
				WID_main_renderer,
				0x80,
				0x80,
				0x80,
				SDL_ALPHA_OPAQUE);

		SDL_RenderClear(WID_main_renderer);

		SDL_RenderPresent(WID_main_renderer);

	    SDL_ShowCursor(SDL_ENABLE);

	    SDL_EventState(SDL_WINDOWEVENT,SDL_IGNORE);

	    bool loop = true;
	    while (loop) {
	    	SDL_PollEvent(&sdlevt);

			if ( sdlevt.type == SDL_QUIT ) {
				loop = false;
			}

			if ((sdlevt.type == SDL_MOUSEBUTTONDOWN) ||
					(sdlevt.type == SDL_MOUSEBUTTONUP) ||
					(sdlevt.type == SDL_MOUSEMOTION)) {
				fprintf(stderr, "SDL event %d, x %d, y %d\n", sdlevt.type, sdlevt.button.x, sdlevt.button.y);
			}


			SDL_FlushEvent(SDL_MOUSEMOTION);

			SDL_Delay(50);
	    }

	    SDL_DestroyRenderer(WID_main_renderer);
	    SDL_DestroyWindow(WID_main_window);
	    SDL_Quit();

		return EXIT_SUCCESS;
	}

I'm using the native allpoint EETI driver and I'm running my application on a 1024x768 screen as shown in the source code.

What is strange anyway is that the first touch seems to be 'a bit' processed as its values are within the 1024x768 range with a maximum of 1023x767. 
Anyway, in my case 1023,767 ismore or less at the center of the screen, not at the bottom right corner.

Here are the results:

The first time I ran the application, with 2 touches in the center of the screen:

        SDL event 1024, x 1023, y 767
	SDL event 1025, x 1023, y 767
	SDL event 1026, x 1023, y 767
	SDL event 1024, x 1968, y 1900
	SDL event 1025, x 1968, y 1900
	SDL event 1026, x 1968, y 1900


The second time I ran the application, with 2 touches close to the top left corner:
	SDL event 1024, x 215, y 271
	SDL event 1025, x 215, y 271
	SDL event 1026, x 215, y 271
	SDL event 1024, x 216, y 308
	SDL event 1025, x 216, y 308
	SDL event 1026, x 216, y 308

The thirdtime I ran the application, with 2 touches close to the bottom right corner:


	SDL event 1024, x 1023, y 767
	SDL event 1025, x 1023, y 767
	SDL event 1026, x 1023, y 767
	SDL event 1024, x 3902, y 3935
	SDL event 1025, x 3902, y 3935
	SDL event 1026, x 3902, y 3935



E
Comment 1 Sylvain 2019-03-14 10:43:05 UTC
maybe this commit is more related: 

https://hg.libsdl.org/SDL/rev/0f79dd727ed6
bug 3157

see min_x/y and range_x/y
Comment 2 Alain 2019-03-14 12:42:34 UTC
Well, I don't really see what to think about your comment Sylvain ?!?

Alain
Comment 3 Alain 2019-03-14 12:45:59 UTC
I see some code supposed to normalize the cordinates but where min_x/y and range_x/y come from?
Comment 4 Sylvain 2019-03-14 13:58:47 UTC
Seems to come from:

ioctl( item->fd, EVIOCGABS( ABS_MT_POSITION_X ), &abs_info );

https://hg.libsdl.org/SDL/file/0f79dd727ed6/src/core/linux/SDL_evdev.c#l614
Comment 5 Alain 2019-03-15 07:44:25 UTC
Ok, but it's not the same thing.
You are talking about normalized coordinates we can get from dX/dY and which are in -1.0, 1.0 range.

I didn't verify this value but I suppose it works as expected.

I'm talking about SDL_SendMouseMotion(mouse->focus, SDL_TOUCH_MOUSEID, 0, abs_x, abs_y); which, according to available data, could be recalculated in the screen resolution range.

I think.
Comment 6 Alain 2019-03-15 08:52:28 UTC
To have a look at we talked about, I modified my loop to take care of finger events as follow:

    while (loop) {
    	if (SDL_PollEvent(&sdlevt)) {

			if ( sdlevt.type == SDL_QUIT ) {
				loop = false;
			}

			if ((sdlevt.type == SDL_MOUSEBUTTONDOWN) ||
					(sdlevt.type == SDL_MOUSEBUTTONUP) ||
					(sdlevt.type == SDL_MOUSEMOTION)) {
				fprintf(stderr, "SDL event %d, x %d, y %d\n", sdlevt.type, sdlevt.button.x, sdlevt.button.y);
			}

			if ((sdlevt.type == SDL_FINGERDOWN) ||
					(sdlevt.type == SDL_FINGERUP) ||
					(sdlevt.type == SDL_FINGERMOTION)) {
				fprintf(stderr, "SDL finger event %d, x %f, y %f, x %d, y %d\n", sdlevt.type, sdlevt.tfinger.x, sdlevt.tfinger.y, (int)(sdlevt.tfinger.x * 1024.0), (int)(sdlevt.tfinger.y * 768.0));
			}
    	}

		SDL_Delay(50);

		SDL_FlushEvent(SDL_MOUSEMOTION);
		SDL_FlushEvent(SDL_FINGERMOTION);
    }

I then touch my screen, make a small displacement and get my finger up. Here is the result:

	SDL event 1024, x 2140, y 2365
	SDL event 1025, x 2140, y 2365
	SDL finger event 1792, x 0.522589, y 0.577534, x 535, y 443
	SDL event 1024, x 2164, y 2367
	SDL event 1024, x 2188, y 2376
	SDL event 1024, x 2212, y 2389
	SDL event 1024, x 2224, y 2398
	SDL event 1024, x 2236, y 2406
	SDL event 1024, x 2252, y 2417
	SDL event 1024, x 2260, y 2424
	SDL event 1024, x 2268, y 2431
	SDL event 1024, x 2272, y 2433
	SDL event 1024, x 2270, y 2427
	SDL event 1026, x 2270, y 2427
	SDL finger event 1793, x 0.554335, y 0.592674, x 567, y 455

That means I receive:
SDL_MOUSEMOTION
SDL_MOUSEBUTTONDOWN
SDL_FINGERDOWN
several SDL_MOUSEMOTION
SDL_MOUSEBUTTONUP
SDL_FINGERUP

I never received any SDL_FINGERMOTION ??

About the coordinates, normalized coordinates are correct within 0.0, 1.0 range and it's quite easy to calculate it according to screen resolution. I agree.

Anyway, touchscreen or mouse are different pointing device to point a location on screen.

For me, it's not cool to prevent an application to not work anymore because we use a touchscreen instead of a mouse or a trackball.

if SDL_MOUSExxxx are generated on touch events, it must report the same coordinates for the same location not depending this location is pointed out by a mouse or by a touchscreen.
Comment 7 Alain 2019-03-15 09:01:23 UTC
What is again more strange about SDL_FINGERMOTION is that even if I didn't get it, if I don't call SDL_FlushEvent(SDL_FINGERMOTION);, my loop freezes.
Comment 8 Sylvain 2019-03-15 12:15:04 UTC
Sorry I misread your example. I read about touchscreen, and thought you were using touch events, but your example is about mouse event.

I'm not sure which sdl setting it builds with. So you need to find out where the mouse event coordinates comes from and see why those coordinates are wrong.
Comment 9 Alain 2019-03-15 15:53:46 UTC
For for me the problem comes from:

https://hg.libsdl.org/SDL/file/f100ca4dd31e/src/core/linux/SDL_evdev.c#l386
and
https://hg.libsdl.org/SDL/file/f100ca4dd31e/src/core/linux/SDL_evdev.c#l387

I don't know SDL data structure enough to write the right code but it should be something like:

 abs_x = (int)(norm_x * window_width); 
 abs_y = (int)(norm_y * window_height);

That way later calls to SDL_SendMouseMotion(mouse->focus, SDL_TOUCH_MOUSEID, 0, abs_x, abs_y); will be fine.
Comment 11 Alain 2019-03-18 07:54:24 UTC
Yes, basically it's exactly what we have to do, though I don't know if we have to separate mouse and touch.
I don't know the whole context so I don't really know what we have to particularly take care about.
On first sight, we should have :
                 
    SDL_GetWindowCoordinates(window, x, y, &window_x, &window_y);
    /* send moved event */
    SDL_SendMouseMotion(window, SDL_TOUCH_MOUSEID, 0, window_x, window_y);

for https://hg.libsdl.org/SDL/rev/f100ca4dd31e#l1.92 and https://hg.libsdl.org/SDL/rev/f100ca4dd31e#l1.115

and with:

static void SDL_GetWindowCoordinates(SDL_Window *window, float x, float y, int *window_x, int *window_y) {
    int window_w, window_h;

    SDL_GetWindowSize(window, &window_w, &window_h);
    *window_x = (int)(x * window_w);
    *window_y = (int)(y * window_h);
}
Comment 12 Alain 2019-03-18 07:56:13 UTC
But I'm surprised tht Ryan C. Gordon or Zach use https://hg.libsdl.org/SDL/rev/f100ca4dd31e like that;
Are we sure it hasn't been fixed already?
Comment 13 Sylvain 2019-03-18 12:35:07 UTC
> Are we sure it hasn't been fixed already?

You're not testing with latest SDL source ? If not, please use latest SDL sources.
Comment 14 Alain 2019-03-18 13:38:51 UTC
Up today the problem remains in the source.
Comment 15 Alain 2019-03-18 14:02:22 UTC
So I suggest for core/linux/SDL_evdev.c:

@@ -224,6 +224,17 @@
 }
 #endif /* SDL_USE_LIBUDEV */
 
+/*
+ * Calculate touchpad coordinates according to window size
+ */
+static void SDL_GetWindowCoordinates(SDL_Window *window, float x, float y, int *window_x, int *window_y) {
+    int window_w, window_h;
+
+    SDL_GetWindowSize(window, &window_w, &window_h);
+    *window_x = (int)(x * window_w);
+    *window_y = (int)(y * window_h);
+}
+
 void 
 SDL_EVDEV_Poll(void)
 {
@@ -383,8 +394,7 @@
                                 norm_pressure = 1.0f;
                             }
 
-                            abs_x = item->touchscreen_data->slots[j].x;
-                            abs_y = item->touchscreen_data->slots[j].y;
+							SDL_GetWindowCoordinates(window, item->touchscreen_data->slots[j].x, item->touchscreen_data->slots[j].y, &abs_x, &abs_y);
 
                             switch(item->touchscreen_data->slots[j].delta) {
                             case EVDEV_TOUCH_SLOTDELTA_DOWN:
Comment 16 Sylvain 2019-03-18 15:27:47 UTC
Check the patch fits your need.

I see that it can be used with following video driver:
video/vivante
video/raspberry
video/kmsdrm
So maybe contact people who could be affected to have some review or to test it
Comment 17 Alain 2019-03-22 08:51:52 UTC
I didn't have lot of replies, but the ones I got are positive.

Anyway, I think it's really necessary to fix the behavior according to what as already been done on Android branche.

As I'm not registered to make any change on SDL, could you please do it according to the patch I suggested just few posts ago?

Thanks in advance.
Alain.
Comment 18 Sylvain 2019-03-23 08:19:21 UTC
Just tried the patch, but it doesn't even compile?
Comment 19 Alain 2019-03-25 12:57:13 UTC
Really sorry Sylvain, I didn't take care about the concept.

I see what to change but I'm just waiting to be able to test it on my hard as I currently don't exactly know the options used to build SDL lib.

I should be able to test it in the next few days thanks to the support to my board provider.

I'l let you know when it's ok.
Alain
Comment 20 Sylvain 2019-03-25 14:01:55 UTC
I also believe it won't fix your issue.
because the window size seems in fact ignore, it's always set as the desktop size:

https://hg.libsdl.org/SDL/file/36ee99107390/src/video/kmsdrm/SDL_kmsdrmvideo.c#l612
Comment 21 Alain 2019-03-26 07:37:11 UTC
I will check for the returned values but as the comment says, I think that most SDL applications run in one window only and certainly mostly in full screen size so it wouldn't really be a problem.
Comment 22 Alain 2019-03-28 15:16:08 UTC
Sylvain,

I discussed with our board provider and it's quite difficult to build SDL alone.
Build system is in fact a yocto project which pull out the SDL source from an URL and takes care about all needed configurations to correctly do the job.

So, would it be possible for you to maybe create a specific branch with the patch below and I'll given them the changeset URL in order they make a build to verify coordinates are now correctly reported.

As I we already talked with you, it seems that SDL library currently considers only one window and this window is in fact accessible from `mouse->focus` in SDL_EVDEV_Poll() function.

Anyway, the only configuration I'll able to test is a SDL window in fullscreen size because we start the application without X server.
The touchscreen is connected through SPI bus and I cannot connect it on a Linux machine via USB to eventually verify what's going on if I create a 800x600 SDL window on a 102x768 Linux Desktop.

I started from the last https://hg.libsdl.org/SDL/rev/d5b4d374a312 changeset, including the following patch and it's now possible to build it successfully (Ubuntu 18.04.2).

@@ -224,6 +224,17 @@
 }
 #endif /* SDL_USE_LIBUDEV */
 
+/*
+ * Calculate touchpad coordinates according to window size
+ */
+static void SDL_GetWindowCoordinates(SDL_Window *window, float x, float y, int *window_x, int *window_y) {
+    int window_w, window_h;
+
+    SDL_GetWindowSize(window, &window_w, &window_h);
+    *window_x = (int)(x * window_w);
+    *window_y = (int)(y * window_h);
+}
+
 void 
 SDL_EVDEV_Poll(void)
 {
@@ -383,8 +394,7 @@
                                 norm_pressure = 1.0f;
                             }
 
-                            abs_x = item->touchscreen_data->slots[j].x;
-                            abs_y = item->touchscreen_data->slots[j].y;
+							SDL_GetWindowCoordinates(mouse->focus, item->touchscreen_data->slots[j].x, item->touchscreen_data->slots[j].y, &abs_x, &abs_y);
 
                             switch(item->touchscreen_data->slots[j].delta) {
                             case EVDEV_TOUCH_SLOTDELTA_DOWN:
Comment 23 Sylvain 2019-03-28 17:25:34 UTC
This is the recipe, isn't it ?

http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-graphics/libsdl2/libsdl2_2.0.9.bb?id=b46dbcad31c990b5556d61357e0a976948a5dede

you can patch 

SRC_URI = "http://www.libsdl.org/release/SDL2-${PV}.tar.gz \
with whatever archive like: 
file://...
Comment 24 Alain 2019-03-29 07:29:13 UTC
Sorry, I cannot tell you more, I'm not aware about it.

Alain
Comment 25 Sylvain 2019-04-02 15:35:26 UTC
Indirectly, this behaviour is in.
SDL_TOUCH_MOUSEID is now handled at higher level, so please give a try.
Comment 27 Alain 2019-04-04 07:48:17 UTC
I currently use https://hg.libsdl.org/SDL/rev/a34ab2e5ddcf and it doesn't work.

To be more precise, the first touch is ok but all subsequent events are reported as raw touchpad coordinates.

Quite strange.
Comment 28 Sylvain 2019-04-04 08:44:00 UTC
Do you have only 1 screen?


Might be an issue with you board drivers or the evdev code that would need specific settings ..  
You definitively want to set up a dev installation and investigate.