We are currently migrating Bugzilla to GitHub issues.
Any changes made to the bug tracker now will be lost, so please do not post new bugs or make changes to them.
When we're done, all bug URLs will redirect to their equivalent location on the new bug tracker.

Bug 4554 - [HIDAPI] Integrate Valve's multi-backend support into SDL
Summary: [HIDAPI] Integrate Valve's multi-backend support into SDL
Status: RESOLVED FIXED
Alias: None
Product: SDL
Classification: Unclassified
Component: joystick (show other bugs)
Version: HG 2.0
Hardware: All All
: P2 enhancement
Assignee: Sam Lantinga
QA Contact: Sam Lantinga
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-17 19:18 UTC by Ethan Lee
Modified: 2019-08-07 02:36 UTC (History)
3 users (show)

See Also:


Attachments
Prototype of the C/loadso port (19.77 KB, text/x-csrc)
2019-03-17 19:18 UTC, Ethan Lee
Details
[PATCH] hidapi: Support multiple hidapi backends (27.08 KB, patch)
2019-07-25 13:17 UTC, Andrew Eikum
Details | Diff
SDL_hidapi support (38.82 KB, patch)
2019-07-31 16:26 UTC, Ethan Lee
Details | Diff
libusb library names patch (1.78 KB, patch)
2019-07-31 19:56 UTC, Ozkan Sezer
Details | Diff
configury patch to allow libusb-less hidapi for macosx (3.59 KB, patch)
2019-08-01 09:01 UTC, Ozkan Sezer
Details | Diff
libusb hid.c Portability Patch (13.34 KB, patch)
2019-08-03 16:23 UTC, Ethan Lee
Details | Diff
Port libusb hid.c to SDL (15.79 KB, patch)
2019-08-04 04:04 UTC, Ethan Lee
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ethan Lee 2019-03-17 19:18:02 UTC
Created attachment 3700 [details]
Prototype of the C/loadso port

Inside SDL's copy of the HIDAPI source is a whole bunch of Valve-related changes, including a very interesting file for Linux specifically:

https://hg.libsdl.org/SDL/file/92e2b144d965/src/hidapi/linux/hid.cpp

Being able to use multiple HIDAPI backends is a really good idea! Three problems with this file though:

1. C++ dependency
2. Linux-specific
3. Links directly to libusb

The C++ side isn't so bad since there's not much to port to C, and the Linux side is mostly in name only. The libusb requirement is definitely a problem though, especially on Windows/macOS where I'd like to ship this.

I've attached a file that in theory solves all three problems. It's a _super_ hacky prototype file, but I wanted to get this in front of the team ASAP since it affects multiple platforms with a rather big chunk of code, and is in fact mandatory for the recent GameCube adapter code to work on Windows and macOS.

The way it works is very simple: Throw SDL_hidapi.c into `src/hidapi`, then compile the one file. It automatically pulls in both the libusb version of hid.c as well as whatever platform hid.c we care about (currently Windows/macOS/Linux-only, not sure if any other platform cares right now?). All the namespacing and separation is done entirely in the file, without adding any preprocessor goo like the `#ifdef NAMESPACE` blocks that are in each file. I did need to add `libusb_get_string_descriptor` into the libusb hid.c so it would use the dynamically-loaded functions, but it seems that's done for FreeBSD anyway so maybe it's not so bad to have...?

AFAIK the code is okay other than absolutely requiring libusb (it'd be better if we just ignored that implementation if the DLL can't be found), but integrating this into the build system will be pretty fun...
Comment 1 Sam Lantinga 2019-06-12 03:16:33 UTC
We don't currently include it because it libusb requires root permission to access devices on many systems, also a C++ dependency wouldn't work for SDL.
Comment 2 Andrew Eikum 2019-07-25 13:17:06 UTC
Created attachment 3897 [details]
[PATCH] hidapi: Support multiple hidapi backends

Here's a new version that is working for me. Note that it depends on the patches from Bug 4736, Bug 4737, and Bug 4738. This definitely needs cross-platform review, I've only tested it on my Arch Linux desktop.
Comment 3 Andrew Eikum 2019-07-25 13:34:53 UTC
This patch fixes the problems Ethan mentioned. It makes both the libusb and platform-specific backends optional at both compile- and run-time. The platform-specific backend can be missing at run-time if libudev cannot be loaded on Linux. For other platforms where the platform backend will always be present, the runtime check is replaced with an if-check for 1, so hopefully the compiler will just remove that entirely at build-time.

I also changed it to use trivial pointer-aliasing with a vtable, instead of traversing a linked list and writing a switch statement for every hidapi device call.

The patch contains changes to both configure and cmake build systems, but more effort may be required for other build systems.
Comment 4 Andrew Eikum 2019-07-25 17:36:53 UTC
Cameron pointed out on Bug 4738 that hid_enumerate should call hid_init. We need to add that to our implementation here.
Comment 5 Ethan Lee 2019-07-31 16:26:13 UTC
Created attachment 3913 [details]
SDL_hidapi support

The patch has been updated and cleaned up; the build should now work on all supported targets (even those that don't use SDL_hidapi.c). This patch still depends on Bug 4737 and Bug 4738.

The only builds that will change are autoconf/CMake builds on Linux/macOS, all other builds should be exactly the same. MinGW builds will eventually join them, but only after somebody ports libusb/hid.c to be OS-agnostic. That's for another day!
Comment 6 Sam Lantinga 2019-07-31 17:24:55 UTC
Patch added!
https://hg.libsdl.org/SDL/rev/0fef4b21fa1d

I'm leaving this bug open to collect feedback and cleanup on this change.
Comment 7 Ozkan Sezer 2019-07-31 18:22:20 UTC
How will this integrate into existing Xcode project files?
Comment 8 Ethan Lee 2019-07-31 18:26:35 UTC
For now it won't affect Xcode projects at all, those files and builds should be exactly the same as before. To add support for the Xcode project we'd have to get libusb directly involved in the SDL packages, which didn't seem like a good idea. The make builds will treat libusb like any other dependency; it'll either include SDL_hidapi and dynamically load libusb, or it'll disable the feature if it's not found.
Comment 9 Ozkan Sezer 2019-07-31 18:34:05 UTC
And for the autofoo builds to catch SDL_LIBUSB_DYNAMIC,
I need to manually build libusb-1.0 beforehand and put
it in my toolchain-seeable paths, yes?
Comment 10 Ethan Lee 2019-07-31 18:57:27 UTC
Yes. You can also add to this block here, if it makes life any easier:

https://hg.libsdl.org/SDL/file/0fef4b21fa1d/src/hidapi/SDL_hidapi.c#l132
Comment 11 Ozkan Sezer 2019-07-31 19:56:02 UTC
Created attachment 3915 [details]
libusb library names patch

(In reply to Ethan Lee from comment #10)
> Yes. You can also add to this block here, if it makes life any easier:
> 
> https://hg.libsdl.org/SDL/file/0fef4b21fa1d/src/hidapi/SDL_hidapi.c#l132

libusb from github current master (1.0.23-rc1 + some more commits)
builds for osx to this:
  libusb-1.0.dylib -> libusb-1.0.1.dylib

... and for linux to this:
  libusb-1.0.so -> libusb-1.0.so.1 -> libusb-1.0.so.1.1.0

An older version (1.0.9-rc1 from CentOS-6.10 / i386) is like this:
  libusb-1.0.so.0 -> libusb-1.0.so.0.1.0
 (libusb-1.0.so symlink comes only with the -devel rpm.)

So I gather that, we need to rely on the presence of libusb-1.0.so
symlink (autofoo should actually detect it correctly) for linux,
and libusb-1.0.dylib symlink for osx (brew should itstall it???).

Is the attached patch OK?
Comment 12 Ethan Lee 2019-07-31 20:03:22 UTC
Interesting, I didn't know they had incremented the version for their next release. I was assuming it was still 1.0.0; I tested with CentOS 7, Fedora 30, and Homebrew for macOS, which are all on 1.0.22 or lower. It looks like the only changes are new APIs, so as long as that still holds up the patch should be safe. We'll probably want to do the same thing for Linux and friends as well:

https://hg.libsdl.org/SDL/file/0fef4b21fa1d/configure.ac#l3268
Comment 13 Ethan Lee 2019-07-31 20:05:00 UTC
Looks like even the libusb team isn't sure about the change:

https://github.com/libusb/libusb/issues/602
Comment 14 Ozkan Sezer 2019-07-31 20:10:19 UTC
(In reply to Ethan Lee from comment #12)
>  We'll probably want to do the same thing for Linux and
> friends as well:
> 
> https://hg.libsdl.org/SDL/file/0fef4b21fa1d/configure.ac#l3268

You suggest that we change libusb-1.0.so.* to libusb-1.0.so* ?

(In reply to Ethan Lee from comment #13)
> Looks like even the libusb team isn't sure about the change:
> 
> https://github.com/libusb/libusb/issues/602

Huh..

What is the final verdict?
Comment 15 Ethan Lee 2019-07-31 20:14:04 UTC
For now let's wait and see what the libusb team has to say about the ABI increment - my hope is that they'll undo the change so we can avoid having to gamble on a symlink, but if they decide to increment then we'll need to apply the patch for this to keep working when 1.0.23 is released.
Comment 16 Ozkan Sezer 2019-07-31 20:27:20 UTC
(In reply to Ethan Lee from comment #15)
> For now let's wait and see what the libusb team has to say about the ABI
> increment - my hope is that they'll undo the change so we can avoid having
> to gamble on a symlink, but if they decide to increment then we'll need to
> apply the patch for this to keep working when 1.0.23 is released.

That sounds reasonable at first, but what happens when they
do make a release (something like 1.1.x) with library versions
like in current 1.0.23-rc?
Comment 17 Ethan Lee 2019-07-31 20:34:14 UTC
I suppose it depends on how SDL has dealt with dependency breakages in the past, which I don't have experience with... other than the nasty libudev.so.0/1 issue, which is just a hardcoded list of names. Ideally the reason for an increment is a legitimate breakage, so in that case we'd still depend on a specific version and have the version-specific code based on the `LIBUSB_API_VERSION` in libusb.h, unless we really needed to support both at once (like udev).

Ryan, Sam, any insight on this?
Comment 18 Ozkan Sezer 2019-07-31 22:43:32 UTC
BTW, this breaks osx builds against 10.8 SDK, because libusb/hid.c
uses clock_gettime() & co, and 10.8 doesn't have it.  IIRC, SDL2.0
still supports building against 10.7+ SDKs.  Needs fixing.
Comment 19 Ozkan Sezer 2019-07-31 22:57:18 UTC
Seems like clock_gettime() is a 10.12+ thing.
libusb has replacement code in libusb/os/darwin_usb.c,
maybe it helps.
Comment 20 Ozkan Sezer 2019-07-31 23:34:01 UTC
A little googling shows clock_gettime() Darwin alternatives here:
https://stackoverflow.com/questions/5167269/
https://developer.apple.com/library/archive/qa/qa1398/_index.html
Comment 21 Ethan Lee 2019-08-01 02:20:22 UTC
libusb's hid.c definitely needs to be SDL-ified, both for macOS and an eventual Windows build. That in particular will probably be replaced with something like SDL_GetTicks, along with other stuff like the explicit use of pthread instead of SDL_Thread.
Comment 22 Ozkan Sezer 2019-08-01 07:36:20 UTC
Configury cannot enable libusb-less hidapi for osx like Xcode
since this merge.  Cmake is _possibly_ affected too (didn't try
it.)  Messing with autofoo now.
Comment 23 Ozkan Sezer 2019-08-01 09:01:59 UTC
Created attachment 3917 [details]
configury patch to allow libusb-less hidapi for macosx

(In reply to Ozkan Sezer from comment #22)
> Configury cannot enable libusb-less hidapi for osx like Xcode
> since this merge.  Cmake is _possibly_ affected too (didn't try
> it.)  Messing with autofoo now.

Attached patch OK?
Comment 24 Ethan Lee 2019-08-01 13:46:48 UTC
Looks good to me! I'll try to work on cleaning up libusb hid.c today.
Comment 25 Ozkan Sezer 2019-08-01 14:43:16 UTC
(In reply to Ethan Lee from comment #24)
> Looks good to me!

Applied: https://hg.libsdl.org/SDL/rev/d2e027c5a389
Cmake will need similar love & care (I won't do that..)

>  I'll try to work on cleaning up libusb hid.c today.

Thanks.
Comment 26 Ethan Lee 2019-08-03 16:23:35 UTC
Created attachment 3919 [details]
libusb hid.c Portability Patch

Attached is a... very pretty patch to make libusb's hid.c portable. This should fix the Mac build, and it also fixes the problems that Windows had, so it's also been added to the MinGW build.

I wrote it in this way because I don't know how far off SDL's hidapi copy is from upstream, so I figured this would be the safest way to compare the two (i.e. just delete all the SDL blocks and diff). If we're sync'd to upstream, we could just nuke all the unportable stuff and put a date/commit at the top of the file that says when we last synchronized with them.
Comment 27 Ethan Lee 2019-08-04 04:04:34 UTC
Created attachment 3921 [details]
Port libusb hid.c to SDL

Patch updated with the changes from upstream hidapi's libusb hid.c (basically some explicit type casts and typo fixes), an added date to mark when their version was last updated, and removed all the non-SDL code paths. The file is now MUCH more legible and it will be easier to check for upstream changes in the future.
Comment 28 Sam Lantinga 2019-08-06 06:36:20 UTC
Okay, I think this is complete!

Ethan, I added your patch so libusb.c works on Windows and Mac OS X:
https://hg.libsdl.org/SDL/rev/a7ca9815bc79

We'll wait on changing libusb library names, that can go in as a separate bug.

Are we waiting on anything else for this bug?
Comment 29 Ethan Lee 2019-08-06 13:03:30 UTC
I believe that was the last thing! I've posted/subscribed to the libusb issue re: the ABI change, may be worth subbing to that for anyone who has a GitHub account:

https://github.com/libusb/libusb/issues/602

If they decide to keep the increment I'll file a new issue for that.
Comment 30 Sam Lantinga 2019-08-07 02:36:36 UTC
Great, thanks!