Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

App fails to start on pre-KitKat devices since 2.0.9 (could not load "libhidapi.so") #3024

Closed
SDLBugzilla opened this issue Feb 11, 2021 · 0 comments

Comments

@SDLBugzilla
Copy link
Collaborator

This bug report was migrated from our old Bugzilla tracker.

These attachments are available in the static archive:

Reported in version: 2.0.9
Reported for operating system, platform: Android (All), ARM

Comments on the original bug report:

On 2018-10-15 23:17:26 +0000, Anthony @ POW Games wrote:

Since 2.0.9 This is only happening on Android 4.2.2 (and all pre-KitKat, I am guessing). The full error on lunch is:

"Error: Cannot load library: soinfo_link_image(linker.cpp:1635): could not load library "libhidapi.so" needed by "libSDL2.so"; caused by load_library(linker.cpp:745): library "libhidapi.so" not found"

On 2018-10-16 07:17:35 +0000, Sylvain wrote:

Since it's only 1 file, it could be included into the libSDL2 build ?
Could it also be converted to c so we don't need the cpp runtime for only 1 file.
Or just deactivated by a define ?

On 2018-11-03 00:20:21 +0000, Sam Lantinga wrote:

Are you not including libhidapi.so in your APK?

On 2018-11-03 01:33:55 +0000, Anthony @ POW Games wrote:

I didn't know I was supposed to? There's no reference of libhidapi.so in any of the 2.0.9 .mk files. I just replaced 2.0.8 with 2.0.9 and rebuilt my project. It builds ok and runs ok on everything after Kit-Kat.

But, I haven't tested this since your first pre-release, so it might have been fixed since. I will get back to you.

On 2018-12-11 05:15:14 +0000, Steve Robinson wrote:

I'm also having this problem with the SDL2 2.0.9 release.

I use CMake to build SDL2 and package it with Conan:
https://gitlab.com/ssrobins/conan-sdl2

Then I use that SDL2 package to build the 'rectangles' examples into a single .so:
https://gitlab.com/ssrobins/sdl2-example

I use Android ABI armeabi-v7a, Android SDK 28, and Android NDK r18b for all builds.

CMake copies the .so plus all the .java files from this SDL2 source tree location:
android-project/app/src/main/java/org/libsdl/app

...and puts them in this location in my Gradle project:
app/src/main/java/org/libsdl/app

Then I run Gradle and everything builds fine.

When installing and running the resulting APK on my Moto X Pure with Android 7.0, I get the 'SDL HIDAPI Error'. Everything worked fine in 2.0.8.

You can download my APK here:
https://gitlab.com/ssrobins/sdl2-example/-/jobs/132014141/artifacts/browse/build_android/_package/

Let me know if there's anything else I can provide to help with this. It's currently preventing me from upgrading.

On 2019-01-03 07:43:25 +0000, wrote:

Running into this same problem, but think I have the root cause:

I think this problem relates to the SDL2 CMakeLists.txt, both Android.mk and configure contain references to building hidapi as a library, but CMakeLists.txt does not. And thus libhidapi is never build and never included in the APK.

So most likely building SDL2 with androids ndk-build would work around this issue.

On 2019-01-05 05:46:45 +0000, Sam Lantinga wrote:

Ahh, that makes sense. I don't use CMake, can you provide a tested patch?

Thanks!

On 2019-01-07 22:11:12 +0000, wrote:

I quickly hacked a few lines into the CMake file to see if I could get it to work.

By no means a final patch, and I'm no CMake wizard at all, but I added:

set(CMAKE_CXX_FLAGS "${CMAKE_C_FLAGS} -std=c++11 -fno-rtti -fno-exceptions")
add_library(hidapi SHARED src/hidapi/android/hid.cpp)
set(_INSTALL_LIBS "hidapi" ${_INSTALL_LIBS})
target_link_libraries(hidapi log)

And modified the project line to remove the C exclusive (cmake defaults to C/C++)

< project(SDL2 C)

project(SDL2)

Then it builds the library for me. However, now I'm getting a segfault. Not sure yet where that is comming from, could also be my own code at this point. Haven't had time to investigate yet.

On 2019-01-28 21:22:28 +0000, Gabriel Kind wrote:

The problem is that hidapi.so is loaded after SDL2.so which breaks it on Android < 4.4 because the lib loader is not smart enough.

To fix this the "mHIDDeviceManager = HIDDeviceManager.acquire(this);" call in SDLActivity.java must be put before "loadLibraries();".

On 2019-01-29 06:58:09 +0000, Steve Robinson wrote:

How about building hidapi into the existing library or at least making a static lib? I'd like to continue to be able to build down to a standalone binary and not worry about missteps in packaging and deployment breaking my program at runtime.

I've started by building hidapi into sdl2, but simply removing the loadlibrary command for hidapi left me with an error loading HIDDeviceManager implementation. I'm pretty ignorant on the Java layer though. Here's what I did so far:
https://gitlab.com/ssrobins/conan-sdl2/tree/test_patch

Happy to work on fixing this if people agree on the approach.

So far, we've only been talking about Android, but you're using this code on other platforms, right? Is it on a specific subset of platforms or 'all platforms'?

On 2019-01-29 14:44:27 +0000, Gabriel Kind wrote:

I would also prefer to have this as a static library (to get a build-failure on link time and not a crash during runtime) or directly embedded into the SDL2 library as I use a custom Android.mk to build my project, therefore this change was non intuitive at all because it was not mentioned in the changelog.

On 2019-02-11 07:06:31 +0000, Steve Robinson wrote:

I haven't forgotten about this. Got a lot of work done this weekend:
https://gitlab.com/ssrobins/conan-sdl2/tree/test_patch

The diff files are patches against existing files in SDL 2.0.9 and CMakeLists-hidapi.txt is a new CMake file that defines the hdiapi static lib and is referenced from the main SDL2 CMakeLists.txt file. I'm using Conan to automate the patch application, building, and packaging process.

I added hid_init() to the rectangles example and can get everything building on Android, iOS, Linux, Mac, and Windows:
https://gitlab.com/ssrobins/sdl2-example/tree/test_sdl2.0.9

I have a few more changes to make and some testing to do. I haven't been using the joystick functionality so if anyone has some test code they can share it would help a lot.

Is there a mailing list or some other way to find out what's coming before it's released? I would have preferred to help with this before it was released.

On 2019-02-18 03:14:11 +0000, Steve Robinson wrote:

The work to add hidapi to the SDL2 CMake builds is here:
https://gitlab.com/ssrobins/conan-sdl2/tree/test_patch

CMakeLists-hidapi.txt
Rename this file to CMakeLists.txt and put it in src/hidapi. It defines the hidapi static library. For linux, I chose the hidraw hid back-end instead of libusb since it supports USB and Bluetooth. More info: https://github.com/signal11/hidapi

The rest of the changes are defined in diff files from existing files from the 2.0.9 source zip. The file path is defined in the first line of the file.

CMakeLists.diff
This is the main CMake file. This adds the joystick/hidapi/*.c files to SDL2 and references the CMake file that defines the hidapi static lib. There are also several build changes to allow a sample program to build and run with SDL2.

HIDDeviceManager.diff
The code to load a hidapi shared library is removed since the SDL2 CMake file defines hidapi as a static library so loading the SDL2 library or a program that statically linked to the SDL2 library will be sufficient to load the hid native functions. Instead of an unhandled exception if the functions aren't found, I now log to logcat and continue. I did this so if someone does a custom build without hid, they won't get an error over something that does not affect their program.

SDL_config.h.diff
Add SDL_JOYSTICK_HIDAPI so the appropriate code is included when this feature is enabled.

SDL_config_iphoneos.diff
Enable SDL_JOYSTICK_HIDAPI for iOS

To test these changes, I took the rectangles example and built it with the above changes on Windows, Linux, Mac, iOS, and Android:
https://gitlab.com/ssrobins/sdl2-example/tree/test_sdl2.0.9

For all platforms except iOS you can download the built programs here:
https://gitlab.com/ssrobins/sdl2-example/pipelines/47924886
Just click on the platform, then either 'Download' or 'Browse'.

I also used the patched SDL 2.0.9 in a closed source project.

I ran the programs on:
Windows 10 Pro ver. 1809
Ubuntu 16.04 LTS 64-bit
macOS High Sierra 10.13.6
Moto X Pure Edition (Android)
iPhone 4S

Programs loaded and ran fine.

I noticed that the Android program no longer has the navigation buttons at the bottom, while it used to on SDL 2.0.8. This is probably due to setting
SDLActivity.mFullscreenModeActive = true in SDLActivity.java. As far as I can tell it's not related to the hidapi work, but is a noticeable behavior change from SDL 2.0.8.

If anyone has any feedback, let me know, otherwise I'll consider this wrapped up.

On 2019-02-18 15:43:40 +0000, Sam Lantinga wrote:

The hidapi code needs to be built as a shared library in some environments. The default can be static linking, but you need to leave an option for dynamic linking.

Can you please attach any patches you would like to be considered to this bug report?

Thanks!

On 2019-03-18 03:29:36 +0000, Steve Robinson wrote:

Created attachment 3702
CMakeLists-hidapi.txt

On 2019-03-18 03:32:01 +0000, Steve Robinson wrote:

Created attachment 3703
CMakeLists.diff

On 2019-03-18 03:32:57 +0000, Steve Robinson wrote:

Created attachment 3704
HIDDeviceManager.diff

On 2019-03-18 03:33:34 +0000, Steve Robinson wrote:

Created attachment 3705
SDL_config.h.diff

On 2019-03-18 03:34:19 +0000, Steve Robinson wrote:

Created attachment 3706
SDL_config_iphoneos.diff

On 2019-03-18 03:40:03 +0000, Steve Robinson wrote:

Ok, I removed the CMake STATIC designation on the the hidapi add_library function so setting the BUILD_SHARED_LIBS CMake variable to ON will switch it to a shared lib.

If you copy the libhidapi.so into the jniLibs directory in the Android Studio project, it'll get loaded with your program. If you build with a static libhidapi, it'll build into your program and load as well. No other changes from what I described above. All files/patches are attached.

On 2019-03-26 20:57:17 +0000, Anthony @ POW Games wrote:

Is there anything special we need to do to build on Android? I've just tried the latest hg and it's not even compiling now. I'm getting undefined references for new and delete from hid.cpp. I liked things better on all platforms before HID was introduced - it's caused a lot of headaches.

On 2019-03-27 01:51:41 +0000, Steve Robinson wrote:

On Android, I had to make this change to build my program with SDL2/hidapi:
https://gitlab.com/ssrobins/sdl2-example/commit/8d29432ccae1d4b397ca845d431309368056c7e8

There is C++ code in the hidapi Android source. That's pretty unexpected when used with a C library like SDL2.

On 2019-03-27 12:41:57 +0000, Anthony @ POW Games wrote:

Ok, I got the hg to compile now (I'd left a "APP_STL := none" in there), but the original problem still persists.

Just to clarify that I'm building with Android Studio (not cmake). The APK builds and runs no problem on newer Android versions, but not on 4.2.2, where the run-time error pop-up occurs.

I've applied Steve's patch but I still get the same error.

On 2019-03-28 02:11:34 +0000, Steve Robinson wrote:

I installed Rectangles-debug.apk on a Samsung Galaxy Nexus running Android 4.3 from:
https://gitlab.com/ssrobins/sdl2-example/-/jobs/179180408/artifacts/browse/build_android-v7a/_package/
Which is using the patched SDL 2.0.9 with hidapi statically linked.

...and it worked fine. That's as old as I got though, so if it's something that's specific to 4.2, I can't help.

Here's more info in case you want to dig into this example:

I build the native code with CMake, but I use this Gradle project template to build the apk:
https://gitlab.com/ssrobins/sdl2-example/tree/test_sdl2.0.9/Android

If you clone the repo and run build_android-v7a.sh it'll try to build the rectangles example:
https://gitlab.com/ssrobins/sdl2-example/tree/test_sdl2.0.9
Be sure to use the test_sdl2.0.9 branch.

The Gradle project gets deposited here:
build_android-v7a/Rectangles/Android/
From there you should be able to open it in Android Studio and use it like any other project. If it works for you, then at least you know it's not SDL 2.0.9, but some sort of build config issue.

On 2019-03-28 15:11:15 +0000, Anthony @ POW Games wrote:

Steve, thanks - I appreciate your hard work on this.

I've tried your apk and it does work on my 4.2.2. But have you accessed the hidapi in this example?

I'm going to compare your project to mine with a fine-tooth comb and see if there's a difference that'll fix it.

On 2019-03-28 15:37:57 +0000, Steve Robinson wrote:

I simply call hid_init() in main():
https://gitlab.com/ssrobins/sdl2-example/blob/test_sdl2.0.9/Rectangles/rectangles.c

On 2019-03-28 16:05:19 +0000, Anthony @ POW Games wrote:

I think I'm missing something here. hidapi.h is burried deep within the SDL2 source directory. There's no way I can include it - it will break my windows build unless I add #ifdefs?

On 2019-03-28 16:22:53 +0000, Anthony @ POW Games wrote:

I don't think this APK is a good example of if it's working or not unless #include "hidapi.h" is removed and replaced with SDL_Init(SDL_INIT_JOYSTICKS); We're supposed to be using SDL as a layer and not using hidapi directly?

On 2019-03-28 16:43:50 +0000, Steve Robinson wrote:

Looked for documentation on how to actually use hidapi within SDL2, didn't find any so I just added code that would definitely call symbols from the hidapi library in the source for a quick test.

I can try SDL_Init(SDL_INIT_JOYSTICKS); later tonight, though I don't have any hardware to test it with.

The hidapi public header is getting published with all the other SDL2 public headers when running 'cmake install' so it took no additional header directories to get it building across several platforms:
https://gitlab.com/ssrobins/sdl2-example/pipelines/52279751

Even if you included hidapi.h from its original directory, why would you need #ifdefs and why would it only break on Windows? While hidapi uses different source files for each platform, there's just the one hidapi.h file, right?

On 2019-03-28 17:07:36 +0000, Anthony @ POW Games wrote:

I get 'hidapi.h': No such file or directory errors if I try to include it because it doesn't exist anywhere other than the SDL2 source code directory src/hidapi/hidapi/hidapi.h. I can add that directory to my includes for Android, but I don't build in Windows from SDL2 source, so it won't work. I don't think you're supposed to include it or access hid_init() directly.

Knowing if the apk works when SDL_INIT_JOYSTICKS is used will be the tell-all. Actual joysticks not needed for it to crash.

I'm now wondering what wizardry you use to get it to build. I can't really understand your project setup.

On 2019-03-28 17:19:45 +0000, Steve Robinson wrote:

I use Conan Package Manager (https://conan.io/) to package SDL2 via this repo:
https://gitlab.com/ssrobins/conan-sdl2/tree/test_patch

The CI process is defined in .gitlab-ci.yml (links to https://gitlab.com/ssrobins/ci/blob/master/conan.yml) so you can follow exactly what it's doing to build on each platform.

It downloads the source from the SDL2 website, patches it, does the CMake build, and then the CMake install. Finally, it pushes the package to the package server.

In https://gitlab.com/ssrobins/sdl2-example/tree/test_sdl2.0.9, it pulls down the SDL2 package, runs CMake to generate the project, and then builds. Again, .gitlab-ci.yml defines the CI process.

I'll try SDL_INIT_JOYSTICKS tonight and post an update.

On 2019-03-28 17:55:26 +0000, Anthony @ POW Games wrote:

I think I found why hidapi will never work on 4.2.2

SDL2 sets the minSdkVersion to 16.

The HID java files (HIDDeviceManager.java, HIDDeviceUSB.java, etc) use stuff that was introduced in API 18. 4.2.2 is API level 17.

The only way to solve this is raise SDL2 min API to 18, or remove HIDAPI. I vote for the latter, HIDAPI is causing a lot of problems and offers very little.

On 2019-03-29 03:47:08 +0000, Steve Robinson wrote:

Created attachment 3717
CMakeLists-hidapi.txt

On 2019-03-29 03:56:52 +0000, Steve Robinson wrote:

I've updated CMakeLists-hidapi.txt to longer treat hidapi.h as a public dependency.

I now call SDL_Init(SDL_INIT_JOYSTICK) rather than hid_init() in the example:
https://gitlab.com/ssrobins/sdl2-example/blob/test_sdl2.0.9/Rectangles/rectangles.c

It still works! You can download my apk here:
https://gitlab.com/ssrobins/sdl2-example/-/jobs/186468940/artifacts/browse/build_android-v7a/_package/

In my example, I set minSdkVersion to 15 and don't seem to run into any problems at build or runtime.

On 2019-03-29 11:12:32 +0000, Anthony @ POW Games wrote:

Steve, thanks again - I now have a much better understanding of what's going on. Here's what I've found:

Your project links statically, so you've bypassed the "could not load library libhidapi.so" error. Your new APK still runs on 4.2.2, but looking at Logcat I'm seeing a lot of errors relating to the HID. Because the HID java code uses API 18 classes and calls, the HID functionality won't work. Here's a snippet of the errors on Logcat:

E/dalvikvm: Could not find class 'android.bluetooth.BluetoothManager', referenced from method org.libsdl.app.HIDDeviceManager.initializeBluetooth

E/dalvikvm: Could not find class 'org.libsdl.app.HIDDeviceBLESteamController', referenced from method org.libsdl.app.HIDDeviceManager.connectBluetoothDevice

And when I plug a USB Xbox controller in:

I/dalvikvm: Could not find method android.hardware.usb.UsbDeviceConnection.controlTransfer, referenced from method org.libsdl.app.HIDDeviceUSB.getFeatureReport

None of the HID native callbacks are working:

W/dalvikvm: VFY: unable to resolve virtual method 105: Landroid/hardware/usb/UsbDeviceConnection;.controlTransfer (IIII[BIII)I

So the problem is still that the new HID Java code isn't compatible with Jellybean (4.2) and below, so the Min APK needs to be raised to 18 to reflect this. The "could not load library libhidapi.so" error is just the first part of a bigger problem.

On 2019-03-29 14:17:26 +0000, Steve Robinson wrote:

Are you using the HIDDeviceManager.java patch?
https://gitlab.com/ssrobins/conan-sdl2/blob/test_patch/HIDDeviceManager.diff

It removes the error when you can't load libhidapi.so since it may be statically linked and it logs an error if HIDDeviceRegisterCallback() fails with the UnsatisfiedLinkError exception, but continues.

On 2019-03-29 14:47:17 +0000, Anthony @ POW Games wrote:

Even with your patch, the main problem will still persist with the API 18 calls failing on Android 4.2. The Logcat errors I showed you were from your APK.

On 2019-03-29 15:39:08 +0000, Anthony @ POW Games wrote:

To clarify: the libhidapi.so SDL_Error popup happens before it reaches HIDDeviceRegisterCallback(). It works on Android versions 4.3 and above, and only fails on 4.2 and below.

On my app which doesn't statically link (and works on 4.3+) this is what Logcat says when I run it on 4.2.

D/dalvikvm: Trying to load lib /data/app-lib/uk.co.powgames.PowBallRen-1/libSDL2.so 0x411783a8

E/dalvikvm: dlopen("/data/app-lib/uk.co.powgames.PowBallRen-1/libSDL2.so") failed: Cannot load library: soinfo_link_image(linker.cpp:1635): could not load library "libhidapi.so" needed by "libSDL2.so"; caused by load_library(linker.cpp:745): library "libhidapi.so" not found

W/System.err: Cannot load library: soinfo_link_image(linker.cpp:1635): could not load library "libhidapi.so" needed by "libSDL2.so"; caused by load_library(linker.cpp:745): library "libhidapi.so" not found

liphadapi.so isn't missing because it loads ok on Android 4.3+. I'm sure this is caused by API 18 being used and non-existent on Android 4.2. Even if it loaded, the functionality will fail after.

On 2019-03-29 16:22:51 +0000, Anthony @ POW Games wrote:

I just added this and it worked - no patches needed (facepalm).

@Override
protected String[] getLibraries() {
    return new String[]{
            "hidapi",
            "SDL2",
            // "SDL2_image",
            "SDL2_mixer",
            // "SDL2_net",
            // "SDL2_ttf",
            "main"
    };
}

On 2019-04-02 07:16:18 +0000, Sylvain wrote:

Not sure what is exactly the issue but:

if the c/cpp shared library doesn't load, maybe :

  • you haven't provided the correct ARCH (armv7a, arm64, x86, etc).

  • it's been compiled with wrong APP_PLATFORM settings:
    it should be: APP_PLATFORM=android-16
    and "minSdkVersion 16" in your build.gradle
    (or make sure: APP_PLATFORM matches minSdkVersion).

    otherwise you'll start to get strange unresolved symbol "atof" etc.

    different combinations are possible lower/greater, but it's more complicated and different purpose.

  • if .java files crash when calling an unresolved API, this would be a mistake, and it needs to be protected and handled with some java code like :
    if (Build.VERSION.SDK_INT >= 26) { ... }

hope it helps

On 2019-04-17 03:01:24 +0000, Sam Lantinga wrote:

Good fix, I added this to the SDL sample project:
https://hg.libsdl.org/SDL/rev/7d349998be25

Thanks!

On 2019-04-18 01:14:36 +0000, Steve Robinson wrote:

How about the CMake changes, are those making it in?

On 2019-04-18 23:23:14 +0000, Sam Lantinga wrote:

Can you create a separate bug for the proposed CMake changes?

On 2019-04-19 02:15:16 +0000, Steve Robinson wrote:

New bug created for the CMake stuff:
https://bugzilla.libsdl.org/show_bug.cgi?id=4601

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant