D8173: Use readelf to find project dependencies
This revision was automatically updated to reflect the committed changes. Closed by commit R240:c91c3650968d: Use readelf to find project dependencies (authored by apol). REPOSITORY R240 Extra CMake Modules CHANGES SINCE LAST UPDATE https://phabricator.kde.org/D8173?vs=24185&id=24192 REVISION DETAIL https://phabricator.kde.org/D8173 AFFECTED FILES tests/CMakeLists.txt tests/ECMToolchainAndroidTest/CMakeLists.txt tests/ECMToolchainAndroidTest/main.c tests/ECMToolchainAndroidTest/testlinkfile/CMakeFiles/testtarget.dir/link.txt tests/ECMToolchainAndroidTest/testlinkfile/outputfake.json toolchain/Android.cmake toolchain/specifydependencies.cmake To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
aacid accepted this revision. This revision is now accepted and ready to land. REPOSITORY R240 Extra CMake Modules BRANCH arcpatch-D8173_2 REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
apol updated this revision to Diff 24185. apol added a comment. Went to albert's place and figured it out REPOSITORY R240 Extra CMake Modules CHANGES SINCE LAST UPDATE https://phabricator.kde.org/D8173?vs=22498&id=24185 BRANCH arcpatch-D8173_2 REVISION DETAIL https://phabricator.kde.org/D8173 AFFECTED FILES tests/CMakeLists.txt tests/ECMToolchainAndroidTest/CMakeLists.txt tests/ECMToolchainAndroidTest/main.c tests/ECMToolchainAndroidTest/testlinkfile/CMakeFiles/testtarget.dir/link.txt tests/ECMToolchainAndroidTest/testlinkfile/outputfake.json toolchain/Android.cmake toolchain/specifydependencies.cmake To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
mart added a comment. here the change works perfectly.. right now, we have building on android broken for everybody, since months, this is not acceptable :/ REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
apol added a comment.
Okay, I gave it another unsuccesful go. Will have to test together if you
have some time.
I tried putting everything in a different prefix like this:
KConfig:
cmake -DCMAKE_INSTALL_PREFIX=/home/user/prefixfw
-DCMAKE_PREFIX_PATH="$QT_ANDROID;/opt/kdeandroid-deps" -DCMAKE_BUILD_TYPE=debug
-DCMAKE_TOOLCHAIN_FILE=/opt/kdeandroid-deps/share/ECM/toolchain/Android.cmake
-DKF5_HOST_TOOLING=/opt/nativetooling/lib/x86_64-linux-gnu/cmake/
-DANDROID_SDK_BUILD_TOOLS_REVISION=21.1.2 /home/user/kconfig/
KTuberling:
cmake -DCMAKE_INSTALL_PREFIX=/home/user/prefixapp
-DECM_ADDITIONAL_FIND_ROOT_PATH="$QT_ANDROID;/opt/kdeandroid-deps;/home/user/prefixfw"
-DCMAKE_BUILD_TYPE=debug
-DCMAKE_TOOLCHAIN_FILE=/opt/kdeandroid-deps/share/ECM/toolchain/Android.cmake
-DKF5_HOST_TOOLING=/opt/nativetooling/lib/x86_64-linux-gnu/cmake/
-DANDROID_SDK_BUILD_TOOLS_REVISION=21.1.2
-DQTANDROID_EXPORTED_TARGET=ktuberling_mobile
-DANDROID_APK_DIR=/home/user/ktuberling/android_data/ /home/user/ktuberling/
Successfully found kconfig:
~/build/ktuberling$ cat ktuberling_mobile-deployment.json
{
"qt": "/opt/Qt/5.9.1/android_armv7",
"sdk": "/opt/android-sdk-linux",
"ndk": "/opt/android-ndk",
"toolchain-prefix": "arm-linux-androideabi",
"tool-prefix": "arm-linux-androideabi",
"toolchain-version": "4.9",
"ndk-host": "linux-x86_64",
"target-architecture": "armeabi-v7a",
"application-binary":
"/home/user/build/ktuberling/ktuberling_mobile_build_apk//libs/armeabi-v7a/libktuberling_mobile.so",
"qml-root-path": "/home/user/ktuberling",
"android-extra-libs": "/home/user/prefixfw/lib/libKF5ConfigCore.so",
"android-extra-plugins": "/home/user/prefixapp/share",
"android-package-source-directory":
"/home/user/ktuberling/android_data/",
"sdkBuildToolsRevision": "21.1.2"
}
REPOSITORY
R240 Extra CMake Modules
REVISION DETAIL
https://phabricator.kde.org/D8173
To: apol, #frameworks, #build_system, aacid
Cc: mart
D8173: Use readelf to find project dependencies
aacid added a comment. still not working in my setup the setenv seems to not be working and complains with - could not find 0x0001 (NEEDED) Shared library: [libKF5ConfigCore.so] in /home/tsdgeos/devel/kde/ktuberling/export//lib or /home/tsdgeos/devel/kde/ktuberling/build_android which is true, the library is not in any of those two paths REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
apol updated this revision to Diff 22498. apol added a comment. Added "" around a variable that may contain ; REPOSITORY R240 Extra CMake Modules CHANGES SINCE LAST UPDATE https://phabricator.kde.org/D8173?vs=22496&id=22498 BRANCH arcpatch-D8173_1 (branched from master) REVISION DETAIL https://phabricator.kde.org/D8173 AFFECTED FILES tests/CMakeLists.txt tests/ECMToolchainAndroidTest/CMakeLists.txt tests/ECMToolchainAndroidTest/main.c tests/ECMToolchainAndroidTest/testlinkfile/CMakeFiles/testtarget.dir/link.txt tests/ECMToolchainAndroidTest/testlinkfile/outputfake.json toolchain/Android.cmake toolchain/specifydependencies.cmake To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
apol updated this revision to Diff 22496. apol added a comment. Pass LANG=C to readelf so it doesn't appear in catalan (or anything other than C) and pass ECM_ADDITIONAL_FIND_ROOT_PATH to look for dependencies REPOSITORY R240 Extra CMake Modules CHANGES SINCE LAST UPDATE https://phabricator.kde.org/D8173?vs=21777&id=22496 BRANCH arcpatch-D8173_1 (branched from master) REVISION DETAIL https://phabricator.kde.org/D8173 AFFECTED FILES tests/CMakeLists.txt tests/ECMToolchainAndroidTest/CMakeLists.txt tests/ECMToolchainAndroidTest/main.c tests/ECMToolchainAndroidTest/testlinkfile/CMakeFiles/testtarget.dir/link.txt tests/ECMToolchainAndroidTest/testlinkfile/outputfake.json toolchain/Android.cmake toolchain/specifydependencies.cmake To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
apol added a comment.
In https://phabricator.kde.org/D8173#167835, @aacid wrote:
> In https://phabricator.kde.org/D8173#167776, @apol wrote:
>
> > I either need more information
>
>
> I'm happy to provide any information you want, i just don't know what you
need other than KConfigCore.so is in the apk without your patch and not with
your patch.
Here's a little patch that you can apply and will offer a bit more
information:
diff --git a/toolchain/specifydependencies.cmake
b/toolchain/specifydependencies.cmake
index 60b7e94..3406d1e 100644
--- a/toolchain/specifydependencies.cmake
+++ b/toolchain/specifydependencies.cmake
@@ -1,3 +1,4 @@
+message(STATUS "executing: readelf --wide --dynamic ${TARGET}")
execute_process(COMMAND readelf --wide --dynamic ${TARGET} ERROR_VARIABLE
readelf_errors OUTPUT_VARIABLE out RESULT_VARIABLE result)
if (NOT result EQUAL 0)
@@ -11,13 +12,16 @@ foreach(line ${lines})
if(matched)
if(EXISTS "${OUTPUT_DIR}/${CMAKE_MATCH_1}")
list(APPEND extralibs "${OUTPUT_DIR}/${CMAKE_MATCH_1}")
-endif()
-if(EXISTS "${EXPORT_DIR}/lib/${CMAKE_MATCH_1}")
+elseif(EXISTS "${EXPORT_DIR}/lib/${CMAKE_MATCH_1}")
list(APPEND extralibs "${EXPORT_DIR}/lib/${CMAKE_MATCH_1}")
+else()
+message(STATUS "could not find ${line} in ${EXPORT_DIR}/lib or
${OUTPUT_DIR}")
endif()
endif()
endforeach()
+message(STATUS "extra libs: ${extralibs}")
+
if(extralibs)
string(REPLACE ";" "," libs "${extralibs}")
set(extralibs "\"android-extra-libs\": \"${libs}\",")
You should get something like (this is a valid output, which you aren't
having):
Generating ktuberling_mobile with
/opt/Qt/5.9.1/android_armv7/bin/androiddeployqt
-- executing: readelf --wide --dynamic
/home/user/build/ktuberling/ktuberling_mobile
-- could not find 0x0001 (NEEDED) Shared library:
[libgnustl_shared.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[liblog.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libz.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libdl.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libQt5Svg.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libQt5Multimedia.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libQt5Xml.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libQt5Widgets.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libQt5Gui.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libQt5Network.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libQt5Core.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libstdc++.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libm.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- could not find 0x0001 (NEEDED) Shared library:
[libc.so] in /opt/kdeandroid-deps/lib or /home/user/build/ktuberling
-- extra libs: /opt/kdeandroid-deps/lib/libKF5ConfigCore.so
Generating Android Package
Then we see how to figure it out.
REPOSITORY
R240 Extra CMake Modules
REVISION DETAIL
https://phabricator.kde.org/D8173
To: apol, #frameworks, #build_system, aacid
Cc: mart
D8173: Use readelf to find project dependencies
aacid added a comment. In https://phabricator.kde.org/D8173#167776, @apol wrote: > I either need more information I'm happy to provide any information you want, i just don't know what you need other than KConfigCore.so is in the apk without your patch and not with your patch. REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
apol added a comment. I've done a build of with and without and I don't see such differences: $ zipinfo ktuberling-patch.apk > patch $ less patch $ zipinfo ktuberling-nopatch.apk > nopatch $ diff patch nopatch 1,2c1,2 < Archive: ktuberling-patch.apk < Zip file size: 31726983 bytes, number of entries: 15236 --- > Archive: ktuberling-nopatch.apk > Zip file size: 31725709 bytes, number of entries: 15235 5,6c5,6 < -rw 2.4 fat 1750077 b- defN 80-000-00 00:00 META-INF/CERT.SF < -rw 2.4 fat 1750034 b- defN 80-000-00 00:00 META-INF/MANIFEST.MF --- > -rw 2.4 fat 1749983 b- defN 80-000-00 00:00 META-INF/CERT.SF > -rw 2.4 fat 1749940 b- defN 80-000-00 00:00 META-INF/MANIFEST.MF 8c8 < -rw 2.4 fat 715326 b- defN 80-000-00 00:00 assets/--Added-by-androiddeployqt--/qt_cache_pregenerated_file_list --- > -rw 2.4 fat 715294 b- defN 80-000-00 00:00 assets/--Added-by-androiddeployqt--/qt_cache_pregenerated_file_list 75,76c75 < -rw 2.4 fat11733 b- defN 80-000-00 00:00 assets/share/ECM/toolchain/Android.cmake --- > -rw 2.4 fat11723 b- defN 80-000-00 00:00 assets/share/ECM/toolchain/Android.cmake 78c77 < -rw 2.4 fat 1551 b- defN 80-000-00 00:00 assets/share/ECM/toolchain/specifydependencies.cmake --- > -rw 2.4 fat 1618 b- defN 80-000-00 00:00 assets/share/ECM/toolchain/specifydependencies.cmake 15238,15239c15237,15238 < -rw-rw-rw- 2.3 unx 1961236 b- stor 80-000-00 00:00 resources.arsc < 15236 files, 63480003 bytes uncompressed, 28709886 bytes compressed: 54.8% --- > -rw-rw-rw- 2.3 unx 1961148 b- stor 80-000-00 00:00 resources.arsc > 15235 files, 63478118 bytes uncompressed, 28708802 bytes compressed: 54.8% I either need more information or we go forward with this and we set infrastructure in the CI to make sure this issue doesn't happen. REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
aacid added a comment. I'm still not getting libkconfigcore.so into the apk with this REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
mart added a comment. +1 REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid Cc: mart
D8173: Use readelf to find project dependencies
apol updated this revision to Diff 21777. apol added a comment. Make sure we get the dependencies from the prefix as well REPOSITORY R240 Extra CMake Modules CHANGES SINCE LAST UPDATE https://phabricator.kde.org/D8173?vs=20434&id=21777 BRANCH arcpatch-D8173 REVISION DETAIL https://phabricator.kde.org/D8173 AFFECTED FILES tests/CMakeLists.txt tests/ECMToolchainAndroidTest/CMakeLists.txt tests/ECMToolchainAndroidTest/main.c tests/ECMToolchainAndroidTest/testlinkfile/CMakeFiles/testtarget.dir/link.txt tests/ECMToolchainAndroidTest/testlinkfile/outputfake.json toolchain/Android.cmake toolchain/specifydependencies.cmake To: apol, #frameworks, #build_system, aacid
D8173: Use readelf to find project dependencies
aacid requested changes to this revision. aacid added a comment. This revision now requires changes to proceed. This breaks KTuberling build for me, libKF5ConfigCore.so is not added to the apk anymore. Two random guesses since i'm not sure i understood the code enough: - does readelf need to be "the android sdk one" or any readelf works? - if readelf is being executed in cmake time isn't that too early since the binary (i.e. ktuberling_mobile) doesn't exist yet and you can't readelf it? Please ignore me if i'm saying stupid stuff :D REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid
D8173: Use readelf to find project dependencies
apol added a reviewer: aacid. REPOSITORY R240 Extra CMake Modules REVISION DETAIL https://phabricator.kde.org/D8173 To: apol, #frameworks, #build_system, aacid
D8173: Use readelf to find project dependencies
apol updated this revision to Diff 20434. apol added a comment. remove unrelated change I introduced REPOSITORY R240 Extra CMake Modules CHANGES SINCE LAST UPDATE https://phabricator.kde.org/D8173?vs=20430&id=20434 BRANCH master REVISION DETAIL https://phabricator.kde.org/D8173 AFFECTED FILES tests/CMakeLists.txt tests/ECMToolchainAndroidTest/CMakeLists.txt tests/ECMToolchainAndroidTest/main.c tests/ECMToolchainAndroidTest/testlinkfile/CMakeFiles/testtarget.dir/link.txt tests/ECMToolchainAndroidTest/testlinkfile/outputfake.json toolchain/Android.cmake toolchain/specifydependencies.cmake To: apol, #frameworks, #build_system
D8173: Use readelf to find project dependencies
apol created this revision. apol added reviewers: Frameworks, Build System. Restricted Application added projects: Frameworks, Build System. REVISION SUMMARY We were using a link.txt file that cmake used to generate, on newer cmake versions it doesn't anymore. Instead use readelf, much like androiddeployqt does, to extract the depenencies. Catch: It relies on having all the binaries being at the same subdirectory, which is the default in ECM since not long ago. TEST PLAN Build kirigamigallery with it REPOSITORY R240 Extra CMake Modules BRANCH master REVISION DETAIL https://phabricator.kde.org/D8173 AFFECTED FILES tests/CMakeLists.txt tests/ECMToolchainAndroidTest/CMakeLists.txt tests/ECMToolchainAndroidTest/main.c tests/ECMToolchainAndroidTest/testlinkfile/CMakeFiles/testtarget.dir/link.txt tests/ECMToolchainAndroidTest/testlinkfile/outputfake.json toolchain/Android.cmake toolchain/specifydependencies.cmake To: apol, #frameworks, #build_system
