[
https://issues.apache.org/jira/browse/IMPALA-12955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088091#comment-18088091
]
ASF subversion and git services commented on IMPALA-12955:
----------------------------------------------------------
Commit d19d209b63928daaea3029a8d71cf5fdf4583a94 in impala's branch
refs/heads/master from Aleksandr Efimov
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d19d209b6 ]
IMPALA-12955: Slim down profile tool deps
Use the RuntimeProfile source split from the parent change to link
impala-profile-tool against smaller dependency subsets. The tool no
longer pulls in unused runtime libraries such as HDFS, JVM, Kudu,
SASL, Kerberos, SSL, and crypto dependencies.
The profile tool now links ProfileToolThrift, ProfileToolUtil, and
StatusMinimal. Keep CMake source lists shared with the full targets to
avoid drift, and keep StatusMinimal in the global Impala link group so
normal backend binaries do not depend on transitive link ordering.
Move GetStackTrace() into util/stack-trace.* so StatusMinimal and Util
can share it without linking debug-util.cc into the profile tool. Move
Status thrift/proto/HS2 conversions out of the Status class into
status-serialization.* free functions.
The archive reader uses local base64 length validation, base64
unescaping, zlib inflate, and local thrift deserialization to avoid
pulling coding-util.cc and thrift-util.cc dependencies into the
profile tool. Error messages distinguish invalid base64 length from
invalid base64 data.
The Linux dependency test checks both direct NEEDED entries and
transitive ldd output so regressions through shared helper libraries
are caught.
Tests:
- git diff --check
- python3 -m py_compile \
tests/observability/test_profile_tool_dependencies.py
- ./buildall.sh -notests -noclean -cmake_only
- make -C be/src/util impala-profile-tool -j 64
- make Catalog Runtime Exec Scheduling Service Statestore -j 64
- bin/create-test-configuration.sh
- bin/impala-py.test -q tests/observability/test_profile_tool.py
- ./buildall.sh -release -so -notests -noclean -cmake_only
- make -C be/src/util impala-profile-tool -j 64
- be/build/latest/util/impala-profile-tool --help
- bin/impala-py.test -q \
tests/observability/test_profile_tool_dependencies.py
Assisted-by: OpenAI Codex
Change-Id: Id476411dcf6c46079e2e2addc5cdd552bc23f8a1
Reviewed-on: http://gerrit.cloudera.org:8080/24358
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> impala-profile-tool depends on unused so files
> ----------------------------------------------
>
> Key: IMPALA-12955
> URL: https://issues.apache.org/jira/browse/IMPALA-12955
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Assignee: Aleksandr Efimov
> Priority: Major
>
> When I want to deploy impala-profile-tool, I have to copy lots of so files.
> Some of them are actually unused, e.g. libjvm.so, libkudu_client.so.0. The
> full dependency list:
> {noformat}
> $ ldd be/build/debug/util/impala-profile-tool
> linux-vdso.so.1 (0x00007ffcb5dae000)
> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> (0x00007fca6dd63000)
> libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2
> (0x00007fca6db48000)
> libjvm.so =>
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> (0x00007fca6cade000)
> libkudu_client.so.0 =>
> /home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/debug/lib/libkudu_client.so.0
> (0x00007fca6c1db000)
> librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fca6bfd3000)
> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fca6bdcf000)
> libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3
> (0x00007fca6baf9000)
> libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2
> (0x00007fca6b8ae000)
> libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1
> (0x00007fca6b621000)
> libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
> (0x00007fca6b156000)
> libstdc++.so.6 =>
> /home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/debug/lib/libstdc++.so.6
> (0x00007fca6dfb6000)
> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fca6adb8000)
> libgcc_s.so.1 =>
> /home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/debug/lib/libgcc_s.so.1
> (0x00007fca6ad9e000)
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fca6a9ad000)
> /lib64/ld-linux-x86-64.so.2 (0x00007fca6df82000)
> libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3
> (0x00007fca6a77b000)
> libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2
> (0x00007fca6a577000)
> libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0
> (0x00007fca6a36c000)
> libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1
> (0x00007fca6a168000)
> libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2
> (0x00007fca69f4e000) {noformat}
> We should remove unused so files in the dependency.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
