[
https://issues.apache.org/jira/browse/HADOOP-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17797928#comment-17797928
]
ASF GitHub Bot commented on HADOOP-19001:
-----------------------------------------
hadoop-yetus commented on PR #6367:
URL: https://github.com/apache/hadoop/pull/6367#issuecomment-1859184739
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 17m 14s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 22s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 38m 0s | | trunk passed |
| +1 :green_heart: | compile | 18m 23s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 48s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | mvnsite | 2m 34s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 12s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 48s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 41m 30s | | branch has no errors
when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 17m 22s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 22s | | the patch passed |
| +1 :green_heart: | compile | 16m 33s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 16m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 2m 29s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 4s | | No new issues. |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 47s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 42m 56s | | patch has no errors
when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 19m 41s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 5m 40s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not
generate ASF License warnings. |
| | | 261m 52s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6367/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6367 |
| Optional Tests | dupname asflicense mvnsite unit codespell detsecrets
shellcheck shelldocs compile javac javadoc mvninstall shadedclient xmllint |
| uname | Linux 5fe6c806baf5 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 723d10ba6f5a32bc6b0b3c3e427ae2e8ede70633 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6367/1/testReport/ |
| Max. process+thread count | 2144 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6367/1/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> LD_LIBRARY_PATH is missing HADOOP_COMMON_LIB_NATIVE_DIR
> -------------------------------------------------------
>
> Key: HADOOP-19001
> URL: https://issues.apache.org/jira/browse/HADOOP-19001
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.2.4
> Reporter: Zilong Zhu
> Priority: Major
> Labels: pull-request-available
>
> When we run a spark job, we find that it cannot load the native library
> successfully.
> We found a difference between hadoop2 and hadoop3.
> hadoop2-Spark-System Properties：
> |java.library.path|:/hadoop/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib|
> hadoop3-Spark-System Properties：
> |java.library.path|:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib|
> The key point is：
> hadoop2-hadoop-config.sh：
> HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH" <--267
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH <--268
>
> hadoop3-hadoop-functions.sh:
> hadoop_add_param HADOOP_OPTS java.library.path \
> "-Djava.library.path=${JAVA_LIBRARY_PATH}"
> export LD_LIBRARY_PATH <--1484
>
> At the same time, the hadoop3 will clear all non-whitelisted environment
> variables.
> I'm not sure if it was intentional. But it makes our spark job unable to find
> the native library on hadoop3.
> Maybe we should modify hadoop-functions.sh(1484) and add LD_LIBRARY_PATH to
> the default configuration item yarn.nodemanager.env-whitelist.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
