[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 8: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 8 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Wed, 10 Apr 2024 20:37:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs During TSAN runs all Impala binaries (including libfesupport.so) are built with TSAN options, which include a reference to the external symbol __tsan_init. This causes a problem for libfesupport.so when it is loaded into Hive during minicluster startup, because the Java VM running Hive's code cannot supply this symbol (the stock JVM is obviously not built with TSAN). Unfortunately this symbol resolution failure causes Hive's JVM simply to abort on Red Hat 8 (or later) and on Ubuntu 20.04 (or later). On earlier versions of the same platforms the JVM turned the same failure into an UnsatisfiedLinkError exception, which is actually handled by Hive. This patch prevents libfesupport.so from being loaded into Hive for TSAN runs so that the minicluster can actually be started. This is achieved by not adding the directory containing libfesupport.so to JAVA_LIBRARY_PATH, preventing the JVM from finding it. Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Reviewed-on: http://gerrit.cloudera.org:8080/21191 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M testdata/bin/run-hive-server.sh 1 file changed, 19 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 9 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10523/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 8 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Wed, 10 Apr 2024 15:29:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 8 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Wed, 10 Apr 2024 15:29:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15837/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 6 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Tue, 09 Apr 2024 22:10:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 7 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Tue, 09 Apr 2024 21:52:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 7: Code-Review+1 Carry +1 votes -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 7 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Tue, 09 Apr 2024 21:48:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/21191/5/testdata/bin/run-hive-server.sh File testdata/bin/run-hive-server.sh: http://gerrit.cloudera.org:8080/#/c/21191/5/testdata/bin/run-hive-server.sh@175 PS5, Line 175: # TSAN and FULL_TSAN runs are detected by reading the build type from > ^ Done -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 7 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Tue, 09 Apr 2024 21:47:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Hello Joe McDonnell, Michael Smith, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21191 to look at the new patch set (#6). Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs During TSAN runs all Impala binaries (including libfesupport.so) are built with TSAN options, which include a reference to the external symbol __tsan_init. This causes a problem for libfesupport.so when it is loaded into Hive during minicluster startup, because the Java VM running Hive's code cannot supply this symbol (the stock JVM is obviously not built with TSAN). Unfortunately this symbol resolution failure causes Hive's JVM simply to abort on Red Hat 8 (or later) and on Ubuntu 20.04 (or later). On earlier versions of the same platforms the JVM turned the same failure into an UnsatisfiedLinkError exception, which is actually handled by Hive. This patch prevents libfesupport.so from being loaded into Hive for TSAN runs so that the minicluster can actually be started. This is achieved by not adding the directory containing libfesupport.so to JAVA_LIBRARY_PATH, preventing the JVM from finding it. Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 --- M testdata/bin/run-hive-server.sh 1 file changed, 19 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/21191/6 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 6 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 5: Code-Review+1 This looks good to me -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 5 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Tue, 09 Apr 2024 05:03:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 5: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/21191/5/testdata/bin/run-hive-server.sh File testdata/bin/run-hive-server.sh: http://gerrit.cloudera.org:8080/#/c/21191/5/testdata/bin/run-hive-server.sh@175 PS5, Line 175: # TSAN and FULL_TSAN runs are detected by reading the build type from the .cmake_build_type > line too long (91 > 90) ^ -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 5 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 08 Apr 2024 18:14:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15800/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 5 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sun, 07 Apr 2024 21:26:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Hello Joe McDonnell, Michael Smith, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21191 to look at the new patch set (#5). Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs During TSAN runs all Impala binaries (including libfesupport.so) are built with TSAN options, which include a reference to the external symbol __tsan_init. This causes a problem for libfesupport.so when it is loaded into Hive during minicluster startup, because the Java VM running Hive's code cannot supply this symbol (the stock JVM is obviously not built with TSAN). Unfortunately this symbol resolution failure causes Hive's JVM simply to abort on Red Hat 8 (or later) and on Ubuntu 20.04 (or later). On earlier versions of the same platforms the JVM turned the same failure into an UnsatisfiedLinkError exception, which is actually handled by Hive. This patch prevents libfesupport.so from being loaded into Hive for TSAN runs so that the minicluster can actually be started. This is achieved by not adding the directory containing libfesupport.so to JAVA_LIBRARY_PATH, preventing the JVM from finding it. Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 --- M testdata/bin/run-hive-server.sh 1 file changed, 18 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/21191/5 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 5 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Laszlo Gaal has posted comments on this change. (
http://gerrit.cloudera.org:8080/21191 )
Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the
minicluster during TSAN runs
..
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21191/4/buildall.sh
File buildall.sh:
http://gerrit.cloudera.org:8080/#/c/21191/4/buildall.sh@344
PS4, Line 344: if [[ ${BUILD_DEBUG_NOOPT} -eq 1 ]]; then
> Randomly added, remove.
Done
--
To view, visit http://gerrit.cloudera.org:8080/21191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20
Gerrit-Change-Number: 21191
Gerrit-PatchSet: 5
Gerrit-Owner: Laszlo Gaal
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Laszlo Gaal
Gerrit-Reviewer: Michael Smith
Gerrit-Comment-Date: Sun, 07 Apr 2024 21:05:48 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/21191/5/testdata/bin/run-hive-server.sh File testdata/bin/run-hive-server.sh: http://gerrit.cloudera.org:8080/#/c/21191/5/testdata/bin/run-hive-server.sh@175 PS5, Line 175: # TSAN and FULL_TSAN runs are detected by reading the build type from the .cmake_build_type line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 5 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sun, 07 Apr 2024 21:04:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15797/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sun, 07 Apr 2024 13:36:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/21191/3/testdata/bin/run-hive-server.sh File testdata/bin/run-hive-server.sh: http://gerrit.cloudera.org:8080/#/c/21191/3/testdata/bin/run-hive-server.sh@175 PS3, Line 175: # TSAN and FULL_TSAN runs are detected by reading the build type from the .cmake_build_type line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sun, 07 Apr 2024 13:13:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/21191/2/testdata/bin/run-hive-server.sh File testdata/bin/run-hive-server.sh: http://gerrit.cloudera.org:8080/#/c/21191/2/testdata/bin/run-hive-server.sh@169 PS2, Line 169: # In TSAN builds libfesupport.so refers to an externa > This check is always false. What we want is either: Done -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 4 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sun, 07 Apr 2024 13:13:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Hello Joe McDonnell, Michael Smith, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21191 to look at the new patch set (#3). Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs During TSAN runs all Impala binaries (including libfesupport.so) are built with TSAN options, which include a reference to the external symbol __tsan_init. This causes a problem for libfesupport.so when it is loaded into Hive during minicluster startup, because the Java VM running Hive's code cannot supply this symbol (the stock JVM is obviously not built with TSAN). Unfortunately this symbol resolution failure causes Hive's JVM simply to abort on Red Hat 8 (or later) and on Ubuntu 20.04 (or later). On earlier versions of the same platforms the JVM turned the same failure into an UnsatisfiedLinkError exception, which is actually handled by Hive. This patch prevents libfesupport.so from being loaded into Hive for TSAN runs so that the minicluster can actually be started. This is achieved by not adding the directory containing libfesupport.so to JAVA_LIBRARY_PATH, preventing the JVM from finding it. Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 --- M buildall.sh M testdata/bin/run-hive-server.sh 2 files changed, 19 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/21191/3 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/21191 )
Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the
minicluster during TSAN runs
..
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21191/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:
http://gerrit.cloudera.org:8080/#/c/21191/2/testdata/bin/run-hive-server.sh@169
PS2, Line 169: if [[ ! ${SKIP_LOADING_LIBFESUPPORT_IN_HIVE} ]]; then
This check is always false. What we want is either:
if ! $SKIP_LOADING_LIBFESUPPORT_IN_HIVE ; then
or
if [[ "$SKIP_LOADING_LIBFESUPPORT_IN_HIVE" != "true" ]]; then
We'll have a problem if SKIP_LOADING_LIBFESUPPORT_IN_HIVE is not set as that
will trip up the "set -euo pipefail" above. So, running this from a place other
than buildall.sh is a problem (and developers do that).
One thing we could do is look at the ${IMPALA_HOME}/.cmake_build_type file
(produced by our CMake build) and see if it contains TSAN. We use that file
from tests/common/environ.py to determine the build type.
if ! grep -q "TSAN" ${IMPALA_HOME}/.cmake_build_type ; then
The file would definitely exist for automated test jobs, and if it doesn't
exist, the code would just go ahead and add libfesupport.so.
--
To view, visit http://gerrit.cloudera.org:8080/21191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20
Gerrit-Change-Number: 21191
Gerrit-PatchSet: 2
Gerrit-Owner: Laszlo Gaal
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Laszlo Gaal
Gerrit-Reviewer: Michael Smith
Gerrit-Comment-Date: Sat, 23 Mar 2024 00:47:29 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15640/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Fri, 22 Mar 2024 22:07:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15639/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Fri, 22 Mar 2024 22:04:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/21191/1/testdata/bin/run-hive-server.sh File testdata/bin/run-hive-server.sh: http://gerrit.cloudera.org:8080/#/c/21191/1/testdata/bin/run-hive-server.sh@176 PS1, Line 176: ex > Ooops Done -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Fri, 22 Mar 2024 21:44:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21191 to look at the new patch set (#2). Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs During TSAN runs all Impala binaries (including libfesupport.so) are built with TSAN options, which include a reference to the external symbol __tsan_init. This causes a problem for libfesupport.so when it is loaded into Hive during minicluster startup, because the Java VM running Hive's code cannot supply this symbol (the stock JVM is obviously not built with TSAN). Unfortunately this symbol resolution failure causes Hive's JVM simply to abort on Red Hat 8 (or later) and on Ubuntu 20.04 (or later), unlike on earlier versions of the same platforms, where the JVM turns the failure into an exception that is actually handled by Hive. This patch prevents libfesupport.so from being loaded into Hive for TSAN runs so that the minicluster can actually be started. This is achieved by not adding the directory containing libfesupport.so to JAVA_LIBRARY_PATH, preventing the JVM from finding it. Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 --- M buildall.sh M testdata/bin/run-hive-server.sh 2 files changed, 18 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/21191/2 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/21191 ) Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. Patch Set 1: Code-Review-1 (1 comment) http://gerrit.cloudera.org:8080/#/c/21191/1/testdata/bin/run-hive-server.sh File testdata/bin/run-hive-server.sh: http://gerrit.cloudera.org:8080/#/c/21191/1/testdata/bin/run-hive-server.sh@176 PS1, Line 176: # Ooops -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Fri, 22 Mar 2024 21:42:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs
Laszlo Gaal has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21191 Change subject: IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs .. IMPALA-12564: Prevent Hive loading libfesupport.so in the minicluster during TSAN runs During TSAN runs all Impala binaries (including libfesupport.so) are built with TSAN options, which include a reference to the external symbol __tsan_init. This causes a problem for libfesupport.so when it is loaded into Hive during minicluster startup, because the Java VM running Hive's code cannot supply this symbol (the stock JVM is obviously not built with TSAN). Unfortunately this symbol resolution failure causes Hive's JVM simply to abort on Red Hat 8 (or later) and on Ubuntu 20.04 (or later), unlike on earlier versions of the same platforms, where the JVM turns the failure into an exception that is actually handled by Hive. This patch prevents libfesupport.so from being loaded into Hive for TSAN runs so that the minicluster can actually be started. This is achieved by not adding the directory containing libfesupport.so to JAVA_LIBRARY_PATH, preventing the JVM from finding it. Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 --- M buildall.sh M testdata/bin/run-hive-server.sh 2 files changed, 18 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/21191/1 -- To view, visit http://gerrit.cloudera.org:8080/21191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie030d9876c297d6e9dae80eba37e525ee2bccb20 Gerrit-Change-Number: 21191 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal
