[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages When using bin/dump_breakpad_symbols.py to dump symbols for RPM/DEB packages, the script extracts the packages to a temporary directory and relies on keeping that directory around until the processing is finished. The parallel processing added in IMPALA-11511 breaks the logic that keeps the temporary directory around, so the script generates errors like: Found debugging info in /tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug Failed to open ELF file '/tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug': No such file or directory Failed to write symbol file. This turns off parallelism for bin/dump_breakpad_symbols.py when processing RPM/DEB packages (i.e. -r/--pkg). This also avoids using a ThreadPool when num_processes <= 1. Testing: - Hand tested with Redhat 7 RPMs Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Reviewed-on: http://gerrit.cloudera.org:8080/20943 Reviewed-by: Impala Public Jenkins Tested-by: Joe McDonnell --- M bin/dump_breakpad_symbols.py 1 file changed, 21 insertions(+), 11 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved Joe McDonnell: Verified -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 3: Verified+1 The test failure is unrelated -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 23:33:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Joe McDonnell has removed a vote on this change. Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Removed Verified-1 by Impala Public Jenkins -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10208/ -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 23:28:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 18:53:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10208/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 18:53:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15089/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 18:28:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 18:15:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/20943/1/bin/dump_breakpad_symbols.py File bin/dump_breakpad_symbols.py: http://gerrit.cloudera.org:8080/#/c/20943/1/bin/dump_breakpad_symbols.py@369 PS1, Line 369: status = 1 > Do we need a break here when something happens? Good point, let's have the behavior match the parallel case. Before the parallelism, this was the old behavior. I filed IMPALA-12766 to add a flag to get the old behavior that keeps going. -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 18:01:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Hello Yida Wu, Michael Smith, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20943 to look at the new patch set (#2). Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages When using bin/dump_breakpad_symbols.py to dump symbols for RPM/DEB packages, the script extracts the packages to a temporary directory and relies on keeping that directory around until the processing is finished. The parallel processing added in IMPALA-11511 breaks the logic that keeps the temporary directory around, so the script generates errors like: Found debugging info in /tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug Failed to open ELF file '/tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug': No such file or directory Failed to write symbol file. This turns off parallelism for bin/dump_breakpad_symbols.py when processing RPM/DEB packages (i.e. -r/--pkg). This also avoids using a ThreadPool when num_processes <= 1. Testing: - Hand tested with Redhat 7 RPMs Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 --- M bin/dump_breakpad_symbols.py 1 file changed, 21 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/20943/2 -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/20943/1/bin/dump_breakpad_symbols.py File bin/dump_breakpad_symbols.py: http://gerrit.cloudera.org:8080/#/c/20943/1/bin/dump_breakpad_symbols.py@369 PS1, Line 369: status = 1 Do we need a break here when something happens? -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 29 Jan 2024 02:28:15 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Wed, 24 Jan 2024 18:35:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20943 ) Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15046/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 24 Jan 2024 03:19:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20943 Change subject: IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages .. IMPALA-12745: Skip parallel symbol dumping with RPM/DEB packages When using bin/dump_breakpad_symbols.py to dump symbols for RPM/DEB packages, the script extracts the packages to a temporary directory and relies on keeping that directory around until the processing is finished. The parallel processing added in IMPALA-11511 breaks the logic that keeps the temporary directory around, so the script generates errors like: Found debugging info in /tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug Failed to open ELF file '/tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug': No such file or directory Failed to write symbol file. This turns off parallelism for bin/dump_breakpad_symbols.py when processing RPM/DEB packages (i.e. -r/--pkg). This also avoids using a ThreadPool when num_processes <= 1. Testing: - Hand tested with Redhat 7 RPMs Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 --- M bin/dump_breakpad_symbols.py 1 file changed, 20 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/20943/1 -- To view, visit http://gerrit.cloudera.org:8080/20943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If2885a9cfb36a4f616b539599e7f744bd23552c3 Gerrit-Change-Number: 20943 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
